niks3-go-unit-tests
checks.x86_64-linux.go-unit-tests
· build #511
· raw
1tribuchet: building on jamie2Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestUploadBuildLog_FileBodyReplayedOnRetry6=== PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry7=== RUN TestRegisterUploadedObjectReusesConnections8=== PAUSE TestRegisterUploadedObjectReusesConnections9=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica10=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica11=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun12=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun13=== RUN TestCaseHackSuffix14=== PAUSE TestCaseHackSuffix15=== RUN TestFilterOversizedClosures16=== PAUSE TestFilterOversizedClosures17=== RUN TestUploadMultipart_PartsInParallel18=== PAUSE TestUploadMultipart_PartsInParallel19=== RUN TestUploadMultipart_ReturnsAfterDroppedBody20=== PAUSE TestUploadMultipart_ReturnsAfterDroppedBody21=== RUN TestUploadMultipart_FailedProducerIsNotCompleted22=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted23=== RUN TestPartSizeForNAR24=== PAUSE TestPartSizeForNAR25=== RUN TestDumpPathCaseHackMatchesNix26--- PASS: TestDumpPathCaseHackMatchesNix (0.03s)27=== RUN TestDumpPathCaseHackCollision28--- PASS: TestDumpPathCaseHackCollision (0.00s)29=== RUN TestDumpPathMatchesNix30=== PAUSE TestDumpPathMatchesNix31=== RUN TestDumpPathSingleFile32=== PAUSE TestDumpPathSingleFile33=== RUN TestDumpPathWriterError34=== PAUSE TestDumpPathWriterError35=== RUN TestEncodeNixBase3236=== PAUSE TestEncodeNixBase3237=== RUN TestEncodeNixBase32WithRealHash38=== PAUSE TestEncodeNixBase32WithRealHash39=== RUN TestConvertHashToNix3240=== PAUSE TestConvertHashToNix3241=== RUN TestGetStorePathHash42=== PAUSE TestGetStorePathHash43=== RUN TestPathInfoHashCompatibility44=== PAUSE TestPathInfoHashCompatibility45=== RUN TestParsePathInfoJSON46=== PAUSE TestParsePathInfoJSON47=== RUN TestParsePathInfoJSONMultiplePaths48=== PAUSE TestParsePathInfoJSONMultiplePaths49=== RUN TestPathInfoCACompatibility50=== PAUSE TestPathInfoCACompatibility51=== RUN TestRateLimiterFeedback52=== PAUSE TestRateLimiterFeedback53=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess54=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess55=== RUN TestRegisterUploadedObject_BoundedAgainstSilentServer56=== PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer57=== RUN TestRegisterUploadedObject_OutlivesPush58=== PAUSE TestRegisterUploadedObject_OutlivesPush59=== RUN TestResolveStorePath60=== PAUSE TestResolveStorePath61=== RUN TestDoWithRetry_BodyReplayedViaGetBody62=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody63=== RUN TestRetryTimeoutOutlastsMaxRetries64=== PAUSE TestRetryTimeoutOutlastsMaxRetries65=== RUN TestDoWithRetry_FinalResponseBodyReadable66=== PAUSE TestDoWithRetry_FinalResponseBodyReadable67=== RUN TestShellSplit68=== PAUSE TestShellSplit69=== RUN TestShellSplitErrors70=== PAUSE TestShellSplitErrors71=== RUN TestStreamPushReportsEveryPath72=== PAUSE TestStreamPushReportsEveryPath73=== RUN TestStreamPushBatchesUnderLoad74=== PAUSE TestStreamPushBatchesUnderLoad75=== RUN TestStreamPushIsolatesFailures76=== PAUSE TestStreamPushIsolatesFailures77=== RUN TestStreamPushGivesUpOnDeadServer78=== PAUSE TestStreamPushGivesUpOnDeadServer79=== RUN TestStreamPushRequestLine80=== PAUSE TestStreamPushRequestLine81=== RUN TestStreamPushReportsSignatures82=== PAUSE TestStreamPushReportsSignatures83=== RUN TestStreamPushReportsSkippedPaths84=== PAUSE TestStreamPushReportsSkippedPaths85=== RUN TestClientSignaturesByStorePath86=== PAUSE TestClientSignaturesByStorePath87=== RUN TestStreamPushAnswersEveryLineWhenCanceled88=== PAUSE TestStreamPushAnswersEveryLineWhenCanceled89=== RUN TestStreamPushStopsWhenCanceled90=== PAUSE TestStreamPushStopsWhenCanceled91=== RUN TestSetClientTLS92=== PAUSE TestSetClientTLS93=== RUN TestSetClientTLSDoesNotMutateDefaultTransport94=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport95=== RUN TestSetClientTLSErrors96=== PAUSE TestSetClientTLSErrors97=== RUN TestStaticToken98=== PAUSE TestStaticToken99=== RUN TestFileTokenReadsAndCaches100=== PAUSE TestFileTokenReadsAndCaches101=== RUN TestFileTokenMissing102=== PAUSE TestFileTokenMissing103=== RUN TestFileTokenEmpty104=== PAUSE TestFileTokenEmpty105=== RUN TestScriptTokenNoExpiryRerunsEveryCall106=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall107=== RUN TestScriptTokenCachesUntilRefresh108=== PAUSE TestScriptTokenCachesUntilRefresh109=== RUN TestScriptTokenEmptyToken110=== PAUSE TestScriptTokenEmptyToken111=== RUN TestScriptTokenBadJSON112=== PAUSE TestScriptTokenBadJSON113=== RUN TestScriptTokenScriptFails114=== PAUSE TestScriptTokenScriptFails115=== RUN TestScriptTokenEmptyCommand116=== PAUSE TestScriptTokenEmptyCommand117=== CONT TestDoServerRequestAttachesToken118=== CONT TestDoWithRetry_FinalResponseBodyReadable119=== CONT TestSetClientTLS120=== CONT TestFileTokenEmpty121=== CONT TestFileTokenMissing122=== CONT TestFileTokenReadsAndCaches123--- PASS: TestFileTokenEmpty (0.00s)124=== CONT TestEncodeNixBase32125=== RUN TestEncodeNixBase32/test_string_hash126=== PAUSE TestEncodeNixBase32/test_string_hash127=== RUN TestEncodeNixBase32/empty_input128=== PAUSE TestEncodeNixBase32/empty_input129=== CONT TestRegisterUploadedObject_OutlivesPush130--- PASS: TestFileTokenMissing (0.00s)131=== CONT TestDumpPathWriterError132=== CONT TestScriptTokenNoExpiryRerunsEveryCall133=== CONT TestStaticToken134--- PASS: TestStaticToken (0.00s)135=== CONT TestRegisterUploadedObject_BoundedAgainstSilentServer136=== CONT TestScriptTokenEmptyCommand137--- PASS: TestFileTokenReadsAndCaches (0.00s)138=== CONT TestDumpPathSingleFile139=== CONT TestSetClientTLSErrors140--- PASS: TestScriptTokenEmptyCommand (0.00s)141=== CONT TestScriptTokenScriptFails142=== CONT TestScriptTokenBadJSON143=== CONT TestScriptTokenEmptyToken144=== CONT TestScriptTokenCachesUntilRefresh145=== CONT TestSetClientTLSDoesNotMutateDefaultTransport146=== CONT TestClientSignaturesByStorePath147=== CONT TestStreamPushStopsWhenCanceled148=== CONT TestStreamPushAnswersEveryLineWhenCanceled149=== CONT TestStreamPushReportsSkippedPaths150=== CONT TestEncodeNixBase32WithRealHash151=== CONT TestRetryTimeoutOutlastsMaxRetries152=== CONT TestUploadMultipart_ReturnsAfterDroppedBody153=== CONT TestDoWithRetry_BodyReplayedViaGetBody154=== CONT TestResolveStorePath155=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess156--- PASS: TestClientSignaturesByStorePath (0.00s)157=== CONT TestDumpPathMatchesNix158--- PASS: TestEncodeNixBase32WithRealHash (0.00s)1592026/11/12 17:02:13 WARN Rate limiter enabled after throttle name=server-test rate=5160--- PASS: TestStreamPushStopsWhenCanceled (0.00s)161=== CONT TestPartSizeForNAR162=== RUN TestPartSizeForNAR/zero_stays_at_minimum163=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum164=== RUN TestPartSizeForNAR/small_stays_at_minimum165=== PAUSE TestPartSizeForNAR/small_stays_at_minimum166=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum167=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum168=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts169=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts170=== RUN TestPartSizeForNAR/1_TiB171=== PAUSE TestPartSizeForNAR/1_TiB172=== RUN TestPartSizeForNAR/5_TiB_S3_max_object1732000/01/01 00:00:00 ERROR Upload failed error="context canceled" count=2174=== RUN TestSetClientTLSErrors/missing_cert_file175--- PASS: TestStreamPushAnswersEveryLineWhenCanceled (0.00s)176=== CONT TestPathInfoCACompatibility177=== CONT TestRateLimiterFeedback178=== RUN TestRateLimiterFeedback/429_enables_limiter179=== RUN TestPathInfoCACompatibility/null_ca_field180=== PAUSE TestRateLimiterFeedback/429_enables_limiter181=== RUN TestRateLimiterFeedback/503_enables_limiter182=== PAUSE TestRateLimiterFeedback/503_enables_limiter183=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter184--- PASS: TestScriptTokenScriptFails (0.01s)1852026/11/12 17:02:13 WARN Rate limiter enabled after throttle name=server-test rate=5186=== PAUSE TestSetClientTLSErrors/missing_cert_file1872026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:33023188=== RUN TestSetClientTLSErrors/missing_key_file189=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter190=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter191=== CONT TestUploadMultipart_FailedProducerIsNotCompleted192=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary193=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary194=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part195=== CONT TestParsePathInfoJSONMultiplePaths196=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object1972026/11/12 17:02:13 WARN Rate limiter backed off name=server-test rate=5198=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths1992026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:33023200=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part2012026/11/12 17:02:13 WARN Rate limiter enabled after throttle name=server-test rate=5202=== PAUSE TestSetClientTLSErrors/missing_key_file203=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter2042026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:33957205--- PASS: TestStreamPushReportsSkippedPaths (0.00s)206--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.01s)207--- PASS: TestDoServerRequestAttachesToken (0.01s)2082026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:36065209=== CONT TestFilterOversizedClosures210=== RUN TestFilterOversizedClosures/no_limit_keeps_everything211=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything212=== RUN TestPartSizeForNAR/capped_at_5_GiB213=== PAUSE TestPartSizeForNAR/capped_at_5_GiB214=== CONT TestConvertHashToNix322152026/11/12 17:02:13 WARN Rate limiter backed off name=server-test rate=5216=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun217=== RUN TestConvertHashToNix32/SRI_format_to_Nix32218=== PAUSE TestPathInfoCACompatibility/null_ca_field219=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths220=== CONT TestPathInfoHashCompatibility221=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths222=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)223=== RUN TestSetClientTLSErrors/missing_ca_file224=== PAUSE TestSetClientTLSErrors/missing_ca_file225=== CONT TestGetStorePathHash226=== RUN TestGetStorePathHash/valid_store_path227=== PAUSE TestGetStorePathHash/valid_store_path228=== CONT TestParsePathInfoJSON229=== CONT TestStreamPushBatchesUnderLoad230--- PASS: TestRegisterUploadedObject_OutlivesPush (0.01s)231=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped232=== CONT TestCaseHackSuffix233=== RUN TestSetClientTLS/rejects_connection_without_client_cert234=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert235=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=false236=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=false237=== CONT TestUploadMultipart_PartsInParallel238=== CONT TestStreamPushReportsSignatures239=== RUN TestPathInfoCACompatibility/old_string_format_-_text240=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text2412026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:33957242=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths243=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)244=== RUN TestSetClientTLSErrors/invalid_ca_file245=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive246=== RUN TestGetStorePathHash/basename_without_hyphen_should_error247=== RUN TestParsePathInfoJSON/Nix_format248--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.01s)249=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped250=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32251=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA252=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=true253=== CONT TestRegisterUploadedObjectReusesConnections254=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon255=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive256--- PASS: TestResolveStorePath (0.00s)257=== PAUSE TestParsePathInfoJSON/Nix_format258=== PAUSE TestSetClientTLSErrors/invalid_ca_file259=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error260=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=true261=== RUN TestFilterOversizedClosures/all_closures_skipped262=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA263=== RUN TestConvertHashToNix32/already_Nix32_format2642026/11/12 17:02:13 ERROR Upload failed error=boom count=12652026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:339572662026/11/12 17:02:13 INFO Garbage collection started267--- PASS: TestDoWithRetry_FinalResponseBodyReadable (0.01s)268=== RUN TestPathInfoCACompatibility/new_structured_format_-_text269=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text270=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error271=== RUN TestSetClientTLS/preserves_debug_logging_transport272=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon273=== CONT TestShellSplit274=== CONT TestStreamPushReportsEveryPath275=== RUN TestParsePathInfoJSON/Lix_format276=== PAUSE TestParsePathInfoJSON/Lix_format277=== RUN TestParsePathInfoJSON/empty_input278=== PAUSE TestParsePathInfoJSON/empty_input279=== CONT TestStreamPushGivesUpOnDeadServer280=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica281=== CONT TestStreamPushRequestLine2822026/11/12 17:02:13 ERROR Upload failed error="connection refused" count=202832026/11/12 17:02:13 ERROR Server seems unavailable, giving up on batch untried=17284=== PAUSE TestFilterOversizedClosures/all_closures_skipped2852026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:33957286=== CONT TestEncodeNixBase32/empty_input2872026/11/12 17:02:13 ERROR Upload failed error=boom count=1288=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary289=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter290=== PAUSE TestConvertHashToNix32/already_Nix32_format2912026/11/12 17:02:13 INFO Garbage collection started292=== RUN TestConvertHashToNix32/invalid_format293=== PAUSE TestConvertHashToNix32/invalid_format294=== CONT TestStreamPushIsolatesFailures295=== CONT TestRateLimiterFeedback/429_enables_limiter296--- PASS: TestUploadMultipart_ReturnsAfterDroppedBody (0.00s)297--- PASS: TestScriptTokenEmptyToken (0.01s)298--- PASS: TestScriptTokenCachesUntilRefresh (0.01s)299--- PASS: TestStreamPushReportsSignatures (0.03s)300--- PASS: TestScriptTokenBadJSON (0.04s)301--- PASS: TestShellSplit (0.00s)302--- PASS: TestStreamPushReportsEveryPath (0.00s)303--- PASS: TestDumpPathSingleFile (0.04s)304--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (0.03s)305=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter306=== CONT TestRateLimiterFeedback/503_enables_limiter3072026/11/12 17:02:13 WARN Rate limiter enabled after throttle name=server-test rate=5308--- PASS: TestStreamPushGivesUpOnDeadServer (0.00s)3092026/11/12 17:02:13 ERROR Upload failed error="bad path" count=3310=== CONT TestShellSplitErrors311=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method3122026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:37679313=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error3142026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:33957315=== PAUSE TestSetClientTLS/preserves_debug_logging_transport3162026/11/12 17:02:13 WARN Rate limiter enabled after throttle name=server-test rate=5317=== CONT TestUploadBuildLog_FileBodyReplayedOnRetry3182026/11/12 17:02:13 WARN Rate limiter backed off name=server-test rate=5319=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI3202026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:45501321=== RUN TestParsePathInfoJSON/whitespace_only322=== CONT TestEncodeNixBase32/test_string_hash3232026/11/12 17:02:13 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=0324=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part325=== CONT TestPartSizeForNAR/zero_stays_at_minimum326=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths3272026/11/12 17:02:13 WARN Rate limiter backed off name=server-test rate=5328--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.04s)329=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method330=== CONT TestPartSizeForNAR/small_stays_at_minimum331=== CONT TestSetClientTLSErrors/missing_key_file332=== CONT TestPartSizeForNAR/capped_at_5_GiB3332026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:33957334=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=true335=== CONT TestPartSizeForNAR/5_TiB_S3_max_object336=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error3372026/11/12 17:02:13 WARN Garbage collection finished on another replica. Its statistics are not available here338=== CONT TestPartSizeForNAR/1_TiB339=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI340=== PAUSE TestParsePathInfoJSON/whitespace_only341=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts342=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum343--- PASS: TestShellSplitErrors (0.00s)344=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths3452026/11/12 17:02:13 WARN Rate limiter enabled after throttle name=s3-test rate=5346=== CONT TestSetClientTLSErrors/missing_cert_file3472026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:33355348=== CONT TestSetClientTLSErrors/missing_ca_file349=== CONT TestSetClientTLSErrors/invalid_ca_file350=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=false351=== CONT TestFilterOversizedClosures/no_limit_keeps_everything352=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error3532026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:33957354=== CONT TestFilterOversizedClosures/all_closures_skipped3552026/11/12 17:02:13 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50356=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped3572026/11/12 17:02:13 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000358=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive359=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512360=== RUN TestParsePathInfoJSON/invalid_JSON361=== CONT TestConvertHashToNix32/SRI_format_to_Nix32362=== CONT TestConvertHashToNix32/invalid_format363--- PASS: TestStreamPushIsolatesFailures (0.00s)364=== CONT TestConvertHashToNix32/already_Nix32_format365=== CONT TestSetClientTLS/rejects_connection_without_client_cert366=== CONT TestSetClientTLS/preserves_debug_logging_transport367=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA368=== CONT TestPathInfoCACompatibility/null_ca_field369=== CONT TestPathInfoCACompatibility/new_structured_format_-_text370=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method3712026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:33957372=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512373=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)374=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI375=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512376=== PAUSE TestParsePathInfoJSON/invalid_JSON377=== CONT TestParsePathInfoJSON/Nix_format378=== CONT TestParsePathInfoJSON/whitespace_only379=== CONT TestGetStorePathHash/valid_store_path380=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error381=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error382--- PASS: TestEncodeNixBase32 (0.00s)383 --- PASS: TestEncodeNixBase32/empty_input (0.00s)384 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)385=== CONT TestGetStorePathHash/basename_without_hyphen_should_error386=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon387=== CONT TestPathInfoCACompatibility/old_string_format_-_text388=== CONT TestParsePathInfoJSON/invalid_JSON389=== CONT TestParsePathInfoJSON/empty_input390=== CONT TestParsePathInfoJSON/Lix_format391--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (0.00s)392--- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (0.00s)393--- PASS: TestFilterOversizedClosures (0.03s)394 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)395 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)396 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)397--- PASS: TestConvertHashToNix32 (0.03s)398 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)399 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)400 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)401--- PASS: TestUploadMultipart_FailedProducerIsNotCompleted (0.00s)402 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary (0.00s)403 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part (0.00s)4042026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:33957405--- PASS: TestStreamPushBatchesUnderLoad (0.00s)406 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=true (0.00s)407 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=false (0.00s)408--- PASS: TestGetStorePathHash (0.04s)409 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)410 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)411 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)412 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)413--- PASS: TestRateLimiterFeedback (0.00s)414 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s)415 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.00s)416 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.00s)417 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s)418--- PASS: TestPathInfoCACompatibility (0.04s)419 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)420 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)421 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)422 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)423 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)424--- PASS: TestParsePathInfoJSON (0.04s)425 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)426 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)427 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)428 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)429 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)430--- PASS: TestParsePathInfoJSONMultiplePaths (0.00s)431 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)432 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)433--- PASS: TestPartSizeForNAR (0.01s)434 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)435 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)436 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)437 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)438 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)439 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)440 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)441--- PASS: TestPathInfoHashCompatibility (0.04s)442 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)443 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)444 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)445 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)446--- PASS: TestSetClientTLSErrors (0.01s)447 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)448 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)449 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)450 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)4512026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:339574522026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:339574532026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:339574542026/11/12 17:02:13 http: TLS handshake error from 127.0.0.1:47346: remote error: tls: bad certificate4552026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:33957456--- PASS: TestSetClientTLS (0.05s)457 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.00s)458 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.00s)459 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.01s)4602026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:33957461--- PASS: TestStreamPushRequestLine (0.02s)4622026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:339574632026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:339574642026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:339574652026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:339574662026/11/12 17:02:13 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:33957467--- PASS: TestRegisterUploadedObjectReusesConnections (0.06s)468--- PASS: TestRetryTimeoutOutlastsMaxRetries (0.06s)469--- PASS: TestCaseHackSuffix (0.06s)470--- PASS: TestDumpPathWriterError (0.08s)4712026/11/12 17:02:13 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:38965/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:38965/api/uploads/complete4722026/11/12 17:02:13 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4732026/11/12 17:02:13 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:38965/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:38965/api/uploads/complete4742026/11/12 17:02:13 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4752026/11/12 17:02:13 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:38965/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:38965/api/uploads/complete4762026/11/12 17:02:13 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"477--- PASS: TestRegisterUploadedObject_BoundedAgainstSilentServer (0.10s)478--- PASS: TestDumpPathMatchesNix (0.12s)479--- PASS: TestUploadMultipart_PartsInParallel (0.67s)480--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)481PASS482Running server tests...483The files belonging to this database system will be owned by user "nixbld".484This user must also own the server process.485486The database cluster will be initialized with locale "C".487The default database encoding has accordingly been set to "SQL_ASCII".488The default text search configuration will be set to "english".489490Data page checksums are enabled.491492creating directory /build/postgres1338061233/data ... ok493creating subdirectories ... ok494selecting dynamic shared memory implementation ... posix495selecting default "max_connections" ... 100496selecting default "shared_buffers" ... 128MB497selecting default time zone ... UTC498creating configuration files ... ok499running bootstrap script ... ok500performing post-bootstrap initialization ... ok501syncing data to disk ... ok502503initdb: warning: enabling "trust" authentication for local connections504initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.505506Success. You can now start the database server using:507508 pg_ctl -D /build/postgres1338061233/data -l logfile start509510/build/postgres1338061233:5432 - no response5112026-11-12 17:02:15.443 UTC [129] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit5122026-11-12 17:02:15.444 UTC [129] LOG: listening on Unix socket "/build/postgres1338061233/.s.PGSQL.5432"5132026-11-12 17:02:15.448 UTC [136] LOG: database system was shut down at 2026-11-12 17:02:15 UTC5142026-11-12 17:02:15.452 UTC [129] LOG: database system is ready to accept connections515/build/postgres1338061233:5432 - accepting connections516{"timestamp":"2026-11-12T17:02:15.75187075Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"1e7983e1-982f-4153-b313-11fcbaa6c35f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(398)"}517=== RUN TestAbortSucceeded518=== PAUSE TestAbortSucceeded519=== RUN TestService_AuthMiddleware520=== PAUSE TestService_AuthMiddleware521=== RUN TestService_AuthMiddleware_MTLSProxyHeader522=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader523=== RUN TestService_AuthMiddleware_MTLSBoundSubjects524=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects525=== RUN TestService_ReadAuthMiddleware526=== PAUSE TestService_ReadAuthMiddleware527=== RUN TestService_AuthMiddleware_OIDC528=== PAUSE TestService_AuthMiddleware_OIDC529=== RUN TestService_RequireScope_OIDC530=== PAUSE TestService_RequireScope_OIDC531=== RUN TestService_ReadScope_PublicByDefault532=== PAUSE TestService_ReadScope_PublicByDefault533=== RUN TestPinScopedPrincipal534=== PAUSE TestPinScopedPrincipal535=== RUN TestCacheConfigHandler536=== PAUSE TestCacheConfigHandler537=== RUN TestCacheStatsHandler538=== PAUSE TestCacheStatsHandler539=== RUN TestClientCADerivations540=== PAUSE TestClientCADerivations541=== RUN TestClientErrorHandling542=== PAUSE TestClientErrorHandling543=== RUN TestClientIntegration544=== PAUSE TestClientIntegration545=== RUN TestClientMultipleUploads546=== PAUSE TestClientMultipleUploads547=== RUN TestClientWithDependencies548=== PAUSE TestClientWithDependencies549=== RUN TestClientSharedPathCommittedMidPush550=== PAUSE TestClientSharedPathCommittedMidPush551=== RUN TestPinProtectsFromGC552=== PAUSE TestPinProtectsFromGC553=== RUN TestClientPushesUseOnePush554=== PAUSE TestClientPushesUseOnePush555=== RUN TestClientFallsBackToClosures556=== PAUSE TestClientFallsBackToClosures557=== RUN TestDeleteClosuresRacingPins558=== PAUSE TestDeleteClosuresRacingPins559=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock560=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock561=== RUN TestResolveDBConnectionString562=== PAUSE TestResolveDBConnectionString563=== RUN TestConnectWaitsForAPeerMigration564=== PAUSE TestConnectWaitsForAPeerMigration565=== RUN TestLeadElectsOneAndHandsOver566=== PAUSE TestLeadElectsOneAndHandsOver567=== RUN TestLeadIncumbentWinsAfterRestart5682026/11/12 17:02:16 INFO lead: acquired remote=192.0.2.1:12345692026/11/12 17:02:16 INFO lead: released remote=192.0.2.1:12345702026/11/12 17:02:16 INFO lead: acquired remote=192.0.2.1:12345712026/11/12 17:02:17 INFO lead: released remote=192.0.2.1:1234572--- PASS: TestLeadIncumbentWinsAfterRestart (1.90s)573=== RUN TestLeadUnannouncedStandbyYieldsToIncumbent574=== PAUSE TestLeadUnannouncedStandbyYieldsToIncumbent575=== RUN TestLeadIncumbentAnnouncesAtOnce576=== PAUSE TestLeadIncumbentAnnouncesAtOnce577=== RUN TestLeadEndsOnShutdown578=== PAUSE TestLeadEndsOnShutdown579=== RUN TestLeadEndsWhenItsConnectionHangs5802026/11/12 17:02:17 INFO lead: acquired remote=192.0.2.1:12345812026-11-12 17:02:19.279 UTC [575] FATAL: terminating connection due to administrator command5822026/11/12 17:02:19 INFO lead: acquired remote=192.0.2.1:12345832026/11/12 17:02:20 WARN lead: lock connection lost error="timeout: context deadline exceeded"5842026/11/12 17:02:20 INFO lead: released remote=192.0.2.1:12345852026/11/12 17:02:20 INFO lead: released remote=192.0.2.1:1234586--- PASS: TestLeadEndsWhenItsConnectionHangs (3.13s)587=== RUN TestGCAdvisoryLockBlocksConcurrentRun588=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun589=== RUN TestGCBugBareHashReferences590=== PAUSE TestGCBugBareHashReferences591=== RUN TestGCMetrics592=== PAUSE TestGCMetrics593=== RUN TestPushDedupSurvivesConcurrentGC594=== PAUSE TestPushDedupSurvivesConcurrentGC595=== RUN TestDeduplicatedObjectsRecordedAsPending596=== PAUSE TestDeduplicatedObjectsRecordedAsPending597=== RUN TestGCSweepSkipsPendingObjects598=== PAUSE TestGCSweepSkipsPendingObjects599=== RUN TestTombstonedObjectOfferedWithoutWaiting600=== PAUSE TestTombstonedObjectOfferedWithoutWaiting601=== RUN TestGCSweepDeliversEachKeyOnce602=== PAUSE TestGCSweepDeliversEachKeyOnce603=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection604=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection605=== RUN TestForceGCDuringPushOffersSweptObject606=== PAUSE TestForceGCDuringPushOffersSweptObject607=== RUN TestFailedS3DeleteKeepsTombstone608=== PAUSE TestFailedS3DeleteKeepsTombstone609=== RUN TestSweepSkipsRowsLockedByPush610=== PAUSE TestSweepSkipsRowsLockedByPush611=== RUN TestRegisterCompletedObjectNeedsPendingRow612=== PAUSE TestRegisterCompletedObjectNeedsPendingRow613=== RUN TestGetExistingObjectsLocksRows614=== PAUSE TestGetExistingObjectsLocksRows615=== RUN TestSweepRechecksPendingAfterLocking616=== PAUSE TestSweepRechecksPendingAfterLocking617=== RUN TestPushWaitsForSweepHoldingRow618=== PAUSE TestPushWaitsForSweepHoldingRow619=== RUN TestCommitRacingPendingCleanupKeepsObjects620=== PAUSE TestCommitRacingPendingCleanupKeepsObjects621=== RUN TestGCEndsOnShutdown622=== PAUSE TestGCEndsOnShutdown623=== RUN TestGCTaskStore_StartNew624=== PAUSE TestGCTaskStore_StartNew625=== RUN TestGCTaskStore_DeduplicateSameParams626=== PAUSE TestGCTaskStore_DeduplicateSameParams627=== RUN TestGCTaskStore_ConflictDifferentParams628=== PAUSE TestGCTaskStore_ConflictDifferentParams629=== RUN TestGCTaskStore_GetEmpty630=== PAUSE TestGCTaskStore_GetEmpty631=== RUN TestGCTaskStore_GetReturnsLatest632=== PAUSE TestGCTaskStore_GetReturnsLatest633=== RUN TestGCTaskStore_CompletedAllowsNewTask634=== PAUSE TestGCTaskStore_CompletedAllowsNewTask635=== RUN TestGCTaskStore_PhaseUpdates636=== PAUSE TestGCTaskStore_PhaseUpdates637=== RUN TestGCTaskStore_Fail638=== PAUSE TestGCTaskStore_Fail639=== RUN TestGracefulShutdownDrainsInflight640=== PAUSE TestGracefulShutdownDrainsInflight641=== RUN TestService_healthCheckHandler642=== PAUSE TestService_healthCheckHandler643=== RUN TestService_readinessHandler644=== PAUSE TestService_readinessHandler645=== RUN TestGenerateLandingPage646=== PAUSE TestGenerateLandingPage647=== RUN TestEnsureAbortUploadsRule648=== PAUSE TestEnsureAbortUploadsRule649=== RUN TestValidatePinReadNeedsWriteSubjects650=== PAUSE TestValidatePinReadNeedsWriteSubjects651=== RUN TestParseArgsReadPinFlag652--- PASS: TestParseArgsReadPinFlag (0.00s)653=== RUN TestCacheConfigHandlerMaxNarSize654=== PAUSE TestCacheConfigHandlerMaxNarSize655=== RUN TestCreatePendingClosureRejectsOversizedNAR656=== PAUSE TestCreatePendingClosureRejectsOversizedNAR657=== RUN TestNARDeduplicationMetadataUploadBug658=== PAUSE TestNARDeduplicationMetadataUploadBug659=== RUN TestMetricsInventory660=== PAUSE TestMetricsInventory661=== RUN TestMetricsCommit662=== PAUSE TestMetricsCommit663=== RUN TestService_NativeMTLS664=== PAUSE TestService_NativeMTLS665=== RUN TestServerTLSConfig666=== PAUSE TestServerTLSConfig667=== RUN TestMultipartCleanup668=== PAUSE TestMultipartCleanup669=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded670=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded671=== RUN TestPendingCleanupUsesOneCutoff672=== PAUSE TestPendingCleanupUsesOneCutoff673=== RUN TestObjectStatsTrigger674=== PAUSE TestObjectStatsTrigger675=== RUN TestObjectStatsUpdatedOncePerCommit676=== PAUSE TestObjectStatsUpdatedOncePerCommit677=== RUN TestOrphanedObjectsGC678=== PAUSE TestOrphanedObjectsGC679=== RUN TestOrphanedObjectsGCStressTest680=== PAUSE TestOrphanedObjectsGCStressTest681=== RUN TestPinObjectsReachable682=== PAUSE TestPinObjectsReachable683=== RUN TestPinObjectsGrace684=== PAUSE TestPinObjectsGrace685=== RUN TestPinObjectsTwoMoves686=== PAUSE TestPinObjectsTwoMoves687=== RUN TestPinObjectsSameTarget688=== PAUSE TestPinObjectsSameTarget689=== RUN TestPinObjectsSelfReference690=== PAUSE TestPinObjectsSelfReference691=== RUN TestPinObjectsOtherPin692=== PAUSE TestPinObjectsOtherPin693=== RUN TestPinObjectsDelete694=== PAUSE TestPinObjectsDelete695=== RUN TestPinObjectsBackfill696=== PAUSE TestPinObjectsBackfill697=== RUN TestPinObjectsSharedTarget698=== PAUSE TestPinObjectsSharedTarget699=== RUN TestGCDeletesExpiredPinObjects700=== PAUSE TestGCDeletesExpiredPinObjects701=== RUN TestGlobToLike702=== PAUSE TestGlobToLike703=== RUN TestPinScopedReads704=== PAUSE TestPinScopedReads705=== RUN TestPinScopedReadRedirect706=== PAUSE TestPinScopedReadRedirect707=== RUN TestFullReaderUnaffected708=== PAUSE TestFullReaderUnaffected709=== RUN TestPinScopedSubstringAttack710=== PAUSE TestPinScopedSubstringAttack711=== RUN TestParsePinReadRule712=== PAUSE TestParsePinReadRule713=== RUN TestCommonName714=== PAUSE TestCommonName715=== RUN TestCreatePin_ReservedPins716=== PAUSE TestCreatePin_ReservedPins717=== RUN TestPresentReportsOnlyClosureRoots718=== PAUSE TestPresentReportsOnlyClosureRoots719=== RUN TestPresentRefreshesOnlyStaleClosures720=== PAUSE TestPresentRefreshesOnlyStaleClosures721=== RUN TestPresentNotReportedWhileGCDeletesClosure722=== PAUSE TestPresentNotReportedWhileGCDeletesClosure723=== RUN TestParseSingleRange724=== PAUSE TestParseSingleRange725=== RUN TestProxyHeadersOnlyTrustedOnSocket7262026/11/12 17:02:20 INFO Starting HTTP server address=127.0.0.1:467137272026/11/12 17:02:20 INFO Starting HTTP server address=proxy.sock7282026/11/12 17:02:20 WARN mTLS auth: subject not in bound subjects subject="CN=someone"7292026/11/12 17:02:21 INFO Shutdown signal received, draining in-flight requests timeout=10s730--- PASS: TestProxyHeadersOnlyTrustedOnSocket (0.11s)731=== RUN TestIsValidCachePath732=== PAUSE TestIsValidCachePath733=== RUN TestReadProxyNarinfo734=== PAUSE TestReadProxyNarinfo735=== RUN TestReadProxyNarinfoAlreadyDecompressed736=== PAUSE TestReadProxyNarinfoAlreadyDecompressed737=== RUN TestReadProxyContentNegotiation738=== PAUSE TestReadProxyContentNegotiation739=== RUN TestReadProxyNarStreaming740=== PAUSE TestReadProxyNarStreaming741=== RUN TestReadProxy404742=== PAUSE TestReadProxy404743=== RUN TestReadProxyInvalidPath744=== PAUSE TestReadProxyInvalidPath745=== RUN TestReadProxyHead746=== PAUSE TestReadProxyHead747=== RUN TestReadProxyOutlastsServerWriteTimeout748=== PAUSE TestReadProxyOutlastsServerWriteTimeout749=== RUN TestReadProxyConditionalGet750=== PAUSE TestReadProxyConditionalGet751=== RUN TestReadProxyRootRedirectsToIndexHTML752=== PAUSE TestReadProxyRootRedirectsToIndexHTML753=== RUN TestReadProxyDisabled754=== PAUSE TestReadProxyDisabled755=== RUN TestReadRedirectNar756=== PAUSE TestReadRedirectNar757=== RUN TestReadRedirectKeepsNarinfoProxied758=== PAUSE TestReadRedirectKeepsNarinfoProxied759=== RUN TestReadProxyRangeRequest760=== PAUSE TestReadProxyRangeRequest761=== RUN TestReadRedirectUsesPublicS3URL762=== PAUSE TestReadRedirectUsesPublicS3URL763=== RUN TestPush_OverlappingRootsStoreOneRowPerKey764=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey765=== RUN TestPush_CompleteCommitsEveryRoot766=== PAUSE TestPush_CompleteCommitsEveryRoot767=== RUN TestPush_RepeatedRootCommitsOnce768=== PAUSE TestPush_RepeatedRootCommitsOnce769=== RUN TestPush_CompleteWaitsForCleanupOfThePush770=== PAUSE TestPush_CompleteWaitsForCleanupOfThePush771=== RUN TestPush_RootsAreLockedInByteOrder772=== PAUSE TestPush_RootsAreLockedInByteOrder773=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit774=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit775=== RUN TestPush_RejectsBadRequests776=== PAUSE TestPush_RejectsBadRequests777=== RUN TestPush_SignsNarinfosOfItsPendingObjects778=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects779=== RUN TestCommitCanBeRepeatedUntilCleanup780=== PAUSE TestCommitCanBeRepeatedUntilCleanup781=== RUN TestSharedNarBothUploadsComplete782=== PAUSE TestSharedNarBothUploadsComplete783=== RUN TestCompleteMultipartUpload_ErrorButObjectExists784=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists785=== RUN TestCompletedNarNotReofferedAcrossClosures786=== PAUSE TestCompletedNarNotReofferedAcrossClosures787=== RUN TestPresignedUploadRegisteredBeforeCommit788=== PAUSE TestPresignedUploadRegisteredBeforeCommit789=== RUN TestService_Rustfstest790=== PAUSE TestService_Rustfstest791=== RUN TestParseSize792=== PAUSE TestParseSize793=== RUN TestSkippedUploadsHandler794=== PAUSE TestSkippedUploadsHandler795=== RUN TestSystemdListenerNotActivated796--- PASS: TestSystemdListenerNotActivated (0.00s)797=== RUN TestWatchdogBeatsWhenHealthy798--- PASS: TestWatchdogBeatsWhenHealthy (0.02s)799=== RUN TestWatchdogSkipsWhenUnhealthy8002026/11/12 17:02:21 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8012026/11/12 17:02:21 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8022026/11/12 17:02:21 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8032026/11/12 17:02:21 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8042026/11/12 17:02:21 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8052026/11/12 17:02:21 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8062026/11/12 17:02:21 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8072026/11/12 17:02:21 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8082026/11/12 17:02:21 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8092026/11/12 17:02:21 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"810--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)811=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle812=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle813=== RUN TestProxyWriteTimeout814=== PAUSE TestProxyWriteTimeout815=== RUN TestPendingClosureWriteTimeout816=== PAUSE TestPendingClosureWriteTimeout817=== RUN TestIsValidUploadKey818=== PAUSE TestIsValidUploadKey819=== RUN TestUploadHandlersRejectInvalidKeys820=== PAUSE TestUploadHandlersRejectInvalidKeys821=== RUN TestUploadHandlersRejectOversizedBody822=== PAUSE TestUploadHandlersRejectOversizedBody823=== RUN TestService_cleanupPendingClosuresHandler824=== PAUSE TestService_cleanupPendingClosuresHandler825=== RUN TestService_createPendingClosureHandler826=== PAUSE TestService_createPendingClosureHandler827=== RUN TestService_verifyS3Integrity828=== PAUSE TestService_verifyS3Integrity829=== RUN TestService_verifyS3ReuploadsStaleNarinfo830=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo831=== RUN TestCompleteMultipartUnregistered832=== PAUSE TestCompleteMultipartUnregistered833=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT834=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT835=== CONT TestAbortSucceeded836=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT837--- PASS: TestAbortSucceeded (0.00s)838=== CONT TestPresentNotReportedWhileGCDeletesClosure839=== RUN TestPresentNotReportedWhileGCDeletesClosure/stale840=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/stale841=== RUN TestPresentNotReportedWhileGCDeletesClosure/fresh842=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/fresh843=== CONT TestPresentRefreshesOnlyStaleClosures844=== CONT TestPresentReportsOnlyClosureRoots845=== CONT TestCreatePin_ReservedPins846=== CONT TestCommonName847=== RUN TestCommonName/CN=web-01,O=acme848=== CONT TestParsePinReadRule849=== CONT TestPinScopedSubstringAttack850=== CONT TestFullReaderUnaffected851=== CONT TestPinScopedReadRedirect852=== CONT TestPinScopedReads853=== CONT TestGlobToLike854=== CONT TestGCDeletesExpiredPinObjects855=== CONT TestPinObjectsSharedTarget856=== CONT TestPinObjectsBackfill857=== CONT TestPinObjectsDelete858=== CONT TestPinObjectsOtherPin859=== CONT TestPinObjectsSelfReference860=== CONT TestPinObjectsSameTarget861=== CONT TestPinObjectsTwoMoves862=== CONT TestPinObjectsGrace863=== CONT TestPinObjectsReachable864=== CONT TestOrphanedObjectsGCStressTest865=== CONT TestOrphanedObjectsGC866=== CONT TestObjectStatsUpdatedOncePerCommit867=== RUN TestParsePinReadRule/CN=*=host-{cn}868=== PAUSE TestParsePinReadRule/CN=*=host-{cn}869=== RUN TestParsePinReadRule/CN=web-*,O=x=web-*870=== PAUSE TestParsePinReadRule/CN=web-*,O=x=web-*871=== RUN TestParsePinReadRule/CN=*=872=== PAUSE TestParsePinReadRule/CN=*=873=== RUN TestParsePinReadRule/=host874=== PAUSE TestParsePinReadRule/=host875=== RUN TestParsePinReadRule/CN=*=host/{cn}876=== PAUSE TestCommonName/CN=web-01,O=acme877--- PASS: TestGlobToLike (0.00s)878=== CONT TestObjectStatsTrigger879=== PAUSE TestParsePinReadRule/CN=*=host/{cn}880=== RUN TestCommonName/O=acme881=== PAUSE TestCommonName/O=acme882=== CONT TestPendingCleanupUsesOneCutoff883=== RUN TestCommonName/O=x\,CN=evil,CN=real884=== PAUSE TestCommonName/O=x\,CN=evil,CN=real885=== RUN TestCommonName/CN=a\,b,O=x886=== PAUSE TestCommonName/CN=a\,b,O=x887=== RUN TestCommonName/CN=a\\,O=b888=== PAUSE TestCommonName/CN=a\\,O=b889=== RUN TestCommonName/CN=a+O=b890=== PAUSE TestCommonName/CN=a+O=b891=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded8922026/11/12 17:02:21 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:41401/oidc893--- PASS: TestPresentRefreshesOnlyStaleClosures (0.38s)894=== CONT TestMultipartCleanup895--- PASS: TestPinObjectsBackfill (0.42s)896=== CONT TestGCSweepSkipsPendingObjects8972026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/host-18982026/11/12 17:02:21 INFO Created/updated pin name=host-1 store_path=/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-test narinfo_key=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo8992026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/host-29002026/11/12 17:02:21 INFO Created/updated pin name=host-2 store_path=/nix/store/cccccccccccccccccccccccccccccccc-test narinfo_key=cccccccccccccccccccccccccccccccc.narinfo901--- PASS: TestPinObjectsOtherPin (0.45s)902=== CONT TestDeduplicatedObjectsRecordedAsPending9032026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/host-db-019042026/11/12 17:02:21 INFO Created/updated pin name=host-db-01 store_path=/nix/store/cccccccccccccccccccccccccccccccc-test narinfo_key=cccccccccccccccccccccccccccccccc.narinfo9052026/11/12 17:02:21 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"906--- PASS: TestPinScopedSubstringAttack (0.48s)907=== CONT TestPushDedupSurvivesConcurrentGC908--- PASS: TestPresentReportsOnlyClosureRoots (0.49s)909=== CONT TestGCMetrics9102026/11/12 17:02:21 INFO Received uploads request method=POST path=/api/pending_closures911--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (0.51s)912=== CONT TestGCBugBareHashReferences9132026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/host-19142026/11/12 17:02:21 INFO Created/updated pin name=host-1 store_path=/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-test narinfo_key=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo9152026/11/12 17:02:21 INFO Received delete pin request method=DELETE path=/api/pins/host-19162026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/host-web-019172026/11/12 17:02:21 INFO Created/updated pin name=host-web-01 store_path=/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-test narinfo_key=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo9182026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/host-db-019192026/11/12 17:02:21 INFO Deleted pin name=host-1920--- PASS: TestPinObjectsDelete (0.57s)921=== CONT TestGCAdvisoryLockBlocksConcurrentRun9222026/11/12 17:02:21 INFO Created/updated pin name=host-db-01 store_path=/nix/store/cccccccccccccccccccccccccccccccc-test narinfo_key=cccccccccccccccccccccccccccccccc.narinfo9232026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/host-web-019242026/11/12 17:02:21 INFO Created/updated pin name=host-web-01 store_path=/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-test narinfo_key=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo9252026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/host-db-019262026/11/12 17:02:21 INFO Created/updated pin name=host-db-01 store_path=/nix/store/cccccccccccccccccccccccccccccccc-test narinfo_key=cccccccccccccccccccccccccccccccc.narinfo9272026/11/12 17:02:21 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"9282026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/host-19292026/11/12 17:02:21 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"9302026/11/12 17:02:21 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"9312026/11/12 17:02:21 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"9322026/11/12 17:02:21 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"9332026/11/12 17:02:21 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"9342026/11/12 17:02:21 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"9352026/11/12 17:02:21 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"9362026/11/12 17:02:21 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"937--- PASS: TestPinScopedReads (0.62s)938=== CONT TestLeadEndsOnShutdown9392026/11/12 17:02:21 INFO Created/updated pin name=host-1 store_path=/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-test narinfo_key=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo9402026/11/12 17:02:21 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"9412026/11/12 17:02:21 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"942--- PASS: TestPinObjectsReachable (0.62s)943=== CONT TestLeadIncumbentAnnouncesAtOnce944--- PASS: TestPinScopedReadRedirect (0.62s)945=== CONT TestLeadUnannouncedStandbyYieldsToIncumbent9462026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/host-web-01947--- PASS: TestObjectStatsUpdatedOncePerCommit (0.67s)948=== CONT TestLeadElectsOneAndHandsOver9492026/11/12 17:02:21 INFO Created/updated pin name=host-web-01 store_path=/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-test narinfo_key=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo9502026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/host-db-019512026/11/12 17:02:21 INFO Received uploads request method=POST path=/api/pending_closures9522026/11/12 17:02:21 INFO Created/updated pin name=host-db-01 store_path=/nix/store/cccccccccccccccccccccccccccccccc-test narinfo_key=cccccccccccccccccccccccccccccccc.narinfo9532026/11/12 17:02:21 INFO Received uploads request method=POST path=/api/pending_closures9542026/11/12 17:02:21 WARN mTLS auth: subject not in bound subjects subject="CN=reader"955--- PASS: TestFullReaderUnaffected (0.70s)956=== CONT TestConnectWaitsForAPeerMigration9572026/11/12 17:02:21 INFO Received cleanup request method=DELETE path=/api/pending_closures9582026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux9592026/11/12 17:02:21 WARN Refused reserved pin name=worker-x86_64-linux9602026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux9612026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/my-app9622026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux963--- PASS: TestCreatePin_ReservedPins (0.74s)964=== CONT TestResolveDBConnectionString965=== RUN TestResolveDBConnectionString/flag_wins966=== PAUSE TestResolveDBConnectionString/flag_wins967=== RUN TestResolveDBConnectionString/file_when_flag_empty968=== PAUSE TestResolveDBConnectionString/file_when_flag_empty969=== RUN TestResolveDBConnectionString/missing_file_is_an_error970=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error971=== RUN TestResolveDBConnectionString/PGHOST_allows_empty972=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty973=== RUN TestResolveDBConnectionString/nothing_configured974=== PAUSE TestResolveDBConnectionString/nothing_configured975=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock9762026/11/12 17:02:21 INFO Received create pin request method=POST path=/api/pins/host-19772026/11/12 17:02:22 INFO Created/updated pin name=host-1 store_path=/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-test narinfo_key=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo9782026/11/12 17:02:22 INFO Received create pin request method=POST path=/api/pins/host-29792026/11/12 17:02:22 INFO Created/updated pin name=host-2 store_path=/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-test narinfo_key=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo9802026/11/12 17:02:22 INFO Received create pin request method=POST path=/api/pins/host-19812026/11/12 17:02:22 INFO Received create pin request method=POST path=/api/pins/host-19822026/11/12 17:02:22 INFO Created/updated pin name=host-1 store_path=/nix/store/cccccccccccccccccccccccccccccccc-test narinfo_key=cccccccccccccccccccccccccccccccc.narinfo9832026/11/12 17:02:22 INFO Created/updated pin name=host-1 store_path=/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-test narinfo_key=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo984--- PASS: TestPinObjectsSelfReference (0.80s)985=== CONT TestDeleteClosuresRacingPins986--- PASS: TestPinObjectsSharedTarget (0.80s)987=== CONT TestClientFallsBackToClosures9882026/11/12 17:02:22 INFO Received create pin request method=POST path=/api/pins/host-19892026/11/12 17:02:22 INFO Created/updated pin name=host-1 store_path=/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-test narinfo_key=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo9902026/11/12 17:02:22 INFO Received create pin request method=POST path=/api/pins/host-19912026/11/12 17:02:22 INFO Created/updated pin name=host-1 store_path=/nix/store/cccccccccccccccccccccccccccccccc-test narinfo_key=cccccccccccccccccccccccccccccccc.narinfo9922026/11/12 17:02:22 INFO Received create pin request method=POST path=/api/pins/host-19932026/11/12 17:02:22 INFO Received create pin request method=POST path=/api/pins/host-19942026/11/12 17:02:22 INFO Created/updated pin name=host-1 store_path=/nix/store/cccccccccccccccccccccccccccccccc-test narinfo_key=cccccccccccccccccccccccccccccccc.narinfo995--- PASS: TestPinObjectsSameTarget (0.87s)996=== CONT TestClientPushesUseOnePush9972026/11/12 17:02:22 INFO Created/updated pin name=host-1 store_path=/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-test narinfo_key=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo9982026/11/12 17:02:22 INFO Received create pin request method=POST path=/api/pins/host-19992026/11/12 17:02:22 INFO Received create pin request method=POST path=/api/pins/host-110002026/11/12 17:02:22 INFO Created/updated pin name=host-1 store_path=/nix/store/cccccccccccccccccccccccccccccccc-test narinfo_key=cccccccccccccccccccccccccccccccc.narinfo1001--- PASS: TestPinObjectsGrace (0.90s)1002=== CONT TestPinProtectsFromGC10032026/11/12 17:02:22 INFO Created/updated pin name=host-1 store_path=/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-test narinfo_key=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo10042026/11/12 17:02:22 INFO Received create pin request method=POST path=/api/pins/host-110052026/11/12 17:02:22 INFO Created/updated pin name=host-1 store_path=/nix/store/cccccccccccccccccccccccccccccccc-test narinfo_key=cccccccccccccccccccccccccccccccc.narinfo10062026/11/12 17:02:22 INFO Received create pin request method=POST path=/api/pins/host-110072026/11/12 17:02:22 INFO Received create pin request method=POST path=/api/pins/host-110082026/11/12 17:02:22 INFO Created/updated pin name=host-1 store_path=/nix/store/dddddddddddddddddddddddddddddddd-test narinfo_key=dddddddddddddddddddddddddddddddd.narinfo1009--- PASS: TestPinObjectsTwoMoves (0.92s)1010=== CONT TestTombstonedObjectOfferedWithoutWaiting10112026/11/12 17:02:22 INFO Created/updated pin name=host-1 store_path=/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-test narinfo_key=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo10122026/11/12 17:02:22 INFO Received create pin request method=POST path=/api/pins/host-110132026/11/12 17:02:22 INFO Received uploads request method=POST path=/api/pending_closures10142026/11/12 17:02:22 INFO Created/updated pin name=host-1 store_path=/nix/store/cccccccccccccccccccccccccccccccc-test narinfo_key=cccccccccccccccccccccccccccccccc.narinfo1015--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (0.95s)1016=== CONT TestClientSharedPathCommittedMidPush10172026/11/12 17:02:22 INFO deleted expired pin objects count=110182026/11/12 17:02:22 INFO Aborted multipart uploads count=0 kept=010192026/11/12 17:02:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010202026/11/12 17:02:22 INFO Vacuumed table table=pending_closures10212026/11/12 17:02:22 INFO Vacuumed table table=commit_receipts10222026/11/12 17:02:22 INFO Vacuumed table table=pending_objects10232026/11/12 17:02:22 INFO Vacuumed table table=multipart_uploads10242026/11/12 17:02:22 INFO Vacuumed table table=closures10252026/11/12 17:02:22 INFO Vacuumed table table=objects1026--- PASS: TestGCDeletesExpiredPinObjects (0.97s)1027=== CONT TestServerTLSConfig1028=== RUN TestServerTLSConfig/no_client_CA1029=== PAUSE TestServerTLSConfig/no_client_CA1030=== RUN TestServerTLSConfig/missing_CA_file1031=== PAUSE TestServerTLSConfig/missing_CA_file1032=== RUN TestServerTLSConfig/not_a_PEM_file1033=== PAUSE TestServerTLSConfig/not_a_PEM_file1034=== CONT TestClientWithDependencies10352026/11/12 17:02:22 INFO Received uploads request method=POST path=/api/pending_closures1036--- PASS: TestObjectStatsTrigger (1.01s)1037=== CONT TestClientMultipleUploads10382026/11/12 17:02:22 INFO Received uploads request method=POST path=/api/pending_closures10392026/11/12 17:02:22 INFO Aborted multipart uploads count=0 kept=010402026/11/12 17:02:22 INFO Received uploads request method=POST path=/api/pending_closures10412026/11/12 17:02:22 WARN Force mode enabled - objects will be deleted immediately without grace period1042--- PASS: TestDeduplicatedObjectsRecordedAsPending (0.61s)1043=== CONT TestService_NativeMTLS10442026/11/12 17:02:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010452026/11/12 17:02:22 INFO Vacuumed table table=pending_closures10462026/11/12 17:02:22 INFO Vacuumed table table=commit_receipts10472026/11/12 17:02:22 INFO Vacuumed table table=pending_objects10482026/11/12 17:02:22 INFO Vacuumed table table=multipart_uploads10492026/11/12 17:02:22 INFO Vacuumed table table=closures10502026/11/12 17:02:22 INFO Vacuumed table table=objects1051--- PASS: TestGCSweepSkipsPendingObjects (0.67s)1052=== CONT TestClientIntegration10532026/11/12 17:02:22 INFO Received cleanup request method=DELETE path=/api/pending_closures10542026/11/12 17:02:22 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket28/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""10552026/11/12 17:02:22 INFO Aborted multipart uploads count=0 kept=11056=== NAME TestOrphanedObjectsGC1057 orphaned_objects_gc_test.go:287: GC Test Summary:1058 orphaned_objects_gc_test.go:288: - Kept: 2 objects from closure A1059 orphaned_objects_gc_test.go:289: - Deleted: 2 objects from closure B1060 orphaned_objects_gc_test.go:290: - Deleted: 6 orphaned chain objects (X1->X2->X3)1061 orphaned_objects_gc_test.go:291: - Deleted: 2 orphaned single objects (Y)1062 orphaned_objects_gc_test.go:292: - Total deleted: 10 objects1063--- PASS: TestOrphanedObjectsGC (1.12s)1064=== CONT TestMetricsCommit10652026/11/12 17:02:22 INFO Received cleanup request method=DELETE path=/api/pending_closures10662026/11/12 17:02:22 INFO Aborted multipart uploads count=1 kept=01067--- PASS: TestMultipartCleanup (0.75s)1068=== CONT TestClientErrorHandling1069=== RUN TestClientErrorHandling/InvalidStorePath1070=== PAUSE TestClientErrorHandling/InvalidStorePath1071=== RUN TestClientErrorHandling/InvalidAuthToken1072=== PAUSE TestClientErrorHandling/InvalidAuthToken1073=== RUN TestClientErrorHandling/ServerNotAvailable1074=== PAUSE TestClientErrorHandling/ServerNotAvailable1075=== CONT TestMetricsInventory10762026/11/12 17:02:22 INFO Received uploads request method=POST path=/api/pending_closures10772026/11/12 17:02:22 INFO Aborted multipart uploads count=0 kept=010782026/11/12 17:02:22 WARN Force mode enabled - objects will be deleted immediately without grace period10792026/11/12 17:02:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010802026/11/12 17:02:22 INFO Vacuumed table table=pending_closures10812026/11/12 17:02:22 INFO Vacuumed table table=commit_receipts10822026/11/12 17:02:22 INFO Vacuumed table table=pending_objects10832026/11/12 17:02:22 INFO Vacuumed table table=multipart_uploads10842026/11/12 17:02:22 INFO Vacuumed table table=closures10852026/11/12 17:02:22 INFO Vacuumed table table=objects1086--- PASS: TestGCMetrics (0.68s)1087=== CONT TestClientCADerivations10882026/11/12 17:02:22 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=nixbld database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"1089--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (0.62s)1090=== CONT TestCacheStatsHandler10912026/11/12 17:02:22 INFO Aborted multipart uploads count=0 kept=010922026/11/12 17:02:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010932026/11/12 17:02:22 INFO Vacuumed table table=pending_closures10942026/11/12 17:02:22 INFO Vacuumed table table=commit_receipts10952026/11/12 17:02:22 INFO Vacuumed table table=pending_objects10962026/11/12 17:02:22 INFO Vacuumed table table=multipart_uploads10972026/11/12 17:02:22 INFO lead: acquired remote=192.0.2.1:123410982026/11/12 17:02:22 INFO Vacuumed table table=closures10992026/11/12 17:02:22 INFO Vacuumed table table=objects11002026/11/12 17:02:22 INFO Aborted multipart uploads count=0 kept=011012026/11/12 17:02:22 WARN Force mode enabled - objects will be deleted immediately without grace period11022026/11/12 17:02:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011032026/11/12 17:02:22 INFO Vacuumed table table=pending_closures11042026/11/12 17:02:22 INFO Vacuumed table table=commit_receipts11052026/11/12 17:02:22 INFO Vacuumed table table=pending_objects11062026/11/12 17:02:22 INFO Vacuumed table table=multipart_uploads11072026/11/12 17:02:22 INFO Vacuumed table table=closures11082026/11/12 17:02:22 INFO Vacuumed table table=objects1109--- PASS: TestPushDedupSurvivesConcurrentGC (0.77s)1110=== CONT TestNARDeduplicationMetadataUploadBug11112026/11/12 17:02:22 INFO lead: acquired remote=192.0.2.1:123411122026/11/12 17:02:22 INFO lead: acquired remote=192.0.2.1:123411132026/11/12 17:02:22 INFO lead: released remote=192.0.2.1:12341114--- PASS: TestLeadIncumbentAnnouncesAtOnce (0.70s)1115=== CONT TestCacheConfigHandler1116=== RUN TestCacheConfigHandler/full_config,_no_issuer1117=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1118=== RUN TestCacheConfigHandler/no_cache_url_configured1119=== PAUSE TestCacheConfigHandler/no_cache_url_configured1120=== RUN TestCacheConfigHandler/no_signing_keys1121=== PAUSE TestCacheConfigHandler/no_signing_keys1122=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1123=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1124=== CONT TestCreatePendingClosureRejectsOversizedNAR11252026/11/12 17:02:22 INFO Received uploads request method=POST path=/api/pending_closures1126--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.00s)1127=== CONT TestPinScopedPrincipal11282026/11/12 17:02:22 INFO lead: acquired remote=192.0.2.1:12341129--- PASS: TestGCBugBareHashReferences (0.89s)1130=== CONT TestService_ReadScope_PublicByDefault11312026/11/12 17:02:22 INFO Received uploads request method=POST path=/api/pending_closures1132--- PASS: TestTombstonedObjectOfferedWithoutWaiting (0.60s)1133=== CONT TestCacheConfigHandlerMaxNarSize1134--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)1135=== CONT TestService_RequireScope_OIDC11362026/11/12 17:02:22 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:42653/oidc1137=== NAME TestPinProtectsFromGC1138 client_integration_test.go:791: Pinned store path: /build/TestPinProtectsFromGC3127227237/001/store/p5lnx5wi030ybq16wdnk00k10a1qy9ym-pinned-file.txt1139 client_integration_test.go:792: Unpinned store path: /build/TestPinProtectsFromGC3127227237/001/store/29bcl3s8x4ngyfmlkf7f6p22x7fcvjcs-unpinned-file.txt1140--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (0.85s)1141=== CONT TestService_AuthMiddleware_OIDC11422026/11/12 17:02:22 WARN mTLS auth: subject not in bound subjects subject="CN=reader"11432026/11/12 17:02:22 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1144--- PASS: TestService_NativeMTLS (0.55s)1145=== CONT TestService_ReadAuthMiddleware11462026/11/12 17:02:22 INFO Received uploads request method=POST path=/api/pending_closures1147=== NAME TestClientMultipleUploads1148 client_integration_test.go:406: Created store path 0: /build/TestClientMultipleUploads1006228265/001/store/3a8vp2ycv249vbzwylarn00y5z60a3yk-test-file-0.txt11492026/11/12 17:02:22 INFO Received uploads request method=POST path=/api/pending_closures11502026/11/12 17:02:22 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)11512026/11/12 17:02:22 INFO Uploading dzpi2gznzda39cffl8h334h8cffkydkd-b (216B)11522026/11/12 17:02:22 INFO Uploading a347d9spsq97pjmi67x8wmh5l5ri5yp3-shared-dep (136B)11532026/11/12 17:02:22 WARN Failed to register uploaded object key=7nw5i6dzrn49l4baxrx4kipp2n9afmjs.ls error="server returned 404: 404 page not found\n"11542026/11/12 17:02:22 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"11552026/11/12 17:02:22 WARN Failed to register uploaded object key=nar/0frw212xhq0sayj1yxgljw15zqavlgmvc66q66awwhkc3d45550d.nar.zst error="server returned 404: 404 page not found\n"11562026/11/12 17:02:22 INFO Received push request method=POST path=/api/pushes11572026/11/12 17:02:22 WARN Failed to register uploaded object key=a347d9spsq97pjmi67x8wmh5l5ri5yp3.ls error="server returned 404: 404 page not found\n"11582026/11/12 17:02:22 WARN Failed to register uploaded object key=dzpi2gznzda39cffl8h334h8cffkydkd.ls error="server returned 404: 404 page not found\n"11592026/11/12 17:02:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign11602026/11/12 17:02:22 INFO Signed narinfos id=1 count=211612026/11/12 17:02:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign11622026/11/12 17:02:22 INFO Signed narinfos id=2 count=211632026/11/12 17:02:22 INFO Uploading 4 narinfos11642026/11/12 17:02:22 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)11652026/11/12 17:02:22 INFO Uploading p5lnx5wi030ybq16wdnk00k10a1qy9ym-pinned-file.txt (128B)1166=== NAME TestClientWithDependencies1167 client_integration_test.go:661: Built derivation: /build/TestClientWithDependencies2076424035/001/store/8zhnkn17ylqq0vvvqz54xziy94c22ldy-test-script11682026/11/12 17:02:22 INFO Received push request method=POST path=/api/pushes11692026/11/12 17:02:22 WARN Failed to register uploaded object key=a347d9spsq97pjmi67x8wmh5l5ri5yp3.narinfo error="server returned 404: 404 page not found\n"11702026/11/12 17:02:22 WARN Failed to register uploaded object key=dzpi2gznzda39cffl8h334h8cffkydkd.narinfo error="server returned 404: 404 page not found\n"11712026/11/12 17:02:22 WARN Failed to register uploaded object key=7nw5i6dzrn49l4baxrx4kipp2n9afmjs.narinfo error="server returned 404: 404 page not found\n"11722026/11/12 17:02:22 WARN Failed to register uploaded object key=p5lnx5wi030ybq16wdnk00k10a1qy9ym.ls error="server returned 404: 404 page not found\n"11732026/11/12 17:02:22 INFO Received push request method=POST path=/api/pushes11742026/11/12 17:02:22 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign11752026/11/12 17:02:22 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete11762026/11/12 17:02:22 WARN Failed to register uploaded object key=a347d9spsq97pjmi67x8wmh5l5ri5yp3.narinfo error="server returned 404: 404 page not found\n"11772026/11/12 17:02:22 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"11782026/11/12 17:02:22 INFO Signed narinfos id=1 count=111792026/11/12 17:02:22 INFO Uploading 1 narinfos1180=== NAME TestClientMultipleUploads11812026/11/12 17:02:22 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)1182 client_integration_test.go:406: Created store path 1: /build/TestClientMultipleUploads1006228265/001/store/k1ldxdb293fanxinfjigflglc9rik2sw-test-file-1.txt11832026/11/12 17:02:22 INFO Uploading 7nac3ydhj14r6dmdx6qwhy3snx60knrf-b (216B)11842026/11/12 17:02:22 INFO Received complete push request method=POST path=/api/pushes/1/complete11852026/11/12 17:02:22 INFO Uploading 94mqzxa32jkvn1005z2l0dpb2rb79zd6-shared-dep (136B)11862026/11/12 17:02:22 WARN Failed to register uploaded object key=p5lnx5wi030ybq16wdnk00k10a1qy9ym.narinfo error="server returned 404: 404 page not found\n"11872026/11/12 17:02:22 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:33357/oidc11882026/11/12 17:02:22 INFO Completed upload id=1 objects=6 duration=9.260598ms11892026/11/12 17:02:22 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete11902026/11/12 17:02:22 INFO Received complete push request method=POST path=/api/pushes/1/complete11912026/11/12 17:02:22 WARN Failed to register uploaded object key=3xph01hvh3kq3nakg1qcp2d02zc26i5z.ls error="server returned 404: 404 page not found\n"11922026/11/12 17:02:22 INFO Completed upload id=2 objects=6 duration=1.70855ms11932026/11/12 17:02:22 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"11942026/11/12 17:02:22 INFO Upload complete. (85ms)1195=== NAME TestClientFallsBackToClosures1196 client_pushes_test.go:112: Retrieved narinfo from S3:1197 StorePath: /build/TestClientFallsBackToClosures2866616703/001/store/a347d9spsq97pjmi67x8wmh5l5ri5yp3-shared-dep1198 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1199 Compression: zstd1200 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821201 NarSize: 1361202 References: 1203 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n12042026/11/12 17:02:22 WARN Failed to register uploaded object key=94mqzxa32jkvn1005z2l0dpb2rb79zd6.ls error="server returned 404: 404 page not found\n"12052026/11/12 17:02:22 WARN Failed to register uploaded object key=7nac3ydhj14r6dmdx6qwhy3snx60knrf.ls error="server returned 404: 404 page not found\n"12062026/11/12 17:02:22 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12072026/11/12 17:02:22 WARN Failed to register uploaded object key=nar/0r59m3hlrhhrks5cjasifjr6sm5vnizjxijwxixgffsrczi8lzgg.nar.zst error="server returned 404: 404 page not found\n"12082026/11/12 17:02:22 INFO Signed narinfos id=1 count=312092026/11/12 17:02:22 INFO Uploading 3 narinfos1210 client_pushes_test.go:112: Retrieved narinfo from S3:1211 StorePath: /build/TestClientFallsBackToClosures2866616703/001/store/7nw5i6dzrn49l4baxrx4kipp2n9afmjs-a1212 URL: nar/0frw212xhq0sayj1yxgljw15zqavlgmvc66q66awwhkc3d45550d.nar.zst1213 Compression: zstd1214 NarHash: sha256:0frw212xhq0sayj1yxgljw15zqavlgmvc66q66awwhkc3d45550d1215 NarSize: 2161216 References: /build/TestClientFallsBackToClosures2866616703/001/store/a347d9spsq97pjmi67x8wmh5l5ri5yp3-shared-dep1217 CA: text:sha256:060h8sj9j3g6baqcv748g1b8c3c39mlxa7x8a0h0ngam0d9f40n21218=== NAME TestClientIntegration1219 client_integration_test.go:292: Created store path: /build/TestClientIntegration1252145864/002/store/4bz49f32pa3hcvxkg9xfk9d3jj2pm9dv-test-file.txt12202026/11/12 17:02:22 INFO Completed push id=1 objects=3 duration=11.461998ms12212026/11/12 17:02:22 INFO Upload complete. (68ms)12222026/11/12 17:02:22 INFO Completed push id=1 objects=4 duration=7.047569ms1223=== NAME TestClientFallsBackToClosures1224 client_pushes_test.go:112: Retrieved narinfo from S3:1225 StorePath: /build/TestClientFallsBackToClosures2866616703/001/store/dzpi2gznzda39cffl8h334h8cffkydkd-b1226 URL: nar/0frw212xhq0sayj1yxgljw15zqavlgmvc66q66awwhkc3d45550d.nar.zst1227 Compression: zstd1228 NarHash: sha256:0frw212xhq0sayj1yxgljw15zqavlgmvc66q66awwhkc3d45550d1229 NarSize: 2161230 References: /build/TestClientFallsBackToClosures2866616703/001/store/a347d9spsq97pjmi67x8wmh5l5ri5yp3-shared-dep1231 CA: text:sha256:060h8sj9j3g6baqcv748g1b8c3c39mlxa7x8a0h0ngam0d9f40n212322026/11/12 17:02:22 WARN Failed to register uploaded object key=3xph01hvh3kq3nakg1qcp2d02zc26i5z.narinfo error="server returned 404: 404 page not found\n"12332026/11/12 17:02:22 INFO Received complete push request method=POST path=/api/pushes/1/complete12342026/11/12 17:02:22 WARN Failed to register uploaded object key=7nac3ydhj14r6dmdx6qwhy3snx60knrf.narinfo error="server returned 404: 404 page not found\n"12352026/11/12 17:02:22 WARN Failed to register uploaded object key=94mqzxa32jkvn1005z2l0dpb2rb79zd6.narinfo error="server returned 404: 404 page not found\n"12362026/11/12 17:02:22 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete12372026/11/12 17:02:22 INFO Completed upload id=2 objects=2 duration=1.08874ms1238=== NAME TestClientWithDependencies1239 client_integration_test.go:663: Found 1 dependencies (including self)1240--- PASS: TestClientFallsBackToClosures (0.89s)1241=== CONT TestValidatePinReadNeedsWriteSubjects1242--- PASS: TestValidatePinReadNeedsWriteSubjects (0.00s)1243=== CONT TestService_AuthMiddleware_MTLSBoundSubjects1244--- PASS: TestMetricsCommit (0.58s)1245=== CONT TestEnsureAbortUploadsRule12462026/11/12 17:02:22 INFO Received push request method=POST path=/api/pushes12472026/11/12 17:02:22 INFO Completed push id=1 objects=8 duration=14.721637ms12482026/11/12 17:02:22 INFO Upload complete. (74ms)1249=== NAME TestClientPushesUseOnePush1250 client_pushes_test.go:97: Retrieved narinfo from S3:1251 StorePath: /build/TestClientPushesUseOnePush2069659192/001/store/94mqzxa32jkvn1005z2l0dpb2rb79zd6-shared-dep12522026/11/12 17:02:22 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)1253 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1254 Compression: zstd12552026/11/12 17:02:22 INFO Uploading l66rw987583s2grxplhk3kfjgnp5ff7d-top (224B)1256 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821257 NarSize: 1361258 References: 12592026/11/12 17:02:22 INFO Uploading frbr7jcjavmc1jblyzzxbl6ybswg5g1b-shared-dep (136B)1260 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1261 client_pushes_test.go:97: Retrieved narinfo from S3:1262 StorePath: /build/TestClientPushesUseOnePush2069659192/001/store/3xph01hvh3kq3nakg1qcp2d02zc26i5z-a1263 URL: nar/0r59m3hlrhhrks5cjasifjr6sm5vnizjxijwxixgffsrczi8lzgg.nar.zst1264 Compression: zstd1265 NarHash: sha256:0r59m3hlrhhrks5cjasifjr6sm5vnizjxijwxixgffsrczi8lzgg1266 NarSize: 2161267 References: /build/TestClientPushesUseOnePush2069659192/001/store/94mqzxa32jkvn1005z2l0dpb2rb79zd6-shared-dep1268 CA: text:sha256:150fha5mdkwhcgqs0mbwvx2q74kiycsdsz8nrb8bdklf7k9cdxmy1269=== NAME TestClientMultipleUploads1270 client_integration_test.go:406: Created store path 2: /build/TestClientMultipleUploads1006228265/001/store/i4ryzlnanbgq9091igwcp8312dgrwbkl-test-file-2.txt12712026/11/12 17:02:22 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"1272--- PASS: TestMetricsInventory (0.58s)1273=== CONT TestGenerateLandingPage1274=== NAME TestClientPushesUseOnePush1275 client_pushes_test.go:97: Retrieved narinfo from S3:1276 StorePath: /build/TestClientPushesUseOnePush2069659192/001/store/7nac3ydhj14r6dmdx6qwhy3snx60knrf-b1277 URL: nar/0r59m3hlrhhrks5cjasifjr6sm5vnizjxijwxixgffsrczi8lzgg.nar.zst1278 Compression: zstd1279 NarHash: sha256:0r59m3hlrhhrks5cjasifjr6sm5vnizjxijwxixgffsrczi8lzgg12802026/11/12 17:02:22 WARN Failed to register uploaded object key=frbr7jcjavmc1jblyzzxbl6ybswg5g1b.ls error="server returned 404: 404 page not found\n"1281 NarSize: 2161282 References: /build/TestClientPushesUseOnePush2069659192/001/store/94mqzxa32jkvn1005z2l0dpb2rb79zd6-shared-dep1283 CA: text:sha256:150fha5mdkwhcgqs0mbwvx2q74kiycsdsz8nrb8bdklf7k9cdxmy12842026/11/12 17:02:22 WARN Failed to register uploaded object key=l66rw987583s2grxplhk3kfjgnp5ff7d.ls error="server returned 404: 404 page not found\n"12852026/11/12 17:02:22 WARN Failed to register uploaded object key=nar/18k4b4aw3laml8vgphhzrs6jm9v5vzb7ysidhw641m92a7bk0arx.nar.zst error="server returned 404: 404 page not found\n"12862026/11/12 17:02:22 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12872026/11/12 17:02:22 INFO Signed narinfos id=1 count=212882026/11/12 17:02:22 INFO Uploading 2 narinfos1289--- PASS: TestGenerateLandingPage (0.00s)1290=== CONT TestService_AuthMiddleware_MTLSProxyHeader1291--- PASS: TestClientPushesUseOnePush (0.85s)12922026/11/12 17:02:22 WARN Failed to register uploaded object key=frbr7jcjavmc1jblyzzxbl6ybswg5g1b.narinfo error="server returned 404: 404 page not found\n"1293=== CONT TestService_readinessHandler12942026/11/12 17:02:22 INFO Received complete push request method=POST path=/api/pushes/1/complete12952026/11/12 17:02:22 WARN Failed to register uploaded object key=l66rw987583s2grxplhk3kfjgnp5ff7d.narinfo error="server returned 404: 404 page not found\n"12962026/11/12 17:02:22 INFO Completed push id=1 objects=6 duration=11.589437ms12972026/11/12 17:02:22 INFO Upload complete. (72ms)1298=== NAME TestClientSharedPathCommittedMidPush1299 client_integration_test.go:740: Retrieved narinfo from S3:1300 StorePath: /build/TestClientSharedPathCommittedMidPush1694615066/001/store/frbr7jcjavmc1jblyzzxbl6ybswg5g1b-shared-dep1301 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1302 Compression: zstd1303 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821304 NarSize: 1361305 References: 1306 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1307 client_integration_test.go:740: Retrieved narinfo from S3:1308 StorePath: /build/TestClientSharedPathCommittedMidPush1694615066/001/store/l66rw987583s2grxplhk3kfjgnp5ff7d-top1309 URL: nar/18k4b4aw3laml8vgphhzrs6jm9v5vzb7ysidhw641m92a7bk0arx.nar.zst1310 Compression: zstd1311 NarHash: sha256:18k4b4aw3laml8vgphhzrs6jm9v5vzb7ysidhw641m92a7bk0arx1312 NarSize: 2241313 References: /build/TestClientSharedPathCommittedMidPush1694615066/001/store/frbr7jcjavmc1jblyzzxbl6ybswg5g1b-shared-dep1314 CA: text:sha256:0365h1kv71845jiiiqzimplyfyf0pavcy80y6176s2ifgacbb6fl1315--- PASS: TestClientSharedPathCommittedMidPush (0.79s)1316=== CONT TestService_AuthMiddleware1317--- PASS: TestCacheStatsHandler (0.54s)1318=== CONT TestService_healthCheckHandler13192026/11/12 17:02:22 INFO Received push request method=POST path=/api/pushes13202026/11/12 17:02:22 INFO Received push request method=POST path=/api/pushes13212026/11/12 17:02:22 INFO Received push request method=POST path=/api/pushes13222026/11/12 17:02:22 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)13232026/11/12 17:02:22 INFO Uploading 4bz49f32pa3hcvxkg9xfk9d3jj2pm9dv-test-file.txt (152B)13242026/11/12 17:02:22 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)13252026/11/12 17:02:22 INFO Uploading 29bcl3s8x4ngyfmlkf7f6p22x7fcvjcs-unpinned-file.txt (128B)13262026/11/12 17:02:23 WARN Failed to register uploaded object key=4bz49f32pa3hcvxkg9xfk9d3jj2pm9dv.ls error="server returned 404: 404 page not found\n"13272026/11/12 17:02:23 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13282026/11/12 17:02:23 INFO Signed narinfos id=1 count=113292026/11/12 17:02:23 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"13302026/11/12 17:02:23 INFO Uploading 1 narinfos13312026/11/12 17:02:23 WARN Failed to register uploaded object key=29bcl3s8x4ngyfmlkf7f6p22x7fcvjcs.ls error="server returned 404: 404 page not found\n"13322026/11/12 17:02:23 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign13332026/11/12 17:02:23 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"13342026/11/12 17:02:23 INFO Signed narinfos id=2 count=113352026/11/12 17:02:23 INFO Uploading 1 narinfos13362026/11/12 17:02:23 INFO Received complete push request method=POST path=/api/pushes/1/complete13372026/11/12 17:02:23 WARN Failed to register uploaded object key=4bz49f32pa3hcvxkg9xfk9d3jj2pm9dv.narinfo error="server returned 404: 404 page not found\n"13382026/11/12 17:02:23 INFO Received complete push request method=POST path=/api/pushes/2/complete13392026/11/12 17:02:23 WARN Failed to register uploaded object key=29bcl3s8x4ngyfmlkf7f6p22x7fcvjcs.narinfo error="server returned 404: 404 page not found\n"13402026/11/12 17:02:23 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)13412026/11/12 17:02:23 INFO Uploading 8zhnkn17ylqq0vvvqz54xziy94c22ldy-test-script (136B)13422026/11/12 17:02:23 WARN Failed to register uploaded object key=8zhnkn17ylqq0vvvqz54xziy94c22ldy.ls error="server returned 404: 404 page not found\n"13432026/11/12 17:02:23 INFO Completed push id=2 objects=3 duration=8.059069ms13442026/11/12 17:02:23 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"13452026/11/12 17:02:23 INFO Upload complete. (68ms)13462026/11/12 17:02:23 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"13472026/11/12 17:02:23 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"13482026/11/12 17:02:23 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"13492026/11/12 17:02:23 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"13502026/11/12 17:02:23 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"13512026/11/12 17:02:23 WARN mTLS auth: subject not in bound subjects subject="CN=web-01"13522026/11/12 17:02:23 WARN mTLS auth: subject not in bound subjects subject="CN=bad/name"13532026/11/12 17:02:23 WARN mTLS auth: subject not in bound subjects subject="CN=bad/name"1354--- PASS: TestPinScopedPrincipal (0.47s)1355=== CONT TestGracefulShutdownDrainsInflight13562026/11/12 17:02:23 INFO Starting HTTP server address=127.0.0.1:4556313572026/11/12 17:02:23 INFO Shutdown signal received, draining in-flight requests timeout=10s13582026/11/12 17:02:23 WARN Failed to register uploaded object key=log/vwyr9hsvg9fcpj9w43q7wh1i7cldw88p-test-script.drv error="server returned 404: 404 page not found\n"13592026/11/12 17:02:23 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13602026/11/12 17:02:23 INFO Signed narinfos id=1 count=113612026/11/12 17:02:23 INFO Uploading 1 narinfos13622026/11/12 17:02:23 INFO Completed push id=1 objects=3 duration=19.463977ms13632026/11/12 17:02:23 INFO Upload complete. (77ms)13642026/11/12 17:02:23 INFO Received complete push request method=POST path=/api/pushes/1/complete13652026/11/12 17:02:23 WARN Failed to register uploaded object key=8zhnkn17ylqq0vvvqz54xziy94c22ldy.narinfo error="server returned 404: 404 page not found\n"1366=== NAME TestClientCADerivations1367 client_ca_test.go:136: Built CA derivation: /build/TestClientCADerivations2429263468/001/store/7d12p7nayny3fkm54j6lywaiq6mnzx1m-ca-test13682026/11/12 17:02:23 INFO Received push request method=POST path=/api/pushes13692026/11/12 17:02:23 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)13702026/11/12 17:02:23 INFO Uploading k1ldxdb293fanxinfjigflglc9rik2sw-test-file-1.txt (160B)13712026/11/12 17:02:23 INFO Uploading 3a8vp2ycv249vbzwylarn00y5z60a3yk-test-file-0.txt (160B)13722026/11/12 17:02:23 INFO Uploading i4ryzlnanbgq9091igwcp8312dgrwbkl-test-file-2.txt (160B)1373=== NAME TestNARDeduplicationMetadataUploadBug1374 metadata_upload_test.go:48: First store path: /build/TestNARDeduplicationMetadataUploadBug635060622/001/store/sxjq3l2pq7fy6c2xsd3n5g9sbn8wwpa7-file1.txt13752026/11/12 17:02:23 INFO Completed push id=1 objects=4 duration=12.660777ms13762026/11/12 17:02:23 INFO Upload complete. (86ms)13772026/11/12 17:02:23 WARN Failed to register uploaded object key=k1ldxdb293fanxinfjigflglc9rik2sw.ls error="server returned 404: 404 page not found\n"13782026/11/12 17:02:23 WARN Failed to register uploaded object key=3a8vp2ycv249vbzwylarn00y5z60a3yk.ls error="server returned 404: 404 page not found\n"13792026/11/12 17:02:23 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"13802026/11/12 17:02:23 WARN Failed to register uploaded object key=i4ryzlnanbgq9091igwcp8312dgrwbkl.ls error="server returned 404: 404 page not found\n"13812026/11/12 17:02:23 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"13822026/11/12 17:02:23 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13832026/11/12 17:02:23 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"13842026/11/12 17:02:23 INFO Signed narinfos id=1 count=313852026/11/12 17:02:23 INFO Uploading 3 narinfos1386--- PASS: TestService_ReadScope_PublicByDefault (0.42s)1387=== CONT TestGCTaskStore_Fail1388--- PASS: TestGCTaskStore_Fail (0.00s)1389=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit13902026/11/12 17:02:23 WARN Failed to register uploaded object key=k1ldxdb293fanxinfjigflglc9rik2sw.narinfo error="server returned 404: 404 page not found\n"13912026/11/12 17:02:23 WARN Failed to register uploaded object key=i4ryzlnanbgq9091igwcp8312dgrwbkl.narinfo error="server returned 404: 404 page not found\n"13922026/11/12 17:02:23 INFO Received complete push request method=POST path=/api/pushes/1/complete13932026/11/12 17:02:23 WARN Failed to register uploaded object key=3a8vp2ycv249vbzwylarn00y5z60a3yk.narinfo error="server returned 404: 404 page not found\n"13942026/11/12 17:02:23 INFO Aborted multipart uploads count=0 kept=013952026/11/12 17:02:23 INFO Starting cleanup of old closures method=DELETE path=/api/closures13962026/11/12 17:02:23 INFO Garbage collection started13972026/11/12 17:02:23 INFO Completed push id=1 objects=9 duration=13.579657ms13982026/11/12 17:02:23 INFO Upload complete. (92ms)1399=== NAME TestClientMultipleUploads1400 client_integration_test.go:417: Uploaded 3 paths in 126.426035ms14012026/11/12 17:02:23 WARN Force mode enabled - objects will be deleted immediately without grace period14022026/11/12 17:02:23 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=014032026/11/12 17:02:23 INFO Vacuumed table table=pending_closures1404=== NAME TestClientCADerivations1405 client_ca_test.go:139: Found 1 dependencies (including self)1406=== CONT TestGCTaskStore_PhaseUpdates1407=== CONT TestGCTaskStore_CompletedAllowsNewTask1408--- PASS: TestClientMultipleUploads (0.84s)1409--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)1410--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)1411=== CONT TestCompleteMultipartUnregistered14122026/11/12 17:02:23 INFO Aborted multipart uploads count=0 kept=014132026/11/12 17:02:23 INFO All 1 paths already cached14142026/11/12 17:02:23 INFO Vacuumed table table=commit_receipts1415=== NAME TestClientIntegration1416 client_integration_test.go:318: Retrieved narinfo from S3:1417 StorePath: /build/TestClientIntegration1252145864/002/store/4bz49f32pa3hcvxkg9xfk9d3jj2pm9dv-test-file.txt1418 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1419 Compression: zstd1420 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11421 NarSize: 1521422 References: 1423 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11424--- PASS: TestGracefulShutdownDrainsInflight (0.07s)1425=== CONT TestService_verifyS3ReuploadsStaleNarinfo1426=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1427=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1428=== RUN TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1429=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1430=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1431=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1432=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1433=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1434=== NAME TestClientIntegration1435 client_integration_test.go:319: Retrieved .ls file from S3 (compressed size: 77 bytes)1436=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1437=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1438=== NAME TestClientIntegration1439 client_integration_test.go:319: Decompressed .ls content (64 bytes):1440 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}1441=== CONT TestService_verifyS3Integrity14422026/11/12 17:02:23 WARN Force mode enabled - objects will be deleted immediately without grace period14432026/11/12 17:02:23 INFO Vacuumed table table=pending_objects14442026/11/12 17:02:23 INFO Vacuumed table table=multipart_uploads14452026/11/12 17:02:23 INFO Vacuumed table table=closures14462026/11/12 17:02:23 INFO Vacuumed table table=objects1447=== NAME TestClientWithDependencies1448 client_integration_test.go:677: Skipping nix copy test - isolated store (/build/TestClientWithDependencies2076424035/001/store) requires matching store prefix1449=== RUN TestService_RequireScope_OIDC/builder_may_write1450=== PAUSE TestService_RequireScope_OIDC/builder_may_write1451=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1452=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1453=== RUN TestService_RequireScope_OIDC/ops_may_admin1454=== PAUSE TestService_RequireScope_OIDC/ops_may_admin1455=== RUN TestService_RequireScope_OIDC/ops_may_not_write1456=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1457=== RUN TestService_RequireScope_OIDC/reader_may_not_write1458=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1459=== RUN TestService_RequireScope_OIDC/static_token_may_admin1460=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1461=== RUN TestService_RequireScope_OIDC/static_token_may_write1462=== PAUSE TestService_RequireScope_OIDC/static_token_may_write1463=== RUN TestService_RequireScope_OIDC/reader_may_read1464=== PAUSE TestService_RequireScope_OIDC/reader_may_read1465=== RUN TestService_RequireScope_OIDC/writer_implies_read1466=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1467=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1468=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read1469=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc1470=== PAUSE TestService_RequireScope_OIDC/reader_may_read_with_netrc1471=== RUN TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1472=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1473=== RUN TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1474=== PAUSE TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1475=== RUN TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1476=== PAUSE TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1477=== CONT TestService_createPendingClosureHandler1478--- PASS: TestClientWithDependencies (0.90s)1479=== CONT TestService_cleanupPendingClosuresHandler1480--- PASS: TestService_ReadAuthMiddleware (0.26s)1481=== CONT TestUploadHandlersRejectOversizedBody14822026/11/12 17:02:23 INFO Received push request method=POST path=/api/pushes14832026/11/12 17:02:23 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)14842026/11/12 17:02:23 INFO Uploading sxjq3l2pq7fy6c2xsd3n5g9sbn8wwpa7-file1.txt (160B)14852026/11/12 17:02:23 INFO Received push request method=POST path=/api/pushes14862026/11/12 17:02:23 INFO Received push request method=POST path=/api/pushes1487=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token1488=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token1489=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1490=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1491=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1492=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1493=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1494=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1495=== CONT TestGCTaskStore_GetReturnsLatest1496--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)1497=== CONT TestUploadHandlersRejectInvalidKeys1498=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1499=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1500=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal1501=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal1502=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key1503=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key1504=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key1505=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key1506=== CONT TestIsValidUploadKey1507=== RUN TestIsValidUploadKey/narinfo1508=== PAUSE TestIsValidUploadKey/narinfo1509=== RUN TestIsValidUploadKey/nar_zst1510=== PAUSE TestIsValidUploadKey/nar_zst1511=== RUN TestIsValidUploadKey/nar_xz1512=== PAUSE TestIsValidUploadKey/nar_xz1513=== RUN TestIsValidUploadKey/nar_plain1514=== PAUSE TestIsValidUploadKey/nar_plain1515=== RUN TestIsValidUploadKey/listing1516=== PAUSE TestIsValidUploadKey/listing1517=== RUN TestIsValidUploadKey/build_log1518=== PAUSE TestIsValidUploadKey/build_log1519=== RUN TestIsValidUploadKey/build_log_home-manager_file1520=== PAUSE TestIsValidUploadKey/build_log_home-manager_file1521=== RUN TestIsValidUploadKey/build_log_plus_in_name1522=== PAUSE TestIsValidUploadKey/build_log_plus_in_name1523=== RUN TestIsValidUploadKey/build_log_question_mark1524=== PAUSE TestIsValidUploadKey/build_log_question_mark1525=== RUN TestIsValidUploadKey/build_log_equals1526=== PAUSE TestIsValidUploadKey/build_log_equals1527=== RUN TestIsValidUploadKey/realisation1528=== PAUSE TestIsValidUploadKey/realisation1529=== RUN TestIsValidUploadKey/realisation_plus_in_output15302026/11/12 17:02:23 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=01531=== PAUSE TestIsValidUploadKey/realisation_plus_in_output1532=== RUN TestIsValidUploadKey/nix-cache-info1533=== PAUSE TestIsValidUploadKey/nix-cache-info1534=== RUN TestIsValidUploadKey/index.html1535=== PAUSE TestIsValidUploadKey/index.html1536=== RUN TestIsValidUploadKey/narinfo_key,_nar_type1537=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type1538=== RUN TestIsValidUploadKey/nar_key,_narinfo_type1539=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type1540=== RUN TestIsValidUploadKey/listing_key,_narinfo_type1541=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type1542=== RUN TestIsValidUploadKey/traversal1543=== PAUSE TestIsValidUploadKey/traversal1544=== RUN TestIsValidUploadKey/traversal_nar1545=== PAUSE TestIsValidUploadKey/traversal_nar1546=== RUN TestIsValidUploadKey/absolute1547=== PAUSE TestIsValidUploadKey/absolute1548=== RUN TestIsValidUploadKey/empty_key1549=== PAUSE TestIsValidUploadKey/empty_key1550=== RUN TestIsValidUploadKey/unknown_type1551=== PAUSE TestIsValidUploadKey/unknown_type1552=== CONT TestPendingClosureWriteTimeout1553=== RUN TestPendingClosureWriteTimeout/empty15542026/11/12 17:02:23 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)1555=== PAUSE TestPendingClosureWriteTimeout/empty15562026/11/12 17:02:23 INFO Object in database but missing from S3 key=4bz49f32pa3hcvxkg9xfk9d3jj2pm9dv.ls1557=== RUN TestPendingClosureWriteTimeout/400_objects15582026/11/12 17:02:23 WARN Found objects in DB but missing from S3, will re-upload count=11559=== PAUSE TestPendingClosureWriteTimeout/400_objects15602026/11/12 17:02:23 INFO Uploading 7d12p7nayny3fkm54j6lywaiq6mnzx1m-ca-test (144B)1561=== RUN TestPendingClosureWriteTimeout/670k_objects1562=== PAUSE TestPendingClosureWriteTimeout/670k_objects1563=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1564=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline15652026/11/12 17:02:23 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)1566=== CONT TestProxyWriteTimeout1567=== RUN TestProxyWriteTimeout/narinfo1568=== PAUSE TestProxyWriteTimeout/narinfo1569=== RUN TestProxyWriteTimeout/1_GiB_nar1570=== PAUSE TestProxyWriteTimeout/1_GiB_nar1571=== RUN TestProxyWriteTimeout/10_GiB_nar1572=== PAUSE TestProxyWriteTimeout/10_GiB_nar1573=== RUN TestProxyWriteTimeout/unknown_size1574=== PAUSE TestProxyWriteTimeout/unknown_size1575=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle15762026/11/12 17:02:23 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"15772026/11/12 17:02:23 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15782026/11/12 17:02:23 WARN Failed to register uploaded object key=sxjq3l2pq7fy6c2xsd3n5g9sbn8wwpa7.ls error="server returned 404: 404 page not found\n"15792026/11/12 17:02:23 INFO Signed narinfos id=1 count=115802026/11/12 17:02:23 INFO Uploading 1 narinfos15812026/11/12 17:02:23 INFO Vacuumed table table=pending_closures15822026/11/12 17:02:23 INFO Vacuumed table table=commit_receipts15832026/11/12 17:02:23 INFO Received complete push request method=POST path=/api/pushes/2/complete15842026/11/12 17:02:23 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"15852026/11/12 17:02:23 WARN Failed to register uploaded object key=7d12p7nayny3fkm54j6lywaiq6mnzx1m.ls error="server returned 404: 404 page not found\n"15862026/11/12 17:02:23 WARN Failed to register uploaded object key=4bz49f32pa3hcvxkg9xfk9d3jj2pm9dv.ls error="server returned 404: 404 page not found\n"15872026/11/12 17:02:23 WARN Failed to register uploaded object key=log/f47yahcaxdbx81g7mq4hs3gafnnnq3p9-ca-test.drv error="server returned 404: 404 page not found\n"15882026/11/12 17:02:23 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15892026/11/12 17:02:23 INFO Signed narinfos id=1 count=115902026/11/12 17:02:23 INFO Uploading 1 narinfos15912026/11/12 17:02:23 WARN Failed to register uploaded object key=sxjq3l2pq7fy6c2xsd3n5g9sbn8wwpa7.narinfo error="server returned 404: 404 page not found\n"15922026/11/12 17:02:23 INFO Received complete push request method=POST path=/api/pushes/1/complete15932026/11/12 17:02:23 INFO Completed push id=2 objects=3 duration=2.656849ms15942026/11/12 17:02:23 INFO Upload complete. (147ms)15952026/11/12 17:02:23 INFO Vacuumed table table=pending_objects1596=== NAME TestClientIntegration1597 client_integration_test.go:336: Retrieved .ls file from S3 (compressed size: 62 bytes)15982026/11/12 17:02:23 INFO Vacuumed table table=multipart_uploads1599 client_integration_test.go:336: Decompressed .ls content (49 bytes):1600 {"version":1,"root":{"type":"regular","size":39}}16012026/11/12 17:02:23 INFO Vacuumed table table=closures16022026/11/12 17:02:23 INFO Received complete push request method=POST path=/api/pushes/1/complete16032026/11/12 17:02:23 WARN Failed to register uploaded object key=7d12p7nayny3fkm54j6lywaiq6mnzx1m.narinfo error="server returned 404: 404 page not found\n"16042026/11/12 17:02:23 INFO Vacuumed table table=objects16052026/11/12 17:02:23 INFO Completed push id=1 objects=3 duration=11.805258ms16062026/11/12 17:02:23 INFO Upload complete. (226ms)1607=== NAME TestNARDeduplicationMetadataUploadBug1608 metadata_upload_test.go:54: Retrieved narinfo from S3:1609 StorePath: /build/TestNARDeduplicationMetadataUploadBug635060622/001/store/sxjq3l2pq7fy6c2xsd3n5g9sbn8wwpa7-file1.txt1610 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1611 Compression: zstd1612 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1613 NarSize: 1601614 References: 1615 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1616 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)1617 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):1618 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}16192026/11/12 17:02:23 INFO Completed push id=1 objects=4 duration=12.090318ms16202026/11/12 17:02:23 INFO Upload complete. (196ms)1621=== NAME TestClientCADerivations1622 client_ca_test.go:180: Narinfo contains CA field: StorePath: /build/TestClientCADerivations2429263468/001/store/7d12p7nayny3fkm54j6lywaiq6mnzx1m-ca-test1623 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1624 Compression: zstd1625 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1626 NarSize: 1441627 References: 1628 Deriver: /build/TestClientCADerivations2429263468/001/store/f47yahcaxdbx81g7mq4hs3gafnnnq3p9-ca-test.drv1629 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1630 client_ca_test.go:185: Checking for realisation files in S3...1631 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1632 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache16332026/11/12 17:02:23 INFO Installed lifecycle rule aborting incomplete NAR uploads bucket=bucket60 days=21634--- PASS: TestEnsureAbortUploadsRule (0.40s)1635=== CONT TestGCTaskStore_GetEmpty1636--- PASS: TestGCTaskStore_GetEmpty (0.00s)1637=== CONT TestSkippedUploadsHandler16382026/11/12 17:02:23 INFO Client skipped oversized paths paths=3 nar_bytes=50000000001639--- PASS: TestSkippedUploadsHandler (0.01s)1640=== CONT TestGCTaskStore_ConflictDifferentParams1641--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)1642=== CONT TestParseSize1643--- PASS: TestParseSize (0.00s)1644=== CONT TestGCTaskStore_DeduplicateSameParams1645--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)1646=== CONT TestService_Rustfstest16472026/11/12 17:02:23 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"16482026/11/12 17:02:23 WARN mTLS auth: bound subjects configured but subject DN unavailable16492026/11/12 17:02:23 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1650--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (0.42s)1651=== CONT TestGCTaskStore_StartNew1652--- PASS: TestGCTaskStore_StartNew (0.00s)1653=== CONT TestPresignedUploadRegisteredBeforeCommit1654=== NAME TestNARDeduplicationMetadataUploadBug1655 metadata_upload_test.go:64: Second store path (same content): /build/TestNARDeduplicationMetadataUploadBug635060622/001/store/x0gci438w3k0vragjfjd3vg7f6vaqap9-file2.txt1656=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart1657=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart1658=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts1659=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts1660=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure1661=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure1662=== CONT TestGCEndsOnShutdown1663=== RUN TestGCEndsOnShutdown/before_the_run1664=== PAUSE TestGCEndsOnShutdown/before_the_run1665=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1666=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1667=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1668=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1669=== CONT TestCompletedNarNotReofferedAcrossClosures1670--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (0.42s)1671=== CONT TestCommitRacingPendingCleanupKeepsObjects16722026/11/12 17:02:23 INFO Received push request method=POST path=/api/pushes16732026/11/12 17:02:23 WARN readiness check failed error="closed pool"16742026/11/12 17:02:23 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)1675--- PASS: TestService_readinessHandler (0.44s)1676=== CONT TestCompleteMultipartUpload_ErrorButObjectExists16772026/11/12 17:02:23 INFO Received complete push request method=POST path=/api/pushes/3/complete16782026/11/12 17:02:23 WARN Failed to register uploaded object key=4bz49f32pa3hcvxkg9xfk9d3jj2pm9dv.ls error="server returned 404: 404 page not found\n"16792026/11/12 17:02:23 INFO Completed push id=3 objects=3 duration=3.413119ms16802026/11/12 17:02:23 INFO Upload complete. (47ms)1681=== NAME TestClientIntegration1682 client_integration_test.go:353: Retrieved .ls file from S3 (compressed size: 62 bytes)1683 client_integration_test.go:353: Decompressed .ls content (49 bytes):1684 {"version":1,"root":{"type":"regular","size":39}}1685 client_integration_test.go:364: Testing garbage collection...16862026/11/12 17:02:23 INFO Received push request method=POST path=/api/pushes16872026/11/12 17:02:23 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)16882026/11/12 17:02:23 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign16892026/11/12 17:02:23 WARN Failed to register uploaded object key=x0gci438w3k0vragjfjd3vg7f6vaqap9.ls error="server returned 404: 404 page not found\n"16902026/11/12 17:02:23 INFO Signed narinfos id=2 count=116912026/11/12 17:02:23 INFO Uploading 1 narinfos16922026/11/12 17:02:23 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"1693--- PASS: TestService_AuthMiddleware (0.45s)1694=== CONT TestPushWaitsForSweepHoldingRow16952026/11/12 17:02:23 INFO Received complete push request method=POST path=/api/pushes/2/complete16962026/11/12 17:02:23 WARN Failed to register uploaded object key=x0gci438w3k0vragjfjd3vg7f6vaqap9.narinfo error="server returned 404: 404 page not found\n"1697=== NAME TestClientCADerivations1698 client_ca_test.go:258: nix copy output: warning: you don't have Internet access; disabling some network-dependent features1699 warning: failed to create TLS context for AWS credential providers; SSO, STS WebIdentity, and ECS container authentication will be unavailable1700 error: binary cache 's3://bucket52?endpoint=http://localhost:33111®ion=eu-west-1' is for Nix stores with prefix '/nix/store', not '/build/TestClientCADerivations2429263468/001/store'1701 client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 117022026/11/12 17:02:23 INFO Completed push id=2 objects=3 duration=10.626078ms17032026/11/12 17:02:23 INFO Upload complete. (60ms)1704=== NAME TestNARDeduplicationMetadataUploadBug1705 metadata_upload_test.go:76: Retrieved narinfo from S3:1706 StorePath: /build/TestNARDeduplicationMetadataUploadBug635060622/001/store/x0gci438w3k0vragjfjd3vg7f6vaqap9-file2.txt1707 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1708 Compression: zstd1709 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1710 NarSize: 1601711 References: 1712 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf17132026/11/12 17:02:23 INFO Starting cleanup of old closures method=DELETE path=/api/closures17142026/11/12 17:02:23 INFO Garbage collection started1715 metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes)1716 metadata_upload_test.go:77: Decompressed .ls content (49 bytes):1717 {"version":1,"root":{"type":"regular","size":44}}1718--- PASS: TestService_healthCheckHandler (0.47s)1719=== CONT TestSharedNarBothUploadsComplete1720--- PASS: TestClientCADerivations (1.04s)1721=== CONT TestSweepRechecksPendingAfterLocking1722--- PASS: TestNARDeduplicationMetadataUploadBug (0.96s)1723=== CONT TestCommitCanBeRepeatedUntilCleanup1724=== RUN TestCommitCanBeRepeatedUntilCleanup/push1725=== PAUSE TestCommitCanBeRepeatedUntilCleanup/push1726=== RUN TestCommitCanBeRepeatedUntilCleanup/pending_closure1727=== PAUSE TestCommitCanBeRepeatedUntilCleanup/pending_closure1728=== CONT TestGetExistingObjectsLocksRows17292026/11/12 17:02:23 INFO Aborted multipart uploads count=0 kept=017302026/11/12 17:02:23 WARN Force mode enabled - objects will be deleted immediately without grace period17312026/11/12 17:02:23 INFO Received uploads request method=POST path=/api/pending_closures17322026/11/12 17:02:23 INFO Received push request method=POST path=/api/pushes17332026/11/12 17:02:23 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=017342026/11/12 17:02:23 INFO Vacuumed table table=pending_closures17352026/11/12 17:02:23 INFO Vacuumed table table=commit_receipts17362026/11/12 17:02:23 INFO Received complete push request method=POST path=/api/pushes/1/complete17372026/11/12 17:02:23 INFO Vacuumed table table=pending_objects17382026/11/12 17:02:23 INFO Vacuumed table table=multipart_uploads17392026/11/12 17:02:23 INFO Received cleanup request method=DELETE path=/api/pending_closures17402026/11/12 17:02:23 INFO Aborted multipart uploads count=0 kept=017412026/11/12 17:02:23 INFO Completed push id=1 objects=4 duration=10.312088ms17422026/11/12 17:02:23 INFO Received push request method=POST path=/api/pushes17432026/11/12 17:02:23 INFO Vacuumed table table=closures17442026/11/12 17:02:23 INFO Received uploads request method=POST path=/api/pending_closures17452026/11/12 17:02:23 INFO Vacuumed table table=objects17462026/11/12 17:02:23 INFO Received cleanup request method=DELETE path=/api/pending_closures17472026/11/12 17:02:23 INFO Aborted multipart uploads count=1 kept=017482026/11/12 17:02:23 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete17492026-11-12 17:02:23.551 UTC [1659] ERROR: Closure does not exist: id=117502026-11-12 17:02:23.551 UTC [1659] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 17 at RAISE17512026-11-12 17:02:23.551 UTC [1659] STATEMENT: -- name: CommitPendingClosure :one1752 SELECT commit_pending_closure($1::bigint)::bigint1753 1754--- PASS: TestService_cleanupPendingClosuresHandler (0.44s)1755=== CONT TestPush_SignsNarinfosOfItsPendingObjects17562026/11/12 17:02:23 INFO Received complete multipart upload request method=POST path=/api/multipart/complete17572026/11/12 17:02:23 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst1758--- PASS: TestCompleteMultipartUnregistered (0.48s)1759=== CONT TestRegisterCompletedObjectNeedsPendingRow17602026/11/12 17:02:23 INFO lead: released remote=192.0.2.1:123417612026/11/12 17:02:23 INFO Received uploads request method=POST path=/api/pending_closures17622026/11/12 17:02:23 INFO Received uploads request method=POST path=/api/pending_closures17632026/11/12 17:02:23 INFO Received uploads request method=POST path=/api/pending_closures17642026/11/12 17:02:23 INFO Aborted multipart uploads count=0 kept=017652026/11/12 17:02:23 WARN Force mode enabled - objects will be deleted immediately without grace period17662026/11/12 17:02:23 INFO lead: acquired remote=192.0.2.1:123417672026/11/12 17:02:23 INFO lead: released remote=192.0.2.1:12341768--- PASS: TestLeadEndsOnShutdown (1.76s)1769=== CONT TestPush_RejectsBadRequests17702026/11/12 17:02:23 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=017712026/11/12 17:02:23 INFO Received uploads request method=POST path=/api/pending_closures17722026/11/12 17:02:23 INFO Vacuumed table table=pending_closures17732026/11/12 17:02:23 INFO Vacuumed table table=commit_receipts17742026/11/12 17:02:23 INFO Vacuumed table table=pending_objects17752026/11/12 17:02:23 INFO Vacuumed table table=multipart_uploads17762026/11/12 17:02:23 INFO Vacuumed table table=closures17772026/11/12 17:02:23 INFO Vacuumed table table=objects17782026/11/12 17:02:23 INFO Received complete push request method=POST path=/api/pushes/2/complete17792026/11/12 17:02:23 INFO Completed push id=2 objects=4 duration=4.922919ms1780--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (0.60s)1781=== CONT TestSweepSkipsRowsLockedByPush1782--- PASS: TestService_Rustfstest (0.32s)1783=== CONT TestFailedS3DeleteKeepsTombstone17842026/11/12 17:02:23 INFO lead: given back to the incumbent remote=192.0.2.1:123417852026/11/12 17:02:23 INFO lead: acquired remote=192.0.2.1:123417862026/11/12 17:02:23 INFO Received uploads request method=POST path=/api/pending_closures17872026/11/12 17:02:23 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst17882026/11/12 17:02:23 INFO Received uploads request method=POST path=/api/pending_closures1789--- PASS: TestPresignedUploadRegisteredBeforeCommit (0.37s)1790=== CONT TestReadProxyRootRedirectsToIndexHTML17912026/11/12 17:02:23 INFO Received complete multipart upload request method=POST path=/api/multipart/complete17922026/11/12 17:02:23 INFO Received uploads request method=POST path=/api/pending_closures17932026/11/12 17:02:23 INFO Aborted multipart uploads count=0 kept=017942026/11/12 17:02:23 WARN Force mode enabled - objects will be deleted immediately without grace period17952026/11/12 17:02:23 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=017962026/11/12 17:02:23 INFO Vacuumed table table=pending_closures17972026/11/12 17:02:23 INFO Vacuumed table table=commit_receipts17982026/11/12 17:02:23 INFO Vacuumed table table=pending_objects17992026/11/12 17:02:23 INFO Vacuumed table table=multipart_uploads18002026/11/12 17:02:23 INFO Received uploads request method=POST path=/api/pending_closures18012026/11/12 17:02:23 INFO Vacuumed table table=closures18022026/11/12 17:02:23 INFO Vacuumed table table=objects1803--- PASS: TestCommitRacingPendingCleanupKeepsObjects (0.41s)1804=== CONT TestForceGCDuringPushOffersSweptObject1805=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows1806=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows1807=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check1808=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check1809=== CONT TestPush_RootsAreLockedInByteOrder18102026/11/12 17:02:23 INFO Received uploads request method=POST path=/api/pending_closures1811--- PASS: TestDeleteClosuresRacingPins (1.78s)1812=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection18132026/11/12 17:02:23 INFO Received complete multipart upload request method=POST path=/api/multipart/complete18142026/11/12 17:02:23 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MjQxNTgzYzQtMzdmMS00YWUwLWExNDAtYzIwOWViYzgwZDcxLmNjYzdmZjZmLTE3NmQtNGU5NC1iOWJkLTA4MDI1MTYxMDIxN3gxNzk0NTAyOTQzNzg1MjkxNTE518152026/11/12 17:02:23 INFO lead: released remote=192.0.2.1:123418162026/11/12 17:02:23 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MjQxNTgzYzQtMzdmMS00YWUwLWExNDAtYzIwOWViYzgwZDcxLmNjYzdmZjZmLTE3NmQtNGU5NC1iOWJkLTA4MDI1MTYxMDIxN3gxNzk0NTAyOTQzNzg1MjkxNTE5 parts=11817--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (0.44s)1818=== CONT TestPush_CompleteWaitsForCleanupOfThePush1819--- PASS: TestGetExistingObjectsLocksRows (0.41s)1820=== CONT TestGCSweepDeliversEachKeyOnce18212026/11/12 17:02:23 INFO lead: acquired remote=192.0.2.1:123418222026/11/12 17:02:23 INFO Aborted multipart uploads count=0 kept=018232026/11/12 17:02:23 INFO Received uploads request method=POST path=/api/pending_closures18242026/11/12 17:02:23 WARN Force mode enabled - objects will be deleted immediately without grace period18252026/11/12 17:02:23 INFO Received uploads request method=POST path=/api/pending_closures18262026/11/12 17:02:23 INFO Received complete multipart upload request method=POST path=/api/multipart/complete18272026/11/12 17:02:23 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=018282026/11/12 17:02:23 INFO Vacuumed table table=pending_closures18292026/11/12 17:02:23 INFO Vacuumed table table=commit_receipts18302026/11/12 17:02:23 INFO Vacuumed table table=pending_objects18312026/11/12 17:02:23 INFO Vacuumed table table=multipart_uploads18322026/11/12 17:02:23 INFO Vacuumed table table=closures18332026/11/12 17:02:23 INFO Vacuumed table table=objects1834--- PASS: TestSweepRechecksPendingAfterLocking (0.52s)1835=== CONT TestPush_RepeatedRootCommitsOnce18362026/11/12 17:02:23 INFO Received push request method=POST path=/api/pushes18372026/11/12 17:02:23 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign18382026/11/12 17:02:23 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=MjQxNTgzYzQtMzdmMS00YWUwLWExNDAtYzIwOWViYzgwZDcxLjFkOTI0Yjg3LWQ4N2MtNDg0NS1hZTk3LWIwMDBjN2YzZTA4M3gxNzk0NTAyOTQzNDc2NDgxMDMw parts=1018392026/11/12 17:02:23 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete18402026/11/12 17:02:23 INFO Signed narinfos id=1 count=118412026/11/12 17:02:23 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign18422026/11/12 17:02:23 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign18432026/11/12 17:02:23 ERROR Failed to sign narinfos signer="index 1" error="signing key cannot be nil"18442026/11/12 17:02:23 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign18452026/11/12 17:02:23 ERROR Failed to sign narinfos signer=niks3-test-1 error="signer failed"18462026/11/12 17:02:23 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign18472026/11/12 17:02:23 ERROR Signer returned unexpected signature count signer=niks3-test-1 got=0 want=118482026/11/12 17:02:23 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign18492026/11/12 17:02:23 ERROR Signer returned unexpected object key signer=niks3-test-1 object_key=unexpected.narinfo1850--- PASS: TestPush_SignsNarinfosOfItsPendingObjects (0.43s)1851=== CONT TestPush_CompleteCommitsEveryRoot18522026/11/12 17:02:23 INFO Completed upload id=1 objects=2 duration=6.190188ms18532026/11/12 17:02:23 INFO Received uploads request method=POST path=/api/pending_closures18542026/11/12 17:02:23 INFO Received uploads request method=POST path=/api/pending_closures1855--- PASS: TestRegisterCompletedObjectNeedsPendingRow (0.44s)18562026/11/12 17:02:23 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo1857=== CONT TestReadRedirectKeepsNarinfoProxied18582026/11/12 17:02:23 WARN Found objects in DB but missing from S3, will re-upload count=11859--- PASS: TestService_verifyS3Integrity (0.91s)1860=== CONT TestReadProxyRangeRequest1861=== RUN TestPush_RejectsBadRequests/no_roots1862=== PAUSE TestPush_RejectsBadRequests/no_roots1863=== RUN TestPush_RejectsBadRequests/no_objects1864=== PAUSE TestPush_RejectsBadRequests/no_objects1865=== RUN TestPush_RejectsBadRequests/bad_root1866=== PAUSE TestPush_RejectsBadRequests/bad_root1867=== RUN TestPush_RejectsBadRequests/root_not_in_objects1868=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects1869=== CONT TestReadRedirectNar18702026/11/12 17:02:24 INFO Received complete multipart upload request method=POST path=/api/multipart/complete18712026/11/12 17:02:24 INFO Received uploads request method=POST path=/api/pending_closures18722026/11/12 17:02:24 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=MjQxNTgzYzQtMzdmMS00YWUwLWExNDAtYzIwOWViYzgwZDcxLmRhNWJhZjUyLTk0ZGYtNDQxOC05YTUwLTA1MjIzNzc3MmIxM3gxNzk0NTAyOTQzNTk3MzQzNTI2 parts=1018732026/11/12 17:02:24 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete18742026/11/12 17:02:24 INFO Completed upload id=1 objects=2 duration=7.843758ms18752026/11/12 17:02:24 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000018762026/11/12 17:02:24 INFO Received uploads request method=POST path=/api/pending_closures18772026/11/12 17:02:24 INFO Starting cleanup of old closures method=DELETE path=/api/closures18782026/11/12 17:02:24 INFO Aborted multipart uploads count=0 kept=018792026/11/12 17:02:24 INFO Aborted multipart uploads count=0 kept=018802026/11/12 17:02:24 WARN Force mode enabled - objects will be deleted immediately without grace period18812026/11/12 17:02:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=018822026/11/12 17:02:24 INFO Vacuumed table table=pending_closures18832026/11/12 17:02:24 INFO Vacuumed table table=commit_receipts18842026/11/12 17:02:24 INFO Vacuumed table table=pending_objects18852026/11/12 17:02:24 INFO Vacuumed table table=multipart_uploads18862026/11/12 17:02:24 INFO Vacuumed table table=closures18872026/11/12 17:02:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=018882026/11/12 17:02:24 INFO Vacuumed table table=objects18892026/11/12 17:02:24 INFO Vacuumed table table=pending_closures18902026/11/12 17:02:24 INFO Vacuumed table table=commit_receipts1891--- PASS: TestReadProxyRootRedirectsToIndexHTML (0.41s)1892=== CONT TestPush_OverlappingRootsStoreOneRowPerKey18932026/11/12 17:02:24 INFO Aborted multipart uploads count=0 kept=018942026/11/12 17:02:24 INFO Vacuumed table table=pending_objects18952026/11/12 17:02:24 INFO Vacuumed table table=multipart_uploads18962026/11/12 17:02:24 WARN Force mode enabled - objects will be deleted immediately without grace period18972026/11/12 17:02:24 INFO Vacuumed table table=closures18982026/11/12 17:02:24 INFO Vacuumed table table=objects18992026/11/12 17:02:24 INFO Aborted multipart uploads count=0 kept=019002026/11/12 17:02:24 WARN Force mode enabled - objects will be deleted immediately without grace period19012026/11/12 17:02:24 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001902--- PASS: TestService_createPendingClosureHandler (1.05s)1903=== CONT TestReadProxyDisabled19042026/11/12 17:02:24 INFO Received push request method=POST path=/api/pushes19052026/11/12 17:02:24 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."19062026/11/12 17:02:24 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."1907--- PASS: TestFailedS3DeleteKeepsTombstone (0.50s)1908=== CONT TestReadRedirectUsesPublicS3URL19092026/11/12 17:02:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=019102026/11/12 17:02:24 INFO Vacuumed table table=pending_closures19112026/11/12 17:02:24 INFO Vacuumed table table=commit_receipts19122026/11/12 17:02:24 INFO Vacuumed table table=pending_objects19132026/11/12 17:02:24 INFO Vacuumed table table=multipart_uploads19142026/11/12 17:02:24 INFO Vacuumed table table=closures19152026/11/12 17:02:24 INFO Vacuumed table table=objects1916--- PASS: TestSweepSkipsRowsLockedByPush (0.52s)1917=== CONT TestReadProxyNarStreaming19182026/11/12 17:02:24 INFO Received push request method=POST path=/api/pushes19192026-11-12 17:02:24.196 UTC [2093] ERROR: could not obtain lock on row in relation "closures"19202026-11-12 17:02:24.196 UTC [2093] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT19212026-11-12 17:02:24.196 UTC [2093] ERROR: could not obtain lock on row in relation "closures"19222026-11-12 17:02:24.196 UTC [2093] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT19232026-11-12 17:02:24.197 UTC [2093] ERROR: could not obtain lock on row in relation "closures"19242026-11-12 17:02:24.197 UTC [2093] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT19252026-11-12 17:02:24.197 UTC [2093] ERROR: could not obtain lock on row in relation "closures"19262026-11-12 17:02:24.197 UTC [2093] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT19272026-11-12 17:02:24.197 UTC [2093] ERROR: could not obtain lock on row in relation "closures"19282026-11-12 17:02:24.197 UTC [2093] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT1929--- PASS: TestPush_RootsAreLockedInByteOrder (0.42s)1930=== CONT TestReadProxyConditionalGet19312026/11/12 17:02:24 INFO Received uploads request method=POST path=/api/pending_closures19322026/11/12 17:02:24 INFO Received push request method=POST path=/api/pushes19332026/11/12 17:02:24 INFO Aborted multipart uploads count=0 kept=019342026/11/12 17:02:24 WARN Force mode enabled - objects will be deleted immediately without grace period19352026/11/12 17:02:24 INFO Received complete push request method=POST path=/api/pushes/1/complete19362026/11/12 17:02:24 INFO Received complete multipart upload request method=POST path=/api/multipart/complete19372026/11/12 17:02:24 INFO Completed push id=1 objects=4 duration=6.111799ms1938--- PASS: TestPush_RepeatedRootCommitsOnce (0.35s)1939=== CONT TestReadProxyOutlastsServerWriteTimeout1940--- PASS: TestPushWaitsForSweepHoldingRow (0.89s)1941=== CONT TestReadProxyHead19422026/11/12 17:02:24 INFO Received push request method=POST path=/api/pushes1943=== NAME TestOrphanedObjectsGCStressTest1944 orphaned_objects_gc_test.go:422: Created 10 active closures, 5 to-delete closures, 20 orphaned chains19452026/11/12 17:02:24 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=MjQxNTgzYzQtMzdmMS00YWUwLWExNDAtYzIwOWViYzgwZDcxLjYwMzczNmY1LWEwODctNGI3My1iZDhiLTIyMjE4ZjUwYjZiNXgxNzk0NTAyOTQzNzU0MTQxNTY1 parts=1219462026/11/12 17:02:24 INFO Received uploads request method=POST path=/api/pending_closures19472026/11/12 17:02:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=01948--- PASS: TestCompletedNarNotReofferedAcrossClosures (0.97s)1949=== CONT TestReadProxyInvalidPath19502026/11/12 17:02:24 INFO Received complete push request method=POST path=/api/pushes/1/complete19512026/11/12 17:02:24 INFO Vacuumed table table=pending_closures19522026/11/12 17:02:24 INFO Vacuumed table table=commit_receipts19532026/11/12 17:02:24 INFO Vacuumed table table=pending_objects19542026/11/12 17:02:24 INFO Vacuumed table table=multipart_uploads19552026/11/12 17:02:24 INFO Vacuumed table table=closures19562026/11/12 17:02:24 INFO Vacuumed table table=objects1957--- PASS: TestGCSweepDeliversEachKeyOnce (0.49s)1958=== CONT TestReadProxy40419592026/11/12 17:02:24 INFO Completed push id=1 objects=6 duration=16.317077ms1960--- PASS: TestPush_CompleteCommitsEveryRoot (0.38s)1961=== CONT TestReadProxyNarinfo1962=== NAME TestOrphanedObjectsGCStressTest1963 orphaned_objects_gc_test.go:440: Marked 210 objects for deletion1964--- PASS: TestReadProxyRangeRequest (0.37s)1965=== CONT TestReadProxyContentNegotiation1966--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (0.62s)1967=== CONT TestReadProxyNarinfoAlreadyDecompressed1968--- PASS: TestReadRedirectKeepsNarinfoProxied (0.58s)1969=== CONT TestIsValidCachePath1970=== RUN TestIsValidCachePath/narinfo1971=== PAUSE TestIsValidCachePath/narinfo1972=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars1973=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars1974=== RUN TestIsValidCachePath/nar_zst1975=== PAUSE TestIsValidCachePath/nar_zst1976=== RUN TestIsValidCachePath/nar_xz1977=== PAUSE TestIsValidCachePath/nar_xz1978=== RUN TestIsValidCachePath/nar_bz21979=== PAUSE TestIsValidCachePath/nar_bz21980=== RUN TestIsValidCachePath/nar_uncompressed1981=== PAUSE TestIsValidCachePath/nar_uncompressed1982=== RUN TestIsValidCachePath/ls1983=== PAUSE TestIsValidCachePath/ls1984=== RUN TestIsValidCachePath/log1985=== PAUSE TestIsValidCachePath/log1986=== RUN TestIsValidCachePath/realisation1987=== PAUSE TestIsValidCachePath/realisation1988=== RUN TestIsValidCachePath/nix-cache-info1989=== PAUSE TestIsValidCachePath/nix-cache-info1990=== RUN TestIsValidCachePath/index.html1991=== PAUSE TestIsValidCachePath/index.html1992=== RUN TestIsValidCachePath/traversal_parent1993=== PAUSE TestIsValidCachePath/traversal_parent1994=== RUN TestIsValidCachePath/traversal_in_middle1995=== PAUSE TestIsValidCachePath/traversal_in_middle1996=== RUN TestIsValidCachePath/invalid_char_e1997=== PAUSE TestIsValidCachePath/invalid_char_e1998=== RUN TestIsValidCachePath/invalid_char_u1999=== PAUSE TestIsValidCachePath/invalid_char_u2000=== RUN TestIsValidCachePath/random_path
loading 1,077 hidden lines…
30782026/11/12 17:02:38 INFO Vacuumed table table=pending_closures30792026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts30802026/11/12 17:02:38 INFO Vacuumed table table=pending_objects30812026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads30822026/11/12 17:02:38 INFO Received complete push request method=POST path=/30832026/11/12 17:02:38 INFO Vacuumed table table=closures30842026/11/12 17:02:38 INFO Received complete push request method=POST path=/30852026/11/12 17:02:38 INFO Vacuumed table table=objects30862026/11/12 17:02:38 INFO Completed push id=4 objects=4 duration=2.02006ms30872026/11/12 17:02:38 INFO Completed push id=6 objects=6 duration=2.24891ms30882026/11/12 17:02:38 INFO Received complete push request method=POST path=/30892026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=030902026/11/12 17:02:38 INFO Vacuumed table table=pending_closures30912026/11/12 17:02:38 INFO Received uploads request method=POST path=/30922026/11/12 17:02:38 INFO Received complete upload request method=POST path=/30932026/11/12 17:02:38 INFO Received uploads request method=POST path=/30942026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts30952026/11/12 17:02:38 INFO Completed push id=6 objects=5 duration=4.381579ms30962026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=030972026/11/12 17:02:38 INFO Vacuumed table table=pending_objects30982026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads30992026/11/12 17:02:38 INFO Completed upload id=10 objects=4 duration=3.23289ms31002026/11/12 17:02:38 INFO Vacuumed table table=closures31012026/11/12 17:02:38 INFO Received push request method=POST path=/31022026/11/12 17:02:38 INFO Received uploads request method=POST path=/31032026/11/12 17:02:38 INFO Vacuumed table table=objects31042026/11/12 17:02:38 INFO Received uploads request method=POST path=/31052026/11/12 17:02:38 INFO Received push request method=POST path=/31062026/11/12 17:02:38 INFO Received push request method=POST path=/31072026/11/12 17:02:38 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."31082026/11/12 17:02:38 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."31092026/11/12 17:02:38 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."31102026/11/12 17:02:38 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."31112026/11/12 17:02:38 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."31122026/11/12 17:02:38 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."31132026/11/12 17:02:38 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."31142026/11/12 17:02:38 INFO Received push request method=POST path=/31152026/11/12 17:02:38 INFO Received uploads request method=POST path=/31162026/11/12 17:02:38 INFO Received uploads request method=POST path=/31172026/11/12 17:02:38 INFO Received push request method=POST path=/31182026/11/12 17:02:38 INFO Received push request method=POST path=/31192026/11/12 17:02:38 INFO Received uploads request method=POST path=/31202026/11/12 17:02:38 INFO Received push request method=POST path=/31212026/11/12 17:02:38 INFO Received complete upload request method=POST path=/31222026/11/12 17:02:38 INFO Completed upload id=9 objects=7 duration=2.92947ms31232026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=031242026/11/12 17:02:38 INFO Received uploads request method=POST path=/31252026/11/12 17:02:38 WARN Force mode enabled - objects will be deleted immediately without grace period31262026/11/12 17:02:38 INFO Received complete upload request method=POST path=/31272026/11/12 17:02:38 INFO Completed upload id=2 objects=2 duration=1.820319ms31282026/11/12 17:02:38 INFO Received push request method=POST path=/31292026/11/12 17:02:38 INFO Received complete upload request method=POST path=/31302026/11/12 17:02:38 INFO Received push request method=POST path=/31312026/11/12 17:02:38 INFO Received uploads request method=POST path=/31322026/11/12 17:02:38 INFO Completed upload id=3 objects=5 duration=1.380749ms31332026/11/12 17:02:38 INFO Received complete upload request method=POST path=/31342026/11/12 17:02:38 INFO Received push request method=POST path=/31352026/11/12 17:02:38 INFO Completed upload id=6 objects=24 duration=4.119769ms31362026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=4 objects-failed-to-delete=031372026/11/12 17:02:38 INFO Received push request method=POST path=/31382026/11/12 17:02:38 INFO Vacuumed table table=pending_closures31392026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts31402026/11/12 17:02:38 INFO Vacuumed table table=pending_objects31412026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads31422026/11/12 17:02:38 INFO Vacuumed table table=closures31432026/11/12 17:02:38 INFO Vacuumed table table=objects31442026/11/12 17:02:38 INFO Received complete upload request method=POST path=/31452026/11/12 17:02:38 INFO Received push request method=POST path=/31462026/11/12 17:02:38 INFO Received complete push request method=POST path=/31472026/11/12 17:02:38 INFO Completed upload id=3 objects=24 duration=1.858269ms31482026/11/12 17:02:38 INFO Received push request method=POST path=/31492026/11/12 17:02:38 INFO Completed push id=7 objects=27 duration=2.0549ms31502026/11/12 17:02:38 INFO Received push request method=POST path=/31512026/11/12 17:02:38 INFO Received push request method=POST path=/31522026/11/12 17:02:38 INFO Received complete push request method=POST path=/31532026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=031542026/11/12 17:02:38 INFO Received complete push request method=POST path=/31552026/11/12 17:02:38 INFO Completed push id=2 objects=7 duration=4.531179ms31562026/11/12 17:02:38 INFO Completed push id=6 objects=5 duration=2.15039ms31572026/11/12 17:02:38 INFO Received push request method=POST path=/31582026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=031592026/11/12 17:02:38 INFO Received complete upload request method=POST path=/31602026/11/12 17:02:38 INFO Received push request method=POST path=/31612026/11/12 17:02:38 INFO Completed upload id=10 objects=6 duration=3.06485ms31622026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=031632026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=031642026/11/12 17:02:38 INFO Vacuumed table table=pending_closures31652026/11/12 17:02:38 INFO Vacuumed table table=pending_closures31662026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts31672026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts31682026/11/12 17:02:38 INFO Vacuumed table table=pending_objects31692026/11/12 17:02:38 INFO Vacuumed table table=pending_objects31702026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads31712026/11/12 17:02:38 INFO Received uploads request method=POST path=/31722026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads31732026/11/12 17:02:38 INFO Vacuumed table table=closures31742026/11/12 17:02:38 INFO Received push request method=POST path=/31752026/11/12 17:02:38 INFO Vacuumed table table=closures31762026/11/12 17:02:38 INFO Vacuumed table table=objects31772026/11/12 17:02:38 INFO Vacuumed table table=objects31782026/11/12 17:02:38 INFO Received uploads request method=POST path=/31792026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=031802026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031812026/11/12 17:02:38 INFO Received push request method=POST path=/31822026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=031832026/11/12 17:02:38 INFO Vacuumed table table=pending_closures31842026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=031852026/11/12 17:02:38 INFO Received push request method=POST path=/31862026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts31872026/11/12 17:02:38 WARN Force mode enabled - objects will be deleted immediately without grace period31882026/11/12 17:02:38 WARN Force mode enabled - objects will be deleted immediately without grace period31892026/11/12 17:02:38 INFO Vacuumed table table=pending_objects31902026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031912026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads31922026/11/12 17:02:38 INFO Vacuumed table table=closures31932026/11/12 17:02:38 INFO Vacuumed table table=pending_closures31942026/11/12 17:02:38 INFO Vacuumed table table=objects31952026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts31962026/11/12 17:02:38 INFO Received complete upload request method=POST path=/31972026/11/12 17:02:38 INFO Vacuumed table table=pending_objects31982026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=031992026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads32002026/11/12 17:02:38 INFO Completed upload id=3 objects=9 duration=1.29832ms32012026/11/12 17:02:38 INFO Vacuumed table table=closures32022026/11/12 17:02:38 INFO Vacuumed table table=objects32032026/11/12 17:02:38 INFO Received complete push request method=POST path=/32042026/11/12 17:02:38 WARN Force mode enabled - objects will be deleted immediately without grace period32052026/11/12 17:02:38 INFO Received complete push request method=POST path=/32062026/11/12 17:02:38 INFO Completed push id=5 objects=3 duration=2.29144ms32072026/11/12 17:02:38 INFO Completed push id=8 objects=6 duration=3.14226ms32082026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=032092026/11/12 17:02:38 INFO Received push request method=POST path=/32102026/11/12 17:02:38 WARN Force mode enabled - objects will be deleted immediately without grace period32112026/11/12 17:02:38 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."32122026/11/12 17:02:38 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."32132026/11/12 17:02:38 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."32142026/11/12 17:02:38 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."32152026/11/12 17:02:38 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."32162026/11/12 17:02:38 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."32172026/11/12 17:02:38 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."32182026/11/12 17:02:38 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."32192026/11/12 17:02:38 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."32202026/11/12 17:02:38 INFO Received uploads request method=POST path=/32212026/11/12 17:02:38 INFO Received push request method=POST path=/32222026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=032232026/11/12 17:02:38 INFO Received push request method=POST path=/32242026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=032252026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032262026/11/12 17:02:38 INFO Received complete upload request method=POST path=/32272026/11/12 17:02:38 INFO Vacuumed table table=pending_closures32282026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts32292026/11/12 17:02:38 INFO Completed upload id=11 objects=14 duration=1.89044ms32302026/11/12 17:02:38 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."32312026/11/12 17:02:38 INFO Vacuumed table table=pending_objects32322026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads32332026/11/12 17:02:38 INFO Vacuumed table table=closures32342026/11/12 17:02:38 INFO Vacuumed table table=objects32352026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=032362026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=032372026/11/12 17:02:38 WARN Force mode enabled - objects will be deleted immediately without grace period32382026/11/12 17:02:38 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."32392026/11/12 17:02:38 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."32402026/11/12 17:02:38 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."32412026/11/12 17:02:38 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."32422026/11/12 17:02:38 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."32432026/11/12 17:02:38 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."32442026/11/12 17:02:38 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."32452026/11/12 17:02:38 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."32462026/11/12 17:02:38 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."32472026/11/12 17:02:38 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."32482026/11/12 17:02:38 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."32492026/11/12 17:02:38 INFO Received push request method=POST path=/32502026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=032512026/11/12 17:02:38 INFO Received complete upload request method=POST path=/32522026/11/12 17:02:38 WARN Force mode enabled - objects will be deleted immediately without grace period32532026/11/12 17:02:38 INFO Completed upload id=9 objects=3 duration=1.509649ms32542026/11/12 17:02:38 INFO Received push request method=POST path=/32552026/11/12 17:02:38 INFO Received complete push request method=POST path=/32562026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032572026/11/12 17:02:38 INFO Vacuumed table table=pending_closures32582026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts32592026/11/12 17:02:38 INFO Received uploads request method=POST path=/32602026/11/12 17:02:38 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."32612026/11/12 17:02:38 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."32622026/11/12 17:02:38 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."32632026/11/12 17:02:38 INFO Vacuumed table table=pending_objects32642026/11/12 17:02:38 INFO Completed push id=2 objects=10 duration=4.562029ms32652026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads32662026/11/12 17:02:38 INFO Vacuumed table table=closures32672026/11/12 17:02:38 INFO Vacuumed table table=objects32682026/11/12 17:02:38 INFO Received push request method=POST path=/32692026/11/12 17:02:38 INFO Received uploads request method=POST path=/32702026/11/12 17:02:38 INFO Received push request method=POST path=/32712026/11/12 17:02:38 INFO Received complete push request method=POST path=/32722026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=032732026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=9 objects-failed-to-delete=032742026/11/12 17:02:38 INFO Completed push id=7 objects=20 duration=3.07927ms32752026/11/12 17:02:38 INFO Received complete push request method=POST path=/32762026/11/12 17:02:38 INFO Vacuumed table table=pending_closures32772026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts32782026/11/12 17:02:38 INFO Completed push id=5 objects=6 duration=1.64857ms32792026/11/12 17:02:38 INFO Vacuumed table table=pending_objects32802026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads32812026/11/12 17:02:38 INFO Received push request method=POST path=/32822026/11/12 17:02:38 INFO Vacuumed table table=closures32832026/11/12 17:02:38 INFO Vacuumed table table=objects32842026/11/12 17:02:38 INFO Received push request method=POST path=/32852026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=032862026/11/12 17:02:38 INFO Vacuumed table table=pending_closures32872026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts32882026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=032892026/11/12 17:02:38 INFO Vacuumed table table=pending_objects32902026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads32912026/11/12 17:02:38 INFO Vacuumed table table=closures32922026/11/12 17:02:38 INFO Vacuumed table table=objects32932026/11/12 17:02:38 INFO Received push request method=POST path=/32942026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=032952026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=4 objects-deleted-after-grace-period=0 objects-failed-to-delete=032962026/11/12 17:02:38 INFO Received push request method=POST path=/32972026/11/12 17:02:38 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."32982026/11/12 17:02:38 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."32992026/11/12 17:02:38 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."33002026/11/12 17:02:38 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."33012026/11/12 17:02:38 INFO Vacuumed table table=pending_closures33022026/11/12 17:02:38 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."33032026/11/12 17:02:38 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."33042026/11/12 17:02:38 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."33052026/11/12 17:02:38 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."33062026/11/12 17:02:38 INFO Received push request method=POST path=/33072026/11/12 17:02:38 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."33082026/11/12 17:02:38 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."33092026/11/12 17:02:38 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."33102026/11/12 17:02:38 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."33112026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts33122026/11/12 17:02:38 INFO Vacuumed table table=pending_objects33132026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads33142026/11/12 17:02:38 INFO Vacuumed table table=closures33152026/11/12 17:02:38 INFO Vacuumed table table=objects33162026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=033172026/11/12 17:02:38 INFO Received push request method=POST path=/33182026/11/12 17:02:38 INFO Received complete push request method=POST path=/33192026/11/12 17:02:38 WARN Force mode enabled - objects will be deleted immediately without grace period33202026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=033212026/11/12 17:02:38 INFO Received push request method=POST path=/33222026/11/12 17:02:38 INFO Completed push id=8 objects=2 duration=1.96694ms33232026/11/12 17:02:38 INFO Received uploads request method=POST path=/33242026/11/12 17:02:38 INFO Received uploads request method=POST path=/33252026/11/12 17:02:38 INFO Received push request method=POST path=/33262026/11/12 17:02:38 INFO Received uploads request method=POST path=/33272026/11/12 17:02:38 INFO Received uploads request method=POST path=/33282026/11/12 17:02:38 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."33292026/11/12 17:02:38 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."33302026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=033312026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033322026/11/12 17:02:38 INFO Received push request method=POST path=/33332026/11/12 17:02:38 INFO Vacuumed table table=pending_closures33342026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts33352026/11/12 17:02:38 INFO Received uploads request method=POST path=/33362026/11/12 17:02:38 INFO Vacuumed table table=pending_objects33372026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads33382026/11/12 17:02:38 INFO Vacuumed table table=closures33392026/11/12 17:02:38 INFO Vacuumed table table=objects33402026/11/12 17:02:38 INFO Received uploads request method=POST path=/33412026/11/12 17:02:38 INFO Received push request method=POST path=/33422026/11/12 17:02:38 INFO Received push request method=POST path=/33432026/11/12 17:02:38 INFO Received complete push request method=POST path=/33442026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=033452026/11/12 17:02:38 INFO Vacuumed table table=pending_closures33462026/11/12 17:02:38 INFO Completed push id=8 objects=7 duration=2.054439ms33472026/11/12 17:02:38 INFO Received complete push request method=POST path=/33482026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts33492026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=033502026/11/12 17:02:38 WARN Force mode enabled - objects will be deleted immediately without grace period33512026/11/12 17:02:38 INFO Vacuumed table table=pending_objects33522026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads33532026/11/12 17:02:38 INFO Completed push id=10 objects=2 duration=2.46925ms33542026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033552026/11/12 17:02:38 INFO Vacuumed table table=closures33562026/11/12 17:02:38 INFO Vacuumed table table=pending_closures33572026/11/12 17:02:38 INFO Vacuumed table table=objects33582026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts33592026/11/12 17:02:38 INFO Received uploads request method=POST path=/33602026/11/12 17:02:38 INFO Received uploads request method=POST path=/33612026/11/12 17:02:38 INFO Vacuumed table table=pending_objects33622026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads33632026/11/12 17:02:38 INFO Received push request method=POST path=/33642026/11/12 17:02:38 INFO Vacuumed table table=closures33652026/11/12 17:02:38 INFO Received uploads request method=POST path=/33662026/11/12 17:02:38 INFO Vacuumed table table=objects33672026/11/12 17:02:38 INFO Received complete push request method=POST path=/33682026/11/12 17:02:38 INFO Received uploads request method=POST path=/33692026/11/12 17:02:38 INFO Received uploads request method=POST path=/33702026/11/12 17:02:38 INFO Completed push id=5 objects=3 duration=3.96256ms33712026/11/12 17:02:38 INFO Received push request method=POST path=/33722026/11/12 17:02:38 INFO Received complete upload request method=POST path=/33732026/11/12 17:02:38 INFO Received uploads request method=POST path=/33742026/11/12 17:02:38 INFO Completed upload id=11 objects=6 duration=2.862009ms33752026/11/12 17:02:38 INFO Received uploads request method=POST path=/33762026/11/12 17:02:38 INFO Received complete upload request method=POST path=/33772026/11/12 17:02:38 INFO Received complete upload request method=POST path=/33782026/11/12 17:02:38 INFO Completed upload id=12 objects=21 duration=1.45064ms33792026/11/12 17:02:38 INFO Received push request method=POST path=/33802026/11/12 17:02:38 INFO Completed upload id=4 objects=14 duration=4.386879ms33812026/11/12 17:02:38 INFO Received push request method=POST path=/33822026/11/12 17:02:38 INFO Received push request method=POST path=/33832026/11/12 17:02:38 INFO Received push request method=POST path=/33842026/11/12 17:02:38 INFO Received uploads request method=POST path=/33852026/11/12 17:02:38 INFO Received uploads request method=POST path=/33862026/11/12 17:02:38 INFO Received complete push request method=POST path=/33872026/11/12 17:02:38 INFO Received push request method=POST path=/33882026/11/12 17:02:38 INFO Completed push id=12 objects=3 duration=2.42077ms33892026/11/12 17:02:38 INFO Received push request method=POST path=/33902026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=033912026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=033922026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=033932026/11/12 17:02:38 INFO Received complete upload request method=POST path=/33942026/11/12 17:02:38 INFO Vacuumed table table=pending_closures33952026/11/12 17:02:38 INFO Completed upload id=9 objects=16 duration=1.851899ms33962026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts33972026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033982026/11/12 17:02:38 INFO Vacuumed table table=pending_objects33992026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads34002026/11/12 17:02:38 INFO Received uploads request method=POST path=/34012026/11/12 17:02:38 INFO Vacuumed table table=pending_closures34022026/11/12 17:02:38 INFO Vacuumed table table=closures34032026/11/12 17:02:38 INFO Received uploads request method=POST path=/34042026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts34052026/11/12 17:02:38 INFO Vacuumed table table=objects34062026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=034072026/11/12 17:02:38 INFO Vacuumed table table=pending_objects34082026/11/12 17:02:38 INFO Vacuumed table table=pending_closures34092026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads34102026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts34112026/11/12 17:02:38 INFO Vacuumed table table=closures34122026/11/12 17:02:38 INFO Vacuumed table table=pending_objects34132026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads34142026/11/12 17:02:38 INFO Vacuumed table table=objects34152026/11/12 17:02:38 INFO Received complete push request method=POST path=/34162026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=034172026/11/12 17:02:38 INFO Vacuumed table table=closures34182026/11/12 17:02:38 WARN Force mode enabled - objects will be deleted immediately without grace period34192026/11/12 17:02:38 INFO Vacuumed table table=objects34202026/11/12 17:02:38 INFO Received uploads request method=POST path=/34212026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=034222026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034232026/11/12 17:02:38 INFO Received uploads request method=POST path=/34242026/11/12 17:02:38 INFO Completed push id=14 objects=3 duration=3.862069ms34252026/11/12 17:02:38 WARN Force mode enabled - objects will be deleted immediately without grace period34262026/11/12 17:02:38 INFO Received push request method=POST path=/34272026/11/12 17:02:38 INFO Vacuumed table table=pending_closures34282026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts34292026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034302026/11/12 17:02:38 INFO Vacuumed table table=pending_objects34312026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads34322026/11/12 17:02:38 INFO Received complete push request method=POST path=/34332026/11/12 17:02:38 INFO Received complete push request method=POST path=/34342026/11/12 17:02:38 INFO Vacuumed table table=pending_closures34352026/11/12 17:02:38 INFO Vacuumed table table=closures34362026/11/12 17:02:38 INFO Received push request method=POST path=/34372026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts34382026/11/12 17:02:38 INFO Completed push id=13 objects=23 duration=1.94016ms34392026/11/12 17:02:38 INFO Vacuumed table table=objects34402026/11/12 17:02:38 INFO Vacuumed table table=pending_objects34412026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads34422026/11/12 17:02:38 INFO Vacuumed table table=closures34432026/11/12 17:02:38 INFO Completed push id=5 objects=9 duration=5.187559ms34442026/11/12 17:02:38 INFO Received push request method=POST path=/34452026/11/12 17:02:38 INFO Vacuumed table table=objects34462026/11/12 17:02:38 INFO Received push request method=POST path=/34472026/11/12 17:02:38 INFO Received uploads request method=POST path=/34482026/11/12 17:02:38 INFO Received push request method=POST path=/34492026/11/12 17:02:38 INFO Received uploads request method=POST path=/34502026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=034512026/11/12 17:02:38 INFO Received push request method=POST path=/34522026/11/12 17:02:38 INFO Received push request method=POST path=/34532026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034542026/11/12 17:02:38 INFO Vacuumed table table=pending_closures34552026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts34562026/11/12 17:02:38 INFO Vacuumed table table=pending_objects34572026/11/12 17:02:38 INFO Received complete push request method=POST path=/34582026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads34592026/11/12 17:02:38 INFO Received complete push request method=POST path=/34602026/11/12 17:02:38 INFO Vacuumed table table=closures34612026/11/12 17:02:38 INFO Vacuumed table table=objects34622026/11/12 17:02:38 INFO Completed push id=8 objects=6 duration=2.240499ms34632026/11/12 17:02:38 INFO Received uploads request method=POST path=/34642026/11/12 17:02:38 INFO Received complete push request method=POST path=/34652026/11/12 17:02:38 INFO Completed push id=13 objects=12 duration=3.857699ms34662026/11/12 17:02:38 INFO Completed push id=5 objects=6 duration=1.97738ms34672026/11/12 17:02:38 INFO Received complete upload request method=POST path=/34682026/11/12 17:02:38 INFO Received complete push request method=POST path=/34692026/11/12 17:02:38 INFO Completed upload id=10 objects=2 duration=2.880749ms34702026/11/12 17:02:38 INFO Completed push id=9 objects=5 duration=2.66147ms34712026/11/12 17:02:38 INFO Received push request method=POST path=/34722026/11/12 17:02:38 INFO Received push request method=POST path=/34732026/11/12 17:02:38 INFO Received uploads request method=POST path=/34742026/11/12 17:02:38 INFO Received push request method=POST path=/34752026/11/12 17:02:38 INFO Received complete push request method=POST path=/34762026/11/12 17:02:38 INFO Completed push id=10 objects=3 duration=1.830709ms34772026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=034782026/11/12 17:02:38 INFO Received push request method=POST path=/34792026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=034802026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=034812026/11/12 17:02:38 INFO Aborted multipart uploads count=0 kept=034822026/11/12 17:02:38 WARN Force mode enabled - objects will be deleted immediately without grace period34832026/11/12 17:02:38 WARN Force mode enabled - objects will be deleted immediately without grace period34842026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034852026/11/12 17:02:38 INFO Received uploads request method=POST path=/34862026/11/12 17:02:38 WARN Force mode enabled - objects will be deleted immediately without grace period34872026/11/12 17:02:38 INFO Vacuumed table table=pending_closures34882026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034892026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts34902026/11/12 17:02:38 INFO Vacuumed table table=pending_closures34912026/11/12 17:02:38 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034922026/11/12 17:02:38 INFO Vacuumed table table=pending_objects34932026/11/12 17:02:38 INFO Vacuumed table table=multipart_uploads34942026/11/12 17:02:38 INFO Vacuumed table table=commit_receipts34952026/11/12 17:02:38 INFO Vacuumed table table=pending_closures34962026/11/12 17:02:39 INFO Vacuumed table table=closures34972026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts34982026/11/12 17:02:39 INFO Vacuumed table table=pending_objects34992026/11/12 17:02:39 INFO Vacuumed table table=objects35002026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads35012026/11/12 17:02:39 INFO Vacuumed table table=pending_objects35022026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads35032026/11/12 17:02:39 INFO Vacuumed table table=closures35042026/11/12 17:02:39 INFO Vacuumed table table=closures35052026/11/12 17:02:39 INFO Vacuumed table table=objects35062026/11/12 17:02:39 INFO Vacuumed table table=objects35072026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=035082026/11/12 17:02:39 INFO Received push request method=POST path=/35092026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=035102026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=035112026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=035122026/11/12 17:02:39 INFO Received uploads request method=POST path=/35132026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=035142026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035152026/11/12 17:02:39 INFO Vacuumed table table=pending_closures35162026/11/12 17:02:39 INFO Received push request method=POST path=/35172026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts35182026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035192026/11/12 17:02:39 INFO Vacuumed table table=pending_closures35202026/11/12 17:02:39 INFO Vacuumed table table=pending_closures35212026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts35222026/11/12 17:02:39 INFO Vacuumed table table=pending_objects35232026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads35242026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts35252026/11/12 17:02:39 INFO Vacuumed table table=pending_objects35262026/11/12 17:02:39 INFO Received push request method=POST path=/35272026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads35282026/11/12 17:02:39 INFO Vacuumed table table=pending_objects35292026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads35302026/11/12 17:02:39 INFO Vacuumed table table=closures35312026/11/12 17:02:39 INFO Vacuumed table table=closures35322026/11/12 17:02:39 INFO Vacuumed table table=closures35332026/11/12 17:02:39 INFO Vacuumed table table=objects35342026/11/12 17:02:39 INFO Vacuumed table table=objects35352026/11/12 17:02:39 INFO Vacuumed table table=objects35362026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=035372026/11/12 17:02:39 INFO Vacuumed table table=pending_closures35382026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts35392026/11/12 17:02:39 INFO Vacuumed table table=pending_objects35402026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads35412026/11/12 17:02:39 INFO Received push request method=POST path=/35422026/11/12 17:02:39 INFO Vacuumed table table=closures35432026/11/12 17:02:39 INFO Vacuumed table table=objects35442026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=035452026/11/12 17:02:39 INFO Received complete upload request method=POST path=/35462026/11/12 17:02:39 INFO Received push request method=POST path=/35472026/11/12 17:02:39 INFO Vacuumed table table=pending_closures35482026/11/12 17:02:39 INFO Received complete push request method=POST path=/35492026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts35502026/11/12 17:02:39 INFO Completed upload id=16 objects=23 duration=1.93434ms35512026/11/12 17:02:39 INFO Vacuumed table table=pending_objects35522026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads35532026/11/12 17:02:39 INFO Received uploads request method=POST path=/35542026/11/12 17:02:39 INFO Vacuumed table table=closures35552026/11/12 17:02:39 INFO Completed push id=9 objects=4 duration=3.952929ms35562026/11/12 17:02:39 INFO Vacuumed table table=objects35572026/11/12 17:02:39 INFO Received uploads request method=POST path=/35582026/11/12 17:02:39 INFO Received push request method=POST path=/35592026/11/12 17:02:39 INFO Received push request method=POST path=/35602026/11/12 17:02:39 INFO Received uploads request method=POST path=/35612026/11/12 17:02:39 INFO Received push request method=POST path=/35622026/11/12 17:02:39 INFO Received complete upload request method=POST path=/35632026/11/12 17:02:39 INFO Completed upload id=10 objects=23 duration=2.41354ms35642026/11/12 17:02:39 INFO Received complete push request method=POST path=/35652026/11/12 17:02:39 INFO Completed push id=5 objects=8 duration=4.013349ms35662026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=035672026/11/12 17:02:39 INFO Received push request method=POST path=/35682026/11/12 17:02:39 INFO Received push request method=POST path=/35692026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035702026/11/12 17:02:39 INFO Received complete upload request method=POST path=/35712026/11/12 17:02:39 INFO Completed upload id=15 objects=6 duration=1.534609ms35722026/11/12 17:02:39 INFO Vacuumed table table=pending_closures35732026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts35742026/11/12 17:02:39 INFO Received push request method=POST path=/35752026/11/12 17:02:39 INFO Vacuumed table table=pending_objects35762026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads35772026/11/12 17:02:39 INFO Vacuumed table table=closures35782026/11/12 17:02:39 INFO Vacuumed table table=objects35792026/11/12 17:02:39 INFO Received uploads request method=POST path=/35802026/11/12 17:02:39 INFO Received push request method=POST path=/35812026/11/12 17:02:39 INFO Received push request method=POST path=/35822026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=035832026/11/12 17:02:39 INFO Received complete push request method=POST path=/35842026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period35852026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035862026/11/12 17:02:39 INFO Received push request method=POST path=/35872026/11/12 17:02:39 INFO Completed push id=8 objects=6 duration=3.443279ms35882026/11/12 17:02:39 INFO Vacuumed table table=pending_closures35892026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts35902026/11/12 17:02:39 INFO Vacuumed table table=pending_objects35912026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads35922026/11/12 17:02:39 INFO Vacuumed table table=closures35932026/11/12 17:02:39 INFO Vacuumed table table=objects35942026/11/12 17:02:39 INFO Received uploads request method=POST path=/35952026/11/12 17:02:39 INFO Received push request method=POST path=/35962026/11/12 17:02:39 INFO Received push request method=POST path=/35972026/11/12 17:02:39 INFO Received uploads request method=POST path=/35982026/11/12 17:02:39 INFO Received push request method=POST path=/35992026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=036002026/11/12 17:02:39 INFO Received uploads request method=POST path=/36012026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period36022026/11/12 17:02:39 INFO Received uploads request method=POST path=/36032026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036042026/11/12 17:02:39 INFO Received uploads request method=POST path=/36052026/11/12 17:02:39 INFO Vacuumed table table=pending_closures36062026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts36072026/11/12 17:02:39 INFO Vacuumed table table=pending_objects36082026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads36092026/11/12 17:02:39 INFO Received uploads request method=POST path=/36102026/11/12 17:02:39 INFO Vacuumed table table=closures36112026/11/12 17:02:39 INFO Vacuumed table table=objects36122026/11/12 17:02:39 INFO Received uploads request method=POST path=/36132026/11/12 17:02:39 INFO Received complete upload request method=POST path=/36142026/11/12 17:02:39 INFO Completed upload id=17 objects=7 duration=1.49337ms36152026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=036162026/11/12 17:02:39 INFO Received complete push request method=POST path=/36172026/11/12 17:02:39 INFO Received push request method=POST path=/36182026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036192026/11/12 17:02:39 INFO Completed push id=11 objects=8 duration=3.583429ms36202026/11/12 17:02:39 INFO Vacuumed table table=pending_closures36212026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts36222026/11/12 17:02:39 INFO Received complete push request method=POST path=/36232026/11/12 17:02:39 INFO Vacuumed table table=pending_objects36242026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads36252026/11/12 17:02:39 INFO Vacuumed table table=closures36262026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=036272026/11/12 17:02:39 INFO Vacuumed table table=objects36282026/11/12 17:02:39 INFO Completed push id=19 objects=6 duration=3.584789ms36292026/11/12 17:02:39 INFO Received complete upload request method=POST path=/36302026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=036312026/11/12 17:02:39 INFO Received complete upload request method=POST path=/36322026/11/12 17:02:39 INFO Completed upload id=11 objects=6 duration=1.476629ms36332026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036342026/11/12 17:02:39 INFO Completed upload id=10 objects=17 duration=1.458819ms36352026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period36362026/11/12 17:02:39 INFO Vacuumed table table=pending_closures36372026/11/12 17:02:39 INFO Received uploads request method=POST path=/36382026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts36392026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036402026/11/12 17:02:39 INFO Vacuumed table table=pending_objects36412026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads36422026/11/12 17:02:39 INFO Vacuumed table table=pending_closures36432026/11/12 17:02:39 INFO Vacuumed table table=closures36442026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts36452026/11/12 17:02:39 INFO Vacuumed table table=objects36462026/11/12 17:02:39 INFO Received uploads request method=POST path=/36472026/11/12 17:02:39 INFO Vacuumed table table=pending_objects36482026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads36492026/11/12 17:02:39 INFO Vacuumed table table=closures36502026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=036512026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=036522026/11/12 17:02:39 INFO Vacuumed table table=objects36532026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036542026/11/12 17:02:39 INFO Vacuumed table table=pending_closures36552026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts36562026/11/12 17:02:39 INFO Vacuumed table table=pending_objects36572026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads36582026/11/12 17:02:39 INFO Received push request method=POST path=/36592026/11/12 17:02:39 INFO Vacuumed table table=closures36602026/11/12 17:02:39 INFO Vacuumed table table=objects36612026/11/12 17:02:39 INFO Received push request method=POST path=/36622026/11/12 17:02:39 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."36632026/11/12 17:02:39 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."36642026/11/12 17:02:39 INFO Received uploads request method=POST path=/36652026/11/12 17:02:39 INFO Received complete upload request method=POST path=/36662026/11/12 17:02:39 INFO Received push request method=POST path=/36672026/11/12 17:02:39 INFO Completed upload id=10 objects=6 duration=3.131339ms36682026/11/12 17:02:39 INFO Received complete upload request method=POST path=/36692026/11/12 17:02:39 INFO Received complete push request method=POST path=/36702026/11/12 17:02:39 INFO Completed upload id=15 objects=4 duration=2.77051ms36712026/11/12 17:02:39 INFO Completed push id=6 objects=26 duration=4.891839ms36722026/11/12 17:02:39 INFO Received uploads request method=POST path=/36732026/11/12 17:02:39 INFO Received push request method=POST path=/36742026/11/12 17:02:39 INFO Received uploads request method=POST path=/36752026/11/12 17:02:39 INFO Received push request method=POST path=/36762026/11/12 17:02:39 INFO Received push request method=POST path=/36772026/11/12 17:02:39 INFO Received complete push request method=POST path=/36782026/11/12 17:02:39 INFO Received complete push request method=POST path=/36792026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=036802026/11/12 17:02:39 INFO Completed push id=12 objects=15 duration=2.14237ms36812026/11/12 17:02:39 INFO Completed push id=13 objects=6 duration=5.136999ms36822026/11/12 17:02:39 INFO Received complete push request method=POST path=/36832026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period36842026/11/12 17:02:39 INFO Completed push id=9 objects=24 duration=1.79789ms36852026/11/12 17:02:39 INFO Received push request method=POST path=/36862026/11/12 17:02:39 INFO Received push request method=POST path=/36872026/11/12 17:02:39 INFO Received uploads request method=POST path=/36882026/11/12 17:02:39 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."36892026/11/12 17:02:39 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."36902026/11/12 17:02:39 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."36912026/11/12 17:02:39 INFO Received push request method=POST path=/36922026/11/12 17:02:39 INFO Received uploads request method=POST path=/36932026/11/12 17:02:39 INFO Received push request method=POST path=/36942026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=036952026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=036962026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period36972026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036982026/11/12 17:02:39 INFO Received push request method=POST path=/36992026/11/12 17:02:39 INFO Received uploads request method=POST path=/37002026/11/12 17:02:39 INFO Vacuumed table table=pending_closures37012026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=037022026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts37032026/11/12 17:02:39 INFO Received complete upload request method=POST path=/37042026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037052026/11/12 17:02:39 INFO Vacuumed table table=pending_objects37062026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads37072026/11/12 17:02:39 INFO Completed upload id=14 objects=6 duration=1.58691ms37082026/11/12 17:02:39 INFO Vacuumed table table=pending_closures37092026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period37102026/11/12 17:02:39 INFO Vacuumed table table=closures37112026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts37122026/11/12 17:02:39 INFO Vacuumed table table=objects37132026/11/12 17:02:39 INFO Vacuumed table table=pending_objects37142026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads37152026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037162026/11/12 17:02:39 INFO Vacuumed table table=closures37172026/11/12 17:02:39 INFO Vacuumed table table=pending_closures37182026/11/12 17:02:39 INFO Vacuumed table table=objects37192026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts37202026/11/12 17:02:39 INFO Received complete push request method=POST path=/37212026/11/12 17:02:39 INFO Received complete push request method=POST path=/37222026/11/12 17:02:39 INFO Vacuumed table table=pending_objects37232026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads37242026/11/12 17:02:39 INFO Vacuumed table table=closures37252026/11/12 17:02:39 INFO Completed push id=17 objects=22 duration=2.902209ms37262026/11/12 17:02:39 INFO Vacuumed table table=objects37272026/11/12 17:02:39 INFO Received push request method=POST path=/37282026/11/12 17:02:39 INFO Completed push id=11 objects=11 duration=5.963359ms37292026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=037302026/11/12 17:02:39 INFO Received complete push request method=POST path=/37312026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period37322026/11/12 17:02:39 INFO Received push request method=POST path=/37332026/11/12 17:02:39 INFO Received complete push request method=POST path=/37342026/11/12 17:02:39 INFO Completed push id=15 objects=24 duration=4.14577ms37352026/11/12 17:02:39 INFO Received complete upload request method=POST path=/37362026/11/12 17:02:39 INFO Received uploads request method=POST path=/37372026/11/12 17:02:39 INFO Completed upload id=16 objects=3 duration=1.403589ms37382026/11/12 17:02:39 INFO Received push request method=POST path=/37392026/11/12 17:02:39 INFO Completed push id=8 objects=6 duration=2.045399ms37402026/11/12 17:02:39 INFO Received complete push request method=POST path=/37412026/11/12 17:02:39 INFO Completed push id=14 objects=6 duration=2.31758ms37422026/11/12 17:02:39 INFO Received uploads request method=POST path=/37432026/11/12 17:02:39 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."37442026/11/12 17:02:39 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."37452026/11/12 17:02:39 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."37462026/11/12 17:02:39 INFO Received complete upload request method=POST path=/37472026/11/12 17:02:39 INFO Received complete push request method=POST path=/37482026/11/12 17:02:39 INFO Completed upload id=8 objects=3 duration=2.907569ms37492026/11/12 17:02:39 INFO Received uploads request method=POST path=/37502026/11/12 17:02:39 INFO Received uploads request method=POST path=/37512026/11/12 17:02:39 INFO Completed push id=9 objects=7 duration=4.09584ms37522026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=037532026/11/12 17:02:39 INFO Received complete upload request method=POST path=/37542026/11/12 17:02:39 INFO Completed upload id=10 objects=10 duration=1.763319ms37552026/11/12 17:02:39 INFO Received push request method=POST path=/37562026/11/12 17:02:39 INFO Received complete push request method=POST path=/37572026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037582026/11/12 17:02:39 INFO Vacuumed table table=pending_closures37592026/11/12 17:02:39 INFO Completed push id=12 objects=7 duration=2.26371ms37602026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts37612026/11/12 17:02:39 INFO Vacuumed table table=pending_objects37622026/11/12 17:02:39 INFO Received complete upload request method=POST path=/37632026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads37642026/11/12 17:02:39 INFO Vacuumed table table=closures37652026/11/12 17:02:39 INFO Completed upload id=7 objects=3 duration=1.55688ms37662026/11/12 17:02:39 INFO Vacuumed table table=objects37672026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=037682026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period37692026/11/12 17:02:39 INFO Received uploads request method=POST path=/37702026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037712026/11/12 17:02:39 INFO Vacuumed table table=pending_closures37722026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts37732026/11/12 17:02:39 INFO Vacuumed table table=pending_objects37742026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads37752026/11/12 17:02:39 INFO Vacuumed table table=closures37762026/11/12 17:02:39 INFO Received uploads request method=POST path=/37772026/11/12 17:02:39 INFO Received complete push request method=POST path=/37782026/11/12 17:02:39 INFO Vacuumed table table=objects37792026/11/12 17:02:39 INFO Completed push id=17 objects=6 duration=2.51397ms37802026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=037812026/11/12 17:02:39 INFO Received push request method=POST path=/37822026/11/12 17:02:39 INFO Received complete push request method=POST path=/37832026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=037842026/11/12 17:02:39 INFO Completed push id=4 objects=6 duration=2.430539ms37852026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period37862026/11/12 17:02:39 INFO Received complete upload request method=POST path=/37872026/11/12 17:02:39 INFO Completed upload id=10 objects=7 duration=1.85988ms37882026/11/12 17:02:39 INFO Received uploads request method=POST path=/37892026/11/12 17:02:39 INFO Received complete push request method=POST path=/37902026/11/12 17:02:39 INFO Received push request method=POST path=/37912026/11/12 17:02:39 INFO Completed push id=18 objects=23 duration=2.13763ms37922026/11/12 17:02:39 INFO Received uploads request method=POST path=/37932026/11/12 17:02:39 INFO Received complete upload request method=POST path=/37942026/11/12 17:02:39 INFO Received complete push request method=POST path=/37952026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=037962026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=037972026/11/12 17:02:39 INFO Completed push id=15 objects=7 duration=2.19595ms37982026/11/12 17:02:39 INFO Completed upload id=17 objects=6 duration=2.917779ms37992026/11/12 17:02:39 INFO Vacuumed table table=pending_closures38002026/11/12 17:02:39 INFO Received uploads request method=POST path=/38012026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts38022026/11/12 17:02:39 INFO Vacuumed table table=pending_objects38032026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads38042026/11/12 17:02:39 INFO Received complete upload request method=POST path=/38052026/11/12 17:02:39 INFO Vacuumed table table=closures38062026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=038072026/11/12 17:02:39 INFO Vacuumed table table=objects38082026/11/12 17:02:39 INFO Completed upload id=7 objects=10 duration=2.175039ms38092026/11/12 17:02:39 INFO Vacuumed table table=pending_closures38102026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts38112026/11/12 17:02:39 INFO Vacuumed table table=pending_objects38122026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads38132026/11/12 17:02:39 INFO Vacuumed table table=closures38142026/11/12 17:02:39 INFO Received uploads request method=POST path=/38152026/11/12 17:02:39 INFO Vacuumed table table=objects38162026/11/12 17:02:39 INFO Received complete push request method=POST path=/38172026/11/12 17:02:39 INFO Completed push id=4 objects=7 duration=1.97123ms38182026/11/12 17:02:39 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."38192026/11/12 17:02:39 INFO Received uploads request method=POST path=/38202026/11/12 17:02:39 INFO Received uploads request method=POST path=/38212026/11/12 17:02:39 INFO Received push request method=POST path=/38222026/11/12 17:02:39 INFO Received complete push request method=POST path=/38232026/11/12 17:02:39 INFO Completed push id=6 objects=6 duration=2.24028ms38242026/11/12 17:02:39 INFO Received complete upload request method=POST path=/38252026/11/12 17:02:39 INFO Received complete upload request method=POST path=/38262026/11/12 17:02:39 INFO Completed upload id=9 objects=2 duration=1.54367ms38272026/11/12 17:02:39 INFO Completed upload id=18 objects=4 duration=2.06171ms38282026/11/12 17:02:39 INFO Received uploads request method=POST path=/38292026/11/12 17:02:39 INFO Received uploads request method=POST path=/38302026/11/12 17:02:39 INFO Received uploads request method=POST path=/38312026/11/12 17:02:39 INFO Received complete upload request method=POST path=/38322026/11/12 17:02:39 INFO Completed upload id=6 objects=6 duration=1.49455ms38332026/11/12 17:02:39 INFO Received push request method=POST path=/38342026/11/12 17:02:39 INFO Received push request method=POST path=/38352026/11/12 17:02:39 INFO Received uploads request method=POST path=/38362026/11/12 17:02:39 INFO Received push request method=POST path=/38372026/11/12 17:02:39 INFO Received complete push request method=POST path=/38382026/11/12 17:02:39 INFO Received complete push request method=POST path=/38392026/11/12 17:02:39 INFO Completed push id=11 objects=9 duration=2.00347ms38402026/11/12 17:02:39 INFO Completed push id=8 objects=3 duration=1.97382ms38412026/11/12 17:02:39 INFO Received uploads request method=POST path=/38422026/11/12 17:02:39 INFO Received uploads request method=POST path=/38432026/11/12 17:02:39 INFO Received push request method=POST path=/38442026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=038452026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period38462026/11/12 17:02:39 INFO Received uploads request method=POST path=/38472026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038482026/11/12 17:02:39 INFO Received push request method=POST path=/38492026/11/12 17:02:39 INFO Received complete upload request method=POST path=/38502026/11/12 17:02:39 INFO Vacuumed table table=pending_closures38512026/11/12 17:02:39 INFO Received push request method=POST path=/38522026/11/12 17:02:39 INFO Received complete push request method=POST path=/38532026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts38542026/11/12 17:02:39 INFO Completed upload id=18 objects=4 duration=2.06065ms38552026/11/12 17:02:39 INFO Vacuumed table table=pending_objects38562026/11/12 17:02:39 INFO Completed push id=17 objects=6 duration=2.31067ms38572026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads38582026/11/12 17:02:39 INFO Vacuumed table table=closures38592026/11/12 17:02:39 INFO Vacuumed table table=objects38602026/11/12 17:02:39 INFO Received complete push request method=POST path=/38612026/11/12 17:02:39 INFO Received complete upload request method=POST path=/38622026/11/12 17:02:39 INFO Completed upload id=19 objects=25 duration=3.35116ms38632026/11/12 17:02:39 INFO Received uploads request method=POST path=/38642026/11/12 17:02:39 INFO Completed push id=3 objects=21 duration=4.847779ms38652026/11/12 17:02:39 INFO Received uploads request method=POST path=/38662026/11/12 17:02:39 INFO Received uploads request method=POST path=/38672026/11/12 17:02:39 INFO Received uploads request method=POST path=/38682026/11/12 17:02:39 INFO Received push request method=POST path=/38692026/11/12 17:02:39 INFO Received push request method=POST path=/38702026/11/12 17:02:39 INFO Received push request method=POST path=/38712026/11/12 17:02:39 INFO Received complete push request method=POST path=/38722026/11/12 17:02:39 INFO Completed push id=10 objects=15 duration=2.33509ms38732026/11/12 17:02:39 INFO Received uploads request method=POST path=/38742026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=038752026/11/12 17:02:39 INFO Received uploads request method=POST path=/38762026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period38772026/11/12 17:02:39 INFO Received uploads request method=POST path=/38782026/11/12 17:02:39 INFO Received complete upload request method=POST path=/38792026/11/12 17:02:39 INFO Completed upload id=13 objects=10 duration=1.06945ms38802026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=038812026/11/12 17:02:39 INFO Vacuumed table table=pending_closures38822026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts38832026/11/12 17:02:39 INFO Received complete upload request method=POST path=/38842026/11/12 17:02:39 INFO Vacuumed table table=pending_objects38852026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads38862026/11/12 17:02:39 INFO Received push request method=POST path=/38872026/11/12 17:02:39 INFO Vacuumed table table=closures38882026/11/12 17:02:39 INFO Completed upload id=11 objects=7 duration=1.80232ms38892026/11/12 17:02:39 INFO Received complete push request method=POST path=/38902026/11/12 17:02:39 INFO Vacuumed table table=objects38912026/11/12 17:02:39 INFO Received complete push request method=POST path=/38922026/11/12 17:02:39 INFO Received push request method=POST path=/38932026/11/12 17:02:39 INFO Received complete upload request method=POST path=/38942026/11/12 17:02:39 INFO Completed push id=7 objects=7 duration=1.60283ms38952026/11/12 17:02:39 INFO Received uploads request method=POST path=/38962026/11/12 17:02:39 INFO Completed push id=14 objects=3 duration=1.622649ms38972026/11/12 17:02:39 INFO Completed upload id=22 objects=2 duration=1.534799ms38982026/11/12 17:02:39 INFO Received uploads request method=POST path=/38992026/11/12 17:02:39 INFO Received push request method=POST path=/39002026/11/12 17:02:39 INFO Received complete push request method=POST path=/39012026/11/12 17:02:39 INFO Received uploads request method=POST path=/39022026/11/12 17:02:39 INFO Completed push id=12 objects=2 duration=3.596109ms39032026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=039042026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period39052026/11/12 17:02:39 INFO Received uploads request method=POST path=/39062026/11/12 17:02:39 INFO Received uploads request method=POST path=/39072026/11/12 17:02:39 INFO Received complete upload request method=POST path=/39082026/11/12 17:02:39 INFO Completed upload id=15 objects=6 duration=1.05144ms39092026/11/12 17:02:39 INFO Received uploads request method=POST path=/39102026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=039112026/11/12 17:02:39 INFO Received push request method=POST path=/39122026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=039132026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=039142026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period39152026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039162026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039172026/11/12 17:02:39 INFO Vacuumed table table=pending_closures39182026/11/12 17:02:39 INFO Received uploads request method=POST path=/39192026/11/12 17:02:39 INFO Vacuumed table table=pending_closures39202026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts39212026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=039222026/11/12 17:02:39 INFO Vacuumed table table=pending_objects39232026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts39242026/11/12 17:02:39 INFO Received push request method=POST path=/39252026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads39262026/11/12 17:02:39 INFO Vacuumed table table=pending_closures39272026/11/12 17:02:39 INFO Vacuumed table table=pending_objects39282026/11/12 17:02:39 INFO Vacuumed table table=closures39292026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts39302026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads39312026/11/12 17:02:39 INFO Received uploads request method=POST path=/39322026/11/12 17:02:39 INFO Vacuumed table table=closures39332026/11/12 17:02:39 INFO Vacuumed table table=pending_objects39342026/11/12 17:02:39 INFO Vacuumed table table=objects39352026/11/12 17:02:39 INFO Vacuumed table table=objects39362026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads39372026/11/12 17:02:39 INFO Vacuumed table table=closures39382026/11/12 17:02:39 INFO Vacuumed table table=objects39392026/11/12 17:02:39 INFO Received uploads request method=POST path=/39402026/11/12 17:02:39 INFO Received push request method=POST path=/39412026/11/12 17:02:39 INFO Received push request method=POST path=/39422026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=039432026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period39442026/11/12 17:02:39 INFO Received uploads request method=POST path=/39452026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039462026/11/12 17:02:39 INFO Vacuumed table table=pending_closures39472026/11/12 17:02:39 INFO Received uploads request method=POST path=/39482026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts39492026/11/12 17:02:39 INFO Vacuumed table table=pending_objects39502026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads39512026/11/12 17:02:39 INFO Vacuumed table table=closures39522026/11/12 17:02:39 INFO Vacuumed table table=objects39532026/11/12 17:02:39 INFO Received push request method=POST path=/39542026/11/12 17:02:39 INFO Received uploads request method=POST path=/39552026/11/12 17:02:39 INFO Received complete upload request method=POST path=/39562026/11/12 17:02:39 INFO Completed upload id=18 objects=11 duration=1.61624ms39572026/11/12 17:02:39 INFO Received uploads request method=POST path=/39582026/11/12 17:02:39 INFO Received uploads request method=POST path=/39592026/11/12 17:02:39 INFO Received uploads request method=POST path=/39602026/11/12 17:02:39 INFO Received push request method=POST path=/39612026/11/12 17:02:39 INFO Received push request method=POST path=/39622026/11/12 17:02:39 INFO Received push request method=POST path=/39632026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=11 objects-failed-to-delete=039642026/11/12 17:02:39 INFO Received uploads request method=POST path=/39652026/11/12 17:02:39 INFO Vacuumed table table=pending_closures39662026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts39672026/11/12 17:02:39 INFO Received uploads request method=POST path=/39682026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=039692026/11/12 17:02:39 INFO Vacuumed table table=pending_objects39702026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads39712026/11/12 17:02:39 INFO Vacuumed table table=closures39722026/11/12 17:02:39 INFO Received complete push request method=POST path=/39732026/11/12 17:02:39 INFO Vacuumed table table=objects39742026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period39752026/11/12 17:02:39 INFO Completed push id=1 objects=11 duration=2.29577ms39762026/11/12 17:02:39 INFO Received complete upload request method=POST path=/39772026/11/12 17:02:39 INFO Completed upload id=15 objects=9 duration=1.49019ms39782026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=039792026/11/12 17:02:39 INFO Received push request method=POST path=/39802026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=039812026/11/12 17:02:39 INFO Vacuumed table table=pending_closures39822026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=039832026/11/12 17:02:39 INFO Received complete push request method=POST path=/39842026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts39852026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period39862026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period39872026/11/12 17:02:39 INFO Vacuumed table table=pending_objects39882026/11/12 17:02:39 INFO Completed push id=15 objects=5 duration=2.07098ms39892026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads39902026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039912026/11/12 17:02:39 INFO Vacuumed table table=closures39922026/11/12 17:02:39 INFO Received uploads request method=POST path=/39932026/11/12 17:02:39 INFO Vacuumed table table=pending_closures39942026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039952026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts39962026/11/12 17:02:39 INFO Vacuumed table table=objects39972026/11/12 17:02:39 INFO Vacuumed table table=pending_objects39982026/11/12 17:02:39 INFO Vacuumed table table=pending_closures39992026/11/12 17:02:39 INFO Received push request method=POST path=/40002026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads40012026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts40022026/11/12 17:02:39 INFO Vacuumed table table=closures40032026/11/12 17:02:39 INFO Received complete push request method=POST path=/40042026/11/12 17:02:39 INFO Vacuumed table table=objects40052026/11/12 17:02:39 INFO Vacuumed table table=pending_objects40062026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads40072026/11/12 17:02:39 INFO Vacuumed table table=closures40082026/11/12 17:02:39 INFO Completed push id=16 objects=27 duration=2.331839ms40092026/11/12 17:02:39 INFO Received push request method=POST path=/40102026/11/12 17:02:39 INFO Vacuumed table table=objects40112026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=040122026/11/12 17:02:39 INFO Received push request method=POST path=/40132026/11/12 17:02:39 INFO Received uploads request method=POST path=/40142026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040152026/11/12 17:02:39 INFO Vacuumed table table=pending_closures40162026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts40172026/11/12 17:02:39 INFO Received push request method=POST path=/40182026/11/12 17:02:39 INFO Vacuumed table table=pending_objects40192026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads40202026/11/12 17:02:39 INFO Vacuumed table table=closures40212026/11/12 17:02:39 INFO Vacuumed table table=objects40222026/11/12 17:02:39 INFO Received complete upload request method=POST path=/40232026/11/12 17:02:39 INFO Completed upload id=16 objects=3 duration=1.08882ms40242026/11/12 17:02:39 INFO Received uploads request method=POST path=/40252026/11/12 17:02:39 INFO Received push request method=POST path=/40262026/11/12 17:02:39 INFO Received push request method=POST path=/40272026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=040282026/11/12 17:02:39 INFO Received uploads request method=POST path=/40292026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period40302026/11/12 17:02:39 INFO Received push request method=POST path=/40312026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040322026/11/12 17:02:39 INFO Vacuumed table table=pending_closures40332026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts40342026/11/12 17:02:39 INFO Received push request method=POST path=/40352026/11/12 17:02:39 INFO Received complete upload request method=POST path=/40362026/11/12 17:02:39 INFO Vacuumed table table=pending_objects40372026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads40382026/11/12 17:02:39 INFO Vacuumed table table=closures40392026/11/12 17:02:39 INFO Received push request method=POST path=/40402026/11/12 17:02:39 INFO Received complete push request method=POST path=/40412026/11/12 17:02:39 INFO Vacuumed table table=objects40422026/11/12 17:02:39 INFO Completed upload id=23 objects=15 duration=3.236429ms40432026/11/12 17:02:39 INFO Received push request method=POST path=/40442026/11/12 17:02:39 INFO Completed push id=19 objects=8 duration=4.275499ms40452026/11/12 17:02:39 INFO Received complete push request method=POST path=/40462026/11/12 17:02:39 INFO Received push request method=POST path=/40472026/11/12 17:02:39 INFO Completed push id=21 objects=17 duration=2.4212ms40482026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=040492026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period40502026/11/12 17:02:39 INFO Received push request method=POST path=/40512026/11/12 17:02:39 INFO Received complete push request method=POST path=/40522026/11/12 17:02:39 INFO Completed push id=14 objects=4 duration=3.227579ms40532026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=040542026/11/12 17:02:39 INFO Received uploads request method=POST path=/40552026/11/12 17:02:39 INFO Received complete push request method=POST path=/40562026/11/12 17:02:39 INFO Received push request method=POST path=/40572026/11/12 17:02:39 INFO Received uploads request method=POST path=/40582026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=040592026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040602026/11/12 17:02:39 INFO Completed push id=13 objects=15 duration=1.93898ms40612026/11/12 17:02:39 INFO Received uploads request method=POST path=/40622026/11/12 17:02:39 INFO Vacuumed table table=pending_closures40632026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period40642026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts40652026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040662026/11/12 17:02:39 INFO Vacuumed table table=pending_objects40672026/11/12 17:02:39 INFO Received uploads request method=POST path=/40682026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads40692026/11/12 17:02:39 INFO Vacuumed table table=pending_closures40702026/11/12 17:02:39 INFO Received complete push request method=POST path=/40712026/11/12 17:02:39 INFO Received push request method=POST path=/40722026/11/12 17:02:39 INFO Received push request method=POST path=/40732026/11/12 17:02:39 INFO Vacuumed table table=closures40742026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts40752026/11/12 17:02:39 INFO Vacuumed table table=objects40762026/11/12 17:02:39 INFO Received push request method=POST path=/40772026/11/12 17:02:39 INFO Vacuumed table table=pending_objects40782026/11/12 17:02:39 INFO Completed push id=19 objects=7 duration=2.086699ms40792026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads40802026/11/12 17:02:39 INFO Vacuumed table table=closures40812026/11/12 17:02:39 INFO Vacuumed table table=objects40822026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=040832026/11/12 17:02:39 INFO Received uploads request method=POST path=/40842026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=040852026/11/12 17:02:39 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."40862026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=040872026/11/12 17:02:39 INFO Received push request method=POST path=/40882026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040892026/11/12 17:02:39 INFO Vacuumed table table=pending_closures40902026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts40912026/11/12 17:02:39 INFO Vacuumed table table=pending_objects40922026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads40932026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040942026/11/12 17:02:39 INFO Vacuumed table table=closures40952026/11/12 17:02:39 INFO Vacuumed table table=objects40962026/11/12 17:02:39 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."40972026/11/12 17:02:39 INFO Vacuumed table table=pending_closures40982026/11/12 17:02:39 INFO Received push request method=POST path=/40992026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=041002026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts41012026/11/12 17:02:39 INFO Received complete push request method=POST path=/41022026/11/12 17:02:39 INFO Vacuumed table table=pending_objects41032026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads41042026/11/12 17:02:39 INFO Completed push id=18 objects=10 duration=2.241729ms41052026/11/12 17:02:39 INFO Vacuumed table table=closures41062026/11/12 17:02:39 INFO Received push request method=POST path=/41072026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period41082026/11/12 17:02:39 INFO Vacuumed table table=objects41092026/11/12 17:02:39 INFO Received complete upload request method=POST path=/41102026/11/12 17:02:39 INFO Received push request method=POST path=/41112026/11/12 17:02:39 INFO Completed upload id=19 objects=6 duration=1.55178ms41122026/11/12 17:02:39 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."41132026/11/12 17:02:39 INFO Received uploads request method=POST path=/41142026/11/12 17:02:39 INFO Received uploads request method=POST path=/41152026/11/12 17:02:39 INFO Received push request method=POST path=/41162026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=041172026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=041182026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period41192026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041202026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period41212026/11/12 17:02:39 INFO Vacuumed table table=pending_closures41222026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts41232026/11/12 17:02:39 INFO Vacuumed table table=pending_objects41242026/11/12 17:02:39 INFO Received complete push request method=POST path=/41252026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads41262026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041272026/11/12 17:02:39 INFO Vacuumed table table=closures41282026/11/12 17:02:39 INFO Vacuumed table table=objects41292026/11/12 17:02:39 INFO Vacuumed table table=pending_closures41302026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts41312026/11/12 17:02:39 INFO Received push request method=POST path=/41322026/11/12 17:02:39 INFO Completed push id=22 objects=3 duration=3.02359ms41332026/11/12 17:02:39 INFO Vacuumed table table=pending_objects41342026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=041352026/11/12 17:02:39 INFO Received uploads request method=POST path=/41362026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads41372026/11/12 17:02:39 INFO Vacuumed table table=closures41382026/11/12 17:02:39 INFO Vacuumed table table=objects41392026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period41402026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041412026/11/12 17:02:39 INFO Vacuumed table table=pending_closures41422026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts41432026/11/12 17:02:39 INFO Vacuumed table table=pending_objects41442026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads41452026/11/12 17:02:39 INFO Received complete upload request method=POST path=/41462026/11/12 17:02:39 INFO Vacuumed table table=closures41472026/11/12 17:02:39 INFO Vacuumed table table=objects41482026/11/12 17:02:39 INFO Completed upload id=22 objects=7 duration=1.83558ms41492026/11/12 17:02:39 INFO Received complete push request method=POST path=/41502026/11/12 17:02:39 INFO Received complete push request method=POST path=/41512026/11/12 17:02:39 INFO Completed push id=20 objects=3 duration=1.8155ms41522026/11/12 17:02:39 INFO Completed push id=16 objects=4 duration=3.505949ms41532026/11/12 17:02:39 INFO Received push request method=POST path=/41542026/11/12 17:02:39 INFO Received uploads request method=POST path=/41552026/11/12 17:02:39 INFO Received complete push request method=POST path=/41562026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=041572026/11/12 17:02:39 INFO Completed push id=15 objects=4 duration=1.99807ms41582026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period41592026/11/12 17:02:39 INFO Received push request method=POST path=/41602026/11/12 17:02:39 INFO Received uploads request method=POST path=/41612026/11/12 17:02:39 INFO Received complete upload request method=POST path=/41622026/11/12 17:02:39 INFO Completed upload id=14 objects=8 duration=2.46619ms41632026/11/12 17:02:39 INFO Received uploads request method=POST path=/41642026/11/12 17:02:39 INFO Received uploads request method=POST path=/41652026/11/12 17:02:39 INFO Received push request method=POST path=/41662026/11/12 17:02:39 INFO Received uploads request method=POST path=/41672026/11/12 17:02:39 INFO Received complete upload request method=POST path=/41682026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=041692026/11/12 17:02:39 INFO Completed upload id=23 objects=16 duration=2.894359ms41702026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=041712026/11/12 17:02:39 INFO Received uploads request method=POST path=/41722026/11/12 17:02:39 WARN Force mode enabled - objects will be deleted immediately without grace period41732026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=5 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041742026/11/12 17:02:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041752026/11/12 17:02:39 INFO Vacuumed table table=pending_closures41762026/11/12 17:02:39 INFO Vacuumed table table=pending_closures41772026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts41782026/11/12 17:02:39 INFO Vacuumed table table=commit_receipts41792026/11/12 17:02:39 INFO Vacuumed table table=pending_objects41802026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads41812026/11/12 17:02:39 INFO Vacuumed table table=pending_objects41822026/11/12 17:02:39 INFO Vacuumed table table=multipart_uploads41832026/11/12 17:02:39 INFO Vacuumed table table=closures41842026/11/12 17:02:39 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."41852026/11/12 17:02:39 INFO Vacuumed table table=closures41862026/11/12 17:02:39 INFO Vacuumed table table=objects41872026/11/12 17:02:39 INFO Aborted multipart uploads count=0 kept=041882026/11/12 17:02:39 INFO Vacuumed table table=objects41892026/11/12 17:02:39 INFO Received complete push request method=POST path=/41902026/11/12 17:02:40 INFO Completed push id=4 objects=6 duration=1.832879ms41912026/11/12 17:02:40 INFO Received uploads request method=POST path=/41922026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period41932026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041942026/11/12 17:02:40 INFO Received uploads request method=POST path=/41952026/11/12 17:02:40 INFO Vacuumed table table=pending_closures41962026/11/12 17:02:40 INFO Received complete upload request method=POST path=/41972026/11/12 17:02:40 INFO Completed upload id=20 objects=5 duration=1.2109ms41982026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts41992026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=042002026/11/12 17:02:40 INFO Vacuumed table table=pending_objects42012026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads42022026/11/12 17:02:40 INFO Received uploads request method=POST path=/42032026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period42042026/11/12 17:02:40 INFO Vacuumed table table=closures42052026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042062026/11/12 17:02:40 INFO Vacuumed table table=objects42072026/11/12 17:02:40 INFO Received uploads request method=POST path=/42082026/11/12 17:02:40 INFO Vacuumed table table=pending_closures42092026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts42102026/11/12 17:02:40 INFO Vacuumed table table=pending_objects42112026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads42122026/11/12 17:02:40 INFO Vacuumed table table=closures42132026/11/12 17:02:40 INFO Vacuumed table table=objects42142026/11/12 17:02:40 INFO Received complete upload request method=POST path=/42152026/11/12 17:02:40 INFO Completed upload id=12 objects=26 duration=3.089209ms42162026/11/12 17:02:40 INFO Received uploads request method=POST path=/42172026/11/12 17:02:40 INFO Received push request method=POST path=/42182026/11/12 17:02:40 INFO Received uploads request method=POST path=/42192026/11/12 17:02:40 INFO Received uploads request method=POST path=/42202026/11/12 17:02:40 INFO Received complete push request method=POST path=/42212026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=042222026/11/12 17:02:40 INFO Received uploads request method=POST path=/42232026/11/12 17:02:40 INFO Completed push id=24 objects=9 duration=2.296299ms42242026/11/12 17:02:40 INFO Received push request method=POST path=/42252026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period42262026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042272026/11/12 17:02:40 INFO Vacuumed table table=pending_closures42282026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts42292026/11/12 17:02:40 INFO Received uploads request method=POST path=/42302026/11/12 17:02:40 INFO Received uploads request method=POST path=/42312026/11/12 17:02:40 INFO Vacuumed table table=pending_objects42322026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads42332026/11/12 17:02:40 INFO Vacuumed table table=closures42342026/11/12 17:02:40 INFO Vacuumed table table=objects42352026/11/12 17:02:40 INFO Received uploads request method=POST path=/42362026/11/12 17:02:40 INFO Received complete push request method=POST path=/42372026/11/12 17:02:40 INFO Completed push id=14 objects=3 duration=1.533539ms42382026/11/12 17:02:40 INFO Received push request method=POST path=/42392026/11/12 17:02:40 INFO Received push request method=POST path=/42402026/11/12 17:02:40 INFO Received uploads request method=POST path=/42412026/11/12 17:02:40 INFO Received push request method=POST path=/42422026/11/12 17:02:40 INFO Received push request method=POST path=/42432026/11/12 17:02:40 INFO Received push request method=POST path=/42442026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=042452026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period42462026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042472026/11/12 17:02:40 INFO Vacuumed table table=pending_closures42482026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts42492026/11/12 17:02:40 INFO Vacuumed table table=pending_objects42502026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads42512026/11/12 17:02:40 INFO Vacuumed table table=closures42522026/11/12 17:02:40 INFO Vacuumed table table=objects42532026/11/12 17:02:40 INFO Received uploads request method=POST path=/42542026/11/12 17:02:40 INFO Received push request method=POST path=/42552026/11/12 17:02:40 INFO Received push request method=POST path=/42562026/11/12 17:02:40 INFO Received uploads request method=POST path=/42572026/11/12 17:02:40 INFO Received uploads request method=POST path=/42582026/11/12 17:02:40 INFO Received complete push request method=POST path=/42592026/11/12 17:02:40 INFO Received push request method=POST path=/42602026/11/12 17:02:40 INFO Received push request method=POST path=/42612026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=042622026/11/12 17:02:40 INFO Completed push id=16 objects=16 duration=4.611809ms42632026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period42642026/11/12 17:02:40 INFO Received push request method=POST path=/42652026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042662026/11/12 17:02:40 INFO Vacuumed table table=pending_closures42672026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts42682026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=042692026/11/12 17:02:40 INFO Vacuumed table table=pending_objects42702026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads42712026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period42722026/11/12 17:02:40 INFO Vacuumed table table=closures42732026/11/12 17:02:40 INFO Vacuumed table table=objects42742026/11/12 17:02:40 INFO Received uploads request method=POST path=/42752026/11/12 17:02:40 INFO Received uploads request method=POST path=/42762026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=042772026/11/12 17:02:40 INFO Received push request method=POST path=/42782026/11/12 17:02:40 INFO Vacuumed table table=pending_closures42792026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts42802026/11/12 17:02:40 INFO Vacuumed table table=pending_objects42812026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads42822026/11/12 17:02:40 INFO Vacuumed table table=closures42832026/11/12 17:02:40 INFO Vacuumed table table=objects42842026/11/12 17:02:40 INFO Received complete push request method=POST path=/42852026/11/12 17:02:40 INFO Completed push id=26 objects=14 duration=2.994159ms42862026/11/12 17:02:40 INFO Received complete push request method=POST path=/42872026/11/12 17:02:40 INFO Received push request method=POST path=/42882026/11/12 17:02:40 INFO Completed push id=16 objects=8 duration=2.42531ms42892026/11/12 17:02:40 INFO Received push request method=POST path=/42902026/11/12 17:02:40 INFO Received uploads request method=POST path=/42912026/11/12 17:02:40 INFO Received complete upload request method=POST path=/42922026/11/12 17:02:40 INFO Completed upload id=20 objects=4 duration=1.11717ms42932026/11/12 17:02:40 INFO Received uploads request method=POST path=/42942026/11/12 17:02:40 INFO Received complete upload request method=POST path=/42952026/11/12 17:02:40 INFO Completed upload id=22 objects=6 duration=1.19383ms42962026/11/12 17:02:40 INFO Received push request method=POST path=/42972026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=042982026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=042992026/11/12 17:02:40 INFO Received complete push request method=POST path=/43002026/11/12 17:02:40 INFO Received uploads request method=POST path=/43012026/11/12 17:02:40 INFO Received push request method=POST path=/43022026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=043032026/11/12 17:02:40 INFO Completed push id=14 objects=3 duration=3.382929ms43042026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period43052026/11/12 17:02:40 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."43062026/11/12 17:02:40 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."43072026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043082026/11/12 17:02:40 INFO Vacuumed table table=pending_closures43092026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts43102026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=043112026/11/12 17:02:40 INFO Vacuumed table table=pending_objects43122026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads43132026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period43142026/11/12 17:02:40 INFO Vacuumed table table=closures43152026/11/12 17:02:40 INFO Vacuumed table table=objects43162026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043172026/11/12 17:02:40 INFO Vacuumed table table=pending_closures43182026/11/12 17:02:40 INFO Received uploads request method=POST path=/43192026/11/12 17:02:40 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."43202026/11/12 17:02:40 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."43212026/11/12 17:02:40 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."43222026/11/12 17:02:40 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."43232026/11/12 17:02:40 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."43242026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts43252026/11/12 17:02:40 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."43262026/11/12 17:02:40 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."43272026/11/12 17:02:40 INFO Received uploads request method=POST path=/43282026/11/12 17:02:40 INFO Vacuumed table table=pending_objects43292026/11/12 17:02:40 INFO Received complete push request method=POST path=/43302026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads43312026/11/12 17:02:40 INFO Vacuumed table table=closures43322026/11/12 17:02:40 INFO Received uploads request method=POST path=/43332026/11/12 17:02:40 INFO Completed push id=18 objects=6 duration=1.830039ms43342026/11/12 17:02:40 INFO Vacuumed table table=objects43352026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=043362026/11/12 17:02:40 INFO Received complete push request method=POST path=/43372026/11/12 17:02:40 INFO Completed push id=16 objects=5 duration=1.40743ms43382026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043392026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=043402026/11/12 17:02:40 INFO Vacuumed table table=pending_closures43412026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts43422026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043432026/11/12 17:02:40 INFO Vacuumed table table=pending_objects43442026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads43452026/11/12 17:02:40 INFO Received complete push request method=POST path=/43462026/11/12 17:02:40 INFO Vacuumed table table=closures43472026/11/12 17:02:40 INFO Vacuumed table table=pending_closures43482026/11/12 17:02:40 INFO Received complete upload request method=POST path=/43492026/11/12 17:02:40 INFO Vacuumed table table=objects43502026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts43512026/11/12 17:02:40 INFO Completed push id=21 objects=9 duration=2.677909ms43522026/11/12 17:02:40 INFO Vacuumed table table=pending_objects43532026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads43542026/11/12 17:02:40 INFO Received uploads request method=POST path=/43552026/11/12 17:02:40 INFO Completed upload id=14 objects=16 duration=2.92666ms43562026/11/12 17:02:40 INFO Vacuumed table table=closures43572026/11/12 17:02:40 INFO Vacuumed table table=objects43582026/11/12 17:02:40 INFO Received complete push request method=POST path=/43592026/11/12 17:02:40 INFO Received uploads request method=POST path=/43602026/11/12 17:02:40 INFO Received complete push request method=POST path=/43612026/11/12 17:02:40 INFO Completed push id=11 objects=6 duration=3.672549ms43622026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=043632026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=043642026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period43652026/11/12 17:02:40 INFO Received complete upload request method=POST path=/43662026/11/12 17:02:40 INFO Completed push id=7 objects=15 duration=3.388469ms43672026/11/12 17:02:40 INFO Completed upload id=23 objects=9 duration=892.51µs43682026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=043692026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043702026/11/12 17:02:40 INFO Vacuumed table table=pending_closures43712026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043722026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts43732026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=043742026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period43752026/11/12 17:02:40 INFO Vacuumed table table=pending_closures43762026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period43772026/11/12 17:02:40 INFO Vacuumed table table=pending_objects43782026/11/12 17:02:40 INFO Received complete upload request method=POST path=/43792026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads43802026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts43812026/11/12 17:02:40 INFO Vacuumed table table=closures43822026/11/12 17:02:40 INFO Vacuumed table table=pending_objects43832026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043842026/11/12 17:02:40 INFO Vacuumed table table=objects43852026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads43862026/11/12 17:02:40 INFO Vacuumed table table=closures43872026/11/12 17:02:40 INFO Completed upload id=16 objects=17 duration=3.31156ms43882026/11/12 17:02:40 INFO Vacuumed table table=pending_closures43892026/11/12 17:02:40 INFO Vacuumed table table=objects43902026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts43912026/11/12 17:02:40 INFO Vacuumed table table=pending_objects43922026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads43932026/11/12 17:02:40 INFO Vacuumed table table=closures43942026/11/12 17:02:40 INFO Received uploads request method=POST path=/43952026/11/12 17:02:40 INFO Vacuumed table table=objects43962026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=043972026/11/12 17:02:40 INFO Received uploads request method=POST path=/43982026/11/12 17:02:40 INFO Received uploads request method=POST path=/43992026/11/12 17:02:40 INFO Vacuumed table table=pending_closures44002026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts44012026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=044022026/11/12 17:02:40 INFO Vacuumed table table=pending_objects44032026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads44042026/11/12 17:02:40 INFO Vacuumed table table=closures44052026/11/12 17:02:40 INFO Vacuumed table table=objects44062026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=044072026/11/12 17:02:40 INFO Received complete push request method=POST path=/44082026/11/12 17:02:40 INFO Vacuumed table table=pending_closures44092026/11/12 17:02:40 INFO Completed push id=18 objects=6 duration=1.509189ms44102026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts44112026/11/12 17:02:40 INFO Vacuumed table table=pending_objects44122026/11/12 17:02:40 INFO Received complete upload request method=POST path=/44132026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads44142026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=044152026/11/12 17:02:40 INFO Vacuumed table table=closures44162026/11/12 17:02:40 INFO Completed upload id=33 objects=5 duration=1.850399ms44172026/11/12 17:02:40 INFO Vacuumed table table=objects44182026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period44192026/11/12 17:02:40 INFO Received push request method=POST path=/44202026/11/12 17:02:40 INFO Received uploads request method=POST path=/44212026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=044222026/11/12 17:02:40 INFO Received push request method=POST path=/44232026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044242026/11/12 17:02:40 INFO Received push request method=POST path=/44252026/11/12 17:02:40 INFO Vacuumed table table=pending_closures44262026/11/12 17:02:40 INFO Received complete upload request method=POST path=/44272026/11/12 17:02:40 INFO Received complete upload request method=POST path=/44282026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts44292026/11/12 17:02:40 INFO Vacuumed table table=pending_objects44302026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads44312026/11/12 17:02:40 INFO Completed upload id=21 objects=7 duration=2.798719ms44322026/11/12 17:02:40 INFO Completed upload id=17 objects=25 duration=3.039089ms44332026/11/12 17:02:40 INFO Vacuumed table table=closures44342026/11/12 17:02:40 INFO Vacuumed table table=objects44352026/11/12 17:02:40 INFO Received complete push request method=POST path=/44362026/11/12 17:02:40 INFO Received push request method=POST path=/44372026/11/12 17:02:40 INFO Completed push id=21 objects=3 duration=1.454819ms44382026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=044392026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=044402026/11/12 17:02:40 INFO Received push request method=POST path=/44412026/11/12 17:02:40 INFO Received complete push request method=POST path=/44422026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044432026/11/12 17:02:40 INFO Completed push id=17 objects=11 duration=2.31803ms44442026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044452026/11/12 17:02:40 INFO Vacuumed table table=pending_closures44462026/11/12 17:02:40 INFO Vacuumed table table=pending_closures44472026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts44482026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts44492026/11/12 17:02:40 INFO Vacuumed table table=pending_objects44502026/11/12 17:02:40 INFO Received push request method=POST path=/44512026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads44522026/11/12 17:02:40 INFO Vacuumed table table=pending_objects44532026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads44542026/11/12 17:02:40 INFO Vacuumed table table=closures44552026/11/12 17:02:40 INFO Received uploads request method=POST path=/44562026/11/12 17:02:40 INFO Vacuumed table table=closures44572026/11/12 17:02:40 INFO Vacuumed table table=objects44582026/11/12 17:02:40 INFO Vacuumed table table=objects44592026/11/12 17:02:40 INFO Received push request method=POST path=/44602026/11/12 17:02:40 INFO Received push request method=POST path=/44612026/11/12 17:02:40 INFO Received push request method=POST path=/44622026/11/12 17:02:40 INFO Received uploads request method=POST path=/44632026/11/12 17:02:40 INFO Received uploads request method=POST path=/44642026/11/12 17:02:40 INFO Received uploads request method=POST path=/44652026/11/12 17:02:40 INFO Received push request method=POST path=/44662026/11/12 17:02:40 INFO Received push request method=POST path=/44672026/11/12 17:02:40 INFO Received uploads request method=POST path=/44682026/11/12 17:02:40 INFO Received uploads request method=POST path=/44692026/11/12 17:02:40 INFO Received complete upload request method=POST path=/44702026/11/12 17:02:40 INFO Completed upload id=20 objects=3 duration=2.7736ms44712026/11/12 17:02:40 INFO Received push request method=POST path=/44722026/11/12 17:02:40 INFO Received complete upload request method=POST path=/44732026/11/12 17:02:40 INFO Completed upload id=21 objects=25 duration=1.32271ms44742026/11/12 17:02:40 INFO Received complete upload request method=POST path=/44752026/11/12 17:02:40 INFO Completed upload id=21 objects=2 duration=1.53005ms44762026/11/12 17:02:40 INFO Received push request method=POST path=/44772026/11/12 17:02:40 INFO Received push request method=POST path=/44782026/11/12 17:02:40 INFO Received uploads request method=POST path=/44792026/11/12 17:02:40 INFO Received uploads request method=POST path=/44802026/11/12 17:02:40 INFO Received push request method=POST path=/44812026/11/12 17:02:40 INFO Received uploads request method=POST path=/44822026/11/12 17:02:40 INFO Received push request method=POST path=/44832026/11/12 17:02:40 INFO Received uploads request method=POST path=/44842026/11/12 17:02:40 INFO Received push request method=POST path=/44852026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=044862026/11/12 17:02:40 INFO Vacuumed table table=pending_closures44872026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts44882026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=044892026/11/12 17:02:40 INFO Vacuumed table table=pending_objects44902026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period44912026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads44922026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=044932026/11/12 17:02:40 INFO Received push request method=POST path=/44942026/11/12 17:02:40 INFO Vacuumed table table=closures44952026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044962026/11/12 17:02:40 INFO Vacuumed table table=objects44972026/11/12 17:02:40 INFO Vacuumed table table=pending_closures44982026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts44992026/11/12 17:02:40 INFO Vacuumed table table=pending_objects45002026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads45012026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=3 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045022026/11/12 17:02:40 INFO Vacuumed table table=closures45032026/11/12 17:02:40 INFO Received push request method=POST path=/45042026/11/12 17:02:40 INFO Vacuumed table table=pending_closures45052026/11/12 17:02:40 INFO Vacuumed table table=objects45062026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts45072026/11/12 17:02:40 INFO Received uploads request method=POST path=/45082026/11/12 17:02:40 INFO Vacuumed table table=pending_objects45092026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads45102026/11/12 17:02:40 INFO Vacuumed table table=closures45112026/11/12 17:02:40 INFO Received push request method=POST path=/45122026/11/12 17:02:40 INFO Vacuumed table table=objects45132026/11/12 17:02:40 INFO Received complete upload request method=POST path=/45142026/11/12 17:02:40 INFO Received uploads request method=POST path=/45152026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=045162026/11/12 17:02:40 INFO Completed upload id=23 objects=18 duration=3.139539ms45172026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period45182026/11/12 17:02:40 INFO Received complete upload request method=POST path=/45192026/11/12 17:02:40 INFO Completed upload id=26 objects=6 duration=1.3703ms45202026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=045212026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045222026/11/12 17:02:40 INFO Vacuumed table table=pending_closures45232026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts45242026/11/12 17:02:40 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."45252026/11/12 17:02:40 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."45262026/11/12 17:02:40 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."45272026/11/12 17:02:40 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."45282026/11/12 17:02:40 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."45292026/11/12 17:02:40 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."45302026/11/12 17:02:40 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."45312026/11/12 17:02:40 INFO Vacuumed table table=pending_objects45322026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads45332026/11/12 17:02:40 INFO Vacuumed table table=closures45342026/11/12 17:02:40 INFO Received uploads request method=POST path=/45352026/11/12 17:02:40 INFO Vacuumed table table=objects45362026/11/12 17:02:40 INFO Received push request method=POST path=/45372026/11/12 17:02:40 INFO Received uploads request method=POST path=/45382026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=045392026/11/12 17:02:40 INFO Received uploads request method=POST path=/45402026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period45412026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045422026/11/12 17:02:40 INFO Received complete push request method=POST path=/45432026/11/12 17:02:40 INFO Vacuumed table table=pending_closures45442026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=045452026/11/12 17:02:40 INFO Completed push id=16 objects=26 duration=1.18402ms45462026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts45472026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period45482026/11/12 17:02:40 INFO Vacuumed table table=pending_objects45492026/11/12 17:02:40 INFO Received complete upload request method=POST path=/45502026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads45512026/11/12 17:02:40 INFO Received complete upload request method=POST path=/45522026/11/12 17:02:40 INFO Received uploads request method=POST path=/45532026/11/12 17:02:40 INFO Vacuumed table table=closures45542026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045552026/11/12 17:02:40 INFO Completed upload id=24 objects=7 duration=1.5148ms45562026/11/12 17:02:40 INFO Completed upload id=12 objects=6 duration=1.3066ms45572026/11/12 17:02:40 INFO Vacuumed table table=objects45582026/11/12 17:02:40 INFO Vacuumed table table=pending_closures45592026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts45602026/11/12 17:02:40 INFO Vacuumed table table=pending_objects45612026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads45622026/11/12 17:02:40 INFO Vacuumed table table=closures45632026/11/12 17:02:40 INFO Vacuumed table table=objects45642026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=045652026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045662026/11/12 17:02:40 INFO Vacuumed table table=pending_closures45672026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts45682026/11/12 17:02:40 INFO Vacuumed table table=pending_objects45692026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads45702026/11/12 17:02:40 INFO Received uploads request method=POST path=/45712026/11/12 17:02:40 INFO Vacuumed table table=closures45722026/11/12 17:02:40 INFO Vacuumed table table=objects45732026/11/12 17:02:40 INFO Received complete push request method=POST path=/45742026/11/12 17:02:40 INFO Received complete upload request method=POST path=/45752026/11/12 17:02:40 INFO Received push request method=POST path=/45762026/11/12 17:02:40 INFO Received push request method=POST path=/45772026/11/12 17:02:40 INFO Completed upload id=27 objects=4 duration=1.50022ms45782026/11/12 17:02:40 INFO Received push request method=POST path=/45792026/11/12 17:02:40 INFO Completed push id=19 objects=14 duration=2.2927ms45802026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=045812026/11/12 17:02:40 INFO Received complete upload request method=POST path=/45822026/11/12 17:02:40 INFO Completed upload id=15 objects=3 duration=2.34314ms45832026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period45842026/11/12 17:02:40 INFO Received complete push request method=POST path=/45852026/11/12 17:02:40 INFO Received complete upload request method=POST path=/45862026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045872026/11/12 17:02:40 INFO Completed push id=17 objects=3 duration=1.837269ms45882026/11/12 17:02:40 INFO Completed upload id=29 objects=5 duration=1.16203ms45892026/11/12 17:02:40 INFO Vacuumed table table=pending_closures45902026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=045912026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts45922026/11/12 17:02:40 INFO Vacuumed table table=pending_objects45932026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads45942026/11/12 17:02:40 INFO Vacuumed table table=closures45952026/11/12 17:02:40 INFO Received complete upload request method=POST path=/45962026/11/12 17:02:40 INFO Vacuumed table table=objects45972026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045982026/11/12 17:02:40 INFO Completed upload id=21 objects=3 duration=1.18861ms45992026/11/12 17:02:40 INFO Vacuumed table table=pending_closures46002026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts46012026/11/12 17:02:40 INFO Vacuumed table table=pending_objects46022026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads46032026/11/12 17:02:40 INFO Vacuumed table table=closures46042026/11/12 17:02:40 INFO Vacuumed table table=objects46052026/11/12 17:02:40 INFO Received complete push request method=POST path=/46062026/11/12 17:02:40 INFO Received push request method=POST path=/46072026/11/12 17:02:40 INFO Received complete push request method=POST path=/46082026/11/12 17:02:40 INFO Completed push id=7 objects=24 duration=3.502379ms46092026/11/12 17:02:40 INFO Completed push id=13 objects=11 duration=1.72239ms46102026/11/12 17:02:40 INFO Received complete push request method=POST path=/46112026/11/12 17:02:40 INFO Received complete upload request method=POST path=/46122026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=046132026/11/12 17:02:40 INFO Completed upload id=22 objects=22 duration=1.31056ms46142026/11/12 17:02:40 INFO Completed push id=30 objects=4 duration=3.38351ms46152026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period46162026/11/12 17:02:40 INFO Received complete push request method=POST path=/46172026/11/12 17:02:40 INFO Completed push id=27 objects=25 duration=1.72024ms46182026/11/12 17:02:40 INFO Received complete push request method=POST path=/46192026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=046202026/11/12 17:02:40 INFO Completed push id=23 objects=5 duration=1.347659ms46212026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period46222026/11/12 17:02:40 INFO Received complete push request method=POST path=/46232026/11/12 17:02:40 INFO Received push request method=POST path=/46242026/11/12 17:02:40 INFO Completed push id=27 objects=6 duration=2.01407ms46252026/11/12 17:02:40 INFO Received push request method=POST path=/46262026/11/12 17:02:40 INFO Received push request method=POST path=/46272026/11/12 17:02:40 INFO Received uploads request method=POST path=/46282026/11/12 17:02:40 INFO Received uploads request method=POST path=/46292026/11/12 17:02:40 INFO Received uploads request method=POST path=/46302026/11/12 17:02:40 INFO Received push request method=POST path=/46312026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=046322026/11/12 17:02:40 INFO Received uploads request method=POST path=/46332026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=7 objects-failed-to-delete=046342026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period46352026/11/12 17:02:40 INFO Vacuumed table table=pending_closures46362026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts46372026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=046382026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046392026/11/12 17:02:40 INFO Vacuumed table table=pending_objects46402026/11/12 17:02:40 INFO Vacuumed table table=pending_closures46412026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads46422026/11/12 17:02:40 INFO Vacuumed table table=closures46432026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts46442026/11/12 17:02:40 INFO Vacuumed table table=objects46452026/11/12 17:02:40 INFO Vacuumed table table=pending_objects46462026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads46472026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046482026/11/12 17:02:40 INFO Vacuumed table table=closures46492026/11/12 17:02:40 INFO Received complete upload request method=POST path=/46502026/11/12 17:02:40 INFO Vacuumed table table=pending_closures46512026/11/12 17:02:40 INFO Received push request method=POST path=/46522026/11/12 17:02:40 INFO Vacuumed table table=objects46532026/11/12 17:02:40 INFO Completed upload id=24 objects=3 duration=1.28356ms46542026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts46552026/11/12 17:02:40 INFO Vacuumed table table=pending_objects46562026/11/12 17:02:40 INFO Received push request method=POST path=/46572026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads46582026/11/12 17:02:40 INFO Vacuumed table table=closures46592026/11/12 17:02:40 INFO Vacuumed table table=objects46602026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=046612026/11/12 17:02:40 INFO Received uploads request method=POST path=/46622026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=046632026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period46642026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period46652026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046662026/11/12 17:02:40 INFO Vacuumed table table=pending_closures46672026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046682026/11/12 17:02:40 INFO Received complete push request method=POST path=/46692026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts46702026/11/12 17:02:40 INFO Vacuumed table table=pending_closures46712026/11/12 17:02:40 INFO Received push request method=POST path=/46722026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts46732026/11/12 17:02:40 INFO Vacuumed table table=pending_objects46742026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=046752026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads46762026/11/12 17:02:40 INFO Completed push id=26 objects=5 duration=2.17285ms46772026/11/12 17:02:40 INFO Vacuumed table table=pending_objects46782026/11/12 17:02:40 INFO Vacuumed table table=closures46792026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads46802026/11/12 17:02:40 INFO Vacuumed table table=pending_closures46812026/11/12 17:02:40 INFO Vacuumed table table=closures46822026/11/12 17:02:40 INFO Vacuumed table table=objects46832026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts46842026/11/12 17:02:40 INFO Vacuumed table table=objects46852026/11/12 17:02:40 INFO Received uploads request method=POST path=/46862026/11/12 17:02:40 INFO Vacuumed table table=pending_objects46872026/11/12 17:02:40 INFO Received push request method=POST path=/46882026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads46892026/11/12 17:02:40 INFO Vacuumed table table=closures46902026/11/12 17:02:40 INFO Received complete push request method=POST path=/46912026/11/12 17:02:40 INFO Received complete upload request method=POST path=/46922026/11/12 17:02:40 INFO Vacuumed table table=objects46932026/11/12 17:02:40 INFO Received complete upload request method=POST path=/46942026/11/12 17:02:40 INFO Completed push id=25 objects=4 duration=1.21477ms46952026/11/12 17:02:40 INFO Completed upload id=15 objects=2 duration=830.18µs46962026/11/12 17:02:40 INFO Completed upload id=13 objects=8 duration=1.76509ms46972026/11/12 17:02:40 INFO Received complete push request method=POST path=/46982026/11/12 17:02:40 INFO Received complete push request method=POST path=/46992026/11/12 17:02:40 INFO Received complete push request method=POST path=/47002026/11/12 17:02:40 INFO Completed push id=31 objects=14 duration=2.757059ms47012026/11/12 17:02:40 INFO Completed push id=16 objects=7 duration=2.29469ms47022026/11/12 17:02:40 INFO Completed push id=24 objects=5 duration=2.73402ms47032026/11/12 17:02:40 INFO Received uploads request method=POST path=/47042026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=047052026/11/12 17:02:40 INFO Received push request method=POST path=/47062026/11/12 17:02:40 INFO Received complete upload request method=POST path=/47072026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period47082026/11/12 17:02:40 INFO Completed upload id=24 objects=14 duration=1.38104ms47092026/11/12 17:02:40 INFO Received push request method=POST path=/47102026/11/12 17:02:40 INFO Received complete push request method=POST path=/47112026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047122026/11/12 17:02:40 INFO Vacuumed table table=pending_closures47132026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts47142026/11/12 17:02:40 INFO Vacuumed table table=pending_objects47152026/11/12 17:02:40 INFO Received push request method=POST path=/47162026/11/12 17:02:40 INFO Completed push id=17 objects=22 duration=2.878959ms47172026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads47182026/11/12 17:02:40 INFO Vacuumed table table=closures47192026/11/12 17:02:40 INFO Vacuumed table table=objects47202026/11/12 17:02:40 INFO Received uploads request method=POST path=/47212026/11/12 17:02:40 INFO Received complete push request method=POST path=/47222026/11/12 17:02:40 INFO Completed push id=23 objects=28 duration=3.964049ms47232026/11/12 17:02:40 INFO Received push request method=POST path=/47242026/11/12 17:02:40 INFO Received uploads request method=POST path=/47252026/11/12 17:02:40 INFO Received complete push request method=POST path=/47262026/11/12 17:02:40 INFO Received uploads request method=POST path=/47272026/11/12 17:02:40 INFO Completed push id=19 objects=3 duration=1.33367ms47282026/11/12 17:02:40 INFO Received complete push request method=POST path=/47292026/11/12 17:02:40 INFO Completed push id=23 objects=6 duration=1.74998ms47302026/11/12 17:02:40 INFO Received push request method=POST path=/47312026/11/12 17:02:40 INFO Received complete push request method=POST path=/47322026/11/12 17:02:40 INFO Received uploads request method=POST path=/47332026/11/12 17:02:40 INFO Received uploads request method=POST path=/47342026/11/12 17:02:40 INFO Completed push id=32 objects=7 duration=1.979669ms47352026/11/12 17:02:40 INFO Received uploads request method=POST path=/47362026/11/12 17:02:40 INFO Received uploads request method=POST path=/47372026/11/12 17:02:40 INFO Received uploads request method=POST path=/47382026/11/12 17:02:40 INFO Received uploads request method=POST path=/47392026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=047402026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047412026/11/12 17:02:40 INFO Received complete push request method=POST path=/47422026/11/12 17:02:40 INFO Vacuumed table table=pending_closures47432026/11/12 17:02:40 INFO Completed push id=26 objects=23 duration=1.59634ms47442026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts47452026/11/12 17:02:40 INFO Vacuumed table table=pending_objects47462026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads47472026/11/12 17:02:40 INFO Vacuumed table table=closures47482026/11/12 17:02:40 INFO Received complete push request method=POST path=/47492026/11/12 17:02:40 INFO Vacuumed table table=objects47502026/11/12 17:02:40 INFO Completed push id=24 objects=6 duration=1.35401ms47512026/11/12 17:02:40 INFO Received complete push request method=POST path=/47522026/11/12 17:02:40 INFO Received complete push request method=POST path=/47532026/11/12 17:02:40 INFO Received complete push request method=POST path=/47542026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=047552026/11/12 17:02:40 INFO Completed push id=26 objects=4 duration=1.54392ms47562026/11/12 17:02:40 INFO Received push request method=POST path=/47572026/11/12 17:02:40 INFO Completed push id=16 objects=14 duration=3.705639ms47582026/11/12 17:02:40 INFO Completed push id=22 objects=3 duration=4.158089ms47592026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047602026/11/12 17:02:40 INFO Received complete upload request method=POST path=/47612026/11/12 17:02:40 INFO Vacuumed table table=pending_closures47622026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts47632026/11/12 17:02:40 INFO Completed upload id=27 objects=8 duration=1.553559ms47642026/11/12 17:02:40 INFO Vacuumed table table=pending_objects47652026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads47662026/11/12 17:02:40 INFO Vacuumed table table=closures47672026/11/12 17:02:40 INFO Received complete push request method=POST path=/47682026/11/12 17:02:40 INFO Vacuumed table table=objects47692026/11/12 17:02:40 INFO Completed push id=13 objects=2 duration=1.09059ms47702026/11/12 17:02:40 INFO Received complete upload request method=POST path=/47712026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=047722026/11/12 17:02:40 INFO Completed upload id=31 objects=3 duration=2.470799ms47732026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period47742026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047752026/11/12 17:02:40 INFO Vacuumed table table=pending_closures47762026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts47772026/11/12 17:02:40 INFO Received uploads request method=POST path=/47782026/11/12 17:02:40 INFO Vacuumed table table=pending_objects47792026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads47802026/11/12 17:02:40 INFO Vacuumed table table=closures47812026/11/12 17:02:40 INFO Received uploads request method=POST path=/47822026/11/12 17:02:40 INFO Vacuumed table table=objects47832026/11/12 17:02:40 INFO Received push request method=POST path=/47842026/11/12 17:02:40 INFO Received complete upload request method=POST path=/47852026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=047862026/11/12 17:02:40 INFO Completed upload id=28 objects=13 duration=1.31345ms47872026/11/12 17:02:40 INFO Received complete push request method=POST path=/47882026/11/12 17:02:40 INFO Received push request method=POST path=/47892026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period47902026/11/12 17:02:40 INFO Completed push id=21 objects=26 duration=1.932389ms47912026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047922026/11/12 17:02:40 INFO Vacuumed table table=pending_closures47932026/11/12 17:02:40 INFO Received complete push request method=POST path=/47942026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts47952026/11/12 17:02:40 INFO Received complete upload request method=POST path=/47962026/11/12 17:02:40 INFO Completed push id=25 objects=7 duration=1.819469ms47972026/11/12 17:02:40 INFO Vacuumed table table=pending_objects47982026/11/12 17:02:40 INFO Completed upload id=27 objects=13 duration=711.2µs47992026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads48002026/11/12 17:02:40 INFO Received complete push request method=POST path=/48012026/11/12 17:02:40 INFO Received complete push request method=POST path=/48022026/11/12 17:02:40 INFO Vacuumed table table=closures48032026/11/12 17:02:40 INFO Vacuumed table table=objects48042026/11/12 17:02:40 INFO Completed push id=7 objects=7 duration=1.76088ms48052026/11/12 17:02:40 INFO Completed push id=27 objects=15 duration=1.77822ms48062026/11/12 17:02:40 INFO Received complete upload request method=POST path=/48072026/11/12 17:02:40 INFO Completed upload id=22 objects=2 duration=985.499µs48082026/11/12 17:02:40 INFO Received complete upload request method=POST path=/48092026/11/12 17:02:40 INFO Completed upload id=19 objects=2 duration=2.537129ms48102026/11/12 17:02:40 INFO Received uploads request method=POST path=/48112026/11/12 17:02:40 INFO Received complete push request method=POST path=/48122026/11/12 17:02:40 INFO Completed push id=29 objects=3 duration=1.356ms48132026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=048142026/11/12 17:02:40 INFO Received complete push request method=POST path=/48152026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=048162026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048172026/11/12 17:02:40 INFO Completed push id=5 objects=28 duration=2.55709ms48182026/11/12 17:02:40 INFO Received complete upload request method=POST path=/48192026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=048202026/11/12 17:02:40 INFO Vacuumed table table=pending_closures48212026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts48222026/11/12 17:02:40 INFO Completed upload id=28 objects=17 duration=1.551419ms48232026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048242026/11/12 17:02:40 INFO Vacuumed table table=pending_objects48252026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048262026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads48272026/11/12 17:02:40 INFO Received complete push request method=POST path=/48282026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=048292026/11/12 17:02:40 INFO Vacuumed table table=pending_closures48302026/11/12 17:02:40 INFO Received uploads request method=POST path=/48312026/11/12 17:02:40 INFO Vacuumed table table=closures48322026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts48332026/11/12 17:02:40 INFO Vacuumed table table=pending_closures48342026/11/12 17:02:40 INFO Completed push id=12 objects=6 duration=1.828479ms48352026/11/12 17:02:40 INFO Vacuumed table table=objects48362026/11/12 17:02:40 INFO Received push request method=POST path=/48372026/11/12 17:02:40 INFO Received push request method=POST path=/48382026/11/12 17:02:40 INFO Vacuumed table table=pending_objects48392026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts48402026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads48412026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period48422026/11/12 17:02:40 INFO Vacuumed table table=closures48432026/11/12 17:02:40 INFO Received uploads request method=POST path=/48442026/11/12 17:02:40 INFO Vacuumed table table=pending_objects48452026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads48462026/11/12 17:02:40 INFO Vacuumed table table=objects48472026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048482026/11/12 17:02:40 INFO Vacuumed table table=closures48492026/11/12 17:02:40 INFO Vacuumed table table=objects48502026/11/12 17:02:40 INFO Vacuumed table table=pending_closures48512026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts48522026/11/12 17:02:40 INFO Received uploads request method=POST path=/48532026/11/12 17:02:40 INFO Vacuumed table table=pending_objects48542026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads48552026/11/12 17:02:40 INFO Received complete upload request method=POST path=/48562026/11/12 17:02:40 INFO Vacuumed table table=closures48572026/11/12 17:02:40 INFO Completed upload id=31 objects=2 duration=1.215559ms48582026/11/12 17:02:40 INFO Vacuumed table table=objects48592026/11/12 17:02:40 INFO Received complete push request method=POST path=/48602026/11/12 17:02:40 INFO Received complete upload request method=POST path=/48612026/11/12 17:02:40 INFO Completed push id=10 objects=2 duration=1.701099ms48622026/11/12 17:02:40 INFO Completed upload id=26 objects=24 duration=1.614389ms48632026/11/12 17:02:40 INFO Received complete push request method=POST path=/48642026/11/12 17:02:40 INFO Completed push id=29 objects=2 duration=1.29883ms48652026/11/12 17:02:40 INFO Received complete push request method=POST path=/48662026/11/12 17:02:40 INFO Completed push id=33 objects=7 duration=1.92134ms48672026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=048682026/11/12 17:02:40 INFO Received complete push request method=POST path=/48692026/11/12 17:02:40 INFO Received complete push request method=POST path=/48702026/11/12 17:02:40 INFO Completed push id=24 objects=3 duration=1.43935ms48712026/11/12 17:02:40 INFO Received push request method=POST path=/48722026/11/12 17:02:40 INFO Received complete upload request method=POST path=/48732026/11/12 17:02:40 INFO Completed push id=19 objects=14 duration=2.0373ms48742026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048752026/11/12 17:02:40 INFO Received uploads request method=POST path=/48762026/11/12 17:02:40 INFO Vacuumed table table=pending_closures48772026/11/12 17:02:40 INFO Received complete upload request method=POST path=/48782026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts48792026/11/12 17:02:40 INFO Completed upload id=25 objects=4 duration=1.55584ms48802026/11/12 17:02:40 INFO Vacuumed table table=pending_objects48812026/11/12 17:02:40 INFO Completed upload id=13 objects=14 duration=3.372629ms48822026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads48832026/11/12 17:02:40 INFO Received complete upload request method=POST path=/48842026/11/12 17:02:40 INFO Received complete push request method=POST path=/48852026/11/12 17:02:40 INFO Vacuumed table table=closures48862026/11/12 17:02:40 INFO Completed upload id=26 objects=8 duration=963.91µs48872026/11/12 17:02:40 INFO Completed push id=28 objects=19 duration=1.45209ms48882026/11/12 17:02:40 INFO Received complete upload request method=POST path=/48892026/11/12 17:02:40 INFO Vacuumed table table=objects48902026/11/12 17:02:40 INFO Completed upload id=18 objects=2 duration=2.3503ms48912026/11/12 17:02:40 INFO Received complete push request method=POST path=/48922026/11/12 17:02:40 INFO Completed push id=33 objects=6 duration=1.30219ms48932026/11/12 17:02:40 INFO Received complete push request method=POST path=/48942026/11/12 17:02:40 INFO Received complete upload request method=POST path=/48952026/11/12 17:02:40 INFO Completed push id=31 objects=6 duration=1.39724ms48962026/11/12 17:02:40 INFO Completed upload id=26 objects=3 duration=1.7153ms48972026/11/12 17:02:40 INFO Received push request method=POST path=/48982026/11/12 17:02:40 INFO Received complete upload request method=POST path=/48992026/11/12 17:02:40 INFO Received uploads request method=POST path=/49002026/11/12 17:02:40 INFO Completed upload id=17 objects=5 duration=1.7047ms49012026/11/12 17:02:40 INFO Received uploads request method=POST path=/49022026/11/12 17:02:40 INFO Received push request method=POST path=/49032026/11/12 17:02:40 INFO Received push request method=POST path=/49042026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=049052026/11/12 17:02:40 INFO Received complete push request method=POST path=/49062026/11/12 17:02:40 INFO Received complete push request method=POST path=/49072026/11/12 17:02:40 INFO Completed push id=13 objects=22 duration=1.51469ms49082026/11/12 17:02:40 INFO Completed push id=28 objects=14 duration=1.0563ms49092026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049102026/11/12 17:02:40 INFO Vacuumed table table=pending_closures49112026/11/12 17:02:40 INFO Received complete upload request method=POST path=/49122026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts49132026/11/12 17:02:40 INFO Vacuumed table table=pending_objects49142026/11/12 17:02:40 INFO Received push request method=POST path=/49152026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads49162026/11/12 17:02:40 INFO Completed upload id=25 objects=16 duration=2.916249ms49172026/11/12 17:02:40 INFO Vacuumed table table=closures49182026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=049192026/11/12 17:02:40 INFO Vacuumed table table=objects49202026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=049212026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period49222026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period49232026/11/12 17:02:40 INFO Received complete push request method=POST path=/49242026/11/12 17:02:40 INFO Received complete upload request method=POST path=/49252026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049262026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049272026/11/12 17:02:40 INFO Completed upload id=27 objects=26 duration=1.50171ms49282026/11/12 17:02:40 INFO Completed push id=32 objects=9 duration=2.08326ms49292026/11/12 17:02:40 INFO Vacuumed table table=pending_closures49302026/11/12 17:02:40 INFO Vacuumed table table=pending_closures49312026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts49322026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts49332026/11/12 17:02:40 INFO Vacuumed table table=pending_objects49342026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads49352026/11/12 17:02:40 INFO Received complete push request method=POST path=/49362026/11/12 17:02:40 INFO Vacuumed table table=pending_objects49372026/11/12 17:02:40 INFO Vacuumed table table=closures49382026/11/12 17:02:40 INFO Received complete upload request method=POST path=/49392026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads49402026/11/12 17:02:40 INFO Completed push id=29 objects=10 duration=1.33185ms49412026/11/12 17:02:40 INFO Vacuumed table table=objects49422026/11/12 17:02:40 INFO Vacuumed table table=closures49432026/11/12 17:02:40 INFO Completed upload id=25 objects=6 duration=1.739239ms49442026/11/12 17:02:40 INFO Vacuumed table table=objects49452026/11/12 17:02:40 INFO Received push request method=POST path=/49462026/11/12 17:02:40 INFO Received push request method=POST path=/49472026/11/12 17:02:40 INFO Received complete push request method=POST path=/49482026/11/12 17:02:40 INFO Received complete push request method=POST path=/49492026/11/12 17:02:40 INFO Completed push id=19 objects=3 duration=1.26909ms49502026/11/12 17:02:40 INFO Completed push id=21 objects=6 duration=1.46436ms49512026/11/12 17:02:40 INFO Received complete upload request method=POST path=/49522026/11/12 17:02:40 INFO Completed upload id=20 objects=6 duration=1.32544ms49532026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=049542026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period49552026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049562026/11/12 17:02:40 INFO Vacuumed table table=pending_closures49572026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts49582026/11/12 17:02:40 INFO Vacuumed table table=pending_objects49592026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads49602026/11/12 17:02:40 INFO Received complete push request method=POST path=/49612026/11/12 17:02:40 INFO Received complete push request method=POST path=/49622026/11/12 17:02:40 INFO Vacuumed table table=closures49632026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=049642026/11/12 17:02:40 INFO Completed push id=28 objects=6 duration=1.430429ms49652026/11/12 17:02:40 INFO Vacuumed table table=objects49662026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period49672026/11/12 17:02:40 INFO Completed push id=31 objects=6 duration=3.476869ms49682026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049692026/11/12 17:02:40 INFO Received complete upload request method=POST path=/49702026/11/12 17:02:40 INFO Vacuumed table table=pending_closures49712026/11/12 17:02:40 INFO Completed upload id=13 objects=7 duration=1.36498ms49722026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts49732026/11/12 17:02:40 INFO Received complete upload request method=POST path=/49742026/11/12 17:02:40 INFO Completed upload id=33 objects=9 duration=1.1085ms49752026/11/12 17:02:40 INFO Vacuumed table table=pending_objects49762026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads49772026/11/12 17:02:40 INFO Received complete push request method=POST path=/49782026/11/12 17:02:40 INFO Received uploads request method=POST path=/49792026/11/12 17:02:40 INFO Vacuumed table table=closures49802026/11/12 17:02:40 INFO Received complete upload request method=POST path=/49812026/11/12 17:02:40 INFO Completed push id=20 objects=8 duration=1.416869ms49822026/11/12 17:02:40 INFO Vacuumed table table=objects49832026/11/12 17:02:40 INFO Completed upload id=30 objects=6 duration=1.23198ms49842026/11/12 17:02:40 INFO Received complete upload request method=POST path=/49852026/11/12 17:02:40 INFO Completed upload id=22 objects=7 duration=1.10741ms49862026/11/12 17:02:40 INFO Received uploads request method=POST path=/49872026/11/12 17:02:40 INFO Received complete push request method=POST path=/49882026/11/12 17:02:40 INFO Completed push id=29 objects=6 duration=1.28993ms49892026/11/12 17:02:40 INFO Received push request method=POST path=/49902026/11/12 17:02:40 INFO Received complete push request method=POST path=/49912026/11/12 17:02:40 INFO Completed push id=20 objects=4 duration=1.284809ms49922026/11/12 17:02:40 INFO Received uploads request method=POST path=/49932026/11/12 17:02:40 INFO Received push request method=POST path=/49942026/11/12 17:02:40 INFO Received complete upload request method=POST path=/49952026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=049962026/11/12 17:02:40 INFO Received complete upload request method=POST path=/49972026/11/12 17:02:40 INFO Received uploads request method=POST path=/49982026/11/12 17:02:40 INFO Received uploads request method=POST path=/49992026/11/12 17:02:40 INFO Completed upload id=20 objects=4 duration=1.54251ms50002026/11/12 17:02:40 INFO Received complete push request method=POST path=/50012026/11/12 17:02:40 INFO Completed upload id=31 objects=5 duration=1.20901ms50022026/11/12 17:02:40 INFO Completed push id=24 objects=4 duration=1.15895ms50032026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period50042026/11/12 17:02:40 INFO Received uploads request method=POST path=/50052026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050062026/11/12 17:02:40 INFO Vacuumed table table=pending_closures50072026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts50082026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=050092026/11/12 17:02:40 INFO Vacuumed table table=pending_objects50102026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads50112026/11/12 17:02:40 INFO Vacuumed table table=closures50122026/11/12 17:02:40 INFO Vacuumed table table=objects50132026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period50142026/11/12 17:02:40 INFO Received push request method=POST path=/50152026/11/12 17:02:40 INFO Received complete push request method=POST path=/50162026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050172026/11/12 17:02:40 INFO Vacuumed table table=pending_closures50182026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts50192026/11/12 17:02:40 INFO Completed push id=9 objects=9 duration=3.132629ms50202026/11/12 17:02:40 INFO Vacuumed table table=pending_objects50212026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads50222026/11/12 17:02:40 INFO Received uploads request method=POST path=/50232026/11/12 17:02:40 INFO Vacuumed table table=closures50242026/11/12 17:02:40 INFO Vacuumed table table=objects50252026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50262026/11/12 17:02:40 INFO Completed upload id=23 objects=7 duration=1.16077ms50272026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50282026/11/12 17:02:40 INFO Completed upload id=34 objects=3 duration=1.02019ms50292026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50302026/11/12 17:02:40 INFO Received complete push request method=POST path=/50312026/11/12 17:02:40 INFO Received push request method=POST path=/50322026/11/12 17:02:40 INFO Completed upload id=29 objects=4 duration=1.296859ms50332026/11/12 17:02:40 INFO Completed push id=25 objects=9 duration=1.34567ms50342026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50352026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50362026/11/12 17:02:40 INFO Completed upload id=22 objects=8 duration=1.05602ms50372026/11/12 17:02:40 INFO Completed upload id=30 objects=8 duration=1.31791ms50382026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=050392026/11/12 17:02:40 INFO Received complete push request method=POST path=/50402026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50412026/11/12 17:02:40 INFO Received uploads request method=POST path=/50422026/11/12 17:02:40 INFO Completed push id=11 objects=9 duration=1.62509ms50432026/11/12 17:02:40 INFO Completed upload id=21 objects=8 duration=1.290209ms50442026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=050452026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period50462026/11/12 17:02:40 INFO Received uploads request method=POST path=/50472026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050482026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50492026/11/12 17:02:40 INFO Completed upload id=26 objects=3 duration=1.444119ms50502026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50512026/11/12 17:02:40 INFO Vacuumed table table=pending_closures50522026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts50532026/11/12 17:02:40 INFO Completed upload id=21 objects=2 duration=1.53021ms50542026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=050552026/11/12 17:02:40 INFO Vacuumed table table=pending_objects50562026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads50572026/11/12 17:02:40 INFO Vacuumed table table=pending_closures50582026/11/12 17:02:40 INFO Received complete push request method=POST path=/50592026/11/12 17:02:40 INFO Vacuumed table table=closures50602026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts50612026/11/12 17:02:40 INFO Completed push id=23 objects=5 duration=1.28791ms50622026/11/12 17:02:40 INFO Vacuumed table table=objects50632026/11/12 17:02:40 INFO Vacuumed table table=pending_objects50642026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads50652026/11/12 17:02:40 INFO Vacuumed table table=closures50662026/11/12 17:02:40 INFO Vacuumed table table=objects50672026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50682026/11/12 17:02:40 INFO Received complete push request method=POST path=/50692026/11/12 17:02:40 INFO Completed upload id=31 objects=6 duration=1.18095ms50702026/11/12 17:02:40 INFO Completed push id=22 objects=6 duration=1.33304ms50712026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50722026/11/12 17:02:40 INFO Completed upload id=21 objects=9 duration=1.09787ms50732026/11/12 17:02:40 INFO Received complete push request method=POST path=/50742026/11/12 17:02:40 INFO Received uploads request method=POST path=/50752026/11/12 17:02:40 INFO Completed push id=15 objects=24 duration=1.68972ms50762026/11/12 17:02:40 INFO Received complete push request method=POST path=/50772026/11/12 17:02:40 INFO Received uploads request method=POST path=/50782026/11/12 17:02:40 INFO Completed push id=27 objects=11 duration=1.223819ms50792026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50802026/11/12 17:02:40 INFO Completed upload id=24 objects=18 duration=1.41682ms50812026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50822026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50832026/11/12 17:02:40 INFO Completed upload id=24 objects=8 duration=1.05077ms50842026/11/12 17:02:40 INFO Completed upload id=27 objects=7 duration=1.20517ms50852026/11/12 17:02:40 INFO Received complete push request method=POST path=/50862026/11/12 17:02:40 INFO Completed push id=1 objects=9 duration=1.80953ms50872026/11/12 17:02:40 INFO Received complete push request method=POST path=/50882026/11/12 17:02:40 INFO Completed push id=24 objects=8 duration=1.260509ms50892026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50902026/11/12 17:02:40 INFO Received uploads request method=POST path=/50912026/11/12 17:02:40 INFO Completed upload id=32 objects=4 duration=1.03249ms50922026/11/12 17:02:40 INFO Received complete push request method=POST path=/50932026/11/12 17:02:40 INFO Completed push id=25 objects=10 duration=1.05147ms50942026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50952026/11/12 17:02:40 INFO Completed upload id=19 objects=2 duration=1.113829ms50962026/11/12 17:02:40 INFO Received complete upload request method=POST path=/50972026/11/12 17:02:40 INFO Received complete push request method=POST path=/50982026/11/12 17:02:40 INFO Received uploads request method=POST path=/50992026/11/12 17:02:40 INFO Received complete push request method=POST path=/51002026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=051012026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51022026/11/12 17:02:40 INFO Completed upload id=29 objects=26 duration=1.29363ms51032026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51042026/11/12 17:02:40 INFO Completed upload id=28 objects=5 duration=1.359429ms51052026/11/12 17:02:40 INFO Completed push id=25 objects=2 duration=2.578209ms51062026/11/12 17:02:40 INFO Completed push id=11 objects=6 duration=2.031649ms51072026/11/12 17:02:40 INFO Completed upload id=28 objects=3 duration=1.221519ms51082026/11/12 17:02:40 WARN Force mode enabled - objects will be deleted immediately without grace period51092026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51102026/11/12 17:02:40 INFO Completed upload id=25 objects=9 duration=1.18094ms51112026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051122026/11/12 17:02:40 INFO Vacuumed table table=pending_closures51132026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts51142026/11/12 17:02:40 INFO Vacuumed table table=pending_objects51152026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads51162026/11/12 17:02:40 INFO Vacuumed table table=closures51172026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51182026/11/12 17:02:40 INFO Vacuumed table table=objects51192026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51202026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51212026/11/12 17:02:40 INFO Completed upload id=22 objects=5 duration=2.357009ms51222026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51232026/11/12 17:02:40 INFO Completed upload id=33 objects=10 duration=1.51954ms51242026/11/12 17:02:40 INFO Completed upload id=29 objects=5 duration=1.02999ms51252026/11/12 17:02:40 INFO Completed upload id=24 objects=7 duration=1.14539ms51262026/11/12 17:02:40 INFO Received complete push request method=POST path=/51272026/11/12 17:02:40 INFO Completed push id=30 objects=8 duration=1.31427ms51282026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51292026/11/12 17:02:40 INFO Received complete push request method=POST path=/51302026/11/12 17:02:40 INFO Completed upload id=26 objects=2 duration=1.918749ms51312026/11/12 17:02:40 INFO Completed push id=27 objects=4 duration=1.06811ms51322026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51332026/11/12 17:02:40 INFO Completed upload id=27 objects=3 duration=1.16621ms51342026/11/12 17:02:40 INFO Received complete push request method=POST path=/51352026/11/12 17:02:40 INFO Completed push id=20 objects=13 duration=968.229µs51362026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51372026/11/12 17:02:40 INFO Received complete push request method=POST path=/51382026/11/12 17:02:40 INFO Completed upload id=30 objects=5 duration=922.84µs51392026/11/12 17:02:40 INFO Completed push id=29 objects=23 duration=1.23923ms51402026/11/12 17:02:40 INFO Received complete push request method=POST path=/51412026/11/12 17:02:40 INFO Received complete push request method=POST path=/51422026/11/12 17:02:40 INFO Completed push id=26 objects=3 duration=1.07602ms51432026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51442026/11/12 17:02:40 INFO Completed push id=28 objects=3 duration=2.01059ms51452026/11/12 17:02:40 INFO Completed upload id=30 objects=2 duration=655.61µs51462026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51472026/11/12 17:02:40 INFO Aborted multipart uploads count=0 kept=051482026/11/12 17:02:40 INFO Completed upload id=31 objects=5 duration=1.21739ms51492026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51502026/11/12 17:02:40 INFO Completed upload id=28 objects=6 duration=1.02398ms51512026/11/12 17:02:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051522026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51532026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51542026/11/12 17:02:40 INFO Vacuumed table table=pending_closures51552026/11/12 17:02:40 INFO Completed upload id=34 objects=23 duration=1.32935ms51562026/11/12 17:02:40 INFO Vacuumed table table=commit_receipts51572026/11/12 17:02:40 INFO Completed upload id=2 objects=7 duration=1.65318ms51582026/11/12 17:02:40 INFO Received uploads request method=POST path=/51592026/11/12 17:02:40 INFO Vacuumed table table=pending_objects51602026/11/12 17:02:40 INFO Vacuumed table table=multipart_uploads51612026/11/12 17:02:40 INFO Vacuumed table table=closures51622026/11/12 17:02:40 INFO Vacuumed table table=objects51632026/11/12 17:02:40 INFO Received complete push request method=POST path=/51642026/11/12 17:02:40 INFO Completed push id=22 objects=4 duration=918.38µs51652026/11/12 17:02:40 INFO Received complete push request method=POST path=/51662026/11/12 17:02:40 INFO Completed push id=30 objects=6 duration=1.10491ms51672026/11/12 17:02:40 INFO Received complete upload request method=POST path=/51682026/11/12 17:02:40 INFO Received complete push request method=POST path=/51692026/11/12 17:02:40 INFO Completed upload id=28 objects=8 duration=1.32235ms51702026/11/12 17:02:40 INFO Received complete push request method=POST path=/51712026/11/12 17:02:40 INFO Completed push id=29 objects=15 duration=1.53438ms51722026/11/12 17:02:40 INFO Received complete push request method=POST path=/51732026/11/12 17:02:40 INFO Completed push id=31 objects=20 duration=798.069µs51742026/11/12 17:02:40 INFO Received complete push request method=POST path=/51752026/11/12 17:02:40 INFO Completed push id=28 objects=26 duration=1.50774ms51762026/11/12 17:02:40 INFO Received complete push request method=POST path=/51772026/11/12 17:02:40 INFO Completed push id=31 objects=3 duration=1.1501ms51782026/11/12 17:02:40 INFO Completed push id=30 objects=4 duration=1.11823ms51792026/11/12 17:02:40 INFO Received complete push request method=POST path=/51802026/11/12 17:02:40 INFO Completed push id=32 objects=3 duration=1.06236ms51812026/11/12 17:02:41 INFO Received push request method=POST path=/51822026/11/12 17:02:41 INFO Received complete push request method=POST path=/51832026/11/12 17:02:41 INFO Completed push id=35 objects=6 duration=1.196279ms51842026/11/12 17:02:41 INFO Received complete upload request method=POST path=/51852026/11/12 17:02:41 INFO Received complete upload request method=POST path=/51862026/11/12 17:02:41 INFO Completed upload id=29 objects=10 duration=1.0765ms51872026/11/12 17:02:41 INFO Completed upload id=23 objects=7 duration=890.24µs51882026/11/12 17:02:41 INFO Received complete upload request method=POST path=/51892026/11/12 17:02:41 INFO Received complete push request method=POST path=/51902026/11/12 17:02:41 INFO Completed upload id=32 objects=5 duration=621.96µs51912026/11/12 17:02:41 INFO Completed push id=31 objects=7 duration=1.133269ms51922026/11/12 17:02:41 INFO Received complete push request method=POST path=/51932026/11/12 17:02:41 INFO Received complete push request method=POST path=/51942026/11/12 17:02:41 INFO Completed push id=30 objects=3 duration=1.06923ms51952026/11/12 17:02:41 INFO Completed push id=30 objects=6 duration=1.53991ms51962026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=051972026/11/12 17:02:41 INFO Received complete push request method=POST path=/51982026/11/12 17:02:41 INFO Completed push id=33 objects=3 duration=1.200519ms51992026/11/12 17:02:41 WARN Force mode enabled - objects will be deleted immediately without grace period52002026/11/12 17:02:41 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052012026/11/12 17:02:41 INFO Received push request method=POST path=/52022026/11/12 17:02:41 INFO Vacuumed table table=pending_closures52032026/11/12 17:02:41 INFO Vacuumed table table=commit_receipts52042026/11/12 17:02:41 INFO Vacuumed table table=pending_objects52052026/11/12 17:02:41 INFO Vacuumed table table=multipart_uploads52062026/11/12 17:02:41 INFO Received complete push request method=POST path=/52072026/11/12 17:02:41 INFO Vacuumed table table=closures52082026/11/12 17:02:41 INFO Vacuumed table table=objects52092026/11/12 17:02:41 INFO Received complete push request method=POST path=/52102026/11/12 17:02:41 INFO Completed push id=32 objects=6 duration=1.12653ms52112026/11/12 17:02:41 INFO Completed push id=24 objects=24 duration=3.85505ms52122026/11/12 17:02:41 INFO Received complete upload request method=POST path=/52132026/11/12 17:02:41 INFO Completed upload id=8 objects=6 duration=1.34104ms52142026/11/12 17:02:41 INFO Received complete upload request method=POST path=/52152026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=052162026/11/12 17:02:41 INFO Completed upload id=30 objects=20 duration=1.114459ms52172026/11/12 17:02:41 INFO Received complete upload request method=POST path=/52182026/11/12 17:02:41 INFO Received complete upload request method=POST path=/52192026/11/12 17:02:41 INFO Received complete push request method=POST path=/52202026/11/12 17:02:41 INFO Completed upload id=33 objects=3 duration=939.06µs52212026/11/12 17:02:41 INFO Received complete push request method=POST path=/52222026/11/12 17:02:41 INFO Completed upload id=32 objects=5 duration=1.0997ms52232026/11/12 17:02:41 INFO Completed push id=24 objects=5 duration=1.11446ms52242026/11/12 17:02:41 WARN Force mode enabled - objects will be deleted immediately without grace period52252026/11/12 17:02:41 INFO Completed push id=32 objects=19 duration=1.17041ms52262026/11/12 17:02:41 INFO Received complete upload request method=POST path=/52272026/11/12 17:02:41 INFO Completed upload id=31 objects=3 duration=1.15469ms52282026/11/12 17:02:41 INFO Received complete push request method=POST path=/52292026/11/12 17:02:41 INFO Completed push id=34 objects=6 duration=936.16µs52302026/11/12 17:02:41 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=052312026/11/12 17:02:41 INFO Vacuumed table table=pending_closures52322026/11/12 17:02:41 INFO Vacuumed table table=commit_receipts52332026/11/12 17:02:41 INFO Vacuumed table table=pending_objects52342026/11/12 17:02:41 INFO Vacuumed table table=multipart_uploads52352026/11/12 17:02:41 INFO Vacuumed table table=closures52362026/11/12 17:02:41 INFO Vacuumed table table=objects52372026/11/12 17:02:41 INFO Received complete upload request method=POST path=/52382026/11/12 17:02:41 INFO Completed upload id=33 objects=7 duration=944.43µs52392026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=052402026/11/12 17:02:41 WARN Force mode enabled - objects will be deleted immediately without grace period52412026/11/12 17:02:41 INFO Received complete push request method=POST path=/52422026/11/12 17:02:41 INFO Received complete upload request method=POST path=/52432026/11/12 17:02:41 INFO Completed push id=31 objects=11 duration=1.19358ms52442026/11/12 17:02:41 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052452026/11/12 17:02:41 INFO Completed upload id=25 objects=2 duration=807.229µs52462026/11/12 17:02:41 INFO Vacuumed table table=pending_closures52472026/11/12 17:02:41 INFO Vacuumed table table=commit_receipts52482026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=052492026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=052502026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=052512026/11/12 17:02:41 INFO Vacuumed table table=pending_objects52522026/11/12 17:02:41 INFO Received complete push request method=POST path=/52532026/11/12 17:02:41 INFO Vacuumed table table=multipart_uploads52542026/11/12 17:02:41 WARN Force mode enabled - objects will be deleted immediately without grace period52552026/11/12 17:02:41 WARN Force mode enabled - objects will be deleted immediately without grace period52562026/11/12 17:02:41 INFO Vacuumed table table=closures52572026/11/12 17:02:41 INFO Completed push id=35 objects=8 duration=1.321249ms52582026/11/12 17:02:41 WARN Force mode enabled - objects will be deleted immediately without grace period52592026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=052602026/11/12 17:02:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052612026/11/12 17:02:41 INFO Vacuumed table table=objects52622026/11/12 17:02:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052632026/11/12 17:02:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052642026/11/12 17:02:41 INFO Vacuumed table table=pending_closures52652026/11/12 17:02:41 INFO Vacuumed table table=pending_closures52662026/11/12 17:02:41 INFO Vacuumed table table=commit_receipts52672026/11/12 17:02:41 INFO Vacuumed table table=commit_receipts52682026/11/12 17:02:41 INFO Vacuumed table table=pending_closures52692026/11/12 17:02:41 INFO Vacuumed table table=commit_receipts52702026/11/12 17:02:41 INFO Vacuumed table table=pending_objects52712026/11/12 17:02:41 INFO Vacuumed table table=pending_objects52722026/11/12 17:02:41 INFO Vacuumed table table=multipart_uploads52732026/11/12 17:02:41 INFO Vacuumed table table=multipart_uploads52742026/11/12 17:02:41 INFO Vacuumed table table=pending_objects52752026/11/12 17:02:41 INFO Vacuumed table table=closures52762026/11/12 17:02:41 INFO Vacuumed table table=multipart_uploads52772026/11/12 17:02:41 INFO Vacuumed table table=closures52782026/11/12 17:02:41 INFO Received complete upload request method=POST path=/52792026/11/12 17:02:41 INFO Vacuumed table table=objects52802026/11/12 17:02:41 INFO Vacuumed table table=closures52812026/11/12 17:02:41 INFO Vacuumed table table=objects52822026/11/12 17:02:41 INFO Completed upload id=34 objects=3 duration=1.24262ms52832026/11/12 17:02:41 INFO Vacuumed table table=objects52842026/11/12 17:02:41 INFO Received complete upload request method=POST path=/52852026/11/12 17:02:41 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."52862026/11/12 17:02:41 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."52872026/11/12 17:02:41 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."52882026/11/12 17:02:41 INFO Received complete upload request method=POST path=/52892026/11/12 17:02:41 INFO Completed upload id=32 objects=2 duration=1.2954ms52902026/11/12 17:02:41 INFO Completed upload id=26 objects=20 duration=773.97µs52912026/11/12 17:02:41 INFO Received complete upload request method=POST path=/52922026/11/12 17:02:41 INFO Completed upload id=36 objects=9 duration=905.43µs52932026/11/12 17:02:41 INFO Received uploads request method=POST path=/52942026/11/12 17:02:41 INFO Received complete upload request method=POST path=/52952026/11/12 17:02:41 INFO Completed upload id=12 objects=9 duration=1.464619ms52962026/11/12 17:02:41 INFO Received complete push request method=POST path=/52972026/11/12 17:02:41 INFO Completed push id=35 objects=11 duration=1.220989ms52982026/11/12 17:02:41 INFO Received push request method=POST path=/52992026/11/12 17:02:41 INFO Received complete push request method=POST path=/53002026/11/12 17:02:41 INFO Received complete upload request method=POST path=/53012026/11/12 17:02:41 INFO Completed push id=33 objects=3 duration=1.60738ms53022026/11/12 17:02:41 INFO Completed upload id=34 objects=24 duration=1.53738ms53032026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=053042026/11/12 17:02:41 WARN Force mode enabled - objects will be deleted immediately without grace period53052026/11/12 17:02:41 INFO Received complete push request method=POST path=/53062026/11/12 17:02:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=053072026/11/12 17:02:41 INFO Completed push id=28 objects=7 duration=876.96µs53082026/11/12 17:02:41 INFO Received uploads request method=POST path=/53092026/11/12 17:02:41 INFO Vacuumed table table=pending_closures53102026/11/12 17:02:41 INFO Vacuumed table table=commit_receipts53112026/11/12 17:02:41 INFO Vacuumed table table=pending_objects53122026/11/12 17:02:41 INFO Vacuumed table table=multipart_uploads53132026/11/12 17:02:41 INFO Vacuumed table table=closures53142026/11/12 17:02:41 INFO Vacuumed table table=objects53152026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=053162026/11/12 17:02:41 INFO Received complete upload request method=POST path=/53172026/11/12 17:02:41 INFO Completed upload id=34 objects=3 duration=938.33µs53182026/11/12 17:02:41 WARN Force mode enabled - objects will be deleted immediately without grace period53192026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=053202026/11/12 17:02:41 WARN Force mode enabled - objects will be deleted immediately without grace period53212026/11/12 17:02:41 INFO Garbage collection completed failed-uploads-deleted=6 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=053222026/11/12 17:02:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=053232026/11/12 17:02:41 INFO Vacuumed table table=pending_closures53242026/11/12 17:02:41 INFO Vacuumed table table=commit_receipts53252026/11/12 17:02:41 INFO Vacuumed table table=pending_closures53262026/11/12 17:02:41 INFO Vacuumed table table=commit_receipts53272026/11/12 17:02:41 INFO Vacuumed table table=pending_objects53282026/11/12 17:02:41 INFO Vacuumed table table=multipart_uploads53292026/11/12 17:02:41 INFO Vacuumed table table=closures53302026/11/12 17:02:41 INFO Received push request method=POST path=/53312026/11/12 17:02:41 INFO Vacuumed table table=pending_objects53322026/11/12 17:02:41 INFO Received complete upload request method=POST path=/53332026/11/12 17:02:41 INFO Vacuumed table table=multipart_uploads53342026/11/12 17:02:41 INFO Vacuumed table table=objects53352026/11/12 17:02:41 INFO Completed upload id=30 objects=6 duration=931.63µs53362026/11/12 17:02:41 INFO Vacuumed table table=closures53372026/11/12 17:02:41 INFO Vacuumed table table=objects53382026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=053392026/11/12 17:02:41 WARN Force mode enabled - objects will be deleted immediately without grace period53402026/11/12 17:02:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=053412026/11/12 17:02:41 INFO Vacuumed table table=pending_closures53422026/11/12 17:02:41 INFO Vacuumed table table=commit_receipts53432026/11/12 17:02:41 INFO Vacuumed table table=pending_objects53442026/11/12 17:02:41 INFO Received push request method=POST path=/53452026/11/12 17:02:41 INFO Vacuumed table table=multipart_uploads53462026/11/12 17:02:41 INFO Vacuumed table table=closures53472026/11/12 17:02:41 INFO Vacuumed table table=objects53482026/11/12 17:02:41 INFO Received complete push request method=POST path=/53492026/11/12 17:02:41 INFO Received complete push request method=POST path=/53502026/11/12 17:02:41 INFO Completed push id=28 objects=11 duration=2.513609ms53512026/11/12 17:02:41 INFO Completed push id=31 objects=5 duration=1.03712ms53522026/11/12 17:02:41 INFO Received uploads request method=POST path=/53532026/11/12 17:02:41 INFO Received complete push request method=POST path=/53542026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=053552026/11/12 17:02:41 INFO Completed push id=32 objects=26 duration=970.819µs53562026/11/12 17:02:41 WARN Force mode enabled - objects will be deleted immediately without grace period53572026/11/12 17:02:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=053582026/11/12 17:02:41 INFO Vacuumed table table=pending_closures53592026/11/12 17:02:41 INFO Vacuumed table table=commit_receipts53602026/11/12 17:02:41 INFO Vacuumed table table=pending_objects53612026/11/12 17:02:41 INFO Vacuumed table table=multipart_uploads53622026/11/12 17:02:41 INFO Vacuumed table table=closures53632026/11/12 17:02:41 INFO Vacuumed table table=objects53642026/11/12 17:02:41 INFO Received complete push request method=POST path=/53652026/11/12 17:02:41 INFO Received complete upload request method=POST path=/53662026/11/12 17:02:41 INFO Completed push id=29 objects=24 duration=1.635229ms53672026/11/12 17:02:41 INFO Completed upload id=18 objects=22 duration=1.46191ms53682026/11/12 17:02:41 INFO Received complete upload request method=POST path=/53692026/11/12 17:02:41 INFO Completed upload id=33 objects=2 duration=938.98µs53702026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=053712026/11/12 17:02:41 WARN Force mode enabled - objects will be deleted immediately without grace period53722026/11/12 17:02:41 INFO Received complete upload request method=POST path=/53732026/11/12 17:02:41 INFO Completed upload id=21 objects=8 duration=1.01846ms53742026/11/12 17:02:41 INFO Received complete upload request method=POST path=/53752026/11/12 17:02:41 INFO Completed upload id=34 objects=5 duration=961.74µs53762026/11/12 17:02:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=053772026/11/12 17:02:41 INFO Vacuumed table table=pending_closures53782026/11/12 17:02:41 INFO Vacuumed table table=commit_receipts53792026/11/12 17:02:41 INFO Vacuumed table table=pending_objects53802026/11/12 17:02:41 INFO Vacuumed table table=multipart_uploads53812026/11/12 17:02:41 INFO Vacuumed table table=closures53822026/11/12 17:02:41 INFO Vacuumed table table=objects53832026/11/12 17:02:41 INFO Received push request method=POST path=/53842026/11/12 17:02:41 INFO Received complete upload request method=POST path=/53852026/11/12 17:02:41 INFO Completed upload id=35 objects=11 duration=910.319µs53862026/11/12 17:02:41 INFO Received complete upload request method=POST path=/53872026/11/12 17:02:41 INFO Completed upload id=22 objects=7 duration=1.24596ms53882026/11/12 17:02:41 INFO Received complete upload request method=POST path=/53892026/11/12 17:02:41 INFO Received complete upload request method=POST path=/53902026/11/12 17:02:41 INFO Completed upload id=36 objects=3 duration=987.7µs53912026/11/12 17:02:41 INFO Completed upload id=23 objects=7 duration=2.735629ms53922026/11/12 17:02:41 INFO Received complete push request method=POST path=/53932026/11/12 17:02:41 INFO Completed push id=24 objects=5 duration=1.25612ms53942026/11/12 17:02:41 INFO Received complete upload request method=POST path=/53952026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=053962026/11/12 17:02:41 WARN Force mode enabled - objects will be deleted immediately without grace period53972026/11/12 17:02:41 INFO Completed upload id=26 objects=5 duration=2.962039ms53982026/11/12 17:02:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=053992026/11/12 17:02:41 INFO Vacuumed table table=pending_closures54002026/11/12 17:02:41 INFO Vacuumed table table=commit_receipts54012026/11/12 17:02:41 INFO Vacuumed table table=pending_objects54022026/11/12 17:02:41 INFO Received complete upload request method=POST path=/54032026/11/12 17:02:41 INFO Vacuumed table table=multipart_uploads54042026/11/12 17:02:41 INFO Completed upload id=25 objects=8 duration=1.25136ms54052026/11/12 17:02:41 INFO Vacuumed table table=closures54062026/11/12 17:02:41 INFO Vacuumed table table=objects54072026/11/12 17:02:41 INFO Received complete push request method=POST path=/54082026/11/12 17:02:41 INFO Completed push id=27 objects=3 duration=2.0259ms54092026/11/12 17:02:41 INFO Received complete push request method=POST path=/54102026/11/12 17:02:41 INFO Completed push id=26 objects=4 duration=3.179409ms54112026/11/12 17:02:41 INFO Received complete upload request method=POST path=/54122026/11/12 17:02:41 INFO Completed upload id=28 objects=5 duration=1.00458ms54132026/11/12 17:02:41 INFO Received complete push request method=POST path=/54142026/11/12 17:02:41 INFO Completed push id=30 objects=2 duration=1.019889ms54152026/11/12 17:02:41 INFO Received complete push request method=POST path=/54162026/11/12 17:02:41 INFO Completed push id=29 objects=17 duration=1.15755ms54172026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=054182026/11/12 17:02:41 WARN Force mode enabled - objects will be deleted immediately without grace period54192026/11/12 17:02:41 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=054202026/11/12 17:02:41 INFO Vacuumed table table=pending_closures54212026/11/12 17:02:41 INFO Received complete push request method=POST path=/54222026/11/12 17:02:41 INFO Vacuumed table table=commit_receipts54232026/11/12 17:02:41 INFO Completed push id=32 objects=7 duration=1.43649ms54242026/11/12 17:02:41 INFO Vacuumed table table=pending_objects54252026/11/12 17:02:41 INFO Vacuumed table table=multipart_uploads54262026/11/12 17:02:41 INFO Vacuumed table table=closures54272026/11/12 17:02:41 INFO Vacuumed table table=objects54282026/11/12 17:02:41 INFO Received complete upload request method=POST path=/54292026/11/12 17:02:41 INFO Completed upload id=33 objects=6 duration=1.049769ms54302026/11/12 17:02:41 INFO Received complete upload request method=POST path=/54312026/11/12 17:02:41 INFO Completed upload id=34 objects=3 duration=849.95µs54322026/11/12 17:02:41 INFO Aborted multipart uploads count=0 kept=054332026/11/12 17:02:41 WARN Force mode enabled - objects will be deleted immediately without grace period54342026/11/12 17:02:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=054352026/11/12 17:02:41 INFO Vacuumed table table=pending_closures54362026/11/12 17:02:41 INFO Vacuumed table table=commit_receipts54372026/11/12 17:02:41 INFO Vacuumed table table=pending_objects54382026/11/12 17:02:41 INFO Vacuumed table table=multipart_uploads54392026/11/12 17:02:41 INFO Vacuumed table table=closures54402026/11/12 17:02:41 INFO Vacuumed table table=objects5441--- PASS: FuzzPushGC (0.00s)5442 --- PASS: FuzzPushGC/seed#14 (3.25s)5443 --- PASS: FuzzPushGC/seed#0 (3.33s)5444 --- PASS: FuzzPushGC/seed#13 (3.37s)5445 --- PASS: FuzzPushGC/seed#3 (3.51s)5446 --- PASS: FuzzPushGC/seed#12 (3.54s)5447 --- PASS: FuzzPushGC/seed#6 (3.55s)5448 --- PASS: FuzzPushGC/seed#7 (3.56s)5449 --- PASS: FuzzPushGC/seed#15 (3.56s)5450 --- PASS: FuzzPushGC/seed#4 (3.59s)5451 --- PASS: FuzzPushGC/seed#8 (3.60s)5452 --- PASS: FuzzPushGC/seed#9 (3.61s)5453 --- PASS: FuzzPushGC/seed#5 (3.62s)5454 --- PASS: FuzzPushGC/seed#2 (3.69s)5455 --- PASS: FuzzPushGC/seed#11 (3.76s)5456 --- PASS: FuzzPushGC/seed#1 (3.84s)5457 --- PASS: FuzzPushGC/seed#10 (3.92s)5458PASS5459{"timestamp":"2026-11-12T17:02:41.468714213Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:45654","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(401)"}54602026-11-12 17:02:41.965 UTC [129] LOG: received smart shutdown request54612026-11-12 17:02:41.969 UTC [129] LOG: background worker "logical replication launcher" (PID 139) exited with exit code 154622026-11-12 17:02:42.527 UTC [134] LOG: checkpoint complete: wrote 1557 buffers (9.5%), wrote 5 SLRU buffers; 0 WAL file(s) added, 0 removed, 33 recycled; write=17.251 s, sync=0.487 s, total=17.814 s; sync files=48810, longest=0.001 s, average=0.001 s; distance=534022 kB, estimate=534022 kB; lsn=0/2BD74808, redo lsn=0/220E784054632026-11-12 17:02:42.529 UTC [134] LOG: shutting down54642026-11-12 17:02:42.529 UTC [134] LOG: checkpoint starting: shutdown immediate54652026-11-12 17:02:43.464 UTC [134] LOG: checkpoint complete: wrote 12883 buffers (78.6%), wrote 4 SLRU buffers; 0 WAL file(s) added, 0 removed, 9 recycled; write=0.242 s, sync=0.672 s, total=0.935 s; sync files=4078, longest=0.070 s, average=0.001 s; distance=160308 kB, estimate=496651 kB; lsn=0/2BD74880, redo lsn=0/2BD7488054662026-11-12 17:02:43.543 UTC [129] LOG: database system is shut down5467Running OIDC tests...5468=== RUN TestAudienceForIssuer5469=== PAUSE TestAudienceForIssuer5470=== RUN TestProviderClientGivesUpOnSilentIssuer5471=== PAUSE TestProviderClientGivesUpOnSilentIssuer5472=== RUN TestGlobMatch5473=== PAUSE TestGlobMatch5474=== RUN TestValidateToken_ValidToken5475=== PAUSE TestValidateToken_ValidToken5476=== RUN TestValidateToken_WrongAudience5477=== PAUSE TestValidateToken_WrongAudience5478=== RUN TestValidateToken_Expired5479=== PAUSE TestValidateToken_Expired5480=== RUN TestValidateToken_BoundClaimsMismatch5481=== PAUSE TestValidateToken_BoundClaimsMismatch5482=== RUN TestValidateToken_BoundSubjectMismatch5483=== PAUSE TestValidateToken_BoundSubjectMismatch5484=== RUN TestValidateToken_MultipleProviders5485=== PAUSE TestValidateToken_MultipleProviders5486=== RUN TestValidateToken_UnknownIssuerFetchesNoKeys5487=== PAUSE TestValidateToken_UnknownIssuerFetchesNoKeys5488=== RUN TestValidateToken_GoogleSchemelessIssuer5489=== PAUSE TestValidateToken_GoogleSchemelessIssuer5490=== RUN TestValidateToken_KubernetesServiceAccount5491=== PAUSE TestValidateToken_KubernetesServiceAccount5492=== RUN TestNewValidator_KubernetesRequiresCA5493=== PAUSE TestNewValidator_KubernetesRequiresCA5494=== RUN TestValidateToken_KubernetesIssuerFromOwnToken5495=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken5496=== RUN TestPins_ReservedForMatchingRule5497=== PAUSE TestPins_ReservedForMatchingRule5498=== RUN TestPins_TopLevelShorthand5499=== PAUSE TestPins_TopLevelShorthand5500=== RUN TestPins_ConfigValidation5501=== PAUSE TestPins_ConfigValidation5502=== RUN TestScopes_LegacyProviderDefaultsToWrite5503=== PAUSE TestScopes_LegacyProviderDefaultsToWrite5504=== RUN TestScopes_Rules5505=== PAUSE TestScopes_Rules5506=== RUN TestScopes_ConfigValidation5507=== PAUSE TestScopes_ConfigValidation5508=== CONT TestAudienceForIssuer5509=== CONT TestValidateToken_GoogleSchemelessIssuer5510--- PASS: TestAudienceForIssuer (0.00s)5511=== CONT TestValidateToken_UnknownIssuerFetchesNoKeys5512=== CONT TestPins_TopLevelShorthand5513=== CONT TestValidateToken_BoundSubjectMismatch5514=== CONT TestValidateToken_BoundClaimsMismatch5515=== CONT TestScopes_Rules55162026/11/12 17:02:46 INFO OIDC provider initialized name=known issuer=https://known.example5517=== CONT TestValidateToken_KubernetesIssuerFromOwnToken5518=== CONT TestNewValidator_KubernetesRequiresCA5519=== CONT TestScopes_ConfigValidation5520=== CONT TestValidateToken_Expired5521=== CONT TestValidateToken_KubernetesServiceAccount5522=== CONT TestValidateToken_MultipleProviders5523=== CONT TestValidateToken_ValidToken5524--- PASS: TestScopes_ConfigValidation (0.00s)5525=== CONT TestValidateToken_WrongAudience5526=== CONT TestScopes_LegacyProviderDefaultsToWrite5527=== CONT TestGlobMatch5528=== RUN TestGlobMatch/foo_foo5529=== PAUSE TestGlobMatch/foo_foo5530=== RUN TestGlobMatch/foo_bar5531=== PAUSE TestGlobMatch/foo_bar5532=== RUN TestGlobMatch/*_5533=== PAUSE TestGlobMatch/*_5534=== RUN TestGlobMatch/*_anything5535=== PAUSE TestGlobMatch/*_anything5536=== RUN TestGlobMatch/foo*_foo5537=== PAUSE TestGlobMatch/foo*_foo5538=== RUN TestGlobMatch/foo*_foobar5539=== PAUSE TestGlobMatch/foo*_foobar5540=== CONT TestProviderClientGivesUpOnSilentIssuer5541=== CONT TestPins_ConfigValidation5542=== CONT TestPins_ReservedForMatchingRule5543=== RUN TestGlobMatch/foo*_bar5544=== PAUSE TestGlobMatch/foo*_bar5545=== RUN TestGlobMatch/*bar_bar5546=== PAUSE TestGlobMatch/*bar_bar5547=== RUN TestGlobMatch/*bar_foobar5548=== PAUSE TestGlobMatch/*bar_foobar5549=== RUN TestGlobMatch/*bar_foo5550=== PAUSE TestGlobMatch/*bar_foo5551=== RUN TestGlobMatch/foo*bar_foobar5552=== PAUSE TestGlobMatch/foo*bar_foobar5553=== RUN TestGlobMatch/foo*bar_foo123bar5554=== PAUSE TestGlobMatch/foo*bar_foo123bar5555=== RUN TestGlobMatch/foo*bar_foobarbaz5556=== PAUSE TestGlobMatch/foo*bar_foobarbaz5557=== RUN TestGlobMatch/*/*_foo/bar5558=== PAUSE TestGlobMatch/*/*_foo/bar5559=== RUN TestGlobMatch/*/*_foo5560=== PAUSE TestGlobMatch/*/*_foo5561=== RUN TestGlobMatch/refs/heads/*_refs/heads/main5562=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main5563=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.05564=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.05565=== RUN TestGlobMatch/refs/*/main_refs/heads/main5566--- PASS: TestProviderClientGivesUpOnSilentIssuer (0.00s)5567=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main5568=== RUN TestGlobMatch/fo?_foo5569=== PAUSE TestGlobMatch/fo?_foo5570=== RUN TestGlobMatch/fo?_fo5571=== PAUSE TestGlobMatch/fo?_fo5572=== RUN TestGlobMatch/fo?_fooo5573=== PAUSE TestGlobMatch/fo?_fooo5574=== RUN TestGlobMatch/?oo_foo5575=== PAUSE TestGlobMatch/?oo_foo5576=== RUN TestGlobMatch/?oo_boo5577=== PAUSE TestGlobMatch/?oo_boo5578=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5579=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5580=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5581=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5582=== CONT TestGlobMatch/foo_foo5583=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5584=== CONT TestGlobMatch/*bar_foo5585=== CONT TestGlobMatch/*_5586=== CONT TestGlobMatch/foo*_foo5587=== CONT TestGlobMatch/*bar_bar5588=== CONT TestGlobMatch/*bar_foobar5589=== CONT TestGlobMatch/foo*_bar5590=== CONT TestGlobMatch/fo?_foo5591=== CONT TestGlobMatch/foo*_foobar5592=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.05593=== CONT TestGlobMatch/fo?_fooo5594=== CONT TestGlobMatch/?oo_boo5595=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5596=== CONT TestGlobMatch/refs/*/main_refs/heads/main5597=== CONT TestGlobMatch/fo?_fo5598=== CONT TestGlobMatch/refs/heads/*_refs/heads/main5599=== CONT TestGlobMatch/*/*_foo5600=== CONT TestGlobMatch/foo*bar_foo123bar5601=== CONT TestGlobMatch/foo*bar_foobar5602=== CONT TestGlobMatch/*_anything5603=== CONT TestGlobMatch/*/*_foo/bar5604=== CONT TestGlobMatch/foo_bar5605=== CONT TestGlobMatch/?oo_foo5606=== CONT TestGlobMatch/foo*bar_foobarbaz5607--- PASS: TestGlobMatch (0.00s)5608 --- PASS: TestGlobMatch/foo_foo (0.00s)5609 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)5610 --- PASS: TestGlobMatch/*bar_foo (0.00s)5611 --- PASS: TestGlobMatch/*_ (0.00s)5612 --- PASS: TestGlobMatch/foo*_foo (0.00s)5613 --- PASS: TestGlobMatch/*bar_bar (0.00s)5614 --- PASS: TestGlobMatch/*bar_foobar (0.00s)5615 --- PASS: TestGlobMatch/foo*_bar (0.00s)5616 --- PASS: TestGlobMatch/fo?_foo (0.00s)5617 --- PASS: TestGlobMatch/foo*_foobar (0.00s)5618 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)5619 --- PASS: TestGlobMatch/fo?_fooo (0.00s)5620 --- PASS: TestGlobMatch/?oo_boo (0.00s)5621 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)5622 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)5623 --- PASS: TestGlobMatch/fo?_fo (0.00s)5624 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)5625 --- PASS: TestGlobMatch/*/*_foo (0.00s)5626 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)5627 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)5628 --- PASS: TestGlobMatch/*_anything (0.00s)5629 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)5630 --- PASS: TestGlobMatch/foo_bar (0.00s)5631 --- PASS: TestGlobMatch/?oo_foo (0.00s)5632 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)5633--- PASS: TestPins_ConfigValidation (0.00s)56342026/11/12 17:02:46 INFO OIDC provider initialized name=google issuer=https://accounts.google.com5635--- PASS: TestValidateToken_GoogleSchemelessIssuer (0.03s)56362026/11/12 17:02:46 http: TLS handshake error from 127.0.0.1:35032: remote error: tls: bad certificate5637--- PASS: TestNewValidator_KubernetesRequiresCA (0.03s)56382026/11/12 17:02:46 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:42967/oidc5639--- PASS: TestScopes_LegacyProviderDefaultsToWrite (0.03s)56402026/11/12 17:02:46 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:32895/oidc5641--- PASS: TestValidateToken_WrongAudience (0.04s)56422026/11/12 17:02:46 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:44955/oidc56432026/11/12 17:02:46 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:39763/oidc5644--- PASS: TestValidateToken_BoundClaimsMismatch (0.05s)5645--- PASS: TestScopes_Rules (0.05s)56462026/11/12 17:02:46 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC1235647--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (0.06s)56482026/11/12 17:02:46 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:36503/oidc56492026/11/12 17:02:46 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:41363/oidc5650--- PASS: TestValidateToken_ValidToken (0.06s)5651--- PASS: TestPins_ReservedForMatchingRule (0.06s)56522026/11/12 17:02:46 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:4366356532026/11/12 17:02:46 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:37103/oidc5654--- PASS: TestValidateToken_Expired (0.08s)5655--- PASS: TestValidateToken_KubernetesServiceAccount (0.08s)56562026/11/12 17:02:46 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:33905/oidc5657--- PASS: TestPins_TopLevelShorthand (0.10s)5658--- PASS: TestValidateToken_UnknownIssuerFetchesNoKeys (0.10s)56592026/11/12 17:02:46 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:46667/oidc5660--- PASS: TestValidateToken_BoundSubjectMismatch (0.12s)56612026/11/12 17:02:46 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:46191/oidc56622026/11/12 17:02:46 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:40721/oidc5663--- PASS: TestValidateToken_MultipleProviders (0.17s)5664PASS5665Running hook tests...5666=== RUN TestSendPathsEmpty5667=== PAUSE TestSendPathsEmpty5668=== RUN TestQueueEnqueueAndFetch5669=== PAUSE TestQueueEnqueueAndFetch5670=== RUN TestQueueDeduplication5671=== PAUSE TestQueueDeduplication5672=== RUN TestQueueRemove5673=== PAUSE TestQueueRemove5674=== RUN TestQueueFetchBatchLimit5675=== PAUSE TestQueueFetchBatchLimit5676=== RUN TestQueueRetryMovesToBack5677=== PAUSE TestQueueRetryMovesToBack5678=== RUN TestQueueFetchRemoveLifecycle5679=== PAUSE TestQueueFetchRemoveLifecycle5680=== RUN TestQueueConcurrentWriters5681=== PAUSE TestQueueConcurrentWriters5682=== RUN TestQueueRemoveLargeClosure5683=== PAUSE TestQueueRemoveLargeClosure5684=== RUN TestServerClientIntegration5685--- PASS: TestServerClientIntegration (0.00s)5686=== RUN TestServerQueueError56872026/11/12 17:02:46 ERROR Failed to queue paths error="permission denied" count=15688--- PASS: TestServerQueueError (0.00s)5689=== RUN TestGetListenerSocketActivation5690 server_test.go:220: === RUN TestGetListenerSocketActivation5691 --- PASS: TestGetListenerSocketActivation (0.00s)5692 PASS5693 5694--- PASS: TestGetListenerSocketActivation (0.01s)5695=== RUN TestServerStalledClientDoesNotBlockShutdown5696=== PAUSE TestServerStalledClientDoesNotBlockShutdown5697=== RUN TestServerBacksOffOnAcceptErrors5698=== PAUSE TestServerBacksOffOnAcceptErrors5699=== RUN TestDrainIsolatesPoisonPath5700=== PAUSE TestDrainIsolatesPoisonPath5701=== RUN TestRunNotBlockedByPoisonHead5702=== PAUSE TestRunNotBlockedByPoisonHead5703=== RUN TestDrainGivesUpWhenServerDown5704=== PAUSE TestDrainGivesUpWhenServerDown5705=== RUN TestFailedPathPrunedByLaterClosure5706=== PAUSE TestFailedPathPrunedByLaterClosure5707=== RUN TestWorkerUploadsAndRemoves5708=== PAUSE TestWorkerUploadsAndRemoves5709=== RUN TestWorkerSkipsGCdPaths5710=== PAUSE TestWorkerSkipsGCdPaths5711=== RUN TestWorkerPrunesClosureDeps5712=== PAUSE TestWorkerPrunesClosureDeps5713=== RUN TestDrainTimeout5714=== PAUSE TestDrainTimeout5715=== RUN TestWorkerRemoveFailureIsNotProgress5716=== PAUSE TestWorkerRemoveFailureIsNotProgress5717=== RUN TestWorkerKeepsPathItCannotStat5718=== PAUSE TestWorkerKeepsPathItCannotStat5719=== CONT TestSendPathsEmpty5720=== CONT TestDrainIsolatesPoisonPath5721--- PASS: TestSendPathsEmpty (0.00s)5722=== CONT TestServerBacksOffOnAcceptErrors5723=== CONT TestServerStalledClientDoesNotBlockShutdown5724=== CONT TestQueueRemoveLargeClosure57252000/01/01 00:00:00 ERROR Accept failed error="too many open files"57262000/01/01 00:00:00 ERROR Accept failed error="too many open files"57272000/01/01 00:00:00 ERROR Accept failed error="too many open files"5728=== CONT TestQueueConcurrentWriters57292000/01/01 00:00:00 ERROR Accept failed error="too many open files"57302000/01/01 00:00:00 ERROR Accept failed error="too many open files"57312000/01/01 00:00:00 ERROR Accept failed error="too many open files"57322000/01/01 00:00:00 ERROR Accept failed error="too many open files"57332000/01/01 00:00:00 ERROR Accept failed error="too many open files"5734=== CONT TestQueueFetchRemoveLifecycle5735--- PASS: TestServerBacksOffOnAcceptErrors (0.00s)5736=== CONT TestQueueDeduplication5737=== CONT TestQueueEnqueueAndFetch5738=== CONT TestWorkerSkipsGCdPaths5739=== CONT TestWorkerKeepsPathItCannotStat5740=== CONT TestWorkerRemoveFailureIsNotProgress5741=== CONT TestDrainTimeout5742=== CONT TestWorkerPrunesClosureDeps5743=== CONT TestFailedPathPrunedByLaterClosure5744=== CONT TestQueueRemove5745=== CONT TestQueueRetryMovesToBack5746=== CONT TestWorkerUploadsAndRemoves5747=== CONT TestQueueFetchBatchLimit5748=== CONT TestDrainGivesUpWhenServerDown5749=== CONT TestRunNotBlockedByPoisonHead57502026/11/12 17:02:46 INFO Upload queue status pending=257512026/11/12 17:02:46 INFO Upload queue status pending=357522026/11/12 17:02:46 INFO Upload queue status pending=257532000/01/01 00:00:00 INFO Uploading batch count=257542026/11/12 17:02:46 INFO Uploading batch count=157552026/11/12 17:02:46 ERROR Upload failed error="upload failed" count=157562000/01/01 00:00:00 ERROR Upload failed error="context deadline exceeded" count=257572026/11/12 17:02:46 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat3980572714/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat3980572714/002/locked/aaa: permission denied"57582026/11/12 17:02:46 INFO Uploading batch count=157592026/11/12 17:02:46 ERROR Upload failed error="upload failed" count=157602026/11/12 17:02:46 INFO Uploading batch count=157612026/11/12 17:02:46 INFO Uploading batch count=457622026/11/12 17:02:46 ERROR Upload failed error="upload failed" count=457632026/11/12 17:02:46 INFO Upload queue status pending=257642000/01/01 00:00:00 ERROR Drain finished with paths left in queue remaining=457652026/11/12 17:02:46 INFO Uploading batch count=257662026/11/12 17:02:46 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerSkipsGCdPaths1167406309/002/nonexistent57672026/11/12 17:02:46 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainIsolatesPoisonPath201048434/002/bbb57682026/11/12 17:02:46 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat3980572714/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat3980572714/002/locked/aaa: permission denied"5769--- PASS: TestQueueFetchBatchLimit (0.02s)5770--- PASS: TestQueueEnqueueAndFetch (0.02s)57712026/11/12 17:02:46 INFO Uploading batch count=257722026/11/12 17:02:46 ERROR Upload failed error="upload failed" count=257732026/11/12 17:02:46 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown2194688570/002/a5774--- PASS: TestQueueRemove (0.02s)57752026/11/12 17:02:46 INFO Uploading batch count=157762026/11/12 17:02:46 INFO Uploading batch count=157772026/11/12 17:02:46 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat3980572714/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat3980572714/002/locked/aaa: permission denied"57782026/11/12 17:02:46 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress1745252674/002/nonexistent5779--- PASS: TestQueueDeduplication (0.02s)5780--- PASS: TestQueueRetryMovesToBack (0.02s)57812026/11/12 17:02:46 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=157822026/11/12 17:02:46 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown2194688570/002/b57832026/11/12 17:02:46 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress1745252674/002/nonexistent5784--- PASS: TestDrainTimeout (0.02s)57852026/11/12 17:02:46 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=157862026/11/12 17:02:46 INFO Uploading batch count=15787--- PASS: TestQueueFetchRemoveLifecycle (0.02s)57882026/11/12 17:02:46 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress1745252674/002/nonexistent57892026/11/12 17:02:46 ERROR Drain finished with paths left in queue remaining=157902026/11/12 17:02:46 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=157912026/11/12 17:02:46 ERROR Drain finished with paths left in queue remaining=157922026/11/12 17:02:46 INFO Uploading batch count=257932026/11/12 17:02:46 ERROR Upload failed error="upload failed" count=257942026/11/12 17:02:46 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown2194688570/002/c57952026/11/12 17:02:46 INFO Uploading batch count=157962026/11/12 17:02:46 ERROR Upload failed error="upload failed" count=15797--- PASS: TestWorkerRemoveFailureIsNotProgress (0.02s)57982026/11/12 17:02:46 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown2194688570/002/d57992026/11/12 17:02:46 INFO Uploading batch count=158002026/11/12 17:02:46 ERROR Upload failed error="upload failed" count=158012026/11/12 17:02:46 INFO Uploading batch count=258022026/11/12 17:02:46 ERROR Upload failed error="upload failed" count=258032026/11/12 17:02:46 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown2194688570/002/e58042026/11/12 17:02:46 INFO Uploading batch count=158052026/11/12 17:02:46 ERROR Upload failed error="upload failed" count=15806--- PASS: TestWorkerKeepsPathItCannotStat (0.02s)5807--- PASS: TestFailedPathPrunedByLaterClosure (0.02s)58082026/11/12 17:02:46 ERROR Drain finished with paths left in queue remaining=158092026/11/12 17:02:46 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown2194688570/002/f58102026/11/12 17:02:46 ERROR Drain finished with paths left in queue remaining=105811--- PASS: TestDrainIsolatesPoisonPath (0.03s)5812--- PASS: TestDrainGivesUpWhenServerDown (0.03s)5813--- PASS: TestWorkerSkipsGCdPaths (0.04s)5814--- PASS: TestWorkerPrunesClosureDeps (0.04s)5815--- PASS: TestWorkerUploadsAndRemoves (0.04s)5816--- PASS: TestQueueRemoveLargeClosure (0.06s)58172026/11/12 17:02:46 ERROR Failed to decode request error="read unix /build/hook2700258312/test.sock->@: i/o timeout"58182026/11/12 17:02:46 ERROR Failed to write response error="write unix /build/hook2700258312/test.sock->@: i/o timeout"5819--- PASS: TestServerStalledClientDoesNotBlockShutdown (0.20s)5820--- PASS: TestQueueConcurrentWriters (0.22s)58212026/11/12 17:02:47 INFO Uploading batch count=158222026/11/12 17:02:47 INFO Uploading batch count=158232026/11/12 17:02:47 INFO Uploading batch count=158242026/11/12 17:02:47 ERROR Upload failed error="upload failed" count=158252026/11/12 17:02:47 INFO Uploading batch count=158262026/11/12 17:02:47 ERROR Upload failed error="upload failed" count=158272026/11/12 17:02:47 INFO Uploading batch count=158282026/11/12 17:02:47 ERROR Upload failed error="upload failed" count=158292026/11/12 17:02:47 INFO Uploading batch count=158302026/11/12 17:02:47 ERROR Upload failed error="upload failed" count=158312026/11/12 17:02:47 ERROR Drain finished with paths left in queue remaining=15832--- PASS: TestRunNotBlockedByPoisonHead (1.04s)5833PASS5834Running signing tests...5835=== RUN TestExternalSigner5836=== PAUSE TestExternalSigner5837=== RUN TestExternalSignerWrongPublicKey5838=== PAUSE TestExternalSignerWrongPublicKey5839=== RUN TestExternalSignerErrors5840=== PAUSE TestExternalSignerErrors5841=== RUN TestExternalSignerRestartsAfterCrash5842=== PAUSE TestExternalSignerRestartsAfterCrash5843=== RUN TestExternalSignerProgramErrorKeepsProcess5844=== PAUSE TestExternalSignerProgramErrorKeepsProcess5845=== RUN TestExternalSignerClose5846=== PAUSE TestExternalSignerClose5847=== RUN TestExternalSignerRejectsInvalidPublicKey5848=== PAUSE TestExternalSignerRejectsInvalidPublicKey5849=== RUN TestExternalSignerCancellation5850=== PAUSE TestExternalSignerCancellation5851=== RUN TestGenerateFingerprint5852=== PAUSE TestGenerateFingerprint5853=== RUN TestParseSigningKey5854=== PAUSE TestParseSigningKey5855=== RUN TestSign5856=== PAUSE TestSign5857=== RUN TestSignBatch5858=== PAUSE TestSignBatch5859=== RUN TestNilKeyPublicKey5860=== PAUSE TestNilKeyPublicKey5861=== RUN TestSignErrors5862=== PAUSE TestSignErrors5863=== CONT TestExternalSigner5864=== CONT TestExternalSignerCancellation5865=== CONT TestParseSigningKey5866=== CONT TestSign5867=== CONT TestExternalSignerProgramErrorKeepsProcess5868=== CONT TestExternalSignerRejectsInvalidPublicKey5869=== RUN TestExternalSignerRejectsInvalidPublicKey/#005870=== PAUSE TestExternalSignerRejectsInvalidPublicKey/#005871=== RUN TestExternalSignerRejectsInvalidPublicKey/no-colon5872=== PAUSE TestExternalSignerRejectsInvalidPublicKey/no-colon5873=== CONT TestExternalSignerClose5874=== CONT TestExternalSignerErrors5875=== RUN TestExternalSignerErrors/exit_at_startup5876=== PAUSE TestExternalSignerErrors/exit_at_startup5877=== RUN TestExternalSignerErrors/exit_on_request5878=== PAUSE TestExternalSignerErrors/exit_on_request5879=== RUN TestExternalSignerErrors/invalid_JSON5880=== PAUSE TestExternalSignerErrors/invalid_JSON5881=== RUN TestExternalSignerErrors/wrong_id5882=== PAUSE TestExternalSignerErrors/wrong_id5883=== CONT TestExternalSignerRestartsAfterCrash5884=== CONT TestSignBatch5885=== CONT TestSignErrors5886=== CONT TestNilKeyPublicKey5887=== CONT TestExternalSignerWrongPublicKey5888=== CONT TestGenerateFingerprint5889=== RUN TestParseSigningKey/valid_32-byte_key5890=== PAUSE TestParseSigningKey/valid_32-byte_key5891=== RUN TestParseSigningKey/valid_32-byte_key_with_different_name5892=== RUN TestExternalSignerErrors/wrong_signatures_shape5893--- PASS: TestNilKeyPublicKey (0.00s)5894=== PAUSE TestParseSigningKey/valid_32-byte_key_with_different_name5895=== RUN TestParseSigningKey/no_colon5896=== PAUSE TestParseSigningKey/no_colon5897=== RUN TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5898=== PAUSE TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5899=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5900=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5901=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5902=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5903=== RUN TestGenerateFingerprint/basic_with_references5904=== PAUSE TestExternalSignerErrors/wrong_signatures_shape5905=== RUN TestExternalSignerErrors/missing_signature_name5906=== CONT TestExternalSignerRejectsInvalidPublicKey/#005907=== PAUSE TestExternalSignerErrors/missing_signature_name5908=== RUN TestExternalSignerErrors/wrong_signature_length5909=== PAUSE TestExternalSignerErrors/wrong_signature_length5910--- PASS: TestSign (0.00s)5911=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5912=== PAUSE TestGenerateFingerprint/basic_with_references5913=== RUN TestGenerateFingerprint/no_references5914=== PAUSE TestGenerateFingerprint/no_references5915=== RUN TestGenerateFingerprint/unsorted_references_get_sorted5916=== PAUSE TestGenerateFingerprint/unsorted_references_get_sorted5917=== RUN TestGenerateFingerprint/invalid_nar_hash_prefix5918=== PAUSE TestGenerateFingerprint/invalid_nar_hash_prefix5919=== RUN TestGenerateFingerprint/invalid_nar_hash_length5920=== PAUSE TestGenerateFingerprint/invalid_nar_hash_length5921=== CONT TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5922=== RUN TestExternalSignerErrors/program_error5923--- PASS: TestSignBatch (0.00s)5924=== PAUSE TestExternalSignerErrors/program_error5925=== RUN TestExternalSignerErrors/wrong_shape5926=== CONT TestExternalSignerRejectsInvalidPublicKey/no-colon5927=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5928=== RUN TestSignErrors/nil_key5929=== PAUSE TestSignErrors/nil_key5930=== RUN TestSignErrors/nil_narinfo5931=== PAUSE TestSignErrors/nil_narinfo5932=== RUN TestGenerateFingerprint/invalid_store_path_prefix5933=== RUN TestParseSigningKey/empty_name5934=== PAUSE TestParseSigningKey/empty_name5935=== PAUSE TestExternalSignerErrors/wrong_shape5936=== RUN TestParseSigningKey/invalid_base645937=== RUN TestExternalSignerErrors/null5938=== PAUSE TestExternalSignerErrors/null5939=== RUN TestExternalSignerErrors/missing_signatures5940=== PAUSE TestExternalSignerErrors/missing_signatures5941=== RUN TestExternalSignerErrors/unrequested_key5942=== PAUSE TestExternalSignerErrors/unrequested_key5943=== RUN TestExternalSignerErrors/wrong_signature_name5944--- PASS: TestExternalSignerRejectsInvalidPublicKey (0.00s)5945 --- PASS: TestExternalSignerRejectsInvalidPublicKey/#00 (0.00s)5946 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:YQ== (0.00s)5947 --- PASS: TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= (0.00s)5948 --- PASS: TestExternalSignerRejectsInvalidPublicKey/no-colon (0.00s)5949 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:invalid! (0.00s)5950=== RUN TestSignErrors/invalid_narinfo5951=== PAUSE TestSignErrors/invalid_narinfo5952=== RUN TestSignErrors/canceled5953=== PAUSE TestGenerateFingerprint/invalid_store_path_prefix5954=== RUN TestGenerateFingerprint/invalid_reference_prefix5955=== PAUSE TestGenerateFingerprint/invalid_reference_prefix5956=== CONT TestGenerateFingerprint/basic_with_references5957=== CONT TestGenerateFingerprint/invalid_nar_hash_prefix5958=== PAUSE TestParseSigningKey/invalid_base645959=== RUN TestParseSigningKey/wrong_length5960=== CONT TestGenerateFingerprint/no_references5961=== PAUSE TestExternalSignerErrors/wrong_signature_name5962=== PAUSE TestSignErrors/canceled5963=== CONT TestGenerateFingerprint/invalid_nar_hash_length5964=== CONT TestGenerateFingerprint/invalid_reference_prefix5965=== CONT TestGenerateFingerprint/invalid_store_path_prefix5966=== CONT TestGenerateFingerprint/unsorted_references_get_sorted5967--- PASS: TestGenerateFingerprint (0.00s)5968 --- PASS: TestGenerateFingerprint/invalid_nar_hash_prefix (0.00s)5969 --- PASS: TestGenerateFingerprint/basic_with_references (0.00s)5970 --- PASS: TestGenerateFingerprint/no_references (0.00s)5971 --- PASS: TestGenerateFingerprint/invalid_nar_hash_length (0.00s)5972 --- PASS: TestGenerateFingerprint/invalid_reference_prefix (0.00s)5973 --- PASS: TestGenerateFingerprint/invalid_store_path_prefix (0.00s)5974 --- PASS: TestGenerateFingerprint/unsorted_references_get_sorted (0.00s)5975=== PAUSE TestParseSigningKey/wrong_length5976=== CONT TestParseSigningKey/valid_32-byte_key5977=== CONT TestParseSigningKey/invalid_base645978=== CONT TestParseSigningKey/no_colon5979=== RUN TestExternalSignerErrors/null_signatures5980=== PAUSE TestExternalSignerErrors/null_signatures5981=== CONT TestSignErrors/nil_key5982=== RUN TestExternalSignerErrors/too_few_signatures5983=== CONT TestParseSigningKey/wrong_length5984=== CONT TestSignErrors/invalid_narinfo5985=== CONT TestSignErrors/nil_narinfo5986=== CONT TestParseSigningKey/empty_name5987=== CONT TestParseSigningKey/valid_32-byte_key_with_different_name5988--- PASS: TestParseSigningKey (0.00s)5989 --- PASS: TestParseSigningKey/invalid_base64 (0.00s)5990 --- PASS: TestParseSigningKey/valid_32-byte_key (0.00s)5991 --- PASS: TestParseSigningKey/no_colon (0.00s)5992 --- PASS: TestParseSigningKey/wrong_length (0.00s)5993 --- PASS: TestParseSigningKey/empty_name (0.00s)5994 --- PASS: TestParseSigningKey/valid_32-byte_key_with_different_name (0.00s)5995=== CONT TestSignErrors/canceled5996--- PASS: TestSignErrors (0.00s)5997 --- PASS: TestSignErrors/nil_key (0.00s)5998 --- PASS: TestSignErrors/invalid_narinfo (0.00s)5999 --- PASS: TestSignErrors/nil_narinfo (0.00s)6000 --- PASS: TestSignErrors/canceled (0.00s)6001=== PAUSE TestExternalSignerErrors/too_few_signatures6002=== RUN TestExternalSignerErrors/invalid_signature_base646003=== PAUSE TestExternalSignerErrors/invalid_signature_base646004=== CONT TestExternalSignerErrors/exit_at_startup6005=== CONT TestExternalSignerErrors/too_few_signatures6006=== CONT TestExternalSignerErrors/unrequested_key6007=== CONT TestExternalSignerErrors/wrong_signature_length6008=== CONT TestExternalSignerErrors/wrong_signatures_shape6009=== CONT TestExternalSignerErrors/wrong_signature_name6010=== CONT TestExternalSignerErrors/null_signatures6011=== CONT TestExternalSignerErrors/missing_signature_name6012=== CONT TestExternalSignerErrors/program_error6013=== CONT TestExternalSignerErrors/missing_signatures6014=== CONT TestExternalSignerErrors/invalid_JSON6015=== CONT TestExternalSignerErrors/null6016=== CONT TestExternalSignerErrors/wrong_id6017=== CONT TestExternalSignerErrors/exit_on_request6018=== CONT TestExternalSignerErrors/invalid_signature_base646019=== CONT TestExternalSignerErrors/wrong_shape6020--- PASS: TestExternalSignerProgramErrorKeepsProcess (0.00s)6021--- PASS: TestExternalSignerWrongPublicKey (0.01s)6022--- PASS: TestExternalSignerClose (0.01s)6023--- PASS: TestExternalSignerRestartsAfterCrash (0.01s)6024--- PASS: TestExternalSigner (0.01s)6025--- PASS: TestExternalSignerErrors (0.00s)6026 --- PASS: TestExternalSignerErrors/unrequested_key (0.00s)6027 --- PASS: TestExternalSignerErrors/wrong_signature_length (0.01s)6028 --- PASS: TestExternalSignerErrors/too_few_signatures (0.01s)6029 --- PASS: TestExternalSignerErrors/invalid_signature_base64 (0.00s)6030 --- PASS: TestExternalSignerErrors/program_error (0.01s)6031 --- PASS: TestExternalSignerErrors/exit_at_startup (0.01s)6032 --- PASS: TestExternalSignerErrors/null (0.01s)6033 --- PASS: TestExternalSignerErrors/exit_on_request (0.00s)6034 --- PASS: TestExternalSignerErrors/wrong_id (0.01s)6035 --- PASS: TestExternalSignerErrors/invalid_JSON (0.01s)6036 --- PASS: TestExternalSignerErrors/wrong_shape (0.00s)6037 --- PASS: TestExternalSignerErrors/missing_signatures (0.01s)6038 --- PASS: TestExternalSignerErrors/null_signatures (0.01s)6039 --- PASS: TestExternalSignerErrors/missing_signature_name (0.01s)6040 --- PASS: TestExternalSignerErrors/wrong_signatures_shape (0.01s)6041 --- PASS: TestExternalSignerErrors/wrong_signature_name (0.01s)6042--- PASS: TestExternalSignerCancellation (0.02s)6043PASS6044Running OpenBao signer tests...6045=== RUN TestSign6046=== PAUSE TestSign6047=== RUN TestSignErrors6048=== PAUSE TestSignErrors6049=== RUN TestServe6050=== PAUSE TestServe6051=== RUN TestServeRejectsGarbage6052=== PAUSE TestServeRejectsGarbage6053=== CONT TestSign6054=== CONT TestSignErrors6055=== CONT TestServe6056=== RUN TestSignErrors/invalid_json6057=== PAUSE TestSignErrors/invalid_json6058=== CONT TestServeRejectsGarbage6059=== RUN TestSignErrors/no_prefix6060=== PAUSE TestSignErrors/no_prefix6061--- PASS: TestServeRejectsGarbage (0.00s)6062=== RUN TestSignErrors/invalid_base646063=== PAUSE TestSignErrors/invalid_base646064=== RUN TestSignErrors/http_error6065=== PAUSE TestSignErrors/http_error6066=== CONT TestSignErrors/invalid_json6067--- PASS: TestServe (0.00s)6068=== CONT TestSignErrors/invalid_base646069=== CONT TestSignErrors/no_prefix6070=== CONT TestSignErrors/http_error6071--- PASS: TestSign (0.01s)6072--- PASS: TestSignErrors (0.00s)6073 --- PASS: TestSignErrors/invalid_json (0.00s)6074 --- PASS: TestSignErrors/http_error (0.00s)6075 --- PASS: TestSignErrors/no_prefix (0.00s)6076 --- PASS: TestSignErrors/invalid_base64 (0.00s)6077PASS