niks3-go-unit-tests
checks.x86_64-linux.go-unit-tests
· build #507
· raw
1tribuchet: building on jamie2Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestUploadBuildLog_FileBodyReplayedOnRetry6=== PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry7=== RUN TestRegisterUploadedObjectReusesConnections8=== PAUSE TestRegisterUploadedObjectReusesConnections9=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica10=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica11=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun12=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun13=== RUN TestCaseHackSuffix14=== PAUSE TestCaseHackSuffix15=== RUN TestFilterOversizedClosures16=== PAUSE TestFilterOversizedClosures17=== RUN TestUploadMultipart_PartsInParallel18=== PAUSE TestUploadMultipart_PartsInParallel19=== RUN TestUploadMultipart_ReturnsAfterDroppedBody20=== PAUSE TestUploadMultipart_ReturnsAfterDroppedBody21=== RUN TestUploadMultipart_FailedProducerIsNotCompleted22=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted23=== RUN TestPartSizeForNAR24=== PAUSE TestPartSizeForNAR25=== RUN TestDumpPathCaseHackMatchesNix26--- PASS: TestDumpPathCaseHackMatchesNix (0.03s)27=== RUN TestDumpPathCaseHackCollision28--- PASS: TestDumpPathCaseHackCollision (0.00s)29=== RUN TestDumpPathMatchesNix30=== PAUSE TestDumpPathMatchesNix31=== RUN TestDumpPathSingleFile32=== PAUSE TestDumpPathSingleFile33=== RUN TestDumpPathWriterError34=== PAUSE TestDumpPathWriterError35=== RUN TestEncodeNixBase3236=== PAUSE TestEncodeNixBase3237=== RUN TestEncodeNixBase32WithRealHash38=== PAUSE TestEncodeNixBase32WithRealHash39=== RUN TestConvertHashToNix3240=== PAUSE TestConvertHashToNix3241=== RUN TestGetStorePathHash42=== PAUSE TestGetStorePathHash43=== RUN TestPathInfoHashCompatibility44=== PAUSE TestPathInfoHashCompatibility45=== RUN TestParsePathInfoJSON46=== PAUSE TestParsePathInfoJSON47=== RUN TestParsePathInfoJSONMultiplePaths48=== PAUSE TestParsePathInfoJSONMultiplePaths49=== RUN TestPathInfoCACompatibility50=== PAUSE TestPathInfoCACompatibility51=== RUN TestRateLimiterFeedback52=== PAUSE TestRateLimiterFeedback53=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess54=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess55=== RUN TestRegisterUploadedObject_BoundedAgainstSilentServer56=== PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer57=== RUN TestRegisterUploadedObject_OutlivesPush58=== PAUSE TestRegisterUploadedObject_OutlivesPush59=== RUN TestResolveStorePath60=== PAUSE TestResolveStorePath61=== RUN TestDoWithRetry_BodyReplayedViaGetBody62=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody63=== RUN TestRetryTimeoutOutlastsMaxRetries64=== PAUSE TestRetryTimeoutOutlastsMaxRetries65=== RUN TestDoWithRetry_FinalResponseBodyReadable66=== PAUSE TestDoWithRetry_FinalResponseBodyReadable67=== RUN TestShellSplit68=== PAUSE TestShellSplit69=== RUN TestShellSplitErrors70=== PAUSE TestShellSplitErrors71=== RUN TestStreamPushReportsEveryPath72=== PAUSE TestStreamPushReportsEveryPath73=== RUN TestStreamPushBatchesUnderLoad74=== PAUSE TestStreamPushBatchesUnderLoad75=== RUN TestStreamPushIsolatesFailures76=== PAUSE TestStreamPushIsolatesFailures77=== RUN TestStreamPushGivesUpOnDeadServer78=== PAUSE TestStreamPushGivesUpOnDeadServer79=== RUN TestStreamPushRequestLine80=== PAUSE TestStreamPushRequestLine81=== RUN TestStreamPushReportsSignatures82=== PAUSE TestStreamPushReportsSignatures83=== RUN TestStreamPushReportsSkippedPaths84=== PAUSE TestStreamPushReportsSkippedPaths85=== RUN TestClientSignaturesByStorePath86=== PAUSE TestClientSignaturesByStorePath87=== RUN TestStreamPushAnswersEveryLineWhenCanceled88=== PAUSE TestStreamPushAnswersEveryLineWhenCanceled89=== RUN TestStreamPushStopsWhenCanceled90=== PAUSE TestStreamPushStopsWhenCanceled91=== RUN TestSetClientTLS92=== PAUSE TestSetClientTLS93=== RUN TestSetClientTLSDoesNotMutateDefaultTransport94=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport95=== RUN TestSetClientTLSErrors96=== PAUSE TestSetClientTLSErrors97=== RUN TestStaticToken98=== PAUSE TestStaticToken99=== RUN TestFileTokenReadsAndCaches100=== PAUSE TestFileTokenReadsAndCaches101=== RUN TestFileTokenMissing102=== PAUSE TestFileTokenMissing103=== RUN TestFileTokenEmpty104=== PAUSE TestFileTokenEmpty105=== RUN TestScriptTokenNoExpiryRerunsEveryCall106=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall107=== RUN TestScriptTokenCachesUntilRefresh108=== PAUSE TestScriptTokenCachesUntilRefresh109=== RUN TestScriptTokenEmptyToken110=== PAUSE TestScriptTokenEmptyToken111=== RUN TestScriptTokenBadJSON112=== PAUSE TestScriptTokenBadJSON113=== RUN TestScriptTokenScriptFails114=== PAUSE TestScriptTokenScriptFails115=== RUN TestScriptTokenEmptyCommand116=== PAUSE TestScriptTokenEmptyCommand117=== CONT TestDoServerRequestAttachesToken118=== CONT TestRetryTimeoutOutlastsMaxRetries119=== CONT TestDumpPathWriterError120=== CONT TestEncodeNixBase32121=== RUN TestEncodeNixBase32/test_string_hash122=== PAUSE TestEncodeNixBase32/test_string_hash123=== RUN TestEncodeNixBase32/empty_input124=== CONT TestDoWithRetry_BodyReplayedViaGetBody125=== CONT TestResolveStorePath126=== CONT TestRegisterUploadedObject_OutlivesPush127=== CONT TestRegisterUploadedObject_BoundedAgainstSilentServer128=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess129=== CONT TestRateLimiterFeedback1302026/11/12 07:15:08 WARN Rate limiter enabled after throttle name=server-test rate=5131=== RUN TestRateLimiterFeedback/429_enables_limiter132=== CONT TestPathInfoCACompatibility133=== PAUSE TestRateLimiterFeedback/429_enables_limiter134=== CONT TestParsePathInfoJSONMultiplePaths135=== CONT TestParsePathInfoJSON136=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths137=== RUN TestParsePathInfoJSON/Nix_format138=== CONT TestPathInfoHashCompatibility139=== CONT TestGetStorePathHash140=== CONT TestConvertHashToNix32141=== CONT TestEncodeNixBase32WithRealHash142=== CONT TestUploadMultipart_PartsInParallel143=== CONT TestScriptTokenEmptyCommand144=== CONT TestScriptTokenScriptFails145=== CONT TestScriptTokenBadJSON146=== CONT TestScriptTokenEmptyToken147=== CONT TestScriptTokenCachesUntilRefresh148=== CONT TestScriptTokenNoExpiryRerunsEveryCall149=== PAUSE TestEncodeNixBase32/empty_input150=== CONT TestFileTokenEmpty151--- PASS: TestResolveStorePath (0.00s)152=== RUN TestPathInfoCACompatibility/null_ca_field153=== RUN TestRateLimiterFeedback/503_enables_limiter154=== CONT TestFileTokenMissing155=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)156=== PAUSE TestRateLimiterFeedback/503_enables_limiter1572026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:35811158=== RUN TestGetStorePathHash/valid_store_path159=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter1602026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:358111612026/11/12 07:15:08 WARN Rate limiter enabled after throttle name=server-test rate=51622026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:40377163=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)164=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon165=== RUN TestConvertHashToNix32/SRI_format_to_Nix32166=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32167=== RUN TestConvertHashToNix32/already_Nix32_format1682026/11/12 07:15:08 WARN Rate limiter backed off name=server-test rate=5169=== PAUSE TestConvertHashToNix32/already_Nix32_format1702026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:40377171=== RUN TestConvertHashToNix32/invalid_format172=== PAUSE TestConvertHashToNix32/invalid_format173=== PAUSE TestGetStorePathHash/valid_store_path174=== RUN TestGetStorePathHash/basename_without_hyphen_should_error175=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error176=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error177=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error178=== CONT TestClientSignaturesByStorePath1792026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:35811180=== CONT TestStreamPushReportsSkippedPaths181=== CONT TestStreamPushReportsSignatures182--- PASS: TestFileTokenMissing (0.00s)183--- PASS: TestScriptTokenEmptyCommand (0.00s)184--- PASS: TestFileTokenEmpty (0.00s)185--- PASS: TestEncodeNixBase32WithRealHash (0.00s)186--- PASS: TestScriptTokenScriptFails (0.00s)187--- PASS: TestDoServerRequestAttachesToken (0.04s)188=== CONT TestStaticToken189--- PASS: TestStaticToken (0.00s)190=== CONT TestStreamPushRequestLine191--- PASS: TestRegisterUploadedObject_OutlivesPush (0.01s)192--- PASS: TestClientSignaturesByStorePath (0.00s)193--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.04s)194=== PAUSE TestPathInfoCACompatibility/null_ca_field195=== CONT TestFileTokenReadsAndCaches196=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter197=== CONT TestSetClientTLSDoesNotMutateDefaultTransport198=== CONT TestSetClientTLSErrors199=== CONT TestStreamPushAnswersEveryLineWhenCanceled200=== CONT TestSetClientTLS201--- PASS: TestScriptTokenBadJSON (0.03s)202=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter203=== CONT TestStreamPushGivesUpOnDeadServer2042000/01/01 00:00:00 ERROR Upload failed error="context canceled" count=2205=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter206=== CONT TestStreamPushStopsWhenCanceled2072026/11/12 07:15:08 ERROR Upload failed error=boom count=12082026/11/12 07:15:08 ERROR Upload failed error=boom count=1209=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths2102026/11/12 07:15:08 ERROR Upload failed error="connection refused" count=20211=== PAUSE TestParsePathInfoJSON/Nix_format2122026/11/12 07:15:08 ERROR Server seems unavailable, giving up on batch untried=17213=== RUN TestPathInfoCACompatibility/old_string_format_-_text214--- PASS: TestStreamPushReportsSkippedPaths (0.00s)215--- PASS: TestStreamPushStopsWhenCanceled (0.00s)216--- PASS: TestStreamPushReportsSignatures (0.00s)2172026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:35811218--- PASS: TestStreamPushAnswersEveryLineWhenCanceled (0.00s)219--- PASS: TestFileTokenReadsAndCaches (0.00s)220--- PASS: TestStreamPushGivesUpOnDeadServer (0.00s)221=== CONT TestStreamPushIsolatesFailures222=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon223=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI224=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI225=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512226=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha5122272026/11/12 07:15:08 ERROR Upload failed error="bad path" count=3228=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error229=== CONT TestStreamPushBatchesUnderLoad230=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=false231=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=false232=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=true233=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=true234=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error235=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun236=== CONT TestShellSplitErrors237=== CONT TestCaseHackSuffix238=== CONT TestRegisterUploadedObjectReusesConnections239=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths240=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths241=== RUN TestParsePathInfoJSON/Lix_format242=== PAUSE TestParsePathInfoJSON/Lix_format243=== RUN TestParsePathInfoJSON/empty_input244=== PAUSE TestParsePathInfoJSON/empty_input245=== RUN TestParsePathInfoJSON/whitespace_only246=== PAUSE TestParsePathInfoJSON/whitespace_only247=== RUN TestParsePathInfoJSON/invalid_JSON248=== PAUSE TestParsePathInfoJSON/invalid_JSON249=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica250=== CONT TestDoWithRetry_FinalResponseBodyReadable251=== CONT TestUploadMultipart_FailedProducerIsNotCompleted252=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part253=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part254=== CONT TestShellSplit255=== CONT TestUploadMultipart_ReturnsAfterDroppedBody256=== CONT TestPartSizeForNAR257=== RUN TestPartSizeForNAR/zero_stays_at_minimum258=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum259=== RUN TestPartSizeForNAR/small_stays_at_minimum2602026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:35811261=== PAUSE TestPartSizeForNAR/small_stays_at_minimum262=== CONT TestDumpPathSingleFile263=== CONT TestStreamPushReportsEveryPath264=== CONT TestDumpPathMatchesNix265--- PASS: TestStreamPushIsolatesFailures (0.00s)266=== CONT TestFilterOversizedClosures267=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text268=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary2692026/11/12 07:15:08 WARN Rate limiter enabled after throttle name=server-test rate=5270=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary2712026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:33989272=== RUN TestSetClientTLSErrors/missing_cert_file273=== PAUSE TestSetClientTLSErrors/missing_cert_file274=== RUN TestSetClientTLSErrors/missing_key_file275=== PAUSE TestSetClientTLSErrors/missing_key_file2762026/11/12 07:15:08 INFO Garbage collection started277=== RUN TestSetClientTLSErrors/missing_ca_file2782026/11/12 07:15:08 INFO Garbage collection started279=== PAUSE TestSetClientTLSErrors/missing_ca_file280=== RUN TestSetClientTLSErrors/invalid_ca_file281=== PAUSE TestSetClientTLSErrors/invalid_ca_file282=== CONT TestEncodeNixBase32/test_string_hash283=== CONT TestEncodeNixBase32/empty_input284=== CONT TestConvertHashToNix32/SRI_format_to_Nix322852026/11/12 07:15:08 WARN Rate limiter backed off name=server-test rate=5286=== CONT TestConvertHashToNix32/invalid_format287=== CONT TestConvertHashToNix32/already_Nix32_format288=== CONT TestRateLimiterFeedback/429_enables_limiter289=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter290=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter291=== RUN TestSetClientTLS/rejects_connection_without_client_cert2922026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:35811293=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert294=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum295--- PASS: TestShellSplitErrors (0.00s)296--- PASS: TestScriptTokenEmptyToken (0.03s)297--- PASS: TestShellSplit (0.00s)298=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA299=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA300=== RUN TestSetClientTLS/preserves_debug_logging_transport301=== PAUSE TestSetClientTLS/preserves_debug_logging_transport3022026/11/12 07:15:08 WARN Rate limiter enabled after throttle name=server-test rate=5303=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum304=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts3052026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:39539306=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts307=== RUN TestPartSizeForNAR/1_TiB308=== PAUSE TestPartSizeForNAR/1_TiB309=== RUN TestPartSizeForNAR/5_TiB_S3_max_object310=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)3112026/11/12 07:15:08 WARN Rate limiter backed off name=server-test rate=5312=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512313=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon314=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=false315=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object316=== RUN TestPartSizeForNAR/capped_at_5_GiB317=== PAUSE TestPartSizeForNAR/capped_at_5_GiB318=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI319=== CONT TestGetStorePathHash/valid_store_path320=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error321=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths322=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths323=== CONT TestGetStorePathHash/basename_without_hyphen_should_error324=== CONT TestParsePathInfoJSON/empty_input3252026/11/12 07:15:08 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=0326=== CONT TestParsePathInfoJSON/whitespace_only3272026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:35811328=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part329=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error330=== CONT TestSetClientTLSErrors/missing_cert_file331=== RUN TestFilterOversizedClosures/no_limit_keeps_everything332--- PASS: TestEncodeNixBase32 (0.01s)333 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)334 --- PASS: TestEncodeNixBase32/empty_input (0.00s)335=== CONT TestRateLimiterFeedback/503_enables_limiter336=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=true337=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive338=== CONT TestUploadBuildLog_FileBodyReplayedOnRetry339--- PASS: TestConvertHashToNix32 (0.03s)340 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)341 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)342 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)343--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.00s)344--- PASS: TestDoWithRetry_FinalResponseBodyReadable (0.00s)345--- PASS: TestUploadMultipart_ReturnsAfterDroppedBody (0.00s)346--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (0.00s)347=== CONT TestSetClientTLSErrors/invalid_ca_file3482026/11/12 07:15:08 WARN Rate limiter enabled after throttle name=server-test rate=53492026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:46429350=== CONT TestSetClientTLSErrors/missing_key_file3512026/11/12 07:15:08 WARN Garbage collection finished on another replica. Its statistics are not available here3522026/11/12 07:15:08 WARN Rate limiter backed off name=server-test rate=5353=== CONT TestSetClientTLS/rejects_connection_without_client_cert3542026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:35811355=== CONT TestPartSizeForNAR/zero_stays_at_minimum356=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive357=== CONT TestSetClientTLS/preserves_debug_logging_transport358=== CONT TestParsePathInfoJSON/Lix_format3592026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:35811360=== RUN TestPathInfoCACompatibility/new_structured_format_-_text361=== CONT TestPartSizeForNAR/5_TiB_S3_max_object362=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything363--- PASS: TestStreamPushReportsEveryPath (0.00s)364=== CONT TestSetClientTLSErrors/missing_ca_file365=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA366=== CONT TestParsePathInfoJSON/Nix_format367=== CONT TestPartSizeForNAR/capped_at_5_GiB368=== CONT TestParsePathInfoJSON/invalid_JSON369=== CONT TestPartSizeForNAR/small_stays_at_minimum370=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary371=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped3722026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:35811373=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped374=== RUN TestFilterOversizedClosures/all_closures_skipped375=== PAUSE TestFilterOversizedClosures/all_closures_skipped376=== CONT TestFilterOversizedClosures/no_limit_keeps_everything377=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts378=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text379=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method380=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method381=== CONT TestPathInfoCACompatibility/null_ca_field382=== CONT TestFilterOversizedClosures/all_closures_skipped3832026/11/12 07:15:08 WARN Rate limiter enabled after throttle name=s3-test rate=5384=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method3852026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:372733862026/11/12 07:15:08 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50387=== CONT TestPathInfoCACompatibility/new_structured_format_-_text388=== CONT TestPartSizeForNAR/1_TiB389=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive390=== CONT TestPathInfoCACompatibility/old_string_format_-_text391=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped3922026/11/12 07:15:08 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000393--- PASS: TestPathInfoHashCompatibility (0.03s)394 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)395 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)396 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)397 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)3982026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:35811399=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum400--- PASS: TestParsePathInfoJSONMultiplePaths (0.04s)401 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)402 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)403--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (0.01s)404--- PASS: TestRateLimiterFeedback (0.04s)405 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s)406 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.00s)407 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.00s)408 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s)409--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.05s)410--- PASS: TestScriptTokenCachesUntilRefresh (0.05s)411--- PASS: TestGetStorePathHash (0.04s)412 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)413 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)414 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)415 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)416--- PASS: TestStreamPushBatchesUnderLoad (0.00s)417 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=false (0.00s)418 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=true (0.01s)419--- PASS: TestSetClientTLSErrors (0.00s)420 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)421 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)422 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)423 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)424--- PASS: TestUploadMultipart_FailedProducerIsNotCompleted (0.00s)425 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part (0.00s)426 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary (0.00s)427--- PASS: TestParsePathInfoJSON (0.04s)428 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)429 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)430 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)431 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)432 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)433--- PASS: TestPartSizeForNAR (0.00s)434 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)435 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)436 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)437 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)438 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)439 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)440 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)441--- PASS: TestPathInfoCACompatibility (0.05s)442 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)443 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)444 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)445 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)446 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)447--- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (0.01s)448--- PASS: TestFilterOversizedClosures (0.01s)449 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)450 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)451 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)4522026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:358114532026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:358114542026/11/12 07:15:08 http: TLS handshake error from 127.0.0.1:45072: remote error: tls: bad certificate4552026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:35811456--- PASS: TestSetClientTLS (0.01s)457 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.01s)458 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.00s)459 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.02s)4602026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:358114612026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:35811462--- PASS: TestStreamPushRequestLine (0.03s)4632026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:358114642026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:358114652026/11/12 07:15:08 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:35811466--- PASS: TestRetryTimeoutOutlastsMaxRetries (0.07s)467--- PASS: TestDumpPathSingleFile (0.03s)468--- PASS: TestRegisterUploadedObjectReusesConnections (0.03s)469--- PASS: TestDumpPathWriterError (0.08s)470--- PASS: TestCaseHackSuffix (0.04s)4712026/11/12 07:15:08 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:44089/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:44089/api/uploads/complete4722026/11/12 07:15:08 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4732026/11/12 07:15:08 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:44089/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:44089/api/uploads/complete4742026/11/12 07:15:08 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4752026/11/12 07:15:08 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:44089/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:44089/api/uploads/complete4762026/11/12 07:15:08 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"477--- PASS: TestRegisterUploadedObject_BoundedAgainstSilentServer (0.10s)478--- PASS: TestDumpPathMatchesNix (0.09s)479--- PASS: TestUploadMultipart_PartsInParallel (0.65s)480--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)481PASS482Running server tests...483The files belonging to this database system will be owned by user "nixbld".484This user must also own the server process.485486The database cluster will be initialized with locale "C".487The default database encoding has accordingly been set to "SQL_ASCII".488The default text search configuration will be set to "english".489490Data page checksums are enabled.491492creating directory /build/postgres2380176725/data ... ok493creating subdirectories ... ok494selecting dynamic shared memory implementation ... posix495selecting default "max_connections" ... 100496selecting default "shared_buffers" ... 128MB497selecting default time zone ... UTC498creating configuration files ... ok499running bootstrap script ... ok500performing post-bootstrap initialization ... ok501syncing data to disk ... ok502503initdb: warning: enabling "trust" authentication for local connections504initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.505506Success. You can now start the database server using:507508 pg_ctl -D /build/postgres2380176725/data -l logfile start509510/build/postgres2380176725:5432 - no response5112026-11-12 07:15:10.580 UTC [130] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit5122026-11-12 07:15:10.581 UTC [130] LOG: listening on Unix socket "/build/postgres2380176725/.s.PGSQL.5432"5132026-11-12 07:15:10.585 UTC [137] LOG: database system was shut down at 2026-11-12 07:15:10 UTC5142026-11-12 07:15:10.588 UTC [130] LOG: database system is ready to accept connections515/build/postgres2380176725:5432 - accepting connections516=== RUN TestAbortSucceeded517=== PAUSE TestAbortSucceeded518=== RUN TestService_AuthMiddleware519=== PAUSE TestService_AuthMiddleware520=== RUN TestService_AuthMiddleware_MTLSProxyHeader521=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader522=== RUN TestService_AuthMiddleware_MTLSBoundSubjects523=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects524=== RUN TestService_ReadAuthMiddleware525=== PAUSE TestService_ReadAuthMiddleware526=== RUN TestService_AuthMiddleware_OIDC527=== PAUSE TestService_AuthMiddleware_OIDC528=== RUN TestService_RequireScope_OIDC529=== PAUSE TestService_RequireScope_OIDC530=== RUN TestService_ReadScope_PublicByDefault531=== PAUSE TestService_ReadScope_PublicByDefault532=== RUN TestCacheConfigHandler533=== PAUSE TestCacheConfigHandler534=== RUN TestCacheStatsHandler535=== PAUSE TestCacheStatsHandler536=== RUN TestClientCADerivations537=== PAUSE TestClientCADerivations538=== RUN TestClientErrorHandling539=== PAUSE TestClientErrorHandling540=== RUN TestClientIntegration541=== PAUSE TestClientIntegration542=== RUN TestClientMultipleUploads543=== PAUSE TestClientMultipleUploads544=== RUN TestClientWithDependencies545=== PAUSE TestClientWithDependencies546=== RUN TestClientSharedPathCommittedMidPush547=== PAUSE TestClientSharedPathCommittedMidPush548=== RUN TestPinProtectsFromGC549=== PAUSE TestPinProtectsFromGC550=== RUN TestClientPushesUseOnePush551=== PAUSE TestClientPushesUseOnePush552=== RUN TestClientFallsBackToClosures553=== PAUSE TestClientFallsBackToClosures554=== RUN TestDeleteClosuresRacingPins555=== PAUSE TestDeleteClosuresRacingPins556=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock557=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock558=== RUN TestResolveDBConnectionString559=== PAUSE TestResolveDBConnectionString560=== RUN TestConnectWaitsForAPeerMigration561=== PAUSE TestConnectWaitsForAPeerMigration562=== RUN TestLeadElectsOneAndHandsOver563=== PAUSE TestLeadElectsOneAndHandsOver564=== RUN TestLeadIncumbentWinsAfterRestart5652026/11/12 07:15:11 INFO lead: acquired remote=192.0.2.1:1234 generation=15662026/11/12 07:15:11 INFO lead: released remote=192.0.2.1:1234 generation=15672026/11/12 07:15:11 INFO lead: acquired remote=192.0.2.1:1234 generation=25682026/11/12 07:15:12 INFO lead: released remote=192.0.2.1:1234 generation=2569--- PASS: TestLeadIncumbentWinsAfterRestart (1.89s)570=== RUN TestLeadUnannouncedStandbyYieldsToIncumbent571=== PAUSE TestLeadUnannouncedStandbyYieldsToIncumbent572=== RUN TestLeadIncumbentAnnouncesAtOnce573=== PAUSE TestLeadIncumbentAnnouncesAtOnce574=== RUN TestLeadGenerationGrowsWithEveryGrant575=== PAUSE TestLeadGenerationGrowsWithEveryGrant576=== RUN TestLeadEndsOnShutdown577=== PAUSE TestLeadEndsOnShutdown578=== RUN TestLeadEndsWhenItsConnectionHangs5792026/11/12 07:15:13 INFO lead: acquired remote=192.0.2.1:1234 generation=15802026-11-12 07:15:14.401 UTC [579] FATAL: terminating connection due to administrator command5812026/11/12 07:15:14 INFO lead: acquired remote=192.0.2.1:1234 generation=25822026/11/12 07:15:15 WARN lead: lock connection lost error="timeout: context deadline exceeded"5832026/11/12 07:15:15 INFO lead: released remote=192.0.2.1:1234 generation=15842026/11/12 07:15:15 INFO lead: released remote=192.0.2.1:1234 generation=2585--- PASS: TestLeadEndsWhenItsConnectionHangs (3.11s)586=== RUN TestGCAdvisoryLockBlocksConcurrentRun587=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun588=== RUN TestGCBugBareHashReferences589=== PAUSE TestGCBugBareHashReferences590=== RUN TestGCMetrics591=== PAUSE TestGCMetrics592=== RUN TestPushDedupSurvivesConcurrentGC593=== PAUSE TestPushDedupSurvivesConcurrentGC594=== RUN TestDeduplicatedObjectsRecordedAsPending595=== PAUSE TestDeduplicatedObjectsRecordedAsPending596=== RUN TestGCSweepSkipsPendingObjects597=== PAUSE TestGCSweepSkipsPendingObjects598=== RUN TestTombstonedObjectOfferedWithoutWaiting599=== PAUSE TestTombstonedObjectOfferedWithoutWaiting600=== RUN TestGCSweepDeliversEachKeyOnce601=== PAUSE TestGCSweepDeliversEachKeyOnce602=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection603=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection604=== RUN TestForceGCDuringPushOffersSweptObject605=== PAUSE TestForceGCDuringPushOffersSweptObject606=== RUN TestFailedS3DeleteKeepsTombstone607=== PAUSE TestFailedS3DeleteKeepsTombstone608=== RUN TestSweepSkipsRowsLockedByPush609=== PAUSE TestSweepSkipsRowsLockedByPush610=== RUN TestRegisterCompletedObjectNeedsPendingRow611=== PAUSE TestRegisterCompletedObjectNeedsPendingRow612=== RUN TestGetExistingObjectsLocksRows613=== PAUSE TestGetExistingObjectsLocksRows614=== RUN TestSweepRechecksPendingAfterLocking615=== PAUSE TestSweepRechecksPendingAfterLocking616=== RUN TestPushWaitsForSweepHoldingRow617=== PAUSE TestPushWaitsForSweepHoldingRow618=== RUN TestCommitRacingPendingCleanupKeepsObjects619=== PAUSE TestCommitRacingPendingCleanupKeepsObjects620=== RUN TestGCEndsOnShutdown621=== PAUSE TestGCEndsOnShutdown622=== RUN TestGCTaskStore_StartNew623=== PAUSE TestGCTaskStore_StartNew624=== RUN TestGCTaskStore_DeduplicateSameParams625=== PAUSE TestGCTaskStore_DeduplicateSameParams626=== RUN TestGCTaskStore_ConflictDifferentParams627=== PAUSE TestGCTaskStore_ConflictDifferentParams628=== RUN TestGCTaskStore_GetEmpty629=== PAUSE TestGCTaskStore_GetEmpty630=== RUN TestGCTaskStore_GetReturnsLatest631=== PAUSE TestGCTaskStore_GetReturnsLatest632=== RUN TestGCTaskStore_CompletedAllowsNewTask633=== PAUSE TestGCTaskStore_CompletedAllowsNewTask634=== RUN TestGCTaskStore_PhaseUpdates635=== PAUSE TestGCTaskStore_PhaseUpdates636=== RUN TestGCTaskStore_Fail637=== PAUSE TestGCTaskStore_Fail638=== RUN TestGracefulShutdownDrainsInflight639=== PAUSE TestGracefulShutdownDrainsInflight640=== RUN TestService_healthCheckHandler641=== PAUSE TestService_healthCheckHandler642=== RUN TestService_readinessHandler643=== PAUSE TestService_readinessHandler644=== RUN TestGenerateLandingPage645=== PAUSE TestGenerateLandingPage646=== RUN TestEnsureAbortUploadsRule647=== PAUSE TestEnsureAbortUploadsRule648=== RUN TestCacheConfigHandlerMaxNarSize649=== PAUSE TestCacheConfigHandlerMaxNarSize650=== RUN TestCreatePendingClosureRejectsOversizedNAR651=== PAUSE TestCreatePendingClosureRejectsOversizedNAR652=== RUN TestNARDeduplicationMetadataUploadBug653=== PAUSE TestNARDeduplicationMetadataUploadBug654=== RUN TestMetricsInventory655=== PAUSE TestMetricsInventory656=== RUN TestMetricsCommit657=== PAUSE TestMetricsCommit658=== RUN TestService_NativeMTLS659=== PAUSE TestService_NativeMTLS660=== RUN TestServerTLSConfig661=== PAUSE TestServerTLSConfig662=== RUN TestMultipartCleanup663=== PAUSE TestMultipartCleanup664=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded665=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded666=== RUN TestPendingCleanupUsesOneCutoff667=== PAUSE TestPendingCleanupUsesOneCutoff668=== RUN TestObjectStatsTrigger669=== PAUSE TestObjectStatsTrigger670=== RUN TestObjectStatsUpdatedOncePerCommit671=== PAUSE TestObjectStatsUpdatedOncePerCommit672=== RUN TestOrphanedObjectsGC673=== PAUSE TestOrphanedObjectsGC674=== RUN TestOrphanedObjectsGCStressTest675=== PAUSE TestOrphanedObjectsGCStressTest676=== RUN TestCreatePin_ReservedPins677=== PAUSE TestCreatePin_ReservedPins678=== RUN TestPresentReportsOnlyClosureRoots679=== PAUSE TestPresentReportsOnlyClosureRoots680=== RUN TestPresentRefreshesOnlyStaleClosures681=== PAUSE TestPresentRefreshesOnlyStaleClosures682=== RUN TestPresentNotReportedWhileGCDeletesClosure683=== PAUSE TestPresentNotReportedWhileGCDeletesClosure684=== RUN TestParseSingleRange685=== PAUSE TestParseSingleRange686=== RUN TestProxyHeadersOnlyTrustedOnSocket6872026/11/12 07:15:16 INFO Starting HTTP server address=proxy.sock6882026/11/12 07:15:16 INFO Starting HTTP server address=127.0.0.1:463356892026/11/12 07:15:16 WARN mTLS auth: subject not in bound subjects subject="CN=someone"6902026/11/12 07:15:16 INFO Shutdown signal received, draining in-flight requests timeout=10s691--- PASS: TestProxyHeadersOnlyTrustedOnSocket (0.11s)692=== RUN TestIsValidCachePath693=== PAUSE TestIsValidCachePath694=== RUN TestReadProxyNarinfo695=== PAUSE TestReadProxyNarinfo696=== RUN TestReadProxyNarinfoAlreadyDecompressed697=== PAUSE TestReadProxyNarinfoAlreadyDecompressed698=== RUN TestReadProxyContentNegotiation699=== PAUSE TestReadProxyContentNegotiation700=== RUN TestReadProxyNarStreaming701=== PAUSE TestReadProxyNarStreaming702=== RUN TestReadProxy404703=== PAUSE TestReadProxy404704=== RUN TestReadProxyInvalidPath705=== PAUSE TestReadProxyInvalidPath706=== RUN TestReadProxyHead707=== PAUSE TestReadProxyHead708=== RUN TestReadProxyOutlastsServerWriteTimeout709=== PAUSE TestReadProxyOutlastsServerWriteTimeout710=== RUN TestReadProxyConditionalGet711=== PAUSE TestReadProxyConditionalGet712=== RUN TestReadProxyRootRedirectsToIndexHTML713=== PAUSE TestReadProxyRootRedirectsToIndexHTML714=== RUN TestReadProxyDisabled715=== PAUSE TestReadProxyDisabled716=== RUN TestReadRedirectNar717=== PAUSE TestReadRedirectNar718=== RUN TestReadRedirectKeepsNarinfoProxied719=== PAUSE TestReadRedirectKeepsNarinfoProxied720=== RUN TestReadProxyRangeRequest721=== PAUSE TestReadProxyRangeRequest722=== RUN TestReadRedirectUsesPublicS3URL723=== PAUSE TestReadRedirectUsesPublicS3URL724=== RUN TestPush_OverlappingRootsStoreOneRowPerKey725=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey726=== RUN TestPush_CompleteCommitsEveryRoot727=== PAUSE TestPush_CompleteCommitsEveryRoot728=== RUN TestPush_RepeatedRootCommitsOnce729=== PAUSE TestPush_RepeatedRootCommitsOnce730=== RUN TestPush_CompleteWaitsForCleanupOfThePush731=== PAUSE TestPush_CompleteWaitsForCleanupOfThePush732=== RUN TestPush_RootsAreLockedInByteOrder733=== PAUSE TestPush_RootsAreLockedInByteOrder734=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit735=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit736=== RUN TestPush_RejectsBadRequests737=== PAUSE TestPush_RejectsBadRequests738=== RUN TestPush_SignsNarinfosOfItsPendingObjects739=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects740=== RUN TestCommitCanBeRepeatedUntilCleanup741=== PAUSE TestCommitCanBeRepeatedUntilCleanup742=== RUN TestSharedNarBothUploadsComplete743=== PAUSE TestSharedNarBothUploadsComplete744=== RUN TestCompleteMultipartUpload_ErrorButObjectExists745=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists746=== RUN TestCompletedNarNotReofferedAcrossClosures747=== PAUSE TestCompletedNarNotReofferedAcrossClosures748=== RUN TestPresignedUploadRegisteredBeforeCommit749=== PAUSE TestPresignedUploadRegisteredBeforeCommit750=== RUN TestService_Rustfstest751=== PAUSE TestService_Rustfstest752=== RUN TestParseSize753=== PAUSE TestParseSize754=== RUN TestSkippedUploadsHandler755=== PAUSE TestSkippedUploadsHandler756=== RUN TestSystemdListenerNotActivated757--- PASS: TestSystemdListenerNotActivated (0.00s)758=== RUN TestWatchdogBeatsWhenHealthy759--- PASS: TestWatchdogBeatsWhenHealthy (0.02s)760=== RUN TestWatchdogSkipsWhenUnhealthy7612026/11/12 07:15:16 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7622026/11/12 07:15:16 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7632026/11/12 07:15:16 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7642026/11/12 07:15:16 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7652026/11/12 07:15:16 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7662026/11/12 07:15:16 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7672026/11/12 07:15:16 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7682026/11/12 07:15:16 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7692026/11/12 07:15:16 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7702026/11/12 07:15:16 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"771--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)772=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle773=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle774=== RUN TestProxyWriteTimeout775=== PAUSE TestProxyWriteTimeout776=== RUN TestPendingClosureWriteTimeout777=== PAUSE TestPendingClosureWriteTimeout778=== RUN TestIsValidUploadKey779=== PAUSE TestIsValidUploadKey780=== RUN TestUploadHandlersRejectInvalidKeys781=== PAUSE TestUploadHandlersRejectInvalidKeys782=== RUN TestUploadHandlersRejectOversizedBody783=== PAUSE TestUploadHandlersRejectOversizedBody784=== RUN TestService_cleanupPendingClosuresHandler785=== PAUSE TestService_cleanupPendingClosuresHandler786=== RUN TestService_createPendingClosureHandler787=== PAUSE TestService_createPendingClosureHandler788=== RUN TestService_verifyS3Integrity789=== PAUSE TestService_verifyS3Integrity790=== RUN TestService_verifyS3ReuploadsStaleNarinfo791=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo792=== RUN TestCompleteMultipartUnregistered793=== PAUSE TestCompleteMultipartUnregistered794=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT795=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT796=== CONT TestAbortSucceeded797=== CONT TestNARDeduplicationMetadataUploadBug798--- PASS: TestAbortSucceeded (0.00s)799=== CONT TestCreatePendingClosureRejectsOversizedNAR800=== CONT TestCacheConfigHandlerMaxNarSize801=== CONT TestEnsureAbortUploadsRule802=== CONT TestGenerateLandingPage8032026/11/12 07:15:16 INFO Received uploads request method=POST path=/api/pending_closures804=== CONT TestService_readinessHandler805=== CONT TestService_healthCheckHandler806=== CONT TestGracefulShutdownDrainsInflight807=== CONT TestGCTaskStore_Fail808=== CONT TestGCSweepDeliversEachKeyOnce809=== CONT TestGCTaskStore_PhaseUpdates810=== CONT TestGCTaskStore_CompletedAllowsNewTask811=== CONT TestGCTaskStore_GetReturnsLatest812=== CONT TestGCTaskStore_GetEmpty813=== CONT TestGCTaskStore_ConflictDifferentParams814=== CONT TestGCTaskStore_DeduplicateSameParams815=== CONT TestGCTaskStore_StartNew816=== CONT TestGCEndsOnShutdown817=== CONT TestCommitRacingPendingCleanupKeepsObjects818=== CONT TestPushWaitsForSweepHoldingRow819=== CONT TestSweepRechecksPendingAfterLocking8202026/11/12 07:15:16 INFO Starting HTTP server address=127.0.0.1:43329821=== CONT TestGetExistingObjectsLocksRows822=== CONT TestRegisterCompletedObjectNeedsPendingRow823=== CONT TestSweepSkipsRowsLockedByPush824=== CONT TestFailedS3DeleteKeepsTombstone825--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)826=== CONT TestForceGCDuringPushOffersSweptObject827=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection828=== CONT TestTombstonedObjectOfferedWithoutWaiting8292026/11/12 07:15:16 INFO Shutdown signal received, draining in-flight requests timeout=10s830=== RUN TestGCEndsOnShutdown/before_the_run831=== PAUSE TestGCEndsOnShutdown/before_the_run832=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete833=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete834=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark835=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark836--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.00s)837--- PASS: TestGCTaskStore_Fail (0.00s)838--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)839--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)840--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)841--- PASS: TestGCTaskStore_StartNew (0.00s)842--- PASS: TestGCTaskStore_GetEmpty (0.00s)843--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)844--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)845=== CONT TestLeadEndsOnShutdown846=== CONT TestDeduplicatedObjectsRecordedAsPending847=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows848=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows849=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check850=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check851=== CONT TestLeadIncumbentAnnouncesAtOnce852=== CONT TestGCSweepSkipsPendingObjects853=== CONT TestPushDedupSurvivesConcurrentGC854=== CONT TestGCBugBareHashReferences855=== CONT TestGCAdvisoryLockBlocksConcurrentRun856--- PASS: TestGenerateLandingPage (0.00s)857=== CONT TestLeadGenerationGrowsWithEveryGrant858=== CONT TestGCMetrics859--- PASS: TestGracefulShutdownDrainsInflight (0.07s)860=== CONT TestLeadUnannouncedStandbyYieldsToIncumbent861=== NAME TestNARDeduplicationMetadataUploadBug862 metadata_upload_test.go:48: First store path: /build/TestNARDeduplicationMetadataUploadBug500885329/001/store/6d9m4x2war4l9hclcysj3yfvwb5khk82-file1.txt8632026/11/12 07:15:16 INFO Received uploads request method=POST path=/api/pending_closures864--- PASS: TestTombstonedObjectOfferedWithoutWaiting (0.34s)865=== CONT TestLeadElectsOneAndHandsOver866--- PASS: TestService_healthCheckHandler (0.36s)867=== CONT TestConnectWaitsForAPeerMigration8682026/11/12 07:15:16 INFO Aborted multipart uploads count=0 kept=08692026/11/12 07:15:16 WARN Force mode enabled - objects will be deleted immediately without grace period8702026/11/12 07:15:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=08712026/11/12 07:15:16 INFO Vacuumed table table=pending_closures8722026/11/12 07:15:16 INFO Vacuumed table table=commit_receipts8732026/11/12 07:15:16 INFO Vacuumed table table=pending_objects8742026/11/12 07:15:16 INFO Vacuumed table table=multipart_uploads8752026/11/12 07:15:16 INFO Vacuumed table table=closures8762026/11/12 07:15:16 INFO Vacuumed table table=objects877--- PASS: TestCommitRacingPendingCleanupKeepsObjects (0.39s)878=== CONT TestResolveDBConnectionString879=== RUN TestResolveDBConnectionString/flag_wins880=== PAUSE TestResolveDBConnectionString/flag_wins881=== RUN TestResolveDBConnectionString/file_when_flag_empty882=== PAUSE TestResolveDBConnectionString/file_when_flag_empty883=== RUN TestResolveDBConnectionString/missing_file_is_an_error884=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error885=== RUN TestResolveDBConnectionString/PGHOST_allows_empty886=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty887=== RUN TestResolveDBConnectionString/nothing_configured888=== PAUSE TestResolveDBConnectionString/nothing_configured889=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock8902026/11/12 07:15:16 INFO Installed lifecycle rule aborting incomplete NAR uploads bucket=bucket8 days=2891--- PASS: TestEnsureAbortUploadsRule (0.40s)892=== CONT TestDeleteClosuresRacingPins8932026/11/12 07:15:16 INFO Received push request method=POST path=/api/pushes8942026/11/12 07:15:16 INFO Received uploads request method=POST path=/api/pending_closures8952026/11/12 07:15:16 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)8962026/11/12 07:15:16 INFO Uploading 6d9m4x2war4l9hclcysj3yfvwb5khk82-file1.txt (160B)8972026/11/12 07:15:16 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"8982026/11/12 07:15:16 WARN Failed to register uploaded object key=6d9m4x2war4l9hclcysj3yfvwb5khk82.ls error="server returned 404: 404 page not found\n"8992026/11/12 07:15:16 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9002026/11/12 07:15:16 INFO Signed narinfos id=1 count=19012026/11/12 07:15:16 INFO Uploading 1 narinfos9022026/11/12 07:15:16 INFO Received complete push request method=POST path=/api/pushes/1/complete9032026/11/12 07:15:16 WARN Failed to register uploaded object key=6d9m4x2war4l9hclcysj3yfvwb5khk82.narinfo error="server returned 404: 404 page not found\n"9042026/11/12 07:15:16 INFO Completed push id=1 objects=3 duration=9.962054ms9052026/11/12 07:15:16 INFO Upload complete. (76ms)906=== NAME TestNARDeduplicationMetadataUploadBug907 metadata_upload_test.go:54: Retrieved narinfo from S3:908 StorePath: /build/TestNARDeduplicationMetadataUploadBug500885329/001/store/6d9m4x2war4l9hclcysj3yfvwb5khk82-file1.txt909 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst910 Compression: zstd911 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf912 NarSize: 160913 References: 914 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf915 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)916 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):917 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}9182026/11/12 07:15:16 INFO Aborted multipart uploads count=0 kept=09192026/11/12 07:15:16 WARN Force mode enabled - objects will be deleted immediately without grace period9202026/11/12 07:15:16 INFO Received uploads request method=POST path=/api/pending_closures9212026/11/12 07:15:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=09222026/11/12 07:15:16 INFO Vacuumed table table=pending_closures9232026/11/12 07:15:16 INFO Vacuumed table table=commit_receipts9242026/11/12 07:15:16 INFO Vacuumed table table=pending_objects9252026/11/12 07:15:16 INFO Vacuumed table table=multipart_uploads9262026/11/12 07:15:16 INFO Vacuumed table table=closures927 metadata_upload_test.go:64: Second store path (same content): /build/TestNARDeduplicationMetadataUploadBug500885329/001/store/3f7aa6xh1j63xg3as24sy21cismk9z6v-file2.txt9282026/11/12 07:15:16 INFO Vacuumed table table=objects929--- PASS: TestDeduplicatedObjectsRecordedAsPending (0.49s)930=== CONT TestClientFallsBackToClosures9312026/11/12 07:15:16 INFO Aborted multipart uploads count=0 kept=09322026/11/12 07:15:16 WARN Force mode enabled - objects will be deleted immediately without grace period9332026/11/12 07:15:16 INFO Aborted multipart uploads count=0 kept=09342026/11/12 07:15:16 WARN Force mode enabled - objects will be deleted immediately without grace period9352026/11/12 07:15:16 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."9362026/11/12 07:15:16 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."937--- PASS: TestFailedS3DeleteKeepsTombstone (0.52s)938=== CONT TestClientPushesUseOnePush9392026/11/12 07:15:16 INFO Aborted multipart uploads count=0 kept=09402026/11/12 07:15:16 WARN readiness check failed error="closed pool"941--- PASS: TestService_readinessHandler (0.53s)942=== CONT TestPinProtectsFromGC9432026/11/12 07:15:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=09442026/11/12 07:15:16 INFO Vacuumed table table=pending_closures9452026/11/12 07:15:16 INFO Vacuumed table table=commit_receipts9462026/11/12 07:15:16 INFO Vacuumed table table=pending_objects9472026/11/12 07:15:16 INFO Vacuumed table table=multipart_uploads9482026/11/12 07:15:16 INFO Vacuumed table table=closures9492026/11/12 07:15:16 WARN Force mode enabled - objects will be deleted immediately without grace period9502026/11/12 07:15:16 INFO Vacuumed table table=objects951--- PASS: TestSweepSkipsRowsLockedByPush (0.54s)952=== CONT TestClientSharedPathCommittedMidPush9532026/11/12 07:15:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=09542026/11/12 07:15:16 INFO Received push request method=POST path=/api/pushes9552026/11/12 07:15:16 INFO Vacuumed table table=pending_closures9562026/11/12 07:15:16 INFO Vacuumed table table=commit_receipts9572026/11/12 07:15:16 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)9582026/11/12 07:15:16 INFO Vacuumed table table=pending_objects9592026/11/12 07:15:16 INFO Vacuumed table table=multipart_uploads9602026/11/12 07:15:16 INFO Vacuumed table table=closures9612026/11/12 07:15:16 INFO Vacuumed table table=objects962--- PASS: TestRegisterCompletedObjectNeedsPendingRow (0.56s)963=== CONT TestClientWithDependencies964--- PASS: TestSweepRechecksPendingAfterLocking (0.56s)965=== CONT TestClientMultipleUploads9662026/11/12 07:15:16 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign9672026/11/12 07:15:16 WARN Failed to register uploaded object key=3f7aa6xh1j63xg3as24sy21cismk9z6v.ls error="server returned 404: 404 page not found\n"9682026/11/12 07:15:16 INFO Signed narinfos id=2 count=19692026/11/12 07:15:16 INFO Uploading 1 narinfos9702026/11/12 07:15:16 INFO Received complete push request method=POST path=/api/pushes/2/complete9712026/11/12 07:15:16 WARN Failed to register uploaded object key=3f7aa6xh1j63xg3as24sy21cismk9z6v.narinfo error="server returned 404: 404 page not found\n"9722026/11/12 07:15:16 INFO Completed push id=2 objects=3 duration=2.845528ms9732026/11/12 07:15:16 INFO Upload complete. (59ms)974=== NAME TestNARDeduplicationMetadataUploadBug975 metadata_upload_test.go:76: Retrieved narinfo from S3:976 StorePath: /build/TestNARDeduplicationMetadataUploadBug500885329/001/store/3f7aa6xh1j63xg3as24sy21cismk9z6v-file2.txt977 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst978 Compression: zstd979 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf980 NarSize: 160981 References: 982 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf983 metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes)984 metadata_upload_test.go:77: Decompressed .ls content (49 bytes):985 {"version":1,"root":{"type":"regular","size":44}}986--- PASS: TestNARDeduplicationMetadataUploadBug (0.59s)987=== CONT TestClientIntegration9882026/11/12 07:15:16 INFO Aborted multipart uploads count=0 kept=09892026/11/12 07:15:16 WARN Force mode enabled - objects will be deleted immediately without grace period9902026/11/12 07:15:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=09912026/11/12 07:15:16 INFO Vacuumed table table=pending_closures9922026/11/12 07:15:16 INFO Vacuumed table table=commit_receipts9932026/11/12 07:15:16 INFO Vacuumed table table=pending_objects9942026/11/12 07:15:16 INFO Vacuumed table table=multipart_uploads9952026/11/12 07:15:16 INFO Vacuumed table table=closures9962026/11/12 07:15:16 INFO Vacuumed table table=objects997--- PASS: TestGCMetrics (0.61s)998=== CONT TestClientErrorHandling999=== RUN TestClientErrorHandling/InvalidStorePath1000=== PAUSE TestClientErrorHandling/InvalidStorePath1001=== RUN TestClientErrorHandling/InvalidAuthToken1002=== PAUSE TestClientErrorHandling/InvalidAuthToken1003=== RUN TestClientErrorHandling/ServerNotAvailable1004=== PAUSE TestClientErrorHandling/ServerNotAvailable1005=== CONT TestClientCADerivations1006--- PASS: TestGetExistingObjectsLocksRows (0.63s)1007=== CONT TestCacheStatsHandler10082026/11/12 07:15:16 INFO lead: acquired remote=192.0.2.1:1234 generation=110092026/11/12 07:15:17 INFO Received uploads request method=POST path=/api/pending_closures10102026/11/12 07:15:17 INFO Received uploads request method=POST path=/api/pending_closures10112026/11/12 07:15:17 INFO Received uploads request method=POST path=/api/pending_closures10122026/11/12 07:15:17 INFO Aborted multipart uploads count=0 kept=010132026/11/12 07:15:17 WARN Force mode enabled - objects will be deleted immediately without grace period10142026/11/12 07:15:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010152026/11/12 07:15:17 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=nixbld database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"10162026/11/12 07:15:17 INFO Vacuumed table table=pending_closures1017--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (0.78s)10182026/11/12 07:15:17 INFO Vacuumed table table=commit_receipts1019=== CONT TestCacheConfigHandler1020=== RUN TestCacheConfigHandler/full_config,_no_issuer1021=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1022=== RUN TestCacheConfigHandler/no_cache_url_configured1023=== PAUSE TestCacheConfigHandler/no_cache_url_configured1024=== RUN TestCacheConfigHandler/no_signing_keys1025=== PAUSE TestCacheConfigHandler/no_signing_keys1026=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1027=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1028=== CONT TestService_ReadScope_PublicByDefault10292026/11/12 07:15:17 INFO Vacuumed table table=pending_objects10302026/11/12 07:15:17 INFO Vacuumed table table=multipart_uploads10312026/11/12 07:15:17 INFO Vacuumed table table=closures10322026/11/12 07:15:17 INFO Vacuumed table table=objects1033--- PASS: TestGCSweepSkipsPendingObjects (0.79s)1034=== CONT TestService_RequireScope_OIDC10352026/11/12 07:15:17 INFO Aborted multipart uploads count=0 kept=010362026/11/12 07:15:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010372026/11/12 07:15:17 INFO lead: acquired remote=192.0.2.1:1234 generation=110382026/11/12 07:15:17 INFO lead: released remote=192.0.2.1:1234 generation=11039--- PASS: TestLeadIncumbentAnnouncesAtOnce (0.81s)1040=== CONT TestService_AuthMiddleware_OIDC10412026/11/12 07:15:17 INFO Vacuumed table table=pending_closures10422026/11/12 07:15:17 INFO Vacuumed table table=commit_receipts10432026/11/12 07:15:17 INFO Vacuumed table table=pending_objects10442026/11/12 07:15:17 INFO Vacuumed table table=multipart_uploads10452026/11/12 07:15:17 INFO Vacuumed table table=closures10462026/11/12 07:15:17 INFO Vacuumed table table=objects10472026/11/12 07:15:17 INFO Aborted multipart uploads count=0 kept=010482026/11/12 07:15:17 WARN Force mode enabled - objects will be deleted immediately without grace period10492026/11/12 07:15:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010502026/11/12 07:15:17 INFO Vacuumed table table=pending_closures10512026/11/12 07:15:17 INFO Vacuumed table table=commit_receipts10522026/11/12 07:15:17 INFO Vacuumed table table=pending_objects10532026/11/12 07:15:17 INFO Vacuumed table table=multipart_uploads10542026/11/12 07:15:17 INFO Vacuumed table table=closures10552026/11/12 07:15:17 INFO Vacuumed table table=objects1056--- PASS: TestPushDedupSurvivesConcurrentGC (0.83s)1057=== CONT TestService_ReadAuthMiddleware10582026/11/12 07:15:17 INFO Aborted multipart uploads count=0 kept=010592026/11/12 07:15:17 INFO Received uploads request method=POST path=/api/pending_closures10602026/11/12 07:15:17 WARN Force mode enabled - objects will be deleted immediately without grace period1061--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (0.88s)1062=== CONT TestService_AuthMiddleware_MTLSBoundSubjects10632026/11/12 07:15:17 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:36801/oidc10642026/11/12 07:15:17 INFO lead: acquired remote=192.0.2.1:1234 generation=110652026/11/12 07:15:17 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:35307/oidc10662026/11/12 07:15:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=010672026/11/12 07:15:17 INFO Vacuumed table table=pending_closures10682026/11/12 07:15:17 INFO Vacuumed table table=commit_receipts10692026/11/12 07:15:17 INFO lead: acquired remote=192.0.2.1:1234 generation=110702026/11/12 07:15:17 INFO Vacuumed table table=pending_objects10712026/11/12 07:15:17 INFO Vacuumed table table=multipart_uploads10722026/11/12 07:15:17 INFO Vacuumed table table=closures10732026/11/12 07:15:17 INFO Vacuumed table table=objects1074--- PASS: TestGCSweepDeliversEachKeyOnce (0.92s)1075=== CONT TestService_AuthMiddleware_MTLSProxyHeader10762026/11/12 07:15:17 INFO lead: acquired remote=192.0.2.1:1234 generation=11077--- PASS: TestGCBugBareHashReferences (0.95s)1078=== CONT TestService_AuthMiddleware1079--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (0.75s)1080=== CONT TestReadProxyRangeRequest1081=== NAME TestPinProtectsFromGC1082 client_integration_test.go:791: Pinned store path: /build/TestPinProtectsFromGC817703581/001/store/0s5m1g39h9cj0cv1312h4zbqi3ld5669-pinned-file.txt1083 client_integration_test.go:792: Unpinned store path: /build/TestPinProtectsFromGC817703581/001/store/i0slx1zz8cdy95pq15ls7kph4kbyx8kz-unpinned-file.txt1084=== NAME TestClientMultipleUploads1085 client_integration_test.go:406: Created store path 0: /build/TestClientMultipleUploads2538324567/001/store/k2vnz0nsqrd8x0ywci63lcl3dnink16l-test-file-0.txt1086 client_integration_test.go:406: Created store path 1: /build/TestClientMultipleUploads2538324567/001/store/h925nr3f9qyn7bd164g7y0rfa3qb49c5-test-file-1.txt10872026/11/12 07:15:17 INFO Received uploads request method=POST path=/api/pending_closures10882026/11/12 07:15:17 INFO Received push request method=POST path=/api/pushes10892026/11/12 07:15:17 INFO Received uploads request method=POST path=/api/pending_closures10902026/11/12 07:15:17 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)10912026/11/12 07:15:17 INFO Uploading yy4smgc9ah2cdaz0qchczhlkwwgvdgld-shared-dep (136B)10922026/11/12 07:15:17 INFO Uploading 5hxx9zblp7s69ww4ifxhwb8vlcx04ypk-a (216B)10932026/11/12 07:15:17 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)10942026/11/12 07:15:17 INFO Uploading 0s5m1g39h9cj0cv1312h4zbqi3ld5669-pinned-file.txt (128B)10952026/11/12 07:15:17 WARN Failed to register uploaded object key=053sdzf2rmgy7fkyqj43vwady90id0lb.ls error="server returned 404: 404 page not found\n"10962026/11/12 07:15:17 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"10972026/11/12 07:15:17 WARN Failed to register uploaded object key=nar/1n8c8yn4clhdg3f82k9cl7a32rf70y7d2vs332683ix9hnkzrks3.nar.zst error="server returned 404: 404 page not found\n"10982026/11/12 07:15:17 WARN Failed to register uploaded object key=yy4smgc9ah2cdaz0qchczhlkwwgvdgld.ls error="server returned 404: 404 page not found\n"10992026/11/12 07:15:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign11002026/11/12 07:15:17 WARN Failed to register uploaded object key=5hxx9zblp7s69ww4ifxhwb8vlcx04ypk.ls error="server returned 404: 404 page not found\n"11012026/11/12 07:15:17 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"11022026/11/12 07:15:17 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign11032026/11/12 07:15:17 INFO Signed narinfos id=2 count=211042026/11/12 07:15:17 WARN Failed to register uploaded object key=0s5m1g39h9cj0cv1312h4zbqi3ld5669.ls error="server returned 404: 404 page not found\n"11052026/11/12 07:15:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign11062026/11/12 07:15:17 INFO Signed narinfos id=1 count=111072026/11/12 07:15:17 INFO Signed narinfos id=1 count=211082026/11/12 07:15:17 INFO Uploading 1 narinfos11092026/11/12 07:15:17 INFO Uploading 4 narinfos1110=== NAME TestClientIntegration1111 client_integration_test.go:292: Created store path: /build/TestClientIntegration223125666/002/store/f7pgvkcl64k7i9bf888y4f75vqnh87fq-test-file.txt11122026/11/12 07:15:17 WARN Failed to register uploaded object key=yy4smgc9ah2cdaz0qchczhlkwwgvdgld.narinfo error="server returned 404: 404 page not found\n"11132026/11/12 07:15:17 WARN Failed to register uploaded object key=5hxx9zblp7s69ww4ifxhwb8vlcx04ypk.narinfo error="server returned 404: 404 page not found\n"11142026/11/12 07:15:17 INFO Received complete push request method=POST path=/api/pushes/1/complete11152026/11/12 07:15:17 WARN Failed to register uploaded object key=0s5m1g39h9cj0cv1312h4zbqi3ld5669.narinfo error="server returned 404: 404 page not found\n"11162026/11/12 07:15:17 WARN Failed to register uploaded object key=053sdzf2rmgy7fkyqj43vwady90id0lb.narinfo error="server returned 404: 404 page not found\n"1117=== NAME TestClientMultipleUploads1118 client_integration_test.go:406: Created store path 2: /build/TestClientMultipleUploads2538324567/001/store/blq82gckjvjp75yldr1gf6ccaf6s7b3i-test-file-2.txt11192026/11/12 07:15:17 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete11202026/11/12 07:15:17 WARN Failed to register uploaded object key=yy4smgc9ah2cdaz0qchczhlkwwgvdgld.narinfo error="server returned 404: 404 page not found\n"11212026/11/12 07:15:17 INFO Received push request method=POST path=/api/pushes11222026/11/12 07:15:17 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)11232026/11/12 07:15:17 INFO Uploading i2ahpaafn0pgsb1zhg4jvy5s7rsyp2zw-a (216B)11242026/11/12 07:15:17 INFO Uploading 0l5n80j6dkg6s32x6idd4mbwjzm14l4w-shared-dep (136B)11252026/11/12 07:15:17 INFO Completed push id=1 objects=3 duration=9.134884ms11262026/11/12 07:15:17 INFO Upload complete. (62ms)1127--- PASS: TestCacheStatsHandler (0.62s)1128=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT11292026/11/12 07:15:17 INFO Completed upload id=1 objects=6 duration=7.757986ms11302026/11/12 07:15:17 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete11312026/11/12 07:15:17 INFO Completed upload id=2 objects=6 duration=1.3442ms11322026/11/12 07:15:17 INFO Upload complete. (74ms)11332026/11/12 07:15:17 WARN Failed to register uploaded object key=i2ahpaafn0pgsb1zhg4jvy5s7rsyp2zw.ls error="server returned 404: 404 page not found\n"1134=== NAME TestClientFallsBackToClosures1135 client_pushes_test.go:112: Retrieved narinfo from S3:1136 StorePath: /build/TestClientFallsBackToClosures2930340720/001/store/yy4smgc9ah2cdaz0qchczhlkwwgvdgld-shared-dep1137 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1138 Compression: zstd11392026/11/12 07:15:17 WARN Failed to register uploaded object key=nar/14hvzpgniikm8bdk6hhhr5sjc69wl9a06ss0ix0fkr04blhx67iz.nar.zst error="server returned 404: 404 page not found\n"1140 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y8211412026/11/12 07:15:17 WARN Failed to register uploaded object key=axfv7l41w9kskwvngxgba3px8lv322y6.ls error="server returned 404: 404 page not found\n"1142 NarSize: 1361143 References: 1144 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1145 client_pushes_test.go:112: Retrieved narinfo from S3:1146 StorePath: /build/TestClientFallsBackToClosures2930340720/001/store/5hxx9zblp7s69ww4ifxhwb8vlcx04ypk-a1147 URL: nar/1n8c8yn4clhdg3f82k9cl7a32rf70y7d2vs332683ix9hnkzrks3.nar.zst1148 Compression: zstd1149 NarHash: sha256:1n8c8yn4clhdg3f82k9cl7a32rf70y7d2vs332683ix9hnkzrks31150 NarSize: 2161151 References: /build/TestClientFallsBackToClosures2930340720/001/store/yy4smgc9ah2cdaz0qchczhlkwwgvdgld-shared-dep1152 CA: text:sha256:1yaqwxmfj2rgd94dzfnijnfcjnnb3frzddn8nvqw8bz0am9d45k311532026/11/12 07:15:17 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"11542026/11/12 07:15:17 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign1155 client_pushes_test.go:112: Retrieved narinfo from S3:11562026/11/12 07:15:17 WARN Failed to register uploaded object key=0l5n80j6dkg6s32x6idd4mbwjzm14l4w.ls error="server returned 404: 404 page not found\n"1157 StorePath: /build/TestClientFallsBackToClosures2930340720/001/store/053sdzf2rmgy7fkyqj43vwady90id0lb-b1158 URL: nar/1n8c8yn4clhdg3f82k9cl7a32rf70y7d2vs332683ix9hnkzrks3.nar.zst1159 Compression: zstd11602026/11/12 07:15:17 INFO Signed narinfos id=1 count=31161 NarHash: sha256:1n8c8yn4clhdg3f82k9cl7a32rf70y7d2vs332683ix9hnkzrks31162 NarSize: 2161163 References: /build/TestClientFallsBackToClosures2930340720/001/store/yy4smgc9ah2cdaz0qchczhlkwwgvdgld-shared-dep1164 CA: text:sha256:1yaqwxmfj2rgd94dzfnijnfcjnnb3frzddn8nvqw8bz0am9d45k311652026/11/12 07:15:17 INFO Uploading 3 narinfos11662026/11/12 07:15:17 WARN Failed to register uploaded object key=i2ahpaafn0pgsb1zhg4jvy5s7rsyp2zw.narinfo error="server returned 404: 404 page not found\n"11672026/11/12 07:15:17 WARN Failed to register uploaded object key=axfv7l41w9kskwvngxgba3px8lv322y6.narinfo error="server returned 404: 404 page not found\n"11682026/11/12 07:15:17 INFO Received complete push request method=POST path=/api/pushes/1/complete11692026/11/12 07:15:17 WARN Failed to register uploaded object key=0l5n80j6dkg6s32x6idd4mbwjzm14l4w.narinfo error="server returned 404: 404 page not found\n"1170--- PASS: TestClientFallsBackToClosures (0.78s)1171=== CONT TestCompleteMultipartUnregistered1172--- PASS: TestService_ReadScope_PublicByDefault (0.48s)1173=== CONT TestService_verifyS3ReuploadsStaleNarinfo1174=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1175=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1176=== RUN TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1177=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1178=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1179=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1180=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1181=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1182=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1183=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1184=== CONT TestService_verifyS3Integrity1185=== NAME TestClientWithDependencies1186 client_integration_test.go:661: Built derivation: /build/TestClientWithDependencies519759306/001/store/3cm0xv06iba0g3vw9fb5jvp963jaxvf2-test-script11872026/11/12 07:15:17 INFO Completed push id=1 objects=8 duration=10.900714ms11882026/11/12 07:15:17 INFO Upload complete. (73ms)1189=== NAME TestClientPushesUseOnePush1190 client_pushes_test.go:97: Retrieved narinfo from S3:1191 StorePath: /build/TestClientPushesUseOnePush1754866365/001/store/0l5n80j6dkg6s32x6idd4mbwjzm14l4w-shared-dep1192 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1193 Compression: zstd1194 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821195 NarSize: 1361196 References: 1197 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1198 client_pushes_test.go:97: Retrieved narinfo from S3:1199 StorePath: /build/TestClientPushesUseOnePush1754866365/001/store/i2ahpaafn0pgsb1zhg4jvy5s7rsyp2zw-a1200 URL: nar/14hvzpgniikm8bdk6hhhr5sjc69wl9a06ss0ix0fkr04blhx67iz.nar.zst1201 Compression: zstd1202 NarHash: sha256:14hvzpgniikm8bdk6hhhr5sjc69wl9a06ss0ix0fkr04blhx67iz1203 NarSize: 2161204 References: /build/TestClientPushesUseOnePush1754866365/001/store/0l5n80j6dkg6s32x6idd4mbwjzm14l4w-shared-dep1205 CA: text:sha256:0qipn48z3mldq32yxvpl7h5df049bvpqd872kzc31vzp4v2r3v9n1206 client_pushes_test.go:97: Retrieved narinfo from S3:1207 StorePath: /build/TestClientPushesUseOnePush1754866365/001/store/axfv7l41w9kskwvngxgba3px8lv322y6-b1208 URL: nar/14hvzpgniikm8bdk6hhhr5sjc69wl9a06ss0ix0fkr04blhx67iz.nar.zst1209 Compression: zstd1210 NarHash: sha256:14hvzpgniikm8bdk6hhhr5sjc69wl9a06ss0ix0fkr04blhx67iz1211 NarSize: 2161212 References: /build/TestClientPushesUseOnePush1754866365/001/store/0l5n80j6dkg6s32x6idd4mbwjzm14l4w-shared-dep1213 CA: text:sha256:0qipn48z3mldq32yxvpl7h5df049bvpqd872kzc31vzp4v2r3v9n12142026/11/12 07:15:17 INFO Received push request method=POST path=/api/pushes1215--- PASS: TestClientPushesUseOnePush (0.77s)1216--- PASS: TestService_ReadAuthMiddleware (0.45s)1217=== CONT TestService_createPendingClosureHandler1218=== CONT TestService_cleanupPendingClosuresHandler12192026/11/12 07:15:17 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)12202026/11/12 07:15:17 INFO Uploading dwz2av6khwanrvdrdfjif4impy1mazfq-shared-dep (136B)12212026/11/12 07:15:17 INFO Uploading hslfmiccr4g9j0yryfipjjn75i4v3x4j-top (224B)12222026/11/12 07:15:17 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"12232026/11/12 07:15:17 WARN Failed to register uploaded object key=hslfmiccr4g9j0yryfipjjn75i4v3x4j.ls error="server returned 404: 404 page not found\n"12242026/11/12 07:15:17 WARN Failed to register uploaded object key=dwz2av6khwanrvdrdfjif4impy1mazfq.ls error="server returned 404: 404 page not found\n"12252026/11/12 07:15:17 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12262026/11/12 07:15:17 WARN Failed to register uploaded object key=nar/0jrdjzk4i6n5qmc0vy0isj5s3b51kkg491qlzf9jfz7rfw2z4z8k.nar.zst error="server returned 404: 404 page not found\n"12272026/11/12 07:15:17 INFO Signed narinfos id=1 count=212282026/11/12 07:15:17 INFO Uploading 2 narinfos1229=== NAME TestClientWithDependencies1230 client_integration_test.go:663: Found 1 dependencies (including self)1231=== NAME TestClientCADerivations1232 client_ca_test.go:136: Built CA derivation: /build/TestClientCADerivations353710617/001/store/6g676cqk59z0zzxdlpllk6dj5p7bakjk-ca-test12332026/11/12 07:15:17 WARN Failed to register uploaded object key=dwz2av6khwanrvdrdfjif4impy1mazfq.narinfo error="server returned 404: 404 page not found\n"12342026/11/12 07:15:17 INFO Received complete push request method=POST path=/api/pushes/1/complete12352026/11/12 07:15:17 WARN Failed to register uploaded object key=hslfmiccr4g9j0yryfipjjn75i4v3x4j.narinfo error="server returned 404: 404 page not found\n"12362026/11/12 07:15:17 INFO Completed push id=1 objects=6 duration=10.430543ms12372026/11/12 07:15:17 INFO Upload complete. (72ms)1238=== NAME TestClientSharedPathCommittedMidPush1239 client_integration_test.go:740: Retrieved narinfo from S3:1240 StorePath: /build/TestClientSharedPathCommittedMidPush2981733742/001/store/dwz2av6khwanrvdrdfjif4impy1mazfq-shared-dep1241 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1242 Compression: zstd1243 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821244 NarSize: 1361245 References: 1246 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n12472026/11/12 07:15:17 INFO Received push request method=POST path=/api/pushes1248 client_integration_test.go:740: Retrieved narinfo from S3:1249 StorePath: /build/TestClientSharedPathCommittedMidPush2981733742/001/store/hslfmiccr4g9j0yryfipjjn75i4v3x4j-top1250 URL: nar/0jrdjzk4i6n5qmc0vy0isj5s3b51kkg491qlzf9jfz7rfw2z4z8k.nar.zst1251 Compression: zstd1252 NarHash: sha256:0jrdjzk4i6n5qmc0vy0isj5s3b51kkg491qlzf9jfz7rfw2z4z8k1253 NarSize: 2241254 References: /build/TestClientSharedPathCommittedMidPush2981733742/001/store/dwz2av6khwanrvdrdfjif4impy1mazfq-shared-dep1255 CA: text:sha256:1ajlh8flr4nmixsh5lk9byvsdklnar6m3a6jiw01fxnck0jy2pfq1256--- PASS: TestClientSharedPathCommittedMidPush (0.79s)1257=== CONT TestUploadHandlersRejectOversizedBody12582026/11/12 07:15:17 INFO Received push request method=POST path=/api/pushes12592026/11/12 07:15:17 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)12602026/11/12 07:15:17 INFO Uploading blq82gckjvjp75yldr1gf6ccaf6s7b3i-test-file-2.txt (160B)12612026/11/12 07:15:17 INFO Uploading h925nr3f9qyn7bd164g7y0rfa3qb49c5-test-file-1.txt (160B)12622026/11/12 07:15:17 INFO Uploading k2vnz0nsqrd8x0ywci63lcl3dnink16l-test-file-0.txt (160B)1263=== RUN TestService_RequireScope_OIDC/builder_may_write1264=== PAUSE TestService_RequireScope_OIDC/builder_may_write1265=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1266=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1267=== RUN TestService_RequireScope_OIDC/ops_may_admin1268=== PAUSE TestService_RequireScope_OIDC/ops_may_admin1269=== RUN TestService_RequireScope_OIDC/ops_may_not_write1270=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1271=== RUN TestService_RequireScope_OIDC/reader_may_not_write1272=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1273=== RUN TestService_RequireScope_OIDC/static_token_may_admin1274=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1275=== RUN TestService_RequireScope_OIDC/static_token_may_write12762026/11/12 07:15:17 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"1277=== PAUSE TestService_RequireScope_OIDC/static_token_may_write12782026/11/12 07:15:17 INFO Received push request method=POST path=/api/pushes1279=== NAME TestClientCADerivations12802026/11/12 07:15:17 WARN mTLS auth: bound subjects configured but subject DN unavailable1281 client_ca_test.go:139: Found 1 dependencies (including self)12822026/11/12 07:15:17 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1283=== RUN TestService_RequireScope_OIDC/reader_may_read12842026/11/12 07:15:17 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)1285=== PAUSE TestService_RequireScope_OIDC/reader_may_read12862026/11/12 07:15:17 INFO Uploading i0slx1zz8cdy95pq15ls7kph4kbyx8kz-unpinned-file.txt (128B)1287--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (0.53s)1288=== CONT TestUploadHandlersRejectInvalidKeys1289=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1290=== RUN TestService_RequireScope_OIDC/writer_implies_read1291=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1292=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1293=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1294=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read1295--- PASS: TestPushWaitsForSweepHoldingRow (1.41s)1296=== CONT TestIsValidUploadKey1297=== RUN TestIsValidUploadKey/narinfo1298=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal1299=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc1300=== PAUSE TestService_RequireScope_OIDC/reader_may_read_with_netrc1301=== PAUSE TestIsValidUploadKey/narinfo1302=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal1303=== RUN TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1304=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1305=== RUN TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1306=== PAUSE TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1307=== RUN TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1308=== PAUSE TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1309=== RUN TestIsValidUploadKey/nar_zst1310=== PAUSE TestIsValidUploadKey/nar_zst1311=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key1312=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key1313=== CONT TestPendingClosureWriteTimeout1314=== RUN TestPendingClosureWriteTimeout/empty1315=== PAUSE TestPendingClosureWriteTimeout/empty1316=== RUN TestPendingClosureWriteTimeout/400_objects1317=== PAUSE TestPendingClosureWriteTimeout/400_objects1318=== RUN TestPendingClosureWriteTimeout/670k_objects1319=== PAUSE TestPendingClosureWriteTimeout/670k_objects1320=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1321=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1322=== RUN TestIsValidUploadKey/nar_xz1323=== PAUSE TestIsValidUploadKey/nar_xz1324=== RUN TestIsValidUploadKey/nar_plain1325=== PAUSE TestIsValidUploadKey/nar_plain1326=== RUN TestIsValidUploadKey/listing1327=== PAUSE TestIsValidUploadKey/listing1328=== RUN TestIsValidUploadKey/build_log1329=== PAUSE TestIsValidUploadKey/build_log1330=== RUN TestIsValidUploadKey/build_log_home-manager_file1331=== PAUSE TestIsValidUploadKey/build_log_home-manager_file1332=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key1333=== CONT TestProxyWriteTimeout1334=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key1335=== RUN TestIsValidUploadKey/build_log_plus_in_name1336=== PAUSE TestIsValidUploadKey/build_log_plus_in_name1337=== RUN TestProxyWriteTimeout/narinfo1338=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle13392026/11/12 07:15:17 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)1340=== RUN TestIsValidUploadKey/build_log_question_mark1341=== PAUSE TestIsValidUploadKey/build_log_question_mark13422026/11/12 07:15:17 INFO Uploading f7pgvkcl64k7i9bf888y4f75vqnh87fq-test-file.txt (152B)1343=== RUN TestIsValidUploadKey/build_log_equals1344=== PAUSE TestIsValidUploadKey/build_log_equals1345=== PAUSE TestProxyWriteTimeout/narinfo1346=== RUN TestIsValidUploadKey/realisation1347=== PAUSE TestIsValidUploadKey/realisation1348=== RUN TestIsValidUploadKey/realisation_plus_in_output1349=== PAUSE TestIsValidUploadKey/realisation_plus_in_output1350=== RUN TestProxyWriteTimeout/1_GiB_nar1351=== RUN TestIsValidUploadKey/nix-cache-info1352=== PAUSE TestIsValidUploadKey/nix-cache-info13532026/11/12 07:15:17 WARN Failed to register uploaded object key=k2vnz0nsqrd8x0ywci63lcl3dnink16l.ls error="server returned 404: 404 page not found\n"1354=== PAUSE TestProxyWriteTimeout/1_GiB_nar13552026/11/12 07:15:17 WARN Failed to register uploaded object key=blq82gckjvjp75yldr1gf6ccaf6s7b3i.ls error="server returned 404: 404 page not found\n"1356=== RUN TestIsValidUploadKey/index.html13572026/11/12 07:15:17 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"1358=== RUN TestProxyWriteTimeout/10_GiB_nar13592026/11/12 07:15:17 WARN Failed to register uploaded object key=h925nr3f9qyn7bd164g7y0rfa3qb49c5.ls error="server returned 404: 404 page not found\n"1360=== PAUSE TestProxyWriteTimeout/10_GiB_nar13612026/11/12 07:15:17 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"1362=== PAUSE TestIsValidUploadKey/index.html1363=== RUN TestIsValidUploadKey/narinfo_key,_nar_type1364=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type1365=== RUN TestIsValidUploadKey/nar_key,_narinfo_type1366=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type1367=== RUN TestIsValidUploadKey/listing_key,_narinfo_type1368=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type1369=== RUN TestIsValidUploadKey/traversal1370=== PAUSE TestIsValidUploadKey/traversal1371=== RUN TestIsValidUploadKey/traversal_nar1372=== PAUSE TestIsValidUploadKey/traversal_nar1373=== RUN TestIsValidUploadKey/absolute1374=== PAUSE TestIsValidUploadKey/absolute1375=== RUN TestIsValidUploadKey/empty_key1376=== PAUSE TestIsValidUploadKey/empty_key1377=== RUN TestProxyWriteTimeout/unknown_size1378=== PAUSE TestProxyWriteTimeout/unknown_size1379=== RUN TestIsValidUploadKey/unknown_type13802026/11/12 07:15:17 WARN Failed to register uploaded object key=i0slx1zz8cdy95pq15ls7kph4kbyx8kz.ls error="server returned 404: 404 page not found\n"1381=== PAUSE TestIsValidUploadKey/unknown_type1382=== CONT TestSkippedUploadsHandler1383=== CONT TestParseSize1384--- PASS: TestParseSize (0.00s)1385=== CONT TestService_Rustfstest13862026/11/12 07:15:17 INFO Client skipped oversized paths paths=3 nar_bytes=500000000013872026/11/12 07:15:17 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign13882026/11/12 07:15:17 INFO Signed narinfos id=2 count=113892026/11/12 07:15:17 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"13902026/11/12 07:15:17 INFO Uploading 1 narinfos13912026/11/12 07:15:17 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13922026/11/12 07:15:17 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"13932026/11/12 07:15:17 WARN Failed to register uploaded object key=f7pgvkcl64k7i9bf888y4f75vqnh87fq.ls error="server returned 404: 404 page not found\n"13942026/11/12 07:15:17 INFO Signed narinfos id=1 count=313952026/11/12 07:15:17 INFO Uploading 3 narinfos13962026/11/12 07:15:17 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"13972026/11/12 07:15:17 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13982026/11/12 07:15:17 INFO Signed narinfos id=1 count=113992026/11/12 07:15:17 INFO Uploading 1 narinfos14002026/11/12 07:15:17 INFO Received complete push request method=POST path=/api/pushes/2/complete14012026/11/12 07:15:17 WARN Failed to register uploaded object key=i0slx1zz8cdy95pq15ls7kph4kbyx8kz.narinfo error="server returned 404: 404 page not found\n"1402=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure14032026/11/12 07:15:17 INFO Completed push id=2 objects=3 duration=61.001754ms1404=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure1405=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart1406=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart1407=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts1408=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts1409=== CONT TestPresignedUploadRegisteredBeforeCommit1410--- PASS: TestSkippedUploadsHandler (0.07s)14112026/11/12 07:15:17 INFO Upload complete. (206ms)1412=== CONT TestCompletedNarNotReofferedAcrossClosures14132026/11/12 07:15:17 WARN Failed to register uploaded object key=f7pgvkcl64k7i9bf888y4f75vqnh87fq.narinfo error="server returned 404: 404 page not found\n"14142026/11/12 07:15:17 INFO Received complete push request method=POST path=/api/pushes/1/complete14152026/11/12 07:15:17 WARN Failed to register uploaded object key=h925nr3f9qyn7bd164g7y0rfa3qb49c5.narinfo error="server returned 404: 404 page not found\n"14162026/11/12 07:15:17 INFO Received complete push request method=POST path=/api/pushes/1/complete14172026/11/12 07:15:17 WARN Failed to register uploaded object key=k2vnz0nsqrd8x0ywci63lcl3dnink16l.narinfo error="server returned 404: 404 page not found\n"14182026/11/12 07:15:17 WARN Failed to register uploaded object key=blq82gckjvjp75yldr1gf6ccaf6s7b3i.narinfo error="server returned 404: 404 page not found\n"1419=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token14202026/11/12 07:15:17 INFO Received push request method=POST path=/api/pushes1421=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token1422=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1423=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1424=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1425=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1426=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1427=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1428=== CONT TestCompleteMultipartUpload_ErrorButObjectExists14292026/11/12 07:15:17 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)14302026/11/12 07:15:17 INFO Uploading 3cm0xv06iba0g3vw9fb5jvp963jaxvf2-test-script (136B)14312026/11/12 07:15:17 INFO Completed push id=1 objects=3 duration=37.773578ms14322026/11/12 07:15:17 INFO Completed push id=1 objects=9 duration=37.526148ms14332026/11/12 07:15:17 INFO Upload complete. (239ms)14342026/11/12 07:15:17 INFO Upload complete. (252ms)1435=== NAME TestClientMultipleUploads1436 client_integration_test.go:417: Uploaded 3 paths in 286.629554ms14372026/11/12 07:15:17 WARN Failed to register uploaded object key=3cm0xv06iba0g3vw9fb5jvp963jaxvf2.ls error="server returned 404: 404 page not found\n"14382026/11/12 07:15:17 INFO Starting cleanup of old closures method=DELETE path=/api/closures14392026/11/12 07:15:17 INFO Garbage collection started14402026/11/12 07:15:17 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"14412026/11/12 07:15:17 WARN Failed to register uploaded object key=log/lxnc4jlv6j2rnmx5bax81lxqq60vbcns-test-script.drv error="server returned 404: 404 page not found\n"14422026/11/12 07:15:17 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign14432026/11/12 07:15:17 INFO Signed narinfos id=1 count=114442026/11/12 07:15:17 INFO Uploading 1 narinfos1445--- PASS: TestClientMultipleUploads (0.97s)1446=== CONT TestSharedNarBothUploadsComplete14472026/11/12 07:15:17 INFO Received complete push request method=POST path=/api/pushes/1/complete14482026/11/12 07:15:17 WARN Failed to register uploaded object key=3cm0xv06iba0g3vw9fb5jvp963jaxvf2.narinfo error="server returned 404: 404 page not found\n"1449--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (0.62s)1450=== CONT TestCommitCanBeRepeatedUntilCleanup1451=== RUN TestCommitCanBeRepeatedUntilCleanup/push1452=== PAUSE TestCommitCanBeRepeatedUntilCleanup/push1453=== RUN TestCommitCanBeRepeatedUntilCleanup/pending_closure1454=== PAUSE TestCommitCanBeRepeatedUntilCleanup/pending_closure1455=== CONT TestPush_SignsNarinfosOfItsPendingObjects14562026/11/12 07:15:17 INFO Aborted multipart uploads count=0 kept=014572026/11/12 07:15:17 INFO Completed push id=1 objects=4 duration=24.039856ms14582026/11/12 07:15:17 INFO Upload complete. (154ms)14592026/11/12 07:15:17 WARN Force mode enabled - objects will be deleted immediately without grace period14602026/11/12 07:15:17 INFO All 1 paths already cached14612026/11/12 07:15:17 INFO Received push request method=POST path=/api/pushes1462=== NAME TestClientIntegration1463 client_integration_test.go:318: Retrieved narinfo from S3:1464 StorePath: /build/TestClientIntegration223125666/002/store/f7pgvkcl64k7i9bf888y4f75vqnh87fq-test-file.txt1465 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1466 Compression: zstd1467 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11468 NarSize: 1521469 References: 1470 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11471 client_integration_test.go:319: Retrieved .ls file from S3 (compressed size: 77 bytes)1472 client_integration_test.go:319: Decompressed .ls content (64 bytes):1473 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}14742026/11/12 07:15:17 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)14752026/11/12 07:15:17 INFO Uploading 6g676cqk59z0zzxdlpllk6dj5p7bakjk-ca-test (144B)14762026/11/12 07:15:17 INFO Aborted multipart uploads count=0 kept=014772026/11/12 07:15:17 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"1478--- PASS: TestService_AuthMiddleware (0.63s)1479=== CONT TestPush_RejectsBadRequests14802026/11/12 07:15:17 WARN Failed to register uploaded object key=6g676cqk59z0zzxdlpllk6dj5p7bakjk.ls error="server returned 404: 404 page not found\n"14812026/11/12 07:15:17 WARN Failed to register uploaded object key=log/vv6hs152mm9j48c2x5861m2dn8yyfjd4-ca-test.drv error="server returned 404: 404 page not found\n"14822026/11/12 07:15:17 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"14832026/11/12 07:15:17 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign14842026/11/12 07:15:17 INFO Signed narinfos id=1 count=114852026/11/12 07:15:17 INFO Uploading 1 narinfos14862026/11/12 07:15:17 WARN Force mode enabled - objects will be deleted immediately without grace period14872026/11/12 07:15:17 INFO Received complete push request method=POST path=/api/pushes/1/complete14882026/11/12 07:15:17 WARN Failed to register uploaded object key=6g676cqk59z0zzxdlpllk6dj5p7bakjk.narinfo error="server returned 404: 404 page not found\n"14892026/11/12 07:15:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=014902026/11/12 07:15:17 INFO Vacuumed table table=pending_closures14912026/11/12 07:15:17 INFO Completed push id=1 objects=4 duration=7.981685ms14922026/11/12 07:15:17 INFO Upload complete. (107ms)1493=== NAME TestClientCADerivations1494 client_ca_test.go:180: Narinfo contains CA field: StorePath: /build/TestClientCADerivations353710617/001/store/6g676cqk59z0zzxdlpllk6dj5p7bakjk-ca-test1495 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1496 Compression: zstd1497 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1498 NarSize: 1441499 References: 1500 Deriver: /build/TestClientCADerivations353710617/001/store/vv6hs152mm9j48c2x5861m2dn8yyfjd4-ca-test.drv1501 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1502 client_ca_test.go:185: Checking for realisation files in S3...15032026/11/12 07:15:17 INFO Vacuumed table table=commit_receipts1504 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1505 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache15062026/11/12 07:15:17 INFO Vacuumed table table=pending_objects15072026/11/12 07:15:17 INFO Vacuumed table table=multipart_uploads15082026/11/12 07:15:17 INFO Vacuumed table table=closures15092026/11/12 07:15:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=01510--- PASS: TestReadProxyRangeRequest (0.47s)1511=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit15122026/11/12 07:15:17 INFO Vacuumed table table=objects1513=== NAME TestClientWithDependencies1514 client_integration_test.go:677: Skipping nix copy test - isolated store (/build/TestClientWithDependencies519759306/001/store) requires matching store prefix15152026/11/12 07:15:17 INFO Vacuumed table table=pending_closures15162026/11/12 07:15:17 INFO Vacuumed table table=commit_receipts1517--- PASS: TestClientWithDependencies (1.07s)1518=== CONT TestPush_RootsAreLockedInByteOrder15192026/11/12 07:15:17 INFO Vacuumed table table=pending_objects15202026/11/12 07:15:17 INFO Vacuumed table table=multipart_uploads15212026/11/12 07:15:17 INFO Received complete multipart upload request method=POST path=/api/multipart/complete15222026/11/12 07:15:17 INFO Vacuumed table table=closures15232026/11/12 07:15:17 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst1524--- PASS: TestCompleteMultipartUnregistered (0.38s)1525=== CONT TestPush_CompleteWaitsForCleanupOfThePush15262026/11/12 07:15:17 INFO Vacuumed table table=objects15272026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures15282026/11/12 07:15:18 INFO Received push request method=POST path=/api/pushes15292026/11/12 07:15:18 INFO Object in database but missing from S3 key=f7pgvkcl64k7i9bf888y4f75vqnh87fq.ls15302026/11/12 07:15:18 WARN Found objects in DB but missing from S3, will re-upload count=115312026/11/12 07:15:18 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)15322026/11/12 07:15:18 INFO Received complete push request method=POST path=/api/pushes/2/complete15332026/11/12 07:15:18 WARN Failed to register uploaded object key=f7pgvkcl64k7i9bf888y4f75vqnh87fq.ls error="server returned 404: 404 page not found\n"15342026/11/12 07:15:18 INFO Completed push id=2 objects=3 duration=3.716968ms15352026/11/12 07:15:18 INFO Upload complete. (67ms)1536=== NAME TestClientIntegration1537 client_integration_test.go:336: Retrieved .ls file from S3 (compressed size: 62 bytes)1538 client_integration_test.go:336: Decompressed .ls content (49 bytes):1539 {"version":1,"root":{"type":"regular","size":39}}15402026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures1541--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (0.46s)1542=== CONT TestPush_RepeatedRootCommitsOnce15432026/11/12 07:15:18 INFO Received cleanup request method=DELETE path=/api/pending_closures15442026/11/12 07:15:18 INFO Aborted multipart uploads count=0 kept=01545=== NAME TestClientCADerivations1546 client_ca_test.go:258: nix copy output: warning: you don't have Internet access; disabling some network-dependent features1547 warning: failed to create TLS context for AWS credential providers; SSO, STS WebIdentity, and ECS container authentication will be unavailable1548 error: binary cache 's3://bucket38?endpoint=http://localhost:35975®ion=eu-west-1' is for Nix stores with prefix '/nix/store', not '/build/TestClientCADerivations353710617/001/store'1549 client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 115502026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures1551--- PASS: TestClientCADerivations (1.12s)1552=== CONT TestPush_CompleteCommitsEveryRoot15532026/11/12 07:15:18 INFO Received cleanup request method=DELETE path=/api/pending_closures15542026/11/12 07:15:18 INFO Aborted multipart uploads count=1 kept=015552026/11/12 07:15:18 INFO lead: released remote=192.0.2.1:1234 generation=115562026/11/12 07:15:18 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete15572026-11-12 07:15:18.094 UTC [1512] ERROR: Closure does not exist: id=115582026-11-12 07:15:18.094 UTC [1512] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 17 at RAISE15592026-11-12 07:15:18.094 UTC [1512] STATEMENT: -- name: CommitPendingClosure :one1560 SELECT commit_pending_closure($1::bigint)::bigint1561 1562--- PASS: TestService_cleanupPendingClosuresHandler (0.46s)1563=== CONT TestPush_OverlappingRootsStoreOneRowPerKey15642026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures15652026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures15662026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures1567--- PASS: TestService_Rustfstest (0.36s)1568=== CONT TestReadRedirectUsesPublicS3URL15692026/11/12 07:15:18 INFO Received push request method=POST path=/api/pushes15702026/11/12 07:15:18 INFO lead: acquired remote=192.0.2.1:1234 generation=215712026/11/12 07:15:18 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)15722026/11/12 07:15:18 INFO Received complete push request method=POST path=/api/pushes/3/complete15732026/11/12 07:15:18 WARN Failed to register uploaded object key=f7pgvkcl64k7i9bf888y4f75vqnh87fq.ls error="server returned 404: 404 page not found\n"15742026/11/12 07:15:18 INFO Completed push id=3 objects=3 duration=3.378038ms15752026/11/12 07:15:18 INFO Upload complete. (52ms)1576=== NAME TestClientIntegration1577 client_integration_test.go:353: Retrieved .ls file from S3 (compressed size: 62 bytes)1578 client_integration_test.go:353: Decompressed .ls content (49 bytes):1579 {"version":1,"root":{"type":"regular","size":39}}1580 client_integration_test.go:364: Testing garbage collection...15812026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures15822026/11/12 07:15:18 INFO lead: given back to the incumbent remote=192.0.2.1:1234 generation=215832026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures15842026/11/12 07:15:18 INFO Starting cleanup of old closures method=DELETE path=/api/closures15852026/11/12 07:15:18 INFO Garbage collection started15862026/11/12 07:15:18 INFO lead: acquired remote=192.0.2.1:1234 generation=315872026/11/12 07:15:18 INFO Aborted multipart uploads count=0 kept=015882026/11/12 07:15:18 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst15892026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures15902026/11/12 07:15:18 WARN Force mode enabled - objects will be deleted immediately without grace period1591--- PASS: TestPresignedUploadRegisteredBeforeCommit (0.37s)1592=== CONT TestParseSingleRange1593=== RUN TestParseSingleRange/none1594=== PAUSE TestParseSingleRange/none1595=== RUN TestParseSingleRange/unknown_unit1596=== PAUSE TestParseSingleRange/unknown_unit1597=== RUN TestParseSingleRange/multi-range_ignored1598=== PAUSE TestParseSingleRange/multi-range_ignored1599=== RUN TestParseSingleRange/malformed_no_dash1600=== PAUSE TestParseSingleRange/malformed_no_dash1601=== RUN TestParseSingleRange/malformed_both_empty1602=== PAUSE TestParseSingleRange/malformed_both_empty1603=== RUN TestParseSingleRange/malformed_end_before_start1604=== PAUSE TestParseSingleRange/malformed_end_before_start1605=== RUN TestParseSingleRange/closed1606=== PAUSE TestParseSingleRange/closed1607=== RUN TestParseSingleRange/open-ended1608=== PAUSE TestParseSingleRange/open-ended1609=== RUN TestParseSingleRange/end_clamped_to_size1610=== PAUSE TestParseSingleRange/end_clamped_to_size1611=== RUN TestParseSingleRange/suffix1612=== PAUSE TestParseSingleRange/suffix1613=== RUN TestParseSingleRange/suffix_exceeds_size1614=== PAUSE TestParseSingleRange/suffix_exceeds_size1615=== RUN TestParseSingleRange/single_byte1616=== PAUSE TestParseSingleRange/single_byte1617=== RUN TestParseSingleRange/start_past_EOF1618=== PAUSE TestParseSingleRange/start_past_EOF1619=== RUN TestParseSingleRange/start_far_past_EOF1620=== PAUSE TestParseSingleRange/start_far_past_EOF1621=== CONT TestReadRedirectKeepsNarinfoProxied16222026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures16232026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures16242026/11/12 07:15:18 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=016252026/11/12 07:15:18 INFO Vacuumed table table=pending_closures16262026/11/12 07:15:18 INFO Received complete multipart upload request method=POST path=/api/multipart/complete16272026/11/12 07:15:18 INFO Vacuumed table table=commit_receipts16282026/11/12 07:15:18 INFO Vacuumed table table=pending_objects16292026/11/12 07:15:18 INFO Vacuumed table table=multipart_uploads16302026/11/12 07:15:18 INFO Vacuumed table table=closures16312026/11/12 07:15:18 INFO Vacuumed table table=objects16322026/11/12 07:15:18 INFO Received push request method=POST path=/api/pushes16332026/11/12 07:15:18 INFO Received complete multipart upload request method=POST path=/api/multipart/complete16342026/11/12 07:15:18 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16352026/11/12 07:15:18 INFO Signed narinfos id=1 count=116362026/11/12 07:15:18 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16372026/11/12 07:15:18 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16382026/11/12 07:15:18 ERROR Failed to sign narinfos signer="index 1" error="signing key cannot be nil"16392026/11/12 07:15:18 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16402026/11/12 07:15:18 ERROR Failed to sign narinfos signer=niks3-test-1 error="signer failed"16412026/11/12 07:15:18 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16422026/11/12 07:15:18 ERROR Signer returned unexpected signature count signer=niks3-test-1 got=0 want=116432026/11/12 07:15:18 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16442026/11/12 07:15:18 ERROR Signer returned unexpected object key signer=niks3-test-1 object_key=unexpected.narinfo1645--- PASS: TestPush_SignsNarinfosOfItsPendingObjects (0.40s)1646=== CONT TestReadRedirectNar16472026/11/12 07:15:18 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MTU4NWZhYmYtZDFkNC00MmVhLThhNDMtZjA1OTQ3YzFhY2YxLmRkZmVmNGNiLTAxMDItNDAzNy05MGRiLTVjNGZiNTlkMGI1N3gxNzk0NDY3NzE4MjUzNTgyNTY216482026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures16492026/11/12 07:15:18 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MTU4NWZhYmYtZDFkNC00MmVhLThhNDMtZjA1OTQ3YzFhY2YxLmRkZmVmNGNiLTAxMDItNDAzNy05MGRiLTVjNGZiNTlkMGI1N3gxNzk0NDY3NzE4MjUzNTgyNTY2 parts=11650--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (0.47s)1651=== CONT TestReadProxyDisabled16522026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures16532026/11/12 07:15:18 INFO lead: released remote=192.0.2.1:1234 generation=11654--- PASS: TestLeadEndsOnShutdown (1.98s)1655=== CONT TestReadProxyRootRedirectsToIndexHTML1656=== RUN TestPush_RejectsBadRequests/root_not_in_objects1657=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects1658=== RUN TestPush_RejectsBadRequests/no_roots1659=== PAUSE TestPush_RejectsBadRequests/no_roots1660=== RUN TestPush_RejectsBadRequests/no_objects1661=== PAUSE TestPush_RejectsBadRequests/no_objects1662=== RUN TestPush_RejectsBadRequests/bad_root1663=== PAUSE TestPush_RejectsBadRequests/bad_root1664=== CONT TestReadProxyConditionalGet16652026/11/12 07:15:18 INFO Received push request method=POST path=/api/pushes16662026/11/12 07:15:18 INFO Received complete push request method=POST path=/api/pushes/1/complete16672026/11/12 07:15:18 INFO lead: released remote=192.0.2.1:1234 generation=116682026/11/12 07:15:18 INFO Received push request method=POST path=/api/pushes16692026/11/12 07:15:18 INFO lead: acquired remote=192.0.2.1:1234 generation=216702026/11/12 07:15:18 INFO Completed push id=1 objects=4 duration=8.279385ms16712026/11/12 07:15:18 INFO Received push request method=POST path=/api/pushes16722026/11/12 07:15:18 INFO Received push request method=POST path=/api/pushes16732026-11-12 07:15:18.450 UTC [2010] ERROR: could not obtain lock on row in relation "closures"16742026-11-12 07:15:18.450 UTC [2010] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT16752026-11-12 07:15:18.450 UTC [2010] ERROR: could not obtain lock on row in relation "closures"16762026-11-12 07:15:18.450 UTC [2010] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT16772026-11-12 07:15:18.450 UTC [2010] ERROR: could not obtain lock on row in relation "closures"16782026-11-12 07:15:18.450 UTC [2010] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT16792026-11-12 07:15:18.451 UTC [2010] ERROR: could not obtain lock on row in relation "closures"16802026-11-12 07:15:18.451 UTC [2010] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT16812026-11-12 07:15:18.451 UTC [2010] ERROR: could not obtain lock on row in relation "closures"16822026-11-12 07:15:18.451 UTC [2010] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT1683--- PASS: TestPush_RootsAreLockedInByteOrder (0.48s)1684=== CONT TestReadProxyOutlastsServerWriteTimeout16852026/11/12 07:15:18 INFO Received push request method=POST path=/api/pushes16862026/11/12 07:15:18 INFO Aborted multipart uploads count=0 kept=016872026/11/12 07:15:18 WARN Force mode enabled - objects will be deleted immediately without grace period1688--- PASS: TestDeleteClosuresRacingPins (1.73s)1689=== CONT TestReadProxyHead16902026/11/12 07:15:18 INFO Received complete push request method=POST path=/api/pushes/1/complete16912026/11/12 07:15:18 INFO Received complete multipart upload request method=POST path=/api/multipart/complete16922026/11/12 07:15:18 INFO Completed push id=1 objects=4 duration=6.368656ms1693--- PASS: TestPush_RepeatedRootCommitsOnce (0.43s)1694=== CONT TestReadProxyInvalidPath16952026/11/12 07:15:18 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=016962026/11/12 07:15:18 INFO Vacuumed table table=pending_closures16972026/11/12 07:15:18 INFO Received push request method=POST path=/api/pushes16982026/11/12 07:15:18 INFO Vacuumed table table=commit_receipts16992026/11/12 07:15:18 INFO Vacuumed table table=pending_objects17002026/11/12 07:15:18 INFO Vacuumed table table=multipart_uploads17012026/11/12 07:15:18 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=MTU4NWZhYmYtZDFkNC00MmVhLThhNDMtZjA1OTQ3YzFhY2YxLmMzMThlNWUzLTkxZWEtNDBkYS1hYmRjLTg1NDk2N2M4MzBiY3gxNzk0NDY3NzE4MDI4NzgyMDQ2 parts=1017022026/11/12 07:15:18 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete17032026/11/12 07:15:18 INFO Vacuumed table table=closures17042026/11/12 07:15:18 INFO Vacuumed table table=objects17052026/11/12 07:15:18 INFO Received complete push request method=POST path=/api/pushes/2/complete17062026/11/12 07:15:18 INFO Received complete push request method=POST path=/api/pushes/1/complete17072026/11/12 07:15:18 INFO Received push request method=POST path=/api/pushes17082026/11/12 07:15:18 INFO Completed upload id=1 objects=2 duration=5.588687ms17092026/11/12 07:15:18 INFO Completed push id=2 objects=4 duration=2.943378ms1710--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (0.56s)1711=== CONT TestReadProxy40417122026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures17132026/11/12 07:15:18 INFO Completed push id=1 objects=6 duration=8.181435ms17142026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures1715--- PASS: TestPush_CompleteCommitsEveryRoot (0.44s)1716=== CONT TestReadProxyNarStreaming17172026/11/12 07:15:18 INFO lead: released remote=192.0.2.1:1234 generation=117182026/11/12 07:15:18 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo17192026/11/12 07:15:18 WARN Found objects in DB but missing from S3, will re-upload count=11720--- PASS: TestService_verifyS3Integrity (0.93s)1721=== CONT TestReadProxyContentNegotiation17222026/11/12 07:15:18 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1723--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (0.45s)1724=== CONT TestReadProxyNarinfoAlreadyDecompressed17252026/11/12 07:15:18 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=MTU4NWZhYmYtZDFkNC00MmVhLThhNDMtZjA1OTQ3YzFhY2YxLjRjNzI5YzJhLTgwMTctNGI1Mi1iM2VlLTJkZTVhZmIxOWY2ZXgxNzk0NDY3NzE4MTA0MTk5Mzgy parts=1017262026/11/12 07:15:18 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete1727--- PASS: TestReadRedirectUsesPublicS3URL (0.44s)1728=== CONT TestReadProxyNarinfo17292026/11/12 07:15:18 INFO Completed upload id=1 objects=2 duration=3.645058ms17302026/11/12 07:15:18 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000017312026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures17322026/11/12 07:15:18 INFO Starting cleanup of old closures method=DELETE path=/api/closures17332026/11/12 07:15:18 INFO Aborted multipart uploads count=0 kept=01734--- PASS: TestReadRedirectKeepsNarinfoProxied (0.38s)1735=== CONT TestIsValidCachePath1736=== RUN TestIsValidCachePath/narinfo1737=== PAUSE TestIsValidCachePath/narinfo1738=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars17392026/11/12 07:15:18 INFO lead: acquired remote=192.0.2.1:1234 generation=21740=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars1741=== RUN TestIsValidCachePath/nar_zst1742=== PAUSE TestIsValidCachePath/nar_zst1743=== RUN TestIsValidCachePath/nar_xz1744=== PAUSE TestIsValidCachePath/nar_xz1745=== RUN TestIsValidCachePath/nar_bz21746=== PAUSE TestIsValidCachePath/nar_bz21747=== RUN TestIsValidCachePath/nar_uncompressed1748=== PAUSE TestIsValidCachePath/nar_uncompressed1749=== RUN TestIsValidCachePath/ls1750=== PAUSE TestIsValidCachePath/ls1751=== RUN TestIsValidCachePath/log1752=== PAUSE TestIsValidCachePath/log1753=== RUN TestIsValidCachePath/realisation1754=== PAUSE TestIsValidCachePath/realisation1755=== RUN TestIsValidCachePath/nix-cache-info1756=== PAUSE TestIsValidCachePath/nix-cache-info1757=== RUN TestIsValidCachePath/index.html1758=== PAUSE TestIsValidCachePath/index.html1759=== RUN TestIsValidCachePath/traversal_parent1760=== PAUSE TestIsValidCachePath/traversal_parent1761=== RUN TestIsValidCachePath/traversal_in_middle1762=== PAUSE TestIsValidCachePath/traversal_in_middle1763=== RUN TestIsValidCachePath/invalid_char_e1764=== PAUSE TestIsValidCachePath/invalid_char_e1765=== RUN TestIsValidCachePath/invalid_char_u1766=== PAUSE TestIsValidCachePath/invalid_char_u1767=== RUN TestIsValidCachePath/random_path1768=== PAUSE TestIsValidCachePath/random_path1769=== RUN TestIsValidCachePath/empty1770=== PAUSE TestIsValidCachePath/empty1771=== RUN TestIsValidCachePath/leading_slash1772=== PAUSE TestIsValidCachePath/leading_slash1773=== RUN TestIsValidCachePath/wrong_extension1774=== PAUSE TestIsValidCachePath/wrong_extension1775=== RUN TestIsValidCachePath/short_hash1776=== PAUSE TestIsValidCachePath/short_hash1777=== CONT TestServerTLSConfig1778=== RUN TestServerTLSConfig/no_client_CA1779=== PAUSE TestServerTLSConfig/no_client_CA1780=== RUN TestServerTLSConfig/missing_CA_file1781=== PAUSE TestServerTLSConfig/missing_CA_file1782=== RUN TestServerTLSConfig/not_a_PEM_file1783=== PAUSE TestServerTLSConfig/not_a_PEM_file1784=== CONT TestPendingCleanupUsesOneCutoff17852026/11/12 07:15:18 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=01786--- PASS: TestReadRedirectNar (0.32s)1787=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded17882026/11/12 07:15:18 INFO Vacuumed table table=pending_closures17892026/11/12 07:15:18 INFO Vacuumed table table=commit_receipts17902026/11/12 07:15:18 INFO Vacuumed table table=pending_objects17912026/11/12 07:15:18 INFO Vacuumed table table=multipart_uploads1792--- PASS: TestReadProxyDisabled (0.32s)17932026/11/12 07:15:18 INFO Vacuumed table table=closures1794=== CONT TestMultipartCleanup17952026/11/12 07:15:18 INFO Vacuumed table table=objects17962026/11/12 07:15:18 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001797--- PASS: TestService_createPendingClosureHandler (1.04s)1798=== CONT TestMetricsCommit1799--- PASS: TestReadProxyRootRedirectsToIndexHTML (0.36s)1800=== CONT TestService_NativeMTLS1801--- PASS: TestReadProxyConditionalGet (0.38s)1802=== CONT TestMetricsInventory18032026/11/12 07:15:18 INFO Received complete multipart upload request method=POST path=/api/multipart/complete18042026-11-12 07:15:18.758 UTC [2012] ERROR: Push does not exist: id=118052026-11-12 07:15:18.758 UTC [2012] CONTEXT: PL/pgSQL function commit_push(bigint) line 15 at RAISE18062026-11-12 07:15:18.758 UTC [2012] STATEMENT: SELECT commit_push($1)1807--- PASS: TestPush_CompleteWaitsForCleanupOfThePush (0.77s)1808=== CONT TestCreatePin_ReservedPins18092026/11/12 07:15:18 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=MTU4NWZhYmYtZDFkNC00MmVhLThhNDMtZjA1OTQ3YzFhY2YxLjRkNmQ0M2UwLTIwNmItNDcxYS05ZmE4LTgyYzY4NGZiODIzMngxNzk0NDY3NzE4MjIxNzIyMjU0 parts=1218102026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures1811--- PASS: TestCompletedNarNotReofferedAcrossClosures (0.94s)1812=== CONT TestPresentNotReportedWhileGCDeletesClosure1813=== RUN TestPresentNotReportedWhileGCDeletesClosure/stale1814=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/stale1815=== RUN TestPresentNotReportedWhileGCDeletesClosure/fresh1816=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/fresh1817=== CONT TestPresentRefreshesOnlyStaleClosures1818--- PASS: TestReadProxyHead (0.30s)1819=== CONT TestPresentReportsOnlyClosureRoots1820--- PASS: TestReadProxyInvalidPath (0.30s)1821=== CONT TestOrphanedObjectsGC18222026/11/12 07:15:18 INFO Received complete multipart upload request method=POST path=/api/multipart/complete18232026/11/12 07:15:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:43591/oidc18242026/11/12 07:15:18 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=MTU4NWZhYmYtZDFkNC00MmVhLThhNDMtZjA1OTQ3YzFhY2YxLjI5ZTMxNDgwLTQ5MzYtNDgwMy04NjkxLTUxMzBmNmI4YTIxMXgxNzk0NDY3NzE4MzE1ODczNTAw parts=121825--- PASS: TestReadProxyNarStreaming (0.33s)1826=== CONT TestOrphanedObjectsGCStressTest1827--- PASS: TestReadProxyContentNegotiation (0.34s)1828=== CONT TestObjectStatsUpdatedOncePerCommit1829--- PASS: TestReadProxyNarinfoAlreadyDecompressed (0.37s)1830=== CONT TestObjectStatsTrigger18312026/11/12 07:15:18 ERROR Failed to decode narinfo key=3hcdxyjf9yiq7qf3i4548drb6sjmwa1v.narinfo limit=16777216 error=<nil>1832--- PASS: TestReadProxyNarinfo (0.39s)1833=== CONT TestGCEndsOnShutdown/before_the_run18342026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures18352026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures18362026/11/12 07:15:18 INFO Received cleanup request method=DELETE path=/api/pending_closures18372026/11/12 07:15:18 INFO Received uploads request method=POST path=/api/pending_closures1838--- PASS: TestReadProxy404 (0.48s)1839=== CONT TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1840--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (0.41s)1841=== CONT TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark18422026/11/12 07:15:19 INFO Received uploads request method=POST path=/api/pending_closures18432026/11/12 07:15:19 INFO Received push request method=POST path=/api/pushes18442026/11/12 07:15:19 INFO Received complete push request method=POST path=/api/pushes/1/complete18452026/11/12 07:15:19 INFO Completed push id=1 objects=4 duration=8.034796ms18462026/11/12 07:15:19 WARN mTLS auth: subject not in bound subjects subject="CN=reader"18472026/11/12 07:15:19 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1848--- PASS: TestService_NativeMTLS (0.40s)1849=== CONT TestForceGCDuringPushOffersSweptObject/before_pending_rows18502026/11/12 07:15:19 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete18512026/11/12 07:15:19 INFO Completed upload id=2 objects=2 duration=1.288929ms1852--- PASS: TestMetricsCommit (0.42s)1853=== CONT TestForceGCDuringPushOffersSweptObject/after_presence_check1854--- PASS: TestMetricsInventory (0.42s)1855=== CONT TestResolveDBConnectionString/flag_wins1856=== CONT TestResolveDBConnectionString/PGHOST_allows_empty1857=== CONT TestResolveDBConnectionString/nothing_configured1858=== CONT TestResolveDBConnectionString/missing_file_is_an_error1859=== CONT TestResolveDBConnectionString/file_when_flag_empty1860=== CONT TestClientErrorHandling/InvalidStorePath1861--- PASS: TestResolveDBConnectionString (0.00s)1862 --- PASS: TestResolveDBConnectionString/flag_wins (0.00s)1863 --- PASS: TestResolveDBConnectionString/PGHOST_allows_empty (0.00s)1864 --- PASS: TestResolveDBConnectionString/nothing_configured (0.00s)1865 --- PASS: TestResolveDBConnectionString/missing_file_is_an_error (0.00s)1866 --- PASS: TestResolveDBConnectionString/file_when_flag_empty (0.00s)18672026/11/12 07:15:19 INFO Received cleanup request method=DELETE path=/api/pending_closures18682026/11/12 07:15:19 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket83/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""18692026/11/12 07:15:19 INFO Aborted multipart uploads count=0 kept=11870--- PASS: TestPresentReportsOnlyClosureRoots (0.38s)1871=== CONT TestClientErrorHandling/ServerNotAvailable18722026/11/12 07:15:19 INFO Received cleanup request method=DELETE path=/api/pending_closures18732026/11/12 07:15:19 INFO Aborted multipart uploads count=1 kept=01874--- PASS: TestMultipartCleanup (0.55s)1875=== CONT TestClientErrorHandling/InvalidAuthToken1876--- PASS: TestPresentRefreshesOnlyStaleClosures (0.42s)1877=== CONT TestCacheConfigHandler/full_config,_no_issuer1878=== CONT TestCacheConfigHandler/no_signing_keys1879=== CONT TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1880=== CONT TestCacheConfigHandler/no_cache_url_configured1881--- PASS: TestCacheConfigHandler (0.00s)1882 --- PASS: TestCacheConfigHandler/full_config,_no_issuer (0.00s)1883 --- PASS: TestCacheConfigHandler/no_signing_keys (0.00s)1884 --- PASS: TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator (0.00s)1885 --- PASS: TestCacheConfigHandler/no_cache_url_configured (0.00s)1886=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed18872026/11/12 07:15:19 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/objects/present18882026/11/12 07:15:19 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux18892026/11/12 07:15:19 WARN Refused reserved pin name=worker-x86_64-linux18902026/11/12 07:15:19 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux18912026/11/12 07:15:19 INFO Received create pin request method=POST path=/api/pins/my-app18922026/11/12 07:15:19 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux1893--- PASS: TestCreatePin_ReservedPins (0.48s)1894=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed18952026/11/12 07:15:19 WARN Request failed, retrying attempt=2 backoff=216.842684ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/objects/present1896--- PASS: TestObjectStatsTrigger (0.43s)1897=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1898--- PASS: TestObjectStatsUpdatedOncePerCommit (0.48s)1899=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost19002026/11/12 07:15:19 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1901=== CONT TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed19022026/11/12 07:15:19 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=MTU4NWZhYmYtZDFkNC00MmVhLThhNDMtZjA1OTQ3YzFhY2YxLmIyNmViNGQyLTQxZDUtNDk5NS05ZTViLWQ3NTBlYTczNWU5OHgxNzk0NDY3NzE4MzI1NzI5MTI0 parts=121903--- PASS: TestSharedNarBothUploadsComplete (1.51s)1904=== CONT TestService_RequireScope_OIDC/builder_may_write1905=== CONT TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read19062026/11/12 07:15:19 WARN Authentication failed token_preview=wrong token_length=5 oidc_error="not a JWT" oidc_provider=""1907=== CONT TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1908=== CONT TestService_RequireScope_OIDC/reader_may_read_with_netrc1909=== CONT TestService_RequireScope_OIDC/anonymous_may_not_read1910=== CONT TestService_RequireScope_OIDC/writer_implies_read1911=== CONT TestService_RequireScope_OIDC/ops_may_not_write1912=== CONT TestService_RequireScope_OIDC/ops_may_admin1913=== CONT TestService_RequireScope_OIDC/reader_may_not_write1914=== CONT TestService_RequireScope_OIDC/builder_may_not_admin1915=== CONT TestService_RequireScope_OIDC/reader_may_read1916=== CONT TestService_RequireScope_OIDC/static_token_may_write1917=== CONT TestService_RequireScope_OIDC/static_token_may_admin1918=== CONT TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1919=== CONT TestPendingClosureWriteTimeout/empty1920=== CONT TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1921--- PASS: TestService_RequireScope_OIDC (0.62s)1922 --- PASS: TestService_RequireScope_OIDC/builder_may_write (0.00s)1923 --- PASS: TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read (0.00s)1924 --- PASS: TestService_RequireScope_OIDC/static_token_may_admin_with_netrc (0.00s)1925 --- PASS: TestService_RequireScope_OIDC/reader_may_read_with_netrc (0.00s)1926 --- PASS: TestService_RequireScope_OIDC/anonymous_may_not_read (0.00s)1927 --- PASS: TestService_RequireScope_OIDC/writer_implies_read (0.00s)1928 --- PASS: TestService_RequireScope_OIDC/ops_may_not_write (0.00s)1929 --- PASS: TestService_RequireScope_OIDC/ops_may_admin (0.00s)1930 --- PASS: TestService_RequireScope_OIDC/reader_may_not_write (0.00s)1931 --- PASS: TestService_RequireScope_OIDC/builder_may_not_admin (0.00s)1932 --- PASS: TestService_RequireScope_OIDC/reader_may_read (0.00s)1933 --- PASS: TestService_RequireScope_OIDC/static_token_may_write (0.00s)1934 --- PASS: TestService_RequireScope_OIDC/static_token_may_admin (0.00s)1935 --- PASS: TestService_RequireScope_OIDC/empty_netrc_password_may_not_read (0.00s)19362026/11/12 07:15:19 INFO Aborted multipart uploads count=0 kept=019372026/11/12 07:15:19 WARN Force mode enabled - objects will be deleted immediately without grace period19382026/11/12 07:15:19 ERROR failed to remove object object=nar/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.nar.zst error="Post \"http://localhost:35975/bucket95/?delete=\": context canceled"1939=== CONT TestPendingClosureWriteTimeout/670k_objects1940=== CONT TestPendingClosureWriteTimeout/400_objects1941=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info19422026/11/12 07:15:19 INFO Received uploads request method=POST path=/1943=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key19442026/11/12 07:15:19 INFO Received request for more parts method=POST path=/1945=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key19462026/11/12 07:15:19 INFO Received complete multipart upload request method=POST path=/1947=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal19482026/11/12 07:15:19 INFO Received uploads request method=POST path=/1949--- PASS: TestUploadHandlersRejectInvalidKeys (0.00s)1950 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)1951 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1952 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1953 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s)1954=== CONT TestProxyWriteTimeout/narinfo1955=== CONT TestProxyWriteTimeout/10_GiB_nar1956=== CONT TestProxyWriteTimeout/1_GiB_nar1957=== CONT TestProxyWriteTimeout/unknown_size1958--- PASS: TestProxyWriteTimeout (0.00s)1959 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1960 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1961 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)1962 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1963=== CONT TestIsValidUploadKey/narinfo1964=== CONT TestIsValidUploadKey/unknown_type1965=== CONT TestIsValidUploadKey/empty_key1966=== CONT TestIsValidUploadKey/absolute1967=== CONT TestIsValidUploadKey/traversal_nar1968=== CONT TestIsValidUploadKey/traversal1969=== CONT TestIsValidUploadKey/listing_key,_narinfo_type1970=== CONT TestIsValidUploadKey/nar_key,_narinfo_type1971=== CONT TestIsValidUploadKey/narinfo_key,_nar_type1972=== CONT TestIsValidUploadKey/index.html1973=== CONT TestIsValidUploadKey/nix-cache-info1974=== CONT TestIsValidUploadKey/realisation_plus_in_output1975=== CONT TestIsValidUploadKey/build_log1976=== CONT TestIsValidUploadKey/listing1977=== CONT TestIsValidUploadKey/nar_plain1978=== CONT TestIsValidUploadKey/nar_xz1979=== CONT TestIsValidUploadKey/nar_zst1980=== CONT TestIsValidUploadKey/build_log_home-manager_file1981=== CONT TestIsValidUploadKey/build_log_equals1982=== CONT TestIsValidUploadKey/realisation1983=== CONT TestIsValidUploadKey/build_log_question_mark1984=== CONT TestIsValidUploadKey/build_log_plus_in_name1985--- PASS: TestIsValidUploadKey (0.01s)1986 --- PASS: TestIsValidUploadKey/narinfo (0.00s)1987 --- PASS: TestIsValidUploadKey/unknown_type (0.00s)1988 --- PASS: TestIsValidUploadKey/empty_key (0.00s)1989 --- PASS: TestIsValidUploadKey/absolute (0.00s)1990 --- PASS: TestIsValidUploadKey/traversal_nar (0.00s)1991 --- PASS: TestIsValidUploadKey/traversal (0.00s)1992 --- PASS: TestIsValidUploadKey/listing_key,_narinfo_type (0.00s)1993 --- PASS: TestIsValidUploadKey/nar_key,_narinfo_type (0.00s)1994 --- PASS: TestIsValidUploadKey/narinfo_key,_nar_type (0.00s)1995 --- PASS: TestIsValidUploadKey/index.html (0.00s)1996 --- PASS: TestIsValidUploadKey/nix-cache-info (0.00s)1997 --- PASS: TestIsValidUploadKey/realisation_plus_in_output (0.00s)1998 --- PASS: TestIsValidUploadKey/build_log (0.00s)1999 --- PASS: TestIsValidUploadKey/listing (0.00s)2000 --- PASS: TestIsValidUploadKey/nar_plain (0.00s)
loading 863 hidden lines…
28642026/11/12 07:15:32 INFO Vacuumed table table=pending_closures28652026/11/12 07:15:32 INFO Vacuumed table table=commit_receipts28662026/11/12 07:15:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=028672026/11/12 07:15:32 INFO Vacuumed table table=pending_objects28682026/11/12 07:15:32 INFO Completed upload id=10 objects=4 duration=3.397498ms28692026/11/12 07:15:32 INFO Vacuumed table table=multipart_uploads28702026/11/12 07:15:32 INFO Vacuumed table table=closures28712026/11/12 07:15:32 INFO Vacuumed table table=pending_closures28722026/11/12 07:15:32 INFO Vacuumed table table=objects28732026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts28742026/11/12 07:15:33 INFO Vacuumed table table=pending_objects28752026/11/12 07:15:33 INFO Received complete push request method=POST path=/28762026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads28772026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=028782026/11/12 07:15:33 INFO Vacuumed table table=closures28792026/11/12 07:15:33 INFO Vacuumed table table=objects28802026/11/12 07:15:33 INFO Received push request method=POST path=/28812026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=028822026/11/12 07:15:33 INFO Received push request method=POST path=/28832026/11/12 07:15:33 INFO Completed push id=6 objects=5 duration=4.030597ms28842026/11/12 07:15:33 INFO Received push request method=POST path=/28852026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028862026/11/12 07:15:33 INFO Vacuumed table table=pending_closures28872026/11/12 07:15:33 INFO Received push request method=POST path=/28882026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts28892026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=028902026/11/12 07:15:33 INFO Vacuumed table table=pending_objects28912026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads28922026/11/12 07:15:33 INFO Vacuumed table table=closures28932026/11/12 07:15:33 INFO Vacuumed table table=objects28942026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period28952026/11/12 07:15:33 INFO Received complete upload request method=POST path=/28962026/11/12 07:15:33 INFO Received push request method=POST path=/28972026/11/12 07:15:33 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."28982026/11/12 07:15:33 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."28992026/11/12 07:15:33 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."29002026/11/12 07:15:33 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."29012026/11/12 07:15:33 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."29022026/11/12 07:15:33 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."29032026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."29042026/11/12 07:15:33 INFO Completed upload id=10 objects=6 duration=2.923478ms29052026/11/12 07:15:33 INFO Received complete upload request method=POST path=/29062026/11/12 07:15:33 INFO Received push request method=POST path=/29072026/11/12 07:15:33 INFO Received complete push request method=POST path=/29082026/11/12 07:15:33 INFO Received push request method=POST path=/29092026/11/12 07:15:33 INFO Received push request method=POST path=/29102026/11/12 07:15:33 INFO Completed upload id=9 objects=7 duration=2.814148ms29112026/11/12 07:15:33 INFO Completed push id=5 objects=3 duration=3.494398ms29122026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=029132026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=029142026/11/12 07:15:33 INFO Vacuumed table table=pending_closures29152026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts29162026/11/12 07:15:33 INFO Received push request method=POST path=/29172026/11/12 07:15:33 INFO Vacuumed table table=pending_objects29182026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads29192026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=029202026/11/12 07:15:33 INFO Vacuumed table table=closures29212026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period29222026/11/12 07:15:33 INFO Vacuumed table table=objects29232026/11/12 07:15:33 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."29242026/11/12 07:15:33 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."29252026/11/12 07:15:33 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."29262026/11/12 07:15:33 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."29272026/11/12 07:15:33 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."29282026/11/12 07:15:33 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."29292026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029302026/11/12 07:15:33 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."29312026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."29322026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."29332026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."29342026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."29352026/11/12 07:15:33 INFO Received push request method=POST path=/29362026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=029372026/11/12 07:15:33 INFO Vacuumed table table=pending_closures29382026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts29392026/11/12 07:15:33 INFO Received push request method=POST path=/29402026/11/12 07:15:33 INFO Vacuumed table table=pending_objects29412026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads29422026/11/12 07:15:33 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."29432026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."29442026/11/12 07:15:33 INFO Vacuumed table table=closures29452026/11/12 07:15:33 INFO Received push request method=POST path=/29462026/11/12 07:15:33 INFO Vacuumed table table=objects29472026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029482026/11/12 07:15:33 INFO Vacuumed table table=pending_closures29492026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts29502026/11/12 07:15:33 INFO Vacuumed table table=pending_objects29512026/11/12 07:15:33 INFO Received complete push request method=POST path=/29522026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads29532026/11/12 07:15:33 INFO Vacuumed table table=closures29542026/11/12 07:15:33 INFO Completed push id=8 objects=7 duration=2.150989ms29552026/11/12 07:15:33 INFO Vacuumed table table=objects29562026/11/12 07:15:33 INFO Received push request method=POST path=/29572026/11/12 07:15:33 INFO Received push request method=POST path=/29582026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=029592026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029602026/11/12 07:15:33 INFO Received push request method=POST path=/29612026/11/12 07:15:33 INFO Vacuumed table table=pending_closures29622026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts29632026/11/12 07:15:33 INFO Received uploads request method=POST path=/29642026/11/12 07:15:33 INFO Vacuumed table table=pending_objects29652026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads29662026/11/12 07:15:33 INFO Vacuumed table table=closures29672026/11/12 07:15:33 INFO Vacuumed table table=objects29682026/11/12 07:15:33 INFO Received uploads request method=POST path=/29692026/11/12 07:15:33 INFO Received push request method=POST path=/29702026/11/12 07:15:33 INFO Received push request method=POST path=/29712026/11/12 07:15:33 INFO Received uploads request method=POST path=/29722026/11/12 07:15:33 INFO Received complete push request method=POST path=/29732026/11/12 07:15:33 INFO Received push request method=POST path=/29742026/11/12 07:15:33 INFO Completed push id=8 objects=2 duration=1.963739ms29752026/11/12 07:15:33 INFO Received uploads request method=POST path=/29762026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=029772026/11/12 07:15:33 INFO Received uploads request method=POST path=/29782026/11/12 07:15:33 INFO Received push request method=POST path=/29792026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=4 objects-deleted-after-grace-period=0 objects-failed-to-delete=029802026/11/12 07:15:33 INFO Vacuumed table table=pending_closures29812026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts29822026/11/12 07:15:33 INFO Vacuumed table table=pending_objects29832026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads29842026/11/12 07:15:33 INFO Vacuumed table table=closures29852026/11/12 07:15:33 INFO Vacuumed table table=objects29862026/11/12 07:15:33 INFO Received push request method=POST path=/29872026/11/12 07:15:33 INFO Received uploads request method=POST path=/29882026/11/12 07:15:33 INFO Received complete push request method=POST path=/29892026/11/12 07:15:33 INFO Received uploads request method=POST path=/29902026/11/12 07:15:33 INFO Received uploads request method=POST path=/29912026/11/12 07:15:33 INFO Completed push id=2 objects=7 duration=5.798477ms29922026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=029932026/11/12 07:15:33 INFO Received complete upload request method=POST path=/29942026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period29952026/11/12 07:15:33 INFO Completed upload id=9 objects=16 duration=1.700109ms29962026/11/12 07:15:33 INFO Received complete push request method=POST path=/29972026/11/12 07:15:33 INFO Received push request method=POST path=/29982026/11/12 07:15:33 INFO Completed push id=12 objects=3 duration=2.064538ms29992026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=030002026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030012026/11/12 07:15:33 INFO Vacuumed table table=pending_closures30022026/11/12 07:15:33 INFO Received push request method=POST path=/30032026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts30042026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=030052026/11/12 07:15:33 INFO Received uploads request method=POST path=/30062026/11/12 07:15:33 INFO Vacuumed table table=pending_objects30072026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads30082026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=030092026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=030102026/11/12 07:15:33 INFO Vacuumed table table=closures30112026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=030122026/11/12 07:15:33 INFO Vacuumed table table=objects30132026/11/12 07:15:33 INFO Received complete upload request method=POST path=/30142026/11/12 07:15:33 INFO Received complete upload request method=POST path=/30152026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period30162026/11/12 07:15:33 INFO Completed upload id=3 objects=24 duration=2.886369ms30172026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030182026/11/12 07:15:33 INFO Completed upload id=11 objects=14 duration=2.372819ms30192026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."30202026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030212026/11/12 07:15:33 INFO Vacuumed table table=pending_closures30222026/11/12 07:15:33 INFO Vacuumed table table=pending_closures30232026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts30242026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts30252026/11/12 07:15:33 INFO Received uploads request method=POST path=/30262026/11/12 07:15:33 INFO Received uploads request method=POST path=/30272026/11/12 07:15:33 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."30282026/11/12 07:15:33 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."30292026/11/12 07:15:33 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."30302026/11/12 07:15:33 INFO Vacuumed table table=pending_objects30312026/11/12 07:15:33 INFO Vacuumed table table=pending_objects30322026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads30332026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads30342026/11/12 07:15:33 INFO Vacuumed table table=closures30352026/11/12 07:15:33 INFO Vacuumed table table=closures30362026/11/12 07:15:33 INFO Vacuumed table table=objects30372026/11/12 07:15:33 INFO Vacuumed table table=objects30382026/11/12 07:15:33 INFO Received push request method=POST path=/30392026/11/12 07:15:33 INFO Received push request method=POST path=/30402026/11/12 07:15:33 INFO Received uploads request method=POST path=/30412026/11/12 07:15:33 INFO Received push request method=POST path=/30422026/11/12 07:15:33 INFO Received uploads request method=POST path=/30432026/11/12 07:15:33 INFO Received complete push request method=POST path=/30442026/11/12 07:15:33 INFO Received uploads request method=POST path=/30452026/11/12 07:15:33 INFO Completed push id=4 objects=4 duration=3.807728ms30462026/11/12 07:15:33 INFO Received push request method=POST path=/30472026/11/12 07:15:33 INFO Received push request method=POST path=/30482026/11/12 07:15:33 INFO Received uploads request method=POST path=/30492026/11/12 07:15:33 INFO Received push request method=POST path=/30502026/11/12 07:15:33 INFO Received push request method=POST path=/30512026/11/12 07:15:33 INFO Received push request method=POST path=/30522026/11/12 07:15:33 INFO Received uploads request method=POST path=/30532026/11/12 07:15:33 INFO Received push request method=POST path=/30542026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=030552026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=030562026/11/12 07:15:33 INFO Received push request method=POST path=/30572026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period30582026/11/12 07:15:33 INFO Received complete push request method=POST path=/30592026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030602026/11/12 07:15:33 INFO Vacuumed table table=pending_closures30612026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts30622026/11/12 07:15:33 INFO Completed push id=6 objects=7 duration=3.572218ms30632026/11/12 07:15:33 INFO Vacuumed table table=pending_objects30642026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads30652026/11/12 07:15:33 INFO Received push request method=POST path=/30662026/11/12 07:15:33 INFO Vacuumed table table=closures30672026/11/12 07:15:33 INFO Received uploads request method=POST path=/30682026/11/12 07:15:33 INFO Vacuumed table table=objects30692026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=030702026/11/12 07:15:33 INFO Received complete upload request method=POST path=/30712026/11/12 07:15:33 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."30722026/11/12 07:15:33 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."30732026/11/12 07:15:33 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."30742026/11/12 07:15:33 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."30752026/11/12 07:15:33 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."30762026/11/12 07:15:33 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."30772026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."30782026/11/12 07:15:33 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."30792026/11/12 07:15:33 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."30802026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."30812026/11/12 07:15:33 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."30822026/11/12 07:15:33 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."30832026/11/12 07:15:33 INFO Received uploads request method=POST path=/30842026/11/12 07:15:33 INFO Received complete push request method=POST path=/30852026/11/12 07:15:33 INFO Completed upload id=11 objects=6 duration=2.997398ms30862026/11/12 07:15:33 INFO Received complete push request method=POST path=/30872026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030882026/11/12 07:15:33 INFO Received uploads request method=POST path=/30892026/11/12 07:15:33 INFO Completed push id=10 objects=3 duration=2.449709ms30902026/11/12 07:15:33 INFO Vacuumed table table=pending_closures30912026/11/12 07:15:33 INFO Completed push id=6 objects=6 duration=2.601539ms30922026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts30932026/11/12 07:15:33 INFO Vacuumed table table=pending_objects30942026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads30952026/11/12 07:15:33 INFO Vacuumed table table=closures30962026/11/12 07:15:33 INFO Vacuumed table table=objects30972026/11/12 07:15:33 INFO Received push request method=POST path=/30982026/11/12 07:15:33 INFO Received uploads request method=POST path=/30992026/11/12 07:15:33 INFO Received uploads request method=POST path=/31002026/11/12 07:15:33 INFO Received uploads request method=POST path=/31012026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=031022026/11/12 07:15:33 INFO Received complete push request method=POST path=/31032026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period31042026/11/12 07:15:33 INFO Received uploads request method=POST path=/31052026/11/12 07:15:33 INFO Completed push id=10 objects=2 duration=2.793019ms31062026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031072026/11/12 07:15:33 INFO Vacuumed table table=pending_closures31082026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=031092026/11/12 07:15:33 INFO Received push request method=POST path=/31102026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts31112026/11/12 07:15:33 INFO Received push request method=POST path=/31122026/11/12 07:15:33 INFO Vacuumed table table=pending_closures31132026/11/12 07:15:33 INFO Vacuumed table table=pending_objects31142026/11/12 07:15:33 INFO Received complete push request method=POST path=/31152026/11/12 07:15:33 INFO Received push request method=POST path=/31162026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads31172026/11/12 07:15:33 INFO Vacuumed table table=closures31182026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts31192026/11/12 07:15:33 INFO Vacuumed table table=objects31202026/11/12 07:15:33 INFO Vacuumed table table=pending_objects31212026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads31222026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=031232026/11/12 07:15:33 INFO Completed push id=7 objects=20 duration=3.463418ms31242026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period31252026/11/12 07:15:33 INFO Received uploads request method=POST path=/31262026/11/12 07:15:33 INFO Vacuumed table table=closures31272026/11/12 07:15:33 INFO Received push request method=POST path=/31282026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031292026/11/12 07:15:33 INFO Vacuumed table table=objects31302026/11/12 07:15:33 INFO Vacuumed table table=pending_closures31312026/11/12 07:15:33 INFO Received uploads request method=POST path=/31322026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts31332026/11/12 07:15:33 INFO Vacuumed table table=pending_objects31342026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads31352026/11/12 07:15:33 INFO Vacuumed table table=closures31362026/11/12 07:15:33 INFO Vacuumed table table=objects31372026/11/12 07:15:33 INFO Received uploads request method=POST path=/31382026/11/12 07:15:33 INFO Received push request method=POST path=/31392026/11/12 07:15:33 INFO Received complete upload request method=POST path=/31402026/11/12 07:15:33 INFO Completed upload id=12 objects=21 duration=1.583349ms31412026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=031422026/11/12 07:15:33 INFO Received complete upload request method=POST path=/31432026/11/12 07:15:33 INFO Received push request method=POST path=/31442026/11/12 07:15:33 INFO Completed upload id=16 objects=23 duration=1.467849ms31452026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031462026/11/12 07:15:33 INFO Vacuumed table table=pending_closures31472026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts31482026/11/12 07:15:33 INFO Vacuumed table table=pending_objects31492026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads31502026/11/12 07:15:33 INFO Vacuumed table table=closures31512026/11/12 07:15:33 INFO Vacuumed table table=objects31522026/11/12 07:15:33 INFO Received push request method=POST path=/31532026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=031542026/11/12 07:15:33 INFO Received push request method=POST path=/31552026/11/12 07:15:33 INFO Received complete push request method=POST path=/31562026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=031572026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=031582026/11/12 07:15:33 INFO Received push request method=POST path=/31592026/11/12 07:15:33 INFO Completed push id=7 objects=27 duration=2.458688ms31602026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=031612026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period31622026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=031632026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period31642026/11/12 07:15:33 INFO Received complete upload request method=POST path=/31652026/11/12 07:15:33 INFO Vacuumed table table=pending_closures31662026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031672026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts31682026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031692026/11/12 07:15:33 INFO Completed upload id=15 objects=6 duration=1.607549ms31702026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031712026/11/12 07:15:33 INFO Vacuumed table table=pending_closures31722026/11/12 07:15:33 INFO Vacuumed table table=pending_closures31732026/11/12 07:15:33 INFO Vacuumed table table=pending_objects31742026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads31752026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts31762026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts31772026/11/12 07:15:33 INFO Vacuumed table table=pending_closures31782026/11/12 07:15:33 INFO Vacuumed table table=closures31792026/11/12 07:15:33 INFO Received uploads request method=POST path=/31802026/11/12 07:15:33 INFO Received push request method=POST path=/31812026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts31822026/11/12 07:15:33 INFO Vacuumed table table=objects31832026/11/12 07:15:33 INFO Vacuumed table table=pending_objects31842026/11/12 07:15:33 INFO Vacuumed table table=pending_objects31852026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads31862026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads31872026/11/12 07:15:33 INFO Vacuumed table table=closures31882026/11/12 07:15:33 INFO Vacuumed table table=pending_objects31892026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads31902026/11/12 07:15:33 INFO Vacuumed table table=closures31912026/11/12 07:15:33 INFO Received complete push request method=POST path=/31922026/11/12 07:15:33 INFO Vacuumed table table=objects31932026/11/12 07:15:33 INFO Vacuumed table table=closures31942026/11/12 07:15:33 INFO Vacuumed table table=objects31952026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=031962026/11/12 07:15:33 INFO Vacuumed table table=objects31972026/11/12 07:15:33 INFO Completed push id=9 objects=5 duration=2.477459ms31982026/11/12 07:15:33 INFO Received push request method=POST path=/31992026/11/12 07:15:33 INFO Received complete upload request method=POST path=/32002026/11/12 07:15:33 INFO Received push request method=POST path=/32012026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=032022026/11/12 07:15:33 INFO Completed upload id=4 objects=14 duration=4.275198ms32032026/11/12 07:15:33 INFO Received uploads request method=POST path=/32042026/11/12 07:15:33 INFO Vacuumed table table=pending_closures32052026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts32062026/11/12 07:15:33 INFO Vacuumed table table=pending_objects32072026/11/12 07:15:33 INFO Received complete push request method=POST path=/32082026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=032092026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads32102026/11/12 07:15:33 INFO Received uploads request method=POST path=/32112026/11/12 07:15:33 INFO Vacuumed table table=closures32122026/11/12 07:15:33 INFO Completed push id=13 objects=23 duration=2.219929ms32132026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period32142026/11/12 07:15:33 INFO Vacuumed table table=objects32152026/11/12 07:15:33 INFO Received complete upload request method=POST path=/32162026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032172026/11/12 07:15:33 INFO Received complete push request method=POST path=/32182026/11/12 07:15:33 INFO Received push request method=POST path=/32192026/11/12 07:15:33 INFO Completed upload id=10 objects=23 duration=2.273899ms32202026/11/12 07:15:33 INFO Received complete upload request method=POST path=/32212026/11/12 07:15:33 INFO Received uploads request method=POST path=/32222026/11/12 07:15:33 INFO Vacuumed table table=pending_closures32232026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts32242026/11/12 07:15:33 INFO Completed upload id=3 objects=9 duration=1.827389ms32252026/11/12 07:15:33 INFO Received push request method=POST path=/32262026/11/12 07:15:33 INFO Received uploads request method=POST path=/32272026/11/12 07:15:33 INFO Received complete push request method=POST path=/32282026/11/12 07:15:33 INFO Completed push id=14 objects=3 duration=3.944058ms32292026/11/12 07:15:33 INFO Vacuumed table table=pending_objects32302026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads32312026/11/12 07:15:33 INFO Vacuumed table table=closures32322026/11/12 07:15:33 INFO Received push request method=POST path=/32332026/11/12 07:15:33 INFO Vacuumed table table=objects32342026/11/12 07:15:33 INFO Received complete push request method=POST path=/32352026/11/12 07:15:33 INFO Completed push id=5 objects=9 duration=5.457937ms32362026/11/12 07:15:33 INFO Completed push id=5 objects=3 duration=3.936347ms32372026/11/12 07:15:33 INFO Received push request method=POST path=/32382026/11/12 07:15:33 INFO Received complete upload request method=POST path=/32392026/11/12 07:15:33 INFO Completed upload id=17 objects=7 duration=1.242809ms32402026/11/12 07:15:33 INFO Received uploads request method=POST path=/32412026/11/12 07:15:33 INFO Received uploads request method=POST path=/32422026/11/12 07:15:33 INFO Received push request method=POST path=/32432026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=032442026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=032452026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period32462026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period32472026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032482026/11/12 07:15:33 INFO Vacuumed table table=pending_closures32492026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts32502026/11/12 07:15:33 INFO Vacuumed table table=pending_objects32512026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads32522026/11/12 07:15:33 INFO Received uploads request method=POST path=/32532026/11/12 07:15:33 INFO Vacuumed table table=closures32542026/11/12 07:15:33 INFO Received push request method=POST path=/32552026/11/12 07:15:33 INFO Vacuumed table table=objects32562026/11/12 07:15:33 INFO Received complete upload request method=POST path=/32572026/11/12 07:15:33 INFO Completed upload id=10 objects=2 duration=2.819929ms32582026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=032592026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=032602026/11/12 07:15:33 INFO Received complete push request method=POST path=/32612026/11/12 07:15:33 INFO Received uploads request method=POST path=/32622026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032632026/11/12 07:15:33 INFO Completed push id=5 objects=6 duration=2.433348ms32642026/11/12 07:15:33 INFO Received uploads request method=POST path=/32652026/11/12 07:15:33 INFO Received push request method=POST path=/32662026/11/12 07:15:33 INFO Received uploads request method=POST path=/32672026/11/12 07:15:33 INFO Vacuumed table table=pending_closures32682026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts32692026/11/12 07:15:33 INFO Vacuumed table table=pending_objects32702026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads32712026/11/12 07:15:33 INFO Vacuumed table table=closures32722026/11/12 07:15:33 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."32732026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."32742026/11/12 07:15:33 INFO Vacuumed table table=objects32752026/11/12 07:15:33 INFO Received push request method=POST path=/32762026/11/12 07:15:33 INFO Received complete push request method=POST path=/32772026/11/12 07:15:33 INFO Completed push id=5 objects=6 duration=1.866309ms32782026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=032792026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=032802026/11/12 07:15:33 INFO Received uploads request method=POST path=/32812026/11/12 07:15:33 INFO Received complete upload request method=POST path=/32822026/11/12 07:15:33 INFO Received uploads request method=POST path=/32832026/11/12 07:15:33 INFO Completed upload id=11 objects=6 duration=1.450819ms32842026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period32852026/11/12 07:15:33 INFO Received push request method=POST path=/32862026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=032872026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032882026/11/12 07:15:33 INFO Vacuumed table table=pending_closures32892026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts32902026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=032912026/11/12 07:15:33 INFO Vacuumed table table=pending_objects32922026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads32932026/11/12 07:15:33 INFO Vacuumed table table=pending_closures32942026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts32952026/11/12 07:15:33 INFO Vacuumed table table=closures32962026/11/12 07:15:33 INFO Vacuumed table table=pending_objects32972026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads32982026/11/12 07:15:33 INFO Vacuumed table table=objects32992026/11/12 07:15:33 INFO Vacuumed table table=closures33002026/11/12 07:15:33 INFO Vacuumed table table=objects33012026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=033022026/11/12 07:15:33 INFO Received push request method=POST path=/33032026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period33042026/11/12 07:15:33 INFO Received complete push request method=POST path=/33052026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=033062026/11/12 07:15:33 INFO Received complete push request method=POST path=/33072026/11/12 07:15:33 INFO Vacuumed table table=pending_closures33082026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts33092026/11/12 07:15:33 INFO Completed push id=9 objects=7 duration=5.032247ms33102026/11/12 07:15:33 INFO Vacuumed table table=pending_objects33112026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads33122026/11/12 07:15:33 INFO Received push request method=POST path=/33132026/11/12 07:15:33 INFO Received complete push request method=POST path=/33142026/11/12 07:15:33 INFO Vacuumed table table=closures33152026/11/12 07:15:33 INFO Completed push id=3 objects=21 duration=5.115257ms33162026/11/12 07:15:33 INFO Vacuumed table table=objects33172026/11/12 07:15:33 INFO Completed push id=8 objects=6 duration=2.237719ms33182026/11/12 07:15:33 INFO Received push request method=POST path=/33192026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=033202026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period33212026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033222026/11/12 07:15:33 INFO Vacuumed table table=pending_closures33232026/11/12 07:15:33 INFO Received push request method=POST path=/33242026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts33252026/11/12 07:15:33 INFO Received uploads request method=POST path=/33262026/11/12 07:15:33 INFO Vacuumed table table=pending_objects33272026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads33282026/11/12 07:15:33 INFO Vacuumed table table=closures33292026/11/12 07:15:33 INFO Vacuumed table table=objects33302026/11/12 07:15:33 INFO Received complete upload request method=POST path=/33312026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=033322026/11/12 07:15:33 INFO Completed upload id=14 objects=6 duration=1.504329ms33332026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033342026/11/12 07:15:33 INFO Vacuumed table table=pending_closures33352026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts33362026/11/12 07:15:33 INFO Received uploads request method=POST path=/33372026/11/12 07:15:33 INFO Vacuumed table table=pending_objects33382026/11/12 07:15:33 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."33392026/11/12 07:15:33 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."33402026/11/12 07:15:33 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."33412026/11/12 07:15:33 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."33422026/11/12 07:15:33 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."33432026/11/12 07:15:33 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."33442026/11/12 07:15:33 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."33452026/11/12 07:15:33 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."33462026/11/12 07:15:33 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."33472026/11/12 07:15:33 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."33482026/11/12 07:15:33 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."33492026/11/12 07:15:33 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."33502026/11/12 07:15:33 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."33512026/11/12 07:15:33 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."33522026/11/12 07:15:33 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."33532026/11/12 07:15:33 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."33542026/11/12 07:15:33 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."33552026/11/12 07:15:33 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."33562026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads33572026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."33582026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."33592026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."33602026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."33612026/11/12 07:15:33 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."33622026/11/12 07:15:33 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."33632026/11/12 07:15:33 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."33642026/11/12 07:15:33 INFO Vacuumed table table=closures33652026/11/12 07:15:33 INFO Vacuumed table table=objects33662026/11/12 07:15:33 INFO Received push request method=POST path=/33672026/11/12 07:15:33 INFO Received push request method=POST path=/33682026/11/12 07:15:33 INFO Received complete upload request method=POST path=/33692026/11/12 07:15:33 INFO Completed upload id=10 objects=7 duration=1.792339ms33702026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=033712026/11/12 07:15:33 INFO Received complete push request method=POST path=/33722026/11/12 07:15:33 INFO Received push request method=POST path=/33732026/11/12 07:15:33 INFO Completed push id=12 objects=15 duration=2.143909ms33742026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033752026/11/12 07:15:33 INFO Vacuumed table table=pending_closures33762026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts33772026/11/12 07:15:33 INFO Vacuumed table table=pending_objects33782026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads33792026/11/12 07:15:33 INFO Vacuumed table table=closures33802026/11/12 07:15:33 INFO Vacuumed table table=objects33812026/11/12 07:15:33 INFO Received complete upload request method=POST path=/33822026/11/12 07:15:33 INFO Received push request method=POST path=/33832026/11/12 07:15:33 INFO Completed upload id=10 objects=6 duration=3.208909ms33842026/11/12 07:15:33 INFO Received push request method=POST path=/33852026/11/12 07:15:33 INFO Received uploads request method=POST path=/33862026/11/12 07:15:33 INFO Received complete push request method=POST path=/33872026/11/12 07:15:33 INFO Received uploads request method=POST path=/33882026/11/12 07:15:33 INFO Completed push id=5 objects=8 duration=3.980608ms33892026/11/12 07:15:33 INFO Received uploads request method=POST path=/33902026/11/12 07:15:33 INFO Received uploads request method=POST path=/33912026/11/12 07:15:33 INFO Received push request method=POST path=/33922026/11/12 07:15:33 INFO Received push request method=POST path=/33932026/11/12 07:15:33 INFO Received uploads request method=POST path=/33942026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=033952026/11/12 07:15:33 INFO Received complete upload request method=POST path=/33962026/11/12 07:15:33 INFO Received uploads request method=POST path=/33972026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period33982026/11/12 07:15:33 INFO Completed upload id=7 objects=3 duration=1.591859ms33992026/11/12 07:15:33 INFO Received complete push request method=POST path=/34002026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034012026/11/12 07:15:33 INFO Received push request method=POST path=/34022026/11/12 07:15:33 INFO Vacuumed table table=pending_closures34032026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts34042026/11/12 07:15:33 INFO Completed push id=11 objects=8 duration=3.336428ms34052026/11/12 07:15:33 INFO Vacuumed table table=pending_objects34062026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads34072026/11/12 07:15:33 INFO Vacuumed table table=closures34082026/11/12 07:15:33 INFO Vacuumed table table=objects34092026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=034102026/11/12 07:15:33 INFO Received uploads request method=POST path=/34112026/11/12 07:15:33 INFO Vacuumed table table=pending_closures34122026/11/12 07:15:33 INFO Received push request method=POST path=/34132026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts34142026/11/12 07:15:33 INFO Vacuumed table table=pending_objects34152026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads34162026/11/12 07:15:33 INFO Vacuumed table table=closures34172026/11/12 07:15:33 INFO Vacuumed table table=objects34182026/11/12 07:15:33 INFO Received push request method=POST path=/34192026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=034202026/11/12 07:15:33 INFO Received uploads request method=POST path=/34212026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034222026/11/12 07:15:33 INFO Vacuumed table table=pending_closures34232026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts34242026/11/12 07:15:33 INFO Received uploads request method=POST path=/34252026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=034262026/11/12 07:15:33 INFO Vacuumed table table=pending_objects34272026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads34282026/11/12 07:15:33 INFO Received uploads request method=POST path=/34292026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period34302026/11/12 07:15:33 INFO Vacuumed table table=closures34312026/11/12 07:15:33 INFO Received uploads request method=POST path=/34322026/11/12 07:15:33 INFO Vacuumed table table=objects34332026/11/12 07:15:33 INFO Received complete upload request method=POST path=/34342026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034352026/11/12 07:15:33 INFO Received complete push request method=POST path=/34362026/11/12 07:15:33 INFO Vacuumed table table=pending_closures34372026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts34382026/11/12 07:15:33 INFO Completed upload id=17 objects=6 duration=3.034148ms34392026/11/12 07:15:33 INFO Received complete upload request method=POST path=/34402026/11/12 07:15:33 INFO Vacuumed table table=pending_objects34412026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads34422026/11/12 07:15:33 INFO Vacuumed table table=closures34432026/11/12 07:15:33 INFO Received push request method=POST path=/34442026/11/12 07:15:33 INFO Completed push id=9 objects=4 duration=4.396258ms34452026/11/12 07:15:33 INFO Vacuumed table table=objects34462026/11/12 07:15:33 INFO Received complete push request method=POST path=/34472026/11/12 07:15:33 INFO Completed upload id=15 objects=4 duration=3.556078ms34482026/11/12 07:15:33 INFO Received complete push request method=POST path=/34492026/11/12 07:15:33 INFO Completed push id=18 objects=23 duration=2.176689ms34502026/11/12 07:15:33 INFO Received push request method=POST path=/34512026/11/12 07:15:33 INFO Received push request method=POST path=/34522026/11/12 07:15:33 INFO Completed push id=15 objects=24 duration=4.282528ms34532026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=034542026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034552026/11/12 07:15:33 INFO Vacuumed table table=pending_closures34562026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts34572026/11/12 07:15:33 INFO Received complete upload request method=POST path=/34582026/11/12 07:15:33 INFO Vacuumed table table=pending_objects34592026/11/12 07:15:33 INFO Received complete push request method=POST path=/34602026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads34612026/11/12 07:15:33 INFO Completed upload id=18 objects=4 duration=2.070369ms34622026/11/12 07:15:33 INFO Vacuumed table table=closures34632026/11/12 07:15:33 INFO Vacuumed table table=objects34642026/11/12 07:15:33 INFO Received push request method=POST path=/34652026/11/12 07:15:33 INFO Received push request method=POST path=/34662026/11/12 07:15:33 INFO Completed push id=6 objects=26 duration=4.744518ms34672026/11/12 07:15:33 INFO Received complete push request method=POST path=/34682026/11/12 07:15:33 INFO Received complete push request method=POST path=/34692026/11/12 07:15:33 INFO Received uploads request method=POST path=/34702026/11/12 07:15:33 INFO Completed push id=8 objects=6 duration=2.138829ms34712026/11/12 07:15:33 INFO Received uploads request method=POST path=/34722026/11/12 07:15:33 INFO Completed push id=13 objects=6 duration=3.149408ms34732026/11/12 07:15:33 INFO Received complete push request method=POST path=/34742026/11/12 07:15:33 INFO Completed push id=15 objects=5 duration=2.290549ms34752026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=034762026/11/12 07:15:33 INFO Received complete upload request method=POST path=/34772026/11/12 07:15:33 INFO Completed upload id=10 objects=17 duration=1.843519ms34782026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period34792026/11/12 07:15:33 INFO Received push request method=POST path=/34802026/11/12 07:15:33 INFO Received push request method=POST path=/34812026/11/12 07:15:33 INFO Received uploads request method=POST path=/34822026/11/12 07:15:33 INFO Received uploads request method=POST path=/34832026/11/12 07:15:33 INFO Received complete push request method=POST path=/34842026/11/12 07:15:33 INFO Completed push id=12 objects=7 duration=1.943299ms34852026/11/12 07:15:33 INFO Received uploads request method=POST path=/34862026/11/12 07:15:33 INFO Received complete push request method=POST path=/34872026/11/12 07:15:33 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."34882026/11/12 07:15:33 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."34892026/11/12 07:15:33 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."34902026/11/12 07:15:33 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."34912026/11/12 07:15:33 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."34922026/11/12 07:15:33 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."34932026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."34942026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."34952026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."34962026/11/12 07:15:33 INFO Completed push id=11 objects=9 duration=2.074569ms34972026/11/12 07:15:33 INFO Received push request method=POST path=/34982026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=034992026/11/12 07:15:33 INFO Received complete push request method=POST path=/35002026/11/12 07:15:33 INFO Received uploads request method=POST path=/35012026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035022026/11/12 07:15:33 INFO Vacuumed table table=pending_closures35032026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts35042026/11/12 07:15:33 INFO Vacuumed table table=pending_objects35052026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads35062026/11/12 07:15:33 INFO Completed push id=11 objects=11 duration=4.414977ms35072026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=035082026/11/12 07:15:33 INFO Vacuumed table table=closures35092026/11/12 07:15:33 INFO Received complete upload request method=POST path=/35102026/11/12 07:15:33 INFO Vacuumed table table=objects35112026/11/12 07:15:33 INFO Completed upload id=16 objects=3 duration=1.324819ms35122026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period35132026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035142026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=035152026/11/12 07:15:33 INFO Vacuumed table table=pending_closures35162026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period35172026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts35182026/11/12 07:15:33 INFO Vacuumed table table=pending_objects35192026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=035202026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads35212026/11/12 07:15:33 INFO Vacuumed table table=closures35222026/11/12 07:15:33 INFO Vacuumed table table=objects35232026/11/12 07:15:33 INFO Received uploads request method=POST path=/35242026/11/12 07:15:33 INFO Received push request method=POST path=/35252026/11/12 07:15:33 INFO Received push request method=POST path=/35262026/11/12 07:15:33 INFO Received uploads request method=POST path=/35272026/11/12 07:15:33 INFO Received push request method=POST path=/35282026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=035292026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=9 objects-failed-to-delete=035302026/11/12 07:15:33 INFO Received uploads request method=POST path=/35312026/11/12 07:15:33 INFO Vacuumed table table=pending_closures35322026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035332026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts35342026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=035352026/11/12 07:15:33 INFO Vacuumed table table=pending_closures35362026/11/12 07:15:33 INFO Vacuumed table table=pending_objects35372026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts35382026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads35392026/11/12 07:15:33 INFO Vacuumed table table=pending_closures35402026/11/12 07:15:33 INFO Received uploads request method=POST path=/35412026/11/12 07:15:33 INFO Vacuumed table table=pending_objects35422026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts35432026/11/12 07:15:33 INFO Vacuumed table table=closures35442026/11/12 07:15:33 INFO Received uploads request method=POST path=/35452026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads35462026/11/12 07:15:33 INFO Vacuumed table table=closures35472026/11/12 07:15:33 INFO Vacuumed table table=objects35482026/11/12 07:15:33 INFO Received push request method=POST path=/35492026/11/12 07:15:33 INFO Vacuumed table table=pending_objects35502026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads35512026/11/12 07:15:33 INFO Vacuumed table table=objects35522026/11/12 07:15:33 INFO Vacuumed table table=closures35532026/11/12 07:15:33 INFO Received push request method=POST path=/35542026/11/12 07:15:33 INFO Vacuumed table table=objects35552026/11/12 07:15:33 INFO Received push request method=POST path=/35562026/11/12 07:15:33 INFO Received push request method=POST path=/35572026/11/12 07:15:33 INFO Received push request method=POST path=/35582026/11/12 07:15:33 INFO Received complete push request method=POST path=/35592026/11/12 07:15:33 INFO Received complete push request method=POST path=/35602026/11/12 07:15:33 INFO Completed push id=17 objects=6 duration=2.407548ms35612026/11/12 07:15:33 INFO Completed push id=8 objects=6 duration=3.229538ms35622026/11/12 07:15:33 INFO Received push request method=POST path=/35632026/11/12 07:15:33 INFO Received complete push request method=POST path=/35642026/11/12 07:15:33 INFO Received complete upload request method=POST path=/35652026/11/12 07:15:33 INFO Completed upload id=6 objects=6 duration=1.696229ms35662026/11/12 07:15:33 INFO Completed push id=4 objects=6 duration=2.945809ms35672026/11/12 07:15:33 INFO Received uploads request method=POST path=/35682026/11/12 07:15:33 INFO Received complete push request method=POST path=/35692026/11/12 07:15:33 INFO Received uploads request method=POST path=/35702026/11/12 07:15:33 INFO Completed push id=14 objects=6 duration=2.285548ms35712026/11/12 07:15:33 INFO Received uploads request method=POST path=/35722026/11/12 07:15:33 INFO Received complete upload request method=POST path=/35732026/11/12 07:15:33 INFO Received uploads request method=POST path=/35742026/11/12 07:15:33 INFO Completed upload id=8 objects=3 duration=2.858008ms35752026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=035762026/11/12 07:15:33 INFO Received push request method=POST path=/35772026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=035782026/11/12 07:15:33 INFO Received push request method=POST path=/35792026/11/12 07:15:33 INFO Received complete push request method=POST path=/35802026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period35812026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=035822026/11/12 07:15:33 INFO Received push request method=POST path=/35832026/11/12 07:15:33 INFO Received complete push request method=POST path=/35842026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period35852026/11/12 07:15:33 INFO Received uploads request method=POST path=/35862026/11/12 07:15:33 INFO Received push request method=POST path=/35872026/11/12 07:15:33 INFO Completed push id=17 objects=22 duration=2.905129ms35882026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period35892026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035902026/11/12 07:15:33 INFO Completed push id=4 objects=7 duration=2.336438ms35912026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035922026/11/12 07:15:33 INFO Vacuumed table table=pending_closures35932026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts35942026/11/12 07:15:33 INFO Vacuumed table table=pending_closures35952026/11/12 07:15:33 INFO Received complete push request method=POST path=/35962026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts35972026/11/12 07:15:33 INFO Vacuumed table table=pending_objects35982026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads35992026/11/12 07:15:33 INFO Vacuumed table table=pending_objects36002026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads36012026/11/12 07:15:33 INFO Vacuumed table table=closures36022026/11/12 07:15:33 INFO Completed push id=1 objects=11 duration=2.964449ms36032026/11/12 07:15:33 INFO Vacuumed table table=closures36042026/11/12 07:15:33 INFO Vacuumed table table=objects36052026/11/12 07:15:33 INFO Vacuumed table table=objects36062026/11/12 07:15:33 INFO Received uploads request method=POST path=/36072026/11/12 07:15:33 INFO Received uploads request method=POST path=/36082026/11/12 07:15:33 INFO Received complete upload request method=POST path=/36092026/11/12 07:15:33 INFO Completed upload id=18 objects=4 duration=2.342979ms36102026/11/12 07:15:33 INFO Received push request method=POST path=/36112026/11/12 07:15:33 INFO Received uploads request method=POST path=/36122026/11/12 07:15:33 INFO Received uploads request method=POST path=/36132026/11/12 07:15:33 INFO Received uploads request method=POST path=/36142026/11/12 07:15:33 INFO Received complete upload request method=POST path=/36152026/11/12 07:15:33 INFO Completed upload id=9 objects=2 duration=1.667369ms36162026/11/12 07:15:33 INFO Received push request method=POST path=/36172026/11/12 07:15:33 INFO Received uploads request method=POST path=/36182026/11/12 07:15:33 INFO Received uploads request method=POST path=/36192026/11/12 07:15:33 INFO Received push request method=POST path=/36202026/11/12 07:15:33 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."36212026/11/12 07:15:33 INFO Received complete upload request method=POST path=/36222026/11/12 07:15:33 INFO Received complete upload request method=POST path=/36232026/11/12 07:15:33 INFO Completed upload id=16 objects=3 duration=1.293109ms36242026/11/12 07:15:33 INFO Completed upload id=15 objects=6 duration=1.257649ms36252026/11/12 07:15:33 INFO Received uploads request method=POST path=/36262026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=036272026/11/12 07:15:33 INFO Received complete push request method=POST path=/36282026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=036292026/11/12 07:15:33 INFO Vacuumed table table=pending_closures36302026/11/12 07:15:33 INFO Received complete upload request method=POST path=/36312026/11/12 07:15:33 INFO Completed push id=8 objects=3 duration=2.056519ms36322026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts36332026/11/12 07:15:33 INFO Completed upload id=15 objects=9 duration=1.254369ms36342026/11/12 07:15:33 INFO Vacuumed table table=pending_objects36352026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads36362026/11/12 07:15:33 INFO Vacuumed table table=closures36372026/11/12 07:15:33 INFO Vacuumed table table=objects36382026/11/12 07:15:33 INFO Received uploads request method=POST path=/36392026/11/12 07:15:33 INFO Received uploads request method=POST path=/36402026/11/12 07:15:33 INFO Received complete push request method=POST path=/36412026/11/12 07:15:33 INFO Received push request method=POST path=/36422026/11/12 07:15:33 INFO Received complete push request method=POST path=/36432026/11/12 07:15:33 INFO Completed push id=17 objects=6 duration=2.730979ms36442026/11/12 07:15:33 INFO Received push request method=POST path=/36452026/11/12 07:15:33 INFO Completed push id=6 objects=6 duration=2.328819ms36462026/11/12 07:15:33 INFO Received complete push request method=POST path=/36472026/11/12 07:15:33 INFO Completed push id=9 objects=24 duration=1.790579ms36482026/11/12 07:15:33 INFO Received uploads request method=POST path=/36492026/11/12 07:15:33 INFO Received push request method=POST path=/36502026/11/12 07:15:33 INFO Received push request method=POST path=/36512026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=036522026/11/12 07:15:33 INFO Received complete upload request method=POST path=/36532026/11/12 07:15:33 INFO Received push request method=POST path=/36542026/11/12 07:15:33 INFO Received push request method=POST path=/36552026/11/12 07:15:33 INFO Received uploads request method=POST path=/36562026/11/12 07:15:33 INFO Completed upload id=7 objects=10 duration=2.024639ms36572026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period36582026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036592026/11/12 07:15:33 INFO Vacuumed table table=pending_closures36602026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=036612026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts36622026/11/12 07:15:33 INFO Received push request method=POST path=/36632026/11/12 07:15:33 INFO Received complete push request method=POST path=/36642026/11/12 07:15:33 INFO Vacuumed table table=pending_objects36652026/11/12 07:15:33 INFO Received uploads request method=POST path=/36662026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period36672026/11/12 07:15:33 INFO Received push request method=POST path=/36682026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads36692026/11/12 07:15:33 INFO Completed push id=21 objects=17 duration=2.180369ms36702026/11/12 07:15:33 INFO Vacuumed table table=closures36712026/11/12 07:15:33 INFO Received complete upload request method=POST path=/36722026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036732026/11/12 07:15:33 INFO Vacuumed table table=objects36742026/11/12 07:15:33 INFO Vacuumed table table=pending_closures36752026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts36762026/11/12 07:15:33 INFO Received complete upload request method=POST path=/36772026/11/12 07:15:33 INFO Completed upload id=19 objects=25 duration=3.458878ms36782026/11/12 07:15:33 INFO Vacuumed table table=pending_objects36792026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads36802026/11/12 07:15:33 INFO Vacuumed table table=closures36812026/11/12 07:15:33 INFO Completed upload id=23 objects=15 duration=2.996618ms36822026/11/12 07:15:33 INFO Vacuumed table table=objects36832026/11/12 07:15:33 INFO Received complete push request method=POST path=/36842026/11/12 07:15:33 INFO Received complete push request method=POST path=/36852026/11/12 07:15:33 INFO Completed push id=7 objects=7 duration=1.880809ms36862026/11/12 07:15:33 INFO Received push request method=POST path=/36872026/11/12 07:15:33 INFO Completed push id=13 objects=12 duration=3.914137ms36882026/11/12 07:15:33 INFO Received complete upload request method=POST path=/36892026/11/12 07:15:33 INFO Completed upload id=13 objects=10 duration=1.383969ms36902026/11/12 07:15:33 INFO Received complete push request method=POST path=/36912026/11/12 07:15:33 INFO Completed push id=19 objects=7 duration=1.835839ms36922026/11/12 07:15:33 INFO Received uploads request method=POST path=/36932026/11/12 07:15:33 INFO Received uploads request method=POST path=/36942026/11/12 07:15:33 INFO Received uploads request method=POST path=/36952026/11/12 07:15:33 INFO Received complete push request method=POST path=/36962026/11/12 07:15:33 INFO Received uploads request method=POST path=/36972026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=036982026/11/12 07:15:33 INFO Completed push id=16 objects=27 duration=2.460458ms36992026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=037002026/11/12 07:15:33 INFO Received push request method=POST path=/37012026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period37022026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=037032026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period37042026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=037052026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=037062026/11/12 07:15:33 INFO Vacuumed table table=pending_closures37072026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts37082026/11/12 07:15:33 INFO Vacuumed table table=pending_objects37092026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037102026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads37112026/11/12 07:15:33 INFO Vacuumed table table=closures37122026/11/12 07:15:33 INFO Vacuumed table table=pending_closures37132026/11/12 07:15:33 INFO Vacuumed table table=objects37142026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts37152026/11/12 07:15:33 INFO Vacuumed table table=pending_objects37162026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads37172026/11/12 07:15:33 INFO Received push request method=POST path=/37182026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=037192026/11/12 07:15:33 INFO Vacuumed table table=closures37202026/11/12 07:15:33 INFO Received push request method=POST path=/37212026/11/12 07:15:33 INFO Vacuumed table table=objects37222026/11/12 07:15:33 INFO Vacuumed table table=pending_closures37232026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts37242026/11/12 07:15:33 INFO Received complete push request method=POST path=/37252026/11/12 07:15:33 INFO Vacuumed table table=pending_objects37262026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads37272026/11/12 07:15:33 INFO Received uploads request method=POST path=/37282026/11/12 07:15:33 INFO Vacuumed table table=closures37292026/11/12 07:15:33 INFO Completed push id=10 objects=15 duration=2.334209ms37302026/11/12 07:15:33 INFO Vacuumed table table=objects37312026/11/12 07:15:33 INFO Received uploads request method=POST path=/37322026/11/12 07:15:33 INFO Received complete upload request method=POST path=/37332026/11/12 07:15:33 INFO Received uploads request method=POST path=/37342026/11/12 07:15:33 INFO Completed upload id=10 objects=10 duration=1.260559ms37352026/11/12 07:15:33 INFO Received uploads request method=POST path=/37362026/11/12 07:15:33 INFO Received complete upload request method=POST path=/37372026/11/12 07:15:33 INFO Completed upload id=19 objects=6 duration=1.437539ms37382026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=037392026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period37402026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037412026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=037422026/11/12 07:15:33 INFO Received complete push request method=POST path=/37432026/11/12 07:15:33 INFO Vacuumed table table=pending_closures37442026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts37452026/11/12 07:15:33 INFO Received complete upload request method=POST path=/37462026/11/12 07:15:33 INFO Vacuumed table table=pending_objects37472026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads37482026/11/12 07:15:33 INFO Completed upload id=18 objects=11 duration=1.669879ms37492026/11/12 07:15:33 INFO Vacuumed table table=closures37502026/11/12 07:15:33 INFO Completed push id=12 objects=2 duration=3.642928ms37512026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period37522026/11/12 07:15:33 INFO Vacuumed table table=objects37532026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037542026/11/12 07:15:33 INFO Received push request method=POST path=/37552026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=037562026/11/12 07:15:33 INFO Vacuumed table table=pending_closures37572026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts37582026/11/12 07:15:33 INFO Vacuumed table table=pending_objects37592026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads37602026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period37612026/11/12 07:15:33 INFO Vacuumed table table=closures37622026/11/12 07:15:33 INFO Received complete push request method=POST path=/37632026/11/12 07:15:33 INFO Vacuumed table table=objects37642026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037652026/11/12 07:15:33 INFO Received push request method=POST path=/37662026/11/12 07:15:33 INFO Vacuumed table table=pending_closures37672026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts37682026/11/12 07:15:33 INFO Completed push id=14 objects=3 duration=3.847087ms37692026/11/12 07:15:33 INFO Vacuumed table table=pending_objects37702026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads37712026/11/12 07:15:33 INFO Vacuumed table table=closures37722026/11/12 07:15:33 INFO Vacuumed table table=objects37732026/11/12 07:15:33 INFO Received complete upload request method=POST path=/37742026/11/12 07:15:33 INFO Received push request method=POST path=/37752026/11/12 07:15:33 INFO Completed upload id=22 objects=7 duration=1.238079ms37762026/11/12 07:15:33 INFO Received uploads request method=POST path=/37772026/11/12 07:15:33 INFO Received complete push request method=POST path=/37782026/11/12 07:15:33 INFO Completed push id=4 objects=6 duration=2.044699ms37792026/11/12 07:15:33 INFO Received complete upload request method=POST path=/37802026/11/12 07:15:33 INFO Received uploads request method=POST path=/37812026/11/12 07:15:33 INFO Completed upload id=11 objects=7 duration=1.542509ms37822026/11/12 07:15:33 INFO Received uploads request method=POST path=/37832026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=11 objects-failed-to-delete=037842026/11/12 07:15:33 INFO Vacuumed table table=pending_closures37852026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts37862026/11/12 07:15:33 INFO Vacuumed table table=pending_objects37872026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads37882026/11/12 07:15:33 INFO Received uploads request method=POST path=/37892026/11/12 07:15:33 INFO Vacuumed table table=closures37902026/11/12 07:15:33 INFO Vacuumed table table=objects37912026/11/12 07:15:33 INFO Received complete push request method=POST path=/37922026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=037932026/11/12 07:15:33 INFO Completed push id=19 objects=8 duration=4.325837ms37942026/11/12 07:15:33 INFO Received complete push request method=POST path=/37952026/11/12 07:15:33 WARN Force mode enabled - objects will be deleted immediately without grace period37962026/11/12 07:15:33 INFO Completed push id=18 objects=10 duration=1.951189ms37972026/11/12 07:15:33 INFO Received complete push request method=POST path=/37982026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037992026/11/12 07:15:33 INFO Received uploads request method=POST path=/38002026/11/12 07:15:33 INFO Received push request method=POST path=/38012026/11/12 07:15:33 INFO Vacuumed table table=pending_closures38022026/11/12 07:15:33 INFO Received uploads request method=POST path=/38032026/11/12 07:15:33 INFO Completed push id=21 objects=9 duration=2.452249ms38042026/11/12 07:15:33 INFO Vacuumed table table=commit_receipts38052026/11/12 07:15:33 INFO Received uploads request method=POST path=/38062026/11/12 07:15:33 INFO Vacuumed table table=pending_objects38072026/11/12 07:15:33 INFO Vacuumed table table=multipart_uploads38082026/11/12 07:15:33 INFO Vacuumed table table=closures38092026/11/12 07:15:33 INFO Vacuumed table table=objects38102026/11/12 07:15:33 INFO Received push request method=POST path=/38112026/11/12 07:15:33 INFO Received push request method=POST path=/38122026/11/12 07:15:33 INFO Received uploads request method=POST path=/38132026/11/12 07:15:33 INFO Received push request method=POST path=/38142026/11/12 07:15:33 INFO Received complete push request method=POST path=/38152026/11/12 07:15:33 INFO Received uploads request method=POST path=/38162026/11/12 07:15:33 INFO Received push request method=POST path=/38172026/11/12 07:15:33 INFO Received uploads request method=POST path=/38182026/11/12 07:15:33 INFO Received complete upload request method=POST path=/38192026/11/12 07:15:34 INFO Received push request method=POST path=/38202026/11/12 07:15:34 INFO Completed push id=19 objects=6 duration=3.971087ms38212026/11/12 07:15:34 INFO Completed upload id=12 objects=26 duration=3.090238ms38222026/11/12 07:15:34 INFO Received push request method=POST path=/38232026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=038242026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=038252026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period38262026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038272026/11/12 07:15:34 INFO Vacuumed table table=pending_closures38282026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts38292026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=038302026/11/12 07:15:34 INFO Vacuumed table table=pending_objects38312026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads38322026/11/12 07:15:34 INFO Vacuumed table table=closures38332026/11/12 07:15:34 INFO Vacuumed table table=objects38342026/11/12 07:15:34 INFO Received uploads request method=POST path=/38352026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period38362026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=038372026/11/12 07:15:34 INFO Received uploads request method=POST path=/38382026/11/12 07:15:34 INFO Vacuumed table table=pending_closures38392026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts38402026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=038412026/11/12 07:15:34 INFO Received complete push request method=POST path=/38422026/11/12 07:15:34 INFO Vacuumed table table=pending_objects38432026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads38442026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=038452026/11/12 07:15:34 INFO Completed push id=24 objects=9 duration=2.248428ms38462026/11/12 07:15:34 INFO Vacuumed table table=closures38472026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period38482026/11/12 07:15:34 INFO Vacuumed table table=objects38492026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038502026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038512026/11/12 07:15:34 INFO Received complete push request method=POST path=/38522026/11/12 07:15:34 INFO Vacuumed table table=pending_closures38532026/11/12 07:15:34 INFO Vacuumed table table=pending_closures38542026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts38552026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts38562026/11/12 07:15:34 INFO Vacuumed table table=pending_objects38572026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads38582026/11/12 07:15:34 INFO Vacuumed table table=pending_objects38592026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads38602026/11/12 07:15:34 INFO Completed push id=14 objects=4 duration=3.812938ms38612026/11/12 07:15:34 INFO Vacuumed table table=closures38622026/11/12 07:15:34 INFO Vacuumed table table=closures38632026/11/12 07:15:34 INFO Vacuumed table table=objects38642026/11/12 07:15:34 INFO Vacuumed table table=objects38652026/11/12 07:15:34 INFO Received push request method=POST path=/38662026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=038672026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period38682026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=038692026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038702026/11/12 07:15:34 INFO Vacuumed table table=pending_closures38712026/11/12 07:15:34 INFO Received push request method=POST path=/38722026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts38732026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period38742026/11/12 07:15:34 INFO Vacuumed table table=pending_objects38752026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads38762026/11/12 07:15:34 INFO Vacuumed table table=closures38772026/11/12 07:15:34 INFO Received push request method=POST path=/38782026/11/12 07:15:34 INFO Vacuumed table table=objects38792026/11/12 07:15:34 INFO Received uploads request method=POST path=/38802026/11/12 07:15:34 INFO Received push request method=POST path=/38812026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=5 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038822026/11/12 07:15:34 INFO Received uploads request method=POST path=/38832026/11/12 07:15:34 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."38842026/11/12 07:15:34 INFO Vacuumed table table=pending_closures38852026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts38862026/11/12 07:15:34 INFO Received push request method=POST path=/38872026/11/12 07:15:34 INFO Received uploads request method=POST path=/38882026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=038892026/11/12 07:15:34 INFO Vacuumed table table=pending_objects38902026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads38912026/11/12 07:15:34 INFO Vacuumed table table=closures38922026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period38932026/11/12 07:15:34 INFO Vacuumed table table=objects38942026/11/12 07:15:34 INFO Received complete upload request method=POST path=/38952026/11/12 07:15:34 INFO Received push request method=POST path=/38962026/11/12 07:15:34 INFO Completed upload id=23 objects=16 duration=2.201439ms38972026/11/12 07:15:34 INFO Received push request method=POST path=/38982026/11/12 07:15:34 INFO Received push request method=POST path=/38992026/11/12 07:15:34 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."39002026/11/12 07:15:34 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."39012026/11/12 07:15:34 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."39022026/11/12 07:15:34 INFO Received complete push request method=POST path=/39032026/11/12 07:15:34 INFO Received push request method=POST path=/39042026/11/12 07:15:34 INFO Received uploads request method=POST path=/39052026/11/12 07:15:34 INFO Completed push id=16 objects=8 duration=2.088909ms39062026/11/12 07:15:34 INFO Received complete push request method=POST path=/39072026/11/12 07:15:34 INFO Completed push id=14 objects=3 duration=1.521799ms39082026/11/12 07:15:34 INFO Received uploads request method=POST path=/39092026/11/12 07:15:34 INFO Received push request method=POST path=/39102026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=039112026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period39122026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039132026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=039142026/11/12 07:15:34 INFO Vacuumed table table=pending_closures39152026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts39162026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period39172026/11/12 07:15:34 INFO Received uploads request method=POST path=/39182026/11/12 07:15:34 INFO Vacuumed table table=pending_objects39192026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads39202026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039212026/11/12 07:15:34 INFO Vacuumed table table=closures39222026/11/12 07:15:34 INFO Vacuumed table table=pending_closures39232026/11/12 07:15:34 INFO Received push request method=POST path=/39242026/11/12 07:15:34 INFO Vacuumed table table=objects39252026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts39262026/11/12 07:15:34 INFO Received uploads request method=POST path=/39272026/11/12 07:15:34 INFO Received uploads request method=POST path=/39282026/11/12 07:15:34 INFO Vacuumed table table=pending_objects39292026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads39302026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=039312026/11/12 07:15:34 INFO Vacuumed table table=closures39322026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period39332026/11/12 07:15:34 INFO Vacuumed table table=objects39342026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=039352026/11/12 07:15:34 INFO Received push request method=POST path=/39362026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period39372026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=039382026/11/12 07:15:34 INFO Received complete push request method=POST path=/39392026/11/12 07:15:34 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."39402026/11/12 07:15:34 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."39412026/11/12 07:15:34 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."39422026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039432026/11/12 07:15:34 INFO Completed push id=15 objects=4 duration=1.478969ms39442026/11/12 07:15:34 INFO Vacuumed table table=pending_closures39452026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039462026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts39472026/11/12 07:15:34 INFO Vacuumed table table=pending_closures39482026/11/12 07:15:34 INFO Received push request method=POST path=/39492026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts39502026/11/12 07:15:34 INFO Vacuumed table table=pending_objects39512026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads39522026/11/12 07:15:34 INFO Vacuumed table table=pending_objects39532026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads39542026/11/12 07:15:34 INFO Vacuumed table table=closures39552026/11/12 07:15:34 INFO Vacuumed table table=closures39562026/11/12 07:15:34 INFO Received uploads request method=POST path=/39572026/11/12 07:15:34 INFO Vacuumed table table=objects39582026/11/12 07:15:34 INFO Vacuumed table table=objects39592026/11/12 07:15:34 INFO Received uploads request method=POST path=/39602026/11/12 07:15:34 INFO Received push request method=POST path=/39612026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=039622026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=039632026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period39642026/11/12 07:15:34 INFO Received uploads request method=POST path=/39652026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039662026/11/12 07:15:34 INFO Received complete push request method=POST path=/39672026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period39682026/11/12 07:15:34 INFO Vacuumed table table=pending_closures39692026/11/12 07:15:34 INFO Received uploads request method=POST path=/39702026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts39712026/11/12 07:15:34 INFO Received push request method=POST path=/39722026/11/12 07:15:34 INFO Completed push id=13 objects=15 duration=2.362708ms39732026/11/12 07:15:34 INFO Vacuumed table table=pending_objects39742026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads39752026/11/12 07:15:34 INFO Vacuumed table table=closures39762026/11/12 07:15:34 INFO Vacuumed table table=objects39772026/11/12 07:15:34 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."39782026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=039792026/11/12 07:15:34 INFO Received uploads request method=POST path=/39802026/11/12 07:15:34 INFO Received complete push request method=POST path=/39812026/11/12 07:15:34 INFO Received push request method=POST path=/39822026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039832026/11/12 07:15:34 INFO Vacuumed table table=pending_closures39842026/11/12 07:15:34 INFO Completed push id=22 objects=3 duration=3.262388ms39852026/11/12 07:15:34 INFO Received complete push request method=POST path=/39862026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts39872026/11/12 07:15:34 INFO Vacuumed table table=pending_objects39882026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads39892026/11/12 07:15:34 INFO Vacuumed table table=closures39902026/11/12 07:15:34 INFO Completed push id=16 objects=4 duration=3.388658ms39912026/11/12 07:15:34 INFO Vacuumed table table=objects39922026/11/12 07:15:34 INFO Received complete push request method=POST path=/39932026/11/12 07:15:34 INFO Received uploads request method=POST path=/39942026/11/12 07:15:34 INFO Completed push id=11 objects=6 duration=3.513428ms39952026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=039962026/11/12 07:15:34 INFO Received push request method=POST path=/39972026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=039982026/11/12 07:15:34 INFO Received push request method=POST path=/39992026/11/12 07:15:34 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."40002026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period40012026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040022026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=040032026/11/12 07:15:34 INFO Vacuumed table table=pending_closures40042026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period40052026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts40062026/11/12 07:15:34 INFO Received push request method=POST path=/40072026/11/12 07:15:34 INFO Received complete upload request method=POST path=/40082026/11/12 07:15:34 INFO Vacuumed table table=pending_objects40092026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads40102026/11/12 07:15:34 INFO Vacuumed table table=closures40112026/11/12 07:15:34 INFO Completed upload id=17 objects=25 duration=3.491818ms40122026/11/12 07:15:34 INFO Received uploads request method=POST path=/40132026/11/12 07:15:34 INFO Received push request method=POST path=/40142026/11/12 07:15:34 INFO Vacuumed table table=objects40152026/11/12 07:15:34 INFO Received complete push request method=POST path=/40162026/11/12 07:15:34 INFO Completed push id=15 objects=7 duration=2.369109ms40172026/11/12 07:15:34 INFO Received complete push request method=POST path=/40182026/11/12 07:15:34 INFO Completed push id=18 objects=6 duration=1.378649ms40192026/11/12 07:15:34 INFO Received uploads request method=POST path=/40202026/11/12 07:15:34 INFO Received complete upload request method=POST path=/40212026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=040222026/11/12 07:15:34 INFO Completed upload id=14 objects=8 duration=2.709969ms40232026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period40242026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040252026/11/12 07:15:34 INFO Vacuumed table table=pending_closures40262026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts40272026/11/12 07:15:34 INFO Received push request method=POST path=/40282026/11/12 07:15:34 INFO Vacuumed table table=pending_objects40292026/11/12 07:15:34 INFO Received complete push request method=POST path=/40302026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads40312026/11/12 07:15:34 INFO Vacuumed table table=closures40322026/11/12 07:15:34 INFO Completed push id=14 objects=3 duration=1.4349ms40332026/11/12 07:15:34 INFO Received push request method=POST path=/40342026/11/12 07:15:34 INFO Received uploads request method=POST path=/40352026/11/12 07:15:34 INFO Vacuumed table table=objects40362026/11/12 07:15:34 INFO Received complete upload request method=POST path=/40372026/11/12 07:15:34 INFO Completed upload id=20 objects=5 duration=1.307259ms40382026/11/12 07:15:34 INFO Received uploads request method=POST path=/40392026/11/12 07:15:34 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."40402026/11/12 07:15:34 INFO Received complete push request method=POST path=/40412026/11/12 07:15:34 INFO Completed push id=20 objects=3 duration=1.839659ms40422026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=040432026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040442026/11/12 07:15:34 INFO Vacuumed table table=pending_closures40452026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts40462026/11/12 07:15:34 INFO Vacuumed table table=pending_objects40472026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads40482026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=040492026/11/12 07:15:34 INFO Vacuumed table table=closures40502026/11/12 07:15:34 INFO Vacuumed table table=objects40512026/11/12 07:15:34 INFO Received uploads request method=POST path=/40522026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040532026/11/12 07:15:34 INFO Received push request method=POST path=/40542026/11/12 07:15:34 INFO Received push request method=POST path=/40552026/11/12 07:15:34 INFO Received complete upload request method=POST path=/40562026/11/12 07:15:34 INFO Vacuumed table table=pending_closures40572026/11/12 07:15:34 INFO Received uploads request method=POST path=/40582026/11/12 07:15:34 INFO Received complete push request method=POST path=/40592026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts40602026/11/12 07:15:34 INFO Completed upload id=21 objects=2 duration=1.611279ms40612026/11/12 07:15:34 INFO Received uploads request method=POST path=/40622026/11/12 07:15:34 INFO Received complete push request method=POST path=/40632026/11/12 07:15:34 INFO Completed push id=21 objects=3 duration=2.039999ms40642026/11/12 07:15:34 INFO Vacuumed table table=pending_objects40652026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads40662026/11/12 07:15:34 INFO Vacuumed table table=closures40672026/11/12 07:15:34 INFO Vacuumed table table=objects40682026/11/12 07:15:34 INFO Completed push id=16 objects=16 duration=4.327287ms40692026/11/12 07:15:34 INFO Received push request method=POST path=/40702026/11/12 07:15:34 INFO Received complete upload request method=POST path=/40712026/11/12 07:15:34 INFO Received uploads request method=POST path=/40722026/11/12 07:15:34 INFO Received complete push request method=POST path=/40732026/11/12 07:15:34 INFO Received complete push request method=POST path=/40742026/11/12 07:15:34 INFO Received uploads request method=POST path=/40752026/11/12 07:15:34 INFO Completed upload id=20 objects=3 duration=2.707708ms40762026/11/12 07:15:34 INFO Completed push id=7 objects=15 duration=2.189459ms40772026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=040782026/11/12 07:15:34 INFO Received uploads request method=POST path=/40792026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period40802026/11/12 07:15:34 INFO Completed push id=26 objects=14 duration=3.433528ms40812026/11/12 07:15:34 INFO Received uploads request method=POST path=/40822026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040832026/11/12 07:15:34 INFO Received complete upload request method=POST path=/40842026/11/12 07:15:34 INFO Completed upload id=20 objects=4 duration=1.478529ms40852026/11/12 07:15:34 INFO Vacuumed table table=pending_closures40862026/11/12 07:15:34 INFO Received complete upload request method=POST path=/40872026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts40882026/11/12 07:15:34 INFO Vacuumed table table=pending_objects40892026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads40902026/11/12 07:15:34 INFO Completed upload id=13 objects=8 duration=1.846809ms40912026/11/12 07:15:34 INFO Vacuumed table table=closures40922026/11/12 07:15:34 INFO Vacuumed table table=objects40932026/11/12 07:15:34 INFO Received complete upload request method=POST path=/40942026/11/12 07:15:34 INFO Received push request method=POST path=/40952026/11/12 07:15:34 INFO Completed upload id=14 objects=16 duration=3.043509ms40962026/11/12 07:15:34 INFO Received uploads request method=POST path=/40972026/11/12 07:15:34 INFO Received push request method=POST path=/40982026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=040992026/11/12 07:15:34 INFO Received push request method=POST path=/41002026/11/12 07:15:34 INFO Received uploads request method=POST path=/41012026/11/12 07:15:34 INFO Received uploads request method=POST path=/41022026/11/12 07:15:34 INFO Received push request method=POST path=/41032026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=041042026/11/12 07:15:34 INFO Received push request method=POST path=/41052026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period41062026/11/12 07:15:34 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."41072026/11/12 07:15:34 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."41082026/11/12 07:15:34 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."41092026/11/12 07:15:34 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."41102026/11/12 07:15:34 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."41112026/11/12 07:15:34 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."41122026/11/12 07:15:34 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."41132026/11/12 07:15:34 INFO Received complete push request method=POST path=/41142026/11/12 07:15:34 INFO Completed push id=16 objects=5 duration=1.301359ms41152026/11/12 07:15:34 INFO Received push request method=POST path=/41162026/11/12 07:15:34 INFO Received push request method=POST path=/41172026/11/12 07:15:34 INFO Received uploads request method=POST path=/41182026/11/12 07:15:34 INFO Received uploads request method=POST path=/41192026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=041202026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=041212026/11/12 07:15:34 INFO Received uploads request method=POST path=/41222026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=041232026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period41242026/11/12 07:15:34 INFO Vacuumed table table=pending_closures41252026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts41262026/11/12 07:15:34 INFO Received complete upload request method=POST path=/41272026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041282026/11/12 07:15:34 INFO Vacuumed table table=pending_objects41292026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads41302026/11/12 07:15:34 INFO Completed upload id=22 objects=2 duration=1.503909ms41312026/11/12 07:15:34 INFO Vacuumed table table=pending_closures41322026/11/12 07:15:34 INFO Vacuumed table table=closures41332026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=3 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041342026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts41352026/11/12 07:15:34 INFO Vacuumed table table=objects41362026/11/12 07:15:34 INFO Vacuumed table table=pending_closures41372026/11/12 07:15:34 INFO Vacuumed table table=pending_objects41382026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads41392026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts41402026/11/12 07:15:34 INFO Vacuumed table table=closures41412026/11/12 07:15:34 INFO Vacuumed table table=objects41422026/11/12 07:15:34 INFO Vacuumed table table=pending_objects41432026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads41442026/11/12 07:15:34 INFO Vacuumed table table=closures41452026/11/12 07:15:34 INFO Vacuumed table table=objects41462026/11/12 07:15:34 INFO Received uploads request method=POST path=/41472026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=041482026/11/12 07:15:34 INFO Received uploads request method=POST path=/41492026/11/12 07:15:34 INFO Received push request method=POST path=/41502026/11/12 07:15:34 INFO Received complete upload request method=POST path=/41512026/11/12 07:15:34 INFO Completed upload id=24 objects=7 duration=1.238739ms41522026/11/12 07:15:34 INFO Received push request method=POST path=/41532026/11/12 07:15:34 INFO Received push request method=POST path=/41542026/11/12 07:15:34 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."41552026/11/12 07:15:34 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."41562026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=041572026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=041582026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041592026/11/12 07:15:34 INFO Vacuumed table table=pending_closures41602026/11/12 07:15:34 INFO Received complete push request method=POST path=/41612026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts41622026/11/12 07:15:34 INFO Received complete push request method=POST path=/41632026/11/12 07:15:34 INFO Vacuumed table table=pending_objects41642026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads41652026/11/12 07:15:34 INFO Completed push id=7 objects=24 duration=4.105738ms41662026/11/12 07:15:34 INFO Completed push id=17 objects=11 duration=2.563909ms41672026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=041682026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=041692026/11/12 07:15:34 INFO Vacuumed table table=closures41702026/11/12 07:15:34 INFO Vacuumed table table=objects41712026/11/12 07:15:34 INFO Vacuumed table table=pending_closures41722026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts41732026/11/12 07:15:34 INFO Vacuumed table table=pending_objects41742026/11/12 07:15:34 INFO Received push request method=POST path=/41752026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads41762026/11/12 07:15:34 INFO Vacuumed table table=closures41772026/11/12 07:15:34 INFO Vacuumed table table=objects41782026/11/12 07:15:34 INFO Received uploads request method=POST path=/41792026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=041802026/11/12 07:15:34 INFO Received push request method=POST path=/41812026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=041822026/11/12 07:15:34 INFO Received uploads request method=POST path=/41832026/11/12 07:15:34 INFO Vacuumed table table=pending_closures41842026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041852026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts41862026/11/12 07:15:34 INFO Vacuumed table table=pending_closures41872026/11/12 07:15:34 INFO Vacuumed table table=pending_objects41882026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads41892026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts41902026/11/12 07:15:34 INFO Received uploads request method=POST path=/41912026/11/12 07:15:34 INFO Vacuumed table table=closures41922026/11/12 07:15:34 INFO Vacuumed table table=pending_objects41932026/11/12 07:15:34 INFO Vacuumed table table=objects41942026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads41952026/11/12 07:15:34 INFO Vacuumed table table=closures41962026/11/12 07:15:34 INFO Vacuumed table table=objects41972026/11/12 07:15:34 INFO Received uploads request method=POST path=/41982026/11/12 07:15:34 INFO Received push request method=POST path=/41992026/11/12 07:15:34 INFO Received uploads request method=POST path=/42002026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=042012026/11/12 07:15:34 INFO Received uploads request method=POST path=/42022026/11/12 07:15:34 INFO Received push request method=POST path=/42032026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period42042026/11/12 07:15:34 INFO Received push request method=POST path=/42052026/11/12 07:15:34 INFO Received complete upload request method=POST path=/42062026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042072026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=042082026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period42092026/11/12 07:15:34 INFO Completed upload id=22 objects=6 duration=1.317399ms42102026/11/12 07:15:34 INFO Vacuumed table table=pending_closures42112026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts42122026/11/12 07:15:34 INFO Received complete upload request method=POST path=/42132026/11/12 07:15:34 INFO Vacuumed table table=pending_objects42142026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads42152026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=042162026/11/12 07:15:34 INFO Vacuumed table table=closures42172026/11/12 07:15:34 INFO Completed upload id=23 objects=18 duration=3.131068ms42182026/11/12 07:15:34 INFO Vacuumed table table=objects42192026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042202026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=042212026/11/12 07:15:34 INFO Received complete push request method=POST path=/42222026/11/12 07:15:34 INFO Vacuumed table table=pending_closures42232026/11/12 07:15:34 INFO Vacuumed table table=pending_closures42242026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts42252026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts42262026/11/12 07:15:34 INFO Vacuumed table table=pending_objects42272026/11/12 07:15:34 INFO Received uploads request method=POST path=/42282026/11/12 07:15:34 INFO Completed push id=5 objects=28 duration=2.872889ms42292026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads42302026/11/12 07:15:34 INFO Vacuumed table table=pending_objects42312026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads42322026/11/12 07:15:34 INFO Vacuumed table table=closures42332026/11/12 07:15:34 INFO Vacuumed table table=closures42342026/11/12 07:15:34 INFO Received push request method=POST path=/42352026/11/12 07:15:34 INFO Received uploads request method=POST path=/42362026/11/12 07:15:34 INFO Vacuumed table table=objects42372026/11/12 07:15:34 INFO Vacuumed table table=objects42382026/11/12 07:15:34 INFO Received push request method=POST path=/42392026/11/12 07:15:34 INFO Received uploads request method=POST path=/42402026/11/12 07:15:34 INFO Received complete upload request method=POST path=/42412026/11/12 07:15:34 INFO Received complete push request method=POST path=/42422026/11/12 07:15:34 INFO Completed upload id=16 objects=17 duration=3.076068ms42432026/11/12 07:15:34 INFO Received uploads request method=POST path=/42442026/11/12 07:15:34 INFO Completed push id=30 objects=4 duration=3.555417ms42452026/11/12 07:15:34 INFO Received uploads request method=POST path=/42462026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=042472026/11/12 07:15:34 INFO Received uploads request method=POST path=/42482026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042492026/11/12 07:15:34 INFO Received push request method=POST path=/42502026/11/12 07:15:34 INFO Vacuumed table table=pending_closures42512026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts42522026/11/12 07:15:34 INFO Vacuumed table table=pending_objects42532026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads42542026/11/12 07:15:34 INFO Received complete upload request method=POST path=/42552026/11/12 07:15:34 INFO Vacuumed table table=closures42562026/11/12 07:15:34 INFO Received complete push request method=POST path=/42572026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=042582026/11/12 07:15:34 INFO Received uploads request method=POST path=/42592026/11/12 07:15:34 INFO Vacuumed table table=objects42602026/11/12 07:15:34 INFO Received uploads request method=POST path=/42612026/11/12 07:15:34 INFO Received push request method=POST path=/42622026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period42632026/11/12 07:15:34 INFO Received uploads request method=POST path=/42642026/11/12 07:15:34 INFO Completed push id=10 objects=2 duration=2.515338ms42652026/11/12 07:15:34 INFO Completed upload id=21 objects=7 duration=3.031978ms42662026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042672026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=042682026/11/12 07:15:34 INFO Vacuumed table table=pending_closures42692026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts42702026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period42712026/11/12 07:15:34 INFO Vacuumed table table=pending_objects42722026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads42732026/11/12 07:15:34 INFO Vacuumed table table=closures42742026/11/12 07:15:34 INFO Received uploads request method=POST path=/42752026/11/12 07:15:34 INFO Vacuumed table table=objects42762026/11/12 07:15:34 INFO Received push request method=POST path=/42772026/11/12 07:15:34 INFO Received uploads request method=POST path=/42782026/11/12 07:15:34 INFO Received push request method=POST path=/42792026/11/12 07:15:34 INFO Received push request method=POST path=/42802026/11/12 07:15:34 INFO Received complete upload request method=POST path=/42812026/11/12 07:15:34 INFO Completed upload id=26 objects=6 duration=1.605549ms42822026/11/12 07:15:34 INFO Received push request method=POST path=/42832026/11/12 07:15:34 INFO Received complete push request method=POST path=/42842026/11/12 07:15:34 INFO Completed push id=19 objects=14 duration=1.618129ms42852026/11/12 07:15:34 INFO Received push request method=POST path=/42862026/11/12 07:15:34 INFO Received uploads request method=POST path=/42872026/11/12 07:15:34 INFO Received complete upload request method=POST path=/42882026/11/12 07:15:34 INFO Completed upload id=17 objects=5 duration=1.509829ms42892026/11/12 07:15:34 INFO Received push request method=POST path=/42902026/11/12 07:15:34 INFO Received push request method=POST path=/42912026/11/12 07:15:34 INFO Received complete push request method=POST path=/42922026/11/12 07:15:34 INFO Received push request method=POST path=/42932026/11/12 07:15:34 INFO Received complete upload request method=POST path=/42942026/11/12 07:15:34 INFO Completed upload id=22 objects=22 duration=1.398319ms42952026/11/12 07:15:34 INFO Completed push id=23 objects=28 duration=3.731368ms42962026/11/12 07:15:34 INFO Received push request method=POST path=/42972026/11/12 07:15:34 INFO Received uploads request method=POST path=/42982026/11/12 07:15:34 INFO Received uploads request method=POST path=/42992026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=043002026/11/12 07:15:34 INFO Received uploads request method=POST path=/43012026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period43022026/11/12 07:15:34 INFO Received push request method=POST path=/43032026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=043042026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043052026/11/12 07:15:34 INFO Received push request method=POST path=/43062026/11/12 07:15:34 INFO Vacuumed table table=pending_closures43072026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts43082026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=043092026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=043102026/11/12 07:15:34 INFO Vacuumed table table=pending_objects43112026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads43122026/11/12 07:15:34 INFO Received complete push request method=POST path=/43132026/11/12 07:15:34 INFO Vacuumed table table=closures43142026/11/12 07:15:34 INFO Received push request method=POST path=/43152026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period43162026/11/12 07:15:34 INFO Vacuumed table table=objects43172026/11/12 07:15:34 INFO Completed push id=19 objects=3 duration=1.910679ms43182026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043192026/11/12 07:15:34 INFO Vacuumed table table=pending_closures43202026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts43212026/11/12 07:15:34 INFO Received complete upload request method=POST path=/43222026/11/12 07:15:34 INFO Vacuumed table table=pending_objects43232026/11/12 07:15:34 INFO Received push request method=POST path=/43242026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads43252026/11/12 07:15:34 INFO Completed upload id=29 objects=5 duration=1.857799ms43262026/11/12 07:15:34 INFO Vacuumed table table=closures43272026/11/12 07:15:34 INFO Vacuumed table table=objects43282026/11/12 07:15:34 INFO Received complete push request method=POST path=/43292026/11/12 07:15:34 INFO Completed push id=13 objects=2 duration=1.528219ms43302026/11/12 07:15:34 INFO Received push request method=POST path=/43312026/11/12 07:15:34 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."43322026/11/12 07:15:34 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."43332026/11/12 07:15:34 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."43342026/11/12 07:15:34 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."43352026/11/12 07:15:34 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."43362026/11/12 07:15:34 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."43372026/11/12 07:15:34 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."43382026/11/12 07:15:34 INFO Received complete push request method=POST path=/43392026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=043402026/11/12 07:15:34 INFO Completed push id=32 objects=7 duration=1.47593ms43412026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043422026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=043432026/11/12 07:15:34 INFO Vacuumed table table=pending_closures43442026/11/12 07:15:34 INFO Vacuumed table table=pending_closures43452026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts43462026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts43472026/11/12 07:15:34 INFO Vacuumed table table=pending_objects43482026/11/12 07:15:34 INFO Received complete upload request method=POST path=/43492026/11/12 07:15:34 INFO Received complete push request method=POST path=/43502026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads43512026/11/12 07:15:34 INFO Received uploads request method=POST path=/43522026/11/12 07:15:34 INFO Vacuumed table table=pending_objects43532026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads43542026/11/12 07:15:34 INFO Vacuumed table table=closures43552026/11/12 07:15:34 INFO Completed upload id=27 objects=8 duration=1.265669ms43562026/11/12 07:15:34 INFO Received push request method=POST path=/43572026/11/12 07:15:34 INFO Vacuumed table table=closures43582026/11/12 07:15:34 INFO Vacuumed table table=objects43592026/11/12 07:15:34 INFO Completed push id=26 objects=5 duration=2.428639ms43602026/11/12 07:15:34 INFO Vacuumed table table=objects43612026/11/12 07:15:34 INFO Received complete upload request method=POST path=/43622026/11/12 07:15:34 INFO Completed upload id=27 objects=13 duration=1.258449ms43632026/11/12 07:15:34 INFO Received complete push request method=POST path=/43642026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=043652026/11/12 07:15:34 INFO Completed push id=18 objects=6 duration=3.358428ms43662026/11/12 07:15:34 INFO Vacuumed table table=pending_closures43672026/11/12 07:15:34 INFO Received complete push request method=POST path=/43682026/11/12 07:15:34 INFO Received uploads request method=POST path=/43692026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts43702026/11/12 07:15:34 INFO Completed push id=20 objects=4 duration=1.438259ms43712026/11/12 07:15:34 INFO Received complete push request method=POST path=/43722026/11/12 07:15:34 INFO Vacuumed table table=pending_objects43732026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads43742026/11/12 07:15:34 INFO Completed push id=26 objects=4 duration=1.24211ms43752026/11/12 07:15:34 INFO Vacuumed table table=closures43762026/11/12 07:15:34 INFO Vacuumed table table=objects43772026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=043782026/11/12 07:15:34 INFO Received uploads request method=POST path=/43792026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period43802026/11/12 07:15:34 INFO Received push request method=POST path=/43812026/11/12 07:15:34 INFO Received push request method=POST path=/43822026/11/12 07:15:34 INFO Received complete upload request method=POST path=/43832026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043842026/11/12 07:15:34 INFO Completed upload id=24 objects=14 duration=1.30688ms43852026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=043862026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=043872026/11/12 07:15:34 INFO Received push request method=POST path=/43882026/11/12 07:15:34 INFO Vacuumed table table=pending_closures43892026/11/12 07:15:34 INFO Received complete upload request method=POST path=/43902026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts43912026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period43922026/11/12 07:15:34 INFO Vacuumed table table=pending_objects43932026/11/12 07:15:34 INFO Completed upload id=21 objects=3 duration=2.537829ms43942026/11/12 07:15:34 INFO Received complete upload request method=POST path=/43952026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads43962026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043972026/11/12 07:15:34 INFO Completed upload id=23 objects=9 duration=1.08626ms43982026/11/12 07:15:34 INFO Vacuumed table table=closures43992026/11/12 07:15:34 INFO Vacuumed table table=pending_closures44002026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044012026/11/12 07:15:34 INFO Received uploads request method=POST path=/44022026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts44032026/11/12 07:15:34 INFO Vacuumed table table=objects44042026/11/12 07:15:34 INFO Vacuumed table table=pending_closures44052026/11/12 07:15:34 INFO Vacuumed table table=pending_objects44062026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts44072026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads44082026/11/12 07:15:34 INFO Vacuumed table table=closures44092026/11/12 07:15:34 INFO Vacuumed table table=pending_objects44102026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads44112026/11/12 07:15:34 INFO Vacuumed table table=objects44122026/11/12 07:15:34 INFO Vacuumed table table=closures44132026/11/12 07:15:34 INFO Vacuumed table table=objects44142026/11/12 07:15:34 INFO Received complete upload request method=POST path=/44152026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=044162026/11/12 07:15:34 INFO Completed upload id=28 objects=13 duration=1.26537ms44172026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period44182026/11/12 07:15:34 INFO Received push request method=POST path=/44192026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044202026/11/12 07:15:34 INFO Received push request method=POST path=/44212026/11/12 07:15:34 INFO Vacuumed table table=pending_closures44222026/11/12 07:15:34 INFO Received complete upload request method=POST path=/44232026/11/12 07:15:34 INFO Received complete push request method=POST path=/44242026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts44252026/11/12 07:15:34 INFO Completed upload id=27 objects=4 duration=1.318469ms44262026/11/12 07:15:34 INFO Vacuumed table table=pending_objects44272026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=044282026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads44292026/11/12 07:15:34 INFO Completed push id=19 objects=3 duration=1.572489ms44302026/11/12 07:15:34 INFO Vacuumed table table=closures44312026/11/12 07:15:34 INFO Vacuumed table table=objects44322026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=044332026/11/12 07:15:34 INFO Received complete upload request method=POST path=/44342026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period44352026/11/12 07:15:34 INFO Completed upload id=22 objects=8 duration=1.197879ms44362026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044372026/11/12 07:15:34 INFO Vacuumed table table=pending_closures44382026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts44392026/11/12 07:15:34 INFO Received uploads request method=POST path=/44402026/11/12 07:15:34 INFO Received complete push request method=POST path=/44412026/11/12 07:15:34 INFO Vacuumed table table=pending_objects44422026/11/12 07:15:34 INFO Received push request method=POST path=/44432026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads44442026/11/12 07:15:34 INFO Vacuumed table table=closures44452026/11/12 07:15:34 INFO Vacuumed table table=objects44462026/11/12 07:15:34 INFO Completed push id=27 objects=25 duration=3.734138ms44472026/11/12 07:15:34 INFO Received uploads request method=POST path=/44482026/11/12 07:15:34 INFO Received complete push request method=POST path=/44492026/11/12 07:15:34 INFO Received uploads request method=POST path=/44502026/11/12 07:15:34 INFO Completed push id=29 objects=3 duration=1.545049ms44512026/11/12 07:15:34 INFO Received uploads request method=POST path=/44522026/11/12 07:15:34 INFO Received push request method=POST path=/44532026/11/12 07:15:34 INFO Received complete push request method=POST path=/44542026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=7 objects-failed-to-delete=044552026/11/12 07:15:34 INFO Received complete upload request method=POST path=/44562026/11/12 07:15:34 INFO Vacuumed table table=pending_closures44572026/11/12 07:15:34 INFO Completed push id=13 objects=11 duration=2.285888ms44582026/11/12 07:15:34 INFO Completed upload id=15 objects=3 duration=2.495398ms44592026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts44602026/11/12 07:15:34 INFO Received complete push request method=POST path=/44612026/11/12 07:15:34 INFO Vacuumed table table=pending_objects44622026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads44632026/11/12 07:15:34 INFO Completed push id=23 objects=5 duration=1.32065ms44642026/11/12 07:15:34 INFO Vacuumed table table=closures44652026/11/12 07:15:34 INFO Vacuumed table table=objects44662026/11/12 07:15:34 INFO Received push request method=POST path=/44672026/11/12 07:15:34 INFO Received uploads request method=POST path=/44682026/11/12 07:15:34 INFO Received push request method=POST path=/44692026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=044702026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=044712026/11/12 07:15:34 INFO Received push request method=POST path=/44722026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044732026/11/12 07:15:34 INFO Received complete upload request method=POST path=/44742026/11/12 07:15:34 INFO Vacuumed table table=pending_closures44752026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044762026/11/12 07:15:34 INFO Completed upload id=31 objects=2 duration=1.2452ms44772026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts44782026/11/12 07:15:34 INFO Vacuumed table table=pending_closures44792026/11/12 07:15:34 INFO Received uploads request method=POST path=/44802026/11/12 07:15:34 INFO Vacuumed table table=pending_objects44812026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts44822026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads44832026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=044842026/11/12 07:15:34 INFO Vacuumed table table=closures44852026/11/12 07:15:34 INFO Vacuumed table table=pending_objects44862026/11/12 07:15:34 INFO Vacuumed table table=objects44872026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads44882026/11/12 07:15:34 INFO Received uploads request method=POST path=/44892026/11/12 07:15:34 INFO Vacuumed table table=closures44902026/11/12 07:15:34 INFO Received uploads request method=POST path=/44912026/11/12 07:15:34 INFO Vacuumed table table=objects44922026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044932026/11/12 07:15:34 INFO Vacuumed table table=pending_closures44942026/11/12 07:15:34 INFO Received complete upload request method=POST path=/44952026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts44962026/11/12 07:15:34 INFO Completed upload id=24 objects=8 duration=1.119039ms44972026/11/12 07:15:34 INFO Vacuumed table table=pending_objects44982026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads44992026/11/12 07:15:34 INFO Vacuumed table table=closures45002026/11/12 07:15:34 INFO Vacuumed table table=objects45012026/11/12 07:15:34 INFO Received uploads request method=POST path=/45022026/11/12 07:15:34 INFO Received uploads request method=POST path=/45032026/11/12 07:15:34 INFO Received uploads request method=POST path=/45042026/11/12 07:15:34 INFO Received uploads request method=POST path=/45052026/11/12 07:15:34 INFO Received push request method=POST path=/45062026/11/12 07:15:34 INFO Received push request method=POST path=/45072026/11/12 07:15:34 INFO Received push request method=POST path=/45082026/11/12 07:15:34 INFO Received push request method=POST path=/45092026/11/12 07:15:34 INFO Received complete push request method=POST path=/45102026/11/12 07:15:34 INFO Completed push id=33 objects=6 duration=1.372009ms45112026/11/12 07:15:34 INFO Received complete upload request method=POST path=/45122026/11/12 07:15:34 INFO Completed upload id=25 objects=16 duration=2.749999ms45132026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=045142026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=045152026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period45162026/11/12 07:15:34 INFO Received complete push request method=POST path=/45172026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period45182026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045192026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045202026/11/12 07:15:34 INFO Vacuumed table table=pending_closures45212026/11/12 07:15:34 INFO Vacuumed table table=pending_closures45222026/11/12 07:15:34 INFO Received complete upload request method=POST path=/45232026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts45242026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts45252026/11/12 07:15:34 INFO Completed push id=24 objects=5 duration=3.534058ms45262026/11/12 07:15:34 INFO Vacuumed table table=pending_objects45272026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads45282026/11/12 07:15:34 INFO Completed upload id=28 objects=5 duration=1.188969ms45292026/11/12 07:15:34 INFO Vacuumed table table=pending_objects45302026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads45312026/11/12 07:15:34 INFO Vacuumed table table=closures45322026/11/12 07:15:34 INFO Vacuumed table table=objects45332026/11/12 07:15:34 INFO Vacuumed table table=closures45342026/11/12 07:15:34 INFO Vacuumed table table=objects45352026/11/12 07:15:34 INFO Received push request method=POST path=/45362026/11/12 07:15:34 INFO Received push request method=POST path=/45372026/11/12 07:15:34 INFO Received complete push request method=POST path=/45382026/11/12 07:15:34 INFO Completed push id=31 objects=14 duration=3.254598ms45392026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=045402026/11/12 07:15:34 INFO Received uploads request method=POST path=/45412026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period45422026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=045432026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045442026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period45452026/11/12 07:15:34 INFO Received uploads request method=POST path=/45462026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045472026/11/12 07:15:34 INFO Vacuumed table table=pending_closures45482026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts45492026/11/12 07:15:34 INFO Vacuumed table table=pending_closures45502026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts45512026/11/12 07:15:34 INFO Vacuumed table table=pending_objects45522026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads45532026/11/12 07:15:34 INFO Vacuumed table table=pending_objects45542026/11/12 07:15:34 INFO Vacuumed table table=closures45552026/11/12 07:15:34 INFO Received push request method=POST path=/45562026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads45572026/11/12 07:15:34 INFO Vacuumed table table=objects45582026/11/12 07:15:34 INFO Vacuumed table table=closures45592026/11/12 07:15:34 INFO Vacuumed table table=objects45602026/11/12 07:15:34 INFO Received push request method=POST path=/45612026/11/12 07:15:34 INFO Received complete upload request method=POST path=/45622026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=045632026/11/12 07:15:34 INFO Received complete upload request method=POST path=/45642026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=045652026/11/12 07:15:34 INFO Completed upload id=30 objects=5 duration=1.227899ms45662026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period45672026/11/12 07:15:34 INFO Completed upload id=31 objects=3 duration=2.585739ms45682026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045692026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045702026/11/12 07:15:34 INFO Vacuumed table table=pending_closures45712026/11/12 07:15:34 INFO Vacuumed table table=pending_closures45722026/11/12 07:15:34 INFO Received uploads request method=POST path=/45732026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts45742026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts45752026/11/12 07:15:34 INFO Vacuumed table table=pending_objects45762026/11/12 07:15:34 INFO Vacuumed table table=pending_objects45772026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads45782026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads45792026/11/12 07:15:34 INFO Vacuumed table table=closures45802026/11/12 07:15:34 INFO Vacuumed table table=closures45812026/11/12 07:15:34 INFO Vacuumed table table=objects45822026/11/12 07:15:34 INFO Vacuumed table table=objects45832026/11/12 07:15:34 INFO Received uploads request method=POST path=/45842026/11/12 07:15:34 INFO Received complete upload request method=POST path=/45852026/11/12 07:15:34 INFO Completed upload id=25 objects=4 duration=1.535989ms45862026/11/12 07:15:34 INFO Received complete push request method=POST path=/45872026/11/12 07:15:34 INFO Received uploads request method=POST path=/45882026/11/12 07:15:34 INFO Completed push id=22 objects=3 duration=3.278748ms45892026/11/12 07:15:34 INFO Received push request method=POST path=/45902026/11/12 07:15:34 INFO Received complete upload request method=POST path=/45912026/11/12 07:15:34 INFO Received complete push request method=POST path=/45922026/11/12 07:15:34 INFO Completed upload id=22 objects=2 duration=1.384869ms45932026/11/12 07:15:34 INFO Received uploads request method=POST path=/45942026/11/12 07:15:34 INFO Completed push id=17 objects=22 duration=3.017138ms45952026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=045962026/11/12 07:15:34 INFO Received complete push request method=POST path=/45972026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period45982026/11/12 07:15:34 INFO Completed push id=31 objects=3 duration=1.276739ms45992026/11/12 07:15:34 INFO Received complete upload request method=POST path=/46002026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046012026/11/12 07:15:34 INFO Vacuumed table table=pending_closures46022026/11/12 07:15:34 INFO Completed upload id=13 objects=14 duration=2.263759ms46032026/11/12 07:15:34 INFO Received complete upload request method=POST path=/46042026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts46052026/11/12 07:15:34 INFO Received uploads request method=POST path=/46062026/11/12 07:15:34 INFO Vacuumed table table=pending_objects46072026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads46082026/11/12 07:15:34 INFO Received push request method=POST path=/46092026/11/12 07:15:34 INFO Vacuumed table table=closures46102026/11/12 07:15:34 INFO Completed upload id=29 objects=4 duration=2.437348ms46112026/11/12 07:15:34 INFO Vacuumed table table=objects46122026/11/12 07:15:34 INFO Received push request method=POST path=/46132026/11/12 07:15:34 INFO Received complete push request method=POST path=/46142026/11/12 07:15:34 INFO Received complete push request method=POST path=/46152026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=046162026/11/12 07:15:34 INFO Completed push id=16 objects=26 duration=1.516389ms46172026/11/12 07:15:34 INFO Completed push id=16 objects=7 duration=2.133859ms46182026/11/12 07:15:34 INFO Received complete push request method=POST path=/46192026/11/12 07:15:34 INFO Received uploads request method=POST path=/46202026/11/12 07:15:34 INFO Received uploads request method=POST path=/46212026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period46222026/11/12 07:15:34 INFO Completed push id=16 objects=14 duration=2.906739ms46232026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046242026/11/12 07:15:34 INFO Received complete push request method=POST path=/46252026/11/12 07:15:34 INFO Vacuumed table table=pending_closures46262026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts46272026/11/12 07:15:34 INFO Completed push id=21 objects=26 duration=1.869499ms46282026/11/12 07:15:34 INFO Vacuumed table table=pending_objects46292026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=046302026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads46312026/11/12 07:15:34 INFO Vacuumed table table=closures46322026/11/12 07:15:34 INFO Vacuumed table table=objects46332026/11/12 07:15:34 INFO Received complete upload request method=POST path=/46342026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046352026/11/12 07:15:34 INFO Received uploads request method=POST path=/46362026/11/12 07:15:34 INFO Vacuumed table table=pending_closures46372026/11/12 07:15:34 INFO Completed upload id=19 objects=2 duration=2.565379ms46382026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts46392026/11/12 07:15:34 INFO Vacuumed table table=pending_objects46402026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads46412026/11/12 07:15:34 INFO Vacuumed table table=closures46422026/11/12 07:15:34 INFO Vacuumed table table=objects46432026/11/12 07:15:34 INFO Received complete push request method=POST path=/46442026/11/12 07:15:34 INFO Completed push id=32 objects=6 duration=1.201529ms46452026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=046462026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period46472026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=046482026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period46492026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046502026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046512026/11/12 07:15:34 INFO Vacuumed table table=pending_closures46522026/11/12 07:15:34 INFO Vacuumed table table=pending_closures46532026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts46542026/11/12 07:15:34 INFO Received complete push request method=POST path=/46552026/11/12 07:15:34 INFO Received push request method=POST path=/46562026/11/12 07:15:34 INFO Received complete upload request method=POST path=/46572026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts46582026/11/12 07:15:34 INFO Received uploads request method=POST path=/46592026/11/12 07:15:34 INFO Vacuumed table table=pending_objects46602026/11/12 07:15:34 INFO Vacuumed table table=pending_objects46612026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads46622026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads46632026/11/12 07:15:34 INFO Completed push id=23 objects=6 duration=2.038419ms46642026/11/12 07:15:34 INFO Vacuumed table table=closures46652026/11/12 07:15:34 INFO Vacuumed table table=closures46662026/11/12 07:15:34 INFO Received uploads request method=POST path=/46672026/11/12 07:15:34 INFO Completed upload id=18 objects=2 duration=3.183169ms46682026/11/12 07:15:34 INFO Received complete push request method=POST path=/46692026/11/12 07:15:34 INFO Vacuumed table table=objects46702026/11/12 07:15:34 INFO Vacuumed table table=objects46712026/11/12 07:15:34 INFO Received complete upload request method=POST path=/46722026/11/12 07:15:34 INFO Received complete push request method=POST path=/46732026/11/12 07:15:34 INFO Completed push id=7 objects=7 duration=1.731289ms46742026/11/12 07:15:34 INFO Completed push id=20 objects=8 duration=1.377059ms46752026/11/12 07:15:34 INFO Completed upload id=28 objects=17 duration=1.494019ms46762026/11/12 07:15:34 INFO Received complete upload request method=POST path=/46772026/11/12 07:15:34 INFO Completed upload id=33 objects=7 duration=1.08354ms46782026/11/12 07:15:34 INFO Received complete push request method=POST path=/46792026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=046802026/11/12 07:15:34 INFO Completed push id=31 objects=6 duration=1.820799ms46812026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046822026/11/12 07:15:34 INFO Vacuumed table table=pending_closures46832026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts46842026/11/12 07:15:34 INFO Received complete push request method=POST path=/46852026/11/12 07:15:34 INFO Completed push id=24 objects=3 duration=1.349389ms46862026/11/12 07:15:34 INFO Vacuumed table table=pending_objects46872026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads46882026/11/12 07:15:34 INFO Vacuumed table table=closures46892026/11/12 07:15:34 INFO Vacuumed table table=objects46902026/11/12 07:15:34 INFO Received push request method=POST path=/46912026/11/12 07:15:34 INFO Received complete push request method=POST path=/46922026/11/12 07:15:34 INFO Received complete push request method=POST path=/46932026/11/12 07:15:34 INFO Completed push id=24 objects=6 duration=1.34632ms46942026/11/12 07:15:34 INFO Completed push id=33 objects=7 duration=2.457989ms46952026/11/12 07:15:34 INFO Received complete push request method=POST path=/46962026/11/12 07:15:34 INFO Received complete push request method=POST path=/46972026/11/12 07:15:34 INFO Received complete push request method=POST path=/46982026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=046992026/11/12 07:15:34 INFO Received uploads request method=POST path=/47002026/11/12 07:15:34 INFO Completed push id=12 objects=6 duration=1.436439ms47012026/11/12 07:15:34 INFO Completed push id=29 objects=2 duration=1.567149ms47022026/11/12 07:15:34 INFO Received uploads request method=POST path=/47032026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047042026/11/12 07:15:34 INFO Received complete upload request method=POST path=/47052026/11/12 07:15:34 INFO Vacuumed table table=pending_closures47062026/11/12 07:15:34 INFO Completed push id=27 objects=6 duration=3.526808ms47072026/11/12 07:15:34 INFO Completed upload id=34 objects=3 duration=1.148939ms47082026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts47092026/11/12 07:15:34 INFO Received uploads request method=POST path=/47102026/11/12 07:15:34 INFO Vacuumed table table=pending_objects47112026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads47122026/11/12 07:15:34 INFO Received complete push request method=POST path=/47132026/11/12 07:15:34 INFO Vacuumed table table=closures47142026/11/12 07:15:34 INFO Vacuumed table table=objects47152026/11/12 07:15:34 INFO Completed push id=11 objects=6 duration=2.117339ms47162026/11/12 07:15:34 INFO Received push request method=POST path=/47172026/11/12 07:15:34 INFO Received complete upload request method=POST path=/47182026/11/12 07:15:34 INFO Completed upload id=26 objects=3 duration=1.21026ms47192026/11/12 07:15:34 INFO Received push request method=POST path=/47202026/11/12 07:15:34 INFO Received complete push request method=POST path=/47212026/11/12 07:15:34 INFO Completed push id=31 objects=6 duration=1.724039ms47222026/11/12 07:15:34 INFO Received complete upload request method=POST path=/47232026/11/12 07:15:34 INFO Received complete push request method=POST path=/47242026/11/12 07:15:34 INFO Received push request method=POST path=/47252026/11/12 07:15:34 INFO Completed upload id=22 objects=5 duration=2.193278ms47262026/11/12 07:15:34 INFO Completed push id=35 objects=11 duration=1.458599ms47272026/11/12 07:15:34 INFO Received push request method=POST path=/47282026/11/12 07:15:34 INFO Received complete push request method=POST path=/47292026/11/12 07:15:34 INFO Completed push id=25 objects=7 duration=1.405289ms47302026/11/12 07:15:34 INFO Received complete upload request method=POST path=/47312026/11/12 07:15:34 INFO Completed upload id=13 objects=7 duration=1.444179ms47322026/11/12 07:15:34 INFO Received complete upload request method=POST path=/47332026/11/12 07:15:34 INFO Received complete push request method=POST path=/47342026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=047352026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period47362026/11/12 07:15:34 INFO Completed push id=20 objects=13 duration=1.743719ms47372026/11/12 07:15:34 INFO Completed upload id=33 objects=5 duration=2.994279ms47382026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047392026/11/12 07:15:34 INFO Vacuumed table table=pending_closures47402026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts47412026/11/12 07:15:34 INFO Vacuumed table table=pending_objects47422026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads47432026/11/12 07:15:34 INFO Vacuumed table table=closures47442026/11/12 07:15:34 INFO Vacuumed table table=objects47452026/11/12 07:15:34 INFO Received complete upload request method=POST path=/47462026/11/12 07:15:34 INFO Received complete push request method=POST path=/47472026/11/12 07:15:34 INFO Completed upload id=15 objects=2 duration=1.342269ms47482026/11/12 07:15:34 INFO Completed push id=32 objects=9 duration=1.420029ms47492026/11/12 07:15:34 INFO Received complete push request method=POST path=/47502026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=047512026/11/12 07:15:34 INFO Received complete push request method=POST path=/47522026/11/12 07:15:34 INFO Received uploads request method=POST path=/47532026/11/12 07:15:34 INFO Completed push id=28 objects=3 duration=1.823479ms47542026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=047552026/11/12 07:15:34 INFO Completed push id=19 objects=14 duration=2.361948ms47562026/11/12 07:15:34 INFO Received complete upload request method=POST path=/47572026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period47582026/11/12 07:15:34 INFO Received complete push request method=POST path=/47592026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period47602026/11/12 07:15:34 INFO Completed upload id=20 objects=4 duration=1.590889ms47612026/11/12 07:15:34 INFO Completed push id=13 objects=22 duration=1.441239ms47622026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=6 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047632026/11/12 07:15:34 INFO Received complete push request method=POST path=/47642026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047652026/11/12 07:15:34 INFO Completed push id=22 objects=4 duration=1.336479ms47662026/11/12 07:15:34 INFO Vacuumed table table=pending_closures47672026/11/12 07:15:34 INFO Vacuumed table table=pending_closures47682026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts47692026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts47702026/11/12 07:15:34 INFO Vacuumed table table=pending_objects47712026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads47722026/11/12 07:15:34 INFO Vacuumed table table=pending_objects47732026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads47742026/11/12 07:15:34 INFO Vacuumed table table=closures47752026/11/12 07:15:34 INFO Vacuumed table table=closures47762026/11/12 07:15:34 INFO Vacuumed table table=objects47772026/11/12 07:15:34 INFO Vacuumed table table=objects47782026/11/12 07:15:34 INFO Received complete push request method=POST path=/47792026/11/12 07:15:34 INFO Completed push id=27 objects=11 duration=1.491739ms47802026/11/12 07:15:34 INFO Received complete push request method=POST path=/47812026/11/12 07:15:34 INFO Received complete upload request method=POST path=/47822026/11/12 07:15:34 INFO Received uploads request method=POST path=/47832026/11/12 07:15:34 INFO Completed push id=29 objects=15 duration=1.409179ms47842026/11/12 07:15:34 INFO Received complete upload request method=POST path=/47852026/11/12 07:15:34 INFO Completed upload id=25 objects=6 duration=2.565078ms47862026/11/12 07:15:34 INFO Received complete upload request method=POST path=/47872026/11/12 07:15:34 INFO Received complete push request method=POST path=/47882026/11/12 07:15:34 INFO Received complete upload request method=POST path=/47892026/11/12 07:15:34 INFO Completed upload id=23 objects=7 duration=1.620299ms47902026/11/12 07:15:34 INFO Completed push id=21 objects=6 duration=1.21841ms47912026/11/12 07:15:34 INFO Completed upload id=33 objects=9 duration=1.179079ms47922026/11/12 07:15:34 INFO Completed upload id=26 objects=24 duration=1.858509ms47932026/11/12 07:15:34 INFO Received complete upload request method=POST path=/47942026/11/12 07:15:34 INFO Completed upload id=23 objects=7 duration=1.250349ms47952026/11/12 07:15:34 INFO Received complete push request method=POST path=/47962026/11/12 07:15:34 INFO Completed push id=1 objects=9 duration=1.358549ms47972026/11/12 07:15:34 INFO Received complete upload request method=POST path=/47982026/11/12 07:15:34 INFO Completed upload id=29 objects=26 duration=1.20452ms47992026/11/12 07:15:34 INFO Received complete push request method=POST path=/48002026/11/12 07:15:34 INFO Completed push id=30 objects=6 duration=1.09068ms48012026/11/12 07:15:34 INFO Received complete upload request method=POST path=/48022026/11/12 07:15:34 INFO Received complete upload request method=POST path=/48032026/11/12 07:15:34 INFO Completed upload id=22 objects=7 duration=1.016409ms48042026/11/12 07:15:34 INFO Completed upload id=21 objects=25 duration=1.940318ms48052026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=048062026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period48072026/11/12 07:15:34 INFO Received complete push request method=POST path=/48082026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048092026/11/12 07:15:34 INFO Completed push id=24 objects=5 duration=1.436719ms48102026/11/12 07:15:34 INFO Received complete push request method=POST path=/48112026/11/12 07:15:34 INFO Received complete upload request method=POST path=/48122026/11/12 07:15:34 INFO Vacuumed table table=pending_closures48132026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts48142026/11/12 07:15:34 INFO Completed upload id=20 objects=6 duration=1.402359ms48152026/11/12 07:15:34 INFO Completed push id=9 objects=9 duration=3.448088ms48162026/11/12 07:15:34 INFO Vacuumed table table=pending_objects48172026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads48182026/11/12 07:15:34 INFO Vacuumed table table=closures48192026/11/12 07:15:34 INFO Received complete push request method=POST path=/48202026/11/12 07:15:34 INFO Vacuumed table table=objects48212026/11/12 07:15:34 INFO Completed push id=30 objects=8 duration=1.33185ms48222026/11/12 07:15:34 INFO Received push request method=POST path=/48232026/11/12 07:15:34 INFO Received complete upload request method=POST path=/48242026/11/12 07:15:34 INFO Completed upload id=31 objects=3 duration=1.420369ms48252026/11/12 07:15:34 INFO Received complete push request method=POST path=/48262026/11/12 07:15:34 INFO Completed push id=24 objects=4 duration=1.055169ms48272026/11/12 07:15:34 INFO Received uploads request method=POST path=/48282026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=048292026/11/12 07:15:34 INFO Received complete upload request method=POST path=/48302026/11/12 07:15:34 INFO Received complete upload request method=POST path=/48312026/11/12 07:15:34 INFO Completed upload id=27 objects=26 duration=1.188569ms48322026/11/12 07:15:34 INFO Completed upload id=25 objects=2 duration=1.07917ms48332026/11/12 07:15:34 INFO Received complete upload request method=POST path=/48342026/11/12 07:15:34 INFO Completed upload id=24 objects=18 duration=1.542989ms48352026/11/12 07:15:34 INFO Received push request method=POST path=/48362026/11/12 07:15:34 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=048372026/11/12 07:15:34 INFO Received complete upload request method=POST path=/48382026/11/12 07:15:34 INFO Received complete upload request method=POST path=/48392026/11/12 07:15:34 INFO Completed upload id=21 objects=8 duration=1.371219ms48402026/11/12 07:15:34 INFO Completed upload id=31 objects=5 duration=1.205069ms48412026/11/12 07:15:34 INFO Vacuumed table table=pending_closures48422026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts48432026/11/12 07:15:34 INFO Received uploads request method=POST path=/48442026/11/12 07:15:34 INFO Vacuumed table table=pending_objects48452026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads48462026/11/12 07:15:34 INFO Aborted multipart uploads count=0 kept=048472026/11/12 07:15:34 INFO Vacuumed table table=closures48482026/11/12 07:15:34 INFO Received complete push request method=POST path=/48492026/11/12 07:15:34 INFO Vacuumed table table=objects48502026/11/12 07:15:34 INFO Completed push id=25 objects=9 duration=1.299239ms48512026/11/12 07:15:34 WARN Force mode enabled - objects will be deleted immediately without grace period48522026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048532026/11/12 07:15:35 INFO Received complete push request method=POST path=/48542026/11/12 07:15:35 INFO Vacuumed table table=pending_closures48552026/11/12 07:15:35 INFO Completed push id=28 objects=6 duration=1.492359ms48562026/11/12 07:15:35 INFO Received complete upload request method=POST path=/48572026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts48582026/11/12 07:15:35 INFO Completed upload id=26 objects=20 duration=1.37075ms48592026/11/12 07:15:35 INFO Vacuumed table table=pending_objects48602026/11/12 07:15:35 INFO Received complete push request method=POST path=/48612026/11/12 07:15:35 INFO Received uploads request method=POST path=/48622026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads48632026/11/12 07:15:35 INFO Vacuumed table table=closures48642026/11/12 07:15:35 INFO Received complete upload request method=POST path=/48652026/11/12 07:15:35 INFO Completed push id=25 objects=2 duration=2.158649ms48662026/11/12 07:15:35 INFO Received complete push request method=POST path=/48672026/11/12 07:15:35 INFO Vacuumed table table=objects48682026/11/12 07:15:35 INFO Completed upload id=2 objects=7 duration=1.091889ms48692026/11/12 07:15:35 INFO Completed push id=22 objects=6 duration=1.431879ms48702026/11/12 07:15:35 INFO Received uploads request method=POST path=/48712026/11/12 07:15:35 INFO Received complete push request method=POST path=/48722026/11/12 07:15:35 INFO Completed push id=32 objects=3 duration=1.059559ms48732026/11/12 07:15:35 INFO Received complete push request method=POST path=/48742026/11/12 07:15:35 INFO Completed push id=11 objects=9 duration=1.738619ms48752026/11/12 07:15:35 INFO Received complete upload request method=POST path=/48762026/11/12 07:15:35 INFO Completed upload id=26 objects=3 duration=1.11842ms48772026/11/12 07:15:35 INFO Received complete push request method=POST path=/48782026/11/12 07:15:35 INFO Completed push id=29 objects=6 duration=1.09742ms48792026/11/12 07:15:35 INFO Received complete upload request method=POST path=/48802026/11/12 07:15:35 INFO Completed upload id=12 objects=6 duration=1.43252ms48812026/11/12 07:15:35 INFO Received complete push request method=POST path=/48822026/11/12 07:15:35 INFO Completed push id=27 objects=4 duration=1.04801ms48832026/11/12 07:15:35 INFO Received complete push request method=POST path=/48842026/11/12 07:15:35 INFO Completed push id=24 objects=8 duration=1.233659ms48852026/11/12 07:15:35 INFO Received complete push request method=POST path=/48862026/11/12 07:15:35 INFO Completed push id=28 objects=7 duration=1.277399ms48872026/11/12 07:15:35 INFO Received uploads request method=POST path=/48882026/11/12 07:15:35 INFO Received uploads request method=POST path=/48892026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=048902026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period48912026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048922026/11/12 07:15:35 INFO Received complete push request method=POST path=/48932026/11/12 07:15:35 INFO Vacuumed table table=pending_closures48942026/11/12 07:15:35 INFO Completed push id=15 objects=24 duration=1.721829ms48952026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts48962026/11/12 07:15:35 INFO Received complete upload request method=POST path=/48972026/11/12 07:15:35 INFO Received complete push request method=POST path=/48982026/11/12 07:15:35 INFO Completed upload id=27 objects=7 duration=987.869µs48992026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49002026/11/12 07:15:35 INFO Vacuumed table table=pending_objects49012026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads49022026/11/12 07:15:35 INFO Completed push id=17 objects=3 duration=1.924199ms49032026/11/12 07:15:35 INFO Completed upload id=25 objects=9 duration=1.146249ms49042026/11/12 07:15:35 INFO Vacuumed table table=closures49052026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49062026/11/12 07:15:35 INFO Vacuumed table table=objects49072026/11/12 07:15:35 INFO Completed upload id=21 objects=2 duration=1.033619ms49082026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49092026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49102026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49112026/11/12 07:15:35 INFO Completed upload id=26 objects=8 duration=1.186409ms49122026/11/12 07:15:35 INFO Completed upload id=8 objects=6 duration=824.87µs49132026/11/12 07:15:35 INFO Completed upload id=26 objects=2 duration=2.254098ms49142026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49152026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49162026/11/12 07:15:35 INFO Completed upload id=30 objects=6 duration=1.10054ms49172026/11/12 07:15:35 INFO Completed upload id=30 objects=8 duration=1.12732ms49182026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49192026/11/12 07:15:35 INFO Completed upload id=19 objects=2 duration=1.054979ms49202026/11/12 07:15:35 INFO Received complete push request method=POST path=/49212026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49222026/11/12 07:15:35 INFO Completed push id=28 objects=26 duration=1.74641ms49232026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49242026/11/12 07:15:35 INFO Completed upload id=27 objects=3 duration=1.194039ms49252026/11/12 07:15:35 INFO Completed upload id=28 objects=3 duration=1.19194ms49262026/11/12 07:15:35 INFO Received uploads request method=POST path=/49272026/11/12 07:15:35 INFO Received complete push request method=POST path=/49282026/11/12 07:15:35 INFO Completed push id=23 objects=5 duration=1.5716ms49292026/11/12 07:15:35 INFO Received complete push request method=POST path=/49302026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=049312026/11/12 07:15:35 INFO Completed push id=28 objects=14 duration=1.479849ms49322026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049332026/11/12 07:15:35 INFO Received complete push request method=POST path=/49342026/11/12 07:15:35 INFO Vacuumed table table=pending_closures49352026/11/12 07:15:35 INFO Completed push id=31 objects=5 duration=1.301729ms49362026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts49372026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49382026/11/12 07:15:35 INFO Vacuumed table table=pending_objects49392026/11/12 07:15:35 INFO Completed upload id=24 objects=7 duration=1.54099ms49402026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads49412026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49422026/11/12 07:15:35 INFO Received complete push request method=POST path=/49432026/11/12 07:15:35 INFO Vacuumed table table=closures49442026/11/12 07:15:35 INFO Completed upload id=28 objects=6 duration=1.07795ms49452026/11/12 07:15:35 INFO Completed push id=30 objects=3 duration=1.479869ms49462026/11/12 07:15:35 INFO Vacuumed table table=objects49472026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49482026/11/12 07:15:35 INFO Completed upload id=31 objects=6 duration=1.101849ms49492026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=049502026/11/12 07:15:35 INFO Received complete push request method=POST path=/49512026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49522026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period49532026/11/12 07:15:35 INFO Completed upload id=24 objects=3 duration=1.223749ms49542026/11/12 07:15:35 INFO Completed push id=29 objects=23 duration=1.846059ms49552026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049562026/11/12 07:15:35 INFO Received push request method=POST path=/49572026/11/12 07:15:35 INFO Vacuumed table table=pending_closures49582026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts49592026/11/12 07:15:35 INFO Vacuumed table table=pending_objects49602026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads49612026/11/12 07:15:35 INFO Vacuumed table table=closures49622026/11/12 07:15:35 INFO Received complete push request method=POST path=/49632026/11/12 07:15:35 INFO Vacuumed table table=objects49642026/11/12 07:15:35 INFO Received complete push request method=POST path=/49652026/11/12 07:15:35 INFO Completed push id=26 objects=3 duration=1.30034ms49662026/11/12 07:15:35 INFO Completed push id=30 objects=4 duration=1.186839ms49672026/11/12 07:15:35 INFO Received complete push request method=POST path=/49682026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49692026/11/12 07:15:35 INFO Completed push id=32 objects=19 duration=1.46719ms49702026/11/12 07:15:35 INFO Completed upload id=32 objects=4 duration=983.519µs49712026/11/12 07:15:35 INFO Received complete push request method=POST path=/49722026/11/12 07:15:35 INFO Received complete push request method=POST path=/49732026/11/12 07:15:35 INFO Received complete push request method=POST path=/49742026/11/12 07:15:35 INFO Completed push id=32 objects=26 duration=1.774009ms49752026/11/12 07:15:35 INFO Completed push id=25 objects=4 duration=1.707809ms49762026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49772026/11/12 07:15:35 INFO Completed push id=30 objects=6 duration=1.334609ms49782026/11/12 07:15:35 INFO Completed upload id=12 objects=9 duration=1.07749ms49792026/11/12 07:15:35 INFO Received uploads request method=POST path=/49802026/11/12 07:15:35 INFO Received complete push request method=POST path=/49812026/11/12 07:15:35 INFO Completed push id=31 objects=7 duration=1.17297ms49822026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49832026/11/12 07:15:35 INFO Received push request method=POST path=/49842026/11/12 07:15:35 INFO Completed upload id=28 objects=8 duration=1.45581ms49852026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=049862026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period49872026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049882026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49892026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49902026/11/12 07:15:35 INFO Vacuumed table table=pending_closures49912026/11/12 07:15:35 INFO Completed upload id=33 objects=2 duration=1.253549ms49922026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts49932026/11/12 07:15:35 INFO Completed upload id=33 objects=10 duration=1.547829ms49942026/11/12 07:15:35 INFO Vacuumed table table=pending_objects49952026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads49962026/11/12 07:15:35 INFO Received complete upload request method=POST path=/49972026/11/12 07:15:35 INFO Vacuumed table table=closures49982026/11/12 07:15:35 INFO Completed upload id=32 objects=5 duration=896.04µs49992026/11/12 07:15:35 INFO Vacuumed table table=objects50002026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=050012026/11/12 07:15:35 INFO Received complete upload request method=POST path=/50022026/11/12 07:15:35 INFO Received complete push request method=POST path=/50032026/11/12 07:15:35 INFO Completed upload id=29 objects=10 duration=1.053499ms50042026/11/12 07:15:35 INFO Received uploads request method=POST path=/50052026/11/12 07:15:35 INFO Completed push id=33 objects=3 duration=1.222769ms50062026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period50072026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050082026/11/12 07:15:35 INFO Vacuumed table table=pending_closures50092026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts50102026/11/12 07:15:35 INFO Received complete upload request method=POST path=/50112026/11/12 07:15:35 INFO Completed upload id=34 objects=5 duration=2.868238ms50122026/11/12 07:15:35 INFO Vacuumed table table=pending_objects50132026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads50142026/11/12 07:15:35 INFO Vacuumed table table=closures50152026/11/12 07:15:35 INFO Vacuumed table table=objects50162026/11/12 07:15:35 INFO Received complete upload request method=POST path=/50172026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=050182026/11/12 07:15:35 INFO Completed upload id=30 objects=20 duration=1.815029ms50192026/11/12 07:15:35 INFO Received complete push request method=POST path=/50202026/11/12 07:15:35 INFO Received push request method=POST path=/50212026/11/12 07:15:35 INFO Completed push id=34 objects=6 duration=1.200359ms50222026/11/12 07:15:35 INFO Received complete upload request method=POST path=/50232026/11/12 07:15:35 INFO Received complete upload request method=POST path=/50242026/11/12 07:15:35 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."50252026/11/12 07:15:35 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."50262026/11/12 07:15:35 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."50272026/11/12 07:15:35 INFO Completed upload id=34 objects=23 duration=7.597876ms50282026/11/12 07:15:35 INFO Received uploads request method=POST path=/50292026/11/12 07:15:35 INFO Received complete push request method=POST path=/50302026/11/12 07:15:35 INFO Received complete push request method=POST path=/50312026/11/12 07:15:35 INFO Completed upload id=35 objects=11 duration=18.877059ms50322026/11/12 07:15:35 INFO Completed push id=35 objects=8 duration=15.81643ms50332026/11/12 07:15:35 INFO Completed push id=31 objects=11 duration=18.899769ms50342026/11/12 07:15:35 INFO Received complete push request method=POST path=/50352026/11/12 07:15:35 INFO Received push request method=POST path=/50362026/11/12 07:15:35 INFO Completed push id=26 objects=23 duration=8.141955ms50372026/11/12 07:15:35 INFO Received complete push request method=POST path=/50382026/11/12 07:15:35 INFO Received complete upload request method=POST path=/50392026/11/12 07:15:35 INFO Received complete push request method=POST path=/50402026/11/12 07:15:35 INFO Received complete upload request method=POST path=/50412026/11/12 07:15:35 INFO Received complete upload request method=POST path=/50422026/11/12 07:15:35 INFO Completed push id=24 objects=24 duration=9.813424ms50432026/11/12 07:15:35 INFO Completed upload id=36 objects=3 duration=8.609645ms50442026/11/12 07:15:35 INFO Received complete upload request method=POST path=/50452026/11/12 07:15:35 INFO Completed upload id=32 objects=2 duration=4.210378ms50462026/11/12 07:15:35 INFO Completed push id=35 objects=6 duration=4.562708ms50472026/11/12 07:15:35 INFO Completed upload id=36 objects=9 duration=3.727678ms50482026/11/12 07:15:35 INFO Completed upload id=21 objects=9 duration=2.440578ms50492026/11/12 07:15:35 INFO Received complete upload request method=POST path=/50502026/11/12 07:15:35 INFO Completed upload id=34 objects=24 duration=1.493389ms50512026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=050522026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=050532026/11/12 07:15:35 INFO Received complete push request method=POST path=/50542026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period50552026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=050562026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period50572026/11/12 07:15:35 INFO Completed push id=33 objects=3 duration=1.48575ms50582026/11/12 07:15:35 INFO Received complete push request method=POST path=/50592026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=050602026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050612026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period50622026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050632026/11/12 07:15:35 INFO Received complete push request method=POST path=/50642026/11/12 07:15:35 INFO Completed push id=25 objects=10 duration=1.811439ms50652026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period50662026/11/12 07:15:35 INFO Vacuumed table table=pending_closures50672026/11/12 07:15:35 INFO Vacuumed table table=pending_closures50682026/11/12 07:15:35 INFO Completed push id=27 objects=15 duration=1.756969ms50692026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts50702026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts50712026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050722026/11/12 07:15:35 INFO Vacuumed table table=pending_objects50732026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads50742026/11/12 07:15:35 INFO Vacuumed table table=pending_objects50752026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads50762026/11/12 07:15:35 INFO Vacuumed table table=closures50772026/11/12 07:15:35 INFO Vacuumed table table=pending_closures50782026/11/12 07:15:35 INFO Vacuumed table table=closures50792026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=050802026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts50812026/11/12 07:15:35 INFO Vacuumed table table=objects50822026/11/12 07:15:35 INFO Vacuumed table table=objects50832026/11/12 07:15:35 INFO Vacuumed table table=pending_closures50842026/11/12 07:15:35 INFO Vacuumed table table=pending_objects50852026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads50862026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts50872026/11/12 07:15:35 INFO Received push request method=POST path=/50882026/11/12 07:15:35 INFO Vacuumed table table=closures50892026/11/12 07:15:35 INFO Vacuumed table table=objects50902026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=050912026/11/12 07:15:35 INFO Vacuumed table table=pending_objects50922026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads50932026/11/12 07:15:35 INFO Received complete upload request method=POST path=/50942026/11/12 07:15:35 INFO Vacuumed table table=closures50952026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period50962026/11/12 07:15:35 INFO Completed upload id=34 objects=3 duration=898.629µs50972026/11/12 07:15:35 INFO Vacuumed table table=objects50982026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050992026/11/12 07:15:35 INFO Vacuumed table table=pending_closures51002026/11/12 07:15:35 INFO Received complete upload request method=POST path=/51012026/11/12 07:15:35 INFO Received complete upload request method=POST path=/51022026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts51032026/11/12 07:15:35 INFO Completed upload id=18 objects=22 duration=980.82µs51042026/11/12 07:15:35 INFO Completed upload id=29 objects=5 duration=1.156819ms51052026/11/12 07:15:35 INFO Vacuumed table table=pending_objects51062026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads51072026/11/12 07:15:35 INFO Vacuumed table table=closures51082026/11/12 07:15:35 INFO Vacuumed table table=objects51092026/11/12 07:15:35 INFO Received uploads request method=POST path=/51102026/11/12 07:15:35 INFO Received complete push request method=POST path=/51112026/11/12 07:15:35 INFO Completed push id=28 objects=11 duration=2.764439ms51122026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=051132026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period51142026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051152026/11/12 07:15:35 INFO Received complete upload request method=POST path=/51162026/11/12 07:15:35 INFO Completed upload id=21 objects=8 duration=1.08267ms51172026/11/12 07:15:35 INFO Vacuumed table table=pending_closures51182026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts51192026/11/12 07:15:35 INFO Received complete upload request method=POST path=/51202026/11/12 07:15:35 INFO Completed upload id=30 objects=2 duration=1.02652ms51212026/11/12 07:15:35 INFO Vacuumed table table=pending_objects51222026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads51232026/11/12 07:15:35 INFO Vacuumed table table=closures51242026/11/12 07:15:35 INFO Vacuumed table table=objects51252026/11/12 07:15:35 INFO Received uploads request method=POST path=/51262026/11/12 07:15:35 INFO Received complete push request method=POST path=/51272026/11/12 07:15:35 INFO Completed push id=28 objects=19 duration=1.751849ms51282026/11/12 07:15:35 INFO Received complete push request method=POST path=/51292026/11/12 07:15:35 INFO Completed push id=31 objects=20 duration=1.559849ms51302026/11/12 07:15:35 INFO Received complete upload request method=POST path=/51312026/11/12 07:15:35 INFO Completed upload id=22 objects=7 duration=829.149µs51322026/11/12 07:15:35 INFO Received complete push request method=POST path=/51332026/11/12 07:15:35 INFO Completed push id=29 objects=24 duration=1.823179ms51342026/11/12 07:15:35 INFO Received complete upload request method=POST path=/51352026/11/12 07:15:35 INFO Received complete push request method=POST path=/51362026/11/12 07:15:35 INFO Completed push id=24 objects=5 duration=753.65µs51372026/11/12 07:15:35 INFO Completed upload id=32 objects=5 duration=1.623199ms51382026/11/12 07:15:35 INFO Received complete push request method=POST path=/51392026/11/12 07:15:35 INFO Received push request method=POST path=/51402026/11/12 07:15:35 INFO Completed push id=29 objects=10 duration=1.42606ms51412026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=051422026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period51432026/11/12 07:15:35 INFO Received complete upload request method=POST path=/51442026/11/12 07:15:35 INFO Completed upload id=33 objects=3 duration=1.228839ms51452026/11/12 07:15:35 INFO Received complete upload request method=POST path=/51462026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=051472026/11/12 07:15:35 INFO Completed upload id=25 objects=8 duration=1.107809ms51482026/11/12 07:15:35 INFO Received uploads request method=POST path=/51492026/11/12 07:15:35 INFO Vacuumed table table=pending_closures51502026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts51512026/11/12 07:15:35 INFO Received complete upload request method=POST path=/51522026/11/12 07:15:35 INFO Vacuumed table table=pending_objects51532026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads51542026/11/12 07:15:35 INFO Completed upload id=30 objects=6 duration=1.45129ms51552026/11/12 07:15:35 INFO Vacuumed table table=closures51562026/11/12 07:15:35 INFO Vacuumed table table=objects51572026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=051582026/11/12 07:15:35 INFO Received complete push request method=POST path=/51592026/11/12 07:15:35 INFO Completed push id=26 objects=4 duration=784.359µs51602026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period51612026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=051622026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051632026/11/12 07:15:35 INFO Vacuumed table table=pending_closures51642026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period51652026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts51662026/11/12 07:15:35 INFO Received complete upload request method=POST path=/51672026/11/12 07:15:35 INFO Completed upload id=31 objects=5 duration=1.040759ms51682026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051692026/11/12 07:15:35 INFO Vacuumed table table=pending_objects51702026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads51712026/11/12 07:15:35 INFO Vacuumed table table=pending_closures51722026/11/12 07:15:35 INFO Vacuumed table table=closures51732026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts51742026/11/12 07:15:35 INFO Vacuumed table table=objects51752026/11/12 07:15:35 INFO Vacuumed table table=pending_objects51762026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads51772026/11/12 07:15:35 INFO Vacuumed table table=closures51782026/11/12 07:15:35 INFO Vacuumed table table=objects51792026/11/12 07:15:35 INFO Received complete push request method=POST path=/51802026/11/12 07:15:35 INFO Completed push id=30 objects=2 duration=730.079µs51812026/11/12 07:15:35 INFO Received complete upload request method=POST path=/51822026/11/12 07:15:35 INFO Completed upload id=34 objects=3 duration=1.138739ms51832026/11/12 07:15:35 INFO Received complete push request method=POST path=/51842026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=051852026/11/12 07:15:35 INFO Completed push id=32 objects=7 duration=927.839µs51862026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period51872026/11/12 07:15:35 INFO Received push request method=POST path=/51882026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051892026/11/12 07:15:35 INFO Vacuumed table table=pending_closures51902026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts51912026/11/12 07:15:35 INFO Vacuumed table table=pending_objects51922026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads51932026/11/12 07:15:35 INFO Vacuumed table table=closures51942026/11/12 07:15:35 INFO Vacuumed table table=objects51952026/11/12 07:15:35 INFO Received complete upload request method=POST path=/51962026/11/12 07:15:35 INFO Completed upload id=33 objects=6 duration=767.32µs51972026/11/12 07:15:35 INFO Received complete upload request method=POST path=/51982026/11/12 07:15:35 INFO Completed upload id=23 objects=7 duration=3.089338ms51992026/11/12 07:15:35 INFO Received complete upload request method=POST path=/52002026/11/12 07:15:35 INFO Completed upload id=34 objects=3 duration=703.57µs52012026/11/12 07:15:35 INFO Received complete upload request method=POST path=/52022026/11/12 07:15:35 INFO Completed upload id=26 objects=5 duration=1.09258ms52032026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=052042026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period52052026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052062026/11/12 07:15:35 INFO Vacuumed table table=pending_closures52072026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts52082026/11/12 07:15:35 INFO Received complete push request method=POST path=/52092026/11/12 07:15:35 INFO Vacuumed table table=pending_objects52102026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads52112026/11/12 07:15:35 INFO Vacuumed table table=closures52122026/11/12 07:15:35 INFO Completed push id=27 objects=3 duration=2.254789ms52132026/11/12 07:15:35 INFO Vacuumed table table=objects52142026/11/12 07:15:35 INFO Received complete upload request method=POST path=/52152026/11/12 07:15:35 INFO Completed upload id=28 objects=5 duration=1.381819ms52162026/11/12 07:15:35 INFO Received complete push request method=POST path=/52172026/11/12 07:15:35 INFO Completed push id=29 objects=17 duration=1.728609ms52182026/11/12 07:15:35 INFO Aborted multipart uploads count=0 kept=052192026/11/12 07:15:35 WARN Force mode enabled - objects will be deleted immediately without grace period52202026/11/12 07:15:35 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052212026/11/12 07:15:35 INFO Vacuumed table table=pending_closures52222026/11/12 07:15:35 INFO Vacuumed table table=commit_receipts52232026/11/12 07:15:35 INFO Vacuumed table table=pending_objects52242026/11/12 07:15:35 INFO Vacuumed table table=multipart_uploads52252026/11/12 07:15:35 INFO Vacuumed table table=closures52262026/11/12 07:15:35 INFO Vacuumed table table=objects5227--- PASS: FuzzPushGC (0.00s)5228 --- PASS: FuzzPushGC/seed#14 (3.07s)5229 --- PASS: FuzzPushGC/seed#8 (3.26s)5230 --- PASS: FuzzPushGC/seed#0 (3.31s)5231 --- PASS: FuzzPushGC/seed#15 (3.35s)5232 --- PASS: FuzzPushGC/seed#3 (3.39s)5233 --- PASS: FuzzPushGC/seed#7 (3.47s)5234 --- PASS: FuzzPushGC/seed#11 (3.59s)5235 --- PASS: FuzzPushGC/seed#4 (3.60s)5236 --- PASS: FuzzPushGC/seed#12 (3.60s)5237 --- PASS: FuzzPushGC/seed#9 (3.61s)5238 --- PASS: FuzzPushGC/seed#5 (3.63s)5239 --- PASS: FuzzPushGC/seed#2 (3.70s)5240 --- PASS: FuzzPushGC/seed#6 (3.72s)5241 --- PASS: FuzzPushGC/seed#13 (3.77s)5242 --- PASS: FuzzPushGC/seed#10 (3.84s)5243 --- PASS: FuzzPushGC/seed#1 (3.93s)5244PASS5245{"timestamp":"2026-11-12T07:15:35.611562168Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:36230","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(399)"}52462026-11-12 07:15:36.049 UTC [130] LOG: received smart shutdown request52472026-11-12 07:15:36.053 UTC [130] LOG: background worker "logical replication launcher" (PID 140) exited with exit code 152482026-11-12 07:15:36.968 UTC [135] LOG: checkpoint complete: wrote 3586 buffers (21.9%), wrote 5 SLRU buffers; 0 WAL file(s) added, 0 removed, 33 recycled; write=16.507 s, sync=0.772 s, total=17.353 s; sync files=43470, longest=0.046 s, average=0.001 s; distance=534253 kB, estimate=534253 kB; lsn=0/279EBB18, redo lsn=0/221211A052492026-11-12 07:15:36.970 UTC [135] LOG: shutting down52502026-11-12 07:15:36.971 UTC [135] LOG: checkpoint starting: shutdown immediate52512026-11-12 07:15:38.143 UTC [135] LOG: checkpoint complete: wrote 9840 buffers (60.1%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 5 recycled; write=0.184 s, sync=0.969 s, total=1.173 s; sync files=3210, longest=0.002 s, average=0.001 s; distance=90922 kB, estimate=489920 kB; lsn=0/279EBB90, redo lsn=0/279EBB9052522026-11-12 07:15:38.239 UTC [130] LOG: database system is shut down5253Running OIDC tests...5254=== RUN TestAudienceForIssuer5255=== PAUSE TestAudienceForIssuer5256=== RUN TestProviderClientGivesUpOnSilentIssuer5257=== PAUSE TestProviderClientGivesUpOnSilentIssuer5258=== RUN TestGlobMatch5259=== PAUSE TestGlobMatch5260=== RUN TestValidateToken_ValidToken5261=== PAUSE TestValidateToken_ValidToken5262=== RUN TestValidateToken_WrongAudience5263=== PAUSE TestValidateToken_WrongAudience5264=== RUN TestValidateToken_Expired5265=== PAUSE TestValidateToken_Expired5266=== RUN TestValidateToken_BoundClaimsMismatch5267=== PAUSE TestValidateToken_BoundClaimsMismatch5268=== RUN TestValidateToken_BoundSubjectMismatch5269=== PAUSE TestValidateToken_BoundSubjectMismatch5270=== RUN TestValidateToken_MultipleProviders5271=== PAUSE TestValidateToken_MultipleProviders5272=== RUN TestValidateToken_UnknownIssuerFetchesNoKeys5273=== PAUSE TestValidateToken_UnknownIssuerFetchesNoKeys5274=== RUN TestValidateToken_GoogleSchemelessIssuer5275=== PAUSE TestValidateToken_GoogleSchemelessIssuer5276=== RUN TestValidateToken_KubernetesServiceAccount5277=== PAUSE TestValidateToken_KubernetesServiceAccount5278=== RUN TestNewValidator_KubernetesRequiresCA5279=== PAUSE TestNewValidator_KubernetesRequiresCA5280=== RUN TestValidateToken_KubernetesIssuerFromOwnToken5281=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken5282=== RUN TestPins_ReservedForMatchingRule5283=== PAUSE TestPins_ReservedForMatchingRule5284=== RUN TestPins_TopLevelShorthand5285=== PAUSE TestPins_TopLevelShorthand5286=== RUN TestPins_ConfigValidation5287=== PAUSE TestPins_ConfigValidation5288=== RUN TestScopes_LegacyProviderDefaultsToWrite5289=== PAUSE TestScopes_LegacyProviderDefaultsToWrite5290=== RUN TestScopes_Rules5291=== PAUSE TestScopes_Rules5292=== RUN TestScopes_ConfigValidation5293=== PAUSE TestScopes_ConfigValidation5294=== CONT TestAudienceForIssuer5295=== CONT TestValidateToken_GoogleSchemelessIssuer5296--- PASS: TestAudienceForIssuer (0.00s)5297=== CONT TestValidateToken_WrongAudience5298=== CONT TestValidateToken_ValidToken5299=== CONT TestGlobMatch5300=== RUN TestGlobMatch/foo_foo5301=== PAUSE TestGlobMatch/foo_foo5302=== RUN TestGlobMatch/foo_bar5303=== PAUSE TestGlobMatch/foo_bar5304=== RUN TestGlobMatch/*_5305=== PAUSE TestGlobMatch/*_5306=== RUN TestGlobMatch/*_anything5307=== PAUSE TestGlobMatch/*_anything5308=== RUN TestGlobMatch/foo*_foo5309=== CONT TestProviderClientGivesUpOnSilentIssuer5310=== CONT TestValidateToken_KubernetesIssuerFromOwnToken5311=== CONT TestPins_ReservedForMatchingRule5312=== CONT TestNewValidator_KubernetesRequiresCA5313=== CONT TestValidateToken_MultipleProviders5314=== CONT TestValidateToken_UnknownIssuerFetchesNoKeys5315=== CONT TestValidateToken_BoundSubjectMismatch5316=== CONT TestScopes_Rules5317=== CONT TestScopes_ConfigValidation5318=== CONT TestValidateToken_KubernetesServiceAccount5319=== CONT TestValidateToken_BoundClaimsMismatch53202026/11/12 07:15:40 INFO OIDC provider initialized name=known issuer=https://known.example5321=== CONT TestScopes_LegacyProviderDefaultsToWrite5322=== CONT TestPins_ConfigValidation5323=== CONT TestValidateToken_Expired5324=== CONT TestPins_TopLevelShorthand5325=== PAUSE TestGlobMatch/foo*_foo5326--- PASS: TestProviderClientGivesUpOnSilentIssuer (0.00s)5327--- PASS: TestScopes_ConfigValidation (0.00s)5328--- PASS: TestPins_ConfigValidation (0.00s)5329=== RUN TestGlobMatch/foo*_foobar5330=== PAUSE TestGlobMatch/foo*_foobar5331=== RUN TestGlobMatch/foo*_bar5332=== PAUSE TestGlobMatch/foo*_bar5333=== RUN TestGlobMatch/*bar_bar5334=== PAUSE TestGlobMatch/*bar_bar5335=== RUN TestGlobMatch/*bar_foobar5336=== PAUSE TestGlobMatch/*bar_foobar5337=== RUN TestGlobMatch/*bar_foo5338=== PAUSE TestGlobMatch/*bar_foo5339=== RUN TestGlobMatch/foo*bar_foobar5340=== PAUSE TestGlobMatch/foo*bar_foobar5341=== RUN TestGlobMatch/foo*bar_foo123bar5342=== PAUSE TestGlobMatch/foo*bar_foo123bar5343=== RUN TestGlobMatch/foo*bar_foobarbaz5344=== PAUSE TestGlobMatch/foo*bar_foobarbaz5345=== RUN TestGlobMatch/*/*_foo/bar5346=== PAUSE TestGlobMatch/*/*_foo/bar5347=== RUN TestGlobMatch/*/*_foo5348=== PAUSE TestGlobMatch/*/*_foo5349=== RUN TestGlobMatch/refs/heads/*_refs/heads/main5350=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main5351=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.05352=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.05353=== RUN TestGlobMatch/refs/*/main_refs/heads/main5354=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main5355=== RUN TestGlobMatch/fo?_foo5356=== PAUSE TestGlobMatch/fo?_foo5357=== RUN TestGlobMatch/fo?_fo5358=== PAUSE TestGlobMatch/fo?_fo5359=== RUN TestGlobMatch/fo?_fooo5360=== PAUSE TestGlobMatch/fo?_fooo5361=== RUN TestGlobMatch/?oo_foo5362=== PAUSE TestGlobMatch/?oo_foo5363=== RUN TestGlobMatch/?oo_boo5364=== PAUSE TestGlobMatch/?oo_boo5365=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5366=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5367=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5368=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5369=== CONT TestGlobMatch/foo_foo5370=== CONT TestGlobMatch/*/*_foo/bar5371=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5372=== CONT TestGlobMatch/foo*bar_foobarbaz5373=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5374=== CONT TestGlobMatch/foo*bar_foo123bar5375=== CONT TestGlobMatch/refs/heads/*_refs/heads/main5376=== CONT TestGlobMatch/?oo_boo5377=== CONT TestGlobMatch/foo*bar_foobar5378=== CONT TestGlobMatch/*bar_foo5379=== CONT TestGlobMatch/?oo_foo5380=== CONT TestGlobMatch/foo*_foo5381=== CONT TestGlobMatch/*_5382=== CONT TestGlobMatch/foo*_bar5383=== CONT TestGlobMatch/*_anything53842026/11/12 07:15:40 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:40319/oidc5385=== CONT TestGlobMatch/foo*_foobar5386=== CONT TestGlobMatch/foo_bar5387=== CONT TestGlobMatch/fo?_fooo5388=== CONT TestGlobMatch/fo?_fo5389=== CONT TestGlobMatch/*bar_bar5390=== CONT TestGlobMatch/fo?_foo5391=== CONT TestGlobMatch/refs/*/main_refs/heads/main5392=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.05393=== CONT TestGlobMatch/*/*_foo5394=== CONT TestGlobMatch/*bar_foobar5395--- PASS: TestGlobMatch (0.01s)5396 --- PASS: TestGlobMatch/foo_foo (0.00s)5397 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)5398 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)5399 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)5400 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)5401 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)5402 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)5403 --- PASS: TestGlobMatch/?oo_boo (0.00s)5404 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)5405 --- PASS: TestGlobMatch/*bar_foo (0.00s)5406 --- PASS: TestGlobMatch/?oo_foo (0.00s)5407 --- PASS: TestGlobMatch/foo*_foo (0.00s)5408 --- PASS: TestGlobMatch/*_ (0.00s)5409 --- PASS: TestGlobMatch/foo*_bar (0.00s)5410 --- PASS: TestGlobMatch/*_anything (0.00s)5411 --- PASS: TestGlobMatch/foo*_foobar (0.00s)5412 --- PASS: TestGlobMatch/foo_bar (0.00s)5413 --- PASS: TestGlobMatch/fo?_fooo (0.00s)5414 --- PASS: TestGlobMatch/fo?_fo (0.00s)5415 --- PASS: TestGlobMatch/*bar_bar (0.00s)5416 --- PASS: TestGlobMatch/fo?_foo (0.00s)5417 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)5418 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)5419 --- PASS: TestGlobMatch/*/*_foo (0.00s)5420 --- PASS: TestGlobMatch/*bar_foobar (0.00s)5421--- PASS: TestValidateToken_ValidToken (0.02s)54222026/11/12 07:15:40 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:45383/oidc5423--- PASS: TestValidateToken_Expired (0.03s)54242026/11/12 07:15:40 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:375435425--- PASS: TestValidateToken_KubernetesServiceAccount (0.03s)54262026/11/12 07:15:40 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC12354272026/11/12 07:15:40 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:35061/oidc5428--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (0.06s)5429--- PASS: TestScopes_LegacyProviderDefaultsToWrite (0.06s)54302026/11/12 07:15:40 INFO OIDC provider initialized name=google issuer=https://accounts.google.com54312026/11/12 07:15:40 http: TLS handshake error from 127.0.0.1:33204: remote error: tls: bad certificate5432--- PASS: TestNewValidator_KubernetesRequiresCA (0.07s)5433--- PASS: TestValidateToken_GoogleSchemelessIssuer (0.07s)54342026/11/12 07:15:40 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:34965/oidc5435--- PASS: TestPins_ReservedForMatchingRule (0.08s)54362026/11/12 07:15:40 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:41585/oidc5437--- PASS: TestValidateToken_UnknownIssuerFetchesNoKeys (0.08s)54382026/11/12 07:15:40 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:45189/oidc5439--- PASS: TestPins_TopLevelShorthand (0.07s)5440--- PASS: TestScopes_Rules (0.08s)54412026/11/12 07:15:40 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:39607/oidc54422026/11/12 07:15:40 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:42109/oidc5443--- PASS: TestValidateToken_WrongAudience (0.12s)54442026/11/12 07:15:40 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:42379/oidc5445--- PASS: TestValidateToken_MultipleProviders (0.12s)54462026/11/12 07:15:40 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:41047/oidc54472026/11/12 07:15:40 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:38311/oidc5448--- PASS: TestValidateToken_BoundClaimsMismatch (0.13s)5449--- PASS: TestValidateToken_BoundSubjectMismatch (0.13s)5450PASS5451Running hook tests...5452=== RUN TestSendPathsEmpty5453=== PAUSE TestSendPathsEmpty5454=== RUN TestQueueEnqueueAndFetch5455=== PAUSE TestQueueEnqueueAndFetch5456=== RUN TestQueueDeduplication5457=== PAUSE TestQueueDeduplication5458=== RUN TestQueueRemove5459=== PAUSE TestQueueRemove5460=== RUN TestQueueFetchBatchLimit5461=== PAUSE TestQueueFetchBatchLimit5462=== RUN TestQueueRetryMovesToBack5463=== PAUSE TestQueueRetryMovesToBack5464=== RUN TestQueueFetchRemoveLifecycle5465=== PAUSE TestQueueFetchRemoveLifecycle5466=== RUN TestQueueConcurrentWriters5467=== PAUSE TestQueueConcurrentWriters5468=== RUN TestQueueRemoveLargeClosure5469=== PAUSE TestQueueRemoveLargeClosure5470=== RUN TestServerClientIntegration5471--- PASS: TestServerClientIntegration (0.00s)5472=== RUN TestServerQueueError54732026/11/12 07:15:40 ERROR Failed to queue paths error="permission denied" count=15474--- PASS: TestServerQueueError (0.00s)5475=== RUN TestGetListenerSocketActivation5476 server_test.go:220: === RUN TestGetListenerSocketActivation5477 --- PASS: TestGetListenerSocketActivation (0.00s)5478 PASS5479 5480--- PASS: TestGetListenerSocketActivation (0.01s)5481=== RUN TestServerStalledClientDoesNotBlockShutdown5482=== PAUSE TestServerStalledClientDoesNotBlockShutdown5483=== RUN TestServerBacksOffOnAcceptErrors5484=== PAUSE TestServerBacksOffOnAcceptErrors5485=== RUN TestDrainIsolatesPoisonPath5486=== PAUSE TestDrainIsolatesPoisonPath5487=== RUN TestRunNotBlockedByPoisonHead5488=== PAUSE TestRunNotBlockedByPoisonHead5489=== RUN TestDrainGivesUpWhenServerDown5490=== PAUSE TestDrainGivesUpWhenServerDown5491=== RUN TestFailedPathPrunedByLaterClosure5492=== PAUSE TestFailedPathPrunedByLaterClosure5493=== RUN TestWorkerUploadsAndRemoves5494=== PAUSE TestWorkerUploadsAndRemoves5495=== RUN TestWorkerSkipsGCdPaths5496=== PAUSE TestWorkerSkipsGCdPaths5497=== RUN TestWorkerPrunesClosureDeps5498=== PAUSE TestWorkerPrunesClosureDeps5499=== RUN TestDrainTimeout5500=== PAUSE TestDrainTimeout5501=== RUN TestWorkerRemoveFailureIsNotProgress5502=== PAUSE TestWorkerRemoveFailureIsNotProgress5503=== RUN TestWorkerKeepsPathItCannotStat5504=== PAUSE TestWorkerKeepsPathItCannotStat5505=== CONT TestSendPathsEmpty5506=== CONT TestDrainGivesUpWhenServerDown5507--- PASS: TestSendPathsEmpty (0.00s)5508=== CONT TestWorkerKeepsPathItCannotStat5509=== CONT TestWorkerRemoveFailureIsNotProgress5510=== CONT TestDrainTimeout5511=== CONT TestWorkerPrunesClosureDeps5512=== CONT TestWorkerSkipsGCdPaths5513=== CONT TestDrainIsolatesPoisonPath5514=== CONT TestServerBacksOffOnAcceptErrors5515=== CONT TestServerStalledClientDoesNotBlockShutdown55162000/01/01 00:00:00 ERROR Accept failed error="too many open files"55172000/01/01 00:00:00 ERROR Accept failed error="too many open files"55182000/01/01 00:00:00 ERROR Accept failed error="too many open files"55192000/01/01 00:00:00 ERROR Accept failed error="too many open files"5520=== CONT TestFailedPathPrunedByLaterClosure55212000/01/01 00:00:00 ERROR Accept failed error="too many open files"55222000/01/01 00:00:00 ERROR Accept failed error="too many open files"55232000/01/01 00:00:00 ERROR Accept failed error="too many open files"55242000/01/01 00:00:00 ERROR Accept failed error="too many open files"5525--- PASS: TestServerBacksOffOnAcceptErrors (0.00s)5526=== CONT TestQueueRemoveLargeClosure5527=== CONT TestQueueConcurrentWriters5528=== CONT TestQueueFetchRemoveLifecycle5529=== CONT TestWorkerUploadsAndRemoves5530=== CONT TestQueueRetryMovesToBack5531=== CONT TestQueueFetchBatchLimit5532=== CONT TestQueueRemove5533=== CONT TestRunNotBlockedByPoisonHead5534=== CONT TestQueueEnqueueAndFetch5535=== CONT TestQueueDeduplication55362026/11/12 07:15:41 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat401863982/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat401863982/002/locked/aaa: permission denied"55372026/11/12 07:15:41 INFO Upload queue status pending=255382026/11/12 07:15:41 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerSkipsGCdPaths1687214193/002/nonexistent55392026/11/12 07:15:41 INFO Upload queue status pending=355402026/11/12 07:15:41 INFO Uploading batch count=255412026/11/12 07:15:41 ERROR Upload failed error="upload failed" count=255422026/11/12 07:15:41 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown3366482971/002/a55432026/11/12 07:15:41 INFO Upload queue status pending=255442000/01/01 00:00:00 INFO Uploading batch count=255452026/11/12 07:15:41 INFO Uploading batch count=455462026/11/12 07:15:41 ERROR Upload failed error="upload failed" count=455472026/11/12 07:15:41 INFO Uploading batch count=155482026/11/12 07:15:41 INFO Uploading batch count=15549--- PASS: TestQueueFetchBatchLimit (0.01s)55502026/11/12 07:15:41 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown3366482971/002/b5551--- PASS: TestQueueEnqueueAndFetch (0.01s)55522026/11/12 07:15:41 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat401863982/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat401863982/002/locked/aaa: permission denied"55532026/11/12 07:15:41 ERROR Upload failed error="upload failed" count=155542026/11/12 07:15:41 INFO Uploading batch count=155552026/11/12 07:15:41 ERROR Upload failed error="upload failed" count=155562000/01/01 00:00:00 ERROR Upload failed error="context deadline exceeded" count=255572026/11/12 07:15:41 INFO Upload queue status pending=255582026/11/12 07:15:41 INFO Uploading batch count=155592026/11/12 07:15:41 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainIsolatesPoisonPath1771562382/002/bbb55602026/11/12 07:15:41 INFO Uploading batch count=255612000/01/01 00:00:00 ERROR Drain finished with paths left in queue remaining=45562--- PASS: TestQueueRemove (0.01s)55632026/11/12 07:15:41 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress614812181/002/nonexistent55642026/11/12 07:15:41 INFO Uploading batch count=255652026/11/12 07:15:41 ERROR Upload failed error="upload failed" count=255662026/11/12 07:15:41 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown3366482971/002/c5567--- PASS: TestQueueDeduplication (0.01s)55682026/11/12 07:15:41 INFO Uploading batch count=15569--- PASS: TestQueueRetryMovesToBack (0.02s)55702026/11/12 07:15:41 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=15571--- PASS: TestQueueFetchRemoveLifecycle (0.02s)55722026/11/12 07:15:41 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat401863982/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat401863982/002/locked/aaa: permission denied"55732026/11/12 07:15:41 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress614812181/002/nonexistent55742026/11/12 07:15:41 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown3366482971/002/d55752026/11/12 07:15:41 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=155762026/11/12 07:15:41 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress614812181/002/nonexistent55772026/11/12 07:15:41 ERROR Drain finished with paths left in queue remaining=155782026/11/12 07:15:41 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=155792026/11/12 07:15:41 ERROR Drain finished with paths left in queue remaining=155802026/11/12 07:15:41 INFO Uploading batch count=155812026/11/12 07:15:41 INFO Uploading batch count=155822026/11/12 07:15:41 ERROR Upload failed error="upload failed" count=155832026/11/12 07:15:41 INFO Uploading batch count=25584--- PASS: TestDrainTimeout (0.02s)55852026/11/12 07:15:41 ERROR Upload failed error="upload failed" count=255862026/11/12 07:15:41 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown3366482971/002/e5587--- PASS: TestWorkerRemoveFailureIsNotProgress (0.02s)55882026/11/12 07:15:41 INFO Uploading batch count=155892026/11/12 07:15:41 ERROR Upload failed error="upload failed" count=155902026/11/12 07:15:41 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown3366482971/002/f5591--- PASS: TestWorkerKeepsPathItCannotStat (0.02s)55922026/11/12 07:15:41 INFO Uploading batch count=155932026/11/12 07:15:41 ERROR Upload failed error="upload failed" count=155942026/11/12 07:15:41 ERROR Drain finished with paths left in queue remaining=105595--- PASS: TestFailedPathPrunedByLaterClosure (0.02s)55962026/11/12 07:15:41 ERROR Drain finished with paths left in queue remaining=15597--- PASS: TestDrainGivesUpWhenServerDown (0.03s)5598--- PASS: TestDrainIsolatesPoisonPath (0.03s)5599--- PASS: TestWorkerSkipsGCdPaths (0.04s)5600--- PASS: TestWorkerPrunesClosureDeps (0.04s)5601--- PASS: TestWorkerUploadsAndRemoves (0.04s)5602--- PASS: TestQueueRemoveLargeClosure (0.07s)56032026/11/12 07:15:41 ERROR Failed to decode request error="read unix /build/hook1172429921/test.sock->@: i/o timeout"56042026/11/12 07:15:41 ERROR Failed to write response error="write unix /build/hook1172429921/test.sock->@: i/o timeout"5605--- PASS: TestServerStalledClientDoesNotBlockShutdown (0.20s)5606--- PASS: TestQueueConcurrentWriters (0.21s)56072026/11/12 07:15:42 INFO Uploading batch count=156082026/11/12 07:15:42 INFO Uploading batch count=156092026/11/12 07:15:42 INFO Uploading batch count=156102026/11/12 07:15:42 ERROR Upload failed error="upload failed" count=156112026/11/12 07:15:42 INFO Uploading batch count=156122026/11/12 07:15:42 ERROR Upload failed error="upload failed" count=156132026/11/12 07:15:42 INFO Uploading batch count=156142026/11/12 07:15:42 ERROR Upload failed error="upload failed" count=156152026/11/12 07:15:42 INFO Uploading batch count=156162026/11/12 07:15:42 ERROR Upload failed error="upload failed" count=156172026/11/12 07:15:42 ERROR Drain finished with paths left in queue remaining=15618--- PASS: TestRunNotBlockedByPoisonHead (1.03s)5619PASS5620Running signing tests...5621=== RUN TestExternalSigner5622=== PAUSE TestExternalSigner5623=== RUN TestExternalSignerWrongPublicKey5624=== PAUSE TestExternalSignerWrongPublicKey5625=== RUN TestExternalSignerErrors5626=== PAUSE TestExternalSignerErrors5627=== RUN TestExternalSignerRestartsAfterCrash5628=== PAUSE TestExternalSignerRestartsAfterCrash5629=== RUN TestExternalSignerProgramErrorKeepsProcess5630=== PAUSE TestExternalSignerProgramErrorKeepsProcess5631=== RUN TestExternalSignerClose5632=== PAUSE TestExternalSignerClose5633=== RUN TestExternalSignerRejectsInvalidPublicKey5634=== PAUSE TestExternalSignerRejectsInvalidPublicKey5635=== RUN TestExternalSignerCancellation5636=== PAUSE TestExternalSignerCancellation5637=== RUN TestGenerateFingerprint5638=== PAUSE TestGenerateFingerprint5639=== RUN TestParseSigningKey5640=== PAUSE TestParseSigningKey5641=== RUN TestSign5642=== PAUSE TestSign5643=== RUN TestSignBatch5644=== PAUSE TestSignBatch5645=== RUN TestNilKeyPublicKey5646=== PAUSE TestNilKeyPublicKey5647=== RUN TestSignErrors5648=== PAUSE TestSignErrors5649=== CONT TestExternalSigner5650=== CONT TestExternalSignerCancellation5651=== CONT TestExternalSignerProgramErrorKeepsProcess5652=== CONT TestExternalSignerRejectsInvalidPublicKey5653=== CONT TestExternalSignerClose5654=== CONT TestSignErrors5655=== RUN TestExternalSignerRejectsInvalidPublicKey/#005656=== CONT TestExternalSignerErrors5657=== CONT TestExternalSignerRestartsAfterCrash5658=== CONT TestNilKeyPublicKey5659--- PASS: TestNilKeyPublicKey (0.00s)5660=== CONT TestExternalSignerWrongPublicKey5661=== CONT TestParseSigningKey5662=== RUN TestSignErrors/nil_key5663=== PAUSE TestSignErrors/nil_key5664=== RUN TestSignErrors/nil_narinfo5665=== PAUSE TestSignErrors/nil_narinfo5666=== RUN TestParseSigningKey/valid_32-byte_key5667=== PAUSE TestParseSigningKey/valid_32-byte_key5668=== CONT TestGenerateFingerprint5669=== CONT TestSignBatch5670=== PAUSE TestExternalSignerRejectsInvalidPublicKey/#005671=== RUN TestExternalSignerErrors/invalid_JSON5672=== RUN TestExternalSignerRejectsInvalidPublicKey/no-colon5673=== PAUSE TestExternalSignerRejectsInvalidPublicKey/no-colon5674=== RUN TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5675=== PAUSE TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5676=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5677=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5678=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5679=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5680=== CONT TestExternalSignerRejectsInvalidPublicKey/#005681=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5682=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5683=== PAUSE TestExternalSignerErrors/invalid_JSON5684=== RUN TestExternalSignerErrors/missing_signatures5685=== PAUSE TestExternalSignerErrors/missing_signatures5686=== RUN TestExternalSignerErrors/wrong_signatures_shape5687=== RUN TestSignErrors/invalid_narinfo5688=== PAUSE TestSignErrors/invalid_narinfo5689=== RUN TestParseSigningKey/valid_32-byte_key_with_different_name5690=== PAUSE TestParseSigningKey/valid_32-byte_key_with_different_name5691=== CONT TestSign5692=== RUN TestGenerateFingerprint/basic_with_references5693=== PAUSE TestGenerateFingerprint/basic_with_references5694=== RUN TestParseSigningKey/no_colon5695=== CONT TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5696=== CONT TestExternalSignerRejectsInvalidPublicKey/no-colon5697=== PAUSE TestExternalSignerErrors/wrong_signatures_shape5698--- PASS: TestSignBatch (0.00s)5699--- PASS: TestSign (0.00s)5700--- PASS: TestExternalSignerRejectsInvalidPublicKey (0.00s)5701 --- PASS: TestExternalSignerRejectsInvalidPublicKey/#00 (0.00s)5702 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:YQ== (0.00s)5703 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:invalid! (0.00s)5704 --- PASS: TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= (0.00s)5705 --- PASS: TestExternalSignerRejectsInvalidPublicKey/no-colon (0.00s)5706=== RUN TestSignErrors/canceled5707=== PAUSE TestSignErrors/canceled5708=== CONT TestSignErrors/nil_key5709=== RUN TestGenerateFingerprint/no_references5710=== CONT TestSignErrors/canceled5711=== PAUSE TestGenerateFingerprint/no_references5712=== RUN TestGenerateFingerprint/unsorted_references_get_sorted5713=== PAUSE TestGenerateFingerprint/unsorted_references_get_sorted5714=== RUN TestGenerateFingerprint/invalid_nar_hash_prefix5715--- PASS: TestExternalSignerProgramErrorKeepsProcess (0.00s)5716=== RUN TestExternalSignerErrors/null_signatures5717=== PAUSE TestExternalSignerErrors/null_signatures5718=== CONT TestSignErrors/invalid_narinfo5719=== PAUSE TestParseSigningKey/no_colon5720=== RUN TestParseSigningKey/empty_name5721=== PAUSE TestParseSigningKey/empty_name5722=== RUN TestParseSigningKey/invalid_base645723=== PAUSE TestParseSigningKey/invalid_base645724=== RUN TestParseSigningKey/wrong_length5725=== PAUSE TestParseSigningKey/wrong_length5726=== CONT TestParseSigningKey/valid_32-byte_key5727=== CONT TestParseSigningKey/no_colon5728=== CONT TestSignErrors/nil_narinfo5729--- PASS: TestSignErrors (0.00s)5730 --- PASS: TestSignErrors/nil_key (0.00s)5731 --- PASS: TestSignErrors/canceled (0.00s)5732 --- PASS: TestSignErrors/invalid_narinfo (0.00s)5733 --- PASS: TestSignErrors/nil_narinfo (0.00s)5734=== PAUSE TestGenerateFingerprint/invalid_nar_hash_prefix5735=== RUN TestExternalSignerErrors/unrequested_key5736=== PAUSE TestExternalSignerErrors/unrequested_key5737=== CONT TestParseSigningKey/empty_name5738=== CONT TestParseSigningKey/wrong_length5739=== CONT TestParseSigningKey/invalid_base645740=== CONT TestParseSigningKey/valid_32-byte_key_with_different_name5741=== RUN TestGenerateFingerprint/invalid_nar_hash_length5742=== PAUSE TestGenerateFingerprint/invalid_nar_hash_length5743--- PASS: TestExternalSignerClose (0.00s)5744=== RUN TestExternalSignerErrors/program_error5745=== PAUSE TestExternalSignerErrors/program_error5746=== RUN TestExternalSignerErrors/exit_at_startup5747=== PAUSE TestExternalSignerErrors/exit_at_startup5748=== RUN TestGenerateFingerprint/invalid_store_path_prefix5749--- PASS: TestParseSigningKey (0.00s)5750 --- PASS: TestParseSigningKey/valid_32-byte_key (0.00s)5751 --- PASS: TestParseSigningKey/no_colon (0.00s)5752 --- PASS: TestParseSigningKey/empty_name (0.00s)5753 --- PASS: TestParseSigningKey/wrong_length (0.00s)5754 --- PASS: TestParseSigningKey/invalid_base64 (0.00s)5755 --- PASS: TestParseSigningKey/valid_32-byte_key_with_different_name (0.00s)5756=== RUN TestExternalSignerErrors/too_few_signatures5757--- PASS: TestExternalSignerWrongPublicKey (0.00s)5758=== PAUSE TestExternalSignerErrors/too_few_signatures5759=== RUN TestExternalSignerErrors/missing_signature_name5760=== PAUSE TestGenerateFingerprint/invalid_store_path_prefix5761=== PAUSE TestExternalSignerErrors/missing_signature_name5762=== RUN TestGenerateFingerprint/invalid_reference_prefix5763=== PAUSE TestGenerateFingerprint/invalid_reference_prefix5764=== CONT TestGenerateFingerprint/basic_with_references5765=== CONT TestGenerateFingerprint/invalid_store_path_prefix5766=== RUN TestExternalSignerErrors/invalid_signature_base645767=== PAUSE TestExternalSignerErrors/invalid_signature_base645768=== RUN TestExternalSignerErrors/exit_on_request5769=== PAUSE TestExternalSignerErrors/exit_on_request5770=== RUN TestExternalSignerErrors/wrong_shape5771=== PAUSE TestExternalSignerErrors/wrong_shape5772=== RUN TestExternalSignerErrors/null5773=== PAUSE TestExternalSignerErrors/null5774=== RUN TestExternalSignerErrors/wrong_id5775=== PAUSE TestExternalSignerErrors/wrong_id5776=== RUN TestExternalSignerErrors/wrong_signature_name5777=== PAUSE TestExternalSignerErrors/wrong_signature_name5778=== RUN TestExternalSignerErrors/wrong_signature_length5779=== PAUSE TestExternalSignerErrors/wrong_signature_length5780=== CONT TestGenerateFingerprint/invalid_nar_hash_prefix5781=== CONT TestGenerateFingerprint/invalid_nar_hash_length5782=== CONT TestGenerateFingerprint/unsorted_references_get_sorted5783=== CONT TestGenerateFingerprint/invalid_reference_prefix5784=== CONT TestGenerateFingerprint/no_references5785--- PASS: TestGenerateFingerprint (0.00s)5786 --- PASS: TestGenerateFingerprint/basic_with_references (0.00s)5787 --- PASS: TestGenerateFingerprint/invalid_store_path_prefix (0.00s)5788 --- PASS: TestGenerateFingerprint/invalid_nar_hash_prefix (0.00s)5789 --- PASS: TestGenerateFingerprint/invalid_nar_hash_length (0.00s)5790 --- PASS: TestGenerateFingerprint/unsorted_references_get_sorted (0.00s)5791 --- PASS: TestGenerateFingerprint/invalid_reference_prefix (0.00s)5792 --- PASS: TestGenerateFingerprint/no_references (0.00s)5793=== CONT TestExternalSignerErrors/invalid_JSON5794=== CONT TestExternalSignerErrors/wrong_signature_length5795=== CONT TestExternalSignerErrors/wrong_signature_name5796=== CONT TestExternalSignerErrors/wrong_id5797=== CONT TestExternalSignerErrors/null5798=== CONT TestExternalSignerErrors/wrong_shape5799=== CONT TestExternalSignerErrors/exit_on_request5800=== CONT TestExternalSignerErrors/invalid_signature_base645801=== CONT TestExternalSignerErrors/missing_signature_name5802=== CONT TestExternalSignerErrors/null_signatures5803=== CONT TestExternalSignerErrors/wrong_signatures_shape5804=== CONT TestExternalSignerErrors/unrequested_key5805=== CONT TestExternalSignerErrors/too_few_signatures5806=== CONT TestExternalSignerErrors/missing_signatures5807=== CONT TestExternalSignerErrors/exit_at_startup5808=== CONT TestExternalSignerErrors/program_error5809--- PASS: TestExternalSigner (0.01s)5810--- PASS: TestExternalSignerRestartsAfterCrash (0.01s)5811--- PASS: TestExternalSignerErrors (0.01s)5812 --- PASS: TestExternalSignerErrors/wrong_signature_length (0.00s)5813 --- PASS: TestExternalSignerErrors/wrong_shape (0.00s)5814 --- PASS: TestExternalSignerErrors/invalid_JSON (0.00s)5815 --- PASS: TestExternalSignerErrors/invalid_signature_base64 (0.00s)5816 --- PASS: TestExternalSignerErrors/wrong_id (0.00s)5817 --- PASS: TestExternalSignerErrors/exit_on_request (0.00s)5818 --- PASS: TestExternalSignerErrors/null (0.01s)5819 --- PASS: TestExternalSignerErrors/null_signatures (0.00s)5820 --- PASS: TestExternalSignerErrors/wrong_signature_name (0.01s)5821 --- PASS: TestExternalSignerErrors/missing_signatures (0.00s)5822 --- PASS: TestExternalSignerErrors/exit_at_startup (0.00s)5823 --- PASS: TestExternalSignerErrors/too_few_signatures (0.01s)5824 --- PASS: TestExternalSignerErrors/program_error (0.00s)5825 --- PASS: TestExternalSignerErrors/missing_signature_name (0.01s)5826 --- PASS: TestExternalSignerErrors/unrequested_key (0.01s)5827 --- PASS: TestExternalSignerErrors/wrong_signatures_shape (0.01s)5828--- PASS: TestExternalSignerCancellation (0.02s)5829PASS5830Running OpenBao signer tests...5831=== RUN TestSign5832=== PAUSE TestSign5833=== RUN TestSignErrors5834=== PAUSE TestSignErrors5835=== RUN TestServe5836=== PAUSE TestServe5837=== RUN TestServeRejectsGarbage5838=== PAUSE TestServeRejectsGarbage5839=== CONT TestSign5840=== CONT TestServe5841=== CONT TestSignErrors5842=== CONT TestServeRejectsGarbage5843=== RUN TestSignErrors/http_error5844=== PAUSE TestSignErrors/http_error5845--- PASS: TestServeRejectsGarbage (0.00s)5846=== RUN TestSignErrors/invalid_json5847=== PAUSE TestSignErrors/invalid_json5848--- PASS: TestServe (0.00s)5849=== RUN TestSignErrors/no_prefix5850=== PAUSE TestSignErrors/no_prefix5851=== RUN TestSignErrors/invalid_base645852=== PAUSE TestSignErrors/invalid_base645853=== CONT TestSignErrors/http_error5854=== CONT TestSignErrors/no_prefix5855=== CONT TestSignErrors/invalid_base645856=== CONT TestSignErrors/invalid_json5857--- PASS: TestSign (0.00s)5858--- PASS: TestSignErrors (0.00s)5859 --- PASS: TestSignErrors/invalid_json (0.00s)5860 --- PASS: TestSignErrors/http_error (0.00s)5861 --- PASS: TestSignErrors/invalid_base64 (0.00s)5862 --- PASS: TestSignErrors/no_prefix (0.00s)5863PASS