niks3-go-unit-tests
checks.aarch64-darwin.go-unit-tests
· build #507
· raw
1tribuchet: building on mac022Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestUploadBuildLog_FileBodyReplayedOnRetry6=== PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry7=== RUN TestRegisterUploadedObjectReusesConnections8=== PAUSE TestRegisterUploadedObjectReusesConnections9=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica10=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica11=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun12=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun13=== RUN TestCaseHackSuffix14=== PAUSE TestCaseHackSuffix15=== RUN TestFilterOversizedClosures16=== PAUSE TestFilterOversizedClosures17=== RUN TestUploadMultipart_PartsInParallel18=== PAUSE TestUploadMultipart_PartsInParallel19=== RUN TestUploadMultipart_ReturnsAfterDroppedBody20=== PAUSE TestUploadMultipart_ReturnsAfterDroppedBody21=== RUN TestUploadMultipart_FailedProducerIsNotCompleted22=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted23=== RUN TestPartSizeForNAR24=== PAUSE TestPartSizeForNAR25=== RUN TestDumpPathCaseHackMatchesNix26--- PASS: TestDumpPathCaseHackMatchesNix (0.13s)27=== RUN TestDumpPathCaseHackCollision28--- PASS: TestDumpPathCaseHackCollision (0.00s)29=== RUN TestDumpPathMatchesNix30=== PAUSE TestDumpPathMatchesNix31=== RUN TestDumpPathSingleFile32=== PAUSE TestDumpPathSingleFile33=== RUN TestDumpPathWriterError34=== PAUSE TestDumpPathWriterError35=== RUN TestEncodeNixBase3236=== PAUSE TestEncodeNixBase3237=== RUN TestEncodeNixBase32WithRealHash38=== PAUSE TestEncodeNixBase32WithRealHash39=== RUN TestConvertHashToNix3240=== PAUSE TestConvertHashToNix3241=== RUN TestGetStorePathHash42=== PAUSE TestGetStorePathHash43=== RUN TestPathInfoHashCompatibility44=== PAUSE TestPathInfoHashCompatibility45=== RUN TestParsePathInfoJSON46=== PAUSE TestParsePathInfoJSON47=== RUN TestParsePathInfoJSONMultiplePaths48=== PAUSE TestParsePathInfoJSONMultiplePaths49=== RUN TestPathInfoCACompatibility50=== PAUSE TestPathInfoCACompatibility51=== RUN TestRateLimiterFeedback52=== PAUSE TestRateLimiterFeedback53=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess54=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess55=== RUN TestRegisterUploadedObject_BoundedAgainstSilentServer56=== PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer57=== RUN TestRegisterUploadedObject_OutlivesPush58=== PAUSE TestRegisterUploadedObject_OutlivesPush59=== RUN TestResolveStorePath60=== PAUSE TestResolveStorePath61=== RUN TestDoWithRetry_BodyReplayedViaGetBody62=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody63=== RUN TestRetryTimeoutOutlastsMaxRetries64=== PAUSE TestRetryTimeoutOutlastsMaxRetries65=== RUN TestDoWithRetry_FinalResponseBodyReadable66=== PAUSE TestDoWithRetry_FinalResponseBodyReadable67=== RUN TestShellSplit68=== PAUSE TestShellSplit69=== RUN TestShellSplitErrors70=== PAUSE TestShellSplitErrors71=== RUN TestStreamPushReportsEveryPath72=== PAUSE TestStreamPushReportsEveryPath73=== RUN TestStreamPushBatchesUnderLoad74=== PAUSE TestStreamPushBatchesUnderLoad75=== RUN TestStreamPushIsolatesFailures76=== PAUSE TestStreamPushIsolatesFailures77=== RUN TestStreamPushGivesUpOnDeadServer78=== PAUSE TestStreamPushGivesUpOnDeadServer79=== RUN TestStreamPushRequestLine80=== PAUSE TestStreamPushRequestLine81=== RUN TestStreamPushReportsSignatures82=== PAUSE TestStreamPushReportsSignatures83=== RUN TestStreamPushReportsSkippedPaths84=== PAUSE TestStreamPushReportsSkippedPaths85=== RUN TestClientSignaturesByStorePath86=== PAUSE TestClientSignaturesByStorePath87=== RUN TestStreamPushAnswersEveryLineWhenCanceled88=== PAUSE TestStreamPushAnswersEveryLineWhenCanceled89=== RUN TestStreamPushStopsWhenCanceled90=== PAUSE TestStreamPushStopsWhenCanceled91=== RUN TestSetClientTLS92=== PAUSE TestSetClientTLS93=== RUN TestSetClientTLSDoesNotMutateDefaultTransport94=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport95=== RUN TestSetClientTLSErrors96=== PAUSE TestSetClientTLSErrors97=== RUN TestStaticToken98=== PAUSE TestStaticToken99=== RUN TestFileTokenReadsAndCaches100=== PAUSE TestFileTokenReadsAndCaches101=== RUN TestFileTokenMissing102=== PAUSE TestFileTokenMissing103=== RUN TestFileTokenEmpty104=== PAUSE TestFileTokenEmpty105=== RUN TestScriptTokenNoExpiryRerunsEveryCall106=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall107=== RUN TestScriptTokenCachesUntilRefresh108=== PAUSE TestScriptTokenCachesUntilRefresh109=== RUN TestScriptTokenEmptyToken110=== PAUSE TestScriptTokenEmptyToken111=== RUN TestScriptTokenBadJSON112=== PAUSE TestScriptTokenBadJSON113=== RUN TestScriptTokenScriptFails114=== PAUSE TestScriptTokenScriptFails115=== RUN TestScriptTokenEmptyCommand116=== PAUSE TestScriptTokenEmptyCommand117=== CONT TestDoServerRequestAttachesToken118=== CONT TestRetryTimeoutOutlastsMaxRetries119=== CONT TestScriptTokenEmptyCommand120--- PASS: TestScriptTokenEmptyCommand (0.00s)121=== CONT TestScriptTokenScriptFails122=== CONT TestStreamPushGivesUpOnDeadServer123=== CONT TestSetClientTLS124=== CONT TestStreamPushIsolatesFailures125=== CONT TestStreamPushBatchesUnderLoad126=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=false127=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=false128=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=true129=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=true130=== CONT TestPathInfoCACompatibility1312026/11/12 07:15:16 ERROR Upload failed error="connection refused" count=201322026/11/12 07:15:16 ERROR Server seems unavailable, giving up on batch untried=171332026/11/12 07:15:16 ERROR Upload failed error="bad path" count=3134=== CONT TestStreamPushReportsEveryPath135=== CONT TestShellSplitErrors136--- PASS: TestShellSplitErrors (0.00s)137=== CONT TestShellSplit138=== CONT TestDoWithRetry_FinalResponseBodyReadable139=== CONT TestClientSignaturesByStorePath140=== CONT TestStreamPushStopsWhenCanceled141=== CONT TestStreamPushAnswersEveryLineWhenCanceled142=== CONT TestStreamPushReportsSignatures143=== CONT TestStreamPushReportsSkippedPaths144=== CONT TestUploadMultipart_PartsInParallel145=== CONT TestDumpPathWriterError146=== CONT TestDumpPathSingleFile147=== CONT TestDumpPathMatchesNix148=== CONT TestPartSizeForNAR149=== CONT TestUploadMultipart_FailedProducerIsNotCompleted150=== CONT TestUploadMultipart_ReturnsAfterDroppedBody151=== CONT TestEncodeNixBase32152=== RUN TestPathInfoCACompatibility/null_ca_field153=== PAUSE TestPathInfoCACompatibility/null_ca_field154=== RUN TestPathInfoCACompatibility/old_string_format_-_text155=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text156=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive157=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive158=== RUN TestPathInfoCACompatibility/new_structured_format_-_text159=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text160=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method161=== CONT TestDoWithRetry_BodyReplayedViaGetBody162--- PASS: TestClientSignaturesByStorePath (0.00s)163--- PASS: TestShellSplit (0.00s)164--- PASS: TestStreamPushStopsWhenCanceled (0.00s)165--- PASS: TestStreamPushIsolatesFailures (0.00s)166=== CONT TestResolveStorePath1672000/01/01 00:00:00 ERROR Upload failed error="context canceled" count=2168=== CONT TestRegisterUploadedObject_OutlivesPush169=== CONT TestRegisterUploadedObject_BoundedAgainstSilentServer170--- PASS: TestStreamPushGivesUpOnDeadServer (0.00s)171=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess172=== CONT TestRateLimiterFeedback173--- PASS: TestResolveStorePath (0.00s)1742026/11/12 07:15:16 ERROR Upload failed error=boom count=1175=== CONT TestFileTokenEmpty176=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part177=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part178=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary179=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary1802026/11/12 07:15:16 WARN Rate limiter enabled after throttle name=server-test rate=5181=== CONT TestScriptTokenEmptyToken182=== CONT TestScriptTokenCachesUntilRefresh183=== CONT TestScriptTokenNoExpiryRerunsEveryCall184--- PASS: TestStreamPushReportsSignatures (0.00s)185--- PASS: TestStreamPushAnswersEveryLineWhenCanceled (0.00s)186=== CONT TestStreamPushRequestLine187--- PASS: TestStreamPushReportsSkippedPaths (0.00s)188--- PASS: TestScriptTokenScriptFails (0.01s)189=== CONT TestScriptTokenBadJSON1902026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:60297191=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method192=== RUN TestRateLimiterFeedback/429_enables_limiter193=== PAUSE TestRateLimiterFeedback/429_enables_limiter194=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun195=== RUN TestRateLimiterFeedback/503_enables_limiter196=== RUN TestPartSizeForNAR/zero_stays_at_minimum197=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum198=== PAUSE TestRateLimiterFeedback/503_enables_limiter199=== RUN TestPartSizeForNAR/small_stays_at_minimum200=== PAUSE TestPartSizeForNAR/small_stays_at_minimum201--- PASS: TestStreamPushReportsEveryPath (0.01s)202=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter203=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter204=== CONT TestFilterOversizedClosures205=== RUN TestFilterOversizedClosures/no_limit_keeps_everything206=== RUN TestEncodeNixBase32/test_string_hash207=== PAUSE TestEncodeNixBase32/test_string_hash208=== RUN TestEncodeNixBase32/empty_input209=== PAUSE TestEncodeNixBase32/empty_input210=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum211=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum212=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts213=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts214=== RUN TestPartSizeForNAR/1_TiB215=== PAUSE TestPartSizeForNAR/1_TiB216=== RUN TestPartSizeForNAR/5_TiB_S3_max_object217=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object218=== RUN TestPartSizeForNAR/capped_at_5_GiB219=== PAUSE TestPartSizeForNAR/capped_at_5_GiB220=== CONT TestCaseHackSuffix2212026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:602972222026/11/12 07:15:16 WARN Rate limiter enabled after throttle name=server-test rate=52232026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:60307224--- PASS: TestRegisterUploadedObject_OutlivesPush (0.00s)225=== CONT TestParsePathInfoJSONMultiplePaths226=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths227=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths228=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths229=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths230=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica231--- PASS: TestDoServerRequestAttachesToken (0.01s)232=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter233=== CONT TestParsePathInfoJSON234=== RUN TestParsePathInfoJSON/Nix_format235=== PAUSE TestParsePathInfoJSON/Nix_format236=== RUN TestParsePathInfoJSON/Lix_format237=== PAUSE TestParsePathInfoJSON/Lix_format238=== RUN TestParsePathInfoJSON/empty_input239=== PAUSE TestParsePathInfoJSON/empty_input2402026/11/12 07:15:16 WARN Rate limiter backed off name=server-test rate=5241=== RUN TestParsePathInfoJSON/whitespace_only2422026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:60307243=== PAUSE TestParsePathInfoJSON/whitespace_only244=== RUN TestParsePathInfoJSON/invalid_JSON245=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything246=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped247=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped248=== RUN TestFilterOversizedClosures/all_closures_skipped249=== PAUSE TestFilterOversizedClosures/all_closures_skipped250=== CONT TestRegisterUploadedObjectReusesConnections251=== CONT TestFileTokenMissing252=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter253=== PAUSE TestParsePathInfoJSON/invalid_JSON254=== CONT TestPathInfoHashCompatibility255=== CONT TestConvertHashToNix32256=== RUN TestConvertHashToNix32/SRI_format_to_Nix32257=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32258=== RUN TestConvertHashToNix32/already_Nix32_format259=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)260=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)261=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon262=== PAUSE TestConvertHashToNix32/already_Nix32_format263=== RUN TestConvertHashToNix32/invalid_format264=== PAUSE TestConvertHashToNix32/invalid_format265=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon266=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI267=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI268=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512269=== CONT TestEncodeNixBase32WithRealHash270--- PASS: TestEncodeNixBase32WithRealHash (0.00s)271=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512272=== CONT TestGetStorePathHash273=== CONT TestStaticToken274--- PASS: TestStaticToken (0.00s)275=== RUN TestGetStorePathHash/valid_store_path276=== CONT TestSetClientTLSErrors277=== PAUSE TestGetStorePathHash/valid_store_path278=== RUN TestGetStorePathHash/basename_without_hyphen_should_error279=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error280=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error281=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error282=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error283=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error284=== CONT TestUploadBuildLog_FileBodyReplayedOnRetry285--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.00s)286=== CONT TestSetClientTLSDoesNotMutateDefaultTransport2872026/11/12 07:15:16 ERROR Upload failed error=boom count=12882026/11/12 07:15:16 INFO Garbage collection started2892026/11/12 07:15:16 WARN Rate limiter enabled after throttle name=server-test rate=52902026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:60298291--- PASS: TestFileTokenMissing (0.00s)292=== CONT TestFileTokenReadsAndCaches293=== RUN TestSetClientTLSErrors/missing_cert_file294=== PAUSE TestSetClientTLSErrors/missing_cert_file295=== RUN TestSetClientTLSErrors/missing_key_file296=== PAUSE TestSetClientTLSErrors/missing_key_file297=== RUN TestSetClientTLSErrors/missing_ca_file298=== PAUSE TestSetClientTLSErrors/missing_ca_file299=== RUN TestSetClientTLSErrors/invalid_ca_file300=== PAUSE TestSetClientTLSErrors/invalid_ca_file301=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=false302=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=true303--- PASS: TestStreamPushBatchesUnderLoad (0.00s)304 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=false (0.00s)305 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=true (0.00s)306=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part307--- PASS: TestFileTokenEmpty (0.01s)308=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary309--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.00s)310=== CONT TestPathInfoCACompatibility/null_ca_field311=== CONT TestPathInfoCACompatibility/new_structured_format_-_text312=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive313=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method314=== CONT TestPathInfoCACompatibility/old_string_format_-_text315--- PASS: TestPathInfoCACompatibility (0.01s)316 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)317 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)318 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)319 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)320 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)321=== CONT TestEncodeNixBase32/test_string_hash322=== CONT TestPartSizeForNAR/zero_stays_at_minimum323=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths324=== CONT TestEncodeNixBase32/empty_input325--- PASS: TestEncodeNixBase32 (0.00s)326 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)327 --- PASS: TestEncodeNixBase32/empty_input (0.00s)328=== CONT TestPartSizeForNAR/capped_at_5_GiB329=== CONT TestPartSizeForNAR/5_TiB_S3_max_object330=== CONT TestPartSizeForNAR/1_TiB331=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts332=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum333=== CONT TestPartSizeForNAR/small_stays_at_minimum334--- PASS: TestPartSizeForNAR (0.00s)335 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)336 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)337 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)338 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)339 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)340 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)341 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)342=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths343--- PASS: TestParsePathInfoJSONMultiplePaths (0.00s)344 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)345 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)346=== CONT TestFilterOversizedClosures/no_limit_keeps_everything347=== CONT TestFilterOversizedClosures/all_closures_skipped3482026/11/12 07:15:16 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50349=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped3502026/11/12 07:15:16 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000351--- PASS: TestFilterOversizedClosures (0.00s)352 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)353 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)354 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)355=== CONT TestRateLimiterFeedback/429_enables_limiter3562026/11/12 07:15:16 WARN Rate limiter enabled after throttle name=s3-test rate=53572026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:603293582026/11/12 07:15:16 INFO Garbage collection started359--- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (0.01s)360=== CONT TestParsePathInfoJSON/Nix_format361=== RUN TestSetClientTLS/rejects_connection_without_client_cert362=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert363=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA364=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA365=== RUN TestSetClientTLS/preserves_debug_logging_transport366=== PAUSE TestSetClientTLS/preserves_debug_logging_transport367=== CONT TestParsePathInfoJSON/invalid_JSON368=== CONT TestParsePathInfoJSON/whitespace_only369=== CONT TestParsePathInfoJSON/empty_input370=== CONT TestParsePathInfoJSON/Lix_format371=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter372--- PASS: TestParsePathInfoJSON (0.00s)373 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)374 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)375 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)376 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)377 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)378=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter379=== CONT TestRateLimiterFeedback/503_enables_limiter380--- PASS: TestFileTokenReadsAndCaches (0.01s)381=== CONT TestConvertHashToNix32/SRI_format_to_Nix32382=== CONT TestConvertHashToNix32/invalid_format383=== CONT TestConvertHashToNix32/already_Nix32_format384--- PASS: TestConvertHashToNix32 (0.00s)385 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)386 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)387 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)388=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)389=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512390=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI391=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon392--- PASS: TestPathInfoHashCompatibility (0.00s)393 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)394 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)395 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)396 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)397=== CONT TestGetStorePathHash/valid_store_path398=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error399=== CONT TestGetStorePathHash/basename_without_hyphen_should_error400=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error401--- PASS: TestGetStorePathHash (0.00s)402 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)403 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)404 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)405 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)406=== CONT TestSetClientTLSErrors/missing_cert_file407=== CONT TestSetClientTLSErrors/invalid_ca_file408=== CONT TestSetClientTLSErrors/missing_ca_file409=== CONT TestSetClientTLSErrors/missing_key_file4102026/11/12 07:15:16 WARN Rate limiter enabled after throttle name=server-test rate=54112026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:603814122026/11/12 07:15:16 WARN Rate limiter enabled after throttle name=server-test rate=54132026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:60327414=== CONT TestSetClientTLS/rejects_connection_without_client_cert415--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (0.01s)416=== CONT TestSetClientTLS/preserves_debug_logging_transport417--- PASS: TestSetClientTLSErrors (0.00s)418 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)419 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)420 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)421 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)4222026/11/12 07:15:16 WARN Rate limiter backed off name=server-test rate=5423=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA4242026/11/12 07:15:16 WARN Rate limiter backed off name=server-test rate=5425--- PASS: TestRateLimiterFeedback (0.00s)426 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.01s)427 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.00s)428 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s)429 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s)4302026/11/12 07:15:16 WARN Rate limiter backed off name=server-test rate=5431--- PASS: TestUploadMultipart_ReturnsAfterDroppedBody (0.02s)4322026/11/12 07:15:16 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=0433--- PASS: TestDoWithRetry_FinalResponseBodyReadable (0.02s)434--- PASS: TestUploadMultipart_FailedProducerIsNotCompleted (0.00s)435 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary (0.01s)436 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part (0.01s)4372026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:602974382026/11/12 07:15:16 WARN Garbage collection finished on another replica. Its statistics are not available here439--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (0.02s)4402026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:602974412026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:602974422026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:602974432026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:60297444--- PASS: TestStreamPushRequestLine (0.03s)4452026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:602974462026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:602974472026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:602974482026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:602974492026/11/12 07:15:16 http: TLS handshake error from 127.0.0.1:60402: read tcp 127.0.0.1:60371->127.0.0.1:60402: use of closed network connection4502026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:60297451--- PASS: TestSetClientTLS (0.02s)452 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.00s)453 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.00s)454 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.03s)455--- PASS: TestRegisterUploadedObjectReusesConnections (0.04s)456--- PASS: TestScriptTokenBadJSON (0.04s)4572026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:60297458--- PASS: TestScriptTokenEmptyToken (0.04s)4592026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:602974602026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:602974612026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:602974622026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:602974632026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:602974642026/11/12 07:15:16 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:60297465--- PASS: TestRetryTimeoutOutlastsMaxRetries (0.06s)466--- PASS: TestDumpPathWriterError (0.06s)467--- PASS: TestScriptTokenCachesUntilRefresh (0.06s)468--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.06s)469--- PASS: TestCaseHackSuffix (0.06s)470--- PASS: TestDumpPathSingleFile (0.07s)471--- PASS: TestDumpPathMatchesNix (0.09s)4722026/11/12 07:15:16 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:60306/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:60306/api/uploads/complete4732026/11/12 07:15:16 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4742026/11/12 07:15:16 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:60306/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:60306/api/uploads/complete4752026/11/12 07:15:16 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4762026/11/12 07:15:16 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:60306/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:60306/api/uploads/complete4772026/11/12 07:15:16 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"478--- PASS: TestRegisterUploadedObject_BoundedAgainstSilentServer (0.10s)479--- PASS: TestUploadMultipart_PartsInParallel (0.61s)480--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)481PASS482Running server tests...483The files belonging to this database system will be owned by user "_tribuchetbld1".484This user must also own the server process.485486The database cluster will be initialized with locale "C".487The default database encoding has accordingly been set to "SQL_ASCII".488The default text search configuration will be set to "english".489490Data page checksums are enabled.491492creating directory /private/var/tb/1/scratch/9cc0db790b9bff21/build/postgres1392460363/data ... ok493creating subdirectories ... ok494selecting dynamic shared memory implementation ... posix495selecting default "max_connections" ... 100496selecting default "shared_buffers" ... 128MB497selecting default time zone ... UTC498creating configuration files ... ok499running bootstrap script ... ok500performing post-bootstrap initialization ... ok501syncing data to disk ... ok502503initdb: warning: enabling "trust" authentication for local connections504initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.505506Success. You can now start the database server using:507508 pg_ctl -D /private/var/tb/1/scratch/9cc0db790b9bff21/build/postgres1392460363/data -l logfile start5095102026-11-12 07:15:19.847 UTC [70726] LOG: starting PostgreSQL 18.6 on aarch64-apple-darwin25.6.0, compiled by clang version 21.1.8, 64-bit5112026-11-12 07:15:19.847 UTC [70726] LOG: listening on Unix socket "/private/var/tb/1/scratch/9cc0db790b9bff21/build/postgres1392460363/.s.PGSQL.5432"5122026-11-12 07:15:19.849 UTC [70733] LOG: database system was shut down at 2026-11-12 07:15:19 UTC5132026-11-12 07:15:19.850 UTC [70726] LOG: database system is ready to accept connections514/private/var/tb/1/scratch/9cc0db790b9bff21/build/postgres1392460363:5432 - accepting connections515{"timestamp":"2026-11-12T07:15:22.157425Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"45450edb-7f9c-43cc-9533-37fe00c10655","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":1,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(9)"}516=== RUN TestAbortSucceeded517=== PAUSE TestAbortSucceeded518=== RUN TestService_AuthMiddleware519=== PAUSE TestService_AuthMiddleware520=== RUN TestService_AuthMiddleware_MTLSProxyHeader521=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader522=== RUN TestService_AuthMiddleware_MTLSBoundSubjects523=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects524=== RUN TestService_ReadAuthMiddleware525=== PAUSE TestService_ReadAuthMiddleware526=== RUN TestService_AuthMiddleware_OIDC527=== PAUSE TestService_AuthMiddleware_OIDC528=== RUN TestService_RequireScope_OIDC529=== PAUSE TestService_RequireScope_OIDC530=== RUN TestService_ReadScope_PublicByDefault531=== PAUSE TestService_ReadScope_PublicByDefault532=== RUN TestCacheConfigHandler533=== PAUSE TestCacheConfigHandler534=== RUN TestCacheStatsHandler535=== PAUSE TestCacheStatsHandler536=== RUN TestClientCADerivations537=== PAUSE TestClientCADerivations538=== RUN TestClientErrorHandling539=== PAUSE TestClientErrorHandling540=== RUN TestClientIntegration541=== PAUSE TestClientIntegration542=== RUN TestClientMultipleUploads543=== PAUSE TestClientMultipleUploads544=== RUN TestClientWithDependencies545=== PAUSE TestClientWithDependencies546=== RUN TestClientSharedPathCommittedMidPush547=== PAUSE TestClientSharedPathCommittedMidPush548=== RUN TestPinProtectsFromGC549=== PAUSE TestPinProtectsFromGC550=== RUN TestClientPushesUseOnePush551=== PAUSE TestClientPushesUseOnePush552=== RUN TestClientFallsBackToClosures553=== PAUSE TestClientFallsBackToClosures554=== RUN TestDeleteClosuresRacingPins555=== PAUSE TestDeleteClosuresRacingPins556=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock557=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock558=== RUN TestResolveDBConnectionString559=== PAUSE TestResolveDBConnectionString560=== RUN TestConnectWaitsForAPeerMigration561=== PAUSE TestConnectWaitsForAPeerMigration562=== RUN TestLeadElectsOneAndHandsOver563=== PAUSE TestLeadElectsOneAndHandsOver564=== RUN TestLeadIncumbentWinsAfterRestart5652026/11/12 07:15:22 INFO lead: acquired remote=192.0.2.1:1234 generation=15662026/11/12 07:15:23 INFO lead: released remote=192.0.2.1:1234 generation=15672026/11/12 07:15:23 INFO lead: acquired remote=192.0.2.1:1234 generation=25682026/11/12 07:15:24 INFO lead: released remote=192.0.2.1:1234 generation=2569--- PASS: TestLeadIncumbentWinsAfterRestart (2.14s)570=== RUN TestLeadUnannouncedStandbyYieldsToIncumbent571=== PAUSE TestLeadUnannouncedStandbyYieldsToIncumbent572=== RUN TestLeadIncumbentAnnouncesAtOnce573=== PAUSE TestLeadIncumbentAnnouncesAtOnce574=== RUN TestLeadGenerationGrowsWithEveryGrant575=== PAUSE TestLeadGenerationGrowsWithEveryGrant576=== RUN TestLeadEndsOnShutdown577=== PAUSE TestLeadEndsOnShutdown578=== RUN TestLeadEndsWhenItsConnectionHangs5792026/11/12 07:15:24 INFO lead: acquired remote=192.0.2.1:1234 generation=15802026-11-12 07:15:25.967 UTC [70769] FATAL: terminating connection due to administrator command5812026/11/12 07:15:26 INFO lead: acquired remote=192.0.2.1:1234 generation=25822026/11/12 07:15:27 WARN lead: lock connection lost error="timeout: context deadline exceeded"5832026/11/12 07:15:27 INFO lead: released remote=192.0.2.1:1234 generation=15842026/11/12 07:15:27 INFO lead: released remote=192.0.2.1:1234 generation=2585--- PASS: TestLeadEndsWhenItsConnectionHangs (2.96s)586=== RUN TestGCAdvisoryLockBlocksConcurrentRun587=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun588=== RUN TestGCBugBareHashReferences589=== PAUSE TestGCBugBareHashReferences590=== RUN TestGCMetrics591=== PAUSE TestGCMetrics592=== RUN TestPushDedupSurvivesConcurrentGC593=== PAUSE TestPushDedupSurvivesConcurrentGC594=== RUN TestDeduplicatedObjectsRecordedAsPending595=== PAUSE TestDeduplicatedObjectsRecordedAsPending596=== RUN TestGCSweepSkipsPendingObjects597=== PAUSE TestGCSweepSkipsPendingObjects598=== RUN TestTombstonedObjectOfferedWithoutWaiting599=== PAUSE TestTombstonedObjectOfferedWithoutWaiting600=== RUN TestGCSweepDeliversEachKeyOnce601=== PAUSE TestGCSweepDeliversEachKeyOnce602=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection603=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection604=== RUN TestForceGCDuringPushOffersSweptObject605=== PAUSE TestForceGCDuringPushOffersSweptObject606=== RUN TestFailedS3DeleteKeepsTombstone607=== PAUSE TestFailedS3DeleteKeepsTombstone608=== RUN TestSweepSkipsRowsLockedByPush609=== PAUSE TestSweepSkipsRowsLockedByPush610=== RUN TestRegisterCompletedObjectNeedsPendingRow611=== PAUSE TestRegisterCompletedObjectNeedsPendingRow612=== RUN TestGetExistingObjectsLocksRows613=== PAUSE TestGetExistingObjectsLocksRows614=== RUN TestSweepRechecksPendingAfterLocking615=== PAUSE TestSweepRechecksPendingAfterLocking616=== RUN TestPushWaitsForSweepHoldingRow617=== PAUSE TestPushWaitsForSweepHoldingRow618=== RUN TestCommitRacingPendingCleanupKeepsObjects619=== PAUSE TestCommitRacingPendingCleanupKeepsObjects620=== RUN TestGCEndsOnShutdown621=== PAUSE TestGCEndsOnShutdown622=== RUN TestGCTaskStore_StartNew623=== PAUSE TestGCTaskStore_StartNew624=== RUN TestGCTaskStore_DeduplicateSameParams625=== PAUSE TestGCTaskStore_DeduplicateSameParams626=== RUN TestGCTaskStore_ConflictDifferentParams627=== PAUSE TestGCTaskStore_ConflictDifferentParams628=== RUN TestGCTaskStore_GetEmpty629=== PAUSE TestGCTaskStore_GetEmpty630=== RUN TestGCTaskStore_GetReturnsLatest631=== PAUSE TestGCTaskStore_GetReturnsLatest632=== RUN TestGCTaskStore_CompletedAllowsNewTask633=== PAUSE TestGCTaskStore_CompletedAllowsNewTask634=== RUN TestGCTaskStore_PhaseUpdates635=== PAUSE TestGCTaskStore_PhaseUpdates636=== RUN TestGCTaskStore_Fail637=== PAUSE TestGCTaskStore_Fail638=== RUN TestGracefulShutdownDrainsInflight639=== PAUSE TestGracefulShutdownDrainsInflight640=== RUN TestService_healthCheckHandler641=== PAUSE TestService_healthCheckHandler642=== RUN TestService_readinessHandler643=== PAUSE TestService_readinessHandler644=== RUN TestGenerateLandingPage645=== PAUSE TestGenerateLandingPage646=== RUN TestEnsureAbortUploadsRule647=== PAUSE TestEnsureAbortUploadsRule648=== RUN TestCacheConfigHandlerMaxNarSize649=== PAUSE TestCacheConfigHandlerMaxNarSize650=== RUN TestCreatePendingClosureRejectsOversizedNAR651=== PAUSE TestCreatePendingClosureRejectsOversizedNAR652=== RUN TestNARDeduplicationMetadataUploadBug653=== PAUSE TestNARDeduplicationMetadataUploadBug654=== RUN TestMetricsInventory655=== PAUSE TestMetricsInventory656=== RUN TestMetricsCommit657=== PAUSE TestMetricsCommit658=== RUN TestService_NativeMTLS659=== PAUSE TestService_NativeMTLS660=== RUN TestServerTLSConfig661=== PAUSE TestServerTLSConfig662=== RUN TestMultipartCleanup663=== PAUSE TestMultipartCleanup664=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded665=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded666=== RUN TestPendingCleanupUsesOneCutoff667=== PAUSE TestPendingCleanupUsesOneCutoff668=== RUN TestObjectStatsTrigger669=== PAUSE TestObjectStatsTrigger670=== RUN TestObjectStatsUpdatedOncePerCommit671=== PAUSE TestObjectStatsUpdatedOncePerCommit672=== RUN TestOrphanedObjectsGC673=== PAUSE TestOrphanedObjectsGC674=== RUN TestOrphanedObjectsGCStressTest675=== PAUSE TestOrphanedObjectsGCStressTest676=== RUN TestCreatePin_ReservedPins677=== PAUSE TestCreatePin_ReservedPins678=== RUN TestPresentReportsOnlyClosureRoots679=== PAUSE TestPresentReportsOnlyClosureRoots680=== RUN TestPresentRefreshesOnlyStaleClosures681=== PAUSE TestPresentRefreshesOnlyStaleClosures682=== RUN TestPresentNotReportedWhileGCDeletesClosure683=== PAUSE TestPresentNotReportedWhileGCDeletesClosure684=== RUN TestParseSingleRange685=== PAUSE TestParseSingleRange686=== RUN TestProxyHeadersOnlyTrustedOnSocket6872026/11/12 07:15:27 INFO Starting HTTP server address=proxy.sock6882026/11/12 07:15:27 INFO Starting HTTP server address=127.0.0.1:604666892026/11/12 07:15:27 WARN mTLS auth: subject not in bound subjects subject="CN=someone"6902026/11/12 07:15:27 INFO Shutdown signal received, draining in-flight requests timeout=10s691--- PASS: TestProxyHeadersOnlyTrustedOnSocket (0.16s)692=== RUN TestIsValidCachePath693=== PAUSE TestIsValidCachePath694=== RUN TestReadProxyNarinfo695=== PAUSE TestReadProxyNarinfo696=== RUN TestReadProxyNarinfoAlreadyDecompressed697=== PAUSE TestReadProxyNarinfoAlreadyDecompressed698=== RUN TestReadProxyContentNegotiation699=== PAUSE TestReadProxyContentNegotiation700=== RUN TestReadProxyNarStreaming701=== PAUSE TestReadProxyNarStreaming702=== RUN TestReadProxy404703=== PAUSE TestReadProxy404704=== RUN TestReadProxyInvalidPath705=== PAUSE TestReadProxyInvalidPath706=== RUN TestReadProxyHead707=== PAUSE TestReadProxyHead708=== RUN TestReadProxyOutlastsServerWriteTimeout709=== PAUSE TestReadProxyOutlastsServerWriteTimeout710=== RUN TestReadProxyConditionalGet711=== PAUSE TestReadProxyConditionalGet712=== RUN TestReadProxyRootRedirectsToIndexHTML713=== PAUSE TestReadProxyRootRedirectsToIndexHTML714=== RUN TestReadProxyDisabled715=== PAUSE TestReadProxyDisabled716=== RUN TestReadRedirectNar717=== PAUSE TestReadRedirectNar718=== RUN TestReadRedirectKeepsNarinfoProxied719=== PAUSE TestReadRedirectKeepsNarinfoProxied720=== RUN TestReadProxyRangeRequest721=== PAUSE TestReadProxyRangeRequest722=== RUN TestReadRedirectUsesPublicS3URL723=== PAUSE TestReadRedirectUsesPublicS3URL724=== RUN TestPush_OverlappingRootsStoreOneRowPerKey725=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey726=== RUN TestPush_CompleteCommitsEveryRoot727=== PAUSE TestPush_CompleteCommitsEveryRoot728=== RUN TestPush_RepeatedRootCommitsOnce729=== PAUSE TestPush_RepeatedRootCommitsOnce730=== RUN TestPush_CompleteWaitsForCleanupOfThePush731=== PAUSE TestPush_CompleteWaitsForCleanupOfThePush732=== RUN TestPush_RootsAreLockedInByteOrder733=== PAUSE TestPush_RootsAreLockedInByteOrder734=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit735=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit736=== RUN TestPush_RejectsBadRequests737=== PAUSE TestPush_RejectsBadRequests738=== RUN TestPush_SignsNarinfosOfItsPendingObjects739=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects740=== RUN TestCommitCanBeRepeatedUntilCleanup741=== PAUSE TestCommitCanBeRepeatedUntilCleanup742=== RUN TestSharedNarBothUploadsComplete743=== PAUSE TestSharedNarBothUploadsComplete744=== RUN TestCompleteMultipartUpload_ErrorButObjectExists745=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists746=== RUN TestCompletedNarNotReofferedAcrossClosures747=== PAUSE TestCompletedNarNotReofferedAcrossClosures748=== RUN TestPresignedUploadRegisteredBeforeCommit749=== PAUSE TestPresignedUploadRegisteredBeforeCommit750=== RUN TestService_Rustfstest751=== PAUSE TestService_Rustfstest752=== RUN TestParseSize753=== PAUSE TestParseSize754=== RUN TestSkippedUploadsHandler755=== PAUSE TestSkippedUploadsHandler756=== RUN TestSystemdListenerNotActivated757--- PASS: TestSystemdListenerNotActivated (0.00s)758=== RUN TestWatchdogBeatsWhenHealthy759--- PASS: TestWatchdogBeatsWhenHealthy (0.03s)760=== RUN TestWatchdogSkipsWhenUnhealthy7612026/11/12 07:15:27 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7622026/11/12 07:15:27 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7632026/11/12 07:15:27 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7642026/11/12 07:15:27 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7652026/11/12 07:15:27 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7662026/11/12 07:15:27 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7672026/11/12 07:15:27 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7682026/11/12 07:15:27 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7692026/11/12 07:15:27 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7702026/11/12 07:15:27 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"771--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)772=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle773=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle774=== RUN TestProxyWriteTimeout775=== PAUSE TestProxyWriteTimeout776=== RUN TestPendingClosureWriteTimeout777=== PAUSE TestPendingClosureWriteTimeout778=== RUN TestIsValidUploadKey779=== PAUSE TestIsValidUploadKey780=== RUN TestUploadHandlersRejectInvalidKeys781=== PAUSE TestUploadHandlersRejectInvalidKeys782=== RUN TestUploadHandlersRejectOversizedBody783=== PAUSE TestUploadHandlersRejectOversizedBody784=== RUN TestService_cleanupPendingClosuresHandler785=== PAUSE TestService_cleanupPendingClosuresHandler786=== RUN TestService_createPendingClosureHandler787=== PAUSE TestService_createPendingClosureHandler788=== RUN TestService_verifyS3Integrity789=== PAUSE TestService_verifyS3Integrity790=== RUN TestService_verifyS3ReuploadsStaleNarinfo791=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo792=== RUN TestCompleteMultipartUnregistered793=== PAUSE TestCompleteMultipartUnregistered794=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT795=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT796=== CONT TestCompletedNarNotReofferedAcrossClosures797=== CONT TestAbortSucceeded798=== CONT TestPush_RepeatedRootCommitsOnce799=== CONT TestUploadHandlersRejectInvalidKeys800=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info801=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info802=== CONT TestCompleteMultipartUpload_ErrorButObjectExists803=== CONT TestSharedNarBothUploadsComplete804=== CONT TestCommitCanBeRepeatedUntilCleanup805=== CONT TestPush_SignsNarinfosOfItsPendingObjects806=== CONT TestPush_RejectsBadRequests807=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit808=== CONT TestPush_RootsAreLockedInByteOrder809=== CONT TestPush_CompleteWaitsForCleanupOfThePush810=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT811=== CONT TestCompleteMultipartUnregistered812=== CONT TestService_verifyS3ReuploadsStaleNarinfo813=== CONT TestService_verifyS3Integrity814--- PASS: TestAbortSucceeded (0.00s)815=== CONT TestService_createPendingClosureHandler816=== CONT TestGCTaskStore_Fail817=== CONT TestService_cleanupPendingClosuresHandler818=== CONT TestGCTaskStore_PhaseUpdates819=== CONT TestSkippedUploadsHandler820=== CONT TestUploadHandlersRejectOversizedBody821=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle822=== CONT TestIsValidUploadKey823=== CONT TestPendingClosureWriteTimeout824=== CONT TestProxyWriteTimeout825=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal826=== RUN TestCommitCanBeRepeatedUntilCleanup/pending_closure827=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed828=== RUN TestIsValidUploadKey/narinfo829=== PAUSE TestIsValidUploadKey/narinfo830=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal831--- PASS: TestGCTaskStore_Fail (0.00s)832--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)833=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed834=== RUN TestIsValidUploadKey/nar_zst835=== CONT TestParseSize836=== PAUSE TestIsValidUploadKey/nar_zst837=== CONT TestService_Rustfstest838=== RUN TestPendingClosureWriteTimeout/empty839=== PAUSE TestPendingClosureWriteTimeout/empty840=== RUN TestProxyWriteTimeout/narinfo841--- PASS: TestParseSize (0.00s)842=== PAUSE TestCommitCanBeRepeatedUntilCleanup/pending_closure843=== PAUSE TestProxyWriteTimeout/narinfo844=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key845=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key846=== RUN TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed847=== RUN TestCommitCanBeRepeatedUntilCleanup/push848=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed849=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key850=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key851=== PAUSE TestCommitCanBeRepeatedUntilCleanup/push852=== CONT TestLeadIncumbentAnnouncesAtOnce853=== CONT TestLeadUnannouncedStandbyYieldsToIncumbent854=== RUN TestIsValidUploadKey/nar_xz855=== RUN TestPendingClosureWriteTimeout/400_objects856=== RUN TestProxyWriteTimeout/1_GiB_nar857=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost858=== PAUSE TestIsValidUploadKey/nar_xz859=== RUN TestIsValidUploadKey/nar_plain8602026/11/12 07:15:27 INFO Client skipped oversized paths paths=3 nar_bytes=5000000000861=== PAUSE TestPendingClosureWriteTimeout/400_objects862=== RUN TestPendingClosureWriteTimeout/670k_objects863=== PAUSE TestPendingClosureWriteTimeout/670k_objects864=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline865=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline866=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost867=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed868=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed869=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed870=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed871=== CONT TestLeadElectsOneAndHandsOver872=== CONT TestPresentRefreshesOnlyStaleClosures873=== PAUSE TestProxyWriteTimeout/1_GiB_nar874=== RUN TestProxyWriteTimeout/10_GiB_nar875=== PAUSE TestProxyWriteTimeout/10_GiB_nar876=== RUN TestProxyWriteTimeout/unknown_size877=== PAUSE TestProxyWriteTimeout/unknown_size878=== CONT TestLeadGenerationGrowsWithEveryGrant879=== PAUSE TestIsValidUploadKey/nar_plain880=== RUN TestIsValidUploadKey/listing881=== PAUSE TestIsValidUploadKey/listing882=== RUN TestIsValidUploadKey/build_log883=== PAUSE TestIsValidUploadKey/build_log884=== RUN TestIsValidUploadKey/build_log_home-manager_file885=== PAUSE TestIsValidUploadKey/build_log_home-manager_file886=== RUN TestIsValidUploadKey/build_log_plus_in_name887=== PAUSE TestIsValidUploadKey/build_log_plus_in_name888=== RUN TestIsValidUploadKey/build_log_question_mark889=== PAUSE TestIsValidUploadKey/build_log_question_mark890=== RUN TestIsValidUploadKey/build_log_equals891=== PAUSE TestIsValidUploadKey/build_log_equals892=== RUN TestIsValidUploadKey/realisation893=== PAUSE TestIsValidUploadKey/realisation894=== RUN TestIsValidUploadKey/realisation_plus_in_output895=== PAUSE TestIsValidUploadKey/realisation_plus_in_output896=== RUN TestIsValidUploadKey/nix-cache-info897=== PAUSE TestIsValidUploadKey/nix-cache-info898=== RUN TestIsValidUploadKey/index.html899=== PAUSE TestIsValidUploadKey/index.html900=== RUN TestIsValidUploadKey/narinfo_key,_nar_type901=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type902=== RUN TestIsValidUploadKey/nar_key,_narinfo_type903=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type904=== RUN TestIsValidUploadKey/listing_key,_narinfo_type905=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type906=== RUN TestIsValidUploadKey/traversal907=== PAUSE TestIsValidUploadKey/traversal908=== RUN TestIsValidUploadKey/traversal_nar909=== PAUSE TestIsValidUploadKey/traversal_nar910=== RUN TestIsValidUploadKey/absolute911=== PAUSE TestIsValidUploadKey/absolute912=== RUN TestIsValidUploadKey/empty_key913=== PAUSE TestIsValidUploadKey/empty_key914=== RUN TestIsValidUploadKey/unknown_type915=== PAUSE TestIsValidUploadKey/unknown_type916=== CONT TestPush_CompleteCommitsEveryRoot917--- PASS: TestSkippedUploadsHandler (0.07s)918=== CONT TestPush_OverlappingRootsStoreOneRowPerKey919=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure920=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure921=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart922=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart923=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts924=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts925=== CONT TestReadRedirectUsesPublicS3URL9262026/11/12 07:15:28 INFO Received push request method=POST path=/api/pushes9272026/11/12 07:15:28 INFO Received complete push request method=POST path=/api/pushes/1/complete9282026/11/12 07:15:28 INFO Completed push id=1 objects=4 duration=79.757833ms929--- PASS: TestPush_RepeatedRootCommitsOnce (0.80s)930=== CONT TestReadProxyRangeRequest9312026/11/12 07:15:28 INFO Received uploads request method=POST path=/api/pending_closures9322026/11/12 07:15:28 INFO Received uploads request method=POST path=/api/pending_closures9332026/11/12 07:15:29 INFO Received uploads request method=POST path=/api/pending_closures9342026/11/12 07:15:29 INFO Received uploads request method=POST path=/api/pending_closures9352026/11/12 07:15:29 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9362026/11/12 07:15:29 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=YTVlNTdlOGYtMmY5My00MjliLTliYzEtMTQxYWIxN2JkOGJlLjNhNWU0NWE5LTQ4MzItNDUxNi05NWQwLWQ2ZWI0ODdkZDdiZXgxNzk0NDY3NzI4OTk5MDczMDAw9372026/11/12 07:15:29 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=YTVlNTdlOGYtMmY5My00MjliLTliYzEtMTQxYWIxN2JkOGJlLjNhNWU0NWE5LTQ4MzItNDUxNi05NWQwLWQ2ZWI0ODdkZDdiZXgxNzk0NDY3NzI4OTk5MDczMDAw parts=1938--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (1.69s)939=== CONT TestReadRedirectKeepsNarinfoProxied940=== RUN TestPush_RejectsBadRequests/no_objects941=== PAUSE TestPush_RejectsBadRequests/no_objects942=== RUN TestPush_RejectsBadRequests/bad_root943=== PAUSE TestPush_RejectsBadRequests/bad_root944=== RUN TestPush_RejectsBadRequests/root_not_in_objects945=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects946=== RUN TestPush_RejectsBadRequests/no_roots947=== PAUSE TestPush_RejectsBadRequests/no_roots948=== CONT TestReadRedirectNar9492026/11/12 07:15:30 INFO Received push request method=POST path=/api/pushes9502026/11/12 07:15:30 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9512026/11/12 07:15:30 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9522026/11/12 07:15:30 INFO Signed narinfos id=1 count=19532026/11/12 07:15:30 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9542026/11/12 07:15:30 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9552026/11/12 07:15:30 ERROR Failed to sign narinfos signer="index 1" error="signing key cannot be nil"9562026/11/12 07:15:30 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9572026/11/12 07:15:30 ERROR Failed to sign narinfos signer=niks3-test-1 error="signer failed"9582026/11/12 07:15:30 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9592026/11/12 07:15:30 ERROR Signer returned unexpected signature count signer=niks3-test-1 got=0 want=19602026/11/12 07:15:30 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9612026/11/12 07:15:30 ERROR Signer returned unexpected object key signer=niks3-test-1 object_key=unexpected.narinfo962--- PASS: TestPush_SignsNarinfosOfItsPendingObjects (2.85s)963=== CONT TestReadProxyDisabled9642026/11/12 07:15:30 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=YTVlNTdlOGYtMmY5My00MjliLTliYzEtMTQxYWIxN2JkOGJlLmNhN2Q5ZWY3LWRiMDYtNDc3My05NDdiLWVmZTZmMWQ0NzBmMngxNzk0NDY3NzI4NjY0MjgwMDAw parts=129652026/11/12 07:15:30 INFO Received uploads request method=POST path=/api/pending_closures966--- PASS: TestCompletedNarNotReofferedAcrossClosures (2.86s)967=== CONT TestReadProxyRootRedirectsToIndexHTML9682026/11/12 07:15:30 INFO Received push request method=POST path=/api/pushes9692026/11/12 07:15:30 INFO Received complete push request method=POST path=/api/pushes/1/complete9702026/11/12 07:15:30 INFO Completed push id=1 objects=4 duration=25.9765ms9712026/11/12 07:15:30 INFO Received push request method=POST path=/api/pushes9722026/11/12 07:15:30 INFO Aborted multipart uploads count=0 kept=09732026/11/12 07:15:30 WARN Force mode enabled - objects will be deleted immediately without grace period9742026/11/12 07:15:30 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=09752026/11/12 07:15:30 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9762026/11/12 07:15:30 INFO Received push request method=POST path=/api/pushes9772026/11/12 07:15:30 INFO Vacuumed table table=pending_closures9782026/11/12 07:15:31 INFO Vacuumed table table=commit_receipts9792026/11/12 07:15:31 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=YTVlNTdlOGYtMmY5My00MjliLTliYzEtMTQxYWIxN2JkOGJlLjhkZjQxZDQ1LTY4MzgtNDhkOS1iMGRiLWY2Mzc5YTYyMjgyYngxNzk0NDY3NzI5MTczMTk2MDAw parts=129802026/11/12 07:15:31 INFO Vacuumed table table=pending_objects9812026/11/12 07:15:31 INFO Vacuumed table table=multipart_uploads9822026-11-12 07:15:31.142 UTC [70860] ERROR: could not obtain lock on row in relation "closures"9832026-11-12 07:15:31.142 UTC [70860] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9842026-11-12 07:15:31.148 UTC [70860] ERROR: could not obtain lock on row in relation "closures"9852026-11-12 07:15:31.148 UTC [70860] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9862026-11-12 07:15:31.152 UTC [70860] ERROR: could not obtain lock on row in relation "closures"9872026-11-12 07:15:31.152 UTC [70860] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9882026/11/12 07:15:31 INFO Vacuumed table table=closures9892026-11-12 07:15:31.158 UTC [70860] ERROR: could not obtain lock on row in relation "closures"9902026-11-12 07:15:31.158 UTC [70860] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9912026-11-12 07:15:31.159 UTC [70860] ERROR: could not obtain lock on row in relation "closures"9922026-11-12 07:15:31.159 UTC [70860] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9932026/11/12 07:15:31 INFO Vacuumed table table=objects9942026/11/12 07:15:31 INFO Received complete push request method=POST path=/api/pushes/2/complete995--- PASS: TestPush_RootsAreLockedInByteOrder (3.42s)996=== CONT TestGCTaskStore_CompletedAllowsNewTask997--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)998=== CONT TestReadProxyConditionalGet9992026/11/12 07:15:31 INFO Completed push id=2 objects=4 duration=10.023209ms1000--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (3.43s)1001=== CONT TestConnectWaitsForAPeerMigration10022026/11/12 07:15:31 INFO Received uploads request method=POST path=/api/pending_closures1003--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (3.55s)1004=== CONT TestGCTaskStore_GetReturnsLatest1005--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)1006=== CONT TestReadProxyOutlastsServerWriteTimeout10072026/11/12 07:15:31 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10082026/11/12 07:15:31 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst1009--- PASS: TestCompleteMultipartUnregistered (3.71s)1010=== CONT TestResolveDBConnectionString1011=== RUN TestResolveDBConnectionString/flag_wins1012=== PAUSE TestResolveDBConnectionString/flag_wins1013=== RUN TestResolveDBConnectionString/file_when_flag_empty1014=== PAUSE TestResolveDBConnectionString/file_when_flag_empty1015=== RUN TestResolveDBConnectionString/missing_file_is_an_error1016=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error1017=== RUN TestResolveDBConnectionString/PGHOST_allows_empty1018=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty1019=== RUN TestResolveDBConnectionString/nothing_configured1020=== PAUSE TestResolveDBConnectionString/nothing_configured1021=== CONT TestReadProxyHead10222026/11/12 07:15:31 INFO Received cleanup request method=DELETE path=/api/pending_closures10232026/11/12 07:15:31 INFO Aborted multipart uploads count=0 kept=010242026/11/12 07:15:31 INFO Received uploads request method=POST path=/api/pending_closures10252026/11/12 07:15:31 INFO Received cleanup request method=DELETE path=/api/pending_closures10262026/11/12 07:15:31 INFO Aborted multipart uploads count=1 kept=010272026/11/12 07:15:31 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10282026-11-12 07:15:31.816 UTC [70836] ERROR: Closure does not exist: id=110292026-11-12 07:15:31.816 UTC [70836] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 17 at RAISE10302026-11-12 07:15:31.816 UTC [70836] STATEMENT: -- name: CommitPendingClosure :one1031 SELECT commit_pending_closure($1::bigint)::bigint1032 1033--- PASS: TestService_cleanupPendingClosuresHandler (4.06s)1034=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock10352026/11/12 07:15:31 INFO Received push request method=POST path=/api/pushes10362026/11/12 07:15:32 INFO Received uploads request method=POST path=/api/pending_closures10372026-11-12 07:15:32.350 UTC [70874] ERROR: Push does not exist: id=110382026-11-12 07:15:32.350 UTC [70874] CONTEXT: PL/pgSQL function commit_push(bigint) line 15 at RAISE10392026-11-12 07:15:32.350 UTC [70874] STATEMENT: SELECT commit_push($1)1040--- PASS: TestPush_CompleteWaitsForCleanupOfThePush (4.59s)1041=== CONT TestReadProxyInvalidPath10422026/11/12 07:15:32 INFO Received uploads request method=POST path=/api/pending_closures10432026/11/12 07:15:32 INFO Received uploads request method=POST path=/api/pending_closures10442026/11/12 07:15:32 INFO Received uploads request method=POST path=/api/pending_closures10452026/11/12 07:15:32 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10462026/11/12 07:15:32 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10472026/11/12 07:15:32 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=YTVlNTdlOGYtMmY5My00MjliLTliYzEtMTQxYWIxN2JkOGJlLjk0YWE0N2NlLWZlMTktNDVjOS1hY2U2LWU1YTEyYTUxYjJkMXgxNzk0NDY3NzI5MjI0MDMxMDAw parts=121048--- PASS: TestSharedNarBothUploadsComplete (4.84s)1049=== CONT TestDeleteClosuresRacingPins10502026/11/12 07:15:32 INFO Received uploads request method=POST path=/api/pending_closures10512026/11/12 07:15:33 INFO lead: acquired remote=192.0.2.1:1234 generation=110522026/11/12 07:15:33 INFO lead: released remote=192.0.2.1:1234 generation=11053--- PASS: TestLeadIncumbentAnnouncesAtOnce (5.27s)1054=== CONT TestReadProxy4041055--- PASS: TestService_Rustfstest (5.53s)1056=== CONT TestClientFallsBackToClosures10572026/11/12 07:15:33 INFO lead: acquired remote=192.0.2.1:1234 generation=110582026/11/12 07:15:33 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10592026/11/12 07:15:33 INFO lead: acquired remote=192.0.2.1:1234 generation=110602026/11/12 07:15:33 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=YTVlNTdlOGYtMmY5My00MjliLTliYzEtMTQxYWIxN2JkOGJlLjI5MWE0NzIyLTg4ZGMtNDczNi04NzlkLWYyMzUzOGViNDc1N3gxNzk0NDY3NzMyNTA0NzE2MDAw parts=1010612026/11/12 07:15:33 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10622026/11/12 07:15:33 INFO Completed upload id=1 objects=2 duration=21.778125ms10632026/11/12 07:15:33 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000010642026/11/12 07:15:33 INFO Received uploads request method=POST path=/api/pending_closures10652026/11/12 07:15:33 INFO Starting cleanup of old closures method=DELETE path=/api/closures10662026/11/12 07:15:33 INFO Aborted multipart uploads count=0 kept=010672026/11/12 07:15:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010682026/11/12 07:15:33 INFO Vacuumed table table=pending_closures10692026/11/12 07:15:34 INFO Vacuumed table table=commit_receipts10702026/11/12 07:15:34 INFO Vacuumed table table=pending_objects10712026/11/12 07:15:34 INFO Vacuumed table table=multipart_uploads10722026/11/12 07:15:34 INFO Vacuumed table table=closures10732026/11/12 07:15:34 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10742026/11/12 07:15:34 INFO Vacuumed table table=objects10752026/11/12 07:15:34 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001076--- PASS: TestService_createPendingClosureHandler (6.44s)1077=== CONT TestReadProxyNarStreaming1078--- PASS: TestPresentRefreshesOnlyStaleClosures (6.40s)1079=== CONT TestGCTaskStore_GetEmpty1080--- PASS: TestGCTaskStore_GetEmpty (0.00s)1081=== CONT TestClientPushesUseOnePush10822026/11/12 07:15:34 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=YTVlNTdlOGYtMmY5My00MjliLTliYzEtMTQxYWIxN2JkOGJlLmE3ZWEzMDllLTQxM2ItNDAwZi04MmVjLWIyNGNkYjA3MjY5MHgxNzk0NDY3NzMyODM2MDExMDAw parts=1010832026/11/12 07:15:34 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10842026/11/12 07:15:34 INFO Completed upload id=1 objects=2 duration=33.543625ms10852026/11/12 07:15:34 INFO Received uploads request method=POST path=/api/pending_closures10862026/11/12 07:15:34 INFO Received uploads request method=POST path=/api/pending_closures10872026/11/12 07:15:34 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo10882026/11/12 07:15:34 WARN Found objects in DB but missing from S3, will re-upload count=11089--- PASS: TestService_verifyS3Integrity (6.53s)1090=== CONT TestReadProxyContentNegotiation10912026/11/12 07:15:34 INFO Received push request method=POST path=/api/pushes10922026/11/12 07:15:34 INFO Received complete push request method=POST path=/api/pushes/1/complete10932026/11/12 07:15:34 INFO Completed push id=1 objects=6 duration=26.31175ms1094--- PASS: TestPush_CompleteCommitsEveryRoot (6.68s)1095=== CONT TestGCTaskStore_ConflictDifferentParams1096--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)1097=== CONT TestPinProtectsFromGC10982026/11/12 07:15:34 INFO lead: acquired remote=192.0.2.1:1234 generation=110992026/11/12 07:15:34 INFO lead: released remote=192.0.2.1:1234 generation=111002026/11/12 07:15:34 INFO lead: acquired remote=192.0.2.1:1234 generation=21101--- PASS: TestReadRedirectUsesPublicS3URL (7.07s)1102=== CONT TestReadProxyNarinfoAlreadyDecompressed11032026/11/12 07:15:35 INFO lead: released remote=192.0.2.1:1234 generation=111042026/11/12 07:15:35 INFO lead: acquired remote=192.0.2.1:1234 generation=211052026/11/12 07:15:35 INFO Received push request method=POST path=/api/pushes1106--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (7.39s)1107=== CONT TestClientSharedPathCommittedMidPush1108--- PASS: TestReadProxyRangeRequest (6.85s)1109=== CONT TestGCTaskStore_DeduplicateSameParams1110--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)1111=== CONT TestReadProxyNarinfo11122026/11/12 07:15:35 INFO lead: released remote=192.0.2.1:1234 generation=11113--- PASS: TestReadRedirectKeepsNarinfoProxied (6.30s)1114=== CONT TestGCTaskStore_StartNew1115--- PASS: TestGCTaskStore_StartNew (0.00s)11162026/11/12 07:15:35 INFO lead: acquired remote=192.0.2.1:1234 generation=21117=== CONT TestIsValidCachePath1118=== RUN TestIsValidCachePath/narinfo1119=== PAUSE TestIsValidCachePath/narinfo1120=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars1121=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars1122=== RUN TestIsValidCachePath/nar_zst1123=== PAUSE TestIsValidCachePath/nar_zst1124=== RUN TestIsValidCachePath/nar_xz1125=== PAUSE TestIsValidCachePath/nar_xz1126=== RUN TestIsValidCachePath/nar_bz21127=== PAUSE TestIsValidCachePath/nar_bz21128=== RUN TestIsValidCachePath/nar_uncompressed1129=== PAUSE TestIsValidCachePath/nar_uncompressed1130=== RUN TestIsValidCachePath/ls1131=== PAUSE TestIsValidCachePath/ls1132=== RUN TestIsValidCachePath/log1133=== PAUSE TestIsValidCachePath/log1134=== RUN TestIsValidCachePath/realisation1135=== PAUSE TestIsValidCachePath/realisation1136=== RUN TestIsValidCachePath/nix-cache-info1137=== PAUSE TestIsValidCachePath/nix-cache-info1138=== RUN TestIsValidCachePath/index.html1139=== PAUSE TestIsValidCachePath/index.html1140=== RUN TestIsValidCachePath/traversal_parent1141=== PAUSE TestIsValidCachePath/traversal_parent1142=== RUN TestIsValidCachePath/traversal_in_middle1143=== PAUSE TestIsValidCachePath/traversal_in_middle1144=== RUN TestIsValidCachePath/invalid_char_e1145=== PAUSE TestIsValidCachePath/invalid_char_e1146=== RUN TestIsValidCachePath/invalid_char_u1147=== PAUSE TestIsValidCachePath/invalid_char_u1148=== RUN TestIsValidCachePath/random_path1149=== PAUSE TestIsValidCachePath/random_path1150=== RUN TestIsValidCachePath/empty1151=== PAUSE TestIsValidCachePath/empty1152=== RUN TestIsValidCachePath/leading_slash1153=== PAUSE TestIsValidCachePath/leading_slash1154=== RUN TestIsValidCachePath/wrong_extension1155=== PAUSE TestIsValidCachePath/wrong_extension1156=== RUN TestIsValidCachePath/short_hash1157=== PAUSE TestIsValidCachePath/short_hash1158=== CONT TestParseSingleRange1159=== RUN TestParseSingleRange/none1160=== PAUSE TestParseSingleRange/none1161=== RUN TestParseSingleRange/unknown_unit1162=== PAUSE TestParseSingleRange/unknown_unit1163=== RUN TestParseSingleRange/multi-range_ignored1164=== PAUSE TestParseSingleRange/multi-range_ignored1165=== RUN TestParseSingleRange/malformed_no_dash1166=== PAUSE TestParseSingleRange/malformed_no_dash1167=== RUN TestParseSingleRange/malformed_both_empty1168=== PAUSE TestParseSingleRange/malformed_both_empty1169=== RUN TestParseSingleRange/malformed_end_before_start1170=== PAUSE TestParseSingleRange/malformed_end_before_start1171=== RUN TestParseSingleRange/closed1172=== PAUSE TestParseSingleRange/closed1173=== RUN TestParseSingleRange/open-ended1174=== PAUSE TestParseSingleRange/open-ended1175=== RUN TestParseSingleRange/end_clamped_to_size1176=== PAUSE TestParseSingleRange/end_clamped_to_size1177=== RUN TestParseSingleRange/suffix1178=== PAUSE TestParseSingleRange/suffix1179=== RUN TestParseSingleRange/suffix_exceeds_size1180=== PAUSE TestParseSingleRange/suffix_exceeds_size1181=== RUN TestParseSingleRange/single_byte1182=== PAUSE TestParseSingleRange/single_byte1183=== RUN TestParseSingleRange/start_past_EOF1184=== PAUSE TestParseSingleRange/start_past_EOF1185=== RUN TestParseSingleRange/start_far_past_EOF1186=== PAUSE TestParseSingleRange/start_far_past_EOF1187=== CONT TestPresentNotReportedWhileGCDeletesClosure1188=== RUN TestPresentNotReportedWhileGCDeletesClosure/stale1189=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/stale1190=== RUN TestPresentNotReportedWhileGCDeletesClosure/fresh1191=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/fresh1192=== CONT TestGCEndsOnShutdown1193=== RUN TestGCEndsOnShutdown/before_the_run1194=== PAUSE TestGCEndsOnShutdown/before_the_run1195=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1196=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1197=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1198=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1199=== CONT TestCommitRacingPendingCleanupKeepsObjects12002026/11/12 07:15:35 INFO lead: given back to the incumbent remote=192.0.2.1:1234 generation=212012026/11/12 07:15:35 INFO lead: acquired remote=192.0.2.1:1234 generation=312022026/11/12 07:15:35 INFO lead: released remote=192.0.2.1:1234 generation=21203--- PASS: TestLeadElectsOneAndHandsOver (8.17s)1204=== CONT TestService_NativeMTLS1205--- PASS: TestReadRedirectNar (5.89s)1206=== CONT TestPresentReportsOnlyClosureRoots12072026/11/12 07:15:36 INFO lead: released remote=192.0.2.1:1234 generation=212082026/11/12 07:15:36 INFO lead: acquired remote=192.0.2.1:1234 generation=312092026/11/12 07:15:36 INFO lead: released remote=192.0.2.1:1234 generation=31210--- PASS: TestLeadGenerationGrowsWithEveryGrant (8.30s)1211=== CONT TestPushWaitsForSweepHoldingRow1212--- PASS: TestReadProxyDisabled (5.66s)1213=== CONT TestClientWithDependencies1214--- PASS: TestReadProxyRootRedirectsToIndexHTML (6.01s)1215=== CONT TestCreatePin_ReservedPins12162026/11/12 07:15:36 WARN Rate limiter enabled after throttle name=s3-test rate=512172026/11/12 07:15:36 WARN S3 rate limit hit operation="complete multipart upload" error="Please reduce your request rate."1218=== NAME TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle1219 throttle_test.go:213: Proxy stats: total=13, throttled=10, completeMultipart=101220 throttle_test.go:215: Rate limiter: enabled=true, rate=5.001221--- PASS: TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle (8.89s)1222=== CONT TestSweepRechecksPendingAfterLocking12232026/11/12 07:15:36 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:60576/oidc1224--- PASS: TestReadProxyConditionalGet (5.75s)1225=== CONT TestClientMultipleUploads1226--- PASS: TestReadProxyHead (6.10s)1227=== CONT TestOrphanedObjectsGCStressTest12282026/11/12 07:15:37 INFO lead: released remote=192.0.2.1:1234 generation=31229--- PASS: TestLeadUnannouncedStandbyYieldsToIncumbent (9.94s)1230=== CONT TestClientIntegration1231--- PASS: TestReadProxyInvalidPath (5.63s)1232=== CONT TestClientErrorHandling1233=== RUN TestClientErrorHandling/InvalidStorePath1234=== PAUSE TestClientErrorHandling/InvalidStorePath1235=== RUN TestClientErrorHandling/InvalidAuthToken1236=== PAUSE TestClientErrorHandling/InvalidAuthToken1237=== RUN TestClientErrorHandling/ServerNotAvailable1238=== PAUSE TestClientErrorHandling/ServerNotAvailable1239=== CONT TestOrphanedObjectsGC1240--- PASS: TestConnectWaitsForAPeerMigration (7.10s)1241=== CONT TestClientCADerivations1242--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (6.77s)1243=== CONT TestObjectStatsUpdatedOncePerCommit1244--- PASS: TestReadProxy404 (5.88s)1245=== CONT TestGetExistingObjectsLocksRows1246--- PASS: TestReadProxyContentNegotiation (4.82s)1247=== CONT TestObjectStatsTrigger12482026/11/12 07:15:39 INFO Received uploads request method=POST path=/api/pending_closures1249--- PASS: TestReadProxyNarStreaming (5.08s)1250=== CONT TestRegisterCompletedObjectNeedsPendingRow12512026/11/12 07:15:39 INFO Received uploads request method=POST path=/api/pending_closures12522026/11/12 07:15:39 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)12532026/11/12 07:15:39 INFO Uploading mxm1c38g0zdgza8lbb79wgv1xkvpdbz9-shared-dep (136B)12542026/11/12 07:15:39 INFO Uploading xasy98bylhx7pmv573h315dxz1k19vbx-b (256B)12552026/11/12 07:15:39 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"12562026/11/12 07:15:39 WARN Failed to register uploaded object key=mxm1c38g0zdgza8lbb79wgv1xkvpdbz9.ls error="server returned 404: 404 page not found\n"12572026/11/12 07:15:39 WARN Failed to register uploaded object key=isxqki7glcd0gjgy8mdy18r1p1jzbv68.ls error="server returned 404: 404 page not found\n"12582026/11/12 07:15:39 WARN Failed to register uploaded object key=nar/1l85rxya83awnbvcanvif7rvjigqs6v8b3c8j5gj5839zyk718x9.nar.zst error="server returned 404: 404 page not found\n"12592026/11/12 07:15:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign12602026/11/12 07:15:39 WARN Failed to register uploaded object key=xasy98bylhx7pmv573h315dxz1k19vbx.ls error="server returned 404: 404 page not found\n"12612026/11/12 07:15:39 INFO Signed narinfos id=1 count=212622026/11/12 07:15:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign12632026/11/12 07:15:39 INFO Signed narinfos id=2 count=212642026/11/12 07:15:39 INFO Uploading 4 narinfos12652026/11/12 07:15:39 WARN Failed to register uploaded object key=mxm1c38g0zdgza8lbb79wgv1xkvpdbz9.narinfo error="server returned 404: 404 page not found\n"12662026/11/12 07:15:39 WARN Failed to register uploaded object key=xasy98bylhx7pmv573h315dxz1k19vbx.narinfo error="server returned 404: 404 page not found\n"12672026/11/12 07:15:39 WARN Failed to register uploaded object key=isxqki7glcd0gjgy8mdy18r1p1jzbv68.narinfo error="server returned 404: 404 page not found\n"12682026/11/12 07:15:39 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete12692026/11/12 07:15:39 WARN Failed to register uploaded object key=mxm1c38g0zdgza8lbb79wgv1xkvpdbz9.narinfo error="server returned 404: 404 page not found\n"12702026/11/12 07:15:39 INFO Completed upload id=1 objects=6 duration=31.577292ms12712026/11/12 07:15:39 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete12722026/11/12 07:15:39 INFO Completed upload id=2 objects=6 duration=1.245666ms12732026/11/12 07:15:39 INFO Upload complete. (355ms)1274=== NAME TestClientFallsBackToClosures1275 client_pushes_test.go:112: Retrieved narinfo from S3:1276 StorePath: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientFallsBackToClosures2086647941/001/store/mxm1c38g0zdgza8lbb79wgv1xkvpdbz9-shared-dep1277 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1278 Compression: zstd1279 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821280 NarSize: 1361281 References: 1282 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1283 client_pushes_test.go:112: Retrieved narinfo from S3:1284 StorePath: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientFallsBackToClosures2086647941/001/store/isxqki7glcd0gjgy8mdy18r1p1jzbv68-a1285 URL: nar/1l85rxya83awnbvcanvif7rvjigqs6v8b3c8j5gj5839zyk718x9.nar.zst1286 Compression: zstd1287 NarHash: sha256:1l85rxya83awnbvcanvif7rvjigqs6v8b3c8j5gj5839zyk718x91288 NarSize: 2561289 References: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientFallsBackToClosures2086647941/001/store/mxm1c38g0zdgza8lbb79wgv1xkvpdbz9-shared-dep1290 CA: text:sha256:1y6a7fhhphbjqriirvi5i41bhz00aw4cdgkwm3v04592nxrmlipm1291 client_pushes_test.go:112: Retrieved narinfo from S3:1292 StorePath: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientFallsBackToClosures2086647941/001/store/xasy98bylhx7pmv573h315dxz1k19vbx-b1293 URL: nar/1l85rxya83awnbvcanvif7rvjigqs6v8b3c8j5gj5839zyk718x9.nar.zst1294 Compression: zstd1295 NarHash: sha256:1l85rxya83awnbvcanvif7rvjigqs6v8b3c8j5gj5839zyk718x91296 NarSize: 2561297 References: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientFallsBackToClosures2086647941/001/store/mxm1c38g0zdgza8lbb79wgv1xkvpdbz9-shared-dep1298 CA: text:sha256:1y6a7fhhphbjqriirvi5i41bhz00aw4cdgkwm3v04592nxrmlipm1299--- PASS: TestClientFallsBackToClosures (6.28s)1300=== CONT TestPendingCleanupUsesOneCutoff1301=== NAME TestPinProtectsFromGC1302 client_integration_test.go:791: Pinned store path: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestPinProtectsFromGC1335971145/001/store/c03rml0qjlyf8wgb5qz29x8i3yqs2r4x-pinned-file.txt1303 client_integration_test.go:792: Unpinned store path: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestPinProtectsFromGC1335971145/001/store/qyb7aww0anhimgf3ln2xns7ybafsg9zj-unpinned-file.txt13042026/11/12 07:15:39 INFO Received push request method=POST path=/api/pushes1305--- PASS: TestReadProxyNarinfoAlreadyDecompressed (5.05s)1306=== CONT TestSweepSkipsRowsLockedByPush13072026/11/12 07:15:39 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)13082026/11/12 07:15:39 INFO Uploading yxynfhkmfy3cxy1mf6n2w7bpmswbfkd2-b (256B)13092026/11/12 07:15:39 INFO Uploading 13mflns0h254bj5izai7kwmscg1zm334-shared-dep (136B)13102026/11/12 07:15:40 WARN Failed to register uploaded object key=6p9lnqd8ffav3v3s3qz2wxnpm62ci8wf.ls error="server returned 404: 404 page not found\n"13112026/11/12 07:15:40 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"13122026/11/12 07:15:40 WARN Failed to register uploaded object key=yxynfhkmfy3cxy1mf6n2w7bpmswbfkd2.ls error="server returned 404: 404 page not found\n"13132026/11/12 07:15:40 WARN Failed to register uploaded object key=nar/18lm1ngjw75ybxk96aws6yf69iy2crm9iclcrq6ax6kkww616p81.nar.zst error="server returned 404: 404 page not found\n"13142026/11/12 07:15:40 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13152026/11/12 07:15:40 WARN Failed to register uploaded object key=13mflns0h254bj5izai7kwmscg1zm334.ls error="server returned 404: 404 page not found\n"13162026/11/12 07:15:40 INFO Signed narinfos id=1 count=313172026/11/12 07:15:40 INFO Uploading 3 narinfos13182026/11/12 07:15:40 INFO Received push request method=POST path=/api/pushes13192026/11/12 07:15:40 WARN Failed to register uploaded object key=13mflns0h254bj5izai7kwmscg1zm334.narinfo error="server returned 404: 404 page not found\n"13202026/11/12 07:15:40 WARN Failed to register uploaded object key=yxynfhkmfy3cxy1mf6n2w7bpmswbfkd2.narinfo error="server returned 404: 404 page not found\n"13212026/11/12 07:15:40 INFO Received complete push request method=POST path=/api/pushes/1/complete13222026/11/12 07:15:40 WARN Failed to register uploaded object key=6p9lnqd8ffav3v3s3qz2wxnpm62ci8wf.narinfo error="server returned 404: 404 page not found\n"13232026/11/12 07:15:40 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)13242026/11/12 07:15:40 INFO Uploading c03rml0qjlyf8wgb5qz29x8i3yqs2r4x-pinned-file.txt (128B)13252026/11/12 07:15:40 INFO Completed push id=1 objects=8 duration=44.466916ms13262026/11/12 07:15:40 INFO Upload complete. (283ms)1327=== NAME TestClientPushesUseOnePush1328 client_pushes_test.go:97: Retrieved narinfo from S3:1329 StorePath: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientPushesUseOnePush878029743/001/store/13mflns0h254bj5izai7kwmscg1zm334-shared-dep1330 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1331 Compression: zstd1332 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821333 NarSize: 1361334 References: 1335 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n13362026/11/12 07:15:40 WARN Failed to register uploaded object key=c03rml0qjlyf8wgb5qz29x8i3yqs2r4x.ls error="server returned 404: 404 page not found\n"1337 client_pushes_test.go:97: Retrieved narinfo from S3:1338 StorePath: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientPushesUseOnePush878029743/001/store/6p9lnqd8ffav3v3s3qz2wxnpm62ci8wf-a1339 URL: nar/18lm1ngjw75ybxk96aws6yf69iy2crm9iclcrq6ax6kkww616p81.nar.zst1340 Compression: zstd1341 NarHash: sha256:18lm1ngjw75ybxk96aws6yf69iy2crm9iclcrq6ax6kkww616p811342 NarSize: 2561343 References: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientPushesUseOnePush878029743/001/store/13mflns0h254bj5izai7kwmscg1zm334-shared-dep1344 CA: text:sha256:0g9nlyi9r4hzx7ngps8m4d5llwd7l7v0c7s6mhg069ghicpk6l951345 client_pushes_test.go:97: Retrieved narinfo from S3:1346 StorePath: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientPushesUseOnePush878029743/001/store/yxynfhkmfy3cxy1mf6n2w7bpmswbfkd2-b1347 URL: nar/18lm1ngjw75ybxk96aws6yf69iy2crm9iclcrq6ax6kkww616p81.nar.zst1348 Compression: zstd1349 NarHash: sha256:18lm1ngjw75ybxk96aws6yf69iy2crm9iclcrq6ax6kkww616p811350 NarSize: 2561351 References: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientPushesUseOnePush878029743/001/store/13mflns0h254bj5izai7kwmscg1zm334-shared-dep1352 CA: text:sha256:0g9nlyi9r4hzx7ngps8m4d5llwd7l7v0c7s6mhg069ghicpk6l9513532026/11/12 07:15:40 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13542026/11/12 07:15:40 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"13552026/11/12 07:15:40 INFO Signed narinfos id=1 count=113562026/11/12 07:15:40 INFO Uploading 1 narinfos13572026/11/12 07:15:40 INFO Received complete push request method=POST path=/api/pushes/1/complete13582026/11/12 07:15:40 WARN Failed to register uploaded object key=c03rml0qjlyf8wgb5qz29x8i3yqs2r4x.narinfo error="server returned 404: 404 page not found\n"1359--- PASS: TestClientPushesUseOnePush (6.03s)1360=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded13612026/11/12 07:15:40 INFO Completed push id=1 objects=3 duration=28.482ms13622026/11/12 07:15:40 INFO Upload complete. (311ms)13632026/11/12 07:15:40 INFO Received push request method=POST path=/api/pushes13642026/11/12 07:15:40 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)13652026/11/12 07:15:40 INFO Uploading qyb7aww0anhimgf3ln2xns7ybafsg9zj-unpinned-file.txt (128B)13662026/11/12 07:15:40 WARN Failed to register uploaded object key=qyb7aww0anhimgf3ln2xns7ybafsg9zj.ls error="server returned 404: 404 page not found\n"13672026/11/12 07:15:40 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign13682026/11/12 07:15:40 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"13692026/11/12 07:15:40 INFO Signed narinfos id=2 count=113702026/11/12 07:15:40 INFO Uploading 1 narinfos13712026/11/12 07:15:40 INFO Received complete push request method=POST path=/api/pushes/2/complete13722026/11/12 07:15:40 WARN Failed to register uploaded object key=qyb7aww0anhimgf3ln2xns7ybafsg9zj.narinfo error="server returned 404: 404 page not found\n"13732026/11/12 07:15:40 INFO Completed push id=2 objects=3 duration=892.084µs13742026/11/12 07:15:40 INFO Upload complete. (154ms)13752026/11/12 07:15:40 INFO Starting cleanup of old closures method=DELETE path=/api/closures13762026/11/12 07:15:40 INFO Garbage collection started13772026/11/12 07:15:40 INFO Aborted multipart uploads count=0 kept=013782026/11/12 07:15:40 WARN Force mode enabled - objects will be deleted immediately without grace period13792026/11/12 07:15:40 ERROR Failed to decode narinfo key=3hcdxyjf9yiq7qf3i4548drb6sjmwa1v.narinfo limit=16777216 error=<nil>1380--- PASS: TestReadProxyNarinfo (5.15s)1381=== CONT TestFailedS3DeleteKeepsTombstone13822026/11/12 07:15:40 INFO Received push request method=POST path=/api/pushes13832026/11/12 07:15:40 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)13842026/11/12 07:15:40 INFO Uploading qg0raz91ad5dn0ymp17k2hv2ivzmvy99-shared-dep (136B)13852026/11/12 07:15:40 INFO Uploading c9cvkf351q3y0h4yqwj477f4y184vv3q-top (264B)13862026/11/12 07:15:40 WARN Failed to register uploaded object key=qg0raz91ad5dn0ymp17k2hv2ivzmvy99.ls error="server returned 404: 404 page not found\n"13872026/11/12 07:15:40 WARN Failed to register uploaded object key=nar/0akqx6xqg0d3a5rxb554wbn7v27rmxx2isrc1iihx9357zl331ws.nar.zst error="server returned 404: 404 page not found\n"13882026/11/12 07:15:40 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"13892026/11/12 07:15:40 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13902026/11/12 07:15:40 WARN Failed to register uploaded object key=c9cvkf351q3y0h4yqwj477f4y184vv3q.ls error="server returned 404: 404 page not found\n"13912026/11/12 07:15:40 INFO Signed narinfos id=1 count=213922026/11/12 07:15:40 INFO Uploading 2 narinfos13932026/11/12 07:15:40 WARN Failed to register uploaded object key=qg0raz91ad5dn0ymp17k2hv2ivzmvy99.narinfo error="server returned 404: 404 page not found\n"13942026/11/12 07:15:40 INFO Received complete push request method=POST path=/api/pushes/1/complete13952026/11/12 07:15:40 WARN Failed to register uploaded object key=c9cvkf351q3y0h4yqwj477f4y184vv3q.narinfo error="server returned 404: 404 page not found\n"13962026/11/12 07:15:40 INFO Completed push id=1 objects=6 duration=77.142833ms13972026/11/12 07:15:40 INFO Upload complete. (349ms)1398=== NAME TestClientSharedPathCommittedMidPush1399 client_integration_test.go:740: Retrieved narinfo from S3:1400 StorePath: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientSharedPathCommittedMidPush978363212/001/store/qg0raz91ad5dn0ymp17k2hv2ivzmvy99-shared-dep1401 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1402 Compression: zstd1403 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821404 NarSize: 1361405 References: 1406 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1407 client_integration_test.go:740: Retrieved narinfo from S3:1408 StorePath: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientSharedPathCommittedMidPush978363212/001/store/c9cvkf351q3y0h4yqwj477f4y184vv3q-top1409 URL: nar/0akqx6xqg0d3a5rxb554wbn7v27rmxx2isrc1iihx9357zl331ws.nar.zst1410 Compression: zstd1411 NarHash: sha256:0akqx6xqg0d3a5rxb554wbn7v27rmxx2isrc1iihx9357zl331ws1412 NarSize: 2641413 References: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientSharedPathCommittedMidPush978363212/001/store/qg0raz91ad5dn0ymp17k2hv2ivzmvy99-shared-dep1414 CA: text:sha256:151myln9rvjz6r5a25p1m2z85xfmgzn353v174bizka950crf0y214152026/11/12 07:15:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=01416--- PASS: TestClientSharedPathCommittedMidPush (5.79s)1417=== CONT TestMultipartCleanup14182026/11/12 07:15:41 INFO Vacuumed table table=pending_closures14192026/11/12 07:15:41 INFO Vacuumed table table=commit_receipts14202026/11/12 07:15:41 INFO Aborted multipart uploads count=0 kept=014212026/11/12 07:15:41 INFO Vacuumed table table=pending_objects14222026/11/12 07:15:41 INFO Vacuumed table table=multipart_uploads14232026/11/12 07:15:41 WARN Force mode enabled - objects will be deleted immediately without grace period14242026/11/12 07:15:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=014252026/11/12 07:15:41 INFO Vacuumed table table=closures14262026/11/12 07:15:41 INFO Vacuumed table table=pending_closures14272026/11/12 07:15:41 INFO Vacuumed table table=commit_receipts14282026/11/12 07:15:41 INFO Vacuumed table table=objects14292026/11/12 07:15:41 INFO Vacuumed table table=pending_objects14302026/11/12 07:15:41 INFO Vacuumed table table=multipart_uploads14312026/11/12 07:15:41 INFO Vacuumed table table=closures14322026/11/12 07:15:41 INFO Vacuumed table table=objects1433--- PASS: TestCommitRacingPendingCleanupKeepsObjects (5.38s)1434=== CONT TestServerTLSConfig1435=== RUN TestServerTLSConfig/no_client_CA1436=== PAUSE TestServerTLSConfig/no_client_CA1437=== RUN TestServerTLSConfig/missing_CA_file1438=== PAUSE TestServerTLSConfig/missing_CA_file1439=== RUN TestServerTLSConfig/not_a_PEM_file1440=== PAUSE TestServerTLSConfig/not_a_PEM_file1441=== CONT TestService_ReadScope_PublicByDefault1442--- PASS: TestPresentReportsOnlyClosureRoots (5.08s)1443=== CONT TestCacheConfigHandler1444=== RUN TestCacheConfigHandler/full_config,_no_issuer1445=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1446=== RUN TestCacheConfigHandler/no_cache_url_configured1447=== PAUSE TestCacheConfigHandler/no_cache_url_configured1448=== RUN TestCacheConfigHandler/no_signing_keys1449=== PAUSE TestCacheConfigHandler/no_signing_keys1450=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1451=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1452=== CONT TestService_RequireScope_OIDC14532026/11/12 07:15:41 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:60685/oidc14542026/11/12 07:15:41 WARN mTLS auth: subject not in bound subjects subject="CN=reader"14552026/11/12 07:15:41 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1456--- PASS: TestService_NativeMTLS (5.33s)1457=== CONT TestPresignedUploadRegisteredBeforeCommit1458--- PASS: TestDeleteClosuresRacingPins (8.84s)1459=== CONT TestDeduplicatedObjectsRecordedAsPending1460--- PASS: TestReadProxyOutlastsServerWriteTimeout (10.17s)1461=== CONT TestForceGCDuringPushOffersSweptObject1462=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows1463=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows1464=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check1465=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check1466=== CONT TestPushDedupSurvivesConcurrentGC14672026/11/12 07:15:41 INFO Received uploads request method=POST path=/api/pending_closures14682026/11/12 07:15:41 INFO Aborted multipart uploads count=0 kept=014692026/11/12 07:15:41 WARN Force mode enabled - objects will be deleted immediately without grace period1470=== NAME TestClientWithDependencies1471 client_integration_test.go:661: Built derivation: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientWithDependencies3328478827/001/store/28piwdn1ljrpr8hmiaq4jqvypn20fj5s-test-script14722026/11/12 07:15:42 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=01473--- PASS: TestPushWaitsForSweepHoldingRow (5.93s)1474=== CONT TestGCMetrics14752026/11/12 07:15:42 INFO Vacuumed table table=pending_closures14762026/11/12 07:15:42 INFO Vacuumed table table=commit_receipts1477=== NAME TestClientWithDependencies1478 client_integration_test.go:663: Found 1 dependencies (including self)14792026/11/12 07:15:42 INFO Vacuumed table table=pending_objects14802026/11/12 07:15:42 INFO Vacuumed table table=multipart_uploads14812026/11/12 07:15:42 INFO Vacuumed table table=closures14822026/11/12 07:15:42 INFO Vacuumed table table=objects1483--- PASS: TestSweepRechecksPendingAfterLocking (5.46s)1484=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection14852026/11/12 07:15:42 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux14862026/11/12 07:15:42 WARN Refused reserved pin name=worker-x86_64-linux14872026/11/12 07:15:42 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux14882026/11/12 07:15:42 INFO Received create pin request method=POST path=/api/pins/my-app14892026/11/12 07:15:42 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux1490--- PASS: TestCreatePin_ReservedPins (5.47s)1491=== CONT TestGCBugBareHashReferences14922026/11/12 07:15:42 INFO Received push request method=POST path=/api/pushes14932026/11/12 07:15:42 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)14942026/11/12 07:15:42 INFO Uploading 28piwdn1ljrpr8hmiaq4jqvypn20fj5s-test-script (136B)14952026/11/12 07:15:42 WARN Failed to register uploaded object key=28piwdn1ljrpr8hmiaq4jqvypn20fj5s.ls error="server returned 404: 404 page not found\n"14962026/11/12 07:15:42 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"14972026/11/12 07:15:42 WARN Failed to register uploaded object key=log/xcr9l907yrl0kijppm18ffcaya8v847w-test-script.drv error="server returned 404: 404 page not found\n"14982026/11/12 07:15:42 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign14992026/11/12 07:15:42 INFO Signed narinfos id=1 count=115002026/11/12 07:15:42 INFO Uploading 1 narinfos15012026/11/12 07:15:42 INFO Received complete push request method=POST path=/api/pushes/1/complete15022026/11/12 07:15:42 WARN Failed to register uploaded object key=28piwdn1ljrpr8hmiaq4jqvypn20fj5s.narinfo error="server returned 404: 404 page not found\n"15032026/11/12 07:15:42 INFO Completed push id=1 objects=4 duration=24.289375ms15042026/11/12 07:15:42 INFO Upload complete. (224ms)15052026/11/12 07:15:42 INFO Aborted multipart uploads count=0 kept=015062026/11/12 07:15:42 WARN Force mode enabled - objects will be deleted immediately without grace period15072026/11/12 07:15:42 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=015082026/11/12 07:15:42 INFO Vacuumed table table=pending_closures15092026/11/12 07:15:42 INFO Vacuumed table table=commit_receipts15102026/11/12 07:15:42 INFO Vacuumed table table=pending_objects15112026/11/12 07:15:42 INFO Vacuumed table table=multipart_uploads15122026/11/12 07:15:42 INFO Vacuumed table table=closures15132026/11/12 07:15:42 INFO Vacuumed table table=objects1514=== NAME TestClientWithDependencies1515 client_integration_test.go:677: Skipping nix copy test - isolated store (/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientWithDependencies3328478827/001/store) requires matching store prefix15162026/11/12 07:15:42 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=015172026/11/12 07:15:42 INFO Received create pin request method=POST path=/api/pins/myapp1518--- PASS: TestClientWithDependencies (6.29s)1519=== CONT TestGCAdvisoryLockBlocksConcurrentRun15202026/11/12 07:15:42 INFO Created/updated pin name=myapp store_path=/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestPinProtectsFromGC1335971145/001/store/c03rml0qjlyf8wgb5qz29x8i3yqs2r4x-pinned-file.txt narinfo_key=c03rml0qjlyf8wgb5qz29x8i3yqs2r4x.narinfo1521=== NAME TestClientMultipleUploads1522 client_integration_test.go:406: Created store path 0: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientMultipleUploads440966431/001/store/fkhz5z43lnvgwvhk3mwi198wd5spyj34-test-file-0.txt15232026/11/12 07:15:42 INFO Starting cleanup of old closures method=DELETE path=/api/closures15242026/11/12 07:15:42 INFO Garbage collection started15252026/11/12 07:15:42 INFO Aborted multipart uploads count=0 kept=015262026/11/12 07:15:42 WARN Force mode enabled - objects will be deleted immediately without grace period15272026/11/12 07:15:42 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=015282026/11/12 07:15:42 INFO Vacuumed table table=pending_closures15292026/11/12 07:15:42 INFO Vacuumed table table=commit_receipts15302026/11/12 07:15:42 INFO Vacuumed table table=pending_objects15312026/11/12 07:15:42 INFO Vacuumed table table=multipart_uploads15322026/11/12 07:15:42 INFO Vacuumed table table=closures1533 client_integration_test.go:406: Created store path 1: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientMultipleUploads440966431/001/store/7rlliqi6aqg1m9fxnfr8zvfnavdhpbxw-test-file-1.txt15342026/11/12 07:15:42 INFO Vacuumed table table=objects1535 client_integration_test.go:406: Created store path 2: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientMultipleUploads440966431/001/store/qvbmzzw5v385zgfy09kaj9w1mh11vlh2-test-file-2.txt1536=== NAME TestClientIntegration1537 client_integration_test.go:292: Created store path: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientIntegration650181799/002/store/ygqw56xvw05q13ipq65dkx9z6xsqs6y8-test-file.txt15382026/11/12 07:15:42 INFO Received push request method=POST path=/api/pushes15392026/11/12 07:15:42 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)15402026/11/12 07:15:42 INFO Uploading qvbmzzw5v385zgfy09kaj9w1mh11vlh2-test-file-2.txt (160B)15412026/11/12 07:15:42 INFO Uploading fkhz5z43lnvgwvhk3mwi198wd5spyj34-test-file-0.txt (160B)15422026/11/12 07:15:42 INFO Uploading 7rlliqi6aqg1m9fxnfr8zvfnavdhpbxw-test-file-1.txt (160B)15432026/11/12 07:15:42 WARN Failed to register uploaded object key=fkhz5z43lnvgwvhk3mwi198wd5spyj34.ls error="server returned 404: 404 page not found\n"15442026/11/12 07:15:42 WARN Failed to register uploaded object key=qvbmzzw5v385zgfy09kaj9w1mh11vlh2.ls error="server returned 404: 404 page not found\n"15452026/11/12 07:15:42 WARN Failed to register uploaded object key=7rlliqi6aqg1m9fxnfr8zvfnavdhpbxw.ls error="server returned 404: 404 page not found\n"15462026/11/12 07:15:42 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"15472026/11/12 07:15:42 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"15482026/11/12 07:15:42 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15492026/11/12 07:15:42 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"15502026/11/12 07:15:42 INFO Signed narinfos id=1 count=315512026/11/12 07:15:42 INFO Uploading 3 narinfos15522026/11/12 07:15:42 INFO Received push request method=POST path=/api/pushes15532026/11/12 07:15:42 WARN Failed to register uploaded object key=qvbmzzw5v385zgfy09kaj9w1mh11vlh2.narinfo error="server returned 404: 404 page not found\n"15542026/11/12 07:15:42 WARN Failed to register uploaded object key=fkhz5z43lnvgwvhk3mwi198wd5spyj34.narinfo error="server returned 404: 404 page not found\n"15552026/11/12 07:15:42 INFO Received complete push request method=POST path=/api/pushes/1/complete15562026/11/12 07:15:42 WARN Failed to register uploaded object key=7rlliqi6aqg1m9fxnfr8zvfnavdhpbxw.narinfo error="server returned 404: 404 page not found\n"15572026/11/12 07:15:42 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)15582026/11/12 07:15:42 INFO Uploading ygqw56xvw05q13ipq65dkx9z6xsqs6y8-test-file.txt (152B)15592026/11/12 07:15:42 INFO Completed push id=1 objects=9 duration=34.71625ms15602026/11/12 07:15:42 INFO Upload complete. (283ms)1561=== NAME TestClientMultipleUploads1562 client_integration_test.go:417: Uploaded 3 paths in 323.466875ms15632026/11/12 07:15:43 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"15642026/11/12 07:15:43 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15652026/11/12 07:15:43 WARN Failed to register uploaded object key=ygqw56xvw05q13ipq65dkx9z6xsqs6y8.ls error="server returned 404: 404 page not found\n"15662026/11/12 07:15:43 INFO Signed narinfos id=1 count=115672026/11/12 07:15:43 INFO Uploading 1 narinfos15682026/11/12 07:15:43 INFO Received complete push request method=POST path=/api/pushes/1/complete15692026/11/12 07:15:43 WARN Failed to register uploaded object key=ygqw56xvw05q13ipq65dkx9z6xsqs6y8.narinfo error="server returned 404: 404 page not found\n"15702026/11/12 07:15:43 INFO Completed push id=1 objects=3 duration=31.863334ms15712026/11/12 07:15:43 INFO Upload complete. (272ms)1572--- PASS: TestClientMultipleUploads (6.15s)1573=== CONT TestLeadEndsOnShutdown15742026/11/12 07:15:43 INFO All 1 paths already cached1575=== NAME TestClientIntegration1576 client_integration_test.go:318: Retrieved narinfo from S3:1577 StorePath: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientIntegration650181799/002/store/ygqw56xvw05q13ipq65dkx9z6xsqs6y8-test-file.txt1578 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1579 Compression: zstd1580 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11581 NarSize: 1521582 References: 1583 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11584 client_integration_test.go:319: Retrieved .ls file from S3 (compressed size: 77 bytes)1585 client_integration_test.go:319: Decompressed .ls content (64 bytes):1586 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}15872026/11/12 07:15:43 INFO Received push request method=POST path=/api/pushes15882026/11/12 07:15:43 INFO Object in database but missing from S3 key=ygqw56xvw05q13ipq65dkx9z6xsqs6y8.ls15892026/11/12 07:15:43 WARN Found objects in DB but missing from S3, will re-upload count=115902026/11/12 07:15:43 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)15912026/11/12 07:15:43 INFO Received complete push request method=POST path=/api/pushes/2/complete15922026/11/12 07:15:43 WARN Failed to register uploaded object key=ygqw56xvw05q13ipq65dkx9z6xsqs6y8.ls error="server returned 404: 404 page not found\n"15932026/11/12 07:15:43 INFO Completed push id=2 objects=3 duration=1.208333ms15942026/11/12 07:15:43 INFO Upload complete. (100ms)1595 client_integration_test.go:336: Retrieved .ls file from S3 (compressed size: 62 bytes)1596 client_integration_test.go:336: Decompressed .ls content (49 bytes):1597 {"version":1,"root":{"type":"regular","size":39}}15982026/11/12 07:15:43 INFO Received push request method=POST path=/api/pushes15992026/11/12 07:15:43 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)16002026/11/12 07:15:43 INFO Received complete push request method=POST path=/api/pushes/3/complete16012026/11/12 07:15:43 WARN Failed to register uploaded object key=ygqw56xvw05q13ipq65dkx9z6xsqs6y8.ls error="server returned 404: 404 page not found\n"16022026/11/12 07:15:43 INFO Completed push id=3 objects=3 duration=674µs16032026/11/12 07:15:43 INFO Upload complete. (125ms)1604 client_integration_test.go:353: Retrieved .ls file from S3 (compressed size: 62 bytes)1605 client_integration_test.go:353: Decompressed .ls content (49 bytes):1606 {"version":1,"root":{"type":"regular","size":39}}1607 client_integration_test.go:364: Testing garbage collection...16082026/11/12 07:15:44 INFO Starting cleanup of old closures method=DELETE path=/api/closures16092026/11/12 07:15:44 INFO Garbage collection started16102026/11/12 07:15:44 INFO Aborted multipart uploads count=0 kept=01611--- PASS: TestObjectStatsUpdatedOncePerCommit (5.45s)1612=== CONT TestCacheConfigHandlerMaxNarSize1613--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)1614=== CONT TestMetricsCommit1615=== NAME TestClientCADerivations1616 client_ca_test.go:136: Built CA derivation: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientCADerivations1548563701/001/store/hji533r7v4yfsy00yhknvjmam60pmb1i-ca-test16172026/11/12 07:15:44 WARN Force mode enabled - objects will be deleted immediately without grace period1618 client_ca_test.go:139: Found 1 dependencies (including self)1619--- PASS: TestGetExistingObjectsLocksRows (5.24s)1620=== CONT TestMetricsInventory16212026/11/12 07:15:44 INFO Received push request method=POST path=/api/pushes16222026/11/12 07:15:44 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)16232026/11/12 07:15:44 INFO Uploading hji533r7v4yfsy00yhknvjmam60pmb1i-ca-test (144B)16242026/11/12 07:15:44 WARN Failed to register uploaded object key=hji533r7v4yfsy00yhknvjmam60pmb1i.ls error="server returned 404: 404 page not found\n"16252026/11/12 07:15:44 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"16262026/11/12 07:15:44 WARN Failed to register uploaded object key=log/dsv0rlp8jip9yy6sd6hrqnk29y8yl0ws-ca-test.drv error="server returned 404: 404 page not found\n"16272026/11/12 07:15:44 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16282026/11/12 07:15:44 INFO Signed narinfos id=1 count=116292026/11/12 07:15:44 INFO Uploading 1 narinfos16302026/11/12 07:15:44 INFO Received complete push request method=POST path=/api/pushes/1/complete16312026/11/12 07:15:44 WARN Failed to register uploaded object key=hji533r7v4yfsy00yhknvjmam60pmb1i.narinfo error="server returned 404: 404 page not found\n"1632=== NAME TestPinProtectsFromGC1633 client_integration_test.go:900: Pin successfully protected closure from garbage collection16342026/11/12 07:15:44 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=016352026/11/12 07:15:44 INFO Completed push id=1 objects=4 duration=64.9265ms16362026/11/12 07:15:44 INFO Upload complete. (482ms)1637=== NAME TestClientCADerivations1638 client_ca_test.go:180: Narinfo contains CA field: StorePath: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientCADerivations1548563701/001/store/hji533r7v4yfsy00yhknvjmam60pmb1i-ca-test1639 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1640 Compression: zstd1641 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1642 NarSize: 1441643 References: 1644 Deriver: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientCADerivations1548563701/001/store/dsv0rlp8jip9yy6sd6hrqnk29y8yl0ws-ca-test.drv1645 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1646 client_ca_test.go:185: Checking for realisation files in S3...1647 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1648 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache16492026/11/12 07:15:44 INFO Vacuumed table table=pending_closures1650--- PASS: TestObjectStatsTrigger (5.59s)1651=== CONT TestNARDeduplicationMetadataUploadBug1652--- PASS: TestPinProtectsFromGC (10.22s)16532026/11/12 07:15:44 INFO Vacuumed table table=commit_receipts1654=== CONT TestGCSweepDeliversEachKeyOnce1655=== NAME TestClientCADerivations1656 client_ca_test.go:258: nix copy output: error: binary cache 's3://bucket59?endpoint=http://localhost:60408®ion=eu-west-1' is for Nix stores with prefix '/nix/store', not '/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestClientCADerivations1548563701/001/store'1657 client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 116582026/11/12 07:15:44 INFO Vacuumed table table=pending_objects16592026/11/12 07:15:44 INFO Vacuumed table table=multipart_uploads16602026/11/12 07:15:44 INFO Vacuumed table table=closures16612026/11/12 07:15:45 INFO Vacuumed table table=objects1662--- PASS: TestClientCADerivations (6.78s)1663=== CONT TestCreatePendingClosureRejectsOversizedNAR16642026/11/12 07:15:45 INFO Received uploads request method=POST path=/api/pending_closures1665--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.01s)1666=== CONT TestTombstonedObjectOfferedWithoutWaiting1667--- PASS: TestRegisterCompletedObjectNeedsPendingRow (5.83s)1668=== CONT TestService_AuthMiddleware_MTLSBoundSubjects1669=== NAME TestOrphanedObjectsGC1670 orphaned_objects_gc_test.go:287: GC Test Summary:1671 orphaned_objects_gc_test.go:288: - Kept: 2 objects from closure A1672 orphaned_objects_gc_test.go:289: - Deleted: 2 objects from closure B1673 orphaned_objects_gc_test.go:290: - Deleted: 6 orphaned chain objects (X1->X2->X3)1674 orphaned_objects_gc_test.go:291: - Deleted: 2 orphaned single objects (Y)1675 orphaned_objects_gc_test.go:292: - Total deleted: 10 objects1676--- PASS: TestOrphanedObjectsGC (7.27s)1677=== CONT TestService_ReadAuthMiddleware16782026/11/12 07:15:45 INFO Received uploads request method=POST path=/api/pending_closures16792026/11/12 07:15:45 INFO Received uploads request method=POST path=/api/pending_closures16802026/11/12 07:15:45 INFO Received cleanup request method=DELETE path=/api/pending_closures16812026/11/12 07:15:45 INFO Aborted multipart uploads count=0 kept=016822026/11/12 07:15:45 WARN Force mode enabled - objects will be deleted immediately without grace period16832026/11/12 07:15:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=016842026/11/12 07:15:45 INFO Vacuumed table table=pending_closures16852026/11/12 07:15:45 INFO Vacuumed table table=commit_receipts16862026/11/12 07:15:45 INFO Vacuumed table table=pending_objects16872026/11/12 07:15:45 INFO Vacuumed table table=multipart_uploads16882026/11/12 07:15:45 INFO Vacuumed table table=closures16892026/11/12 07:15:46 INFO Vacuumed table table=objects16902026/11/12 07:15:46 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=01691=== NAME TestClientIntegration1692 client_integration_test.go:371: Objects in database after GC:1693 client_integration_test.go:371: Successfully deleted all objects with GC --force16942026/11/12 07:15:46 INFO Aborted multipart uploads count=0 kept=016952026/11/12 07:15:46 WARN Force mode enabled - objects will be deleted immediately without grace period16962026/11/12 07:15:46 INFO Received uploads request method=POST path=/api/pending_closures1697--- PASS: TestClientIntegration (8.45s)1698=== CONT TestGCSweepSkipsPendingObjects1699--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (6.01s)1700=== CONT TestService_AuthMiddleware_MTLSProxyHeader17012026/11/12 07:15:46 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=017022026/11/12 07:15:46 INFO Vacuumed table table=pending_closures17032026/11/12 07:15:46 INFO Vacuumed table table=commit_receipts17042026/11/12 07:15:46 INFO Vacuumed table table=pending_objects17052026/11/12 07:15:46 INFO Vacuumed table table=multipart_uploads17062026/11/12 07:15:46 INFO Vacuumed table table=closures17072026/11/12 07:15:46 INFO Vacuumed table table=objects1708--- PASS: TestSweepSkipsRowsLockedByPush (6.37s)1709=== CONT TestService_AuthMiddleware17102026/11/12 07:15:46 INFO Received uploads request method=POST path=/api/pending_closures17112026/11/12 07:15:46 INFO Aborted multipart uploads count=0 kept=017122026/11/12 07:15:46 INFO Received uploads request method=POST path=/api/pending_closures17132026/11/12 07:15:46 WARN Force mode enabled - objects will be deleted immediately without grace period17142026/11/12 07:15:46 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."17152026/11/12 07:15:46 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."1716--- PASS: TestFailedS3DeleteKeepsTombstone (6.38s)1717=== CONT TestService_AuthMiddleware_OIDC17182026/11/12 07:15:47 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:60793/oidc17192026/11/12 07:15:47 INFO Received cleanup request method=DELETE path=/api/pending_closures17202026/11/12 07:15:47 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket68/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""17212026/11/12 07:15:47 INFO Aborted multipart uploads count=0 kept=117222026/11/12 07:15:47 INFO Received cleanup request method=DELETE path=/api/pending_closures17232026/11/12 07:15:47 INFO Aborted multipart uploads count=1 kept=01724--- PASS: TestService_ReadScope_PublicByDefault (6.04s)1725=== CONT TestEnsureAbortUploadsRule1726--- PASS: TestMultipartCleanup (6.21s)1727=== CONT TestGenerateLandingPage1728--- PASS: TestGenerateLandingPage (0.00s)1729=== CONT TestService_readinessHandler17302026/11/12 07:15:47 INFO Received uploads request method=POST path=/api/pending_closures17312026/11/12 07:15:47 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst17322026/11/12 07:15:47 INFO Received uploads request method=POST path=/api/pending_closures1733--- PASS: TestPresignedUploadRegisteredBeforeCommit (6.50s)1734=== CONT TestService_healthCheckHandler1735=== RUN TestService_RequireScope_OIDC/builder_may_write1736=== PAUSE TestService_RequireScope_OIDC/builder_may_write1737=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1738=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1739=== RUN TestService_RequireScope_OIDC/ops_may_admin1740=== PAUSE TestService_RequireScope_OIDC/ops_may_admin1741=== RUN TestService_RequireScope_OIDC/ops_may_not_write1742=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1743=== RUN TestService_RequireScope_OIDC/reader_may_not_write1744=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1745=== RUN TestService_RequireScope_OIDC/static_token_may_admin1746=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1747=== RUN TestService_RequireScope_OIDC/static_token_may_write1748=== PAUSE TestService_RequireScope_OIDC/static_token_may_write1749=== RUN TestService_RequireScope_OIDC/reader_may_read1750=== PAUSE TestService_RequireScope_OIDC/reader_may_read1751=== RUN TestService_RequireScope_OIDC/writer_implies_read1752=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1753=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1754=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read1755=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc1756=== PAUSE TestService_RequireScope_OIDC/reader_may_read_with_netrc1757=== RUN TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1758=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1759=== RUN TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1760=== PAUSE TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1761=== RUN TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1762=== PAUSE TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1763=== CONT TestGracefulShutdownDrainsInflight17642026/11/12 07:15:48 INFO Starting HTTP server address=127.0.0.1:6080417652026/11/12 07:15:48 INFO Shutdown signal received, draining in-flight requests timeout=10s1766--- PASS: TestGracefulShutdownDrainsInflight (0.07s)1767=== CONT TestCacheStatsHandler17682026/11/12 07:15:48 INFO Received uploads request method=POST path=/api/pending_closures1769--- PASS: TestDeduplicatedObjectsRecordedAsPending (7.02s)1770=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info17712026/11/12 07:15:48 INFO Received uploads request method=POST path=/1772=== CONT TestCommitCanBeRepeatedUntilCleanup/pending_closure17732026/11/12 07:15:49 INFO Received uploads request method=POST path=/api/pending_closures17742026/11/12 07:15:49 INFO Aborted multipart uploads count=0 kept=017752026/11/12 07:15:49 INFO Aborted multipart uploads count=0 kept=017762026/11/12 07:15:49 WARN Force mode enabled - objects will be deleted immediately without grace period17772026/11/12 07:15:49 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=017782026/11/12 07:15:49 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=017792026/11/12 07:15:49 INFO Vacuumed table table=pending_closures17802026/11/12 07:15:49 INFO Vacuumed table table=commit_receipts17812026/11/12 07:15:49 INFO Vacuumed table table=pending_closures17822026/11/12 07:15:49 INFO Vacuumed table table=pending_objects17832026/11/12 07:15:49 INFO Vacuumed table table=multipart_uploads17842026/11/12 07:15:49 INFO Vacuumed table table=closures17852026/11/12 07:15:49 INFO Vacuumed table table=objects1786--- PASS: TestGCMetrics (7.35s)1787=== CONT TestCommitCanBeRepeatedUntilCleanup/push17882026/11/12 07:15:49 INFO Vacuumed table table=commit_receipts17892026/11/12 07:15:49 INFO Vacuumed table table=pending_objects17902026/11/12 07:15:49 INFO Vacuumed table table=multipart_uploads17912026/11/12 07:15:49 INFO Vacuumed table table=closures17922026/11/12 07:15:49 INFO Vacuumed table table=objects17932026/11/12 07:15:49 INFO Aborted multipart uploads count=0 kept=017942026/11/12 07:15:49 WARN Force mode enabled - objects will be deleted immediately without grace period17952026/11/12 07:15:49 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=017962026/11/12 07:15:49 INFO Vacuumed table table=pending_closures17972026/11/12 07:15:49 INFO Vacuumed table table=commit_receipts17982026/11/12 07:15:49 INFO Vacuumed table table=pending_objects17992026/11/12 07:15:49 INFO Vacuumed table table=multipart_uploads18002026/11/12 07:15:49 INFO Vacuumed table table=closures18012026/11/12 07:15:49 INFO Vacuumed table table=objects1802--- PASS: TestPushDedupSurvivesConcurrentGC (8.35s)1803=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key18042026/11/12 07:15:49 INFO Received complete multipart upload request method=POST path=/1805=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key18062026/11/12 07:15:49 INFO Received request for more parts method=POST path=/1807=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal18082026/11/12 07:15:49 INFO Received uploads request method=POST path=/1809--- PASS: TestUploadHandlersRejectInvalidKeys (0.01s)1810 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.01s)1811 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1812 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1813 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.01s)1814=== CONT TestPendingClosureWriteTimeout/empty1815=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1816--- PASS: TestGCBugBareHashReferences (8.00s)1817=== CONT TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline18182026/11/12 07:15:50 INFO Received uploads request method=POST path=/api/pending_closures1819--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (8.31s)1820=== CONT TestPendingClosureWriteTimeout/670k_objects1821=== CONT TestPendingClosureWriteTimeout/400_objects1822=== CONT TestProxyWriteTimeout/narinfo1823=== CONT TestProxyWriteTimeout/10_GiB_nar1824=== CONT TestProxyWriteTimeout/1_GiB_nar1825=== CONT TestProxyWriteTimeout/unknown_size1826--- PASS: TestProxyWriteTimeout (0.05s)1827 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1828 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1829 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)1830 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1831=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost18322026/11/12 07:15:50 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=_tribuchetbld1 database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"1833--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (8.03s)1834=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed18352026/11/12 07:15:50 INFO lead: acquired remote=192.0.2.1:1234 generation=118362026/11/12 07:15:51 INFO Received push request method=POST path=/api/pushes18372026/11/12 07:15:51 INFO Received complete push request method=POST path=/api/pushes/1/complete18382026/11/12 07:15:51 INFO Completed push id=1 objects=4 duration=102.287125ms18392026/11/12 07:15:51 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete18402026/11/12 07:15:51 INFO Aborted multipart uploads count=1 kept=018412026/11/12 07:15:51 INFO Completed upload id=2 objects=2 duration=1.963208ms1842--- PASS: TestMetricsCommit (7.66s)1843=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed18442026/11/12 07:15:51 INFO Received cleanup request method=DELETE path=/api/pending_closures18452026/11/12 07:15:51 INFO Aborted multipart uploads count=1 kept=01846--- PASS: TestPendingCleanupUsesOneCutoff (12.20s)1847=== CONT TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1848--- PASS: TestMetricsInventory (7.78s)1849=== CONT TestIsValidUploadKey/narinfo1850=== CONT TestIsValidUploadKey/realisation_plus_in_output1851=== CONT TestIsValidUploadKey/build_log1852=== CONT TestIsValidUploadKey/realisation1853=== CONT TestIsValidUploadKey/build_log_equals1854=== CONT TestIsValidUploadKey/build_log_question_mark1855=== CONT TestIsValidUploadKey/build_log_plus_in_name1856=== CONT TestIsValidUploadKey/build_log_home-manager_file1857=== CONT TestIsValidUploadKey/nar_xz1858=== CONT TestIsValidUploadKey/listing1859=== CONT TestIsValidUploadKey/nar_plain1860=== CONT TestIsValidUploadKey/traversal1861=== CONT TestIsValidUploadKey/unknown_type1862=== CONT TestIsValidUploadKey/empty_key1863=== CONT TestIsValidUploadKey/absolute1864=== CONT TestIsValidUploadKey/traversal_nar1865=== CONT TestIsValidUploadKey/narinfo_key,_nar_type1866=== CONT TestIsValidUploadKey/listing_key,_narinfo_type1867=== CONT TestIsValidUploadKey/nar_key,_narinfo_type1868=== CONT TestIsValidUploadKey/nar_zst1869=== CONT TestIsValidUploadKey/index.html1870=== CONT TestIsValidUploadKey/nix-cache-info1871--- PASS: TestIsValidUploadKey (0.06s)1872 --- PASS: TestIsValidUploadKey/narinfo (0.00s)1873 --- PASS: TestIsValidUploadKey/realisation_plus_in_output (0.00s)1874 --- PASS: TestIsValidUploadKey/build_log (0.00s)1875 --- PASS: TestIsValidUploadKey/realisation (0.00s)1876 --- PASS: TestIsValidUploadKey/build_log_equals (0.00s)1877 --- PASS: TestIsValidUploadKey/build_log_question_mark (0.00s)1878 --- PASS: TestIsValidUploadKey/build_log_plus_in_name (0.00s)1879 --- PASS: TestIsValidUploadKey/build_log_home-manager_file (0.00s)1880 --- PASS: TestIsValidUploadKey/nar_xz (0.00s)1881 --- PASS: TestIsValidUploadKey/listing (0.00s)1882 --- PASS: TestIsValidUploadKey/nar_plain (0.00s)1883 --- PASS: TestIsValidUploadKey/traversal (0.00s)1884 --- PASS: TestIsValidUploadKey/unknown_type (0.00s)1885 --- PASS: TestIsValidUploadKey/empty_key (0.00s)1886 --- PASS: TestIsValidUploadKey/absolute (0.00s)1887 --- PASS: TestIsValidUploadKey/traversal_nar (0.00s)1888 --- PASS: TestIsValidUploadKey/narinfo_key,_nar_type (0.00s)1889 --- PASS: TestIsValidUploadKey/listing_key,_narinfo_type (0.00s)1890 --- PASS: TestIsValidUploadKey/nar_key,_narinfo_type (0.00s)1891 --- PASS: TestIsValidUploadKey/nar_zst (0.00s)1892 --- PASS: TestIsValidUploadKey/index.html (0.00s)1893 --- PASS: TestIsValidUploadKey/nix-cache-info (0.00s)1894=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure18952026/11/12 07:15:52 INFO Received uploads request method=POST path=/18962026/11/12 07:15:52 INFO lead: released remote=192.0.2.1:1234 generation=11897--- PASS: TestLeadEndsOnShutdown (8.94s)1898=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts18992026/11/12 07:15:52 INFO Received request for more parts method=POST path=/1900=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart19012026/11/12 07:15:52 INFO Received complete multipart upload request method=POST path=/1902=== CONT TestPush_RejectsBadRequests/no_objects19032026/11/12 07:15:52 INFO Received push request method=POST path=/api/pushes1904=== CONT TestPush_RejectsBadRequests/root_not_in_objects19052026/11/12 07:15:52 INFO Received push request method=POST path=/api/pushes1906=== CONT TestPush_RejectsBadRequests/no_roots19072026/11/12 07:15:52 INFO Received push request method=POST path=/api/pushes1908=== CONT TestPush_RejectsBadRequests/bad_root19092026/11/12 07:15:52 INFO Received push request method=POST path=/api/pushes1910=== CONT TestResolveDBConnectionString/flag_wins1911=== CONT TestResolveDBConnectionString/PGHOST_allows_empty1912=== CONT TestResolveDBConnectionString/nothing_configured1913=== CONT TestResolveDBConnectionString/missing_file_is_an_error1914--- PASS: TestPush_RejectsBadRequests (2.45s)1915 --- PASS: TestPush_RejectsBadRequests/no_objects (0.00s)1916 --- PASS: TestPush_RejectsBadRequests/root_not_in_objects (0.00s)1917 --- PASS: TestPush_RejectsBadRequests/no_roots (0.00s)1918 --- PASS: TestPush_RejectsBadRequests/bad_root (0.00s)1919=== CONT TestResolveDBConnectionString/file_when_flag_empty1920=== CONT TestIsValidCachePath/narinfo1921=== CONT TestIsValidCachePath/index.html1922=== CONT TestIsValidCachePath/short_hash1923=== CONT TestIsValidCachePath/wrong_extension1924=== CONT TestIsValidCachePath/leading_slash1925=== CONT TestIsValidCachePath/empty1926=== CONT TestIsValidCachePath/random_path1927=== CONT TestIsValidCachePath/invalid_char_u1928=== CONT TestIsValidCachePath/invalid_char_e1929=== CONT TestIsValidCachePath/traversal_in_middle1930=== CONT TestIsValidCachePath/traversal_parent1931=== CONT TestIsValidCachePath/nar_uncompressed1932=== CONT TestIsValidCachePath/nix-cache-info1933=== CONT TestIsValidCachePath/realisation1934=== CONT TestIsValidCachePath/log1935=== CONT TestIsValidCachePath/ls1936=== CONT TestIsValidCachePath/nar_xz1937=== CONT TestIsValidCachePath/nar_bz21938=== CONT TestIsValidCachePath/nar_zst1939=== CONT TestIsValidCachePath/narinfo_all_nix_base32_chars1940--- PASS: TestIsValidCachePath (0.00s)1941 --- PASS: TestIsValidCachePath/narinfo (0.00s)1942 --- PASS: TestIsValidCachePath/index.html (0.00s)1943 --- PASS: TestIsValidCachePath/short_hash (0.00s)1944 --- PASS: TestIsValidCachePath/wrong_extension (0.00s)1945 --- PASS: TestIsValidCachePath/leading_slash (0.00s)1946 --- PASS: TestIsValidCachePath/empty (0.00s)1947 --- PASS: TestIsValidCachePath/random_path (0.00s)1948 --- PASS: TestIsValidCachePath/invalid_char_u (0.00s)1949 --- PASS: TestIsValidCachePath/invalid_char_e (0.00s)1950 --- PASS: TestIsValidCachePath/traversal_in_middle (0.00s)1951 --- PASS: TestIsValidCachePath/traversal_parent (0.00s)1952 --- PASS: TestIsValidCachePath/nar_uncompressed (0.00s)1953 --- PASS: TestIsValidCachePath/nix-cache-info (0.00s)1954 --- PASS: TestIsValidCachePath/realisation (0.00s)1955 --- PASS: TestIsValidCachePath/log (0.00s)1956 --- PASS: TestIsValidCachePath/ls (0.00s)1957 --- PASS: TestIsValidCachePath/nar_xz (0.00s)1958 --- PASS: TestIsValidCachePath/nar_bz2 (0.00s)1959 --- PASS: TestIsValidCachePath/nar_zst (0.00s)1960 --- PASS: TestIsValidCachePath/narinfo_all_nix_base32_chars (0.00s)1961=== CONT TestParseSingleRange/none1962=== CONT TestParseSingleRange/open-ended1963=== CONT TestParseSingleRange/start_far_past_EOF1964=== CONT TestParseSingleRange/start_past_EOF1965=== CONT TestParseSingleRange/single_byte1966=== CONT TestParseSingleRange/suffix_exceeds_size1967=== CONT TestParseSingleRange/suffix1968=== CONT TestParseSingleRange/end_clamped_to_size1969=== CONT TestParseSingleRange/malformed_both_empty1970=== CONT TestParseSingleRange/closed1971=== CONT TestParseSingleRange/malformed_end_before_start1972=== CONT TestParseSingleRange/unknown_unit1973=== CONT TestParseSingleRange/malformed_no_dash1974=== CONT TestParseSingleRange/multi-range_ignored1975--- PASS: TestParseSingleRange (0.01s)1976 --- PASS: TestParseSingleRange/none (0.00s)1977 --- PASS: TestParseSingleRange/open-ended (0.00s)1978 --- PASS: TestParseSingleRange/start_far_past_EOF (0.00s)1979 --- PASS: TestParseSingleRange/start_past_EOF (0.00s)1980 --- PASS: TestParseSingleRange/single_byte (0.00s)1981 --- PASS: TestParseSingleRange/suffix_exceeds_size (0.00s)1982 --- PASS: TestParseSingleRange/suffix (0.00s)1983 --- PASS: TestParseSingleRange/end_clamped_to_size (0.00s)1984 --- PASS: TestParseSingleRange/malformed_both_empty (0.00s)1985 --- PASS: TestParseSingleRange/closed (0.00s)1986 --- PASS: TestParseSingleRange/malformed_end_before_start (0.00s)1987 --- PASS: TestParseSingleRange/unknown_unit (0.00s)1988 --- PASS: TestParseSingleRange/malformed_no_dash (0.00s)1989 --- PASS: TestParseSingleRange/multi-range_ignored (0.00s)1990=== CONT TestPresentNotReportedWhileGCDeletesClosure/stale1991--- PASS: TestResolveDBConnectionString (0.02s)1992 --- PASS: TestResolveDBConnectionString/flag_wins (0.00s)1993 --- PASS: TestResolveDBConnectionString/PGHOST_allows_empty (0.00s)1994 --- PASS: TestResolveDBConnectionString/nothing_configured (0.00s)1995 --- PASS: TestResolveDBConnectionString/missing_file_is_an_error (0.00s)1996 --- PASS: TestResolveDBConnectionString/file_when_flag_empty (0.00s)1997=== CONT TestPresentNotReportedWhileGCDeletesClosure/fresh1998--- PASS: TestUploadHandlersRejectOversizedBody (0.08s)1999 --- PASS: TestUploadHandlersRejectOversizedBody/request_more_parts (0.03s)2000 --- PASS: TestUploadHandlersRejectOversizedBody/complete_multipart (0.02s)
loading 900 hidden lines…
29012026/11/12 07:16:15 INFO Aborted multipart uploads count=0 kept=029022026/11/12 07:16:15 INFO Completed upload id=3 objects=25 duration=471.625µs29032026/11/12 07:16:15 INFO Vacuumed table table=multipart_uploads29042026/11/12 07:16:15 INFO Aborted multipart uploads count=0 kept=029052026/11/12 07:16:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=029062026/11/12 07:16:15 INFO Vacuumed table table=pending_closures29072026/11/12 07:16:15 WARN Force mode enabled - objects will be deleted immediately without grace period29082026/11/12 07:16:15 INFO Received complete upload request method=POST path=/29092026/11/12 07:16:15 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=029102026/11/12 07:16:15 INFO Completed upload id=17 objects=6 duration=817.875µs29112026/11/12 07:16:15 INFO Vacuumed table table=closures29122026/11/12 07:16:15 INFO Vacuumed table table=objects29132026/11/12 07:16:15 INFO Vacuumed table table=commit_receipts29142026/11/12 07:16:15 INFO Received push request method=POST path=/29152026/11/12 07:16:15 INFO Vacuumed table table=pending_objects29162026/11/12 07:16:15 INFO Vacuumed table table=multipart_uploads29172026/11/12 07:16:15 INFO Vacuumed table table=pending_closures29182026/11/12 07:16:15 INFO Received push request method=POST path=/29192026/11/12 07:16:15 INFO Received uploads request method=POST path=/29202026/11/12 07:16:15 INFO Received push request method=POST path=/29212026/11/12 07:16:15 INFO Vacuumed table table=closures29222026/11/12 07:16:15 INFO Received uploads request method=POST path=/29232026/11/12 07:16:15 INFO Received complete upload request method=POST path=/29242026/11/12 07:16:15 INFO Vacuumed table table=commit_receipts29252026/11/12 07:16:15 INFO Completed upload id=15 objects=9 duration=6.015375ms29262026/11/12 07:16:15 INFO Vacuumed table table=objects29272026/11/12 07:16:15 INFO Vacuumed table table=pending_objects29282026/11/12 07:16:15 INFO Vacuumed table table=multipart_uploads29292026/11/12 07:16:15 INFO Received complete push request method=POST path=/29302026/11/12 07:16:15 INFO Vacuumed table table=closures29312026/11/12 07:16:15 INFO Completed push id=21 objects=17 duration=595.167µs29322026/11/12 07:16:15 INFO Received push request method=POST path=/29332026/11/12 07:16:15 INFO Received complete push request method=POST path=/29342026/11/12 07:16:15 INFO Completed push id=19 objects=7 duration=485.791µs29352026/11/12 07:16:15 INFO Received push request method=POST path=/29362026/11/12 07:16:15 INFO Vacuumed table table=objects29372026/11/12 07:16:15 INFO Received complete push request method=POST path=/29382026/11/12 07:16:15 INFO Received push request method=POST path=/29392026/11/12 07:16:15 INFO Aborted multipart uploads count=0 kept=029402026/11/12 07:16:15 INFO Completed push id=15 objects=24 duration=5.802458ms29412026/11/12 07:16:15 INFO Received complete upload request method=POST path=/29422026/11/12 07:16:15 WARN Force mode enabled - objects will be deleted immediately without grace period29432026/11/12 07:16:15 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029442026/11/12 07:16:15 INFO Completed upload id=10 objects=6 duration=7.323875ms29452026/11/12 07:16:15 INFO Vacuumed table table=pending_closures29462026/11/12 07:16:15 INFO Received uploads request method=POST path=/29472026/11/12 07:16:15 INFO Vacuumed table table=commit_receipts29482026/11/12 07:16:15 INFO Vacuumed table table=pending_objects29492026/11/12 07:16:15 INFO Vacuumed table table=multipart_uploads29502026/11/12 07:16:15 INFO Received uploads request method=POST path=/29512026/11/12 07:16:15 INFO Received uploads request method=POST path=/29522026/11/12 07:16:15 INFO Vacuumed table table=closures29532026/11/12 07:16:15 INFO Received push request method=POST path=/29542026/11/12 07:16:15 INFO Vacuumed table table=objects29552026/11/12 07:16:15 INFO Received push request method=POST path=/29562026/11/12 07:16:15 INFO Received complete push request method=POST path=/29572026/11/12 07:16:15 INFO Received uploads request method=POST path=/29582026/11/12 07:16:15 INFO Received uploads request method=POST path=/29592026/11/12 07:16:15 INFO Completed push id=8 objects=6 duration=507.666µs29602026/11/12 07:16:16 INFO Received complete push request method=POST path=/29612026/11/12 07:16:16 INFO Received complete push request method=POST path=/29622026/11/12 07:16:16 INFO Completed push id=6 objects=8 duration=929.5µs29632026/11/12 07:16:16 INFO Received uploads request method=POST path=/29642026/11/12 07:16:16 INFO Received uploads request method=POST path=/29652026/11/12 07:16:16 INFO Received uploads request method=POST path=/29662026/11/12 07:16:16 INFO Completed push id=3 objects=8 duration=38.21775ms29672026/11/12 07:16:16 INFO Received complete upload request method=POST path=/29682026/11/12 07:16:16 INFO Received complete upload request method=POST path=/29692026/11/12 07:16:16 INFO Completed upload id=6 objects=6 duration=554.875µs29702026/11/12 07:16:16 INFO Received push request method=POST path=/29712026/11/12 07:16:16 INFO Completed upload id=7 objects=4 duration=481.917µs29722026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=029732026/11/12 07:16:16 INFO Received complete push request method=POST path=/29742026/11/12 07:16:16 WARN Force mode enabled - objects will be deleted immediately without grace period29752026/11/12 07:16:16 INFO Completed push id=4 objects=7 duration=533µs29762026/11/12 07:16:16 INFO Received push request method=POST path=/29772026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029782026/11/12 07:16:16 INFO Vacuumed table table=pending_closures29792026/11/12 07:16:16 INFO Received uploads request method=POST path=/29802026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts29812026/11/12 07:16:16 INFO Vacuumed table table=pending_objects29822026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=029832026/11/12 07:16:16 INFO Received uploads request method=POST path=/29842026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads29852026/11/12 07:16:16 INFO Received complete push request method=POST path=/29862026/11/12 07:16:16 WARN Force mode enabled - objects will be deleted immediately without grace period29872026/11/12 07:16:16 INFO Vacuumed table table=closures29882026/11/12 07:16:16 INFO Completed push id=12 objects=7 duration=546.667µs29892026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029902026/11/12 07:16:16 INFO Received push request method=POST path=/29912026/11/12 07:16:16 INFO Received complete upload request method=POST path=/29922026/11/12 07:16:16 INFO Vacuumed table table=objects29932026/11/12 07:16:16 INFO Vacuumed table table=pending_closures29942026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=029952026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=029962026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts29972026/11/12 07:16:16 INFO Received push request method=POST path=/29982026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=029992026/11/12 07:16:16 INFO Received uploads request method=POST path=/30002026/11/12 07:16:16 INFO Vacuumed table table=pending_objects30012026/11/12 07:16:16 WARN Force mode enabled - objects will be deleted immediately without grace period30022026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads30032026/11/12 07:16:16 INFO Received complete upload request method=POST path=/30042026/11/12 07:16:16 INFO Completed upload id=2 objects=26 duration=34.080083ms30052026/11/12 07:16:16 INFO Completed upload id=12 objects=26 duration=6.372791ms30062026/11/12 07:16:16 INFO Vacuumed table table=closures30072026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=030082026/11/12 07:16:16 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."30092026/11/12 07:16:16 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."30102026/11/12 07:16:16 INFO Vacuumed table table=objects30112026/11/12 07:16:16 INFO Received push request method=POST path=/30122026/11/12 07:16:16 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."30132026/11/12 07:16:16 INFO Vacuumed table table=pending_closures30142026/11/12 07:16:16 WARN Force mode enabled - objects will be deleted immediately without grace period30152026/11/12 07:16:16 INFO Received uploads request method=POST path=/30162026/11/12 07:16:16 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."30172026/11/12 07:16:16 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."30182026/11/12 07:16:16 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."30192026/11/12 07:16:16 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."30202026/11/12 07:16:16 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."30212026/11/12 07:16:16 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."30222026/11/12 07:16:16 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."30232026/11/12 07:16:16 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."30242026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts30252026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030262026/11/12 07:16:16 INFO Vacuumed table table=pending_closures30272026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts30282026/11/12 07:16:16 INFO Received push request method=POST path=/30292026/11/12 07:16:16 INFO Received push request method=POST path=/30302026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=11 objects-failed-to-delete=030312026/11/12 07:16:16 INFO Received complete push request method=POST path=/30322026/11/12 07:16:16 INFO Received uploads request method=POST path=/30332026/11/12 07:16:16 INFO Received complete push request method=POST path=/30342026/11/12 07:16:16 INFO Vacuumed table table=pending_objects30352026/11/12 07:16:16 INFO Vacuumed table table=pending_closures30362026/11/12 07:16:16 INFO Completed push id=16 objects=8 duration=527.542µs30372026/11/12 07:16:16 INFO Completed push id=8 objects=2 duration=438.5µs30382026/11/12 07:16:16 INFO Received complete upload request method=POST path=/30392026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads30402026/11/12 07:16:16 INFO Vacuumed table table=pending_objects30412026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts30422026/11/12 07:16:16 INFO Received push request method=POST path=/30432026/11/12 07:16:16 INFO Vacuumed table table=closures30442026/11/12 07:16:16 INFO Completed upload id=9 objects=7 duration=1.32925ms30452026/11/12 07:16:16 INFO Received uploads request method=POST path=/30462026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=030472026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads30482026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=030492026/11/12 07:16:16 INFO Vacuumed table table=objects30502026/11/12 07:16:16 INFO Vacuumed table table=pending_objects30512026/11/12 07:16:16 WARN Force mode enabled - objects will be deleted immediately without grace period30522026/11/12 07:16:16 INFO Vacuumed table table=closures30532026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=030542026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads30552026/11/12 07:16:16 INFO Vacuumed table table=objects30562026/11/12 07:16:16 INFO Vacuumed table table=pending_closures30572026/11/12 07:16:16 INFO Vacuumed table table=closures30582026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030592026/11/12 07:16:16 INFO Received uploads request method=POST path=/30602026/11/12 07:16:16 INFO Vacuumed table table=objects30612026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=030622026/11/12 07:16:16 INFO Received push request method=POST path=/30632026/11/12 07:16:16 INFO Vacuumed table table=pending_closures30642026/11/12 07:16:16 INFO Received uploads request method=POST path=/30652026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts30662026/11/12 07:16:16 INFO Vacuumed table table=pending_objects30672026/11/12 07:16:16 INFO Received push request method=POST path=/30682026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts30692026/11/12 07:16:16 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."30702026/11/12 07:16:16 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."30712026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads30722026/11/12 07:16:16 INFO Received uploads request method=POST path=/30732026/11/12 07:16:16 INFO Received uploads request method=POST path=/30742026/11/12 07:16:16 INFO Received push request method=POST path=/30752026/11/12 07:16:16 INFO Vacuumed table table=pending_objects30762026/11/12 07:16:16 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."30772026/11/12 07:16:16 INFO Vacuumed table table=closures30782026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads30792026/11/12 07:16:16 INFO Received complete push request method=POST path=/30802026/11/12 07:16:16 INFO Received push request method=POST path=/30812026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=030822026/11/12 07:16:16 INFO Received uploads request method=POST path=/30832026/11/12 07:16:16 INFO Vacuumed table table=objects30842026/11/12 07:16:16 INFO Completed push id=17 objects=22 duration=5.466042ms30852026/11/12 07:16:16 WARN Force mode enabled - objects will be deleted immediately without grace period30862026/11/12 07:16:16 INFO Received uploads request method=POST path=/30872026/11/12 07:16:16 INFO Received uploads request method=POST path=/30882026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030892026/11/12 07:16:16 INFO Vacuumed table table=pending_closures30902026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts30912026/11/12 07:16:16 INFO Received complete push request method=POST path=/30922026/11/12 07:16:16 INFO Vacuumed table table=pending_objects30932026/11/12 07:16:16 INFO Completed push id=10 objects=2 duration=651.708µs30942026/11/12 07:16:16 INFO Received push request method=POST path=/30952026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads30962026/11/12 07:16:16 INFO Vacuumed table table=closures30972026/11/12 07:16:16 INFO Vacuumed table table=closures30982026/11/12 07:16:16 INFO Received uploads request method=POST path=/30992026/11/12 07:16:16 INFO Vacuumed table table=objects31002026/11/12 07:16:16 INFO Vacuumed table table=objects31012026/11/12 07:16:16 INFO Received complete push request method=POST path=/31022026/11/12 07:16:16 INFO Received push request method=POST path=/31032026/11/12 07:16:16 INFO Completed push id=14 objects=4 duration=1.03025ms31042026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=031052026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=031062026/11/12 07:16:16 INFO Received push request method=POST path=/31072026/11/12 07:16:16 WARN Force mode enabled - objects will be deleted immediately without grace period31082026/11/12 07:16:16 WARN Force mode enabled - objects will be deleted immediately without grace period31092026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031102026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031112026/11/12 07:16:16 INFO Received push request method=POST path=/31122026/11/12 07:16:16 INFO Received push request method=POST path=/31132026/11/12 07:16:16 INFO Received complete push request method=POST path=/31142026/11/12 07:16:16 INFO Vacuumed table table=pending_closures31152026/11/12 07:16:16 INFO Received push request method=POST path=/31162026/11/12 07:16:16 INFO Received complete push request method=POST path=/31172026/11/12 07:16:16 INFO Received uploads request method=POST path=/31182026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts31192026/11/12 07:16:16 INFO Completed push id=11 objects=6 duration=1.164459ms31202026/11/12 07:16:16 INFO Vacuumed table table=pending_closures31212026/11/12 07:16:16 INFO Completed push id=2 objects=7 duration=2.138334ms31222026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts31232026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=031242026/11/12 07:16:16 INFO Vacuumed table table=pending_objects31252026/11/12 07:16:16 INFO Received complete upload request method=POST path=/31262026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads31272026/11/12 07:16:16 INFO Completed upload id=12 objects=21 duration=620.083µs31282026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031292026/11/12 07:16:16 INFO Received push request method=POST path=/31302026/11/12 07:16:16 INFO Vacuumed table table=pending_objects31312026/11/12 07:16:16 INFO Vacuumed table table=closures31322026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=031332026/11/12 07:16:16 INFO Received push request method=POST path=/31342026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads31352026/11/12 07:16:16 INFO Vacuumed table table=objects31362026/11/12 07:16:16 INFO Received push request method=POST path=/31372026/11/12 07:16:16 WARN Force mode enabled - objects will be deleted immediately without grace period31382026/11/12 07:16:16 INFO Vacuumed table table=closures31392026/11/12 07:16:16 INFO Vacuumed table table=pending_closures31402026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031412026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts31422026/11/12 07:16:16 INFO Received push request method=POST path=/31432026/11/12 07:16:16 INFO Received push request method=POST path=/31442026/11/12 07:16:16 INFO Vacuumed table table=objects31452026/11/12 07:16:16 INFO Vacuumed table table=pending_closures31462026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts31472026/11/12 07:16:16 INFO Received uploads request method=POST path=/31482026/11/12 07:16:16 INFO Vacuumed table table=pending_objects31492026/11/12 07:16:16 INFO Received uploads request method=POST path=/31502026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads31512026/11/12 07:16:16 INFO Vacuumed table table=closures31522026/11/12 07:16:16 INFO Vacuumed table table=pending_objects31532026/11/12 07:16:16 INFO Received push request method=POST path=/31542026/11/12 07:16:16 INFO Vacuumed table table=objects31552026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads31562026/11/12 07:16:16 INFO Received complete push request method=POST path=/31572026/11/12 07:16:16 INFO Received push request method=POST path=/31582026/11/12 07:16:16 INFO Received complete push request method=POST path=/31592026/11/12 07:16:16 INFO Completed push id=13 objects=23 duration=613.709µs31602026/11/12 07:16:16 INFO Vacuumed table table=closures31612026/11/12 07:16:16 INFO Received uploads request method=POST path=/31622026/11/12 07:16:16 INFO Completed push id=18 objects=6 duration=539.083µs31632026/11/12 07:16:16 INFO Vacuumed table table=objects31642026/11/12 07:16:16 INFO Received push request method=POST path=/31652026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=031662026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=031672026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=031682026/11/12 07:16:16 INFO Received complete push request method=POST path=/31692026/11/12 07:16:16 INFO Received uploads request method=POST path=/31702026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031712026/11/12 07:16:16 WARN Force mode enabled - objects will be deleted immediately without grace period31722026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=031732026/11/12 07:16:16 INFO Received uploads request method=POST path=/31742026/11/12 07:16:16 INFO Vacuumed table table=pending_closures31752026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031762026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts31772026/11/12 07:16:16 INFO Vacuumed table table=pending_objects31782026/11/12 07:16:16 INFO Received push request method=POST path=/31792026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads31802026/11/12 07:16:16 INFO Completed push id=1 objects=10 duration=52.624833ms31812026/11/12 07:16:16 INFO Received push request method=POST path=/31822026/11/12 07:16:16 INFO Vacuumed table table=pending_closures31832026/11/12 07:16:16 INFO Vacuumed table table=closures31842026/11/12 07:16:16 INFO Vacuumed table table=pending_closures31852026/11/12 07:16:16 INFO Received uploads request method=POST path=/31862026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts31872026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts31882026/11/12 07:16:16 INFO Vacuumed table table=objects31892026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=031902026/11/12 07:16:16 INFO Received push request method=POST path=/31912026/11/12 07:16:16 INFO Received complete upload request method=POST path=/31922026/11/12 07:16:16 WARN Force mode enabled - objects will be deleted immediately without grace period31932026/11/12 07:16:16 INFO Vacuumed table table=pending_objects31942026/11/12 07:16:16 INFO Vacuumed table table=pending_objects31952026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads31962026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031972026/11/12 07:16:16 INFO Completed upload id=20 objects=3 duration=972.083µs31982026/11/12 07:16:16 INFO Received push request method=POST path=/31992026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads32002026/11/12 07:16:16 INFO Vacuumed table table=closures32012026/11/12 07:16:16 INFO Vacuumed table table=pending_closures32022026/11/12 07:16:16 INFO Received complete push request method=POST path=/32032026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts32042026/11/12 07:16:16 INFO Received complete push request method=POST path=/32052026/11/12 07:16:16 INFO Vacuumed table table=objects32062026/11/12 07:16:16 INFO Vacuumed table table=closures32072026/11/12 07:16:16 INFO Received push request method=POST path=/32082026/11/12 07:16:16 INFO Received complete upload request method=POST path=/32092026/11/12 07:16:16 INFO Completed upload id=3 objects=5 duration=642.042µs32102026/11/12 07:16:16 INFO Completed push id=19 objects=8 duration=1.554834ms32112026/11/12 07:16:16 INFO Vacuumed table table=objects32122026/11/12 07:16:16 INFO Vacuumed table table=pending_objects32132026/11/12 07:16:16 INFO Received complete upload request method=POST path=/32142026/11/12 07:16:16 INFO Received push request method=POST path=/32152026/11/12 07:16:16 INFO Received uploads request method=POST path=/32162026/11/12 07:16:16 INFO Received complete upload request method=POST path=/32172026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads32182026/11/12 07:16:16 INFO Completed upload id=23 objects=16 duration=1.086875ms32192026/11/12 07:16:16 INFO Received push request method=POST path=/32202026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=032212026/11/12 07:16:16 INFO Completed upload id=24 objects=7 duration=276.334µs32222026/11/12 07:16:16 INFO Completed push id=1 objects=5 duration=45.278625ms32232026/11/12 07:16:16 INFO Vacuumed table table=closures32242026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032252026/11/12 07:16:16 INFO Vacuumed table table=objects32262026/11/12 07:16:16 INFO Vacuumed table table=pending_closures32272026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=032282026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=032292026/11/12 07:16:16 INFO Received push request method=POST path=/32302026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts32312026/11/12 07:16:16 WARN Force mode enabled - objects will be deleted immediately without grace period32322026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032332026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=032342026/11/12 07:16:16 INFO Received uploads request method=POST path=/32352026/11/12 07:16:16 INFO Vacuumed table table=pending_objects32362026/11/12 07:16:16 INFO Vacuumed table table=pending_closures32372026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032382026/11/12 07:16:16 WARN Force mode enabled - objects will be deleted immediately without grace period32392026/11/12 07:16:16 INFO Received uploads request method=POST path=/32402026/11/12 07:16:16 INFO Vacuumed table table=pending_closures32412026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts32422026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads32432026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032442026/11/12 07:16:16 INFO Received complete upload request method=POST path=/32452026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts32462026/11/12 07:16:16 INFO Completed upload id=15 objects=4 duration=4.347959ms32472026/11/12 07:16:16 INFO Vacuumed table table=pending_closures32482026/11/12 07:16:16 INFO Vacuumed table table=pending_objects32492026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads32502026/11/12 07:16:16 INFO Vacuumed table table=commit_receipts32512026/11/12 07:16:16 INFO Vacuumed table table=pending_objects32522026/11/12 07:16:16 INFO Received push request method=POST path=/32532026/11/12 07:16:16 INFO Received complete upload request method=POST path=/32542026/11/12 07:16:16 INFO Vacuumed table table=closures32552026/11/12 07:16:16 INFO Received push request method=POST path=/32562026/11/12 07:16:16 INFO Vacuumed table table=closures32572026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads32582026/11/12 07:16:16 INFO Completed upload id=6 objects=24 duration=1.090917ms32592026/11/12 07:16:16 INFO Received uploads request method=POST path=/32602026/11/12 07:16:16 INFO Vacuumed table table=objects32612026/11/12 07:16:16 INFO Vacuumed table table=closures32622026/11/12 07:16:16 INFO Vacuumed table table=objects32632026/11/12 07:16:16 INFO Vacuumed table table=objects32642026/11/12 07:16:16 INFO Vacuumed table table=pending_objects32652026/11/12 07:16:16 INFO Received push request method=POST path=/32662026/11/12 07:16:16 INFO Received complete upload request method=POST path=/32672026/11/12 07:16:16 INFO Received uploads request method=POST path=/32682026/11/12 07:16:16 INFO Completed upload id=16 objects=3 duration=302.084µs32692026/11/12 07:16:16 INFO Vacuumed table table=multipart_uploads32702026/11/12 07:16:16 INFO Vacuumed table table=closures32712026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=032722026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=032732026/11/12 07:16:16 INFO Vacuumed table table=objects32742026/11/12 07:16:16 WARN Force mode enabled - objects will be deleted immediately without grace period32752026/11/12 07:16:16 INFO Aborted multipart uploads count=0 kept=032762026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032772026/11/12 07:16:16 INFO Received uploads request method=POST path=/32782026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032792026/11/12 07:16:16 WARN Force mode enabled - objects will be deleted immediately without grace period32802026/11/12 07:16:16 INFO Received push request method=POST path=/32812026/11/12 07:16:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032822026/11/12 07:16:17 INFO Received complete push request method=POST path=/32832026/11/12 07:16:17 INFO Received complete push request method=POST path=/32842026/11/12 07:16:17 INFO Vacuumed table table=pending_closures32852026/11/12 07:16:17 INFO Vacuumed table table=pending_closures32862026/11/12 07:16:17 INFO Vacuumed table table=pending_closures32872026/11/12 07:16:17 INFO Completed push id=4 objects=3 duration=19.688083ms32882026/11/12 07:16:17 INFO Completed push id=8 objects=6 duration=27.037209ms32892026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts32902026/11/12 07:16:17 INFO Received complete push request method=POST path=/32912026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts32922026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts32932026/11/12 07:16:17 INFO Vacuumed table table=pending_objects32942026/11/12 07:16:17 INFO Completed push id=16 objects=16 duration=1.805333ms32952026/11/12 07:16:17 INFO Vacuumed table table=multipart_uploads32962026/11/12 07:16:17 INFO Received push request method=POST path=/32972026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=032982026/11/12 07:16:17 INFO Vacuumed table table=pending_objects32992026/11/12 07:16:17 INFO Vacuumed table table=closures33002026/11/12 07:16:17 INFO Received push request method=POST path=/33012026/11/12 07:16:17 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."33022026/11/12 07:16:17 INFO Vacuumed table table=multipart_uploads33032026/11/12 07:16:17 INFO Vacuumed table table=pending_objects33042026/11/12 07:16:17 INFO Vacuumed table table=objects33052026/11/12 07:16:17 INFO Received push request method=POST path=/33062026/11/12 07:16:17 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."33072026/11/12 07:16:17 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."33082026/11/12 07:16:17 INFO Vacuumed table table=closures33092026/11/12 07:16:17 INFO Vacuumed table table=multipart_uploads33102026/11/12 07:16:17 INFO Received complete push request method=POST path=/33112026/11/12 07:16:17 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."33122026/11/12 07:16:17 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."33132026/11/12 07:16:17 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."33142026/11/12 07:16:17 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."33152026/11/12 07:16:17 INFO Vacuumed table table=closures33162026/11/12 07:16:17 INFO Completed push id=2 objects=4 duration=1.04125ms33172026/11/12 07:16:17 INFO Received push request method=POST path=/33182026/11/12 07:16:17 INFO Received uploads request method=POST path=/33192026/11/12 07:16:17 INFO Received uploads request method=POST path=/33202026/11/12 07:16:17 INFO Vacuumed table table=objects33212026/11/12 07:16:17 INFO Vacuumed table table=objects33222026/11/12 07:16:17 INFO Received uploads request method=POST path=/33232026/11/12 07:16:17 INFO Received complete upload request method=POST path=/33242026/11/12 07:16:17 INFO Completed upload id=9 objects=3 duration=1.121833ms33252026/11/12 07:16:17 INFO Received uploads request method=POST path=/33262026/11/12 07:16:17 INFO Received uploads request method=POST path=/33272026/11/12 07:16:17 INFO Received uploads request method=POST path=/33282026/11/12 07:16:17 INFO Received push request method=POST path=/33292026/11/12 07:16:17 INFO Received uploads request method=POST path=/33302026/11/12 07:16:17 INFO Received uploads request method=POST path=/33312026/11/12 07:16:17 INFO Received uploads request method=POST path=/33322026/11/12 07:16:17 INFO Received push request method=POST path=/33332026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=033342026/11/12 07:16:17 INFO Received push request method=POST path=/33352026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=033362026/11/12 07:16:17 WARN Force mode enabled - objects will be deleted immediately without grace period33372026/11/12 07:16:17 INFO Received complete upload request method=POST path=/33382026/11/12 07:16:17 INFO Received uploads request method=POST path=/33392026/11/12 07:16:17 INFO Received uploads request method=POST path=/33402026/11/12 07:16:17 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."33412026/11/12 07:16:17 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."33422026/11/12 07:16:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033432026/11/12 07:16:17 INFO Completed upload id=18 objects=4 duration=847.625µs33442026/11/12 07:16:17 INFO Received uploads request method=POST path=/33452026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=033462026/11/12 07:16:17 INFO Received uploads request method=POST path=/33472026/11/12 07:16:17 WARN Force mode enabled - objects will be deleted immediately without grace period33482026/11/12 07:16:17 INFO Received complete push request method=POST path=/33492026/11/12 07:16:17 INFO Completed push id=17 objects=6 duration=532.167µs33502026/11/12 07:16:17 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."33512026/11/12 07:16:17 INFO Received complete push request method=POST path=/33522026/11/12 07:16:17 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."33532026/11/12 07:16:17 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."33542026/11/12 07:16:17 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."33552026/11/12 07:16:17 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."33562026/11/12 07:16:17 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."33572026/11/12 07:16:17 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."33582026/11/12 07:16:17 INFO Received push request method=POST path=/33592026/11/12 07:16:17 INFO Received complete push request method=POST path=/33602026/11/12 07:16:17 INFO Completed push id=12 objects=3 duration=663.25µs33612026/11/12 07:16:17 INFO Vacuumed table table=pending_closures33622026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=033632026/11/12 07:16:17 INFO Received complete upload request method=POST path=/33642026/11/12 07:16:17 INFO Completed push id=3 objects=21 duration=6.818708ms33652026/11/12 07:16:17 INFO Received uploads request method=POST path=/33662026/11/12 07:16:17 INFO Received complete push request method=POST path=/33672026/11/12 07:16:17 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033682026/11/12 07:16:17 INFO Completed upload id=21 objects=3 duration=277.292µs33692026/11/12 07:16:17 INFO Received uploads request method=POST path=/33702026/11/12 07:16:17 INFO Received uploads request method=POST path=/33712026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts33722026/11/12 07:16:17 INFO Completed push id=23 objects=28 duration=963.334µs33732026/11/12 07:16:17 INFO Vacuumed table table=pending_closures33742026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=033752026/11/12 07:16:17 INFO Received push request method=POST path=/33762026/11/12 07:16:17 INFO Received complete upload request method=POST path=/33772026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts33782026/11/12 07:16:17 WARN Force mode enabled - objects will be deleted immediately without grace period33792026/11/12 07:16:17 INFO Vacuumed table table=pending_objects33802026/11/12 07:16:17 INFO Completed upload id=7 objects=10 duration=811.042µs33812026/11/12 07:16:17 INFO Received complete upload request method=POST path=/33822026/11/12 07:16:17 INFO Received uploads request method=POST path=/33832026/11/12 07:16:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=7 objects-failed-to-delete=033842026/11/12 07:16:17 INFO Vacuumed table table=multipart_uploads33852026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=033862026/11/12 07:16:17 INFO Completed upload id=27 objects=8 duration=446.708µs33872026/11/12 07:16:17 INFO Vacuumed table table=closures33882026/11/12 07:16:17 WARN Force mode enabled - objects will be deleted immediately without grace period33892026/11/12 07:16:17 INFO Received uploads request method=POST path=/33902026/11/12 07:16:17 INFO Received push request method=POST path=/33912026/11/12 07:16:17 INFO Vacuumed table table=objects33922026/11/12 07:16:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033932026/11/12 07:16:17 INFO Vacuumed table table=pending_objects33942026/11/12 07:16:17 INFO Received complete push request method=POST path=/33952026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=033962026/11/12 07:16:17 INFO Received complete upload request method=POST path=/33972026/11/12 07:16:17 INFO Vacuumed table table=pending_closures33982026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts33992026/11/12 07:16:17 INFO Vacuumed table table=multipart_uploads34002026/11/12 07:16:17 INFO Completed upload id=10 objects=23 duration=715.375µs34012026/11/12 07:16:17 INFO Vacuumed table table=pending_closures34022026/11/12 07:16:17 INFO Completed push id=15 objects=5 duration=851.75µs34032026/11/12 07:16:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034042026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts34052026/11/12 07:16:17 INFO Received uploads request method=POST path=/34062026/11/12 07:16:17 INFO Vacuumed table table=pending_objects34072026/11/12 07:16:17 INFO Vacuumed table table=pending_closures34082026/11/12 07:16:17 INFO Vacuumed table table=multipart_uploads34092026/11/12 07:16:17 INFO Received complete upload request method=POST path=/34102026/11/12 07:16:17 INFO Received push request method=POST path=/34112026/11/12 07:16:17 INFO Received uploads request method=POST path=/34122026/11/12 07:16:17 INFO Vacuumed table table=closures34132026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts34142026/11/12 07:16:17 INFO Vacuumed table table=pending_objects34152026/11/12 07:16:17 INFO Completed upload id=16 objects=23 duration=598.333µs34162026/11/12 07:16:17 INFO Vacuumed table table=closures34172026/11/12 07:16:17 INFO Vacuumed table table=objects34182026/11/12 07:16:17 INFO Vacuumed table table=multipart_uploads34192026/11/12 07:16:17 INFO Received push request method=POST path=/34202026/11/12 07:16:17 INFO Received complete upload request method=POST path=/34212026/11/12 07:16:17 INFO Received push request method=POST path=/34222026/11/12 07:16:17 INFO Vacuumed table table=pending_objects34232026/11/12 07:16:17 INFO Completed upload id=15 objects=6 duration=393.292µs34242026/11/12 07:16:17 INFO Vacuumed table table=closures34252026/11/12 07:16:17 INFO Vacuumed table table=multipart_uploads34262026/11/12 07:16:17 INFO Received uploads request method=POST path=/34272026/11/12 07:16:17 INFO Received push request method=POST path=/34282026/11/12 07:16:17 INFO Vacuumed table table=closures34292026/11/12 07:16:17 INFO Vacuumed table table=objects34302026/11/12 07:16:17 INFO Received uploads request method=POST path=/34312026/11/12 07:16:17 INFO Received uploads request method=POST path=/34322026/11/12 07:16:17 INFO Vacuumed table table=objects34332026/11/12 07:16:17 INFO Vacuumed table table=objects34342026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=034352026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=034362026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=034372026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=034382026/11/12 07:16:17 WARN Force mode enabled - objects will be deleted immediately without grace period34392026/11/12 07:16:17 WARN Force mode enabled - objects will be deleted immediately without grace period34402026/11/12 07:16:17 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034412026/11/12 07:16:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034422026/11/12 07:16:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034432026/11/12 07:16:17 INFO Received complete push request method=POST path=/34442026/11/12 07:16:17 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034452026/11/12 07:16:17 INFO Vacuumed table table=pending_closures34462026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts34472026/11/12 07:16:17 INFO Vacuumed table table=pending_closures34482026/11/12 07:16:17 INFO Vacuumed table table=pending_closures34492026/11/12 07:16:17 INFO Received complete upload request method=POST path=/34502026/11/12 07:16:17 INFO Vacuumed table table=pending_closures34512026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts34522026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts34532026/11/12 07:16:17 INFO Received push request method=POST path=/34542026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts34552026/11/12 07:16:17 INFO Completed upload id=11 objects=6 duration=384.333µs34562026/11/12 07:16:17 INFO Vacuumed table table=pending_objects34572026/11/12 07:16:17 INFO Vacuumed table table=pending_objects34582026/11/12 07:16:17 INFO Vacuumed table table=pending_objects34592026/11/12 07:16:17 INFO Vacuumed table table=pending_objects34602026/11/12 07:16:17 INFO Received uploads request method=POST path=/34612026/11/12 07:16:17 INFO Received push request method=POST path=/34622026/11/12 07:16:17 INFO Vacuumed table table=multipart_uploads34632026/11/12 07:16:17 INFO Vacuumed table table=multipart_uploads34642026/11/12 07:16:17 INFO Vacuumed table table=multipart_uploads34652026/11/12 07:16:17 INFO Completed push id=1 objects=7 duration=41.166542ms34662026/11/12 07:16:17 INFO Vacuumed table table=multipart_uploads34672026/11/12 07:16:17 INFO Received complete upload request method=POST path=/34682026/11/12 07:16:17 INFO Vacuumed table table=closures34692026/11/12 07:16:17 INFO Vacuumed table table=closures34702026/11/12 07:16:17 INFO Vacuumed table table=closures34712026/11/12 07:16:17 INFO Completed upload id=17 objects=7 duration=369.334µs34722026/11/12 07:16:17 INFO Vacuumed table table=objects34732026/11/12 07:16:17 INFO Vacuumed table table=objects34742026/11/12 07:16:17 INFO Vacuumed table table=objects34752026/11/12 07:16:17 INFO Vacuumed table table=closures34762026/11/12 07:16:17 INFO Vacuumed table table=objects34772026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=034782026/11/12 07:16:17 INFO Received uploads request method=POST path=/34792026/11/12 07:16:17 INFO Received push request method=POST path=/34802026/11/12 07:16:17 INFO Received push request method=POST path=/34812026/11/12 07:16:17 INFO Received complete upload request method=POST path=/34822026/11/12 07:16:17 INFO Completed upload id=25 objects=16 duration=899.041µs34832026/11/12 07:16:17 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."34842026/11/12 07:16:17 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."34852026/11/12 07:16:17 INFO Received uploads request method=POST path=/34862026/11/12 07:16:17 INFO Received push request method=POST path=/34872026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=034882026/11/12 07:16:17 INFO Received uploads request method=POST path=/34892026/11/12 07:16:17 INFO Received complete push request method=POST path=/34902026/11/12 07:16:17 INFO Received push request method=POST path=/34912026/11/12 07:16:17 WARN Force mode enabled - objects will be deleted immediately without grace period34922026/11/12 07:16:17 INFO Received push request method=POST path=/34932026/11/12 07:16:17 INFO Received complete push request method=POST path=/34942026/11/12 07:16:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034952026/11/12 07:16:17 INFO Completed push id=22 objects=3 duration=1.055042ms34962026/11/12 07:16:17 INFO Received push request method=POST path=/34972026/11/12 07:16:17 INFO Completed push id=8 objects=13 duration=511.083µs34982026/11/12 07:16:17 INFO Received complete push request method=POST path=/34992026/11/12 07:16:17 INFO Vacuumed table table=pending_closures35002026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts35012026/11/12 07:16:17 INFO Completed push id=26 objects=14 duration=1.075209ms35022026/11/12 07:16:17 INFO Received push request method=POST path=/35032026/11/12 07:16:17 INFO Vacuumed table table=pending_objects35042026/11/12 07:16:17 INFO Received uploads request method=POST path=/35052026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=035062026/11/12 07:16:17 INFO Vacuumed table table=multipart_uploads35072026/11/12 07:16:17 INFO Received push request method=POST path=/35082026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=035092026/11/12 07:16:17 INFO Received push request method=POST path=/35102026/11/12 07:16:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035112026/11/12 07:16:17 INFO Vacuumed table table=closures35122026/11/12 07:16:17 WARN Force mode enabled - objects will be deleted immediately without grace period35132026/11/12 07:16:17 INFO Received complete push request method=POST path=/35142026/11/12 07:16:17 INFO Received uploads request method=POST path=/35152026/11/12 07:16:17 INFO Completed push id=24 objects=3 duration=390.459µs35162026/11/12 07:16:17 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035172026/11/12 07:16:17 INFO Vacuumed table table=objects35182026/11/12 07:16:17 INFO Received complete upload request method=POST path=/35192026/11/12 07:16:17 INFO Received push request method=POST path=/35202026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=035212026/11/12 07:16:17 INFO Completed upload id=14 objects=6 duration=320.542µs35222026/11/12 07:16:17 INFO Received push request method=POST path=/35232026/11/12 07:16:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035242026/11/12 07:16:17 INFO Vacuumed table table=pending_closures35252026/11/12 07:16:17 INFO Vacuumed table table=pending_closures35262026/11/12 07:16:17 INFO Received push request method=POST path=/35272026/11/12 07:16:17 INFO Received push request method=POST path=/35282026/11/12 07:16:17 INFO Aborted multipart uploads count=0 kept=035292026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts35302026/11/12 07:16:17 INFO Received complete push request method=POST path=/35312026/11/12 07:16:17 WARN Force mode enabled - objects will be deleted immediately without grace period35322026/11/12 07:16:17 INFO Received complete push request method=POST path=/35332026/11/12 07:16:17 INFO Vacuumed table table=pending_objects35342026/11/12 07:16:17 INFO Received uploads request method=POST path=/35352026/11/12 07:16:17 INFO Completed push id=12 objects=15 duration=643.792µs35362026/11/12 07:16:17 INFO Vacuumed table table=pending_closures35372026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts35382026/11/12 07:16:17 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035392026/11/12 07:16:17 INFO Completed push id=14 objects=3 duration=913.667µs35402026/11/12 07:16:17 INFO Vacuumed table table=multipart_uploads35412026/11/12 07:16:17 INFO Received push request method=POST path=/35422026/11/12 07:16:17 INFO Received complete upload request method=POST path=/35432026/11/12 07:16:17 INFO Vacuumed table table=closures35442026/11/12 07:16:17 INFO Vacuumed table table=objects35452026/11/12 07:16:17 INFO Completed upload id=22 objects=5 duration=6.110375ms35462026/11/12 07:16:17 INFO Vacuumed table table=pending_closures35472026/11/12 07:16:17 INFO Vacuumed table table=pending_objects35482026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts35492026/11/12 07:16:17 INFO Vacuumed table table=commit_receipts35502026/11/12 07:16:17 INFO Vacuumed table table=multipart_uploads35512026/11/12 07:16:17 INFO Received complete push request method=POST path=/35522026/11/12 07:16:17 INFO Received push request method=POST path=/35532026/11/12 07:16:17 INFO Completed push id=28 objects=3 duration=614.833µs35542026/11/12 07:16:17 INFO Received push request method=POST path=/35552026/11/12 07:16:17 INFO Received uploads request method=POST path=/35562026/11/12 07:16:17 INFO Received complete upload request method=POST path=/35572026/11/12 07:16:17 INFO Received complete push request method=POST path=/35582026/11/12 07:16:17 INFO Received push request method=POST path=/35592026/11/12 07:16:17 INFO Completed push id=29 objects=15 duration=353.5µs35602026/11/12 07:16:17 INFO Vacuumed table table=pending_objects35612026/11/12 07:16:17 INFO Completed upload id=4 objects=14 duration=5.444333ms35622026/11/12 07:16:18 INFO Vacuumed table table=pending_objects35632026/11/12 07:16:18 INFO Vacuumed table table=multipart_uploads35642026/11/12 07:16:18 INFO Received push request method=POST path=/35652026/11/12 07:16:18 INFO Vacuumed table table=multipart_uploads35662026/11/12 07:16:18 INFO Received complete push request method=POST path=/35672026/11/12 07:16:18 INFO Vacuumed table table=closures35682026/11/12 07:16:18 INFO Vacuumed table table=closures35692026/11/12 07:16:18 INFO Received uploads request method=POST path=/35702026/11/12 07:16:18 INFO Aborted multipart uploads count=0 kept=035712026/11/12 07:16:18 INFO Received complete upload request method=POST path=/35722026/11/12 07:16:18 INFO Received push request method=POST path=/35732026/11/12 07:16:18 INFO Received complete push request method=POST path=/35742026/11/12 07:16:18 INFO Completed upload id=7 objects=3 duration=276.875µs35752026/11/12 07:16:18 INFO Vacuumed table table=objects35762026/11/12 07:16:18 INFO Completed push id=30 objects=6 duration=534.208µs35772026/11/12 07:16:18 INFO Completed push id=21 objects=9 duration=731.375µs35782026/11/12 07:16:18 WARN Force mode enabled - objects will be deleted immediately without grace period35792026/11/12 07:16:18 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035802026/11/12 07:16:18 INFO Vacuumed table table=closures35812026/11/12 07:16:18 INFO Received complete upload request method=POST path=/35822026/11/12 07:16:18 INFO Received complete upload request method=POST path=/35832026/11/12 07:16:18 INFO Received complete upload request method=POST path=/35842026/11/12 07:16:18 INFO Received uploads request method=POST path=/35852026/11/12 07:16:18 INFO Completed upload id=6 objects=7 duration=1.083583ms35862026/11/12 07:16:18 INFO Completed upload id=31 objects=3 duration=588.625µs35872026/11/12 07:16:18 INFO Received uploads request method=POST path=/35882026/11/12 07:16:18 INFO Vacuumed table table=objects35892026/11/12 07:16:18 INFO Completed upload id=14 objects=16 duration=2.216542ms35902026/11/12 07:16:18 INFO Aborted multipart uploads count=0 kept=035912026/11/12 07:16:18 INFO Received complete upload request method=POST path=/35922026/11/12 07:16:18 INFO Aborted multipart uploads count=0 kept=035932026/11/12 07:16:18 INFO Aborted multipart uploads count=0 kept=035942026/11/12 07:16:18 INFO Received push request method=POST path=/35952026/11/12 07:16:18 INFO Vacuumed table table=pending_closures35962026/11/12 07:16:18 INFO Vacuumed table table=objects35972026/11/12 07:16:18 INFO Received push request method=POST path=/35982026/11/12 07:16:18 INFO Completed upload id=15 objects=6 duration=381.667µs35992026/11/12 07:16:18 INFO Vacuumed table table=commit_receipts36002026/11/12 07:16:18 WARN Force mode enabled - objects will be deleted immediately without grace period36012026/11/12 07:16:18 WARN Force mode enabled - objects will be deleted immediately without grace period36022026/11/12 07:16:18 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=036032026/11/12 07:16:18 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036042026/11/12 07:16:18 INFO Received complete push request method=POST path=/36052026/11/12 07:16:18 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036062026/11/12 07:16:18 INFO Received uploads request method=POST path=/36072026/11/12 07:16:18 INFO Received push request method=POST path=/36082026/11/12 07:16:18 INFO Received push request method=POST path=/36092026/11/12 07:16:18 INFO Completed push id=7 objects=15 duration=885.417µs36102026/11/12 07:16:18 INFO Vacuumed table table=pending_objects36112026/11/12 07:16:18 INFO Vacuumed table table=pending_closures36122026/11/12 07:16:18 INFO Vacuumed table table=pending_closures36132026/11/12 07:16:18 INFO Vacuumed table table=multipart_uploads36142026/11/12 07:16:18 INFO Vacuumed table table=commit_receipts36152026/11/12 07:16:18 INFO Received complete push request method=POST path=/36162026/11/12 07:16:18 INFO Received push request method=POST path=/36172026/11/12 07:16:18 INFO Vacuumed table table=commit_receipts36182026/11/12 07:16:18 INFO Vacuumed table table=closures36192026/11/12 07:16:18 INFO Completed push id=27 objects=25 duration=1.112834ms36202026/11/12 07:16:18 INFO Received complete upload request method=POST path=/36212026/11/12 07:16:18 INFO Received push request method=POST path=/36222026/11/12 07:16:18 INFO Vacuumed table table=objects36232026/11/12 07:16:18 INFO Vacuumed table table=pending_closures36242026/11/12 07:16:18 INFO Aborted multipart uploads count=0 kept=036252026/11/12 07:16:18 INFO Completed upload id=13 objects=8 duration=349.666µs36262026/11/12 07:16:18 INFO Vacuumed table table=pending_objects36272026/11/12 07:16:18 INFO Received complete push request method=POST path=/36282026/11/12 07:16:18 INFO Vacuumed table table=commit_receipts36292026/11/12 07:16:18 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036302026/11/12 07:16:18 INFO Vacuumed table table=pending_objects36312026/11/12 07:16:18 INFO Vacuumed table table=multipart_uploads36322026/11/12 07:16:18 INFO Completed push id=6 objects=5 duration=2.647917ms36332026/11/12 07:16:18 INFO Received uploads request method=POST path=/36342026/11/12 07:16:18 INFO Vacuumed table table=pending_closures36352026/11/12 07:16:18 INFO Vacuumed table table=multipart_uploads36362026/11/12 07:16:18 INFO Received push request method=POST path=/36372026/11/12 07:16:18 INFO Vacuumed table table=closures36382026/11/12 07:16:18 INFO Received push request method=POST path=/36392026/11/12 07:16:18 INFO Received uploads request method=POST path=/36402026/11/12 07:16:18 INFO Received complete push request method=POST path=/36412026/11/12 07:16:18 INFO Vacuumed table table=commit_receipts36422026/11/12 07:16:18 INFO Vacuumed table table=closures36432026/11/12 07:16:18 INFO Vacuumed table table=objects36442026/11/12 07:16:18 INFO Completed push id=18 objects=23 duration=485.375µs36452026/11/12 07:16:18 INFO Received push request method=POST path=/36462026/11/12 07:16:18 INFO Vacuumed table table=pending_objects36472026/11/12 07:16:18 INFO Vacuumed table table=objects36482026/11/12 07:16:18 INFO Received complete upload request method=POST path=/36492026/11/12 07:16:18 INFO Vacuumed table table=pending_objects36502026/11/12 07:16:18 INFO Vacuumed table table=multipart_uploads36512026/11/12 07:16:18 INFO Completed upload id=11 objects=7 duration=502.083µs36522026/11/12 07:16:18 INFO Aborted multipart uploads count=0 kept=036532026/11/12 07:16:18 INFO Aborted multipart uploads count=0 kept=036542026/11/12 07:16:18 INFO Received uploads request method=POST path=/36552026/11/12 07:16:18 INFO Vacuumed table table=multipart_uploads36562026/11/12 07:16:18 INFO Vacuumed table table=closures36572026/11/12 07:16:18 INFO Received uploads request method=POST path=/36582026/11/12 07:16:18 WARN Force mode enabled - objects will be deleted immediately without grace period36592026/11/12 07:16:18 INFO Received uploads request method=POST path=/36602026/11/12 07:16:18 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036612026/11/12 07:16:18 INFO Vacuumed table table=closures36622026/11/12 07:16:18 INFO Vacuumed table table=objects36632026/11/12 07:16:18 INFO Received uploads request method=POST path=/36642026/11/12 07:16:18 INFO Vacuumed table table=objects36652026/11/12 07:16:18 INFO Received complete upload request method=POST path=/36662026/11/12 07:16:18 INFO Received complete upload request method=POST path=/36672026/11/12 07:16:18 INFO Completed upload id=26 objects=2 duration=706.541µs36682026/11/12 07:16:18 INFO Completed upload id=10 objects=4 duration=915.833µs36692026/11/12 07:16:18 INFO Received complete push request method=POST path=/36702026/11/12 07:16:18 INFO Vacuumed table table=pending_closures36712026/11/12 07:16:18 INFO Received complete upload request method=POST path=/36722026/11/12 07:16:18 INFO Completed push id=24 objects=5 duration=1.098416ms36732026/11/12 07:16:18 INFO Received push request method=POST path=/36742026/11/12 07:16:18 INFO Aborted multipart uploads count=0 kept=036752026/11/12 07:16:18 INFO Vacuumed table table=commit_receipts36762026/11/12 07:16:18 INFO Received push request method=POST path=/36772026/11/12 07:16:18 INFO Received push request method=POST path=/36782026/11/12 07:16:18 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036792026/11/12 07:16:18 INFO Received uploads request method=POST path=/36802026/11/12 07:16:18 INFO Received push request method=POST path=/36812026/11/12 07:16:18 INFO Received push request method=POST path=/36822026/11/12 07:16:18 INFO Vacuumed table table=pending_objects36832026/11/12 07:16:18 INFO Vacuumed table table=multipart_uploads36842026/11/12 07:16:18 INFO Received uploads request method=POST path=/36852026/11/12 07:16:18 INFO Vacuumed table table=closures36862026/11/12 07:16:18 INFO Vacuumed table table=objects36872026/11/12 07:16:18 INFO Completed upload id=1 objects=9 duration=61.707625ms36882026/11/12 07:16:18 INFO Received push request method=POST path=/36892026/11/12 07:16:18 INFO Received uploads request method=POST path=/36902026/11/12 07:16:18 INFO Vacuumed table table=pending_closures36912026/11/12 07:16:18 INFO Vacuumed table table=commit_receipts36922026/11/12 07:16:18 INFO Received complete push request method=POST path=/36932026/11/12 07:16:18 INFO Received complete push request method=POST path=/36942026/11/12 07:16:18 INFO Completed push id=11 objects=9 duration=500.083µs36952026/11/12 07:16:18 INFO Completed push id=17 objects=11 duration=848.292µs36962026/11/12 07:16:18 INFO Received push request method=POST path=/36972026/11/12 07:16:18 INFO Received push request method=POST path=/36982026/11/12 07:16:18 INFO Received push request method=POST path=/36992026/11/12 07:16:18 INFO Received push request method=POST path=/37002026/11/12 07:16:18 INFO Vacuumed table table=pending_objects37012026/11/12 07:16:18 INFO Received uploads request method=POST path=/37022026/11/12 07:16:18 INFO Received complete push request method=POST path=/37032026/11/12 07:16:18 INFO Vacuumed table table=multipart_uploads37042026/11/12 07:16:18 INFO Received complete upload request method=POST path=/37052026/11/12 07:16:18 INFO Aborted multipart uploads count=0 kept=037062026/11/12 07:16:18 INFO Completed push id=16 objects=27 duration=804.042µs37072026/11/12 07:16:18 INFO Vacuumed table table=closures37082026/11/12 07:16:18 INFO Completed upload id=26 objects=6 duration=320.083µs37092026/11/12 07:16:18 WARN Force mode enabled - objects will be deleted immediately without grace period37102026/11/12 07:16:18 INFO Vacuumed table table=objects37112026/11/12 07:16:18 INFO Received complete push request method=POST path=/37122026/11/12 07:16:18 INFO Received uploads request method=POST path=/37132026/11/12 07:16:18 INFO Received uploads request method=POST path=/37142026/11/12 07:16:18 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037152026/11/12 07:16:18 INFO Received push request method=POST path=/37162026/11/12 07:16:18 INFO Completed push id=5 objects=8 duration=1.606459ms37172026/11/12 07:16:18 INFO Vacuumed table table=pending_closures37182026/11/12 07:16:18 INFO Received push request method=POST path=/37192026/11/12 07:16:18 INFO Received complete upload request method=POST path=/37202026/11/12 07:16:18 INFO Received push request method=POST path=/37212026/11/12 07:16:18 INFO Vacuumed table table=commit_receipts37222026/11/12 07:16:18 INFO Aborted multipart uploads count=0 kept=037232026/11/12 07:16:18 INFO Received uploads request method=POST path=/37242026/11/12 07:16:18 INFO Completed upload id=29 objects=5 duration=280.125µs37252026/11/12 07:16:18 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."37262026/11/12 07:16:18 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."37272026/11/12 07:16:18 INFO Aborted multipart uploads count=0 kept=037282026/11/12 07:16:18 INFO Vacuumed table table=pending_objects37292026/11/12 07:16:18 INFO Aborted multipart uploads count=0 kept=037302026/11/12 07:16:18 INFO Received push request method=POST path=/37312026/11/12 07:16:18 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."37322026/11/12 07:16:18 INFO Vacuumed table table=multipart_uploads37332026/11/12 07:16:18 WARN Force mode enabled - objects will be deleted immediately without grace period37342026/11/12 07:16:18 INFO Vacuumed table table=closures37352026/11/12 07:16:18 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037362026/11/12 07:16:18 INFO Received push request method=POST path=/37372026/11/12 07:16:18 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037382026/11/12 07:16:18 INFO Vacuumed table table=objects37392026/11/12 07:16:18 INFO Vacuumed table table=pending_closures37402026/11/12 07:16:18 INFO Vacuumed table table=pending_closures37412026/11/12 07:16:18 INFO Received push request method=POST path=/37422026/11/12 07:16:18 INFO Received complete upload request method=POST path=/37432026/11/12 07:16:18 INFO Vacuumed table table=commit_receipts37442026/11/12 07:16:18 INFO Received uploads request method=POST path=/37452026/11/12 07:16:18 INFO Vacuumed table table=commit_receipts37462026/11/12 07:16:18 INFO Completed upload id=11 objects=14 duration=411.375µs37472026/11/12 07:16:18 INFO Received complete push request method=POST path=/37482026/11/12 07:16:18 INFO Vacuumed table table=pending_objects37492026/11/12 07:16:18 INFO Completed push id=28 objects=11 duration=10.569583ms37502026/11/12 07:16:18 INFO Vacuumed table table=pending_objects37512026/11/12 07:16:18 INFO Vacuumed table table=multipart_uploads37522026/11/12 07:16:18 INFO Received push request method=POST path=/37532026/11/12 07:16:18 INFO Received push request method=POST path=/37542026/11/12 07:16:18 INFO Vacuumed table table=multipart_uploads37552026/11/12 07:16:18 INFO Received push request method=POST path=/37562026/11/12 07:16:18 INFO Vacuumed table table=closures37572026/11/12 07:16:18 INFO Received push request method=POST path=/37582026/11/12 07:16:18 INFO Received complete push request method=POST path=/37592026/11/12 07:16:18 INFO Vacuumed table table=closures37602026/11/12 07:16:18 INFO Received complete upload request method=POST path=/37612026/11/12 07:16:18 INFO Vacuumed table table=objects37622026/11/12 07:16:18 INFO Completed upload id=23 objects=15 duration=6.164417ms37632026/11/12 07:16:18 INFO Completed push id=21 objects=26 duration=6.27375ms37642026/11/12 07:16:18 INFO Vacuumed table table=objects37652026/11/12 07:16:18 INFO Received push request method=POST path=/37662026/11/12 07:16:18 INFO Received push request method=POST path=/37672026/11/12 07:16:18 INFO Aborted multipart uploads count=0 kept=037682026/11/12 07:16:18 INFO Received uploads request method=POST path=/37692026/11/12 07:16:18 INFO Received uploads request method=POST path=/37702026/11/12 07:16:18 INFO Received complete push request method=POST path=/37712026/11/12 07:16:18 WARN Force mode enabled - objects will be deleted immediately without grace period37722026/11/12 07:16:18 INFO Received uploads request method=POST path=/37732026/11/12 07:16:18 INFO Completed push id=18 objects=10 duration=547.834µs37742026/11/12 07:16:18 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037752026/11/12 07:16:18 INFO Received complete upload request method=POST path=/37762026/11/12 07:16:18 INFO Received push request method=POST path=/37772026/11/12 07:16:18 INFO Vacuumed table table=pending_closures37782026/11/12 07:16:18 INFO Aborted multipart uploads count=0 kept=037792026/11/12 07:16:18 INFO Completed upload id=31 objects=3 duration=771.709µs37802026/11/12 07:16:18 INFO Received complete upload request method=POST path=/37812026/11/12 07:16:18 INFO Vacuumed table table=commit_receipts37822026/11/12 07:16:18 WARN Force mode enabled - objects will be deleted immediately without grace period37832026/11/12 07:16:18 INFO Received complete upload request method=POST path=/37842026/11/12 07:16:18 INFO Completed upload id=8 objects=3 duration=709.875µs37852026/11/12 07:16:18 INFO Vacuumed table table=pending_objects37862026/11/12 07:16:18 INFO Received complete upload request method=POST path=/37872026/11/12 07:16:18 INFO Garbage collection completed failed-uploads-deleted=5 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037882026/11/12 07:16:18 INFO Completed upload id=22 objects=2 duration=334.542µs37892026/11/12 07:16:18 INFO Vacuumed table table=multipart_uploads37902026/11/12 07:16:18 INFO Completed upload id=19 objects=6 duration=619.125µs37912026/11/12 07:16:18 INFO Received uploads request method=POST path=/37922026/11/12 07:16:18 INFO Aborted multipart uploads count=0 kept=037932026/11/12 07:16:18 WARN Force mode enabled - objects will be deleted immediately without grace period37942026/11/12 07:16:18 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037952026/11/12 07:16:18 INFO Vacuumed table table=pending_closures37962026/11/12 07:16:18 INFO Vacuumed table table=commit_receipts37972026/11/12 07:16:18 INFO Received complete push request method=POST path=/37982026/11/12 07:16:18 INFO Received complete upload request method=POST path=/37992026/11/12 07:16:18 INFO Vacuumed table table=closures38002026/11/12 07:16:18 INFO Vacuumed table table=pending_objects38012026/11/12 07:16:18 INFO Completed push id=6 objects=6 duration=621.25µs38022026/11/12 07:16:18 INFO Completed upload id=28 objects=17 duration=340µs38032026/11/12 07:16:18 INFO Vacuumed table table=objects38042026/11/12 07:16:18 INFO Vacuumed table table=multipart_uploads38052026/11/12 07:16:18 INFO Received complete push request method=POST path=/38062026/11/12 07:16:18 INFO Received push request method=POST path=/38072026/11/12 07:16:18 INFO Received complete push request method=POST path=/38082026/11/12 07:16:18 INFO Vacuumed table table=pending_closures38092026/11/12 07:16:18 INFO Vacuumed table table=closures38102026/11/12 07:16:18 INFO Received uploads request method=POST path=/38112026/11/12 07:16:18 INFO Completed push id=5 objects=28 duration=489.292µs38122026/11/12 07:16:18 INFO Completed push id=29 objects=2 duration=358.666µs38132026/11/12 07:16:18 INFO Vacuumed table table=objects38142026/11/12 07:16:18 INFO Received complete push request method=POST path=/38152026/11/12 07:16:18 INFO Received complete push request method=POST path=/38162026/11/12 07:16:18 INFO Received complete push request method=POST path=/38172026/11/12 07:16:18 INFO Completed push id=31 objects=6 duration=410.5µs38182026/11/12 07:16:18 INFO Completed push id=10 objects=2 duration=499.75µs38192026/11/12 07:16:18 INFO Completed push id=14 objects=3 duration=1.366291ms38202026/11/12 07:16:18 INFO Received complete upload request method=POST path=/38212026/11/12 07:16:18 INFO Received complete push request method=POST path=/38222026/11/12 07:16:18 INFO Vacuumed table table=commit_receipts38232026/11/12 07:16:18 INFO Completed push id=32 objects=9 duration=345.875µs38242026/11/12 07:16:18 INFO Completed upload id=18 objects=2 duration=6.791458ms38252026/11/12 07:16:18 INFO Received complete push request method=POST path=/38262026/11/12 07:16:18 INFO Received complete upload request method=POST path=/38272026/11/12 07:16:18 INFO Received push request method=POST path=/38282026/11/12 07:16:18 INFO Completed upload id=22 objects=7 duration=326.917µs38292026/11/12 07:16:18 INFO Completed push id=5 objects=6 duration=5.935625ms38302026/11/12 07:16:19 INFO Aborted multipart uploads count=0 kept=038312026/11/12 07:16:19 INFO Vacuumed table table=pending_objects38322026/11/12 07:16:19 INFO Vacuumed table table=multipart_uploads38332026/11/12 07:16:19 WARN Force mode enabled - objects will be deleted immediately without grace period38342026/11/12 07:16:19 INFO Received complete push request method=POST path=/38352026/11/12 07:16:19 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038362026/11/12 07:16:19 INFO Received complete push request method=POST path=/38372026/11/12 07:16:19 INFO Vacuumed table table=pending_closures38382026/11/12 07:16:19 INFO Completed push id=24 objects=9 duration=718.125µs38392026/11/12 07:16:19 INFO Completed push id=19 objects=14 duration=472.625µs38402026/11/12 07:16:19 INFO Vacuumed table table=commit_receipts38412026/11/12 07:16:19 INFO Vacuumed table table=pending_objects38422026/11/12 07:16:19 INFO Vacuumed table table=multipart_uploads38432026/11/12 07:16:19 INFO Received push request method=POST path=/38442026/11/12 07:16:19 INFO Aborted multipart uploads count=0 kept=038452026/11/12 07:16:19 INFO Received complete upload request method=POST path=/38462026/11/12 07:16:19 INFO Vacuumed table table=closures38472026/11/12 07:16:19 WARN Force mode enabled - objects will be deleted immediately without grace period38482026/11/12 07:16:19 INFO Received push request method=POST path=/38492026/11/12 07:16:19 INFO Vacuumed table table=closures38502026/11/12 07:16:19 INFO Completed upload id=17 objects=5 duration=276.833µs38512026/11/12 07:16:19 INFO Received complete upload request method=POST path=/38522026/11/12 07:16:19 INFO Vacuumed table table=objects38532026/11/12 07:16:19 INFO Completed upload id=33 objects=9 duration=537.208µs38542026/11/12 07:16:19 INFO Received complete push request method=POST path=/38552026/11/12 07:16:19 INFO Received uploads request method=POST path=/38562026/11/12 07:16:19 INFO Aborted multipart uploads count=0 kept=038572026/11/12 07:16:19 INFO Completed push id=10 objects=15 duration=570.25µs38582026/11/12 07:16:19 INFO Received complete push request method=POST path=/38592026/11/12 07:16:19 WARN Force mode enabled - objects will be deleted immediately without grace period38602026/11/12 07:16:19 INFO Completed push id=29 objects=24 duration=486.542µs38612026/11/12 07:16:19 INFO Received complete push request method=POST path=/38622026/11/12 07:16:19 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038632026/11/12 07:16:19 INFO Vacuumed table table=objects38642026/11/12 07:16:19 INFO Completed push id=21 objects=3 duration=10.046417ms38652026/11/12 07:16:19 INFO Aborted multipart uploads count=0 kept=038662026/11/12 07:16:19 INFO Aborted multipart uploads count=0 kept=038672026/11/12 07:16:19 WARN Force mode enabled - objects will be deleted immediately without grace period38682026/11/12 07:16:19 INFO Received uploads request method=POST path=/38692026/11/12 07:16:19 WARN Force mode enabled - objects will be deleted immediately without grace period38702026/11/12 07:16:19 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=038712026/11/12 07:16:19 INFO Vacuumed table table=pending_closures38722026/11/12 07:16:19 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=038732026/11/12 07:16:19 INFO Aborted multipart uploads count=0 kept=038742026/11/12 07:16:19 INFO Vacuumed table table=commit_receipts38752026/11/12 07:16:19 INFO Received push request method=POST path=/38762026/11/12 07:16:19 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=3 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038772026/11/12 07:16:19 INFO Vacuumed table table=pending_closures38782026/11/12 07:16:19 INFO Vacuumed table table=commit_receipts38792026/11/12 07:16:19 INFO Received push request method=POST path=/38802026/11/12 07:16:19 INFO Vacuumed table table=pending_objects38812026/11/12 07:16:19 INFO Vacuumed table table=pending_objects38822026/11/12 07:16:19 INFO Vacuumed table table=pending_closures38832026/11/12 07:16:19 INFO Vacuumed table table=multipart_uploads38842026/11/12 07:16:19 INFO Vacuumed table table=pending_closures38852026/11/12 07:16:19 INFO Received push request method=POST path=/38862026/11/12 07:16:19 INFO Vacuumed table table=multipart_uploads38872026/11/12 07:16:19 INFO Vacuumed table table=commit_receipts38882026/11/12 07:16:19 INFO Vacuumed table table=closures38892026/11/12 07:16:19 INFO Vacuumed table table=commit_receipts38902026/11/12 07:16:19 INFO Vacuumed table table=closures38912026/11/12 07:16:19 INFO Vacuumed table table=objects38922026/11/12 07:16:19 INFO Aborted multipart uploads count=0 kept=038932026/11/12 07:16:19 INFO Received complete upload request method=POST path=/38942026/11/12 07:16:19 INFO Vacuumed table table=objects38952026/11/12 07:16:19 INFO Completed upload id=13 objects=7 duration=10.075625ms38962026/11/12 07:16:19 INFO Vacuumed table table=pending_objects38972026/11/12 07:16:19 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038982026/11/12 07:16:19 INFO Received complete upload request method=POST path=/38992026/11/12 07:16:19 INFO Received push request method=POST path=/39002026/11/12 07:16:19 INFO Completed upload id=20 objects=4 duration=310.708µs39012026/11/12 07:16:19 INFO Vacuumed table table=multipart_uploads39022026/11/12 07:16:19 INFO Received push request method=POST path=/39032026/11/12 07:16:19 INFO Received complete push request method=POST path=/39042026/11/12 07:16:19 INFO Vacuumed table table=closures39052026/11/12 07:16:19 INFO Vacuumed table table=pending_objects39062026/11/12 07:16:19 INFO Received complete upload request method=POST path=/39072026/11/12 07:16:19 INFO Received complete push request method=POST path=/39082026/11/12 07:16:19 INFO Vacuumed table table=pending_closures39092026/11/12 07:16:19 INFO Completed push id=19 objects=3 duration=505.5µs39102026/11/12 07:16:19 INFO Vacuumed table table=objects39112026/11/12 07:16:19 INFO Vacuumed table table=multipart_uploads39122026/11/12 07:16:19 INFO Completed upload id=23 objects=7 duration=423.75µs39132026/11/12 07:16:19 INFO Completed push id=30 objects=4 duration=984.375µs39142026/11/12 07:16:19 INFO Vacuumed table table=commit_receipts39152026/11/12 07:16:19 INFO Vacuumed table table=closures39162026/11/12 07:16:19 INFO Vacuumed table table=objects39172026/11/12 07:16:19 INFO Received push request method=POST path=/39182026/11/12 07:16:19 INFO Received uploads request method=POST path=/39192026/11/12 07:16:19 INFO Received push request method=POST path=/39202026/11/12 07:16:19 INFO Received push request method=POST path=/39212026/11/12 07:16:19 INFO Received uploads request method=POST path=/39222026/11/12 07:16:19 INFO Received complete push request method=POST path=/39232026/11/12 07:16:19 INFO Completed push id=32 objects=7 duration=407.25µs39242026/11/12 07:16:19 INFO Aborted multipart uploads count=0 kept=039252026/11/12 07:16:19 INFO Vacuumed table table=pending_objects39262026/11/12 07:16:19 INFO Vacuumed table table=multipart_uploads39272026/11/12 07:16:19 WARN Force mode enabled - objects will be deleted immediately without grace period39282026/11/12 07:16:19 INFO Received complete push request method=POST path=/39292026/11/12 07:16:19 INFO Received complete push request method=POST path=/39302026/11/12 07:16:19 INFO Vacuumed table table=closures39312026/11/12 07:16:19 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."39322026/11/12 07:16:19 INFO Completed push id=26 objects=4 duration=452µs39332026/11/12 07:16:19 INFO Vacuumed table table=objects39342026/11/12 07:16:19 INFO Received complete upload request method=POST path=/39352026/11/12 07:16:19 INFO Completed upload id=28 objects=13 duration=516.958µs39362026/11/12 07:16:19 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."39372026/11/12 07:16:19 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."39382026/11/12 07:16:19 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."39392026/11/12 07:16:19 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."39402026/11/12 07:16:19 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."39412026/11/12 07:16:19 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."39422026/11/12 07:16:19 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."39432026/11/12 07:16:19 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."39442026/11/12 07:16:19 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."39452026/11/12 07:16:19 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."39462026/11/12 07:16:19 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."39472026/11/12 07:16:19 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."39482026/11/12 07:16:19 INFO Received complete push request method=POST path=/39492026/11/12 07:16:19 INFO Completed push id=29 objects=3 duration=434.041µs39502026/11/12 07:16:19 INFO Received uploads request method=POST path=/39512026/11/12 07:16:19 INFO Received uploads request method=POST path=/39522026/11/12 07:16:19 INFO Received complete upload request method=POST path=/39532026/11/12 07:16:19 INFO Received uploads request method=POST path=/39542026/11/12 07:16:19 INFO Completed push id=1 objects=6 duration=57.290584ms39552026/11/12 07:16:19 INFO Completed upload id=31 objects=2 duration=268.292µs39562026/11/12 07:16:19 INFO Received push request method=POST path=/39572026/11/12 07:16:19 INFO Received complete push request method=POST path=/39582026/11/12 07:16:19 INFO Received uploads request method=POST path=/39592026/11/12 07:16:19 INFO Received uploads request method=POST path=/39602026/11/12 07:16:19 INFO Completed push id=33 objects=6 duration=307.042µs39612026/11/12 07:16:19 INFO Received push request method=POST path=/39622026/11/12 07:16:19 INFO Received complete upload request method=POST path=/39632026/11/12 07:16:19 INFO Aborted multipart uploads count=0 kept=039642026/11/12 07:16:19 INFO Received push request method=POST path=/39652026/11/12 07:16:19 INFO Completed upload id=23 objects=18 duration=704µs39662026/11/12 07:16:19 WARN Force mode enabled - objects will be deleted immediately without grace period39672026/11/12 07:16:19 INFO Received complete upload request method=POST path=/39682026/11/12 07:16:19 INFO Aborted multipart uploads count=0 kept=039692026/11/12 07:16:19 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039702026/11/12 07:16:19 INFO Completed upload id=14 objects=8 duration=11.024625ms39712026/11/12 07:16:19 INFO Received uploads request method=POST path=/39722026/11/12 07:16:19 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=039732026/11/12 07:16:19 INFO Received complete upload request method=POST path=/39742026/11/12 07:16:19 INFO Received push request method=POST path=/39752026/11/12 07:16:19 INFO Received uploads request method=POST path=/39762026/11/12 07:16:19 INFO Received complete push request method=POST path=/39772026/11/12 07:16:19 INFO Vacuumed table table=pending_closures39782026/11/12 07:16:19 INFO Vacuumed table table=pending_closures39792026/11/12 07:16:19 INFO Completed upload id=3 objects=24 duration=471.667µs39802026/11/12 07:16:19 INFO Completed push id=20 objects=4 duration=490µs39812026/11/12 07:16:19 INFO Received complete upload request method=POST path=/39822026/11/12 07:16:19 INFO Vacuumed table table=commit_receipts39832026/11/12 07:16:19 INFO Aborted multipart uploads count=0 kept=039842026/11/12 07:16:19 INFO Vacuumed table table=commit_receipts39852026/11/12 07:16:19 INFO Completed upload id=22 objects=22 duration=776.208µs39862026/11/12 07:16:19 WARN Force mode enabled - objects will be deleted immediately without grace period39872026/11/12 07:16:19 INFO Vacuumed table table=pending_objects39882026/11/12 07:16:19 INFO Vacuumed table table=multipart_uploads39892026/11/12 07:16:19 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=039902026/11/12 07:16:19 INFO Received push request method=POST path=/39912026/11/12 07:16:19 INFO Vacuumed table table=pending_objects39922026/11/12 07:16:19 INFO Vacuumed table table=multipart_uploads39932026/11/12 07:16:19 INFO Received complete push request method=POST path=/39942026/11/12 07:16:19 INFO Vacuumed table table=closures39952026/11/12 07:16:19 INFO Completed push id=26 objects=5 duration=4.636583ms39962026/11/12 07:16:19 INFO Vacuumed table table=closures39972026/11/12 07:16:19 INFO Vacuumed table table=objects39982026/11/12 07:16:19 INFO Vacuumed table table=pending_closures39992026/11/12 07:16:19 INFO Received complete upload request method=POST path=/40002026/11/12 07:16:19 INFO Vacuumed table table=commit_receipts40012026/11/12 07:16:19 INFO Completed upload id=24 objects=14 duration=6.383125ms40022026/11/12 07:16:19 INFO Vacuumed table table=objects40032026/11/12 07:16:19 INFO Received complete push request method=POST path=/40042026/11/12 07:16:19 INFO Received push request method=POST path=/40052026/11/12 07:16:19 INFO Completed push id=19 objects=3 duration=520.791µs40062026/11/12 07:16:19 INFO Vacuumed table table=pending_objects40072026/11/12 07:16:19 INFO Received complete upload request method=POST path=/40082026/11/12 07:16:19 INFO Vacuumed table table=multipart_uploads40092026/11/12 07:16:19 INFO Received uploads request method=POST path=/40102026/11/12 07:16:19 INFO Vacuumed table table=closures40112026/11/12 07:16:19 INFO Vacuumed table table=objects40122026/11/12 07:16:19 INFO Completed upload id=10 objects=2 duration=18.670416ms40132026/11/12 07:16:19 INFO Received uploads request method=POST path=/40142026/11/12 07:16:19 INFO Received push request method=POST path=/40152026/11/12 07:16:19 INFO Aborted multipart uploads count=0 kept=040162026/11/12 07:16:19 INFO Received complete upload request method=POST path=/40172026/11/12 07:16:19 INFO Completed upload id=25 objects=4 duration=342.792µs40182026/11/12 07:16:19 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=040192026/11/12 07:16:19 INFO Received uploads request method=POST path=/40202026/11/12 07:16:19 INFO Received push request method=POST path=/40212026/11/12 07:16:19 INFO Vacuumed table table=pending_closures40222026/11/12 07:16:19 INFO Vacuumed table table=commit_receipts40232026/11/12 07:16:19 INFO Vacuumed table table=pending_objects40242026/11/12 07:16:19 INFO Received complete push request method=POST path=/40252026/11/12 07:16:19 INFO Vacuumed table table=multipart_uploads40262026/11/12 07:16:19 INFO Completed push id=20 objects=8 duration=407.125µs40272026/11/12 07:16:19 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=040282026/11/12 07:16:19 INFO Received uploads request method=POST path=/40292026/11/12 07:16:19 INFO Vacuumed table table=pending_closures40302026/11/12 07:16:19 INFO Vacuumed table table=closures40312026/11/12 07:16:19 INFO Received push request method=POST path=/40322026/11/12 07:16:19 INFO Vacuumed table table=commit_receipts40332026/11/12 07:16:19 INFO Vacuumed table table=objects40342026/11/12 07:16:19 INFO Received complete push request method=POST path=/40352026/11/12 07:16:19 INFO Received complete upload request method=POST path=/40362026/11/12 07:16:19 INFO Completed push id=27 objects=11 duration=559.334µs40372026/11/12 07:16:19 INFO Vacuumed table table=pending_objects40382026/11/12 07:16:19 INFO Received push request method=POST path=/40392026/11/12 07:16:19 INFO Completed upload id=22 objects=8 duration=324.792µs40402026/11/12 07:16:19 INFO Vacuumed table table=multipart_uploads40412026/11/12 07:16:19 INFO Received complete upload request method=POST path=/40422026/11/12 07:16:19 INFO Vacuumed table table=closures40432026/11/12 07:16:19 INFO Received complete push request method=POST path=/40442026/11/12 07:16:19 INFO Completed upload id=29 objects=26 duration=249.583µs40452026/11/12 07:16:19 INFO Vacuumed table table=objects40462026/11/12 07:16:19 INFO Completed push id=23 objects=5 duration=351.334µs40472026/11/12 07:16:20 INFO Received complete push request method=POST path=/40482026/11/12 07:16:20 INFO Completed push id=30 objects=8 duration=853.583µs40492026/11/12 07:16:20 INFO Received push request method=POST path=/40502026/11/12 07:16:20 INFO Received complete upload request method=POST path=/40512026/11/12 07:16:20 INFO Received complete push request method=POST path=/40522026/11/12 07:16:20 INFO Received complete upload request method=POST path=/40532026/11/12 07:16:20 INFO Received uploads request method=POST path=/40542026/11/12 07:16:20 INFO Completed upload id=24 objects=8 duration=315.916µs40552026/11/12 07:16:20 INFO Completed upload id=31 objects=5 duration=323.708µs40562026/11/12 07:16:20 INFO Completed push id=2 objects=4 duration=719.042µs40572026/11/12 07:16:20 INFO Aborted multipart uploads count=0 kept=040582026/11/12 07:16:20 INFO Received complete upload request method=POST path=/40592026/11/12 07:16:20 INFO Received complete push request method=POST path=/40602026/11/12 07:16:20 INFO Completed upload id=28 objects=5 duration=346.25µs40612026/11/12 07:16:20 INFO Completed push id=32 objects=3 duration=6.076792ms40622026/11/12 07:16:20 INFO Received complete upload request method=POST path=/40632026/11/12 07:16:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040642026/11/12 07:16:20 INFO Vacuumed table table=pending_closures40652026/11/12 07:16:20 INFO Completed upload id=16 objects=17 duration=1.235708ms40662026/11/12 07:16:20 INFO Vacuumed table table=commit_receipts40672026/11/12 07:16:20 INFO Aborted multipart uploads count=0 kept=040682026/11/12 07:16:20 INFO Vacuumed table table=pending_objects40692026/11/12 07:16:20 WARN Force mode enabled - objects will be deleted immediately without grace period40702026/11/12 07:16:20 INFO Aborted multipart uploads count=0 kept=040712026/11/12 07:16:20 INFO Vacuumed table table=multipart_uploads40722026/11/12 07:16:20 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040732026/11/12 07:16:20 INFO Vacuumed table table=closures40742026/11/12 07:16:20 WARN Force mode enabled - objects will be deleted immediately without grace period40752026/11/12 07:16:20 INFO Vacuumed table table=objects40762026/11/12 07:16:20 INFO Received complete upload request method=POST path=/40772026/11/12 07:16:20 INFO Completed upload id=24 objects=18 duration=638.625µs40782026/11/12 07:16:20 INFO Vacuumed table table=pending_closures40792026/11/12 07:16:20 INFO Received complete push request method=POST path=/40802026/11/12 07:16:20 INFO Vacuumed table table=commit_receipts40812026/11/12 07:16:20 INFO Completed push id=25 objects=2 duration=5.942875ms40822026/11/12 07:16:20 INFO Received complete push request method=POST path=/40832026/11/12 07:16:20 INFO Completed push id=2 objects=25 duration=885.917µs40842026/11/12 07:16:20 INFO Received complete push request method=POST path=/40852026/11/12 07:16:20 INFO Completed push id=27 objects=4 duration=462.833µs40862026/11/12 07:16:20 INFO Vacuumed table table=pending_objects40872026/11/12 07:16:20 INFO Received push request method=POST path=/40882026/11/12 07:16:20 INFO Vacuumed table table=multipart_uploads40892026/11/12 07:16:20 INFO Vacuumed table table=closures40902026/11/12 07:16:20 INFO Aborted multipart uploads count=0 kept=040912026/11/12 07:16:20 INFO Vacuumed table table=objects40922026/11/12 07:16:20 INFO Received complete upload request method=POST path=/40932026/11/12 07:16:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040942026/11/12 07:16:20 INFO Completed upload id=30 objects=5 duration=345.417µs40952026/11/12 07:16:20 INFO Vacuumed table table=pending_closures40962026/11/12 07:16:20 INFO Aborted multipart uploads count=0 kept=040972026/11/12 07:16:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040982026/11/12 07:16:20 INFO Vacuumed table table=commit_receipts40992026/11/12 07:16:20 INFO Received complete push request method=POST path=/41002026/11/12 07:16:20 INFO Completed push id=31 objects=3 duration=531.833µs41012026/11/12 07:16:20 INFO Vacuumed table table=pending_objects41022026/11/12 07:16:20 INFO Vacuumed table table=multipart_uploads41032026/11/12 07:16:20 INFO Vacuumed table table=pending_closures41042026/11/12 07:16:20 INFO Vacuumed table table=closures41052026/11/12 07:16:20 INFO Vacuumed table table=commit_receipts41062026/11/12 07:16:20 INFO Received complete push request method=POST path=/41072026/11/12 07:16:20 INFO Received complete push request method=POST path=/41082026/11/12 07:16:20 INFO Received complete push request method=POST path=/41092026/11/12 07:16:20 INFO Completed push id=13 objects=15 duration=897.292µs41102026/11/12 07:16:20 INFO Completed push id=28 objects=26 duration=1.056959ms41112026/11/12 07:16:20 INFO Completed push id=11 objects=8 duration=11.980792ms41122026/11/12 07:16:20 INFO Aborted multipart uploads count=0 kept=041132026/11/12 07:16:20 INFO Received complete push request method=POST path=/41142026/11/12 07:16:20 INFO Completed push id=30 objects=3 duration=466.167µs41152026/11/12 07:16:20 INFO Vacuumed table table=objects41162026/11/12 07:16:20 INFO Aborted multipart uploads count=0 kept=041172026/11/12 07:16:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041182026/11/12 07:16:20 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."41192026/11/12 07:16:20 INFO Vacuumed table table=pending_closures41202026/11/12 07:16:20 INFO Received complete push request method=POST path=/41212026/11/12 07:16:20 INFO Vacuumed table table=commit_receipts41222026/11/12 07:16:20 INFO Completed push id=32 objects=19 duration=498.917µs41232026/11/12 07:16:20 INFO Vacuumed table table=pending_objects41242026/11/12 07:16:20 INFO Received push request method=POST path=/41252026/11/12 07:16:20 INFO Vacuumed table table=multipart_uploads41262026/11/12 07:16:20 INFO Received complete push request method=POST path=/41272026/11/12 07:16:20 INFO Vacuumed table table=closures41282026/11/12 07:16:20 INFO Vacuumed table table=pending_objects41292026/11/12 07:16:20 INFO Aborted multipart uploads count=0 kept=041302026/11/12 07:16:20 INFO Vacuumed table table=multipart_uploads41312026/11/12 07:16:20 WARN Force mode enabled - objects will be deleted immediately without grace period41322026/11/12 07:16:20 INFO Vacuumed table table=objects41332026/11/12 07:16:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041342026/11/12 07:16:20 INFO Received push request method=POST path=/41352026/11/12 07:16:20 INFO Aborted multipart uploads count=0 kept=041362026/11/12 07:16:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041372026/11/12 07:16:20 INFO Vacuumed table table=closures41382026/11/12 07:16:20 INFO Received complete push request method=POST path=/41392026/11/12 07:16:20 INFO Vacuumed table table=pending_closures41402026/11/12 07:16:20 INFO Completed push id=3 objects=22 duration=42.580541ms41412026/11/12 07:16:20 INFO Vacuumed table table=pending_closures41422026/11/12 07:16:20 INFO Vacuumed table table=commit_receipts41432026/11/12 07:16:20 INFO Vacuumed table table=commit_receipts41442026/11/12 07:16:20 INFO Completed push id=32 objects=6 duration=673.5µs41452026/11/12 07:16:20 INFO Vacuumed table table=pending_objects41462026/11/12 07:16:20 INFO Received push request method=POST path=/41472026/11/12 07:16:20 INFO Vacuumed table table=multipart_uploads41482026/11/12 07:16:20 INFO Received complete upload request method=POST path=/41492026/11/12 07:16:20 INFO Completed upload id=33 objects=7 duration=532.834µs41502026/11/12 07:16:20 INFO Vacuumed table table=closures41512026/11/12 07:16:20 INFO Vacuumed table table=objects41522026/11/12 07:16:20 INFO Vacuumed table table=objects41532026/11/12 07:16:20 INFO Vacuumed table table=pending_objects41542026/11/12 07:16:20 INFO Vacuumed table table=multipart_uploads41552026/11/12 07:16:20 INFO Received complete upload request method=POST path=/41562026/11/12 07:16:20 INFO Vacuumed table table=closures41572026/11/12 07:16:20 INFO Received complete push request method=POST path=/41582026/11/12 07:16:20 INFO Completed upload id=34 objects=3 duration=484.125µs41592026/11/12 07:16:20 INFO Received push request method=POST path=/41602026/11/12 07:16:20 INFO Completed push id=4 objects=4 duration=1.812042ms41612026/11/12 07:16:20 INFO Received push request method=POST path=/41622026/11/12 07:16:20 INFO Vacuumed table table=objects41632026/11/12 07:16:20 INFO Received complete push request method=POST path=/41642026/11/12 07:16:20 INFO Completed push id=35 objects=11 duration=406.584µs41652026/11/12 07:16:20 INFO Aborted multipart uploads count=0 kept=041662026/11/12 07:16:20 WARN Force mode enabled - objects will be deleted immediately without grace period41672026/11/12 07:16:20 INFO Received push request method=POST path=/41682026/11/12 07:16:20 INFO Garbage collection completed failed-uploads-deleted=6 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041692026/11/12 07:16:20 INFO Received uploads request method=POST path=/41702026/11/12 07:16:20 INFO Received push request method=POST path=/41712026/11/12 07:16:20 INFO Received complete push request method=POST path=/41722026/11/12 07:16:20 INFO Vacuumed table table=pending_closures41732026/11/12 07:16:20 INFO Completed push id=6 objects=7 duration=5.730708ms41742026/11/12 07:16:20 INFO Vacuumed table table=commit_receipts41752026/11/12 07:16:20 INFO Received uploads request method=POST path=/41762026/11/12 07:16:20 INFO Received push request method=POST path=/41772026/11/12 07:16:20 INFO Received uploads request method=POST path=/41782026/11/12 07:16:20 INFO Vacuumed table table=pending_objects41792026/11/12 07:16:20 INFO Vacuumed table table=multipart_uploads41802026/11/12 07:16:20 INFO Vacuumed table table=closures41812026/11/12 07:16:20 INFO Vacuumed table table=objects41822026/11/12 07:16:20 INFO Aborted multipart uploads count=0 kept=041832026/11/12 07:16:20 WARN Force mode enabled - objects will be deleted immediately without grace period41842026/11/12 07:16:20 INFO Received complete push request method=POST path=/41852026/11/12 07:16:20 INFO Completed push id=7 objects=20 duration=734.5µs41862026/11/12 07:16:20 INFO Aborted multipart uploads count=0 kept=041872026/11/12 07:16:20 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041882026/11/12 07:16:20 INFO Received complete push request method=POST path=/41892026/11/12 07:16:20 INFO Vacuumed table table=pending_closures41902026/11/12 07:16:20 INFO Completed push id=6 objects=6 duration=495.417µs41912026/11/12 07:16:20 INFO Vacuumed table table=commit_receipts41922026/11/12 07:16:20 INFO Received push request method=POST path=/41932026/11/12 07:16:20 INFO Received complete push request method=POST path=/41942026/11/12 07:16:20 INFO Vacuumed table table=pending_objects41952026/11/12 07:16:20 INFO Completed push id=20 objects=3 duration=9.863833ms41962026/11/12 07:16:20 INFO Vacuumed table table=multipart_uploads41972026/11/12 07:16:20 INFO Received complete upload request method=POST path=/41982026/11/12 07:16:20 INFO Vacuumed table table=closures41992026/11/12 07:16:20 INFO Completed upload id=2 objects=2 duration=1.127042ms42002026/11/12 07:16:20 INFO Vacuumed table table=objects42012026/11/12 07:16:20 INFO Received complete push request method=POST path=/42022026/11/12 07:16:20 INFO Received push request method=POST path=/42032026/11/12 07:16:20 INFO Completed push id=13 objects=6 duration=1.031083ms42042026/11/12 07:16:20 INFO Received push request method=POST path=/42052026/11/12 07:16:20 INFO Received push request method=POST path=/42062026/11/12 07:16:20 INFO Received push request method=POST path=/42072026/11/12 07:16:20 INFO Received push request method=POST path=/42082026/11/12 07:16:20 INFO Received complete upload request method=POST path=/42092026/11/12 07:16:20 INFO Completed upload id=22 objects=6 duration=322.541µs42102026/11/12 07:16:20 INFO Received uploads request method=POST path=/42112026/11/12 07:16:20 INFO Received uploads request method=POST path=/42122026/11/12 07:16:20 INFO Received complete push request method=POST path=/42132026/11/12 07:16:20 INFO Received push request method=POST path=/42142026/11/12 07:16:20 INFO Completed push id=6 objects=5 duration=494.5µs42152026/11/12 07:16:20 INFO Received push request method=POST path=/42162026/11/12 07:16:20 INFO Received push request method=POST path=/42172026/11/12 07:16:20 INFO Received uploads request method=POST path=/42182026/11/12 07:16:20 INFO Received push request method=POST path=/42192026/11/12 07:16:20 INFO Received uploads request method=POST path=/42202026/11/12 07:16:20 INFO Received complete push request method=POST path=/42212026/11/12 07:16:20 INFO Received push request method=POST path=/42222026/11/12 07:16:20 INFO Completed push id=5 objects=3 duration=8.763833ms42232026/11/12 07:16:20 INFO Received uploads request method=POST path=/42242026/11/12 07:16:21 INFO Received push request method=POST path=/42252026/11/12 07:16:21 INFO Received uploads request method=POST path=/42262026/11/12 07:16:21 INFO Received complete push request method=POST path=/42272026/11/12 07:16:21 INFO Received uploads request method=POST path=/42282026/11/12 07:16:21 INFO Completed push id=8 objects=6 duration=377.375µs42292026/11/12 07:16:21 INFO Received uploads request method=POST path=/42302026/11/12 07:16:21 INFO Received push request method=POST path=/42312026/11/12 07:16:21 INFO Received push request method=POST path=/42322026/11/12 07:16:21 INFO Received uploads request method=POST path=/42332026/11/12 07:16:21 INFO Received push request method=POST path=/42342026/11/12 07:16:21 INFO Received push request method=POST path=/42352026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=042362026/11/12 07:16:21 INFO Received push request method=POST path=/42372026/11/12 07:16:21 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042382026/11/12 07:16:21 INFO Vacuumed table table=pending_closures42392026/11/12 07:16:21 INFO Received complete push request method=POST path=/42402026/11/12 07:16:21 INFO Completed push id=7 objects=27 duration=537.167µs42412026/11/12 07:16:21 INFO Vacuumed table table=commit_receipts42422026/11/12 07:16:21 INFO Vacuumed table table=pending_objects42432026/11/12 07:16:21 INFO Vacuumed table table=multipart_uploads42442026/11/12 07:16:21 INFO Vacuumed table table=closures42452026/11/12 07:16:21 INFO Received complete upload request method=POST path=/42462026/11/12 07:16:21 INFO Completed upload id=13 objects=14 duration=1.146458ms42472026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=042482026/11/12 07:16:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042492026/11/12 07:16:21 INFO Vacuumed table table=pending_closures42502026/11/12 07:16:21 INFO Vacuumed table table=commit_receipts42512026/11/12 07:16:21 INFO Vacuumed table table=pending_objects42522026/11/12 07:16:21 INFO Vacuumed table table=multipart_uploads42532026/11/12 07:16:21 INFO Vacuumed table table=objects42542026/11/12 07:16:21 INFO Vacuumed table table=closures42552026/11/12 07:16:21 INFO Vacuumed table table=objects42562026/11/12 07:16:21 INFO Received uploads request method=POST path=/42572026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=042582026/11/12 07:16:21 INFO Received complete upload request method=POST path=/42592026/11/12 07:16:21 WARN Force mode enabled - objects will be deleted immediately without grace period42602026/11/12 07:16:21 INFO Completed upload id=10 objects=17 duration=494.084µs42612026/11/12 07:16:21 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042622026/11/12 07:16:21 INFO Vacuumed table table=pending_closures42632026/11/12 07:16:21 INFO Received push request method=POST path=/42642026/11/12 07:16:21 INFO Vacuumed table table=commit_receipts42652026/11/12 07:16:21 INFO Received push request method=POST path=/42662026/11/12 07:16:21 INFO Vacuumed table table=pending_objects42672026/11/12 07:16:21 INFO Vacuumed table table=multipart_uploads42682026/11/12 07:16:21 INFO Received complete push request method=POST path=/42692026/11/12 07:16:21 INFO Vacuumed table table=closures42702026/11/12 07:16:21 INFO Received complete push request method=POST path=/42712026/11/12 07:16:21 INFO Completed push id=14 objects=6 duration=452.083µs42722026/11/12 07:16:21 INFO Vacuumed table table=objects42732026/11/12 07:16:21 INFO Completed push id=6 objects=26 duration=5.278583ms42742026/11/12 07:16:21 INFO Received push request method=POST path=/42752026/11/12 07:16:21 INFO Received complete push request method=POST path=/42762026/11/12 07:16:21 INFO Received push request method=POST path=/42772026/11/12 07:16:21 INFO Completed push id=31 objects=6 duration=1.55425ms42782026/11/12 07:16:21 INFO Received push request method=POST path=/42792026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=042802026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=042812026/11/12 07:16:21 WARN Force mode enabled - objects will be deleted immediately without grace period42822026/11/12 07:16:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042832026/11/12 07:16:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042842026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=042852026/11/12 07:16:21 INFO Received uploads request method=POST path=/42862026/11/12 07:16:21 INFO Vacuumed table table=pending_closures42872026/11/12 07:16:21 INFO Vacuumed table table=commit_receipts42882026/11/12 07:16:21 INFO Vacuumed table table=pending_closures42892026/11/12 07:16:21 INFO Vacuumed table table=commit_receipts42902026/11/12 07:16:21 INFO Vacuumed table table=pending_objects42912026/11/12 07:16:21 INFO Vacuumed table table=multipart_uploads42922026/11/12 07:16:21 INFO Vacuumed table table=pending_objects42932026/11/12 07:16:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=042942026/11/12 07:16:21 INFO Vacuumed table table=closures42952026/11/12 07:16:21 INFO Vacuumed table table=multipart_uploads42962026/11/12 07:16:21 INFO Vacuumed table table=pending_closures42972026/11/12 07:16:21 INFO Vacuumed table table=commit_receipts42982026/11/12 07:16:21 INFO Vacuumed table table=objects42992026/11/12 07:16:21 INFO Vacuumed table table=closures43002026/11/12 07:16:21 INFO Vacuumed table table=pending_objects43012026/11/12 07:16:21 INFO Vacuumed table table=objects43022026/11/12 07:16:21 INFO Received push request method=POST path=/43032026/11/12 07:16:21 INFO Vacuumed table table=multipart_uploads43042026/11/12 07:16:21 INFO Received complete upload request method=POST path=/43052026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=043062026/11/12 07:16:21 INFO Received uploads request method=POST path=/43072026/11/12 07:16:21 INFO Vacuumed table table=closures43082026/11/12 07:16:21 INFO Completed upload id=3 objects=9 duration=383.417µs43092026/11/12 07:16:21 INFO Vacuumed table table=objects43102026/11/12 07:16:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043112026/11/12 07:16:21 INFO Vacuumed table table=pending_closures43122026/11/12 07:16:21 INFO Vacuumed table table=commit_receipts43132026/11/12 07:16:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=043142026/11/12 07:16:21 INFO Vacuumed table table=pending_objects43152026/11/12 07:16:21 INFO Vacuumed table table=pending_closures43162026/11/12 07:16:21 INFO Vacuumed table table=multipart_uploads43172026/11/12 07:16:21 INFO Received push request method=POST path=/43182026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=043192026/11/12 07:16:21 INFO Vacuumed table table=commit_receipts43202026/11/12 07:16:21 INFO Vacuumed table table=closures43212026/11/12 07:16:21 INFO Vacuumed table table=pending_objects43222026/11/12 07:16:21 INFO Vacuumed table table=objects43232026/11/12 07:16:21 WARN Force mode enabled - objects will be deleted immediately without grace period43242026/11/12 07:16:21 INFO Received uploads request method=POST path=/43252026/11/12 07:16:21 INFO Vacuumed table table=multipart_uploads43262026/11/12 07:16:21 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."43272026/11/12 07:16:21 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."43282026/11/12 07:16:21 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."43292026/11/12 07:16:21 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."43302026/11/12 07:16:21 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."43312026/11/12 07:16:21 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."43322026/11/12 07:16:21 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."43332026/11/12 07:16:21 INFO Vacuumed table table=closures43342026/11/12 07:16:21 INFO Received push request method=POST path=/43352026/11/12 07:16:21 INFO Received uploads request method=POST path=/43362026/11/12 07:16:21 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."43372026/11/12 07:16:21 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."43382026/11/12 07:16:21 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."43392026/11/12 07:16:21 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."43402026/11/12 07:16:21 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."43412026/11/12 07:16:21 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."43422026/11/12 07:16:21 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."43432026/11/12 07:16:21 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."43442026/11/12 07:16:21 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."43452026/11/12 07:16:21 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."43462026/11/12 07:16:21 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."43472026/11/12 07:16:21 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."43482026/11/12 07:16:21 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."43492026/11/12 07:16:21 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."43502026/11/12 07:16:21 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."43512026/11/12 07:16:21 INFO Vacuumed table table=objects43522026/11/12 07:16:21 INFO Received uploads request method=POST path=/43532026/11/12 07:16:21 INFO Received complete push request method=POST path=/43542026/11/12 07:16:21 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."43552026/11/12 07:16:21 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."43562026/11/12 07:16:21 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."43572026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=043582026/11/12 07:16:21 INFO Completed push id=12 objects=2 duration=6.74275ms43592026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=043602026/11/12 07:16:21 WARN Force mode enabled - objects will be deleted immediately without grace period43612026/11/12 07:16:21 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043622026/11/12 07:16:21 INFO Received uploads request method=POST path=/43632026/11/12 07:16:21 INFO Received uploads request method=POST path=/43642026/11/12 07:16:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043652026/11/12 07:16:21 INFO Vacuumed table table=pending_closures43662026/11/12 07:16:21 INFO Vacuumed table table=commit_receipts43672026/11/12 07:16:21 INFO Vacuumed table table=pending_closures43682026/11/12 07:16:21 INFO Received complete push request method=POST path=/43692026/11/12 07:16:21 INFO Vacuumed table table=commit_receipts43702026/11/12 07:16:21 INFO Received push request method=POST path=/43712026/11/12 07:16:21 INFO Completed push id=17 objects=6 duration=1.129416ms43722026/11/12 07:16:21 INFO Vacuumed table table=pending_objects43732026/11/12 07:16:21 INFO Vacuumed table table=pending_objects43742026/11/12 07:16:21 INFO Vacuumed table table=multipart_uploads43752026/11/12 07:16:21 INFO Vacuumed table table=multipart_uploads43762026/11/12 07:16:21 INFO Received uploads request method=POST path=/43772026/11/12 07:16:21 INFO Received uploads request method=POST path=/43782026/11/12 07:16:21 INFO Vacuumed table table=closures43792026/11/12 07:16:21 INFO Received complete push request method=POST path=/43802026/11/12 07:16:21 INFO Vacuumed table table=closures43812026/11/12 07:16:21 INFO Vacuumed table table=objects43822026/11/12 07:16:21 INFO Received push request method=POST path=/43832026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=043842026/11/12 07:16:21 INFO Completed push id=5 objects=6 duration=388.125µs43852026/11/12 07:16:21 INFO Vacuumed table table=objects43862026/11/12 07:16:21 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043872026/11/12 07:16:21 INFO Vacuumed table table=pending_closures43882026/11/12 07:16:21 INFO Received push request method=POST path=/43892026/11/12 07:16:21 INFO Received complete upload request method=POST path=/43902026/11/12 07:16:21 INFO Vacuumed table table=commit_receipts43912026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=043922026/11/12 07:16:21 INFO Completed upload id=15 objects=3 duration=708.208µs43932026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=043942026/11/12 07:16:21 WARN Force mode enabled - objects will be deleted immediately without grace period43952026/11/12 07:16:21 INFO Vacuumed table table=pending_objects43962026/11/12 07:16:21 WARN Force mode enabled - objects will be deleted immediately without grace period43972026/11/12 07:16:21 INFO Received push request method=POST path=/43982026/11/12 07:16:21 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."43992026/11/12 07:16:21 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."44002026/11/12 07:16:21 INFO Vacuumed table table=multipart_uploads44012026/11/12 07:16:21 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."44022026/11/12 07:16:21 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."44032026/11/12 07:16:21 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."44042026/11/12 07:16:21 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."44052026/11/12 07:16:21 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."44062026/11/12 07:16:21 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."44072026/11/12 07:16:21 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."44082026/11/12 07:16:21 INFO Vacuumed table table=closures44092026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=044102026/11/12 07:16:21 WARN Force mode enabled - objects will be deleted immediately without grace period44112026/11/12 07:16:21 INFO Vacuumed table table=objects44122026/11/12 07:16:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=9 objects-failed-to-delete=044132026/11/12 07:16:21 INFO Received push request method=POST path=/44142026/11/12 07:16:21 INFO Received uploads request method=POST path=/44152026/11/12 07:16:21 INFO Vacuumed table table=pending_closures44162026/11/12 07:16:21 INFO Vacuumed table table=commit_receipts44172026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=044182026/11/12 07:16:21 INFO Vacuumed table table=pending_objects44192026/11/12 07:16:21 INFO Vacuumed table table=multipart_uploads44202026/11/12 07:16:21 INFO Received complete push request method=POST path=/44212026/11/12 07:16:21 INFO Vacuumed table table=closures44222026/11/12 07:16:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044232026/11/12 07:16:21 INFO Vacuumed table table=pending_closures44242026/11/12 07:16:21 INFO Completed push id=9 objects=7 duration=6.3665ms44252026/11/12 07:16:21 INFO Vacuumed table table=objects44262026/11/12 07:16:21 INFO Vacuumed table table=commit_receipts44272026/11/12 07:16:21 INFO Vacuumed table table=pending_objects44282026/11/12 07:16:21 INFO Vacuumed table table=multipart_uploads44292026/11/12 07:16:21 INFO Vacuumed table table=closures44302026/11/12 07:16:21 INFO Received push request method=POST path=/44312026/11/12 07:16:21 INFO Vacuumed table table=objects44322026/11/12 07:16:21 INFO Received uploads request method=POST path=/44332026/11/12 07:16:21 INFO Received complete push request method=POST path=/44342026/11/12 07:16:21 INFO Completed push id=22 objects=3 duration=1.659ms44352026/11/12 07:16:21 INFO Received push request method=POST path=/44362026/11/12 07:16:21 INFO Aborted multipart uploads count=0 kept=044372026/11/12 07:16:21 WARN Force mode enabled - objects will be deleted immediately without grace period44382026/11/12 07:16:21 INFO Received uploads request method=POST path=/44392026/11/12 07:16:21 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."44402026/11/12 07:16:21 INFO Received push request method=POST path=/44412026/11/12 07:16:21 INFO Received complete push request method=POST path=/44422026/11/12 07:16:21 INFO Received complete push request method=POST path=/44432026/11/12 07:16:21 INFO Completed push id=4 objects=7 duration=615.416µs44442026/11/12 07:16:21 INFO Completed push id=16 objects=7 duration=620.75µs44452026/11/12 07:16:21 INFO Received uploads request method=POST path=/44462026/11/12 07:16:21 INFO Received uploads request method=POST path=/44472026/11/12 07:16:21 INFO Received complete upload request method=POST path=/44482026/11/12 07:16:21 INFO Completed upload id=10 objects=7 duration=468.041µs44492026/11/12 07:16:21 INFO Received uploads request method=POST path=/44502026/11/12 07:16:21 INFO Received uploads request method=POST path=/44512026/11/12 07:16:21 INFO Received complete upload request method=POST path=/44522026/11/12 07:16:21 INFO Completed upload id=18 objects=4 duration=310.666µs44532026/11/12 07:16:22 INFO Received complete push request method=POST path=/44542026/11/12 07:16:22 INFO Completed push id=23 objects=6 duration=506.541µs44552026/11/12 07:16:22 INFO Received complete push request method=POST path=/44562026/11/12 07:16:22 INFO Completed push id=24 objects=6 duration=345.542µs44572026/11/12 07:16:22 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."44582026/11/12 07:16:22 INFO Received complete push request method=POST path=/44592026/11/12 07:16:22 INFO Received uploads request method=POST path=/44602026/11/12 07:16:22 INFO Completed push id=17 objects=22 duration=965.792µs44612026/11/12 07:16:22 INFO Received uploads request method=POST path=/44622026/11/12 07:16:22 INFO Received uploads request method=POST path=/44632026/11/12 07:16:22 INFO Received push request method=POST path=/44642026/11/12 07:16:22 INFO Received complete push request method=POST path=/44652026/11/12 07:16:22 INFO Received complete push request method=POST path=/44662026/11/12 07:16:22 INFO Completed push id=25 objects=7 duration=5.726417ms44672026/11/12 07:16:22 INFO Aborted multipart uploads count=0 kept=044682026/11/12 07:16:22 INFO Completed push id=19 objects=14 duration=635.916µs44692026/11/12 07:16:22 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."44702026/11/12 07:16:22 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."44712026/11/12 07:16:22 INFO Received uploads request method=POST path=/44722026/11/12 07:16:22 INFO Aborted multipart uploads count=0 kept=044732026/11/12 07:16:22 WARN Force mode enabled - objects will be deleted immediately without grace period44742026/11/12 07:16:22 INFO Received complete push request method=POST path=/44752026/11/12 07:16:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=044762026/11/12 07:16:22 INFO Completed push id=1 objects=9 duration=399.333µs44772026/11/12 07:16:22 INFO Vacuumed table table=pending_closures44782026/11/12 07:16:22 INFO Vacuumed table table=commit_receipts44792026/11/12 07:16:22 INFO Vacuumed table table=pending_objects44802026/11/12 07:16:22 INFO Vacuumed table table=multipart_uploads44812026/11/12 07:16:22 INFO Vacuumed table table=closures44822026/11/12 07:16:22 INFO Vacuumed table table=objects44832026/11/12 07:16:22 INFO Received complete upload request method=POST path=/44842026/11/12 07:16:22 INFO Completed upload id=21 objects=7 duration=811.375µs44852026/11/12 07:16:22 INFO Received push request method=POST path=/44862026/11/12 07:16:22 INFO Received uploads request method=POST path=/44872026/11/12 07:16:22 INFO Received uploads request method=POST path=/44882026/11/12 07:16:22 INFO Received complete upload request method=POST path=/44892026/11/12 07:16:22 INFO Completed upload id=27 objects=4 duration=276.167µs44902026/11/12 07:16:22 INFO Received complete upload request method=POST path=/44912026/11/12 07:16:22 INFO Completed upload id=20 objects=6 duration=560.542µs44922026/11/12 07:16:22 INFO Received complete push request method=POST path=/44932026/11/12 07:16:22 INFO Completed push id=13 objects=11 duration=539.208µs44942026/11/12 07:16:22 INFO Received uploads request method=POST path=/44952026/11/12 07:16:22 INFO Received uploads request method=POST path=/44962026/11/12 07:16:22 INFO Aborted multipart uploads count=0 kept=044972026/11/12 07:16:22 WARN Force mode enabled - objects will be deleted immediately without grace period44982026/11/12 07:16:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044992026/11/12 07:16:22 INFO Received complete upload request method=POST path=/45002026/11/12 07:16:22 INFO Vacuumed table table=pending_closures45012026/11/12 07:16:22 INFO Completed upload id=21 objects=2 duration=342.292µs45022026/11/12 07:16:22 INFO Received complete upload request method=POST path=/45032026/11/12 07:16:22 INFO Vacuumed table table=commit_receipts45042026/11/12 07:16:22 INFO Vacuumed table table=pending_objects45052026/11/12 07:16:22 INFO Completed upload id=19 objects=25 duration=6.873709ms45062026/11/12 07:16:22 INFO Vacuumed table table=multipart_uploads45072026/11/12 07:16:22 INFO Received uploads request method=POST path=/45082026/11/12 07:16:22 INFO Vacuumed table table=closures45092026/11/12 07:16:22 INFO Vacuumed table table=objects45102026/11/12 07:16:22 INFO Aborted multipart uploads count=0 kept=045112026/11/12 07:16:22 WARN Force mode enabled - objects will be deleted immediately without grace period45122026/11/12 07:16:22 INFO Aborted multipart uploads count=0 kept=045132026/11/12 07:16:22 WARN Force mode enabled - objects will be deleted immediately without grace period45142026/11/12 07:16:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=045152026/11/12 07:16:22 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045162026/11/12 07:16:22 INFO Received uploads request method=POST path=/45172026/11/12 07:16:22 INFO Vacuumed table table=pending_closures45182026/11/12 07:16:22 INFO Vacuumed table table=pending_closures45192026/11/12 07:16:22 INFO Vacuumed table table=commit_receipts45202026/11/12 07:16:22 INFO Vacuumed table table=commit_receipts45212026/11/12 07:16:22 INFO Vacuumed table table=pending_objects45222026/11/12 07:16:22 INFO Vacuumed table table=multipart_uploads45232026/11/12 07:16:22 INFO Vacuumed table table=pending_objects45242026/11/12 07:16:22 INFO Vacuumed table table=multipart_uploads45252026/11/12 07:16:22 INFO Vacuumed table table=closures45262026/11/12 07:16:22 INFO Vacuumed table table=closures45272026/11/12 07:16:22 INFO Vacuumed table table=objects45282026/11/12 07:16:22 INFO Vacuumed table table=objects45292026/11/12 07:16:22 INFO Received push request method=POST path=/45302026/11/12 07:16:22 INFO Received uploads request method=POST path=/45312026/11/12 07:16:22 INFO Received complete push request method=POST path=/45322026/11/12 07:16:22 INFO Received uploads request method=POST path=/45332026/11/12 07:16:22 INFO Received push request method=POST path=/45342026/11/12 07:16:22 INFO Completed push id=7 objects=7 duration=381.792µs45352026/11/12 07:16:22 INFO Received uploads request method=POST path=/45362026/11/12 07:16:22 INFO Received complete push request method=POST path=/45372026/11/12 07:16:22 INFO Received uploads request method=POST path=/45382026/11/12 07:16:22 INFO Received push request method=POST path=/45392026/11/12 07:16:22 INFO Completed push id=24 objects=24 duration=1.009167ms45402026/11/12 07:16:22 INFO Aborted multipart uploads count=0 kept=045412026/11/12 07:16:22 INFO Received push request method=POST path=/45422026/11/12 07:16:22 WARN Force mode enabled - objects will be deleted immediately without grace period45432026/11/12 07:16:22 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045442026/11/12 07:16:22 INFO Vacuumed table table=pending_closures45452026/11/12 07:16:22 INFO Vacuumed table table=commit_receipts45462026/11/12 07:16:22 INFO Received complete push request method=POST path=/45472026/11/12 07:16:22 INFO Received uploads request method=POST path=/45482026/11/12 07:16:22 INFO Completed push id=13 objects=12 duration=1.132833ms45492026/11/12 07:16:22 INFO Aborted multipart uploads count=0 kept=045502026/11/12 07:16:22 INFO Vacuumed table table=pending_objects45512026/11/12 07:16:22 INFO Received complete push request method=POST path=/45522026/11/12 07:16:22 INFO Vacuumed table table=multipart_uploads45532026/11/12 07:16:22 INFO Vacuumed table table=closures45542026/11/12 07:16:22 INFO Completed push id=16 objects=4 duration=7.102125ms45552026/11/12 07:16:22 INFO Vacuumed table table=objects45562026/11/12 07:16:22 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=045572026/11/12 07:16:22 INFO Received uploads request method=POST path=/45582026/11/12 07:16:22 INFO Received uploads request method=POST path=/45592026/11/12 07:16:22 INFO Vacuumed table table=pending_closures45602026/11/12 07:16:22 INFO Received complete upload request method=POST path=/45612026/11/12 07:16:22 INFO Completed upload id=20 objects=5 duration=289.959µs45622026/11/12 07:16:22 INFO Vacuumed table table=commit_receipts45632026/11/12 07:16:22 INFO Received uploads request method=POST path=/45642026/11/12 07:16:22 INFO Vacuumed table table=pending_objects45652026/11/12 07:16:22 INFO Received complete upload request method=POST path=/45662026/11/12 07:16:22 INFO Received complete push request method=POST path=/45672026/11/12 07:16:22 INFO Completed upload id=26 objects=24 duration=716.041µs45682026/11/12 07:16:22 INFO Vacuumed table table=multipart_uploads45692026/11/12 07:16:22 INFO Completed push id=16 objects=14 duration=1.842125ms45702026/11/12 07:16:22 INFO Received uploads request method=POST path=/45712026/11/12 07:16:22 INFO Vacuumed table table=closures45722026/11/12 07:16:22 INFO Received push request method=POST path=/45732026/11/12 07:16:22 INFO Aborted multipart uploads count=0 kept=045742026/11/12 07:16:22 INFO Received push request method=POST path=/45752026/11/12 07:16:22 INFO Vacuumed table table=objects45762026/11/12 07:16:22 WARN Force mode enabled - objects will be deleted immediately without grace period45772026/11/12 07:16:22 INFO Received uploads request method=POST path=/45782026/11/12 07:16:22 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045792026/11/12 07:16:22 INFO Vacuumed table table=pending_closures45802026/11/12 07:16:22 INFO Received push request method=POST path=/45812026/11/12 07:16:22 INFO Received push request method=POST path=/45822026/11/12 07:16:22 INFO Aborted multipart uploads count=0 kept=045832026/11/12 07:16:22 INFO Vacuumed table table=commit_receipts45842026/11/12 07:16:22 INFO Vacuumed table table=pending_objects45852026/11/12 07:16:22 INFO Vacuumed table table=multipart_uploads45862026/11/12 07:16:22 WARN Force mode enabled - objects will be deleted immediately without grace period45872026/11/12 07:16:22 INFO Vacuumed table table=closures45882026/11/12 07:16:22 INFO Vacuumed table table=objects45892026/11/12 07:16:22 INFO Received push request method=POST path=/45902026/11/12 07:16:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045912026/11/12 07:16:22 INFO Aborted multipart uploads count=0 kept=045922026/11/12 07:16:22 INFO Vacuumed table table=pending_closures45932026/11/12 07:16:22 INFO Vacuumed table table=commit_receipts45942026/11/12 07:16:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045952026/11/12 07:16:22 INFO Vacuumed table table=pending_objects45962026/11/12 07:16:22 INFO Vacuumed table table=pending_closures45972026/11/12 07:16:22 INFO Vacuumed table table=multipart_uploads45982026/11/12 07:16:22 INFO Aborted multipart uploads count=0 kept=045992026/11/12 07:16:22 INFO Vacuumed table table=commit_receipts46002026/11/12 07:16:22 INFO Vacuumed table table=closures46012026/11/12 07:16:22 INFO Vacuumed table table=pending_objects46022026/11/12 07:16:22 WARN Force mode enabled - objects will be deleted immediately without grace period46032026/11/12 07:16:22 INFO Vacuumed table table=multipart_uploads46042026/11/12 07:16:22 INFO Vacuumed table table=closures46052026/11/12 07:16:22 INFO Vacuumed table table=objects46062026/11/12 07:16:22 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046072026/11/12 07:16:22 INFO Vacuumed table table=objects46082026/11/12 07:16:22 INFO Vacuumed table table=pending_closures46092026/11/12 07:16:22 INFO Vacuumed table table=commit_receipts46102026/11/12 07:16:22 INFO Vacuumed table table=pending_objects46112026/11/12 07:16:22 INFO Received push request method=POST path=/46122026/11/12 07:16:22 INFO Vacuumed table table=multipart_uploads46132026/11/12 07:16:22 INFO Received push request method=POST path=/46142026/11/12 07:16:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=046152026/11/12 07:16:22 INFO Vacuumed table table=closures46162026/11/12 07:16:22 INFO Received uploads request method=POST path=/46172026/11/12 07:16:22 INFO Received complete upload request method=POST path=/46182026/11/12 07:16:22 INFO Vacuumed table table=objects46192026/11/12 07:16:22 INFO Completed upload id=25 objects=6 duration=5.933166ms46202026/11/12 07:16:22 INFO Vacuumed table table=pending_closures46212026/11/12 07:16:22 INFO Aborted multipart uploads count=0 kept=046222026/11/12 07:16:22 INFO Vacuumed table table=commit_receipts46232026/11/12 07:16:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046242026/11/12 07:16:22 INFO Vacuumed table table=pending_closures46252026/11/12 07:16:22 INFO Received push request method=POST path=/46262026/11/12 07:16:22 INFO Vacuumed table table=commit_receipts46272026/11/12 07:16:22 INFO Received uploads request method=POST path=/46282026/11/12 07:16:22 INFO Vacuumed table table=pending_objects46292026/11/12 07:16:22 INFO Received complete push request method=POST path=/46302026/11/12 07:16:22 INFO Vacuumed table table=multipart_uploads46312026/11/12 07:16:22 INFO Completed push id=11 objects=9 duration=4.293625ms46322026/11/12 07:16:22 INFO Vacuumed table table=pending_objects46332026/11/12 07:16:22 INFO Vacuumed table table=closures46342026/11/12 07:16:22 INFO Received complete upload request method=POST path=/46352026/11/12 07:16:22 INFO Received complete upload request method=POST path=/46362026/11/12 07:16:22 INFO Vacuumed table table=objects46372026/11/12 07:16:22 INFO Vacuumed table table=multipart_uploads46382026/11/12 07:16:22 INFO Completed upload id=18 objects=11 duration=766.5µs46392026/11/12 07:16:22 INFO Completed upload id=23 objects=7 duration=1.30025ms46402026/11/12 07:16:22 INFO Vacuumed table table=closures46412026/11/12 07:16:22 INFO Received push request method=POST path=/46422026/11/12 07:16:22 INFO Received uploads request method=POST path=/46432026/11/12 07:16:22 INFO Received complete upload request method=POST path=/46442026/11/12 07:16:22 INFO Completed upload id=26 objects=5 duration=500.541µs46452026/11/12 07:16:22 INFO Received push request method=POST path=/46462026/11/12 07:16:22 INFO Vacuumed table table=objects46472026/11/12 07:16:22 INFO Received push request method=POST path=/46482026/11/12 07:16:22 INFO Received complete push request method=POST path=/46492026/11/12 07:16:22 INFO Completed push id=27 objects=3 duration=554.458µs46502026/11/12 07:16:22 INFO Aborted multipart uploads count=0 kept=046512026/11/12 07:16:22 INFO Aborted multipart uploads count=0 kept=046522026/11/12 07:16:22 INFO Aborted multipart uploads count=0 kept=046532026/11/12 07:16:22 INFO Received complete upload request method=POST path=/46542026/11/12 07:16:22 WARN Force mode enabled - objects will be deleted immediately without grace period46552026/11/12 07:16:22 WARN Force mode enabled - objects will be deleted immediately without grace period46562026/11/12 07:16:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046572026/11/12 07:16:22 INFO Completed upload id=28 objects=5 duration=287.25µs46582026/11/12 07:16:22 INFO Received complete push request method=POST path=/46592026/11/12 07:16:22 INFO Vacuumed table table=pending_closures46602026/11/12 07:16:22 INFO Completed push id=15 objects=24 duration=5.638625ms46612026/11/12 07:16:22 INFO Vacuumed table table=commit_receipts46622026/11/12 07:16:22 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046632026/11/12 07:16:22 INFO Vacuumed table table=pending_objects46642026/11/12 07:16:22 INFO Vacuumed table table=pending_closures46652026/11/12 07:16:22 INFO Received complete push request method=POST path=/46662026/11/12 07:16:23 INFO Received complete upload request method=POST path=/46672026/11/12 07:16:23 INFO Vacuumed table table=commit_receipts46682026/11/12 07:16:23 INFO Vacuumed table table=multipart_uploads46692026/11/12 07:16:23 INFO Completed push id=29 objects=17 duration=8.205708ms46702026/11/12 07:16:23 INFO Completed upload id=19 objects=2 duration=8.202083ms46712026/11/12 07:16:23 INFO Vacuumed table table=pending_objects46722026/11/12 07:16:23 INFO Vacuumed table table=closures46732026/11/12 07:16:23 INFO Vacuumed table table=multipart_uploads46742026/11/12 07:16:23 INFO Vacuumed table table=objects46752026/11/12 07:16:23 INFO Aborted multipart uploads count=0 kept=046762026/11/12 07:16:23 INFO Received complete upload request method=POST path=/46772026/11/12 07:16:23 INFO Vacuumed table table=closures46782026/11/12 07:16:23 WARN Force mode enabled - objects will be deleted immediately without grace period46792026/11/12 07:16:23 INFO Received push request method=POST path=/46802026/11/12 07:16:23 INFO Vacuumed table table=objects46812026/11/12 07:16:23 INFO Completed upload id=24 objects=7 duration=5.720375ms46822026/11/12 07:16:23 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046832026/11/12 07:16:23 INFO Received complete push request method=POST path=/46842026/11/12 07:16:23 INFO Received complete upload request method=POST path=/46852026/11/12 07:16:23 INFO Completed push id=26 objects=3 duration=328.083µs46862026/11/12 07:16:23 INFO Received complete push request method=POST path=/46872026/11/12 07:16:23 INFO Received push request method=POST path=/46882026/11/12 07:16:23 INFO Completed upload id=2 objects=7 duration=477.708µs46892026/11/12 07:16:23 INFO Completed push id=8 objects=6 duration=6.199833ms46902026/11/12 07:16:23 INFO Vacuumed table table=pending_closures46912026/11/12 07:16:23 INFO Received complete upload request method=POST path=/46922026/11/12 07:16:23 INFO Vacuumed table table=commit_receipts46932026/11/12 07:16:23 INFO Completed upload id=28 objects=8 duration=250.917µs46942026/11/12 07:16:23 INFO Received push request method=POST path=/46952026/11/12 07:16:23 INFO Received push request method=POST path=/46962026/11/12 07:16:23 INFO Received complete upload request method=POST path=/46972026/11/12 07:16:23 INFO Received complete upload request method=POST path=/46982026/11/12 07:16:23 INFO Completed upload id=29 objects=10 duration=805.875µs46992026/11/12 07:16:23 INFO Received complete push request method=POST path=/47002026/11/12 07:16:23 INFO Received complete push request method=POST path=/47012026/11/12 07:16:23 INFO Vacuumed table table=pending_objects47022026/11/12 07:16:23 INFO Completed upload id=27 objects=26 duration=334.5µs47032026/11/12 07:16:23 INFO Completed push id=31 objects=14 duration=8.257ms47042026/11/12 07:16:23 INFO Vacuumed table table=multipart_uploads47052026/11/12 07:16:23 INFO Completed push id=1 objects=11 duration=8.32075ms47062026/11/12 07:16:23 INFO Vacuumed table table=closures47072026/11/12 07:16:23 INFO Received complete upload request method=POST path=/47082026/11/12 07:16:23 INFO Completed upload id=30 objects=20 duration=361.25µs47092026/11/12 07:16:23 INFO Vacuumed table table=objects47102026/11/12 07:16:23 INFO Received complete push request method=POST path=/47112026/11/12 07:16:23 INFO Received complete push request method=POST path=/47122026/11/12 07:16:23 INFO Completed push id=28 objects=6 duration=674.958µs47132026/11/12 07:16:23 INFO Received uploads request method=POST path=/47142026/11/12 07:16:23 INFO Completed push id=19 objects=6 duration=1.6475ms47152026/11/12 07:16:23 INFO Received complete push request method=POST path=/47162026/11/12 07:16:23 INFO Received complete push request method=POST path=/47172026/11/12 07:16:23 INFO Completed push id=31 objects=11 duration=247µs47182026/11/12 07:16:23 INFO Aborted multipart uploads count=0 kept=047192026/11/12 07:16:23 INFO Completed push id=29 objects=6 duration=343.584µs47202026/11/12 07:16:23 WARN Force mode enabled - objects will be deleted immediately without grace period47212026/11/12 07:16:23 INFO Received complete upload request method=POST path=/47222026/11/12 07:16:23 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."47232026/11/12 07:16:23 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."47242026/11/12 07:16:23 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."47252026/11/12 07:16:23 INFO Completed upload id=32 objects=2 duration=301.25µs47262026/11/12 07:16:23 INFO Aborted multipart uploads count=0 kept=047272026/11/12 07:16:23 INFO Received complete push request method=POST path=/47282026/11/12 07:16:23 WARN Force mode enabled - objects will be deleted immediately without grace period47292026/11/12 07:16:23 INFO Completed push id=33 objects=3 duration=318.875µs47302026/11/12 07:16:23 INFO Received complete upload request method=POST path=/47312026/11/12 07:16:23 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."47322026/11/12 07:16:23 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."47332026/11/12 07:16:23 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."47342026/11/12 07:16:23 INFO Completed upload id=16 objects=3 duration=522.916µs47352026/11/12 07:16:23 INFO Received complete upload request method=POST path=/47362026/11/12 07:16:23 INFO Completed upload id=34 objects=3 duration=185.334µs47372026/11/12 07:16:23 INFO Received push request method=POST path=/47382026/11/12 07:16:23 INFO Aborted multipart uploads count=0 kept=047392026/11/12 07:16:23 WARN Force mode enabled - objects will be deleted immediately without grace period47402026/11/12 07:16:23 INFO Received push request method=POST path=/47412026/11/12 07:16:23 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047422026/11/12 07:16:23 INFO Vacuumed table table=pending_closures47432026/11/12 07:16:23 INFO Vacuumed table table=commit_receipts47442026/11/12 07:16:23 INFO Vacuumed table table=pending_objects47452026/11/12 07:16:23 INFO Vacuumed table table=multipart_uploads47462026/11/12 07:16:23 INFO Vacuumed table table=closures47472026/11/12 07:16:23 INFO Vacuumed table table=objects47482026/11/12 07:16:23 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."47492026/11/12 07:16:23 INFO Received uploads request method=POST path=/47502026/11/12 07:16:23 INFO Received complete push request method=POST path=/47512026/11/12 07:16:23 INFO Completed push id=15 objects=4 duration=545.667µs47522026/11/12 07:16:23 INFO Received push request method=POST path=/47532026/11/12 07:16:23 INFO Received complete upload request method=POST path=/47542026/11/12 07:16:23 INFO Received uploads request method=POST path=/47552026/11/12 07:16:23 INFO Completed upload id=30 objects=8 duration=277.458µs47562026/11/12 07:16:23 INFO Received complete upload request method=POST path=/47572026/11/12 07:16:23 INFO Completed upload id=8 objects=6 duration=347.75µs47582026/11/12 07:16:23 INFO Received complete upload request method=POST path=/47592026/11/12 07:16:23 INFO Completed upload id=31 objects=6 duration=398.833µs47602026/11/12 07:16:23 INFO Received complete push request method=POST path=/47612026/11/12 07:16:23 INFO Received complete upload request method=POST path=/47622026/11/12 07:16:23 INFO Completed upload id=32 objects=4 duration=409.542µs47632026/11/12 07:16:23 INFO Completed push id=7 objects=7 duration=7.094125ms47642026/11/12 07:16:23 INFO Received complete push request method=POST path=/47652026/11/12 07:16:23 INFO Completed push id=12 objects=6 duration=454.042µs47662026/11/12 07:16:23 INFO Received complete push request method=POST path=/47672026/11/12 07:16:23 INFO Completed push id=9 objects=24 duration=500.167µs47682026/11/12 07:16:23 INFO Received uploads request method=POST path=/47692026/11/12 07:16:23 INFO Received uploads request method=POST path=/47702026/11/12 07:16:23 INFO Received complete upload request method=POST path=/47712026/11/12 07:16:23 INFO Completed upload id=10 objects=10 duration=311.375µs47722026/11/12 07:16:23 INFO Received complete upload request method=POST path=/47732026/11/12 07:16:23 INFO Received uploads request method=POST path=/47742026/11/12 07:16:23 INFO Completed upload id=33 objects=10 duration=360.167µs47752026/11/12 07:16:23 INFO Received push request method=POST path=/47762026/11/12 07:16:23 INFO Received complete push request method=POST path=/47772026/11/12 07:16:23 INFO Received push request method=POST path=/47782026/11/12 07:16:23 INFO Completed push id=14 objects=3 duration=419.084µs47792026/11/12 07:16:23 INFO Received uploads request method=POST path=/47802026/11/12 07:16:23 INFO Received complete upload request method=POST path=/47812026/11/12 07:16:23 INFO Completed upload id=20 objects=4 duration=259.542µs47822026/11/12 07:16:23 INFO Received complete push request method=POST path=/47832026/11/12 07:16:23 INFO Completed push id=14 objects=3 duration=272.458µs47842026/11/12 07:16:23 INFO Received push request method=POST path=/47852026/11/12 07:16:23 INFO Received push request method=POST path=/47862026/11/12 07:16:23 INFO Received complete push request method=POST path=/47872026/11/12 07:16:23 INFO Aborted multipart uploads count=0 kept=047882026/11/12 07:16:23 INFO Completed push id=4 objects=6 duration=457.875µs47892026/11/12 07:16:23 WARN Force mode enabled - objects will be deleted immediately without grace period47902026/11/12 07:16:23 INFO Received uploads request method=POST path=/47912026/11/12 07:16:23 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047922026/11/12 07:16:23 INFO Received push request method=POST path=/47932026/11/12 07:16:23 INFO Vacuumed table table=pending_closures47942026/11/12 07:16:23 INFO Vacuumed table table=commit_receipts47952026/11/12 07:16:23 INFO Aborted multipart uploads count=0 kept=047962026/11/12 07:16:23 INFO Received complete push request method=POST path=/47972026/11/12 07:16:23 WARN Force mode enabled - objects will be deleted immediately without grace period47982026/11/12 07:16:23 INFO Completed push id=15 objects=7 duration=523.083µs47992026/11/12 07:16:23 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048002026/11/12 07:16:23 INFO Vacuumed table table=pending_objects48012026/11/12 07:16:23 INFO Received uploads request method=POST path=/48022026/11/12 07:16:23 INFO Vacuumed table table=multipart_uploads48032026/11/12 07:16:23 INFO Vacuumed table table=pending_closures48042026/11/12 07:16:23 INFO Vacuumed table table=closures48052026/11/12 07:16:23 INFO Vacuumed table table=commit_receipts48062026/11/12 07:16:23 INFO Received uploads request method=POST path=/48072026/11/12 07:16:23 INFO Vacuumed table table=objects48082026/11/12 07:16:23 INFO Vacuumed table table=pending_objects48092026/11/12 07:16:23 INFO Vacuumed table table=multipart_uploads48102026/11/12 07:16:23 INFO Received push request method=POST path=/48112026/11/12 07:16:23 INFO Received complete upload request method=POST path=/48122026/11/12 07:16:23 INFO Vacuumed table table=closures48132026/11/12 07:16:23 INFO Received complete upload request method=POST path=/48142026/11/12 07:16:23 INFO Completed upload id=34 objects=23 duration=333.292µs48152026/11/12 07:16:23 INFO Vacuumed table table=objects48162026/11/12 07:16:23 INFO Completed upload id=22 objects=2 duration=559.708µs48172026/11/12 07:16:23 INFO Received push request method=POST path=/48182026/11/12 07:16:23 INFO Received uploads request method=POST path=/48192026/11/12 07:16:23 INFO Aborted multipart uploads count=0 kept=048202026/11/12 07:16:23 INFO Received uploads request method=POST path=/48212026/11/12 07:16:23 INFO Aborted multipart uploads count=0 kept=048222026/11/12 07:16:23 WARN Force mode enabled - objects will be deleted immediately without grace period48232026/11/12 07:16:23 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048242026/11/12 07:16:23 INFO Received uploads request method=POST path=/48252026/11/12 07:16:23 INFO Vacuumed table table=pending_closures48262026/11/12 07:16:23 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=048272026/11/12 07:16:23 INFO Vacuumed table table=commit_receipts48282026/11/12 07:16:23 INFO Vacuumed table table=pending_closures48292026/11/12 07:16:23 INFO Aborted multipart uploads count=0 kept=048302026/11/12 07:16:23 INFO Vacuumed table table=pending_objects48312026/11/12 07:16:23 INFO Vacuumed table table=commit_receipts48322026/11/12 07:16:23 WARN Force mode enabled - objects will be deleted immediately without grace period48332026/11/12 07:16:23 INFO Vacuumed table table=multipart_uploads48342026/11/12 07:16:23 INFO Vacuumed table table=pending_objects48352026/11/12 07:16:23 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048362026/11/12 07:16:23 INFO Vacuumed table table=closures48372026/11/12 07:16:23 INFO Vacuumed table table=multipart_uploads48382026/11/12 07:16:23 INFO Vacuumed table table=objects48392026/11/12 07:16:23 INFO Vacuumed table table=pending_closures48402026/11/12 07:16:23 INFO Vacuumed table table=commit_receipts48412026/11/12 07:16:23 INFO Vacuumed table table=closures48422026/11/12 07:16:23 INFO Received complete push request method=POST path=/48432026/11/12 07:16:23 INFO Vacuumed table table=pending_objects48442026/11/12 07:16:23 INFO Vacuumed table table=objects48452026/11/12 07:16:23 INFO Completed push id=35 objects=6 duration=899.75µs48462026/11/12 07:16:23 INFO Vacuumed table table=multipart_uploads48472026/11/12 07:16:23 INFO Vacuumed table table=closures48482026/11/12 07:16:23 INFO Aborted multipart uploads count=0 kept=048492026/11/12 07:16:23 INFO Received uploads request method=POST path=/48502026/11/12 07:16:23 INFO Aborted multipart uploads count=0 kept=048512026/11/12 07:16:23 INFO Vacuumed table table=objects48522026/11/12 07:16:23 WARN Force mode enabled - objects will be deleted immediately without grace period48532026/11/12 07:16:23 WARN Force mode enabled - objects will be deleted immediately without grace period48542026/11/12 07:16:23 INFO Received push request method=POST path=/48552026/11/12 07:16:23 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048562026/11/12 07:16:23 INFO Vacuumed table table=pending_closures48572026/11/12 07:16:23 INFO Vacuumed table table=commit_receipts48582026/11/12 07:16:23 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=048592026/11/12 07:16:23 INFO Received uploads request method=POST path=/48602026/11/12 07:16:23 INFO Received push request method=POST path=/48612026/11/12 07:16:23 INFO Received complete push request method=POST path=/48622026/11/12 07:16:23 INFO Vacuumed table table=pending_objects48632026/11/12 07:16:23 INFO Received uploads request method=POST path=/48642026/11/12 07:16:23 INFO Received push request method=POST path=/48652026/11/12 07:16:23 INFO Completed push id=16 objects=5 duration=315.041µs48662026/11/12 07:16:23 INFO Vacuumed table table=multipart_uploads48672026/11/12 07:16:23 INFO Vacuumed table table=closures48682026/11/12 07:16:23 INFO Received complete upload request method=POST path=/48692026/11/12 07:16:23 INFO Vacuumed table table=pending_closures48702026/11/12 07:16:23 INFO Received uploads request method=POST path=/48712026/11/12 07:16:23 INFO Aborted multipart uploads count=0 kept=048722026/11/12 07:16:23 INFO Vacuumed table table=objects48732026/11/12 07:16:23 INFO Vacuumed table table=commit_receipts48742026/11/12 07:16:23 INFO Received complete push request method=POST path=/48752026/11/12 07:16:23 INFO Completed upload id=12 objects=9 duration=1.263333ms48762026/11/12 07:16:23 INFO Completed push id=13 objects=22 duration=1.000125ms48772026/11/12 07:16:23 INFO Received uploads request method=POST path=/48782026/11/12 07:16:23 INFO Vacuumed table table=pending_objects48792026/11/12 07:16:23 INFO Vacuumed table table=multipart_uploads48802026/11/12 07:16:23 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048812026/11/12 07:16:23 INFO Vacuumed table table=closures48822026/11/12 07:16:23 INFO Vacuumed table table=pending_closures48832026/11/12 07:16:23 INFO Vacuumed table table=objects48842026/11/12 07:16:23 INFO Vacuumed table table=commit_receipts48852026/11/12 07:16:23 INFO Vacuumed table table=pending_objects48862026/11/12 07:16:23 INFO Vacuumed table table=multipart_uploads48872026/11/12 07:16:23 INFO Vacuumed table table=closures48882026/11/12 07:16:23 INFO Vacuumed table table=objects48892026/11/12 07:16:23 INFO Aborted multipart uploads count=0 kept=048902026/11/12 07:16:23 INFO Received push request method=POST path=/48912026/11/12 07:16:23 INFO Received complete push request method=POST path=/48922026/11/12 07:16:23 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=048932026/11/12 07:16:23 INFO Completed push id=21 objects=6 duration=334.875µs48942026/11/12 07:16:23 INFO Vacuumed table table=pending_closures48952026/11/12 07:16:23 INFO Received uploads request method=POST path=/48962026/11/12 07:16:23 INFO Vacuumed table table=commit_receipts48972026/11/12 07:16:23 INFO Vacuumed table table=pending_objects48982026/11/12 07:16:23 INFO Received complete upload request method=POST path=/48992026/11/12 07:16:23 INFO Received push request method=POST path=/49002026/11/12 07:16:23 INFO Received complete push request method=POST path=/49012026/11/12 07:16:23 INFO Vacuumed table table=multipart_uploads49022026/11/12 07:16:23 INFO Completed upload id=22 objects=7 duration=6.471417ms49032026/11/12 07:16:23 INFO Completed push id=18 objects=6 duration=6.422709ms49042026/11/12 07:16:23 INFO Vacuumed table table=closures49052026/11/12 07:16:23 INFO Vacuumed table table=objects49062026/11/12 07:16:23 INFO Received complete push request method=POST path=/49072026/11/12 07:16:23 INFO Received complete upload request method=POST path=/49082026/11/12 07:16:23 INFO Completed push id=24 objects=4 duration=387.459µs49092026/11/12 07:16:23 INFO Aborted multipart uploads count=0 kept=049102026/11/12 07:16:23 INFO Received uploads request method=POST path=/49112026/11/12 07:16:23 INFO Completed upload id=23 objects=9 duration=237.833µs49122026/11/12 07:16:23 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049132026/11/12 07:16:23 INFO Received complete push request method=POST path=/49142026/11/12 07:16:23 INFO Completed push id=25 objects=9 duration=251.417µs49152026/11/12 07:16:23 INFO Vacuumed table table=pending_closures49162026/11/12 07:16:23 INFO Vacuumed table table=commit_receipts49172026/11/12 07:16:23 INFO Vacuumed table table=pending_objects49182026/11/12 07:16:24 INFO Vacuumed table table=multipart_uploads49192026/11/12 07:16:24 INFO Vacuumed table table=closures49202026/11/12 07:16:24 INFO Vacuumed table table=objects49212026/11/12 07:16:24 INFO Received uploads request method=POST path=/49222026/11/12 07:16:24 INFO Received uploads request method=POST path=/49232026/11/12 07:16:24 INFO Received complete upload request method=POST path=/49242026/11/12 07:16:24 INFO Completed upload id=26 objects=3 duration=176.417µs49252026/11/12 07:16:24 INFO Received push request method=POST path=/49262026/11/12 07:16:24 INFO Received push request method=POST path=/49272026/11/12 07:16:24 INFO Received uploads request method=POST path=/49282026/11/12 07:16:24 INFO Aborted multipart uploads count=0 kept=049292026/11/12 07:16:24 INFO Received push request method=POST path=/49302026/11/12 07:16:24 INFO Received complete upload request method=POST path=/49312026/11/12 07:16:24 WARN Force mode enabled - objects will be deleted immediately without grace period49322026/11/12 07:16:24 INFO Received uploads request method=POST path=/49332026/11/12 07:16:24 INFO Completed upload id=17 objects=25 duration=1.541625ms49342026/11/12 07:16:24 INFO Aborted multipart uploads count=0 kept=049352026/11/12 07:16:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049362026/11/12 07:16:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049372026/11/12 07:16:24 INFO Vacuumed table table=pending_closures49382026/11/12 07:16:24 INFO Vacuumed table table=pending_closures49392026/11/12 07:16:24 INFO Received push request method=POST path=/49402026/11/12 07:16:24 INFO Vacuumed table table=commit_receipts49412026/11/12 07:16:24 INFO Vacuumed table table=commit_receipts49422026/11/12 07:16:24 INFO Vacuumed table table=pending_objects49432026/11/12 07:16:24 INFO Vacuumed table table=multipart_uploads49442026/11/12 07:16:24 INFO Vacuumed table table=pending_objects49452026/11/12 07:16:24 INFO Vacuumed table table=closures49462026/11/12 07:16:24 INFO Vacuumed table table=multipart_uploads49472026/11/12 07:16:24 INFO Vacuumed table table=objects49482026/11/12 07:16:24 INFO Vacuumed table table=closures49492026/11/12 07:16:24 INFO Vacuumed table table=objects49502026/11/12 07:16:24 INFO Received push request method=POST path=/49512026/11/12 07:16:24 INFO Received complete upload request method=POST path=/49522026/11/12 07:16:24 INFO Received complete upload request method=POST path=/49532026/11/12 07:16:24 INFO Aborted multipart uploads count=0 kept=049542026/11/12 07:16:24 INFO Completed upload id=27 objects=7 duration=319.542µs49552026/11/12 07:16:24 INFO Completed upload id=21 objects=2 duration=481.375µs49562026/11/12 07:16:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049572026/11/12 07:16:24 INFO Received uploads request method=POST path=/49582026/11/12 07:16:24 INFO Received complete upload request method=POST path=/49592026/11/12 07:16:24 INFO Vacuumed table table=pending_closures49602026/11/12 07:16:24 INFO Completed upload id=28 objects=3 duration=256.5µs49612026/11/12 07:16:24 INFO Vacuumed table table=commit_receipts49622026/11/12 07:16:24 INFO Vacuumed table table=pending_objects49632026/11/12 07:16:24 INFO Vacuumed table table=multipart_uploads49642026/11/12 07:16:24 INFO Vacuumed table table=closures49652026/11/12 07:16:24 INFO Received complete push request method=POST path=/49662026/11/12 07:16:24 INFO Aborted multipart uploads count=0 kept=049672026/11/12 07:16:24 WARN Force mode enabled - objects will be deleted immediately without grace period49682026/11/12 07:16:24 INFO Completed push id=16 objects=26 duration=545.166µs49692026/11/12 07:16:24 INFO Vacuumed table table=objects49702026/11/12 07:16:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049712026/11/12 07:16:24 INFO Aborted multipart uploads count=0 kept=049722026/11/12 07:16:24 INFO Vacuumed table table=pending_closures49732026/11/12 07:16:24 INFO Vacuumed table table=commit_receipts49742026/11/12 07:16:24 INFO Aborted multipart uploads count=0 kept=049752026/11/12 07:16:24 INFO Vacuumed table table=pending_objects49762026/11/12 07:16:24 INFO Vacuumed table table=multipart_uploads49772026/11/12 07:16:24 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049782026/11/12 07:16:24 WARN Force mode enabled - objects will be deleted immediately without grace period49792026/11/12 07:16:24 INFO Vacuumed table table=closures49802026/11/12 07:16:24 INFO Vacuumed table table=pending_closures49812026/11/12 07:16:24 INFO Received push request method=POST path=/49822026/11/12 07:16:24 INFO Vacuumed table table=objects49832026/11/12 07:16:24 INFO Vacuumed table table=commit_receipts49842026/11/12 07:16:24 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049852026/11/12 07:16:24 INFO Received push request method=POST path=/49862026/11/12 07:16:24 INFO Received complete push request method=POST path=/49872026/11/12 07:16:24 INFO Vacuumed table table=pending_objects49882026/11/12 07:16:24 INFO Vacuumed table table=pending_closures49892026/11/12 07:16:24 INFO Received uploads request method=POST path=/49902026/11/12 07:16:24 INFO Completed push id=29 objects=23 duration=368.917µs49912026/11/12 07:16:24 INFO Vacuumed table table=multipart_uploads49922026/11/12 07:16:24 INFO Vacuumed table table=commit_receipts49932026/11/12 07:16:24 INFO Vacuumed table table=closures49942026/11/12 07:16:24 INFO Vacuumed table table=objects49952026/11/12 07:16:24 INFO Vacuumed table table=pending_objects49962026/11/12 07:16:24 INFO Vacuumed table table=multipart_uploads49972026/11/12 07:16:24 INFO Vacuumed table table=closures49982026/11/12 07:16:24 INFO Received push request method=POST path=/49992026/11/12 07:16:24 INFO Vacuumed table table=objects50002026/11/12 07:16:24 INFO Received complete push request method=POST path=/50012026/11/12 07:16:24 INFO Completed push id=30 objects=6 duration=303.708µs50022026/11/12 07:16:24 INFO Received complete push request method=POST path=/50032026/11/12 07:16:24 INFO Completed push id=27 objects=6 duration=1.402292ms50042026/11/12 07:16:24 INFO Aborted multipart uploads count=0 kept=050052026/11/12 07:16:24 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050062026/11/12 07:16:24 INFO Vacuumed table table=pending_closures50072026/11/12 07:16:24 INFO Vacuumed table table=commit_receipts50082026/11/12 07:16:24 INFO Received complete upload request method=POST path=/50092026/11/12 07:16:24 INFO Vacuumed table table=pending_objects50102026/11/12 07:16:24 INFO Vacuumed table table=multipart_uploads50112026/11/12 07:16:24 INFO Completed upload id=15 objects=2 duration=6.222542ms50122026/11/12 07:16:24 INFO Vacuumed table table=closures50132026/11/12 07:16:24 INFO Received complete upload request method=POST path=/50142026/11/12 07:16:24 INFO Vacuumed table table=objects50152026/11/12 07:16:24 INFO Completed upload id=33 objects=5 duration=909µs50162026/11/12 07:16:24 INFO Received push request method=POST path=/50172026/11/12 07:16:24 INFO Aborted multipart uploads count=0 kept=050182026/11/12 07:16:24 INFO Received uploads request method=POST path=/50192026/11/12 07:16:24 WARN Force mode enabled - objects will be deleted immediately without grace period50202026/11/12 07:16:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050212026/11/12 07:16:24 INFO Vacuumed table table=pending_closures50222026/11/12 07:16:24 INFO Received uploads request method=POST path=/50232026/11/12 07:16:24 INFO Vacuumed table table=commit_receipts50242026/11/12 07:16:24 INFO Received complete push request method=POST path=/50252026/11/12 07:16:24 INFO Vacuumed table table=pending_objects50262026/11/12 07:16:24 INFO Received complete upload request method=POST path=/50272026/11/12 07:16:24 INFO Vacuumed table table=multipart_uploads50282026/11/12 07:16:24 INFO Completed upload id=26 objects=8 duration=6.399833ms50292026/11/12 07:16:24 INFO Completed push id=33 objects=3 duration=6.469916ms50302026/11/12 07:16:24 INFO Vacuumed table table=closures50312026/11/12 07:16:24 INFO Vacuumed table table=objects50322026/11/12 07:16:24 INFO Received complete push request method=POST path=/50332026/11/12 07:16:24 INFO Received complete push request method=POST path=/50342026/11/12 07:16:24 INFO Completed push id=34 objects=6 duration=274.208µs50352026/11/12 07:16:24 INFO Completed push id=28 objects=14 duration=264.792µs50362026/11/12 07:16:24 INFO Received complete upload request method=POST path=/50372026/11/12 07:16:24 INFO Completed upload id=19 objects=2 duration=1.198541ms50382026/11/12 07:16:24 INFO Received push request method=POST path=/50392026/11/12 07:16:24 INFO Received complete push request method=POST path=/50402026/11/12 07:16:24 INFO Received uploads request method=POST path=/50412026/11/12 07:16:24 INFO Completed push id=35 objects=8 duration=260.458µs50422026/11/12 07:16:24 INFO Received complete push request method=POST path=/50432026/11/12 07:16:24 INFO Received uploads request method=POST path=/50442026/11/12 07:16:24 INFO Completed push id=33 objects=7 duration=421.875µs50452026/11/12 07:16:24 INFO Received complete upload request method=POST path=/50462026/11/12 07:16:24 INFO Aborted multipart uploads count=0 kept=050472026/11/12 07:16:24 INFO Completed upload id=36 objects=9 duration=246.542µs50482026/11/12 07:16:24 INFO Received complete upload request method=POST path=/50492026/11/12 07:16:24 INFO Completed upload id=26 objects=3 duration=196.625µs50502026/11/12 07:16:24 WARN Force mode enabled - objects will be deleted immediately without grace period50512026/11/12 07:16:24 INFO Aborted multipart uploads count=0 kept=050522026/11/12 07:16:24 WARN Force mode enabled - objects will be deleted immediately without grace period50532026/11/12 07:16:24 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050542026/11/12 07:16:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050552026/11/12 07:16:24 INFO Vacuumed table table=pending_closures50562026/11/12 07:16:24 INFO Aborted multipart uploads count=0 kept=050572026/11/12 07:16:24 INFO Vacuumed table table=commit_receipts50582026/11/12 07:16:24 WARN Force mode enabled - objects will be deleted immediately without grace period50592026/11/12 07:16:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050602026/11/12 07:16:24 INFO Vacuumed table table=pending_objects50612026/11/12 07:16:24 INFO Vacuumed table table=pending_closures50622026/11/12 07:16:24 INFO Vacuumed table table=pending_closures50632026/11/12 07:16:24 INFO Vacuumed table table=multipart_uploads50642026/11/12 07:16:24 INFO Vacuumed table table=commit_receipts50652026/11/12 07:16:24 INFO Vacuumed table table=commit_receipts50662026/11/12 07:16:24 INFO Vacuumed table table=closures50672026/11/12 07:16:24 INFO Vacuumed table table=objects50682026/11/12 07:16:24 INFO Vacuumed table table=pending_objects50692026/11/12 07:16:24 INFO Vacuumed table table=pending_objects50702026/11/12 07:16:24 INFO Vacuumed table table=multipart_uploads50712026/11/12 07:16:24 INFO Vacuumed table table=multipart_uploads50722026/11/12 07:16:24 INFO Received uploads request method=POST path=/50732026/11/12 07:16:24 INFO Vacuumed table table=closures50742026/11/12 07:16:24 INFO Vacuumed table table=objects50752026/11/12 07:16:24 INFO Vacuumed table table=closures50762026/11/12 07:16:24 INFO Vacuumed table table=objects50772026/11/12 07:16:24 INFO Received complete upload request method=POST path=/50782026/11/12 07:16:24 INFO Completed upload id=21 objects=9 duration=645.458µs50792026/11/12 07:16:24 INFO Received complete push request method=POST path=/50802026/11/12 07:16:24 INFO Completed push id=7 objects=24 duration=1.183792ms50812026/11/12 07:16:24 INFO Received complete push request method=POST path=/50822026/11/12 07:16:24 INFO Completed push id=25 objects=10 duration=389.375µs50832026/11/12 07:16:24 INFO Received uploads request method=POST path=/50842026/11/12 07:16:24 INFO Received complete upload request method=POST path=/50852026/11/12 07:16:24 INFO Completed upload id=29 objects=5 duration=317.625µs50862026/11/12 07:16:24 INFO Received uploads request method=POST path=/50872026/11/12 07:16:24 INFO Received complete upload request method=POST path=/50882026/11/12 07:16:24 INFO Received uploads request method=POST path=/50892026/11/12 07:16:24 INFO Completed upload id=30 objects=2 duration=278.666µs50902026/11/12 07:16:24 INFO Received complete push request method=POST path=/50912026/11/12 07:16:24 INFO Received push request method=POST path=/50922026/11/12 07:16:24 INFO Completed push id=31 objects=20 duration=256.041µs50932026/11/12 07:16:24 INFO Received uploads request method=POST path=/50942026/11/12 07:16:24 INFO Received complete upload request method=POST path=/50952026/11/12 07:16:24 INFO Completed upload id=32 objects=5 duration=195.875µs50962026/11/12 07:16:24 INFO Received complete upload request method=POST path=/50972026/11/12 07:16:24 INFO Received complete push request method=POST path=/50982026/11/12 07:16:24 INFO Completed upload id=33 objects=3 duration=307.25µs50992026/11/12 07:16:24 INFO Completed push id=13 objects=2 duration=219.125µs51002026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51012026/11/12 07:16:24 INFO Aborted multipart uploads count=0 kept=051022026/11/12 07:16:24 INFO Completed upload id=27 objects=13 duration=338.792µs51032026/11/12 07:16:24 WARN Force mode enabled - objects will be deleted immediately without grace period51042026/11/12 07:16:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051052026/11/12 07:16:24 INFO Aborted multipart uploads count=0 kept=051062026/11/12 07:16:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051072026/11/12 07:16:24 INFO Vacuumed table table=pending_closures51082026/11/12 07:16:24 INFO Vacuumed table table=commit_receipts51092026/11/12 07:16:24 INFO Vacuumed table table=pending_closures51102026/11/12 07:16:24 INFO Vacuumed table table=commit_receipts51112026/11/12 07:16:24 INFO Vacuumed table table=pending_objects51122026/11/12 07:16:24 INFO Vacuumed table table=multipart_uploads51132026/11/12 07:16:24 INFO Vacuumed table table=closures51142026/11/12 07:16:24 INFO Vacuumed table table=objects51152026/11/12 07:16:24 INFO Vacuumed table table=pending_objects51162026/11/12 07:16:24 INFO Vacuumed table table=multipart_uploads51172026/11/12 07:16:24 INFO Vacuumed table table=closures51182026/11/12 07:16:24 INFO Received uploads request method=POST path=/51192026/11/12 07:16:24 INFO Vacuumed table table=objects51202026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51212026/11/12 07:16:24 INFO Received push request method=POST path=/51222026/11/12 07:16:24 INFO Completed upload id=21 objects=25 duration=511.25µs51232026/11/12 07:16:24 INFO Received push request method=POST path=/51242026/11/12 07:16:24 INFO Received uploads request method=POST path=/51252026/11/12 07:16:24 INFO Received uploads request method=POST path=/51262026/11/12 07:16:24 INFO Received uploads request method=POST path=/51272026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51282026/11/12 07:16:24 INFO Completed upload id=29 objects=4 duration=518.208µs51292026/11/12 07:16:24 INFO Received uploads request method=POST path=/51302026/11/12 07:16:24 INFO Received uploads request method=POST path=/51312026/11/12 07:16:24 INFO Received complete push request method=POST path=/51322026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51332026/11/12 07:16:24 INFO Completed upload id=18 objects=22 duration=344.916µs51342026/11/12 07:16:24 INFO Completed push id=11 objects=6 duration=6.125958ms51352026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51362026/11/12 07:16:24 INFO Completed upload id=21 objects=8 duration=293.542µs51372026/11/12 07:16:24 INFO Received complete push request method=POST path=/51382026/11/12 07:16:24 INFO Completed push id=20 objects=13 duration=347.542µs51392026/11/12 07:16:24 INFO Received complete push request method=POST path=/51402026/11/12 07:16:24 INFO Completed push id=22 objects=4 duration=268.541µs51412026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51422026/11/12 07:16:24 INFO Completed upload id=22 objects=7 duration=189.375µs51432026/11/12 07:16:24 INFO Received complete push request method=POST path=/51442026/11/12 07:16:24 INFO Completed push id=24 objects=5 duration=179.959µs51452026/11/12 07:16:24 INFO Received complete push request method=POST path=/51462026/11/12 07:16:24 INFO Completed push id=9 objects=9 duration=1.04425ms51472026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51482026/11/12 07:16:24 INFO Completed upload id=12 objects=6 duration=228.958µs51492026/11/12 07:16:24 INFO Received complete push request method=POST path=/51502026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51512026/11/12 07:16:24 INFO Completed push id=17 objects=3 duration=390.084µs51522026/11/12 07:16:24 INFO Completed upload id=23 objects=7 duration=203.792µs51532026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51542026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51552026/11/12 07:16:24 INFO Completed upload id=21 objects=8 duration=314.834µs51562026/11/12 07:16:24 INFO Completed upload id=25 objects=8 duration=5.387708ms51572026/11/12 07:16:24 INFO Received complete push request method=POST path=/51582026/11/12 07:16:24 INFO Completed push id=24 objects=5 duration=232.625µs51592026/11/12 07:16:24 INFO Received complete push request method=POST path=/51602026/11/12 07:16:24 INFO Received complete push request method=POST path=/51612026/11/12 07:16:24 INFO Completed push id=22 objects=6 duration=224.625µs51622026/11/12 07:16:24 INFO Completed push id=26 objects=4 duration=184.584µs51632026/11/12 07:16:24 INFO Received complete push request method=POST path=/51642026/11/12 07:16:24 INFO Completed push id=23 objects=5 duration=244.75µs51652026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51662026/11/12 07:16:24 INFO Completed upload id=25 objects=2 duration=153.041µs51672026/11/12 07:16:24 INFO Received complete push request method=POST path=/51682026/11/12 07:16:24 INFO Completed push id=24 objects=8 duration=242.5µs51692026/11/12 07:16:24 INFO Received complete push request method=POST path=/51702026/11/12 07:16:24 INFO Completed push id=30 objects=2 duration=191.375µs51712026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51722026/11/12 07:16:24 INFO Completed upload id=24 objects=3 duration=304.416µs51732026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51742026/11/12 07:16:24 INFO Completed upload id=26 objects=20 duration=170.959µs51752026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51762026/11/12 07:16:24 INFO Completed upload id=25 objects=9 duration=173.167µs51772026/11/12 07:16:24 INFO Received complete push request method=POST path=/51782026/11/12 07:16:24 INFO Completed push id=25 objects=4 duration=290.792µs51792026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51802026/11/12 07:16:24 INFO Completed upload id=27 objects=3 duration=171.958µs51812026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51822026/11/12 07:16:24 INFO Completed upload id=28 objects=6 duration=140.666µs51832026/11/12 07:16:24 INFO Received complete push request method=POST path=/51842026/11/12 07:16:24 INFO Completed push id=30 objects=4 duration=159.208µs51852026/11/12 07:16:24 INFO Received complete push request method=POST path=/51862026/11/12 07:16:24 INFO Completed push id=31 objects=7 duration=155.417µs51872026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51882026/11/12 07:16:24 INFO Completed upload id=32 objects=5 duration=149.625µs51892026/11/12 07:16:24 INFO Received complete push request method=POST path=/51902026/11/12 07:16:24 INFO Completed push id=32 objects=7 duration=524.917µs51912026/11/12 07:16:24 INFO Received complete push request method=POST path=/51922026/11/12 07:16:24 INFO Completed push id=28 objects=7 duration=235.667µs51932026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51942026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51952026/11/12 07:16:24 INFO Completed upload id=33 objects=6 duration=222.667µs51962026/11/12 07:16:24 INFO Completed upload id=30 objects=6 duration=5.171417ms51972026/11/12 07:16:24 INFO Received complete upload request method=POST path=/51982026/11/12 07:16:24 INFO Received complete push request method=POST path=/51992026/11/12 07:16:24 INFO Completed upload id=34 objects=3 duration=146.959µs52002026/11/12 07:16:24 INFO Completed push id=31 objects=5 duration=528.833µs52012026/11/12 07:16:24 INFO Aborted multipart uploads count=0 kept=052022026/11/12 07:16:24 WARN Force mode enabled - objects will be deleted immediately without grace period52032026/11/12 07:16:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052042026/11/12 07:16:24 INFO Vacuumed table table=pending_closures52052026/11/12 07:16:24 INFO Vacuumed table table=commit_receipts52062026/11/12 07:16:24 INFO Vacuumed table table=pending_objects52072026/11/12 07:16:24 INFO Vacuumed table table=multipart_uploads52082026/11/12 07:16:24 INFO Vacuumed table table=closures52092026/11/12 07:16:24 INFO Vacuumed table table=objects52102026/11/12 07:16:24 INFO Received complete push request method=POST path=/52112026/11/12 07:16:24 INFO Completed push id=32 objects=26 duration=445.417µs52122026/11/12 07:16:24 INFO Received complete upload request method=POST path=/52132026/11/12 07:16:24 INFO Completed upload id=33 objects=2 duration=288.834µs52142026/11/12 07:16:25 INFO Received complete upload request method=POST path=/52152026/11/12 07:16:25 INFO Completed upload id=34 objects=5 duration=162.959µs52162026/11/12 07:16:25 INFO Received complete upload request method=POST path=/52172026/11/12 07:16:25 INFO Completed upload id=34 objects=24 duration=597.875µs52182026/11/12 07:16:25 INFO Received complete upload request method=POST path=/52192026/11/12 07:16:25 INFO Completed upload id=35 objects=11 duration=172.542µs52202026/11/12 07:16:25 INFO Aborted multipart uploads count=0 kept=052212026/11/12 07:16:25 INFO Received complete upload request method=POST path=/52222026/11/12 07:16:25 INFO Received complete push request method=POST path=/52232026/11/12 07:16:25 INFO Completed upload id=36 objects=3 duration=168.917µs52242026/11/12 07:16:25 WARN Force mode enabled - objects will be deleted immediately without grace period52252026/11/12 07:16:25 INFO Completed push id=26 objects=23 duration=322.083µs52262026/11/12 07:16:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052272026/11/12 07:16:25 INFO Vacuumed table table=pending_closures52282026/11/12 07:16:25 INFO Vacuumed table table=commit_receipts52292026/11/12 07:16:25 INFO Vacuumed table table=pending_objects52302026/11/12 07:16:25 INFO Vacuumed table table=multipart_uploads52312026/11/12 07:16:25 INFO Vacuumed table table=closures52322026/11/12 07:16:25 INFO Vacuumed table table=objects52332026/11/12 07:16:25 INFO Aborted multipart uploads count=0 kept=052342026/11/12 07:16:25 WARN Force mode enabled - objects will be deleted immediately without grace period52352026/11/12 07:16:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052362026/11/12 07:16:25 INFO Vacuumed table table=pending_closures52372026/11/12 07:16:25 INFO Vacuumed table table=commit_receipts52382026/11/12 07:16:25 INFO Vacuumed table table=pending_objects52392026/11/12 07:16:25 INFO Vacuumed table table=multipart_uploads52402026/11/12 07:16:25 INFO Vacuumed table table=closures52412026/11/12 07:16:25 INFO Vacuumed table table=objects52422026/11/12 07:16:25 INFO Received complete push request method=POST path=/52432026/11/12 07:16:25 INFO Completed push id=27 objects=15 duration=279.375µs52442026/11/12 07:16:25 INFO Received complete push request method=POST path=/52452026/11/12 07:16:25 INFO Completed push id=28 objects=19 duration=363.75µs52462026/11/12 07:16:25 INFO Received complete push request method=POST path=/52472026/11/12 07:16:25 INFO Completed push id=29 objects=10 duration=301.834µs52482026/11/12 07:16:25 INFO Received complete upload request method=POST path=/52492026/11/12 07:16:25 INFO Completed upload id=30 objects=6 duration=223.875µs52502026/11/12 07:16:25 INFO Received complete upload request method=POST path=/52512026/11/12 07:16:25 INFO Completed upload id=31 objects=5 duration=181.875µs52522026/11/12 07:16:25 INFO Received complete upload request method=POST path=/52532026/11/12 07:16:25 INFO Completed upload id=34 objects=3 duration=183.792µs52542026/11/12 07:16:25 INFO Aborted multipart uploads count=0 kept=052552026/11/12 07:16:25 WARN Force mode enabled - objects will be deleted immediately without grace period52562026/11/12 07:16:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052572026/11/12 07:16:25 INFO Vacuumed table table=pending_closures52582026/11/12 07:16:25 INFO Vacuumed table table=commit_receipts52592026/11/12 07:16:25 INFO Vacuumed table table=pending_objects52602026/11/12 07:16:25 INFO Vacuumed table table=multipart_uploads52612026/11/12 07:16:25 INFO Vacuumed table table=closures52622026/11/12 07:16:25 INFO Vacuumed table table=objects5263--- PASS: FuzzPushGC (0.02s)5264 --- PASS: FuzzPushGC/seed#15 (7.48s)5265 --- PASS: FuzzPushGC/seed#0 (8.54s)5266 --- PASS: FuzzPushGC/seed#2 (8.59s)5267 --- PASS: FuzzPushGC/seed#14 (8.97s)5268 --- PASS: FuzzPushGC/seed#3 (9.38s)5269 --- PASS: FuzzPushGC/seed#7 (9.80s)5270 --- PASS: FuzzPushGC/seed#8 (9.95s)5271 --- PASS: FuzzPushGC/seed#1 (12.30s)5272 --- PASS: FuzzPushGC/seed#5 (12.50s)5273 --- PASS: FuzzPushGC/seed#12 (13.18s)5274 --- PASS: FuzzPushGC/seed#4 (13.55s)5275 --- PASS: FuzzPushGC/seed#6 (13.69s)5276 --- PASS: FuzzPushGC/seed#10 (14.17s)5277 --- PASS: FuzzPushGC/seed#9 (14.22s)5278 --- PASS: FuzzPushGC/seed#11 (14.25s)5279 --- PASS: FuzzPushGC/seed#13 (14.49s)5280PASS5281{"timestamp":"2026-11-12T07:16:25.267161Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:60945","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(6)"}5282{"timestamp":"2026-11-12T07:16:25.267204Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:60964","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(6)"}5283{"timestamp":"2026-11-12T07:16:25.267616Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:60531","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(8)"}5284{"timestamp":"2026-11-12T07:16:25.267659Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:60746","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(7)"}52852026-11-12 07:16:25.405 UTC [70726] LOG: received smart shutdown request52862026-11-12 07:16:25.406 UTC [70726] LOG: background worker "logical replication launcher" (PID 70736) exited with exit code 152872026-11-12 07:16:26.330 UTC [70731] LOG: checkpoint complete: wrote 3120 buffers (19.0%), wrote 5 SLRU buffers; 0 WAL file(s) added, 0 removed, 33 recycled; write=30.760 s, sync=0.828 s, total=31.690 s; sync files=43473, longest=0.001 s, average=0.001 s; distance=533206 kB, estimate=533206 kB; lsn=0/2851F358, redo lsn=0/220429E052882026-11-12 07:16:26.333 UTC [70731] LOG: shutting down52892026-11-12 07:16:26.333 UTC [70731] LOG: checkpoint starting: shutdown immediate52902026-11-12 07:16:26.890 UTC [70731] LOG: checkpoint complete: wrote 9759 buffers (59.6%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 6 recycled; write=0.504 s, sync=0.050 s, total=0.558 s; sync files=3262, longest=0.001 s, average=0.001 s; distance=103282 kB, estimate=490214 kB; lsn=0/2851F3D0, redo lsn=0/2851F3D052912026-11-12 07:16:26.897 UTC [70726] LOG: database system is shut down5292Running OIDC tests...5293=== RUN TestAudienceForIssuer5294=== PAUSE TestAudienceForIssuer5295=== RUN TestProviderClientGivesUpOnSilentIssuer5296=== PAUSE TestProviderClientGivesUpOnSilentIssuer5297=== RUN TestGlobMatch5298=== PAUSE TestGlobMatch5299=== RUN TestValidateToken_ValidToken5300=== PAUSE TestValidateToken_ValidToken5301=== RUN TestValidateToken_WrongAudience5302=== PAUSE TestValidateToken_WrongAudience5303=== RUN TestValidateToken_Expired5304=== PAUSE TestValidateToken_Expired5305=== RUN TestValidateToken_BoundClaimsMismatch5306=== PAUSE TestValidateToken_BoundClaimsMismatch5307=== RUN TestValidateToken_BoundSubjectMismatch5308=== PAUSE TestValidateToken_BoundSubjectMismatch5309=== RUN TestValidateToken_MultipleProviders5310=== PAUSE TestValidateToken_MultipleProviders5311=== RUN TestValidateToken_UnknownIssuerFetchesNoKeys5312=== PAUSE TestValidateToken_UnknownIssuerFetchesNoKeys5313=== RUN TestValidateToken_GoogleSchemelessIssuer5314=== PAUSE TestValidateToken_GoogleSchemelessIssuer5315=== RUN TestValidateToken_KubernetesServiceAccount5316=== PAUSE TestValidateToken_KubernetesServiceAccount5317=== RUN TestNewValidator_KubernetesRequiresCA5318=== PAUSE TestNewValidator_KubernetesRequiresCA5319=== RUN TestValidateToken_KubernetesIssuerFromOwnToken5320=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken5321=== RUN TestPins_ReservedForMatchingRule5322=== PAUSE TestPins_ReservedForMatchingRule5323=== RUN TestPins_TopLevelShorthand5324=== PAUSE TestPins_TopLevelShorthand5325=== RUN TestPins_ConfigValidation5326=== PAUSE TestPins_ConfigValidation5327=== RUN TestScopes_LegacyProviderDefaultsToWrite5328=== PAUSE TestScopes_LegacyProviderDefaultsToWrite5329=== RUN TestScopes_Rules5330=== PAUSE TestScopes_Rules5331=== RUN TestScopes_ConfigValidation5332=== PAUSE TestScopes_ConfigValidation5333=== CONT TestAudienceForIssuer5334=== CONT TestValidateToken_GoogleSchemelessIssuer5335--- PASS: TestAudienceForIssuer (0.00s)5336=== CONT TestPins_TopLevelShorthand5337=== CONT TestValidateToken_UnknownIssuerFetchesNoKeys5338=== CONT TestScopes_ConfigValidation5339=== CONT TestValidateToken_MultipleProviders5340=== CONT TestScopes_Rules5341=== CONT TestScopes_LegacyProviderDefaultsToWrite53422026/11/12 07:16:28 INFO OIDC provider initialized name=known issuer=https://known.example5343=== CONT TestValidateToken_BoundSubjectMismatch5344=== CONT TestValidateToken_BoundClaimsMismatch5345=== CONT TestPins_ConfigValidation5346=== CONT TestValidateToken_Expired5347=== CONT TestValidateToken_WrongAudience5348--- PASS: TestScopes_ConfigValidation (0.00s)5349=== CONT TestValidateToken_KubernetesIssuerFromOwnToken5350--- PASS: TestPins_ConfigValidation (0.00s)5351=== CONT TestValidateToken_ValidToken5352=== CONT TestGlobMatch5353=== RUN TestGlobMatch/foo_foo5354=== PAUSE TestGlobMatch/foo_foo5355=== RUN TestGlobMatch/foo_bar5356=== PAUSE TestGlobMatch/foo_bar5357=== RUN TestGlobMatch/*_5358=== PAUSE TestGlobMatch/*_5359=== RUN TestGlobMatch/*_anything5360=== PAUSE TestGlobMatch/*_anything5361=== RUN TestGlobMatch/foo*_foo5362=== PAUSE TestGlobMatch/foo*_foo5363=== RUN TestGlobMatch/foo*_foobar5364=== PAUSE TestGlobMatch/foo*_foobar5365=== RUN TestGlobMatch/foo*_bar5366=== PAUSE TestGlobMatch/foo*_bar5367=== RUN TestGlobMatch/*bar_bar5368=== PAUSE TestGlobMatch/*bar_bar5369=== RUN TestGlobMatch/*bar_foobar5370=== PAUSE TestGlobMatch/*bar_foobar5371=== RUN TestGlobMatch/*bar_foo5372=== PAUSE TestGlobMatch/*bar_foo5373=== RUN TestGlobMatch/foo*bar_foobar5374=== PAUSE TestGlobMatch/foo*bar_foobar5375=== RUN TestGlobMatch/foo*bar_foo123bar5376=== PAUSE TestGlobMatch/foo*bar_foo123bar5377=== RUN TestGlobMatch/foo*bar_foobarbaz5378=== PAUSE TestGlobMatch/foo*bar_foobarbaz5379=== RUN TestGlobMatch/*/*_foo/bar5380=== PAUSE TestGlobMatch/*/*_foo/bar5381=== RUN TestGlobMatch/*/*_foo5382=== PAUSE TestGlobMatch/*/*_foo5383=== RUN TestGlobMatch/refs/heads/*_refs/heads/main5384=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main5385=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.05386=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.05387=== RUN TestGlobMatch/refs/*/main_refs/heads/main5388=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main5389=== RUN TestGlobMatch/fo?_foo5390=== PAUSE TestGlobMatch/fo?_foo5391=== RUN TestGlobMatch/fo?_fo5392=== PAUSE TestGlobMatch/fo?_fo5393=== RUN TestGlobMatch/fo?_fooo5394=== PAUSE TestGlobMatch/fo?_fooo5395=== RUN TestGlobMatch/?oo_foo5396=== PAUSE TestGlobMatch/?oo_foo5397=== RUN TestGlobMatch/?oo_boo5398=== PAUSE TestGlobMatch/?oo_boo5399=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5400=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5401=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5402=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5403=== CONT TestGlobMatch/foo_foo5404=== CONT TestPins_ReservedForMatchingRule5405=== CONT TestGlobMatch/*/*_foo/bar5406=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5407=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5408=== CONT TestGlobMatch/?oo_boo5409=== CONT TestGlobMatch/?oo_foo5410=== CONT TestGlobMatch/fo?_fooo5411=== CONT TestGlobMatch/fo?_fo5412=== CONT TestGlobMatch/fo?_foo5413=== CONT TestGlobMatch/refs/*/main_refs/heads/main5414=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.05415=== CONT TestGlobMatch/refs/heads/*_refs/heads/main5416=== CONT TestGlobMatch/*/*_foo5417=== CONT TestGlobMatch/*bar_bar5418=== CONT TestGlobMatch/foo*bar_foobarbaz5419=== CONT TestGlobMatch/foo*bar_foo123bar5420=== CONT TestGlobMatch/foo*bar_foobar5421=== CONT TestGlobMatch/*bar_foo5422=== CONT TestGlobMatch/*bar_foobar5423=== CONT TestGlobMatch/foo*_foo5424=== CONT TestGlobMatch/foo*_bar5425=== CONT TestGlobMatch/foo*_foobar5426=== CONT TestGlobMatch/*_5427=== CONT TestGlobMatch/*_anything5428=== CONT TestGlobMatch/foo_bar5429--- PASS: TestGlobMatch (0.00s)5430 --- PASS: TestGlobMatch/foo_foo (0.00s)5431 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)5432 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)5433 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)5434 --- PASS: TestGlobMatch/?oo_boo (0.00s)5435 --- PASS: TestGlobMatch/?oo_foo (0.00s)5436 --- PASS: TestGlobMatch/fo?_fooo (0.00s)5437 --- PASS: TestGlobMatch/fo?_fo (0.00s)5438 --- PASS: TestGlobMatch/fo?_foo (0.00s)5439 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)5440 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)5441 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)5442 --- PASS: TestGlobMatch/*/*_foo (0.00s)5443 --- PASS: TestGlobMatch/*bar_bar (0.00s)5444 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)5445 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)5446 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)5447 --- PASS: TestGlobMatch/*bar_foo (0.00s)5448 --- PASS: TestGlobMatch/*bar_foobar (0.00s)5449 --- PASS: TestGlobMatch/foo*_foo (0.00s)5450 --- PASS: TestGlobMatch/foo*_bar (0.00s)5451 --- PASS: TestGlobMatch/foo*_foobar (0.00s)5452 --- PASS: TestGlobMatch/*_ (0.00s)5453 --- PASS: TestGlobMatch/*_anything (0.00s)5454 --- PASS: TestGlobMatch/foo_bar (0.00s)5455=== CONT TestProviderClientGivesUpOnSilentIssuer5456=== CONT TestNewValidator_KubernetesRequiresCA5457--- PASS: TestProviderClientGivesUpOnSilentIssuer (0.00s)5458=== CONT TestValidateToken_KubernetesServiceAccount54592026/11/12 07:16:28 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:61760/oidc54602026/11/12 07:16:28 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:61758/oidc5461--- PASS: TestPins_TopLevelShorthand (0.06s)54622026/11/12 07:16:29 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:61762/oidc54632026/11/12 07:16:29 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:61763/oidc5464--- PASS: TestValidateToken_UnknownIssuerFetchesNoKeys (0.06s)5465--- PASS: TestScopes_Rules (0.06s)5466--- PASS: TestValidateToken_ValidToken (0.06s)5467--- PASS: TestScopes_LegacyProviderDefaultsToWrite (0.06s)54682026/11/12 07:16:29 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:61767/oidc5469--- PASS: TestValidateToken_Expired (0.07s)54702026/11/12 07:16:29 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC12354712026/11/12 07:16:29 http: TLS handshake error from 127.0.0.1:61770: read tcp 127.0.0.1:61769->127.0.0.1:61770: use of closed network connection5472--- PASS: TestNewValidator_KubernetesRequiresCA (0.08s)5473--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (0.09s)54742026/11/12 07:16:29 INFO OIDC provider initialized name=google issuer=https://accounts.google.com5475--- PASS: TestValidateToken_GoogleSchemelessIssuer (0.11s)54762026/11/12 07:16:29 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:617755477--- PASS: TestValidateToken_KubernetesServiceAccount (0.10s)54782026/11/12 07:16:29 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:61778/oidc5479--- PASS: TestValidateToken_WrongAudience (0.11s)54802026/11/12 07:16:29 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:61780/oidc54812026/11/12 07:16:29 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:61782/oidc5482--- PASS: TestValidateToken_BoundClaimsMismatch (0.14s)5483--- PASS: TestPins_ReservedForMatchingRule (0.14s)54842026/11/12 07:16:29 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:61784/oidc5485--- PASS: TestValidateToken_BoundSubjectMismatch (0.15s)54862026/11/12 07:16:29 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:61777/oidc54872026/11/12 07:16:29 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:61786/oidc5488--- PASS: TestValidateToken_MultipleProviders (0.19s)5489PASS5490Running hook tests...5491=== RUN TestSendPathsEmpty5492=== PAUSE TestSendPathsEmpty5493=== RUN TestQueueEnqueueAndFetch5494=== PAUSE TestQueueEnqueueAndFetch5495=== RUN TestQueueDeduplication5496=== PAUSE TestQueueDeduplication5497=== RUN TestQueueRemove5498=== PAUSE TestQueueRemove5499=== RUN TestQueueFetchBatchLimit5500=== PAUSE TestQueueFetchBatchLimit5501=== RUN TestQueueRetryMovesToBack5502=== PAUSE TestQueueRetryMovesToBack5503=== RUN TestQueueFetchRemoveLifecycle5504=== PAUSE TestQueueFetchRemoveLifecycle5505=== RUN TestQueueConcurrentWriters5506=== PAUSE TestQueueConcurrentWriters5507=== RUN TestQueueRemoveLargeClosure5508=== PAUSE TestQueueRemoveLargeClosure5509=== RUN TestServerClientIntegration5510--- PASS: TestServerClientIntegration (0.00s)5511=== RUN TestServerQueueError55122026/11/12 07:16:29 ERROR Failed to queue paths error="permission denied" count=15513--- PASS: TestServerQueueError (0.00s)5514=== RUN TestGetListenerSocketActivation5515 server_test.go:147: systemd socket activation does not exist on darwin5516--- SKIP: TestGetListenerSocketActivation (0.00s)5517=== RUN TestServerStalledClientDoesNotBlockShutdown5518=== PAUSE TestServerStalledClientDoesNotBlockShutdown5519=== RUN TestServerBacksOffOnAcceptErrors5520=== PAUSE TestServerBacksOffOnAcceptErrors5521=== RUN TestDrainIsolatesPoisonPath5522=== PAUSE TestDrainIsolatesPoisonPath5523=== RUN TestRunNotBlockedByPoisonHead5524=== PAUSE TestRunNotBlockedByPoisonHead5525=== RUN TestDrainGivesUpWhenServerDown5526=== PAUSE TestDrainGivesUpWhenServerDown5527=== RUN TestFailedPathPrunedByLaterClosure5528=== PAUSE TestFailedPathPrunedByLaterClosure5529=== RUN TestWorkerUploadsAndRemoves5530=== PAUSE TestWorkerUploadsAndRemoves5531=== RUN TestWorkerSkipsGCdPaths5532=== PAUSE TestWorkerSkipsGCdPaths5533=== RUN TestWorkerPrunesClosureDeps5534=== PAUSE TestWorkerPrunesClosureDeps5535=== RUN TestDrainTimeout5536=== PAUSE TestDrainTimeout5537=== RUN TestWorkerRemoveFailureIsNotProgress5538=== PAUSE TestWorkerRemoveFailureIsNotProgress5539=== RUN TestWorkerKeepsPathItCannotStat5540=== PAUSE TestWorkerKeepsPathItCannotStat5541=== CONT TestSendPathsEmpty5542=== CONT TestServerBacksOffOnAcceptErrors5543=== CONT TestQueueRemoveLargeClosure5544=== CONT TestWorkerKeepsPathItCannotStat55452000/01/01 00:00:00 ERROR Accept failed error="too many open files"55462000/01/01 00:00:00 ERROR Accept failed error="too many open files"5547=== CONT TestWorkerRemoveFailureIsNotProgress55482000/01/01 00:00:00 ERROR Accept failed error="too many open files"55492000/01/01 00:00:00 ERROR Accept failed error="too many open files"55502000/01/01 00:00:00 ERROR Accept failed error="too many open files"55512000/01/01 00:00:00 ERROR Accept failed error="too many open files"55522000/01/01 00:00:00 ERROR Accept failed error="too many open files"55532000/01/01 00:00:00 ERROR Accept failed error="too many open files"5554=== CONT TestDrainTimeout5555=== CONT TestWorkerPrunesClosureDeps5556=== CONT TestWorkerSkipsGCdPaths5557=== CONT TestRunNotBlockedByPoisonHead5558=== CONT TestFailedPathPrunedByLaterClosure5559=== CONT TestDrainGivesUpWhenServerDown5560=== CONT TestServerStalledClientDoesNotBlockShutdown5561--- PASS: TestSendPathsEmpty (0.00s)5562=== CONT TestQueueFetchBatchLimit5563--- PASS: TestServerBacksOffOnAcceptErrors (0.00s)5564=== CONT TestWorkerUploadsAndRemoves5565=== CONT TestQueueConcurrentWriters5566=== CONT TestQueueRemove5567=== CONT TestQueueFetchRemoveLifecycle5568=== CONT TestQueueDeduplication5569=== CONT TestQueueRetryMovesToBack5570=== CONT TestQueueEnqueueAndFetch5571=== CONT TestDrainIsolatesPoisonPath55722026/11/12 07:16:29 INFO Upload queue status pending=255732026/11/12 07:16:29 INFO Uploading batch count=25574--- PASS: TestQueueRemove (0.01s)55752026/11/12 07:16:29 INFO Uploading batch count=155762026/11/12 07:16:29 ERROR Upload failed error="upload failed" count=155772000/01/01 00:00:00 INFO Uploading batch count=255782000/01/01 00:00:00 ERROR Upload failed error="context deadline exceeded" count=255792000/01/01 00:00:00 ERROR Drain finished with paths left in queue remaining=45580--- PASS: TestQueueDeduplication (0.01s)5581--- PASS: TestQueueEnqueueAndFetch (0.01s)55822026/11/12 07:16:29 INFO Upload queue status pending=355832026/11/12 07:16:29 INFO Uploading batch count=155842026/11/12 07:16:29 ERROR Upload failed error="upload failed" count=155852026/11/12 07:16:29 INFO Uploading batch count=155862026/11/12 07:16:29 INFO Upload queue status pending=255872026/11/12 07:16:29 WARN Cannot stat store path, will retry later path=/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestWorkerKeepsPathItCannotStat389873231/002/locked/aaa error="stat /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestWorkerKeepsPathItCannotStat389873231/002/locked/aaa: permission denied"55882026/11/12 07:16:29 INFO Uploading batch count=155892026/11/12 07:16:29 INFO Upload queue status pending=255902026/11/12 07:16:29 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestWorkerSkipsGCdPaths3914644578/002/nonexistent5591--- PASS: TestQueueFetchBatchLimit (0.01s)55922026/11/12 07:16:29 INFO Uploading batch count=155932026/11/12 07:16:29 WARN Cannot stat store path, will retry later path=/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestWorkerKeepsPathItCannotStat389873231/002/locked/aaa error="stat /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestWorkerKeepsPathItCannotStat389873231/002/locked/aaa: permission denied"55942026/11/12 07:16:29 INFO Uploading batch count=155952026/11/12 07:16:29 INFO Uploading batch count=455962026/11/12 07:16:29 ERROR Upload failed error="upload failed" count=455972026/11/12 07:16:29 WARN Cannot stat store path, will retry later path=/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestWorkerKeepsPathItCannotStat389873231/002/locked/aaa error="stat /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestWorkerKeepsPathItCannotStat389873231/002/locked/aaa: permission denied"5598--- PASS: TestDrainTimeout (0.02s)55992026/11/12 07:16:29 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestDrainIsolatesPoisonPath1322242611/002/bbb5600--- PASS: TestQueueRetryMovesToBack (0.01s)5601--- PASS: TestQueueFetchRemoveLifecycle (0.01s)56022026/11/12 07:16:29 INFO Uploading batch count=256032026/11/12 07:16:29 ERROR Drain finished with paths left in queue remaining=156042026/11/12 07:16:29 ERROR Upload failed error="upload failed" count=256052026/11/12 07:16:29 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestDrainGivesUpWhenServerDown2325464758/002/a56062026/11/12 07:16:29 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestDrainGivesUpWhenServerDown2325464758/002/b56072026/11/12 07:16:29 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestWorkerRemoveFailureIsNotProgress2756630533/002/nonexistent56082026/11/12 07:16:29 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=156092026/11/12 07:16:29 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestWorkerRemoveFailureIsNotProgress2756630533/002/nonexistent56102026/11/12 07:16:29 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=156112026/11/12 07:16:29 INFO Uploading batch count=256122026/11/12 07:16:29 ERROR Upload failed error="upload failed" count=256132026/11/12 07:16:29 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestDrainGivesUpWhenServerDown2325464758/002/c56142026/11/12 07:16:29 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestWorkerRemoveFailureIsNotProgress2756630533/002/nonexistent56152026/11/12 07:16:29 INFO Uploading batch count=156162026/11/12 07:16:29 ERROR Upload failed error="upload failed" count=156172026/11/12 07:16:29 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=156182026/11/12 07:16:29 ERROR Drain finished with paths left in queue remaining=15619--- PASS: TestFailedPathPrunedByLaterClosure (0.02s)56202026/11/12 07:16:29 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestDrainGivesUpWhenServerDown2325464758/002/d56212026/11/12 07:16:29 INFO Uploading batch count=156222026/11/12 07:16:29 ERROR Upload failed error="upload failed" count=156232026/11/12 07:16:29 INFO Uploading batch count=256242026/11/12 07:16:29 ERROR Upload failed error="upload failed" count=256252026/11/12 07:16:29 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestDrainGivesUpWhenServerDown2325464758/002/e56262026/11/12 07:16:29 INFO Uploading batch count=156272026/11/12 07:16:29 ERROR Upload failed error="upload failed" count=156282026/11/12 07:16:29 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestDrainGivesUpWhenServerDown2325464758/002/f5629--- PASS: TestWorkerKeepsPathItCannotStat (0.02s)56302026/11/12 07:16:29 ERROR Drain finished with paths left in queue remaining=156312026/11/12 07:16:29 ERROR Drain finished with paths left in queue remaining=105632--- PASS: TestWorkerRemoveFailureIsNotProgress (0.02s)5633--- PASS: TestDrainIsolatesPoisonPath (0.02s)5634--- PASS: TestDrainGivesUpWhenServerDown (0.02s)5635--- PASS: TestWorkerUploadsAndRemoves (0.03s)5636--- PASS: TestWorkerPrunesClosureDeps (0.04s)5637--- PASS: TestWorkerSkipsGCdPaths (0.04s)5638--- PASS: TestQueueRemoveLargeClosure (0.05s)5639--- PASS: TestQueueConcurrentWriters (0.06s)56402026/11/12 07:16:29 ERROR Failed to decode request error="read unix /private/var/tb/1/scratch/9cc0db790b9bff21/build/hook364437033/test.sock->: i/o timeout"56412026/11/12 07:16:29 ERROR Failed to write response error="write unix /private/var/tb/1/scratch/9cc0db790b9bff21/build/hook364437033/test.sock->: i/o timeout"5642--- PASS: TestServerStalledClientDoesNotBlockShutdown (0.20s)56432026/11/12 07:16:30 INFO Uploading batch count=156442026/11/12 07:16:30 INFO Uploading batch count=156452026/11/12 07:16:30 INFO Uploading batch count=156462026/11/12 07:16:30 ERROR Upload failed error="upload failed" count=156472026/11/12 07:16:30 INFO Uploading batch count=156482026/11/12 07:16:30 ERROR Upload failed error="upload failed" count=156492026/11/12 07:16:30 INFO Uploading batch count=156502026/11/12 07:16:30 ERROR Upload failed error="upload failed" count=156512026/11/12 07:16:30 INFO Uploading batch count=156522026/11/12 07:16:30 ERROR Upload failed error="upload failed" count=156532026/11/12 07:16:30 ERROR Drain finished with paths left in queue remaining=15654--- PASS: TestRunNotBlockedByPoisonHead (1.04s)5655PASS5656Running signing tests...5657=== RUN TestExternalSigner5658=== PAUSE TestExternalSigner5659=== RUN TestExternalSignerWrongPublicKey5660=== PAUSE TestExternalSignerWrongPublicKey5661=== RUN TestExternalSignerErrors5662=== PAUSE TestExternalSignerErrors5663=== RUN TestExternalSignerRestartsAfterCrash5664=== PAUSE TestExternalSignerRestartsAfterCrash5665=== RUN TestExternalSignerProgramErrorKeepsProcess5666=== PAUSE TestExternalSignerProgramErrorKeepsProcess5667=== RUN TestExternalSignerClose5668=== PAUSE TestExternalSignerClose5669=== RUN TestExternalSignerRejectsInvalidPublicKey5670=== PAUSE TestExternalSignerRejectsInvalidPublicKey5671=== RUN TestExternalSignerCancellation5672=== PAUSE TestExternalSignerCancellation5673=== RUN TestGenerateFingerprint5674=== PAUSE TestGenerateFingerprint5675=== RUN TestParseSigningKey5676=== PAUSE TestParseSigningKey5677=== RUN TestSign5678=== PAUSE TestSign5679=== RUN TestSignBatch5680=== PAUSE TestSignBatch5681=== RUN TestNilKeyPublicKey5682=== PAUSE TestNilKeyPublicKey5683=== RUN TestSignErrors5684=== PAUSE TestSignErrors5685=== CONT TestExternalSigner5686=== CONT TestExternalSignerCancellation5687=== CONT TestExternalSignerRejectsInvalidPublicKey5688=== CONT TestSign5689=== CONT TestExternalSignerClose5690=== CONT TestSignBatch5691=== CONT TestSignErrors5692=== CONT TestNilKeyPublicKey5693=== CONT TestExternalSignerErrors5694=== RUN TestExternalSignerRejectsInvalidPublicKey/#005695=== PAUSE TestExternalSignerRejectsInvalidPublicKey/#005696--- PASS: TestNilKeyPublicKey (0.00s)5697=== RUN TestExternalSignerRejectsInvalidPublicKey/no-colon5698=== CONT TestParseSigningKey5699=== RUN TestParseSigningKey/valid_32-byte_key5700=== CONT TestExternalSignerWrongPublicKey5701=== CONT TestExternalSignerRestartsAfterCrash5702=== CONT TestGenerateFingerprint5703=== RUN TestGenerateFingerprint/basic_with_references5704=== PAUSE TestGenerateFingerprint/basic_with_references5705=== RUN TestGenerateFingerprint/no_references5706=== PAUSE TestGenerateFingerprint/no_references5707=== RUN TestGenerateFingerprint/unsorted_references_get_sorted5708=== PAUSE TestGenerateFingerprint/unsorted_references_get_sorted5709=== RUN TestGenerateFingerprint/invalid_nar_hash_prefix5710=== PAUSE TestGenerateFingerprint/invalid_nar_hash_prefix5711=== RUN TestGenerateFingerprint/invalid_nar_hash_length5712=== PAUSE TestGenerateFingerprint/invalid_nar_hash_length5713=== RUN TestGenerateFingerprint/invalid_store_path_prefix5714=== CONT TestExternalSignerProgramErrorKeepsProcess5715=== RUN TestExternalSignerErrors/null_signatures5716=== PAUSE TestExternalSignerErrors/null_signatures5717=== RUN TestExternalSignerErrors/wrong_signatures_shape5718=== PAUSE TestExternalSignerErrors/wrong_signatures_shape5719=== RUN TestExternalSignerErrors/wrong_signature_name5720=== PAUSE TestExternalSignerErrors/wrong_signature_name5721=== RUN TestExternalSignerErrors/wrong_signature_length5722=== PAUSE TestExternalSignerErrors/wrong_signature_length5723=== RUN TestExternalSignerErrors/null5724=== PAUSE TestExternalSignerErrors/null5725=== RUN TestExternalSignerErrors/too_few_signatures5726=== PAUSE TestExternalSignerErrors/too_few_signatures5727=== RUN TestExternalSignerErrors/program_error5728=== PAUSE TestExternalSignerErrors/program_error5729=== RUN TestExternalSignerErrors/exit_on_request5730=== PAUSE TestExternalSignerErrors/exit_on_request5731=== RUN TestExternalSignerErrors/wrong_shape5732=== PAUSE TestExternalSignerErrors/wrong_shape5733=== RUN TestExternalSignerErrors/unrequested_key5734=== PAUSE TestExternalSignerErrors/unrequested_key5735=== RUN TestExternalSignerErrors/missing_signature_name5736=== PAUSE TestExternalSignerErrors/missing_signature_name5737=== RUN TestExternalSignerErrors/invalid_signature_base645738=== PAUSE TestExternalSignerErrors/invalid_signature_base645739=== PAUSE TestGenerateFingerprint/invalid_store_path_prefix5740=== PAUSE TestExternalSignerRejectsInvalidPublicKey/no-colon5741=== RUN TestGenerateFingerprint/invalid_reference_prefix5742=== PAUSE TestGenerateFingerprint/invalid_reference_prefix5743=== PAUSE TestParseSigningKey/valid_32-byte_key5744=== RUN TestExternalSignerErrors/exit_at_startup5745=== RUN TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5746=== PAUSE TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5747=== CONT TestGenerateFingerprint/invalid_store_path_prefix5748=== CONT TestGenerateFingerprint/invalid_reference_prefix5749=== CONT TestGenerateFingerprint/basic_with_references5750=== CONT TestGenerateFingerprint/invalid_nar_hash_prefix5751=== CONT TestGenerateFingerprint/unsorted_references_get_sorted5752=== CONT TestGenerateFingerprint/invalid_nar_hash_length5753=== CONT TestGenerateFingerprint/no_references5754--- PASS: TestGenerateFingerprint (0.00s)5755 --- PASS: TestGenerateFingerprint/invalid_store_path_prefix (0.00s)5756 --- PASS: TestGenerateFingerprint/invalid_reference_prefix (0.00s)5757 --- PASS: TestGenerateFingerprint/basic_with_references (0.00s)5758 --- PASS: TestGenerateFingerprint/invalid_nar_hash_prefix (0.00s)5759 --- PASS: TestGenerateFingerprint/unsorted_references_get_sorted (0.00s)5760 --- PASS: TestGenerateFingerprint/invalid_nar_hash_length (0.00s)5761 --- PASS: TestGenerateFingerprint/no_references (0.00s)5762=== RUN TestParseSigningKey/valid_32-byte_key_with_different_name5763=== PAUSE TestParseSigningKey/valid_32-byte_key_with_different_name5764=== PAUSE TestExternalSignerErrors/exit_at_startup5765=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5766=== RUN TestExternalSignerErrors/invalid_JSON5767=== PAUSE TestExternalSignerErrors/invalid_JSON5768=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5769=== RUN TestParseSigningKey/no_colon5770=== PAUSE TestParseSigningKey/no_colon5771=== RUN TestParseSigningKey/empty_name5772=== RUN TestExternalSignerErrors/wrong_id5773=== PAUSE TestExternalSignerErrors/wrong_id5774=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5775=== RUN TestExternalSignerErrors/missing_signatures5776=== PAUSE TestParseSigningKey/empty_name5777=== PAUSE TestExternalSignerErrors/missing_signatures5778=== CONT TestExternalSignerErrors/null_signatures5779=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5780=== CONT TestExternalSignerRejectsInvalidPublicKey/#005781=== CONT TestExternalSignerErrors/wrong_shape5782=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5783=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5784=== CONT TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5785=== CONT TestExternalSignerErrors/null5786=== CONT TestExternalSignerErrors/program_error5787=== CONT TestExternalSignerErrors/too_few_signatures5788--- PASS: TestSign (0.00s)5789=== CONT TestExternalSignerRejectsInvalidPublicKey/no-colon5790--- PASS: TestExternalSignerRejectsInvalidPublicKey (0.00s)5791 --- PASS: TestExternalSignerRejectsInvalidPublicKey/#00 (0.00s)5792 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:YQ== (0.00s)5793 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:invalid! (0.00s)5794 --- PASS: TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= (0.00s)5795 --- PASS: TestExternalSignerRejectsInvalidPublicKey/no-colon (0.00s)5796=== CONT TestExternalSignerErrors/exit_at_startup5797=== CONT TestExternalSignerErrors/wrong_signature_name5798=== CONT TestExternalSignerErrors/wrong_signature_length5799=== CONT TestExternalSignerErrors/wrong_signatures_shape5800=== CONT TestExternalSignerErrors/missing_signatures5801=== CONT TestExternalSignerErrors/wrong_id5802=== CONT TestExternalSignerErrors/invalid_JSON5803=== RUN TestParseSigningKey/invalid_base645804=== PAUSE TestParseSigningKey/invalid_base645805=== CONT TestExternalSignerErrors/exit_on_request5806=== RUN TestParseSigningKey/wrong_length5807=== PAUSE TestParseSigningKey/wrong_length5808=== CONT TestExternalSignerErrors/unrequested_key5809=== CONT TestExternalSignerErrors/missing_signature_name5810=== CONT TestExternalSignerErrors/invalid_signature_base645811=== RUN TestSignErrors/nil_key5812--- PASS: TestSignBatch (0.00s)5813=== CONT TestParseSigningKey/valid_32-byte_key5814=== PAUSE TestSignErrors/nil_key5815=== RUN TestSignErrors/nil_narinfo5816=== CONT TestParseSigningKey/empty_name5817=== CONT TestParseSigningKey/wrong_length5818=== CONT TestParseSigningKey/invalid_base645819=== PAUSE TestSignErrors/nil_narinfo5820=== CONT TestParseSigningKey/no_colon5821=== RUN TestSignErrors/invalid_narinfo5822=== PAUSE TestSignErrors/invalid_narinfo5823=== RUN TestSignErrors/canceled5824=== CONT TestParseSigningKey/valid_32-byte_key_with_different_name5825=== PAUSE TestSignErrors/canceled5826=== CONT TestSignErrors/nil_key5827=== CONT TestSignErrors/invalid_narinfo5828--- PASS: TestParseSigningKey (0.00s)5829 --- PASS: TestParseSigningKey/valid_32-byte_key (0.00s)5830 --- PASS: TestParseSigningKey/empty_name (0.00s)5831 --- PASS: TestParseSigningKey/wrong_length (0.00s)5832 --- PASS: TestParseSigningKey/invalid_base64 (0.00s)5833 --- PASS: TestParseSigningKey/no_colon (0.00s)5834 --- PASS: TestParseSigningKey/valid_32-byte_key_with_different_name (0.00s)5835=== CONT TestSignErrors/nil_narinfo5836=== CONT TestSignErrors/canceled5837--- PASS: TestSignErrors (0.00s)5838 --- PASS: TestSignErrors/nil_key (0.00s)5839 --- PASS: TestSignErrors/invalid_narinfo (0.00s)5840 --- PASS: TestSignErrors/nil_narinfo (0.00s)5841 --- PASS: TestSignErrors/canceled (0.00s)5842--- PASS: TestExternalSignerRestartsAfterCrash (0.24s)5843/private/var/tb/1/scratch/9cc0db790b9bff21/build/TestExternalSignerRestartsAfterCrash4165645805/001/sign program: line 7: /private/var/tb/1/scratch/9cc0db790b9bff21/build/TestExternalSignerRestartsAfterCrash4165645805/001/sign program.starts: No such file or directory5844--- PASS: TestExternalSignerWrongPublicKey (1.72s)5845--- PASS: TestExternalSigner (1.92s)5846--- PASS: TestExternalSignerCancellation (2.12s)5847--- PASS: TestExternalSignerClose (2.29s)5848--- PASS: TestExternalSignerProgramErrorKeepsProcess (2.47s)5849--- PASS: TestExternalSignerErrors (0.00s)5850 --- PASS: TestExternalSignerErrors/null (0.40s)5851 --- PASS: TestExternalSignerErrors/wrong_shape (0.59s)5852 --- PASS: TestExternalSignerErrors/program_error (0.78s)5853 --- PASS: TestExternalSignerErrors/invalid_signature_base64 (0.97s)5854 --- PASS: TestExternalSignerErrors/null_signatures (1.16s)5855 --- PASS: TestExternalSignerErrors/wrong_signature_length (1.35s)5856 --- PASS: TestExternalSignerErrors/wrong_signatures_shape (1.53s)5857 --- PASS: TestExternalSignerErrors/too_few_signatures (2.66s)5858 --- PASS: TestExternalSignerErrors/wrong_id (2.84s)5859 --- PASS: TestExternalSignerErrors/invalid_JSON (3.03s)5860 --- PASS: TestExternalSignerErrors/exit_at_startup (3.25s)5861 --- PASS: TestExternalSignerErrors/wrong_signature_name (3.44s)5862 --- PASS: TestExternalSignerErrors/exit_on_request (3.62s)5863 --- PASS: TestExternalSignerErrors/missing_signature_name (3.81s)5864 --- PASS: TestExternalSignerErrors/unrequested_key (4.01s)5865 --- PASS: TestExternalSignerErrors/missing_signatures (4.20s)5866PASS5867Running OpenBao signer tests...5868=== RUN TestSign5869=== PAUSE TestSign5870=== RUN TestSignErrors5871=== PAUSE TestSignErrors5872=== RUN TestServe5873=== PAUSE TestServe5874=== RUN TestServeRejectsGarbage5875=== PAUSE TestServeRejectsGarbage5876=== CONT TestSign5877=== CONT TestSignErrors5878=== RUN TestSignErrors/http_error5879=== PAUSE TestSignErrors/http_error5880=== RUN TestSignErrors/invalid_json5881=== CONT TestServeRejectsGarbage5882=== PAUSE TestSignErrors/invalid_json5883=== CONT TestServe5884--- PASS: TestServeRejectsGarbage (0.00s)5885=== RUN TestSignErrors/no_prefix5886=== PAUSE TestSignErrors/no_prefix5887=== RUN TestSignErrors/invalid_base645888=== PAUSE TestSignErrors/invalid_base645889=== CONT TestSignErrors/http_error5890=== CONT TestSignErrors/no_prefix5891=== CONT TestSignErrors/invalid_base645892=== CONT TestSignErrors/invalid_json5893--- PASS: TestServe (0.00s)5894--- PASS: TestSign (0.00s)5895--- PASS: TestSignErrors (0.00s)5896 --- PASS: TestSignErrors/http_error (0.00s)5897 --- PASS: TestSignErrors/no_prefix (0.00s)5898 --- PASS: TestSignErrors/invalid_base64 (0.00s)5899 --- PASS: TestSignErrors/invalid_json (0.00s)5900PASS