niks3-go-unit-tests log

succeeded `niks3-go-unit-tests` `checks.aarch64-linux.go-unit-tests` · build #506 · raw

1tribuchet: building on eliza2Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestUploadBuildLog_FileBodyReplayedOnRetry6=== PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry7=== RUN TestRegisterUploadedObjectReusesConnections8=== PAUSE TestRegisterUploadedObjectReusesConnections9=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica10=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica11=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun12=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun13=== RUN TestCaseHackSuffix14=== PAUSE TestCaseHackSuffix15=== RUN TestFilterOversizedClosures16=== PAUSE TestFilterOversizedClosures17=== RUN TestUploadMultipart_PartsInParallel18=== PAUSE TestUploadMultipart_PartsInParallel19=== RUN TestUploadMultipart_ReturnsAfterDroppedBody20=== PAUSE TestUploadMultipart_ReturnsAfterDroppedBody21=== RUN TestUploadMultipart_FailedProducerIsNotCompleted22=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted23=== RUN TestPartSizeForNAR24=== PAUSE TestPartSizeForNAR25=== RUN TestDumpPathCaseHackMatchesNix26--- PASS: TestDumpPathCaseHackMatchesNix (0.93s)27=== RUN TestDumpPathCaseHackCollision28--- PASS: TestDumpPathCaseHackCollision (0.00s)29=== RUN TestDumpPathMatchesNix30=== PAUSE TestDumpPathMatchesNix31=== RUN TestDumpPathSingleFile32=== PAUSE TestDumpPathSingleFile33=== RUN TestDumpPathWriterError34=== PAUSE TestDumpPathWriterError35=== RUN TestEncodeNixBase3236=== PAUSE TestEncodeNixBase3237=== RUN TestEncodeNixBase32WithRealHash38=== PAUSE TestEncodeNixBase32WithRealHash39=== RUN TestConvertHashToNix3240=== PAUSE TestConvertHashToNix3241=== RUN TestGetStorePathHash42=== PAUSE TestGetStorePathHash43=== RUN TestPathInfoHashCompatibility44=== PAUSE TestPathInfoHashCompatibility45=== RUN TestParsePathInfoJSON46=== PAUSE TestParsePathInfoJSON47=== RUN TestParsePathInfoJSONMultiplePaths48=== PAUSE TestParsePathInfoJSONMultiplePaths49=== RUN TestPathInfoCACompatibility50=== PAUSE TestPathInfoCACompatibility51=== RUN TestRateLimiterFeedback52=== PAUSE TestRateLimiterFeedback53=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess54=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess55=== RUN TestRegisterUploadedObject_BoundedAgainstSilentServer56=== PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer57=== RUN TestRegisterUploadedObject_OutlivesPush58=== PAUSE TestRegisterUploadedObject_OutlivesPush59=== RUN TestResolveStorePath60=== PAUSE TestResolveStorePath61=== RUN TestDoWithRetry_BodyReplayedViaGetBody62=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody63=== RUN TestRetryTimeoutOutlastsMaxRetries64=== PAUSE TestRetryTimeoutOutlastsMaxRetries65=== RUN TestDoWithRetry_FinalResponseBodyReadable66=== PAUSE TestDoWithRetry_FinalResponseBodyReadable67=== RUN TestShellSplit68=== PAUSE TestShellSplit69=== RUN TestShellSplitErrors70=== PAUSE TestShellSplitErrors71=== RUN TestStreamPushReportsEveryPath72=== PAUSE TestStreamPushReportsEveryPath73=== RUN TestStreamPushBatchesUnderLoad74=== PAUSE TestStreamPushBatchesUnderLoad75=== RUN TestStreamPushIsolatesFailures76=== PAUSE TestStreamPushIsolatesFailures77=== RUN TestStreamPushGivesUpOnDeadServer78=== PAUSE TestStreamPushGivesUpOnDeadServer79=== RUN TestStreamPushRequestLine80=== PAUSE TestStreamPushRequestLine81=== RUN TestStreamPushReportsSignatures82=== PAUSE TestStreamPushReportsSignatures83=== RUN TestStreamPushReportsSkippedPaths84=== PAUSE TestStreamPushReportsSkippedPaths85=== RUN TestClientSignaturesByStorePath86=== PAUSE TestClientSignaturesByStorePath87=== RUN TestStreamPushAnswersEveryLineWhenCanceled88=== PAUSE TestStreamPushAnswersEveryLineWhenCanceled89=== RUN TestStreamPushStopsWhenCanceled90=== PAUSE TestStreamPushStopsWhenCanceled91=== RUN TestSetClientTLS92=== PAUSE TestSetClientTLS93=== RUN TestSetClientTLSDoesNotMutateDefaultTransport94=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport95=== RUN TestSetClientTLSErrors96=== PAUSE TestSetClientTLSErrors97=== RUN TestStaticToken98=== PAUSE TestStaticToken99=== RUN TestFileTokenReadsAndCaches100=== PAUSE TestFileTokenReadsAndCaches101=== RUN TestFileTokenMissing102=== PAUSE TestFileTokenMissing103=== RUN TestFileTokenEmpty104=== PAUSE TestFileTokenEmpty105=== RUN TestScriptTokenNoExpiryRerunsEveryCall106=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall107=== RUN TestScriptTokenCachesUntilRefresh108=== PAUSE TestScriptTokenCachesUntilRefresh109=== RUN TestScriptTokenEmptyToken110=== PAUSE TestScriptTokenEmptyToken111=== RUN TestScriptTokenBadJSON112=== PAUSE TestScriptTokenBadJSON113=== RUN TestScriptTokenScriptFails114=== PAUSE TestScriptTokenScriptFails115=== RUN TestScriptTokenEmptyCommand116=== PAUSE TestScriptTokenEmptyCommand117=== CONT TestDoServerRequestAttachesToken118=== CONT TestRetryTimeoutOutlastsMaxRetries119=== CONT TestScriptTokenEmptyCommand120--- PASS: TestScriptTokenEmptyCommand (0.00s)121=== CONT TestScriptTokenScriptFails122=== CONT TestScriptTokenBadJSON123=== CONT TestDoWithRetry_BodyReplayedViaGetBody124=== CONT TestResolveStorePath125--- PASS: TestResolveStorePath (0.00s)126=== CONT TestRegisterUploadedObject_OutlivesPush127=== CONT TestRegisterUploadedObject_BoundedAgainstSilentServer128=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess129=== CONT TestRateLimiterFeedback130=== RUN TestRateLimiterFeedback/429_enables_limiter131=== PAUSE TestRateLimiterFeedback/429_enables_limiter132=== RUN TestRateLimiterFeedback/503_enables_limiter133=== PAUSE TestRateLimiterFeedback/503_enables_limiter134=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter135=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter136=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter137=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter138=== CONT TestRateLimiterFeedback/429_enables_limiter139=== CONT TestPathInfoCACompatibility140=== RUN TestPathInfoCACompatibility/null_ca_field141=== PAUSE TestPathInfoCACompatibility/null_ca_field142=== RUN TestPathInfoCACompatibility/old_string_format_-_text143=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text144=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive145=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive146=== RUN TestPathInfoCACompatibility/new_structured_format_-_text147=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text148=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method149=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method150=== CONT TestPathInfoCACompatibility/null_ca_field151=== CONT TestParsePathInfoJSONMultiplePaths152=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths153=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths154=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths155=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths156=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths157=== CONT TestParsePathInfoJSON158=== RUN TestParsePathInfoJSON/Nix_format159=== PAUSE TestParsePathInfoJSON/Nix_format160=== RUN TestParsePathInfoJSON/Lix_format161=== PAUSE TestParsePathInfoJSON/Lix_format162=== RUN TestParsePathInfoJSON/empty_input163=== PAUSE TestParsePathInfoJSON/empty_input164=== RUN TestParsePathInfoJSON/whitespace_only165=== PAUSE TestParsePathInfoJSON/whitespace_only166=== RUN TestParsePathInfoJSON/invalid_JSON167=== PAUSE TestParsePathInfoJSON/invalid_JSON168=== CONT TestParsePathInfoJSON/Nix_format169=== CONT TestPathInfoHashCompatibility170=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)171=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)172=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon173=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon174=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI175=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI176=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512177=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512178=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)179=== CONT TestGetStorePathHash180=== RUN TestGetStorePathHash/valid_store_path181=== PAUSE TestGetStorePathHash/valid_store_path182=== RUN TestGetStorePathHash/basename_without_hyphen_should_error183=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error184=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error185=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error186=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error187=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error188=== CONT TestGetStorePathHash/valid_store_path189=== CONT TestConvertHashToNix32190=== RUN TestConvertHashToNix32/SRI_format_to_Nix32191=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32192=== RUN TestConvertHashToNix32/already_Nix32_format193=== PAUSE TestConvertHashToNix32/already_Nix32_format194=== RUN TestConvertHashToNix32/invalid_format195=== PAUSE TestConvertHashToNix32/invalid_format196=== CONT TestConvertHashToNix32/SRI_format_to_Nix32197=== CONT TestEncodeNixBase32WithRealHash198--- PASS: TestEncodeNixBase32WithRealHash (0.00s)199=== CONT TestEncodeNixBase32200=== RUN TestEncodeNixBase32/test_string_hash201=== PAUSE TestEncodeNixBase32/test_string_hash202=== RUN TestEncodeNixBase32/empty_input203=== PAUSE TestEncodeNixBase32/empty_input204=== CONT TestEncodeNixBase32/test_string_hash205=== CONT TestDumpPathWriterError206=== CONT TestScriptTokenEmptyToken207=== CONT TestStreamPushAnswersEveryLineWhenCanceled208=== CONT TestScriptTokenCachesUntilRefresh209=== CONT TestScriptTokenNoExpiryRerunsEveryCall210=== CONT TestFileTokenEmpty211=== CONT TestFileTokenReadsAndCaches2122026/11/12 00:44:34 WARN Rate limiter enabled after throttle name=server-test rate=5213=== CONT TestStreamPushReportsSkippedPaths214=== CONT TestStreamPushReportsSignatures215=== CONT TestStreamPushRequestLine216=== CONT TestStaticToken217=== CONT TestStreamPushGivesUpOnDeadServer218=== CONT TestSetClientTLSErrors219=== CONT TestSetClientTLSDoesNotMutateDefaultTransport220=== CONT TestSetClientTLS221=== CONT TestClientSignaturesByStorePath222=== CONT TestPathInfoCACompatibility/new_structured_format_-_text223=== CONT TestStreamPushStopsWhenCanceled224=== CONT TestPartSizeForNAR225=== CONT TestUploadMultipart_FailedProducerIsNotCompleted226=== RUN TestPartSizeForNAR/zero_stays_at_minimum227=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary228=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum229=== CONT TestDumpPathMatchesNix230=== RUN TestPartSizeForNAR/small_stays_at_minimum231=== PAUSE TestPartSizeForNAR/small_stays_at_minimum232=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum233=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum234=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts235=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts236=== RUN TestPartSizeForNAR/1_TiB237=== PAUSE TestPartSizeForNAR/1_TiB238=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary239=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part240=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part241=== CONT TestDumpPathSingleFile242=== CONT TestUploadMultipart_ReturnsAfterDroppedBody2432026/11/12 00:44:35 ERROR Upload failed error=boom count=1244=== CONT TestStreamPushIsolatesFailures245=== RUN TestPartSizeForNAR/5_TiB_S3_max_object246--- PASS: TestFileTokenEmpty (0.00s)247--- PASS: TestFileTokenReadsAndCaches (0.00s)248=== CONT TestFileTokenMissing2492000/01/01 00:00:00 ERROR Upload failed error="context canceled" count=22502026/11/12 00:44:35 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:38245/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:38245/api/uploads/complete2512026/11/12 00:44:35 ERROR Upload failed error="connection refused" count=202522026/11/12 00:44:37 ERROR Server seems unavailable, giving up on batch untried=172532026/11/12 00:44:35 ERROR Upload failed error=boom count=12542026/11/12 00:44:37 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"2552026/11/12 00:44:37 ERROR Upload failed error="bad path" count=3256--- PASS: TestScriptTokenScriptFails (1.03s)2572026/11/12 00:44:38 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:38245/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:38245/api/uploads/complete2582026/11/12 00:44:38 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"2592026/11/12 00:44:38 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:38245/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:38245/api/uploads/complete260--- PASS: TestScriptTokenBadJSON (1.02s)2612026/11/12 00:44:38 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"262--- PASS: TestStaticToken (0.00s)263--- PASS: TestStreamPushReportsSkippedPaths (1.21s)264=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object265=== CONT TestUploadMultipart_PartsInParallel266=== CONT TestFilterOversizedClosures2672026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:42561268=== CONT TestShellSplitErrors2692026/11/12 00:44:38 WARN Rate limiter enabled after throttle name=server-test rate=52702026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:45709271=== CONT TestStreamPushReportsEveryPath272=== RUN TestFilterOversizedClosures/no_limit_keeps_everything273=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything274--- PASS: TestClientSignaturesByStorePath (0.00s)275=== RUN TestPartSizeForNAR/capped_at_5_GiB276=== CONT TestCaseHackSuffix277=== CONT TestStreamPushBatchesUnderLoad278=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=false279=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=false280=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=true281=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=true282=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun283=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica2842026/11/12 00:44:38 WARN Rate limiter backed off name=server-test rate=5285=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped286=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped287=== CONT TestPathInfoCACompatibility/old_string_format_-_text288=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths2892026/11/12 00:44:38 WARN Rate limiter enabled after throttle name=server-test rate=52902026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:34985291=== CONT TestRegisterUploadedObjectReusesConnections292=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter293=== CONT TestUploadBuildLog_FileBodyReplayedOnRetry294=== CONT TestParsePathInfoJSON/invalid_JSON295=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512296=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter297=== CONT TestParsePathInfoJSON/whitespace_only298=== CONT TestRateLimiterFeedback/503_enables_limiter299=== PAUSE TestPartSizeForNAR/capped_at_5_GiB300=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive3012026/11/12 00:44:38 INFO Garbage collection started3022026/11/12 00:44:38 WARN Rate limiter backed off name=server-test rate=53032026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:34985304=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method3052026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:42561306=== CONT TestParsePathInfoJSON/Lix_format307=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error308=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI309=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon310=== CONT TestConvertHashToNix32/invalid_format311=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error312=== CONT TestGetStorePathHash/basename_without_hyphen_should_error313=== CONT TestConvertHashToNix32/already_Nix32_format314=== CONT TestEncodeNixBase32/empty_input315=== CONT TestDoWithRetry_FinalResponseBodyReadable316=== CONT TestShellSplit317=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary318=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part319=== RUN TestFilterOversizedClosures/all_closures_skipped320--- PASS: TestStreamPushStopsWhenCanceled (0.00s)321--- PASS: TestStreamPushGivesUpOnDeadServer (3.85s)322--- PASS: TestStreamPushReportsSignatures (3.85s)323--- PASS: TestStreamPushIsolatesFailures (0.17s)324--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (4.03s)325--- PASS: TestScriptTokenCachesUntilRefresh (4.03s)326--- PASS: TestFileTokenMissing (0.00s)327--- PASS: TestShellSplitErrors (0.00s)328--- PASS: TestScriptTokenEmptyToken (4.03s)329--- PASS: TestRegisterUploadedObject_BoundedAgainstSilentServer (5.03s)330--- PASS: TestDoServerRequestAttachesToken (5.05s)331--- PASS: TestRegisterUploadedObject_OutlivesPush (5.03s)332--- PASS: TestStreamPushReportsEveryPath (0.00s)333--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (5.04s)334--- PASS: TestShellSplit (0.00s)335--- PASS: TestUploadMultipart_ReturnsAfterDroppedBody (2.81s)336=== PAUSE TestFilterOversizedClosures/all_closures_skipped337=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=true338=== CONT TestPartSizeForNAR/zero_stays_at_minimum339=== CONT TestPartSizeForNAR/1_TiB340=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum341=== CONT TestPartSizeForNAR/capped_at_5_GiB342=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts343=== CONT TestPartSizeForNAR/5_TiB_S3_max_object344=== CONT TestPartSizeForNAR/small_stays_at_minimum345--- PASS: TestPartSizeForNAR (2.81s)346 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)347 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)348 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)349 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)350 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)351 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)352 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)353=== CONT TestFilterOversizedClosures/no_limit_keeps_everything354=== CONT TestFilterOversizedClosures/all_closures_skipped3552026/11/12 00:44:38 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50356=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped3572026/11/12 00:44:38 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000358--- PASS: TestFilterOversizedClosures (0.04s)359 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)360 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)361 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)362=== CONT TestParsePathInfoJSON/empty_input363--- PASS: TestParsePathInfoJSON (0.00s)364 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)365 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)366 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)367 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)368 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)369--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (2.85s)3702026/11/12 00:44:38 WARN Rate limiter enabled after throttle name=server-test rate=53712026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:384433722026/11/12 00:44:38 WARN Rate limiter backed off name=server-test rate=5373--- PASS: TestDoWithRetry_FinalResponseBodyReadable (0.04s)3742026/11/12 00:44:38 INFO Garbage collection started375--- PASS: TestStreamPushAnswersEveryLineWhenCanceled (4.04s)376=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=false377--- PASS: TestStreamPushBatchesUnderLoad (0.00s)378 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=true (0.00s)379 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=false (0.00s)380--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (0.05s)3812026/11/12 00:44:38 WARN Rate limiter enabled after throttle name=server-test rate=53822026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:398813832026/11/12 00:44:38 WARN Rate limiter backed off name=server-test rate=53842026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:42561385--- PASS: TestPathInfoHashCompatibility (0.04s)386 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)387 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)388 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)389 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)390--- PASS: TestGetStorePathHash (0.04s)391 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)392 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)393 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)394 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)395--- PASS: TestUploadMultipart_FailedProducerIsNotCompleted (0.00s)396 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part (0.04s)397 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary (0.05s)3982026/11/12 00:44:38 WARN Rate limiter enabled after throttle name=s3-test rate=5399--- PASS: TestConvertHashToNix32 (0.04s)400 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)401 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)402 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)4032026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:41493404--- PASS: TestRateLimiterFeedback (0.00s)405 --- PASS: TestRateLimiterFeedback/429_enables_limiter (5.03s)406 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.01s)407 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.05s)408 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.05s)409--- PASS: TestEncodeNixBase32 (0.04s)410 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)411 --- PASS: TestEncodeNixBase32/empty_input (0.00s)412--- PASS: TestPathInfoCACompatibility (0.00s)413 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)414 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)415 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)416 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)417 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.05s)418--- PASS: TestParsePathInfoJSONMultiplePaths (0.04s)419 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)420 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)421=== RUN TestSetClientTLSErrors/missing_cert_file422=== PAUSE TestSetClientTLSErrors/missing_cert_file423=== RUN TestSetClientTLSErrors/missing_key_file424=== PAUSE TestSetClientTLSErrors/missing_key_file425=== RUN TestSetClientTLSErrors/missing_ca_file426=== PAUSE TestSetClientTLSErrors/missing_ca_file427=== RUN TestSetClientTLSErrors/invalid_ca_file428=== PAUSE TestSetClientTLSErrors/invalid_ca_file429=== CONT TestSetClientTLSErrors/missing_cert_file430=== CONT TestSetClientTLSErrors/invalid_ca_file431--- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (0.05s)4322026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:42561433=== CONT TestSetClientTLSErrors/missing_ca_file434=== CONT TestSetClientTLSErrors/missing_key_file435--- PASS: TestSetClientTLSErrors (2.86s)436 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)437 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)438 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)439 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)4402026/11/12 00:44:38 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=04412026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:42561442=== RUN TestSetClientTLS/rejects_connection_without_client_cert443=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert444=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA445=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA446=== RUN TestSetClientTLS/preserves_debug_logging_transport447=== PAUSE TestSetClientTLS/preserves_debug_logging_transport448=== CONT TestSetClientTLS/rejects_connection_without_client_cert4492026/11/12 00:44:38 WARN Garbage collection finished on another replica. Its statistics are not available here450--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (0.06s)451=== CONT TestSetClientTLS/preserves_debug_logging_transport452=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA4532026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:425614542026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:42561455--- PASS: TestStreamPushRequestLine (4.10s)4562026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:425614572026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:425614582026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:425614592026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:425614602026/11/12 00:44:38 http: TLS handshake error from 127.0.0.1:54622: read tcp 127.0.0.1:40821->127.0.0.1:54622: use of closed network connection4612026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:42561462--- PASS: TestSetClientTLS (2.86s)463 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.02s)464 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.02s)465 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.04s)4662026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:425614672026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:425614682026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:425614692026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:425614702026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:425614712026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:425614722026/11/12 00:44:38 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:42561473--- PASS: TestRetryTimeoutOutlastsMaxRetries (5.15s)474--- PASS: TestDumpPathWriterError (5.14s)475--- PASS: TestRegisterUploadedObjectReusesConnections (0.11s)476--- PASS: TestDumpPathSingleFile (6.71s)477--- PASS: TestCaseHackSuffix (3.90s)478--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (8.94s)479--- PASS: TestDumpPathMatchesNix (6.73s)480--- PASS: TestUploadMultipart_PartsInParallel (4.47s)481PASS482Running server tests...483The files belonging to this database system will be owned by user "nixbld".484This user must also own the server process.485486The database cluster will be initialized with locale "C".487The default database encoding has accordingly been set to "SQL_ASCII".488The default text search configuration will be set to "english".489490Data page checksums are enabled.491492creating directory /build/postgres2115268374/data ... ok493creating subdirectories ... ok494selecting dynamic shared memory implementation ... posix495selecting default "max_connections" ... 100496selecting default "shared_buffers" ... 128MB497selecting default time zone ... UTC498creating configuration files ... ok499running bootstrap script ... ok500performing post-bootstrap initialization ... ok501syncing data to disk ... ok502503initdb: warning: enabling "trust" authentication for local connections504initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.505506Success. You can now start the database server using:507508 pg_ctl -D /build/postgres2115268374/data -l logfile start509510/build/postgres2115268374:5432 - no response511/build/postgres2115268374:5432 - no response512/build/postgres2115268374:5432 - no response513/build/postgres2115268374:5432 - no response514/build/postgres2115268374:5432 - no response515/build/postgres2115268374:5432 - no response516/build/postgres2115268374:5432 - no response517/build/postgres2115268374:5432 - no response518/build/postgres2115268374:5432 - no response519/build/postgres2115268374:5432 - no response520/build/postgres2115268374:5432 - no response521/build/postgres2115268374:5432 - no response522/build/postgres2115268374:5432 - no response523/build/postgres2115268374:5432 - no response524/build/postgres2115268374:5432 - no response525/build/postgres2115268374:5432 - no response526/build/postgres2115268374:5432 - no response527/build/postgres2115268374:5432 - no response528/build/postgres2115268374:5432 - no response529/build/postgres2115268374:5432 - no response530/build/postgres2115268374:5432 - no response5312026-11-12 00:45:05.070 UTC [130] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit5322026-11-12 00:45:05.071 UTC [130] LOG: listening on Unix socket "/build/postgres2115268374/.s.PGSQL.5432"5332026-11-12 00:45:05.086 UTC [157] LOG: database system was shut down at 2026-11-12 00:45:02 UTC5342026-11-12 00:45:05.094 UTC [130] LOG: database system is ready to accept connections535/build/postgres2115268374:5432 - accepting connections536{"timestamp":"2026-11-12T00:45:36.481423935Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a2dcc740-979b-467f-b998-5fa85376e789","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":20,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(196)"}537{"timestamp":"2026-11-12T00:45:36.605653001Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"fdc442f9-ca7d-4f0d-aeb3-9aa4ad2cd9fa","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(200)"}538{"timestamp":"2026-11-12T00:45:36.709123597Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8cad9f63-4172-4bd4-90ae-823b273d2a2e","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(201)"}539=== RUN TestAbortSucceeded540=== PAUSE TestAbortSucceeded541=== RUN TestService_AuthMiddleware542=== PAUSE TestService_AuthMiddleware543=== RUN TestService_AuthMiddleware_MTLSProxyHeader544=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader545=== RUN TestService_AuthMiddleware_MTLSBoundSubjects546=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects547=== RUN TestService_ReadAuthMiddleware548=== PAUSE TestService_ReadAuthMiddleware549=== RUN TestService_AuthMiddleware_OIDC550=== PAUSE TestService_AuthMiddleware_OIDC551=== RUN TestService_RequireScope_OIDC552=== PAUSE TestService_RequireScope_OIDC553=== RUN TestService_ReadScope_PublicByDefault554=== PAUSE TestService_ReadScope_PublicByDefault555=== RUN TestCacheConfigHandler556=== PAUSE TestCacheConfigHandler557=== RUN TestCacheStatsHandler558=== PAUSE TestCacheStatsHandler559=== RUN TestClientCADerivations560=== PAUSE TestClientCADerivations561=== RUN TestClientErrorHandling562=== PAUSE TestClientErrorHandling563=== RUN TestClientIntegration564=== PAUSE TestClientIntegration565=== RUN TestClientMultipleUploads566=== PAUSE TestClientMultipleUploads567=== RUN TestClientWithDependencies568=== PAUSE TestClientWithDependencies569=== RUN TestClientSharedPathCommittedMidPush570=== PAUSE TestClientSharedPathCommittedMidPush571=== RUN TestPinProtectsFromGC572=== PAUSE TestPinProtectsFromGC573=== RUN TestClientPushesUseOnePush574=== PAUSE TestClientPushesUseOnePush575=== RUN TestClientFallsBackToClosures576=== PAUSE TestClientFallsBackToClosures577=== RUN TestDeleteClosuresRacingPins578=== PAUSE TestDeleteClosuresRacingPins579=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock580=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock581=== RUN TestResolveDBConnectionString582=== PAUSE TestResolveDBConnectionString583=== RUN TestConnectWaitsForAPeerMigration584=== PAUSE TestConnectWaitsForAPeerMigration585=== RUN TestLeadElectsOneAndHandsOver586=== PAUSE TestLeadElectsOneAndHandsOver587=== RUN TestLeadIncumbentWinsAfterRestart5882026/11/12 00:45:38 INFO lead: acquired remote=192.0.2.1:1234 generation=15892026/11/12 00:45:39 INFO lead: released remote=192.0.2.1:1234 generation=15902026/11/12 00:45:39 INFO lead: acquired remote=192.0.2.1:1234 generation=25912026/11/12 00:45:40 INFO lead: released remote=192.0.2.1:1234 generation=2592--- PASS: TestLeadIncumbentWinsAfterRestart (3.40s)593=== RUN TestLeadUnannouncedStandbyYieldsToIncumbent594=== PAUSE TestLeadUnannouncedStandbyYieldsToIncumbent595=== RUN TestLeadIncumbentAnnouncesAtOnce596=== PAUSE TestLeadIncumbentAnnouncesAtOnce597=== RUN TestLeadGenerationGrowsWithEveryGrant598=== PAUSE TestLeadGenerationGrowsWithEveryGrant599=== RUN TestLeadEndsOnShutdown600=== PAUSE TestLeadEndsOnShutdown601=== RUN TestLeadEndsWhenItsConnectionHangs6022026/11/12 00:45:40 INFO lead: acquired remote=192.0.2.1:1234 generation=16032026-11-12 00:45:42.162 UTC [405] FATAL: terminating connection due to administrator command6042026/11/12 00:45:42 INFO lead: acquired remote=192.0.2.1:1234 generation=26052026/11/12 00:45:43 WARN lead: lock connection lost error="timeout: context deadline exceeded"6062026/11/12 00:45:43 INFO lead: released remote=192.0.2.1:1234 generation=16072026/11/12 00:45:43 INFO lead: released remote=192.0.2.1:1234 generation=2608--- PASS: TestLeadEndsWhenItsConnectionHangs (3.43s)609=== RUN TestGCAdvisoryLockBlocksConcurrentRun610=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun611=== RUN TestGCBugBareHashReferences612=== PAUSE TestGCBugBareHashReferences613=== RUN TestGCMetrics614=== PAUSE TestGCMetrics615=== RUN TestPushDedupSurvivesConcurrentGC616=== PAUSE TestPushDedupSurvivesConcurrentGC617=== RUN TestDeduplicatedObjectsRecordedAsPending618=== PAUSE TestDeduplicatedObjectsRecordedAsPending619=== RUN TestGCSweepSkipsPendingObjects620=== PAUSE TestGCSweepSkipsPendingObjects621=== RUN TestTombstonedObjectOfferedWithoutWaiting622=== PAUSE TestTombstonedObjectOfferedWithoutWaiting623=== RUN TestGCSweepDeliversEachKeyOnce624=== PAUSE TestGCSweepDeliversEachKeyOnce625=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection626=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection627=== RUN TestForceGCDuringPushOffersSweptObject628=== PAUSE TestForceGCDuringPushOffersSweptObject629=== RUN TestFailedS3DeleteKeepsTombstone630=== PAUSE TestFailedS3DeleteKeepsTombstone631=== RUN TestSweepSkipsRowsLockedByPush632=== PAUSE TestSweepSkipsRowsLockedByPush633=== RUN TestRegisterCompletedObjectNeedsPendingRow634=== PAUSE TestRegisterCompletedObjectNeedsPendingRow635=== RUN TestGetExistingObjectsLocksRows636=== PAUSE TestGetExistingObjectsLocksRows637=== RUN TestSweepRechecksPendingAfterLocking638=== PAUSE TestSweepRechecksPendingAfterLocking639=== RUN TestPushWaitsForSweepHoldingRow640=== PAUSE TestPushWaitsForSweepHoldingRow641=== RUN TestCommitRacingPendingCleanupKeepsObjects642=== PAUSE TestCommitRacingPendingCleanupKeepsObjects643=== RUN TestGCEndsOnShutdown644=== PAUSE TestGCEndsOnShutdown645=== RUN TestGCTaskStore_StartNew646=== PAUSE TestGCTaskStore_StartNew647=== RUN TestGCTaskStore_DeduplicateSameParams648=== PAUSE TestGCTaskStore_DeduplicateSameParams649=== RUN TestGCTaskStore_ConflictDifferentParams650=== PAUSE TestGCTaskStore_ConflictDifferentParams651=== RUN TestGCTaskStore_GetEmpty652=== PAUSE TestGCTaskStore_GetEmpty653=== RUN TestGCTaskStore_GetReturnsLatest654=== PAUSE TestGCTaskStore_GetReturnsLatest655=== RUN TestGCTaskStore_CompletedAllowsNewTask656=== PAUSE TestGCTaskStore_CompletedAllowsNewTask657=== RUN TestGCTaskStore_PhaseUpdates658=== PAUSE TestGCTaskStore_PhaseUpdates659=== RUN TestGCTaskStore_Fail660=== PAUSE TestGCTaskStore_Fail661=== RUN TestGracefulShutdownDrainsInflight662=== PAUSE TestGracefulShutdownDrainsInflight663=== RUN TestService_healthCheckHandler664=== PAUSE TestService_healthCheckHandler665=== RUN TestService_readinessHandler666=== PAUSE TestService_readinessHandler667=== RUN TestGenerateLandingPage668=== PAUSE TestGenerateLandingPage669=== RUN TestEnsureAbortUploadsRule670=== PAUSE TestEnsureAbortUploadsRule671=== RUN TestCacheConfigHandlerMaxNarSize672=== PAUSE TestCacheConfigHandlerMaxNarSize673=== RUN TestCreatePendingClosureRejectsOversizedNAR674=== PAUSE TestCreatePendingClosureRejectsOversizedNAR675=== RUN TestNARDeduplicationMetadataUploadBug676=== PAUSE TestNARDeduplicationMetadataUploadBug677=== RUN TestMetricsInventory678=== PAUSE TestMetricsInventory679=== RUN TestMetricsCommit680=== PAUSE TestMetricsCommit681=== RUN TestService_NativeMTLS682=== PAUSE TestService_NativeMTLS683=== RUN TestServerTLSConfig684=== PAUSE TestServerTLSConfig685=== RUN TestMultipartCleanup686=== PAUSE TestMultipartCleanup687=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded688=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded689=== RUN TestPendingCleanupUsesOneCutoff690=== PAUSE TestPendingCleanupUsesOneCutoff691=== RUN TestObjectStatsTrigger692=== PAUSE TestObjectStatsTrigger693=== RUN TestObjectStatsUpdatedOncePerCommit694=== PAUSE TestObjectStatsUpdatedOncePerCommit695=== RUN TestOrphanedObjectsGC696=== PAUSE TestOrphanedObjectsGC697=== RUN TestOrphanedObjectsGCStressTest698=== PAUSE TestOrphanedObjectsGCStressTest699=== RUN TestCreatePin_ReservedPins700=== PAUSE TestCreatePin_ReservedPins701=== RUN TestPresentReportsOnlyClosureRoots702=== PAUSE TestPresentReportsOnlyClosureRoots703=== RUN TestPresentRefreshesOnlyStaleClosures704=== PAUSE TestPresentRefreshesOnlyStaleClosures705=== RUN TestPresentNotReportedWhileGCDeletesClosure706=== PAUSE TestPresentNotReportedWhileGCDeletesClosure707=== RUN TestParseSingleRange708=== PAUSE TestParseSingleRange709=== RUN TestProxyHeadersOnlyTrustedOnSocket7102026/11/12 00:45:45 INFO Starting HTTP server address=proxy.sock7112026/11/12 00:45:45 INFO Starting HTTP server address=127.0.0.1:440097122026/11/12 00:45:45 WARN mTLS auth: subject not in bound subjects subject="CN=someone"7132026/11/12 00:45:45 INFO Shutdown signal received, draining in-flight requests timeout=10s714--- PASS: TestProxyHeadersOnlyTrustedOnSocket (2.16s)715=== RUN TestIsValidCachePath716=== PAUSE TestIsValidCachePath717=== RUN TestReadProxyNarinfo718=== PAUSE TestReadProxyNarinfo719=== RUN TestReadProxyNarinfoAlreadyDecompressed720=== PAUSE TestReadProxyNarinfoAlreadyDecompressed721=== RUN TestReadProxyContentNegotiation722=== PAUSE TestReadProxyContentNegotiation723=== RUN TestReadProxyNarStreaming724=== PAUSE TestReadProxyNarStreaming725=== RUN TestReadProxy404726=== PAUSE TestReadProxy404727=== RUN TestReadProxyInvalidPath728=== PAUSE TestReadProxyInvalidPath729=== RUN TestReadProxyHead730=== PAUSE TestReadProxyHead731=== RUN TestReadProxyOutlastsServerWriteTimeout732=== PAUSE TestReadProxyOutlastsServerWriteTimeout733=== RUN TestReadProxyConditionalGet734=== PAUSE TestReadProxyConditionalGet735=== RUN TestReadProxyRootRedirectsToIndexHTML736=== PAUSE TestReadProxyRootRedirectsToIndexHTML737=== RUN TestReadProxyDisabled738=== PAUSE TestReadProxyDisabled739=== RUN TestReadRedirectNar740=== PAUSE TestReadRedirectNar741=== RUN TestReadRedirectKeepsNarinfoProxied742=== PAUSE TestReadRedirectKeepsNarinfoProxied743=== RUN TestReadProxyRangeRequest744=== PAUSE TestReadProxyRangeRequest745=== RUN TestReadRedirectUsesPublicS3URL746=== PAUSE TestReadRedirectUsesPublicS3URL747=== RUN TestPush_OverlappingRootsStoreOneRowPerKey748=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey749=== RUN TestPush_CompleteCommitsEveryRoot750=== PAUSE TestPush_CompleteCommitsEveryRoot751=== RUN TestPush_RepeatedRootCommitsOnce752=== PAUSE TestPush_RepeatedRootCommitsOnce753=== RUN TestPush_CompleteWaitsForCleanupOfThePush754=== PAUSE TestPush_CompleteWaitsForCleanupOfThePush755=== RUN TestPush_RootsAreLockedInByteOrder756=== PAUSE TestPush_RootsAreLockedInByteOrder757=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit758=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit759=== RUN TestPush_RejectsBadRequests760=== PAUSE TestPush_RejectsBadRequests761=== RUN TestPush_SignsNarinfosOfItsPendingObjects762=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects763=== RUN TestCommitCanBeRepeatedUntilCleanup764=== PAUSE TestCommitCanBeRepeatedUntilCleanup765=== RUN TestSharedNarBothUploadsComplete766=== PAUSE TestSharedNarBothUploadsComplete767=== RUN TestCompleteMultipartUpload_ErrorButObjectExists768=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists769=== RUN TestCompletedNarNotReofferedAcrossClosures770=== PAUSE TestCompletedNarNotReofferedAcrossClosures771=== RUN TestPresignedUploadRegisteredBeforeCommit772=== PAUSE TestPresignedUploadRegisteredBeforeCommit773=== RUN TestService_Rustfstest774=== PAUSE TestService_Rustfstest775=== RUN TestParseSize776=== PAUSE TestParseSize777=== RUN TestSkippedUploadsHandler778=== PAUSE TestSkippedUploadsHandler779=== RUN TestSystemdListenerNotActivated780--- PASS: TestSystemdListenerNotActivated (0.00s)781=== RUN TestWatchdogBeatsWhenHealthy782--- PASS: TestWatchdogBeatsWhenHealthy (0.02s)783=== RUN TestWatchdogSkipsWhenUnhealthy7842026/11/12 00:45:46 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7852026/11/12 00:45:46 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7862026/11/12 00:45:46 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7872026/11/12 00:45:46 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7882026/11/12 00:45:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7892026/11/12 00:45:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7902026/11/12 00:45:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7912026/11/12 00:45:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7922026/11/12 00:45:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7932026/11/12 00:45:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"794--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)795=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle796=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle797=== RUN TestProxyWriteTimeout798=== PAUSE TestProxyWriteTimeout799=== RUN TestPendingClosureWriteTimeout800=== PAUSE TestPendingClosureWriteTimeout801=== RUN TestIsValidUploadKey802=== PAUSE TestIsValidUploadKey803=== RUN TestUploadHandlersRejectInvalidKeys804=== PAUSE TestUploadHandlersRejectInvalidKeys805=== RUN TestUploadHandlersRejectOversizedBody806=== PAUSE TestUploadHandlersRejectOversizedBody807=== RUN TestService_cleanupPendingClosuresHandler808=== PAUSE TestService_cleanupPendingClosuresHandler809=== RUN TestService_createPendingClosureHandler810=== PAUSE TestService_createPendingClosureHandler811=== RUN TestService_verifyS3Integrity812=== PAUSE TestService_verifyS3Integrity813=== RUN TestService_verifyS3ReuploadsStaleNarinfo814=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo815=== RUN TestCompleteMultipartUnregistered816=== PAUSE TestCompleteMultipartUnregistered817=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT818=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT819=== CONT TestPendingClosureWriteTimeout820=== CONT TestPush_CompleteCommitsEveryRoot821=== CONT TestService_verifyS3Integrity822=== CONT TestProxyWriteTimeout823=== RUN TestProxyWriteTimeout/narinfo824=== PAUSE TestProxyWriteTimeout/narinfo825=== RUN TestProxyWriteTimeout/1_GiB_nar826=== PAUSE TestProxyWriteTimeout/1_GiB_nar827=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle828=== CONT TestSkippedUploadsHandler829=== CONT TestParseSize830=== CONT TestService_Rustfstest831=== CONT TestPresignedUploadRegisteredBeforeCommit832=== CONT TestCompletedNarNotReofferedAcrossClosures833=== CONT TestCompleteMultipartUpload_ErrorButObjectExists834=== CONT TestSharedNarBothUploadsComplete835=== CONT TestCommitCanBeRepeatedUntilCleanup836=== CONT TestPush_SignsNarinfosOfItsPendingObjects837=== CONT TestPush_RejectsBadRequests838=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit839=== CONT TestPush_RootsAreLockedInByteOrder840=== CONT TestService_createPendingClosureHandler841=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT842=== CONT TestCompleteMultipartUnregistered843=== CONT TestPush_CompleteWaitsForCleanupOfThePush844=== CONT TestPush_RepeatedRootCommitsOnce845=== CONT TestService_verifyS3ReuploadsStaleNarinfo846=== CONT TestAbortSucceeded847=== RUN TestPendingClosureWriteTimeout/empty848=== RUN TestProxyWriteTimeout/10_GiB_nar849=== PAUSE TestProxyWriteTimeout/10_GiB_nar850=== RUN TestProxyWriteTimeout/unknown_size851=== PAUSE TestProxyWriteTimeout/unknown_size852=== RUN TestCommitCanBeRepeatedUntilCleanup/pending_closure853=== PAUSE TestCommitCanBeRepeatedUntilCleanup/pending_closure854=== RUN TestCommitCanBeRepeatedUntilCleanup/push855=== PAUSE TestCommitCanBeRepeatedUntilCleanup/push856--- PASS: TestParseSize (0.00s)857=== CONT TestPush_OverlappingRootsStoreOneRowPerKey8582026/11/12 00:45:47 INFO Client skipped oversized paths paths=3 nar_bytes=5000000000859=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed860=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed861=== RUN TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed862=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed863=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost864=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost865=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed866=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed867=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed868=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed869--- PASS: TestSkippedUploadsHandler (0.01s)870=== CONT TestReadRedirectKeepsNarinfoProxied871=== CONT TestReadRedirectUsesPublicS3URL872=== CONT TestGCTaskStore_ConflictDifferentParams873--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)874=== CONT TestReadRedirectNar875=== CONT TestReadProxyRangeRequest876=== PAUSE TestPendingClosureWriteTimeout/empty877=== RUN TestPendingClosureWriteTimeout/400_objects878=== PAUSE TestPendingClosureWriteTimeout/400_objects879=== RUN TestPendingClosureWriteTimeout/670k_objects880=== PAUSE TestPendingClosureWriteTimeout/670k_objects881=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline882=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline883=== CONT TestReadProxyDisabled884--- PASS: TestAbortSucceeded (0.00s)885=== CONT TestReadProxyRootRedirectsToIndexHTML8862026/11/12 00:45:51 INFO Received uploads request method=POST path=/api/pending_closures8872026/11/12 00:45:51 INFO Received uploads request method=POST path=/api/pending_closures8882026/11/12 00:45:51 INFO Received push request method=POST path=/api/pushes8892026/11/12 00:45:51 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign8902026/11/12 00:45:51 INFO Signed narinfos id=1 count=18912026/11/12 00:45:51 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign8922026/11/12 00:45:51 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst8932026/11/12 00:45:51 INFO Received uploads request method=POST path=/api/pending_closures8942026/11/12 00:45:51 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign895--- PASS: TestPresignedUploadRegisteredBeforeCommit (4.25s)896=== CONT TestReadProxyConditionalGet8972026/11/12 00:45:51 ERROR Failed to sign narinfos signer="index 1" error="signing key cannot be nil"8982026/11/12 00:45:51 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign8992026/11/12 00:45:51 ERROR Failed to sign narinfos signer=niks3-test-1 error="signer failed"9002026/11/12 00:45:51 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9012026/11/12 00:45:51 ERROR Signer returned unexpected signature count signer=niks3-test-1 got=0 want=19022026/11/12 00:45:51 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9032026/11/12 00:45:51 ERROR Signer returned unexpected object key signer=niks3-test-1 object_key=unexpected.narinfo904--- PASS: TestPush_SignsNarinfosOfItsPendingObjects (4.25s)905=== CONT TestReadProxyOutlastsServerWriteTimeout9062026/11/12 00:45:51 INFO Received uploads request method=POST path=/api/pending_closures907--- PASS: TestReadProxyRangeRequest (1.59s)908=== CONT TestReadProxyHead9092026/11/12 00:45:51 INFO Received push request method=POST path=/api/pushes9102026/11/12 00:45:51 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9112026/11/12 00:45:51 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=YWUyZDIzYjctYWY5Zi00NTE4LThmNTctMWNmYWNlN2VjZjgzLjcwOGVhZjEwLTlhNGUtNGE1Yy1hMDYxLTRlNDgwMGY2NzIzY3gxNzk0NDQ0MzUxMzQyMTEwODM09122026/11/12 00:45:51 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=YWUyZDIzYjctYWY5Zi00NTE4LThmNTctMWNmYWNlN2VjZjgzLjcwOGVhZjEwLTlhNGUtNGE1Yy1hMDYxLTRlNDgwMGY2NzIzY3gxNzk0NDQ0MzUxMzQyMTEwODM0 parts=1913--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (4.30s)914=== CONT TestUploadHandlersRejectOversizedBody9152026/11/12 00:45:51 INFO Received uploads request method=POST path=/api/pending_closures9162026/11/12 00:45:51 INFO Received complete push request method=POST path=/api/pushes/1/complete917--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (1.67s)918=== CONT TestReadProxyInvalidPath9192026/11/12 00:45:51 INFO Received uploads request method=POST path=/api/pending_closures9202026/11/12 00:45:51 INFO Received uploads request method=POST path=/api/pending_closures9212026/11/12 00:45:51 INFO Received uploads request method=POST path=/api/pending_closures9222026/11/12 00:45:51 INFO Received complete multipart upload request method=POST path=/api/multipart/complete923--- PASS: TestReadProxyDisabled (1.71s)924=== CONT TestService_cleanupPendingClosuresHandler9252026/11/12 00:45:51 INFO Received uploads request method=POST path=/api/pending_closures9262026/11/12 00:45:51 INFO Received push request method=POST path=/api/pushes927--- PASS: TestReadProxyRootRedirectsToIndexHTML (1.84s)928=== CONT TestReadProxy4049292026/11/12 00:45:51 INFO Received uploads request method=POST path=/api/pending_closures930--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (4.52s)931=== CONT TestUploadHandlersRejectInvalidKeys932=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info933=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info934=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal935=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal936=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key937=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key938=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key939=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key940=== CONT TestReadProxyNarStreaming941=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure942=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure943=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart944=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart945=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts946=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts947=== CONT TestReadProxyContentNegotiation9482026/11/12 00:45:51 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9492026/11/12 00:45:51 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst950--- PASS: TestCompleteMultipartUnregistered (1.89s)951=== CONT TestLeadUnannouncedStandbyYieldsToIncumbent9522026/11/12 00:45:51 INFO Received uploads request method=POST path=/api/pending_closures9532026/11/12 00:45:51 INFO Completed push id=1 objects=4 duration=242.80469ms9542026/11/12 00:45:51 INFO Received push request method=POST path=/api/pushes955--- PASS: TestReadRedirectNar (1.97s)956=== CONT TestReadProxyNarinfoAlreadyDecompressed9572026/11/12 00:45:51 INFO Aborted multipart uploads count=0 kept=09582026/11/12 00:45:51 WARN Force mode enabled - objects will be deleted immediately without grace period959=== RUN TestPush_RejectsBadRequests/root_not_in_objects960=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects961=== RUN TestPush_RejectsBadRequests/no_roots962=== PAUSE TestPush_RejectsBadRequests/no_roots963=== RUN TestPush_RejectsBadRequests/no_objects964=== PAUSE TestPush_RejectsBadRequests/no_objects965=== RUN TestPush_RejectsBadRequests/bad_root966=== PAUSE TestPush_RejectsBadRequests/bad_root967=== CONT TestGCTaskStore_DeduplicateSameParams968--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)969=== CONT TestReadProxyNarinfo9702026/11/12 00:45:51 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=09712026/11/12 00:45:51 INFO Vacuumed table table=pending_closures9722026/11/12 00:45:51 INFO Vacuumed table table=commit_receipts9732026/11/12 00:45:51 INFO Vacuumed table table=pending_objects9742026/11/12 00:45:51 INFO Vacuumed table table=multipart_uploads975--- PASS: TestReadRedirectUsesPublicS3URL (2.18s)976=== CONT TestGCTaskStore_StartNew977--- PASS: TestGCTaskStore_StartNew (0.00s)978=== CONT TestIsValidCachePath979=== RUN TestIsValidCachePath/narinfo980=== PAUSE TestIsValidCachePath/narinfo981=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars982=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars983=== RUN TestIsValidCachePath/nar_zst984=== PAUSE TestIsValidCachePath/nar_zst985=== RUN TestIsValidCachePath/nar_xz986=== PAUSE TestIsValidCachePath/nar_xz987=== RUN TestIsValidCachePath/nar_bz2988=== PAUSE TestIsValidCachePath/nar_bz2989=== RUN TestIsValidCachePath/nar_uncompressed990=== PAUSE TestIsValidCachePath/nar_uncompressed991=== RUN TestIsValidCachePath/ls992=== PAUSE TestIsValidCachePath/ls993=== RUN TestIsValidCachePath/log994=== PAUSE TestIsValidCachePath/log995=== RUN TestIsValidCachePath/realisation996=== PAUSE TestIsValidCachePath/realisation997=== RUN TestIsValidCachePath/nix-cache-info998=== PAUSE TestIsValidCachePath/nix-cache-info999=== RUN TestIsValidCachePath/index.html1000=== PAUSE TestIsValidCachePath/index.html1001=== RUN TestIsValidCachePath/traversal_parent1002=== PAUSE TestIsValidCachePath/traversal_parent1003=== RUN TestIsValidCachePath/traversal_in_middle1004=== PAUSE TestIsValidCachePath/traversal_in_middle1005=== RUN TestIsValidCachePath/invalid_char_e1006=== PAUSE TestIsValidCachePath/invalid_char_e1007=== RUN TestIsValidCachePath/invalid_char_u1008=== PAUSE TestIsValidCachePath/invalid_char_u1009=== RUN TestIsValidCachePath/random_path10102026/11/12 00:45:51 INFO Vacuumed table table=closures1011=== PAUSE TestIsValidCachePath/random_path1012=== RUN TestIsValidCachePath/empty1013=== PAUSE TestIsValidCachePath/empty1014=== RUN TestIsValidCachePath/leading_slash1015=== PAUSE TestIsValidCachePath/leading_slash1016=== RUN TestIsValidCachePath/wrong_extension1017=== PAUSE TestIsValidCachePath/wrong_extension1018=== RUN TestIsValidCachePath/short_hash1019=== PAUSE TestIsValidCachePath/short_hash1020=== CONT TestGCEndsOnShutdown1021=== RUN TestGCEndsOnShutdown/before_the_run1022=== PAUSE TestGCEndsOnShutdown/before_the_run1023=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1024=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1025=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1026=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1027=== CONT TestParseSingleRange1028=== RUN TestParseSingleRange/none1029=== PAUSE TestParseSingleRange/none1030=== RUN TestParseSingleRange/unknown_unit1031=== PAUSE TestParseSingleRange/unknown_unit1032=== RUN TestParseSingleRange/multi-range_ignored1033=== PAUSE TestParseSingleRange/multi-range_ignored1034=== RUN TestParseSingleRange/malformed_no_dash1035=== PAUSE TestParseSingleRange/malformed_no_dash1036=== RUN TestParseSingleRange/malformed_both_empty1037=== PAUSE TestParseSingleRange/malformed_both_empty1038=== RUN TestParseSingleRange/malformed_end_before_start1039=== PAUSE TestParseSingleRange/malformed_end_before_start1040=== RUN TestParseSingleRange/closed1041=== PAUSE TestParseSingleRange/closed1042=== RUN TestParseSingleRange/open-ended1043=== PAUSE TestParseSingleRange/open-ended1044=== RUN TestParseSingleRange/end_clamped_to_size1045=== PAUSE TestParseSingleRange/end_clamped_to_size1046=== RUN TestParseSingleRange/suffix1047=== PAUSE TestParseSingleRange/suffix1048=== RUN TestParseSingleRange/suffix_exceeds_size1049=== PAUSE TestParseSingleRange/suffix_exceeds_size1050=== RUN TestParseSingleRange/single_byte1051=== PAUSE TestParseSingleRange/single_byte1052=== RUN TestParseSingleRange/start_past_EOF1053=== PAUSE TestParseSingleRange/start_past_EOF1054=== RUN TestParseSingleRange/start_far_past_EOF1055=== PAUSE TestParseSingleRange/start_far_past_EOF1056=== CONT TestCommitRacingPendingCleanupKeepsObjects10572026/11/12 00:45:51 INFO Vacuumed table table=objects10582026/11/12 00:45:51 INFO Received complete push request method=POST path=/api/pushes/2/complete10592026/11/12 00:45:51 INFO Completed push id=2 objects=4 duration=5.077691ms1060--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (2.20s)1061=== CONT TestPresentNotReportedWhileGCDeletesClosure1062=== RUN TestPresentNotReportedWhileGCDeletesClosure/stale1063=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/stale1064=== RUN TestPresentNotReportedWhileGCDeletesClosure/fresh1065=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/fresh1066=== CONT TestPushWaitsForSweepHoldingRow1067--- PASS: TestReadRedirectKeepsNarinfoProxied (4.86s)1068=== CONT TestPresentRefreshesOnlyStaleClosures1069--- PASS: TestService_Rustfstest (4.88s)1070=== CONT TestSweepRechecksPendingAfterLocking10712026/11/12 00:45:52 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10722026/11/12 00:45:52 INFO Received push request method=POST path=/api/pushes10732026/11/12 00:45:52 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=YWUyZDIzYjctYWY5Zi00NTE4LThmNTctMWNmYWNlN2VjZjgzLmIxMzVmYzcwLTBmOTEtNDdhNC04NmVhLTViMjI2NWQxNjhjNHgxNzk0NDQ0MzUxNDcxOTY1MDEz parts=1010742026/11/12 00:45:52 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10752026/11/12 00:45:52 INFO Received complete push request method=POST path=/api/pushes/1/complete10762026/11/12 00:45:52 INFO Completed upload id=1 objects=2 duration=4.753467ms10772026/11/12 00:45:52 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000010782026/11/12 00:45:52 INFO Received uploads request method=POST path=/api/pending_closures10792026/11/12 00:45:52 INFO Received push request method=POST path=/api/pushes10802026/11/12 00:45:52 INFO Starting cleanup of old closures method=DELETE path=/api/closures10812026/11/12 00:45:52 INFO Completed push id=1 objects=4 duration=9.651577ms1082--- PASS: TestPush_RepeatedRootCommitsOnce (2.32s)1083=== CONT TestPresentReportsOnlyClosureRoots10842026/11/12 00:45:52 INFO Aborted multipart uploads count=0 kept=010852026/11/12 00:45:52 INFO Received push request method=POST path=/api/pushes10862026/11/12 00:45:52 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10872026/11/12 00:45:52 INFO Received uploads request method=POST path=/api/pending_closures10882026/11/12 00:45:52 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=YWUyZDIzYjctYWY5Zi00NTE4LThmNTctMWNmYWNlN2VjZjgzLmFiNTJiYTMyLWNmYTEtNDA0MS04MTMxLWY4ZDM4OTEyMTU5ZngxNzk0NDQ0MzUxNTQ3MDA3NTY0 parts=1210892026/11/12 00:45:52 INFO Received uploads request method=POST path=/api/pending_closures1090--- PASS: TestCompletedNarNotReofferedAcrossClosures (5.08s)1091=== CONT TestGetExistingObjectsLocksRows10922026/11/12 00:45:52 INFO Received push request method=POST path=/api/pushes10932026/11/12 00:45:52 INFO Received complete push request method=POST path=/api/pushes/1/complete1094--- PASS: TestReadProxyHead (0.91s)1095=== CONT TestCreatePin_ReservedPins10962026/11/12 00:45:52 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1097--- PASS: TestReadProxyInvalidPath (0.86s)1098=== CONT TestRegisterCompletedObjectNeedsPendingRow10992026/11/12 00:45:52 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=YWUyZDIzYjctYWY5Zi00NTE4LThmNTctMWNmYWNlN2VjZjgzLjU2ZTQxNDkzLTQ4OGItNDZmZS1hYzJiLTdkNWNmNWRmMGEyNXgxNzk0NDQ0MzUxNjU3ODM3MDM0 parts=1211002026/11/12 00:45:52 INFO Received cleanup request method=DELETE path=/api/pending_closures11012026/11/12 00:45:52 INFO Aborted multipart uploads count=0 kept=011022026/11/12 00:45:52 INFO Received uploads request method=POST path=/api/pending_closures11032026/11/12 00:45:52 INFO Received cleanup request method=DELETE path=/api/pending_closures11042026/11/12 00:45:52 INFO Aborted multipart uploads count=1 kept=011052026/11/12 00:45:52 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete11062026-11-12 00:45:52.373 UTC [520] ERROR: Closure does not exist: id=111072026-11-12 00:45:52.373 UTC [520] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 17 at RAISE11082026-11-12 00:45:52.373 UTC [520] STATEMENT: -- name: CommitPendingClosure :one1109 SELECT commit_pending_closure($1::bigint)::bigint1110 1111--- PASS: TestService_cleanupPendingClosuresHandler (0.88s)1112=== CONT TestOrphanedObjectsGCStressTest11132026/11/12 00:45:52 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:33807/oidc1114--- PASS: TestReadProxyConditionalGet (1.06s)1115=== CONT TestSweepSkipsRowsLockedByPush11162026-11-12 00:45:52.449 UTC [571] ERROR: Push does not exist: id=111172026-11-12 00:45:52.449 UTC [571] CONTEXT: PL/pgSQL function commit_push(bigint) line 15 at RAISE11182026-11-12 00:45:52.449 UTC [571] STATEMENT: SELECT commit_push($1)1119--- PASS: TestPush_CompleteWaitsForCleanupOfThePush (2.68s)1120=== CONT TestOrphanedObjectsGC11212026-11-12 00:45:52.464 UTC [584] ERROR: could not obtain lock on row in relation "closures"11222026-11-12 00:45:52.464 UTC [584] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT11232026-11-12 00:45:52.465 UTC [584] ERROR: could not obtain lock on row in relation "closures"11242026-11-12 00:45:52.465 UTC [584] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT11252026-11-12 00:45:52.465 UTC [584] ERROR: could not obtain lock on row in relation "closures"11262026-11-12 00:45:52.465 UTC [584] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT11272026-11-12 00:45:52.466 UTC [584] ERROR: could not obtain lock on row in relation "closures"11282026-11-12 00:45:52.466 UTC [584] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT11292026-11-12 00:45:52.466 UTC [584] ERROR: could not obtain lock on row in relation "closures"11302026-11-12 00:45:52.466 UTC [584] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT11312026/11/12 00:45:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011322026/11/12 00:45:52 INFO Completed push id=1 objects=6 duration=213.526617ms1133--- PASS: TestPush_CompleteCommitsEveryRoot (5.35s)1134=== CONT TestFailedS3DeleteKeepsTombstone11352026/11/12 00:45:52 INFO Vacuumed table table=pending_closures11362026/11/12 00:45:52 INFO Vacuumed table table=commit_receipts1137--- PASS: TestPush_RootsAreLockedInByteOrder (2.70s)1138=== CONT TestObjectStatsUpdatedOncePerCommit11392026/11/12 00:45:52 INFO Vacuumed table table=pending_objects11402026/11/12 00:45:52 INFO Vacuumed table table=multipart_uploads11412026/11/12 00:45:52 INFO Vacuumed table table=closures11422026/11/12 00:45:52 INFO Vacuumed table table=objects11432026/11/12 00:45:52 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001144--- PASS: TestService_createPendingClosureHandler (2.78s)1145=== CONT TestForceGCDuringPushOffersSweptObject1146=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows1147=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows1148=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check1149=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check1150=== CONT TestObjectStatsTrigger1151--- PASS: TestReadProxyNarStreaming (0.93s)1152=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection1153--- PASS: TestReadProxyContentNegotiation (0.92s)1154=== CONT TestPendingCleanupUsesOneCutoff11552026/11/12 00:45:52 INFO lead: acquired remote=192.0.2.1:1234 generation=111562026/11/12 00:45:52 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1157--- PASS: TestReadProxy404 (1.01s)1158=== CONT TestGCSweepDeliversEachKeyOnce1159--- PASS: TestReadProxyNarinfoAlreadyDecompressed (0.88s)1160=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded11612026/11/12 00:45:52 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=YWUyZDIzYjctYWY5Zi00NTE4LThmNTctMWNmYWNlN2VjZjgzLjRhZTgyZTRmLTdjYzctNDZkYy05ZGYxLWFkN2M0NTAzMDk5ZXgxNzk0NDQ0MzUyMTg0MTkyMDYx parts=1011622026/11/12 00:45:52 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete11632026/11/12 00:45:52 INFO Completed upload id=1 objects=2 duration=22.347484ms11642026/11/12 00:45:52 INFO Received uploads request method=POST path=/api/pending_closures11652026/11/12 00:45:52 INFO Received uploads request method=POST path=/api/pending_closures11662026/11/12 00:45:52 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo11672026/11/12 00:45:52 WARN Found objects in DB but missing from S3, will re-upload count=11168--- PASS: TestService_verifyS3Integrity (5.60s)1169=== CONT TestTombstonedObjectOfferedWithoutWaiting11702026/11/12 00:45:52 ERROR Failed to decode narinfo key=3hcdxyjf9yiq7qf3i4548drb6sjmwa1v.narinfo limit=16777216 error=<nil>1171--- PASS: TestReadProxyNarinfo (0.90s)1172=== CONT TestMultipartCleanup11732026/11/12 00:45:52 INFO Aborted multipart uploads count=0 kept=011742026/11/12 00:45:52 INFO Aborted multipart uploads count=0 kept=011752026/11/12 00:45:52 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1176--- PASS: TestGetExistingObjectsLocksRows (0.67s)1177=== CONT TestGCSweepSkipsPendingObjects11782026/11/12 00:45:52 WARN Force mode enabled - objects will be deleted immediately without grace period11792026/11/12 00:45:52 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=YWUyZDIzYjctYWY5Zi00NTE4LThmNTctMWNmYWNlN2VjZjgzLjRiMDY3MDRhLWU1ODEtNDMxNS1hODIyLTU3ZDdhNjUyNDg5NHgxNzk0NDQ0MzUxNjkyNDg5Mzgw parts=1211802026/11/12 00:45:52 WARN Force mode enabled - objects will be deleted immediately without grace period1181--- PASS: TestSharedNarBothUploadsComplete (5.75s)1182=== CONT TestServerTLSConfig1183=== RUN TestServerTLSConfig/no_client_CA1184=== PAUSE TestServerTLSConfig/no_client_CA1185=== RUN TestServerTLSConfig/missing_CA_file1186=== PAUSE TestServerTLSConfig/missing_CA_file1187=== RUN TestServerTLSConfig/not_a_PEM_file1188=== PAUSE TestServerTLSConfig/not_a_PEM_file1189=== CONT TestDeduplicatedObjectsRecordedAsPending11902026/11/12 00:45:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=01191--- PASS: TestPresentReportsOnlyClosureRoots (0.78s)1192=== CONT TestService_NativeMTLS11932026/11/12 00:45:52 INFO Vacuumed table table=pending_closures11942026/11/12 00:45:52 INFO Vacuumed table table=commit_receipts11952026/11/12 00:45:52 INFO Vacuumed table table=pending_objects11962026/11/12 00:45:52 INFO Vacuumed table table=multipart_uploads11972026/11/12 00:45:52 INFO Vacuumed table table=closures11982026/11/12 00:45:52 INFO Vacuumed table table=objects11992026/11/12 00:45:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012002026/11/12 00:45:52 INFO Vacuumed table table=pending_closures12012026/11/12 00:45:52 INFO Vacuumed table table=commit_receipts12022026/11/12 00:45:52 INFO Vacuumed table table=pending_objects12032026/11/12 00:45:52 INFO Vacuumed table table=multipart_uploads12042026/11/12 00:45:52 INFO Vacuumed table table=closures12052026/11/12 00:45:52 INFO Received uploads request method=POST path=/api/pending_closures12062026/11/12 00:45:52 INFO Vacuumed table table=objects1207--- PASS: TestSweepRechecksPendingAfterLocking (0.92s)1208=== CONT TestPushDedupSurvivesConcurrentGC1209--- PASS: TestRegisterCompletedObjectNeedsPendingRow (0.63s)1210=== CONT TestMetricsCommit1211--- PASS: TestCommitRacingPendingCleanupKeepsObjects (1.01s)1212=== CONT TestGCMetrics1213--- PASS: TestPresentRefreshesOnlyStaleClosures (0.99s)1214=== CONT TestMetricsInventory12152026/11/12 00:45:53 INFO Aborted multipart uploads count=0 kept=012162026/11/12 00:45:53 INFO Received uploads request method=POST path=/api/pending_closures12172026/11/12 00:45:53 WARN Force mode enabled - objects will be deleted immediately without grace period12182026/11/12 00:45:53 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012192026/11/12 00:45:53 INFO Vacuumed table table=pending_closures12202026/11/12 00:45:53 INFO Vacuumed table table=commit_receipts12212026/11/12 00:45:53 INFO Vacuumed table table=pending_objects12222026/11/12 00:45:53 INFO Vacuumed table table=multipart_uploads12232026/11/12 00:45:53 INFO Vacuumed table table=closures12242026/11/12 00:45:53 INFO Vacuumed table table=objects1225--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (0.45s)1226=== CONT TestGCBugBareHashReferences12272026/11/12 00:45:53 INFO Aborted multipart uploads count=0 kept=012282026/11/12 00:45:53 INFO Received uploads request method=POST path=/api/pending_closures12292026/11/12 00:45:53 INFO Received uploads request method=POST path=/api/pending_closures1230--- PASS: TestObjectStatsTrigger (0.60s)1231=== CONT TestNARDeduplicationMetadataUploadBug12322026/11/12 00:45:53 INFO Received uploads request method=POST path=/api/pending_closures12332026/11/12 00:45:53 INFO Received cleanup request method=DELETE path=/api/pending_closures12342026/11/12 00:45:53 INFO Aborted multipart uploads count=0 kept=012352026/11/12 00:45:53 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux12362026/11/12 00:45:53 WARN Refused reserved pin name=worker-x86_64-linux12372026/11/12 00:45:53 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux12382026/11/12 00:45:53 INFO Received create pin request method=POST path=/api/pins/my-app12392026/11/12 00:45:53 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux1240--- PASS: TestCreatePin_ReservedPins (0.91s)1241=== CONT TestGCAdvisoryLockBlocksConcurrentRun12422026/11/12 00:45:53 INFO Received uploads request method=POST path=/api/pending_closures1243--- PASS: TestObjectStatsUpdatedOncePerCommit (0.81s)1244=== CONT TestCreatePendingClosureRejectsOversizedNAR12452026/11/12 00:45:53 INFO Received uploads request method=POST path=/api/pending_closures1246--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.00s)1247=== CONT TestClientIntegration12482026/11/12 00:45:53 INFO Received uploads request method=POST path=/api/pending_closures12492026/11/12 00:45:53 INFO Aborted multipart uploads count=0 kept=01250--- PASS: TestTombstonedObjectOfferedWithoutWaiting (0.61s)1251=== CONT TestCacheConfigHandlerMaxNarSize1252--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)1253=== CONT TestEnsureAbortUploadsRule12542026/11/12 00:45:53 INFO Received uploads request method=POST path=/api/pending_closures12552026/11/12 00:45:53 INFO Received uploads request method=POST path=/api/pending_closures12562026/11/12 00:45:53 INFO Aborted multipart uploads count=0 kept=012572026/11/12 00:45:53 INFO Received uploads request method=POST path=/api/pending_closures1258--- PASS: TestPushWaitsForSweepHoldingRow (1.45s)1259=== CONT TestLeadElectsOneAndHandsOver1260--- PASS: TestDeduplicatedObjectsRecordedAsPending (0.55s)1261=== CONT TestLeadEndsOnShutdown12622026/11/12 00:45:53 WARN mTLS auth: subject not in bound subjects subject="CN=reader"12632026/11/12 00:45:53 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1264--- PASS: TestService_NativeMTLS (0.55s)1265=== CONT TestGenerateLandingPage1266--- PASS: TestGenerateLandingPage (0.01s)1267=== CONT TestConnectWaitsForAPeerMigration12682026/11/12 00:45:53 INFO Received cleanup request method=DELETE path=/api/pending_closures12692026/11/12 00:45:53 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket58/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""12702026/11/12 00:45:53 INFO Aborted multipart uploads count=0 kept=11271--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (0.90s)1272=== CONT TestLeadGenerationGrowsWithEveryGrant12732026/11/12 00:45:53 INFO Received cleanup request method=DELETE path=/api/pending_closures12742026/11/12 00:45:53 INFO Aborted multipart uploads count=1 kept=012752026/11/12 00:45:53 INFO Received push request method=POST path=/api/pushes1276--- PASS: TestMultipartCleanup (0.71s)1277=== CONT TestService_readinessHandler12782026/11/12 00:45:53 INFO Received uploads request method=POST path=/api/pending_closures12792026/11/12 00:45:53 INFO Received complete push request method=POST path=/api/pushes/1/complete12802026/11/12 00:45:53 INFO Aborted multipart uploads count=0 kept=012812026/11/12 00:45:53 INFO Aborted multipart uploads count=0 kept=012822026/11/12 00:45:53 INFO lead: released remote=192.0.2.1:1234 generation=112832026/11/12 00:45:53 INFO lead: acquired remote=192.0.2.1:1234 generation=212842026/11/12 00:45:53 INFO lead: given back to the incumbent remote=192.0.2.1:1234 generation=212852026/11/12 00:45:53 INFO lead: acquired remote=192.0.2.1:1234 generation=312862026/11/12 00:45:54 WARN Force mode enabled - objects will be deleted immediately without grace period12872026/11/12 00:45:54 WARN Force mode enabled - objects will be deleted immediately without grace period12882026/11/12 00:45:54 WARN Force mode enabled - objects will be deleted immediately without grace period12892026/11/12 00:45:54 WARN Force mode enabled - objects will be deleted immediately without grace period12902026/11/12 00:45:54 INFO Completed push id=1 objects=4 duration=553.66384ms12912026/11/12 00:45:54 WARN Force mode enabled - objects will be deleted immediately without grace period12922026/11/12 00:45:54 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012932026/11/12 00:45:54 INFO Vacuumed table table=pending_closures12942026/11/12 00:45:54 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012952026/11/12 00:45:54 INFO Vacuumed table table=commit_receipts12962026/11/12 00:45:54 INFO Vacuumed table table=pending_objects12972026/11/12 00:45:54 INFO Vacuumed table table=multipart_uploads12982026/11/12 00:45:54 INFO Vacuumed table table=closures12992026/11/12 00:45:54 INFO Vacuumed table table=objects13002026/11/12 00:45:54 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete13012026/11/12 00:45:54 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=01302--- PASS: TestGCMetrics (1.12s)13032026/11/12 00:45:54 INFO Vacuumed table table=pending_closures1304=== CONT TestResolveDBConnectionString13052026/11/12 00:45:54 INFO Vacuumed table table=commit_receipts1306=== RUN TestResolveDBConnectionString/flag_wins1307=== PAUSE TestResolveDBConnectionString/flag_wins1308=== RUN TestResolveDBConnectionString/file_when_flag_empty1309=== PAUSE TestResolveDBConnectionString/file_when_flag_empty1310=== RUN TestResolveDBConnectionString/missing_file_is_an_error1311=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error1312=== RUN TestResolveDBConnectionString/PGHOST_allows_empty1313=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty1314=== RUN TestResolveDBConnectionString/nothing_configured1315=== PAUSE TestResolveDBConnectionString/nothing_configured1316=== CONT TestLeadIncumbentAnnouncesAtOnce13172026/11/12 00:45:54 INFO Completed upload id=2 objects=2 duration=5.838258ms13182026/11/12 00:45:54 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."13192026/11/12 00:45:54 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."13202026/11/12 00:45:54 INFO Vacuumed table table=pending_objects13212026/11/12 00:45:54 INFO Vacuumed table table=pending_closures13222026/11/12 00:45:54 INFO Vacuumed table table=multipart_uploads13232026/11/12 00:45:54 INFO Vacuumed table table=closures1324--- PASS: TestMetricsCommit (1.15s)1325=== CONT TestService_healthCheckHandler13262026/11/12 00:45:54 INFO Vacuumed table table=commit_receipts13272026/11/12 00:45:54 INFO Vacuumed table table=objects1328--- PASS: TestFailedS3DeleteKeepsTombstone (1.63s)1329=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock13302026/11/12 00:45:54 INFO Vacuumed table table=pending_objects13312026/11/12 00:45:54 INFO Vacuumed table table=multipart_uploads13322026/11/12 00:45:54 INFO Vacuumed table table=closures1333--- PASS: TestGCSweepSkipsPendingObjects (1.25s)1334=== CONT TestGracefulShutdownDrainsInflight13352026/11/12 00:45:54 INFO Starting HTTP server address=127.0.0.1:4568113362026/11/12 00:45:54 INFO Shutdown signal received, draining in-flight requests timeout=10s13372026/11/12 00:45:54 INFO Vacuumed table table=objects13382026/11/12 00:45:54 INFO Aborted multipart uploads count=0 kept=013392026/11/12 00:45:54 WARN Force mode enabled - objects will be deleted immediately without grace period13402026/11/12 00:45:54 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013412026/11/12 00:45:54 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=013422026/11/12 00:45:54 INFO Vacuumed table table=pending_closures13432026/11/12 00:45:54 INFO Vacuumed table table=commit_receipts13442026/11/12 00:45:54 INFO Vacuumed table table=pending_objects13452026/11/12 00:45:54 INFO Vacuumed table table=multipart_uploads13462026/11/12 00:45:54 INFO Vacuumed table table=closures13472026/11/12 00:45:54 INFO Vacuumed table table=pending_closures13482026/11/12 00:45:54 INFO Vacuumed table table=commit_receipts13492026/11/12 00:45:54 INFO Vacuumed table table=objects1350--- PASS: TestSweepSkipsRowsLockedByPush (1.72s)1351=== CONT TestService_RequireScope_OIDC13522026/11/12 00:45:54 INFO Vacuumed table table=pending_objects13532026/11/12 00:45:54 INFO Vacuumed table table=multipart_uploads13542026/11/12 00:45:54 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=013552026/11/12 00:45:54 INFO Vacuumed table table=pending_closures13562026/11/12 00:45:54 INFO Vacuumed table table=commit_receipts13572026/11/12 00:45:54 INFO Vacuumed table table=pending_objects13582026/11/12 00:45:54 INFO Vacuumed table table=multipart_uploads13592026/11/12 00:45:54 INFO Vacuumed table table=closures13602026/11/12 00:45:54 INFO Vacuumed table table=closures13612026/11/12 00:45:54 INFO Vacuumed table table=objects1362--- PASS: TestPushDedupSurvivesConcurrentGC (1.24s)1363=== CONT TestDeleteClosuresRacingPins13642026/11/12 00:45:54 INFO Vacuumed table table=objects1365--- PASS: TestGCSweepDeliversEachKeyOnce (1.54s)1366=== CONT TestClientErrorHandling1367=== RUN TestClientErrorHandling/InvalidStorePath1368=== PAUSE TestClientErrorHandling/InvalidStorePath1369=== RUN TestClientErrorHandling/InvalidAuthToken1370=== PAUSE TestClientErrorHandling/InvalidAuthToken1371=== RUN TestClientErrorHandling/ServerNotAvailable1372=== PAUSE TestClientErrorHandling/ServerNotAvailable1373=== CONT TestGCTaskStore_Fail1374--- PASS: TestGCTaskStore_Fail (0.00s)1375=== CONT TestClientCADerivations1376--- PASS: TestGracefulShutdownDrainsInflight (0.07s)1377=== CONT TestGCTaskStore_PhaseUpdates1378--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)1379=== CONT TestCacheStatsHandler13802026/11/12 00:45:54 INFO lead: acquired remote=192.0.2.1:1234 generation=11381=== NAME TestOrphanedObjectsGC1382 orphaned_objects_gc_test.go:287: GC Test Summary:1383 orphaned_objects_gc_test.go:288: - Kept: 2 objects from closure A1384 orphaned_objects_gc_test.go:289: - Deleted: 2 objects from closure B1385 orphaned_objects_gc_test.go:290: - Deleted: 6 orphaned chain objects (X1->X2->X3)1386 orphaned_objects_gc_test.go:291: - Deleted: 2 orphaned single objects (Y)1387 orphaned_objects_gc_test.go:292: - Total deleted: 10 objects1388--- PASS: TestOrphanedObjectsGC (1.75s)1389=== CONT TestGCTaskStore_CompletedAllowsNewTask1390--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)1391=== CONT TestCacheConfigHandler1392=== RUN TestCacheConfigHandler/full_config,_no_issuer1393=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1394=== RUN TestCacheConfigHandler/no_cache_url_configured1395=== PAUSE TestCacheConfigHandler/no_cache_url_configured1396=== RUN TestCacheConfigHandler/no_signing_keys1397=== PAUSE TestCacheConfigHandler/no_signing_keys1398=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1399=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1400=== CONT TestGCTaskStore_GetReturnsLatest1401--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)1402=== CONT TestClientSharedPathCommittedMidPush14032026/11/12 00:45:54 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:37219/oidc14042026/11/12 00:45:54 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=nixbld database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"1405--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (1.02s)1406=== CONT TestGCTaskStore_GetEmpty1407--- PASS: TestGCTaskStore_GetEmpty (0.00s)1408=== CONT TestService_ReadScope_PublicByDefault1409--- PASS: TestMetricsInventory (1.32s)1410=== CONT TestClientWithDependencies14112026/11/12 00:45:54 WARN readiness check failed error="closed pool"1412--- PASS: TestService_readinessHandler (0.81s)1413=== CONT TestPinProtectsFromGC14142026/11/12 00:45:54 INFO lead: acquired remote=192.0.2.1:1234 generation=114152026/11/12 00:45:54 INFO lead: acquired remote=192.0.2.1:1234 generation=114162026/11/12 00:45:54 INFO Installed lifecycle rule aborting incomplete NAR uploads bucket=bucket74 days=21417--- PASS: TestEnsureAbortUploadsRule (1.13s)1418=== CONT TestClientMultipleUploads1419=== NAME TestNARDeduplicationMetadataUploadBug1420 metadata_upload_test.go:48: First store path: /build/TestNARDeduplicationMetadataUploadBug2821813440/001/store/vb3lg49kq3bfc8msnpw7rr1jd11wlgnk-file1.txt1421=== NAME TestClientIntegration1422 client_integration_test.go:292: Created store path: /build/TestClientIntegration2141685899/002/store/ggp9i0yhhl7xp3fnmr37hvaysm0zf4mc-test-file.txt14232026/11/12 00:45:55 WARN Rate limiter enabled after throttle name=s3-test rate=514242026/11/12 00:45:55 WARN S3 rate limit hit operation="complete multipart upload" error="Please reduce your request rate."1425=== NAME TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle1426 throttle_test.go:213: Proxy stats: total=13, throttled=10, completeMultipart=101427 throttle_test.go:215: Rate limiter: enabled=true, rate=5.001428--- PASS: TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle (8.02s)1429=== CONT TestClientFallsBackToClosures1430--- PASS: TestService_ReadScope_PublicByDefault (0.92s)1431=== CONT TestService_AuthMiddleware_MTLSBoundSubjects14322026/11/12 00:45:55 INFO lead: acquired remote=192.0.2.1:1234 generation=114332026/11/12 00:45:55 INFO lead: released remote=192.0.2.1:1234 generation=11434--- PASS: TestLeadIncumbentAnnouncesAtOnce (1.17s)1435=== CONT TestClientPushesUseOnePush14362026/11/12 00:45:55 INFO lead: released remote=192.0.2.1:1234 generation=11437=== NAME TestOrphanedObjectsGCStressTest1438 orphaned_objects_gc_test.go:422: Created 10 active closures, 5 to-delete closures, 20 orphaned chains1439=== RUN TestService_RequireScope_OIDC/builder_may_write1440=== PAUSE TestService_RequireScope_OIDC/builder_may_write1441=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1442=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1443=== RUN TestService_RequireScope_OIDC/ops_may_admin1444=== PAUSE TestService_RequireScope_OIDC/ops_may_admin1445=== RUN TestService_RequireScope_OIDC/ops_may_not_write1446=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1447=== RUN TestService_RequireScope_OIDC/reader_may_not_write1448=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1449=== RUN TestService_RequireScope_OIDC/static_token_may_admin1450=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1451=== RUN TestService_RequireScope_OIDC/static_token_may_write1452=== PAUSE TestService_RequireScope_OIDC/static_token_may_write1453=== RUN TestService_RequireScope_OIDC/reader_may_read1454=== PAUSE TestService_RequireScope_OIDC/reader_may_read1455=== RUN TestService_RequireScope_OIDC/writer_implies_read1456=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1457=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1458=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read1459=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc1460=== PAUSE TestService_RequireScope_OIDC/reader_may_read_with_netrc1461=== RUN TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1462=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1463=== RUN TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1464=== PAUSE TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1465=== RUN TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1466=== PAUSE TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1467=== CONT TestIsValidUploadKey1468=== RUN TestIsValidUploadKey/narinfo1469=== PAUSE TestIsValidUploadKey/narinfo1470=== RUN TestIsValidUploadKey/nar_zst1471=== PAUSE TestIsValidUploadKey/nar_zst1472=== RUN TestIsValidUploadKey/nar_xz1473=== PAUSE TestIsValidUploadKey/nar_xz1474=== RUN TestIsValidUploadKey/nar_plain1475=== PAUSE TestIsValidUploadKey/nar_plain1476=== RUN TestIsValidUploadKey/listing1477=== PAUSE TestIsValidUploadKey/listing1478=== RUN TestIsValidUploadKey/build_log1479=== PAUSE TestIsValidUploadKey/build_log1480=== RUN TestIsValidUploadKey/build_log_home-manager_file1481=== PAUSE TestIsValidUploadKey/build_log_home-manager_file1482=== RUN TestIsValidUploadKey/build_log_plus_in_name1483=== PAUSE TestIsValidUploadKey/build_log_plus_in_name1484=== RUN TestIsValidUploadKey/build_log_question_mark1485=== PAUSE TestIsValidUploadKey/build_log_question_mark1486=== RUN TestIsValidUploadKey/build_log_equals1487=== PAUSE TestIsValidUploadKey/build_log_equals1488=== RUN TestIsValidUploadKey/realisation1489=== PAUSE TestIsValidUploadKey/realisation1490=== RUN TestIsValidUploadKey/realisation_plus_in_output1491=== PAUSE TestIsValidUploadKey/realisation_plus_in_output1492=== RUN TestIsValidUploadKey/nix-cache-info1493=== PAUSE TestIsValidUploadKey/nix-cache-info1494=== RUN TestIsValidUploadKey/index.html1495=== PAUSE TestIsValidUploadKey/index.html1496=== RUN TestIsValidUploadKey/narinfo_key,_nar_type1497=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type1498=== RUN TestIsValidUploadKey/nar_key,_narinfo_type1499=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type1500=== RUN TestIsValidUploadKey/listing_key,_narinfo_type1501=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type1502=== RUN TestIsValidUploadKey/traversal1503=== PAUSE TestIsValidUploadKey/traversal1504=== RUN TestIsValidUploadKey/traversal_nar1505=== PAUSE TestIsValidUploadKey/traversal_nar1506=== RUN TestIsValidUploadKey/absolute1507=== PAUSE TestIsValidUploadKey/absolute1508=== RUN TestIsValidUploadKey/empty_key1509=== PAUSE TestIsValidUploadKey/empty_key1510=== RUN TestIsValidUploadKey/unknown_type1511=== PAUSE TestIsValidUploadKey/unknown_type1512=== CONT TestService_AuthMiddleware_MTLSProxyHeader1513--- PASS: TestCacheStatsHandler (1.28s)1514=== CONT TestService_AuthMiddleware_OIDC15152026/11/12 00:45:55 INFO lead: released remote=192.0.2.1:1234 generation=115162026/11/12 00:45:55 INFO lead: acquired remote=192.0.2.1:1234 generation=215172026/11/12 00:45:55 INFO lead: released remote=192.0.2.1:1234 generation=11518--- PASS: TestLeadEndsOnShutdown (2.06s)1519=== CONT TestService_ReadAuthMiddleware15202026/11/12 00:45:55 INFO lead: acquired remote=192.0.2.1:1234 generation=21521--- PASS: TestGCBugBareHashReferences (2.42s)1522=== CONT TestService_AuthMiddleware1523=== NAME TestOrphanedObjectsGCStressTest1524 orphaned_objects_gc_test.go:440: Marked 210 objects for deletion1525=== NAME TestClientWithDependencies1526 client_integration_test.go:661: Built derivation: /build/TestClientWithDependencies2448754965/001/store/l6vdd3l06gi02aqlpwxsnaprmm4yv5yp-test-script15272026/11/12 00:45:55 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:38553/oidc15282026/11/12 00:45:55 INFO lead: released remote=192.0.2.1:1234 generation=31529--- PASS: TestLeadUnannouncedStandbyYieldsToIncumbent (4.03s)1530=== CONT TestProxyWriteTimeout/narinfo1531=== CONT TestCommitCanBeRepeatedUntilCleanup/pending_closure1532=== NAME TestClientWithDependencies1533 client_integration_test.go:663: Found 1 dependencies (including self)1534=== NAME TestClientMultipleUploads1535 client_integration_test.go:406: Created store path 0: /build/TestClientMultipleUploads3852259728/001/store/raxznbda0xxq9iwp8ziwsi0gkh6vs36c-test-file-0.txt1536=== NAME TestClientCADerivations1537 client_ca_test.go:136: Built CA derivation: /build/TestClientCADerivations509630867/001/store/g6x3zk00g6vkhikhbpx22azmfxf9rj1y-ca-test15382026/11/12 00:45:55 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"15392026/11/12 00:45:55 WARN mTLS auth: bound subjects configured but subject DN unavailable15402026/11/12 00:45:55 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1541--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (0.59s)1542=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1543--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (1.66s)1544=== CONT TestProxyWriteTimeout/unknown_size1545=== CONT TestProxyWriteTimeout/10_GiB_nar1546=== CONT TestProxyWriteTimeout/1_GiB_nar1547--- PASS: TestProxyWriteTimeout (0.00s)1548 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1549 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1550 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1551 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)1552=== CONT TestCommitCanBeRepeatedUntilCleanup/push15532026/11/12 00:45:55 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"1554--- PASS: TestService_AuthMiddleware (0.29s)1555=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1556=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token1557=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token1558=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1559=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1560=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1561=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1562=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1563=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1564=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1565--- PASS: TestService_ReadAuthMiddleware (0.36s)1566=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1567--- PASS: TestService_healthCheckHandler (1.79s)1568=== CONT TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed15692026/11/12 00:45:56 INFO Received uploads request method=POST path=/api/pending_closures15702026/11/12 00:45:56 INFO Received push request method=POST path=/api/pushes15712026/11/12 00:45:56 INFO Received complete upload request method=POST path=/complete15722026/11/12 00:45:56 INFO Received complete push request method=POST path=/complete15732026/11/12 00:45:56 INFO Completed upload id=1 objects=2 duration=5.734637ms15742026/11/12 00:45:56 INFO Received complete upload request method=POST path=/complete15752026/11/12 00:45:56 INFO Completed upload id=1 objects=0 duration=402.164µs15762026/11/12 00:45:56 INFO Received cleanup request method=DELETE path=/api/pending_closures15772026/11/12 00:45:56 INFO Aborted multipart uploads count=0 kept=015782026/11/12 00:45:56 INFO Received complete upload request method=POST path=/complete15792026/11/12 00:45:56 INFO Completed push id=1 objects=2 duration=6.0831ms15802026/11/12 00:45:56 INFO Received complete push request method=POST path=/complete15812026-11-12 00:45:56.043 UTC [1040] ERROR: Closure does not exist: id=115822026-11-12 00:45:56.043 UTC [1040] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 17 at RAISE15832026-11-12 00:45:56.043 UTC [1040] STATEMENT: -- name: CommitPendingClosure :one1584 SELECT commit_pending_closure($1::bigint)::bigint1585 15862026/11/12 00:45:56 INFO Completed push id=1 objects=0 duration=498.405µs15872026/11/12 00:45:56 INFO Received cleanup request method=DELETE path=/api/pending_closures1588=== CONT TestPendingClosureWriteTimeout/empty1589=== CONT TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline15902026/11/12 00:45:56 INFO Aborted multipart uploads count=0 kept=015912026/11/12 00:45:56 INFO Received complete push request method=POST path=/complete15922026-11-12 00:45:56.047 UTC [1046] ERROR: Push does not exist: id=115932026-11-12 00:45:56.047 UTC [1046] CONTEXT: PL/pgSQL function commit_push(bigint) line 15 at RAISE15942026-11-12 00:45:56.047 UTC [1046] STATEMENT: -- name: CommitPush :one1595 SELECT commit_push($1::bigint)::bigint1596 15972026/11/12 00:45:56 WARN Push not found id=11598--- PASS: TestCommitCanBeRepeatedUntilCleanup (0.00s)1599 --- PASS: TestCommitCanBeRepeatedUntilCleanup/pending_closure (0.35s)1600 --- PASS: TestCommitCanBeRepeatedUntilCleanup/push (0.29s)1601=== CONT TestPendingClosureWriteTimeout/670k_objects1602=== CONT TestPendingClosureWriteTimeout/400_objects1603=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info16042026/11/12 00:45:56 INFO Received uploads request method=POST path=/1605=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key16062026/11/12 00:45:56 INFO Received complete multipart upload request method=POST path=/1607=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key16082026/11/12 00:45:56 INFO Received request for more parts method=POST path=/1609=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal16102026/11/12 00:45:56 INFO Received uploads request method=POST path=/1611--- PASS: TestUploadHandlersRejectInvalidKeys (0.00s)1612 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)1613 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1614 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1615 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s)1616=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure16172026/11/12 00:45:56 INFO Received uploads request method=POST path=/16182026/11/12 00:45:56 INFO Received push request method=POST path=/api/pushes1619=== NAME TestClientCADerivations1620 client_ca_test.go:139: Found 1 dependencies (including self)16212026/11/12 00:45:56 INFO Received push request method=POST path=/api/pushes16222026/11/12 00:45:56 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)16232026/11/12 00:45:56 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)16242026/11/12 00:45:56 INFO Uploading ggp9i0yhhl7xp3fnmr37hvaysm0zf4mc-test-file.txt (152B)16252026/11/12 00:45:56 INFO Uploading vb3lg49kq3bfc8msnpw7rr1jd11wlgnk-file1.txt (160B)16262026/11/12 00:45:56 WARN Failed to register uploaded object key=vb3lg49kq3bfc8msnpw7rr1jd11wlgnk.ls error="server returned 404: 404 page not found\n"16272026/11/12 00:45:56 WARN Failed to register uploaded object key=ggp9i0yhhl7xp3fnmr37hvaysm0zf4mc.ls error="server returned 404: 404 page not found\n"16282026/11/12 00:45:56 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16292026/11/12 00:45:56 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"16302026/11/12 00:45:56 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16312026/11/12 00:45:56 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"16322026/11/12 00:45:56 INFO Signed narinfos id=1 count=116332026/11/12 00:45:56 INFO Signed narinfos id=1 count=116342026/11/12 00:45:56 INFO Uploading 1 narinfos16352026/11/12 00:45:56 INFO Uploading 1 narinfos16362026/11/12 00:45:56 INFO Received complete push request method=POST path=/api/pushes/1/complete16372026/11/12 00:45:56 INFO Received complete push request method=POST path=/api/pushes/1/complete16382026/11/12 00:45:56 WARN Failed to register uploaded object key=ggp9i0yhhl7xp3fnmr37hvaysm0zf4mc.narinfo error="server returned 404: 404 page not found\n"16392026/11/12 00:45:56 WARN Failed to register uploaded object key=vb3lg49kq3bfc8msnpw7rr1jd11wlgnk.narinfo error="server returned 404: 404 page not found\n"16402026/11/12 00:45:56 INFO Completed push id=1 objects=3 duration=8.0177ms16412026/11/12 00:45:56 INFO Upload complete. (748ms)16422026/11/12 00:45:56 INFO Completed push id=1 objects=3 duration=10.824988ms16432026/11/12 00:45:56 INFO Upload complete. (604ms)1644=== NAME TestNARDeduplicationMetadataUploadBug1645 metadata_upload_test.go:54: Retrieved narinfo from S3:1646 StorePath: /build/TestNARDeduplicationMetadataUploadBug2821813440/001/store/vb3lg49kq3bfc8msnpw7rr1jd11wlgnk-file1.txt1647 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1648 Compression: zstd1649 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1650 NarSize: 1601651 References: 1652 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1653 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)1654 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):1655 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}16562026/11/12 00:45:56 INFO Received uploads request method=POST path=/api/pending_closures1657=== NAME TestClientMultipleUploads1658 client_integration_test.go:406: Created store path 1: /build/TestClientMultipleUploads3852259728/001/store/xcygrjpb5nlqr8ndc7l13ns336j01v49-test-file-1.txt16592026/11/12 00:45:56 INFO Received uploads request method=POST path=/api/pending_closures1660=== NAME TestPinProtectsFromGC1661 client_integration_test.go:791: Pinned store path: /build/TestPinProtectsFromGC1561845854/001/store/qnq5ppy8r48i9gydsbvfba0kaynh6kvx-pinned-file.txt1662 client_integration_test.go:792: Unpinned store path: /build/TestPinProtectsFromGC1561845854/001/store/41zxmycigsixxmsgbwvqqv8kp3172c9f-unpinned-file.txt16632026/11/12 00:45:56 INFO Received uploads request method=POST path=/api/pending_closures16642026/11/12 00:45:56 INFO All 1 paths already cached1665=== NAME TestClientIntegration1666 client_integration_test.go:318: Retrieved narinfo from S3:1667 StorePath: /build/TestClientIntegration2141685899/002/store/ggp9i0yhhl7xp3fnmr37hvaysm0zf4mc-test-file.txt1668 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1669 Compression: zstd1670 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11671 NarSize: 1521672 References: 1673 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11674 client_integration_test.go:319: Retrieved .ls file from S3 (compressed size: 77 bytes)1675 client_integration_test.go:319: Decompressed .ls content (64 bytes):1676 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}1677=== NAME TestClientMultipleUploads1678 client_integration_test.go:406: Created store path 2: /build/TestClientMultipleUploads3852259728/001/store/canvz3xa04ygvdj9qyiq1wcb0bn2h93f-test-file-2.txt1679--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (0.96s)1680=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts16812026/11/12 00:45:56 INFO Received request for more parts method=POST path=/16822026/11/12 00:45:56 INFO Received uploads request method=POST path=/api/pending_closures1683=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart16842026/11/12 00:45:56 INFO Received complete multipart upload request method=POST path=/16852026/11/12 00:45:56 INFO Received uploads request method=POST path=/api/pending_closures16862026/11/12 00:45:56 INFO Received uploads request method=POST path=/api/pending_closures1687=== CONT TestPush_RejectsBadRequests/root_not_in_objects16882026/11/12 00:45:56 INFO Received push request method=POST path=/api/pushes1689=== CONT TestPush_RejectsBadRequests/bad_root16902026/11/12 00:45:56 INFO Received push request method=POST path=/api/pushes1691=== CONT TestPush_RejectsBadRequests/no_objects16922026/11/12 00:45:56 INFO Received push request method=POST path=/api/pushes1693=== CONT TestPush_RejectsBadRequests/no_roots16942026/11/12 00:45:56 INFO Received push request method=POST path=/api/pushes1695=== CONT TestIsValidCachePath/narinfo1696--- PASS: TestPush_RejectsBadRequests (4.76s)1697 --- PASS: TestPush_RejectsBadRequests/root_not_in_objects (0.00s)1698 --- PASS: TestPush_RejectsBadRequests/bad_root (0.00s)1699 --- PASS: TestPush_RejectsBadRequests/no_objects (0.00s)1700 --- PASS: TestPush_RejectsBadRequests/no_roots (0.00s)1701=== CONT TestIsValidCachePath/index.html1702=== CONT TestIsValidCachePath/short_hash1703=== CONT TestIsValidCachePath/wrong_extension1704=== CONT TestIsValidCachePath/leading_slash1705=== CONT TestIsValidCachePath/empty1706=== CONT TestIsValidCachePath/random_path1707=== CONT TestIsValidCachePath/invalid_char_u1708=== CONT TestIsValidCachePath/nar_uncompressed1709=== CONT TestIsValidCachePath/invalid_char_e1710=== CONT TestIsValidCachePath/nix-cache-info1711=== CONT TestIsValidCachePath/traversal_parent1712=== CONT TestIsValidCachePath/realisation1713=== CONT TestIsValidCachePath/traversal_in_middle1714=== CONT TestIsValidCachePath/nar_xz1715=== CONT TestIsValidCachePath/log1716=== CONT TestIsValidCachePath/nar_bz21717=== CONT TestIsValidCachePath/ls1718=== CONT TestIsValidCachePath/narinfo_all_nix_base32_chars1719=== CONT TestIsValidCachePath/nar_zst1720--- PASS: TestIsValidCachePath (0.00s)1721 --- PASS: TestIsValidCachePath/narinfo (0.00s)1722 --- PASS: TestIsValidCachePath/index.html (0.00s)1723 --- PASS: TestIsValidCachePath/short_hash (0.00s)1724 --- PASS: TestIsValidCachePath/wrong_extension (0.00s)1725 --- PASS: TestIsValidCachePath/leading_slash (0.00s)1726 --- PASS: TestIsValidCachePath/empty (0.00s)1727 --- PASS: TestIsValidCachePath/random_path (0.00s)1728 --- PASS: TestIsValidCachePath/invalid_char_u (0.00s)1729 --- PASS: TestIsValidCachePath/nar_uncompressed (0.00s)1730 --- PASS: TestIsValidCachePath/invalid_char_e (0.00s)1731 --- PASS: TestIsValidCachePath/nix-cache-info (0.00s)1732 --- PASS: TestIsValidCachePath/traversal_parent (0.00s)1733 --- PASS: TestIsValidCachePath/realisation (0.00s)1734 --- PASS: TestIsValidCachePath/traversal_in_middle (0.00s)1735 --- PASS: TestIsValidCachePath/nar_xz (0.00s)1736 --- PASS: TestIsValidCachePath/log (0.00s)1737 --- PASS: TestIsValidCachePath/nar_bz2 (0.00s)1738 --- PASS: TestIsValidCachePath/ls (0.00s)1739 --- PASS: TestIsValidCachePath/narinfo_all_nix_base32_chars (0.00s)1740 --- PASS: TestIsValidCachePath/nar_zst (0.00s)1741=== CONT TestGCEndsOnShutdown/before_the_run1742=== NAME TestOrphanedObjectsGCStressTest1743 orphaned_objects_gc_test.go:503: Stress test completed successfully:1744 orphaned_objects_gc_test.go:504: - Active objects preserved: 201745 orphaned_objects_gc_test.go:505: - Objects deleted: 2101746 orphaned_objects_gc_test.go:506: - Total GC'd: 2101747--- PASS: TestOrphanedObjectsGCStressTest (4.19s)1748=== CONT TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark17492026/11/12 00:45:56 INFO lead: released remote=192.0.2.1:1234 generation=217502026/11/12 00:45:56 INFO lead: acquired remote=192.0.2.1:1234 generation=317512026/11/12 00:45:56 INFO lead: released remote=192.0.2.1:1234 generation=31752--- PASS: TestLeadGenerationGrowsWithEveryGrant (3.12s)1753=== CONT TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete17542026/11/12 00:45:56 INFO Received push request method=POST path=/api/pushes1755--- PASS: TestReadProxyOutlastsServerWriteTimeout (5.23s)1756=== CONT TestParseSingleRange/none1757=== CONT TestParseSingleRange/start_far_past_EOF1758=== CONT TestParseSingleRange/start_past_EOF1759=== CONT TestParseSingleRange/single_byte1760=== CONT TestParseSingleRange/suffix_exceeds_size17612026/11/12 00:45:56 INFO lead: released remote=192.0.2.1:1234 generation=21762=== CONT TestParseSingleRange/suffix1763=== CONT TestParseSingleRange/end_clamped_to_size1764=== CONT TestParseSingleRange/open-ended1765=== CONT TestParseSingleRange/closed1766=== CONT TestParseSingleRange/malformed_both_empty1767=== CONT TestParseSingleRange/malformed_no_dash1768=== CONT TestParseSingleRange/multi-range_ignored1769=== CONT TestParseSingleRange/unknown_unit1770--- PASS: TestLeadElectsOneAndHandsOver (3.18s)1771=== CONT TestParseSingleRange/malformed_end_before_start1772--- PASS: TestParseSingleRange (0.00s)1773 --- PASS: TestParseSingleRange/none (0.00s)1774 --- PASS: TestParseSingleRange/start_far_past_EOF (0.00s)1775 --- PASS: TestParseSingleRange/start_past_EOF (0.00s)1776 --- PASS: TestParseSingleRange/single_byte (0.00s)1777 --- PASS: TestParseSingleRange/suffix_exceeds_size (0.00s)1778 --- PASS: TestParseSingleRange/suffix (0.00s)1779 --- PASS: TestParseSingleRange/end_clamped_to_size (0.00s)1780 --- PASS: TestParseSingleRange/open-ended (0.00s)1781 --- PASS: TestParseSingleRange/closed (0.00s)1782 --- PASS: TestParseSingleRange/malformed_both_empty (0.00s)1783 --- PASS: TestParseSingleRange/malformed_no_dash (0.00s)1784 --- PASS: TestParseSingleRange/multi-range_ignored (0.00s)1785 --- PASS: TestParseSingleRange/unknown_unit (0.00s)1786 --- PASS: TestParseSingleRange/malformed_end_before_start (0.00s)1787=== CONT TestPresentNotReportedWhileGCDeletesClosure/fresh1788=== CONT TestPresentNotReportedWhileGCDeletesClosure/stale17892026/11/12 00:45:56 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)17902026/11/12 00:45:56 INFO Uploading l6vdd3l06gi02aqlpwxsnaprmm4yv5yp-test-script (136B)1791=== NAME TestNARDeduplicationMetadataUploadBug1792 metadata_upload_test.go:64: Second store path (same content): /build/TestNARDeduplicationMetadataUploadBug2821813440/001/store/0sqjcb9cq23xc6rwhvc6m7v4h1y38kjr-file2.txt17932026/11/12 00:45:56 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"17942026/11/12 00:45:56 WARN Failed to register uploaded object key=l6vdd3l06gi02aqlpwxsnaprmm4yv5yp.ls error="server returned 404: 404 page not found\n"17952026/11/12 00:45:56 WARN Failed to register uploaded object key=log/fbqi5ix8gvjp57jf70n5r9dmj37nw1ca-test-script.drv error="server returned 404: 404 page not found\n"17962026/11/12 00:45:56 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign17972026/11/12 00:45:56 INFO Signed narinfos id=1 count=117982026/11/12 00:45:56 INFO Uploading 1 narinfos17992026/11/12 00:45:56 INFO Received complete push request method=POST path=/api/pushes/1/complete18002026/11/12 00:45:56 WARN Failed to register uploaded object key=l6vdd3l06gi02aqlpwxsnaprmm4yv5yp.narinfo error="server returned 404: 404 page not found\n"18012026/11/12 00:45:56 INFO Completed push id=1 objects=4 duration=15.176491ms18022026/11/12 00:45:56 INFO Upload complete. (352ms)18032026/11/12 00:45:56 INFO Aborted multipart uploads count=0 kept=018042026/11/12 00:45:56 INFO Received complete multipart upload request method=POST path=/api/multipart/complete18052026/11/12 00:45:56 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000400000000000000000000.nar.zst upload_id=YWUyZDIzYjctYWY5Zi00NTE4LThmNTctMWNmYWNlN2VjZjgzLjczNjFlNDAyLWFjMTktNGU5MS1hMDk1LWVmN2YyNzBmMDg0N3gxNzk0NDQ0MzU2MzM1NTEyMjE2 parts=101806=== CONT TestForceGCDuringPushOffersSweptObject/before_pending_rows18072026/11/12 00:45:56 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete18082026/11/12 00:45:56 INFO Received complete multipart upload request method=POST path=/api/multipart/complete18092026/11/12 00:45:56 INFO Completed upload id=1 objects=3 duration=4.517465ms18102026/11/12 00:45:56 INFO Received uploads request method=POST path=/api/pending_closures18112026/11/12 00:45:56 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000500000000000000000000.nar.zst upload_id=YWUyZDIzYjctYWY5Zi00NTE4LThmNTctMWNmYWNlN2VjZjgzLjNkMThiOTQ4LWVkYmQtNDgyYi1iM2RkLTJkOGY3NmJmNmQzNngxNzk0NDQ0MzU2MzkzMjY3MzM0 parts=101812=== CONT TestForceGCDuringPushOffersSweptObject/after_presence_check1813--- PASS: TestUploadHandlersRejectOversizedBody (0.24s)1814 --- PASS: TestUploadHandlersRejectOversizedBody/request_more_parts (0.06s)1815 --- PASS: TestUploadHandlersRejectOversizedBody/complete_multipart (0.06s)1816 --- PASS: TestUploadHandlersRejectOversizedBody/create_pending_closure (0.85s)18172026/11/12 00:45:56 INFO Received push request method=POST path=/api/pushes18182026/11/12 00:45:56 INFO Received push request method=POST path=/api/pushes18192026/11/12 00:45:56 WARN Force mode enabled - objects will be deleted immediately without grace period18202026/11/12 00:45:56 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete18212026/11/12 00:45:56 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)18222026/11/12 00:45:56 INFO Uploading qnq5ppy8r48i9gydsbvfba0kaynh6kvx-pinned-file.txt (128B)18232026/11/12 00:45:56 INFO Received uploads request method=POST path=/api/pending_closures18242026/11/12 00:45:56 INFO Received push request method=POST path=/api/pushes18252026/11/12 00:45:56 INFO Received push request method=POST path=/api/pushes18262026/11/12 00:45:56 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)18272026/11/12 00:45:56 INFO Uploading dpywiifw9x6xsab8bqahyjlnn5l3frmr-shared-dep (136B)18282026/11/12 00:45:56 INFO Uploading 7dlfciaw5nqmmmhx298yykf0b6kxgqrv-top (224B)18292026/11/12 00:45:56 INFO Completed upload id=1 objects=3 duration=7.673357ms18302026/11/12 00:45:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=018312026/11/12 00:45:56 INFO Received uploads request method=POST path=/api/pending_closures18322026/11/12 00:45:56 WARN Failed to register uploaded object key=qnq5ppy8r48i9gydsbvfba0kaynh6kvx.ls error="server returned 404: 404 page not found\n"18332026/11/12 00:45:56 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign18342026/11/12 00:45:56 WARN Failed to register uploaded object key=dpywiifw9x6xsab8bqahyjlnn5l3frmr.ls error="server returned 404: 404 page not found\n"18352026/11/12 00:45:56 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"18362026/11/12 00:45:56 INFO Signed narinfos id=1 count=118372026/11/12 00:45:56 INFO Object in database but missing from S3 key=ggp9i0yhhl7xp3fnmr37hvaysm0zf4mc.ls18382026/11/12 00:45:56 WARN Found objects in DB but missing from S3, will re-upload count=118392026/11/12 00:45:56 WARN Failed to register uploaded object key=nar/1aliyqi8yr0ribhaq0yknn5m53gn6fkgvkxxp71kcx9wyrb3s2pc.nar.zst error="server returned 404: 404 page not found\n"18402026/11/12 00:45:56 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"18412026/11/12 00:45:56 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)18422026/11/12 00:45:56 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)18432026/11/12 00:45:56 INFO Vacuumed table table=pending_closures18442026/11/12 00:45:56 INFO Uploading h681s2ls8dafzzpsfx1x9na3iszanxyq-shared-dep (136B)18452026/11/12 00:45:56 INFO Uploading d59wxvs98i57mc47vajgx0wjz48bxzdc-a (216B)18462026/11/12 00:45:56 WARN Failed to register uploaded object key=7dlfciaw5nqmmmhx298yykf0b6kxgqrv.ls error="server returned 404: 404 page not found\n"18472026/11/12 00:45:56 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)18482026/11/12 00:45:56 INFO Uploading g6x3zk00g6vkhikhbpx22azmfxf9rj1y-ca-test (144B)18492026/11/12 00:45:56 INFO Uploading 1 narinfos18502026/11/12 00:45:56 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign18512026/11/12 00:45:56 INFO Signed narinfos id=1 count=218522026/11/12 00:45:56 INFO Uploading 2 narinfos18532026/11/12 00:45:56 INFO Vacuumed table table=commit_receipts18542026/11/12 00:45:56 INFO Received complete push request method=POST path=/api/pushes/1/complete18552026/11/12 00:45:56 INFO Received complete push request method=POST path=/api/pushes/2/complete18562026/11/12 00:45:56 WARN Failed to register uploaded object key=qnq5ppy8r48i9gydsbvfba0kaynh6kvx.narinfo error="server returned 404: 404 page not found\n"18572026/11/12 00:45:56 WARN Failed to register uploaded object key=7lvd1326ls8362cb1k9hzb94682z1bmy.ls error="server returned 404: 404 page not found\n"18582026/11/12 00:45:56 WARN Failed to register uploaded object key=g6x3zk00g6vkhikhbpx22azmfxf9rj1y.ls error="server returned 404: 404 page not found\n"18592026/11/12 00:45:56 WARN Failed to register uploaded object key=d59wxvs98i57mc47vajgx0wjz48bxzdc.ls error="server returned 404: 404 page not found\n"18602026/11/12 00:45:56 WARN Failed to register uploaded object key=ggp9i0yhhl7xp3fnmr37hvaysm0zf4mc.ls error="server returned 404: 404 page not found\n"18612026/11/12 00:45:56 WARN Failed to register uploaded object key=7dlfciaw5nqmmmhx298yykf0b6kxgqrv.narinfo error="server returned 404: 404 page not found\n"18622026/11/12 00:45:56 WARN Failed to register uploaded object key=h681s2ls8dafzzpsfx1x9na3iszanxyq.ls error="server returned 404: 404 page not found\n"18632026/11/12 00:45:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign18642026/11/12 00:45:56 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"18652026/11/12 00:45:56 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"18662026/11/12 00:45:56 WARN Failed to register uploaded object key=nar/1b79s2h01i4i3jkazmrxayw83s0f4pbgkwrmv7wsphy9513qv9k5.nar.zst error="server returned 404: 404 page not found\n"18672026/11/12 00:45:56 INFO Received complete push request method=POST path=/api/pushes/1/complete18682026/11/12 00:45:56 INFO Signed narinfos id=2 count=218692026/11/12 00:45:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign18702026/11/12 00:45:56 INFO Signed narinfos id=1 count=218712026/11/12 00:45:56 INFO Uploading 4 narinfos18722026/11/12 00:45:56 WARN Failed to register uploaded object key=dpywiifw9x6xsab8bqahyjlnn5l3frmr.narinfo error="server returned 404: 404 page not found\n"18732026/11/12 00:45:56 WARN Failed to register uploaded object key=log/qf1c2s950qv7fb4r3mx7yzrbhgfndxq5-ca-test.drv error="server returned 404: 404 page not found\n"18742026/11/12 00:45:56 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign18752026/11/12 00:45:56 INFO Signed narinfos id=1 count=118762026/11/12 00:45:56 INFO Uploading 1 narinfos18772026/11/12 00:45:56 INFO Vacuumed table table=pending_objects18782026/11/12 00:45:56 INFO Completed push id=2 objects=3 duration=4.367704ms18792026/11/12 00:45:56 INFO Upload complete. (283ms)1880=== NAME TestClientIntegration1881 client_integration_test.go:336: Retrieved .ls file from S3 (compressed size: 62 bytes)1882 client_integration_test.go:336: Decompressed .ls content (49 bytes):1883 {"version":1,"root":{"type":"regular","size":39}}18842026/11/12 00:45:56 INFO Vacuumed table table=multipart_uploads18852026/11/12 00:45:56 INFO Completed push id=1 objects=3 duration=11.202592ms18862026/11/12 00:45:56 INFO Upload complete. (339ms)18872026/11/12 00:45:56 INFO Vacuumed table table=closures18882026/11/12 00:45:56 INFO Received complete push request method=POST path=/api/pushes/1/complete18892026/11/12 00:45:56 WARN Failed to register uploaded object key=d59wxvs98i57mc47vajgx0wjz48bxzdc.narinfo error="server returned 404: 404 page not found\n"18902026/11/12 00:45:56 WARN Failed to register uploaded object key=h681s2ls8dafzzpsfx1x9na3iszanxyq.narinfo error="server returned 404: 404 page not found\n"18912026/11/12 00:45:56 WARN Failed to register uploaded object key=7lvd1326ls8362cb1k9hzb94682z1bmy.narinfo error="server returned 404: 404 page not found\n"18922026/11/12 00:45:56 WARN Failed to register uploaded object key=g6x3zk00g6vkhikhbpx22azmfxf9rj1y.narinfo error="server returned 404: 404 page not found\n"18932026/11/12 00:45:56 INFO Completed push id=1 objects=6 duration=15.509475ms18942026/11/12 00:45:56 INFO Upload complete. (608ms)18952026/11/12 00:45:56 INFO Vacuumed table table=objects1896=== NAME TestClientSharedPathCommittedMidPush1897 client_integration_test.go:740: Retrieved narinfo from S3:1898 StorePath: /build/TestClientSharedPathCommittedMidPush3963069071/001/store/dpywiifw9x6xsab8bqahyjlnn5l3frmr-shared-dep1899 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1900 Compression: zstd1901 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821902 NarSize: 1361903 References: 1904 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1905=== NAME TestClientWithDependencies1906 client_integration_test.go:677: Skipping nix copy test - isolated store (/build/TestClientWithDependencies2448754965/001/store) requires matching store prefix19072026/11/12 00:45:56 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1908=== NAME TestClientSharedPathCommittedMidPush1909 client_integration_test.go:740: Retrieved narinfo from S3:1910 StorePath: /build/TestClientSharedPathCommittedMidPush3963069071/001/store/7dlfciaw5nqmmmhx298yykf0b6kxgqrv-top1911 URL: nar/1aliyqi8yr0ribhaq0yknn5m53gn6fkgvkxxp71kcx9wyrb3s2pc.nar.zst1912 Compression: zstd1913 NarHash: sha256:1aliyqi8yr0ribhaq0yknn5m53gn6fkgvkxxp71kcx9wyrb3s2pc1914 NarSize: 2241915 References: /build/TestClientSharedPathCommittedMidPush3963069071/001/store/dpywiifw9x6xsab8bqahyjlnn5l3frmr-shared-dep1916 CA: text:sha256:1mdvh7z5m4qkqvplbvb548sr4zc0dh178y7gplhxvn009amrzwwm19172026/11/12 00:45:56 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete19182026/11/12 00:45:56 WARN Failed to register uploaded object key=h681s2ls8dafzzpsfx1x9na3iszanxyq.narinfo error="server returned 404: 404 page not found\n"19192026/11/12 00:45:56 INFO Completed push id=1 objects=4 duration=10.803529ms19202026/11/12 00:45:56 INFO Upload complete. (628ms)1921=== NAME TestClientCADerivations1922 client_ca_test.go:180: Narinfo contains CA field: StorePath: /build/TestClientCADerivations509630867/001/store/g6x3zk00g6vkhikhbpx22azmfxf9rj1y-ca-test1923 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1924 Compression: zstd1925 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1926 NarSize: 1441927 References: 1928 Deriver: /build/TestClientCADerivations509630867/001/store/qf1c2s950qv7fb4r3mx7yzrbhgfndxq5-ca-test.drv1929 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1930 client_ca_test.go:185: Checking for realisation files in S3...19312026/11/12 00:45:56 INFO Completed upload id=1 objects=6 duration=8.220303ms19322026/11/12 00:45:56 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete19332026/11/12 00:45:56 INFO Received uploads request method=POST path=/api/pending_closures19342026/11/12 00:45:56 INFO Completed upload id=2 objects=6 duration=3.510915ms19352026/11/12 00:45:56 INFO Upload complete. (630ms)19362026/11/12 00:45:56 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000200000000000000000000.nar.zst upload_id=YWUyZDIzYjctYWY5Zi00NTE4LThmNTctMWNmYWNlN2VjZjgzLjUxMzA0ZTE2LTg4ZGUtNGNiYS1hNWM0LWMwOWU3ZDE2ZjYzNngxNzk0NDQ0MzU2NDU4NTUxMjg3 parts=1019372026/11/12 00:45:56 INFO Object in database but missing from S3 key=nar/dadb44fdadb44fdadb44fdadb44f000500000000000000000000.nar.zst19382026/11/12 00:45:56 WARN Found objects in DB but missing from S3, will re-upload count=11939=== NAME TestClientFallsBackToClosures1940 client_pushes_test.go:112: Retrieved narinfo from S3:1941 StorePath: /build/TestClientFallsBackToClosures112928693/001/store/h681s2ls8dafzzpsfx1x9na3iszanxyq-shared-dep1942 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1943 Compression: zstd1944 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821945 NarSize: 1361946 References: 1947 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1948 client_pushes_test.go:112: Retrieved narinfo from S3:1949 StorePath: /build/TestClientFallsBackToClosures112928693/001/store/d59wxvs98i57mc47vajgx0wjz48bxzdc-a1950 URL: nar/1b79s2h01i4i3jkazmrxayw83s0f4pbgkwrmv7wsphy9513qv9k5.nar.zst1951 Compression: zstd1952 NarHash: sha256:1b79s2h01i4i3jkazmrxayw83s0f4pbgkwrmv7wsphy9513qv9k51953 NarSize: 2161954 References: /build/TestClientFallsBackToClosures112928693/001/store/h681s2ls8dafzzpsfx1x9na3iszanxyq-shared-dep1955 CA: text:sha256:12si24msmdk2sfgh591rk1gs3xljf82v6sqx1qvhbfshfvwyll911956=== NAME TestClientCADerivations1957 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1958 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache1959=== NAME TestClientFallsBackToClosures1960 client_pushes_test.go:112: Retrieved narinfo from S3:1961 StorePath: /build/TestClientFallsBackToClosures112928693/001/store/7lvd1326ls8362cb1k9hzb94682z1bmy-b1962 URL: nar/1b79s2h01i4i3jkazmrxayw83s0f4pbgkwrmv7wsphy9513qv9k5.nar.zst1963 Compression: zstd1964 NarHash: sha256:1b79s2h01i4i3jkazmrxayw83s0f4pbgkwrmv7wsphy9513qv9k51965 NarSize: 2161966 References: /build/TestClientFallsBackToClosures112928693/001/store/h681s2ls8dafzzpsfx1x9na3iszanxyq-shared-dep1967 CA: text:sha256:12si24msmdk2sfgh591rk1gs3xljf82v6sqx1qvhbfshfvwyll911968=== CONT TestServerTLSConfig/no_client_CA1969=== CONT TestServerTLSConfig/not_a_PEM_file19702026/11/12 00:45:56 INFO Received complete multipart upload request method=POST path=/api/multipart/complete19712026/11/12 00:45:56 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete19722026/11/12 00:45:56 INFO Completed upload id=1 objects=3 duration=4.310103ms19732026/11/12 00:45:56 INFO Received uploads request method=POST path=/api/pending_closures19742026/11/12 00:45:57 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000100000000000000000000.nar.zst upload_id=YWUyZDIzYjctYWY5Zi00NTE4LThmNTctMWNmYWNlN2VjZjgzLjgzZjc1NWE5LTYwNmMtNDEyZi05ZjYzLWM4MDk3MmEwNGJlMXgxNzk0NDQ0MzU2NDkwMzYyOTA1 parts=1019752026/11/12 00:45:57 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete1976=== CONT TestServerTLSConfig/missing_CA_file1977=== CONT TestResolveDBConnectionString/flag_wins1978=== CONT TestResolveDBConnectionString/nothing_configured1979=== CONT TestResolveDBConnectionString/PGHOST_allows_empty1980=== CONT TestResolveDBConnectionString/missing_file_is_an_error1981=== CONT TestResolveDBConnectionString/file_when_flag_empty1982=== CONT TestClientErrorHandling/InvalidStorePath1983--- PASS: TestServerTLSConfig (0.00s)1984 --- PASS: TestServerTLSConfig/no_client_CA (0.00s)1985 --- PASS: TestServerTLSConfig/missing_CA_file (0.00s)1986 --- PASS: TestServerTLSConfig/not_a_PEM_file (0.03s)1987--- PASS: TestResolveDBConnectionString (0.00s)1988 --- PASS: TestResolveDBConnectionString/flag_wins (0.00s)1989 --- PASS: TestResolveDBConnectionString/nothing_configured (0.00s)1990 --- PASS: TestResolveDBConnectionString/PGHOST_allows_empty (0.00s)1991 --- PASS: TestResolveDBConnectionString/missing_file_is_an_error (0.00s)1992 --- PASS: TestResolveDBConnectionString/file_when_flag_empty (0.00s)1993=== CONT TestClientErrorHandling/InvalidAuthToken19942026/11/12 00:45:57 INFO Completed upload id=1 objects=3 duration=3.569756ms1995--- PASS: TestClientSharedPathCommittedMidPush (2.81s)1996=== CONT TestClientErrorHandling/ServerNotAvailable1997--- PASS: TestClientFallsBackToClosures (1.88s)1998=== CONT TestCacheConfigHandler/full_config,_no_issuer1999=== CONT TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator2000=== CONT TestCacheConfigHandler/no_signing_keys

loading 894 hidden lines…

28952026/11/12 00:46:15 INFO Vacuumed table table=pending_closures28962026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts28972026/11/12 00:46:15 INFO Vacuumed table table=pending_objects28982026/11/12 00:46:15 INFO Received push request method=POST path=/28992026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads29002026/11/12 00:46:15 INFO Vacuumed table table=closures29012026/11/12 00:46:15 INFO Received push request method=POST path=/29022026/11/12 00:46:15 INFO Vacuumed table table=objects29032026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=029042026/11/12 00:46:15 INFO Received complete push request method=POST path=/29052026/11/12 00:46:15 INFO Completed push id=8 objects=2 duration=2.02292ms29062026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029072026/11/12 00:46:15 INFO Vacuumed table table=pending_closures29082026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts29092026/11/12 00:46:15 INFO Received complete upload request method=POST path=/29102026/11/12 00:46:15 INFO Received complete upload request method=POST path=/29112026/11/12 00:46:15 INFO Received uploads request method=POST path=/29122026/11/12 00:46:15 INFO Completed upload id=16 objects=3 duration=1.657017ms29132026/11/12 00:46:15 INFO Received push request method=POST path=/29142026/11/12 00:46:15 INFO Vacuumed table table=pending_objects29152026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads29162026/11/12 00:46:15 INFO Completed upload id=10 objects=4 duration=2.772768ms29172026/11/12 00:46:15 INFO Vacuumed table table=closures29182026/11/12 00:46:15 INFO Vacuumed table table=objects29192026/11/12 00:46:15 INFO Received push request method=POST path=/29202026/11/12 00:46:15 INFO Received uploads request method=POST path=/29212026/11/12 00:46:15 INFO Received uploads request method=POST path=/29222026/11/12 00:46:15 INFO Received push request method=POST path=/29232026/11/12 00:46:15 INFO Received push request method=POST path=/29242026/11/12 00:46:15 INFO Received push request method=POST path=/29252026/11/12 00:46:15 INFO Received complete upload request method=POST path=/29262026/11/12 00:46:15 INFO Completed upload id=6 objects=7 duration=2.360704ms29272026/11/12 00:46:15 INFO Received push request method=POST path=/29282026/11/12 00:46:15 INFO Received complete upload request method=POST path=/29292026/11/12 00:46:15 INFO Completed upload id=2 objects=7 duration=5.597896ms29302026/11/12 00:46:15 INFO Received uploads request method=POST path=/29312026/11/12 00:46:15 INFO Received push request method=POST path=/29322026/11/12 00:46:15 INFO Received complete upload request method=POST path=/29332026/11/12 00:46:15 INFO Received push request method=POST path=/29342026/11/12 00:46:15 INFO Completed upload id=3 objects=6 duration=5.237532ms29352026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=029362026/11/12 00:46:15 INFO Received push request method=POST path=/29372026/11/12 00:46:15 INFO Received push request method=POST path=/29382026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period29392026/11/12 00:46:15 INFO Received complete push request method=POST path=/29402026/11/12 00:46:15 INFO Received push request method=POST path=/29412026/11/12 00:46:15 INFO Received complete push request method=POST path=/29422026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=029432026/11/12 00:46:15 INFO Completed push id=1 objects=2 duration=2.662807ms29442026/11/12 00:46:15 INFO Completed push id=6 objects=8 duration=3.734837ms29452026/11/12 00:46:15 INFO Received uploads request method=POST path=/29462026/11/12 00:46:15 INFO Received complete upload request method=POST path=/29472026/11/12 00:46:15 INFO Received uploads request method=POST path=/29482026/11/12 00:46:15 INFO Received uploads request method=POST path=/29492026/11/12 00:46:15 INFO Completed upload id=6 objects=24 duration=3.093211ms29502026/11/12 00:46:15 INFO Received uploads request method=POST path=/29512026/11/12 00:46:15 INFO Received push request method=POST path=/29522026/11/12 00:46:15 INFO Received uploads request method=POST path=/29532026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=029542026/11/12 00:46:15 INFO Received push request method=POST path=/29552026/11/12 00:46:15 INFO Received push request method=POST path=/29562026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029572026/11/12 00:46:15 INFO Vacuumed table table=pending_closures29582026/11/12 00:46:15 INFO Received complete upload request method=POST path=/29592026/11/12 00:46:15 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo error="Access Denied."29602026/11/12 00:46:15 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."29612026/11/12 00:46:15 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."29622026/11/12 00:46:15 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."29632026/11/12 00:46:15 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."29642026/11/12 00:46:15 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."29652026/11/12 00:46:15 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."29662026/11/12 00:46:15 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."29672026/11/12 00:46:15 ERROR failed to remove object object=nar/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.nar.zst error="Access Denied."29682026/11/12 00:46:15 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."29692026/11/12 00:46:15 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."29702026/11/12 00:46:15 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."29712026/11/12 00:46:15 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."29722026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."29732026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."29742026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."29752026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts29762026/11/12 00:46:15 INFO Received complete upload request method=POST path=/29772026/11/12 00:46:15 INFO Vacuumed table table=pending_objects29782026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads29792026/11/12 00:46:15 INFO Received uploads request method=POST path=/29802026/11/12 00:46:15 INFO Vacuumed table table=closures29812026/11/12 00:46:15 INFO Completed upload id=11 objects=14 duration=2.668267ms29822026/11/12 00:46:15 INFO Received uploads request method=POST path=/29832026/11/12 00:46:15 INFO Completed upload id=1 objects=15 duration=6.621087ms29842026/11/12 00:46:15 INFO Received complete upload request method=POST path=/29852026/11/12 00:46:15 INFO Vacuumed table table=objects29862026/11/12 00:46:15 INFO Received complete push request method=POST path=/29872026/11/12 00:46:15 INFO Received complete push request method=POST path=/29882026/11/12 00:46:15 INFO Received uploads request method=POST path=/29892026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=029902026/11/12 00:46:15 INFO Completed push id=6 objects=7 duration=3.690257ms29912026/11/12 00:46:15 INFO Completed push id=8 objects=6 duration=2.065241ms29922026/11/12 00:46:15 INFO Completed upload id=2 objects=26 duration=7.93994ms29932026/11/12 00:46:15 INFO Received uploads request method=POST path=/29942026/11/12 00:46:15 INFO Received push request method=POST path=/29952026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=029962026/11/12 00:46:15 INFO Received uploads request method=POST path=/29972026/11/12 00:46:15 INFO Received uploads request method=POST path=/29982026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029992026/11/12 00:46:15 INFO Vacuumed table table=pending_closures30002026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts30012026/11/12 00:46:15 INFO Vacuumed table table=pending_objects30022026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads30032026/11/12 00:46:15 INFO Vacuumed table table=closures30042026/11/12 00:46:15 INFO Vacuumed table table=objects30052026/11/12 00:46:15 INFO Received complete push request method=POST path=/30062026/11/12 00:46:15 INFO Completed push id=4 objects=6 duration=1.673077ms30072026/11/12 00:46:15 INFO Received uploads request method=POST path=/30082026/11/12 00:46:15 INFO Received push request method=POST path=/30092026/11/12 00:46:15 INFO Received complete upload request method=POST path=/30102026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=030112026/11/12 00:46:15 INFO Completed upload id=9 objects=3 duration=1.145591ms30122026/11/12 00:46:15 INFO Received complete upload request method=POST path=/30132026/11/12 00:46:15 INFO Vacuumed table table=pending_closures30142026/11/12 00:46:15 INFO Received complete upload request method=POST path=/30152026/11/12 00:46:15 INFO Received uploads request method=POST path=/30162026/11/12 00:46:15 INFO Completed upload id=7 objects=4 duration=1.444274ms30172026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts30182026/11/12 00:46:15 INFO Completed upload id=5 objects=7 duration=3.08635ms30192026/11/12 00:46:15 INFO Vacuumed table table=pending_objects30202026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads30212026/11/12 00:46:15 INFO Received uploads request method=POST path=/30222026/11/12 00:46:15 INFO Vacuumed table table=closures30232026/11/12 00:46:15 INFO Received uploads request method=POST path=/30242026/11/12 00:46:15 INFO Vacuumed table table=objects30252026/11/12 00:46:15 INFO Received push request method=POST path=/30262026/11/12 00:46:15 INFO Received complete upload request method=POST path=/30272026/11/12 00:46:15 INFO Completed upload id=3 objects=24 duration=1.854638ms30282026/11/12 00:46:15 INFO Received push request method=POST path=/30292026/11/12 00:46:15 INFO Received push request method=POST path=/30302026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=030312026/11/12 00:46:15 INFO Received complete upload request method=POST path=/30322026/11/12 00:46:15 INFO Completed upload id=10 objects=23 duration=1.872119ms30332026/11/12 00:46:15 INFO Received complete upload request method=POST path=/30342026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=030352026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=030362026/11/12 00:46:15 INFO Completed upload id=6 objects=9 duration=1.400354ms30372026/11/12 00:46:15 INFO Received push request method=POST path=/30382026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period30392026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period30402026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030412026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=030422026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=030432026/11/12 00:46:15 INFO Vacuumed table table=pending_closures30442026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts30452026/11/12 00:46:15 INFO Received push request method=POST path=/30462026/11/12 00:46:15 INFO Vacuumed table table=pending_objects30472026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads30482026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period30492026/11/12 00:46:15 INFO Vacuumed table table=closures30502026/11/12 00:46:15 INFO Vacuumed table table=objects30512026/11/12 00:46:15 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."30522026/11/12 00:46:15 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."30532026/11/12 00:46:15 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."30542026/11/12 00:46:15 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."30552026/11/12 00:46:15 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."30562026/11/12 00:46:15 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."30572026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."30582026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030592026/11/12 00:46:15 INFO Vacuumed table table=pending_closures30602026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts30612026/11/12 00:46:15 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."30622026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."30632026/11/12 00:46:15 INFO Vacuumed table table=pending_objects30642026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads30652026/11/12 00:46:15 INFO Received complete upload request method=POST path=/30662026/11/12 00:46:15 INFO Vacuumed table table=closures30672026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=030682026/11/12 00:46:15 INFO Received push request method=POST path=/30692026/11/12 00:46:15 INFO Vacuumed table table=objects30702026/11/12 00:46:15 INFO Completed upload id=10 objects=6 duration=3.587656ms30712026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period30722026/11/12 00:46:15 INFO Received uploads request method=POST path=/30732026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=030742026/11/12 00:46:15 INFO Received uploads request method=POST path=/30752026/11/12 00:46:15 INFO Received uploads request method=POST path=/30762026/11/12 00:46:15 INFO Received complete push request method=POST path=/30772026/11/12 00:46:15 INFO Received uploads request method=POST path=/30782026/11/12 00:46:15 INFO Received uploads request method=POST path=/30792026/11/12 00:46:15 INFO Completed push id=2 objects=7 duration=4.734287ms30802026/11/12 00:46:15 INFO Received uploads request method=POST path=/30812026/11/12 00:46:15 INFO Received push request method=POST path=/30822026/11/12 00:46:15 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."30832026/11/12 00:46:15 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."30842026/11/12 00:46:15 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."30852026/11/12 00:46:15 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."30862026/11/12 00:46:15 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."30872026/11/12 00:46:15 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."30882026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."30892026/11/12 00:46:15 INFO Received complete upload request method=POST path=/30902026/11/12 00:46:15 INFO Received complete upload request method=POST path=/30912026/11/12 00:46:15 INFO Completed upload id=11 objects=6 duration=1.578015ms30922026/11/12 00:46:15 INFO Completed upload id=3 objects=5 duration=1.224952ms30932026/11/12 00:46:15 INFO Received uploads request method=POST path=/30942026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=030952026/11/12 00:46:15 INFO Received complete push request method=POST path=/30962026/11/12 00:46:15 INFO Received complete upload request method=POST path=/30972026/11/12 00:46:15 INFO Received complete push request method=POST path=/30982026/11/12 00:46:15 INFO Completed push id=7 objects=20 duration=2.796828ms30992026/11/12 00:46:15 INFO Completed upload id=9 objects=7 duration=3.061871ms31002026/11/12 00:46:15 INFO Received push request method=POST path=/31012026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=031022026/11/12 00:46:15 INFO Received push request method=POST path=/31032026/11/12 00:46:15 INFO Completed push id=14 objects=3 duration=4.576486ms31042026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=031052026/11/12 00:46:15 INFO Received push request method=POST path=/31062026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period31072026/11/12 00:46:15 INFO Received complete push request method=POST path=/31082026/11/12 00:46:15 INFO Received push request method=POST path=/31092026/11/12 00:46:15 INFO Completed push id=12 objects=3 duration=2.443125ms31102026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=031112026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=031122026/11/12 00:46:15 INFO Received complete push request method=POST path=/31132026/11/12 00:46:15 INFO Vacuumed table table=pending_closures31142026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts31152026/11/12 00:46:15 INFO Vacuumed table table=pending_objects31162026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads31172026/11/12 00:46:15 INFO Completed push id=8 objects=6 duration=2.188721ms31182026/11/12 00:46:15 INFO Vacuumed table table=closures31192026/11/12 00:46:15 INFO Vacuumed table table=objects31202026/11/12 00:46:15 INFO Received uploads request method=POST path=/31212026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=031222026/11/12 00:46:15 INFO Received complete push request method=POST path=/31232026/11/12 00:46:15 INFO Received complete push request method=POST path=/31242026/11/12 00:46:15 INFO Completed push id=12 objects=15 duration=1.853798ms31252026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031262026/11/12 00:46:15 INFO Completed push id=5 objects=6 duration=2.206462ms31272026/11/12 00:46:15 INFO Vacuumed table table=pending_closures31282026/11/12 00:46:15 INFO Received push request method=POST path=/31292026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts31302026/11/12 00:46:15 INFO Vacuumed table table=pending_objects31312026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads31322026/11/12 00:46:15 INFO Vacuumed table table=closures31332026/11/12 00:46:15 INFO Vacuumed table table=objects31342026/11/12 00:46:15 INFO Received complete push request method=POST path=/31352026/11/12 00:46:15 INFO Completed push id=12 objects=7 duration=2.167402ms31362026/11/12 00:46:15 INFO Received push request method=POST path=/31372026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=031382026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=031392026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period31402026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031412026/11/12 00:46:15 INFO Received uploads request method=POST path=/31422026/11/12 00:46:15 INFO Vacuumed table table=pending_closures31432026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts31442026/11/12 00:46:15 INFO Vacuumed table table=pending_objects31452026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads31462026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."31472026/11/12 00:46:15 INFO Vacuumed table table=closures31482026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=031492026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=031502026/11/12 00:46:15 INFO Vacuumed table table=objects31512026/11/12 00:46:15 INFO Received complete push request method=POST path=/31522026/11/12 00:46:15 INFO Received complete push request method=POST path=/31532026/11/12 00:46:15 INFO Completed push id=5 objects=3 duration=3.690737ms31542026/11/12 00:46:15 INFO Completed push id=2 objects=10 duration=3.623756ms31552026/11/12 00:46:15 INFO Received uploads request method=POST path=/31562026/11/12 00:46:15 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."31572026/11/12 00:46:15 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."31582026/11/12 00:46:15 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."31592026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=031602026/11/12 00:46:15 INFO Vacuumed table table=pending_closures31612026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts31622026/11/12 00:46:15 INFO Vacuumed table table=pending_objects31632026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads31642026/11/12 00:46:15 INFO Vacuumed table table=closures31652026/11/12 00:46:15 INFO Vacuumed table table=objects31662026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=031672026/11/12 00:46:15 INFO Received push request method=POST path=/31682026/11/12 00:46:15 INFO Received uploads request method=POST path=/31692026/11/12 00:46:15 INFO Received push request method=POST path=/31702026/11/12 00:46:15 INFO Received complete upload request method=POST path=/31712026/11/12 00:46:15 INFO Received complete push request method=POST path=/31722026/11/12 00:46:15 INFO Completed upload id=16 objects=23 duration=1.492534ms31732026/11/12 00:46:15 INFO Completed push id=8 objects=7 duration=1.841238ms31742026/11/12 00:46:15 INFO Received complete push request method=POST path=/31752026/11/12 00:46:15 INFO Completed push id=10 objects=2 duration=2.419544ms31762026/11/12 00:46:15 INFO Received complete push request method=POST path=/31772026/11/12 00:46:15 INFO Completed push id=8 objects=6 duration=1.635996ms31782026/11/12 00:46:15 INFO Received uploads request method=POST path=/31792026/11/12 00:46:15 INFO Received complete upload request method=POST path=/31802026/11/12 00:46:15 INFO Completed upload id=15 objects=6 duration=1.106171ms31812026/11/12 00:46:15 INFO Received uploads request method=POST path=/31822026/11/12 00:46:15 INFO Received uploads request method=POST path=/31832026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=031842026/11/12 00:46:15 INFO Vacuumed table table=pending_closures31852026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts31862026/11/12 00:46:15 INFO Received uploads request method=POST path=/31872026/11/12 00:46:15 INFO Vacuumed table table=pending_objects31882026/11/12 00:46:15 INFO Received uploads request method=POST path=/31892026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads31902026/11/12 00:46:15 INFO Received complete upload request method=POST path=/31912026/11/12 00:46:15 INFO Completed upload id=12 objects=21 duration=2.137681ms31922026/11/12 00:46:15 INFO Vacuumed table table=closures31932026/11/12 00:46:15 INFO Vacuumed table table=objects31942026/11/12 00:46:15 INFO Received uploads request method=POST path=/31952026/11/12 00:46:15 INFO Received push request method=POST path=/31962026/11/12 00:46:15 INFO Received complete upload request method=POST path=/31972026/11/12 00:46:15 INFO Received uploads request method=POST path=/31982026/11/12 00:46:15 INFO Received push request method=POST path=/31992026/11/12 00:46:15 INFO Received complete upload request method=POST path=/32002026/11/12 00:46:15 INFO Completed upload id=9 objects=16 duration=2.00976ms32012026/11/12 00:46:15 INFO Completed upload id=17 objects=7 duration=1.400934ms32022026/11/12 00:46:15 INFO Received uploads request method=POST path=/32032026/11/12 00:46:15 INFO Received uploads request method=POST path=/32042026/11/12 00:46:15 INFO Received push request method=POST path=/32052026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=032062026/11/12 00:46:15 INFO Received complete push request method=POST path=/32072026/11/12 00:46:15 INFO Received push request method=POST path=/32082026/11/12 00:46:15 INFO Received complete push request method=POST path=/32092026/11/12 00:46:15 INFO Completed push id=13 objects=23 duration=2.134921ms32102026/11/12 00:46:15 INFO Completed push id=6 objects=26 duration=6.415925ms32112026/11/12 00:46:15 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."32122026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."32132026/11/12 00:46:15 INFO Received push request method=POST path=/32142026/11/12 00:46:15 INFO Received push request method=POST path=/32152026/11/12 00:46:15 INFO Received push request method=POST path=/32162026/11/12 00:46:15 INFO Received complete upload request method=POST path=/32172026/11/12 00:46:15 INFO Completed upload id=15 objects=6 duration=958.93µs32182026/11/12 00:46:15 INFO Received uploads request method=POST path=/32192026/11/12 00:46:15 INFO Received complete push request method=POST path=/32202026/11/12 00:46:15 INFO Received complete push request method=POST path=/32212026/11/12 00:46:15 INFO Completed push id=10 objects=3 duration=2.113922ms32222026/11/12 00:46:15 INFO Completed push id=5 objects=9 duration=5.907719ms32232026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=032242026/11/12 00:46:15 INFO Received push request method=POST path=/32252026/11/12 00:46:15 INFO Received complete upload request method=POST path=/32262026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period32272026/11/12 00:46:15 INFO Completed upload id=10 objects=17 duration=2.682987ms32282026/11/12 00:46:15 INFO Received uploads request method=POST path=/32292026/11/12 00:46:15 INFO Received push request method=POST path=/32302026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032312026/11/12 00:46:15 INFO Received complete upload request method=POST path=/32322026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=032332026/11/12 00:46:15 INFO Vacuumed table table=pending_closures32342026/11/12 00:46:15 INFO Completed upload id=14 objects=6 duration=1.135931ms32352026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts32362026/11/12 00:46:15 INFO Vacuumed table table=pending_objects32372026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads32382026/11/12 00:46:15 INFO Vacuumed table table=closures32392026/11/12 00:46:15 INFO Vacuumed table table=objects32402026/11/12 00:46:15 INFO Received push request method=POST path=/32412026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=032422026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=032432026/11/12 00:46:15 INFO Received push request method=POST path=/32442026/11/12 00:46:15 INFO Vacuumed table table=pending_closures32452026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts32462026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period32472026/11/12 00:46:15 INFO Vacuumed table table=pending_objects32482026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads32492026/11/12 00:46:15 INFO Vacuumed table table=closures32502026/11/12 00:46:15 INFO Vacuumed table table=objects32512026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032522026/11/12 00:46:15 INFO Received push request method=POST path=/32532026/11/12 00:46:15 INFO Vacuumed table table=pending_closures32542026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts32552026/11/12 00:46:15 INFO Received uploads request method=POST path=/32562026/11/12 00:46:15 INFO Received push request method=POST path=/32572026/11/12 00:46:15 INFO Vacuumed table table=pending_objects32582026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads32592026/11/12 00:46:15 INFO Vacuumed table table=closures32602026/11/12 00:46:15 INFO Vacuumed table table=objects32612026/11/12 00:46:15 INFO Received push request method=POST path=/32622026/11/12 00:46:15 INFO Received uploads request method=POST path=/32632026/11/12 00:46:15 INFO Received push request method=POST path=/32642026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=032652026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period32662026/11/12 00:46:15 INFO Received push request method=POST path=/32672026/11/12 00:46:15 INFO Received complete upload request method=POST path=/32682026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032692026/11/12 00:46:15 INFO Received complete push request method=POST path=/32702026/11/12 00:46:15 INFO Completed upload id=7 objects=3 duration=789.528µs32712026/11/12 00:46:15 INFO Vacuumed table table=pending_closures32722026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts32732026/11/12 00:46:15 INFO Completed push id=16 objects=27 duration=1.799918ms32742026/11/12 00:46:15 INFO Vacuumed table table=pending_objects32752026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads32762026/11/12 00:46:15 INFO Vacuumed table table=closures32772026/11/12 00:46:15 INFO Received complete push request method=POST path=/32782026/11/12 00:46:15 INFO Vacuumed table table=objects32792026/11/12 00:46:15 INFO Received uploads request method=POST path=/32802026/11/12 00:46:15 INFO Completed push id=19 objects=8 duration=3.99408ms32812026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=032822026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period32832026/11/12 00:46:15 INFO Received complete push request method=POST path=/32842026/11/12 00:46:15 INFO Received uploads request method=POST path=/32852026/11/12 00:46:15 INFO Completed push id=14 objects=6 duration=2.407944ms32862026/11/12 00:46:15 INFO Received push request method=POST path=/32872026/11/12 00:46:15 INFO Received uploads request method=POST path=/32882026/11/12 00:46:15 INFO Received uploads request method=POST path=/32892026/11/12 00:46:15 INFO Received uploads request method=POST path=/32902026/11/12 00:46:15 INFO Received complete push request method=POST path=/32912026/11/12 00:46:15 INFO Completed push id=18 objects=23 duration=2.907369ms32922026/11/12 00:46:15 INFO Received push request method=POST path=/32932026/11/12 00:46:15 INFO Received push request method=POST path=/32942026/11/12 00:46:15 INFO Received uploads request method=POST path=/32952026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=032962026/11/12 00:46:15 INFO Received uploads request method=POST path=/32972026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=032982026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032992026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period33002026/11/12 00:46:15 INFO Vacuumed table table=pending_closures33012026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts33022026/11/12 00:46:15 INFO Vacuumed table table=pending_objects33032026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads33042026/11/12 00:46:15 INFO Vacuumed table table=closures33052026/11/12 00:46:15 INFO Received push request method=POST path=/33062026/11/12 00:46:15 INFO Vacuumed table table=objects33072026/11/12 00:46:15 INFO Received uploads request method=POST path=/33082026/11/12 00:46:15 INFO Received complete upload request method=POST path=/33092026/11/12 00:46:15 INFO Completed upload id=15 objects=4 duration=2.464684ms33102026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=033112026/11/12 00:46:15 INFO Received complete push request method=POST path=/33122026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period33132026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=033142026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033152026/11/12 00:46:15 INFO Completed push id=11 objects=9 duration=2.173461ms33162026/11/12 00:46:15 INFO Vacuumed table table=pending_closures33172026/11/12 00:46:15 INFO Received complete push request method=POST path=/33182026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts33192026/11/12 00:46:15 INFO Vacuumed table table=pending_objects33202026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads33212026/11/12 00:46:15 INFO Completed push id=18 objects=10 duration=1.824758ms33222026/11/12 00:46:15 INFO Vacuumed table table=closures33232026/11/12 00:46:15 INFO Vacuumed table table=objects33242026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=033252026/11/12 00:46:15 INFO Vacuumed table table=pending_closures33262026/11/12 00:46:15 INFO Received push request method=POST path=/33272026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts33282026/11/12 00:46:15 INFO Vacuumed table table=pending_objects33292026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads33302026/11/12 00:46:15 INFO Vacuumed table table=closures33312026/11/12 00:46:15 INFO Vacuumed table table=objects33322026/11/12 00:46:15 INFO Received push request method=POST path=/33332026/11/12 00:46:15 INFO Received push request method=POST path=/33342026/11/12 00:46:15 INFO Received push request method=POST path=/33352026/11/12 00:46:15 INFO Received uploads request method=POST path=/33362026/11/12 00:46:15 INFO Received uploads request method=POST path=/33372026/11/12 00:46:15 INFO Received push request method=POST path=/33382026/11/12 00:46:15 INFO Received complete upload request method=POST path=/33392026/11/12 00:46:15 INFO Completed upload id=16 objects=3 duration=906.349µs33402026/11/12 00:46:15 INFO Received push request method=POST path=/33412026/11/12 00:46:15 INFO Received uploads request method=POST path=/33422026/11/12 00:46:15 INFO Received uploads request method=POST path=/33432026/11/12 00:46:15 INFO Received uploads request method=POST path=/33442026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=033452026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=033462026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period33472026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033482026/11/12 00:46:15 INFO Received uploads request method=POST path=/33492026/11/12 00:46:15 INFO Received uploads request method=POST path=/33502026/11/12 00:46:15 INFO Vacuumed table table=pending_closures33512026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts33522026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=033532026/11/12 00:46:15 INFO Vacuumed table table=pending_objects33542026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads33552026/11/12 00:46:15 INFO Vacuumed table table=closures33562026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period33572026/11/12 00:46:15 INFO Received push request method=POST path=/33582026/11/12 00:46:15 INFO Received uploads request method=POST path=/33592026/11/12 00:46:15 INFO Vacuumed table table=objects33602026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033612026/11/12 00:46:15 INFO Vacuumed table table=pending_closures33622026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts33632026/11/12 00:46:15 INFO Vacuumed table table=pending_objects33642026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads33652026/11/12 00:46:15 INFO Vacuumed table table=closures33662026/11/12 00:46:15 INFO Vacuumed table table=objects33672026/11/12 00:46:15 INFO Received uploads request method=POST path=/33682026/11/12 00:46:15 INFO Received uploads request method=POST path=/33692026/11/12 00:46:15 INFO Received complete push request method=POST path=/33702026/11/12 00:46:15 INFO Received uploads request method=POST path=/33712026/11/12 00:46:15 INFO Completed push id=12 objects=2 duration=3.185232ms33722026/11/12 00:46:15 INFO Received complete upload request method=POST path=/33732026/11/12 00:46:15 INFO Completed upload id=17 objects=6 duration=3.233472ms33742026/11/12 00:46:15 INFO Received uploads request method=POST path=/33752026/11/12 00:46:15 INFO Received complete upload request method=POST path=/33762026/11/12 00:46:15 INFO Completed upload id=18 objects=4 duration=1.526675ms33772026/11/12 00:46:15 INFO Received push request method=POST path=/33782026/11/12 00:46:15 INFO Received push request method=POST path=/33792026/11/12 00:46:15 INFO Received uploads request method=POST path=/33802026/11/12 00:46:15 INFO Received complete upload request method=POST path=/33812026/11/12 00:46:15 INFO Completed upload id=23 objects=15 duration=3.252632ms33822026/11/12 00:46:15 INFO Received complete push request method=POST path=/33832026/11/12 00:46:15 INFO Completed push id=17 objects=6 duration=5.10203ms33842026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=033852026/11/12 00:46:15 INFO Received complete upload request method=POST path=/33862026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033872026/11/12 00:46:15 INFO Completed upload id=14 objects=16 duration=4.497605ms33882026/11/12 00:46:15 INFO Vacuumed table table=pending_closures33892026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts33902026/11/12 00:46:15 INFO Received complete upload request method=POST path=/33912026/11/12 00:46:15 INFO Vacuumed table table=pending_objects33922026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads33932026/11/12 00:46:15 INFO Completed upload id=19 objects=6 duration=1.719297ms33942026/11/12 00:46:15 INFO Vacuumed table table=closures33952026/11/12 00:46:15 INFO Vacuumed table table=objects33962026/11/12 00:46:15 INFO Received push request method=POST path=/33972026/11/12 00:46:15 INFO Received push request method=POST path=/33982026/11/12 00:46:15 INFO Received complete upload request method=POST path=/33992026/11/12 00:46:15 INFO Received push request method=POST path=/34002026/11/12 00:46:15 INFO Completed upload id=22 objects=7 duration=1.195792ms34012026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034022026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034032026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034042026/11/12 00:46:15 INFO Vacuumed table table=pending_closures34052026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034062026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034072026/11/12 00:46:15 INFO Vacuumed table table=pending_closures34082026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts34092026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts34102026/11/12 00:46:15 INFO Vacuumed table table=pending_closures34112026/11/12 00:46:15 INFO Vacuumed table table=pending_objects34122026/11/12 00:46:15 INFO Vacuumed table table=pending_closures34132026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads34142026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts34152026/11/12 00:46:15 INFO Vacuumed table table=pending_objects34162026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=5 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034172026/11/12 00:46:15 INFO Vacuumed table table=pending_closures34182026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads34192026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=034202026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts34212026/11/12 00:46:15 INFO Vacuumed table table=closures34222026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts34232026/11/12 00:46:15 INFO Vacuumed table table=closures34242026/11/12 00:46:15 INFO Vacuumed table table=pending_objects34252026/11/12 00:46:15 INFO Vacuumed table table=objects34262026/11/12 00:46:15 INFO Vacuumed table table=pending_objects34272026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads34282026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads34292026/11/12 00:46:15 INFO Vacuumed table table=objects34302026/11/12 00:46:15 INFO Received uploads request method=POST path=/34312026/11/12 00:46:15 INFO Vacuumed table table=closures34322026/11/12 00:46:15 INFO Vacuumed table table=closures34332026/11/12 00:46:15 INFO Vacuumed table table=pending_objects34342026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads34352026/11/12 00:46:15 INFO Vacuumed table table=objects34362026/11/12 00:46:15 INFO Vacuumed table table=objects34372026/11/12 00:46:15 INFO Vacuumed table table=closures34382026/11/12 00:46:15 INFO Vacuumed table table=pending_closures34392026/11/12 00:46:15 INFO Vacuumed table table=objects34402026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034412026/11/12 00:46:15 INFO Vacuumed table table=pending_closures34422026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts34432026/11/12 00:46:15 INFO Received uploads request method=POST path=/34442026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts34452026/11/12 00:46:15 INFO Vacuumed table table=pending_objects34462026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads34472026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=034482026/11/12 00:46:15 INFO Vacuumed table table=closures34492026/11/12 00:46:15 INFO Vacuumed table table=pending_closures34502026/11/12 00:46:15 INFO Vacuumed table table=objects34512026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts34522026/11/12 00:46:15 INFO Vacuumed table table=pending_objects34532026/11/12 00:46:15 INFO Vacuumed table table=pending_objects34542026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=034552026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads34562026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads34572026/11/12 00:46:15 INFO Received uploads request method=POST path=/34582026/11/12 00:46:15 INFO Received push request method=POST path=/34592026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period34602026/11/12 00:46:15 INFO Vacuumed table table=closures34612026/11/12 00:46:15 INFO Vacuumed table table=closures34622026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034632026/11/12 00:46:15 INFO Received push request method=POST path=/34642026/11/12 00:46:15 INFO Received complete push request method=POST path=/34652026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=034662026/11/12 00:46:15 INFO Vacuumed table table=objects34672026/11/12 00:46:15 INFO Vacuumed table table=pending_closures34682026/11/12 00:46:15 INFO Vacuumed table table=objects34692026/11/12 00:46:15 INFO Received uploads request method=POST path=/34702026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts34712026/11/12 00:46:15 INFO Completed push id=17 objects=11 duration=1.947439ms34722026/11/12 00:46:15 INFO Vacuumed table table=pending_objects34732026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads34742026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034752026/11/12 00:46:15 INFO Vacuumed table table=closures34762026/11/12 00:46:15 INFO Vacuumed table table=objects34772026/11/12 00:46:15 INFO Vacuumed table table=pending_closures34782026/11/12 00:46:15 INFO Received push request method=POST path=/34792026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts34802026/11/12 00:46:15 INFO Vacuumed table table=pending_objects34812026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads34822026/11/12 00:46:15 INFO Vacuumed table table=closures34832026/11/12 00:46:15 INFO Vacuumed table table=objects34842026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=034852026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period34862026/11/12 00:46:15 INFO Received complete push request method=POST path=/34872026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034882026/11/12 00:46:15 INFO Vacuumed table table=pending_closures34892026/11/12 00:46:15 INFO Completed push id=8 objects=6 duration=2.285263ms34902026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts34912026/11/12 00:46:15 INFO Received push request method=POST path=/34922026/11/12 00:46:15 INFO Vacuumed table table=pending_objects34932026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads34942026/11/12 00:46:15 INFO Vacuumed table table=closures34952026/11/12 00:46:15 INFO Received push request method=POST path=/34962026/11/12 00:46:15 INFO Vacuumed table table=objects34972026/11/12 00:46:15 INFO Received uploads request method=POST path=/34982026/11/12 00:46:15 INFO Received complete push request method=POST path=/34992026/11/12 00:46:15 INFO Completed push id=24 objects=9 duration=1.538475ms35002026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=035012026/11/12 00:46:15 INFO Received uploads request method=POST path=/35022026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period35032026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035042026/11/12 00:46:15 INFO Vacuumed table table=pending_closures35052026/11/12 00:46:15 INFO Received complete push request method=POST path=/35062026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts35072026/11/12 00:46:15 INFO Vacuumed table table=pending_objects35082026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads35092026/11/12 00:46:15 INFO Vacuumed table table=closures35102026/11/12 00:46:15 INFO Completed push id=22 objects=3 duration=2.856769ms35112026/11/12 00:46:15 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."35122026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."35132026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."35142026/11/12 00:46:15 INFO Vacuumed table table=objects35152026/11/12 00:46:15 INFO Received push request method=POST path=/35162026/11/12 00:46:15 INFO Received push request method=POST path=/35172026/11/12 00:46:15 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."35182026/11/12 00:46:15 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."35192026/11/12 00:46:15 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."35202026/11/12 00:46:15 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."35212026/11/12 00:46:15 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."35222026/11/12 00:46:15 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."35232026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."35242026/11/12 00:46:15 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."35252026/11/12 00:46:15 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."35262026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."35272026/11/12 00:46:15 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."35282026/11/12 00:46:15 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."35292026/11/12 00:46:15 INFO Received push request method=POST path=/35302026/11/12 00:46:15 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."35312026/11/12 00:46:15 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."35322026/11/12 00:46:15 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."35332026/11/12 00:46:15 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."35342026/11/12 00:46:15 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."35352026/11/12 00:46:15 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."35362026/11/12 00:46:15 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."35372026/11/12 00:46:15 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."35382026/11/12 00:46:15 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."35392026/11/12 00:46:15 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."35402026/11/12 00:46:15 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."35412026/11/12 00:46:15 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."35422026/11/12 00:46:15 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."35432026/11/12 00:46:15 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."35442026/11/12 00:46:15 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."35452026/11/12 00:46:15 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."35462026/11/12 00:46:15 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."35472026/11/12 00:46:15 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."35482026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."35492026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."35502026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."35512026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."35522026/11/12 00:46:15 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."35532026/11/12 00:46:15 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."35542026/11/12 00:46:15 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."35552026/11/12 00:46:15 INFO Received complete upload request method=POST path=/35562026/11/12 00:46:15 INFO Received push request method=POST path=/35572026/11/12 00:46:15 INFO Completed upload id=15 objects=9 duration=1.765358ms35582026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=035592026/11/12 00:46:15 INFO Received uploads request method=POST path=/35602026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period35612026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=035622026/11/12 00:46:15 INFO Received push request method=POST path=/35632026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period35642026/11/12 00:46:15 INFO Received push request method=POST path=/35652026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035662026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=035672026/11/12 00:46:15 INFO Vacuumed table table=pending_closures35682026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts35692026/11/12 00:46:15 INFO Received complete push request method=POST path=/35702026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period35712026/11/12 00:46:15 INFO Vacuumed table table=pending_objects35722026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads35732026/11/12 00:46:15 INFO Vacuumed table table=closures35742026/11/12 00:46:15 INFO Received push request method=POST path=/35752026/11/12 00:46:15 INFO Completed push id=11 objects=11 duration=4.080101ms35762026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035772026/11/12 00:46:15 INFO Received push request method=POST path=/35782026/11/12 00:46:15 INFO Vacuumed table table=objects35792026/11/12 00:46:15 INFO Vacuumed table table=pending_closures35802026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts35812026/11/12 00:46:15 INFO Vacuumed table table=pending_objects35822026/11/12 00:46:15 INFO Received complete upload request method=POST path=/35832026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads35842026/11/12 00:46:15 INFO Vacuumed table table=closures35852026/11/12 00:46:15 INFO Received uploads request method=POST path=/35862026/11/12 00:46:15 INFO Completed upload id=3 objects=8 duration=7.513456ms35872026/11/12 00:46:15 INFO Received complete upload request method=POST path=/35882026/11/12 00:46:15 INFO Vacuumed table table=objects35892026/11/12 00:46:15 INFO Completed upload id=26 objects=6 duration=2.726247ms35902026/11/12 00:46:15 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."35912026/11/12 00:46:15 INFO Received push request method=POST path=/35922026/11/12 00:46:15 INFO Received uploads request method=POST path=/35932026/11/12 00:46:15 INFO Received push request method=POST path=/35942026/11/12 00:46:15 INFO Received push request method=POST path=/35952026/11/12 00:46:15 INFO Received uploads request method=POST path=/35962026/11/12 00:46:15 INFO Received uploads request method=POST path=/35972026/11/12 00:46:15 INFO Received complete push request method=POST path=/35982026/11/12 00:46:15 INFO Received push request method=POST path=/35992026/11/12 00:46:15 INFO Received complete upload request method=POST path=/36002026/11/12 00:46:15 INFO Received complete upload request method=POST path=/36012026/11/12 00:46:15 INFO Received push request method=POST path=/36022026/11/12 00:46:15 INFO Completed push id=21 objects=17 duration=2.307283ms36032026/11/12 00:46:15 INFO Completed upload id=19 objects=25 duration=2.257063ms36042026/11/12 00:46:15 INFO Received uploads request method=POST path=/36052026/11/12 00:46:15 INFO Received push request method=POST path=/36062026/11/12 00:46:15 INFO Completed upload id=17 objects=25 duration=3.457575ms36072026/11/12 00:46:15 INFO Received push request method=POST path=/36082026/11/12 00:46:15 INFO Received uploads request method=POST path=/36092026/11/12 00:46:15 INFO Received complete upload request method=POST path=/36102026/11/12 00:46:15 INFO Received push request method=POST path=/36112026/11/12 00:46:15 INFO Completed upload id=29 objects=5 duration=918.569µs36122026/11/12 00:46:15 INFO Received uploads request method=POST path=/36132026/11/12 00:46:15 INFO Received uploads request method=POST path=/36142026/11/12 00:46:15 INFO Received complete push request method=POST path=/36152026/11/12 00:46:15 INFO Received complete push request method=POST path=/36162026/11/12 00:46:15 INFO Received complete upload request method=POST path=/36172026/11/12 00:46:15 INFO Completed push id=9 objects=24 duration=1.320453ms36182026/11/12 00:46:15 INFO Completed push id=4 objects=6 duration=1.822798ms36192026/11/12 00:46:15 INFO Received complete push request method=POST path=/36202026/11/12 00:46:15 INFO Completed upload id=4 objects=14 duration=3.735297ms36212026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=036222026/11/12 00:46:15 INFO Completed push id=19 objects=7 duration=2.286263ms36232026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period36242026/11/12 00:46:15 INFO Received push request method=POST path=/36252026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=036262026/11/12 00:46:15 INFO Received push request method=POST path=/36272026/11/12 00:46:15 INFO Received push request method=POST path=/36282026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=036292026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036302026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=036312026/11/12 00:46:15 INFO Received uploads request method=POST path=/36322026/11/12 00:46:15 INFO Vacuumed table table=pending_closures36332026/11/12 00:46:15 INFO Vacuumed table table=pending_closures36342026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts36352026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts36362026/11/12 00:46:15 INFO Received uploads request method=POST path=/36372026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=036382026/11/12 00:46:15 INFO Vacuumed table table=pending_objects36392026/11/12 00:46:15 INFO Vacuumed table table=pending_objects36402026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads36412026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads36422026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=036432026/11/12 00:46:15 INFO Vacuumed table table=closures36442026/11/12 00:46:15 INFO Vacuumed table table=closures36452026/11/12 00:46:15 INFO Vacuumed table table=pending_closures36462026/11/12 00:46:15 INFO Received push request method=POST path=/36472026/11/12 00:46:15 INFO Vacuumed table table=objects36482026/11/12 00:46:15 INFO Vacuumed table table=objects36492026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts36502026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period36512026/11/12 00:46:15 INFO Received push request method=POST path=/36522026/11/12 00:46:15 INFO Vacuumed table table=pending_objects36532026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads36542026/11/12 00:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036552026/11/12 00:46:15 INFO Received push request method=POST path=/36562026/11/12 00:46:15 INFO Vacuumed table table=closures36572026/11/12 00:46:15 INFO Vacuumed table table=pending_closures36582026/11/12 00:46:15 INFO Vacuumed table table=commit_receipts36592026/11/12 00:46:15 INFO Vacuumed table table=objects36602026/11/12 00:46:15 INFO Vacuumed table table=pending_objects36612026/11/12 00:46:15 INFO Vacuumed table table=multipart_uploads36622026/11/12 00:46:15 INFO Received complete upload request method=POST path=/36632026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=036642026/11/12 00:46:15 INFO Vacuumed table table=closures36652026/11/12 00:46:15 INFO Completed upload id=21 objects=2 duration=979.53µs36662026/11/12 00:46:15 INFO Received push request method=POST path=/36672026/11/12 00:46:15 INFO Vacuumed table table=objects36682026/11/12 00:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period36692026/11/12 00:46:15 INFO Aborted multipart uploads count=0 kept=036702026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036712026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=036722026/11/12 00:46:16 INFO Vacuumed table table=pending_closures36732026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period36742026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts36752026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period36762026/11/12 00:46:16 INFO Received uploads request method=POST path=/36772026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=036782026/11/12 00:46:16 INFO Vacuumed table table=pending_objects36792026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036802026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads36812026/11/12 00:46:16 INFO Received uploads request method=POST path=/36822026/11/12 00:46:16 INFO Vacuumed table table=closures36832026/11/12 00:46:16 INFO Received push request method=POST path=/36842026/11/12 00:46:16 INFO Vacuumed table table=pending_closures36852026/11/12 00:46:16 INFO Vacuumed table table=objects36862026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts36872026/11/12 00:46:16 INFO Vacuumed table table=pending_objects36882026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads36892026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=036902026/11/12 00:46:16 INFO Vacuumed table table=closures36912026/11/12 00:46:16 INFO Received complete upload request method=POST path=/36922026/11/12 00:46:16 INFO Received push request method=POST path=/36932026/11/12 00:46:16 INFO Vacuumed table table=objects36942026/11/12 00:46:16 INFO Completed upload id=9 objects=2 duration=1.121371ms36952026/11/12 00:46:16 INFO Received push request method=POST path=/36962026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period36972026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=036982026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period36992026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037002026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=037012026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037022026/11/12 00:46:16 INFO Vacuumed table table=pending_closures37032026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts37042026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period37052026/11/12 00:46:16 INFO Vacuumed table table=pending_closures37062026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts37072026/11/12 00:46:16 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."37082026/11/12 00:46:16 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."37092026/11/12 00:46:16 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."37102026/11/12 00:46:16 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."37112026/11/12 00:46:16 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."37122026/11/12 00:46:16 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."37132026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."37142026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."37152026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."37162026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=037172026/11/12 00:46:16 INFO Vacuumed table table=pending_objects37182026/11/12 00:46:16 INFO Vacuumed table table=pending_objects37192026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037202026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads37212026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads37222026/11/12 00:46:16 INFO Vacuumed table table=closures37232026/11/12 00:46:16 INFO Vacuumed table table=closures37242026/11/12 00:46:16 INFO Vacuumed table table=pending_closures37252026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts37262026/11/12 00:46:16 INFO Vacuumed table table=objects37272026/11/12 00:46:16 INFO Vacuumed table table=objects37282026/11/12 00:46:16 INFO Vacuumed table table=pending_objects37292026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads37302026/11/12 00:46:16 INFO Vacuumed table table=closures37312026/11/12 00:46:16 INFO Vacuumed table table=objects37322026/11/12 00:46:16 INFO Received push request method=POST path=/37332026/11/12 00:46:16 INFO Received complete push request method=POST path=/37342026/11/12 00:46:16 INFO Received complete upload request method=POST path=/37352026/11/12 00:46:16 INFO Received push request method=POST path=/37362026/11/12 00:46:16 INFO Completed upload id=10 objects=10 duration=868.669µs37372026/11/12 00:46:16 INFO Completed push id=8 objects=3 duration=1.453614ms37382026/11/12 00:46:16 INFO Received uploads request method=POST path=/37392026/11/12 00:46:16 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."37402026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."37412026/11/12 00:46:16 INFO Received complete upload request method=POST path=/37422026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=037432026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=037442026/11/12 00:46:16 INFO Completed upload id=10 objects=2 duration=2.090741ms37452026/11/12 00:46:16 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."37462026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."37472026/11/12 00:46:16 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."37482026/11/12 00:46:16 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."37492026/11/12 00:46:16 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."37502026/11/12 00:46:16 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."37512026/11/12 00:46:16 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."37522026/11/12 00:46:16 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."37532026/11/12 00:46:16 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."37542026/11/12 00:46:16 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."37552026/11/12 00:46:16 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."37562026/11/12 00:46:16 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."37572026/11/12 00:46:16 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."37582026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period37592026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:aaaaaaaa!out.doi error="Access Denied."37602026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."37612026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."37622026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."37632026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."37642026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."37652026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."37662026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037672026/11/12 00:46:16 INFO Vacuumed table table=pending_closures37682026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts37692026/11/12 00:46:16 INFO Vacuumed table table=pending_objects37702026/11/12 00:46:16 INFO Received complete push request method=POST path=/37712026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads37722026/11/12 00:46:16 INFO Received push request method=POST path=/37732026/11/12 00:46:16 INFO Completed push id=21 objects=3 duration=1.09607ms37742026/11/12 00:46:16 INFO Received push request method=POST path=/37752026/11/12 00:46:16 INFO Vacuumed table table=closures37762026/11/12 00:46:16 INFO Vacuumed table table=objects37772026/11/12 00:46:16 INFO Received uploads request method=POST path=/37782026/11/12 00:46:16 INFO Received push request method=POST path=/37792026/11/12 00:46:16 INFO Received push request method=POST path=/37802026/11/12 00:46:16 INFO Received uploads request method=POST path=/37812026/11/12 00:46:16 INFO Received uploads request method=POST path=/37822026/11/12 00:46:16 INFO Received uploads request method=POST path=/37832026/11/12 00:46:16 INFO Received push request method=POST path=/37842026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=9 objects-failed-to-delete=037852026/11/12 00:46:16 INFO Vacuumed table table=pending_closures37862026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts37872026/11/12 00:46:16 INFO Vacuumed table table=pending_objects37882026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads37892026/11/12 00:46:16 INFO Received push request method=POST path=/37902026/11/12 00:46:16 INFO Received uploads request method=POST path=/37912026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=037922026/11/12 00:46:16 INFO Vacuumed table table=closures37932026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=037942026/11/12 00:46:16 INFO Received uploads request method=POST path=/37952026/11/12 00:46:16 INFO Vacuumed table table=objects37962026/11/12 00:46:16 INFO Received uploads request method=POST path=/37972026/11/12 00:46:16 INFO Received uploads request method=POST path=/37982026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037992026/11/12 00:46:16 INFO Received uploads request method=POST path=/38002026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=038012026/11/12 00:46:16 INFO Vacuumed table table=pending_closures38022026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts38032026/11/12 00:46:16 INFO Received push request method=POST path=/38042026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period38052026/11/12 00:46:16 INFO Vacuumed table table=pending_objects38062026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads38072026/11/12 00:46:16 INFO Vacuumed table table=closures38082026/11/12 00:46:16 INFO Vacuumed table table=objects38092026/11/12 00:46:16 INFO Received push request method=POST path=/38102026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=038112026/11/12 00:46:16 INFO Vacuumed table table=pending_closures38122026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts38132026/11/12 00:46:16 INFO Received complete upload request method=POST path=/38142026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=038152026/11/12 00:46:16 INFO Vacuumed table table=pending_objects38162026/11/12 00:46:16 INFO Received complete upload request method=POST path=/38172026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads38182026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period38192026/11/12 00:46:16 INFO Completed upload id=13 objects=10 duration=829.129µs38202026/11/12 00:46:16 INFO Received uploads request method=POST path=/38212026/11/12 00:46:16 INFO Received push request method=POST path=/38222026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=038232026/11/12 00:46:16 INFO Completed upload id=12 objects=26 duration=3.528255ms38242026/11/12 00:46:16 INFO Vacuumed table table=closures38252026/11/12 00:46:16 INFO Vacuumed table table=pending_closures38262026/11/12 00:46:16 INFO Vacuumed table table=objects38272026/11/12 00:46:16 INFO Received uploads request method=POST path=/38282026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts38292026/11/12 00:46:16 INFO Vacuumed table table=pending_objects38302026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads38312026/11/12 00:46:16 INFO Vacuumed table table=closures38322026/11/12 00:46:16 INFO Received push request method=POST path=/38332026/11/12 00:46:16 INFO Received uploads request method=POST path=/38342026/11/12 00:46:16 INFO Received complete push request method=POST path=/38352026/11/12 00:46:16 INFO Received uploads request method=POST path=/38362026/11/12 00:46:16 INFO Vacuumed table table=objects38372026/11/12 00:46:16 INFO Completed push id=7 objects=24 duration=3.470455ms38382026/11/12 00:46:16 INFO Received complete push request method=POST path=/38392026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=038402026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period38412026/11/12 00:46:16 INFO Received complete upload request method=POST path=/38422026/11/12 00:46:16 INFO Completed push id=5 objects=8 duration=3.668757ms38432026/11/12 00:46:16 INFO Received push request method=POST path=/38442026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=038452026/11/12 00:46:16 INFO Received push request method=POST path=/38462026/11/12 00:46:16 INFO Completed upload id=23 objects=16 duration=2.334283ms38472026/11/12 00:46:16 INFO Received push request method=POST path=/38482026/11/12 00:46:16 INFO Received complete upload request method=POST path=/38492026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=038502026/11/12 00:46:16 INFO Received push request method=POST path=/38512026/11/12 00:46:16 INFO Completed upload id=31 objects=3 duration=1.952239ms38522026/11/12 00:46:16 INFO Vacuumed table table=pending_closures38532026/11/12 00:46:16 INFO Received complete upload request method=POST path=/38542026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts38552026/11/12 00:46:16 INFO Received push request method=POST path=/38562026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=3 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038572026/11/12 00:46:16 INFO Vacuumed table table=pending_objects38582026/11/12 00:46:16 INFO Received complete upload request method=POST path=/38592026/11/12 00:46:16 INFO Vacuumed table table=pending_closures38602026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads38612026/11/12 00:46:16 INFO Received uploads request method=POST path=/38622026/11/12 00:46:16 INFO Completed upload id=21 objects=7 duration=2.308163ms38632026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts38642026/11/12 00:46:16 INFO Vacuumed table table=closures38652026/11/12 00:46:16 INFO Vacuumed table table=objects38662026/11/12 00:46:16 INFO Completed upload id=23 objects=18 duration=3.04593ms38672026/11/12 00:46:16 INFO Received push request method=POST path=/38682026/11/12 00:46:16 INFO Vacuumed table table=pending_objects38692026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads38702026/11/12 00:46:16 INFO Vacuumed table table=closures38712026/11/12 00:46:16 INFO Vacuumed table table=objects38722026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=038732026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period38742026/11/12 00:46:16 INFO Received uploads request method=POST path=/38752026/11/12 00:46:16 INFO Received push request method=POST path=/38762026/11/12 00:46:16 INFO Received complete push request method=POST path=/38772026/11/12 00:46:16 INFO Received push request method=POST path=/38782026/11/12 00:46:16 INFO Received complete upload request method=POST path=/38792026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038802026/11/12 00:46:16 INFO Completed upload id=22 objects=2 duration=1.02999ms38812026/11/12 00:46:16 INFO Received uploads request method=POST path=/38822026/11/12 00:46:16 INFO Received push request method=POST path=/38832026/11/12 00:46:16 INFO Vacuumed table table=pending_closures38842026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts38852026/11/12 00:46:16 INFO Completed push id=26 objects=14 duration=3.00865ms38862026/11/12 00:46:16 INFO Vacuumed table table=pending_objects38872026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads38882026/11/12 00:46:16 INFO Received uploads request method=POST path=/38892026/11/12 00:46:16 INFO Vacuumed table table=closures38902026/11/12 00:46:16 INFO Received uploads request method=POST path=/38912026/11/12 00:46:16 INFO Vacuumed table table=objects38922026/11/12 00:46:16 INFO Received uploads request method=POST path=/38932026/11/12 00:46:16 INFO Received push request method=POST path=/38942026/11/12 00:46:16 INFO Received uploads request method=POST path=/38952026/11/12 00:46:16 INFO Received push request method=POST path=/38962026/11/12 00:46:16 INFO Received uploads request method=POST path=/38972026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=038982026/11/12 00:46:16 INFO Received push request method=POST path=/38992026/11/12 00:46:16 INFO Received complete push request method=POST path=/39002026/11/12 00:46:16 INFO Received push request method=POST path=/39012026/11/12 00:46:16 INFO Completed push id=16 objects=8 duration=1.838698ms39022026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period39032026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=039042026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=11 objects-failed-to-delete=039052026/11/12 00:46:16 INFO Vacuumed table table=pending_closures39062026/11/12 00:46:16 INFO Received complete push request method=POST path=/39072026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039082026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts39092026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039102026/11/12 00:46:16 INFO Received uploads request method=POST path=/39112026/11/12 00:46:16 INFO Received push request method=POST path=/39122026/11/12 00:46:16 INFO Vacuumed table table=pending_closures39132026/11/12 00:46:16 INFO Vacuumed table table=pending_closures39142026/11/12 00:46:16 INFO Completed push id=11 objects=8 duration=3.174711ms39152026/11/12 00:46:16 INFO Received complete push request method=POST path=/39162026/11/12 00:46:16 INFO Received push request method=POST path=/39172026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts39182026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts39192026/11/12 00:46:16 INFO Vacuumed table table=pending_objects39202026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads39212026/11/12 00:46:16 INFO Received uploads request method=POST path=/39222026/11/12 00:46:16 INFO Completed push id=14 objects=3 duration=1.078171ms39232026/11/12 00:46:16 INFO Vacuumed table table=pending_objects39242026/11/12 00:46:16 INFO Vacuumed table table=pending_objects39252026/11/12 00:46:16 INFO Vacuumed table table=closures39262026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads39272026/11/12 00:46:16 INFO Received uploads request method=POST path=/39282026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads39292026/11/12 00:46:16 INFO Vacuumed table table=closures39302026/11/12 00:46:16 INFO Vacuumed table table=closures39312026/11/12 00:46:16 INFO Vacuumed table table=objects39322026/11/12 00:46:16 INFO Vacuumed table table=objects39332026/11/12 00:46:16 INFO Vacuumed table table=objects39342026/11/12 00:46:16 INFO Received uploads request method=POST path=/39352026/11/12 00:46:16 INFO Received uploads request method=POST path=/39362026/11/12 00:46:16 INFO Received push request method=POST path=/39372026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=039382026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=039392026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period39402026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039412026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039422026/11/12 00:46:16 INFO Vacuumed table table=pending_closures39432026/11/12 00:46:16 INFO Vacuumed table table=pending_closures39442026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts39452026/11/12 00:46:16 INFO Received push request method=POST path=/39462026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts39472026/11/12 00:46:16 INFO Vacuumed table table=pending_objects39482026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads39492026/11/12 00:46:16 INFO Vacuumed table table=pending_objects39502026/11/12 00:46:16 INFO Vacuumed table table=closures39512026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads39522026/11/12 00:46:16 INFO Vacuumed table table=closures39532026/11/12 00:46:16 INFO Vacuumed table table=objects39542026/11/12 00:46:16 INFO Received uploads request method=POST path=/39552026/11/12 00:46:16 INFO Received complete push request method=POST path=/39562026/11/12 00:46:16 INFO Received complete upload request method=POST path=/39572026/11/12 00:46:16 INFO Received push request method=POST path=/39582026/11/12 00:46:16 INFO Received push request method=POST path=/39592026/11/12 00:46:16 INFO Received complete upload request method=POST path=/39602026/11/12 00:46:16 INFO Received push request method=POST path=/39612026/11/12 00:46:16 INFO Vacuumed table table=objects39622026/11/12 00:46:16 INFO Completed upload id=22 objects=22 duration=1.150311ms39632026/11/12 00:46:16 INFO Completed upload id=18 objects=2 duration=2.080601ms39642026/11/12 00:46:16 INFO Completed push id=30 objects=4 duration=2.96869ms39652026/11/12 00:46:16 INFO Received complete push request method=POST path=/39662026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=039672026/11/12 00:46:16 INFO Completed push id=16 objects=4 duration=2.345583ms39682026/11/12 00:46:16 INFO Received uploads request method=POST path=/39692026/11/12 00:46:16 INFO Received push request method=POST path=/39702026/11/12 00:46:16 INFO Received push request method=POST path=/39712026/11/12 00:46:16 INFO Received uploads request method=POST path=/39722026/11/12 00:46:16 INFO Received push request method=POST path=/39732026/11/12 00:46:16 INFO Received uploads request method=POST path=/39742026/11/12 00:46:16 INFO Received push request method=POST path=/39752026/11/12 00:46:16 INFO Received push request method=POST path=/39762026/11/12 00:46:16 INFO Received uploads request method=POST path=/39772026/11/12 00:46:16 INFO Received complete push request method=POST path=/39782026/11/12 00:46:16 INFO Received complete push request method=POST path=/39792026/11/12 00:46:16 INFO Received uploads request method=POST path=/39802026/11/12 00:46:16 INFO Completed push id=11 objects=6 duration=2.477725ms39812026/11/12 00:46:16 INFO Received complete push request method=POST path=/39822026/11/12 00:46:16 INFO Received uploads request method=POST path=/39832026/11/12 00:46:16 INFO Completed push id=13 objects=12 duration=3.443474ms39842026/11/12 00:46:16 INFO Received uploads request method=POST path=/39852026/11/12 00:46:16 INFO Completed push id=13 objects=6 duration=2.747188ms39862026/11/12 00:46:16 INFO Received push request method=POST path=/39872026/11/12 00:46:16 INFO Received complete upload request method=POST path=/39882026/11/12 00:46:16 INFO Completed upload id=27 objects=4 duration=866.129µs39892026/11/12 00:46:16 INFO Received complete push request method=POST path=/39902026/11/12 00:46:16 INFO Received complete push request method=POST path=/39912026/11/12 00:46:16 INFO Received push request method=POST path=/39922026/11/12 00:46:16 INFO Completed push id=14 objects=4 duration=2.582866ms39932026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=039942026/11/12 00:46:16 INFO Completed push id=13 objects=2 duration=1.221892ms39952026/11/12 00:46:16 INFO Received push request method=POST path=/39962026/11/12 00:46:16 INFO Received complete push request method=POST path=/39972026/11/12 00:46:16 INFO Received complete upload request method=POST path=/39982026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=039992026/11/12 00:46:16 INFO Completed upload id=20 objects=5 duration=1.326533ms40002026/11/12 00:46:16 INFO Received push request method=POST path=/40012026/11/12 00:46:16 INFO Completed push id=26 objects=5 duration=2.133381ms40022026/11/12 00:46:16 INFO Received push request method=POST path=/40032026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period40042026/11/12 00:46:16 INFO Received complete upload request method=POST path=/40052026/11/12 00:46:16 INFO Received complete push request method=POST path=/40062026/11/12 00:46:16 INFO Completed upload id=13 objects=7 duration=1.00573ms40072026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040082026/11/12 00:46:16 INFO Completed push id=18 objects=6 duration=1.118491ms40092026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=040102026/11/12 00:46:16 INFO Vacuumed table table=pending_closures40112026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts40122026/11/12 00:46:16 INFO Received complete push request method=POST path=/40132026/11/12 00:46:16 INFO Vacuumed table table=pending_objects40142026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period40152026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads40162026/11/12 00:46:16 INFO Vacuumed table table=closures40172026/11/12 00:46:16 INFO Completed push id=13 objects=11 duration=1.640457ms40182026/11/12 00:46:16 INFO Vacuumed table table=objects40192026/11/12 00:46:16 INFO Received push request method=POST path=/40202026/11/12 00:46:16 INFO Received push request method=POST path=/40212026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040222026/11/12 00:46:16 INFO Received complete upload request method=POST path=/40232026/11/12 00:46:16 INFO Completed upload id=27 objects=13 duration=1.697657ms40242026/11/12 00:46:16 INFO Vacuumed table table=pending_closures40252026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=040262026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts40272026/11/12 00:46:16 INFO Received push request method=POST path=/40282026/11/12 00:46:16 INFO Vacuumed table table=pending_closures40292026/11/12 00:46:16 INFO Vacuumed table table=pending_objects40302026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts40312026/11/12 00:46:16 INFO Received complete upload request method=POST path=/40322026/11/12 00:46:16 INFO Received uploads request method=POST path=/40332026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads40342026/11/12 00:46:16 INFO Received complete upload request method=POST path=/40352026/11/12 00:46:16 INFO Completed upload id=24 objects=14 duration=1.059651ms40362026/11/12 00:46:16 INFO Vacuumed table table=pending_objects40372026/11/12 00:46:16 INFO Vacuumed table table=closures40382026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads40392026/11/12 00:46:16 INFO Vacuumed table table=objects40402026/11/12 00:46:16 INFO Received push request method=POST path=/40412026/11/12 00:46:16 INFO Received complete upload request method=POST path=/40422026/11/12 00:46:16 INFO Vacuumed table table=closures40432026/11/12 00:46:16 INFO Vacuumed table table=objects40442026/11/12 00:46:16 INFO Completed upload id=20 objects=4 duration=1.094831ms40452026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=040462026/11/12 00:46:16 INFO Completed upload id=8 objects=3 duration=3.646617ms40472026/11/12 00:46:16 INFO Received uploads request method=POST path=/40482026/11/12 00:46:16 INFO Received uploads request method=POST path=/40492026/11/12 00:46:16 INFO Received complete push request method=POST path=/40502026/11/12 00:46:16 INFO Completed push id=7 objects=15 duration=2.175101ms40512026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=040522026/11/12 00:46:16 INFO Received uploads request method=POST path=/40532026/11/12 00:46:16 INFO Received complete push request method=POST path=/40542026/11/12 00:46:16 INFO Received push request method=POST path=/40552026/11/12 00:46:16 INFO Received complete push request method=POST path=/40562026/11/12 00:46:16 INFO Completed push id=32 objects=7 duration=1.05819ms40572026/11/12 00:46:16 INFO Received uploads request method=POST path=/40582026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040592026/11/12 00:46:16 INFO Completed push id=19 objects=3 duration=1.05323ms40602026/11/12 00:46:16 INFO Vacuumed table table=pending_closures40612026/11/12 00:46:16 INFO Received push request method=POST path=/40622026/11/12 00:46:16 INFO Received complete upload request method=POST path=/40632026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts40642026/11/12 00:46:16 INFO Completed upload id=23 objects=7 duration=928.889µs40652026/11/12 00:46:16 INFO Vacuumed table table=pending_objects40662026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=040672026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads40682026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=040692026/11/12 00:46:16 INFO Vacuumed table table=closures40702026/11/12 00:46:16 INFO Vacuumed table table=objects40712026/11/12 00:46:16 INFO Received push request method=POST path=/40722026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period40732026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period40742026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040752026/11/12 00:46:16 INFO Vacuumed table table=pending_closures40762026/11/12 00:46:16 INFO Received push request method=POST path=/40772026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts40782026/11/12 00:46:16 INFO Vacuumed table table=pending_objects40792026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads40802026/11/12 00:46:16 INFO Vacuumed table table=closures40812026/11/12 00:46:16 INFO Received complete push request method=POST path=/40822026/11/12 00:46:16 INFO Vacuumed table table=objects40832026/11/12 00:46:16 INFO Completed push id=26 objects=4 duration=1.292073ms40842026/11/12 00:46:16 INFO Received push request method=POST path=/40852026/11/12 00:46:16 INFO Received complete push request method=POST path=/40862026/11/12 00:46:16 INFO Received uploads request method=POST path=/40872026/11/12 00:46:16 INFO Completed push id=17 objects=6 duration=1.768818ms40882026/11/12 00:46:16 INFO Received uploads request method=POST path=/40892026/11/12 00:46:16 INFO Received push request method=POST path=/40902026/11/12 00:46:16 INFO Received push request method=POST path=/40912026/11/12 00:46:16 INFO Received uploads request method=POST path=/40922026/11/12 00:46:16 INFO Received complete upload request method=POST path=/40932026/11/12 00:46:16 INFO Received push request method=POST path=/40942026/11/12 00:46:16 INFO Completed upload id=28 objects=13 duration=1.08149ms40952026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=040962026/11/12 00:46:16 INFO Received uploads request method=POST path=/40972026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=040982026/11/12 00:46:16 INFO Received push request method=POST path=/40992026/11/12 00:46:16 INFO Received complete push request method=POST path=/41002026/11/12 00:46:16 INFO Received push request method=POST path=/41012026/11/12 00:46:16 INFO Received complete upload request method=POST path=/41022026/11/12 00:46:16 INFO Completed push id=6 objects=6 duration=1.856379ms41032026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period41042026/11/12 00:46:16 INFO Completed upload id=24 objects=18 duration=1.01183ms41052026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041062026/11/12 00:46:16 INFO Vacuumed table table=pending_closures41072026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts41082026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=041092026/11/12 00:46:16 INFO Vacuumed table table=pending_closures41102026/11/12 00:46:16 INFO Vacuumed table table=pending_objects41112026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts41122026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads41132026/11/12 00:46:16 INFO Vacuumed table table=closures41142026/11/12 00:46:16 INFO Received complete push request method=POST path=/41152026/11/12 00:46:16 INFO Received complete push request method=POST path=/41162026/11/12 00:46:16 INFO Vacuumed table table=pending_objects41172026/11/12 00:46:16 INFO Vacuumed table table=objects41182026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads41192026/11/12 00:46:16 INFO Vacuumed table table=closures41202026/11/12 00:46:16 INFO Received complete push request method=POST path=/41212026/11/12 00:46:16 INFO Vacuumed table table=objects41222026/11/12 00:46:16 INFO Completed push id=29 objects=3 duration=967.669µs41232026/11/12 00:46:16 INFO Completed push id=4 objects=7 duration=2.324063ms41242026/11/12 00:46:16 INFO Completed push id=19 objects=6 duration=3.183252ms41252026/11/12 00:46:16 INFO Received push request method=POST path=/41262026/11/12 00:46:16 INFO Received complete push request method=POST path=/41272026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=041282026/11/12 00:46:16 INFO Received push request method=POST path=/41292026/11/12 00:46:16 INFO Completed push id=25 objects=2 duration=1.521715ms41302026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period41312026/11/12 00:46:16 INFO Received push request method=POST path=/41322026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041332026/11/12 00:46:16 INFO Received uploads request method=POST path=/41342026/11/12 00:46:16 INFO Vacuumed table table=pending_closures41352026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts41362026/11/12 00:46:16 INFO Vacuumed table table=pending_objects41372026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads41382026/11/12 00:46:16 INFO Received uploads request method=POST path=/41392026/11/12 00:46:16 INFO Vacuumed table table=closures41402026/11/12 00:46:16 INFO Received complete upload request method=POST path=/41412026/11/12 00:46:16 INFO Vacuumed table table=objects41422026/11/12 00:46:16 INFO Completed upload id=31 objects=2 duration=1.172352ms41432026/11/12 00:46:16 INFO Received push request method=POST path=/41442026/11/12 00:46:16 INFO Received uploads request method=POST path=/41452026/11/12 00:46:16 INFO Received uploads request method=POST path=/41462026/11/12 00:46:16 INFO Received complete push request method=POST path=/41472026/11/12 00:46:16 INFO Received complete upload request method=POST path=/41482026/11/12 00:46:16 INFO Received complete push request method=POST path=/41492026/11/12 00:46:16 INFO Completed upload id=25 objects=4 duration=852.549µs41502026/11/12 00:46:16 INFO Received complete push request method=POST path=/41512026/11/12 00:46:16 INFO Completed push id=33 objects=6 duration=824.288µs41522026/11/12 00:46:16 INFO Received uploads request method=POST path=/41532026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=041542026/11/12 00:46:16 INFO Completed push id=16 objects=16 duration=3.362574ms41552026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=041562026/11/12 00:46:16 INFO Completed push id=3 objects=21 duration=6.264603ms41572026/11/12 00:46:16 INFO Received complete push request method=POST path=/41582026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period41592026/11/12 00:46:16 INFO Completed push id=10 objects=15 duration=1.862418ms41602026/11/12 00:46:16 INFO Received uploads request method=POST path=/41612026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period41622026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041632026/11/12 00:46:16 INFO Received uploads request method=POST path=/41642026/11/12 00:46:16 INFO Vacuumed table table=pending_closures41652026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts41662026/11/12 00:46:16 INFO Vacuumed table table=pending_objects41672026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads41682026/11/12 00:46:16 INFO Vacuumed table table=closures41692026/11/12 00:46:16 INFO Received push request method=POST path=/41702026/11/12 00:46:16 INFO Vacuumed table table=objects41712026/11/12 00:46:16 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."41722026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."41732026/11/12 00:46:16 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."41742026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=041752026/11/12 00:46:16 INFO Received uploads request method=POST path=/41762026/11/12 00:46:16 INFO Received uploads request method=POST path=/41772026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period41782026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041792026/11/12 00:46:16 INFO Received complete upload request method=POST path=/41802026/11/12 00:46:16 INFO Vacuumed table table=pending_closures41812026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts41822026/11/12 00:46:16 INFO Completed upload id=29 objects=4 duration=2.719367ms41832026/11/12 00:46:16 INFO Received uploads request method=POST path=/41842026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=041852026/11/12 00:46:16 INFO Vacuumed table table=pending_objects41862026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads41872026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period41882026/11/12 00:46:16 INFO Vacuumed table table=closures41892026/11/12 00:46:16 INFO Vacuumed table table=objects41902026/11/12 00:46:16 INFO Received complete push request method=POST path=/41912026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=041922026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=041932026/11/12 00:46:16 INFO Completed push id=27 objects=4 duration=1.120871ms41942026/11/12 00:46:16 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."41952026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."41962026/11/12 00:46:16 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."41972026/11/12 00:46:16 INFO Received push request method=POST path=/41982026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041992026/11/12 00:46:16 INFO Received complete push request method=POST path=/42002026/11/12 00:46:16 INFO Vacuumed table table=pending_closures42012026/11/12 00:46:16 INFO Received complete push request method=POST path=/42022026/11/12 00:46:16 INFO Completed push id=7 objects=7 duration=1.273173ms42032026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts42042026/11/12 00:46:16 INFO Completed push id=19 objects=14 duration=1.529636ms42052026/11/12 00:46:16 INFO Vacuumed table table=pending_objects42062026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads42072026/11/12 00:46:16 INFO Vacuumed table table=closures42082026/11/12 00:46:16 INFO Received uploads request method=POST path=/42092026/11/12 00:46:16 INFO Vacuumed table table=objects42102026/11/12 00:46:16 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."42112026/11/12 00:46:16 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."42122026/11/12 00:46:16 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."42132026/11/12 00:46:16 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."42142026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."42152026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."42162026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."42172026/11/12 00:46:16 INFO Received uploads request method=POST path=/42182026/11/12 00:46:16 INFO Received complete push request method=POST path=/42192026/11/12 00:46:16 INFO Completed push id=20 objects=8 duration=854.289µs42202026/11/12 00:46:16 INFO Received uploads request method=POST path=/42212026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=042222026/11/12 00:46:16 INFO Received uploads request method=POST path=/42232026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period42242026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=042252026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042262026/11/12 00:46:16 INFO Vacuumed table table=pending_closures42272026/11/12 00:46:16 INFO Received uploads request method=POST path=/42282026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts42292026/11/12 00:46:16 INFO Received uploads request method=POST path=/42302026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period42312026/11/12 00:46:16 INFO Vacuumed table table=pending_objects42322026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads42332026/11/12 00:46:16 INFO Received push request method=POST path=/42342026/11/12 00:46:16 INFO Vacuumed table table=closures42352026/11/12 00:46:16 INFO Received complete push request method=POST path=/42362026/11/12 00:46:16 INFO Vacuumed table table=objects42372026/11/12 00:46:16 INFO Completed push id=28 objects=26 duration=1.363094ms42382026/11/12 00:46:16 INFO Received uploads request method=POST path=/42392026/11/12 00:46:16 INFO Received uploads request method=POST path=/42402026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=042412026/11/12 00:46:16 INFO Received uploads request method=POST path=/42422026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=042432026/11/12 00:46:16 INFO Received complete push request method=POST path=/42442026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period42452026/11/12 00:46:16 INFO Completed push id=30 objects=3 duration=1.01829ms42462026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042472026/11/12 00:46:16 INFO Received push request method=POST path=/42482026/11/12 00:46:16 INFO Received uploads request method=POST path=/42492026/11/12 00:46:16 INFO Vacuumed table table=pending_closures42502026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts42512026/11/12 00:46:16 INFO Received complete push request method=POST path=/42522026/11/12 00:46:16 INFO Received complete upload request method=POST path=/42532026/11/12 00:46:16 INFO Vacuumed table table=pending_objects42542026/11/12 00:46:16 INFO Completed push id=11 objects=6 duration=1.618676ms42552026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads42562026/11/12 00:46:16 INFO Completed upload id=18 objects=11 duration=1.207672ms42572026/11/12 00:46:16 INFO Vacuumed table table=closures42582026/11/12 00:46:16 INFO Vacuumed table table=objects42592026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=042602026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=042612026/11/12 00:46:16 INFO Vacuumed table table=pending_closures42622026/11/12 00:46:16 INFO Received complete push request method=POST path=/42632026/11/12 00:46:16 INFO Vacuumed table table=pending_closures42642026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts42652026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts42662026/11/12 00:46:16 INFO Vacuumed table table=pending_objects42672026/11/12 00:46:16 INFO Completed push id=15 objects=7 duration=2.249682ms42682026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads42692026/11/12 00:46:16 INFO Vacuumed table table=pending_objects42702026/11/12 00:46:16 INFO Vacuumed table table=closures42712026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads42722026/11/12 00:46:16 INFO Received push request method=POST path=/42732026/11/12 00:46:16 INFO Vacuumed table table=closures42742026/11/12 00:46:16 INFO Vacuumed table table=objects42752026/11/12 00:46:16 INFO Received complete push request method=POST path=/42762026/11/12 00:46:16 INFO Received push request method=POST path=/42772026/11/12 00:46:16 INFO Vacuumed table table=objects42782026/11/12 00:46:16 INFO Completed push id=32 objects=19 duration=983.009µs42792026/11/12 00:46:16 INFO Received push request method=POST path=/42802026/11/12 00:46:16 INFO Received uploads request method=POST path=/42812026/11/12 00:46:16 INFO Received uploads request method=POST path=/42822026/11/12 00:46:16 INFO Received complete push request method=POST path=/42832026/11/12 00:46:16 INFO Received complete push request method=POST path=/42842026/11/12 00:46:16 INFO Received push request method=POST path=/42852026/11/12 00:46:16 INFO Completed push id=20 objects=13 duration=1.961359ms42862026/11/12 00:46:16 INFO Completed push id=15 objects=5 duration=2.498206ms42872026/11/12 00:46:16 INFO Received push request method=POST path=/42882026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=042892026/11/12 00:46:16 INFO Received push request method=POST path=/42902026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period42912026/11/12 00:46:16 INFO Received complete push request method=POST path=/42922026/11/12 00:46:16 INFO Received complete push request method=POST path=/42932026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042942026/11/12 00:46:16 INFO Completed push id=27 objects=11 duration=916.75µs42952026/11/12 00:46:16 INFO Completed push id=18 objects=6 duration=3.122191ms42962026/11/12 00:46:16 INFO Vacuumed table table=pending_closures42972026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts42982026/11/12 00:46:16 INFO Received uploads request method=POST path=/42992026/11/12 00:46:16 INFO Vacuumed table table=pending_objects43002026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads43012026/11/12 00:46:16 INFO Vacuumed table table=closures43022026/11/12 00:46:16 INFO Vacuumed table table=objects43032026/11/12 00:46:16 INFO Received uploads request method=POST path=/43042026/11/12 00:46:16 INFO Received complete push request method=POST path=/43052026/11/12 00:46:16 INFO Received uploads request method=POST path=/43062026/11/12 00:46:16 INFO Completed push id=22 objects=4 duration=1.038231ms43072026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=043082026/11/12 00:46:16 INFO Received uploads request method=POST path=/43092026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period43102026/11/12 00:46:16 INFO Received complete push request method=POST path=/43112026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043122026/11/12 00:46:16 INFO Vacuumed table table=pending_closures43132026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts43142026/11/12 00:46:16 INFO Completed push id=9 objects=4 duration=3.320474ms43152026/11/12 00:46:16 INFO Received complete upload request method=POST path=/43162026/11/12 00:46:16 INFO Received complete upload request method=POST path=/43172026/11/12 00:46:16 INFO Vacuumed table table=pending_objects43182026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads43192026/11/12 00:46:16 INFO Completed upload id=23 objects=9 duration=983.77µs43202026/11/12 00:46:16 INFO Completed upload id=29 objects=26 duration=1.208572ms43212026/11/12 00:46:16 INFO Vacuumed table table=closures43222026/11/12 00:46:16 INFO Vacuumed table table=objects43232026/11/12 00:46:16 INFO Received uploads request method=POST path=/43242026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=043252026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period43262026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=043272026/11/12 00:46:16 INFO Received complete upload request method=POST path=/43282026/11/12 00:46:16 INFO Completed upload id=23 objects=7 duration=969.39µs43292026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043302026/11/12 00:46:16 INFO Vacuumed table table=pending_closures43312026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts43322026/11/12 00:46:16 INFO Vacuumed table table=pending_objects43332026/11/12 00:46:16 INFO Received complete push request method=POST path=/43342026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads43352026/11/12 00:46:16 INFO Vacuumed table table=closures43362026/11/12 00:46:16 INFO Completed push id=30 objects=8 duration=1.02037ms43372026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=043382026/11/12 00:46:16 INFO Vacuumed table table=objects43392026/11/12 00:46:16 INFO Received push request method=POST path=/43402026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period43412026/11/12 00:46:16 INFO Received push request method=POST path=/43422026/11/12 00:46:16 INFO Received complete push request method=POST path=/43432026/11/12 00:46:16 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."43442026/11/12 00:46:16 INFO Completed push id=24 objects=5 duration=1.06505ms43452026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."43462026/11/12 00:46:16 INFO Received push request method=POST path=/43472026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=043482026/11/12 00:46:16 INFO Received push request method=POST path=/43492026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period43502026/11/12 00:46:16 INFO Received complete upload request method=POST path=/43512026/11/12 00:46:16 INFO Completed upload id=31 objects=5 duration=719.687µs43522026/11/12 00:46:16 INFO Received complete upload request method=POST path=/43532026/11/12 00:46:16 INFO Completed upload id=22 objects=2 duration=1.248612ms43542026/11/12 00:46:16 INFO Received complete upload request method=POST path=/43552026/11/12 00:46:16 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."43562026/11/12 00:46:16 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."43572026/11/12 00:46:16 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."43582026/11/12 00:46:16 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."43592026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."43602026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."43612026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."43622026/11/12 00:46:16 INFO Received uploads request method=POST path=/43632026/11/12 00:46:16 INFO Completed upload id=25 objects=2 duration=1.01273ms43642026/11/12 00:46:16 INFO Received uploads request method=POST path=/43652026/11/12 00:46:16 INFO Received push request method=POST path=/43662026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=043672026/11/12 00:46:16 INFO Received complete push request method=POST path=/43682026/11/12 00:46:16 INFO Completed push id=32 objects=3 duration=859.049µs43692026/11/12 00:46:16 INFO Received complete upload request method=POST path=/43702026/11/12 00:46:16 INFO Received complete upload request method=POST path=/43712026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=043722026/11/12 00:46:16 INFO Completed upload id=26 objects=20 duration=975.829µs43732026/11/12 00:46:16 INFO Vacuumed table table=pending_closures43742026/11/12 00:46:16 INFO Completed upload id=14 objects=8 duration=2.204742ms43752026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts43762026/11/12 00:46:16 INFO Received complete push request method=POST path=/43772026/11/12 00:46:16 INFO Vacuumed table table=pending_objects43782026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads43792026/11/12 00:46:16 INFO Completed push id=15 objects=4 duration=1.207352ms43802026/11/12 00:46:16 INFO Vacuumed table table=closures43812026/11/12 00:46:16 INFO Vacuumed table table=objects43822026/11/12 00:46:16 INFO Received complete push request method=POST path=/43832026/11/12 00:46:16 INFO Received uploads request method=POST path=/43842026/11/12 00:46:16 INFO Completed push id=31 objects=14 duration=2.549966ms43852026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=043862026/11/12 00:46:16 INFO Received complete upload request method=POST path=/43872026/11/12 00:46:16 INFO Completed upload id=21 objects=3 duration=911.349µs43882026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period43892026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043902026/11/12 00:46:16 INFO Vacuumed table table=pending_closures43912026/11/12 00:46:16 INFO Received uploads request method=POST path=/43922026/11/12 00:46:16 INFO Received push request method=POST path=/43932026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts43942026/11/12 00:46:16 INFO Vacuumed table table=pending_objects43952026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads43962026/11/12 00:46:16 INFO Vacuumed table table=closures43972026/11/12 00:46:16 INFO Received complete push request method=POST path=/43982026/11/12 00:46:16 INFO Vacuumed table table=objects43992026/11/12 00:46:16 INFO Received complete upload request method=POST path=/44002026/11/12 00:46:16 INFO Received push request method=POST path=/44012026/11/12 00:46:16 INFO Completed push id=28 objects=7 duration=1.764818ms44022026/11/12 00:46:16 INFO Completed upload id=13 objects=14 duration=2.563726ms44032026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=044042026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period44052026/11/12 00:46:16 INFO Received uploads request method=POST path=/44062026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=7 objects-failed-to-delete=044072026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=044082026/11/12 00:46:16 INFO Received uploads request method=POST path=/44092026/11/12 00:46:16 INFO Vacuumed table table=pending_closures44102026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts44112026/11/12 00:46:16 INFO Received uploads request method=POST path=/44122026/11/12 00:46:16 INFO Vacuumed table table=pending_objects44132026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads44142026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044152026/11/12 00:46:16 INFO Vacuumed table table=closures44162026/11/12 00:46:16 INFO Vacuumed table table=pending_closures44172026/11/12 00:46:16 INFO Vacuumed table table=objects44182026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts44192026/11/12 00:46:16 INFO Received complete upload request method=POST path=/44202026/11/12 00:46:16 INFO Received push request method=POST path=/44212026/11/12 00:46:16 INFO Completed upload id=30 objects=6 duration=1.02737ms44222026/11/12 00:46:16 INFO Vacuumed table table=pending_objects44232026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads44242026/11/12 00:46:16 INFO Vacuumed table table=closures44252026/11/12 00:46:16 INFO Vacuumed table table=objects44262026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044272026/11/12 00:46:16 INFO Vacuumed table table=pending_closures44282026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts44292026/11/12 00:46:16 INFO Vacuumed table table=pending_objects44302026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads44312026/11/12 00:46:16 INFO Vacuumed table table=closures44322026/11/12 00:46:16 INFO Vacuumed table table=objects44332026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=044342026/11/12 00:46:16 INFO Received complete push request method=POST path=/44352026/11/12 00:46:16 INFO Received complete push request method=POST path=/44362026/11/12 00:46:16 INFO Completed push id=17 objects=22 duration=1.601717ms44372026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period44382026/11/12 00:46:16 INFO Completed push id=16 objects=26 duration=1.660437ms44392026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044402026/11/12 00:46:16 INFO Vacuumed table table=pending_closures44412026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts44422026/11/12 00:46:16 INFO Received push request method=POST path=/44432026/11/12 00:46:16 INFO Received complete push request method=POST path=/44442026/11/12 00:46:16 INFO Vacuumed table table=pending_objects44452026/11/12 00:46:16 INFO Received complete push request method=POST path=/44462026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads44472026/11/12 00:46:16 INFO Completed push id=31 objects=5 duration=952.429µs44482026/11/12 00:46:16 INFO Completed push id=14 objects=3 duration=1.05161ms44492026/11/12 00:46:16 INFO Vacuumed table table=closures44502026/11/12 00:46:16 INFO Vacuumed table table=objects44512026/11/12 00:46:16 INFO Received complete upload request method=POST path=/44522026/11/12 00:46:16 INFO Received complete push request method=POST path=/44532026/11/12 00:46:16 INFO Completed upload id=20 objects=3 duration=2.215522ms44542026/11/12 00:46:16 INFO Completed push id=31 objects=6 duration=3.649837ms44552026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=044562026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=044572026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period44582026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period44592026/11/12 00:46:16 INFO Received push request method=POST path=/44602026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=044612026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044622026/11/12 00:46:16 INFO Vacuumed table table=pending_closures44632026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts44642026/11/12 00:46:16 INFO Vacuumed table table=pending_objects44652026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads44662026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period44672026/11/12 00:46:16 INFO Vacuumed table table=closures44682026/11/12 00:46:16 INFO Vacuumed table table=objects44692026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044702026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=044712026/11/12 00:46:16 INFO Vacuumed table table=pending_closures44722026/11/12 00:46:16 INFO Vacuumed table table=pending_closures44732026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts44742026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts44752026/11/12 00:46:16 INFO Received complete push request method=POST path=/44762026/11/12 00:46:16 INFO Vacuumed table table=pending_objects44772026/11/12 00:46:16 INFO Vacuumed table table=pending_objects44782026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads44792026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads44802026/11/12 00:46:16 INFO Completed push id=7 objects=7 duration=1.433414ms44812026/11/12 00:46:16 INFO Vacuumed table table=closures44822026/11/12 00:46:16 INFO Vacuumed table table=closures44832026/11/12 00:46:16 INFO Vacuumed table table=objects44842026/11/12 00:46:16 INFO Vacuumed table table=objects44852026/11/12 00:46:16 INFO Received push request method=POST path=/44862026/11/12 00:46:16 INFO Received complete push request method=POST path=/44872026/11/12 00:46:16 INFO Received uploads request method=POST path=/44882026/11/12 00:46:16 INFO Completed push id=32 objects=26 duration=1.307934ms44892026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=044902026/11/12 00:46:16 INFO Received push request method=POST path=/44912026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044922026/11/12 00:46:16 INFO Received uploads request method=POST path=/44932026/11/12 00:46:16 INFO Vacuumed table table=pending_closures44942026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts44952026/11/12 00:46:16 INFO Vacuumed table table=pending_objects44962026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads44972026/11/12 00:46:16 INFO Vacuumed table table=closures44982026/11/12 00:46:16 INFO Received complete upload request method=POST path=/44992026/11/12 00:46:16 INFO Received complete push request method=POST path=/45002026/11/12 00:46:16 INFO Vacuumed table table=objects45012026/11/12 00:46:16 INFO Received complete push request method=POST path=/45022026/11/12 00:46:16 INFO Completed upload id=20 objects=4 duration=986.349µs45032026/11/12 00:46:16 INFO Completed push id=12 objects=6 duration=1.260613ms45042026/11/12 00:46:16 INFO Received complete upload request method=POST path=/45052026/11/12 00:46:16 INFO Completed push id=27 objects=6 duration=3.102191ms45062026/11/12 00:46:16 INFO Completed upload id=33 objects=2 duration=1.05411ms45072026/11/12 00:46:16 INFO Received complete push request method=POST path=/45082026/11/12 00:46:16 INFO Completed push id=22 objects=3 duration=2.904629ms45092026/11/12 00:46:16 INFO Received push request method=POST path=/45102026/11/12 00:46:16 INFO Received uploads request method=POST path=/45112026/11/12 00:46:16 INFO Received complete upload request method=POST path=/45122026/11/12 00:46:16 INFO Completed upload id=7 objects=10 duration=1.421634ms45132026/11/12 00:46:16 INFO Received complete upload request method=POST path=/45142026/11/12 00:46:16 INFO Received complete push request method=POST path=/45152026/11/12 00:46:16 INFO Received uploads request method=POST path=/45162026/11/12 00:46:16 INFO Received complete upload request method=POST path=/45172026/11/12 00:46:16 INFO Completed upload id=24 objects=7 duration=2.590266ms45182026/11/12 00:46:16 INFO Completed upload id=34 objects=5 duration=914.829µs45192026/11/12 00:46:16 INFO Received complete push request method=POST path=/45202026/11/12 00:46:16 INFO Completed push id=14 objects=3 duration=3.373593ms45212026/11/12 00:46:16 INFO Completed push id=16 objects=5 duration=966.909µs45222026/11/12 00:46:16 INFO Received push request method=POST path=/45232026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045242026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045252026/11/12 00:46:16 INFO Vacuumed table table=pending_closures45262026/11/12 00:46:16 INFO Vacuumed table table=pending_closures45272026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts45282026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts45292026/11/12 00:46:16 INFO Vacuumed table table=pending_objects45302026/11/12 00:46:16 INFO Vacuumed table table=pending_objects45312026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads45322026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads45332026/11/12 00:46:16 INFO Vacuumed table table=closures45342026/11/12 00:46:16 INFO Vacuumed table table=closures45352026/11/12 00:46:16 INFO Vacuumed table table=objects45362026/11/12 00:46:16 INFO Vacuumed table table=objects45372026/11/12 00:46:16 INFO Received complete upload request method=POST path=/45382026/11/12 00:46:16 INFO Completed upload id=15 objects=2 duration=981.91µs45392026/11/12 00:46:16 INFO Received uploads request method=POST path=/45402026/11/12 00:46:16 INFO Received complete upload request method=POST path=/45412026/11/12 00:46:16 INFO Received complete upload request method=POST path=/45422026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=045432026/11/12 00:46:16 INFO Completed upload id=35 objects=11 duration=1.100951ms45442026/11/12 00:46:16 INFO Completed upload id=16 objects=17 duration=3.254432ms45452026/11/12 00:46:16 INFO Received complete push request method=POST path=/45462026/11/12 00:46:16 INFO Received complete push request method=POST path=/45472026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045482026/11/12 00:46:16 INFO Completed push id=13 objects=22 duration=1.426735ms45492026/11/12 00:46:16 INFO Vacuumed table table=pending_closures45502026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts45512026/11/12 00:46:16 INFO Completed push id=21 objects=9 duration=1.92764ms45522026/11/12 00:46:16 INFO Vacuumed table table=pending_objects45532026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads45542026/11/12 00:46:16 INFO Received complete push request method=POST path=/45552026/11/12 00:46:16 INFO Vacuumed table table=closures45562026/11/12 00:46:16 INFO Vacuumed table table=objects45572026/11/12 00:46:16 INFO Completed push id=1 objects=9 duration=1.123371ms45582026/11/12 00:46:16 INFO Received push request method=POST path=/45592026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=045602026/11/12 00:46:16 INFO Received uploads request method=POST path=/45612026/11/12 00:46:16 INFO Received complete upload request method=POST path=/45622026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=045632026/11/12 00:46:16 INFO Completed upload id=36 objects=3 duration=819.608µs45642026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period45652026/11/12 00:46:16 INFO Received complete push request method=POST path=/45662026/11/12 00:46:16 INFO Completed push id=21 objects=6 duration=1.078931ms45672026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=045682026/11/12 00:46:16 INFO Vacuumed table table=pending_closures45692026/11/12 00:46:16 INFO Received uploads request method=POST path=/45702026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts45712026/11/12 00:46:16 INFO Received push request method=POST path=/45722026/11/12 00:46:16 INFO Received uploads request method=POST path=/45732026/11/12 00:46:16 INFO Vacuumed table table=pending_objects45742026/11/12 00:46:16 INFO Received complete push request method=POST path=/45752026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads45762026/11/12 00:46:16 INFO Vacuumed table table=closures45772026/11/12 00:46:16 INFO Completed push id=24 objects=3 duration=1.123792ms45782026/11/12 00:46:16 INFO Vacuumed table table=objects45792026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=045802026/11/12 00:46:16 INFO Received complete upload request method=POST path=/45812026/11/12 00:46:16 INFO Received uploads request method=POST path=/45822026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period45832026/11/12 00:46:16 INFO Completed upload id=22 objects=7 duration=1.01317ms45842026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045852026/11/12 00:46:16 INFO Vacuumed table table=pending_closures45862026/11/12 00:46:16 INFO Received push request method=POST path=/45872026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts45882026/11/12 00:46:16 INFO Received uploads request method=POST path=/45892026/11/12 00:46:16 INFO Received push request method=POST path=/45902026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=045912026/11/12 00:46:16 INFO Vacuumed table table=pending_objects45922026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads45932026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=045942026/11/12 00:46:16 INFO Vacuumed table table=closures45952026/11/12 00:46:16 INFO Vacuumed table table=objects45962026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period45972026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045982026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045992026/11/12 00:46:16 INFO Vacuumed table table=pending_closures46002026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts46012026/11/12 00:46:16 INFO Vacuumed table table=pending_closures46022026/11/12 00:46:16 INFO Received complete push request method=POST path=/46032026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts46042026/11/12 00:46:16 INFO Vacuumed table table=pending_objects46052026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads46062026/11/12 00:46:16 INFO Completed push id=16 objects=14 duration=2.94071ms46072026/11/12 00:46:16 INFO Vacuumed table table=pending_objects46082026/11/12 00:46:16 INFO Vacuumed table table=closures46092026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads46102026/11/12 00:46:16 INFO Received complete push request method=POST path=/46112026/11/12 00:46:16 INFO Vacuumed table table=closures46122026/11/12 00:46:16 INFO Vacuumed table table=objects46132026/11/12 00:46:16 INFO Vacuumed table table=objects46142026/11/12 00:46:16 INFO Completed push id=24 objects=4 duration=887.289µs46152026/11/12 00:46:16 INFO Received complete upload request method=POST path=/46162026/11/12 00:46:16 INFO Received push request method=POST path=/46172026/11/12 00:46:16 INFO Completed upload id=11 objects=7 duration=1.081991ms46182026/11/12 00:46:16 INFO Received push request method=POST path=/46192026/11/12 00:46:16 INFO Received push request method=POST path=/46202026/11/12 00:46:16 INFO Received complete upload request method=POST path=/46212026/11/12 00:46:16 INFO Completed upload id=2 objects=7 duration=1.236173ms46222026/11/12 00:46:16 INFO Received uploads request method=POST path=/46232026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=046242026/11/12 00:46:16 INFO Vacuumed table table=pending_closures46252026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=046262026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts46272026/11/12 00:46:16 INFO Vacuumed table table=pending_objects46282026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads46292026/11/12 00:46:16 INFO Received complete push request method=POST path=/46302026/11/12 00:46:16 INFO Vacuumed table table=closures46312026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period46322026/11/12 00:46:16 INFO Completed push id=25 objects=9 duration=829.028µs46332026/11/12 00:46:16 INFO Vacuumed table table=objects46342026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046352026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=046362026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=046372026/11/12 00:46:16 INFO Vacuumed table table=pending_closures46382026/11/12 00:46:16 INFO Received uploads request method=POST path=/46392026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period46402026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts46412026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period46422026/11/12 00:46:16 INFO Vacuumed table table=pending_objects46432026/11/12 00:46:16 INFO Received complete push request method=POST path=/46442026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads46452026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046462026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046472026/11/12 00:46:16 INFO Vacuumed table table=closures46482026/11/12 00:46:16 INFO Vacuumed table table=pending_closures46492026/11/12 00:46:16 INFO Received uploads request method=POST path=/46502026/11/12 00:46:16 INFO Vacuumed table table=objects46512026/11/12 00:46:16 INFO Vacuumed table table=pending_closures46522026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts46532026/11/12 00:46:16 INFO Completed push id=23 objects=28 duration=2.681106ms46542026/11/12 00:46:16 INFO Received uploads request method=POST path=/46552026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts46562026/11/12 00:46:16 INFO Vacuumed table table=pending_objects46572026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads46582026/11/12 00:46:16 INFO Vacuumed table table=pending_objects46592026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads46602026/11/12 00:46:16 INFO Vacuumed table table=closures46612026/11/12 00:46:16 INFO Vacuumed table table=closures46622026/11/12 00:46:16 INFO Received push request method=POST path=/46632026/11/12 00:46:16 INFO Vacuumed table table=objects46642026/11/12 00:46:16 INFO Vacuumed table table=objects46652026/11/12 00:46:16 INFO Received complete upload request method=POST path=/46662026/11/12 00:46:16 INFO Completed upload id=26 objects=3 duration=778.068µs46672026/11/12 00:46:16 INFO Received uploads request method=POST path=/46682026/11/12 00:46:16 INFO Received push request method=POST path=/46692026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=046702026/11/12 00:46:16 INFO Received uploads request method=POST path=/46712026/11/12 00:46:16 INFO Received push request method=POST path=/46722026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=046732026/11/12 00:46:16 INFO Received complete upload request method=POST path=/46742026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period46752026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046762026/11/12 00:46:16 INFO Completed upload id=8 objects=6 duration=1.117092ms46772026/11/12 00:46:16 INFO Vacuumed table table=pending_closures46782026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046792026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts46802026/11/12 00:46:16 INFO Vacuumed table table=pending_closures46812026/11/12 00:46:16 INFO Vacuumed table table=pending_objects46822026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts46832026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads46842026/11/12 00:46:16 INFO Vacuumed table table=pending_objects46852026/11/12 00:46:16 INFO Vacuumed table table=closures46862026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads46872026/11/12 00:46:16 INFO Vacuumed table table=closures46882026/11/12 00:46:16 INFO Vacuumed table table=objects46892026/11/12 00:46:16 INFO Received complete upload request method=POST path=/46902026/11/12 00:46:16 INFO Vacuumed table table=objects46912026/11/12 00:46:16 INFO Received uploads request method=POST path=/46922026/11/12 00:46:16 INFO Completed upload id=27 objects=7 duration=1.121652ms46932026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=046942026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046952026/11/12 00:46:16 INFO Vacuumed table table=pending_closures46962026/11/12 00:46:16 INFO Received complete upload request method=POST path=/46972026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts46982026/11/12 00:46:16 INFO Received complete upload request method=POST path=/46992026/11/12 00:46:16 INFO Vacuumed table table=pending_objects47002026/11/12 00:46:16 INFO Completed upload id=26 objects=8 duration=1.216772ms47012026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads47022026/11/12 00:46:16 INFO Completed upload id=27 objects=8 duration=1.538655ms47032026/11/12 00:46:16 INFO Vacuumed table table=closures47042026/11/12 00:46:16 INFO Vacuumed table table=objects47052026/11/12 00:46:16 INFO Received complete upload request method=POST path=/47062026/11/12 00:46:16 INFO Received push request method=POST path=/47072026/11/12 00:46:16 INFO Completed upload id=15 objects=3 duration=1.800398ms47082026/11/12 00:46:16 INFO Received complete upload request method=POST path=/47092026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=047102026/11/12 00:46:16 INFO Received push request method=POST path=/47112026/11/12 00:46:16 INFO Completed upload id=28 objects=3 duration=985.51µs47122026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=047132026/11/12 00:46:16 INFO Received complete push request method=POST path=/47142026/11/12 00:46:16 INFO Completed push id=28 objects=14 duration=1.549536ms47152026/11/12 00:46:16 INFO Received push request method=POST path=/47162026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=047172026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047182026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=047192026/11/12 00:46:16 INFO Vacuumed table table=pending_closures47202026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts47212026/11/12 00:46:16 INFO Vacuumed table table=pending_closures47222026/11/12 00:46:16 INFO Received uploads request method=POST path=/47232026/11/12 00:46:16 INFO Vacuumed table table=pending_objects47242026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts47252026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads47262026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047272026/11/12 00:46:16 INFO Vacuumed table table=closures47282026/11/12 00:46:16 INFO Vacuumed table table=pending_objects47292026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads47302026/11/12 00:46:16 INFO Vacuumed table table=pending_closures47312026/11/12 00:46:16 INFO Vacuumed table table=objects47322026/11/12 00:46:16 INFO Vacuumed table table=closures47332026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts47342026/11/12 00:46:16 INFO Vacuumed table table=objects47352026/11/12 00:46:16 INFO Vacuumed table table=pending_objects47362026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads47372026/11/12 00:46:16 INFO Vacuumed table table=closures47382026/11/12 00:46:16 INFO Vacuumed table table=objects47392026/11/12 00:46:16 INFO Received complete push request method=POST path=/47402026/11/12 00:46:16 INFO Received complete push request method=POST path=/47412026/11/12 00:46:16 INFO Completed push id=29 objects=23 duration=1.150272ms47422026/11/12 00:46:16 INFO Received complete upload request method=POST path=/47432026/11/12 00:46:16 INFO Received push request method=POST path=/47442026/11/12 00:46:16 INFO Received push request method=POST path=/47452026/11/12 00:46:16 INFO Completed push id=27 objects=25 duration=2.834348ms47462026/11/12 00:46:16 INFO Completed upload id=12 objects=9 duration=1.770398ms47472026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=047482026/11/12 00:46:16 INFO Received complete upload request method=POST path=/47492026/11/12 00:46:16 INFO Completed upload id=25 objects=6 duration=1.557316ms47502026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047512026/11/12 00:46:16 INFO Vacuumed table table=pending_closures47522026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts47532026/11/12 00:46:16 INFO Vacuumed table table=pending_objects47542026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads47552026/11/12 00:46:16 INFO Vacuumed table table=closures47562026/11/12 00:46:16 INFO Received complete push request method=POST path=/47572026/11/12 00:46:16 INFO Vacuumed table table=objects47582026/11/12 00:46:16 INFO Completed push id=30 objects=6 duration=1.904959ms47592026/11/12 00:46:16 INFO Received push request method=POST path=/47602026/11/12 00:46:16 INFO Received uploads request method=POST path=/47612026/11/12 00:46:16 INFO Received uploads request method=POST path=/47622026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=047632026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047642026/11/12 00:46:16 INFO Vacuumed table table=pending_closures47652026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts47662026/11/12 00:46:16 INFO Received complete upload request method=POST path=/47672026/11/12 00:46:16 INFO Vacuumed table table=pending_objects47682026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads47692026/11/12 00:46:16 INFO Vacuumed table table=closures47702026/11/12 00:46:16 INFO Vacuumed table table=objects47712026/11/12 00:46:16 INFO Completed upload id=33 objects=5 duration=2.383743ms47722026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=047732026/11/12 00:46:16 INFO Received push request method=POST path=/47742026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047752026/11/12 00:46:16 INFO Received push request method=POST path=/47762026/11/12 00:46:16 INFO Vacuumed table table=pending_closures47772026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts47782026/11/12 00:46:16 INFO Received complete push request method=POST path=/47792026/11/12 00:46:16 INFO Received complete upload request method=POST path=/47802026/11/12 00:46:16 INFO Vacuumed table table=pending_objects47812026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads47822026/11/12 00:46:16 INFO Completed push id=33 objects=3 duration=1.279053ms47832026/11/12 00:46:16 INFO Vacuumed table table=closures47842026/11/12 00:46:16 INFO Completed upload id=26 objects=2 duration=1.516415ms47852026/11/12 00:46:16 INFO Received complete push request method=POST path=/47862026/11/12 00:46:16 INFO Vacuumed table table=objects47872026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=047882026/11/12 00:46:16 INFO Completed push id=13 objects=15 duration=1.680217ms47892026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period47902026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047912026/11/12 00:46:16 INFO Vacuumed table table=pending_closures47922026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts47932026/11/12 00:46:16 INFO Received complete upload request method=POST path=/47942026/11/12 00:46:16 INFO Received complete push request method=POST path=/47952026/11/12 00:46:16 INFO Received complete push request method=POST path=/47962026/11/12 00:46:16 INFO Vacuumed table table=pending_objects47972026/11/12 00:46:16 INFO Received uploads request method=POST path=/47982026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads47992026/11/12 00:46:16 INFO Completed upload id=13 objects=8 duration=1.371594ms48002026/11/12 00:46:16 INFO Vacuumed table table=closures48012026/11/12 00:46:16 INFO Completed push id=17 objects=22 duration=2.240483ms48022026/11/12 00:46:16 INFO Vacuumed table table=objects48032026/11/12 00:46:16 INFO Completed push id=24 objects=5 duration=3.452474ms48042026/11/12 00:46:16 INFO Received uploads request method=POST path=/48052026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=048062026/11/12 00:46:16 INFO Received complete push request method=POST path=/48072026/11/12 00:46:16 INFO Completed push id=34 objects=6 duration=1.00109ms48082026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=048092026/11/12 00:46:16 INFO Received complete upload request method=POST path=/48102026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048112026/11/12 00:46:16 INFO Completed upload id=25 objects=16 duration=3.287353ms48122026/11/12 00:46:16 INFO Vacuumed table table=pending_closures48132026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts48142026/11/12 00:46:16 INFO Vacuumed table table=pending_objects48152026/11/12 00:46:16 INFO Received push request method=POST path=/48162026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads48172026/11/12 00:46:16 INFO Received uploads request method=POST path=/48182026/11/12 00:46:16 INFO Received uploads request method=POST path=/48192026/11/12 00:46:16 INFO Vacuumed table table=closures48202026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."48212026/11/12 00:46:16 INFO Vacuumed table table=objects48222026/11/12 00:46:16 INFO Received uploads request method=POST path=/48232026/11/12 00:46:16 INFO Received complete push request method=POST path=/48242026/11/12 00:46:16 INFO Completed push id=11 objects=9 duration=1.323733ms48252026/11/12 00:46:16 INFO Received complete push request method=POST path=/48262026/11/12 00:46:16 INFO Completed push id=35 objects=8 duration=1.131831ms48272026/11/12 00:46:16 INFO Received complete upload request method=POST path=/48282026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=048292026/11/12 00:46:16 INFO Completed upload id=21 objects=25 duration=1.676796ms48302026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period48312026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048322026/11/12 00:46:16 INFO Vacuumed table table=pending_closures48332026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts48342026/11/12 00:46:16 INFO Received uploads request method=POST path=/48352026/11/12 00:46:16 INFO Vacuumed table table=pending_objects48362026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads48372026/11/12 00:46:16 INFO Vacuumed table table=closures48382026/11/12 00:46:16 INFO Received complete upload request method=POST path=/48392026/11/12 00:46:16 INFO Received uploads request method=POST path=/48402026/11/12 00:46:16 INFO Vacuumed table table=objects48412026/11/12 00:46:16 INFO Received complete upload request method=POST path=/48422026/11/12 00:46:16 INFO Completed upload id=21 objects=9 duration=1.897679ms48432026/11/12 00:46:16 INFO Completed upload id=18 objects=22 duration=1.299453ms48442026/11/12 00:46:16 INFO Received complete upload request method=POST path=/48452026/11/12 00:46:16 INFO Completed upload id=36 objects=9 duration=902.229µs48462026/11/12 00:46:16 INFO Received complete push request method=POST path=/48472026/11/12 00:46:16 INFO Completed push id=15 objects=24 duration=1.145591ms48482026/11/12 00:46:16 INFO Received uploads request method=POST path=/48492026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=048502026/11/12 00:46:16 INFO Received push request method=POST path=/48512026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048522026/11/12 00:46:16 INFO Received complete push request method=POST path=/48532026/11/12 00:46:16 INFO Vacuumed table table=pending_closures48542026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts48552026/11/12 00:46:16 INFO Received push request method=POST path=/48562026/11/12 00:46:16 INFO Completed push id=20 objects=3 duration=1.752438ms48572026/11/12 00:46:16 INFO Received push request method=POST path=/48582026/11/12 00:46:16 INFO Vacuumed table table=pending_objects48592026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads48602026/11/12 00:46:16 INFO Received complete push request method=POST path=/48612026/11/12 00:46:16 INFO Vacuumed table table=closures48622026/11/12 00:46:16 INFO Vacuumed table table=objects48632026/11/12 00:46:16 INFO Received complete upload request method=POST path=/48642026/11/12 00:46:16 INFO Completed push id=25 objects=10 duration=1.873419ms48652026/11/12 00:46:16 INFO Completed upload id=21 objects=8 duration=1.02735ms48662026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=048672026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period48682026/11/12 00:46:16 INFO Received complete upload request method=POST path=/48692026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048702026/11/12 00:46:16 INFO Completed upload id=19 objects=2 duration=1.020531ms48712026/11/12 00:46:16 INFO Vacuumed table table=pending_closures48722026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts48732026/11/12 00:46:16 INFO Vacuumed table table=pending_objects48742026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads48752026/11/12 00:46:16 INFO Vacuumed table table=closures48762026/11/12 00:46:16 INFO Vacuumed table table=objects48772026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=048782026/11/12 00:46:16 INFO Received push request method=POST path=/48792026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period48802026/11/12 00:46:16 INFO Received complete upload request method=POST path=/48812026/11/12 00:46:16 INFO Completed upload id=29 objects=5 duration=997.63µs48822026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048832026/11/12 00:46:16 INFO Received complete push request method=POST path=/48842026/11/12 00:46:16 INFO Vacuumed table table=pending_closures48852026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts48862026/11/12 00:46:16 INFO Completed push id=21 objects=26 duration=1.575416ms48872026/11/12 00:46:16 INFO Vacuumed table table=pending_objects48882026/11/12 00:46:16 INFO Received complete upload request method=POST path=/48892026/11/12 00:46:16 INFO Received push request method=POST path=/48902026/11/12 00:46:16 INFO Received complete upload request method=POST path=/48912026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads48922026/11/12 00:46:16 INFO Vacuumed table table=closures48932026/11/12 00:46:16 INFO Completed upload id=24 objects=7 duration=1.327593ms48942026/11/12 00:46:16 INFO Completed upload id=22 objects=7 duration=1.07583ms48952026/11/12 00:46:16 INFO Vacuumed table table=objects48962026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=048972026/11/12 00:46:16 INFO Received complete upload request method=POST path=/48982026/11/12 00:46:16 INFO Completed upload id=12 objects=6 duration=1.133852ms48992026/11/12 00:46:16 INFO Received complete upload request method=POST path=/49002026/11/12 00:46:16 INFO Completed upload id=30 objects=2 duration=955.289µs49012026/11/12 00:46:16 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."49022026/11/12 00:46:16 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."49032026/11/12 00:46:16 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."49042026/11/12 00:46:16 INFO Received complete upload request method=POST path=/49052026/11/12 00:46:16 INFO Received complete push request method=POST path=/49062026/11/12 00:46:16 INFO Received complete push request method=POST path=/49072026/11/12 00:46:16 INFO Completed upload id=19 objects=2 duration=2.0346ms49082026/11/12 00:46:16 INFO Received complete push request method=POST path=/49092026/11/12 00:46:16 INFO Completed push id=26 objects=3 duration=823.488µs49102026/11/12 00:46:16 INFO Completed push id=19 objects=14 duration=1.639696ms49112026/11/12 00:46:16 INFO Completed push id=24 objects=5 duration=1.00367ms49122026/11/12 00:46:16 INFO Received complete push request method=POST path=/49132026/11/12 00:46:16 INFO Received complete upload request method=POST path=/49142026/11/12 00:46:16 INFO Completed push id=5 objects=28 duration=1.623536ms49152026/11/12 00:46:16 INFO Completed upload id=28 objects=17 duration=889.029µs49162026/11/12 00:46:16 INFO Received complete push request method=POST path=/49172026/11/12 00:46:16 INFO Completed push id=17 objects=3 duration=1.909579ms49182026/11/12 00:46:16 INFO Received push request method=POST path=/49192026/11/12 00:46:16 INFO Received uploads request method=POST path=/49202026/11/12 00:46:16 INFO Received complete push request method=POST path=/49212026/11/12 00:46:16 INFO Received complete upload request method=POST path=/49222026/11/12 00:46:16 INFO Received uploads request method=POST path=/49232026/11/12 00:46:16 INFO Completed push id=31 objects=20 duration=1.497255ms49242026/11/12 00:46:16 INFO Completed upload id=28 objects=8 duration=875.509µs49252026/11/12 00:46:16 INFO Received push request method=POST path=/49262026/11/12 00:46:16 INFO Received complete push request method=POST path=/49272026/11/12 00:46:16 INFO Completed push id=10 objects=2 duration=1.588796ms49282026/11/12 00:46:16 INFO Received complete push request method=POST path=/49292026/11/12 00:46:16 INFO Completed push id=29 objects=2 duration=1.181812ms49302026/11/12 00:46:16 INFO Received complete upload request method=POST path=/49312026/11/12 00:46:16 INFO Completed upload id=25 objects=8 duration=1.01259ms49322026/11/12 00:46:16 INFO Received complete push request method=POST path=/49332026/11/12 00:46:16 INFO Received uploads request method=POST path=/49342026/11/12 00:46:16 INFO Received complete push request method=POST path=/49352026/11/12 00:46:16 INFO Completed push id=33 objects=7 duration=1.781498ms49362026/11/12 00:46:16 INFO Completed push id=23 objects=5 duration=1.513315ms49372026/11/12 00:46:16 INFO Received complete upload request method=POST path=/49382026/11/12 00:46:16 INFO Received complete upload request method=POST path=/49392026/11/12 00:46:16 INFO Completed upload id=29 objects=10 duration=893.469µs49402026/11/12 00:46:16 INFO Completed upload id=32 objects=5 duration=1.086251ms49412026/11/12 00:46:16 INFO Received push request method=POST path=/49422026/11/12 00:46:16 INFO Received complete push request method=POST path=/49432026/11/12 00:46:16 INFO Received push request method=POST path=/49442026/11/12 00:46:16 INFO Received complete push request method=POST path=/49452026/11/12 00:46:16 INFO Completed push id=26 objects=4 duration=1.02303ms49462026/11/12 00:46:16 INFO Received complete upload request method=POST path=/49472026/11/12 00:46:16 INFO Received complete upload request method=POST path=/49482026/11/12 00:46:16 INFO Received complete upload request method=POST path=/49492026/11/12 00:46:16 INFO Completed push id=31 objects=6 duration=1.637037ms49502026/11/12 00:46:16 INFO Completed upload id=17 objects=5 duration=1.536056ms49512026/11/12 00:46:16 INFO Completed upload id=26 objects=3 duration=975.47µs49522026/11/12 00:46:16 INFO Completed upload id=20 objects=6 duration=1.219032ms49532026/11/12 00:46:16 INFO Received complete upload request method=POST path=/49542026/11/12 00:46:16 INFO Received complete upload request method=POST path=/49552026/11/12 00:46:16 INFO Completed upload id=24 objects=3 duration=906.849µs49562026/11/12 00:46:16 INFO Completed upload id=22 objects=5 duration=1.640156ms49572026/11/12 00:46:16 INFO Received complete upload request method=POST path=/49582026/11/12 00:46:16 INFO Completed upload id=30 objects=20 duration=950.35µs49592026/11/12 00:46:16 INFO Received push request method=POST path=/49602026/11/12 00:46:16 INFO Received complete upload request method=POST path=/49612026/11/12 00:46:16 INFO Completed upload id=33 objects=3 duration=790.988µs49622026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=049632026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period49642026/11/12 00:46:16 INFO Received complete push request method=POST path=/49652026/11/12 00:46:16 INFO Received complete push request method=POST path=/49662026/11/12 00:46:16 INFO Completed push id=30 objects=2 duration=969.13µs49672026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049682026/11/12 00:46:16 INFO Completed push id=28 objects=11 duration=2.00122ms49692026/11/12 00:46:16 INFO Received complete push request method=POST path=/49702026/11/12 00:46:16 INFO Vacuumed table table=pending_closures49712026/11/12 00:46:16 INFO Completed push id=25 objects=4 duration=1.130631ms49722026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts49732026/11/12 00:46:16 INFO Received complete push request method=POST path=/49742026/11/12 00:46:16 INFO Completed push id=32 objects=9 duration=1.120951ms49752026/11/12 00:46:16 INFO Received complete push request method=POST path=/49762026/11/12 00:46:16 INFO Vacuumed table table=pending_objects49772026/11/12 00:46:16 INFO Completed push id=28 objects=3 duration=1.515815ms49782026/11/12 00:46:16 INFO Received complete push request method=POST path=/49792026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads49802026/11/12 00:46:16 INFO Received complete push request method=POST path=/49812026/11/12 00:46:16 INFO Vacuumed table table=closures49822026/11/12 00:46:16 INFO Completed push id=31 objects=11 duration=1.250453ms49832026/11/12 00:46:16 INFO Vacuumed table table=objects49842026/11/12 00:46:16 INFO Received complete upload request method=POST path=/49852026/11/12 00:46:16 INFO Completed push id=19 objects=3 duration=1.134891ms49862026/11/12 00:46:16 INFO Completed upload id=22 objects=6 duration=927.349µs49872026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=049882026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period49892026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049902026/11/12 00:46:16 INFO Vacuumed table table=pending_closures49912026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts49922026/11/12 00:46:16 INFO Received uploads request method=POST path=/49932026/11/12 00:46:16 INFO Vacuumed table table=pending_objects49942026/11/12 00:46:16 INFO Vacuumed table table=multipart_uploads49952026/11/12 00:46:16 INFO Vacuumed table table=closures49962026/11/12 00:46:16 INFO Vacuumed table table=objects49972026/11/12 00:46:16 INFO Received complete push request method=POST path=/49982026/11/12 00:46:16 INFO Completed push id=29 objects=15 duration=1.112851ms49992026/11/12 00:46:16 INFO Received complete upload request method=POST path=/50002026/11/12 00:46:16 INFO Received uploads request method=POST path=/50012026/11/12 00:46:16 INFO Completed upload id=32 objects=2 duration=844.289µs50022026/11/12 00:46:16 INFO Received complete upload request method=POST path=/50032026/11/12 00:46:16 INFO Completed upload id=33 objects=9 duration=825.668µs50042026/11/12 00:46:16 INFO Received complete push request method=POST path=/50052026/11/12 00:46:16 INFO Completed push id=32 objects=7 duration=1.179292ms50062026/11/12 00:46:16 INFO Received complete push request method=POST path=/50072026/11/12 00:46:16 INFO Completed push id=20 objects=4 duration=1.343873ms50082026/11/12 00:46:16 INFO Received complete push request method=POST path=/50092026/11/12 00:46:16 INFO Completed push id=29 objects=24 duration=1.566135ms50102026/11/12 00:46:16 INFO Received complete push request method=POST path=/50112026/11/12 00:46:16 INFO Received complete push request method=POST path=/50122026/11/12 00:46:16 INFO Completed push id=33 objects=3 duration=1.077191ms50132026/11/12 00:46:16 INFO Completed push id=30 objects=6 duration=1.416474ms50142026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=050152026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period50162026/11/12 00:46:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050172026/11/12 00:46:16 INFO Received complete push request method=POST path=/50182026/11/12 00:46:16 INFO Received complete upload request method=POST path=/50192026/11/12 00:46:16 INFO Received complete upload request method=POST path=/50202026/11/12 00:46:16 INFO Completed upload id=33 objects=6 duration=1.296873ms50212026/11/12 00:46:16 INFO Completed push id=26 objects=23 duration=1.598116ms50222026/11/12 00:46:16 INFO Completed upload id=21 objects=2 duration=1.181392ms50232026/11/12 00:46:16 INFO Vacuumed table table=pending_closures50242026/11/12 00:46:16 INFO Vacuumed table table=commit_receipts50252026/11/12 00:46:16 INFO Aborted multipart uploads count=0 kept=050262026/11/12 00:46:16 INFO Vacuumed table table=pending_objects50272026/11/12 00:46:16 INFO Received uploads request method=POST path=/50282026/11/12 00:46:16 WARN Force mode enabled - objects will be deleted immediately without grace period50292026/11/12 00:46:17 INFO Vacuumed table table=multipart_uploads50302026/11/12 00:46:17 INFO Vacuumed table table=closures50312026/11/12 00:46:17 INFO Received complete upload request method=POST path=/50322026/11/12 00:46:17 INFO Vacuumed table table=objects50332026/11/12 00:46:17 INFO Received complete upload request method=POST path=/50342026/11/12 00:46:17 INFO Completed upload id=34 objects=3 duration=673.346µs50352026/11/12 00:46:17 INFO Completed upload id=31 objects=3 duration=827.428µs50362026/11/12 00:46:17 INFO Received complete push request method=POST path=/50372026/11/12 00:46:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=050382026/11/12 00:46:17 INFO Vacuumed table table=pending_closures50392026/11/12 00:46:17 INFO Vacuumed table table=commit_receipts50402026/11/12 00:46:17 INFO Completed push id=9 objects=9 duration=2.702967ms50412026/11/12 00:46:17 INFO Received complete upload request method=POST path=/50422026/11/12 00:46:17 INFO Vacuumed table table=pending_objects50432026/11/12 00:46:17 INFO Vacuumed table table=multipart_uploads50442026/11/12 00:46:17 INFO Completed upload id=22 objects=8 duration=1.03443ms50452026/11/12 00:46:17 INFO Vacuumed table table=closures50462026/11/12 00:46:17 INFO Received complete upload request method=POST path=/50472026/11/12 00:46:17 INFO Vacuumed table table=objects50482026/11/12 00:46:17 INFO Received uploads request method=POST path=/50492026/11/12 00:46:17 INFO Completed upload id=34 objects=3 duration=1.352193ms50502026/11/12 00:46:17 INFO Received push request method=POST path=/50512026/11/12 00:46:17 INFO Aborted multipart uploads count=0 kept=050522026/11/12 00:46:17 WARN Force mode enabled - objects will be deleted immediately without grace period50532026/11/12 00:46:17 INFO Aborted multipart uploads count=0 kept=050542026/11/12 00:46:17 INFO Received complete push request method=POST path=/50552026/11/12 00:46:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050562026/11/12 00:46:17 INFO Received complete upload request method=POST path=/50572026/11/12 00:46:17 WARN Force mode enabled - objects will be deleted immediately without grace period50582026/11/12 00:46:17 INFO Completed push id=23 objects=5 duration=1.238413ms50592026/11/12 00:46:17 INFO Vacuumed table table=pending_closures50602026/11/12 00:46:17 INFO Completed upload id=21 objects=8 duration=1.266953ms50612026/11/12 00:46:17 INFO Vacuumed table table=commit_receipts50622026/11/12 00:46:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050632026/11/12 00:46:17 INFO Received complete push request method=POST path=/50642026/11/12 00:46:17 INFO Aborted multipart uploads count=0 kept=050652026/11/12 00:46:17 INFO Aborted multipart uploads count=0 kept=050662026/11/12 00:46:17 INFO Completed push id=27 objects=15 duration=1.125811ms50672026/11/12 00:46:17 INFO Vacuumed table table=pending_closures50682026/11/12 00:46:17 INFO Vacuumed table table=pending_objects50692026/11/12 00:46:17 INFO Vacuumed table table=multipart_uploads50702026/11/12 00:46:17 INFO Vacuumed table table=commit_receipts50712026/11/12 00:46:17 WARN Force mode enabled - objects will be deleted immediately without grace period50722026/11/12 00:46:17 INFO Vacuumed table table=closures50732026/11/12 00:46:17 WARN Force mode enabled - objects will be deleted immediately without grace period50742026/11/12 00:46:17 INFO Vacuumed table table=objects50752026/11/12 00:46:17 INFO Vacuumed table table=pending_objects50762026/11/12 00:46:17 INFO Vacuumed table table=multipart_uploads50772026/11/12 00:46:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050782026/11/12 00:46:17 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050792026/11/12 00:46:17 INFO Vacuumed table table=closures50802026/11/12 00:46:17 INFO Vacuumed table table=pending_closures50812026/11/12 00:46:17 INFO Vacuumed table table=objects50822026/11/12 00:46:17 INFO Vacuumed table table=pending_closures50832026/11/12 00:46:17 INFO Vacuumed table table=commit_receipts50842026/11/12 00:46:17 INFO Vacuumed table table=commit_receipts50852026/11/12 00:46:17 INFO Vacuumed table table=pending_objects50862026/11/12 00:46:17 INFO Vacuumed table table=multipart_uploads50872026/11/12 00:46:17 INFO Vacuumed table table=pending_objects50882026/11/12 00:46:17 INFO Vacuumed table table=multipart_uploads50892026/11/12 00:46:17 INFO Received push request method=POST path=/50902026/11/12 00:46:17 INFO Vacuumed table table=closures50912026/11/12 00:46:17 INFO Vacuumed table table=closures50922026/11/12 00:46:17 INFO Vacuumed table table=objects50932026/11/12 00:46:17 INFO Received complete push request method=POST path=/50942026/11/12 00:46:17 INFO Vacuumed table table=objects50952026/11/12 00:46:17 INFO Completed push id=22 objects=6 duration=1.257212ms50962026/11/12 00:46:17 INFO Received complete upload request method=POST path=/50972026/11/12 00:46:17 INFO Completed upload id=24 objects=8 duration=1.03545ms50982026/11/12 00:46:17 INFO Received uploads request method=POST path=/50992026/11/12 00:46:17 INFO Received complete push request method=POST path=/51002026/11/12 00:46:17 INFO Received complete upload request method=POST path=/51012026/11/12 00:46:17 INFO Completed push id=24 objects=8 duration=1.328533ms51022026/11/12 00:46:17 INFO Completed upload id=28 objects=5 duration=989.53µs51032026/11/12 00:46:17 INFO Received uploads request method=POST path=/51042026/11/12 00:46:17 INFO Received complete upload request method=POST path=/51052026/11/12 00:46:17 INFO Received complete push request method=POST path=/51062026/11/12 00:46:17 INFO Received uploads request method=POST path=/51072026/11/12 00:46:17 INFO Completed upload id=25 objects=9 duration=982.27µs51082026/11/12 00:46:17 INFO Completed push id=28 objects=19 duration=1.218432ms51092026/11/12 00:46:17 INFO Received complete upload request method=POST path=/51102026/11/12 00:46:17 INFO Completed upload id=30 objects=5 duration=946.43µs51112026/11/12 00:46:17 INFO Received complete upload request method=POST path=/51122026/11/12 00:46:17 INFO Completed upload id=27 objects=3 duration=821.969µs51132026/11/12 00:46:17 INFO Received uploads request method=POST path=/51142026/11/12 00:46:17 INFO Received push request method=POST path=/51152026/11/12 00:46:17 INFO Received complete upload request method=POST path=/51162026/11/12 00:46:17 INFO Completed upload id=28 objects=6 duration=760.847µs51172026/11/12 00:46:17 INFO Received complete push request method=POST path=/51182026/11/12 00:46:17 INFO Completed push id=29 objects=10 duration=1.208652ms51192026/11/12 00:46:17 INFO Received uploads request method=POST path=/51202026/11/12 00:46:17 INFO Received complete push request method=POST path=/51212026/11/12 00:46:17 INFO Completed push id=31 objects=3 duration=1.091411ms51222026/11/12 00:46:17 INFO Received complete push request method=POST path=/51232026/11/12 00:46:17 INFO Completed push id=24 objects=24 duration=2.909829ms51242026/11/12 00:46:17 INFO Received complete push request method=POST path=/51252026/11/12 00:46:17 INFO Completed push id=30 objects=4 duration=915.729µs51262026/11/12 00:46:17 INFO Received uploads request method=POST path=/51272026/11/12 00:46:17 INFO Received complete upload request method=POST path=/51282026/11/12 00:46:17 INFO Aborted multipart uploads count=0 kept=051292026/11/12 00:46:17 INFO Completed upload id=30 objects=6 duration=1.063051ms51302026/11/12 00:46:17 INFO Received complete push request method=POST path=/51312026/11/12 00:46:17 INFO Completed push id=31 objects=7 duration=1.136211ms51322026/11/12 00:46:17 WARN Force mode enabled - objects will be deleted immediately without grace period51332026/11/12 00:46:17 INFO Received complete push request method=POST path=/51342026/11/12 00:46:17 INFO Completed push id=32 objects=6 duration=1.124611ms51352026/11/12 00:46:17 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051362026/11/12 00:46:17 INFO Vacuumed table table=pending_closures51372026/11/12 00:46:17 INFO Vacuumed table table=commit_receipts51382026/11/12 00:46:17 INFO Vacuumed table table=pending_objects51392026/11/12 00:46:17 INFO Vacuumed table table=multipart_uploads51402026/11/12 00:46:17 INFO Vacuumed table table=closures51412026/11/12 00:46:17 INFO Received push request method=POST path=/51422026/11/12 00:46:17 INFO Vacuumed table table=objects51432026/11/12 00:46:17 INFO Received complete upload request method=POST path=/51442026/11/12 00:46:17 INFO Completed upload id=32 objects=5 duration=902.229µs51452026/11/12 00:46:17 INFO Received complete upload request method=POST path=/51462026/11/12 00:46:17 INFO Completed upload id=31 objects=5 duration=815.289µs51472026/11/12 00:46:17 INFO Received complete upload request method=POST path=/51482026/11/12 00:46:17 INFO Completed upload id=33 objects=7 duration=757.708µs51492026/11/12 00:46:17 INFO Received uploads request method=POST path=/51502026/11/12 00:46:17 INFO Received complete upload request method=POST path=/51512026/11/12 00:46:17 INFO Completed upload id=34 objects=3 duration=540.505µs51522026/11/12 00:46:17 INFO Received complete upload request method=POST path=/51532026/11/12 00:46:17 INFO Completed upload id=34 objects=3 duration=747.247µs51542026/11/12 00:46:17 INFO Received uploads request method=POST path=/51552026/11/12 00:46:17 INFO Aborted multipart uploads count=0 kept=051562026/11/12 00:46:17 WARN Force mode enabled - objects will be deleted immediately without grace period51572026/11/12 00:46:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051582026/11/12 00:46:17 INFO Vacuumed table table=pending_closures51592026/11/12 00:46:17 INFO Vacuumed table table=commit_receipts51602026/11/12 00:46:17 INFO Received complete push request method=POST path=/51612026/11/12 00:46:17 INFO Vacuumed table table=pending_objects51622026/11/12 00:46:17 INFO Vacuumed table table=multipart_uploads51632026/11/12 00:46:17 INFO Completed push id=35 objects=11 duration=1.07879ms51642026/11/12 00:46:17 INFO Vacuumed table table=closures51652026/11/12 00:46:17 INFO Vacuumed table table=objects51662026/11/12 00:46:17 INFO Received complete upload request method=POST path=/51672026/11/12 00:46:17 INFO Received complete push request method=POST path=/51682026/11/12 00:46:17 INFO Completed upload id=34 objects=24 duration=1.00991ms51692026/11/12 00:46:17 INFO Completed push id=16 objects=7 duration=1.735417ms51702026/11/12 00:46:17 INFO Aborted multipart uploads count=0 kept=051712026/11/12 00:46:17 INFO Received push request method=POST path=/51722026/11/12 00:46:17 INFO Aborted multipart uploads count=0 kept=051732026/11/12 00:46:17 WARN Force mode enabled - objects will be deleted immediately without grace period51742026/11/12 00:46:17 WARN Force mode enabled - objects will be deleted immediately without grace period51752026/11/12 00:46:17 INFO Garbage collection completed failed-uploads-deleted=6 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051762026/11/12 00:46:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051772026/11/12 00:46:17 INFO Vacuumed table table=pending_closures51782026/11/12 00:46:17 INFO Vacuumed table table=commit_receipts51792026/11/12 00:46:17 INFO Vacuumed table table=pending_closures51802026/11/12 00:46:17 INFO Vacuumed table table=commit_receipts51812026/11/12 00:46:17 INFO Vacuumed table table=pending_objects51822026/11/12 00:46:17 INFO Vacuumed table table=multipart_uploads51832026/11/12 00:46:17 INFO Vacuumed table table=pending_objects51842026/11/12 00:46:17 INFO Vacuumed table table=closures51852026/11/12 00:46:17 INFO Vacuumed table table=multipart_uploads51862026/11/12 00:46:17 INFO Vacuumed table table=closures51872026/11/12 00:46:17 INFO Vacuumed table table=objects51882026/11/12 00:46:17 INFO Received complete push request method=POST path=/51892026/11/12 00:46:17 INFO Vacuumed table table=objects51902026/11/12 00:46:17 INFO Completed push id=23 objects=6 duration=1.203613ms51912026/11/12 00:46:17 INFO Received uploads request method=POST path=/51922026/11/12 00:46:17 INFO Received complete push request method=POST path=/51932026/11/12 00:46:17 INFO Completed push id=24 objects=6 duration=1.119851ms51942026/11/12 00:46:17 INFO Aborted multipart uploads count=0 kept=051952026/11/12 00:46:17 WARN Force mode enabled - objects will be deleted immediately without grace period51962026/11/12 00:46:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051972026/11/12 00:46:17 INFO Vacuumed table table=pending_closures51982026/11/12 00:46:17 INFO Vacuumed table table=commit_receipts51992026/11/12 00:46:17 INFO Vacuumed table table=pending_objects52002026/11/12 00:46:17 INFO Vacuumed table table=multipart_uploads52012026/11/12 00:46:17 INFO Vacuumed table table=closures52022026/11/12 00:46:17 INFO Vacuumed table table=objects52032026/11/12 00:46:17 INFO Received complete push request method=POST path=/52042026/11/12 00:46:17 INFO Completed push id=25 objects=7 duration=1.240413ms52052026/11/12 00:46:17 INFO Received push request method=POST path=/52062026/11/12 00:46:17 INFO Received complete upload request method=POST path=/52072026/11/12 00:46:17 INFO Received complete upload request method=POST path=/52082026/11/12 00:46:17 INFO Completed upload id=26 objects=24 duration=1.142232ms52092026/11/12 00:46:17 INFO Completed upload id=23 objects=7 duration=2.367804ms52102026/11/12 00:46:17 INFO Received complete upload request method=POST path=/52112026/11/12 00:46:17 INFO Completed upload id=26 objects=5 duration=1.434415ms52122026/11/12 00:46:17 INFO Received complete push request method=POST path=/52132026/11/12 00:46:17 INFO Completed push id=27 objects=3 duration=1.783838ms52142026/11/12 00:46:17 INFO Received complete upload request method=POST path=/52152026/11/12 00:46:17 INFO Completed upload id=27 objects=26 duration=1.186772ms52162026/11/12 00:46:17 INFO Received complete upload request method=POST path=/52172026/11/12 00:46:17 INFO Completed upload id=28 objects=5 duration=1.04485ms52182026/11/12 00:46:17 INFO Received complete push request method=POST path=/52192026/11/12 00:46:17 INFO Completed push id=28 objects=6 duration=939.609µs52202026/11/12 00:46:17 INFO Received complete push request method=POST path=/52212026/11/12 00:46:17 INFO Completed push id=29 objects=17 duration=1.195492ms52222026/11/12 00:46:17 INFO Received complete push request method=POST path=/52232026/11/12 00:46:17 INFO Completed push id=29 objects=6 duration=991.63µs52242026/11/12 00:46:17 INFO Aborted multipart uploads count=0 kept=052252026/11/12 00:46:17 WARN Force mode enabled - objects will be deleted immediately without grace period52262026/11/12 00:46:17 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052272026/11/12 00:46:17 INFO Vacuumed table table=pending_closures52282026/11/12 00:46:17 INFO Vacuumed table table=commit_receipts52292026/11/12 00:46:17 INFO Vacuumed table table=pending_objects52302026/11/12 00:46:17 INFO Vacuumed table table=multipart_uploads52312026/11/12 00:46:17 INFO Vacuumed table table=closures52322026/11/12 00:46:17 INFO Vacuumed table table=objects52332026/11/12 00:46:17 INFO Received complete upload request method=POST path=/52342026/11/12 00:46:17 INFO Completed upload id=30 objects=8 duration=1.01563ms52352026/11/12 00:46:17 INFO Received complete upload request method=POST path=/52362026/11/12 00:46:17 INFO Completed upload id=31 objects=6 duration=1.126171ms52372026/11/12 00:46:17 INFO Received complete upload request method=POST path=/52382026/11/12 00:46:17 INFO Completed upload id=32 objects=4 duration=829.588µs52392026/11/12 00:46:17 INFO Received complete upload request method=POST path=/52402026/11/12 00:46:17 INFO Completed upload id=33 objects=10 duration=1.060411ms52412026/11/12 00:46:17 INFO Received complete upload request method=POST path=/52422026/11/12 00:46:17 INFO Completed upload id=34 objects=23 duration=1.230452ms52432026/11/12 00:46:17 INFO Received complete push request method=POST path=/52442026/11/12 00:46:17 INFO Completed push id=35 objects=6 duration=1.00671ms52452026/11/12 00:46:17 INFO Aborted multipart uploads count=0 kept=052462026/11/12 00:46:17 WARN Force mode enabled - objects will be deleted immediately without grace period52472026/11/12 00:46:17 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=052482026/11/12 00:46:17 INFO Vacuumed table table=pending_closures52492026/11/12 00:46:17 INFO Vacuumed table table=commit_receipts52502026/11/12 00:46:17 INFO Vacuumed table table=pending_objects52512026/11/12 00:46:17 INFO Vacuumed table table=multipart_uploads52522026/11/12 00:46:17 INFO Vacuumed table table=closures52532026/11/12 00:46:17 INFO Vacuumed table table=objects5254--- PASS: FuzzPushGC (0.01s)5255 --- PASS: FuzzPushGC/seed#14 (3.08s)5256 --- PASS: FuzzPushGC/seed#7 (3.16s)5257 --- PASS: FuzzPushGC/seed#3 (3.24s)5258 --- PASS: FuzzPushGC/seed#11 (3.41s)5259 --- PASS: FuzzPushGC/seed#4 (3.62s)5260 --- PASS: FuzzPushGC/seed#6 (3.71s)5261 --- PASS: FuzzPushGC/seed#0 (3.75s)5262 --- PASS: FuzzPushGC/seed#2 (3.75s)5263 --- PASS: FuzzPushGC/seed#5 (3.77s)5264 --- PASS: FuzzPushGC/seed#15 (3.78s)5265 --- PASS: FuzzPushGC/seed#10 (3.78s)5266 --- PASS: FuzzPushGC/seed#13 (3.91s)5267 --- PASS: FuzzPushGC/seed#8 (3.94s)5268 --- PASS: FuzzPushGC/seed#9 (3.94s)5269 --- PASS: FuzzPushGC/seed#1 (4.10s)5270 --- PASS: FuzzPushGC/seed#12 (4.31s)5271PASS5272{"timestamp":"2026-11-12T00:46:17.589454742Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:45662","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(200)"}5273{"timestamp":"2026-11-12T00:46:17.589460082Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:44658","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(204)"}5274{"timestamp":"2026-11-12T00:46:17.589461262Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:45652","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(198)"}5275{"timestamp":"2026-11-12T00:46:17.589460182Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:45520","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}5276{"timestamp":"2026-11-12T00:46:17.589464982Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:45642","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(206)"}52772026-11-12 00:46:18.044 UTC [130] LOG: received smart shutdown request52782026-11-12 00:46:18.054 UTC [130] LOG: background worker "logical replication launcher" (PID 160) exited with exit code 152792026-11-12 00:46:19.041 UTC [155] LOG: checkpoint complete: wrote 4071 buffers (24.8%), wrote 5 SLRU buffers; 0 WAL file(s) added, 0 removed, 33 recycled; write=19.472 s, sync=0.709 s, total=20.338 s; sync files=43471, longest=0.004 s, average=0.001 s; distance=533427 kB, estimate=533427 kB; lsn=0/27954FB8, redo lsn=0/22052BA052802026-11-12 00:46:19.047 UTC [155] LOG: shutting down52812026-11-12 00:46:19.048 UTC [155] LOG: checkpoint starting: shutdown immediate52822026-11-12 00:46:19.633 UTC [155] LOG: checkpoint complete: wrote 8745 buffers (53.4%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 5 recycled; write=0.180 s, sync=0.395 s, total=0.586 s; sync files=2873, longest=0.004 s, average=0.001 s; distance=91145 kB, estimate=489199 kB; lsn=0/27955030, redo lsn=0/2795503052832026-11-12 00:46:19.833 UTC [130] LOG: database system is shut down5284Running OIDC tests...5285=== RUN TestAudienceForIssuer5286=== PAUSE TestAudienceForIssuer5287=== RUN TestProviderClientGivesUpOnSilentIssuer5288=== PAUSE TestProviderClientGivesUpOnSilentIssuer5289=== RUN TestGlobMatch5290=== PAUSE TestGlobMatch5291=== RUN TestValidateToken_ValidToken5292=== PAUSE TestValidateToken_ValidToken5293=== RUN TestValidateToken_WrongAudience5294=== PAUSE TestValidateToken_WrongAudience5295=== RUN TestValidateToken_Expired5296=== PAUSE TestValidateToken_Expired5297=== RUN TestValidateToken_BoundClaimsMismatch5298=== PAUSE TestValidateToken_BoundClaimsMismatch5299=== RUN TestValidateToken_BoundSubjectMismatch5300=== PAUSE TestValidateToken_BoundSubjectMismatch5301=== RUN TestValidateToken_MultipleProviders5302=== PAUSE TestValidateToken_MultipleProviders5303=== RUN TestValidateToken_UnknownIssuerFetchesNoKeys5304=== PAUSE TestValidateToken_UnknownIssuerFetchesNoKeys5305=== RUN TestValidateToken_GoogleSchemelessIssuer5306=== PAUSE TestValidateToken_GoogleSchemelessIssuer5307=== RUN TestValidateToken_KubernetesServiceAccount5308=== PAUSE TestValidateToken_KubernetesServiceAccount5309=== RUN TestNewValidator_KubernetesRequiresCA5310=== PAUSE TestNewValidator_KubernetesRequiresCA5311=== RUN TestValidateToken_KubernetesIssuerFromOwnToken5312=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken5313=== RUN TestPins_ReservedForMatchingRule5314=== PAUSE TestPins_ReservedForMatchingRule5315=== RUN TestPins_TopLevelShorthand5316=== PAUSE TestPins_TopLevelShorthand5317=== RUN TestPins_ConfigValidation5318=== PAUSE TestPins_ConfigValidation5319=== RUN TestScopes_LegacyProviderDefaultsToWrite5320=== PAUSE TestScopes_LegacyProviderDefaultsToWrite5321=== RUN TestScopes_Rules5322=== PAUSE TestScopes_Rules5323=== RUN TestScopes_ConfigValidation5324=== PAUSE TestScopes_ConfigValidation5325=== CONT TestAudienceForIssuer5326--- PASS: TestAudienceForIssuer (0.00s)5327=== CONT TestScopes_ConfigValidation5328--- PASS: TestScopes_ConfigValidation (0.00s)5329=== CONT TestScopes_Rules5330=== CONT TestScopes_LegacyProviderDefaultsToWrite5331=== CONT TestPins_ConfigValidation5332=== CONT TestPins_TopLevelShorthand5333=== CONT TestPins_ReservedForMatchingRule5334--- PASS: TestPins_ConfigValidation (0.01s)53352026/11/12 00:46:23 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:40349/oidc53362026/11/12 00:46:23 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:34015/oidc5337=== CONT TestValidateToken_BoundSubjectMismatch53382026/11/12 00:46:23 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:33021/oidc5339--- PASS: TestPins_ReservedForMatchingRule (0.08s)5340--- PASS: TestScopes_Rules (0.13s)5341--- PASS: TestScopes_LegacyProviderDefaultsToWrite (0.12s)5342=== CONT TestValidateToken_KubernetesIssuerFromOwnToken5343=== CONT TestValidateToken_WrongAudience5344=== CONT TestValidateToken_ValidToken5345=== CONT TestGlobMatch5346=== RUN TestGlobMatch/foo_foo5347=== PAUSE TestGlobMatch/foo_foo5348=== RUN TestGlobMatch/foo_bar5349=== PAUSE TestGlobMatch/foo_bar5350=== RUN TestGlobMatch/*_5351=== PAUSE TestGlobMatch/*_5352=== RUN TestGlobMatch/*_anything5353=== PAUSE TestGlobMatch/*_anything5354=== RUN TestGlobMatch/foo*_foo5355=== PAUSE TestGlobMatch/foo*_foo5356=== RUN TestGlobMatch/foo*_foobar5357=== PAUSE TestGlobMatch/foo*_foobar5358=== RUN TestGlobMatch/foo*_bar5359=== PAUSE TestGlobMatch/foo*_bar5360=== RUN TestGlobMatch/*bar_bar5361=== CONT TestProviderClientGivesUpOnSilentIssuer5362=== PAUSE TestGlobMatch/*bar_bar5363=== CONT TestNewValidator_KubernetesRequiresCA5364=== CONT TestValidateToken_UnknownIssuerFetchesNoKeys5365=== CONT TestValidateToken_KubernetesServiceAccount5366=== CONT TestValidateToken_BoundClaimsMismatch5367=== CONT TestValidateToken_Expired5368=== CONT TestValidateToken_MultipleProviders5369=== CONT TestValidateToken_GoogleSchemelessIssuer5370--- PASS: TestProviderClientGivesUpOnSilentIssuer (0.00s)5371=== RUN TestGlobMatch/*bar_foobar5372=== PAUSE TestGlobMatch/*bar_foobar5373=== RUN TestGlobMatch/*bar_foo5374=== PAUSE TestGlobMatch/*bar_foo5375=== RUN TestGlobMatch/foo*bar_foobar5376=== PAUSE TestGlobMatch/foo*bar_foobar5377=== RUN TestGlobMatch/foo*bar_foo123bar5378=== PAUSE TestGlobMatch/foo*bar_foo123bar5379=== RUN TestGlobMatch/foo*bar_foobarbaz5380=== PAUSE TestGlobMatch/foo*bar_foobarbaz5381=== RUN TestGlobMatch/*/*_foo/bar5382=== PAUSE TestGlobMatch/*/*_foo/bar5383=== RUN TestGlobMatch/*/*_foo5384=== PAUSE TestGlobMatch/*/*_foo5385=== RUN TestGlobMatch/refs/heads/*_refs/heads/main5386=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main5387=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.05388=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.05389=== RUN TestGlobMatch/refs/*/main_refs/heads/main5390=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main5391=== RUN TestGlobMatch/fo?_foo5392=== PAUSE TestGlobMatch/fo?_foo5393=== RUN TestGlobMatch/fo?_fo5394=== PAUSE TestGlobMatch/fo?_fo5395=== RUN TestGlobMatch/fo?_fooo5396=== PAUSE TestGlobMatch/fo?_fooo5397=== RUN TestGlobMatch/?oo_foo5398=== PAUSE TestGlobMatch/?oo_foo5399=== RUN TestGlobMatch/?oo_boo5400=== PAUSE TestGlobMatch/?oo_boo5401=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5402=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5403=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5404=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5405=== CONT TestGlobMatch/foo_foo5406=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5407=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5408=== CONT TestGlobMatch/?oo_boo5409=== CONT TestGlobMatch/?oo_foo5410=== CONT TestGlobMatch/fo?_fooo5411=== CONT TestGlobMatch/fo?_fo5412=== CONT TestGlobMatch/fo?_foo5413=== CONT TestGlobMatch/refs/*/main_refs/heads/main5414=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.05415=== CONT TestGlobMatch/refs/heads/*_refs/heads/main5416=== CONT TestGlobMatch/*/*_foo5417=== CONT TestGlobMatch/*/*_foo/bar5418=== CONT TestGlobMatch/foo*bar_foobarbaz5419=== CONT TestGlobMatch/foo*bar_foo123bar5420=== CONT TestGlobMatch/foo*bar_foobar5421=== CONT TestGlobMatch/*bar_foo5422=== CONT TestGlobMatch/*bar_foobar5423=== CONT TestGlobMatch/*bar_bar5424=== CONT TestGlobMatch/foo*_bar5425=== CONT TestGlobMatch/foo*_foobar5426=== CONT TestGlobMatch/foo*_foo5427=== CONT TestGlobMatch/*_anything5428=== CONT TestGlobMatch/*_5429=== CONT TestGlobMatch/foo_bar5430--- PASS: TestGlobMatch (0.02s)5431 --- PASS: TestGlobMatch/foo_foo (0.00s)5432 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)5433 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)5434 --- PASS: TestGlobMatch/?oo_boo (0.00s)5435 --- PASS: TestGlobMatch/?oo_foo (0.00s)5436 --- PASS: TestGlobMatch/fo?_fooo (0.00s)5437 --- PASS: TestGlobMatch/fo?_fo (0.00s)5438 --- PASS: TestGlobMatch/fo?_foo (0.00s)5439 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)5440 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)5441 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)5442 --- PASS: TestGlobMatch/*/*_foo (0.00s)5443 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)5444 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)5445 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)5446 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)5447 --- PASS: TestGlobMatch/*bar_foo (0.00s)5448 --- PASS: TestGlobMatch/*bar_foobar (0.00s)5449 --- PASS: TestGlobMatch/*bar_bar (0.00s)5450 --- PASS: TestGlobMatch/foo*_bar (0.00s)5451 --- PASS: TestGlobMatch/foo*_foobar (0.00s)5452 --- PASS: TestGlobMatch/foo*_foo (0.00s)5453 --- PASS: TestGlobMatch/*_anything (0.00s)5454 --- PASS: TestGlobMatch/*_ (0.00s)5455 --- PASS: TestGlobMatch/foo_bar (0.00s)54562026/11/12 00:46:24 INFO OIDC provider initialized name=known issuer=https://known.example54572026/11/12 00:46:24 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC12354582026/11/12 00:46:24 INFO OIDC provider initialized name=google issuer=https://accounts.google.com54592026/11/12 00:46:24 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:33741/oidc54602026/11/12 00:46:24 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:35557/oidc5461--- PASS: TestValidateToken_BoundSubjectMismatch (0.91s)54622026/11/12 00:46:24 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:39081/oidc5463--- PASS: TestValidateToken_BoundClaimsMismatch (0.25s)5464--- PASS: TestPins_TopLevelShorthand (2.10s)5465--- PASS: TestValidateToken_UnknownIssuerFetchesNoKeys (1.30s)5466--- PASS: TestValidateToken_GoogleSchemelessIssuer (1.31s)54672026/11/12 00:46:25 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:382475468--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (1.53s)5469--- PASS: TestValidateToken_KubernetesServiceAccount (1.30s)54702026/11/12 00:46:25 http: TLS handshake error from 127.0.0.1:32914: remote error: tls: bad certificate5471--- PASS: TestNewValidator_KubernetesRequiresCA (1.32s)54722026/11/12 00:46:25 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:45459/oidc5473--- PASS: TestValidateToken_Expired (1.38s)54742026/11/12 00:46:25 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:41693/oidc5475--- PASS: TestValidateToken_ValidToken (1.43s)54762026/11/12 00:46:25 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:40347/oidc5477--- PASS: TestValidateToken_WrongAudience (1.46s)54782026/11/12 00:46:25 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:39717/oidc54792026/11/12 00:46:25 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:39657/oidc5480--- PASS: TestValidateToken_MultipleProviders (1.74s)5481PASS5482Running hook tests...5483=== RUN TestSendPathsEmpty5484=== PAUSE TestSendPathsEmpty5485=== RUN TestQueueEnqueueAndFetch5486=== PAUSE TestQueueEnqueueAndFetch5487=== RUN TestQueueDeduplication5488=== PAUSE TestQueueDeduplication5489=== RUN TestQueueRemove5490=== PAUSE TestQueueRemove5491=== RUN TestQueueFetchBatchLimit5492=== PAUSE TestQueueFetchBatchLimit5493=== RUN TestQueueRetryMovesToBack5494=== PAUSE TestQueueRetryMovesToBack5495=== RUN TestQueueFetchRemoveLifecycle5496=== PAUSE TestQueueFetchRemoveLifecycle5497=== RUN TestQueueConcurrentWriters5498=== PAUSE TestQueueConcurrentWriters5499=== RUN TestQueueRemoveLargeClosure5500=== PAUSE TestQueueRemoveLargeClosure5501=== RUN TestServerClientIntegration5502--- PASS: TestServerClientIntegration (0.01s)5503=== RUN TestServerQueueError55042026/11/12 00:46:26 ERROR Failed to queue paths error="permission denied" count=15505--- PASS: TestServerQueueError (0.00s)5506=== RUN TestGetListenerSocketActivation5507 server_test.go:220: === RUN TestGetListenerSocketActivation5508 --- PASS: TestGetListenerSocketActivation (0.00s)5509 PASS5510 5511--- PASS: TestGetListenerSocketActivation (0.14s)5512=== RUN TestServerStalledClientDoesNotBlockShutdown5513=== PAUSE TestServerStalledClientDoesNotBlockShutdown5514=== RUN TestServerBacksOffOnAcceptErrors5515=== PAUSE TestServerBacksOffOnAcceptErrors5516=== RUN TestDrainIsolatesPoisonPath5517=== PAUSE TestDrainIsolatesPoisonPath5518=== RUN TestRunNotBlockedByPoisonHead5519=== PAUSE TestRunNotBlockedByPoisonHead5520=== RUN TestDrainGivesUpWhenServerDown5521=== PAUSE TestDrainGivesUpWhenServerDown5522=== RUN TestFailedPathPrunedByLaterClosure5523=== PAUSE TestFailedPathPrunedByLaterClosure5524=== RUN TestWorkerUploadsAndRemoves5525=== PAUSE TestWorkerUploadsAndRemoves5526=== RUN TestWorkerSkipsGCdPaths5527=== PAUSE TestWorkerSkipsGCdPaths5528=== RUN TestWorkerPrunesClosureDeps5529=== PAUSE TestWorkerPrunesClosureDeps5530=== RUN TestDrainTimeout5531=== PAUSE TestDrainTimeout5532=== RUN TestWorkerRemoveFailureIsNotProgress5533=== PAUSE TestWorkerRemoveFailureIsNotProgress5534=== RUN TestWorkerKeepsPathItCannotStat5535=== PAUSE TestWorkerKeepsPathItCannotStat5536=== CONT TestSendPathsEmpty5537=== CONT TestDrainIsolatesPoisonPath5538=== CONT TestWorkerRemoveFailureIsNotProgress5539--- PASS: TestSendPathsEmpty (0.00s)5540=== CONT TestServerBacksOffOnAcceptErrors55412000/01/01 00:00:00 ERROR Accept failed error="too many open files"55422000/01/01 00:00:00 ERROR Accept failed error="too many open files"55432000/01/01 00:00:00 ERROR Accept failed error="too many open files"55442000/01/01 00:00:00 ERROR Accept failed error="too many open files"55452000/01/01 00:00:00 ERROR Accept failed error="too many open files"55462000/01/01 00:00:00 ERROR Accept failed error="too many open files"55472000/01/01 00:00:00 ERROR Accept failed error="too many open files"55482000/01/01 00:00:00 ERROR Accept failed error="too many open files"5549=== CONT TestServerStalledClientDoesNotBlockShutdown5550--- PASS: TestServerBacksOffOnAcceptErrors (0.00s)5551=== CONT TestQueueRemoveLargeClosure5552=== CONT TestQueueConcurrentWriters5553=== CONT TestQueueFetchRemoveLifecycle5554=== CONT TestQueueRetryMovesToBack5555=== CONT TestQueueFetchBatchLimit5556=== CONT TestQueueRemove5557=== CONT TestQueueDeduplication5558=== CONT TestQueueEnqueueAndFetch5559=== CONT TestWorkerSkipsGCdPaths5560=== CONT TestDrainTimeout5561=== CONT TestWorkerPrunesClosureDeps5562=== CONT TestFailedPathPrunedByLaterClosure5563=== CONT TestWorkerUploadsAndRemoves5564=== CONT TestWorkerKeepsPathItCannotStat5565=== CONT TestDrainGivesUpWhenServerDown5566=== CONT TestRunNotBlockedByPoisonHead5567--- PASS: TestQueueFetchBatchLimit (0.03s)5568--- PASS: TestQueueRemove (0.03s)5569--- PASS: TestQueueDeduplication (0.03s)5570--- PASS: TestQueueEnqueueAndFetch (0.04s)55712026/11/12 00:46:26 INFO Upload queue status pending=255722026/11/12 00:46:26 INFO Uploading batch count=155732026/11/12 00:46:26 ERROR Upload failed error="upload failed" count=155742026/11/12 00:46:26 INFO Uploading batch count=155752026/11/12 00:46:26 INFO Upload queue status pending=255762026/11/12 00:46:26 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerSkipsGCdPaths4120191497/002/nonexistent55772026/11/12 00:46:26 INFO Uploading batch count=155782026/11/12 00:46:26 INFO Uploading batch count=155792026/11/12 00:46:26 INFO Uploading batch count=155802026/11/12 00:46:26 INFO Upload queue status pending=255812026/11/12 00:46:26 INFO Uploading batch count=255822026/11/12 00:46:26 INFO Uploading batch count=455832026/11/12 00:46:26 ERROR Upload failed error="upload failed" count=455842026/11/12 00:46:26 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainIsolatesPoisonPath3850597632/002/bbb55852026/11/12 00:46:26 INFO Uploading batch count=155862026/11/12 00:46:26 ERROR Upload failed error="upload failed" count=155872026/11/12 00:46:26 INFO Uploading batch count=155882026/11/12 00:46:26 ERROR Upload failed error="upload failed" count=155892026/11/12 00:46:26 INFO Uploading batch count=155902026/11/12 00:46:26 ERROR Upload failed error="upload failed" count=155912026/11/12 00:46:26 ERROR Drain finished with paths left in queue remaining=15592--- PASS: TestDrainIsolatesPoisonPath (0.06s)55932026/11/12 00:46:26 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress3613513944/002/nonexistent55942026/11/12 00:46:26 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=155952026/11/12 00:46:26 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress3613513944/002/nonexistent55962026/11/12 00:46:26 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=155972026/11/12 00:46:26 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress3613513944/002/nonexistent55982026/11/12 00:46:26 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=155992026/11/12 00:46:26 ERROR Drain finished with paths left in queue remaining=15600--- PASS: TestWorkerRemoveFailureIsNotProgress (0.06s)56012000/01/01 00:00:00 INFO Uploading batch count=256022000/01/01 00:00:00 ERROR Upload failed error="context deadline exceeded" count=256032000/01/01 00:00:00 ERROR Drain finished with paths left in queue remaining=45604--- PASS: TestFailedPathPrunedByLaterClosure (0.06s)5605--- PASS: TestWorkerUploadsAndRemoves (0.07s)5606--- PASS: TestDrainTimeout (0.07s)5607--- PASS: TestQueueFetchRemoveLifecycle (0.07s)56082026/11/12 00:46:27 ERROR Failed to decode request error="read unix /build/hook842726920/test.sock->@: i/o timeout"56092026/11/12 00:46:27 ERROR Failed to write response error="write unix /build/hook842726920/test.sock->@: i/o timeout"5610--- PASS: TestServerStalledClientDoesNotBlockShutdown (1.79s)56112026/11/12 00:46:27 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat3315847952/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat3315847952/002/locked/aaa: permission denied"56122026/11/12 00:46:27 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat3315847952/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat3315847952/002/locked/aaa: permission denied"5613--- PASS: TestWorkerSkipsGCdPaths (1.80s)56142026/11/12 00:46:27 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat3315847952/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat3315847952/002/locked/aaa: permission denied"5615--- PASS: TestWorkerPrunesClosureDeps (1.80s)56162026/11/12 00:46:27 ERROR Drain finished with paths left in queue remaining=156172026/11/12 00:46:27 INFO Uploading batch count=256182026/11/12 00:46:27 ERROR Upload failed error="upload failed" count=256192026/11/12 00:46:27 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown2435661522/002/a5620--- PASS: TestQueueRetryMovesToBack (1.81s)56212026/11/12 00:46:27 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown2435661522/002/b5622--- PASS: TestWorkerKeepsPathItCannotStat (1.81s)56232026/11/12 00:46:27 INFO Uploading batch count=256242026/11/12 00:46:27 ERROR Upload failed error="upload failed" count=256252026/11/12 00:46:27 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown2435661522/002/c56262026/11/12 00:46:27 INFO Upload queue status pending=356272026/11/12 00:46:27 INFO Uploading batch count=156282026/11/12 00:46:27 ERROR Upload failed error="upload failed" count=156292026/11/12 00:46:27 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown2435661522/002/d56302026/11/12 00:46:27 INFO Uploading batch count=256312026/11/12 00:46:27 ERROR Upload failed error="upload failed" count=256322026/11/12 00:46:27 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown2435661522/002/e56332026/11/12 00:46:27 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown2435661522/002/f56342026/11/12 00:46:27 ERROR Drain finished with paths left in queue remaining=105635--- PASS: TestDrainGivesUpWhenServerDown (1.81s)5636--- PASS: TestQueueConcurrentWriters (1.91s)5637--- PASS: TestQueueRemoveLargeClosure (1.92s)56382026/11/12 00:46:28 INFO Uploading batch count=156392026/11/12 00:46:28 INFO Uploading batch count=156402026/11/12 00:46:28 INFO Uploading batch count=156412026/11/12 00:46:28 ERROR Upload failed error="upload failed" count=156422026/11/12 00:46:28 INFO Uploading batch count=156432026/11/12 00:46:28 ERROR Upload failed error="upload failed" count=156442026/11/12 00:46:28 INFO Uploading batch count=156452026/11/12 00:46:28 ERROR Upload failed error="upload failed" count=156462026/11/12 00:46:28 INFO Uploading batch count=156472026/11/12 00:46:28 ERROR Upload failed error="upload failed" count=156482026/11/12 00:46:28 ERROR Drain finished with paths left in queue remaining=15649--- PASS: TestRunNotBlockedByPoisonHead (2.82s)5650PASS5651Running signing tests...5652=== RUN TestExternalSigner5653=== PAUSE TestExternalSigner5654=== RUN TestExternalSignerWrongPublicKey5655=== PAUSE TestExternalSignerWrongPublicKey5656=== RUN TestExternalSignerErrors5657=== PAUSE TestExternalSignerErrors5658=== RUN TestExternalSignerRestartsAfterCrash5659=== PAUSE TestExternalSignerRestartsAfterCrash5660=== RUN TestExternalSignerProgramErrorKeepsProcess5661=== PAUSE TestExternalSignerProgramErrorKeepsProcess5662=== RUN TestExternalSignerClose5663=== PAUSE TestExternalSignerClose5664=== RUN TestExternalSignerRejectsInvalidPublicKey5665=== PAUSE TestExternalSignerRejectsInvalidPublicKey5666=== RUN TestExternalSignerCancellation5667=== PAUSE TestExternalSignerCancellation5668=== RUN TestGenerateFingerprint5669=== PAUSE TestGenerateFingerprint5670=== RUN TestParseSigningKey5671=== PAUSE TestParseSigningKey5672=== RUN TestSign5673=== PAUSE TestSign5674=== RUN TestSignBatch5675=== PAUSE TestSignBatch5676=== RUN TestNilKeyPublicKey5677=== PAUSE TestNilKeyPublicKey5678=== RUN TestSignErrors5679=== PAUSE TestSignErrors5680=== CONT TestExternalSigner5681=== CONT TestExternalSignerProgramErrorKeepsProcess5682=== CONT TestExternalSignerRejectsInvalidPublicKey5683=== RUN TestExternalSignerRejectsInvalidPublicKey/#005684=== PAUSE TestExternalSignerRejectsInvalidPublicKey/#005685=== RUN TestExternalSignerRejectsInvalidPublicKey/no-colon5686=== PAUSE TestExternalSignerRejectsInvalidPublicKey/no-colon5687=== RUN TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5688=== PAUSE TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5689=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5690=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5691=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5692=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5693=== CONT TestExternalSignerRejectsInvalidPublicKey/#005694=== CONT TestExternalSignerClose5695=== CONT TestExternalSignerCancellation5696=== CONT TestSignBatch5697=== CONT TestSign5698=== CONT TestExternalSignerErrors5699=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5700=== CONT TestExternalSignerRestartsAfterCrash5701=== RUN TestExternalSignerErrors/exit_on_request5702=== PAUSE TestExternalSignerErrors/exit_on_request5703=== RUN TestExternalSignerErrors/invalid_JSON5704=== CONT TestParseSigningKey5705=== RUN TestParseSigningKey/valid_32-byte_key5706=== PAUSE TestParseSigningKey/valid_32-byte_key5707=== RUN TestParseSigningKey/valid_32-byte_key_with_different_name5708=== PAUSE TestParseSigningKey/valid_32-byte_key_with_different_name5709=== CONT TestSignErrors5710=== CONT TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5711=== CONT TestGenerateFingerprint5712=== RUN TestGenerateFingerprint/basic_with_references5713=== PAUSE TestGenerateFingerprint/basic_with_references5714=== RUN TestGenerateFingerprint/no_references5715=== PAUSE TestGenerateFingerprint/no_references5716=== CONT TestExternalSignerRejectsInvalidPublicKey/no-colon5717=== CONT TestNilKeyPublicKey5718--- PASS: TestNilKeyPublicKey (0.00s)5719=== CONT TestExternalSignerWrongPublicKey5720=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5721--- PASS: TestExternalSignerRejectsInvalidPublicKey (0.00s)5722 --- PASS: TestExternalSignerRejectsInvalidPublicKey/#00 (0.00s)5723 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:invalid! (0.00s)5724 --- PASS: TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= (0.00s)5725 --- PASS: TestExternalSignerRejectsInvalidPublicKey/no-colon (0.00s)5726 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:YQ== (0.00s)5727=== PAUSE TestExternalSignerErrors/invalid_JSON5728=== RUN TestExternalSignerErrors/wrong_id5729=== RUN TestParseSigningKey/no_colon5730=== PAUSE TestParseSigningKey/no_colon5731=== RUN TestParseSigningKey/empty_name5732=== PAUSE TestParseSigningKey/empty_name5733=== RUN TestParseSigningKey/invalid_base645734=== PAUSE TestParseSigningKey/invalid_base645735=== RUN TestParseSigningKey/wrong_length5736=== RUN TestGenerateFingerprint/unsorted_references_get_sorted5737=== PAUSE TestGenerateFingerprint/unsorted_references_get_sorted5738=== PAUSE TestExternalSignerErrors/wrong_id5739=== PAUSE TestParseSigningKey/wrong_length5740=== CONT TestParseSigningKey/valid_32-byte_key5741=== CONT TestParseSigningKey/invalid_base645742=== RUN TestExternalSignerErrors/missing_signatures5743=== PAUSE TestExternalSignerErrors/missing_signatures5744=== CONT TestParseSigningKey/wrong_length5745=== CONT TestParseSigningKey/empty_name5746=== RUN TestGenerateFingerprint/invalid_nar_hash_prefix5747=== CONT TestParseSigningKey/valid_32-byte_key_with_different_name5748=== RUN TestExternalSignerErrors/too_few_signatures5749=== PAUSE TestExternalSignerErrors/too_few_signatures5750=== RUN TestExternalSignerErrors/null5751=== PAUSE TestExternalSignerErrors/null5752=== RUN TestExternalSignerErrors/null_signatures5753=== PAUSE TestExternalSignerErrors/null_signatures5754=== RUN TestExternalSignerErrors/wrong_signatures_shape5755=== PAUSE TestExternalSignerErrors/wrong_signatures_shape5756=== RUN TestExternalSignerErrors/missing_signature_name5757=== PAUSE TestExternalSignerErrors/missing_signature_name5758=== RUN TestExternalSignerErrors/wrong_signature_name5759=== PAUSE TestExternalSignerErrors/wrong_signature_name5760=== RUN TestExternalSignerErrors/invalid_signature_base645761=== PAUSE TestExternalSignerErrors/invalid_signature_base645762=== RUN TestExternalSignerErrors/wrong_signature_length5763=== PAUSE TestExternalSignerErrors/wrong_signature_length5764=== RUN TestExternalSignerErrors/program_error5765=== PAUSE TestExternalSignerErrors/program_error5766=== RUN TestExternalSignerErrors/exit_at_startup5767=== PAUSE TestExternalSignerErrors/exit_at_startup5768=== RUN TestExternalSignerErrors/wrong_shape5769=== PAUSE TestExternalSignerErrors/wrong_shape5770=== RUN TestExternalSignerErrors/unrequested_key5771=== PAUSE TestExternalSignerErrors/unrequested_key5772=== CONT TestExternalSignerErrors/exit_on_request5773=== CONT TestParseSigningKey/no_colon5774=== PAUSE TestGenerateFingerprint/invalid_nar_hash_prefix5775=== RUN TestGenerateFingerprint/invalid_nar_hash_length5776=== PAUSE TestGenerateFingerprint/invalid_nar_hash_length5777=== RUN TestGenerateFingerprint/invalid_store_path_prefix5778=== PAUSE TestGenerateFingerprint/invalid_store_path_prefix5779=== RUN TestGenerateFingerprint/invalid_reference_prefix5780=== PAUSE TestGenerateFingerprint/invalid_reference_prefix5781=== CONT TestGenerateFingerprint/basic_with_references5782=== CONT TestExternalSignerErrors/unrequested_key5783=== CONT TestExternalSignerErrors/wrong_shape5784=== CONT TestExternalSignerErrors/exit_at_startup5785=== CONT TestExternalSignerErrors/program_error5786=== CONT TestGenerateFingerprint/invalid_store_path_prefix5787=== CONT TestGenerateFingerprint/unsorted_references_get_sorted5788=== CONT TestExternalSignerErrors/missing_signature_name5789=== CONT TestGenerateFingerprint/invalid_nar_hash_prefix5790=== CONT TestExternalSignerErrors/invalid_signature_base645791=== CONT TestExternalSignerErrors/wrong_id5792=== CONT TestGenerateFingerprint/invalid_nar_hash_length5793=== CONT TestExternalSignerErrors/wrong_signature_name5794=== CONT TestExternalSignerErrors/missing_signatures5795--- PASS: TestSign (0.00s)5796=== CONT TestExternalSignerErrors/wrong_signature_length5797=== RUN TestSignErrors/nil_key5798=== PAUSE TestSignErrors/nil_key5799=== RUN TestSignErrors/nil_narinfo5800=== CONT TestGenerateFingerprint/no_references5801=== CONT TestExternalSignerErrors/invalid_JSON5802=== CONT TestGenerateFingerprint/invalid_reference_prefix5803=== CONT TestExternalSignerErrors/null_signatures5804=== PAUSE TestSignErrors/nil_narinfo5805=== RUN TestSignErrors/invalid_narinfo5806=== PAUSE TestSignErrors/invalid_narinfo5807=== RUN TestSignErrors/canceled5808=== PAUSE TestSignErrors/canceled5809=== CONT TestExternalSignerErrors/null5810--- PASS: TestSignBatch (0.00s)5811=== CONT TestExternalSignerErrors/too_few_signatures5812--- PASS: TestGenerateFingerprint (0.00s)5813 --- PASS: TestGenerateFingerprint/basic_with_references (0.00s)5814 --- PASS: TestGenerateFingerprint/invalid_store_path_prefix (0.00s)5815 --- PASS: TestGenerateFingerprint/unsorted_references_get_sorted (0.00s)5816 --- PASS: TestGenerateFingerprint/invalid_nar_hash_prefix (0.00s)5817 --- PASS: TestGenerateFingerprint/invalid_nar_hash_length (0.00s)5818 --- PASS: TestGenerateFingerprint/invalid_reference_prefix (0.00s)5819 --- PASS: TestGenerateFingerprint/no_references (0.00s)5820=== CONT TestSignErrors/nil_key5821=== CONT TestSignErrors/nil_narinfo5822--- PASS: TestParseSigningKey (0.00s)5823 --- PASS: TestParseSigningKey/invalid_base64 (0.00s)5824 --- PASS: TestParseSigningKey/wrong_length (0.00s)5825 --- PASS: TestParseSigningKey/empty_name (0.00s)5826 --- PASS: TestParseSigningKey/no_colon (0.00s)5827 --- PASS: TestParseSigningKey/valid_32-byte_key (0.00s)5828 --- PASS: TestParseSigningKey/valid_32-byte_key_with_different_name (0.00s)5829=== CONT TestExternalSignerErrors/wrong_signatures_shape5830=== CONT TestSignErrors/invalid_narinfo5831=== CONT TestSignErrors/canceled5832--- PASS: TestSignErrors (0.00s)5833 --- PASS: TestSignErrors/nil_key (0.00s)5834 --- PASS: TestSignErrors/nil_narinfo (0.00s)5835 --- PASS: TestSignErrors/invalid_narinfo (0.00s)5836 --- PASS: TestSignErrors/canceled (0.00s)5837--- PASS: TestExternalSigner (0.02s)5838--- PASS: TestExternalSignerProgramErrorKeepsProcess (0.03s)5839--- PASS: TestExternalSignerWrongPublicKey (0.02s)5840--- PASS: TestExternalSignerClose (0.03s)5841--- PASS: TestExternalSignerErrors (0.00s)5842 --- PASS: TestExternalSignerErrors/exit_at_startup (0.01s)5843 --- PASS: TestExternalSignerErrors/wrong_signature_name (0.01s)5844 --- PASS: TestExternalSignerErrors/wrong_id (0.02s)5845 --- PASS: TestExternalSignerErrors/wrong_shape (0.02s)5846 --- PASS: TestExternalSignerErrors/unrequested_key (0.02s)5847 --- PASS: TestExternalSignerErrors/invalid_JSON (0.02s)5848 --- PASS: TestExternalSignerErrors/null (0.02s)5849 --- PASS: TestExternalSignerErrors/missing_signature_name (0.02s)5850 --- PASS: TestExternalSignerErrors/null_signatures (0.02s)5851 --- PASS: TestExternalSignerErrors/program_error (0.02s)5852 --- PASS: TestExternalSignerErrors/invalid_signature_base64 (0.02s)5853 --- PASS: TestExternalSignerErrors/missing_signatures (0.02s)5854 --- PASS: TestExternalSignerErrors/too_few_signatures (0.02s)5855 --- PASS: TestExternalSignerErrors/exit_on_request (0.02s)5856 --- PASS: TestExternalSignerErrors/wrong_signatures_shape (0.02s)5857 --- PASS: TestExternalSignerErrors/wrong_signature_length (0.02s)5858--- PASS: TestExternalSignerRestartsAfterCrash (0.03s)5859--- PASS: TestExternalSignerCancellation (0.04s)5860PASS5861Running OpenBao signer tests...5862=== RUN TestSign5863=== PAUSE TestSign5864=== RUN TestSignErrors5865=== PAUSE TestSignErrors5866=== RUN TestServe5867=== PAUSE TestServe5868=== RUN TestServeRejectsGarbage5869=== PAUSE TestServeRejectsGarbage5870=== CONT TestSign5871=== CONT TestServe5872=== CONT TestServeRejectsGarbage5873--- PASS: TestServeRejectsGarbage (0.00s)5874=== CONT TestSignErrors5875=== RUN TestSignErrors/http_error5876=== PAUSE TestSignErrors/http_error5877=== RUN TestSignErrors/invalid_json5878=== PAUSE TestSignErrors/invalid_json5879=== RUN TestSignErrors/no_prefix5880=== PAUSE TestSignErrors/no_prefix5881=== RUN TestSignErrors/invalid_base645882=== PAUSE TestSignErrors/invalid_base645883=== CONT TestSignErrors/http_error5884=== CONT TestSignErrors/invalid_base645885=== CONT TestSignErrors/no_prefix5886=== CONT TestSignErrors/invalid_json5887--- PASS: TestServe (0.00s)5888--- PASS: TestSign (1.29s)5889--- PASS: TestSignErrors (0.00s)5890 --- PASS: TestSignErrors/invalid_base64 (1.29s)5891 --- PASS: TestSignErrors/http_error (1.29s)5892 --- PASS: TestSignErrors/no_prefix (1.29s)5893 --- PASS: TestSignErrors/invalid_json (1.29s)5894PASS

succeeded niks3-go-unit-tests checks.aarch64-linux.go-unit-tests · build #506 · raw

succeeded `niks3-go-unit-tests` `checks.aarch64-linux.go-unit-tests` · build #506 · raw