niks3-go-unit-tests
checks.aarch64-linux.go-unit-tests
· build #510
· raw
1tribuchet: building on eliza2Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestUploadBuildLog_FileBodyReplayedOnRetry6=== PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry7=== RUN TestRegisterUploadedObjectReusesConnections8=== PAUSE TestRegisterUploadedObjectReusesConnections9=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica10=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica11=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun12=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun13=== RUN TestCaseHackSuffix14=== PAUSE TestCaseHackSuffix15=== RUN TestFilterOversizedClosures16=== PAUSE TestFilterOversizedClosures17=== RUN TestUploadMultipart_PartsInParallel18=== PAUSE TestUploadMultipart_PartsInParallel19=== RUN TestUploadMultipart_ReturnsAfterDroppedBody20=== PAUSE TestUploadMultipart_ReturnsAfterDroppedBody21=== RUN TestUploadMultipart_FailedProducerIsNotCompleted22=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted23=== RUN TestPartSizeForNAR24=== PAUSE TestPartSizeForNAR25=== RUN TestDumpPathCaseHackMatchesNix26--- PASS: TestDumpPathCaseHackMatchesNix (0.03s)27=== RUN TestDumpPathCaseHackCollision28--- PASS: TestDumpPathCaseHackCollision (0.00s)29=== RUN TestDumpPathMatchesNix30=== PAUSE TestDumpPathMatchesNix31=== RUN TestDumpPathSingleFile32=== PAUSE TestDumpPathSingleFile33=== RUN TestDumpPathWriterError34=== PAUSE TestDumpPathWriterError35=== RUN TestEncodeNixBase3236=== PAUSE TestEncodeNixBase3237=== RUN TestEncodeNixBase32WithRealHash38=== PAUSE TestEncodeNixBase32WithRealHash39=== RUN TestConvertHashToNix3240=== PAUSE TestConvertHashToNix3241=== RUN TestGetStorePathHash42=== PAUSE TestGetStorePathHash43=== RUN TestPathInfoHashCompatibility44=== PAUSE TestPathInfoHashCompatibility45=== RUN TestParsePathInfoJSON46=== PAUSE TestParsePathInfoJSON47=== RUN TestParsePathInfoJSONMultiplePaths48=== PAUSE TestParsePathInfoJSONMultiplePaths49=== RUN TestPathInfoCACompatibility50=== PAUSE TestPathInfoCACompatibility51=== RUN TestRateLimiterFeedback52=== PAUSE TestRateLimiterFeedback53=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess54=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess55=== RUN TestRegisterUploadedObject_BoundedAgainstSilentServer56=== PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer57=== RUN TestRegisterUploadedObject_OutlivesPush58=== PAUSE TestRegisterUploadedObject_OutlivesPush59=== RUN TestResolveStorePath60=== PAUSE TestResolveStorePath61=== RUN TestDoWithRetry_BodyReplayedViaGetBody62=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody63=== RUN TestRetryTimeoutOutlastsMaxRetries64=== PAUSE TestRetryTimeoutOutlastsMaxRetries65=== RUN TestDoWithRetry_FinalResponseBodyReadable66=== PAUSE TestDoWithRetry_FinalResponseBodyReadable67=== RUN TestShellSplit68=== PAUSE TestShellSplit69=== RUN TestShellSplitErrors70=== PAUSE TestShellSplitErrors71=== RUN TestStreamPushReportsEveryPath72=== PAUSE TestStreamPushReportsEveryPath73=== RUN TestStreamPushBatchesUnderLoad74=== PAUSE TestStreamPushBatchesUnderLoad75=== RUN TestStreamPushIsolatesFailures76=== PAUSE TestStreamPushIsolatesFailures77=== RUN TestStreamPushGivesUpOnDeadServer78=== PAUSE TestStreamPushGivesUpOnDeadServer79=== RUN TestStreamPushRequestLine80=== PAUSE TestStreamPushRequestLine81=== RUN TestStreamPushReportsSignatures82=== PAUSE TestStreamPushReportsSignatures83=== RUN TestStreamPushReportsSkippedPaths84=== PAUSE TestStreamPushReportsSkippedPaths85=== RUN TestClientSignaturesByStorePath86=== PAUSE TestClientSignaturesByStorePath87=== RUN TestStreamPushAnswersEveryLineWhenCanceled88=== PAUSE TestStreamPushAnswersEveryLineWhenCanceled89=== RUN TestStreamPushStopsWhenCanceled90=== PAUSE TestStreamPushStopsWhenCanceled91=== RUN TestSetClientTLS92=== PAUSE TestSetClientTLS93=== RUN TestSetClientTLSDoesNotMutateDefaultTransport94=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport95=== RUN TestSetClientTLSErrors96=== PAUSE TestSetClientTLSErrors97=== RUN TestStaticToken98=== PAUSE TestStaticToken99=== RUN TestFileTokenReadsAndCaches100=== PAUSE TestFileTokenReadsAndCaches101=== RUN TestFileTokenMissing102=== PAUSE TestFileTokenMissing103=== RUN TestFileTokenEmpty104=== PAUSE TestFileTokenEmpty105=== RUN TestScriptTokenNoExpiryRerunsEveryCall106=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall107=== RUN TestScriptTokenCachesUntilRefresh108=== PAUSE TestScriptTokenCachesUntilRefresh109=== RUN TestScriptTokenEmptyToken110=== PAUSE TestScriptTokenEmptyToken111=== RUN TestScriptTokenBadJSON112=== PAUSE TestScriptTokenBadJSON113=== RUN TestScriptTokenScriptFails114=== PAUSE TestScriptTokenScriptFails115=== RUN TestScriptTokenEmptyCommand116=== PAUSE TestScriptTokenEmptyCommand117=== CONT TestDoServerRequestAttachesToken118=== CONT TestScriptTokenScriptFails119=== CONT TestEncodeNixBase32120=== CONT TestDoWithRetry_BodyReplayedViaGetBody121=== CONT TestScriptTokenBadJSON122=== CONT TestScriptTokenEmptyToken123=== CONT TestScriptTokenCachesUntilRefresh124=== CONT TestScriptTokenNoExpiryRerunsEveryCall125=== CONT TestFileTokenEmpty126=== CONT TestFileTokenMissing127=== CONT TestShellSplitErrors128=== CONT TestShellSplit129=== CONT TestFileTokenReadsAndCaches130=== CONT TestStaticToken131=== CONT TestSetClientTLSErrors132=== CONT TestSetClientTLSDoesNotMutateDefaultTransport133=== CONT TestSetClientTLS134=== CONT TestStreamPushStopsWhenCanceled135=== CONT TestStreamPushAnswersEveryLineWhenCanceled136=== CONT TestClientSignaturesByStorePath137=== CONT TestStreamPushReportsSkippedPaths138=== CONT TestStreamPushReportsSignatures139=== CONT TestStreamPushRequestLine140=== CONT TestStreamPushGivesUpOnDeadServer141=== CONT TestStreamPushIsolatesFailures142=== CONT TestStreamPushBatchesUnderLoad143=== RUN TestEncodeNixBase32/test_string_hash144--- PASS: TestScriptTokenScriptFails (0.00s)145=== CONT TestStreamPushReportsEveryPath146=== CONT TestDoWithRetry_FinalResponseBodyReadable147=== CONT TestRetryTimeoutOutlastsMaxRetries148=== CONT TestUploadMultipart_PartsInParallel149=== CONT TestDumpPathWriterError150--- PASS: TestScriptTokenBadJSON (0.00s)151=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=false152=== CONT TestDumpPathMatchesNix1532026/11/12 16:57:00 WARN Rate limiter enabled after throttle name=server-test rate=51542026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:382751552026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:355191562026/11/12 16:57:00 ERROR Upload failed error="bad path" count=3157=== PAUSE TestEncodeNixBase32/test_string_hash1582026/11/12 16:57:00 ERROR Upload failed error=boom count=1159=== CONT TestPathInfoCACompatibility1602026/11/12 16:57:00 ERROR Upload failed error=boom count=1161=== CONT TestRegisterUploadedObject_OutlivesPush1622026/11/12 16:57:00 ERROR Upload failed error="connection refused" count=201632026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:35519164=== CONT TestResolveStorePath1652000/01/01 00:00:00 ERROR Upload failed error="context canceled" count=21662026/11/12 16:57:00 WARN Rate limiter backed off name=server-test rate=5167=== RUN TestPathInfoCACompatibility/null_ca_field1682026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:38275169=== PAUSE TestPathInfoCACompatibility/null_ca_field170=== RUN TestPathInfoCACompatibility/old_string_format_-_text1712026/11/12 16:57:00 ERROR Server seems unavailable, giving up on batch untried=17172=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text173=== CONT TestPartSizeForNAR174=== CONT TestUploadMultipart_FailedProducerIsNotCompleted1752026/11/12 16:57:00 WARN Rate limiter enabled after throttle name=server-test rate=5176--- PASS: TestShellSplitErrors (0.00s)177=== CONT TestUploadMultipart_ReturnsAfterDroppedBody178=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=false179=== RUN TestEncodeNixBase32/empty_input1802026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:45155181=== PAUSE TestEncodeNixBase32/empty_input182=== CONT TestDumpPathSingleFile183=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive184=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive185=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun1862026/11/12 16:57:00 WARN Rate limiter backed off name=server-test rate=5187=== CONT TestRegisterUploadedObject_BoundedAgainstSilentServer188=== CONT TestFilterOversizedClosures189=== RUN TestPartSizeForNAR/zero_stays_at_minimum190=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum191=== RUN TestPartSizeForNAR/small_stays_at_minimum192=== PAUSE TestPartSizeForNAR/small_stays_at_minimum193=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum194=== CONT TestCaseHackSuffix195=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part196=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum197--- PASS: TestFileTokenEmpty (0.00s)198--- PASS: TestShellSplit (0.00s)199--- PASS: TestClientSignaturesByStorePath (0.00s)200--- PASS: TestFileTokenMissing (0.00s)201--- PASS: TestStaticToken (0.00s)202--- PASS: TestStreamPushStopsWhenCanceled (0.00s)203--- PASS: TestStreamPushReportsSkippedPaths (0.00s)204=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess205=== CONT TestParsePathInfoJSONMultiplePaths206=== CONT TestPathInfoHashCompatibility207=== CONT TestRateLimiterFeedback208=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths209=== RUN TestRateLimiterFeedback/429_enables_limiter210=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)211=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)212=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon213=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon214=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI215=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI2162026/11/12 16:57:00 WARN Rate limiter enabled after throttle name=server-test rate=5217=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512218=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512219=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths220=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths221=== CONT TestParsePathInfoJSON2222026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:355192232026/11/12 16:57:00 INFO Garbage collection started224=== PAUSE TestRateLimiterFeedback/429_enables_limiter225=== RUN TestPathInfoCACompatibility/new_structured_format_-_text226=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text227=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method228=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method229=== RUN TestFilterOversizedClosures/no_limit_keeps_everything230=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part231=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts232=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary233=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica234=== CONT TestUploadBuildLog_FileBodyReplayedOnRetry235=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts236=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary237=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=true238=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=true239=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths240=== RUN TestSetClientTLSErrors/missing_cert_file241=== CONT TestEncodeNixBase32WithRealHash242=== CONT TestScriptTokenEmptyCommand243=== CONT TestEncodeNixBase32/test_string_hash244=== CONT TestEncodeNixBase32/empty_input245=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)246=== PAUSE TestSetClientTLSErrors/missing_cert_file247=== RUN TestSetClientTLSErrors/missing_key_file248=== RUN TestRateLimiterFeedback/503_enables_limiter249=== CONT TestRegisterUploadedObjectReusesConnections250=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything251=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped252=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped253--- PASS: TestScriptTokenEmptyToken (0.01s)254=== CONT TestConvertHashToNix322552026/11/12 16:57:00 INFO Garbage collection started256--- PASS: TestFileTokenReadsAndCaches (0.00s)257--- PASS: TestStreamPushReportsEveryPath (0.03s)258--- PASS: TestStreamPushIsolatesFailures (0.03s)259--- PASS: TestStreamPushReportsSignatures (0.03s)260--- PASS: TestStreamPushAnswersEveryLineWhenCanceled (0.03s)261=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon262=== RUN TestConvertHashToNix32/SRI_format_to_Nix322632026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:35519264=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512265=== CONT TestPathInfoCACompatibility/null_ca_field266=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method267=== CONT TestPathInfoCACompatibility/old_string_format_-_text268=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive269=== CONT TestPathInfoCACompatibility/new_structured_format_-_text270=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary271=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32272=== RUN TestConvertHashToNix32/already_Nix32_format273=== PAUSE TestConvertHashToNix32/already_Nix32_format274=== RUN TestConvertHashToNix32/invalid_format275=== PAUSE TestConvertHashToNix32/invalid_format2762026/11/12 16:57:00 WARN Rate limiter enabled after throttle name=s3-test rate=5277=== RUN TestParsePathInfoJSON/Nix_format2782026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:43331279=== PAUSE TestParsePathInfoJSON/Nix_format280=== RUN TestSetClientTLS/rejects_connection_without_client_cert281=== PAUSE TestSetClientTLSErrors/missing_key_file282=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI283=== PAUSE TestRateLimiterFeedback/503_enables_limiter284=== RUN TestFilterOversizedClosures/all_closures_skipped285=== RUN TestPartSizeForNAR/1_TiB286--- PASS: TestDoServerRequestAttachesToken (0.04s)2872026/11/12 16:57:00 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=0288=== CONT TestGetStorePathHash289=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part290=== RUN TestParsePathInfoJSON/Lix_format291=== PAUSE TestParsePathInfoJSON/Lix_format292=== RUN TestParsePathInfoJSON/empty_input293=== PAUSE TestParsePathInfoJSON/empty_input294=== RUN TestParsePathInfoJSON/whitespace_only295=== PAUSE TestParsePathInfoJSON/whitespace_only296=== RUN TestParsePathInfoJSON/invalid_JSON297=== PAUSE TestParsePathInfoJSON/invalid_JSON298=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert299=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA300=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA301=== RUN TestSetClientTLS/preserves_debug_logging_transport302=== PAUSE TestSetClientTLS/preserves_debug_logging_transport303=== RUN TestSetClientTLSErrors/missing_ca_file304=== PAUSE TestSetClientTLSErrors/missing_ca_file305=== RUN TestSetClientTLSErrors/invalid_ca_file306=== PAUSE TestSetClientTLSErrors/invalid_ca_file307=== PAUSE TestPartSizeForNAR/1_TiB308=== RUN TestPartSizeForNAR/5_TiB_S3_max_object3092026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:35519310=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object311=== RUN TestPartSizeForNAR/capped_at_5_GiB312=== PAUSE TestPartSizeForNAR/capped_at_5_GiB313=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths314=== CONT TestSetClientTLS/rejects_connection_without_client_cert315=== RUN TestGetStorePathHash/valid_store_path316=== PAUSE TestGetStorePathHash/valid_store_path317=== RUN TestGetStorePathHash/basename_without_hyphen_should_error318=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error319=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error320=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error321=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error322=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error323=== CONT TestConvertHashToNix32/SRI_format_to_Nix32324=== CONT TestPartSizeForNAR/zero_stays_at_minimum325=== CONT TestSetClientTLSErrors/invalid_ca_file326=== CONT TestParsePathInfoJSON/invalid_JSON327=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths328=== CONT TestParsePathInfoJSON/empty_input329=== CONT TestParsePathInfoJSON/Lix_format330=== CONT TestSetClientTLSErrors/missing_key_file331=== CONT TestPartSizeForNAR/5_TiB_S3_max_object3322026/11/12 16:57:00 WARN Garbage collection finished on another replica. Its statistics are not available here333=== CONT TestPartSizeForNAR/1_TiB334=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts335=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum336=== CONT TestPartSizeForNAR/small_stays_at_minimum337=== CONT TestGetStorePathHash/valid_store_path338=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error339=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error340=== CONT TestGetStorePathHash/basename_without_hyphen_should_error341=== CONT TestSetClientTLS/preserves_debug_logging_transport342=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA343=== CONT TestConvertHashToNix32/already_Nix32_format344=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=true345--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.03s)346--- PASS: TestStreamPushGivesUpOnDeadServer (0.03s)3472026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:35519348--- PASS: TestDoWithRetry_FinalResponseBodyReadable (0.03s)349--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.04s)350--- PASS: TestRegisterUploadedObject_OutlivesPush (0.00s)351--- PASS: TestResolveStorePath (0.00s)352--- PASS: TestUploadMultipart_ReturnsAfterDroppedBody (0.00s)353--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.03s)354--- PASS: TestEncodeNixBase32WithRealHash (0.00s)355--- PASS: TestScriptTokenEmptyCommand (0.00s)356=== CONT TestPartSizeForNAR/capped_at_5_GiB357=== CONT TestSetClientTLSErrors/missing_ca_file358=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter359=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter360=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter361=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter362=== CONT TestRateLimiterFeedback/429_enables_limiter363--- PASS: TestEncodeNixBase32 (0.04s)364 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)365 --- PASS: TestEncodeNixBase32/empty_input (0.00s)366=== PAUSE TestFilterOversizedClosures/all_closures_skipped367=== CONT TestFilterOversizedClosures/no_limit_keeps_everything368=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter369=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter370=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=false371=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped3722026/11/12 16:57:00 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000373=== CONT TestFilterOversizedClosures/all_closures_skipped3742026/11/12 16:57:00 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50375=== CONT TestParsePathInfoJSON/Nix_format376=== CONT TestSetClientTLSErrors/missing_cert_file377=== CONT TestConvertHashToNix32/invalid_format3782026/11/12 16:57:00 WARN Rate limiter enabled after throttle name=server-test rate=5379=== CONT TestRateLimiterFeedback/503_enables_limiter3802026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:393533812026/11/12 16:57:00 WARN Rate limiter backed off name=server-test rate=5382--- PASS: TestFilterOversizedClosures (0.01s)383 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)384 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)385 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)386=== CONT TestParsePathInfoJSON/whitespace_only387--- PASS: TestStreamPushBatchesUnderLoad (0.03s)388 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=true (0.00s)389 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=false (0.00s)390--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (0.00s)391--- PASS: TestPathInfoCACompatibility (0.00s)392 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)393 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)394 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)395 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)396 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)397--- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (0.00s)398--- PASS: TestScriptTokenCachesUntilRefresh (0.04s)399--- PASS: TestParsePathInfoJSONMultiplePaths (0.00s)400 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)401 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)402--- PASS: TestUploadMultipart_FailedProducerIsNotCompleted (0.00s)403 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary (0.00s)404 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part (0.00s)405--- PASS: TestParsePathInfoJSON (0.00s)406 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)407 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)408 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)409 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)410 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)4112026/11/12 16:57:00 WARN Rate limiter enabled after throttle name=server-test rate=5412--- PASS: TestPartSizeForNAR (0.00s)413 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)414 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)415 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)416 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)417 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)418 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)419 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)4202026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:43397421--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (0.00s)422--- PASS: TestGetStorePathHash (0.00s)423 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)424 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)425 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)426 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)427--- PASS: TestConvertHashToNix32 (0.00s)428 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)429 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)430 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)431--- PASS: TestPathInfoHashCompatibility (0.00s)432 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)433 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)434 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)435 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)4362026/11/12 16:57:00 WARN Rate limiter backed off name=server-test rate=5437--- PASS: TestSetClientTLSErrors (0.03s)438 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)439 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)440 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)441 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)442--- PASS: TestRateLimiterFeedback (0.00s)443 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s)444 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.00s)445 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.00s)446 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s)4472026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:355194482026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:355194492026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:355194502026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:355194512026/11/12 16:57:00 http: TLS handshake error from 127.0.0.1:45274: remote error: tls: bad certificate4522026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:35519453--- PASS: TestSetClientTLS (0.04s)454 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.00s)455 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.00s)456 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.02s)4572026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:355194582026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:355194592026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:35519460--- PASS: TestRegisterUploadedObjectReusesConnections (0.04s)461--- PASS: TestDumpPathSingleFile (0.04s)4622026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:35519463--- PASS: TestCaseHackSuffix (0.04s)464--- PASS: TestStreamPushRequestLine (0.07s)4652026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:355194662026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:355194672026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:355194682026/11/12 16:57:00 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:35519469--- PASS: TestRetryTimeoutOutlastsMaxRetries (0.08s)470--- PASS: TestDumpPathWriterError (0.08s)471--- PASS: TestDumpPathMatchesNix (0.12s)4722026/11/12 16:57:00 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:41659/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:41659/api/uploads/complete4732026/11/12 16:57:00 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4742026/11/12 16:57:00 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:41659/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:41659/api/uploads/complete4752026/11/12 16:57:00 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4762026/11/12 16:57:00 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:41659/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:41659/api/uploads/complete4772026/11/12 16:57:00 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"478--- PASS: TestRegisterUploadedObject_BoundedAgainstSilentServer (0.10s)479--- PASS: TestUploadMultipart_PartsInParallel (0.66s)480--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)481PASS482Running server tests...483The files belonging to this database system will be owned by user "nixbld".484This user must also own the server process.485486The database cluster will be initialized with locale "C".487The default database encoding has accordingly been set to "SQL_ASCII".488The default text search configuration will be set to "english".489490Data page checksums are enabled.491492creating directory /build/postgres1290530788/data ... ok493creating subdirectories ... ok494selecting dynamic shared memory implementation ... posix495selecting default "max_connections" ... 100496selecting default "shared_buffers" ... 128MB497selecting default time zone ... UTC498creating configuration files ... ok499running bootstrap script ... ok500performing post-bootstrap initialization ... ok501syncing data to disk ... ok502503initdb: warning: enabling "trust" authentication for local connections504initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.505506Success. You can now start the database server using:507508 pg_ctl -D /build/postgres1290530788/data -l logfile start509510/build/postgres1290530788:5432 - no response5112026-11-12 16:57:01.905 UTC [128] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit5122026-11-12 16:57:01.905 UTC [128] LOG: listening on Unix socket "/build/postgres1290530788/.s.PGSQL.5432"5132026-11-12 16:57:01.910 UTC [135] LOG: database system was shut down at 2026-11-12 16:57:01 UTC5142026-11-12 16:57:01.913 UTC [128] LOG: database system is ready to accept connections515/build/postgres1290530788:5432 - accepting connections516{"timestamp":"2026-11-12T16:57:02.214398398Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"56628dd4-d0d2-4463-84cb-f276fba5cdd6","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":3,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(204)"}517=== RUN TestAbortSucceeded518=== PAUSE TestAbortSucceeded519=== RUN TestService_AuthMiddleware520=== PAUSE TestService_AuthMiddleware521=== RUN TestService_AuthMiddleware_MTLSProxyHeader522=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader523=== RUN TestService_AuthMiddleware_MTLSBoundSubjects524=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects525=== RUN TestService_ReadAuthMiddleware526=== PAUSE TestService_ReadAuthMiddleware527=== RUN TestService_AuthMiddleware_OIDC528=== PAUSE TestService_AuthMiddleware_OIDC529=== RUN TestService_RequireScope_OIDC530=== PAUSE TestService_RequireScope_OIDC531=== RUN TestService_ReadScope_PublicByDefault532=== PAUSE TestService_ReadScope_PublicByDefault533=== RUN TestCacheConfigHandler534=== PAUSE TestCacheConfigHandler535=== RUN TestCacheStatsHandler536=== PAUSE TestCacheStatsHandler537=== RUN TestClientCADerivations538=== PAUSE TestClientCADerivations539=== RUN TestClientErrorHandling540=== PAUSE TestClientErrorHandling541=== RUN TestClientIntegration542=== PAUSE TestClientIntegration543=== RUN TestClientMultipleUploads544=== PAUSE TestClientMultipleUploads545=== RUN TestClientWithDependencies546=== PAUSE TestClientWithDependencies547=== RUN TestClientSharedPathCommittedMidPush548=== PAUSE TestClientSharedPathCommittedMidPush549=== RUN TestPinProtectsFromGC550=== PAUSE TestPinProtectsFromGC551=== RUN TestClientPushesUseOnePush552=== PAUSE TestClientPushesUseOnePush553=== RUN TestClientFallsBackToClosures554=== PAUSE TestClientFallsBackToClosures555=== RUN TestDeleteClosuresRacingPins556=== PAUSE TestDeleteClosuresRacingPins557=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock558=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock559=== RUN TestResolveDBConnectionString560=== PAUSE TestResolveDBConnectionString561=== RUN TestConnectWaitsForAPeerMigration562=== PAUSE TestConnectWaitsForAPeerMigration563=== RUN TestLeadElectsOneAndHandsOver564=== PAUSE TestLeadElectsOneAndHandsOver565=== RUN TestLeadIncumbentWinsAfterRestart5662026/11/12 16:57:02 INFO lead: acquired remote=192.0.2.1:12345672026/11/12 16:57:03 INFO lead: released remote=192.0.2.1:12345682026/11/12 16:57:03 INFO lead: acquired remote=192.0.2.1:12345692026/11/12 16:57:04 INFO lead: released remote=192.0.2.1:1234570--- PASS: TestLeadIncumbentWinsAfterRestart (1.86s)571=== RUN TestLeadUnannouncedStandbyYieldsToIncumbent572=== PAUSE TestLeadUnannouncedStandbyYieldsToIncumbent573=== RUN TestLeadIncumbentAnnouncesAtOnce574=== PAUSE TestLeadIncumbentAnnouncesAtOnce575=== RUN TestLeadEndsOnShutdown576=== PAUSE TestLeadEndsOnShutdown577=== RUN TestLeadEndsWhenItsConnectionHangs5782026/11/12 16:57:04 INFO lead: acquired remote=192.0.2.1:12345792026-11-12 16:57:05.693 UTC [385] FATAL: terminating connection due to administrator command5802026/11/12 16:57:05 INFO lead: acquired remote=192.0.2.1:12345812026/11/12 16:57:06 WARN lead: lock connection lost error="timeout: context deadline exceeded"5822026/11/12 16:57:06 INFO lead: released remote=192.0.2.1:12345832026/11/12 16:57:07 INFO lead: released remote=192.0.2.1:1234584--- PASS: TestLeadEndsWhenItsConnectionHangs (3.11s)585=== RUN TestGCAdvisoryLockBlocksConcurrentRun586=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun587=== RUN TestGCBugBareHashReferences588=== PAUSE TestGCBugBareHashReferences589=== RUN TestGCMetrics590=== PAUSE TestGCMetrics591=== RUN TestPushDedupSurvivesConcurrentGC592=== PAUSE TestPushDedupSurvivesConcurrentGC593=== RUN TestDeduplicatedObjectsRecordedAsPending594=== PAUSE TestDeduplicatedObjectsRecordedAsPending595=== RUN TestGCSweepSkipsPendingObjects596=== PAUSE TestGCSweepSkipsPendingObjects597=== RUN TestTombstonedObjectOfferedWithoutWaiting598=== PAUSE TestTombstonedObjectOfferedWithoutWaiting599=== RUN TestGCSweepDeliversEachKeyOnce600=== PAUSE TestGCSweepDeliversEachKeyOnce601=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection602=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection603=== RUN TestForceGCDuringPushOffersSweptObject604=== PAUSE TestForceGCDuringPushOffersSweptObject605=== RUN TestFailedS3DeleteKeepsTombstone606=== PAUSE TestFailedS3DeleteKeepsTombstone607=== RUN TestSweepSkipsRowsLockedByPush608=== PAUSE TestSweepSkipsRowsLockedByPush609=== RUN TestRegisterCompletedObjectNeedsPendingRow610=== PAUSE TestRegisterCompletedObjectNeedsPendingRow611=== RUN TestGetExistingObjectsLocksRows612=== PAUSE TestGetExistingObjectsLocksRows613=== RUN TestSweepRechecksPendingAfterLocking614=== PAUSE TestSweepRechecksPendingAfterLocking615=== RUN TestPushWaitsForSweepHoldingRow616=== PAUSE TestPushWaitsForSweepHoldingRow617=== RUN TestCommitRacingPendingCleanupKeepsObjects618=== PAUSE TestCommitRacingPendingCleanupKeepsObjects619=== RUN TestGCEndsOnShutdown620=== PAUSE TestGCEndsOnShutdown621=== RUN TestGCTaskStore_StartNew622=== PAUSE TestGCTaskStore_StartNew623=== RUN TestGCTaskStore_DeduplicateSameParams624=== PAUSE TestGCTaskStore_DeduplicateSameParams625=== RUN TestGCTaskStore_ConflictDifferentParams626=== PAUSE TestGCTaskStore_ConflictDifferentParams627=== RUN TestGCTaskStore_GetEmpty628=== PAUSE TestGCTaskStore_GetEmpty629=== RUN TestGCTaskStore_GetReturnsLatest630=== PAUSE TestGCTaskStore_GetReturnsLatest631=== RUN TestGCTaskStore_CompletedAllowsNewTask632=== PAUSE TestGCTaskStore_CompletedAllowsNewTask633=== RUN TestGCTaskStore_PhaseUpdates634=== PAUSE TestGCTaskStore_PhaseUpdates635=== RUN TestGCTaskStore_Fail636=== PAUSE TestGCTaskStore_Fail637=== RUN TestGracefulShutdownDrainsInflight638=== PAUSE TestGracefulShutdownDrainsInflight639=== RUN TestService_healthCheckHandler640=== PAUSE TestService_healthCheckHandler641=== RUN TestService_readinessHandler642=== PAUSE TestService_readinessHandler643=== RUN TestGenerateLandingPage644=== PAUSE TestGenerateLandingPage645=== RUN TestEnsureAbortUploadsRule646=== PAUSE TestEnsureAbortUploadsRule647=== RUN TestCacheConfigHandlerMaxNarSize648=== PAUSE TestCacheConfigHandlerMaxNarSize649=== RUN TestCreatePendingClosureRejectsOversizedNAR650=== PAUSE TestCreatePendingClosureRejectsOversizedNAR651=== RUN TestNARDeduplicationMetadataUploadBug652=== PAUSE TestNARDeduplicationMetadataUploadBug653=== RUN TestMetricsInventory654=== PAUSE TestMetricsInventory655=== RUN TestMetricsCommit656=== PAUSE TestMetricsCommit657=== RUN TestService_NativeMTLS658=== PAUSE TestService_NativeMTLS659=== RUN TestServerTLSConfig660=== PAUSE TestServerTLSConfig661=== RUN TestMultipartCleanup662=== PAUSE TestMultipartCleanup663=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded664=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded665=== RUN TestPendingCleanupUsesOneCutoff666=== PAUSE TestPendingCleanupUsesOneCutoff667=== RUN TestObjectStatsTrigger668=== PAUSE TestObjectStatsTrigger669=== RUN TestObjectStatsUpdatedOncePerCommit670=== PAUSE TestObjectStatsUpdatedOncePerCommit671=== RUN TestOrphanedObjectsGC672=== PAUSE TestOrphanedObjectsGC673=== RUN TestOrphanedObjectsGCStressTest674=== PAUSE TestOrphanedObjectsGCStressTest675=== RUN TestCreatePin_ReservedPins676=== PAUSE TestCreatePin_ReservedPins677=== RUN TestPresentReportsOnlyClosureRoots678=== PAUSE TestPresentReportsOnlyClosureRoots679=== RUN TestPresentRefreshesOnlyStaleClosures680=== PAUSE TestPresentRefreshesOnlyStaleClosures681=== RUN TestPresentNotReportedWhileGCDeletesClosure682=== PAUSE TestPresentNotReportedWhileGCDeletesClosure683=== RUN TestParseSingleRange684=== PAUSE TestParseSingleRange685=== RUN TestProxyHeadersOnlyTrustedOnSocket6862026/11/12 16:57:07 INFO Starting HTTP server address=127.0.0.1:372436872026/11/12 16:57:07 INFO Starting HTTP server address=proxy.sock6882026/11/12 16:57:07 WARN mTLS auth: subject not in bound subjects subject="CN=someone"6892026/11/12 16:57:07 INFO Shutdown signal received, draining in-flight requests timeout=10s690--- PASS: TestProxyHeadersOnlyTrustedOnSocket (0.10s)691=== RUN TestIsValidCachePath692=== PAUSE TestIsValidCachePath693=== RUN TestReadProxyNarinfo694=== PAUSE TestReadProxyNarinfo695=== RUN TestReadProxyNarinfoAlreadyDecompressed696=== PAUSE TestReadProxyNarinfoAlreadyDecompressed697=== RUN TestReadProxyContentNegotiation698=== PAUSE TestReadProxyContentNegotiation699=== RUN TestReadProxyNarStreaming700=== PAUSE TestReadProxyNarStreaming701=== RUN TestReadProxy404702=== PAUSE TestReadProxy404703=== RUN TestReadProxyInvalidPath704=== PAUSE TestReadProxyInvalidPath705=== RUN TestReadProxyHead706=== PAUSE TestReadProxyHead707=== RUN TestReadProxyOutlastsServerWriteTimeout708=== PAUSE TestReadProxyOutlastsServerWriteTimeout709=== RUN TestReadProxyConditionalGet710=== PAUSE TestReadProxyConditionalGet711=== RUN TestReadProxyRootRedirectsToIndexHTML712=== PAUSE TestReadProxyRootRedirectsToIndexHTML713=== RUN TestReadProxyDisabled714=== PAUSE TestReadProxyDisabled715=== RUN TestReadRedirectNar716=== PAUSE TestReadRedirectNar717=== RUN TestReadRedirectKeepsNarinfoProxied718=== PAUSE TestReadRedirectKeepsNarinfoProxied719=== RUN TestReadProxyRangeRequest720=== PAUSE TestReadProxyRangeRequest721=== RUN TestReadRedirectUsesPublicS3URL722=== PAUSE TestReadRedirectUsesPublicS3URL723=== RUN TestPush_OverlappingRootsStoreOneRowPerKey724=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey725=== RUN TestPush_CompleteCommitsEveryRoot726=== PAUSE TestPush_CompleteCommitsEveryRoot727=== RUN TestPush_RepeatedRootCommitsOnce728=== PAUSE TestPush_RepeatedRootCommitsOnce729=== RUN TestPush_CompleteWaitsForCleanupOfThePush730=== PAUSE TestPush_CompleteWaitsForCleanupOfThePush731=== RUN TestPush_RootsAreLockedInByteOrder732=== PAUSE TestPush_RootsAreLockedInByteOrder733=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit734=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit735=== RUN TestPush_RejectsBadRequests736=== PAUSE TestPush_RejectsBadRequests737=== RUN TestPush_SignsNarinfosOfItsPendingObjects738=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects739=== RUN TestCommitCanBeRepeatedUntilCleanup740=== PAUSE TestCommitCanBeRepeatedUntilCleanup741=== RUN TestSharedNarBothUploadsComplete742=== PAUSE TestSharedNarBothUploadsComplete743=== RUN TestCompleteMultipartUpload_ErrorButObjectExists744=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists745=== RUN TestCompletedNarNotReofferedAcrossClosures746=== PAUSE TestCompletedNarNotReofferedAcrossClosures747=== RUN TestPresignedUploadRegisteredBeforeCommit748=== PAUSE TestPresignedUploadRegisteredBeforeCommit749=== RUN TestService_Rustfstest750=== PAUSE TestService_Rustfstest751=== RUN TestParseSize752=== PAUSE TestParseSize753=== RUN TestSkippedUploadsHandler754=== PAUSE TestSkippedUploadsHandler755=== RUN TestSystemdListenerNotActivated756--- PASS: TestSystemdListenerNotActivated (0.00s)757=== RUN TestWatchdogBeatsWhenHealthy758--- PASS: TestWatchdogBeatsWhenHealthy (0.02s)759=== RUN TestWatchdogSkipsWhenUnhealthy7602026/11/12 16:57:07 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7612026/11/12 16:57:07 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7622026/11/12 16:57:07 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7632026/11/12 16:57:07 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7642026/11/12 16:57:07 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7652026/11/12 16:57:07 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7662026/11/12 16:57:07 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7672026/11/12 16:57:07 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7682026/11/12 16:57:07 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7692026/11/12 16:57:07 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"770--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)771=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle772=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle773=== RUN TestProxyWriteTimeout774=== PAUSE TestProxyWriteTimeout775=== RUN TestPendingClosureWriteTimeout776=== PAUSE TestPendingClosureWriteTimeout777=== RUN TestIsValidUploadKey778=== PAUSE TestIsValidUploadKey779=== RUN TestUploadHandlersRejectInvalidKeys780=== PAUSE TestUploadHandlersRejectInvalidKeys781=== RUN TestUploadHandlersRejectOversizedBody782=== PAUSE TestUploadHandlersRejectOversizedBody783=== RUN TestService_cleanupPendingClosuresHandler784=== PAUSE TestService_cleanupPendingClosuresHandler785=== RUN TestService_createPendingClosureHandler786=== PAUSE TestService_createPendingClosureHandler787=== RUN TestService_verifyS3Integrity788=== PAUSE TestService_verifyS3Integrity789=== RUN TestService_verifyS3ReuploadsStaleNarinfo790=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo791=== RUN TestCompleteMultipartUnregistered792=== PAUSE TestCompleteMultipartUnregistered793=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT794=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT795=== CONT TestPendingClosureWriteTimeout796=== CONT TestCompleteMultipartUpload_ErrorButObjectExists797=== RUN TestPendingClosureWriteTimeout/empty798=== CONT TestPush_SignsNarinfosOfItsPendingObjects799=== CONT TestGCTaskStore_Fail800--- PASS: TestGCTaskStore_Fail (0.00s)801=== CONT TestProxyWriteTimeout802=== RUN TestProxyWriteTimeout/narinfo803=== PAUSE TestProxyWriteTimeout/narinfo804=== RUN TestProxyWriteTimeout/1_GiB_nar805=== PAUSE TestProxyWriteTimeout/1_GiB_nar806=== RUN TestProxyWriteTimeout/10_GiB_nar807=== PAUSE TestProxyWriteTimeout/10_GiB_nar808=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle809=== CONT TestSkippedUploadsHandler810=== CONT TestParseSize811--- PASS: TestParseSize (0.00s)812=== CONT TestGCTaskStore_ConflictDifferentParams813--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)814=== CONT TestGCTaskStore_DeduplicateSameParams815--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)816=== CONT TestUploadHandlersRejectOversizedBody817=== CONT TestService_Rustfstest818=== CONT TestPresignedUploadRegisteredBeforeCommit819=== CONT TestCompletedNarNotReofferedAcrossClosures820=== CONT TestSharedNarBothUploadsComplete821=== CONT TestCommitCanBeRepeatedUntilCleanup822=== RUN TestCommitCanBeRepeatedUntilCleanup/push823=== PAUSE TestCommitCanBeRepeatedUntilCleanup/push824=== CONT TestService_createPendingClosureHandler825=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT826=== CONT TestCompleteMultipartUnregistered827=== CONT TestService_verifyS3ReuploadsStaleNarinfo828=== CONT TestService_verifyS3Integrity829=== CONT TestLeadEndsOnShutdown830=== CONT TestGCTaskStore_PhaseUpdates831=== CONT TestGCTaskStore_CompletedAllowsNewTask832=== PAUSE TestPendingClosureWriteTimeout/empty833=== CONT TestGCTaskStore_GetReturnsLatest834=== CONT TestService_cleanupPendingClosuresHandler835=== CONT TestAbortSucceeded836=== CONT TestGCTaskStore_GetEmpty837=== RUN TestProxyWriteTimeout/unknown_size838=== PAUSE TestProxyWriteTimeout/unknown_size8392026/11/12 16:57:07 INFO Client skipped oversized paths paths=3 nar_bytes=5000000000840=== RUN TestCommitCanBeRepeatedUntilCleanup/pending_closure841=== PAUSE TestCommitCanBeRepeatedUntilCleanup/pending_closure842=== CONT TestGCEndsOnShutdown843=== CONT TestGCTaskStore_StartNew844=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection845--- PASS: TestGCTaskStore_GetEmpty (0.00s)846--- PASS: TestGCTaskStore_StartNew (0.00s)847=== CONT TestGCSweepDeliversEachKeyOnce848=== RUN TestPendingClosureWriteTimeout/400_objects849=== RUN TestGCEndsOnShutdown/before_the_run850--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)851=== CONT TestIsValidUploadKey852=== RUN TestIsValidUploadKey/narinfo853=== PAUSE TestIsValidUploadKey/narinfo854=== RUN TestIsValidUploadKey/nar_zst855=== PAUSE TestIsValidUploadKey/nar_zst856=== RUN TestIsValidUploadKey/nar_xz857=== PAUSE TestIsValidUploadKey/nar_xz858=== RUN TestIsValidUploadKey/nar_plain859=== PAUSE TestIsValidUploadKey/nar_plain860--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)861--- PASS: TestAbortSucceeded (0.00s)862=== CONT TestUploadHandlersRejectInvalidKeys863=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info864=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info865=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal866=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal867=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key868=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key869=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key870=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key871--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)872=== CONT TestCommitRacingPendingCleanupKeepsObjects873=== PAUSE TestPendingClosureWriteTimeout/400_objects874=== RUN TestPendingClosureWriteTimeout/670k_objects875=== PAUSE TestPendingClosureWriteTimeout/670k_objects876=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline877=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline878=== PAUSE TestGCEndsOnShutdown/before_the_run879=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete880=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete881=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark882=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark883=== CONT TestSweepRechecksPendingAfterLocking884=== CONT TestParseSingleRange885=== RUN TestParseSingleRange/none886=== PAUSE TestParseSingleRange/none887=== RUN TestParseSingleRange/unknown_unit888=== PAUSE TestParseSingleRange/unknown_unit889=== RUN TestParseSingleRange/multi-range_ignored890=== PAUSE TestParseSingleRange/multi-range_ignored891=== RUN TestParseSingleRange/malformed_no_dash892=== PAUSE TestParseSingleRange/malformed_no_dash893=== RUN TestParseSingleRange/malformed_both_empty894=== PAUSE TestParseSingleRange/malformed_both_empty895=== CONT TestPushWaitsForSweepHoldingRow896=== RUN TestIsValidUploadKey/listing897=== PAUSE TestIsValidUploadKey/listing898=== RUN TestIsValidUploadKey/build_log899=== PAUSE TestIsValidUploadKey/build_log900=== RUN TestIsValidUploadKey/build_log_home-manager_file901=== PAUSE TestIsValidUploadKey/build_log_home-manager_file902=== RUN TestIsValidUploadKey/build_log_plus_in_name903=== PAUSE TestIsValidUploadKey/build_log_plus_in_name904=== RUN TestIsValidUploadKey/build_log_question_mark905=== PAUSE TestIsValidUploadKey/build_log_question_mark906=== RUN TestIsValidUploadKey/build_log_equals907=== PAUSE TestIsValidUploadKey/build_log_equals908=== RUN TestIsValidUploadKey/realisation909=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed910=== CONT TestTombstonedObjectOfferedWithoutWaiting911=== RUN TestParseSingleRange/malformed_end_before_start912=== PAUSE TestParseSingleRange/malformed_end_before_start913=== PAUSE TestIsValidUploadKey/realisation914=== RUN TestIsValidUploadKey/realisation_plus_in_output915=== PAUSE TestIsValidUploadKey/realisation_plus_in_output916=== RUN TestIsValidUploadKey/nix-cache-info917=== PAUSE TestIsValidUploadKey/nix-cache-info918=== RUN TestIsValidUploadKey/index.html919=== PAUSE TestIsValidUploadKey/index.html920=== RUN TestIsValidUploadKey/narinfo_key,_nar_type921=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type922=== RUN TestIsValidUploadKey/nar_key,_narinfo_type923=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type924=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed925=== RUN TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed926=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed927=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost928=== RUN TestParseSingleRange/closed929=== PAUSE TestParseSingleRange/closed930=== RUN TestParseSingleRange/open-ended931=== PAUSE TestParseSingleRange/open-ended932=== RUN TestParseSingleRange/end_clamped_to_size933=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost934=== PAUSE TestParseSingleRange/end_clamped_to_size935=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed936=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed937=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed938=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed939=== RUN TestIsValidUploadKey/listing_key,_narinfo_type940=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type941=== CONT TestPush_RejectsBadRequests942--- PASS: TestSkippedUploadsHandler (0.08s)943=== CONT TestGetExistingObjectsLocksRows944=== RUN TestParseSingleRange/suffix945=== RUN TestIsValidUploadKey/traversal946=== PAUSE TestIsValidUploadKey/traversal947=== PAUSE TestParseSingleRange/suffix948=== RUN TestParseSingleRange/suffix_exceeds_size949=== PAUSE TestParseSingleRange/suffix_exceeds_size950=== RUN TestIsValidUploadKey/traversal_nar951=== PAUSE TestIsValidUploadKey/traversal_nar952=== RUN TestIsValidUploadKey/absolute953=== RUN TestParseSingleRange/single_byte954=== PAUSE TestParseSingleRange/single_byte955=== RUN TestParseSingleRange/start_past_EOF956=== PAUSE TestParseSingleRange/start_past_EOF957=== RUN TestParseSingleRange/start_far_past_EOF958=== PAUSE TestIsValidUploadKey/absolute959=== PAUSE TestParseSingleRange/start_far_past_EOF960=== CONT TestRegisterCompletedObjectNeedsPendingRow961=== RUN TestIsValidUploadKey/empty_key962=== PAUSE TestIsValidUploadKey/empty_key963=== RUN TestIsValidUploadKey/unknown_type964=== PAUSE TestIsValidUploadKey/unknown_type965=== CONT TestSweepSkipsRowsLockedByPush966=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts967=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts968=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure969=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure970=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart971=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart972=== CONT TestFailedS3DeleteKeepsTombstone9732026/11/12 16:57:07 INFO Received uploads request method=POST path=/api/pending_closures9742026/11/12 16:57:07 INFO Received uploads request method=POST path=/api/pending_closures9752026/11/12 16:57:08 INFO Received uploads request method=POST path=/api/pending_closures9762026/11/12 16:57:08 INFO Received uploads request method=POST path=/api/pending_closures9772026/11/12 16:57:08 INFO Received push request method=POST path=/api/pushes9782026/11/12 16:57:08 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9792026/11/12 16:57:08 INFO Signed narinfos id=1 count=19802026/11/12 16:57:08 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9812026/11/12 16:57:08 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9822026/11/12 16:57:08 ERROR Failed to sign narinfos signer="index 1" error="signing key cannot be nil"9832026/11/12 16:57:08 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9842026/11/12 16:57:08 ERROR Failed to sign narinfos signer=niks3-test-1 error="signer failed"9852026/11/12 16:57:08 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9862026/11/12 16:57:08 ERROR Signer returned unexpected signature count signer=niks3-test-1 got=0 want=19872026/11/12 16:57:08 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9882026/11/12 16:57:08 ERROR Signer returned unexpected object key signer=niks3-test-1 object_key=unexpected.narinfo989--- PASS: TestPush_SignsNarinfosOfItsPendingObjects (0.55s)990=== CONT TestForceGCDuringPushOffersSweptObject991=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows992=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows993=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check994=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check995=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit9962026/11/12 16:57:08 INFO Received uploads request method=POST path=/api/pending_closures9972026/11/12 16:57:08 INFO Received complete multipart upload request method=POST path=/api/multipart/complete998--- PASS: TestService_Rustfstest (0.75s)999=== CONT TestServerTLSConfig1000=== RUN TestServerTLSConfig/no_client_CA1001=== PAUSE TestServerTLSConfig/no_client_CA1002=== RUN TestServerTLSConfig/missing_CA_file1003=== PAUSE TestServerTLSConfig/missing_CA_file1004=== RUN TestServerTLSConfig/not_a_PEM_file1005=== PAUSE TestServerTLSConfig/not_a_PEM_file1006=== CONT TestPush_RootsAreLockedInByteOrder10072026/11/12 16:57:08 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10082026/11/12 16:57:08 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=NzY3YTMxNTYtMmRkOS00OTE4LWFhOTctZGU1NmYxZWQwYjI4LjYxYjIwMGVlLTYyMGYtNDY0OS04ZDIzLTA1MmQzZDJmMjliOXgxNzk0NTAyNjI4MjY3OTYyODU210092026/11/12 16:57:08 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=NzY3YTMxNTYtMmRkOS00OTE4LWFhOTctZGU1NmYxZWQwYjI4LjYxYjIwMGVlLTYyMGYtNDY0OS04ZDIzLTA1MmQzZDJmMjliOXgxNzk0NTAyNjI4MjY3OTYyODU2 parts=11010--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (0.83s)1011=== CONT TestPresentNotReportedWhileGCDeletesClosure1012=== RUN TestPresentNotReportedWhileGCDeletesClosure/stale1013=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/stale1014=== RUN TestPresentNotReportedWhileGCDeletesClosure/fresh1015=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/fresh1016=== CONT TestGCSweepSkipsPendingObjects10172026/11/12 16:57:08 INFO Received uploads request method=POST path=/api/pending_closures10182026/11/12 16:57:08 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst10192026/11/12 16:57:08 INFO Received uploads request method=POST path=/api/pending_closures1020--- PASS: TestPresignedUploadRegisteredBeforeCommit (0.94s)1021=== CONT TestPush_CompleteWaitsForCleanupOfThePush10222026/11/12 16:57:08 INFO Received uploads request method=POST path=/api/pending_closures10232026/11/12 16:57:08 INFO Received uploads request method=POST path=/api/pending_closures10242026/11/12 16:57:08 INFO Received uploads request method=POST path=/api/pending_closures10252026/11/12 16:57:08 INFO Received uploads request method=POST path=/api/pending_closures1026--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (1.16s)1027=== CONT TestPresentRefreshesOnlyStaleClosures10282026/11/12 16:57:09 INFO Aborted multipart uploads count=0 kept=010292026/11/12 16:57:09 WARN Force mode enabled - objects will be deleted immediately without grace period10302026/11/12 16:57:09 INFO lead: acquired remote=192.0.2.1:123410312026/11/12 16:57:09 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10322026/11/12 16:57:09 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=010332026/11/12 16:57:09 INFO Vacuumed table table=pending_closures10342026/11/12 16:57:09 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst1035--- PASS: TestCompleteMultipartUnregistered (1.47s)1036=== CONT TestDeduplicatedObjectsRecordedAsPending10372026/11/12 16:57:09 INFO Vacuumed table table=commit_receipts10382026/11/12 16:57:09 INFO Vacuumed table table=pending_objects10392026/11/12 16:57:09 INFO Vacuumed table table=multipart_uploads10402026/11/12 16:57:09 INFO Vacuumed table table=closures10412026/11/12 16:57:09 INFO Vacuumed table table=objects1042--- PASS: TestGCSweepDeliversEachKeyOnce (1.48s)1043=== CONT TestPush_RepeatedRootCommitsOnce10442026/11/12 16:57:09 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10452026/11/12 16:57:09 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=NzY3YTMxNTYtMmRkOS00OTE4LWFhOTctZGU1NmYxZWQwYjI4LmRkZGRlMjI5LTI3MDQtNDRmYi1hMTc2LTc4NmFjMWEzODYwMHgxNzk0NTAyNjI3OTI0MDM5MDk2 parts=1210462026/11/12 16:57:09 INFO Received uploads request method=POST path=/api/pending_closures1047--- PASS: TestCompletedNarNotReofferedAcrossClosures (1.63s)1048=== CONT TestPushDedupSurvivesConcurrentGC10492026/11/12 16:57:09 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10502026/11/12 16:57:09 INFO Received cleanup request method=DELETE path=/api/pending_closures10512026/11/12 16:57:09 INFO Aborted multipart uploads count=0 kept=010522026/11/12 16:57:09 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=NzY3YTMxNTYtMmRkOS00OTE4LWFhOTctZGU1NmYxZWQwYjI4LmRlMzY3ZjE0LWNhZWItNDYyZC04NDcyLTllYzVhYmM2ODQ0Y3gxNzk0NTAyNjI3OTg3Mjg3NTg1 parts=1210532026/11/12 16:57:09 INFO Received uploads request method=POST path=/api/pending_closures10542026/11/12 16:57:09 INFO Aborted multipart uploads count=0 kept=010552026/11/12 16:57:09 WARN Force mode enabled - objects will be deleted immediately without grace period10562026/11/12 16:57:09 INFO Received cleanup request method=DELETE path=/api/pending_closures10572026/11/12 16:57:09 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010582026/11/12 16:57:09 INFO Vacuumed table table=pending_closures10592026/11/12 16:57:09 INFO Vacuumed table table=commit_receipts10602026/11/12 16:57:09 INFO Vacuumed table table=pending_objects10612026/11/12 16:57:09 INFO Vacuumed table table=multipart_uploads10622026/11/12 16:57:09 INFO Vacuumed table table=closures10632026/11/12 16:57:09 INFO Vacuumed table table=objects10642026/11/12 16:57:09 INFO Aborted multipart uploads count=1 kept=010652026/11/12 16:57:09 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10662026-11-12 16:57:09.408 UTC [471] ERROR: Closure does not exist: id=110672026-11-12 16:57:09.408 UTC [471] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 17 at RAISE10682026-11-12 16:57:09.408 UTC [471] STATEMENT: -- name: CommitPendingClosure :one1069 SELECT commit_pending_closure($1::bigint)::bigint1070 1071--- PASS: TestService_cleanupPendingClosuresHandler (1.72s)1072=== CONT TestPush_CompleteCommitsEveryRoot1073--- PASS: TestCommitRacingPendingCleanupKeepsObjects (1.74s)1074=== CONT TestPresentReportsOnlyClosureRoots10752026/11/12 16:57:09 INFO Received uploads request method=POST path=/api/pending_closures10762026/11/12 16:57:09 INFO Received uploads request method=POST path=/api/pending_closures10772026/11/12 16:57:09 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10782026/11/12 16:57:09 INFO Aborted multipart uploads count=0 kept=010792026/11/12 16:57:09 INFO Received uploads request method=POST path=/api/pending_closures10802026/11/12 16:57:09 WARN Force mode enabled - objects will be deleted immediately without grace period1081--- PASS: TestTombstonedObjectOfferedWithoutWaiting (1.88s)1082=== CONT TestPush_OverlappingRootsStoreOneRowPerKey10832026/11/12 16:57:09 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=NzY3YTMxNTYtMmRkOS00OTE4LWFhOTctZGU1NmYxZWQwYjI4LmI5ZDZhNDE1LTJmMzYtNDY1YS1iMmE0LTk0YzRjYTI3NDU4ZngxNzk0NTAyNjI4NjEyMjE4ODk4 parts=1010842026/11/12 16:57:09 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10852026/11/12 16:57:09 INFO Completed upload id=1 objects=2 duration=2.911227ms10862026/11/12 16:57:09 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000010872026/11/12 16:57:09 INFO Received uploads request method=POST path=/api/pending_closures10882026/11/12 16:57:09 INFO Starting cleanup of old closures method=DELETE path=/api/closures10892026/11/12 16:57:09 INFO Aborted multipart uploads count=0 kept=010902026/11/12 16:57:09 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010912026/11/12 16:57:09 INFO Vacuumed table table=pending_closures1092--- PASS: TestRegisterCompletedObjectNeedsPendingRow (1.89s)1093=== CONT TestGCMetrics10942026/11/12 16:57:09 INFO Vacuumed table table=commit_receipts10952026/11/12 16:57:09 INFO Vacuumed table table=pending_objects10962026/11/12 16:57:09 INFO Vacuumed table table=multipart_uploads10972026/11/12 16:57:09 INFO Vacuumed table table=closures10982026/11/12 16:57:09 INFO Vacuumed table table=objects1099--- PASS: TestSweepRechecksPendingAfterLocking (1.90s)1100=== CONT TestReadRedirectUsesPublicS3URL11012026/11/12 16:57:09 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011022026/11/12 16:57:09 INFO Vacuumed table table=pending_closures11032026/11/12 16:57:09 INFO Vacuumed table table=commit_receipts11042026/11/12 16:57:09 INFO Vacuumed table table=pending_objects11052026/11/12 16:57:09 INFO Vacuumed table table=multipart_uploads1106=== RUN TestPush_RejectsBadRequests/no_roots1107=== PAUSE TestPush_RejectsBadRequests/no_roots1108=== RUN TestPush_RejectsBadRequests/no_objects1109=== PAUSE TestPush_RejectsBadRequests/no_objects1110=== RUN TestPush_RejectsBadRequests/bad_root1111=== PAUSE TestPush_RejectsBadRequests/bad_root1112=== RUN TestPush_RejectsBadRequests/root_not_in_objects1113=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects1114=== CONT TestCreatePin_ReservedPins11152026/11/12 16:57:09 INFO Vacuumed table table=closures11162026/11/12 16:57:09 INFO Vacuumed table table=objects11172026/11/12 16:57:09 INFO Received uploads request method=POST path=/api/pending_closures11182026/11/12 16:57:09 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001119--- PASS: TestService_createPendingClosureHandler (2.06s)1120=== CONT TestGCBugBareHashReferences11212026/11/12 16:57:09 INFO Aborted multipart uploads count=0 kept=011222026/11/12 16:57:09 WARN Force mode enabled - objects will be deleted immediately without grace period11232026/11/12 16:57:09 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011242026/11/12 16:57:09 INFO Vacuumed table table=pending_closures11252026/11/12 16:57:09 INFO Vacuumed table table=commit_receipts11262026/11/12 16:57:09 INFO Vacuumed table table=pending_objects11272026/11/12 16:57:09 INFO Vacuumed table table=multipart_uploads11282026/11/12 16:57:09 INFO Vacuumed table table=closures1129--- PASS: TestGetExistingObjectsLocksRows (2.04s)1130=== CONT TestReadProxyRangeRequest11312026/11/12 16:57:09 INFO Vacuumed table table=objects11322026/11/12 16:57:09 INFO Aborted multipart uploads count=0 kept=011332026/11/12 16:57:09 INFO Received uploads request method=POST path=/api/pending_closures11342026/11/12 16:57:09 WARN Force mode enabled - objects will be deleted immediately without grace period11352026/11/12 16:57:09 INFO Received push request method=POST path=/api/pushes11362026/11/12 16:57:09 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=011372026/11/12 16:57:09 INFO Vacuumed table table=pending_closures11382026/11/12 16:57:09 INFO Vacuumed table table=commit_receipts11392026/11/12 16:57:09 INFO Vacuumed table table=pending_objects11402026/11/12 16:57:09 INFO Vacuumed table table=multipart_uploads11412026/11/12 16:57:09 INFO Vacuumed table table=closures11422026/11/12 16:57:09 INFO Vacuumed table table=objects1143--- PASS: TestSweepSkipsRowsLockedByPush (2.08s)1144=== CONT TestOrphanedObjectsGCStressTest11452026/11/12 16:57:09 INFO Received complete push request method=POST path=/api/pushes/1/complete11462026/11/12 16:57:09 INFO Received push request method=POST path=/api/pushes11472026/11/12 16:57:09 INFO Completed push id=1 objects=4 duration=10.874761ms11482026/11/12 16:57:09 INFO Received push request method=POST path=/api/pushes11492026/11/12 16:57:09 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:36155/oidc11502026/11/12 16:57:09 INFO Received complete multipart upload request method=POST path=/api/multipart/complete11512026/11/12 16:57:09 INFO Received uploads request method=POST path=/api/pending_closures11522026/11/12 16:57:09 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=NzY3YTMxNTYtMmRkOS00OTE4LWFhOTctZGU1NmYxZWQwYjI4LjRlNTM4ZWI5LWU5MmItNGRlYi05NTBiLTgyNDRjMjBjZjcxM3gxNzk0NTAyNjI4MDEyODg5MjYz parts=121153--- PASS: TestSharedNarBothUploadsComplete (2.20s)1154=== CONT TestGCAdvisoryLockBlocksConcurrentRun11552026/11/12 16:57:09 INFO Aborted multipart uploads count=0 kept=011562026-11-12 16:57:09.831 UTC [566] ERROR: could not obtain lock on row in relation "closures"11572026-11-12 16:57:09.831 UTC [566] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT11582026-11-12 16:57:09.832 UTC [566] ERROR: could not obtain lock on row in relation "closures"11592026-11-12 16:57:09.832 UTC [566] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT11602026-11-12 16:57:09.832 UTC [566] ERROR: could not obtain lock on row in relation "closures"11612026-11-12 16:57:09.832 UTC [566] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT11622026-11-12 16:57:09.832 UTC [566] ERROR: could not obtain lock on row in relation "closures"11632026-11-12 16:57:09.832 UTC [566] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT11642026-11-12 16:57:09.832 UTC [566] ERROR: could not obtain lock on row in relation "closures"11652026-11-12 16:57:09.832 UTC [566] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT11662026/11/12 16:57:09 WARN Force mode enabled - objects will be deleted immediately without grace period1167--- PASS: TestPush_RootsAreLockedInByteOrder (1.46s)1168=== CONT TestReadRedirectKeepsNarinfoProxied11692026/11/12 16:57:09 INFO Aborted multipart uploads count=0 kept=011702026/11/12 16:57:09 INFO Received push request method=POST path=/api/pushes11712026/11/12 16:57:09 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."11722026/11/12 16:57:09 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."11732026/11/12 16:57:09 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1174--- PASS: TestFailedS3DeleteKeepsTombstone (2.06s)1175=== CONT TestOrphanedObjectsGC11762026/11/12 16:57:09 WARN Force mode enabled - objects will be deleted immediately without grace period11772026/11/12 16:57:09 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011782026/11/12 16:57:09 INFO Vacuumed table table=pending_closures11792026/11/12 16:57:09 INFO Vacuumed table table=commit_receipts11802026/11/12 16:57:09 INFO Vacuumed table table=pending_objects11812026/11/12 16:57:09 INFO Vacuumed table table=multipart_uploads11822026/11/12 16:57:09 INFO Aborted multipart uploads count=0 kept=011832026/11/12 16:57:09 INFO Vacuumed table table=closures11842026/11/12 16:57:09 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=NzY3YTMxNTYtMmRkOS00OTE4LWFhOTctZGU1NmYxZWQwYjI4Ljk5NWU4ZGViLTA5MDktNDljYi05NGYzLTJiNmI2NzQyYmI1M3gxNzk0NTAyNjI5NTQxNjMwNjA1 parts=1011852026/11/12 16:57:09 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete11862026/11/12 16:57:09 INFO Vacuumed table table=objects11872026/11/12 16:57:09 WARN Force mode enabled - objects will be deleted immediately without grace period11882026/11/12 16:57:09 INFO Completed upload id=1 objects=2 duration=4.035637ms11892026/11/12 16:57:09 INFO Received uploads request method=POST path=/api/pending_closures1190--- PASS: TestGCSweepSkipsPendingObjects (1.42s)1191=== CONT TestReadRedirectNar11922026/11/12 16:57:09 INFO Received uploads request method=POST path=/api/pending_closures11932026/11/12 16:57:09 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=011942026/11/12 16:57:09 INFO Vacuumed table table=pending_closures11952026/11/12 16:57:09 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo11962026/11/12 16:57:09 WARN Found objects in DB but missing from S3, will re-upload count=11197--- PASS: TestPresentRefreshesOnlyStaleClosures (1.09s)1198=== CONT TestObjectStatsUpdatedOncePerCommit1199--- PASS: TestService_verifyS3Integrity (2.19s)1200=== CONT TestClientMultipleUploads12012026/11/12 16:57:09 INFO Vacuumed table table=commit_receipts12022026/11/12 16:57:09 INFO Vacuumed table table=pending_objects12032026/11/12 16:57:09 INFO Vacuumed table table=multipart_uploads1204--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (2.20s)1205=== CONT TestObjectStatsTrigger12062026/11/12 16:57:09 INFO Vacuumed table table=closures12072026/11/12 16:57:09 INFO Vacuumed table table=objects12082026/11/12 16:57:09 INFO Received uploads request method=POST path=/api/pending_closures12092026/11/12 16:57:09 INFO Received complete push request method=POST path=/api/pushes/2/complete12102026/11/12 16:57:09 INFO Completed push id=2 objects=4 duration=3.733394ms1211--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (1.72s)1212=== CONT TestLeadIncumbentAnnouncesAtOnce1213--- PASS: TestDeduplicatedObjectsRecordedAsPending (0.74s)1214=== CONT TestPendingCleanupUsesOneCutoff12152026/11/12 16:57:09 INFO Received push request method=POST path=/api/pushes12162026/11/12 16:57:09 INFO Received complete push request method=POST path=/api/pushes/1/complete12172026/11/12 16:57:09 INFO Completed push id=1 objects=4 duration=12.239654ms1218--- PASS: TestPush_RepeatedRootCommitsOnce (0.77s)1219=== CONT TestLeadUnannouncedStandbyYieldsToIncumbent12202026/11/12 16:57:09 INFO Received push request method=POST path=/api/pushes12212026/11/12 16:57:09 INFO Received uploads request method=POST path=/api/pending_closures12222026/11/12 16:57:09 INFO Received complete push request method=POST path=/api/pushes/1/complete1223--- PASS: TestPushWaitsForSweepHoldingRow (2.28s)1224=== CONT TestReadProxyDisabled12252026/11/12 16:57:10 INFO Completed push id=1 objects=6 duration=23.120436ms1226--- PASS: TestPush_CompleteCommitsEveryRoot (0.59s)1227=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded1228--- PASS: TestPresentReportsOnlyClosureRoots (0.57s)1229=== CONT TestReadProxyRootRedirectsToIndexHTML12302026/11/12 16:57:10 INFO Received push request method=POST path=/api/pushes12312026/11/12 16:57:10 INFO Aborted multipart uploads count=0 kept=01232--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (0.47s)1233=== CONT TestLeadElectsOneAndHandsOver12342026/11/12 16:57:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012352026/11/12 16:57:10 INFO Aborted multipart uploads count=0 kept=012362026/11/12 16:57:10 INFO Vacuumed table table=pending_closures12372026/11/12 16:57:10 INFO Vacuumed table table=commit_receipts12382026/11/12 16:57:10 INFO Vacuumed table table=pending_objects12392026/11/12 16:57:10 INFO Vacuumed table table=multipart_uploads12402026/11/12 16:57:10 WARN Force mode enabled - objects will be deleted immediately without grace period12412026/11/12 16:57:10 INFO Vacuumed table table=closures12422026/11/12 16:57:10 INFO Vacuumed table table=objects12432026/11/12 16:57:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012442026/11/12 16:57:10 INFO Vacuumed table table=pending_closures12452026/11/12 16:57:10 INFO Vacuumed table table=commit_receipts12462026/11/12 16:57:10 INFO Vacuumed table table=pending_objects12472026/11/12 16:57:10 INFO Vacuumed table table=multipart_uploads12482026/11/12 16:57:10 INFO Vacuumed table table=closures12492026/11/12 16:57:10 INFO Vacuumed table table=objects12502026/11/12 16:57:10 INFO Aborted multipart uploads count=0 kept=012512026/11/12 16:57:10 WARN Force mode enabled - objects will be deleted immediately without grace period1252--- PASS: TestGCMetrics (0.48s)1253=== CONT TestReadProxyConditionalGet12542026/11/12 16:57:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012552026/11/12 16:57:10 INFO Vacuumed table table=pending_closures12562026/11/12 16:57:10 INFO Vacuumed table table=commit_receipts12572026/11/12 16:57:10 INFO Vacuumed table table=pending_objects12582026/11/12 16:57:10 INFO Vacuumed table table=multipart_uploads12592026/11/12 16:57:10 INFO Vacuumed table table=closures12602026/11/12 16:57:10 INFO Vacuumed table table=objects1261--- PASS: TestPushDedupSurvivesConcurrentGC (0.83s)1262=== CONT TestMultipartCleanup1263--- PASS: TestReadRedirectUsesPublicS3URL (0.49s)1264=== CONT TestConnectWaitsForAPeerMigration12652026/11/12 16:57:10 INFO lead: released remote=192.0.2.1:12341266--- PASS: TestLeadEndsOnShutdown (2.42s)1267=== CONT TestReadProxyOutlastsServerWriteTimeout1268--- PASS: TestReadProxyRangeRequest (0.39s)1269=== CONT TestResolveDBConnectionString1270=== RUN TestResolveDBConnectionString/flag_wins1271=== PAUSE TestResolveDBConnectionString/flag_wins1272=== RUN TestResolveDBConnectionString/file_when_flag_empty1273=== PAUSE TestResolveDBConnectionString/file_when_flag_empty1274=== RUN TestResolveDBConnectionString/missing_file_is_an_error1275=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error1276=== RUN TestResolveDBConnectionString/PGHOST_allows_empty1277=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty1278=== RUN TestResolveDBConnectionString/nothing_configured1279=== PAUSE TestResolveDBConnectionString/nothing_configured1280=== CONT TestService_ReadScope_PublicByDefault12812026-11-12 16:57:10.159 UTC [578] ERROR: Push does not exist: id=112822026-11-12 16:57:10.159 UTC [578] CONTEXT: PL/pgSQL function commit_push(bigint) line 15 at RAISE12832026-11-12 16:57:10.159 UTC [578] STATEMENT: SELECT commit_push($1)1284--- PASS: TestPush_CompleteWaitsForCleanupOfThePush (1.59s)1285=== CONT TestReadProxyHead12862026/11/12 16:57:10 INFO Received uploads request method=POST path=/api/pending_closures1287--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (0.18s)1288=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock12892026/11/12 16:57:10 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux12902026/11/12 16:57:10 WARN Refused reserved pin name=worker-x86_64-linux12912026/11/12 16:57:10 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux12922026/11/12 16:57:10 INFO Received create pin request method=POST path=/api/pins/my-app12932026/11/12 16:57:10 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux1294--- PASS: TestCreatePin_ReservedPins (0.58s)1295=== CONT TestReadProxyInvalidPath12962026/11/12 16:57:10 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=nixbld database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"1297--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (0.39s)1298=== CONT TestDeleteClosuresRacingPins1299--- PASS: TestReadRedirectKeepsNarinfoProxied (0.40s)1300=== CONT TestClientFallsBackToClosures1301--- PASS: TestReadRedirectNar (0.42s)1302=== CONT TestClientPushesUseOnePush1303--- PASS: TestGCBugBareHashReferences (0.63s)1304=== CONT TestReadProxy4041305--- PASS: TestObjectStatsTrigger (0.45s)1306=== CONT TestPinProtectsFromGC1307--- PASS: TestObjectStatsUpdatedOncePerCommit (0.46s)1308=== CONT TestReadProxyNarStreaming13092026/11/12 16:57:10 INFO lead: acquired remote=192.0.2.1:123413102026/11/12 16:57:10 INFO lead: released remote=192.0.2.1:12341311--- PASS: TestLeadIncumbentAnnouncesAtOnce (0.48s)1312=== CONT TestClientSharedPathCommittedMidPush13132026/11/12 16:57:10 INFO Received uploads request method=POST path=/api/pending_closures1314=== NAME TestClientMultipleUploads1315 client_integration_test.go:406: Created store path 0: /build/TestClientMultipleUploads598844951/001/store/53ayx7vmsqsbjnjzzmbf25qckjfqqjgm-test-file-0.txt13162026/11/12 16:57:10 INFO Received uploads request method=POST path=/api/pending_closures13172026/11/12 16:57:10 INFO Received cleanup request method=DELETE path=/api/pending_closures13182026/11/12 16:57:10 INFO lead: acquired remote=192.0.2.1:12341319 client_integration_test.go:406: Created store path 1: /build/TestClientMultipleUploads598844951/001/store/n72bsdyw5c1k8mv8awv4s8w1xy2ik9mn-test-file-1.txt1320 client_integration_test.go:406: Created store path 2: /build/TestClientMultipleUploads598844951/001/store/hj2d72ziqpr3rmnpibshbxgslkm2k4jp-test-file-2.txt1321--- PASS: TestReadProxyDisabled (0.52s)1322=== CONT TestReadProxyContentNegotiation1323--- PASS: TestReadProxyRootRedirectsToIndexHTML (0.52s)1324=== CONT TestCacheStatsHandler13252026/11/12 16:57:10 INFO lead: acquired remote=192.0.2.1:12341326--- PASS: TestReadProxyConditionalGet (0.50s)1327=== CONT TestReadProxyNarinfoAlreadyDecompressed13282026/11/12 16:57:10 INFO Received push request method=POST path=/api/pushes13292026/11/12 16:57:10 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)13302026/11/12 16:57:10 INFO Uploading hj2d72ziqpr3rmnpibshbxgslkm2k4jp-test-file-2.txt (160B)13312026/11/12 16:57:10 INFO Uploading 53ayx7vmsqsbjnjzzmbf25qckjfqqjgm-test-file-0.txt (160B)13322026/11/12 16:57:10 INFO Uploading n72bsdyw5c1k8mv8awv4s8w1xy2ik9mn-test-file-1.txt (160B)13332026/11/12 16:57:10 INFO Received uploads request method=POST path=/api/pending_closures13342026/11/12 16:57:10 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"13352026/11/12 16:57:10 WARN Failed to register uploaded object key=53ayx7vmsqsbjnjzzmbf25qckjfqqjgm.ls error="server returned 404: 404 page not found\n"13362026/11/12 16:57:10 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"13372026/11/12 16:57:10 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"13382026/11/12 16:57:10 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13392026/11/12 16:57:10 WARN Failed to register uploaded object key=n72bsdyw5c1k8mv8awv4s8w1xy2ik9mn.ls error="server returned 404: 404 page not found\n"13402026/11/12 16:57:10 WARN Failed to register uploaded object key=hj2d72ziqpr3rmnpibshbxgslkm2k4jp.ls error="server returned 404: 404 page not found\n"13412026/11/12 16:57:10 INFO Signed narinfos id=1 count=313422026/11/12 16:57:10 INFO Uploading 3 narinfos13432026/11/12 16:57:10 INFO Received complete push request method=POST path=/api/pushes/1/complete13442026/11/12 16:57:10 WARN Failed to register uploaded object key=53ayx7vmsqsbjnjzzmbf25qckjfqqjgm.narinfo error="server returned 404: 404 page not found\n"13452026/11/12 16:57:10 WARN Failed to register uploaded object key=hj2d72ziqpr3rmnpibshbxgslkm2k4jp.narinfo error="server returned 404: 404 page not found\n"13462026/11/12 16:57:10 WARN Failed to register uploaded object key=n72bsdyw5c1k8mv8awv4s8w1xy2ik9mn.narinfo error="server returned 404: 404 page not found\n"13472026/11/12 16:57:10 INFO Completed push id=1 objects=9 duration=7.191147ms13482026/11/12 16:57:10 INFO Upload complete. (66ms)1349=== NAME TestClientMultipleUploads1350 client_integration_test.go:417: Uploaded 3 paths in 105.3538ms1351--- PASS: TestClientMultipleUploads (0.72s)1352=== CONT TestClientWithDependencies1353=== NAME TestOrphanedObjectsGC1354 orphaned_objects_gc_test.go:287: GC Test Summary:1355 orphaned_objects_gc_test.go:288: - Kept: 2 objects from closure A1356 orphaned_objects_gc_test.go:289: - Deleted: 2 objects from closure B1357 orphaned_objects_gc_test.go:290: - Deleted: 6 orphaned chain objects (X1->X2->X3)1358 orphaned_objects_gc_test.go:291: - Deleted: 2 orphaned single objects (Y)1359 orphaned_objects_gc_test.go:292: - Total deleted: 10 objects1360--- PASS: TestOrphanedObjectsGC (0.77s)1361=== CONT TestCacheConfigHandler1362=== RUN TestCacheConfigHandler/full_config,_no_issuer1363=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1364=== RUN TestCacheConfigHandler/no_cache_url_configured1365=== PAUSE TestCacheConfigHandler/no_cache_url_configured1366=== RUN TestCacheConfigHandler/no_signing_keys1367=== PAUSE TestCacheConfigHandler/no_signing_keys1368=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1369=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1370=== CONT TestClientCADerivations1371--- PASS: TestService_ReadScope_PublicByDefault (0.49s)1372=== CONT TestCacheConfigHandlerMaxNarSize1373--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)1374=== CONT TestService_ReadAuthMiddleware1375--- PASS: TestReadProxyHead (0.49s)1376=== CONT TestClientIntegration1377--- PASS: TestReadProxyInvalidPath (0.49s)1378=== CONT TestService_RequireScope_OIDC13792026/11/12 16:57:10 INFO Received cleanup request method=DELETE path=/api/pending_closures13802026/11/12 16:57:10 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket60/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""13812026/11/12 16:57:10 INFO Aborted multipart uploads count=0 kept=113822026/11/12 16:57:10 INFO Received cleanup request method=DELETE path=/api/pending_closures13832026/11/12 16:57:10 INFO Aborted multipart uploads count=1 kept=01384--- PASS: TestMultipartCleanup (0.62s)1385=== CONT TestClientErrorHandling1386=== RUN TestClientErrorHandling/InvalidStorePath1387=== PAUSE TestClientErrorHandling/InvalidStorePath1388=== RUN TestClientErrorHandling/InvalidAuthToken1389=== PAUSE TestClientErrorHandling/InvalidAuthToken1390=== RUN TestClientErrorHandling/ServerNotAvailable1391=== PAUSE TestClientErrorHandling/ServerNotAvailable1392=== CONT TestService_AuthMiddleware_OIDC1393--- PASS: TestReadProxyNarStreaming (0.46s)1394=== CONT TestService_NativeMTLS13952026/11/12 16:57:10 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:44679/oidc1396--- PASS: TestReadProxy404 (0.50s)1397=== CONT TestService_readinessHandler13982026/11/12 16:57:10 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:41043/oidc1399--- PASS: TestCacheStatsHandler (0.36s)1400=== CONT TestMetricsCommit1401=== NAME TestPinProtectsFromGC1402 client_integration_test.go:791: Pinned store path: /build/TestPinProtectsFromGC4264332258/001/store/8zc727anxzaaz9rgagmv18xz0myvrm41-pinned-file.txt1403 client_integration_test.go:792: Unpinned store path: /build/TestPinProtectsFromGC4264332258/001/store/hwd9m24mqi0ldgapb3m3gxcz91zz7w3z-unpinned-file.txt1404--- PASS: TestReadProxyNarinfoAlreadyDecompressed (0.34s)1405=== CONT TestService_AuthMiddleware_MTLSProxyHeader14062026/11/12 16:57:10 INFO Received uploads request method=POST path=/api/pending_closures14072026/11/12 16:57:10 INFO Received uploads request method=POST path=/api/pending_closures1408--- PASS: TestReadProxyContentNegotiation (0.42s)1409=== CONT TestReadProxyNarinfo14102026/11/12 16:57:10 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)14112026/11/12 16:57:10 INFO Uploading ygminpdw5plgxpnndz0liai8wz4jg3jn-a (216B)14122026/11/12 16:57:10 INFO Uploading gcyhjhpi88s6prrcl1fhn8rmd4mxb0nd-shared-dep (136B)14132026/11/12 16:57:10 WARN Failed to register uploaded object key=gy7j6rjg0n0j76bvjkn4w2nrbvhqiqyx.ls error="server returned 404: 404 page not found\n"14142026/11/12 16:57:10 WARN Failed to register uploaded object key=gcyhjhpi88s6prrcl1fhn8rmd4mxb0nd.ls error="server returned 404: 404 page not found\n"14152026/11/12 16:57:10 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"14162026/11/12 16:57:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign14172026/11/12 16:57:10 WARN Failed to register uploaded object key=nar/0lfyzv7477f4l2yq34hayqsb9jq8y58vxc1528iqqfb32hwkmw5h.nar.zst error="server returned 404: 404 page not found\n"14182026/11/12 16:57:10 WARN Failed to register uploaded object key=ygminpdw5plgxpnndz0liai8wz4jg3jn.ls error="server returned 404: 404 page not found\n"14192026/11/12 16:57:10 INFO Signed narinfos id=1 count=214202026/11/12 16:57:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign14212026/11/12 16:57:10 INFO Signed narinfos id=2 count=214222026/11/12 16:57:10 INFO Received push request method=POST path=/api/pushes14232026/11/12 16:57:10 INFO Uploading 4 narinfos14242026/11/12 16:57:10 WARN Failed to register uploaded object key=gcyhjhpi88s6prrcl1fhn8rmd4mxb0nd.narinfo error="server returned 404: 404 page not found\n"14252026/11/12 16:57:10 WARN Failed to register uploaded object key=gy7j6rjg0n0j76bvjkn4w2nrbvhqiqyx.narinfo error="server returned 404: 404 page not found\n"14262026/11/12 16:57:10 WARN Failed to register uploaded object key=ygminpdw5plgxpnndz0liai8wz4jg3jn.narinfo error="server returned 404: 404 page not found\n"14272026/11/12 16:57:10 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete14282026/11/12 16:57:10 WARN Failed to register uploaded object key=gcyhjhpi88s6prrcl1fhn8rmd4mxb0nd.narinfo error="server returned 404: 404 page not found\n"14292026/11/12 16:57:10 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)14302026/11/12 16:57:10 INFO Uploading xz1sgafsxcp3s394kika42wb2fwrd3kz-a (216B)14312026/11/12 16:57:10 INFO Uploading kqgb14k8b9b8km9ygqqiq46j4dv9wif6-shared-dep (136B)14322026/11/12 16:57:10 INFO Completed upload id=1 objects=6 duration=6.994965ms14332026/11/12 16:57:10 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete14342026/11/12 16:57:10 WARN Failed to register uploaded object key=09kkmph2k98whs05szrrj4sz1chm3j1w.ls error="server returned 404: 404 page not found\n"14352026/11/12 16:57:10 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"14362026/11/12 16:57:10 INFO Completed upload id=2 objects=6 duration=1.839817ms14372026/11/12 16:57:10 WARN Failed to register uploaded object key=kqgb14k8b9b8km9ygqqiq46j4dv9wif6.ls error="server returned 404: 404 page not found\n"14382026/11/12 16:57:10 WARN Failed to register uploaded object key=nar/06qwxpx808pchcx7j6cd6kl4kwm6lxy274a435j6lm02l1dr262p.nar.zst error="server returned 404: 404 page not found\n"14392026/11/12 16:57:10 INFO Upload complete. (74ms)14402026/11/12 16:57:10 WARN Failed to register uploaded object key=xz1sgafsxcp3s394kika42wb2fwrd3kz.ls error="server returned 404: 404 page not found\n"14412026/11/12 16:57:10 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign14422026/11/12 16:57:10 INFO Signed narinfos id=1 count=314432026/11/12 16:57:10 INFO Uploading 3 narinfos1444=== NAME TestClientFallsBackToClosures1445 client_pushes_test.go:112: Retrieved narinfo from S3:1446 StorePath: /build/TestClientFallsBackToClosures1086080864/001/store/gcyhjhpi88s6prrcl1fhn8rmd4mxb0nd-shared-dep1447 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1448 Compression: zstd1449 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821450 NarSize: 1361451 References: 1452 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1453 client_pushes_test.go:112: Retrieved narinfo from S3:1454 StorePath: /build/TestClientFallsBackToClosures1086080864/001/store/ygminpdw5plgxpnndz0liai8wz4jg3jn-a1455 URL: nar/0lfyzv7477f4l2yq34hayqsb9jq8y58vxc1528iqqfb32hwkmw5h.nar.zst1456 Compression: zstd1457 NarHash: sha256:0lfyzv7477f4l2yq34hayqsb9jq8y58vxc1528iqqfb32hwkmw5h1458 NarSize: 2161459 References: /build/TestClientFallsBackToClosures1086080864/001/store/gcyhjhpi88s6prrcl1fhn8rmd4mxb0nd-shared-dep1460 CA: text:sha256:07klf4lqnwlqh3qsdzn6jj54khyrzdvmwa90z32z8zl4yjzb3zj014612026/11/12 16:57:10 WARN Failed to register uploaded object key=09kkmph2k98whs05szrrj4sz1chm3j1w.narinfo error="server returned 404: 404 page not found\n"14622026/11/12 16:57:10 WARN Failed to register uploaded object key=xz1sgafsxcp3s394kika42wb2fwrd3kz.narinfo error="server returned 404: 404 page not found\n"14632026/11/12 16:57:10 INFO Received complete push request method=POST path=/api/pushes/1/complete14642026/11/12 16:57:10 WARN Failed to register uploaded object key=kqgb14k8b9b8km9ygqqiq46j4dv9wif6.narinfo error="server returned 404: 404 page not found\n"1465 client_pushes_test.go:112: Retrieved narinfo from S3:1466 StorePath: /build/TestClientFallsBackToClosures1086080864/001/store/gy7j6rjg0n0j76bvjkn4w2nrbvhqiqyx-b1467 URL: nar/0lfyzv7477f4l2yq34hayqsb9jq8y58vxc1528iqqfb32hwkmw5h.nar.zst1468 Compression: zstd1469 NarHash: sha256:0lfyzv7477f4l2yq34hayqsb9jq8y58vxc1528iqqfb32hwkmw5h1470 NarSize: 2161471 References: /build/TestClientFallsBackToClosures1086080864/001/store/gcyhjhpi88s6prrcl1fhn8rmd4mxb0nd-shared-dep1472 CA: text:sha256:07klf4lqnwlqh3qsdzn6jj54khyrzdvmwa90z32z8zl4yjzb3zj014732026/11/12 16:57:10 INFO Received push request method=POST path=/api/pushes1474--- PASS: TestClientFallsBackToClosures (0.72s)1475=== CONT TestService_AuthMiddleware_MTLSBoundSubjects1476--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (0.78s)1477=== CONT TestIsValidCachePath1478=== RUN TestIsValidCachePath/narinfo1479=== PAUSE TestIsValidCachePath/narinfo1480=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars1481=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars1482=== RUN TestIsValidCachePath/nar_zst1483=== PAUSE TestIsValidCachePath/nar_zst14842026/11/12 16:57:10 INFO Completed push id=1 objects=8 duration=7.791952ms1485=== RUN TestIsValidCachePath/nar_xz1486=== PAUSE TestIsValidCachePath/nar_xz1487=== RUN TestIsValidCachePath/nar_bz21488=== PAUSE TestIsValidCachePath/nar_bz21489=== RUN TestIsValidCachePath/nar_uncompressed1490=== PAUSE TestIsValidCachePath/nar_uncompressed14912026/11/12 16:57:10 INFO Upload complete. (72ms)1492=== RUN TestIsValidCachePath/ls1493=== PAUSE TestIsValidCachePath/ls1494=== RUN TestIsValidCachePath/log1495=== PAUSE TestIsValidCachePath/log1496=== RUN TestIsValidCachePath/realisation1497=== PAUSE TestIsValidCachePath/realisation1498=== RUN TestIsValidCachePath/nix-cache-info1499=== PAUSE TestIsValidCachePath/nix-cache-info1500=== RUN TestIsValidCachePath/index.html1501=== PAUSE TestIsValidCachePath/index.html1502=== RUN TestIsValidCachePath/pin1503=== PAUSE TestIsValidCachePath/pin1504=== RUN TestIsValidCachePath/pin_all_alphabet_chars1505=== PAUSE TestIsValidCachePath/pin_all_alphabet_chars1506=== RUN TestIsValidCachePath/traversal_parent1507=== PAUSE TestIsValidCachePath/traversal_parent1508=== RUN TestIsValidCachePath/traversal_in_middle1509=== PAUSE TestIsValidCachePath/traversal_in_middle1510=== RUN TestIsValidCachePath/pin_traversal1511=== PAUSE TestIsValidCachePath/pin_traversal1512=== RUN TestIsValidCachePath/invalid_char_e1513=== PAUSE TestIsValidCachePath/invalid_char_e1514=== RUN TestIsValidCachePath/invalid_char_u1515=== PAUSE TestIsValidCachePath/invalid_char_u1516=== RUN TestIsValidCachePath/random_path1517=== PAUSE TestIsValidCachePath/random_path1518=== RUN TestIsValidCachePath/empty1519=== PAUSE TestIsValidCachePath/empty1520=== RUN TestIsValidCachePath/leading_slash1521=== PAUSE TestIsValidCachePath/leading_slash1522=== RUN TestIsValidCachePath/wrong_extension1523=== PAUSE TestIsValidCachePath/wrong_extension1524=== RUN TestIsValidCachePath/short_hash1525=== PAUSE TestIsValidCachePath/short_hash1526=== RUN TestIsValidCachePath/pin_empty_name1527=== PAUSE TestIsValidCachePath/pin_empty_name1528=== RUN TestIsValidCachePath/pin_nested1529=== PAUSE TestIsValidCachePath/pin_nested1530=== RUN TestIsValidCachePath/pin_invalid_char1531=== PAUSE TestIsValidCachePath/pin_invalid_char1532=== CONT TestService_healthCheckHandler15332026/11/12 16:57:10 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)1534=== NAME TestClientPushesUseOnePush15352026/11/12 16:57:10 INFO Uploading 8zc727anxzaaz9rgagmv18xz0myvrm41-pinned-file.txt (128B)1536 client_pushes_test.go:97: Retrieved narinfo from S3:1537 StorePath: /build/TestClientPushesUseOnePush2622817318/001/store/kqgb14k8b9b8km9ygqqiq46j4dv9wif6-shared-dep1538 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1539 Compression: zstd1540 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821541 NarSize: 1361542 References: 1543 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1544 client_pushes_test.go:97: Retrieved narinfo from S3:1545 StorePath: /build/TestClientPushesUseOnePush2622817318/001/store/xz1sgafsxcp3s394kika42wb2fwrd3kz-a1546 URL: nar/06qwxpx808pchcx7j6cd6kl4kwm6lxy274a435j6lm02l1dr262p.nar.zst1547 Compression: zstd1548 NarHash: sha256:06qwxpx808pchcx7j6cd6kl4kwm6lxy274a435j6lm02l1dr262p1549 NarSize: 2161550 References: /build/TestClientPushesUseOnePush2622817318/001/store/kqgb14k8b9b8km9ygqqiq46j4dv9wif6-shared-dep1551 CA: text:sha256:1sy59269nipx12xhrz4rc9hc1v8vz54xc6vyw2gnq86nfkxl5gpz15522026/11/12 16:57:10 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"15532026/11/12 16:57:10 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign1554 client_pushes_test.go:97: Retrieved narinfo from S3:1555 StorePath: /build/TestClientPushesUseOnePush2622817318/001/store/09kkmph2k98whs05szrrj4sz1chm3j1w-b1556 URL: nar/06qwxpx808pchcx7j6cd6kl4kwm6lxy274a435j6lm02l1dr262p.nar.zst1557 Compression: zstd1558 NarHash: sha256:06qwxpx808pchcx7j6cd6kl4kwm6lxy274a435j6lm02l1dr262p1559 NarSize: 2161560 References: /build/TestClientPushesUseOnePush2622817318/001/store/kqgb14k8b9b8km9ygqqiq46j4dv9wif6-shared-dep1561 CA: text:sha256:1sy59269nipx12xhrz4rc9hc1v8vz54xc6vyw2gnq86nfkxl5gpz15622026/11/12 16:57:10 WARN Failed to register uploaded object key=8zc727anxzaaz9rgagmv18xz0myvrm41.ls error="server returned 404: 404 page not found\n"15632026/11/12 16:57:10 INFO Signed narinfos id=1 count=115642026/11/12 16:57:10 INFO Uploading 1 narinfos1565--- PASS: TestService_ReadAuthMiddleware (0.35s)1566=== CONT TestEnsureAbortUploadsRule1567--- PASS: TestClientPushesUseOnePush (0.69s)1568=== CONT TestGenerateLandingPage15692026/11/12 16:57:10 INFO Received complete push request method=POST path=/api/pushes/1/complete15702026/11/12 16:57:10 WARN Failed to register uploaded object key=8zc727anxzaaz9rgagmv18xz0myvrm41.narinfo error="server returned 404: 404 page not found\n"1571--- PASS: TestGenerateLandingPage (0.00s)1572=== CONT TestGracefulShutdownDrainsInflight15732026/11/12 16:57:10 INFO Starting HTTP server address=127.0.0.1:3587315742026/11/12 16:57:10 INFO Shutdown signal received, draining in-flight requests timeout=10s15752026/11/12 16:57:10 INFO Completed push id=1 objects=3 duration=7.167567ms15762026/11/12 16:57:10 INFO Upload complete. (63ms)15772026/11/12 16:57:10 INFO Received push request method=POST path=/api/pushes15782026/11/12 16:57:10 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)15792026/11/12 16:57:10 INFO Uploading jim3ly7nrl39al0zam67nqpr8vwb2b12-shared-dep (136B)15802026/11/12 16:57:10 INFO Uploading fs9ykq5p8dp0p1z63aphj26nlpf5zsdi-top (224B)15812026/11/12 16:57:10 WARN Failed to register uploaded object key=jim3ly7nrl39al0zam67nqpr8vwb2b12.ls error="server returned 404: 404 page not found\n"15822026/11/12 16:57:10 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15832026/11/12 16:57:10 WARN Failed to register uploaded object key=fs9ykq5p8dp0p1z63aphj26nlpf5zsdi.ls error="server returned 404: 404 page not found\n"15842026/11/12 16:57:10 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"15852026/11/12 16:57:10 WARN Failed to register uploaded object key=nar/0nw1hmsjya7n13f9z16amw6pr1z1bs9nydzi11cib024biv7mwk3.nar.zst error="server returned 404: 404 page not found\n"15862026/11/12 16:57:11 INFO Signed narinfos id=1 count=215872026/11/12 16:57:11 INFO Uploading 2 narinfos15882026/11/12 16:57:11 WARN Failed to register uploaded object key=fs9ykq5p8dp0p1z63aphj26nlpf5zsdi.narinfo error="server returned 404: 404 page not found\n"15892026/11/12 16:57:11 INFO Received complete push request method=POST path=/api/pushes/1/complete15902026/11/12 16:57:11 WARN Failed to register uploaded object key=jim3ly7nrl39al0zam67nqpr8vwb2b12.narinfo error="server returned 404: 404 page not found\n"15912026/11/12 16:57:11 INFO Completed push id=1 objects=6 duration=7.594091ms15922026/11/12 16:57:11 INFO Upload complete. (68ms)1593=== NAME TestClientSharedPathCommittedMidPush1594 client_integration_test.go:740: Retrieved narinfo from S3:1595 StorePath: /build/TestClientSharedPathCommittedMidPush2881268188/001/store/jim3ly7nrl39al0zam67nqpr8vwb2b12-shared-dep1596 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1597 Compression: zstd1598 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821599 NarSize: 1361600 References: 1601 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1602=== NAME TestClientWithDependencies1603 client_integration_test.go:661: Built derivation: /build/TestClientWithDependencies4189719816/001/store/r4qxmpjsqd7ih6nm95fl5f3xdqz143fz-test-script1604=== NAME TestClientSharedPathCommittedMidPush1605 client_integration_test.go:740: Retrieved narinfo from S3:1606 StorePath: /build/TestClientSharedPathCommittedMidPush2881268188/001/store/fs9ykq5p8dp0p1z63aphj26nlpf5zsdi-top1607 URL: nar/0nw1hmsjya7n13f9z16amw6pr1z1bs9nydzi11cib024biv7mwk3.nar.zst1608 Compression: zstd1609 NarHash: sha256:0nw1hmsjya7n13f9z16amw6pr1z1bs9nydzi11cib024biv7mwk31610 NarSize: 2241611 References: /build/TestClientSharedPathCommittedMidPush2881268188/001/store/jim3ly7nrl39al0zam67nqpr8vwb2b12-shared-dep1612 CA: text:sha256:069p2awjm9rzazxbbi64rj5vmgrmnircj3xc016dxdkdwvajlqw016132026/11/12 16:57:11 WARN mTLS auth: subject not in bound subjects subject="CN=reader"16142026/11/12 16:57:11 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1615--- PASS: TestService_NativeMTLS (0.22s)1616=== CONT TestService_AuthMiddleware1617--- PASS: TestClientSharedPathCommittedMidPush (0.65s)1618=== CONT TestNARDeduplicationMetadataUploadBug1619=== NAME TestClientIntegration1620 client_integration_test.go:292: Created store path: /build/TestClientIntegration3056814026/002/store/acdmyb3ksvqgghmr2aydxfdfayfzvdfm-test-file.txt1621=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token1622=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token1623=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1624=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1625=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1626=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1627=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1628=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1629=== CONT TestCreatePendingClosureRejectsOversizedNAR16302026/11/12 16:57:11 INFO Received uploads request method=POST path=/api/pending_closures1631--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.00s)1632=== CONT TestMetricsInventory1633--- PASS: TestGracefulShutdownDrainsInflight (0.07s)1634=== CONT TestProxyWriteTimeout/narinfo1635=== CONT TestCommitCanBeRepeatedUntilCleanup/push1636=== NAME TestClientCADerivations1637 client_ca_test.go:136: Built CA derivation: /build/TestClientCADerivations966050048/001/store/7n6q36pa1c0a4gp99s9lssqaym6gs2ym-ca-test16382026/11/12 16:57:11 INFO Received push request method=POST path=/api/pushes1639=== NAME TestClientWithDependencies1640 client_integration_test.go:663: Found 1 dependencies (including self)16412026/11/12 16:57:11 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)16422026/11/12 16:57:11 INFO Uploading hwd9m24mqi0ldgapb3m3gxcz91zz7w3z-unpinned-file.txt (128B)16432026/11/12 16:57:11 WARN Failed to register uploaded object key=hwd9m24mqi0ldgapb3m3gxcz91zz7w3z.ls error="server returned 404: 404 page not found\n"16442026/11/12 16:57:11 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign16452026/11/12 16:57:11 INFO Signed narinfos id=2 count=116462026/11/12 16:57:11 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"16472026/11/12 16:57:11 INFO Uploading 1 narinfos16482026/11/12 16:57:11 INFO Received complete push request method=POST path=/api/pushes/2/complete16492026/11/12 16:57:11 WARN Failed to register uploaded object key=hwd9m24mqi0ldgapb3m3gxcz91zz7w3z.narinfo error="server returned 404: 404 page not found\n"16502026/11/12 16:57:11 INFO Completed push id=2 objects=3 duration=2.633624ms16512026/11/12 16:57:11 WARN readiness check failed error="closed pool"16522026/11/12 16:57:11 INFO Upload complete. (54ms)1653--- PASS: TestService_readinessHandler (0.25s)1654=== CONT TestCommitCanBeRepeatedUntilCleanup/pending_closure1655=== NAME TestClientCADerivations1656 client_ca_test.go:139: Found 1 dependencies (including self)16572026/11/12 16:57:11 INFO Starting cleanup of old closures method=DELETE path=/api/closures16582026/11/12 16:57:11 INFO Garbage collection started1659=== RUN TestService_RequireScope_OIDC/builder_may_write1660=== PAUSE TestService_RequireScope_OIDC/builder_may_write1661=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1662=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1663=== RUN TestService_RequireScope_OIDC/ops_may_admin1664=== PAUSE TestService_RequireScope_OIDC/ops_may_admin1665=== RUN TestService_RequireScope_OIDC/ops_may_not_write1666=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1667=== RUN TestService_RequireScope_OIDC/reader_may_not_write1668=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1669=== RUN TestService_RequireScope_OIDC/static_token_may_admin1670=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1671=== RUN TestService_RequireScope_OIDC/static_token_may_write1672=== PAUSE TestService_RequireScope_OIDC/static_token_may_write1673=== RUN TestService_RequireScope_OIDC/reader_may_read1674=== PAUSE TestService_RequireScope_OIDC/reader_may_read1675=== RUN TestService_RequireScope_OIDC/writer_implies_read1676=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1677=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1678=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read1679=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc1680=== PAUSE TestService_RequireScope_OIDC/reader_may_read_with_netrc1681=== RUN TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1682=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1683=== RUN TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1684=== PAUSE TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1685=== RUN TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1686=== PAUSE TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1687=== CONT TestProxyWriteTimeout/10_GiB_nar1688=== CONT TestProxyWriteTimeout/unknown_size1689=== CONT TestProxyWriteTimeout/1_GiB_nar1690=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1691--- PASS: TestProxyWriteTimeout (0.00s)1692 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1693 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1694 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1695 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)16962026/11/12 16:57:11 INFO Received uploads request method=POST path=/1697=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key16982026/11/12 16:57:11 INFO Received request for more parts method=POST path=/1699=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key17002026/11/12 16:57:11 INFO Received complete multipart upload request method=POST path=/1701=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal17022026/11/12 16:57:11 INFO Received uploads request method=POST path=/1703--- PASS: TestUploadHandlersRejectInvalidKeys (0.00s)1704 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)1705 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1706 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1707 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s)1708=== CONT TestPendingClosureWriteTimeout/empty1709=== CONT TestGCEndsOnShutdown/before_the_run17102026/11/12 16:57:11 INFO Received push request method=POST path=/api/pushes17112026/11/12 16:57:11 INFO Received push request method=POST path=/api/pushes17122026/11/12 16:57:11 INFO Aborted multipart uploads count=0 kept=017132026/11/12 16:57:11 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)17142026/11/12 16:57:11 INFO Uploading acdmyb3ksvqgghmr2aydxfdfayfzvdfm-test-file.txt (152B)17152026/11/12 16:57:11 INFO Received push request method=POST path=/api/pushes17162026/11/12 16:57:11 WARN Failed to register uploaded object key=acdmyb3ksvqgghmr2aydxfdfayfzvdfm.ls error="server returned 404: 404 page not found\n"17172026/11/12 16:57:11 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"17182026/11/12 16:57:11 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign17192026/11/12 16:57:11 INFO Signed narinfos id=1 count=117202026/11/12 16:57:11 INFO Uploading 1 narinfos17212026/11/12 16:57:11 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)17222026/11/12 16:57:11 INFO Uploading r4qxmpjsqd7ih6nm95fl5f3xdqz143fz-test-script (136B)17232026/11/12 16:57:11 WARN Force mode enabled - objects will be deleted immediately without grace period17242026/11/12 16:57:11 INFO Received complete push request method=POST path=/api/pushes/1/complete17252026/11/12 16:57:11 INFO Received complete push request method=POST path=/api/pushes/1/complete17262026/11/12 16:57:11 WARN Failed to register uploaded object key=acdmyb3ksvqgghmr2aydxfdfayfzvdfm.narinfo error="server returned 404: 404 page not found\n"17272026/11/12 16:57:11 WARN Failed to register uploaded object key=r4qxmpjsqd7ih6nm95fl5f3xdqz143fz.ls error="server returned 404: 404 page not found\n"17282026/11/12 16:57:11 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"17292026/11/12 16:57:11 WARN Failed to register uploaded object key=log/w5j8gmd7wmnfd46kdjg4grvgqbm83dlk-test-script.drv error="server returned 404: 404 page not found\n"17302026/11/12 16:57:11 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign17312026/11/12 16:57:11 INFO Signed narinfos id=1 count=117322026/11/12 16:57:11 INFO Uploading 1 narinfos17332026/11/12 16:57:11 INFO Completed push id=1 objects=4 duration=8.707021ms1734--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (0.24s)1735=== CONT TestPendingClosureWriteTimeout/670k_objects1736=== CONT TestPendingClosureWriteTimeout/400_objects1737=== CONT TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline17382026/11/12 16:57:11 INFO Completed push id=1 objects=3 duration=8.184057ms17392026/11/12 16:57:11 INFO Upload complete. (70ms)17402026/11/12 16:57:11 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete17412026/11/12 16:57:11 INFO Received complete push request method=POST path=/api/pushes/1/complete17422026/11/12 16:57:11 WARN Failed to register uploaded object key=r4qxmpjsqd7ih6nm95fl5f3xdqz143fz.narinfo error="server returned 404: 404 page not found\n"17432026/11/12 16:57:11 INFO Completed upload id=2 objects=2 duration=1.718576ms1744--- PASS: TestMetricsCommit (0.27s)1745=== CONT TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark17462026/11/12 16:57:11 INFO Completed push id=1 objects=4 duration=7.896574ms17472026/11/12 16:57:11 INFO Upload complete. (68ms)17482026/11/12 16:57:11 INFO Aborted multipart uploads count=0 kept=017492026/11/12 16:57:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=017502026/11/12 16:57:11 WARN Force mode enabled - objects will be deleted immediately without grace period17512026/11/12 16:57:11 INFO Vacuumed table table=pending_closures17522026/11/12 16:57:11 INFO Vacuumed table table=commit_receipts17532026/11/12 16:57:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=017542026/11/12 16:57:11 INFO Vacuumed table table=pending_objects17552026/11/12 16:57:11 INFO Vacuumed table table=multipart_uploads17562026/11/12 16:57:11 INFO All 1 paths already cached17572026/11/12 16:57:11 INFO Vacuumed table table=pending_closures17582026/11/12 16:57:11 INFO Vacuumed table table=closures1759=== NAME TestClientIntegration1760 client_integration_test.go:318: Retrieved narinfo from S3:1761 StorePath: /build/TestClientIntegration3056814026/002/store/acdmyb3ksvqgghmr2aydxfdfayfzvdfm-test-file.txt1762 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1763 Compression: zstd1764 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11765 NarSize: 1521766 References: 1767 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk117682026/11/12 16:57:11 INFO Vacuumed table table=objects17692026/11/12 16:57:11 INFO Vacuumed table table=commit_receipts1770 client_integration_test.go:319: Retrieved .ls file from S3 (compressed size: 77 bytes)1771 client_integration_test.go:319: Decompressed .ls content (64 bytes):1772 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}17732026/11/12 16:57:11 INFO Vacuumed table table=pending_objects17742026/11/12 16:57:11 INFO Vacuumed table table=multipart_uploads17752026/11/12 16:57:11 INFO Received push request method=POST path=/api/pushes17762026/11/12 16:57:11 INFO Vacuumed table table=closures17772026/11/12 16:57:11 INFO Vacuumed table table=objects1778=== NAME TestClientWithDependencies1779 client_integration_test.go:677: Skipping nix copy test - isolated store (/build/TestClientWithDependencies4189719816/001/store) requires matching store prefix17802026/11/12 16:57:11 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)17812026/11/12 16:57:11 INFO Uploading 7n6q36pa1c0a4gp99s9lssqaym6gs2ym-ca-test (144B)1782--- PASS: TestService_healthCheckHandler (0.23s)1783=== CONT TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1784--- PASS: TestClientWithDependencies (0.59s)1785=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed17862026/11/12 16:57:11 WARN Failed to register uploaded object key=7n6q36pa1c0a4gp99s9lssqaym6gs2ym.ls error="server returned 404: 404 page not found\n"17872026/11/12 16:57:11 WARN Failed to register uploaded object key=log/h9a410idz2vs1g2yhwkf5mvj9yal0iil-ca-test.drv error="server returned 404: 404 page not found\n"17882026/11/12 16:57:11 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign17892026/11/12 16:57:11 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"17902026/11/12 16:57:11 INFO Signed narinfos id=1 count=117912026/11/12 16:57:11 INFO Uploading 1 narinfos17922026/11/12 16:57:11 INFO Received complete push request method=POST path=/api/pushes/1/complete17932026/11/12 16:57:11 WARN Failed to register uploaded object key=7n6q36pa1c0a4gp99s9lssqaym6gs2ym.narinfo error="server returned 404: 404 page not found\n"17942026/11/12 16:57:11 INFO Completed push id=1 objects=4 duration=7.309848ms17952026/11/12 16:57:11 INFO Upload complete. (101ms)1796=== NAME TestClientCADerivations1797 client_ca_test.go:180: Narinfo contains CA field: StorePath: /build/TestClientCADerivations966050048/001/store/7n6q36pa1c0a4gp99s9lssqaym6gs2ym-ca-test1798 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1799 Compression: zstd1800 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1801 NarSize: 1441802 References: 1803 Deriver: /build/TestClientCADerivations966050048/001/store/h9a410idz2vs1g2yhwkf5mvj9yal0iil-ca-test.drv1804 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1805 client_ca_test.go:185: Checking for realisation files in S3...1806 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1807 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache18082026/11/12 16:57:11 INFO Installed lifecycle rule aborting incomplete NAR uploads bucket=bucket88 days=21809--- PASS: TestEnsureAbortUploadsRule (0.26s)1810=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed18112026/11/12 16:57:11 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"18122026/11/12 16:57:11 WARN mTLS auth: bound subjects configured but subject DN unavailable18132026/11/12 16:57:11 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1814--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (0.28s)1815=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed18162026/11/12 16:57:11 ERROR Failed to decode narinfo key=3hcdxyjf9yiq7qf3i4548drb6sjmwa1v.narinfo limit=16777216 error=<nil>1817--- PASS: TestReadProxyNarinfo (0.33s)1818=== CONT TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed18192026/11/12 16:57:11 INFO Received push request method=POST path=/api/pushes18202026/11/12 16:57:11 INFO Object in database but missing from S3 key=acdmyb3ksvqgghmr2aydxfdfayfzvdfm.ls18212026/11/12 16:57:11 WARN Found objects in DB but missing from S3, will re-upload count=118222026/11/12 16:57:11 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)18232026/11/12 16:57:11 INFO Received complete push request method=POST path=/api/pushes/2/complete18242026/11/12 16:57:11 WARN Failed to register uploaded object key=acdmyb3ksvqgghmr2aydxfdfayfzvdfm.ls error="server returned 404: 404 page not found\n"18252026/11/12 16:57:11 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"1826--- PASS: TestService_AuthMiddleware (0.27s)1827=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost18282026/11/12 16:57:11 INFO Completed push id=2 objects=3 duration=2.310862ms18292026/11/12 16:57:11 INFO Upload complete. (54ms)1830=== NAME TestClientIntegration1831 client_integration_test.go:336: Retrieved .ls file from S3 (compressed size: 62 bytes)1832 client_integration_test.go:336: Decompressed .ls content (49 bytes):1833 {"version":1,"root":{"type":"regular","size":39}}1834=== NAME TestNARDeduplicationMetadataUploadBug1835 metadata_upload_test.go:48: First store path: /build/TestNARDeduplicationMetadataUploadBug2882856396/001/store/dv3ib3b3m552fw1vr0cry01zvbsnp29y-file1.txt1836=== NAME TestClientCADerivations1837 client_ca_test.go:258: nix copy output: warning: you don't have Internet access; disabling some network-dependent features1838 warning: failed to create TLS context for AWS credential providers; SSO, STS WebIdentity, and ECS container authentication will be unavailable1839 error: binary cache 's3://bucket77?endpoint=http://localhost:43373®ion=eu-west-1' is for Nix stores with prefix '/nix/store', not '/build/TestClientCADerivations966050048/001/store'1840 client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 11841--- PASS: TestClientCADerivations (0.70s)1842=== CONT TestParseSingleRange/none1843=== CONT TestParseSingleRange/open-ended1844=== CONT TestParseSingleRange/start_far_past_EOF1845=== CONT TestParseSingleRange/start_past_EOF1846=== CONT TestParseSingleRange/single_byte1847=== CONT TestParseSingleRange/suffix_exceeds_size1848=== CONT TestParseSingleRange/suffix1849=== CONT TestParseSingleRange/end_clamped_to_size1850=== CONT TestParseSingleRange/malformed_both_empty1851=== CONT TestParseSingleRange/closed1852=== CONT TestParseSingleRange/malformed_end_before_start1853=== CONT TestParseSingleRange/multi-range_ignored1854=== CONT TestParseSingleRange/malformed_no_dash1855=== CONT TestParseSingleRange/unknown_unit1856--- PASS: TestParseSingleRange (0.00s)1857 --- PASS: TestParseSingleRange/none (0.00s)1858 --- PASS: TestParseSingleRange/open-ended (0.00s)1859 --- PASS: TestParseSingleRange/start_far_past_EOF (0.00s)1860 --- PASS: TestParseSingleRange/start_past_EOF (0.00s)1861 --- PASS: TestParseSingleRange/single_byte (0.00s)1862 --- PASS: TestParseSingleRange/suffix_exceeds_size (0.00s)1863 --- PASS: TestParseSingleRange/suffix (0.00s)1864 --- PASS: TestParseSingleRange/end_clamped_to_size (0.00s)1865 --- PASS: TestParseSingleRange/malformed_both_empty (0.00s)1866 --- PASS: TestParseSingleRange/closed (0.00s)1867 --- PASS: TestParseSingleRange/malformed_end_before_start (0.00s)1868 --- PASS: TestParseSingleRange/multi-range_ignored (0.00s)1869 --- PASS: TestParseSingleRange/malformed_no_dash (0.00s)1870 --- PASS: TestParseSingleRange/unknown_unit (0.00s)1871=== CONT TestIsValidUploadKey/narinfo1872=== CONT TestIsValidUploadKey/realisation_plus_in_output1873=== CONT TestIsValidUploadKey/unknown_type1874=== CONT TestIsValidUploadKey/empty_key1875=== CONT TestIsValidUploadKey/absolute1876=== CONT TestIsValidUploadKey/traversal_nar1877=== CONT TestIsValidUploadKey/traversal1878=== CONT TestIsValidUploadKey/listing_key,_narinfo_type1879=== CONT TestIsValidUploadKey/nar_key,_narinfo_type1880=== CONT TestIsValidUploadKey/narinfo_key,_nar_type1881=== CONT TestIsValidUploadKey/index.html1882=== CONT TestIsValidUploadKey/nix-cache-info1883=== CONT TestIsValidUploadKey/nar_plain1884=== CONT TestIsValidUploadKey/build_log1885=== CONT TestIsValidUploadKey/listing1886=== CONT TestIsValidUploadKey/build_log_equals1887=== CONT TestIsValidUploadKey/realisation1888=== CONT TestIsValidUploadKey/nar_xz1889=== CONT TestIsValidUploadKey/nar_zst1890=== CONT TestIsValidUploadKey/build_log_question_mark1891=== CONT TestIsValidUploadKey/build_log_plus_in_name1892=== CONT TestIsValidUploadKey/build_log_home-manager_file1893--- PASS: TestIsValidUploadKey (0.00s)1894 --- PASS: TestIsValidUploadKey/narinfo (0.00s)1895 --- PASS: TestIsValidUploadKey/realisation_plus_in_output (0.00s)1896 --- PASS: TestIsValidUploadKey/unknown_type (0.00s)1897 --- PASS: TestIsValidUploadKey/empty_key (0.00s)1898 --- PASS: TestIsValidUploadKey/absolute (0.00s)1899 --- PASS: TestIsValidUploadKey/traversal_nar (0.00s)1900 --- PASS: TestIsValidUploadKey/traversal (0.00s)1901 --- PASS: TestIsValidUploadKey/listing_key,_narinfo_type (0.00s)1902 --- PASS: TestIsValidUploadKey/nar_key,_narinfo_type (0.00s)1903 --- PASS: TestIsValidUploadKey/narinfo_key,_nar_type (0.00s)1904 --- PASS: TestIsValidUploadKey/index.html (0.00s)1905 --- PASS: TestIsValidUploadKey/nix-cache-info (0.00s)1906 --- PASS: TestIsValidUploadKey/nar_plain (0.00s)1907 --- PASS: TestIsValidUploadKey/build_log (0.00s)1908 --- PASS: TestIsValidUploadKey/listing (0.00s)1909 --- PASS: TestIsValidUploadKey/build_log_equals (0.00s)1910 --- PASS: TestIsValidUploadKey/realisation (0.00s)1911 --- PASS: TestIsValidUploadKey/nar_xz (0.00s)1912 --- PASS: TestIsValidUploadKey/nar_zst (0.00s)1913 --- PASS: TestIsValidUploadKey/build_log_question_mark (0.00s)1914 --- PASS: TestIsValidUploadKey/build_log_plus_in_name (0.00s)1915 --- PASS: TestIsValidUploadKey/build_log_home-manager_file (0.00s)1916=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts19172026/11/12 16:57:11 INFO Received request for more parts method=POST path=/1918--- PASS: TestMetricsInventory (0.29s)1919=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart19202026/11/12 16:57:11 INFO Received complete multipart upload request method=POST path=/19212026/11/12 16:57:11 INFO Received push request method=POST path=/api/pushes19222026/11/12 16:57:11 INFO Received complete push request method=POST path=/complete19232026/11/12 16:57:11 INFO Completed push id=1 objects=2 duration=7.924334ms19242026/11/12 16:57:11 INFO Received complete push request method=POST path=/complete19252026/11/12 16:57:11 INFO Completed push id=1 objects=0 duration=565.725µs19262026/11/12 16:57:11 INFO Received cleanup request method=DELETE path=/api/pending_closures19272026/11/12 16:57:11 INFO Aborted multipart uploads count=0 kept=019282026/11/12 16:57:11 INFO Received complete push request method=POST path=/complete19292026-11-12 16:57:11.363 UTC [1431] ERROR: Push does not exist: id=119302026-11-12 16:57:11.363 UTC [1431] CONTEXT: PL/pgSQL function commit_push(bigint) line 15 at RAISE19312026-11-12 16:57:11.363 UTC [1431] STATEMENT: -- name: CommitPush :one1932 SELECT commit_push($1::bigint)::bigint1933 19342026/11/12 16:57:11 WARN Push not found id=11935=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure19362026/11/12 16:57:11 INFO Received uploads request method=POST path=/19372026/11/12 16:57:11 INFO Received uploads request method=POST path=/api/pending_closures19382026/11/12 16:57:11 INFO Received complete upload request method=POST path=/complete19392026/11/12 16:57:11 INFO Completed upload id=1 objects=2 duration=4.27148ms19402026/11/12 16:57:11 INFO Received complete upload request method=POST path=/complete19412026/11/12 16:57:11 INFO Completed upload id=1 objects=0 duration=367.764µs19422026/11/12 16:57:11 INFO Received cleanup request method=DELETE path=/api/pending_closures19432026/11/12 16:57:11 INFO Aborted multipart uploads count=0 kept=019442026/11/12 16:57:11 INFO Received complete upload request method=POST path=/complete19452026-11-12 16:57:11.381 UTC [1433] ERROR: Closure does not exist: id=119462026-11-12 16:57:11.381 UTC [1433] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 17 at RAISE19472026-11-12 16:57:11.381 UTC [1433] STATEMENT: -- name: CommitPendingClosure :one1948 SELECT commit_pending_closure($1::bigint)::bigint1949 1950--- PASS: TestCommitCanBeRepeatedUntilCleanup (0.00s)1951 --- PASS: TestCommitCanBeRepeatedUntilCleanup/push (0.32s)1952 --- PASS: TestCommitCanBeRepeatedUntilCleanup/pending_closure (0.31s)1953=== CONT TestForceGCDuringPushOffersSweptObject/before_pending_rows1954=== CONT TestForceGCDuringPushOffersSweptObject/after_presence_check19552026/11/12 16:57:11 INFO Received push request method=POST path=/api/pushes19562026/11/12 16:57:11 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)19572026/11/12 16:57:11 INFO Received push request method=POST path=/api/pushes1958=== CONT TestServerTLSConfig/no_client_CA19592026/11/12 16:57:11 INFO Received complete push request method=POST path=/api/pushes/3/complete1960=== CONT TestServerTLSConfig/not_a_PEM_file1961=== CONT TestServerTLSConfig/missing_CA_file1962=== CONT TestPresentNotReportedWhileGCDeletesClosure/stale19632026/11/12 16:57:11 WARN Failed to register uploaded object key=acdmyb3ksvqgghmr2aydxfdfayfzvdfm.ls error="server returned 404: 404 page not found\n"1964--- PASS: TestServerTLSConfig (0.00s)1965 --- PASS: TestServerTLSConfig/no_client_CA (0.00s)1966 --- PASS: TestServerTLSConfig/missing_CA_file (0.00s)1967 --- PASS: TestServerTLSConfig/not_a_PEM_file (0.00s)1968=== CONT TestPresentNotReportedWhileGCDeletesClosure/fresh19692026/11/12 16:57:11 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)19702026/11/12 16:57:11 INFO Uploading dv3ib3b3m552fw1vr0cry01zvbsnp29y-file1.txt (160B)19712026/11/12 16:57:11 INFO Completed push id=3 objects=3 duration=2.167241ms19722026/11/12 16:57:11 INFO Upload complete. (46ms)1973=== NAME TestClientIntegration1974 client_integration_test.go:353: Retrieved .ls file from S3 (compressed size: 62 bytes)1975 client_integration_test.go:353: Decompressed .ls content (49 bytes):1976 {"version":1,"root":{"type":"regular","size":39}}1977 client_integration_test.go:364: Testing garbage collection...19782026/11/12 16:57:11 WARN Failed to register uploaded object key=dv3ib3b3m552fw1vr0cry01zvbsnp29y.ls error="server returned 404: 404 page not found\n"19792026/11/12 16:57:11 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign19802026/11/12 16:57:11 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"19812026/11/12 16:57:11 INFO Signed narinfos id=1 count=119822026/11/12 16:57:11 INFO Uploading 1 narinfos19832026/11/12 16:57:11 INFO Received complete push request method=POST path=/api/pushes/1/complete19842026/11/12 16:57:11 WARN Failed to register uploaded object key=dv3ib3b3m552fw1vr0cry01zvbsnp29y.narinfo error="server returned 404: 404 page not found\n"19852026/11/12 16:57:11 INFO Received uploads request method=POST path=/api/pending_closures19862026/11/12 16:57:11 INFO Completed push id=1 objects=3 duration=6.804263ms19872026/11/12 16:57:11 INFO Upload complete. (63ms)1988=== NAME TestNARDeduplicationMetadataUploadBug1989 metadata_upload_test.go:54: Retrieved narinfo from S3:1990 StorePath: /build/TestNARDeduplicationMetadataUploadBug2882856396/001/store/dv3ib3b3m552fw1vr0cry01zvbsnp29y-file1.txt1991 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1992 Compression: zstd1993 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1994 NarSize: 1601995 References: 1996 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1997 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)1998 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):1999 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}20002026/11/12 16:57:11 INFO Starting cleanup of old closures method=DELETE path=/api/closures
loading 936 hidden lines…
29372026/11/12 16:57:24 INFO Received uploads request method=POST path=/29382026/11/12 16:57:24 INFO Received push request method=POST path=/29392026/11/12 16:57:24 INFO Received complete push request method=POST path=/29402026/11/12 16:57:24 INFO Completed push id=4 objects=7 duration=1.506934ms29412026/11/12 16:57:24 INFO Received uploads request method=POST path=/29422026/11/12 16:57:24 INFO Received complete upload request method=POST path=/29432026/11/12 16:57:24 INFO Received complete push request method=POST path=/29442026/11/12 16:57:24 INFO Received complete push request method=POST path=/29452026/11/12 16:57:24 INFO Completed push id=9 objects=5 duration=1.352513ms29462026/11/12 16:57:24 INFO Completed push id=4 objects=4 duration=1.594095ms29472026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=029482026/11/12 16:57:24 INFO Completed upload id=3 objects=6 duration=3.945317ms29492026/11/12 16:57:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=029502026/11/12 16:57:24 INFO Vacuumed table table=pending_closures29512026/11/12 16:57:24 WARN Force mode enabled - objects will be deleted immediately without grace period29522026/11/12 16:57:24 INFO Received push request method=POST path=/29532026/11/12 16:57:24 INFO Vacuumed table table=commit_receipts29542026/11/12 16:57:24 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029552026/11/12 16:57:24 INFO Vacuumed table table=pending_objects29562026/11/12 16:57:24 INFO Vacuumed table table=multipart_uploads29572026/11/12 16:57:24 INFO Vacuumed table table=pending_closures29582026/11/12 16:57:24 INFO Vacuumed table table=commit_receipts29592026/11/12 16:57:24 INFO Vacuumed table table=closures29602026/11/12 16:57:24 INFO Vacuumed table table=pending_objects29612026/11/12 16:57:24 INFO Vacuumed table table=objects29622026/11/12 16:57:24 INFO Vacuumed table table=multipart_uploads29632026/11/12 16:57:24 INFO Vacuumed table table=closures29642026/11/12 16:57:24 INFO Vacuumed table table=objects29652026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=029662026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=029672026/11/12 16:57:24 WARN Force mode enabled - objects will be deleted immediately without grace period29682026/11/12 16:57:24 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029692026/11/12 16:57:24 INFO Received uploads request method=POST path=/29702026/11/12 16:57:24 INFO Vacuumed table table=pending_closures29712026/11/12 16:57:24 INFO Received complete upload request method=POST path=/29722026/11/12 16:57:24 INFO Vacuumed table table=commit_receipts29732026/11/12 16:57:24 INFO Received push request method=POST path=/29742026/11/12 16:57:24 INFO Received complete upload request method=POST path=/29752026/11/12 16:57:24 INFO Vacuumed table table=pending_objects29762026/11/12 16:57:24 INFO Vacuumed table table=multipart_uploads29772026/11/12 16:57:24 INFO Completed upload id=3 objects=24 duration=1.120811ms29782026/11/12 16:57:24 INFO Received push request method=POST path=/29792026/11/12 16:57:24 INFO Received complete upload request method=POST path=/29802026/11/12 16:57:24 INFO Received uploads request method=POST path=/29812026/11/12 16:57:24 INFO Vacuumed table table=closures29822026/11/12 16:57:24 INFO Completed upload id=2 objects=26 duration=4.2632ms29832026/11/12 16:57:24 INFO Received uploads request method=POST path=/29842026/11/12 16:57:24 INFO Vacuumed table table=objects29852026/11/12 16:57:24 INFO Completed upload id=6 objects=24 duration=2.834407ms29862026/11/12 16:57:24 INFO Received push request method=POST path=/29872026/11/12 16:57:24 INFO Received uploads request method=POST path=/29882026/11/12 16:57:24 INFO Received push request method=POST path=/29892026/11/12 16:57:24 INFO Received complete push request method=POST path=/29902026/11/12 16:57:24 INFO Received push request method=POST path=/29912026/11/12 16:57:24 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo error="Access Denied."29922026/11/12 16:57:24 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."29932026/11/12 16:57:24 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."29942026/11/12 16:57:24 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."29952026/11/12 16:57:24 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."29962026/11/12 16:57:24 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."29972026/11/12 16:57:24 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."29982026/11/12 16:57:24 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."29992026/11/12 16:57:24 ERROR failed to remove object object=nar/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.nar.zst error="Access Denied."30002026/11/12 16:57:24 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."30012026/11/12 16:57:24 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."30022026/11/12 16:57:24 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."30032026/11/12 16:57:24 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."30042026/11/12 16:57:24 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."30052026/11/12 16:57:24 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."30062026/11/12 16:57:24 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."30072026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=030082026/11/12 16:57:24 INFO Completed push id=2 objects=25 duration=2.565604ms30092026/11/12 16:57:24 INFO Received uploads request method=POST path=/30102026/11/12 16:57:24 INFO Received push request method=POST path=/30112026/11/12 16:57:24 WARN Force mode enabled - objects will be deleted immediately without grace period30122026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=030132026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=030142026/11/12 16:57:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030152026/11/12 16:57:24 INFO Vacuumed table table=pending_closures30162026/11/12 16:57:24 INFO Received push request method=POST path=/30172026/11/12 16:57:24 INFO Vacuumed table table=commit_receipts30182026/11/12 16:57:24 INFO Vacuumed table table=pending_objects30192026/11/12 16:57:24 INFO Vacuumed table table=multipart_uploads30202026/11/12 16:57:24 INFO Vacuumed table table=closures30212026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=030222026/11/12 16:57:24 INFO Received complete upload request method=POST path=/30232026/11/12 16:57:24 INFO Vacuumed table table=objects30242026/11/12 16:57:24 INFO Received uploads request method=POST path=/30252026/11/12 16:57:24 INFO Completed upload id=11 objects=14 duration=1.359233ms30262026/11/12 16:57:24 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."30272026/11/12 16:57:24 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."30282026/11/12 16:57:24 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."30292026/11/12 16:57:24 INFO Received push request method=POST path=/30302026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=030312026/11/12 16:57:24 INFO Received push request method=POST path=/30322026/11/12 16:57:24 WARN Force mode enabled - objects will be deleted immediately without grace period30332026/11/12 16:57:24 INFO Received complete push request method=POST path=/30342026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=030352026/11/12 16:57:24 INFO Completed push id=8 objects=6 duration=1.683316ms30362026/11/12 16:57:24 INFO Received push request method=POST path=/30372026/11/12 16:57:24 WARN Force mode enabled - objects will be deleted immediately without grace period30382026/11/12 16:57:24 INFO Received push request method=POST path=/30392026/11/12 16:57:24 INFO Received push request method=POST path=/30402026/11/12 16:57:24 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030412026/11/12 16:57:24 INFO Vacuumed table table=pending_closures30422026/11/12 16:57:24 INFO Vacuumed table table=commit_receipts30432026/11/12 16:57:24 INFO Vacuumed table table=pending_objects30442026/11/12 16:57:24 INFO Vacuumed table table=multipart_uploads30452026/11/12 16:57:24 INFO Vacuumed table table=closures30462026/11/12 16:57:24 INFO Received push request method=POST path=/30472026/11/12 16:57:24 INFO Vacuumed table table=objects30482026/11/12 16:57:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=030492026/11/12 16:57:24 INFO Received push request method=POST path=/30502026/11/12 16:57:24 INFO Vacuumed table table=pending_closures30512026/11/12 16:57:24 INFO Vacuumed table table=commit_receipts30522026/11/12 16:57:24 INFO Vacuumed table table=pending_objects30532026/11/12 16:57:24 INFO Vacuumed table table=multipart_uploads30542026/11/12 16:57:24 INFO Vacuumed table table=closures30552026/11/12 16:57:24 INFO Vacuumed table table=objects30562026/11/12 16:57:24 INFO Received push request method=POST path=/30572026/11/12 16:57:24 INFO Received push request method=POST path=/30582026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=030592026/11/12 16:57:24 INFO Received complete push request method=POST path=/30602026/11/12 16:57:24 INFO Received push request method=POST path=/30612026/11/12 16:57:24 INFO Received complete upload request method=POST path=/30622026/11/12 16:57:24 INFO Received uploads request method=POST path=/30632026/11/12 16:57:24 INFO Completed push id=5 objects=3 duration=1.539915ms30642026/11/12 16:57:24 INFO Completed upload id=9 objects=3 duration=1.10855ms30652026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=030662026/11/12 16:57:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030672026/11/12 16:57:24 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."30682026/11/12 16:57:24 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."30692026/11/12 16:57:24 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."30702026/11/12 16:57:24 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."30712026/11/12 16:57:24 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."30722026/11/12 16:57:24 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."30732026/11/12 16:57:24 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."30742026/11/12 16:57:24 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."30752026/11/12 16:57:24 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."30762026/11/12 16:57:24 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."30772026/11/12 16:57:24 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."30782026/11/12 16:57:24 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."30792026/11/12 16:57:24 INFO Received uploads request method=POST path=/30802026/11/12 16:57:24 INFO Vacuumed table table=pending_closures30812026/11/12 16:57:24 WARN Force mode enabled - objects will be deleted immediately without grace period30822026/11/12 16:57:24 INFO Vacuumed table table=commit_receipts30832026/11/12 16:57:24 INFO Received complete push request method=POST path=/30842026/11/12 16:57:24 INFO Vacuumed table table=pending_objects30852026/11/12 16:57:24 INFO Vacuumed table table=multipart_uploads30862026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=030872026/11/12 16:57:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030882026/11/12 16:57:24 INFO Completed push id=7 objects=20 duration=1.493974ms30892026/11/12 16:57:24 INFO Vacuumed table table=closures30902026/11/12 16:57:24 INFO Vacuumed table table=pending_closures30912026/11/12 16:57:24 INFO Vacuumed table table=objects30922026/11/12 16:57:24 INFO Vacuumed table table=commit_receipts30932026/11/12 16:57:24 INFO Vacuumed table table=pending_objects30942026/11/12 16:57:24 INFO Vacuumed table table=multipart_uploads30952026/11/12 16:57:24 INFO Vacuumed table table=closures30962026/11/12 16:57:24 INFO Vacuumed table table=objects30972026/11/12 16:57:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030982026/11/12 16:57:24 INFO Received uploads request method=POST path=/30992026/11/12 16:57:24 INFO Vacuumed table table=pending_closures31002026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=031012026/11/12 16:57:24 INFO Received uploads request method=POST path=/31022026/11/12 16:57:24 INFO Vacuumed table table=commit_receipts31032026/11/12 16:57:24 INFO Vacuumed table table=pending_objects31042026/11/12 16:57:24 INFO Vacuumed table table=multipart_uploads31052026/11/12 16:57:24 INFO Received push request method=POST path=/31062026/11/12 16:57:24 INFO Received push request method=POST path=/31072026/11/12 16:57:24 INFO Vacuumed table table=closures31082026/11/12 16:57:24 INFO Vacuumed table table=objects31092026/11/12 16:57:24 WARN Force mode enabled - objects will be deleted immediately without grace period31102026/11/12 16:57:24 INFO Received complete push request method=POST path=/31112026/11/12 16:57:24 INFO Completed push id=10 objects=2 duration=1.673916ms31122026/11/12 16:57:24 INFO Received push request method=POST path=/31132026/11/12 16:57:24 INFO Received push request method=POST path=/31142026/11/12 16:57:24 INFO Received push request method=POST path=/31152026/11/12 16:57:24 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."31162026/11/12 16:57:24 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."31172026/11/12 16:57:24 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."31182026/11/12 16:57:24 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."31192026/11/12 16:57:24 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."31202026/11/12 16:57:24 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."31212026/11/12 16:57:24 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."31222026/11/12 16:57:24 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."31232026/11/12 16:57:24 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."31242026/11/12 16:57:24 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."31252026/11/12 16:57:24 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."31262026/11/12 16:57:24 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."31272026/11/12 16:57:24 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."31282026/11/12 16:57:24 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."31292026/11/12 16:57:24 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."31302026/11/12 16:57:24 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."31312026/11/12 16:57:24 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."31322026/11/12 16:57:24 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."31332026/11/12 16:57:24 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."31342026/11/12 16:57:24 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."31352026/11/12 16:57:24 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."31362026/11/12 16:57:24 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."31372026/11/12 16:57:24 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."31382026/11/12 16:57:24 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."31392026/11/12 16:57:24 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."31402026/11/12 16:57:24 INFO Received uploads request method=POST path=/31412026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=031422026/11/12 16:57:24 WARN Force mode enabled - objects will be deleted immediately without grace period31432026/11/12 16:57:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031442026/11/12 16:57:24 INFO Received push request method=POST path=/31452026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=031462026/11/12 16:57:24 INFO Received push request method=POST path=/31472026/11/12 16:57:24 INFO Vacuumed table table=pending_closures31482026/11/12 16:57:24 INFO Vacuumed table table=commit_receipts31492026/11/12 16:57:24 INFO Received push request method=POST path=/31502026/11/12 16:57:24 INFO Received uploads request method=POST path=/31512026/11/12 16:57:24 INFO Vacuumed table table=pending_objects31522026/11/12 16:57:24 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."31532026/11/12 16:57:24 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."31542026/11/12 16:57:24 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."31552026/11/12 16:57:24 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."31562026/11/12 16:57:24 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."31572026/11/12 16:57:24 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."31582026/11/12 16:57:24 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."31592026/11/12 16:57:24 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."31602026/11/12 16:57:24 INFO Vacuumed table table=multipart_uploads31612026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=031622026/11/12 16:57:24 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."31632026/11/12 16:57:24 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."31642026/11/12 16:57:24 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."31652026/11/12 16:57:24 INFO Received uploads request method=POST path=/31662026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=031672026/11/12 16:57:24 INFO Vacuumed table table=closures31682026/11/12 16:57:24 INFO Received complete upload request method=POST path=/31692026/11/12 16:57:24 INFO Vacuumed table table=objects31702026/11/12 16:57:24 INFO Received complete push request method=POST path=/31712026/11/12 16:57:24 INFO Completed upload id=12 objects=21 duration=1.08515ms31722026/11/12 16:57:24 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031732026/11/12 16:57:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=4 objects-deleted-after-grace-period=0 objects-failed-to-delete=031742026/11/12 16:57:24 INFO Vacuumed table table=pending_closures31752026/11/12 16:57:24 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."31762026/11/12 16:57:24 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."31772026/11/12 16:57:24 INFO Vacuumed table table=commit_receipts31782026/11/12 16:57:24 INFO Completed push id=2 objects=7 duration=3.430652ms31792026/11/12 16:57:24 INFO Vacuumed table table=pending_closures31802026/11/12 16:57:24 INFO Vacuumed table table=commit_receipts31812026/11/12 16:57:24 INFO Vacuumed table table=pending_objects31822026/11/12 16:57:24 INFO Received uploads request method=POST path=/31832026/11/12 16:57:24 INFO Vacuumed table table=multipart_uploads31842026/11/12 16:57:24 INFO Received push request method=POST path=/31852026/11/12 16:57:24 INFO Vacuumed table table=closures31862026/11/12 16:57:24 INFO Vacuumed table table=pending_objects31872026/11/12 16:57:24 INFO Vacuumed table table=objects31882026/11/12 16:57:24 INFO Vacuumed table table=multipart_uploads31892026/11/12 16:57:24 INFO Received push request method=POST path=/31902026/11/12 16:57:24 INFO Received uploads request method=POST path=/31912026/11/12 16:57:24 INFO Received push request method=POST path=/31922026/11/12 16:57:24 INFO Vacuumed table table=closures31932026/11/12 16:57:24 INFO Vacuumed table table=objects31942026/11/12 16:57:24 INFO Aborted multipart uploads count=0 kept=031952026/11/12 16:57:24 INFO Received complete upload request method=POST path=/31962026/11/12 16:57:24 INFO Received complete push request method=POST path=/31972026/11/12 16:57:24 INFO Received push request method=POST path=/31982026/11/12 16:57:24 INFO Completed upload id=5 objects=7 duration=1.875377ms31992026/11/12 16:57:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032002026/11/12 16:57:24 INFO Received push request method=POST path=/32012026/11/12 16:57:24 INFO Received uploads request method=POST path=/32022026/11/12 16:57:24 INFO Vacuumed table table=pending_closures32032026/11/12 16:57:24 INFO Completed push id=11 objects=11 duration=3.23707ms32042026/11/12 16:57:24 INFO Vacuumed table table=commit_receipts32052026/11/12 16:57:24 INFO Vacuumed table table=pending_objects32062026/11/12 16:57:24 INFO Vacuumed table table=multipart_uploads32072026/11/12 16:57:24 INFO Vacuumed table table=closures32082026/11/12 16:57:25 INFO Vacuumed table table=objects32092026/11/12 16:57:25 INFO Received push request method=POST path=/32102026/11/12 16:57:25 INFO Received push request method=POST path=/32112026/11/12 16:57:25 INFO Received complete push request method=POST path=/32122026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=032132026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=032142026/11/12 16:57:25 INFO Received uploads request method=POST path=/32152026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=032162026/11/12 16:57:25 INFO Completed push id=5 objects=3 duration=2.656024ms32172026/11/12 16:57:25 INFO Received complete push request method=POST path=/32182026/11/12 16:57:25 INFO Completed push id=8 objects=2 duration=1.569614ms32192026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=032202026/11/12 16:57:25 INFO Vacuumed table table=pending_closures32212026/11/12 16:57:25 INFO Received push request method=POST path=/32222026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032232026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts32242026/11/12 16:57:25 INFO Received uploads request method=POST path=/32252026/11/12 16:57:25 INFO Vacuumed table table=pending_closures32262026/11/12 16:57:25 INFO Received uploads request method=POST path=/32272026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts32282026/11/12 16:57:25 INFO Vacuumed table table=pending_objects32292026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads32302026/11/12 16:57:25 INFO Received push request method=POST path=/32312026/11/12 16:57:25 INFO Received uploads request method=POST path=/32322026/11/12 16:57:25 INFO Vacuumed table table=pending_objects32332026/11/12 16:57:25 INFO Received complete upload request method=POST path=/32342026/11/12 16:57:25 INFO Received complete push request method=POST path=/32352026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads32362026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=032372026/11/12 16:57:25 INFO Vacuumed table table=closures32382026/11/12 16:57:25 INFO Vacuumed table table=closures32392026/11/12 16:57:25 INFO Vacuumed table table=pending_closures32402026/11/12 16:57:25 INFO Completed upload id=6 objects=9 duration=1.349372ms32412026/11/12 16:57:25 INFO Vacuumed table table=objects32422026/11/12 16:57:25 INFO Received uploads request method=POST path=/32432026/11/12 16:57:25 INFO Completed push id=13 objects=23 duration=2.057059ms32442026/11/12 16:57:25 INFO Vacuumed table table=objects32452026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts32462026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=032472026/11/12 16:57:25 INFO Vacuumed table table=pending_objects32482026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads32492026/11/12 16:57:25 INFO Vacuumed table table=closures32502026/11/12 16:57:25 INFO Vacuumed table table=objects32512026/11/12 16:57:25 INFO Received complete push request method=POST path=/32522026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032532026/11/12 16:57:25 INFO Received push request method=POST path=/32542026/11/12 16:57:25 INFO Vacuumed table table=pending_closures32552026/11/12 16:57:25 INFO Completed push id=14 objects=3 duration=2.07314ms32562026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts32572026/11/12 16:57:25 INFO Vacuumed table table=pending_objects32582026/11/12 16:57:25 INFO Received complete push request method=POST path=/32592026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads32602026/11/12 16:57:25 INFO Vacuumed table table=closures32612026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=032622026/11/12 16:57:25 INFO Completed push id=12 objects=3 duration=1.356753ms32632026/11/12 16:57:25 INFO Vacuumed table table=objects32642026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period32652026/11/12 16:57:25 INFO Received complete push request method=POST path=/32662026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032672026/11/12 16:57:25 INFO Vacuumed table table=pending_closures32682026/11/12 16:57:25 INFO Received push request method=POST path=/32692026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts32702026/11/12 16:57:25 INFO Vacuumed table table=pending_objects32712026/11/12 16:57:25 INFO Completed push id=15 objects=24 duration=2.910007ms32722026/11/12 16:57:25 INFO Received push request method=POST path=/32732026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads32742026/11/12 16:57:25 INFO Vacuumed table table=closures32752026/11/12 16:57:25 INFO Vacuumed table table=objects32762026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=032772026/11/12 16:57:25 INFO Received push request method=POST path=/32782026/11/12 16:57:25 INFO Received complete push request method=POST path=/32792026/11/12 16:57:25 INFO Completed push id=4 objects=6 duration=1.384093ms32802026/11/12 16:57:25 INFO Received complete push request method=POST path=/32812026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=032822026/11/12 16:57:25 INFO Received uploads request method=POST path=/32832026/11/12 16:57:25 INFO Received complete push request method=POST path=/32842026/11/12 16:57:25 INFO Vacuumed table table=pending_closures32852026/11/12 16:57:25 INFO Completed push id=9 objects=7 duration=3.007828ms32862026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts32872026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=032882026/11/12 16:57:25 INFO Vacuumed table table=pending_objects32892026/11/12 16:57:25 INFO Received uploads request method=POST path=/32902026/11/12 16:57:25 INFO Completed push id=4 objects=4 duration=2.813927ms32912026/11/12 16:57:25 INFO Received uploads request method=POST path=/32922026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads32932026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period32942026/11/12 16:57:25 INFO Vacuumed table table=closures32952026/11/12 16:57:25 INFO Received push request method=POST path=/32962026/11/12 16:57:25 INFO Vacuumed table table=objects32972026/11/12 16:57:25 INFO Received push request method=POST path=/32982026/11/12 16:57:25 INFO Received complete upload request method=POST path=/32992026/11/12 16:57:25 INFO Received complete push request method=POST path=/33002026/11/12 16:57:25 INFO Completed upload id=10 objects=2 duration=1.982698ms33012026/11/12 16:57:25 INFO Completed push id=5 objects=6 duration=1.223651ms33022026/11/12 16:57:25 INFO Received uploads request method=POST path=/33032026/11/12 16:57:25 INFO Received uploads request method=POST path=/33042026/11/12 16:57:25 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."33052026/11/12 16:57:25 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."33062026/11/12 16:57:25 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."33072026/11/12 16:57:25 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."33082026/11/12 16:57:25 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."33092026/11/12 16:57:25 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."33102026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."33112026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."33122026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."33132026/11/12 16:57:25 INFO Received push request method=POST path=/33142026/11/12 16:57:25 INFO Received uploads request method=POST path=/33152026/11/12 16:57:25 INFO Received push request method=POST path=/33162026/11/12 16:57:25 INFO Received complete upload request method=POST path=/33172026/11/12 16:57:25 INFO Received complete push request method=POST path=/33182026/11/12 16:57:25 INFO Completed upload id=3 objects=5 duration=1.02641ms33192026/11/12 16:57:25 INFO Received push request method=POST path=/33202026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=033212026/11/12 16:57:25 INFO Completed push id=8 objects=6 duration=1.278872ms33222026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=033232026/11/12 16:57:25 INFO Received complete upload request method=POST path=/33242026/11/12 16:57:25 INFO Received complete push request method=POST path=/33252026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=033262026/11/12 16:57:25 INFO Completed upload id=10 objects=7 duration=1.07491ms33272026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period33282026/11/12 16:57:25 INFO Completed push id=6 objects=6 duration=1.577275ms33292026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period33302026/11/12 16:57:25 INFO Received push request method=POST path=/33312026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=033322026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033332026/11/12 16:57:25 INFO Received complete upload request method=POST path=/33342026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=033352026/11/12 16:57:25 INFO Vacuumed table table=pending_closures33362026/11/12 16:57:25 INFO Completed upload id=9 objects=2 duration=1.128411ms33372026/11/12 16:57:25 INFO Received uploads request method=POST path=/33382026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts33392026/11/12 16:57:25 INFO Vacuumed table table=pending_closures33402026/11/12 16:57:25 INFO Vacuumed table table=pending_objects33412026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts33422026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads33432026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=033442026/11/12 16:57:25 INFO Vacuumed table table=closures33452026/11/12 16:57:25 INFO Vacuumed table table=pending_objects33462026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads33472026/11/12 16:57:25 INFO Vacuumed table table=objects33482026/11/12 16:57:25 INFO Vacuumed table table=closures33492026/11/12 16:57:25 INFO Vacuumed table table=objects33502026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=9 objects-failed-to-delete=033512026/11/12 16:57:25 INFO Received uploads request method=POST path=/33522026/11/12 16:57:25 INFO Received uploads request method=POST path=/33532026/11/12 16:57:25 INFO Vacuumed table table=pending_closures33542026/11/12 16:57:25 INFO Received push request method=POST path=/33552026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts33562026/11/12 16:57:25 INFO Vacuumed table table=pending_objects33572026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads33582026/11/12 16:57:25 INFO Vacuumed table table=closures33592026/11/12 16:57:25 INFO Received complete push request method=POST path=/33602026/11/12 16:57:25 INFO Vacuumed table table=objects33612026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=033622026/11/12 16:57:25 INFO Received uploads request method=POST path=/33632026/11/12 16:57:25 INFO Completed push id=8 objects=3 duration=1.519935ms33642026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=033652026/11/12 16:57:25 INFO Vacuumed table table=pending_closures33662026/11/12 16:57:25 INFO Received push request method=POST path=/33672026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts33682026/11/12 16:57:25 INFO Vacuumed table table=pending_closures33692026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts33702026/11/12 16:57:25 INFO Vacuumed table table=pending_objects33712026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads33722026/11/12 16:57:25 INFO Vacuumed table table=pending_objects33732026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads33742026/11/12 16:57:25 INFO Vacuumed table table=closures33752026/11/12 16:57:25 INFO Vacuumed table table=closures33762026/11/12 16:57:25 INFO Vacuumed table table=objects33772026/11/12 16:57:25 INFO Vacuumed table table=objects33782026/11/12 16:57:25 INFO Received uploads request method=POST path=/33792026/11/12 16:57:25 INFO Received push request method=POST path=/33802026/11/12 16:57:25 INFO Received push request method=POST path=/33812026/11/12 16:57:25 INFO Received complete upload request method=POST path=/33822026/11/12 16:57:25 INFO Received push request method=POST path=/33832026/11/12 16:57:25 INFO Completed upload id=4 objects=14 duration=2.409802ms33842026/11/12 16:57:25 INFO Received complete upload request method=POST path=/33852026/11/12 16:57:25 INFO Received uploads request method=POST path=/33862026/11/12 16:57:25 INFO Completed upload id=16 objects=23 duration=1.027549ms33872026/11/12 16:57:25 INFO Received complete upload request method=POST path=/33882026/11/12 16:57:25 INFO Received uploads request method=POST path=/33892026/11/12 16:57:25 INFO Completed upload id=6 objects=6 duration=1.188371ms33902026/11/12 16:57:25 INFO Received push request method=POST path=/33912026/11/12 16:57:25 INFO Received uploads request method=POST path=/33922026/11/12 16:57:25 INFO Received uploads request method=POST path=/33932026/11/12 16:57:25 INFO Received push request method=POST path=/33942026/11/12 16:57:25 INFO Received complete push request method=POST path=/33952026/11/12 16:57:25 INFO Received push request method=POST path=/33962026/11/12 16:57:25 INFO Received complete upload request method=POST path=/33972026/11/12 16:57:25 INFO Completed push id=7 objects=27 duration=1.986739ms33982026/11/12 16:57:25 INFO Completed upload id=15 objects=6 duration=1.14377ms33992026/11/12 16:57:25 INFO Received push request method=POST path=/34002026/11/12 16:57:25 INFO Received push request method=POST path=/34012026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=034022026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=034032026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period34042026/11/12 16:57:25 INFO Received complete push request method=POST path=/34052026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=034062026/11/12 16:57:25 INFO Received complete upload request method=POST path=/34072026/11/12 16:57:25 INFO Received complete upload request method=POST path=/34082026/11/12 16:57:25 INFO Received uploads request method=POST path=/34092026/11/12 16:57:25 INFO Received uploads request method=POST path=/34102026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period34112026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period34122026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034132026/11/12 16:57:25 INFO Completed push id=2 objects=10 duration=2.889067ms34142026/11/12 16:57:25 INFO Completed upload id=18 objects=4 duration=1.788717ms34152026/11/12 16:57:25 INFO Completed upload id=13 objects=10 duration=1.221511ms34162026/11/12 16:57:25 INFO Received push request method=POST path=/34172026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034182026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034192026/11/12 16:57:25 INFO Vacuumed table table=pending_closures34202026/11/12 16:57:25 INFO Received uploads request method=POST path=/34212026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts34222026/11/12 16:57:25 INFO Received push request method=POST path=/34232026/11/12 16:57:25 INFO Vacuumed table table=pending_closures34242026/11/12 16:57:25 INFO Vacuumed table table=pending_closures34252026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts34262026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts34272026/11/12 16:57:25 INFO Vacuumed table table=pending_objects34282026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads34292026/11/12 16:57:25 INFO Vacuumed table table=pending_objects34302026/11/12 16:57:25 INFO Vacuumed table table=closures34312026/11/12 16:57:25 INFO Vacuumed table table=pending_objects34322026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads34332026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads34342026/11/12 16:57:25 INFO Vacuumed table table=closures34352026/11/12 16:57:25 INFO Vacuumed table table=objects34362026/11/12 16:57:25 INFO Vacuumed table table=closures34372026/11/12 16:57:25 INFO Vacuumed table table=objects34382026/11/12 16:57:25 INFO Vacuumed table table=objects34392026/11/12 16:57:25 INFO Received uploads request method=POST path=/34402026/11/12 16:57:25 INFO Received complete upload request method=POST path=/34412026/11/12 16:57:25 INFO Received uploads request method=POST path=/34422026/11/12 16:57:25 INFO Completed upload id=3 objects=9 duration=1.13373ms34432026/11/12 16:57:25 INFO Received complete push request method=POST path=/34442026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=034452026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period34462026/11/12 16:57:25 INFO Received push request method=POST path=/34472026/11/12 16:57:25 INFO Received complete upload request method=POST path=/34482026/11/12 16:57:25 INFO Received uploads request method=POST path=/34492026/11/12 16:57:25 INFO Completed push id=11 objects=8 duration=2.701565ms34502026/11/12 16:57:25 INFO Completed upload id=10 objects=23 duration=1.506394ms34512026/11/12 16:57:25 INFO Received push request method=POST path=/34522026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=034532026/11/12 16:57:25 INFO Received complete upload request method=POST path=/34542026/11/12 16:57:25 INFO Received push request method=POST path=/34552026/11/12 16:57:25 INFO Completed upload id=17 objects=7 duration=793.647µs34562026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034572026/11/12 16:57:25 INFO Received complete upload request method=POST path=/34582026/11/12 16:57:25 INFO Vacuumed table table=pending_closures34592026/11/12 16:57:25 INFO Received uploads request method=POST path=/34602026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=034612026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts34622026/11/12 16:57:25 INFO Completed upload id=10 objects=17 duration=1.09203ms34632026/11/12 16:57:25 INFO Received push request method=POST path=/34642026/11/12 16:57:25 INFO Vacuumed table table=pending_objects34652026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads34662026/11/12 16:57:25 INFO Vacuumed table table=closures34672026/11/12 16:57:25 INFO Vacuumed table table=objects34682026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034692026/11/12 16:57:25 INFO Vacuumed table table=pending_closures34702026/11/12 16:57:25 INFO Received uploads request method=POST path=/34712026/11/12 16:57:25 INFO Received complete upload request method=POST path=/34722026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts34732026/11/12 16:57:25 INFO Received push request method=POST path=/34742026/11/12 16:57:25 INFO Vacuumed table table=pending_objects34752026/11/12 16:57:25 INFO Completed upload id=15 objects=4 duration=1.918738ms34762026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads34772026/11/12 16:57:25 INFO Vacuumed table table=closures34782026/11/12 16:57:25 INFO Vacuumed table table=objects34792026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=034802026/11/12 16:57:25 INFO Received uploads request method=POST path=/34812026/11/12 16:57:25 INFO Received push request method=POST path=/34822026/11/12 16:57:25 INFO Received complete push request method=POST path=/34832026/11/12 16:57:25 INFO Received uploads request method=POST path=/34842026/11/12 16:57:25 INFO Received complete push request method=POST path=/34852026/11/12 16:57:25 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."34862026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."34872026/11/12 16:57:25 INFO Completed push id=8 objects=7 duration=1.514634ms34882026/11/12 16:57:25 INFO Completed push id=6 objects=26 duration=3.368731ms34892026/11/12 16:57:25 INFO Received uploads request method=POST path=/34902026/11/12 16:57:25 INFO Received push request method=POST path=/34912026/11/12 16:57:25 INFO Received complete push request method=POST path=/34922026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=034932026/11/12 16:57:25 INFO Completed push id=5 objects=6 duration=1.679296ms34942026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=11 objects-failed-to-delete=034952026/11/12 16:57:25 INFO Vacuumed table table=pending_closures34962026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts34972026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034982026/11/12 16:57:25 INFO Received push request method=POST path=/34992026/11/12 16:57:25 INFO Vacuumed table table=pending_closures35002026/11/12 16:57:25 INFO Vacuumed table table=pending_objects35012026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts35022026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads35032026/11/12 16:57:25 INFO Received uploads request method=POST path=/35042026/11/12 16:57:25 INFO Vacuumed table table=pending_objects35052026/11/12 16:57:25 INFO Vacuumed table table=closures35062026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads35072026/11/12 16:57:25 INFO Vacuumed table table=closures35082026/11/12 16:57:25 INFO Received complete push request method=POST path=/35092026/11/12 16:57:25 INFO Vacuumed table table=objects35102026/11/12 16:57:25 INFO Vacuumed table table=objects35112026/11/12 16:57:25 INFO Received complete upload request method=POST path=/35122026/11/12 16:57:25 INFO Received push request method=POST path=/35132026/11/12 16:57:25 INFO Completed upload id=11 objects=6 duration=1.395293ms35142026/11/12 16:57:25 INFO Completed push id=13 objects=6 duration=2.377962ms35152026/11/12 16:57:25 INFO Received uploads request method=POST path=/35162026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=035172026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=035182026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period35192026/11/12 16:57:25 INFO Received complete push request method=POST path=/35202026/11/12 16:57:25 INFO Received complete upload request method=POST path=/35212026/11/12 16:57:25 INFO Received complete push request method=POST path=/35222026/11/12 16:57:25 INFO Received push request method=POST path=/35232026/11/12 16:57:25 INFO Received push request method=POST path=/35242026/11/12 16:57:25 INFO Completed upload id=10 objects=6 duration=1.963719ms35252026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period35262026/11/12 16:57:25 INFO Received push request method=POST path=/35272026/11/12 16:57:25 INFO Completed push id=9 objects=4 duration=2.618985ms35282026/11/12 16:57:25 INFO Completed push id=13 objects=12 duration=2.920647ms35292026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035302026/11/12 16:57:25 INFO Received push request method=POST path=/35312026/11/12 16:57:25 INFO Vacuumed table table=pending_closures35322026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts35332026/11/12 16:57:25 INFO Received push request method=POST path=/35342026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=035352026/11/12 16:57:25 INFO Received complete upload request method=POST path=/35362026/11/12 16:57:25 INFO Received complete push request method=POST path=/35372026/11/12 16:57:25 INFO Vacuumed table table=pending_objects35382026/11/12 16:57:25 INFO Received complete upload request method=POST path=/35392026/11/12 16:57:25 INFO Completed upload id=14 objects=6 duration=913.129µs35402026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads35412026/11/12 16:57:25 INFO Completed upload id=16 objects=3 duration=863.368µs35422026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period35432026/11/12 16:57:25 INFO Vacuumed table table=closures35442026/11/12 16:57:25 INFO Vacuumed table table=objects35452026/11/12 16:57:25 INFO Completed push id=5 objects=8 duration=2.992688ms35462026/11/12 16:57:25 INFO Received push request method=POST path=/35472026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035482026/11/12 16:57:25 INFO Received complete upload request method=POST path=/35492026/11/12 16:57:25 INFO Vacuumed table table=pending_closures35502026/11/12 16:57:25 INFO Completed upload id=9 objects=16 duration=1.02219ms35512026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts35522026/11/12 16:57:25 INFO Received push request method=POST path=/35532026/11/12 16:57:25 INFO Vacuumed table table=pending_objects35542026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=035552026/11/12 16:57:25 INFO Received complete push request method=POST path=/35562026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads35572026/11/12 16:57:25 INFO Vacuumed table table=closures35582026/11/12 16:57:25 INFO Completed push id=14 objects=6 duration=1.160331ms35592026/11/12 16:57:25 INFO Vacuumed table table=objects35602026/11/12 16:57:25 INFO Received complete push request method=POST path=/35612026/11/12 16:57:25 INFO Completed push id=1 objects=11 duration=1.884577ms35622026/11/12 16:57:25 INFO Received push request method=POST path=/35632026/11/12 16:57:25 INFO Received push request method=POST path=/35642026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=035652026/11/12 16:57:25 INFO Received push request method=POST path=/35662026/11/12 16:57:25 INFO Received complete push request method=POST path=/35672026/11/12 16:57:25 INFO Received push request method=POST path=/35682026/11/12 16:57:25 INFO Completed push id=12 objects=15 duration=1.388853ms35692026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035702026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=035712026/11/12 16:57:25 INFO Vacuumed table table=pending_closures35722026/11/12 16:57:25 INFO Vacuumed table table=pending_closures35732026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts35742026/11/12 16:57:25 INFO Received push request method=POST path=/35752026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts35762026/11/12 16:57:25 INFO Vacuumed table table=pending_objects35772026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads35782026/11/12 16:57:25 INFO Vacuumed table table=pending_objects35792026/11/12 16:57:25 INFO Vacuumed table table=closures35802026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads35812026/11/12 16:57:25 INFO Received uploads request method=POST path=/35822026/11/12 16:57:25 INFO Vacuumed table table=objects35832026/11/12 16:57:25 INFO Vacuumed table table=closures35842026/11/12 16:57:25 INFO Received complete push request method=POST path=/35852026/11/12 16:57:25 INFO Received push request method=POST path=/35862026/11/12 16:57:25 INFO Vacuumed table table=objects35872026/11/12 16:57:25 INFO Completed push id=14 objects=4 duration=2.392502ms35882026/11/12 16:57:25 INFO Received uploads request method=POST path=/35892026/11/12 16:57:25 INFO Received uploads request method=POST path=/35902026/11/12 16:57:25 INFO Received complete upload request method=POST path=/35912026/11/12 16:57:25 INFO Received push request method=POST path=/35922026/11/12 16:57:25 INFO Received push request method=POST path=/35932026/11/12 16:57:25 INFO Received push request method=POST path=/35942026/11/12 16:57:25 INFO Completed upload id=16 objects=3 duration=706.026µs35952026/11/12 16:57:25 INFO Received complete upload request method=POST path=/35962026/11/12 16:57:25 INFO Completed upload id=7 objects=3 duration=764.927µs35972026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=035982026/11/12 16:57:25 INFO Received push request method=POST path=/35992026/11/12 16:57:25 INFO Received uploads request method=POST path=/36002026/11/12 16:57:25 INFO Vacuumed table table=pending_closures36012026/11/12 16:57:25 INFO Received complete push request method=POST path=/36022026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts36032026/11/12 16:57:25 INFO Received uploads request method=POST path=/36042026/11/12 16:57:25 INFO Completed push id=17 objects=6 duration=1.506574ms36052026/11/12 16:57:25 INFO Vacuumed table table=pending_objects36062026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads36072026/11/12 16:57:25 INFO Vacuumed table table=closures36082026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=036092026/11/12 16:57:25 INFO Vacuumed table table=objects36102026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036112026/11/12 16:57:25 INFO Received push request method=POST path=/36122026/11/12 16:57:25 INFO Vacuumed table table=pending_closures36132026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts36142026/11/12 16:57:25 INFO Received complete push request method=POST path=/36152026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=036162026/11/12 16:57:25 INFO Received push request method=POST path=/36172026/11/12 16:57:25 INFO Vacuumed table table=pending_objects36182026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads36192026/11/12 16:57:25 INFO Completed push id=8 objects=6 duration=1.421853ms36202026/11/12 16:57:25 INFO Vacuumed table table=closures36212026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period36222026/11/12 16:57:25 INFO Vacuumed table table=objects36232026/11/12 16:57:25 INFO Received complete push request method=POST path=/36242026/11/12 16:57:25 INFO Received push request method=POST path=/36252026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036262026/11/12 16:57:25 INFO Vacuumed table table=pending_closures36272026/11/12 16:57:25 INFO Completed push id=10 objects=3 duration=1.362673ms36282026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=036292026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts36302026/11/12 16:57:25 INFO Vacuumed table table=pending_objects36312026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads36322026/11/12 16:57:25 INFO Received uploads request method=POST path=/36332026/11/12 16:57:25 INFO Vacuumed table table=closures36342026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=036352026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=036362026/11/12 16:57:25 INFO Vacuumed table table=objects36372026/11/12 16:57:25 INFO Vacuumed table table=pending_closures36382026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period36392026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts36402026/11/12 16:57:25 INFO Received complete push request method=POST path=/36412026/11/12 16:57:25 INFO Vacuumed table table=pending_objects36422026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads36432026/11/12 16:57:25 INFO Completed push id=18 objects=23 duration=1.548274ms36442026/11/12 16:57:25 INFO Vacuumed table table=closures36452026/11/12 16:57:25 INFO Received push request method=POST path=/36462026/11/12 16:57:25 INFO Vacuumed table table=objects36472026/11/12 16:57:25 INFO Received complete push request method=POST path=/36482026/11/12 16:57:25 INFO Received uploads request method=POST path=/36492026/11/12 16:57:25 INFO Completed push id=12 objects=7 duration=1.216491ms36502026/11/12 16:57:25 INFO Received uploads request method=POST path=/36512026/11/12 16:57:25 INFO Received complete upload request method=POST path=/36522026/11/12 16:57:25 INFO Received complete push request method=POST path=/36532026/11/12 16:57:25 INFO Completed upload id=18 objects=4 duration=1.564315ms36542026/11/12 16:57:25 INFO Completed push id=4 objects=7 duration=1.347553ms36552026/11/12 16:57:25 INFO Received push request method=POST path=/36562026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=036572026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period36582026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036592026/11/12 16:57:25 INFO Received uploads request method=POST path=/36602026/11/12 16:57:25 INFO Received uploads request method=POST path=/36612026/11/12 16:57:25 INFO Vacuumed table table=pending_closures36622026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts36632026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."36642026/11/12 16:57:25 INFO Vacuumed table table=pending_objects36652026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads36662026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=036672026/11/12 16:57:25 INFO Vacuumed table table=closures36682026/11/12 16:57:25 INFO Vacuumed table table=objects36692026/11/12 16:57:25 INFO Received uploads request method=POST path=/36702026/11/12 16:57:25 INFO Received complete push request method=POST path=/36712026/11/12 16:57:25 INFO Received push request method=POST path=/36722026/11/12 16:57:25 INFO Received push request method=POST path=/36732026/11/12 16:57:25 INFO Completed push id=19 objects=6 duration=2.894967ms36742026/11/12 16:57:25 INFO Received complete push request method=POST path=/36752026/11/12 16:57:25 INFO Received uploads request method=POST path=/36762026/11/12 16:57:25 INFO Received push request method=POST path=/36772026/11/12 16:57:25 INFO Completed push id=17 objects=22 duration=2.010399ms36782026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=036792026/11/12 16:57:25 INFO Received push request method=POST path=/36802026/11/12 16:57:25 INFO Vacuumed table table=pending_closures36812026/11/12 16:57:25 INFO Received uploads request method=POST path=/36822026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts36832026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=036842026/11/12 16:57:25 INFO Received complete push request method=POST path=/36852026/11/12 16:57:25 INFO Received complete push request method=POST path=/36862026/11/12 16:57:25 INFO Vacuumed table table=pending_objects36872026/11/12 16:57:25 INFO Completed push id=11 objects=9 duration=1.261452ms36882026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads36892026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=036902026/11/12 16:57:25 INFO Vacuumed table table=closures36912026/11/12 16:57:25 INFO Completed push id=17 objects=6 duration=1.817156ms36922026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036932026/11/12 16:57:25 INFO Vacuumed table table=objects36942026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=036952026/11/12 16:57:25 INFO Vacuumed table table=pending_closures36962026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=036972026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts36982026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period36992026/11/12 16:57:25 INFO Vacuumed table table=pending_objects37002026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads37012026/11/12 16:57:25 INFO Vacuumed table table=closures37022026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period37032026/11/12 16:57:25 INFO Vacuumed table table=objects37042026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037052026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period37062026/11/12 16:57:25 INFO Received push request method=POST path=/37072026/11/12 16:57:25 INFO Vacuumed table table=pending_closures37082026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037092026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts37102026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037112026/11/12 16:57:25 INFO Vacuumed table table=pending_objects37122026/11/12 16:57:25 INFO Vacuumed table table=pending_closures37132026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads37142026/11/12 16:57:25 INFO Vacuumed table table=pending_closures37152026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts37162026/11/12 16:57:25 INFO Vacuumed table table=closures37172026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts37182026/11/12 16:57:25 INFO Vacuumed table table=objects37192026/11/12 16:57:25 INFO Vacuumed table table=pending_objects37202026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads37212026/11/12 16:57:25 INFO Vacuumed table table=closures37222026/11/12 16:57:25 INFO Received uploads request method=POST path=/37232026/11/12 16:57:25 INFO Vacuumed table table=pending_objects37242026/11/12 16:57:25 INFO Vacuumed table table=objects37252026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads37262026/11/12 16:57:25 INFO Vacuumed table table=closures37272026/11/12 16:57:25 INFO Vacuumed table table=objects37282026/11/12 16:57:25 INFO Received push request method=POST path=/37292026/11/12 16:57:25 INFO Received uploads request method=POST path=/37302026/11/12 16:57:25 INFO Received push request method=POST path=/37312026/11/12 16:57:25 INFO Received complete upload request method=POST path=/37322026/11/12 16:57:25 INFO Completed upload id=8 objects=3 duration=1.801537ms37332026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=037342026/11/12 16:57:25 INFO Received complete upload request method=POST path=/37352026/11/12 16:57:25 INFO Received uploads request method=POST path=/37362026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period37372026/11/12 16:57:25 INFO Received uploads request method=POST path=/37382026/11/12 16:57:25 INFO Completed upload id=19 objects=25 duration=2.143ms37392026/11/12 16:57:25 INFO Received push request method=POST path=/37402026/11/12 16:57:25 INFO Received push request method=POST path=/37412026/11/12 16:57:25 INFO Received uploads request method=POST path=/37422026/11/12 16:57:25 INFO Received complete push request method=POST path=/37432026/11/12 16:57:25 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."37442026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."37452026/11/12 16:57:25 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."37462026/11/12 16:57:25 INFO Received uploads request method=POST path=/37472026/11/12 16:57:25 INFO Completed push id=8 objects=6 duration=2.510703ms37482026/11/12 16:57:25 INFO Received complete push request method=POST path=/37492026/11/12 16:57:25 INFO Completed push id=4 objects=6 duration=1.177071ms37502026/11/12 16:57:25 INFO Received complete upload request method=POST path=/37512026/11/12 16:57:25 INFO Received uploads request method=POST path=/37522026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=037532026/11/12 16:57:25 INFO Received complete push request method=POST path=/37542026/11/12 16:57:25 INFO Completed push id=7 objects=7 duration=904.269µs37552026/11/12 16:57:25 INFO Completed upload id=23 objects=16 duration=1.829777ms37562026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period37572026/11/12 16:57:25 INFO Received complete push request method=POST path=/37582026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037592026/11/12 16:57:25 INFO Vacuumed table table=pending_closures37602026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=037612026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts37622026/11/12 16:57:25 INFO Received uploads request method=POST path=/37632026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=037642026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period37652026/11/12 16:57:25 INFO Vacuumed table table=pending_objects37662026/11/12 16:57:25 INFO Completed push id=12 objects=2 duration=2.589044ms37672026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads37682026/11/12 16:57:25 INFO Vacuumed table table=closures37692026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=037702026/11/12 16:57:25 INFO Vacuumed table table=objects37712026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period37722026/11/12 16:57:25 INFO Received complete push request method=POST path=/37732026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037742026/11/12 16:57:25 INFO Received complete upload request method=POST path=/37752026/11/12 16:57:25 INFO Vacuumed table table=pending_closures37762026/11/12 16:57:25 INFO Completed push id=16 objects=16 duration=2.658804ms37772026/11/12 16:57:25 INFO Received uploads request method=POST path=/37782026/11/12 16:57:25 INFO Completed upload id=7 objects=10 duration=1.461474ms37792026/11/12 16:57:25 INFO Received uploads request method=POST path=/37802026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts37812026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=037822026/11/12 16:57:25 INFO Vacuumed table table=pending_objects37832026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads37842026/11/12 16:57:25 INFO Vacuumed table table=pending_closures37852026/11/12 16:57:25 INFO Received uploads request method=POST path=/37862026/11/12 16:57:25 INFO Vacuumed table table=closures37872026/11/12 16:57:25 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."37882026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."37892026/11/12 16:57:25 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."37902026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts37912026/11/12 16:57:25 INFO Vacuumed table table=objects37922026/11/12 16:57:25 INFO Vacuumed table table=pending_objects37932026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads37942026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=037952026/11/12 16:57:25 INFO Vacuumed table table=closures37962026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period37972026/11/12 16:57:25 INFO Vacuumed table table=objects37982026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037992026/11/12 16:57:25 INFO Received uploads request method=POST path=/38002026/11/12 16:57:25 INFO Received uploads request method=POST path=/38012026/11/12 16:57:25 INFO Vacuumed table table=pending_closures38022026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts38032026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=038042026/11/12 16:57:25 INFO Received push request method=POST path=/38052026/11/12 16:57:25 INFO Received push request method=POST path=/38062026/11/12 16:57:25 INFO Vacuumed table table=pending_objects38072026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads38082026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period38092026/11/12 16:57:25 INFO Vacuumed table table=closures38102026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038112026/11/12 16:57:25 INFO Vacuumed table table=objects38122026/11/12 16:57:25 INFO Vacuumed table table=pending_closures38132026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts38142026/11/12 16:57:25 INFO Received push request method=POST path=/38152026/11/12 16:57:25 INFO Vacuumed table table=pending_objects38162026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads38172026/11/12 16:57:25 INFO Vacuumed table table=closures38182026/11/12 16:57:25 INFO Vacuumed table table=objects38192026/11/12 16:57:25 INFO Received uploads request method=POST path=/38202026/11/12 16:57:25 INFO Received complete push request method=POST path=/38212026/11/12 16:57:25 INFO Completed push id=6 objects=6 duration=1.526094ms38222026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=038232026/11/12 16:57:25 INFO Received uploads request method=POST path=/38242026/11/12 16:57:25 INFO Received complete upload request method=POST path=/38252026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=038262026/11/12 16:57:25 INFO Received complete upload request method=POST path=/38272026/11/12 16:57:25 INFO Completed upload id=18 objects=11 duration=1.158351ms38282026/11/12 16:57:25 INFO Received uploads request method=POST path=/38292026/11/12 16:57:25 INFO Completed upload id=15 objects=6 duration=1.471734ms38302026/11/12 16:57:25 INFO Received uploads request method=POST path=/38312026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period38322026/11/12 16:57:25 INFO Received push request method=POST path=/38332026/11/12 16:57:25 INFO Received push request method=POST path=/38342026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038352026/11/12 16:57:25 INFO Received push request method=POST path=/38362026/11/12 16:57:25 INFO Received uploads request method=POST path=/38372026/11/12 16:57:25 INFO Vacuumed table table=pending_closures38382026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts38392026/11/12 16:57:25 INFO Vacuumed table table=pending_objects38402026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads38412026/11/12 16:57:25 INFO Vacuumed table table=closures38422026/11/12 16:57:25 INFO Vacuumed table table=objects38432026/11/12 16:57:25 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."38442026/11/12 16:57:25 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."38452026/11/12 16:57:25 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."38462026/11/12 16:57:25 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."38472026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."38482026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."38492026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."38502026/11/12 16:57:25 INFO Received uploads request method=POST path=/38512026/11/12 16:57:25 INFO Received push request method=POST path=/38522026/11/12 16:57:25 INFO Received complete push request method=POST path=/38532026/11/12 16:57:25 INFO Completed push id=15 objects=7 duration=1.326192ms38542026/11/12 16:57:25 INFO Received uploads request method=POST path=/38552026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=038562026/11/12 16:57:25 INFO Received uploads request method=POST path=/38572026/11/12 16:57:25 INFO Received uploads request method=POST path=/38582026/11/12 16:57:25 INFO Received complete push request method=POST path=/38592026/11/12 16:57:25 INFO Received push request method=POST path=/38602026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038612026/11/12 16:57:25 INFO Completed push id=9 objects=24 duration=1.292852ms38622026/11/12 16:57:25 INFO Vacuumed table table=pending_closures38632026/11/12 16:57:25 INFO Received push request method=POST path=/38642026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts38652026/11/12 16:57:25 INFO Vacuumed table table=pending_objects38662026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads38672026/11/12 16:57:25 INFO Received uploads request method=POST path=/38682026/11/12 16:57:25 INFO Vacuumed table table=closures38692026/11/12 16:57:25 INFO Received uploads request method=POST path=/38702026/11/12 16:57:25 INFO Vacuumed table table=objects38712026/11/12 16:57:25 INFO Received complete push request method=POST path=/38722026/11/12 16:57:25 INFO Received push request method=POST path=/38732026/11/12 16:57:25 INFO Received uploads request method=POST path=/38742026/11/12 16:57:25 INFO Received complete upload request method=POST path=/38752026/11/12 16:57:25 INFO Completed push id=3 objects=21 duration=3.333311ms38762026/11/12 16:57:25 INFO Received complete push request method=POST path=/38772026/11/12 16:57:25 INFO Completed upload id=11 objects=7 duration=992.889µs38782026/11/12 16:57:25 INFO Completed push id=10 objects=15 duration=1.510794ms38792026/11/12 16:57:25 INFO Received uploads request method=POST path=/38802026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=038812026/11/12 16:57:25 INFO Received uploads request method=POST path=/38822026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period38832026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038842026/11/12 16:57:25 INFO Vacuumed table table=pending_closures38852026/11/12 16:57:25 INFO Received push request method=POST path=/38862026/11/12 16:57:25 INFO Received push request method=POST path=/38872026/11/12 16:57:25 INFO Received uploads request method=POST path=/38882026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts38892026/11/12 16:57:25 INFO Vacuumed table table=pending_objects38902026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads38912026/11/12 16:57:25 INFO Received complete upload request method=POST path=/38922026/11/12 16:57:25 INFO Vacuumed table table=closures38932026/11/12 16:57:25 INFO Vacuumed table table=objects38942026/11/12 16:57:25 INFO Received uploads request method=POST path=/38952026/11/12 16:57:25 INFO Completed upload id=23 objects=15 duration=1.676956ms38962026/11/12 16:57:25 INFO Received uploads request method=POST path=/38972026/11/12 16:57:25 INFO Received uploads request method=POST path=/38982026/11/12 16:57:25 INFO Received push request method=POST path=/38992026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=039002026/11/12 16:57:25 INFO Received uploads request method=POST path=/39012026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period39022026/11/12 16:57:25 INFO Received push request method=POST path=/39032026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=039042026/11/12 16:57:25 INFO Received complete push request method=POST path=/39052026/11/12 16:57:25 INFO Received uploads request method=POST path=/39062026/11/12 16:57:25 INFO Completed push id=16 objects=27 duration=1.562974ms39072026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period39082026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=039092026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039102026/11/12 16:57:25 INFO Received push request method=POST path=/39112026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period39122026/11/12 16:57:25 INFO Vacuumed table table=pending_closures39132026/11/12 16:57:25 INFO Received push request method=POST path=/39142026/11/12 16:57:25 INFO Received complete upload request method=POST path=/39152026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts39162026/11/12 16:57:25 INFO Received uploads request method=POST path=/39172026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039182026/11/12 16:57:25 INFO Vacuumed table table=pending_objects39192026/11/12 16:57:25 INFO Vacuumed table table=pending_closures39202026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads39212026/11/12 16:57:25 INFO Completed upload id=17 objects=6 duration=2.197801ms39222026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts39232026/11/12 16:57:25 INFO Vacuumed table table=closures39242026/11/12 16:57:25 INFO Received complete upload request method=POST path=/39252026/11/12 16:57:25 INFO Vacuumed table table=objects39262026/11/12 16:57:25 INFO Vacuumed table table=pending_objects39272026/11/12 16:57:25 INFO Completed upload id=10 objects=10 duration=794.687µs39282026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads39292026/11/12 16:57:25 INFO Vacuumed table table=closures39302026/11/12 16:57:25 INFO Vacuumed table table=objects39312026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."39322026/11/12 16:57:25 INFO Received uploads request method=POST path=/39332026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=039342026/11/12 16:57:25 INFO Received complete push request method=POST path=/39352026/11/12 16:57:25 INFO Received push request method=POST path=/39362026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039372026/11/12 16:57:25 INFO Completed push id=7 objects=15 duration=2.389102ms39382026/11/12 16:57:25 INFO Vacuumed table table=pending_closures39392026/11/12 16:57:25 INFO Received uploads request method=POST path=/39402026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts39412026/11/12 16:57:25 INFO Received complete upload request method=POST path=/39422026/11/12 16:57:25 INFO Received uploads request method=POST path=/39432026/11/12 16:57:25 INFO Vacuumed table table=pending_objects39442026/11/12 16:57:25 INFO Completed upload id=19 objects=6 duration=914.888µs39452026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads39462026/11/12 16:57:25 INFO Vacuumed table table=closures39472026/11/12 16:57:25 INFO Vacuumed table table=objects39482026/11/12 16:57:25 INFO Received push request method=POST path=/39492026/11/12 16:57:25 INFO Received push request method=POST path=/39502026/11/12 16:57:25 INFO Received uploads request method=POST path=/39512026/11/12 16:57:25 INFO Received push request method=POST path=/39522026/11/12 16:57:25 INFO Received complete push request method=POST path=/39532026/11/12 16:57:25 INFO Received complete upload request method=POST path=/39542026/11/12 16:57:25 INFO Completed push id=19 objects=8 duration=3.035928ms39552026/11/12 16:57:25 INFO Received uploads request method=POST path=/39562026/11/12 16:57:25 INFO Received push request method=POST path=/39572026/11/12 16:57:25 INFO Completed upload id=17 objects=25 duration=1.835837ms39582026/11/12 16:57:25 INFO Received complete push request method=POST path=/39592026/11/12 16:57:25 INFO Received complete upload request method=POST path=/39602026/11/12 16:57:25 INFO Received uploads request method=POST path=/39612026/11/12 16:57:25 INFO Completed upload id=22 objects=2 duration=771.287µs39622026/11/12 16:57:25 INFO Completed push id=22 objects=3 duration=2.404362ms39632026/11/12 16:57:25 INFO Received complete upload request method=POST path=/39642026/11/12 16:57:25 INFO Received push request method=POST path=/39652026/11/12 16:57:25 INFO Completed upload id=22 objects=7 duration=652.606µs39662026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=039672026/11/12 16:57:25 INFO Received complete push request method=POST path=/39682026/11/12 16:57:25 INFO Received push request method=POST path=/39692026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period39702026/11/12 16:57:25 INFO Completed push id=15 objects=4 duration=1.422873ms39712026/11/12 16:57:25 INFO Received uploads request method=POST path=/39722026/11/12 16:57:25 INFO Received uploads request method=POST path=/39732026/11/12 16:57:25 INFO Received push request method=POST path=/39742026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=039752026/11/12 16:57:25 INFO Received complete push request method=POST path=/39762026/11/12 16:57:25 INFO Received push request method=POST path=/39772026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=039782026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=039792026/11/12 16:57:25 INFO Completed push id=18 objects=10 duration=1.281772ms39802026/11/12 16:57:25 INFO Vacuumed table table=pending_closures39812026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=039822026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts39832026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period39842026/11/12 16:57:25 INFO Received push request method=POST path=/39852026/11/12 16:57:25 INFO Vacuumed table table=pending_objects39862026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads39872026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=039882026/11/12 16:57:25 INFO Vacuumed table table=closures39892026/11/12 16:57:25 INFO Vacuumed table table=pending_closures39902026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039912026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts39922026/11/12 16:57:25 INFO Received push request method=POST path=/39932026/11/12 16:57:25 INFO Vacuumed table table=objects39942026/11/12 16:57:25 INFO Vacuumed table table=pending_closures39952026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=039962026/11/12 16:57:25 INFO Vacuumed table table=pending_objects39972026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads39982026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts39992026/11/12 16:57:25 INFO Vacuumed table table=closures40002026/11/12 16:57:25 INFO Received complete push request method=POST path=/40012026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period40022026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=040032026/11/12 16:57:25 INFO Vacuumed table table=pending_objects40042026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads40052026/11/12 16:57:25 INFO Vacuumed table table=objects40062026/11/12 16:57:25 INFO Vacuumed table table=closures40072026/11/12 16:57:25 INFO Vacuumed table table=objects40082026/11/12 16:57:25 INFO Completed push id=16 objects=4 duration=2.188781ms40092026/11/12 16:57:25 INFO Received complete push request method=POST path=/40102026/11/12 16:57:25 INFO Received complete upload request method=POST path=/40112026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040122026/11/12 16:57:25 INFO Completed upload id=21 objects=2 duration=581.106µs40132026/11/12 16:57:25 INFO Vacuumed table table=pending_closures40142026/11/12 16:57:25 INFO Completed push id=15 objects=5 duration=1.473834ms40152026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts40162026/11/12 16:57:25 INFO Vacuumed table table=pending_objects40172026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads40182026/11/12 16:57:25 INFO Received push request method=POST path=/40192026/11/12 16:57:25 INFO Vacuumed table table=closures40202026/11/12 16:57:25 INFO Vacuumed table table=objects40212026/11/12 16:57:25 INFO Received uploads request method=POST path=/40222026/11/12 16:57:25 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."40232026/11/12 16:57:25 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."40242026/11/12 16:57:25 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."40252026/11/12 16:57:25 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."40262026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."40272026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."40282026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."40292026/11/12 16:57:25 INFO Received push request method=POST path=/40302026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."40312026/11/12 16:57:25 INFO Received uploads request method=POST path=/40322026/11/12 16:57:25 INFO Received push request method=POST path=/40332026/11/12 16:57:25 INFO Received uploads request method=POST path=/40342026/11/12 16:57:25 INFO Received push request method=POST path=/40352026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=040362026/11/12 16:57:25 INFO Received uploads request method=POST path=/40372026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period40382026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040392026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=040402026/11/12 16:57:25 INFO Received complete push request method=POST path=/40412026/11/12 16:57:25 INFO Vacuumed table table=pending_closures40422026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period40432026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts40442026/11/12 16:57:25 INFO Received complete push request method=POST path=/40452026/11/12 16:57:25 INFO Completed push id=24 objects=9 duration=1.338912ms40462026/11/12 16:57:25 INFO Vacuumed table table=pending_objects40472026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040482026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads40492026/11/12 16:57:25 INFO Vacuumed table table=closures40502026/11/12 16:57:25 INFO Vacuumed table table=pending_closures40512026/11/12 16:57:25 INFO Completed push id=13 objects=15 duration=1.443513ms40522026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts40532026/11/12 16:57:25 INFO Vacuumed table table=objects40542026/11/12 16:57:25 INFO Vacuumed table table=pending_objects40552026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads40562026/11/12 16:57:25 INFO Vacuumed table table=closures40572026/11/12 16:57:25 INFO Received uploads request method=POST path=/40582026/11/12 16:57:25 INFO Vacuumed table table=objects40592026/11/12 16:57:25 INFO Received uploads request method=POST path=/40602026/11/12 16:57:25 INFO Received complete upload request method=POST path=/40612026/11/12 16:57:25 INFO Received uploads request method=POST path=/40622026/11/12 16:57:25 INFO Completed upload id=20 objects=5 duration=790.087µs40632026/11/12 16:57:25 INFO Received complete push request method=POST path=/40642026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=040652026/11/12 16:57:25 INFO Received uploads request method=POST path=/40662026/11/12 16:57:25 INFO Completed push id=14 objects=3 duration=894.209µs40672026/11/12 16:57:25 INFO Received complete upload request method=POST path=/40682026/11/12 16:57:25 INFO Completed upload id=21 objects=3 duration=1.04337ms40692026/11/12 16:57:25 INFO Received push request method=POST path=/40702026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period40712026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=5 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040722026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=040732026/11/12 16:57:25 INFO Received uploads request method=POST path=/40742026/11/12 16:57:25 INFO Received push request method=POST path=/40752026/11/12 16:57:25 INFO Received push request method=POST path=/40762026/11/12 16:57:25 INFO Vacuumed table table=pending_closures40772026/11/12 16:57:25 INFO Received uploads request method=POST path=/40782026/11/12 16:57:25 INFO Received push request method=POST path=/40792026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts40802026/11/12 16:57:25 INFO Received push request method=POST path=/40812026/11/12 16:57:25 INFO Vacuumed table table=pending_objects40822026/11/12 16:57:25 INFO Received push request method=POST path=/40832026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads40842026/11/12 16:57:25 INFO Vacuumed table table=closures40852026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."40862026/11/12 16:57:25 INFO Vacuumed table table=objects40872026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=040882026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period40892026/11/12 16:57:25 INFO Received complete push request method=POST path=/40902026/11/12 16:57:25 INFO Completed push id=14 objects=3 duration=767.547µs40912026/11/12 16:57:25 INFO Received push request method=POST path=/40922026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=040932026/11/12 16:57:25 INFO Received push request method=POST path=/40942026/11/12 16:57:25 INFO Received uploads request method=POST path=/40952026/11/12 16:57:25 INFO Received uploads request method=POST path=/40962026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period40972026/11/12 16:57:25 INFO Received complete push request method=POST path=/40982026/11/12 16:57:25 INFO Received uploads request method=POST path=/40992026/11/12 16:57:25 INFO Received push request method=POST path=/41002026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=7 objects-failed-to-delete=041012026/11/12 16:57:25 INFO Completed push id=26 objects=14 duration=2.072679ms41022026/11/12 16:57:25 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."41032026/11/12 16:57:25 INFO Vacuumed table table=pending_closures41042026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts41052026/11/12 16:57:25 INFO Received push request method=POST path=/41062026/11/12 16:57:25 INFO Vacuumed table table=pending_objects41072026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads41082026/11/12 16:57:25 INFO Vacuumed table table=closures41092026/11/12 16:57:25 INFO Vacuumed table table=objects41102026/11/12 16:57:25 INFO Received push request method=POST path=/41112026/11/12 16:57:25 INFO Received uploads request method=POST path=/41122026/11/12 16:57:25 INFO Received complete upload request method=POST path=/41132026/11/12 16:57:25 INFO Completed upload id=20 objects=4 duration=821.847µs41142026/11/12 16:57:25 INFO Received complete push request method=POST path=/41152026/11/12 16:57:25 INFO Received complete push request method=POST path=/41162026/11/12 16:57:25 INFO Received complete upload request method=POST path=/41172026/11/12 16:57:25 INFO Completed push id=19 objects=14 duration=1.513974ms41182026/11/12 16:57:25 INFO Received complete push request method=POST path=/41192026/11/12 16:57:25 INFO Received push request method=POST path=/41202026/11/12 16:57:25 INFO Received push request method=POST path=/41212026/11/12 16:57:25 INFO Completed upload id=15 objects=9 duration=814.748µs41222026/11/12 16:57:25 INFO Completed push id=7 objects=24 duration=2.784085ms41232026/11/12 16:57:25 INFO Completed push id=20 objects=3 duration=1.300512ms41242026/11/12 16:57:25 INFO Received uploads request method=POST path=/41252026/11/12 16:57:25 INFO Received uploads request method=POST path=/41262026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=041272026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=041282026/11/12 16:57:25 INFO Received uploads request method=POST path=/41292026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=041302026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041312026/11/12 16:57:25 INFO Vacuumed table table=pending_closures41322026/11/12 16:57:25 INFO Received uploads request method=POST path=/41332026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts41342026/11/12 16:57:25 INFO Received push request method=POST path=/41352026/11/12 16:57:25 INFO Received uploads request method=POST path=/41362026/11/12 16:57:25 INFO Received complete upload request method=POST path=/41372026/11/12 16:57:25 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."41382026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."41392026/11/12 16:57:25 INFO Received push request method=POST path=/41402026/11/12 16:57:25 INFO Vacuumed table table=pending_objects41412026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period41422026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads41432026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=041442026/11/12 16:57:25 INFO Vacuumed table table=closures41452026/11/12 16:57:25 INFO Completed upload id=14 objects=8 duration=1.783797ms41462026/11/12 16:57:25 INFO Vacuumed table table=objects41472026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period41482026/11/12 16:57:25 INFO Received push request method=POST path=/41492026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041502026/11/12 16:57:25 INFO Vacuumed table table=pending_closures41512026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts41522026/11/12 16:57:25 INFO Vacuumed table table=pending_objects41532026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads41542026/11/12 16:57:25 INFO Received complete push request method=POST path=/41552026/11/12 16:57:25 INFO Vacuumed table table=closures41562026/11/12 16:57:25 INFO Received push request method=POST path=/41572026/11/12 16:57:25 INFO Vacuumed table table=objects41582026/11/12 16:57:25 INFO Completed push id=16 objects=5 duration=1.029649ms41592026/11/12 16:57:25 INFO Received uploads request method=POST path=/41602026/11/12 16:57:25 INFO Received complete push request method=POST path=/41612026/11/12 16:57:25 INFO Received push request method=POST path=/41622026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=041632026/11/12 16:57:25 INFO Completed push id=21 objects=17 duration=1.629315ms41642026/11/12 16:57:25 INFO Received push request method=POST path=/41652026/11/12 16:57:25 INFO Received push request method=POST path=/41662026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=041672026/11/12 16:57:25 INFO Received uploads request method=POST path=/41682026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041692026/11/12 16:57:25 INFO Received uploads request method=POST path=/41702026/11/12 16:57:25 INFO Vacuumed table table=pending_closures41712026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period41722026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts41732026/11/12 16:57:25 INFO Vacuumed table table=pending_objects41742026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads41752026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041762026/11/12 16:57:25 INFO Vacuumed table table=closures41772026/11/12 16:57:25 INFO Vacuumed table table=pending_closures41782026/11/12 16:57:25 INFO Vacuumed table table=objects41792026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts41802026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=041812026/11/12 16:57:25 INFO Vacuumed table table=pending_objects41822026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads41832026/11/12 16:57:25 INFO Vacuumed table table=pending_closures41842026/11/12 16:57:25 INFO Vacuumed table table=closures41852026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts41862026/11/12 16:57:25 INFO Vacuumed table table=objects41872026/11/12 16:57:25 INFO Received uploads request method=POST path=/41882026/11/12 16:57:25 INFO Vacuumed table table=pending_objects41892026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads41902026/11/12 16:57:25 INFO Vacuumed table table=closures41912026/11/12 16:57:25 INFO Received push request method=POST path=/41922026/11/12 16:57:25 INFO Vacuumed table table=objects41932026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=041942026/11/12 16:57:25 INFO Received complete push request method=POST path=/41952026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period41962026/11/12 16:57:25 INFO Received push request method=POST path=/41972026/11/12 16:57:25 INFO Completed push id=19 objects=7 duration=1.238652ms41982026/11/12 16:57:25 INFO Received complete push request method=POST path=/41992026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=042002026/11/12 16:57:25 INFO Received push request method=POST path=/42012026/11/12 16:57:25 INFO Completed push id=22 objects=3 duration=2.512084ms42022026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=042032026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=042042026/11/12 16:57:25 INFO Received uploads request method=POST path=/42052026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period42062026/11/12 16:57:25 INFO Vacuumed table table=pending_closures42072026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts42082026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=042092026/11/12 16:57:25 INFO Vacuumed table table=pending_closures42102026/11/12 16:57:25 INFO Vacuumed table table=pending_objects42112026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads42122026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts42132026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042142026/11/12 16:57:25 INFO Received push request method=POST path=/42152026/11/12 16:57:25 INFO Vacuumed table table=closures42162026/11/12 16:57:25 INFO Vacuumed table table=pending_objects42172026/11/12 16:57:25 INFO Vacuumed table table=pending_closures42182026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads42192026/11/12 16:57:25 INFO Vacuumed table table=objects42202026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts42212026/11/12 16:57:25 INFO Received push request method=POST path=/42222026/11/12 16:57:25 INFO Vacuumed table table=closures42232026/11/12 16:57:25 INFO Received push request method=POST path=/42242026/11/12 16:57:25 INFO Vacuumed table table=objects42252026/11/12 16:57:25 INFO Vacuumed table table=pending_objects42262026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads42272026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=042282026/11/12 16:57:25 INFO Vacuumed table table=closures42292026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=042302026/11/12 16:57:25 INFO Received push request method=POST path=/42312026/11/12 16:57:25 INFO Vacuumed table table=objects42322026/11/12 16:57:25 INFO Received complete push request method=POST path=/42332026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period42342026/11/12 16:57:25 INFO Received uploads request method=POST path=/42352026/11/12 16:57:25 INFO Completed push id=21 objects=3 duration=1.00755ms42362026/11/12 16:57:25 INFO Received push request method=POST path=/42372026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042382026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042392026/11/12 16:57:25 INFO Received push request method=POST path=/42402026/11/12 16:57:25 INFO Vacuumed table table=pending_closures42412026/11/12 16:57:25 INFO Vacuumed table table=pending_closures42422026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts42432026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts42442026/11/12 16:57:25 INFO Vacuumed table table=pending_objects42452026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads42462026/11/12 16:57:25 INFO Received uploads request method=POST path=/42472026/11/12 16:57:25 INFO Vacuumed table table=pending_objects42482026/11/12 16:57:25 INFO Vacuumed table table=closures42492026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads42502026/11/12 16:57:25 INFO Vacuumed table table=objects42512026/11/12 16:57:25 INFO Vacuumed table table=closures42522026/11/12 16:57:25 INFO Vacuumed table table=objects42532026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=042542026/11/12 16:57:25 INFO Received complete upload request method=POST path=/42552026/11/12 16:57:25 INFO Received uploads request method=POST path=/42562026/11/12 16:57:25 INFO Received uploads request method=POST path=/42572026/11/12 16:57:25 INFO Completed upload id=21 objects=7 duration=1.546235ms42582026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042592026/11/12 16:57:25 INFO Received uploads request method=POST path=/42602026/11/12 16:57:25 INFO Vacuumed table table=pending_closures42612026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts42622026/11/12 16:57:25 INFO Vacuumed table table=pending_objects42632026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads42642026/11/12 16:57:25 INFO Vacuumed table table=closures42652026/11/12 16:57:25 INFO Vacuumed table table=objects42662026/11/12 16:57:25 INFO Received uploads request method=POST path=/42672026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=042682026/11/12 16:57:25 INFO Received push request method=POST path=/42692026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period42702026/11/12 16:57:25 INFO Received push request method=POST path=/42712026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=042722026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042732026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period42742026/11/12 16:57:25 INFO Received uploads request method=POST path=/42752026/11/12 16:57:25 INFO Vacuumed table table=pending_closures42762026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=042772026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts42782026/11/12 16:57:25 INFO Vacuumed table table=pending_objects42792026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads42802026/11/12 16:57:25 INFO Vacuumed table table=closures42812026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042822026/11/12 16:57:25 INFO Vacuumed table table=objects42832026/11/12 16:57:25 INFO Vacuumed table table=pending_closures42842026/11/12 16:57:25 INFO Received push request method=POST path=/42852026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts42862026/11/12 16:57:25 INFO Received complete push request method=POST path=/42872026/11/12 16:57:25 INFO Vacuumed table table=pending_objects42882026/11/12 16:57:25 INFO Received uploads request method=POST path=/42892026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=042902026/11/12 16:57:25 INFO Completed push id=13 objects=2 duration=795.188µs42912026/11/12 16:57:25 INFO Received uploads request method=POST path=/42922026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads42932026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=042942026/11/12 16:57:25 INFO Received uploads request method=POST path=/42952026/11/12 16:57:25 INFO Vacuumed table table=closures42962026/11/12 16:57:25 INFO Vacuumed table table=pending_closures42972026/11/12 16:57:25 INFO Vacuumed table table=objects42982026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts42992026/11/12 16:57:25 INFO Vacuumed table table=pending_objects43002026/11/12 16:57:25 INFO Received complete upload request method=POST path=/43012026/11/12 16:57:25 INFO Received uploads request method=POST path=/43022026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads43032026/11/12 16:57:25 INFO Received complete upload request method=POST path=/43042026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043052026/11/12 16:57:25 INFO Completed upload id=14 objects=16 duration=2.1564ms43062026/11/12 16:57:25 INFO Vacuumed table table=closures43072026/11/12 16:57:25 INFO Completed upload id=22 objects=6 duration=895.489µs43082026/11/12 16:57:25 INFO Vacuumed table table=pending_closures43092026/11/12 16:57:25 INFO Received push request method=POST path=/43102026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts43112026/11/12 16:57:25 INFO Vacuumed table table=objects43122026/11/12 16:57:25 INFO Vacuumed table table=pending_objects43132026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads43142026/11/12 16:57:25 INFO Vacuumed table table=closures43152026/11/12 16:57:25 INFO Vacuumed table table=objects43162026/11/12 16:57:25 INFO Received push request method=POST path=/43172026/11/12 16:57:25 INFO Received complete push request method=POST path=/43182026/11/12 16:57:25 INFO Received uploads request method=POST path=/43192026/11/12 16:57:25 INFO Completed push id=24 objects=3 duration=1.00547ms43202026/11/12 16:57:25 INFO Received complete upload request method=POST path=/43212026/11/12 16:57:25 INFO Received uploads request method=POST path=/43222026/11/12 16:57:25 INFO Completed upload id=27 objects=13 duration=1.020049ms43232026/11/12 16:57:25 INFO Received push request method=POST path=/43242026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=043252026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=043262026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period43272026/11/12 16:57:25 INFO Received push request method=POST path=/43282026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043292026/11/12 16:57:25 INFO Vacuumed table table=pending_closures43302026/11/12 16:57:25 INFO Received uploads request method=POST path=/43312026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts43322026/11/12 16:57:25 INFO Vacuumed table table=pending_objects43332026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=3 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043342026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads43352026/11/12 16:57:25 INFO Vacuumed table table=closures43362026/11/12 16:57:25 INFO Vacuumed table table=pending_closures43372026/11/12 16:57:25 INFO Received complete push request method=POST path=/43382026/11/12 16:57:25 INFO Received uploads request method=POST path=/43392026/11/12 16:57:25 INFO Vacuumed table table=objects43402026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts43412026/11/12 16:57:25 INFO Received push request method=POST path=/43422026/11/12 16:57:25 INFO Completed push id=14 objects=3 duration=2.620705ms43432026/11/12 16:57:25 INFO Vacuumed table table=pending_objects43442026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads43452026/11/12 16:57:25 INFO Received push request method=POST path=/43462026/11/12 16:57:25 INFO Vacuumed table table=closures43472026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=043482026/11/12 16:57:25 INFO Vacuumed table table=objects43492026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=043502026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043512026/11/12 16:57:25 INFO Received uploads request method=POST path=/43522026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period43532026/11/12 16:57:25 INFO Vacuumed table table=pending_closures43542026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts43552026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043562026/11/12 16:57:25 INFO Vacuumed table table=pending_objects43572026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads43582026/11/12 16:57:25 INFO Vacuumed table table=pending_closures43592026/11/12 16:57:25 INFO Vacuumed table table=closures43602026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts43612026/11/12 16:57:25 INFO Received complete upload request method=POST path=/43622026/11/12 16:57:25 INFO Vacuumed table table=objects43632026/11/12 16:57:25 INFO Vacuumed table table=pending_objects43642026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads43652026/11/12 16:57:25 INFO Completed upload id=12 objects=26 duration=1.780297ms43662026/11/12 16:57:25 INFO Vacuumed table table=closures43672026/11/12 16:57:25 INFO Received uploads request method=POST path=/43682026/11/12 16:57:25 INFO Vacuumed table table=objects43692026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=043702026/11/12 16:57:25 INFO Received push request method=POST path=/43712026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043722026/11/12 16:57:25 INFO Received push request method=POST path=/43732026/11/12 16:57:25 INFO Received complete push request method=POST path=/43742026/11/12 16:57:25 INFO Vacuumed table table=pending_closures43752026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts43762026/11/12 16:57:25 INFO Completed push id=21 objects=9 duration=1.382973ms43772026/11/12 16:57:25 INFO Vacuumed table table=pending_objects43782026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads43792026/11/12 16:57:25 INFO Received complete push request method=POST path=/43802026/11/12 16:57:25 INFO Vacuumed table table=closures43812026/11/12 16:57:25 INFO Vacuumed table table=objects43822026/11/12 16:57:25 INFO Received uploads request method=POST path=/43832026/11/12 16:57:25 INFO Completed push id=17 objects=11 duration=1.436993ms43842026/11/12 16:57:25 INFO Received uploads request method=POST path=/43852026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=043862026/11/12 16:57:25 INFO Received uploads request method=POST path=/43872026/11/12 16:57:25 INFO Received uploads request method=POST path=/43882026/11/12 16:57:25 INFO Received uploads request method=POST path=/43892026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period43902026/11/12 16:57:25 INFO Received complete upload request method=POST path=/43912026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043922026/11/12 16:57:25 INFO Received push request method=POST path=/43932026/11/12 16:57:25 INFO Completed upload id=16 objects=17 duration=2.1431ms43942026/11/12 16:57:25 INFO Vacuumed table table=pending_closures43952026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts43962026/11/12 16:57:25 INFO Vacuumed table table=pending_objects43972026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads43982026/11/12 16:57:25 INFO Vacuumed table table=closures43992026/11/12 16:57:25 INFO Vacuumed table table=objects44002026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=044012026/11/12 16:57:25 INFO Received push request method=POST path=/44022026/11/12 16:57:25 INFO Received push request method=POST path=/44032026/11/12 16:57:25 INFO Received uploads request method=POST path=/44042026/11/12 16:57:25 INFO Received complete upload request method=POST path=/44052026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044062026/11/12 16:57:25 INFO Received push request method=POST path=/44072026/11/12 16:57:25 INFO Vacuumed table table=pending_closures44082026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts44092026/11/12 16:57:25 INFO Completed upload id=13 objects=14 duration=1.944358ms44102026/11/12 16:57:25 INFO Vacuumed table table=pending_objects44112026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads44122026/11/12 16:57:25 INFO Received complete upload request method=POST path=/44132026/11/12 16:57:25 INFO Vacuumed table table=closures44142026/11/12 16:57:25 INFO Received push request method=POST path=/44152026/11/12 16:57:25 INFO Vacuumed table table=objects44162026/11/12 16:57:25 INFO Completed upload id=27 objects=4 duration=1.04317ms44172026/11/12 16:57:25 INFO Received uploads request method=POST path=/44182026/11/12 16:57:25 INFO Received complete upload request method=POST path=/44192026/11/12 16:57:25 INFO Completed upload id=23 objects=18 duration=1.785836ms44202026/11/12 16:57:25 INFO Received complete push request method=POST path=/44212026/11/12 16:57:25 INFO Received uploads request method=POST path=/44222026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=044232026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=044242026/11/12 16:57:25 INFO Received uploads request method=POST path=/44252026/11/12 16:57:25 INFO Completed push id=30 objects=4 duration=1.949858ms44262026/11/12 16:57:25 INFO Received push request method=POST path=/44272026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period44282026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period44292026/11/12 16:57:25 INFO Received push request method=POST path=/44302026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044312026/11/12 16:57:25 INFO Vacuumed table table=pending_closures44322026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts44332026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=044342026/11/12 16:57:25 INFO Vacuumed table table=pending_objects44352026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads44362026/11/12 16:57:25 INFO Vacuumed table table=closures44372026/11/12 16:57:25 INFO Received complete push request method=POST path=/44382026/11/12 16:57:25 INFO Vacuumed table table=objects44392026/11/12 16:57:25 INFO Received complete push request method=POST path=/44402026/11/12 16:57:25 INFO Received push request method=POST path=/44412026/11/12 16:57:25 INFO Received complete push request method=POST path=/44422026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044432026/11/12 16:57:25 INFO Completed push id=13 objects=11 duration=1.345313ms44442026/11/12 16:57:25 INFO Vacuumed table table=pending_closures44452026/11/12 16:57:25 INFO Completed push id=16 objects=8 duration=1.742477ms44462026/11/12 16:57:25 INFO Completed push id=27 objects=25 duration=2.473303ms44472026/11/12 16:57:25 INFO Received uploads request method=POST path=/44482026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts44492026/11/12 16:57:25 INFO Vacuumed table table=pending_objects44502026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads44512026/11/12 16:57:25 INFO Vacuumed table table=closures44522026/11/12 16:57:25 INFO Vacuumed table table=objects44532026/11/12 16:57:25 INFO Received uploads request method=POST path=/44542026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=044552026/11/12 16:57:25 INFO Received uploads request method=POST path=/44562026/11/12 16:57:25 INFO Received uploads request method=POST path=/44572026/11/12 16:57:25 INFO Received push request method=POST path=/44582026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044592026/11/12 16:57:25 INFO Received uploads request method=POST path=/44602026/11/12 16:57:25 INFO Vacuumed table table=pending_closures44612026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts44622026/11/12 16:57:25 INFO Vacuumed table table=pending_objects44632026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=044642026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads44652026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=044662026/11/12 16:57:25 INFO Vacuumed table table=closures44672026/11/12 16:57:25 INFO Received uploads request method=POST path=/44682026/11/12 16:57:25 INFO Vacuumed table table=objects44692026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=044702026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period44712026/11/12 16:57:25 INFO Received push request method=POST path=/44722026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044732026/11/12 16:57:25 INFO Received push request method=POST path=/44742026/11/12 16:57:25 INFO Vacuumed table table=pending_closures44752026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts44762026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044772026/11/12 16:57:25 INFO Vacuumed table table=pending_objects44782026/11/12 16:57:25 INFO Vacuumed table table=pending_closures44792026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads44802026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts44812026/11/12 16:57:25 INFO Vacuumed table table=closures44822026/11/12 16:57:25 INFO Vacuumed table table=objects44832026/11/12 16:57:25 INFO Received complete upload request method=POST path=/44842026/11/12 16:57:25 INFO Vacuumed table table=pending_objects44852026/11/12 16:57:25 INFO Received complete push request method=POST path=/44862026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads44872026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=044882026/11/12 16:57:25 INFO Vacuumed table table=closures44892026/11/12 16:57:25 INFO Completed upload id=26 objects=6 duration=836.028µs44902026/11/12 16:57:25 INFO Received push request method=POST path=/44912026/11/12 16:57:25 INFO Vacuumed table table=objects44922026/11/12 16:57:25 INFO Vacuumed table table=pending_closures44932026/11/12 16:57:25 INFO Received uploads request method=POST path=/44942026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts44952026/11/12 16:57:25 INFO Completed push id=31 objects=6 duration=2.390502ms44962026/11/12 16:57:25 INFO Received push request method=POST path=/44972026/11/12 16:57:25 INFO Vacuumed table table=pending_objects44982026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads44992026/11/12 16:57:25 INFO Vacuumed table table=closures45002026/11/12 16:57:25 INFO Vacuumed table table=objects45012026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=045022026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period45032026/11/12 16:57:25 INFO Received uploads request method=POST path=/45042026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045052026/11/12 16:57:25 INFO Received complete upload request method=POST path=/45062026/11/12 16:57:25 INFO Vacuumed table table=pending_closures45072026/11/12 16:57:25 INFO Completed upload id=22 objects=22 duration=793.687µs45082026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts45092026/11/12 16:57:25 INFO Vacuumed table table=pending_objects45102026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads45112026/11/12 16:57:25 INFO Received push request method=POST path=/45122026/11/12 16:57:25 INFO Received push request method=POST path=/45132026/11/12 16:57:25 INFO Vacuumed table table=closures45142026/11/12 16:57:25 INFO Vacuumed table table=objects45152026/11/12 16:57:25 INFO Received uploads request method=POST path=/45162026/11/12 16:57:25 INFO Received uploads request method=POST path=/45172026/11/12 16:57:25 INFO Received complete push request method=POST path=/45182026/11/12 16:57:25 INFO Received uploads request method=POST path=/45192026/11/12 16:57:25 INFO Received uploads request method=POST path=/45202026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=045212026/11/12 16:57:25 INFO Received uploads request method=POST path=/45222026/11/12 16:57:25 INFO Received push request method=POST path=/45232026/11/12 16:57:25 INFO Vacuumed table table=pending_closures45242026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts45252026/11/12 16:57:25 INFO Completed push id=24 objects=5 duration=2.332981ms45262026/11/12 16:57:25 INFO Vacuumed table table=pending_objects45272026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads45282026/11/12 16:57:25 INFO Received complete push request method=POST path=/45292026/11/12 16:57:25 INFO Vacuumed table table=closures45302026/11/12 16:57:25 INFO Vacuumed table table=objects45312026/11/12 16:57:25 INFO Received complete upload request method=POST path=/45322026/11/12 16:57:25 INFO Received complete upload request method=POST path=/45332026/11/12 16:57:25 INFO Completed push id=11 objects=6 duration=2.665325ms45342026/11/12 16:57:25 INFO Completed upload id=29 objects=4 duration=1.435354ms45352026/11/12 16:57:25 INFO Completed upload id=33 objects=5 duration=2.16194ms45362026/11/12 16:57:25 INFO Received push request method=POST path=/45372026/11/12 16:57:25 INFO Received complete push request method=POST path=/45382026/11/12 16:57:25 INFO Received push request method=POST path=/45392026/11/12 16:57:25 INFO Completed push id=32 objects=7 duration=993.409µs45402026/11/12 16:57:25 INFO Received uploads request method=POST path=/45412026/11/12 16:57:25 INFO Received push request method=POST path=/45422026/11/12 16:57:25 INFO Received complete upload request method=POST path=/45432026/11/12 16:57:25 INFO Received uploads request method=POST path=/45442026/11/12 16:57:25 INFO Completed upload id=29 objects=5 duration=998.009µs45452026/11/12 16:57:25 INFO Received complete push request method=POST path=/45462026/11/12 16:57:25 INFO Completed push id=31 objects=14 duration=1.914678ms45472026/11/12 16:57:25 INFO Received uploads request method=POST path=/45482026/11/12 16:57:25 INFO Received uploads request method=POST path=/45492026/11/12 16:57:25 INFO Received complete push request method=POST path=/45502026/11/12 16:57:25 INFO Completed push id=26 objects=5 duration=1.488594ms45512026/11/12 16:57:25 INFO Received complete push request method=POST path=/45522026/11/12 16:57:25 INFO Received complete push request method=POST path=/45532026/11/12 16:57:25 INFO Completed push id=26 objects=4 duration=899.528µs45542026/11/12 16:57:25 INFO Received complete upload request method=POST path=/45552026/11/12 16:57:25 INFO Completed push id=18 objects=6 duration=1.408273ms45562026/11/12 16:57:25 INFO Completed upload id=26 objects=2 duration=1.636016ms45572026/11/12 16:57:25 INFO Received uploads request method=POST path=/45582026/11/12 16:57:25 INFO Received uploads request method=POST path=/45592026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=045602026/11/12 16:57:25 INFO Received complete push request method=POST path=/45612026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period45622026/11/12 16:57:25 INFO Received uploads request method=POST path=/45632026/11/12 16:57:25 INFO Completed push id=18 objects=6 duration=2.009579ms45642026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045652026/11/12 16:57:25 INFO Vacuumed table table=pending_closures45662026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts45672026/11/12 16:57:25 INFO Vacuumed table table=pending_objects45682026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads45692026/11/12 16:57:25 INFO Vacuumed table table=closures45702026/11/12 16:57:25 INFO Received complete upload request method=POST path=/45712026/11/12 16:57:25 INFO Vacuumed table table=objects45722026/11/12 16:57:25 INFO Completed upload id=24 objects=14 duration=891.288µs45732026/11/12 16:57:25 INFO Received complete push request method=POST path=/45742026/11/12 16:57:25 INFO Received complete upload request method=POST path=/45752026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=045762026/11/12 16:57:25 INFO Completed upload id=28 objects=13 duration=850.108µs45772026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=045782026/11/12 16:57:25 INFO Completed push id=16 objects=14 duration=2.225421ms45792026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=045802026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=045812026/11/12 16:57:25 INFO Received push request method=POST path=/45822026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045832026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045842026/11/12 16:57:25 INFO Vacuumed table table=pending_closures45852026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period45862026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts45872026/11/12 16:57:25 INFO Received uploads request method=POST path=/45882026/11/12 16:57:25 INFO Received complete upload request method=POST path=/45892026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045902026/11/12 16:57:25 INFO Vacuumed table table=pending_objects45912026/11/12 16:57:25 INFO Vacuumed table table=pending_closures45922026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045932026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads45942026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts45952026/11/12 16:57:25 INFO Vacuumed table table=closures45962026/11/12 16:57:25 INFO Completed upload id=21 objects=25 duration=1.179771ms45972026/11/12 16:57:25 INFO Vacuumed table table=pending_closures45982026/11/12 16:57:25 INFO Vacuumed table table=pending_closures45992026/11/12 16:57:25 INFO Vacuumed table table=objects46002026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts46012026/11/12 16:57:25 INFO Received complete upload request method=POST path=/46022026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts46032026/11/12 16:57:25 INFO Vacuumed table table=pending_objects46042026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads46052026/11/12 16:57:25 INFO Vacuumed table table=pending_objects46062026/11/12 16:57:25 INFO Completed upload id=23 objects=9 duration=1.062929ms46072026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads46082026/11/12 16:57:25 INFO Vacuumed table table=pending_objects46092026/11/12 16:57:25 INFO Vacuumed table table=closures46102026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads46112026/11/12 16:57:25 INFO Vacuumed table table=closures46122026/11/12 16:57:25 INFO Received complete upload request method=POST path=/46132026/11/12 16:57:25 INFO Vacuumed table table=objects46142026/11/12 16:57:25 INFO Vacuumed table table=closures46152026/11/12 16:57:25 INFO Vacuumed table table=objects46162026/11/12 16:57:25 INFO Vacuumed table table=objects46172026/11/12 16:57:25 INFO Completed upload id=15 objects=3 duration=1.291972ms46182026/11/12 16:57:25 INFO Received complete push request method=POST path=/46192026/11/12 16:57:25 INFO Received complete push request method=POST path=/46202026/11/12 16:57:25 INFO Received complete push request method=POST path=/46212026/11/12 16:57:25 INFO Completed push id=19 objects=3 duration=830.888µs46222026/11/12 16:57:25 INFO Completed push id=11 objects=6 duration=1.179991ms46232026/11/12 16:57:25 INFO Completed push id=1 objects=9 duration=1.024069ms46242026/11/12 16:57:25 INFO Received complete push request method=POST path=/46252026/11/12 16:57:25 INFO Completed push id=29 objects=3 duration=786.007µs46262026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=046272026/11/12 16:57:25 INFO Received complete push request method=POST path=/46282026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period46292026/11/12 16:57:25 INFO Completed push id=21 objects=26 duration=1.284452ms46302026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046312026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=046322026/11/12 16:57:25 INFO Received uploads request method=POST path=/46332026/11/12 16:57:25 INFO Vacuumed table table=pending_closures46342026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts46352026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period46362026/11/12 16:57:25 INFO Vacuumed table table=pending_objects46372026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads46382026/11/12 16:57:25 INFO Vacuumed table table=closures46392026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046402026/11/12 16:57:25 INFO Vacuumed table table=objects46412026/11/12 16:57:25 INFO Vacuumed table table=pending_closures46422026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts46432026/11/12 16:57:25 INFO Vacuumed table table=pending_objects46442026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads46452026/11/12 16:57:25 INFO Received push request method=POST path=/46462026/11/12 16:57:25 INFO Vacuumed table table=closures46472026/11/12 16:57:25 INFO Vacuumed table table=objects46482026/11/12 16:57:25 INFO Received complete upload request method=POST path=/46492026/11/12 16:57:25 INFO Completed upload id=20 objects=3 duration=1.947758ms46502026/11/12 16:57:25 INFO Received complete upload request method=POST path=/46512026/11/12 16:57:25 INFO Completed upload id=31 objects=2 duration=797.267µs46522026/11/12 16:57:25 INFO Received complete push request method=POST path=/46532026/11/12 16:57:25 INFO Completed push id=20 objects=13 duration=1.04403ms46542026/11/12 16:57:25 INFO Received push request method=POST path=/46552026/11/12 16:57:25 INFO Received push request method=POST path=/46562026/11/12 16:57:25 INFO Received complete upload request method=POST path=/46572026/11/12 16:57:25 INFO Completed upload id=28 objects=17 duration=1.114751ms46582026/11/12 16:57:25 INFO Received complete upload request method=POST path=/46592026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=046602026/11/12 16:57:25 INFO Received push request method=POST path=/46612026/11/12 16:57:25 INFO Completed upload id=19 objects=2 duration=1.597415ms46622026/11/12 16:57:25 INFO Received complete push request method=POST path=/46632026/11/12 16:57:25 INFO Received uploads request method=POST path=/46642026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046652026/11/12 16:57:25 INFO Received push request method=POST path=/46662026/11/12 16:57:25 INFO Vacuumed table table=pending_closures46672026/11/12 16:57:25 INFO Completed push id=7 objects=7 duration=1.368093ms46682026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts46692026/11/12 16:57:25 INFO Vacuumed table table=pending_objects46702026/11/12 16:57:25 INFO Received complete push request method=POST path=/46712026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads46722026/11/12 16:57:25 INFO Vacuumed table table=closures46732026/11/12 16:57:25 INFO Completed push id=33 objects=6 duration=664.866µs46742026/11/12 16:57:25 INFO Received complete push request method=POST path=/46752026/11/12 16:57:25 INFO Vacuumed table table=objects46762026/11/12 16:57:25 INFO Received complete push request method=POST path=/46772026/11/12 16:57:25 INFO Completed push id=16 objects=7 duration=1.386993ms46782026/11/12 16:57:25 INFO Completed push id=22 objects=4 duration=988.489µs46792026/11/12 16:57:25 INFO Received complete push request method=POST path=/46802026/11/12 16:57:25 INFO Received uploads request method=POST path=/46812026/11/12 16:57:25 INFO Completed push id=29 objects=2 duration=1.030869ms46822026/11/12 16:57:25 INFO Received complete upload request method=POST path=/46832026/11/12 16:57:25 INFO Received complete upload request method=POST path=/46842026/11/12 16:57:25 INFO Received uploads request method=POST path=/46852026/11/12 16:57:25 INFO Received uploads request method=POST path=/46862026/11/12 16:57:25 INFO Completed upload id=12 objects=6 duration=1.292072ms46872026/11/12 16:57:25 INFO Completed upload id=13 objects=8 duration=1.294712ms46882026/11/12 16:57:25 INFO Received complete push request method=POST path=/46892026/11/12 16:57:25 INFO Completed push id=12 objects=6 duration=946.229µs46902026/11/12 16:57:25 INFO Received complete upload request method=POST path=/46912026/11/12 16:57:25 INFO Completed upload id=2 objects=7 duration=749.487µs46922026/11/12 16:57:25 INFO Received push request method=POST path=/46932026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=046942026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period46952026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=046962026/11/12 16:57:25 INFO Received complete push request method=POST path=/46972026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046982026/11/12 16:57:25 INFO Completed push id=31 objects=6 duration=871.008µs46992026/11/12 16:57:25 INFO Received complete push request method=POST path=/47002026/11/12 16:57:25 INFO Received complete upload request method=POST path=/47012026/11/12 16:57:25 INFO Vacuumed table table=pending_closures47022026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts47032026/11/12 16:57:25 INFO Completed upload id=23 objects=7 duration=825.308µs47042026/11/12 16:57:25 INFO Completed push id=33 objects=7 duration=1.422053ms47052026/11/12 16:57:25 INFO Received complete push request method=POST path=/47062026/11/12 16:57:25 INFO Vacuumed table table=pending_objects47072026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads47082026/11/12 16:57:25 INFO Received uploads request method=POST path=/47092026/11/12 16:57:25 INFO Vacuumed table table=closures47102026/11/12 16:57:25 INFO Completed push id=17 objects=3 duration=1.430714ms47112026/11/12 16:57:25 INFO Vacuumed table table=objects47122026/11/12 16:57:25 INFO Received complete push request method=POST path=/47132026/11/12 16:57:25 INFO Received uploads request method=POST path=/47142026/11/12 16:57:25 INFO Received complete upload request method=POST path=/47152026/11/12 16:57:25 INFO Completed push id=23 objects=6 duration=1.007549ms47162026/11/12 16:57:25 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."47172026/11/12 16:57:25 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."47182026/11/12 16:57:25 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."47192026/11/12 16:57:25 INFO Completed upload id=24 objects=7 duration=945.749µs47202026/11/12 16:57:25 INFO Received uploads request method=POST path=/47212026/11/12 16:57:25 INFO Received complete push request method=POST path=/47222026/11/12 16:57:25 INFO Completed push id=17 objects=22 duration=1.816917ms47232026/11/12 16:57:25 INFO Received complete push request method=POST path=/47242026/11/12 16:57:25 INFO Received complete upload request method=POST path=/47252026/11/12 16:57:25 INFO Completed push id=32 objects=9 duration=952.229µs47262026/11/12 16:57:25 INFO Received complete push request method=POST path=/47272026/11/12 16:57:25 INFO Completed upload id=26 objects=3 duration=795.408µs47282026/11/12 16:57:25 INFO Completed push id=24 objects=5 duration=804.548µs47292026/11/12 16:57:25 INFO Received complete upload request method=POST path=/47302026/11/12 16:57:25 INFO Received uploads request method=POST path=/47312026/11/12 16:57:25 INFO Completed upload id=31 objects=3 duration=2.082739ms47322026/11/12 16:57:25 INFO Received complete push request method=POST path=/47332026/11/12 16:57:25 INFO Received push request method=POST path=/47342026/11/12 16:57:25 INFO Completed push id=24 objects=6 duration=805.167µs47352026/11/12 16:57:25 INFO Received push request method=POST path=/47362026/11/12 16:57:25 INFO Received complete upload request method=POST path=/47372026/11/12 16:57:25 INFO Completed upload id=25 objects=4 duration=849.728µs47382026/11/12 16:57:25 INFO Received complete upload request method=POST path=/47392026/11/12 16:57:25 INFO Received complete push request method=POST path=/47402026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=047412026/11/12 16:57:25 INFO Completed upload id=25 objects=6 duration=1.277072ms47422026/11/12 16:57:25 INFO Completed push id=23 objects=5 duration=1.163911ms47432026/11/12 16:57:25 INFO Received complete upload request method=POST path=/47442026/11/12 16:57:25 INFO Received uploads request method=POST path=/47452026/11/12 16:57:25 INFO Completed upload id=8 objects=6 duration=835.748µs47462026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=047472026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period47482026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047492026/11/12 16:57:25 INFO Vacuumed table table=pending_closures47502026/11/12 16:57:25 INFO Received complete push request method=POST path=/47512026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047522026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts47532026/11/12 16:57:25 INFO Received complete upload request method=POST path=/47542026/11/12 16:57:25 INFO Completed push id=13 objects=22 duration=1.188851ms47552026/11/12 16:57:25 INFO Completed upload id=25 objects=2 duration=941.649µs47562026/11/12 16:57:25 INFO Vacuumed table table=pending_closures47572026/11/12 16:57:25 INFO Vacuumed table table=pending_objects47582026/11/12 16:57:25 INFO Received complete upload request method=POST path=/47592026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads47602026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts47612026/11/12 16:57:25 INFO Vacuumed table table=closures47622026/11/12 16:57:25 INFO Completed upload id=22 objects=2 duration=738.646µs47632026/11/12 16:57:25 INFO Received push request method=POST path=/47642026/11/12 16:57:25 INFO Vacuumed table table=pending_objects47652026/11/12 16:57:25 INFO Vacuumed table table=objects47662026/11/12 16:57:25 INFO Received complete upload request method=POST path=/47672026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads47682026/11/12 16:57:25 INFO Vacuumed table table=closures47692026/11/12 16:57:25 INFO Completed upload id=33 objects=9 duration=870.108µs47702026/11/12 16:57:25 INFO Vacuumed table table=objects47712026/11/12 16:57:25 INFO Received push request method=POST path=/47722026/11/12 16:57:25 INFO Received complete upload request method=POST path=/47732026/11/12 16:57:25 INFO Completed upload id=24 objects=3 duration=749.047µs47742026/11/12 16:57:25 INFO Received complete push request method=POST path=/47752026/11/12 16:57:25 INFO Completed push id=25 objects=7 duration=916.129µs47762026/11/12 16:57:25 INFO Received complete upload request method=POST path=/47772026/11/12 16:57:25 INFO Received complete push request method=POST path=/47782026/11/12 16:57:25 INFO Completed upload id=26 objects=20 duration=793.987µs47792026/11/12 16:57:25 INFO Completed push id=21 objects=6 duration=1.001009ms47802026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=047812026/11/12 16:57:25 INFO Received push request method=POST path=/47822026/11/12 16:57:25 INFO Received uploads request method=POST path=/47832026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=047842026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period47852026/11/12 16:57:25 INFO Received complete push request method=POST path=/47862026/11/12 16:57:25 INFO Received push request method=POST path=/47872026/11/12 16:57:25 INFO Completed push id=16 objects=26 duration=1.05339ms47882026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period47892026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047902026/11/12 16:57:25 INFO Vacuumed table table=pending_closures47912026/11/12 16:57:25 INFO Received complete push request method=POST path=/47922026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts47932026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047942026/11/12 16:57:25 INFO Completed push id=25 objects=4 duration=838.408µs47952026/11/12 16:57:25 INFO Vacuumed table table=pending_objects47962026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads47972026/11/12 16:57:25 INFO Vacuumed table table=pending_closures47982026/11/12 16:57:25 INFO Vacuumed table table=closures47992026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts48002026/11/12 16:57:25 INFO Vacuumed table table=objects48012026/11/12 16:57:25 INFO Vacuumed table table=pending_objects48022026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads48032026/11/12 16:57:25 INFO Vacuumed table table=closures48042026/11/12 16:57:25 INFO Vacuumed table table=objects48052026/11/12 16:57:25 INFO Received complete upload request method=POST path=/48062026/11/12 16:57:25 INFO Completed upload id=22 objects=7 duration=622.165µs48072026/11/12 16:57:25 INFO Received complete push request method=POST path=/48082026/11/12 16:57:25 INFO Completed push id=28 objects=11 duration=1.929298ms48092026/11/12 16:57:25 INFO Received complete push request method=POST path=/48102026/11/12 16:57:25 INFO Completed push id=28 objects=7 duration=827.388µs48112026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=048122026/11/12 16:57:25 INFO Received uploads request method=POST path=/48132026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=048142026/11/12 16:57:25 INFO Received complete upload request method=POST path=/48152026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period48162026/11/12 16:57:25 INFO Received complete push request method=POST path=/48172026/11/12 16:57:25 INFO Completed upload id=12 objects=9 duration=834.568µs48182026/11/12 16:57:25 INFO Completed push id=20 objects=8 duration=757.987µs48192026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048202026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period48212026/11/12 16:57:25 INFO Received complete upload request method=POST path=/48222026/11/12 16:57:25 INFO Vacuumed table table=pending_closures48232026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts48242026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048252026/11/12 16:57:25 INFO Vacuumed table table=pending_objects48262026/11/12 16:57:25 INFO Completed upload id=18 objects=2 duration=1.544574ms48272026/11/12 16:57:25 INFO Received complete push request method=POST path=/48282026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads48292026/11/12 16:57:25 INFO Vacuumed table table=pending_closures48302026/11/12 16:57:25 INFO Vacuumed table table=closures48312026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts48322026/11/12 16:57:25 INFO Vacuumed table table=objects48332026/11/12 16:57:25 INFO Received complete upload request method=POST path=/48342026/11/12 16:57:25 INFO Received complete push request method=POST path=/48352026/11/12 16:57:25 INFO Completed push id=5 objects=28 duration=1.741236ms48362026/11/12 16:57:25 INFO Vacuumed table table=pending_objects48372026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads48382026/11/12 16:57:25 INFO Completed push id=24 objects=4 duration=822.628µs48392026/11/12 16:57:25 INFO Completed upload id=26 objects=24 duration=1.11381ms48402026/11/12 16:57:25 INFO Vacuumed table table=closures48412026/11/12 16:57:25 INFO Vacuumed table table=objects48422026/11/12 16:57:25 INFO Received push request method=POST path=/48432026/11/12 16:57:25 INFO Received complete push request method=POST path=/48442026/11/12 16:57:25 INFO Received complete push request method=POST path=/48452026/11/12 16:57:25 INFO Received complete upload request method=POST path=/48462026/11/12 16:57:25 INFO Received uploads request method=POST path=/48472026/11/12 16:57:25 INFO Completed push id=9 objects=9 duration=1.964838ms48482026/11/12 16:57:25 INFO Completed upload id=30 objects=6 duration=660.126µs48492026/11/12 16:57:25 INFO Completed push id=11 objects=9 duration=1.406213ms48502026/11/12 16:57:25 INFO Received push request method=POST path=/48512026/11/12 16:57:25 INFO Received complete push request method=POST path=/48522026/11/12 16:57:25 INFO Received complete push request method=POST path=/48532026/11/12 16:57:25 INFO Completed push id=10 objects=2 duration=1.187971ms48542026/11/12 16:57:25 INFO Completed push id=25 objects=9 duration=727.327µs48552026/11/12 16:57:25 INFO Received complete push request method=POST path=/48562026/11/12 16:57:25 INFO Received complete push request method=POST path=/48572026/11/12 16:57:25 INFO Completed push id=27 objects=6 duration=2.454603ms48582026/11/12 16:57:25 INFO Received complete push request method=POST path=/48592026/11/12 16:57:25 INFO Received complete push request method=POST path=/48602026/11/12 16:57:25 INFO Received complete push request method=POST path=/48612026/11/12 16:57:25 INFO Completed push id=23 objects=28 duration=2.799866ms48622026/11/12 16:57:25 INFO Completed push id=19 objects=14 duration=1.405333ms48632026/11/12 16:57:25 INFO Completed push id=29 objects=24 duration=997.049µs48642026/11/12 16:57:25 INFO Completed push id=26 objects=23 duration=1.06353ms48652026/11/12 16:57:25 INFO Received complete push request method=POST path=/48662026/11/12 16:57:25 INFO Completed push id=31 objects=5 duration=636.346µs48672026/11/12 16:57:25 INFO Received push request method=POST path=/48682026/11/12 16:57:25 INFO Received complete upload request method=POST path=/48692026/11/12 16:57:25 INFO Completed upload id=21 objects=8 duration=932.089µs48702026/11/12 16:57:25 INFO Received complete push request method=POST path=/48712026/11/12 16:57:25 INFO Completed push id=15 objects=24 duration=1.211451ms48722026/11/12 16:57:25 INFO Received complete upload request method=POST path=/48732026/11/12 16:57:25 INFO Completed upload id=26 objects=3 duration=884.449µs48742026/11/12 16:57:25 INFO Received complete upload request method=POST path=/48752026/11/12 16:57:25 INFO Completed upload id=17 objects=5 duration=688.746µs48762026/11/12 16:57:25 INFO Received complete upload request method=POST path=/48772026/11/12 16:57:25 INFO Received uploads request method=POST path=/48782026/11/12 16:57:25 INFO Received complete upload request method=POST path=/48792026/11/12 16:57:25 INFO Completed upload id=13 objects=7 duration=846.168µs48802026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=048812026/11/12 16:57:25 INFO Completed upload id=27 objects=26 duration=818.468µs48822026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period48832026/11/12 16:57:25 INFO Received complete push request method=POST path=/48842026/11/12 16:57:25 INFO Completed push id=22 objects=6 duration=679.727µs48852026/11/12 16:57:25 INFO Received complete upload request method=POST path=/48862026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=048872026/11/12 16:57:25 INFO Completed upload id=19 objects=2 duration=771.387µs48882026/11/12 16:57:25 INFO Vacuumed table table=pending_closures48892026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts48902026/11/12 16:57:25 INFO Vacuumed table table=pending_objects48912026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads48922026/11/12 16:57:25 INFO Received complete push request method=POST path=/48932026/11/12 16:57:25 INFO Vacuumed table table=closures48942026/11/12 16:57:25 INFO Vacuumed table table=objects48952026/11/12 16:57:25 INFO Completed push id=32 objects=26 duration=1.04363ms48962026/11/12 16:57:25 INFO Received complete upload request method=POST path=/48972026/11/12 16:57:25 INFO Received complete upload request method=POST path=/48982026/11/12 16:57:25 INFO Completed upload id=20 objects=4 duration=641.346µs48992026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49002026/11/12 16:57:25 INFO Completed upload id=15 objects=2 duration=899.769µs49012026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49022026/11/12 16:57:25 INFO Received complete push request method=POST path=/49032026/11/12 16:57:25 INFO Completed upload id=27 objects=7 duration=791.127µs49042026/11/12 16:57:25 INFO Received complete push request method=POST path=/49052026/11/12 16:57:25 INFO Completed upload id=27 objects=8 duration=1.15343ms49062026/11/12 16:57:25 INFO Received complete push request method=POST path=/49072026/11/12 16:57:25 INFO Completed push id=28 objects=6 duration=872.608µs49082026/11/12 16:57:25 INFO Completed push id=27 objects=15 duration=815.948µs49092026/11/12 16:57:25 INFO Completed push id=24 objects=8 duration=831.707µs49102026/11/12 16:57:25 INFO Received complete push request method=POST path=/49112026/11/12 16:57:25 INFO Received complete push request method=POST path=/49122026/11/12 16:57:25 INFO Completed push id=19 objects=3 duration=757.327µs49132026/11/12 16:57:25 INFO Completed push id=27 objects=11 duration=953.369µs49142026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49152026/11/12 16:57:25 INFO Completed upload id=24 objects=7 duration=1.04779ms49162026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49172026/11/12 16:57:25 INFO Completed upload id=20 objects=6 duration=866.548µs49182026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49192026/11/12 16:57:25 INFO Completed upload id=33 objects=2 duration=681.266µs49202026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49212026/11/12 16:57:25 INFO Completed upload id=23 objects=7 duration=799.648µs49222026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49232026/11/12 16:57:25 INFO Completed upload id=28 objects=3 duration=574.665µs49242026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49252026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49262026/11/12 16:57:25 INFO Received complete push request method=POST path=/49272026/11/12 16:57:25 INFO Received push request method=POST path=/49282026/11/12 16:57:25 INFO Completed upload id=25 objects=9 duration=960.849µs49292026/11/12 16:57:25 INFO Completed upload id=18 objects=22 duration=1.013889ms49302026/11/12 16:57:25 INFO Completed push id=29 objects=6 duration=971.169µs49312026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49322026/11/12 16:57:25 INFO Completed upload id=29 objects=26 duration=721.807µs49332026/11/12 16:57:25 INFO Received complete push request method=POST path=/49342026/11/12 16:57:25 INFO Completed push id=26 objects=3 duration=977.689µs49352026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49362026/11/12 16:57:25 INFO Completed upload id=34 objects=5 duration=717.087µs49372026/11/12 16:57:25 INFO Received complete push request method=POST path=/49382026/11/12 16:57:25 INFO Completed push id=20 objects=4 duration=948.029µs49392026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49402026/11/12 16:57:25 INFO Completed upload id=27 objects=3 duration=736.946µs49412026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49422026/11/12 16:57:25 INFO Completed upload id=21 objects=8 duration=650.886µs49432026/11/12 16:57:25 INFO Received complete push request method=POST path=/49442026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=049452026/11/12 16:57:25 INFO Received complete push request method=POST path=/49462026/11/12 16:57:25 INFO Completed push id=29 objects=23 duration=986.729µs49472026/11/12 16:57:25 INFO Completed push id=30 objects=8 duration=828.027µs49482026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49492026/11/12 16:57:25 INFO Received uploads request method=POST path=/49502026/11/12 16:57:25 INFO Completed upload id=28 objects=8 duration=787.588µs49512026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049522026/11/12 16:57:25 INFO Vacuumed table table=pending_closures49532026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts49542026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49552026/11/12 16:57:25 INFO Received complete push request method=POST path=/49562026/11/12 16:57:25 INFO Completed upload id=35 objects=11 duration=562.486µs49572026/11/12 16:57:25 INFO Completed push id=28 objects=19 duration=985.269µs49582026/11/12 16:57:25 INFO Vacuumed table table=pending_objects49592026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads49602026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49612026/11/12 16:57:25 INFO Vacuumed table table=closures49622026/11/12 16:57:25 INFO Completed upload id=28 objects=6 duration=677.166µs49632026/11/12 16:57:25 INFO Vacuumed table table=objects49642026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49652026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49662026/11/12 16:57:25 INFO Completed upload id=30 objects=8 duration=853.768µs49672026/11/12 16:57:25 INFO Completed upload id=31 objects=5 duration=658.106µs49682026/11/12 16:57:25 INFO Received complete push request method=POST path=/49692026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49702026/11/12 16:57:25 INFO Completed push id=30 objects=6 duration=782.628µs49712026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49722026/11/12 16:57:25 INFO Completed upload id=24 objects=18 duration=928.569µs49732026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49742026/11/12 16:57:25 INFO Completed upload id=29 objects=10 duration=646.466µs49752026/11/12 16:57:25 INFO Completed upload id=22 objects=7 duration=827.128µs49762026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49772026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49782026/11/12 16:57:25 INFO Completed upload id=36 objects=3 duration=756.267µs49792026/11/12 16:57:25 INFO Completed upload id=22 objects=8 duration=708.987µs49802026/11/12 16:57:25 INFO Received complete push request method=POST path=/49812026/11/12 16:57:25 INFO Completed push id=30 objects=4 duration=687.807µs49822026/11/12 16:57:25 INFO Received push request method=POST path=/49832026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=049842026/11/12 16:57:25 INFO Received complete push request method=POST path=/49852026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049862026/11/12 16:57:25 INFO Completed push id=32 objects=3 duration=804.408µs49872026/11/12 16:57:25 INFO Vacuumed table table=pending_closures49882026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts49892026/11/12 16:57:25 INFO Received complete push request method=POST path=/49902026/11/12 16:57:25 INFO Vacuumed table table=pending_objects49912026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads49922026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49932026/11/12 16:57:25 INFO Received complete upload request method=POST path=/49942026/11/12 16:57:25 INFO Vacuumed table table=closures49952026/11/12 16:57:25 INFO Completed push id=25 objects=2 duration=1.360513ms49962026/11/12 16:57:25 INFO Received complete push request method=POST path=/49972026/11/12 16:57:25 INFO Vacuumed table table=objects49982026/11/12 16:57:25 INFO Completed upload id=21 objects=2 duration=907.049µs49992026/11/12 16:57:25 INFO Received complete upload request method=POST path=/50002026/11/12 16:57:25 INFO Completed upload id=30 objects=20 duration=963.168µs50012026/11/12 16:57:25 INFO Received complete push request method=POST path=/50022026/11/12 16:57:25 INFO Completed push id=24 objects=5 duration=942.488µs50032026/11/12 16:57:25 INFO Completed upload id=31 objects=6 duration=793.827µs50042026/11/12 16:57:25 INFO Received complete push request method=POST path=/50052026/11/12 16:57:25 INFO Completed push id=29 objects=10 duration=773.427µs50062026/11/12 16:57:25 INFO Received complete push request method=POST path=/50072026/11/12 16:57:25 INFO Completed push id=33 objects=3 duration=740.287µs50082026/11/12 16:57:25 INFO Received complete push request method=POST path=/50092026/11/12 16:57:25 INFO Completed push id=31 objects=7 duration=604.406µs50102026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=050112026/11/12 16:57:25 INFO Completed push id=23 objects=5 duration=669.146µs50122026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period50132026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050142026/11/12 16:57:25 INFO Received uploads request method=POST path=/50152026/11/12 16:57:25 INFO Vacuumed table table=pending_closures50162026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts50172026/11/12 16:57:25 INFO Aborted multipart uploads count=0 kept=050182026/11/12 16:57:25 INFO Vacuumed table table=pending_objects50192026/11/12 16:57:25 INFO Received complete push request method=POST path=/50202026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads50212026/11/12 16:57:25 INFO Vacuumed table table=closures50222026/11/12 16:57:25 INFO Completed push id=27 objects=4 duration=894.688µs50232026/11/12 16:57:25 INFO Vacuumed table table=objects50242026/11/12 16:57:25 WARN Force mode enabled - objects will be deleted immediately without grace period50252026/11/12 16:57:25 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050262026/11/12 16:57:25 INFO Received uploads request method=POST path=/50272026/11/12 16:57:25 INFO Received complete upload request method=POST path=/50282026/11/12 16:57:25 INFO Received complete upload request method=POST path=/50292026/11/12 16:57:25 INFO Received complete upload request method=POST path=/50302026/11/12 16:57:25 INFO Received complete push request method=POST path=/50312026/11/12 16:57:25 INFO Vacuumed table table=pending_closures50322026/11/12 16:57:25 INFO Vacuumed table table=commit_receipts50332026/11/12 16:57:25 INFO Completed upload id=32 objects=5 duration=1.03843ms50342026/11/12 16:57:25 INFO Completed upload id=32 objects=4 duration=843.727µs50352026/11/12 16:57:25 INFO Received complete push request method=POST path=/50362026/11/12 16:57:25 INFO Completed push id=31 objects=11 duration=1.149331ms50372026/11/12 16:57:25 INFO Completed upload id=25 objects=16 duration=2.043979ms50382026/11/12 16:57:25 INFO Received complete upload request method=POST path=/50392026/11/12 16:57:25 INFO Completed push id=34 objects=6 duration=762.707µs50402026/11/12 16:57:25 INFO Vacuumed table table=pending_objects50412026/11/12 16:57:25 INFO Completed upload id=24 objects=8 duration=584.505µs50422026/11/12 16:57:25 INFO Vacuumed table table=multipart_uploads50432026/11/12 16:57:25 INFO Vacuumed table table=closures50442026/11/12 16:57:25 INFO Received complete upload request method=POST path=/50452026/11/12 16:57:25 INFO Vacuumed table table=objects50462026/11/12 16:57:25 INFO Completed upload id=25 objects=8 duration=840.948µs50472026/11/12 16:57:25 INFO Received complete upload request method=POST path=/50482026/11/12 16:57:25 INFO Completed upload id=30 objects=6 duration=759.787µs50492026/11/12 16:57:25 INFO Received uploads request method=POST path=/50502026/11/12 16:57:26 INFO Received complete upload request method=POST path=/50512026/11/12 16:57:26 INFO Completed upload id=32 objects=2 duration=630.246µs50522026/11/12 16:57:26 INFO Received complete push request method=POST path=/50532026/11/12 16:57:26 INFO Received complete upload request method=POST path=/50542026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=050552026/11/12 16:57:26 INFO Completed push id=35 objects=8 duration=851.208µs50562026/11/12 16:57:26 INFO Completed upload id=28 objects=5 duration=945.509µs50572026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=050582026/11/12 16:57:26 INFO Received complete push request method=POST path=/50592026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period50602026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period50612026/11/12 16:57:26 INFO Received complete upload request method=POST path=/50622026/11/12 16:57:26 INFO Received complete upload request method=POST path=/50632026/11/12 16:57:26 INFO Completed push id=26 objects=4 duration=921.689µs50642026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050652026/11/12 16:57:26 INFO Received complete push request method=POST path=/50662026/11/12 16:57:26 INFO Completed upload id=33 objects=10 duration=605.466µs50672026/11/12 16:57:26 INFO Completed upload id=31 objects=5 duration=807.167µs50682026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050692026/11/12 16:57:26 INFO Vacuumed table table=pending_closures50702026/11/12 16:57:26 INFO Completed push id=28 objects=26 duration=1.01939ms50712026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts50722026/11/12 16:57:26 INFO Received complete upload request method=POST path=/50732026/11/12 16:57:26 INFO Vacuumed table table=pending_closures50742026/11/12 16:57:26 INFO Vacuumed table table=pending_objects50752026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts50762026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads50772026/11/12 16:57:26 INFO Completed upload id=26 objects=8 duration=836.487µs50782026/11/12 16:57:26 INFO Vacuumed table table=closures50792026/11/12 16:57:26 INFO Vacuumed table table=pending_objects50802026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads50812026/11/12 16:57:26 INFO Vacuumed table table=objects50822026/11/12 16:57:26 INFO Vacuumed table table=closures50832026/11/12 16:57:26 INFO Vacuumed table table=objects50842026/11/12 16:57:26 INFO Received complete push request method=POST path=/50852026/11/12 16:57:26 INFO Completed push id=33 objects=3 duration=757.107µs50862026/11/12 16:57:26 INFO Received complete upload request method=POST path=/50872026/11/12 16:57:26 INFO Completed upload id=36 objects=9 duration=794.827µs50882026/11/12 16:57:26 INFO Received complete push request method=POST path=/50892026/11/12 16:57:26 INFO Completed push id=30 objects=2 duration=755.927µs50902026/11/12 16:57:26 INFO Received complete upload request method=POST path=/50912026/11/12 16:57:26 INFO Received complete push request method=POST path=/50922026/11/12 16:57:26 INFO Received complete upload request method=POST path=/50932026/11/12 16:57:26 INFO Completed upload id=34 objects=3 duration=777.287µs50942026/11/12 16:57:26 INFO Completed push id=30 objects=3 duration=849.007µs50952026/11/12 16:57:26 INFO Completed upload id=30 objects=5 duration=770.967µs50962026/11/12 16:57:26 INFO Received complete push request method=POST path=/50972026/11/12 16:57:26 INFO Received push request method=POST path=/50982026/11/12 16:57:26 INFO Completed push id=28 objects=14 duration=1.10493ms50992026/11/12 16:57:26 INFO Received complete upload request method=POST path=/51002026/11/12 16:57:26 INFO Received complete upload request method=POST path=/51012026/11/12 16:57:26 INFO Completed upload id=34 objects=3 duration=764.507µs51022026/11/12 16:57:26 INFO Completed upload id=34 objects=24 duration=1.217111ms51032026/11/12 16:57:26 INFO Received complete upload request method=POST path=/51042026/11/12 16:57:26 INFO Completed upload id=34 objects=23 duration=793.087µs51052026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=051062026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period51072026/11/12 16:57:26 INFO Received complete push request method=POST path=/51082026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051092026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=051102026/11/12 16:57:26 INFO Completed push id=32 objects=19 duration=1.107791ms51112026/11/12 16:57:26 INFO Vacuumed table table=pending_closures51122026/11/12 16:57:26 INFO Received push request method=POST path=/51132026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period51142026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts51152026/11/12 16:57:26 INFO Received uploads request method=POST path=/51162026/11/12 16:57:26 INFO Vacuumed table table=pending_objects51172026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051182026/11/12 16:57:26 INFO Received push request method=POST path=/51192026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads51202026/11/12 16:57:26 INFO Received complete push request method=POST path=/51212026/11/12 16:57:26 INFO Vacuumed table table=closures51222026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=051232026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=051242026/11/12 16:57:26 INFO Vacuumed table table=pending_closures51252026/11/12 16:57:26 INFO Completed push id=31 objects=3 duration=968.889µs51262026/11/12 16:57:26 INFO Received complete push request method=POST path=/51272026/11/12 16:57:26 INFO Vacuumed table table=objects51282026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts51292026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period51302026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period51312026/11/12 16:57:26 INFO Completed push id=32 objects=7 duration=753.447µs51322026/11/12 16:57:26 INFO Vacuumed table table=pending_objects51332026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051342026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051352026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads51362026/11/12 16:57:26 INFO Vacuumed table table=closures51372026/11/12 16:57:26 INFO Vacuumed table table=pending_closures51382026/11/12 16:57:26 INFO Vacuumed table table=pending_closures51392026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts51402026/11/12 16:57:26 INFO Vacuumed table table=objects51412026/11/12 16:57:26 INFO Received complete push request method=POST path=/51422026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts51432026/11/12 16:57:26 INFO Vacuumed table table=pending_objects51442026/11/12 16:57:26 INFO Completed push id=35 objects=6 duration=946.529µs51452026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads51462026/11/12 16:57:26 INFO Vacuumed table table=pending_objects51472026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads51482026/11/12 16:57:26 INFO Vacuumed table table=closures51492026/11/12 16:57:26 INFO Vacuumed table table=closures51502026/11/12 16:57:26 INFO Vacuumed table table=objects51512026/11/12 16:57:26 INFO Vacuumed table table=objects51522026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=051532026/11/12 16:57:26 INFO Received uploads request method=POST path=/51542026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period51552026/11/12 16:57:26 INFO Received push request method=POST path=/51562026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051572026/11/12 16:57:26 INFO Vacuumed table table=pending_closures51582026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts51592026/11/12 16:57:26 INFO Vacuumed table table=pending_objects51602026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads51612026/11/12 16:57:26 INFO Vacuumed table table=closures51622026/11/12 16:57:26 INFO Vacuumed table table=objects51632026/11/12 16:57:26 INFO Received complete upload request method=POST path=/51642026/11/12 16:57:26 INFO Completed upload id=33 objects=6 duration=753.467µs51652026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=051662026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period51672026/11/12 16:57:26 INFO Received complete push request method=POST path=/51682026/11/12 16:57:26 INFO Completed push id=32 objects=6 duration=918.069µs51692026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=051702026/11/12 16:57:26 INFO Received complete push request method=POST path=/51712026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period51722026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=051732026/11/12 16:57:26 INFO Completed push id=24 objects=24 duration=2.093259ms51742026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051752026/11/12 16:57:26 INFO Vacuumed table table=pending_closures51762026/11/12 16:57:26 INFO Vacuumed table table=pending_closures51772026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts51782026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts51792026/11/12 16:57:26 INFO Vacuumed table table=pending_objects51802026/11/12 16:57:26 INFO Vacuumed table table=pending_objects51812026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads51822026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads51832026/11/12 16:57:26 INFO Vacuumed table table=closures51842026/11/12 16:57:26 INFO Received complete upload request method=POST path=/51852026/11/12 16:57:26 INFO Vacuumed table table=closures51862026/11/12 16:57:26 INFO Vacuumed table table=objects51872026/11/12 16:57:26 INFO Completed upload id=34 objects=3 duration=610.866µs51882026/11/12 16:57:26 INFO Vacuumed table table=objects51892026/11/12 16:57:26 INFO Received complete upload request method=POST path=/51902026/11/12 16:57:26 INFO Completed upload id=33 objects=7 duration=737.187µs51912026/11/12 16:57:26 INFO Received uploads request method=POST path=/51922026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=051932026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period51942026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051952026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=051962026/11/12 16:57:26 INFO Vacuumed table table=pending_closures51972026/11/12 16:57:26 INFO Received uploads request method=POST path=/51982026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts51992026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period52002026/11/12 16:57:26 INFO Received complete upload request method=POST path=/52012026/11/12 16:57:26 INFO Vacuumed table table=pending_objects52022026/11/12 16:57:26 INFO Completed upload id=34 objects=3 duration=638.126µs52032026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052042026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads52052026/11/12 16:57:26 INFO Vacuumed table table=closures52062026/11/12 16:57:26 INFO Vacuumed table table=objects52072026/11/12 16:57:26 INFO Vacuumed table table=pending_closures52082026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts52092026/11/12 16:57:26 INFO Received complete upload request method=POST path=/52102026/11/12 16:57:26 INFO Vacuumed table table=pending_objects52112026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads52122026/11/12 16:57:26 INFO Vacuumed table table=closures52132026/11/12 16:57:26 INFO Completed upload id=22 objects=5 duration=1.312913ms52142026/11/12 16:57:26 INFO Vacuumed table table=objects52152026/11/12 16:57:26 INFO Received complete upload request method=POST path=/52162026/11/12 16:57:26 INFO Completed upload id=21 objects=9 duration=1.507154ms52172026/11/12 16:57:26 INFO Received complete push request method=POST path=/52182026/11/12 16:57:26 INFO Received uploads request method=POST path=/52192026/11/12 16:57:26 INFO Completed push id=35 objects=11 duration=910.189µs52202026/11/12 16:57:26 INFO Received complete push request method=POST path=/52212026/11/12 16:57:26 INFO Completed push id=28 objects=3 duration=1.309552ms52222026/11/12 16:57:26 INFO Received complete push request method=POST path=/52232026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=052242026/11/12 16:57:26 INFO Received uploads request method=POST path=/52252026/11/12 16:57:26 INFO Completed push id=25 objects=10 duration=1.547035ms52262026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period52272026/11/12 16:57:26 INFO Received complete push request method=POST path=/52282026/11/12 16:57:26 INFO Completed push id=29 objects=15 duration=765.107µs52292026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=6 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052302026/11/12 16:57:26 INFO Vacuumed table table=pending_closures52312026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts52322026/11/12 16:57:26 INFO Vacuumed table table=pending_objects52332026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads52342026/11/12 16:57:26 INFO Vacuumed table table=closures52352026/11/12 16:57:26 INFO Vacuumed table table=objects52362026/11/12 16:57:26 INFO Received complete upload request method=POST path=/52372026/11/12 16:57:26 INFO Completed upload id=29 objects=5 duration=641.746µs52382026/11/12 16:57:26 INFO Received complete push request method=POST path=/52392026/11/12 16:57:26 INFO Completed push id=30 objects=6 duration=757.867µs52402026/11/12 16:57:26 INFO Received complete upload request method=POST path=/52412026/11/12 16:57:26 INFO Received push request method=POST path=/52422026/11/12 16:57:26 INFO Completed upload id=30 objects=2 duration=498.065µs52432026/11/12 16:57:26 INFO Received complete upload request method=POST path=/52442026/11/12 16:57:26 INFO Completed upload id=31 objects=3 duration=644.326µs52452026/11/12 16:57:26 INFO Received complete push request method=POST path=/52462026/11/12 16:57:26 INFO Received uploads request method=POST path=/52472026/11/12 16:57:26 INFO Completed push id=31 objects=20 duration=750.927µs52482026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=052492026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period52502026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052512026/11/12 16:57:26 INFO Vacuumed table table=pending_closures52522026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts52532026/11/12 16:57:26 INFO Vacuumed table table=pending_objects52542026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads52552026/11/12 16:57:26 INFO Vacuumed table table=closures52562026/11/12 16:57:26 INFO Vacuumed table table=objects52572026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=052582026/11/12 16:57:26 INFO Received complete upload request method=POST path=/52592026/11/12 16:57:26 INFO Completed upload id=32 objects=5 duration=671.786µs52602026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period52612026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052622026/11/12 16:57:26 INFO Vacuumed table table=pending_closures52632026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts52642026/11/12 16:57:26 INFO Vacuumed table table=pending_objects52652026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads52662026/11/12 16:57:26 INFO Vacuumed table table=closures52672026/11/12 16:57:26 INFO Vacuumed table table=objects52682026/11/12 16:57:26 INFO Received complete upload request method=POST path=/52692026/11/12 16:57:26 INFO Completed upload id=33 objects=3 duration=609.265µs52702026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=052712026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period52722026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052732026/11/12 16:57:26 INFO Vacuumed table table=pending_closures52742026/11/12 16:57:26 INFO Received push request method=POST path=/52752026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts52762026/11/12 16:57:26 INFO Vacuumed table table=pending_objects52772026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads52782026/11/12 16:57:26 INFO Vacuumed table table=closures52792026/11/12 16:57:26 INFO Vacuumed table table=objects52802026/11/12 16:57:26 INFO Received complete upload request method=POST path=/52812026/11/12 16:57:26 INFO Completed upload id=23 objects=7 duration=1.932858ms52822026/11/12 16:57:26 INFO Received complete upload request method=POST path=/52832026/11/12 16:57:26 INFO Completed upload id=26 objects=5 duration=628.306µs52842026/11/12 16:57:26 INFO Received complete push request method=POST path=/52852026/11/12 16:57:26 INFO Completed push id=27 objects=3 duration=1.178031ms52862026/11/12 16:57:26 INFO Received complete upload request method=POST path=/52872026/11/12 16:57:26 INFO Completed upload id=28 objects=5 duration=662.227µs52882026/11/12 16:57:26 INFO Received complete push request method=POST path=/52892026/11/12 16:57:26 INFO Completed push id=29 objects=17 duration=959.528µs52902026/11/12 16:57:26 INFO Aborted multipart uploads count=0 kept=052912026/11/12 16:57:26 WARN Force mode enabled - objects will be deleted immediately without grace period52922026/11/12 16:57:26 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052932026/11/12 16:57:26 INFO Vacuumed table table=pending_closures52942026/11/12 16:57:26 INFO Vacuumed table table=commit_receipts52952026/11/12 16:57:26 INFO Vacuumed table table=pending_objects52962026/11/12 16:57:26 INFO Vacuumed table table=multipart_uploads52972026/11/12 16:57:26 INFO Vacuumed table table=closures52982026/11/12 16:57:26 INFO Vacuumed table table=objects5299--- PASS: FuzzPushGC (0.00s)5300 --- PASS: FuzzPushGC/seed#14 (1.80s)5301 --- PASS: FuzzPushGC/seed#0 (1.84s)5302 --- PASS: FuzzPushGC/seed#2 (1.90s)5303 --- PASS: FuzzPushGC/seed#11 (1.96s)5304 --- PASS: FuzzPushGC/seed#3 (1.97s)5305 --- PASS: FuzzPushGC/seed#4 (2.01s)5306 --- PASS: FuzzPushGC/seed#13 (2.02s)5307 --- PASS: FuzzPushGC/seed#5 (2.02s)5308 --- PASS: FuzzPushGC/seed#9 (2.02s)5309 --- PASS: FuzzPushGC/seed#7 (2.03s)5310 --- PASS: FuzzPushGC/seed#12 (2.04s)5311 --- PASS: FuzzPushGC/seed#10 (2.06s)5312 --- PASS: FuzzPushGC/seed#8 (2.08s)5313 --- PASS: FuzzPushGC/seed#15 (2.12s)5314 --- PASS: FuzzPushGC/seed#6 (2.15s)5315 --- PASS: FuzzPushGC/seed#1 (2.22s)5316PASS5317{"timestamp":"2026-11-12T16:57:26.264181363Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:36806","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(197)"}5318{"timestamp":"2026-11-12T16:57:26.264254743Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:52564","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}53192026-11-12 16:57:26.554 UTC [128] LOG: received smart shutdown request53202026-11-12 16:57:26.559 UTC [128] LOG: background worker "logical replication launcher" (PID 138) exited with exit code 153212026-11-12 16:57:27.828 UTC [133] LOG: checkpoint complete: wrote 3968 buffers (24.2%), wrote 5 SLRU buffers; 0 WAL file(s) added, 0 removed, 33 recycled; write=14.946 s, sync=1.161 s, total=16.158 s; sync files=43130, longest=0.002 s, average=0.001 s; distance=533234 kB, estimate=533234 kB; lsn=0/276BC1E8, redo lsn=0/2202253853222026-11-12 16:57:27.833 UTC [133] LOG: shutting down53232026-11-12 16:57:27.833 UTC [133] LOG: checkpoint starting: shutdown immediate53242026-11-12 16:57:28.323 UTC [133] LOG: checkpoint complete: wrote 9836 buffers (60.0%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 5 recycled; write=0.130 s, sync=0.353 s, total=0.490 s; sync files=3109, longest=0.002 s, average=0.001 s; distance=88679 kB, estimate=488778 kB; lsn=0/276BC260, redo lsn=0/276BC26053252026-11-12 16:57:28.404 UTC [128] LOG: database system is shut down5326Running OIDC tests...5327=== RUN TestAudienceForIssuer5328=== PAUSE TestAudienceForIssuer5329=== RUN TestProviderClientGivesUpOnSilentIssuer5330=== PAUSE TestProviderClientGivesUpOnSilentIssuer5331=== RUN TestGlobMatch5332=== PAUSE TestGlobMatch5333=== RUN TestValidateToken_ValidToken5334=== PAUSE TestValidateToken_ValidToken5335=== RUN TestValidateToken_WrongAudience5336=== PAUSE TestValidateToken_WrongAudience5337=== RUN TestValidateToken_Expired5338=== PAUSE TestValidateToken_Expired5339=== RUN TestValidateToken_BoundClaimsMismatch5340=== PAUSE TestValidateToken_BoundClaimsMismatch5341=== RUN TestValidateToken_BoundSubjectMismatch5342=== PAUSE TestValidateToken_BoundSubjectMismatch5343=== RUN TestValidateToken_MultipleProviders5344=== PAUSE TestValidateToken_MultipleProviders5345=== RUN TestValidateToken_UnknownIssuerFetchesNoKeys5346=== PAUSE TestValidateToken_UnknownIssuerFetchesNoKeys5347=== RUN TestValidateToken_GoogleSchemelessIssuer5348=== PAUSE TestValidateToken_GoogleSchemelessIssuer5349=== RUN TestValidateToken_KubernetesServiceAccount5350=== PAUSE TestValidateToken_KubernetesServiceAccount5351=== RUN TestNewValidator_KubernetesRequiresCA5352=== PAUSE TestNewValidator_KubernetesRequiresCA5353=== RUN TestValidateToken_KubernetesIssuerFromOwnToken5354=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken5355=== RUN TestPins_ReservedForMatchingRule5356=== PAUSE TestPins_ReservedForMatchingRule5357=== RUN TestPins_TopLevelShorthand5358=== PAUSE TestPins_TopLevelShorthand5359=== RUN TestPins_ConfigValidation5360=== PAUSE TestPins_ConfigValidation5361=== RUN TestScopes_LegacyProviderDefaultsToWrite5362=== PAUSE TestScopes_LegacyProviderDefaultsToWrite5363=== RUN TestScopes_Rules5364=== PAUSE TestScopes_Rules5365=== RUN TestScopes_ConfigValidation5366=== PAUSE TestScopes_ConfigValidation5367=== CONT TestAudienceForIssuer5368=== CONT TestPins_TopLevelShorthand5369=== CONT TestValidateToken_GoogleSchemelessIssuer5370--- PASS: TestAudienceForIssuer (0.00s)5371=== CONT TestValidateToken_UnknownIssuerFetchesNoKeys5372=== CONT TestValidateToken_MultipleProviders5373=== CONT TestScopes_Rules5374=== CONT TestValidateToken_BoundSubjectMismatch5375=== CONT TestValidateToken_BoundClaimsMismatch5376=== CONT TestValidateToken_Expired53772026/11/12 16:57:30 INFO OIDC provider initialized name=known issuer=https://known.example5378=== CONT TestValidateToken_WrongAudience5379=== CONT TestValidateToken_ValidToken5380=== CONT TestGlobMatch5381=== RUN TestGlobMatch/foo_foo5382=== PAUSE TestGlobMatch/foo_foo5383=== RUN TestGlobMatch/foo_bar5384=== PAUSE TestGlobMatch/foo_bar5385=== RUN TestGlobMatch/*_5386=== PAUSE TestGlobMatch/*_5387=== RUN TestGlobMatch/*_anything5388=== PAUSE TestGlobMatch/*_anything5389=== RUN TestGlobMatch/foo*_foo5390=== CONT TestProviderClientGivesUpOnSilentIssuer5391=== CONT TestScopes_LegacyProviderDefaultsToWrite5392--- PASS: TestProviderClientGivesUpOnSilentIssuer (0.00s)5393=== CONT TestScopes_ConfigValidation5394=== CONT TestValidateToken_KubernetesIssuerFromOwnToken5395=== CONT TestPins_ReservedForMatchingRule5396--- PASS: TestScopes_ConfigValidation (0.00s)5397=== CONT TestNewValidator_KubernetesRequiresCA5398=== CONT TestPins_ConfigValidation5399=== CONT TestValidateToken_KubernetesServiceAccount5400=== PAUSE TestGlobMatch/foo*_foo5401=== RUN TestGlobMatch/foo*_foobar5402=== PAUSE TestGlobMatch/foo*_foobar5403=== RUN TestGlobMatch/foo*_bar5404=== PAUSE TestGlobMatch/foo*_bar5405=== RUN TestGlobMatch/*bar_bar5406=== PAUSE TestGlobMatch/*bar_bar5407=== RUN TestGlobMatch/*bar_foobar5408=== PAUSE TestGlobMatch/*bar_foobar5409=== RUN TestGlobMatch/*bar_foo5410=== PAUSE TestGlobMatch/*bar_foo5411=== RUN TestGlobMatch/foo*bar_foobar5412--- PASS: TestPins_ConfigValidation (0.00s)5413=== PAUSE TestGlobMatch/foo*bar_foobar5414=== RUN TestGlobMatch/foo*bar_foo123bar5415=== PAUSE TestGlobMatch/foo*bar_foo123bar5416=== RUN TestGlobMatch/foo*bar_foobarbaz5417=== PAUSE TestGlobMatch/foo*bar_foobarbaz5418=== RUN TestGlobMatch/*/*_foo/bar5419=== PAUSE TestGlobMatch/*/*_foo/bar5420=== RUN TestGlobMatch/*/*_foo5421=== PAUSE TestGlobMatch/*/*_foo5422=== RUN TestGlobMatch/refs/heads/*_refs/heads/main5423=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main5424=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.05425=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.05426=== RUN TestGlobMatch/refs/*/main_refs/heads/main5427=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main5428=== RUN TestGlobMatch/fo?_foo5429=== PAUSE TestGlobMatch/fo?_foo5430=== RUN TestGlobMatch/fo?_fo5431=== PAUSE TestGlobMatch/fo?_fo5432=== RUN TestGlobMatch/fo?_fooo5433=== PAUSE TestGlobMatch/fo?_fooo5434=== RUN TestGlobMatch/?oo_foo5435=== PAUSE TestGlobMatch/?oo_foo5436=== RUN TestGlobMatch/?oo_boo5437=== PAUSE TestGlobMatch/?oo_boo5438=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5439=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5440=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5441=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5442=== CONT TestGlobMatch/foo_foo5443=== CONT TestGlobMatch/*bar_foo5444=== CONT TestGlobMatch/foo*_foobar5445=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5446=== CONT TestGlobMatch/foo*_bar5447=== CONT TestGlobMatch/?oo_boo5448=== CONT TestGlobMatch/?oo_foo5449=== CONT TestGlobMatch/fo?_fooo5450=== CONT TestGlobMatch/fo?_fo5451=== CONT TestGlobMatch/*bar_foobar5452=== CONT TestGlobMatch/refs/heads/*_refs/heads/main5453=== CONT TestGlobMatch/refs/*/main_refs/heads/main5454=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.05455=== CONT TestGlobMatch/*bar_bar5456=== CONT TestGlobMatch/*/*_foo5457=== CONT TestGlobMatch/*/*_foo/bar5458=== CONT TestGlobMatch/foo*bar_foobarbaz5459=== CONT TestGlobMatch/foo*bar_foo123bar5460=== CONT TestGlobMatch/foo*bar_foobar5461=== CONT TestGlobMatch/foo*_foo5462=== CONT TestGlobMatch/*_anything5463=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5464=== CONT TestGlobMatch/*_5465=== CONT TestGlobMatch/foo_bar5466=== CONT TestGlobMatch/fo?_foo5467--- PASS: TestGlobMatch (0.00s)5468 --- PASS: TestGlobMatch/foo_foo (0.00s)5469 --- PASS: TestGlobMatch/*bar_foo (0.00s)5470 --- PASS: TestGlobMatch/foo*_foobar (0.00s)5471 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)5472 --- PASS: TestGlobMatch/foo*_bar (0.00s)5473 --- PASS: TestGlobMatch/?oo_boo (0.00s)5474 --- PASS: TestGlobMatch/?oo_foo (0.00s)5475 --- PASS: TestGlobMatch/fo?_fooo (0.00s)5476 --- PASS: TestGlobMatch/fo?_fo (0.00s)5477 --- PASS: TestGlobMatch/*bar_foobar (0.00s)5478 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)5479 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)5480 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)5481 --- PASS: TestGlobMatch/*bar_bar (0.00s)5482 --- PASS: TestGlobMatch/*/*_foo (0.00s)5483 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)5484 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)5485 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)5486 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)5487 --- PASS: TestGlobMatch/foo*_foo (0.00s)5488 --- PASS: TestGlobMatch/*_anything (0.00s)5489 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)5490 --- PASS: TestGlobMatch/*_ (0.00s)5491 --- PASS: TestGlobMatch/foo_bar (0.00s)5492 --- PASS: TestGlobMatch/fo?_foo (0.00s)54932026/11/12 16:57:30 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC12354942026/11/12 16:57:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:41133/oidc54952026/11/12 16:57:30 INFO OIDC provider initialized name=google issuer=https://accounts.google.com5496--- PASS: TestValidateToken_GoogleSchemelessIssuer (0.05s)5497--- PASS: TestPins_ReservedForMatchingRule (0.05s)5498--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (0.05s)54992026/11/12 16:57:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:40037/oidc5500--- PASS: TestValidateToken_WrongAudience (0.07s)55012026/11/12 16:57:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:35147/oidc5502--- PASS: TestValidateToken_BoundSubjectMismatch (0.10s)55032026/11/12 16:57:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:40329/oidc5504--- PASS: TestPins_TopLevelShorthand (0.14s)55052026/11/12 16:57:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:34913/oidc55062026/11/12 16:57:30 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:44673/oidc55072026/11/12 16:57:30 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:41355/oidc5508--- PASS: TestValidateToken_Expired (0.14s)5509--- PASS: TestValidateToken_MultipleProviders (0.14s)55102026/11/12 16:57:30 http: TLS handshake error from 127.0.0.1:51656: remote error: tls: bad certificate5511--- PASS: TestNewValidator_KubernetesRequiresCA (0.14s)55122026/11/12 16:57:30 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:3493355132026/11/12 16:57:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:41619/oidc5514--- PASS: TestValidateToken_KubernetesServiceAccount (0.15s)5515--- PASS: TestScopes_Rules (0.16s)55162026/11/12 16:57:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:41971/oidc5517--- PASS: TestScopes_LegacyProviderDefaultsToWrite (0.17s)55182026/11/12 16:57:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:43867/oidc55192026/11/12 16:57:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:37743/oidc5520--- PASS: TestValidateToken_ValidToken (0.18s)5521--- PASS: TestValidateToken_BoundClaimsMismatch (0.18s)5522--- PASS: TestValidateToken_UnknownIssuerFetchesNoKeys (0.28s)5523PASS5524Running hook tests...5525=== RUN TestSendPathsEmpty5526=== PAUSE TestSendPathsEmpty5527=== RUN TestQueueEnqueueAndFetch5528=== PAUSE TestQueueEnqueueAndFetch5529=== RUN TestQueueDeduplication5530=== PAUSE TestQueueDeduplication5531=== RUN TestQueueRemove5532=== PAUSE TestQueueRemove5533=== RUN TestQueueFetchBatchLimit5534=== PAUSE TestQueueFetchBatchLimit5535=== RUN TestQueueRetryMovesToBack5536=== PAUSE TestQueueRetryMovesToBack5537=== RUN TestQueueFetchRemoveLifecycle5538=== PAUSE TestQueueFetchRemoveLifecycle5539=== RUN TestQueueConcurrentWriters5540=== PAUSE TestQueueConcurrentWriters5541=== RUN TestQueueRemoveLargeClosure5542=== PAUSE TestQueueRemoveLargeClosure5543=== RUN TestServerClientIntegration5544--- PASS: TestServerClientIntegration (0.00s)5545=== RUN TestServerQueueError55462026/11/12 16:57:30 ERROR Failed to queue paths error="permission denied" count=15547--- PASS: TestServerQueueError (0.00s)5548=== RUN TestGetListenerSocketActivation5549 server_test.go:220: === RUN TestGetListenerSocketActivation5550 --- PASS: TestGetListenerSocketActivation (0.00s)5551 PASS5552 5553--- PASS: TestGetListenerSocketActivation (0.01s)5554=== RUN TestServerStalledClientDoesNotBlockShutdown5555=== PAUSE TestServerStalledClientDoesNotBlockShutdown5556=== RUN TestServerBacksOffOnAcceptErrors5557=== PAUSE TestServerBacksOffOnAcceptErrors5558=== RUN TestDrainIsolatesPoisonPath5559=== PAUSE TestDrainIsolatesPoisonPath5560=== RUN TestRunNotBlockedByPoisonHead5561=== PAUSE TestRunNotBlockedByPoisonHead5562=== RUN TestDrainGivesUpWhenServerDown5563=== PAUSE TestDrainGivesUpWhenServerDown5564=== RUN TestFailedPathPrunedByLaterClosure5565=== PAUSE TestFailedPathPrunedByLaterClosure5566=== RUN TestWorkerUploadsAndRemoves5567=== PAUSE TestWorkerUploadsAndRemoves5568=== RUN TestWorkerSkipsGCdPaths5569=== PAUSE TestWorkerSkipsGCdPaths5570=== RUN TestWorkerPrunesClosureDeps5571=== PAUSE TestWorkerPrunesClosureDeps5572=== RUN TestDrainTimeout5573=== PAUSE TestDrainTimeout5574=== RUN TestWorkerRemoveFailureIsNotProgress5575=== PAUSE TestWorkerRemoveFailureIsNotProgress5576=== RUN TestWorkerKeepsPathItCannotStat5577=== PAUSE TestWorkerKeepsPathItCannotStat5578=== CONT TestSendPathsEmpty5579=== CONT TestWorkerPrunesClosureDeps5580--- PASS: TestSendPathsEmpty (0.00s)5581=== CONT TestWorkerKeepsPathItCannotStat5582=== CONT TestServerBacksOffOnAcceptErrors5583=== CONT TestDrainTimeout55842000/01/01 00:00:00 ERROR Accept failed error="too many open files"55852000/01/01 00:00:00 ERROR Accept failed error="too many open files"5586=== CONT TestQueueRetryMovesToBack55872000/01/01 00:00:00 ERROR Accept failed error="too many open files"55882000/01/01 00:00:00 ERROR Accept failed error="too many open files"55892000/01/01 00:00:00 ERROR Accept failed error="too many open files"55902000/01/01 00:00:00 ERROR Accept failed error="too many open files"55912000/01/01 00:00:00 ERROR Accept failed error="too many open files"55922000/01/01 00:00:00 ERROR Accept failed error="too many open files"5593=== CONT TestWorkerRemoveFailureIsNotProgress5594=== CONT TestServerStalledClientDoesNotBlockShutdown5595=== CONT TestQueueFetchBatchLimit5596=== CONT TestQueueRemoveLargeClosure5597=== CONT TestQueueRemove5598=== CONT TestQueueConcurrentWriters5599=== CONT TestQueueDeduplication5600=== CONT TestFailedPathPrunedByLaterClosure5601=== CONT TestWorkerSkipsGCdPaths5602=== CONT TestQueueEnqueueAndFetch5603=== CONT TestWorkerUploadsAndRemoves5604=== CONT TestQueueFetchRemoveLifecycle5605=== CONT TestRunNotBlockedByPoisonHead5606=== CONT TestDrainIsolatesPoisonPath5607=== CONT TestDrainGivesUpWhenServerDown5608--- PASS: TestServerBacksOffOnAcceptErrors (0.00s)56092026/11/12 16:57:30 INFO Upload queue status pending=256102026/11/12 16:57:30 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerSkipsGCdPaths841709428/002/nonexistent56112026/11/12 16:57:30 INFO Upload queue status pending=256122026/11/12 16:57:30 INFO Uploading batch count=15613--- PASS: TestQueueDeduplication (0.02s)5614--- PASS: TestQueueEnqueueAndFetch (0.01s)56152026/11/12 16:57:30 INFO Uploading batch count=256162026/11/12 16:57:30 ERROR Upload failed error="upload failed" count=256172026/11/12 16:57:30 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown275951013/002/a56182026/11/12 16:57:30 INFO Uploading batch count=15619--- PASS: TestQueueRetryMovesToBack (0.02s)5620--- PASS: TestQueueFetchRemoveLifecycle (0.02s)56212000/01/01 00:00:00 INFO Uploading batch count=256222000/01/01 00:00:00 ERROR Upload failed error="context deadline exceeded" count=256232000/01/01 00:00:00 ERROR Drain finished with paths left in queue remaining=456242026/11/12 16:57:30 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat2242245306/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat2242245306/002/locked/aaa: permission denied"5625--- PASS: TestQueueFetchBatchLimit (0.02s)56262026/11/12 16:57:30 INFO Upload queue status pending=256272026/11/12 16:57:30 INFO Uploading batch count=256282026/11/12 16:57:30 INFO Uploading batch count=456292026/11/12 16:57:30 ERROR Upload failed error="upload failed" count=456302026/11/12 16:57:30 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainIsolatesPoisonPath320231705/002/bbb56312026/11/12 16:57:30 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown275951013/002/b56322026/11/12 16:57:30 INFO Uploading batch count=156332026/11/12 16:57:30 ERROR Upload failed error="upload failed" count=156342026/11/12 16:57:30 INFO Upload queue status pending=356352026/11/12 16:57:30 INFO Uploading batch count=156362026/11/12 16:57:30 ERROR Upload failed error="upload failed" count=15637--- PASS: TestDrainTimeout (0.02s)56382026/11/12 16:57:30 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat2242245306/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat2242245306/002/locked/aaa: permission denied"5639--- PASS: TestQueueRemove (0.02s)56402026/11/12 16:57:30 INFO Uploading batch count=156412026/11/12 16:57:30 INFO Uploading batch count=256422026/11/12 16:57:30 ERROR Upload failed error="upload failed" count=256432026/11/12 16:57:30 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown275951013/002/c56442026/11/12 16:57:30 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat2242245306/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat2242245306/002/locked/aaa: permission denied"56452026/11/12 16:57:30 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown275951013/002/d56462026/11/12 16:57:30 INFO Uploading batch count=156472026/11/12 16:57:30 ERROR Upload failed error="upload failed" count=156482026/11/12 16:57:30 INFO Uploading batch count=156492026/11/12 16:57:30 INFO Uploading batch count=256502026/11/12 16:57:30 ERROR Upload failed error="upload failed" count=256512026/11/12 16:57:30 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown275951013/002/e56522026/11/12 16:57:30 ERROR Drain finished with paths left in queue remaining=156532026/11/12 16:57:30 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress2232503571/002/nonexistent56542026/11/12 16:57:30 INFO Uploading batch count=156552026/11/12 16:57:30 ERROR Upload failed error="upload failed" count=156562026/11/12 16:57:30 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=156572026/11/12 16:57:30 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress2232503571/002/nonexistent56582026/11/12 16:57:30 INFO Uploading batch count=156592026/11/12 16:57:30 ERROR Upload failed error="upload failed" count=156602026/11/12 16:57:30 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=156612026/11/12 16:57:30 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress2232503571/002/nonexistent56622026/11/12 16:57:30 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown275951013/002/f56632026/11/12 16:57:30 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=156642026/11/12 16:57:30 ERROR Drain finished with paths left in queue remaining=15665--- PASS: TestFailedPathPrunedByLaterClosure (0.02s)56662026/11/12 16:57:30 ERROR Drain finished with paths left in queue remaining=156672026/11/12 16:57:30 ERROR Drain finished with paths left in queue remaining=105668--- PASS: TestWorkerKeepsPathItCannotStat (0.02s)5669--- PASS: TestWorkerRemoveFailureIsNotProgress (0.02s)5670--- PASS: TestDrainIsolatesPoisonPath (0.02s)5671--- PASS: TestDrainGivesUpWhenServerDown (0.02s)5672--- PASS: TestWorkerSkipsGCdPaths (0.03s)5673--- PASS: TestWorkerPrunesClosureDeps (0.04s)5674--- PASS: TestWorkerUploadsAndRemoves (0.04s)5675--- PASS: TestQueueConcurrentWriters (0.10s)5676--- PASS: TestQueueRemoveLargeClosure (0.11s)56772026/11/12 16:57:30 ERROR Failed to decode request error="read unix /build/hook165430708/test.sock->@: i/o timeout"56782026/11/12 16:57:30 ERROR Failed to write response error="write unix /build/hook165430708/test.sock->@: i/o timeout"5679--- PASS: TestServerStalledClientDoesNotBlockShutdown (0.20s)56802026/11/12 16:57:31 INFO Uploading batch count=156812026/11/12 16:57:31 INFO Uploading batch count=156822026/11/12 16:57:31 INFO Uploading batch count=156832026/11/12 16:57:31 ERROR Upload failed error="upload failed" count=156842026/11/12 16:57:31 INFO Uploading batch count=156852026/11/12 16:57:31 ERROR Upload failed error="upload failed" count=156862026/11/12 16:57:31 INFO Uploading batch count=156872026/11/12 16:57:31 ERROR Upload failed error="upload failed" count=156882026/11/12 16:57:31 INFO Uploading batch count=156892026/11/12 16:57:31 ERROR Upload failed error="upload failed" count=156902026/11/12 16:57:31 ERROR Drain finished with paths left in queue remaining=15691--- PASS: TestRunNotBlockedByPoisonHead (1.03s)5692PASS5693Running signing tests...5694=== RUN TestExternalSigner5695=== PAUSE TestExternalSigner5696=== RUN TestExternalSignerWrongPublicKey5697=== PAUSE TestExternalSignerWrongPublicKey5698=== RUN TestExternalSignerErrors5699=== PAUSE TestExternalSignerErrors5700=== RUN TestExternalSignerRestartsAfterCrash5701=== PAUSE TestExternalSignerRestartsAfterCrash5702=== RUN TestExternalSignerProgramErrorKeepsProcess5703=== PAUSE TestExternalSignerProgramErrorKeepsProcess5704=== RUN TestExternalSignerClose5705=== PAUSE TestExternalSignerClose5706=== RUN TestExternalSignerRejectsInvalidPublicKey5707=== PAUSE TestExternalSignerRejectsInvalidPublicKey5708=== RUN TestExternalSignerCancellation5709=== PAUSE TestExternalSignerCancellation5710=== RUN TestGenerateFingerprint5711=== PAUSE TestGenerateFingerprint5712=== RUN TestParseSigningKey5713=== PAUSE TestParseSigningKey5714=== RUN TestSign5715=== PAUSE TestSign5716=== RUN TestSignBatch5717=== PAUSE TestSignBatch5718=== RUN TestNilKeyPublicKey5719=== PAUSE TestNilKeyPublicKey5720=== RUN TestSignErrors5721=== PAUSE TestSignErrors5722=== CONT TestExternalSigner5723=== CONT TestExternalSignerCancellation5724=== CONT TestSign5725=== CONT TestExternalSignerProgramErrorKeepsProcess5726=== CONT TestSignBatch5727=== CONT TestSignErrors5728=== CONT TestNilKeyPublicKey5729--- PASS: TestNilKeyPublicKey (0.00s)5730=== CONT TestParseSigningKey5731=== RUN TestParseSigningKey/valid_32-byte_key5732=== PAUSE TestParseSigningKey/valid_32-byte_key5733=== RUN TestParseSigningKey/valid_32-byte_key_with_different_name5734=== PAUSE TestParseSigningKey/valid_32-byte_key_with_different_name5735=== RUN TestParseSigningKey/no_colon5736=== CONT TestExternalSignerErrors5737=== PAUSE TestParseSigningKey/no_colon5738=== RUN TestParseSigningKey/empty_name5739=== PAUSE TestParseSigningKey/empty_name5740=== RUN TestParseSigningKey/invalid_base645741=== PAUSE TestParseSigningKey/invalid_base645742=== CONT TestExternalSignerWrongPublicKey5743=== RUN TestExternalSignerErrors/exit_on_request5744=== CONT TestExternalSignerRestartsAfterCrash5745=== CONT TestExternalSignerRejectsInvalidPublicKey5746=== RUN TestExternalSignerRejectsInvalidPublicKey/#005747=== CONT TestExternalSignerClose5748=== PAUSE TestExternalSignerErrors/exit_on_request5749=== RUN TestParseSigningKey/wrong_length5750=== PAUSE TestParseSigningKey/wrong_length5751=== CONT TestGenerateFingerprint5752=== PAUSE TestExternalSignerRejectsInvalidPublicKey/#005753=== RUN TestExternalSignerErrors/invalid_JSON5754=== PAUSE TestExternalSignerErrors/invalid_JSON5755=== CONT TestParseSigningKey/valid_32-byte_key5756=== CONT TestParseSigningKey/wrong_length5757=== CONT TestParseSigningKey/invalid_base645758=== CONT TestParseSigningKey/valid_32-byte_key_with_different_name5759=== CONT TestParseSigningKey/empty_name5760=== CONT TestParseSigningKey/no_colon5761=== RUN TestGenerateFingerprint/basic_with_references5762=== PAUSE TestGenerateFingerprint/basic_with_references5763=== RUN TestGenerateFingerprint/no_references5764=== RUN TestExternalSignerRejectsInvalidPublicKey/no-colon5765=== RUN TestExternalSignerErrors/wrong_shape5766=== PAUSE TestExternalSignerErrors/wrong_shape5767=== PAUSE TestExternalSignerRejectsInvalidPublicKey/no-colon5768=== PAUSE TestGenerateFingerprint/no_references5769=== RUN TestExternalSignerErrors/missing_signatures5770=== PAUSE TestExternalSignerErrors/missing_signatures5771=== RUN TestExternalSignerErrors/wrong_signatures_shape5772=== PAUSE TestExternalSignerErrors/wrong_signatures_shape5773=== RUN TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5774=== PAUSE TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5775=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5776=== RUN TestGenerateFingerprint/unsorted_references_get_sorted5777=== PAUSE TestGenerateFingerprint/unsorted_references_get_sorted5778=== RUN TestExternalSignerErrors/missing_signature_name5779=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5780=== RUN TestGenerateFingerprint/invalid_nar_hash_prefix5781=== PAUSE TestGenerateFingerprint/invalid_nar_hash_prefix5782=== RUN TestGenerateFingerprint/invalid_nar_hash_length5783=== PAUSE TestGenerateFingerprint/invalid_nar_hash_length5784=== RUN TestGenerateFingerprint/invalid_store_path_prefix5785=== PAUSE TestExternalSignerErrors/missing_signature_name5786=== RUN TestExternalSignerErrors/wrong_id5787=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5788=== PAUSE TestGenerateFingerprint/invalid_store_path_prefix5789=== RUN TestGenerateFingerprint/invalid_reference_prefix5790=== PAUSE TestExternalSignerErrors/wrong_id5791=== RUN TestExternalSignerErrors/null_signatures5792=== PAUSE TestExternalSignerErrors/null_signatures5793=== RUN TestExternalSignerErrors/program_error5794=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5795=== CONT TestExternalSignerRejectsInvalidPublicKey/#005796=== CONT TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5797=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5798=== PAUSE TestExternalSignerErrors/program_error5799=== RUN TestExternalSignerErrors/wrong_signature_name5800--- PASS: TestSign (0.00s)5801=== PAUSE TestGenerateFingerprint/invalid_reference_prefix5802=== CONT TestGenerateFingerprint/basic_with_references5803=== CONT TestGenerateFingerprint/invalid_nar_hash_length5804=== CONT TestGenerateFingerprint/invalid_store_path_prefix5805=== CONT TestGenerateFingerprint/unsorted_references_get_sorted5806=== CONT TestExternalSignerRejectsInvalidPublicKey/no-colon5807=== RUN TestSignErrors/nil_key5808=== PAUSE TestSignErrors/nil_key5809=== RUN TestSignErrors/nil_narinfo5810=== PAUSE TestExternalSignerErrors/wrong_signature_name5811--- PASS: TestParseSigningKey (0.00s)5812 --- PASS: TestParseSigningKey/wrong_length (0.00s)5813 --- PASS: TestParseSigningKey/invalid_base64 (0.00s)5814 --- PASS: TestParseSigningKey/empty_name (0.00s)5815 --- PASS: TestParseSigningKey/no_colon (0.00s)5816 --- PASS: TestParseSigningKey/valid_32-byte_key (0.00s)5817 --- PASS: TestParseSigningKey/valid_32-byte_key_with_different_name (0.00s)5818=== RUN TestExternalSignerErrors/invalid_signature_base645819=== PAUSE TestExternalSignerErrors/invalid_signature_base645820=== RUN TestExternalSignerErrors/wrong_signature_length5821=== CONT TestGenerateFingerprint/invalid_nar_hash_prefix5822=== PAUSE TestExternalSignerErrors/wrong_signature_length5823=== CONT TestGenerateFingerprint/no_references5824=== CONT TestGenerateFingerprint/invalid_reference_prefix5825--- PASS: TestGenerateFingerprint (0.00s)5826 --- PASS: TestGenerateFingerprint/basic_with_references (0.00s)5827 --- PASS: TestGenerateFingerprint/invalid_nar_hash_length (0.00s)5828 --- PASS: TestGenerateFingerprint/invalid_store_path_prefix (0.00s)5829 --- PASS: TestGenerateFingerprint/unsorted_references_get_sorted (0.00s)5830 --- PASS: TestGenerateFingerprint/invalid_nar_hash_prefix (0.00s)5831 --- PASS: TestGenerateFingerprint/no_references (0.00s)5832 --- PASS: TestGenerateFingerprint/invalid_reference_prefix (0.00s)5833=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5834--- PASS: TestExternalSignerRejectsInvalidPublicKey (0.00s)5835 --- PASS: TestExternalSignerRejectsInvalidPublicKey/#00 (0.00s)5836 --- PASS: TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= (0.00s)5837 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:invalid! (0.00s)5838 --- PASS: TestExternalSignerRejectsInvalidPublicKey/no-colon (0.00s)5839 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:YQ== (0.00s)5840=== PAUSE TestSignErrors/nil_narinfo5841=== RUN TestSignErrors/invalid_narinfo5842=== PAUSE TestSignErrors/invalid_narinfo5843=== RUN TestSignErrors/canceled5844=== PAUSE TestSignErrors/canceled5845=== CONT TestSignErrors/nil_key5846=== RUN TestExternalSignerErrors/exit_at_startup5847=== CONT TestSignErrors/nil_narinfo5848=== PAUSE TestExternalSignerErrors/exit_at_startup5849=== CONT TestSignErrors/canceled5850=== CONT TestSignErrors/invalid_narinfo5851--- PASS: TestSignBatch (0.00s)5852--- PASS: TestSignErrors (0.00s)5853 --- PASS: TestSignErrors/nil_key (0.00s)5854 --- PASS: TestSignErrors/nil_narinfo (0.00s)5855 --- PASS: TestSignErrors/canceled (0.00s)5856 --- PASS: TestSignErrors/invalid_narinfo (0.00s)5857=== RUN TestExternalSignerErrors/null5858=== PAUSE TestExternalSignerErrors/null5859--- PASS: TestExternalSignerProgramErrorKeepsProcess (0.00s)5860=== RUN TestExternalSignerErrors/too_few_signatures5861=== PAUSE TestExternalSignerErrors/too_few_signatures5862=== RUN TestExternalSignerErrors/unrequested_key5863=== PAUSE TestExternalSignerErrors/unrequested_key5864=== CONT TestExternalSignerErrors/exit_on_request5865=== CONT TestExternalSignerErrors/program_error5866=== CONT TestExternalSignerErrors/exit_at_startup5867=== CONT TestExternalSignerErrors/unrequested_key5868=== CONT TestExternalSignerErrors/too_few_signatures5869=== CONT TestExternalSignerErrors/invalid_signature_base645870=== CONT TestExternalSignerErrors/null5871=== CONT TestExternalSignerErrors/wrong_signature_length5872=== CONT TestExternalSignerErrors/wrong_signatures_shape5873--- PASS: TestExternalSignerClose (0.00s)5874=== CONT TestExternalSignerErrors/wrong_signature_name5875=== CONT TestExternalSignerErrors/null_signatures5876=== CONT TestExternalSignerErrors/wrong_id5877=== CONT TestExternalSignerErrors/missing_signature_name5878=== CONT TestExternalSignerErrors/wrong_shape5879--- PASS: TestExternalSignerWrongPublicKey (0.00s)5880=== CONT TestExternalSignerErrors/invalid_JSON5881=== CONT TestExternalSignerErrors/missing_signatures5882--- PASS: TestExternalSigner (0.01s)5883--- PASS: TestExternalSignerRestartsAfterCrash (0.01s)5884--- PASS: TestExternalSignerErrors (0.00s)5885 --- PASS: TestExternalSignerErrors/exit_at_startup (0.00s)5886 --- PASS: TestExternalSignerErrors/exit_on_request (0.00s)5887 --- PASS: TestExternalSignerErrors/program_error (0.00s)5888 --- PASS: TestExternalSignerErrors/unrequested_key (0.01s)5889 --- PASS: TestExternalSignerErrors/missing_signature_name (0.00s)5890 --- PASS: TestExternalSignerErrors/null (0.01s)5891 --- PASS: TestExternalSignerErrors/too_few_signatures (0.01s)5892 --- PASS: TestExternalSignerErrors/wrong_signature_length (0.01s)5893 --- PASS: TestExternalSignerErrors/wrong_signatures_shape (0.01s)5894 --- PASS: TestExternalSignerErrors/wrong_shape (0.01s)5895 --- PASS: TestExternalSignerErrors/wrong_signature_name (0.01s)5896 --- PASS: TestExternalSignerErrors/invalid_signature_base64 (0.01s)5897 --- PASS: TestExternalSignerErrors/wrong_id (0.01s)5898 --- PASS: TestExternalSignerErrors/null_signatures (0.01s)5899 --- PASS: TestExternalSignerErrors/invalid_JSON (0.01s)5900 --- PASS: TestExternalSignerErrors/missing_signatures (0.01s)5901--- PASS: TestExternalSignerCancellation (0.02s)5902PASS5903Running OpenBao signer tests...5904=== RUN TestSign5905=== PAUSE TestSign5906=== RUN TestSignErrors5907=== PAUSE TestSignErrors5908=== RUN TestServe5909=== PAUSE TestServe5910=== RUN TestServeRejectsGarbage5911=== PAUSE TestServeRejectsGarbage5912=== CONT TestSign5913=== CONT TestServe5914=== CONT TestSignErrors5915=== RUN TestSignErrors/invalid_json5916=== PAUSE TestSignErrors/invalid_json5917=== RUN TestSignErrors/no_prefix5918=== PAUSE TestSignErrors/no_prefix5919=== RUN TestSignErrors/invalid_base645920=== PAUSE TestSignErrors/invalid_base645921=== CONT TestServeRejectsGarbage5922--- PASS: TestServeRejectsGarbage (0.00s)5923=== RUN TestSignErrors/http_error5924=== PAUSE TestSignErrors/http_error5925=== CONT TestSignErrors/invalid_json5926--- PASS: TestServe (0.00s)5927=== CONT TestSignErrors/invalid_base645928=== CONT TestSignErrors/no_prefix5929=== CONT TestSignErrors/http_error5930--- PASS: TestSign (0.01s)5931--- PASS: TestSignErrors (0.00s)5932 --- PASS: TestSignErrors/http_error (0.00s)5933 --- PASS: TestSignErrors/invalid_json (0.00s)5934 --- PASS: TestSignErrors/no_prefix (0.00s)5935 --- PASS: TestSignErrors/invalid_base64 (0.00s)5936PASS