niks3-go-unit-tests
checks.x86_64-linux.go-unit-tests
· build #508
· raw
1tribuchet: building on jamie2Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestUploadBuildLog_FileBodyReplayedOnRetry6=== PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry7=== RUN TestRegisterUploadedObjectReusesConnections8=== PAUSE TestRegisterUploadedObjectReusesConnections9=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica10=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica11=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun12=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun13=== RUN TestCaseHackSuffix14=== PAUSE TestCaseHackSuffix15=== RUN TestFilterOversizedClosures16=== PAUSE TestFilterOversizedClosures17=== RUN TestUploadMultipart_PartsInParallel18=== PAUSE TestUploadMultipart_PartsInParallel19=== RUN TestUploadMultipart_ReturnsAfterDroppedBody20=== PAUSE TestUploadMultipart_ReturnsAfterDroppedBody21=== RUN TestUploadMultipart_FailedProducerIsNotCompleted22=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted23=== RUN TestPartSizeForNAR24=== PAUSE TestPartSizeForNAR25=== RUN TestDumpPathCaseHackMatchesNix26--- PASS: TestDumpPathCaseHackMatchesNix (0.03s)27=== RUN TestDumpPathCaseHackCollision28--- PASS: TestDumpPathCaseHackCollision (0.00s)29=== RUN TestDumpPathMatchesNix30=== PAUSE TestDumpPathMatchesNix31=== RUN TestDumpPathSingleFile32=== PAUSE TestDumpPathSingleFile33=== RUN TestDumpPathWriterError34=== PAUSE TestDumpPathWriterError35=== RUN TestEncodeNixBase3236=== PAUSE TestEncodeNixBase3237=== RUN TestEncodeNixBase32WithRealHash38=== PAUSE TestEncodeNixBase32WithRealHash39=== RUN TestConvertHashToNix3240=== PAUSE TestConvertHashToNix3241=== RUN TestGetStorePathHash42=== PAUSE TestGetStorePathHash43=== RUN TestPathInfoHashCompatibility44=== PAUSE TestPathInfoHashCompatibility45=== RUN TestParsePathInfoJSON46=== PAUSE TestParsePathInfoJSON47=== RUN TestParsePathInfoJSONMultiplePaths48=== PAUSE TestParsePathInfoJSONMultiplePaths49=== RUN TestPathInfoCACompatibility50=== PAUSE TestPathInfoCACompatibility51=== RUN TestRateLimiterFeedback52=== PAUSE TestRateLimiterFeedback53=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess54=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess55=== RUN TestRegisterUploadedObject_BoundedAgainstSilentServer56=== PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer57=== RUN TestRegisterUploadedObject_OutlivesPush58=== PAUSE TestRegisterUploadedObject_OutlivesPush59=== RUN TestResolveStorePath60=== PAUSE TestResolveStorePath61=== RUN TestDoWithRetry_BodyReplayedViaGetBody62=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody63=== RUN TestRetryTimeoutOutlastsMaxRetries64=== PAUSE TestRetryTimeoutOutlastsMaxRetries65=== RUN TestDoWithRetry_FinalResponseBodyReadable66=== PAUSE TestDoWithRetry_FinalResponseBodyReadable67=== RUN TestShellSplit68=== PAUSE TestShellSplit69=== RUN TestShellSplitErrors70=== PAUSE TestShellSplitErrors71=== RUN TestStreamPushReportsEveryPath72=== PAUSE TestStreamPushReportsEveryPath73=== RUN TestStreamPushBatchesUnderLoad74=== PAUSE TestStreamPushBatchesUnderLoad75=== RUN TestStreamPushIsolatesFailures76=== PAUSE TestStreamPushIsolatesFailures77=== RUN TestStreamPushGivesUpOnDeadServer78=== PAUSE TestStreamPushGivesUpOnDeadServer79=== RUN TestStreamPushRequestLine80=== PAUSE TestStreamPushRequestLine81=== RUN TestStreamPushReportsSignatures82=== PAUSE TestStreamPushReportsSignatures83=== RUN TestStreamPushReportsSkippedPaths84=== PAUSE TestStreamPushReportsSkippedPaths85=== RUN TestClientSignaturesByStorePath86=== PAUSE TestClientSignaturesByStorePath87=== RUN TestStreamPushAnswersEveryLineWhenCanceled88=== PAUSE TestStreamPushAnswersEveryLineWhenCanceled89=== RUN TestStreamPushStopsWhenCanceled90=== PAUSE TestStreamPushStopsWhenCanceled91=== RUN TestSetClientTLS92=== PAUSE TestSetClientTLS93=== RUN TestSetClientTLSDoesNotMutateDefaultTransport94=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport95=== RUN TestSetClientTLSErrors96=== PAUSE TestSetClientTLSErrors97=== RUN TestStaticToken98=== PAUSE TestStaticToken99=== RUN TestFileTokenReadsAndCaches100=== PAUSE TestFileTokenReadsAndCaches101=== RUN TestFileTokenMissing102=== PAUSE TestFileTokenMissing103=== RUN TestFileTokenEmpty104=== PAUSE TestFileTokenEmpty105=== RUN TestScriptTokenNoExpiryRerunsEveryCall106=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall107=== RUN TestScriptTokenCachesUntilRefresh108=== PAUSE TestScriptTokenCachesUntilRefresh109=== RUN TestScriptTokenEmptyToken110=== PAUSE TestScriptTokenEmptyToken111=== RUN TestScriptTokenBadJSON112=== PAUSE TestScriptTokenBadJSON113=== RUN TestScriptTokenScriptFails114=== PAUSE TestScriptTokenScriptFails115=== RUN TestScriptTokenEmptyCommand116=== PAUSE TestScriptTokenEmptyCommand117=== CONT TestDoServerRequestAttachesToken118=== CONT TestRetryTimeoutOutlastsMaxRetries119=== CONT TestFileTokenEmpty120=== CONT TestScriptTokenBadJSON121=== CONT TestScriptTokenEmptyCommand122--- PASS: TestScriptTokenEmptyCommand (0.00s)123=== CONT TestPartSizeForNAR124=== RUN TestPartSizeForNAR/zero_stays_at_minimum125=== CONT TestSetClientTLS126=== CONT TestScriptTokenScriptFails127--- PASS: TestFileTokenEmpty (0.00s)128=== CONT TestShellSplitErrors129--- PASS: TestShellSplitErrors (0.00s)130=== CONT TestUploadMultipart_FailedProducerIsNotCompleted131=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part132=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part133=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary134=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary135=== CONT TestShellSplit136--- PASS: TestShellSplit (0.00s)137=== CONT TestUploadMultipart_ReturnsAfterDroppedBody138=== CONT TestScriptTokenEmptyToken139=== CONT TestScriptTokenCachesUntilRefresh140--- PASS: TestScriptTokenScriptFails (0.00s)141=== CONT TestStreamPushGivesUpOnDeadServer142=== CONT TestStreamPushStopsWhenCanceled143=== CONT TestStreamPushAnswersEveryLineWhenCanceled144=== CONT TestClientSignaturesByStorePath145=== CONT TestStreamPushReportsSkippedPaths146=== CONT TestStreamPushReportsSignatures147=== CONT TestStreamPushRequestLine148=== CONT TestUploadMultipart_PartsInParallel149=== CONT TestScriptTokenNoExpiryRerunsEveryCall150=== CONT TestDumpPathWriterError151=== CONT TestStreamPushIsolatesFailures152=== CONT TestDumpPathSingleFile153=== CONT TestStreamPushBatchesUnderLoad154=== CONT TestDumpPathMatchesNix155=== CONT TestStreamPushReportsEveryPath156=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum157=== CONT TestDoWithRetry_FinalResponseBodyReadable158=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun159=== RUN TestPartSizeForNAR/small_stays_at_minimum160=== PAUSE TestPartSizeForNAR/small_stays_at_minimum161=== CONT TestRegisterUploadedObjectReusesConnections162=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum163=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum164=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts165--- PASS: TestStreamPushStopsWhenCanceled (0.00s)166--- PASS: TestClientSignaturesByStorePath (0.00s)167--- PASS: TestStreamPushReportsSkippedPaths (0.00s)168=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica169=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts170=== RUN TestPartSizeForNAR/1_TiB171=== PAUSE TestPartSizeForNAR/1_TiB172=== RUN TestPartSizeForNAR/5_TiB_S3_max_object173=== CONT TestCaseHackSuffix174--- PASS: TestScriptTokenEmptyToken (0.00s)175=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object176=== RUN TestPartSizeForNAR/capped_at_5_GiB177=== PAUSE TestPartSizeForNAR/capped_at_5_GiB178=== CONT TestUploadBuildLog_FileBodyReplayedOnRetry179=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=false180=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=false181=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=true182=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=true1832026/11/12 11:45:49 INFO Garbage collection started184=== CONT TestFilterOversizedClosures185=== CONT TestPathInfoCACompatibility186=== RUN TestPathInfoCACompatibility/null_ca_field1872026/11/12 11:45:49 ERROR Upload failed error="bad path" count=3188=== PAUSE TestPathInfoCACompatibility/null_ca_field189=== RUN TestFilterOversizedClosures/no_limit_keeps_everything190=== RUN TestPathInfoCACompatibility/old_string_format_-_text1912026/11/12 11:45:49 ERROR Upload failed error="connection refused" count=20192--- PASS: TestScriptTokenBadJSON (0.00s)1932026/11/12 11:45:49 ERROR Upload failed error=boom count=1194--- PASS: TestStreamPushIsolatesFailures (0.00s)195=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything1962026/11/12 11:45:49 ERROR Server seems unavailable, giving up on batch untried=17197=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text198=== CONT TestPathInfoHashCompatibility199=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive200=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped201--- PASS: TestStreamPushReportsEveryPath (0.00s)202=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)203=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped204--- PASS: TestStreamPushGivesUpOnDeadServer (0.01s)205=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)206=== RUN TestFilterOversizedClosures/all_closures_skipped207=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon208=== RUN TestSetClientTLS/rejects_connection_without_client_cert2092000/01/01 00:00:00 ERROR Upload failed error="context canceled" count=22102026/11/12 11:45:49 INFO Garbage collection started2112026/11/12 11:45:49 ERROR Upload failed error=boom count=1212=== CONT TestDoWithRetry_BodyReplayedViaGetBody213=== CONT TestParsePathInfoJSONMultiplePaths214=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths215=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths216=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths217=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths218=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive219=== RUN TestPathInfoCACompatibility/new_structured_format_-_text220=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text221=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method222=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method223=== PAUSE TestFilterOversizedClosures/all_closures_skipped224=== CONT TestRegisterUploadedObject_OutlivesPush225=== CONT TestRegisterUploadedObject_BoundedAgainstSilentServer226=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess2272026/11/12 11:45:49 WARN Rate limiter enabled after throttle name=server-test rate=52282026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:44397229=== CONT TestParsePathInfoJSON230=== RUN TestParsePathInfoJSON/Nix_format2312026/11/12 11:45:49 WARN Rate limiter enabled after throttle name=server-test rate=5232=== PAUSE TestParsePathInfoJSON/Nix_format233=== RUN TestParsePathInfoJSON/Lix_format234=== PAUSE TestParsePathInfoJSON/Lix_format235--- PASS: TestStreamPushAnswersEveryLineWhenCanceled (0.03s)236--- PASS: TestRegisterUploadedObject_OutlivesPush (0.00s)2372026/11/12 11:45:49 WARN Rate limiter backed off name=server-test rate=5238=== CONT TestStaticToken2392026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:44397240=== CONT TestSetClientTLSErrors241=== CONT TestFileTokenMissing242=== CONT TestRateLimiterFeedback243=== RUN TestRateLimiterFeedback/429_enables_limiter244=== PAUSE TestRateLimiterFeedback/429_enables_limiter245=== RUN TestRateLimiterFeedback/503_enables_limiter246--- PASS: TestStreamPushReportsSignatures (0.03s)247--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (0.03s)248=== CONT TestResolveStorePath249=== RUN TestParsePathInfoJSON/empty_input250=== PAUSE TestParsePathInfoJSON/empty_input251=== RUN TestParsePathInfoJSON/whitespace_only252=== PAUSE TestRateLimiterFeedback/503_enables_limiter253=== CONT TestSetClientTLSDoesNotMutateDefaultTransport254=== CONT TestFileTokenReadsAndCaches255=== RUN TestSetClientTLSErrors/missing_cert_file256=== PAUSE TestSetClientTLSErrors/missing_cert_file257=== RUN TestSetClientTLSErrors/missing_key_file258=== PAUSE TestSetClientTLSErrors/missing_key_file259=== RUN TestSetClientTLSErrors/missing_ca_file260=== PAUSE TestSetClientTLSErrors/missing_ca_file261=== RUN TestSetClientTLSErrors/invalid_ca_file262=== PAUSE TestSetClientTLSErrors/invalid_ca_file263=== CONT TestConvertHashToNix32264=== RUN TestConvertHashToNix32/SRI_format_to_Nix32265=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32266=== RUN TestConvertHashToNix32/already_Nix32_format267=== PAUSE TestConvertHashToNix32/already_Nix32_format268=== RUN TestConvertHashToNix32/invalid_format269=== PAUSE TestConvertHashToNix32/invalid_format270=== CONT TestGetStorePathHash271=== RUN TestGetStorePathHash/valid_store_path272=== PAUSE TestGetStorePathHash/valid_store_path273=== RUN TestGetStorePathHash/basename_without_hyphen_should_error274=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error275=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error276=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error277=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error278=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error279=== CONT TestEncodeNixBase32WithRealHash280=== CONT TestEncodeNixBase32281=== RUN TestEncodeNixBase32/test_string_hash282=== PAUSE TestEncodeNixBase32/test_string_hash2832026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:37497284=== RUN TestEncodeNixBase32/empty_input285=== PAUSE TestEncodeNixBase32/empty_input286=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part287=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary2882026/11/12 11:45:49 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=0289=== CONT TestPartSizeForNAR/zero_stays_at_minimum290=== CONT TestPartSizeForNAR/1_TiB291=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum292--- PASS: TestUploadMultipart_ReturnsAfterDroppedBody (0.04s)293--- PASS: TestStaticToken (0.00s)294--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.00s)295--- PASS: TestFileTokenMissing (0.00s)296=== CONT TestPartSizeForNAR/small_stays_at_minimum297=== CONT TestPartSizeForNAR/capped_at_5_GiB298=== CONT TestPartSizeForNAR/5_TiB_S3_max_object299--- PASS: TestEncodeNixBase32WithRealHash (0.00s)300--- PASS: TestFileTokenReadsAndCaches (0.00s)301=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter302=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts303=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=true304--- PASS: TestResolveStorePath (0.00s)305=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter306=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter307=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter308=== PAUSE TestParsePathInfoJSON/whitespace_only309=== RUN TestParsePathInfoJSON/invalid_JSON310=== PAUSE TestParsePathInfoJSON/invalid_JSON311=== CONT TestFilterOversizedClosures/no_limit_keeps_everything312=== CONT TestPathInfoCACompatibility/old_string_format_-_text313=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method3142026/11/12 11:45:49 WARN Rate limiter enabled after throttle name=server-test rate=5315=== CONT TestPathInfoCACompatibility/new_structured_format_-_text316--- PASS: TestDoServerRequestAttachesToken (0.04s)3172026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:37389318=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths319=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert320=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA3212026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:37497322=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA323=== RUN TestSetClientTLS/preserves_debug_logging_transport324=== PAUSE TestSetClientTLS/preserves_debug_logging_transport3252026/11/12 11:45:49 WARN Rate limiter backed off name=server-test rate=5326--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.00s)327=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped328=== CONT TestFilterOversizedClosures/all_closures_skipped3292026/11/12 11:45:49 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000330=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths3312026/11/12 11:45:49 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50332=== CONT TestSetClientTLSErrors/missing_key_file333=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=false334=== CONT TestPathInfoCACompatibility/null_ca_field335=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive336=== CONT TestConvertHashToNix32/already_Nix32_format337=== CONT TestGetStorePathHash/valid_store_path338=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error339=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error3402026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:37497341--- PASS: TestDoWithRetry_FinalResponseBodyReadable (0.04s)342--- PASS: TestParsePathInfoJSONMultiplePaths (0.00s)343 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)344 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)345=== CONT TestSetClientTLSErrors/missing_ca_file346=== CONT TestRateLimiterFeedback/429_enables_limiter347=== CONT TestSetClientTLSErrors/invalid_ca_file348=== CONT TestConvertHashToNix32/SRI_format_to_Nix32349=== CONT TestConvertHashToNix32/invalid_format350=== CONT TestSetClientTLSErrors/missing_cert_file3512026/11/12 11:45:49 WARN Garbage collection finished on another replica. Its statistics are not available here352=== CONT TestGetStorePathHash/basename_without_hyphen_should_error353=== CONT TestEncodeNixBase32/test_string_hash354=== CONT TestRateLimiterFeedback/503_enables_limiter355=== CONT TestParsePathInfoJSON/invalid_JSON356=== CONT TestParsePathInfoJSON/whitespace_only357=== CONT TestSetClientTLS/rejects_connection_without_client_cert3582026/11/12 11:45:49 WARN Rate limiter enabled after throttle name=server-test rate=5359--- PASS: TestFilterOversizedClosures (0.03s)360 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)361 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)362 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)3632026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:42983364--- PASS: TestPathInfoCACompatibility (0.03s)365 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)366 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)367 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)368 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)369 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)3702026/11/12 11:45:49 WARN Rate limiter enabled after throttle name=server-test rate=5371=== CONT TestParsePathInfoJSON/Nix_format3722026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:421053732026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:374973742026/11/12 11:45:49 WARN Rate limiter enabled after throttle name=s3-test rate=5375=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA3762026/11/12 11:45:49 WARN Rate limiter backed off name=server-test rate=5377=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter3782026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:42513379=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter3802026/11/12 11:45:49 WARN Rate limiter backed off name=server-test rate=5381=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon382=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI383=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI384=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512385=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512386=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)387=== CONT TestParsePathInfoJSON/empty_input388=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI389=== CONT TestParsePathInfoJSON/Lix_format390=== CONT TestEncodeNixBase32/empty_input391--- PASS: TestStreamPushBatchesUnderLoad (0.00s)392 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=true (0.00s)393 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=false (0.00s)394--- PASS: TestUploadMultipart_FailedProducerIsNotCompleted (0.00s)395 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part (0.00s)396 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary (0.00s)397--- PASS: TestPartSizeForNAR (0.01s)398 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)399 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)400 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)401 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)402 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)403 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)404 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)405--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (0.04s)406=== CONT TestSetClientTLS/preserves_debug_logging_transport407=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon408=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512409--- PASS: TestConvertHashToNix32 (0.00s)410 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)411 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)412 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)413--- PASS: TestEncodeNixBase32 (0.00s)414 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)415 --- PASS: TestEncodeNixBase32/empty_input (0.00s)4162026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:37497417--- PASS: TestParsePathInfoJSON (0.00s)418 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)419 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)420 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)421 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)422 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)423--- PASS: TestRateLimiterFeedback (0.00s)424 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s)425 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.00s)426 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s)427 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.00s)428--- PASS: TestGetStorePathHash (0.00s)429 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)430 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)431 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)432 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)433--- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (0.04s)434--- PASS: TestPathInfoHashCompatibility (0.04s)435 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)436 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)437 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)438 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)439--- PASS: TestSetClientTLSErrors (0.00s)440 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)441 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)442 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)443 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)4442026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:37497445--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.04s)446--- PASS: TestScriptTokenCachesUntilRefresh (0.05s)4472026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:374974482026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:374974492026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:374974502026/11/12 11:45:49 http: TLS handshake error from 127.0.0.1:37046: remote error: tls: bad certificate4512026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:37497452--- PASS: TestStreamPushRequestLine (0.05s)453--- PASS: TestSetClientTLS (0.04s)454 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.00s)455 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.00s)456 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.01s)4572026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:374974582026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:374974592026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:374974602026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:374974612026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:37497462--- PASS: TestRegisterUploadedObjectReusesConnections (0.06s)4632026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:374974642026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:37497465--- PASS: TestCaseHackSuffix (0.06s)4662026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:374974672026/11/12 11:45:49 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:37497468--- PASS: TestRetryTimeoutOutlastsMaxRetries (0.07s)469--- PASS: TestDumpPathSingleFile (0.07s)470--- PASS: TestDumpPathWriterError (0.07s)471--- PASS: TestDumpPathMatchesNix (0.12s)4722026/11/12 11:45:49 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:37855/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:37855/api/uploads/complete4732026/11/12 11:45:49 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4742026/11/12 11:45:49 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:37855/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:37855/api/uploads/complete4752026/11/12 11:45:49 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4762026/11/12 11:45:49 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:37855/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:37855/api/uploads/complete4772026/11/12 11:45:49 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"478--- PASS: TestRegisterUploadedObject_BoundedAgainstSilentServer (0.10s)479--- PASS: TestUploadMultipart_PartsInParallel (0.65s)480--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)481PASS482Running server tests...483The files belonging to this database system will be owned by user "nixbld".484This user must also own the server process.485486The database cluster will be initialized with locale "C".487The default database encoding has accordingly been set to "SQL_ASCII".488The default text search configuration will be set to "english".489490Data page checksums are enabled.491492creating directory /build/postgres344967288/data ... ok493creating subdirectories ... ok494selecting dynamic shared memory implementation ... posix495selecting default "max_connections" ... 100496selecting default "shared_buffers" ... 128MB497selecting default time zone ... UTC498creating configuration files ... ok499running bootstrap script ... ok500performing post-bootstrap initialization ... ok501syncing data to disk ... ok502503initdb: warning: enabling "trust" authentication for local connections504initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.505506Success. You can now start the database server using:507508 pg_ctl -D /build/postgres344967288/data -l logfile start509510/build/postgres344967288:5432 - no response5112026-11-12 11:45:51.236 UTC [131] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit5122026-11-12 11:45:51.237 UTC [131] LOG: listening on Unix socket "/build/postgres344967288/.s.PGSQL.5432"5132026-11-12 11:45:51.241 UTC [138] LOG: database system was shut down at 2026-11-12 11:45:50 UTC5142026-11-12 11:45:51.245 UTC [131] LOG: database system is ready to accept connections515/build/postgres344967288:5432 - accepting connections516{"timestamp":"2026-11-12T11:45:51.542236592Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"dc1b1320-3e80-4962-8257-808a50b9a491","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}517=== RUN TestAbortSucceeded518=== PAUSE TestAbortSucceeded519=== RUN TestService_AuthMiddleware520=== PAUSE TestService_AuthMiddleware521=== RUN TestService_AuthMiddleware_MTLSProxyHeader522=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader523=== RUN TestService_AuthMiddleware_MTLSBoundSubjects524=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects525=== RUN TestService_ReadAuthMiddleware526=== PAUSE TestService_ReadAuthMiddleware527=== RUN TestService_AuthMiddleware_OIDC528=== PAUSE TestService_AuthMiddleware_OIDC529=== RUN TestService_RequireScope_OIDC530=== PAUSE TestService_RequireScope_OIDC531=== RUN TestService_ReadScope_PublicByDefault532=== PAUSE TestService_ReadScope_PublicByDefault533=== RUN TestCacheConfigHandler534=== PAUSE TestCacheConfigHandler535=== RUN TestCacheStatsHandler536=== PAUSE TestCacheStatsHandler537=== RUN TestClientCADerivations538=== PAUSE TestClientCADerivations539=== RUN TestClientErrorHandling540=== PAUSE TestClientErrorHandling541=== RUN TestClientIntegration542=== PAUSE TestClientIntegration543=== RUN TestClientMultipleUploads544=== PAUSE TestClientMultipleUploads545=== RUN TestClientWithDependencies546=== PAUSE TestClientWithDependencies547=== RUN TestClientSharedPathCommittedMidPush548=== PAUSE TestClientSharedPathCommittedMidPush549=== RUN TestPinProtectsFromGC550=== PAUSE TestPinProtectsFromGC551=== RUN TestClientPushesUseOnePush552=== PAUSE TestClientPushesUseOnePush553=== RUN TestClientFallsBackToClosures554=== PAUSE TestClientFallsBackToClosures555=== RUN TestDeleteClosuresRacingPins556=== PAUSE TestDeleteClosuresRacingPins557=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock558=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock559=== RUN TestResolveDBConnectionString560=== PAUSE TestResolveDBConnectionString561=== RUN TestConnectWaitsForAPeerMigration562=== PAUSE TestConnectWaitsForAPeerMigration563=== RUN TestLeadElectsOneAndHandsOver564=== PAUSE TestLeadElectsOneAndHandsOver565=== RUN TestLeadIncumbentWinsAfterRestart5662026/11/12 11:45:51 INFO lead: acquired remote=192.0.2.1:12345672026/11/12 11:45:52 INFO lead: released remote=192.0.2.1:12345682026/11/12 11:45:52 INFO lead: acquired remote=192.0.2.1:12345692026/11/12 11:45:53 INFO lead: released remote=192.0.2.1:1234570--- PASS: TestLeadIncumbentWinsAfterRestart (1.91s)571=== RUN TestLeadUnannouncedStandbyYieldsToIncumbent572=== PAUSE TestLeadUnannouncedStandbyYieldsToIncumbent573=== RUN TestLeadIncumbentAnnouncesAtOnce574=== PAUSE TestLeadIncumbentAnnouncesAtOnce575=== RUN TestLeadEndsOnShutdown576=== PAUSE TestLeadEndsOnShutdown577=== RUN TestLeadEndsWhenItsConnectionHangs5782026/11/12 11:45:53 INFO lead: acquired remote=192.0.2.1:12345792026-11-12 11:45:55.077 UTC [576] FATAL: terminating connection due to administrator command5802026/11/12 11:45:55 INFO lead: acquired remote=192.0.2.1:12345812026/11/12 11:45:56 WARN lead: lock connection lost error="timeout: context deadline exceeded"5822026/11/12 11:45:56 INFO lead: released remote=192.0.2.1:12345832026/11/12 11:45:56 INFO lead: released remote=192.0.2.1:1234584--- PASS: TestLeadEndsWhenItsConnectionHangs (3.12s)585=== RUN TestGCAdvisoryLockBlocksConcurrentRun586=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun587=== RUN TestGCBugBareHashReferences588=== PAUSE TestGCBugBareHashReferences589=== RUN TestGCMetrics590=== PAUSE TestGCMetrics591=== RUN TestPushDedupSurvivesConcurrentGC592=== PAUSE TestPushDedupSurvivesConcurrentGC593=== RUN TestDeduplicatedObjectsRecordedAsPending594=== PAUSE TestDeduplicatedObjectsRecordedAsPending595=== RUN TestGCSweepSkipsPendingObjects596=== PAUSE TestGCSweepSkipsPendingObjects597=== RUN TestTombstonedObjectOfferedWithoutWaiting598=== PAUSE TestTombstonedObjectOfferedWithoutWaiting599=== RUN TestGCSweepDeliversEachKeyOnce600=== PAUSE TestGCSweepDeliversEachKeyOnce601=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection602=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection603=== RUN TestForceGCDuringPushOffersSweptObject604=== PAUSE TestForceGCDuringPushOffersSweptObject605=== RUN TestFailedS3DeleteKeepsTombstone606=== PAUSE TestFailedS3DeleteKeepsTombstone607=== RUN TestSweepSkipsRowsLockedByPush608=== PAUSE TestSweepSkipsRowsLockedByPush609=== RUN TestRegisterCompletedObjectNeedsPendingRow610=== PAUSE TestRegisterCompletedObjectNeedsPendingRow611=== RUN TestGetExistingObjectsLocksRows612=== PAUSE TestGetExistingObjectsLocksRows613=== RUN TestSweepRechecksPendingAfterLocking614=== PAUSE TestSweepRechecksPendingAfterLocking615=== RUN TestPushWaitsForSweepHoldingRow616=== PAUSE TestPushWaitsForSweepHoldingRow617=== RUN TestCommitRacingPendingCleanupKeepsObjects618=== PAUSE TestCommitRacingPendingCleanupKeepsObjects619=== RUN TestGCEndsOnShutdown620=== PAUSE TestGCEndsOnShutdown621=== RUN TestGCTaskStore_StartNew622=== PAUSE TestGCTaskStore_StartNew623=== RUN TestGCTaskStore_DeduplicateSameParams624=== PAUSE TestGCTaskStore_DeduplicateSameParams625=== RUN TestGCTaskStore_ConflictDifferentParams626=== PAUSE TestGCTaskStore_ConflictDifferentParams627=== RUN TestGCTaskStore_GetEmpty628=== PAUSE TestGCTaskStore_GetEmpty629=== RUN TestGCTaskStore_GetReturnsLatest630=== PAUSE TestGCTaskStore_GetReturnsLatest631=== RUN TestGCTaskStore_CompletedAllowsNewTask632=== PAUSE TestGCTaskStore_CompletedAllowsNewTask633=== RUN TestGCTaskStore_PhaseUpdates634=== PAUSE TestGCTaskStore_PhaseUpdates635=== RUN TestGCTaskStore_Fail636=== PAUSE TestGCTaskStore_Fail637=== RUN TestGracefulShutdownDrainsInflight638=== PAUSE TestGracefulShutdownDrainsInflight639=== RUN TestService_healthCheckHandler640=== PAUSE TestService_healthCheckHandler641=== RUN TestService_readinessHandler642=== PAUSE TestService_readinessHandler643=== RUN TestGenerateLandingPage644=== PAUSE TestGenerateLandingPage645=== RUN TestEnsureAbortUploadsRule646=== PAUSE TestEnsureAbortUploadsRule647=== RUN TestCacheConfigHandlerMaxNarSize648=== PAUSE TestCacheConfigHandlerMaxNarSize649=== RUN TestCreatePendingClosureRejectsOversizedNAR650=== PAUSE TestCreatePendingClosureRejectsOversizedNAR651=== RUN TestNARDeduplicationMetadataUploadBug652=== PAUSE TestNARDeduplicationMetadataUploadBug653=== RUN TestMetricsInventory654=== PAUSE TestMetricsInventory655=== RUN TestMetricsCommit656=== PAUSE TestMetricsCommit657=== RUN TestService_NativeMTLS658=== PAUSE TestService_NativeMTLS659=== RUN TestServerTLSConfig660=== PAUSE TestServerTLSConfig661=== RUN TestMultipartCleanup662=== PAUSE TestMultipartCleanup663=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded664=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded665=== RUN TestPendingCleanupUsesOneCutoff666=== PAUSE TestPendingCleanupUsesOneCutoff667=== RUN TestObjectStatsTrigger668=== PAUSE TestObjectStatsTrigger669=== RUN TestObjectStatsUpdatedOncePerCommit670=== PAUSE TestObjectStatsUpdatedOncePerCommit671=== RUN TestOrphanedObjectsGC672=== PAUSE TestOrphanedObjectsGC673=== RUN TestOrphanedObjectsGCStressTest674=== PAUSE TestOrphanedObjectsGCStressTest675=== RUN TestCreatePin_ReservedPins676=== PAUSE TestCreatePin_ReservedPins677=== RUN TestPresentReportsOnlyClosureRoots678=== PAUSE TestPresentReportsOnlyClosureRoots679=== RUN TestPresentRefreshesOnlyStaleClosures680=== PAUSE TestPresentRefreshesOnlyStaleClosures681=== RUN TestPresentNotReportedWhileGCDeletesClosure682=== PAUSE TestPresentNotReportedWhileGCDeletesClosure683=== RUN TestParseSingleRange684=== PAUSE TestParseSingleRange685=== RUN TestProxyHeadersOnlyTrustedOnSocket6862026/11/12 11:45:56 INFO Starting HTTP server address=proxy.sock6872026/11/12 11:45:56 INFO Starting HTTP server address=127.0.0.1:453336882026/11/12 11:45:56 WARN mTLS auth: subject not in bound subjects subject="CN=someone"6892026/11/12 11:45:56 INFO Shutdown signal received, draining in-flight requests timeout=10s690--- PASS: TestProxyHeadersOnlyTrustedOnSocket (0.12s)691=== RUN TestIsValidCachePath692=== PAUSE TestIsValidCachePath693=== RUN TestReadProxyNarinfo694=== PAUSE TestReadProxyNarinfo695=== RUN TestReadProxyNarinfoAlreadyDecompressed696=== PAUSE TestReadProxyNarinfoAlreadyDecompressed697=== RUN TestReadProxyContentNegotiation698=== PAUSE TestReadProxyContentNegotiation699=== RUN TestReadProxyNarStreaming700=== PAUSE TestReadProxyNarStreaming701=== RUN TestReadProxy404702=== PAUSE TestReadProxy404703=== RUN TestReadProxyInvalidPath704=== PAUSE TestReadProxyInvalidPath705=== RUN TestReadProxyHead706=== PAUSE TestReadProxyHead707=== RUN TestReadProxyOutlastsServerWriteTimeout708=== PAUSE TestReadProxyOutlastsServerWriteTimeout709=== RUN TestReadProxyConditionalGet710=== PAUSE TestReadProxyConditionalGet711=== RUN TestReadProxyRootRedirectsToIndexHTML712=== PAUSE TestReadProxyRootRedirectsToIndexHTML713=== RUN TestReadProxyDisabled714=== PAUSE TestReadProxyDisabled715=== RUN TestReadRedirectNar716=== PAUSE TestReadRedirectNar717=== RUN TestReadRedirectKeepsNarinfoProxied718=== PAUSE TestReadRedirectKeepsNarinfoProxied719=== RUN TestReadProxyRangeRequest720=== PAUSE TestReadProxyRangeRequest721=== RUN TestReadRedirectUsesPublicS3URL722=== PAUSE TestReadRedirectUsesPublicS3URL723=== RUN TestPush_OverlappingRootsStoreOneRowPerKey724=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey725=== RUN TestPush_CompleteCommitsEveryRoot726=== PAUSE TestPush_CompleteCommitsEveryRoot727=== RUN TestPush_RepeatedRootCommitsOnce728=== PAUSE TestPush_RepeatedRootCommitsOnce729=== RUN TestPush_CompleteWaitsForCleanupOfThePush730=== PAUSE TestPush_CompleteWaitsForCleanupOfThePush731=== RUN TestPush_RootsAreLockedInByteOrder732=== PAUSE TestPush_RootsAreLockedInByteOrder733=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit734=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit735=== RUN TestPush_RejectsBadRequests736=== PAUSE TestPush_RejectsBadRequests737=== RUN TestPush_SignsNarinfosOfItsPendingObjects738=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects739=== RUN TestCommitCanBeRepeatedUntilCleanup740=== PAUSE TestCommitCanBeRepeatedUntilCleanup741=== RUN TestSharedNarBothUploadsComplete742=== PAUSE TestSharedNarBothUploadsComplete743=== RUN TestCompleteMultipartUpload_ErrorButObjectExists744=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists745=== RUN TestCompletedNarNotReofferedAcrossClosures746=== PAUSE TestCompletedNarNotReofferedAcrossClosures747=== RUN TestPresignedUploadRegisteredBeforeCommit748=== PAUSE TestPresignedUploadRegisteredBeforeCommit749=== RUN TestService_Rustfstest750=== PAUSE TestService_Rustfstest751=== RUN TestParseSize752=== PAUSE TestParseSize753=== RUN TestSkippedUploadsHandler754=== PAUSE TestSkippedUploadsHandler755=== RUN TestSystemdListenerNotActivated756--- PASS: TestSystemdListenerNotActivated (0.00s)757=== RUN TestWatchdogBeatsWhenHealthy758--- PASS: TestWatchdogBeatsWhenHealthy (0.02s)759=== RUN TestWatchdogSkipsWhenUnhealthy7602026/11/12 11:45:56 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7612026/11/12 11:45:56 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7622026/11/12 11:45:56 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7632026/11/12 11:45:56 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7642026/11/12 11:45:56 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7652026/11/12 11:45:56 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7662026/11/12 11:45:56 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7672026/11/12 11:45:56 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7682026/11/12 11:45:57 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7692026/11/12 11:45:57 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"770--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)771=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle772=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle773=== RUN TestProxyWriteTimeout774=== PAUSE TestProxyWriteTimeout775=== RUN TestPendingClosureWriteTimeout776=== PAUSE TestPendingClosureWriteTimeout777=== RUN TestIsValidUploadKey778=== PAUSE TestIsValidUploadKey779=== RUN TestUploadHandlersRejectInvalidKeys780=== PAUSE TestUploadHandlersRejectInvalidKeys781=== RUN TestUploadHandlersRejectOversizedBody782=== PAUSE TestUploadHandlersRejectOversizedBody783=== RUN TestService_cleanupPendingClosuresHandler784=== PAUSE TestService_cleanupPendingClosuresHandler785=== RUN TestService_createPendingClosureHandler786=== PAUSE TestService_createPendingClosureHandler787=== RUN TestService_verifyS3Integrity788=== PAUSE TestService_verifyS3Integrity789=== RUN TestService_verifyS3ReuploadsStaleNarinfo790=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo791=== RUN TestCompleteMultipartUnregistered792=== PAUSE TestCompleteMultipartUnregistered793=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT794=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT795=== CONT TestAbortSucceeded796=== CONT TestPush_SignsNarinfosOfItsPendingObjects797--- PASS: TestAbortSucceeded (0.00s)798=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT799=== CONT TestCompleteMultipartUnregistered800=== CONT TestService_verifyS3ReuploadsStaleNarinfo801=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed802=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed803=== RUN TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed804=== CONT TestService_verifyS3Integrity805=== CONT TestService_createPendingClosureHandler806=== CONT TestService_cleanupPendingClosuresHandler807=== CONT TestUploadHandlersRejectOversizedBody808=== CONT TestUploadHandlersRejectInvalidKeys809=== CONT TestIsValidUploadKey810=== CONT TestPendingClosureWriteTimeout811=== CONT TestProxyWriteTimeout812=== RUN TestPendingClosureWriteTimeout/empty813=== PAUSE TestPendingClosureWriteTimeout/empty814=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle815=== CONT TestSkippedUploadsHandler816=== CONT TestParseSize817=== CONT TestService_Rustfstest818=== CONT TestPresignedUploadRegisteredBeforeCommit819=== CONT TestCompletedNarNotReofferedAcrossClosures820=== CONT TestCompleteMultipartUpload_ErrorButObjectExists821=== CONT TestSharedNarBothUploadsComplete822=== CONT TestCommitCanBeRepeatedUntilCleanup823=== CONT TestOrphanedObjectsGC824=== CONT TestObjectStatsUpdatedOncePerCommit825=== CONT TestObjectStatsTrigger826=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed827=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info828=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info829--- PASS: TestParseSize (0.00s)830=== RUN TestProxyWriteTimeout/narinfo831=== CONT TestPendingCleanupUsesOneCutoff8322026/11/12 11:45:57 INFO Client skipped oversized paths paths=3 nar_bytes=5000000000833=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost834=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost835=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed836=== RUN TestIsValidUploadKey/narinfo837=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal838=== PAUSE TestProxyWriteTimeout/narinfo839=== RUN TestPendingClosureWriteTimeout/400_objects840=== RUN TestProxyWriteTimeout/1_GiB_nar841=== RUN TestCommitCanBeRepeatedUntilCleanup/push842=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed843=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed844=== PAUSE TestIsValidUploadKey/narinfo845=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal846=== PAUSE TestPendingClosureWriteTimeout/400_objects847=== PAUSE TestCommitCanBeRepeatedUntilCleanup/push848=== RUN TestPendingClosureWriteTimeout/670k_objects849=== RUN TestCommitCanBeRepeatedUntilCleanup/pending_closure850=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed851=== RUN TestIsValidUploadKey/nar_zst852=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key853=== PAUSE TestProxyWriteTimeout/1_GiB_nar854=== RUN TestProxyWriteTimeout/10_GiB_nar855=== PAUSE TestPendingClosureWriteTimeout/670k_objects856=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded857=== PAUSE TestIsValidUploadKey/nar_zst858=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key859=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key860=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key861=== PAUSE TestProxyWriteTimeout/10_GiB_nar862=== RUN TestProxyWriteTimeout/unknown_size863=== PAUSE TestProxyWriteTimeout/unknown_size864--- PASS: TestSkippedUploadsHandler (0.09s)865=== CONT TestMultipartCleanup866=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline867=== RUN TestIsValidUploadKey/nar_xz868=== PAUSE TestIsValidUploadKey/nar_xz869=== PAUSE TestCommitCanBeRepeatedUntilCleanup/pending_closure870=== CONT TestService_NativeMTLS871=== CONT TestServerTLSConfig872=== RUN TestServerTLSConfig/no_client_CA873=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline874=== RUN TestIsValidUploadKey/nar_plain875=== PAUSE TestIsValidUploadKey/nar_plain876=== RUN TestIsValidUploadKey/listing877=== PAUSE TestIsValidUploadKey/listing878=== RUN TestIsValidUploadKey/build_log879=== PAUSE TestIsValidUploadKey/build_log880=== RUN TestIsValidUploadKey/build_log_home-manager_file881=== CONT TestMetricsCommit882=== PAUSE TestServerTLSConfig/no_client_CA883=== CONT TestMetricsInventory884=== PAUSE TestIsValidUploadKey/build_log_home-manager_file885=== RUN TestServerTLSConfig/missing_CA_file886=== PAUSE TestServerTLSConfig/missing_CA_file887=== RUN TestServerTLSConfig/not_a_PEM_file888=== PAUSE TestServerTLSConfig/not_a_PEM_file889=== RUN TestIsValidUploadKey/build_log_plus_in_name890=== PAUSE TestIsValidUploadKey/build_log_plus_in_name891=== RUN TestIsValidUploadKey/build_log_question_mark892=== CONT TestNARDeduplicationMetadataUploadBug893=== PAUSE TestIsValidUploadKey/build_log_question_mark894=== RUN TestIsValidUploadKey/build_log_equals895=== PAUSE TestIsValidUploadKey/build_log_equals896=== RUN TestIsValidUploadKey/realisation897=== PAUSE TestIsValidUploadKey/realisation898=== RUN TestIsValidUploadKey/realisation_plus_in_output899=== PAUSE TestIsValidUploadKey/realisation_plus_in_output900=== RUN TestIsValidUploadKey/nix-cache-info901=== PAUSE TestIsValidUploadKey/nix-cache-info902=== RUN TestIsValidUploadKey/index.html903=== PAUSE TestIsValidUploadKey/index.html904=== RUN TestIsValidUploadKey/narinfo_key,_nar_type905=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type906=== RUN TestIsValidUploadKey/nar_key,_narinfo_type907=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type908=== RUN TestIsValidUploadKey/listing_key,_narinfo_type909=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type910=== RUN TestIsValidUploadKey/traversal911=== PAUSE TestIsValidUploadKey/traversal912=== RUN TestIsValidUploadKey/traversal_nar913=== PAUSE TestIsValidUploadKey/traversal_nar914=== RUN TestIsValidUploadKey/absolute915=== PAUSE TestIsValidUploadKey/absolute916=== RUN TestIsValidUploadKey/empty_key917=== PAUSE TestIsValidUploadKey/empty_key918=== RUN TestIsValidUploadKey/unknown_type919=== PAUSE TestIsValidUploadKey/unknown_type920=== CONT TestCreatePendingClosureRejectsOversizedNAR9212026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures922--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.00s)923=== CONT TestCacheConfigHandlerMaxNarSize924--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)925=== CONT TestEnsureAbortUploadsRule926=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure927=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure928=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart929=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart930=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts931=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts932=== CONT TestGenerateLandingPage933--- PASS: TestGenerateLandingPage (0.00s)934=== CONT TestService_readinessHandler9352026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures936--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (0.34s)937=== CONT TestService_healthCheckHandler9382026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures9392026/11/12 11:45:57 INFO Received cleanup request method=DELETE path=/api/pending_closures9402026/11/12 11:45:57 INFO Aborted multipart uploads count=0 kept=09412026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures9422026/11/12 11:45:57 INFO Received push request method=POST path=/api/pushes9432026/11/12 11:45:57 INFO Received cleanup request method=DELETE path=/api/pending_closures9442026/11/12 11:45:57 INFO Aborted multipart uploads count=1 kept=09452026/11/12 11:45:57 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete9462026-11-12 11:45:57.427 UTC [649] ERROR: Closure does not exist: id=19472026-11-12 11:45:57.427 UTC [649] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 17 at RAISE9482026-11-12 11:45:57.427 UTC [649] STATEMENT: -- name: CommitPendingClosure :one949 SELECT commit_pending_closure($1::bigint)::bigint950 951--- PASS: TestService_cleanupPendingClosuresHandler (0.40s)952=== CONT TestLeadEndsOnShutdown9532026/11/12 11:45:57 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9542026/11/12 11:45:57 INFO Signed narinfos id=1 count=19552026/11/12 11:45:57 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9562026/11/12 11:45:57 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9572026/11/12 11:45:57 ERROR Failed to sign narinfos signer="index 1" error="signing key cannot be nil"9582026/11/12 11:45:57 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9592026/11/12 11:45:57 ERROR Failed to sign narinfos signer=niks3-test-1 error="signer failed"9602026/11/12 11:45:57 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9612026/11/12 11:45:57 ERROR Signer returned unexpected signature count signer=niks3-test-1 got=0 want=19622026/11/12 11:45:57 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9632026/11/12 11:45:57 ERROR Signer returned unexpected object key signer=niks3-test-1 object_key=unexpected.narinfo964--- PASS: TestPush_SignsNarinfosOfItsPendingObjects (0.40s)965=== CONT TestLeadIncumbentAnnouncesAtOnce966--- PASS: TestService_Rustfstest (0.41s)967=== CONT TestLeadUnannouncedStandbyYieldsToIncumbent9682026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures9692026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures9702026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures9712026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures972--- PASS: TestObjectStatsTrigger (0.49s)973=== CONT TestLeadElectsOneAndHandsOver9742026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures9752026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures9762026/11/12 11:45:57 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9772026/11/12 11:45:57 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=OThmM2QzNGUtZjU1NS00MmRiLTgzMzMtMDU2MDlhMjZjMmNkLjIxNjZkMjE3LTc2NDYtNGE5OS05MTg1LTg4ZWI2ZjYyYjQ5ZngxNzk0NDgzOTU3NTkxMDM1NjUw978--- PASS: TestObjectStatsUpdatedOncePerCommit (0.60s)9792026/11/12 11:45:57 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=OThmM2QzNGUtZjU1NS00MmRiLTgzMzMtMDU2MDlhMjZjMmNkLjIxNjZkMjE3LTc2NDYtNGE5OS05MTg1LTg4ZWI2ZjYyYjQ5ZngxNzk0NDgzOTU3NTkxMDM1NjUw parts=1980=== CONT TestConnectWaitsForAPeerMigration981--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (0.60s)982=== CONT TestResolveDBConnectionString983=== RUN TestResolveDBConnectionString/flag_wins984=== PAUSE TestResolveDBConnectionString/flag_wins985=== RUN TestResolveDBConnectionString/file_when_flag_empty986=== PAUSE TestResolveDBConnectionString/file_when_flag_empty987=== RUN TestResolveDBConnectionString/missing_file_is_an_error988=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error989=== RUN TestResolveDBConnectionString/PGHOST_allows_empty990=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty991=== RUN TestResolveDBConnectionString/nothing_configured992=== PAUSE TestResolveDBConnectionString/nothing_configured993=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock9942026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures9952026/11/12 11:45:57 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9962026/11/12 11:45:57 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst9972026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures998--- PASS: TestPresignedUploadRegisteredBeforeCommit (0.63s)999=== CONT TestDeleteClosuresRacingPins10002026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures10012026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures10022026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures10032026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures10042026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures10052026/11/12 11:45:57 INFO Received cleanup request method=DELETE path=/api/pending_closures1006--- PASS: TestMetricsInventory (0.65s)1007=== CONT TestClientFallsBackToClosures10082026/11/12 11:45:57 INFO Received push request method=POST path=/api/pushes10092026/11/12 11:45:57 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10102026/11/12 11:45:57 INFO Received complete push request method=POST path=/api/pushes/1/complete10112026/11/12 11:45:57 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10122026/11/12 11:45:57 INFO Completed push id=1 objects=4 duration=7.63377ms10132026/11/12 11:45:57 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete10142026/11/12 11:45:57 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst1015--- PASS: TestCompleteMultipartUnregistered (0.79s)1016=== CONT TestClientPushesUseOnePush10172026/11/12 11:45:57 INFO Completed upload id=2 objects=2 duration=1.37119ms10182026/11/12 11:45:57 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=OThmM2QzNGUtZjU1NS00MmRiLTgzMzMtMDU2MDlhMjZjMmNkLjRmZDYxOTI2LWYzNGMtNDJjMC1iNWU4LThiNTQyYzI4YmZkMngxNzk0NDgzOTU3Mzg4MjM4ODE0 parts=1010192026/11/12 11:45:57 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete1020--- PASS: TestMetricsCommit (0.70s)1021=== CONT TestPinProtectsFromGC10222026/11/12 11:45:57 INFO Completed upload id=1 objects=2 duration=5.38564ms10232026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures10242026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures10252026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures10262026/11/12 11:45:57 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo10272026/11/12 11:45:57 WARN Found objects in DB but missing from S3, will re-upload count=11028--- PASS: TestService_verifyS3Integrity (0.81s)1029=== CONT TestClientSharedPathCommittedMidPush10302026/11/12 11:45:57 INFO Received cleanup request method=DELETE path=/api/pending_closures10312026/11/12 11:45:57 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket18/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""10322026/11/12 11:45:57 INFO Aborted multipart uploads count=0 kept=110332026/11/12 11:45:57 INFO Received cleanup request method=DELETE path=/api/pending_closures1034--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (0.74s)1035=== CONT TestClientWithDependencies10362026/11/12 11:45:57 INFO Aborted multipart uploads count=1 kept=01037--- PASS: TestMultipartCleanup (0.74s)1038=== CONT TestClientMultipleUploads10392026/11/12 11:45:57 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10402026/11/12 11:45:57 INFO Installed lifecycle rule aborting incomplete NAR uploads bucket=bucket24 days=21041--- PASS: TestEnsureAbortUploadsRule (0.79s)1042=== CONT TestClientIntegration10432026/11/12 11:45:57 WARN readiness check failed error="closed pool"1044--- PASS: TestService_readinessHandler (0.72s)1045=== CONT TestClientErrorHandling1046=== RUN TestClientErrorHandling/InvalidStorePath1047=== PAUSE TestClientErrorHandling/InvalidStorePath1048=== RUN TestClientErrorHandling/InvalidAuthToken1049=== PAUSE TestClientErrorHandling/InvalidAuthToken1050=== RUN TestClientErrorHandling/ServerNotAvailable1051=== PAUSE TestClientErrorHandling/ServerNotAvailable1052=== CONT TestClientCADerivations10532026/11/12 11:45:57 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=OThmM2QzNGUtZjU1NS00MmRiLTgzMzMtMDU2MDlhMjZjMmNkLjFmM2IxY2VmLWNlYjAtNDY1MS1iNDZmLTkyMTYzNzI4OGU0NXgxNzk0NDgzOTU3NDc4MTc4NjIy parts=1010542026/11/12 11:45:57 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10552026/11/12 11:45:57 INFO Completed upload id=1 objects=2 duration=5.81648ms10562026/11/12 11:45:57 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000010572026/11/12 11:45:57 INFO Received uploads request method=POST path=/api/pending_closures10582026/11/12 11:45:57 INFO Starting cleanup of old closures method=DELETE path=/api/closures10592026/11/12 11:45:57 INFO Aborted multipart uploads count=0 kept=010602026/11/12 11:45:57 WARN mTLS auth: subject not in bound subjects subject="CN=reader"10612026/11/12 11:45:57 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1062--- PASS: TestService_NativeMTLS (0.84s)1063=== CONT TestCacheStatsHandler10642026/11/12 11:45:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010652026/11/12 11:45:57 INFO Vacuumed table table=pending_closures10662026/11/12 11:45:57 INFO Vacuumed table table=commit_receipts10672026/11/12 11:45:57 INFO Vacuumed table table=pending_objects10682026/11/12 11:45:57 INFO Vacuumed table table=multipart_uploads10692026/11/12 11:45:57 INFO Vacuumed table table=closures10702026/11/12 11:45:57 INFO Vacuumed table table=objects10712026/11/12 11:45:57 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001072--- PASS: TestService_createPendingClosureHandler (0.96s)1073=== CONT TestCacheConfigHandler1074=== RUN TestCacheConfigHandler/full_config,_no_issuer1075=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1076=== RUN TestCacheConfigHandler/no_cache_url_configured1077=== PAUSE TestCacheConfigHandler/no_cache_url_configured1078=== RUN TestCacheConfigHandler/no_signing_keys1079=== PAUSE TestCacheConfigHandler/no_signing_keys1080=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1081=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1082=== CONT TestService_ReadScope_PublicByDefault1083--- PASS: TestService_healthCheckHandler (0.65s)1084=== CONT TestService_RequireScope_OIDC10852026/11/12 11:45:58 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1086=== NAME TestNARDeduplicationMetadataUploadBug1087 metadata_upload_test.go:48: First store path: /build/TestNARDeduplicationMetadataUploadBug1257523792/001/store/mjwyn67k6p52m9mi4pq21h600gar334n-file1.txt10882026/11/12 11:45:58 INFO lead: acquired remote=192.0.2.1:123410892026/11/12 11:45:58 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=OThmM2QzNGUtZjU1NS00MmRiLTgzMzMtMDU2MDlhMjZjMmNkLjJmZmI3Zjg5LTg0YWItNDcyZC04MGQzLWYwYjdiNTZlMWU0NngxNzk0NDgzOTU3NTI3Mjc3ODUx parts=1210902026/11/12 11:45:58 INFO Received uploads request method=POST path=/api/pending_closures1091--- PASS: TestCompletedNarNotReofferedAcrossClosures (1.03s)1092=== CONT TestService_AuthMiddleware_OIDC10932026/11/12 11:45:58 INFO lead: acquired remote=192.0.2.1:123410942026/11/12 11:45:58 INFO lead: released remote=192.0.2.1:12341095--- PASS: TestLeadIncumbentAnnouncesAtOnce (0.65s)1096=== CONT TestService_ReadAuthMiddleware10972026/11/12 11:45:58 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:38677/oidc10982026/11/12 11:45:58 INFO lead: acquired remote=192.0.2.1:123410992026/11/12 11:45:58 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:45657/oidc11002026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes11012026/11/12 11:45:58 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)11022026/11/12 11:45:58 INFO Uploading mjwyn67k6p52m9mi4pq21h600gar334n-file1.txt (160B)11032026/11/12 11:45:58 INFO lead: acquired remote=192.0.2.1:123411042026/11/12 11:45:58 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"11052026/11/12 11:45:58 WARN Failed to register uploaded object key=mjwyn67k6p52m9mi4pq21h600gar334n.ls error="server returned 404: 404 page not found\n"11062026/11/12 11:45:58 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign11072026/11/12 11:45:58 INFO Signed narinfos id=1 count=111082026/11/12 11:45:58 INFO Uploading 1 narinfos11092026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/1/complete11102026/11/12 11:45:58 WARN Failed to register uploaded object key=mjwyn67k6p52m9mi4pq21h600gar334n.narinfo error="server returned 404: 404 page not found\n"11112026/11/12 11:45:58 INFO Completed push id=1 objects=3 duration=7.34139ms11122026/11/12 11:45:58 INFO Upload complete. (73ms)1113=== NAME TestNARDeduplicationMetadataUploadBug1114 metadata_upload_test.go:54: Retrieved narinfo from S3:1115 StorePath: /build/TestNARDeduplicationMetadataUploadBug1257523792/001/store/mjwyn67k6p52m9mi4pq21h600gar334n-file1.txt1116 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1117 Compression: zstd1118 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1119 NarSize: 1601120 References: 1121 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1122 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)1123 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):1124 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}11252026/11/12 11:45:58 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1126 metadata_upload_test.go:64: Second store path (same content): /build/TestNARDeduplicationMetadataUploadBug1257523792/001/store/sq68qgkpxnx4cmc319gqgncszphc8xw2-file2.txt11272026/11/12 11:45:58 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=OThmM2QzNGUtZjU1NS00MmRiLTgzMzMtMDU2MDlhMjZjMmNkLjdlNzU4YWU2LWQyYmUtNGI4MC04NzVmLTIyYTVlOTEyNWMyNHgxNzk0NDgzOTU3NjcyOTA0ODY5 parts=1211282026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes11292026/11/12 11:45:58 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)11302026/11/12 11:45:58 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign11312026/11/12 11:45:58 INFO Signed narinfos id=2 count=111322026/11/12 11:45:58 WARN Failed to register uploaded object key=sq68qgkpxnx4cmc319gqgncszphc8xw2.ls error="server returned 404: 404 page not found\n"11332026/11/12 11:45:58 INFO Uploading 1 narinfos1134=== NAME TestOrphanedObjectsGC1135 orphaned_objects_gc_test.go:287: GC Test Summary:1136 orphaned_objects_gc_test.go:288: - Kept: 2 objects from closure A1137 orphaned_objects_gc_test.go:289: - Deleted: 2 objects from closure B1138 orphaned_objects_gc_test.go:290: - Deleted: 6 orphaned chain objects (X1->X2->X3)1139 orphaned_objects_gc_test.go:291: - Deleted: 2 orphaned single objects (Y)1140 orphaned_objects_gc_test.go:292: - Total deleted: 10 objects1141--- PASS: TestOrphanedObjectsGC (1.25s)1142=== CONT TestService_AuthMiddleware_MTLSBoundSubjects11432026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/2/complete11442026/11/12 11:45:58 WARN Failed to register uploaded object key=sq68qgkpxnx4cmc319gqgncszphc8xw2.narinfo error="server returned 404: 404 page not found\n"11452026/11/12 11:45:58 INFO Completed push id=2 objects=3 duration=1.46663ms11462026/11/12 11:45:58 INFO Upload complete. (50ms)1147=== NAME TestNARDeduplicationMetadataUploadBug1148 metadata_upload_test.go:76: Retrieved narinfo from S3:1149 StorePath: /build/TestNARDeduplicationMetadataUploadBug1257523792/001/store/sq68qgkpxnx4cmc319gqgncszphc8xw2-file2.txt1150 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1151 Compression: zstd1152 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1153 NarSize: 1601154 References: 1155 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1156 metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes)1157 metadata_upload_test.go:77: Decompressed .ls content (49 bytes):1158 {"version":1,"root":{"type":"regular","size":44}}1159--- PASS: TestNARDeduplicationMetadataUploadBug (1.17s)1160=== CONT TestGCAdvisoryLockBlocksConcurrentRun1161=== NAME TestPinProtectsFromGC1162 client_integration_test.go:791: Pinned store path: /build/TestPinProtectsFromGC1396743254/001/store/8axdyd6dw61j38gznhz4jdnvwcpn2bbp-pinned-file.txt1163 client_integration_test.go:792: Unpinned store path: /build/TestPinProtectsFromGC1396743254/001/store/b85sxmg674mhnlin6l7yrs7vi8nkcwjm-unpinned-file.txt1164=== NAME TestClientMultipleUploads1165 client_integration_test.go:406: Created store path 0: /build/TestClientMultipleUploads162250291/001/store/vk8sq4cwrim12qq4l6a4ivly0h9yf50h-test-file-0.txt1166--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (0.76s)1167=== CONT TestService_AuthMiddleware_MTLSProxyHeader11682026/11/12 11:45:58 INFO Received uploads request method=POST path=/api/pending_closures11692026/11/12 11:45:58 INFO Received uploads request method=POST path=/api/pending_closures1170=== NAME TestClientWithDependencies1171 client_integration_test.go:661: Built derivation: /build/TestClientWithDependencies3485559823/001/store/399g33csklg36jqzfhbl8k0s38gqp5ni-test-script11722026/11/12 11:45:58 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)11732026/11/12 11:45:58 INFO Uploading jjk27960d16i2c6n5rl8dwqkfas6bqhk-shared-dep (136B)11742026/11/12 11:45:58 INFO Uploading jmpc28h9bxfrdk3nhrgxnwkx5c0mw9fd-a (216B)1175=== NAME TestClientIntegration1176 client_integration_test.go:292: Created store path: /build/TestClientIntegration2282523218/002/store/ari43awr51k3ybfk41rkla86cg4xgxsp-test-file.txt11772026/11/12 11:45:58 WARN Failed to register uploaded object key=4yf8vy0sk8arxy5ykscv5ggz8h6zb8sy.ls error="server returned 404: 404 page not found\n"11782026/11/12 11:45:58 WARN Failed to register uploaded object key=jjk27960d16i2c6n5rl8dwqkfas6bqhk.ls error="server returned 404: 404 page not found\n"11792026/11/12 11:45:58 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"11802026/11/12 11:45:58 WARN Failed to register uploaded object key=jmpc28h9bxfrdk3nhrgxnwkx5c0mw9fd.ls error="server returned 404: 404 page not found\n"11812026/11/12 11:45:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign11822026/11/12 11:45:58 WARN Failed to register uploaded object key=nar/0jn886m1hcwwnqd4si5hp01hzixh1rlx2r8kjvvvr813q0fq38v9.nar.zst error="server returned 404: 404 page not found\n"11832026/11/12 11:45:58 INFO Signed narinfos id=1 count=211842026/11/12 11:45:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign11852026/11/12 11:45:58 INFO Signed narinfos id=2 count=211862026/11/12 11:45:58 INFO Uploading 4 narinfos11872026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes11882026/11/12 11:45:58 WARN Failed to register uploaded object key=jmpc28h9bxfrdk3nhrgxnwkx5c0mw9fd.narinfo error="server returned 404: 404 page not found\n"11892026/11/12 11:45:58 WARN Failed to register uploaded object key=jjk27960d16i2c6n5rl8dwqkfas6bqhk.narinfo error="server returned 404: 404 page not found\n"1190=== NAME TestClientMultipleUploads1191 client_integration_test.go:406: Created store path 1: /build/TestClientMultipleUploads162250291/001/store/qgqfr80wh1gv1xc3synhp6fy7df32qy6-test-file-1.txt1192--- PASS: TestCacheStatsHandler (0.48s)11932026/11/12 11:45:58 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)1194=== CONT TestService_AuthMiddleware11952026/11/12 11:45:58 INFO Uploading 8axdyd6dw61j38gznhz4jdnvwcpn2bbp-pinned-file.txt (128B)11962026/11/12 11:45:58 WARN Failed to register uploaded object key=4yf8vy0sk8arxy5ykscv5ggz8h6zb8sy.narinfo error="server returned 404: 404 page not found\n"11972026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes11982026/11/12 11:45:58 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete11992026/11/12 11:45:58 WARN Failed to register uploaded object key=jjk27960d16i2c6n5rl8dwqkfas6bqhk.narinfo error="server returned 404: 404 page not found\n"12002026/11/12 11:45:58 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"12012026/11/12 11:45:58 WARN Failed to register uploaded object key=8axdyd6dw61j38gznhz4jdnvwcpn2bbp.ls error="server returned 404: 404 page not found\n"12022026/11/12 11:45:58 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12032026/11/12 11:45:58 INFO Signed narinfos id=1 count=112042026/11/12 11:45:58 INFO Uploading 1 narinfos12052026/11/12 11:45:58 INFO Completed upload id=1 objects=6 duration=4.77808ms12062026/11/12 11:45:58 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)12072026/11/12 11:45:58 INFO Uploading 847sklk31d3qc9pfm6fl2mxp9kd9b1ry-shared-dep (136B)12082026/11/12 11:45:58 INFO Uploading dh3inhy2nv7mrafrc2hfas5k8m6ql0rz-b (216B)12092026/11/12 11:45:58 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete12102026/11/12 11:45:58 INFO Completed upload id=2 objects=6 duration=755.02µs12112026/11/12 11:45:58 INFO Upload complete. (72ms)1212=== NAME TestClientFallsBackToClosures1213 client_pushes_test.go:112: Retrieved narinfo from S3:1214 StorePath: /build/TestClientFallsBackToClosures1411482507/001/store/jjk27960d16i2c6n5rl8dwqkfas6bqhk-shared-dep1215 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1216 Compression: zstd1217 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821218 NarSize: 1361219 References: 1220 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n12212026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/1/complete12222026/11/12 11:45:58 WARN Failed to register uploaded object key=8axdyd6dw61j38gznhz4jdnvwcpn2bbp.narinfo error="server returned 404: 404 page not found\n"1223--- PASS: TestService_ReadScope_PublicByDefault (0.46s)1224=== CONT TestGracefulShutdownDrainsInflight12252026/11/12 11:45:58 INFO Starting HTTP server address=127.0.0.1:3747312262026/11/12 11:45:58 INFO Shutdown signal received, draining in-flight requests timeout=10s1227=== NAME TestClientFallsBackToClosures1228 client_pushes_test.go:112: Retrieved narinfo from S3:1229 StorePath: /build/TestClientFallsBackToClosures1411482507/001/store/jmpc28h9bxfrdk3nhrgxnwkx5c0mw9fd-a1230 URL: nar/0jn886m1hcwwnqd4si5hp01hzixh1rlx2r8kjvvvr813q0fq38v9.nar.zst1231 Compression: zstd1232 NarHash: sha256:0jn886m1hcwwnqd4si5hp01hzixh1rlx2r8kjvvvr813q0fq38v91233 NarSize: 21612342026/11/12 11:45:58 WARN Failed to register uploaded object key=847sklk31d3qc9pfm6fl2mxp9kd9b1ry.ls error="server returned 404: 404 page not found\n"1235 References: /build/TestClientFallsBackToClosures1411482507/001/store/jjk27960d16i2c6n5rl8dwqkfas6bqhk-shared-dep1236 CA: text:sha256:120wd9nciikc16i9fvdbl0nz8gq2waypw60la8m4713123rpj2pa12372026/11/12 11:45:58 WARN Failed to register uploaded object key=40cs9b4aq5b1wybz0xipilgzfyjljxip.ls error="server returned 404: 404 page not found\n"12382026/11/12 11:45:58 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"12392026/11/12 11:45:58 WARN Failed to register uploaded object key=dh3inhy2nv7mrafrc2hfas5k8m6ql0rz.ls error="server returned 404: 404 page not found\n"1240=== NAME TestClientWithDependencies1241 client_integration_test.go:663: Found 1 dependencies (including self)12422026/11/12 11:45:58 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12432026/11/12 11:45:58 WARN Failed to register uploaded object key=nar/0fg3cr5alxvrvb306sxnc2m4fc21793b7945iv4xjf1sda4xpk46.nar.zst error="server returned 404: 404 page not found\n"12442026/11/12 11:45:58 INFO Signed narinfos id=1 count=312452026/11/12 11:45:58 INFO Uploading 3 narinfos1246=== NAME TestClientFallsBackToClosures1247 client_pushes_test.go:112: Retrieved narinfo from S3:1248 StorePath: /build/TestClientFallsBackToClosures1411482507/001/store/4yf8vy0sk8arxy5ykscv5ggz8h6zb8sy-b1249 URL: nar/0jn886m1hcwwnqd4si5hp01hzixh1rlx2r8kjvvvr813q0fq38v9.nar.zst1250 Compression: zstd1251 NarHash: sha256:0jn886m1hcwwnqd4si5hp01hzixh1rlx2r8kjvvvr813q0fq38v91252 NarSize: 2161253 References: /build/TestClientFallsBackToClosures1411482507/001/store/jjk27960d16i2c6n5rl8dwqkfas6bqhk-shared-dep1254 CA: text:sha256:120wd9nciikc16i9fvdbl0nz8gq2waypw60la8m4713123rpj2pa12552026/11/12 11:45:58 INFO Completed push id=1 objects=3 duration=6.12326ms12562026/11/12 11:45:58 INFO Upload complete. (65ms)12572026/11/12 11:45:58 WARN Failed to register uploaded object key=847sklk31d3qc9pfm6fl2mxp9kd9b1ry.narinfo error="server returned 404: 404 page not found\n"12582026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/1/complete12592026/11/12 11:45:58 WARN Failed to register uploaded object key=dh3inhy2nv7mrafrc2hfas5k8m6ql0rz.narinfo error="server returned 404: 404 page not found\n"12602026/11/12 11:45:58 WARN Failed to register uploaded object key=40cs9b4aq5b1wybz0xipilgzfyjljxip.narinfo error="server returned 404: 404 page not found\n"1261--- PASS: TestClientFallsBackToClosures (0.69s)1262=== CONT TestGCTaskStore_Fail1263--- PASS: TestGCTaskStore_Fail (0.00s)1264=== CONT TestReadProxyOutlastsServerWriteTimeout12652026/11/12 11:45:58 INFO Completed push id=1 objects=8 duration=10.90701ms12662026/11/12 11:45:58 INFO Upload complete. (68ms)1267=== NAME TestClientMultipleUploads1268 client_integration_test.go:406: Created store path 2: /build/TestClientMultipleUploads162250291/001/store/l3qp3kwbf8zal4k8kq6q0psz74w79w7a-test-file-2.txt1269=== NAME TestClientPushesUseOnePush1270 client_pushes_test.go:97: Retrieved narinfo from S3:1271 StorePath: /build/TestClientPushesUseOnePush3245861239/001/store/847sklk31d3qc9pfm6fl2mxp9kd9b1ry-shared-dep1272 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1273 Compression: zstd1274 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821275 NarSize: 1361276 References: 1277 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1278--- PASS: TestService_ReadAuthMiddleware (0.39s)1279=== CONT TestGCTaskStore_PhaseUpdates1280--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)1281=== CONT TestGCTaskStore_CompletedAllowsNewTask1282--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)1283=== CONT TestGCTaskStore_GetReturnsLatest1284--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)1285=== CONT TestPush_RejectsBadRequests1286=== NAME TestClientPushesUseOnePush1287 client_pushes_test.go:97: Retrieved narinfo from S3:1288 StorePath: /build/TestClientPushesUseOnePush3245861239/001/store/40cs9b4aq5b1wybz0xipilgzfyjljxip-a1289 URL: nar/0fg3cr5alxvrvb306sxnc2m4fc21793b7945iv4xjf1sda4xpk46.nar.zst1290 Compression: zstd1291 NarHash: sha256:0fg3cr5alxvrvb306sxnc2m4fc21793b7945iv4xjf1sda4xpk461292 NarSize: 2161293 References: /build/TestClientPushesUseOnePush3245861239/001/store/847sklk31d3qc9pfm6fl2mxp9kd9b1ry-shared-dep1294 CA: text:sha256:0ndb8d7brm6p9670jp06wmjxqqbn00yv3rzf26hqjbjhvkw5sqk61295 client_pushes_test.go:97: Retrieved narinfo from S3:1296 StorePath: /build/TestClientPushesUseOnePush3245861239/001/store/dh3inhy2nv7mrafrc2hfas5k8m6ql0rz-b1297 URL: nar/0fg3cr5alxvrvb306sxnc2m4fc21793b7945iv4xjf1sda4xpk46.nar.zst1298 Compression: zstd1299 NarHash: sha256:0fg3cr5alxvrvb306sxnc2m4fc21793b7945iv4xjf1sda4xpk461300 NarSize: 2161301 References: /build/TestClientPushesUseOnePush3245861239/001/store/847sklk31d3qc9pfm6fl2mxp9kd9b1ry-shared-dep1302 CA: text:sha256:0ndb8d7brm6p9670jp06wmjxqqbn00yv3rzf26hqjbjhvkw5sqk61303--- PASS: TestClientPushesUseOnePush (0.67s)1304=== CONT TestGCTaskStore_GetEmpty1305--- PASS: TestGCTaskStore_GetEmpty (0.00s)1306=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit13072026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes13082026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes1309=== NAME TestClientCADerivations13102026/11/12 11:45:58 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)1311 client_ca_test.go:136: Built CA derivation: /build/TestClientCADerivations3550462514/001/store/m3x4r2zsksfacza1mpqaa7im731y1y3v-ca-test13122026/11/12 11:45:58 INFO Uploading mpk2ysxhzr0ijw5jbsyqngwbh610knrl-shared-dep (136B)13132026/11/12 11:45:58 INFO Uploading 2729bsdkcf8vkcvhjn1xi2mc256dh36h-top (224B)13142026/11/12 11:45:58 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)13152026/11/12 11:45:58 INFO Uploading ari43awr51k3ybfk41rkla86cg4xgxsp-test-file.txt (152B)13162026/11/12 11:45:58 WARN Failed to register uploaded object key=mpk2ysxhzr0ijw5jbsyqngwbh610knrl.ls error="server returned 404: 404 page not found\n"13172026/11/12 11:45:58 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13182026/11/12 11:45:58 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"13192026/11/12 11:45:58 WARN Failed to register uploaded object key=ari43awr51k3ybfk41rkla86cg4xgxsp.ls error="server returned 404: 404 page not found\n"13202026/11/12 11:45:58 INFO Signed narinfos id=1 count=113212026/11/12 11:45:58 INFO Uploading 1 narinfos13222026/11/12 11:45:58 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"13232026/11/12 11:45:58 WARN Failed to register uploaded object key=nar/0b0gnvfpa0x4zv8jgzlwq4mm14wsnv0kdhq7vr8x80dyp06p3y76.nar.zst error="server returned 404: 404 page not found\n"13242026/11/12 11:45:58 WARN Failed to register uploaded object key=2729bsdkcf8vkcvhjn1xi2mc256dh36h.ls error="server returned 404: 404 page not found\n"13252026/11/12 11:45:58 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13262026/11/12 11:45:58 INFO Signed narinfos id=1 count=213272026/11/12 11:45:58 INFO Uploading 2 narinfos13282026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/1/complete13292026/11/12 11:45:58 WARN Failed to register uploaded object key=ari43awr51k3ybfk41rkla86cg4xgxsp.narinfo error="server returned 404: 404 page not found\n"13302026/11/12 11:45:58 WARN Failed to register uploaded object key=mpk2ysxhzr0ijw5jbsyqngwbh610knrl.narinfo error="server returned 404: 404 page not found\n"13312026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/1/complete13322026/11/12 11:45:58 WARN Failed to register uploaded object key=2729bsdkcf8vkcvhjn1xi2mc256dh36h.narinfo error="server returned 404: 404 page not found\n"13332026/11/12 11:45:58 INFO Completed push id=1 objects=3 duration=6.50009ms13342026/11/12 11:45:58 INFO Upload complete. (59ms)1335--- PASS: TestGracefulShutdownDrainsInflight (0.07s)1336=== CONT TestGCTaskStore_ConflictDifferentParams1337--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)1338=== CONT TestPush_RootsAreLockedInByteOrder13392026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes1340=== RUN TestService_RequireScope_OIDC/builder_may_write13412026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes1342=== PAUSE TestService_RequireScope_OIDC/builder_may_write1343=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1344=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1345=== RUN TestService_RequireScope_OIDC/ops_may_admin1346=== PAUSE TestService_RequireScope_OIDC/ops_may_admin1347=== RUN TestService_RequireScope_OIDC/ops_may_not_write1348=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1349=== RUN TestService_RequireScope_OIDC/reader_may_not_write1350=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1351=== RUN TestService_RequireScope_OIDC/static_token_may_admin1352=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1353=== RUN TestService_RequireScope_OIDC/static_token_may_write1354=== PAUSE TestService_RequireScope_OIDC/static_token_may_write1355=== RUN TestService_RequireScope_OIDC/reader_may_read1356=== PAUSE TestService_RequireScope_OIDC/reader_may_read1357=== RUN TestService_RequireScope_OIDC/writer_implies_read1358=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1359=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1360=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read1361=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc1362=== PAUSE TestService_RequireScope_OIDC/reader_may_read_with_netrc1363=== RUN TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1364=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1365=== RUN TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read13662026/11/12 11:45:58 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)1367=== PAUSE TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1368=== RUN TestService_RequireScope_OIDC/empty_netrc_password_may_not_read13692026/11/12 11:45:58 INFO Uploading b85sxmg674mhnlin6l7yrs7vi8nkcwjm-unpinned-file.txt (128B)1370=== PAUSE TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1371=== CONT TestPush_CompleteWaitsForCleanupOfThePush13722026/11/12 11:45:58 INFO Completed push id=1 objects=6 duration=11.77437ms13732026/11/12 11:45:58 INFO Upload complete. (81ms)1374=== NAME TestClientSharedPathCommittedMidPush1375 client_integration_test.go:740: Retrieved narinfo from S3:1376 StorePath: /build/TestClientSharedPathCommittedMidPush3072981855/001/store/mpk2ysxhzr0ijw5jbsyqngwbh610knrl-shared-dep1377 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1378 Compression: zstd1379 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821380 NarSize: 1361381 References: 1382 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n13832026/11/12 11:45:58 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)13842026/11/12 11:45:58 INFO Uploading 399g33csklg36jqzfhbl8k0s38gqp5ni-test-script (136B)1385=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token1386=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token1387=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1388=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1389=== NAME TestClientSharedPathCommittedMidPush1390 client_integration_test.go:740: Retrieved narinfo from S3:1391 StorePath: /build/TestClientSharedPathCommittedMidPush3072981855/001/store/2729bsdkcf8vkcvhjn1xi2mc256dh36h-top1392 URL: nar/0b0gnvfpa0x4zv8jgzlwq4mm14wsnv0kdhq7vr8x80dyp06p3y76.nar.zst1393 Compression: zstd1394 NarHash: sha256:0b0gnvfpa0x4zv8jgzlwq4mm14wsnv0kdhq7vr8x80dyp06p3y761395 NarSize: 2241396 References: /build/TestClientSharedPathCommittedMidPush3072981855/001/store/mpk2ysxhzr0ijw5jbsyqngwbh610knrl-shared-dep1397 CA: text:sha256:02wx8qhprw6jkb4cfb5wan8idpjl2y4aay0v92n9vqigaql781ij1398=== NAME TestClientCADerivations1399 client_ca_test.go:139: Found 1 dependencies (including self)1400=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1401=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1402=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1403=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1404=== CONT TestGCTaskStore_DeduplicateSameParams1405--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)1406=== CONT TestPush_RepeatedRootCommitsOnce14072026/11/12 11:45:58 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign14082026/11/12 11:45:58 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"14092026/11/12 11:45:58 WARN Failed to register uploaded object key=b85sxmg674mhnlin6l7yrs7vi8nkcwjm.ls error="server returned 404: 404 page not found\n"14102026/11/12 11:45:58 INFO Signed narinfos id=2 count=114112026/11/12 11:45:58 INFO Uploading 1 narinfos14122026/11/12 11:45:58 WARN Failed to register uploaded object key=399g33csklg36jqzfhbl8k0s38gqp5ni.ls error="server returned 404: 404 page not found\n"14132026/11/12 11:45:58 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"14142026/11/12 11:45:58 WARN Failed to register uploaded object key=log/nr6jzc44zcc0mxhd8a14rj9qlmc80p8a-test-script.drv error="server returned 404: 404 page not found\n"14152026/11/12 11:45:58 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign14162026/11/12 11:45:58 INFO Signed narinfos id=1 count=11417--- PASS: TestClientSharedPathCommittedMidPush (0.69s)14182026/11/12 11:45:58 INFO Uploading 1 narinfos1419=== CONT TestPush_CompleteCommitsEveryRoot14202026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/2/complete14212026/11/12 11:45:58 WARN Failed to register uploaded object key=b85sxmg674mhnlin6l7yrs7vi8nkcwjm.narinfo error="server returned 404: 404 page not found\n"14222026/11/12 11:45:58 INFO Completed push id=2 objects=3 duration=3.40213ms14232026/11/12 11:45:58 INFO Upload complete. (51ms)14242026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/1/complete14252026/11/12 11:45:58 WARN Failed to register uploaded object key=399g33csklg36jqzfhbl8k0s38gqp5ni.narinfo error="server returned 404: 404 page not found\n"14262026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes14272026/11/12 11:45:58 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)14282026/11/12 11:45:58 INFO Uploading qgqfr80wh1gv1xc3synhp6fy7df32qy6-test-file-1.txt (160B)14292026/11/12 11:45:58 INFO Uploading vk8sq4cwrim12qq4l6a4ivly0h9yf50h-test-file-0.txt (160B)14302026/11/12 11:45:58 INFO Uploading l3qp3kwbf8zal4k8kq6q0psz74w79w7a-test-file-2.txt (160B)14312026/11/12 11:45:58 INFO Completed push id=1 objects=4 duration=6.8018ms14322026/11/12 11:45:58 INFO Upload complete. (69ms)14332026/11/12 11:45:58 WARN Failed to register uploaded object key=qgqfr80wh1gv1xc3synhp6fy7df32qy6.ls error="server returned 404: 404 page not found\n"14342026/11/12 11:45:58 WARN Failed to register uploaded object key=l3qp3kwbf8zal4k8kq6q0psz74w79w7a.ls error="server returned 404: 404 page not found\n"14352026/11/12 11:45:58 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"14362026/11/12 11:45:58 WARN Failed to register uploaded object key=vk8sq4cwrim12qq4l6a4ivly0h9yf50h.ls error="server returned 404: 404 page not found\n"14372026/11/12 11:45:58 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"14382026/11/12 11:45:58 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign14392026/11/12 11:45:58 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"14402026/11/12 11:45:58 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"14412026/11/12 11:45:58 INFO Signed narinfos id=1 count=314422026/11/12 11:45:58 WARN mTLS auth: bound subjects configured but subject DN unavailable14432026/11/12 11:45:58 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1444--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (0.27s)14452026/11/12 11:45:58 INFO Uploading 3 narinfos1446=== CONT TestGCTaskStore_StartNew1447--- PASS: TestGCTaskStore_StartNew (0.00s)1448=== CONT TestPush_OverlappingRootsStoreOneRowPerKey14492026/11/12 11:45:58 WARN Failed to register uploaded object key=qgqfr80wh1gv1xc3synhp6fy7df32qy6.narinfo error="server returned 404: 404 page not found\n"14502026/11/12 11:45:58 WARN Failed to register uploaded object key=vk8sq4cwrim12qq4l6a4ivly0h9yf50h.narinfo error="server returned 404: 404 page not found\n"14512026/11/12 11:45:58 INFO Aborted multipart uploads count=0 kept=014522026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/1/complete14532026/11/12 11:45:58 WARN Failed to register uploaded object key=l3qp3kwbf8zal4k8kq6q0psz74w79w7a.narinfo error="server returned 404: 404 page not found\n"14542026/11/12 11:45:58 INFO Completed push id=1 objects=9 duration=10.260329ms14552026/11/12 11:45:58 INFO Upload complete. (71ms)1456=== NAME TestClientMultipleUploads1457 client_integration_test.go:417: Uploaded 3 paths in 103.393428ms14582026/11/12 11:45:58 WARN Force mode enabled - objects will be deleted immediately without grace period14592026/11/12 11:45:58 INFO Starting cleanup of old closures method=DELETE path=/api/closures14602026/11/12 11:45:58 INFO Garbage collection started14612026/11/12 11:45:58 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=014622026/11/12 11:45:58 INFO All 1 paths already cached1463=== NAME TestClientIntegration1464 client_integration_test.go:318: Retrieved narinfo from S3:1465 StorePath: /build/TestClientIntegration2282523218/002/store/ari43awr51k3ybfk41rkla86cg4xgxsp-test-file.txt1466 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1467 Compression: zstd1468 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11469 NarSize: 1521470 References: 1471 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11472--- PASS: TestClientMultipleUploads (0.72s)1473=== CONT TestGCEndsOnShutdown14742026/11/12 11:45:58 INFO Vacuumed table table=pending_closures1475=== RUN TestGCEndsOnShutdown/before_the_run1476=== PAUSE TestGCEndsOnShutdown/before_the_run1477=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1478=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1479=== NAME TestClientIntegration1480 client_integration_test.go:319: Retrieved .ls file from S3 (compressed size: 77 bytes)1481=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1482=== NAME TestClientIntegration1483 client_integration_test.go:319: Decompressed .ls content (64 bytes):1484 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}1485=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1486=== CONT TestReadRedirectUsesPublicS3URL14872026/11/12 11:45:58 INFO Aborted multipart uploads count=0 kept=014882026/11/12 11:45:58 INFO Vacuumed table table=commit_receipts14892026/11/12 11:45:58 INFO Vacuumed table table=pending_objects14902026/11/12 11:45:58 INFO Vacuumed table table=multipart_uploads14912026/11/12 11:45:58 WARN Force mode enabled - objects will be deleted immediately without grace period14922026/11/12 11:45:58 INFO Vacuumed table table=closures14932026/11/12 11:45:58 INFO Vacuumed table table=objects1494=== NAME TestClientWithDependencies1495 client_integration_test.go:677: Skipping nix copy test - isolated store (/build/TestClientWithDependencies3485559823/001/store) requires matching store prefix1496--- PASS: TestClientWithDependencies (0.75s)1497=== CONT TestCommitRacingPendingCleanupKeepsObjects14982026/11/12 11:45:58 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=nixbld database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"1499--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (0.32s)1500=== CONT TestReadProxyRangeRequest15012026/11/12 11:45:58 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=01502--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (0.24s)1503=== CONT TestPushWaitsForSweepHoldingRow15042026/11/12 11:45:58 INFO Vacuumed table table=pending_closures15052026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes15062026/11/12 11:45:58 INFO Vacuumed table table=commit_receipts15072026/11/12 11:45:58 INFO Vacuumed table table=pending_objects15082026/11/12 11:45:58 INFO Vacuumed table table=multipart_uploads15092026/11/12 11:45:58 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)15102026/11/12 11:45:58 INFO Uploading m3x4r2zsksfacza1mpqaa7im731y1y3v-ca-test (144B)15112026/11/12 11:45:58 INFO Vacuumed table table=closures15122026/11/12 11:45:58 WARN Failed to register uploaded object key=m3x4r2zsksfacza1mpqaa7im731y1y3v.ls error="server returned 404: 404 page not found\n"15132026/11/12 11:45:58 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"15142026/11/12 11:45:58 INFO Vacuumed table table=objects15152026/11/12 11:45:58 WARN Failed to register uploaded object key=log/2mjzlfi1pivid7pjgi8p7q0942ymwh0y-ca-test.drv error="server returned 404: 404 page not found\n"15162026/11/12 11:45:58 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15172026/11/12 11:45:58 INFO Signed narinfos id=1 count=115182026/11/12 11:45:58 INFO Uploading 1 narinfos15192026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/1/complete15202026/11/12 11:45:58 WARN Failed to register uploaded object key=m3x4r2zsksfacza1mpqaa7im731y1y3v.narinfo error="server returned 404: 404 page not found\n"15212026/11/12 11:45:58 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"1522--- PASS: TestService_AuthMiddleware (0.23s)1523=== CONT TestReadRedirectKeepsNarinfoProxied15242026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes15252026/11/12 11:45:58 INFO Completed push id=1 objects=4 duration=10.81184ms15262026/11/12 11:45:58 INFO Upload complete. (114ms)15272026/11/12 11:45:58 INFO Object in database but missing from S3 key=ari43awr51k3ybfk41rkla86cg4xgxsp.ls1528=== NAME TestClientCADerivations15292026/11/12 11:45:58 WARN Found objects in DB but missing from S3, will re-upload count=11530 client_ca_test.go:180: Narinfo contains CA field: StorePath: /build/TestClientCADerivations3550462514/001/store/m3x4r2zsksfacza1mpqaa7im731y1y3v-ca-test1531 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1532 Compression: zstd1533 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1534 NarSize: 1441535 References: 1536 Deriver: /build/TestClientCADerivations3550462514/001/store/2mjzlfi1pivid7pjgi8p7q0942ymwh0y-ca-test.drv1537 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n15382026/11/12 11:45:58 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)1539 client_ca_test.go:185: Checking for realisation files in S3...1540 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1541 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache15422026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/2/complete15432026/11/12 11:45:58 WARN Failed to register uploaded object key=ari43awr51k3ybfk41rkla86cg4xgxsp.ls error="server returned 404: 404 page not found\n"15442026/11/12 11:45:58 INFO Completed push id=2 objects=3 duration=4.38101ms15452026/11/12 11:45:58 INFO Upload complete. (54ms)1546=== NAME TestClientIntegration1547 client_integration_test.go:336: Retrieved .ls file from S3 (compressed size: 62 bytes)1548 client_integration_test.go:336: Decompressed .ls content (49 bytes):1549 {"version":1,"root":{"type":"regular","size":39}}15502026/11/12 11:45:58 INFO Received complete multipart upload request method=POST path=/api/multipart/complete15512026/11/12 11:45:58 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=OThmM2QzNGUtZjU1NS00MmRiLTgzMzMtMDU2MDlhMjZjMmNkLmJmYTQ3MjljLWVhMTQtNDUwZS05MzlkLWY1ZjFjYmViOTk3OHgxNzk0NDgzOTU3Njc5NzgxNDc5 parts=121552--- PASS: TestSharedNarBothUploadsComplete (1.70s)1553=== CONT TestSweepRechecksPendingAfterLocking1554=== RUN TestPush_RejectsBadRequests/no_objects1555=== PAUSE TestPush_RejectsBadRequests/no_objects1556=== RUN TestPush_RejectsBadRequests/bad_root1557=== PAUSE TestPush_RejectsBadRequests/bad_root1558=== RUN TestPush_RejectsBadRequests/root_not_in_objects1559=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects1560=== RUN TestPush_RejectsBadRequests/no_roots1561=== PAUSE TestPush_RejectsBadRequests/no_roots1562=== CONT TestReadRedirectNar15632026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes15642026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/1/complete15652026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes15662026/11/12 11:45:58 INFO Completed push id=1 objects=4 duration=10.20332ms15672026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes15682026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes15692026/11/12 11:45:58 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)15702026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/3/complete15712026/11/12 11:45:58 WARN Failed to register uploaded object key=ari43awr51k3ybfk41rkla86cg4xgxsp.ls error="server returned 404: 404 page not found\n"15722026/11/12 11:45:58 INFO Completed push id=3 objects=3 duration=2.82254ms15732026/11/12 11:45:58 INFO Upload complete. (44ms)1574=== NAME TestClientIntegration1575 client_integration_test.go:353: Retrieved .ls file from S3 (compressed size: 62 bytes)1576 client_integration_test.go:353: Decompressed .ls content (49 bytes):1577 {"version":1,"root":{"type":"regular","size":39}}1578=== NAME TestClientCADerivations1579 client_ca_test.go:258: nix copy output: warning: you don't have Internet access; disabling some network-dependent features1580 warning: failed to create TLS context for AWS credential providers; SSO, STS WebIdentity, and ECS container authentication will be unavailable1581 error: binary cache 's3://bucket42?endpoint=http://localhost:43069®ion=eu-west-1' is for Nix stores with prefix '/nix/store', not '/build/TestClientCADerivations3550462514/001/store'1582 client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 11583=== NAME TestClientIntegration1584 client_integration_test.go:364: Testing garbage collection...15852026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes1586--- PASS: TestClientCADerivations (0.89s)1587=== CONT TestReadProxyDisabled15882026-11-12 11:45:58.823 UTC [1967] ERROR: could not obtain lock on row in relation "closures"15892026-11-12 11:45:58.823 UTC [1967] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT15902026-11-12 11:45:58.824 UTC [1967] ERROR: could not obtain lock on row in relation "closures"15912026-11-12 11:45:58.824 UTC [1967] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT15922026-11-12 11:45:58.824 UTC [1967] ERROR: could not obtain lock on row in relation "closures"15932026-11-12 11:45:58.824 UTC [1967] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT15942026-11-12 11:45:58.824 UTC [1967] ERROR: could not obtain lock on row in relation "closures"15952026-11-12 11:45:58.824 UTC [1967] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT15962026-11-12 11:45:58.825 UTC [1967] ERROR: could not obtain lock on row in relation "closures"15972026-11-12 11:45:58.825 UTC [1967] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT15982026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes1599--- PASS: TestPush_RootsAreLockedInByteOrder (0.31s)1600=== CONT TestReadProxyRootRedirectsToIndexHTML16012026/11/12 11:45:58 INFO Starting cleanup of old closures method=DELETE path=/api/closures16022026/11/12 11:45:58 INFO Garbage collection started16032026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/1/complete16042026/11/12 11:45:58 INFO Aborted multipart uploads count=0 kept=016052026/11/12 11:45:58 INFO Aborted multipart uploads count=0 kept=016062026/11/12 11:45:58 INFO Completed push id=1 objects=6 duration=10.85975ms16072026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes1608--- PASS: TestPush_CompleteCommitsEveryRoot (0.32s)1609=== CONT TestGetExistingObjectsLocksRows16102026/11/12 11:45:58 WARN Force mode enabled - objects will be deleted immediately without grace period16112026/11/12 11:45:58 WARN Force mode enabled - objects will be deleted immediately without grace period16122026/11/12 11:45:58 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=016132026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/1/complete16142026/11/12 11:45:58 INFO Received push request method=POST path=/api/pushes16152026/11/12 11:45:58 INFO Vacuumed table table=pending_closures16162026/11/12 11:45:58 INFO Vacuumed table table=commit_receipts16172026/11/12 11:45:58 INFO Completed push id=1 objects=4 duration=9.44504ms1618--- PASS: TestPush_RepeatedRootCommitsOnce (0.36s)1619=== CONT TestReadProxyConditionalGet16202026/11/12 11:45:58 INFO Vacuumed table table=pending_objects16212026/11/12 11:45:58 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=016222026/11/12 11:45:58 INFO Vacuumed table table=multipart_uploads16232026/11/12 11:45:58 INFO Vacuumed table table=pending_closures16242026/11/12 11:45:58 INFO Vacuumed table table=closures16252026/11/12 11:45:58 INFO Vacuumed table table=commit_receipts16262026/11/12 11:45:58 INFO Vacuumed table table=objects16272026/11/12 11:45:58 INFO Received complete push request method=POST path=/api/pushes/2/complete16282026/11/12 11:45:58 INFO Vacuumed table table=pending_objects16292026/11/12 11:45:58 INFO Vacuumed table table=multipart_uploads1630--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (0.35s)1631=== CONT TestRegisterCompletedObjectNeedsPendingRow16322026/11/12 11:45:58 INFO Completed push id=2 objects=4 duration=5.39863ms16332026/11/12 11:45:58 INFO Vacuumed table table=closures1634--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (0.43s)1635=== CONT TestGCSweepSkipsPendingObjects16362026/11/12 11:45:58 INFO Vacuumed table table=objects1637--- PASS: TestReadRedirectUsesPublicS3URL (0.35s)1638=== CONT TestTombstonedObjectOfferedWithoutWaiting1639--- PASS: TestReadProxyRangeRequest (0.37s)1640=== CONT TestDeduplicatedObjectsRecordedAsPending16412026/11/12 11:45:58 INFO Aborted multipart uploads count=0 kept=016422026/11/12 11:45:58 WARN Force mode enabled - objects will be deleted immediately without grace period16432026/11/12 11:45:58 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=016442026/11/12 11:45:58 INFO Received uploads request method=POST path=/api/pending_closures16452026/11/12 11:45:59 INFO Vacuumed table table=pending_closures16462026/11/12 11:45:59 INFO Vacuumed table table=commit_receipts16472026/11/12 11:45:59 INFO Vacuumed table table=pending_objects16482026/11/12 11:45:59 INFO Vacuumed table table=multipart_uploads16492026/11/12 11:45:59 INFO Vacuumed table table=closures16502026/11/12 11:45:59 INFO Vacuumed table table=objects1651--- PASS: TestReadRedirectKeepsNarinfoProxied (0.36s)1652--- PASS: TestCommitRacingPendingCleanupKeepsObjects (0.42s)1653=== CONT TestSweepSkipsRowsLockedByPush1654=== CONT TestPushDedupSurvivesConcurrentGC16552026/11/12 11:45:59 INFO Aborted multipart uploads count=0 kept=016562026/11/12 11:45:59 WARN Force mode enabled - objects will be deleted immediately without grace period1657--- PASS: TestReadRedirectNar (0.35s)1658=== CONT TestFailedS3DeleteKeepsTombstone16592026/11/12 11:45:59 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=016602026/11/12 11:45:59 INFO Vacuumed table table=pending_closures16612026/11/12 11:45:59 INFO Vacuumed table table=commit_receipts1662--- PASS: TestReadProxyDisabled (0.29s)1663=== CONT TestGCMetrics16642026/11/12 11:45:59 INFO Vacuumed table table=pending_objects16652026/11/12 11:45:59 INFO Vacuumed table table=multipart_uploads16662026/11/12 11:45:59 INFO Vacuumed table table=closures16672026/11/12 11:45:59 INFO Vacuumed table table=objects1668--- PASS: TestSweepRechecksPendingAfterLocking (0.38s)1669=== CONT TestForceGCDuringPushOffersSweptObject1670=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows1671=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows1672=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check1673=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check1674=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection16752026-11-12 11:45:59.126 UTC [1974] ERROR: Push does not exist: id=116762026-11-12 11:45:59.126 UTC [1974] CONTEXT: PL/pgSQL function commit_push(bigint) line 15 at RAISE16772026-11-12 11:45:59.126 UTC [1974] STATEMENT: SELECT commit_push($1)1678--- PASS: TestPush_CompleteWaitsForCleanupOfThePush (0.60s)1679=== CONT TestGCBugBareHashReferences1680--- PASS: TestReadProxyRootRedirectsToIndexHTML (0.30s)1681=== CONT TestGCSweepDeliversEachKeyOnce1682--- PASS: TestDeleteClosuresRacingPins (1.48s)1683=== CONT TestReadProxyNarinfo16842026/11/12 11:45:59 INFO lead: released remote=192.0.2.1:12341685--- PASS: TestLeadEndsOnShutdown (1.72s)1686=== CONT TestPresentRefreshesOnlyStaleClosures1687--- PASS: TestGetExistingObjectsLocksRows (0.31s)1688=== CONT TestReadProxyHead1689--- PASS: TestReadProxyConditionalGet (0.30s)1690=== CONT TestReadProxyInvalidPath16912026/11/12 11:45:59 INFO lead: released remote=192.0.2.1:123416922026/11/12 11:45:59 INFO Received uploads request method=POST path=/api/pending_closures1693--- PASS: TestRegisterCompletedObjectNeedsPendingRow (0.33s)1694=== CONT TestIsValidCachePath1695=== RUN TestIsValidCachePath/narinfo1696=== PAUSE TestIsValidCachePath/narinfo1697=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars1698=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars1699=== RUN TestIsValidCachePath/nar_zst1700=== PAUSE TestIsValidCachePath/nar_zst1701=== RUN TestIsValidCachePath/nar_xz1702=== PAUSE TestIsValidCachePath/nar_xz1703=== RUN TestIsValidCachePath/nar_bz21704=== PAUSE TestIsValidCachePath/nar_bz21705=== RUN TestIsValidCachePath/nar_uncompressed1706=== PAUSE TestIsValidCachePath/nar_uncompressed1707=== RUN TestIsValidCachePath/ls1708=== PAUSE TestIsValidCachePath/ls1709=== RUN TestIsValidCachePath/log1710=== PAUSE TestIsValidCachePath/log1711=== RUN TestIsValidCachePath/realisation1712=== PAUSE TestIsValidCachePath/realisation1713=== RUN TestIsValidCachePath/nix-cache-info1714=== PAUSE TestIsValidCachePath/nix-cache-info1715=== RUN TestIsValidCachePath/index.html1716=== PAUSE TestIsValidCachePath/index.html1717=== RUN TestIsValidCachePath/traversal_parent1718=== PAUSE TestIsValidCachePath/traversal_parent1719=== RUN TestIsValidCachePath/traversal_in_middle1720=== PAUSE TestIsValidCachePath/traversal_in_middle1721=== RUN TestIsValidCachePath/invalid_char_e1722=== PAUSE TestIsValidCachePath/invalid_char_e1723=== RUN TestIsValidCachePath/invalid_char_u1724=== PAUSE TestIsValidCachePath/invalid_char_u1725=== RUN TestIsValidCachePath/random_path1726=== PAUSE TestIsValidCachePath/random_path1727=== RUN TestIsValidCachePath/empty1728=== PAUSE TestIsValidCachePath/empty1729=== RUN TestIsValidCachePath/leading_slash1730=== PAUSE TestIsValidCachePath/leading_slash1731=== RUN TestIsValidCachePath/wrong_extension1732=== PAUSE TestIsValidCachePath/wrong_extension1733=== RUN TestIsValidCachePath/short_hash1734=== PAUSE TestIsValidCachePath/short_hash1735=== CONT TestReadProxy40417362026/11/12 11:45:59 INFO Aborted multipart uploads count=0 kept=017372026/11/12 11:45:59 WARN Force mode enabled - objects will be deleted immediately without grace period17382026/11/12 11:45:59 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=017392026/11/12 11:45:59 INFO lead: acquired remote=192.0.2.1:123417402026/11/12 11:45:59 INFO Vacuumed table table=pending_closures17412026/11/12 11:45:59 INFO Vacuumed table table=commit_receipts17422026/11/12 11:45:59 INFO Received uploads request method=POST path=/api/pending_closures17432026/11/12 11:45:59 INFO Vacuumed table table=pending_objects17442026/11/12 11:45:59 INFO Vacuumed table table=multipart_uploads17452026/11/12 11:45:59 INFO Vacuumed table table=closures1746--- PASS: TestTombstonedObjectOfferedWithoutWaiting (0.33s)1747=== CONT TestCreatePin_ReservedPins17482026/11/12 11:45:59 INFO Vacuumed table table=objects17492026/11/12 11:45:59 INFO Received uploads request method=POST path=/api/pending_closures1750--- PASS: TestGCSweepSkipsPendingObjects (0.36s)1751=== CONT TestParseSingleRange1752=== RUN TestParseSingleRange/none1753=== PAUSE TestParseSingleRange/none1754=== RUN TestParseSingleRange/unknown_unit1755=== PAUSE TestParseSingleRange/unknown_unit1756=== RUN TestParseSingleRange/multi-range_ignored1757=== PAUSE TestParseSingleRange/multi-range_ignored1758=== RUN TestParseSingleRange/malformed_no_dash1759=== PAUSE TestParseSingleRange/malformed_no_dash1760=== RUN TestParseSingleRange/malformed_both_empty1761=== PAUSE TestParseSingleRange/malformed_both_empty1762=== RUN TestParseSingleRange/malformed_end_before_start1763=== PAUSE TestParseSingleRange/malformed_end_before_start1764=== RUN TestParseSingleRange/closed1765=== PAUSE TestParseSingleRange/closed1766=== RUN TestParseSingleRange/open-ended1767=== PAUSE TestParseSingleRange/open-ended1768=== RUN TestParseSingleRange/end_clamped_to_size1769=== PAUSE TestParseSingleRange/end_clamped_to_size1770=== RUN TestParseSingleRange/suffix1771=== PAUSE TestParseSingleRange/suffix1772=== RUN TestParseSingleRange/suffix_exceeds_size1773=== PAUSE TestParseSingleRange/suffix_exceeds_size1774=== RUN TestParseSingleRange/single_byte1775=== PAUSE TestParseSingleRange/single_byte1776=== RUN TestParseSingleRange/start_past_EOF1777=== PAUSE TestParseSingleRange/start_past_EOF1778=== RUN TestParseSingleRange/start_far_past_EOF1779=== PAUSE TestParseSingleRange/start_far_past_EOF1780=== CONT TestReadProxyNarStreaming17812026/11/12 11:45:59 INFO lead: given back to the incumbent remote=192.0.2.1:123417822026/11/12 11:45:59 INFO Received uploads request method=POST path=/api/pending_closures17832026/11/12 11:45:59 INFO lead: acquired remote=192.0.2.1:12341784--- PASS: TestDeduplicatedObjectsRecordedAsPending (0.33s)1785=== CONT TestPresentReportsOnlyClosureRoots17862026/11/12 11:45:59 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:44109/oidc17872026/11/12 11:45:59 INFO Aborted multipart uploads count=0 kept=017882026/11/12 11:45:59 WARN Force mode enabled - objects will be deleted immediately without grace period17892026/11/12 11:45:59 INFO Aborted multipart uploads count=0 kept=017902026/11/12 11:45:59 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=017912026/11/12 11:45:59 INFO Vacuumed table table=pending_closures17922026/11/12 11:45:59 INFO Vacuumed table table=commit_receipts17932026/11/12 11:45:59 INFO Vacuumed table table=pending_objects17942026/11/12 11:45:59 INFO Vacuumed table table=multipart_uploads17952026/11/12 11:45:59 INFO Vacuumed table table=closures17962026/11/12 11:45:59 WARN Force mode enabled - objects will be deleted immediately without grace period17972026/11/12 11:45:59 INFO Vacuumed table table=objects17982026/11/12 11:45:59 INFO Received uploads request method=POST path=/api/pending_closures17992026/11/12 11:45:59 INFO Aborted multipart uploads count=0 kept=018002026/11/12 11:45:59 INFO Aborted multipart uploads count=0 kept=018012026/11/12 11:45:59 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."18022026/11/12 11:45:59 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."18032026/11/12 11:45:59 WARN Force mode enabled - objects will be deleted immediately without grace period1804--- PASS: TestFailedS3DeleteKeepsTombstone (0.28s)1805=== CONT TestPresentNotReportedWhileGCDeletesClosure1806=== RUN TestPresentNotReportedWhileGCDeletesClosure/fresh1807=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/fresh1808=== RUN TestPresentNotReportedWhileGCDeletesClosure/stale1809=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/stale1810=== CONT TestOrphanedObjectsGCStressTest18112026/11/12 11:45:59 WARN Force mode enabled - objects will be deleted immediately without grace period18122026/11/12 11:45:59 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=018132026/11/12 11:45:59 INFO Received uploads request method=POST path=/api/pending_closures18142026/11/12 11:45:59 INFO Vacuumed table table=pending_closures18152026/11/12 11:45:59 INFO Vacuumed table table=commit_receipts18162026/11/12 11:45:59 INFO Vacuumed table table=pending_objects18172026/11/12 11:45:59 INFO Vacuumed table table=multipart_uploads18182026/11/12 11:45:59 INFO Vacuumed table table=closures18192026/11/12 11:45:59 INFO Vacuumed table table=objects1820--- PASS: TestGCMetrics (0.28s)1821=== CONT TestReadProxyContentNegotiation18222026/11/12 11:45:59 INFO lead: released remote=192.0.2.1:123418232026/11/12 11:45:59 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=018242026/11/12 11:45:59 INFO Vacuumed table table=pending_closures18252026/11/12 11:45:59 INFO Vacuumed table table=commit_receipts18262026/11/12 11:45:59 INFO Vacuumed table table=pending_objects18272026/11/12 11:45:59 INFO Vacuumed table table=multipart_uploads18282026/11/12 11:45:59 INFO Vacuumed table table=closures18292026/11/12 11:45:59 INFO Vacuumed table table=objects1830--- PASS: TestSweepSkipsRowsLockedByPush (0.37s)1831=== CONT TestReadProxyNarinfoAlreadyDecompressed18322026/11/12 11:45:59 INFO Aborted multipart uploads count=0 kept=018332026/11/12 11:45:59 INFO lead: acquired remote=192.0.2.1:123418342026/11/12 11:45:59 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=018352026/11/12 11:45:59 INFO Aborted multipart uploads count=0 kept=018362026/11/12 11:45:59 WARN Force mode enabled - objects will be deleted immediately without grace period18372026/11/12 11:45:59 INFO Vacuumed table table=pending_closures18382026/11/12 11:45:59 INFO Vacuumed table table=commit_receipts18392026/11/12 11:45:59 INFO Vacuumed table table=pending_objects18402026/11/12 11:45:59 INFO Vacuumed table table=multipart_uploads18412026/11/12 11:45:59 INFO Vacuumed table table=closures18422026/11/12 11:45:59 INFO Vacuumed table table=objects18432026/11/12 11:45:59 INFO Aborted multipart uploads count=0 kept=018442026/11/12 11:45:59 WARN Force mode enabled - objects will be deleted immediately without grace period18452026/11/12 11:45:59 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=018462026/11/12 11:45:59 INFO Vacuumed table table=pending_closures18472026/11/12 11:45:59 INFO Vacuumed table table=commit_receipts18482026/11/12 11:45:59 INFO Vacuumed table table=pending_objects18492026/11/12 11:45:59 INFO Vacuumed table table=multipart_uploads1850--- PASS: TestPresentRefreshesOnlyStaleClosures (0.33s)1851=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed18522026/11/12 11:45:59 INFO Vacuumed table table=closures18532026/11/12 11:45:59 INFO Vacuumed table table=objects1854--- PASS: TestPushDedupSurvivesConcurrentGC (0.45s)1855=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed18562026/11/12 11:45:59 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=018572026/11/12 11:45:59 INFO Vacuumed table table=pending_closures18582026/11/12 11:45:59 INFO Vacuumed table table=commit_receipts18592026/11/12 11:45:59 INFO Vacuumed table table=pending_objects18602026/11/12 11:45:59 INFO Vacuumed table table=multipart_uploads18612026/11/12 11:45:59 INFO Vacuumed table table=closures18622026/11/12 11:45:59 INFO Vacuumed table table=objects1863--- PASS: TestGCSweepDeliversEachKeyOnce (0.37s)1864=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1865--- PASS: TestPushWaitsForSweepHoldingRow (0.86s)1866=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1867--- PASS: TestReadProxyHead (0.34s)1868=== CONT TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed18692026/11/12 11:45:59 ERROR Failed to decode narinfo key=3hcdxyjf9yiq7qf3i4548drb6sjmwa1v.narinfo limit=16777216 error=<nil>1870--- PASS: TestReadProxyNarinfo (0.38s)1871=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info18722026/11/12 11:45:59 INFO Received uploads request method=POST path=/1873=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key18742026/11/12 11:45:59 INFO Received request for more parts method=POST path=/1875=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal18762026/11/12 11:45:59 INFO Received uploads request method=POST path=/1877=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key18782026/11/12 11:45:59 INFO Received complete multipart upload request method=POST path=/1879--- PASS: TestUploadHandlersRejectInvalidKeys (0.09s)1880 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)1881 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1882 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s)1883 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1884=== CONT TestProxyWriteTimeout/narinfo1885=== CONT TestProxyWriteTimeout/unknown_size1886=== CONT TestProxyWriteTimeout/10_GiB_nar1887=== CONT TestProxyWriteTimeout/1_GiB_nar1888--- PASS: TestProxyWriteTimeout (0.09s)1889 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1890 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1891 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1892 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)1893=== CONT TestCommitCanBeRepeatedUntilCleanup/push1894--- PASS: TestReadProxyInvalidPath (0.34s)1895=== CONT TestCommitCanBeRepeatedUntilCleanup/pending_closure1896--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (0.48s)1897=== CONT TestPendingClosureWriteTimeout/empty1898=== CONT TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1899--- PASS: TestReadProxyNarStreaming (0.31s)1900=== CONT TestPendingClosureWriteTimeout/670k_objects1901=== CONT TestPendingClosureWriteTimeout/400_objects1902=== CONT TestServerTLSConfig/no_client_CA1903=== CONT TestServerTLSConfig/not_a_PEM_file1904=== CONT TestServerTLSConfig/missing_CA_file1905=== CONT TestIsValidUploadKey/narinfo1906--- PASS: TestServerTLSConfig (0.00s)1907 --- PASS: TestServerTLSConfig/no_client_CA (0.00s)1908 --- PASS: TestServerTLSConfig/not_a_PEM_file (0.00s)1909 --- PASS: TestServerTLSConfig/missing_CA_file (0.00s)1910=== CONT TestIsValidUploadKey/realisation_plus_in_output1911=== CONT TestIsValidUploadKey/unknown_type1912=== CONT TestIsValidUploadKey/empty_key1913=== CONT TestIsValidUploadKey/absolute1914=== CONT TestIsValidUploadKey/traversal_nar1915=== CONT TestIsValidUploadKey/traversal1916=== CONT TestIsValidUploadKey/listing_key,_narinfo_type1917=== CONT TestIsValidUploadKey/index.html1918=== CONT TestIsValidUploadKey/nix-cache-info1919=== CONT TestIsValidUploadKey/build_log_home-manager_file1920=== CONT TestIsValidUploadKey/realisation1921=== CONT TestIsValidUploadKey/build_log_equals1922=== CONT TestIsValidUploadKey/build_log_question_mark1923=== CONT TestIsValidUploadKey/build_log_plus_in_name1924=== CONT TestIsValidUploadKey/nar_key,_narinfo_type1925=== CONT TestIsValidUploadKey/nar_plain1926=== CONT TestIsValidUploadKey/build_log1927=== CONT TestIsValidUploadKey/listing1928=== CONT TestIsValidUploadKey/nar_xz1929=== CONT TestIsValidUploadKey/nar_zst1930=== CONT TestIsValidUploadKey/narinfo_key,_nar_type1931=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure1932--- PASS: TestIsValidUploadKey (0.09s)1933 --- PASS: TestIsValidUploadKey/narinfo (0.00s)1934 --- PASS: TestIsValidUploadKey/realisation_plus_in_output (0.00s)1935 --- PASS: TestIsValidUploadKey/unknown_type (0.00s)1936 --- PASS: TestIsValidUploadKey/empty_key (0.00s)1937 --- PASS: TestIsValidUploadKey/absolute (0.00s)1938 --- PASS: TestIsValidUploadKey/traversal_nar (0.00s)1939 --- PASS: TestIsValidUploadKey/traversal (0.00s)1940 --- PASS: TestIsValidUploadKey/listing_key,_narinfo_type (0.00s)1941 --- PASS: TestIsValidUploadKey/index.html (0.00s)1942 --- PASS: TestIsValidUploadKey/nix-cache-info (0.00s)1943 --- PASS: TestIsValidUploadKey/build_log_home-manager_file (0.00s)1944 --- PASS: TestIsValidUploadKey/realisation (0.00s)1945 --- PASS: TestIsValidUploadKey/build_log_equals (0.00s)1946 --- PASS: TestIsValidUploadKey/build_log_question_mark (0.00s)1947 --- PASS: TestIsValidUploadKey/build_log_plus_in_name (0.00s)1948 --- PASS: TestIsValidUploadKey/nar_key,_narinfo_type (0.00s)1949 --- PASS: TestIsValidUploadKey/nar_plain (0.00s)1950 --- PASS: TestIsValidUploadKey/build_log (0.00s)1951 --- PASS: TestIsValidUploadKey/listing (0.00s)1952 --- PASS: TestIsValidUploadKey/nar_xz (0.00s)1953 --- PASS: TestIsValidUploadKey/nar_zst (0.00s)1954 --- PASS: TestIsValidUploadKey/narinfo_key,_nar_type (0.00s)19552026/11/12 11:45:59 INFO Received uploads request method=POST path=/1956--- PASS: TestReadProxy404 (0.37s)1957=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts19582026/11/12 11:45:59 INFO Received request for more parts method=POST path=/1959--- PASS: TestGCBugBareHashReferences (0.51s)1960=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart19612026/11/12 11:45:59 INFO Received complete multipart upload request method=POST path=/1962--- PASS: TestPresentReportsOnlyClosureRoots (0.32s)1963=== CONT TestResolveDBConnectionString/flag_wins1964=== CONT TestResolveDBConnectionString/PGHOST_allows_empty1965=== CONT TestResolveDBConnectionString/nothing_configured1966=== CONT TestResolveDBConnectionString/missing_file_is_an_error1967=== CONT TestResolveDBConnectionString/file_when_flag_empty1968=== CONT TestClientErrorHandling/InvalidStorePath1969--- PASS: TestResolveDBConnectionString (0.00s)1970 --- PASS: TestResolveDBConnectionString/flag_wins (0.00s)1971 --- PASS: TestResolveDBConnectionString/PGHOST_allows_empty (0.00s)1972 --- PASS: TestResolveDBConnectionString/nothing_configured (0.00s)1973 --- PASS: TestResolveDBConnectionString/missing_file_is_an_error (0.00s)1974 --- PASS: TestResolveDBConnectionString/file_when_flag_empty (0.00s)19752026/11/12 11:45:59 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux19762026/11/12 11:45:59 WARN Refused reserved pin name=worker-x86_64-linux19772026/11/12 11:45:59 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux19782026/11/12 11:45:59 INFO Received create pin request method=POST path=/api/pins/my-app19792026/11/12 11:45:59 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux1980--- PASS: TestCreatePin_ReservedPins (0.38s)1981=== CONT TestClientErrorHandling/ServerNotAvailable1982=== CONT TestClientErrorHandling/InvalidAuthToken1983=== CONT TestCacheConfigHandler/full_config,_no_issuer1984=== CONT TestCacheConfigHandler/no_signing_keys1985=== CONT TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1986=== CONT TestCacheConfigHandler/no_cache_url_configured1987--- PASS: TestCacheConfigHandler (0.00s)1988 --- PASS: TestCacheConfigHandler/full_config,_no_issuer (0.00s)1989 --- PASS: TestCacheConfigHandler/no_signing_keys (0.00s)1990 --- PASS: TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator (0.00s)1991 --- PASS: TestCacheConfigHandler/no_cache_url_configured (0.00s)1992=== CONT TestService_RequireScope_OIDC/builder_may_write1993=== CONT TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1994=== CONT TestService_RequireScope_OIDC/reader_may_read1995=== CONT TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1996=== CONT TestService_RequireScope_OIDC/reader_may_read_with_netrc1997=== CONT TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read19982026/11/12 11:45:59 WARN Authentication failed token_preview=wrong token_length=5 oidc_error="not a JWT" oidc_provider=""1999=== CONT TestService_RequireScope_OIDC/anonymous_may_not_read2000=== CONT TestService_RequireScope_OIDC/writer_implies_read
loading 865 hidden lines…
28662026/11/12 11:46:13 INFO Received push request method=POST path=/28672026/11/12 11:46:13 INFO Received uploads request method=POST path=/28682026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=028692026/11/12 11:46:13 INFO Vacuumed table table=pending_closures28702026/11/12 11:46:13 INFO Received push request method=POST path=/28712026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts28722026/11/12 11:46:13 INFO Vacuumed table table=pending_objects28732026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads28742026/11/12 11:46:13 INFO Vacuumed table table=closures28752026/11/12 11:46:13 INFO Vacuumed table table=objects28762026/11/12 11:46:13 INFO Received uploads request method=POST path=/28772026/11/12 11:46:13 INFO Received complete upload request method=POST path=/28782026/11/12 11:46:13 INFO Received push request method=POST path=/28792026/11/12 11:46:13 INFO Received uploads request method=POST path=/28802026/11/12 11:46:13 INFO Received push request method=POST path=/28812026/11/12 11:46:13 INFO Completed upload id=6 objects=7 duration=3.904129ms28822026/11/12 11:46:13 INFO Received complete push request method=POST path=/28832026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=028842026/11/12 11:46:13 INFO Received push request method=POST path=/28852026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period28862026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=028872026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028882026/11/12 11:46:13 INFO Received uploads request method=POST path=/28892026/11/12 11:46:13 INFO Vacuumed table table=pending_closures28902026/11/12 11:46:13 INFO Completed push id=3 objects=22 duration=7.443919ms28912026/11/12 11:46:13 INFO Vacuumed table table=pending_closures28922026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts28932026/11/12 11:46:13 INFO Received push request method=POST path=/28942026/11/12 11:46:13 INFO Vacuumed table table=pending_objects28952026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts28962026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads28972026/11/12 11:46:13 INFO Vacuumed table table=closures28982026/11/12 11:46:13 INFO Vacuumed table table=pending_objects28992026/11/12 11:46:13 INFO Received push request method=POST path=/29002026/11/12 11:46:13 INFO Vacuumed table table=objects29012026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads29022026/11/12 11:46:13 INFO Vacuumed table table=closures29032026/11/12 11:46:13 INFO Received push request method=POST path=/29042026/11/12 11:46:13 INFO Vacuumed table table=objects29052026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=029062026/11/12 11:46:13 INFO Received uploads request method=POST path=/29072026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029082026/11/12 11:46:13 INFO Received push request method=POST path=/29092026/11/12 11:46:13 INFO Received push request method=POST path=/29102026/11/12 11:46:13 INFO Received uploads request method=POST path=/29112026/11/12 11:46:13 INFO Received uploads request method=POST path=/29122026/11/12 11:46:13 INFO Received push request method=POST path=/29132026/11/12 11:46:13 INFO Received uploads request method=POST path=/29142026/11/12 11:46:13 INFO Received push request method=POST path=/29152026/11/12 11:46:13 INFO Received push request method=POST path=/29162026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=029172026/11/12 11:46:13 INFO Vacuumed table table=pending_closures29182026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts29192026/11/12 11:46:13 INFO Vacuumed table table=pending_objects29202026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads29212026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period29222026/11/12 11:46:13 INFO Received complete upload request method=POST path=/29232026/11/12 11:46:13 INFO Vacuumed table table=closures29242026/11/12 11:46:13 INFO Vacuumed table table=objects29252026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029262026/11/12 11:46:13 INFO Completed upload id=5 objects=7 duration=4.3939ms29272026/11/12 11:46:13 INFO Vacuumed table table=pending_closures29282026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts29292026/11/12 11:46:13 INFO Vacuumed table table=pending_objects29302026/11/12 11:46:13 INFO Received complete push request method=POST path=/29312026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads29322026/11/12 11:46:13 INFO Received uploads request method=POST path=/29332026/11/12 11:46:13 INFO Received uploads request method=POST path=/29342026/11/12 11:46:13 INFO Vacuumed table table=closures29352026/11/12 11:46:13 INFO Vacuumed table table=objects29362026/11/12 11:46:13 INFO Completed push id=2 objects=7 duration=5.016179ms29372026/11/12 11:46:13 INFO Received complete upload request method=POST path=/29382026/11/12 11:46:13 INFO Received push request method=POST path=/29392026/11/12 11:46:13 INFO Received complete push request method=POST path=/29402026/11/12 11:46:13 INFO Completed upload id=10 objects=23 duration=2.20787ms29412026/11/12 11:46:13 INFO Received uploads request method=POST path=/29422026/11/12 11:46:13 INFO Received push request method=POST path=/29432026/11/12 11:46:13 INFO Completed push id=14 objects=3 duration=4.08959ms29442026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=029452026/11/12 11:46:13 INFO Received uploads request method=POST path=/29462026/11/12 11:46:13 INFO Received complete push request method=POST path=/29472026/11/12 11:46:13 INFO Completed push id=4 objects=4 duration=3.66181ms29482026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=029492026/11/12 11:46:13 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."29502026/11/12 11:46:13 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."29512026/11/12 11:46:13 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."29522026/11/12 11:46:13 INFO Received complete push request method=POST path=/29532026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029542026/11/12 11:46:13 INFO Received complete upload request method=POST path=/29552026/11/12 11:46:13 INFO Vacuumed table table=pending_closures29562026/11/12 11:46:13 INFO Completed upload id=6 objects=9 duration=1.79004ms29572026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts29582026/11/12 11:46:13 INFO Completed push id=6 objects=5 duration=4.26438ms29592026/11/12 11:46:13 INFO Received uploads request method=POST path=/29602026/11/12 11:46:13 INFO Vacuumed table table=pending_objects29612026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads29622026/11/12 11:46:13 INFO Vacuumed table table=closures29632026/11/12 11:46:13 INFO Received uploads request method=POST path=/29642026/11/12 11:46:13 INFO Vacuumed table table=objects29652026/11/12 11:46:13 INFO Received push request method=POST path=/29662026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=029672026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=029682026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period29692026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=029702026/11/12 11:46:13 INFO Received push request method=POST path=/29712026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period29722026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029732026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=029742026/11/12 11:46:13 INFO Vacuumed table table=pending_closures29752026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts29762026/11/12 11:46:13 INFO Vacuumed table table=pending_closures29772026/11/12 11:46:13 INFO Received uploads request method=POST path=/29782026/11/12 11:46:13 INFO Vacuumed table table=pending_objects29792026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads29802026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts29812026/11/12 11:46:13 INFO Received uploads request method=POST path=/29822026/11/12 11:46:13 INFO Received uploads request method=POST path=/29832026/11/12 11:46:13 INFO Received push request method=POST path=/29842026/11/12 11:46:13 INFO Vacuumed table table=closures29852026/11/12 11:46:13 INFO Vacuumed table table=pending_objects29862026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads29872026/11/12 11:46:13 INFO Vacuumed table table=objects29882026/11/12 11:46:13 INFO Received complete push request method=POST path=/29892026/11/12 11:46:13 INFO Vacuumed table table=closures29902026/11/12 11:46:13 INFO Received complete push request method=POST path=/29912026/11/12 11:46:13 INFO Completed push id=5 objects=6 duration=2.2796ms29922026/11/12 11:46:13 INFO Vacuumed table table=objects29932026/11/12 11:46:13 INFO Received uploads request method=POST path=/29942026/11/12 11:46:13 INFO Completed push id=6 objects=6 duration=2.27127ms29952026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=029962026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period29972026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=029982026/11/12 11:46:13 INFO Received push request method=POST path=/29992026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=030002026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period30012026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030022026/11/12 11:46:13 INFO Received uploads request method=POST path=/30032026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period30042026/11/12 11:46:13 INFO Vacuumed table table=pending_closures30052026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030062026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts30072026/11/12 11:46:13 INFO Received uploads request method=POST path=/30082026/11/12 11:46:13 INFO Vacuumed table table=pending_closures30092026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=030102026/11/12 11:46:13 INFO Vacuumed table table=pending_objects30112026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads30122026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts30132026/11/12 11:46:13 INFO Vacuumed table table=closures30142026/11/12 11:46:13 INFO Received push request method=POST path=/30152026/11/12 11:46:13 INFO Vacuumed table table=pending_objects30162026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads30172026/11/12 11:46:13 INFO Vacuumed table table=objects30182026/11/12 11:46:13 INFO Received push request method=POST path=/30192026/11/12 11:46:13 INFO Vacuumed table table=closures30202026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period30212026/11/12 11:46:13 INFO Received complete upload request method=POST path=/30222026/11/12 11:46:13 INFO Vacuumed table table=objects30232026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=4 objects-failed-to-delete=030242026/11/12 11:46:13 INFO Completed upload id=11 objects=6 duration=1.58309ms30252026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030262026/11/12 11:46:13 INFO Vacuumed table table=pending_closures30272026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts30282026/11/12 11:46:13 INFO Vacuumed table table=pending_closures30292026/11/12 11:46:13 INFO Received uploads request method=POST path=/30302026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts30312026/11/12 11:46:13 INFO Vacuumed table table=pending_objects30322026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads30332026/11/12 11:46:13 INFO Vacuumed table table=closures30342026/11/12 11:46:13 INFO Received complete upload request method=POST path=/30352026/11/12 11:46:13 INFO Vacuumed table table=pending_objects30362026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads30372026/11/12 11:46:13 INFO Vacuumed table table=objects30382026/11/12 11:46:13 INFO Vacuumed table table=closures30392026/11/12 11:46:13 INFO Completed upload id=2 objects=2 duration=1.89629ms30402026/11/12 11:46:13 INFO Vacuumed table table=objects30412026/11/12 11:46:13 INFO Received complete push request method=POST path=/30422026/11/12 11:46:13 INFO Completed push id=10 objects=2 duration=2.52104ms30432026/11/12 11:46:13 INFO Received complete upload request method=POST path=/30442026/11/12 11:46:13 INFO Received complete upload request method=POST path=/30452026/11/12 11:46:13 INFO Completed upload id=3 objects=5 duration=1.64394ms30462026/11/12 11:46:13 INFO Completed upload id=3 objects=24 duration=2.19038ms30472026/11/12 11:46:13 INFO Received complete upload request method=POST path=/30482026/11/12 11:46:13 INFO Completed upload id=10 objects=6 duration=2.94979ms30492026/11/12 11:46:13 INFO Received push request method=POST path=/30502026/11/12 11:46:13 INFO Received push request method=POST path=/30512026/11/12 11:46:13 INFO Received push request method=POST path=/30522026/11/12 11:46:13 INFO Received push request method=POST path=/30532026/11/12 11:46:13 INFO Received complete push request method=POST path=/30542026/11/12 11:46:13 INFO Completed push id=6 objects=5 duration=2.20194ms30552026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=030562026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period30572026/11/12 11:46:13 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."30582026/11/12 11:46:13 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."30592026/11/12 11:46:13 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."30602026/11/12 11:46:13 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."30612026/11/12 11:46:13 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."30622026/11/12 11:46:13 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."30632026/11/12 11:46:13 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."30642026/11/12 11:46:13 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."30652026/11/12 11:46:13 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."30662026/11/12 11:46:13 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."30672026/11/12 11:46:13 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."30682026/11/12 11:46:13 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."30692026/11/12 11:46:13 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."30702026/11/12 11:46:13 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."30712026/11/12 11:46:13 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."30722026/11/12 11:46:13 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."30732026/11/12 11:46:13 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."30742026/11/12 11:46:13 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."30752026/11/12 11:46:13 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."30762026/11/12 11:46:13 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."30772026/11/12 11:46:13 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."30782026/11/12 11:46:13 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."30792026/11/12 11:46:13 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."30802026/11/12 11:46:13 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."30812026/11/12 11:46:13 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."30822026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030832026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=030842026/11/12 11:46:13 INFO Vacuumed table table=pending_closures30852026/11/12 11:46:13 INFO Received push request method=POST path=/30862026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts30872026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period30882026/11/12 11:46:13 INFO Vacuumed table table=pending_objects30892026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads30902026/11/12 11:46:13 INFO Vacuumed table table=closures30912026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=030922026/11/12 11:46:13 INFO Vacuumed table table=objects30932026/11/12 11:46:13 INFO Received complete upload request method=POST path=/30942026/11/12 11:46:13 INFO Received uploads request method=POST path=/30952026/11/12 11:46:13 INFO Completed upload id=12 objects=21 duration=1.61732ms30962026/11/12 11:46:13 INFO Received complete push request method=POST path=/30972026/11/12 11:46:13 INFO Completed push id=7 objects=27 duration=2.61735ms30982026/11/12 11:46:13 INFO Received push request method=POST path=/30992026/11/12 11:46:13 INFO Received uploads request method=POST path=/31002026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=031012026/11/12 11:46:13 INFO Received complete push request method=POST path=/31022026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=031032026/11/12 11:46:13 INFO Vacuumed table table=pending_closures31042026/11/12 11:46:13 INFO Completed push id=12 objects=15 duration=2.51956ms31052026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts31062026/11/12 11:46:13 INFO Vacuumed table table=pending_objects31072026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031082026/11/12 11:46:13 INFO Received push request method=POST path=/31092026/11/12 11:46:13 INFO Received uploads request method=POST path=/31102026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads31112026/11/12 11:46:13 INFO Received complete push request method=POST path=/31122026/11/12 11:46:13 INFO Vacuumed table table=pending_closures31132026/11/12 11:46:13 INFO Vacuumed table table=closures31142026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts31152026/11/12 11:46:13 INFO Vacuumed table table=objects31162026/11/12 11:46:13 INFO Vacuumed table table=pending_objects31172026/11/12 11:46:13 INFO Received uploads request method=POST path=/31182026/11/12 11:46:13 INFO Received push request method=POST path=/31192026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads31202026/11/12 11:46:13 INFO Vacuumed table table=closures31212026/11/12 11:46:13 INFO Completed push id=11 objects=11 duration=4.50849ms31222026/11/12 11:46:13 INFO Vacuumed table table=objects31232026/11/12 11:46:13 INFO Received push request method=POST path=/31242026/11/12 11:46:13 INFO Received push request method=POST path=/31252026/11/12 11:46:13 INFO Received complete upload request method=POST path=/31262026/11/12 11:46:13 INFO Received uploads request method=POST path=/31272026/11/12 11:46:13 INFO Completed upload id=3 objects=9 duration=1.450969ms31282026/11/12 11:46:13 INFO Received uploads request method=POST path=/31292026/11/12 11:46:13 INFO Received uploads request method=POST path=/31302026/11/12 11:46:13 INFO Received push request method=POST path=/31312026/11/12 11:46:13 INFO Received push request method=POST path=/31322026/11/12 11:46:13 INFO Received push request method=POST path=/31332026/11/12 11:46:13 INFO Received complete push request method=POST path=/31342026/11/12 11:46:13 INFO Received complete upload request method=POST path=/31352026/11/12 11:46:13 INFO Completed push id=13 objects=23 duration=2.24024ms31362026/11/12 11:46:13 INFO Completed upload id=4 objects=14 duration=3.46266ms31372026/11/12 11:46:13 INFO Received uploads request method=POST path=/31382026/11/12 11:46:13 INFO Received uploads request method=POST path=/31392026/11/12 11:46:13 INFO Received push request method=POST path=/31402026/11/12 11:46:13 INFO Received complete upload request method=POST path=/31412026/11/12 11:46:13 INFO Received complete push request method=POST path=/31422026/11/12 11:46:13 INFO Received complete push request method=POST path=/31432026/11/12 11:46:13 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."31442026/11/12 11:46:13 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."31452026/11/12 11:46:13 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."31462026/11/12 11:46:13 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."31472026/11/12 11:46:13 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."31482026/11/12 11:46:13 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."31492026/11/12 11:46:13 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."31502026/11/12 11:46:13 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."31512026/11/12 11:46:13 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."31522026/11/12 11:46:13 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."31532026/11/12 11:46:13 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."31542026/11/12 11:46:13 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."31552026/11/12 11:46:13 INFO Completed push id=7 objects=20 duration=3.25479ms31562026/11/12 11:46:13 INFO Completed upload id=6 objects=24 duration=4.00114ms31572026/11/12 11:46:13 INFO Completed push id=2 objects=10 duration=4.25463ms31582026/11/12 11:46:13 INFO Received push request method=POST path=/31592026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=031602026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=031612026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period31622026/11/12 11:46:13 INFO Received uploads request method=POST path=/31632026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031642026/11/12 11:46:13 INFO Received push request method=POST path=/31652026/11/12 11:46:13 INFO Received uploads request method=POST path=/31662026/11/12 11:46:13 INFO Received push request method=POST path=/31672026/11/12 11:46:13 INFO Vacuumed table table=pending_closures31682026/11/12 11:46:13 INFO Received complete push request method=POST path=/31692026/11/12 11:46:13 INFO Received push request method=POST path=/31702026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts31712026/11/12 11:46:13 INFO Received complete push request method=POST path=/31722026/11/12 11:46:13 INFO Completed push id=4 objects=6 duration=2.76607ms31732026/11/12 11:46:13 INFO Vacuumed table table=pending_objects31742026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads31752026/11/12 11:46:13 INFO Completed push id=5 objects=6 duration=2.18434ms31762026/11/12 11:46:13 INFO Vacuumed table table=closures31772026/11/12 11:46:13 INFO Vacuumed table table=objects31782026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=031792026/11/12 11:46:13 INFO Vacuumed table table=pending_closures31802026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts31812026/11/12 11:46:13 INFO Vacuumed table table=pending_objects31822026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads31832026/11/12 11:46:13 INFO Vacuumed table table=closures31842026/11/12 11:46:13 INFO Vacuumed table table=objects31852026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=031862026/11/12 11:46:13 INFO Received push request method=POST path=/31872026/11/12 11:46:13 INFO Received uploads request method=POST path=/31882026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period31892026/11/12 11:46:13 INFO Received push request method=POST path=/31902026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=031912026/11/12 11:46:13 INFO Received complete upload request method=POST path=/31922026/11/12 11:46:13 INFO Received push request method=POST path=/31932026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=031942026/11/12 11:46:13 INFO Completed upload id=10 objects=6 duration=3.12673ms31952026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031962026/11/12 11:46:13 INFO Vacuumed table table=pending_closures31972026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts31982026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=031992026/11/12 11:46:13 INFO Received push request method=POST path=/32002026/11/12 11:46:13 INFO Received uploads request method=POST path=/32012026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=032022026/11/12 11:46:13 INFO Vacuumed table table=pending_objects32032026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads32042026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period32052026/11/12 11:46:13 INFO Received complete push request method=POST path=/32062026/11/12 11:46:13 INFO Vacuumed table table=closures32072026/11/12 11:46:13 INFO Vacuumed table table=pending_closures32082026/11/12 11:46:13 INFO Received complete push request method=POST path=/32092026/11/12 11:46:13 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."32102026/11/12 11:46:13 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."32112026/11/12 11:46:13 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."32122026/11/12 11:46:13 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."32132026/11/12 11:46:13 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."32142026/11/12 11:46:13 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."32152026/11/12 11:46:13 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."32162026/11/12 11:46:13 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."32172026/11/12 11:46:13 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."32182026/11/12 11:46:13 INFO Vacuumed table table=objects32192026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts32202026/11/12 11:46:13 INFO Received complete upload request method=POST path=/32212026/11/12 11:46:13 INFO Completed push id=8 objects=6 duration=2.50042ms32222026/11/12 11:46:13 INFO Vacuumed table table=pending_objects32232026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads32242026/11/12 11:46:13 INFO Completed push id=6 objects=26 duration=5.07167ms32252026/11/12 11:46:13 INFO Completed upload id=11 objects=6 duration=2.39427ms32262026/11/12 11:46:13 INFO Vacuumed table table=closures32272026/11/12 11:46:13 INFO Vacuumed table table=objects32282026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=032292026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period32302026/11/12 11:46:13 INFO Received push request method=POST path=/32312026/11/12 11:46:13 INFO Received push request method=POST path=/32322026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032332026/11/12 11:46:13 INFO Received complete push request method=POST path=/32342026/11/12 11:46:13 INFO Vacuumed table table=pending_closures32352026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts32362026/11/12 11:46:13 INFO Received push request method=POST path=/32372026/11/12 11:46:13 INFO Completed push id=8 objects=7 duration=2.1951ms32382026/11/12 11:46:13 INFO Vacuumed table table=pending_objects32392026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads32402026/11/12 11:46:13 INFO Vacuumed table table=closures32412026/11/12 11:46:13 INFO Vacuumed table table=objects32422026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=032432026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period32442026/11/12 11:46:13 INFO Received uploads request method=POST path=/32452026/11/12 11:46:13 INFO Received complete upload request method=POST path=/32462026/11/12 11:46:13 INFO Received push request method=POST path=/32472026/11/12 11:46:13 INFO Completed upload id=9 objects=2 duration=1.53924ms32482026/11/12 11:46:13 INFO Received push request method=POST path=/32492026/11/12 11:46:13 INFO Received complete push request method=POST path=/32502026/11/12 11:46:13 INFO Received push request method=POST path=/32512026/11/12 11:46:13 INFO Received complete upload request method=POST path=/32522026/11/12 11:46:13 INFO Received uploads request method=POST path=/32532026/11/12 11:46:13 INFO Completed upload id=9 objects=3 duration=1.99436ms32542026/11/12 11:46:13 INFO Completed push id=5 objects=3 duration=4.26564ms32552026/11/12 11:46:13 INFO Received uploads request method=POST path=/32562026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=032572026/11/12 11:46:13 INFO Received uploads request method=POST path=/32582026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=9 objects-failed-to-delete=032592026/11/12 11:46:13 INFO Vacuumed table table=pending_closures32602026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032612026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts32622026/11/12 11:46:13 INFO Vacuumed table table=pending_closures32632026/11/12 11:46:13 INFO Vacuumed table table=pending_objects32642026/11/12 11:46:13 INFO Received uploads request method=POST path=/32652026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts32662026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads32672026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=032682026/11/12 11:46:13 INFO Vacuumed table table=closures32692026/11/12 11:46:13 INFO Vacuumed table table=pending_objects32702026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads32712026/11/12 11:46:13 INFO Received complete push request method=POST path=/32722026/11/12 11:46:13 INFO Vacuumed table table=objects32732026/11/12 11:46:13 INFO Vacuumed table table=closures32742026/11/12 11:46:13 INFO Vacuumed table table=objects32752026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=032762026/11/12 11:46:13 INFO Completed push id=8 objects=3 duration=2.26129ms32772026/11/12 11:46:13 INFO Received push request method=POST path=/32782026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032792026/11/12 11:46:13 INFO Received uploads request method=POST path=/32802026/11/12 11:46:13 INFO Vacuumed table table=pending_closures32812026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=032822026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts32832026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=032842026/11/12 11:46:13 INFO Received push request method=POST path=/32852026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period32862026/11/12 11:46:13 INFO Vacuumed table table=pending_objects32872026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads32882026/11/12 11:46:13 INFO Vacuumed table table=pending_closures32892026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032902026/11/12 11:46:13 INFO Vacuumed table table=closures32912026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts32922026/11/12 11:46:13 INFO Vacuumed table table=pending_closures32932026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts32942026/11/12 11:46:13 INFO Vacuumed table table=pending_objects32952026/11/12 11:46:13 INFO Vacuumed table table=objects32962026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads32972026/11/12 11:46:13 INFO Vacuumed table table=pending_objects32982026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads32992026/11/12 11:46:13 INFO Vacuumed table table=closures33002026/11/12 11:46:13 INFO Vacuumed table table=closures33012026/11/12 11:46:13 INFO Vacuumed table table=objects33022026/11/12 11:46:13 INFO Received push request method=POST path=/33032026/11/12 11:46:13 INFO Vacuumed table table=objects33042026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=033052026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period33062026/11/12 11:46:13 INFO Received uploads request method=POST path=/33072026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033082026/11/12 11:46:13 INFO Vacuumed table table=pending_closures33092026/11/12 11:46:13 INFO Received push request method=POST path=/33102026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts33112026/11/12 11:46:13 INFO Vacuumed table table=pending_objects33122026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads33132026/11/12 11:46:13 INFO Vacuumed table table=closures33142026/11/12 11:46:13 INFO Received complete push request method=POST path=/33152026/11/12 11:46:13 INFO Vacuumed table table=objects33162026/11/12 11:46:13 INFO Received complete upload request method=POST path=/33172026/11/12 11:46:13 INFO Received complete push request method=POST path=/33182026/11/12 11:46:13 INFO Completed upload id=9 objects=16 duration=1.80354ms33192026/11/12 11:46:13 INFO Completed push id=8 objects=6 duration=2.93684ms33202026/11/12 11:46:13 INFO Completed push id=5 objects=8 duration=3.62614ms33212026/11/12 11:46:13 INFO Received push request method=POST path=/33222026/11/12 11:46:13 INFO Received push request method=POST path=/33232026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=033242026/11/12 11:46:13 INFO Received uploads request method=POST path=/33252026/11/12 11:46:13 INFO Received uploads request method=POST path=/33262026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=033272026/11/12 11:46:13 INFO Vacuumed table table=pending_closures33282026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts33292026/11/12 11:46:13 INFO Received uploads request method=POST path=/33302026/11/12 11:46:13 INFO Vacuumed table table=pending_objects33312026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads33322026/11/12 11:46:13 INFO Vacuumed table table=closures33332026/11/12 11:46:13 INFO Vacuumed table table=objects33342026/11/12 11:46:13 INFO Received uploads request method=POST path=/33352026/11/12 11:46:13 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."33362026/11/12 11:46:13 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."33372026/11/12 11:46:13 INFO Received complete push request method=POST path=/33382026/11/12 11:46:13 INFO Received push request method=POST path=/33392026/11/12 11:46:13 INFO Completed push id=9 objects=5 duration=2.28265ms33402026/11/12 11:46:13 INFO Received push request method=POST path=/33412026/11/12 11:46:13 INFO Received push request method=POST path=/33422026/11/12 11:46:13 INFO Received push request method=POST path=/33432026/11/12 11:46:13 INFO Received push request method=POST path=/33442026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=033452026/11/12 11:46:13 INFO Received complete push request method=POST path=/33462026/11/12 11:46:13 INFO Received uploads request method=POST path=/33472026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period33482026/11/12 11:46:13 INFO Completed push id=12 objects=7 duration=2.46303ms33492026/11/12 11:46:13 INFO Received complete upload request method=POST path=/33502026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033512026/11/12 11:46:13 INFO Received uploads request method=POST path=/33522026/11/12 11:46:13 INFO Completed upload id=10 objects=2 duration=2.96072ms33532026/11/12 11:46:13 INFO Vacuumed table table=pending_closures33542026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts33552026/11/12 11:46:13 INFO Vacuumed table table=pending_objects33562026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads33572026/11/12 11:46:13 INFO Vacuumed table table=closures33582026/11/12 11:46:13 INFO Vacuumed table table=objects33592026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=033602026/11/12 11:46:13 INFO Received complete upload request method=POST path=/33612026/11/12 11:46:13 INFO Received uploads request method=POST path=/33622026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period33632026/11/12 11:46:13 INFO Completed upload id=13 objects=10 duration=1.42451ms33642026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033652026/11/12 11:46:13 INFO Received uploads request method=POST path=/33662026/11/12 11:46:13 INFO Vacuumed table table=pending_closures33672026/11/12 11:46:13 INFO Received push request method=POST path=/33682026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts33692026/11/12 11:46:13 INFO Vacuumed table table=pending_objects33702026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads33712026/11/12 11:46:13 INFO Received complete push request method=POST path=/33722026/11/12 11:46:13 INFO Received push request method=POST path=/33732026/11/12 11:46:13 INFO Vacuumed table table=closures33742026/11/12 11:46:13 INFO Completed push id=8 objects=6 duration=1.13976ms33752026/11/12 11:46:13 INFO Vacuumed table table=objects33762026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=033772026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=033782026/11/12 11:46:13 INFO Received uploads request method=POST path=/33792026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=033802026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period33812026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033822026/11/12 11:46:13 INFO Vacuumed table table=pending_closures33832026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033842026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts33852026/11/12 11:46:13 INFO Received complete upload request method=POST path=/33862026/11/12 11:46:13 INFO Vacuumed table table=pending_closures33872026/11/12 11:46:13 INFO Vacuumed table table=pending_objects33882026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts33892026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads33902026/11/12 11:46:13 INFO Vacuumed table table=closures33912026/11/12 11:46:13 INFO Vacuumed table table=pending_objects33922026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads33932026/11/12 11:46:13 INFO Completed upload id=15 objects=4 duration=3.64491ms33942026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=033952026/11/12 11:46:13 INFO Vacuumed table table=objects33962026/11/12 11:46:13 INFO Vacuumed table table=closures33972026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period33982026/11/12 11:46:13 INFO Vacuumed table table=objects33992026/11/12 11:46:13 INFO Received uploads request method=POST path=/34002026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=034012026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=034022026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=034032026/11/12 11:46:13 INFO Received complete push request method=POST path=/34042026/11/12 11:46:13 INFO Received push request method=POST path=/34052026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period34062026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034072026/11/12 11:46:13 INFO Completed push id=10 objects=3 duration=1.88382ms34082026/11/12 11:46:13 INFO Vacuumed table table=pending_closures34092026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts34102026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=034112026/11/12 11:46:13 INFO Vacuumed table table=pending_objects34122026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads34132026/11/12 11:46:13 INFO Vacuumed table table=pending_closures34142026/11/12 11:46:13 INFO Vacuumed table table=closures34152026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts34162026/11/12 11:46:13 INFO Received push request method=POST path=/34172026/11/12 11:46:13 INFO Vacuumed table table=objects34182026/11/12 11:46:13 INFO Vacuumed table table=pending_objects34192026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads34202026/11/12 11:46:13 INFO Vacuumed table table=closures34212026/11/12 11:46:13 INFO Vacuumed table table=objects34222026/11/12 11:46:13 INFO Received push request method=POST path=/34232026/11/12 11:46:13 INFO Received complete upload request method=POST path=/34242026/11/12 11:46:13 INFO Completed upload id=15 objects=6 duration=1.281ms34252026/11/12 11:46:13 INFO Received push request method=POST path=/34262026/11/12 11:46:13 INFO Received complete push request method=POST path=/34272026/11/12 11:46:13 INFO Received push request method=POST path=/34282026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=034292026/11/12 11:46:13 INFO Completed push id=12 objects=3 duration=2.36871ms34302026/11/12 11:46:13 INFO Vacuumed table table=pending_closures34312026/11/12 11:46:13 INFO Received push request method=POST path=/34322026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts34332026/11/12 11:46:13 INFO Vacuumed table table=pending_objects34342026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads34352026/11/12 11:46:13 INFO Received uploads request method=POST path=/34362026/11/12 11:46:13 INFO Vacuumed table table=closures34372026/11/12 11:46:13 INFO Vacuumed table table=objects34382026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=034392026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034402026/11/12 11:46:13 INFO Received complete push request method=POST path=/34412026/11/12 11:46:13 INFO Vacuumed table table=pending_closures34422026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts34432026/11/12 11:46:13 INFO Vacuumed table table=pending_objects34442026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads34452026/11/12 11:46:13 INFO Vacuumed table table=closures34462026/11/12 11:46:13 INFO Vacuumed table table=objects34472026/11/12 11:46:13 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."34482026/11/12 11:46:13 INFO Completed push id=3 objects=21 duration=5.11515ms34492026/11/12 11:46:13 INFO Received uploads request method=POST path=/34502026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=034512026/11/12 11:46:13 INFO Received uploads request method=POST path=/34522026/11/12 11:46:13 INFO Received uploads request method=POST path=/34532026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034542026/11/12 11:46:13 INFO Vacuumed table table=pending_closures34552026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts34562026/11/12 11:46:13 INFO Received push request method=POST path=/34572026/11/12 11:46:13 INFO Vacuumed table table=pending_objects34582026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads34592026/11/12 11:46:13 INFO Vacuumed table table=closures34602026/11/12 11:46:13 INFO Vacuumed table table=objects34612026/11/12 11:46:13 INFO Received complete push request method=POST path=/34622026/11/12 11:46:13 INFO Received uploads request method=POST path=/34632026/11/12 11:46:13 INFO Received push request method=POST path=/34642026/11/12 11:46:13 INFO Completed push id=9 objects=4 duration=3.71129ms34652026/11/12 11:46:13 INFO Received push request method=POST path=/34662026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=11 objects-failed-to-delete=034672026/11/12 11:46:13 INFO Received push request method=POST path=/34682026/11/12 11:46:13 INFO Vacuumed table table=pending_closures34692026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts34702026/11/12 11:46:13 INFO Received uploads request method=POST path=/34712026/11/12 11:46:13 INFO Vacuumed table table=pending_objects34722026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads34732026/11/12 11:46:13 INFO Received uploads request method=POST path=/34742026/11/12 11:46:13 INFO Vacuumed table table=closures34752026/11/12 11:46:13 INFO Vacuumed table table=objects34762026/11/12 11:46:13 INFO Received complete upload request method=POST path=/34772026/11/12 11:46:13 INFO Received push request method=POST path=/34782026/11/12 11:46:13 INFO Completed upload id=16 objects=3 duration=1.49978ms34792026/11/12 11:46:13 INFO Received push request method=POST path=/34802026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=034812026/11/12 11:46:13 INFO Received uploads request method=POST path=/34822026/11/12 11:46:13 INFO Received uploads request method=POST path=/34832026/11/12 11:46:13 INFO Received complete push request method=POST path=/34842026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period34852026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034862026/11/12 11:46:13 INFO Completed push id=13 objects=12 duration=4.9904ms34872026/11/12 11:46:13 INFO Vacuumed table table=pending_closures34882026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts34892026/11/12 11:46:13 INFO Received push request method=POST path=/34902026/11/12 11:46:13 INFO Received complete push request method=POST path=/34912026/11/12 11:46:13 INFO Vacuumed table table=pending_objects34922026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads34932026/11/12 11:46:13 INFO Received push request method=POST path=/34942026/11/12 11:46:13 INFO Vacuumed table table=closures34952026/11/12 11:46:13 INFO Completed push id=8 objects=6 duration=3.64766ms34962026/11/12 11:46:13 INFO Vacuumed table table=objects34972026/11/12 11:46:13 INFO Received push request method=POST path=/34982026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=034992026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=035002026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035012026/11/12 11:46:13 INFO Received push request method=POST path=/35022026/11/12 11:46:13 INFO Vacuumed table table=pending_closures35032026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=035042026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts35052026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035062026/11/12 11:46:13 INFO Received uploads request method=POST path=/35072026/11/12 11:46:13 INFO Vacuumed table table=pending_closures35082026/11/12 11:46:13 INFO Vacuumed table table=pending_objects35092026/11/12 11:46:13 INFO Received complete push request method=POST path=/35102026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads35112026/11/12 11:46:13 INFO Received push request method=POST path=/35122026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts35132026/11/12 11:46:13 INFO Vacuumed table table=closures35142026/11/12 11:46:13 INFO Vacuumed table table=pending_objects35152026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads35162026/11/12 11:46:13 INFO Vacuumed table table=objects35172026/11/12 11:46:13 INFO Vacuumed table table=closures35182026/11/12 11:46:13 INFO Received complete push request method=POST path=/35192026/11/12 11:46:13 INFO Completed push id=11 objects=8 duration=4.32179ms35202026/11/12 11:46:13 INFO Vacuumed table table=objects35212026/11/12 11:46:13 INFO Received complete upload request method=POST path=/35222026/11/12 11:46:13 INFO Completed push id=16 objects=27 duration=2.12791ms35232026/11/12 11:46:13 INFO Received uploads request method=POST path=/35242026/11/12 11:46:13 INFO Received uploads request method=POST path=/35252026/11/12 11:46:13 INFO Completed upload id=19 objects=25 duration=3.54505ms35262026/11/12 11:46:13 INFO Received complete upload request method=POST path=/35272026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=035282026/11/12 11:46:13 INFO Completed upload id=8 objects=3 duration=2.48583ms35292026/11/12 11:46:13 INFO Vacuumed table table=pending_closures35302026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts35312026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=035322026/11/12 11:46:13 INFO Vacuumed table table=pending_objects35332026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads35342026/11/12 11:46:13 INFO Received complete upload request method=POST path=/35352026/11/12 11:46:13 INFO Received uploads request method=POST path=/35362026/11/12 11:46:13 INFO Vacuumed table table=closures35372026/11/12 11:46:13 INFO Completed upload id=10 objects=17 duration=1.574549ms35382026/11/12 11:46:13 INFO Vacuumed table table=objects35392026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035402026/11/12 11:46:13 INFO Received push request method=POST path=/35412026/11/12 11:46:13 INFO Received uploads request method=POST path=/35422026/11/12 11:46:13 INFO Received push request method=POST path=/35432026/11/12 11:46:13 INFO Vacuumed table table=pending_closures35442026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts35452026/11/12 11:46:13 INFO Vacuumed table table=pending_objects35462026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads35472026/11/12 11:46:13 INFO Received uploads request method=POST path=/35482026/11/12 11:46:13 INFO Received complete push request method=POST path=/35492026/11/12 11:46:13 INFO Vacuumed table table=closures35502026/11/12 11:46:13 INFO Vacuumed table table=objects35512026/11/12 11:46:13 INFO Received complete upload request method=POST path=/35522026/11/12 11:46:13 INFO Aborted multipart uploads count=0 kept=035532026/11/12 11:46:13 INFO Completed push id=14 objects=4 duration=3.46371ms35542026/11/12 11:46:13 INFO Received uploads request method=POST path=/35552026/11/12 11:46:13 INFO Completed upload id=16 objects=23 duration=1.42688ms35562026/11/12 11:46:13 WARN Force mode enabled - objects will be deleted immediately without grace period35572026/11/12 11:46:13 INFO Received uploads request method=POST path=/35582026/11/12 11:46:13 INFO Received uploads request method=POST path=/35592026/11/12 11:46:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=035602026/11/12 11:46:13 INFO Vacuumed table table=pending_closures35612026/11/12 11:46:13 INFO Vacuumed table table=commit_receipts35622026/11/12 11:46:13 INFO Vacuumed table table=pending_objects35632026/11/12 11:46:13 INFO Vacuumed table table=multipart_uploads35642026/11/12 11:46:13 INFO Vacuumed table table=closures35652026/11/12 11:46:13 INFO Received uploads request method=POST path=/35662026/11/12 11:46:13 INFO Vacuumed table table=objects35672026/11/12 11:46:14 INFO Received push request method=POST path=/35682026/11/12 11:46:14 INFO Received complete upload request method=POST path=/35692026/11/12 11:46:14 INFO Completed upload id=15 objects=6 duration=1.17537ms35702026/11/12 11:46:14 INFO Received uploads request method=POST path=/35712026/11/12 11:46:14 INFO Received push request method=POST path=/35722026/11/12 11:46:14 INFO Received push request method=POST path=/35732026/11/12 11:46:14 INFO Received complete push request method=POST path=/35742026/11/12 11:46:14 INFO Received push request method=POST path=/35752026/11/12 11:46:14 INFO Completed push id=13 objects=6 duration=3.28763ms35762026/11/12 11:46:14 INFO Received push request method=POST path=/35772026/11/12 11:46:14 INFO Received complete upload request method=POST path=/35782026/11/12 11:46:14 INFO Completed upload id=18 objects=4 duration=2.78032ms35792026/11/12 11:46:14 INFO Received uploads request method=POST path=/35802026/11/12 11:46:14 INFO Received complete push request method=POST path=/35812026/11/12 11:46:14 INFO Received complete push request method=POST path=/35822026/11/12 11:46:14 INFO Completed push id=15 objects=24 duration=4.30052ms35832026/11/12 11:46:14 INFO Completed push id=15 objects=5 duration=2.70998ms35842026/11/12 11:46:14 INFO Received push request method=POST path=/35852026/11/12 11:46:14 INFO Received uploads request method=POST path=/35862026/11/12 11:46:14 INFO Received push request method=POST path=/35872026/11/12 11:46:14 INFO Received complete push request method=POST path=/35882026/11/12 11:46:14 INFO Received uploads request method=POST path=/35892026/11/12 11:46:14 INFO Completed push id=18 objects=10 duration=2.19077ms35902026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=035912026/11/12 11:46:14 INFO Received push request method=POST path=/35922026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035932026/11/12 11:46:14 INFO Received complete push request method=POST path=/35942026/11/12 11:46:14 INFO Vacuumed table table=pending_closures35952026/11/12 11:46:14 INFO Completed push id=9 objects=24 duration=2.00464ms35962026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts35972026/11/12 11:46:14 INFO Received complete push request method=POST path=/35982026/11/12 11:46:14 INFO Vacuumed table table=pending_objects35992026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads36002026/11/12 11:46:14 INFO Vacuumed table table=closures36012026/11/12 11:46:14 INFO Completed push id=6 objects=6 duration=2.31387ms36022026/11/12 11:46:14 INFO Received push request method=POST path=/36032026/11/12 11:46:14 INFO Vacuumed table table=objects36042026/11/12 11:46:14 INFO Received uploads request method=POST path=/36052026/11/12 11:46:14 INFO Received push request method=POST path=/36062026/11/12 11:46:14 INFO Received complete upload request method=POST path=/36072026/11/12 11:46:14 INFO Received uploads request method=POST path=/36082026/11/12 11:46:14 INFO Received complete push request method=POST path=/36092026/11/12 11:46:14 INFO Received push request method=POST path=/36102026/11/12 11:46:14 INFO Completed upload id=17 objects=7 duration=1.33605ms36112026/11/12 11:46:14 INFO Completed push id=17 objects=22 duration=3.13965ms36122026/11/12 11:46:14 INFO Received complete push request method=POST path=/36132026/11/12 11:46:14 INFO Received complete push request method=POST path=/36142026/11/12 11:46:14 INFO Completed push id=14 objects=6 duration=1.92841ms36152026/11/12 11:46:14 INFO Completed push id=19 objects=6 duration=3.35464ms36162026/11/12 11:46:14 INFO Received complete push request method=POST path=/36172026/11/12 11:46:14 INFO Completed push id=17 objects=6 duration=2.19174ms36182026/11/12 11:46:14 INFO Received uploads request method=POST path=/36192026/11/12 11:46:14 INFO Received push request method=POST path=/36202026/11/12 11:46:14 INFO Received uploads request method=POST path=/36212026/11/12 11:46:14 INFO Received uploads request method=POST path=/36222026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=036232026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=036242026/11/12 11:46:14 INFO Received push request method=POST path=/36252026/11/12 11:46:14 INFO Received complete push request method=POST path=/36262026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period36272026/11/12 11:46:14 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."36282026/11/12 11:46:14 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."36292026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036302026/11/12 11:46:14 INFO Vacuumed table table=pending_closures36312026/11/12 11:46:14 INFO Completed push id=9 objects=7 duration=5.33373ms36322026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts36332026/11/12 11:46:14 INFO Vacuumed table table=pending_objects36342026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads36352026/11/12 11:46:14 INFO Vacuumed table table=closures36362026/11/12 11:46:14 INFO Vacuumed table table=objects36372026/11/12 11:46:14 INFO Received uploads request method=POST path=/36382026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=036392026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period36402026/11/12 11:46:14 INFO Received uploads request method=POST path=/36412026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036422026/11/12 11:46:14 INFO Received push request method=POST path=/36432026/11/12 11:46:14 INFO Received complete push request method=POST path=/36442026/11/12 11:46:14 INFO Vacuumed table table=pending_closures36452026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts36462026/11/12 11:46:14 INFO Completed push id=17 objects=6 duration=3.28592ms36472026/11/12 11:46:14 INFO Received push request method=POST path=/36482026/11/12 11:46:14 INFO Vacuumed table table=pending_objects36492026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads36502026/11/12 11:46:14 INFO Vacuumed table table=closures36512026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=036522026/11/12 11:46:14 INFO Vacuumed table table=objects36532026/11/12 11:46:14 INFO Received push request method=POST path=/36542026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period36552026/11/12 11:46:14 INFO Received uploads request method=POST path=/36562026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=5 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036572026/11/12 11:46:14 INFO Vacuumed table table=pending_closures36582026/11/12 11:46:14 INFO Received uploads request method=POST path=/36592026/11/12 11:46:14 INFO Received complete upload request method=POST path=/36602026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts36612026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=036622026/11/12 11:46:14 INFO Completed upload id=10 objects=10 duration=1.79983ms36632026/11/12 11:46:14 INFO Received complete upload request method=POST path=/36642026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period36652026/11/12 11:46:14 INFO Vacuumed table table=pending_objects36662026/11/12 11:46:14 INFO Received complete push request method=POST path=/36672026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads36682026/11/12 11:46:14 INFO Completed upload id=10 objects=7 duration=1.75102ms36692026/11/12 11:46:14 INFO Vacuumed table table=closures36702026/11/12 11:46:14 INFO Completed push id=10 objects=15 duration=2.25862ms36712026/11/12 11:46:14 INFO Received complete push request method=POST path=/36722026/11/12 11:46:14 INFO Vacuumed table table=objects36732026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=036742026/11/12 11:46:14 INFO Completed push id=19 objects=8 duration=4.36403ms36752026/11/12 11:46:14 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."36762026/11/12 11:46:14 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."36772026/11/12 11:46:14 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."36782026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=036792026/11/12 11:46:14 INFO Received complete upload request method=POST path=/36802026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=036812026/11/12 11:46:14 INFO Received push request method=POST path=/36822026/11/12 11:46:14 INFO Completed upload id=14 objects=6 duration=1.227269ms36832026/11/12 11:46:14 INFO Vacuumed table table=pending_closures36842026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts36852026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036862026/11/12 11:46:14 INFO Vacuumed table table=pending_objects36872026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads36882026/11/12 11:46:14 INFO Vacuumed table table=pending_closures36892026/11/12 11:46:14 INFO Vacuumed table table=closures36902026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts36912026/11/12 11:46:14 INFO Vacuumed table table=objects36922026/11/12 11:46:14 INFO Vacuumed table table=pending_objects36932026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads36942026/11/12 11:46:14 INFO Vacuumed table table=closures36952026/11/12 11:46:14 INFO Vacuumed table table=objects36962026/11/12 11:46:14 INFO Received uploads request method=POST path=/36972026/11/12 11:46:14 INFO Received complete push request method=POST path=/36982026/11/12 11:46:14 INFO Received complete upload request method=POST path=/36992026/11/12 11:46:14 INFO Completed push id=4 objects=7 duration=2.43993ms37002026/11/12 11:46:14 INFO Completed upload id=7 objects=10 duration=2.36583ms37012026/11/12 11:46:14 INFO Received complete upload request method=POST path=/37022026/11/12 11:46:14 INFO Received uploads request method=POST path=/37032026/11/12 11:46:14 INFO Received complete upload request method=POST path=/37042026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=037052026/11/12 11:46:14 INFO Completed upload id=6 objects=6 duration=1.96969ms37062026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period37072026/11/12 11:46:14 INFO Completed upload id=17 objects=6 duration=3.36124ms37082026/11/12 11:46:14 INFO Received push request method=POST path=/37092026/11/12 11:46:14 INFO Received push request method=POST path=/37102026/11/12 11:46:14 INFO Received complete push request method=POST path=/37112026/11/12 11:46:14 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."37122026/11/12 11:46:14 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."37132026/11/12 11:46:14 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."37142026/11/12 11:46:14 INFO Completed push id=16 objects=16 duration=3.99495ms37152026/11/12 11:46:14 INFO Received uploads request method=POST path=/37162026/11/12 11:46:14 INFO Received uploads request method=POST path=/37172026/11/12 11:46:14 INFO Received uploads request method=POST path=/37182026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=037192026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=037202026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period37212026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037222026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037232026/11/12 11:46:14 INFO Vacuumed table table=pending_closures37242026/11/12 11:46:14 INFO Vacuumed table table=pending_closures37252026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts37262026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts37272026/11/12 11:46:14 INFO Vacuumed table table=pending_objects37282026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads37292026/11/12 11:46:14 INFO Vacuumed table table=pending_objects37302026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads37312026/11/12 11:46:14 INFO Vacuumed table table=closures37322026/11/12 11:46:14 INFO Vacuumed table table=closures37332026/11/12 11:46:14 INFO Vacuumed table table=objects37342026/11/12 11:46:14 INFO Vacuumed table table=objects37352026/11/12 11:46:14 INFO Received complete upload request method=POST path=/37362026/11/12 11:46:14 INFO Completed upload id=7 objects=3 duration=1.23461ms37372026/11/12 11:46:14 INFO Received uploads request method=POST path=/37382026/11/12 11:46:14 INFO Received push request method=POST path=/37392026/11/12 11:46:14 INFO Received uploads request method=POST path=/37402026/11/12 11:46:14 INFO Received push request method=POST path=/37412026/11/12 11:46:14 INFO Received push request method=POST path=/37422026/11/12 11:46:14 INFO Received push request method=POST path=/37432026/11/12 11:46:14 INFO Received push request method=POST path=/37442026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=037452026/11/12 11:46:14 INFO Received uploads request method=POST path=/37462026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period37472026/11/12 11:46:14 INFO Received uploads request method=POST path=/37482026/11/12 11:46:14 INFO Received complete upload request method=POST path=/37492026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=037502026/11/12 11:46:14 INFO Received push request method=POST path=/37512026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037522026/11/12 11:46:14 INFO Completed upload id=18 objects=4 duration=1.74109ms37532026/11/12 11:46:14 INFO Vacuumed table table=pending_closures37542026/11/12 11:46:14 INFO Received push request method=POST path=/37552026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts37562026/11/12 11:46:14 INFO Vacuumed table table=pending_objects37572026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads37582026/11/12 11:46:14 INFO Vacuumed table table=closures37592026/11/12 11:46:14 INFO Vacuumed table table=objects37602026/11/12 11:46:14 INFO Received complete push request method=POST path=/37612026/11/12 11:46:14 INFO Completed push id=18 objects=23 duration=1.88293ms37622026/11/12 11:46:14 INFO Received complete push request method=POST path=/37632026/11/12 11:46:14 INFO Completed push id=15 objects=7 duration=1.26473ms37642026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=037652026/11/12 11:46:14 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."37662026/11/12 11:46:14 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."37672026/11/12 11:46:14 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."37682026/11/12 11:46:14 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."37692026/11/12 11:46:14 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."37702026/11/12 11:46:14 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."37712026/11/12 11:46:14 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."37722026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period37732026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=037742026/11/12 11:46:14 INFO Received uploads request method=POST path=/37752026/11/12 11:46:14 INFO Received complete push request method=POST path=/37762026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period37772026/11/12 11:46:14 INFO Received uploads request method=POST path=/37782026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037792026/11/12 11:46:14 INFO Completed push id=16 objects=4 duration=3.62746ms37802026/11/12 11:46:14 INFO Received push request method=POST path=/37812026/11/12 11:46:14 INFO Vacuumed table table=pending_closures37822026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=037832026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts37842026/11/12 11:46:14 INFO Received push request method=POST path=/37852026/11/12 11:46:14 INFO Vacuumed table table=pending_objects37862026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads37872026/11/12 11:46:14 INFO Vacuumed table table=pending_closures37882026/11/12 11:46:14 INFO Vacuumed table table=closures37892026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts37902026/11/12 11:46:14 INFO Vacuumed table table=objects37912026/11/12 11:46:14 INFO Vacuumed table table=pending_objects37922026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads37932026/11/12 11:46:14 INFO Received uploads request method=POST path=/37942026/11/12 11:46:14 INFO Vacuumed table table=closures37952026/11/12 11:46:14 INFO Received uploads request method=POST path=/37962026/11/12 11:46:14 INFO Vacuumed table table=objects37972026/11/12 11:46:14 INFO Received uploads request method=POST path=/37982026/11/12 11:46:14 INFO Received complete push request method=POST path=/37992026/11/12 11:46:14 INFO Completed push id=7 objects=7 duration=1.74317ms38002026/11/12 11:46:14 INFO Received complete upload request method=POST path=/38012026/11/12 11:46:14 INFO Completed upload id=11 objects=7 duration=1.42848ms38022026/11/12 11:46:14 INFO Received uploads request method=POST path=/38032026/11/12 11:46:14 INFO Received complete push request method=POST path=/38042026/11/12 11:46:14 INFO Received uploads request method=POST path=/38052026/11/12 11:46:14 INFO Received uploads request method=POST path=/38062026/11/12 11:46:14 INFO Completed push id=12 objects=2 duration=3.49396ms38072026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=038082026/11/12 11:46:14 INFO Received push request method=POST path=/38092026/11/12 11:46:14 INFO Received uploads request method=POST path=/38102026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038112026/11/12 11:46:14 INFO Received push request method=POST path=/38122026/11/12 11:46:14 INFO Vacuumed table table=pending_closures38132026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts38142026/11/12 11:46:14 INFO Received push request method=POST path=/38152026/11/12 11:46:14 INFO Vacuumed table table=pending_objects38162026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads38172026/11/12 11:46:14 INFO Received uploads request method=POST path=/38182026/11/12 11:46:14 INFO Vacuumed table table=closures38192026/11/12 11:46:14 INFO Received uploads request method=POST path=/38202026/11/12 11:46:14 INFO Vacuumed table table=objects38212026/11/12 11:46:14 INFO Received complete push request method=POST path=/38222026/11/12 11:46:14 INFO Completed push id=11 objects=9 duration=1.40517ms38232026/11/12 11:46:14 INFO Received uploads request method=POST path=/38242026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=038252026/11/12 11:46:14 INFO Received uploads request method=POST path=/38262026/11/12 11:46:14 INFO Received complete push request method=POST path=/38272026/11/12 11:46:14 INFO Received uploads request method=POST path=/38282026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period38292026/11/12 11:46:14 INFO Completed push id=14 objects=3 duration=969.44µs38302026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038312026/11/12 11:46:14 INFO Vacuumed table table=pending_closures38322026/11/12 11:46:14 INFO Received complete upload request method=POST path=/38332026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts38342026/11/12 11:46:14 INFO Received uploads request method=POST path=/38352026/11/12 11:46:14 INFO Received uploads request method=POST path=/38362026/11/12 11:46:14 INFO Completed upload id=20 objects=5 duration=1.32037ms38372026/11/12 11:46:14 INFO Vacuumed table table=pending_objects38382026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads38392026/11/12 11:46:14 INFO Vacuumed table table=closures38402026/11/12 11:46:14 INFO Vacuumed table table=objects38412026/11/12 11:46:14 INFO Received push request method=POST path=/38422026/11/12 11:46:14 INFO Received uploads request method=POST path=/38432026/11/12 11:46:14 INFO Received push request method=POST path=/38442026/11/12 11:46:14 INFO Received uploads request method=POST path=/38452026/11/12 11:46:14 INFO Received uploads request method=POST path=/38462026/11/12 11:46:14 INFO Received push request method=POST path=/38472026/11/12 11:46:14 INFO Received uploads request method=POST path=/38482026/11/12 11:46:14 INFO Received uploads request method=POST path=/38492026/11/12 11:46:14 INFO Received push request method=POST path=/38502026/11/12 11:46:14 INFO Received push request method=POST path=/38512026/11/12 11:46:14 INFO Received complete upload request method=POST path=/38522026/11/12 11:46:14 INFO Received push request method=POST path=/38532026/11/12 11:46:14 INFO Completed upload id=18 objects=11 duration=2.13782ms38542026/11/12 11:46:14 INFO Received complete push request method=POST path=/38552026/11/12 11:46:14 INFO Completed push id=14 objects=3 duration=3.72255ms38562026/11/12 11:46:14 INFO Received push request method=POST path=/38572026/11/12 11:46:14 INFO Received push request method=POST path=/38582026/11/12 11:46:14 INFO Received push request method=POST path=/38592026/11/12 11:46:14 INFO Received uploads request method=POST path=/38602026/11/12 11:46:14 INFO Received push request method=POST path=/38612026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=038622026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period38632026/11/12 11:46:14 INFO Received uploads request method=POST path=/38642026/11/12 11:46:14 INFO Received uploads request method=POST path=/38652026/11/12 11:46:14 INFO Received uploads request method=POST path=/38662026/11/12 11:46:14 INFO Received uploads request method=POST path=/38672026/11/12 11:46:14 INFO Received push request method=POST path=/38682026/11/12 11:46:14 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."38692026/11/12 11:46:14 INFO Received uploads request method=POST path=/38702026/11/12 11:46:14 INFO Received complete upload request method=POST path=/38712026/11/12 11:46:14 INFO Received uploads request method=POST path=/38722026/11/12 11:46:14 INFO Received uploads request method=POST path=/38732026/11/12 11:46:14 INFO Completed upload id=22 objects=2 duration=1.83575ms38742026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=038752026/11/12 11:46:14 INFO Received complete push request method=POST path=/38762026/11/12 11:46:14 INFO Received push request method=POST path=/38772026/11/12 11:46:14 INFO Received complete push request method=POST path=/38782026/11/12 11:46:14 INFO Completed push id=21 objects=9 duration=2.84618ms38792026/11/12 11:46:14 INFO Completed push id=1 objects=11 duration=2.28017ms38802026/11/12 11:46:14 INFO Received push request method=POST path=/38812026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038822026/11/12 11:46:14 INFO Vacuumed table table=pending_closures38832026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts38842026/11/12 11:46:14 INFO Vacuumed table table=pending_objects38852026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads38862026/11/12 11:46:14 INFO Vacuumed table table=closures38872026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=038882026/11/12 11:46:14 INFO Received push request method=POST path=/38892026/11/12 11:46:14 INFO Vacuumed table table=objects38902026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period38912026/11/12 11:46:14 INFO Received complete upload request method=POST path=/38922026/11/12 11:46:14 INFO Completed upload id=15 objects=9 duration=1.75775ms38932026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038942026/11/12 11:46:14 INFO Vacuumed table table=pending_closures38952026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=038962026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts38972026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=038982026/11/12 11:46:14 INFO Vacuumed table table=pending_objects38992026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads39002026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=039012026/11/12 11:46:14 INFO Vacuumed table table=closures39022026/11/12 11:46:14 INFO Vacuumed table table=objects39032026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period39042026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039052026/11/12 11:46:14 INFO Vacuumed table table=pending_closures39062026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039072026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts39082026/11/12 11:46:14 INFO Vacuumed table table=pending_closures39092026/11/12 11:46:14 INFO Vacuumed table table=pending_objects39102026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads39112026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts39122026/11/12 11:46:14 INFO Vacuumed table table=closures39132026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=039142026/11/12 11:46:14 INFO Vacuumed table table=pending_objects39152026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads39162026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=039172026/11/12 11:46:14 INFO Vacuumed table table=objects39182026/11/12 11:46:14 INFO Vacuumed table table=closures39192026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period39202026/11/12 11:46:14 INFO Vacuumed table table=pending_closures39212026/11/12 11:46:14 INFO Vacuumed table table=objects39222026/11/12 11:46:14 INFO Received push request method=POST path=/39232026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts39242026/11/12 11:46:14 INFO Received complete upload request method=POST path=/39252026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039262026/11/12 11:46:14 INFO Received push request method=POST path=/39272026/11/12 11:46:14 INFO Completed upload id=16 objects=3 duration=1.24266ms39282026/11/12 11:46:14 INFO Vacuumed table table=pending_objects39292026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads39302026/11/12 11:46:14 INFO Vacuumed table table=pending_closures39312026/11/12 11:46:14 INFO Vacuumed table table=closures39322026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts39332026/11/12 11:46:14 INFO Vacuumed table table=objects39342026/11/12 11:46:14 INFO Vacuumed table table=pending_objects39352026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads39362026/11/12 11:46:14 INFO Vacuumed table table=closures39372026/11/12 11:46:14 INFO Vacuumed table table=objects39382026/11/12 11:46:14 INFO Received uploads request method=POST path=/39392026/11/12 11:46:14 INFO Received complete upload request method=POST path=/39402026/11/12 11:46:14 INFO Completed upload id=14 objects=8 duration=2.60241ms39412026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=039422026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period39432026/11/12 11:46:14 INFO Received uploads request method=POST path=/39442026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039452026/11/12 11:46:14 INFO Vacuumed table table=pending_closures39462026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts39472026/11/12 11:46:14 INFO Received push request method=POST path=/39482026/11/12 11:46:14 INFO Vacuumed table table=pending_objects39492026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads39502026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=039512026/11/12 11:46:14 INFO Vacuumed table table=closures39522026/11/12 11:46:14 INFO Vacuumed table table=objects39532026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period39542026/11/12 11:46:14 INFO Received uploads request method=POST path=/39552026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=039562026/11/12 11:46:14 INFO Received push request method=POST path=/39572026/11/12 11:46:14 INFO Received uploads request method=POST path=/39582026/11/12 11:46:14 INFO Received uploads request method=POST path=/39592026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period39602026/11/12 11:46:14 INFO Received push request method=POST path=/39612026/11/12 11:46:14 INFO Received complete push request method=POST path=/39622026/11/12 11:46:14 INFO Received complete upload request method=POST path=/39632026/11/12 11:46:14 INFO Completed push id=21 objects=17 duration=2.35695ms39642026/11/12 11:46:14 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."39652026/11/12 11:46:14 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."39662026/11/12 11:46:14 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."39672026/11/12 11:46:14 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."39682026/11/12 11:46:14 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."39692026/11/12 11:46:14 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."39702026/11/12 11:46:14 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."39712026/11/12 11:46:14 INFO Completed upload id=23 objects=18 duration=2.87869ms39722026/11/12 11:46:14 INFO Received push request method=POST path=/39732026/11/12 11:46:14 INFO Received uploads request method=POST path=/39742026/11/12 11:46:14 INFO Received complete push request method=POST path=/39752026/11/12 11:46:14 INFO Completed push id=13 objects=15 duration=2.13501ms39762026/11/12 11:46:14 INFO Received push request method=POST path=/39772026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=039782026/11/12 11:46:14 INFO Received complete upload request method=POST path=/39792026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039802026/11/12 11:46:14 INFO Completed upload id=14 objects=16 duration=2.88211ms39812026/11/12 11:46:14 INFO Received uploads request method=POST path=/39822026/11/12 11:46:14 INFO Vacuumed table table=pending_closures39832026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts39842026/11/12 11:46:14 INFO Vacuumed table table=pending_objects39852026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads39862026/11/12 11:46:14 INFO Received complete push request method=POST path=/39872026/11/12 11:46:14 INFO Vacuumed table table=closures39882026/11/12 11:46:14 INFO Completed push id=19 objects=7 duration=1.65124ms39892026/11/12 11:46:14 INFO Vacuumed table table=objects39902026/11/12 11:46:14 INFO Received uploads request method=POST path=/39912026/11/12 11:46:14 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."39922026/11/12 11:46:14 INFO Received push request method=POST path=/39932026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=039942026/11/12 11:46:14 INFO Received push request method=POST path=/39952026/11/12 11:46:14 INFO Received push request method=POST path=/39962026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=039972026/11/12 11:46:14 INFO Received uploads request method=POST path=/39982026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period39992026/11/12 11:46:14 INFO Received complete upload request method=POST path=/40002026/11/12 11:46:14 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."40012026/11/12 11:46:14 INFO Completed upload id=21 objects=3 duration=1.3433ms40022026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040032026/11/12 11:46:14 INFO Vacuumed table table=pending_closures40042026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts40052026/11/12 11:46:14 INFO Vacuumed table table=pending_objects40062026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads40072026/11/12 11:46:14 INFO Received uploads request method=POST path=/40082026/11/12 11:46:14 INFO Vacuumed table table=closures40092026/11/12 11:46:14 INFO Received complete push request method=POST path=/40102026/11/12 11:46:14 INFO Received push request method=POST path=/40112026/11/12 11:46:14 INFO Vacuumed table table=objects40122026/11/12 11:46:14 INFO Received uploads request method=POST path=/40132026/11/12 11:46:14 INFO Received push request method=POST path=/40142026/11/12 11:46:14 INFO Completed push id=22 objects=3 duration=3.18903ms40152026/11/12 11:46:14 INFO Received push request method=POST path=/40162026/11/12 11:46:14 INFO Received push request method=POST path=/40172026/11/12 11:46:14 INFO Received complete push request method=POST path=/40182026/11/12 11:46:14 INFO Completed push id=26 objects=14 duration=3.24776ms40192026/11/12 11:46:14 INFO Received complete upload request method=POST path=/40202026/11/12 11:46:14 INFO Completed upload id=23 objects=15 duration=1.79998ms40212026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=040222026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period40232026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=040242026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=040252026/11/12 11:46:14 INFO Received complete upload request method=POST path=/40262026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period40272026/11/12 11:46:14 INFO Received push request method=POST path=/40282026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=040292026/11/12 11:46:14 INFO Received complete upload request method=POST path=/40302026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040312026/11/12 11:46:14 INFO Received push request method=POST path=/40322026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=040332026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040342026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period40352026/11/12 11:46:14 INFO Vacuumed table table=pending_closures40362026/11/12 11:46:14 INFO Completed upload id=22 objects=22 duration=1.67845ms40372026/11/12 11:46:14 INFO Completed upload id=23 objects=16 duration=3.15705ms40382026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts40392026/11/12 11:46:14 INFO Received complete push request method=POST path=/40402026/11/12 11:46:14 INFO Received push request method=POST path=/40412026/11/12 11:46:14 INFO Vacuumed table table=pending_closures40422026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts40432026/11/12 11:46:14 INFO Vacuumed table table=pending_objects40442026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads40452026/11/12 11:46:14 INFO Completed push id=15 objects=4 duration=2.08914ms40462026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period40472026/11/12 11:46:14 INFO Vacuumed table table=pending_objects40482026/11/12 11:46:14 INFO Vacuumed table table=closures40492026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads40502026/11/12 11:46:14 INFO Vacuumed table table=objects40512026/11/12 11:46:14 INFO Vacuumed table table=closures40522026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=7 objects-failed-to-delete=040532026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040542026/11/12 11:46:14 INFO Received complete push request method=POST path=/40552026/11/12 11:46:14 INFO Vacuumed table table=objects40562026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=040572026/11/12 11:46:14 INFO Vacuumed table table=pending_closures40582026/11/12 11:46:14 INFO Received complete push request method=POST path=/40592026/11/12 11:46:14 INFO Vacuumed table table=pending_closures40602026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period40612026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts40622026/11/12 11:46:14 INFO Completed push id=17 objects=11 duration=2.61054ms40632026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts40642026/11/12 11:46:14 INFO Completed push id=20 objects=3 duration=2.31755ms40652026/11/12 11:46:14 INFO Vacuumed table table=pending_objects40662026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads40672026/11/12 11:46:14 INFO Vacuumed table table=pending_objects40682026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads40692026/11/12 11:46:14 INFO Vacuumed table table=closures40702026/11/12 11:46:14 INFO Vacuumed table table=closures40712026/11/12 11:46:14 INFO Vacuumed table table=objects40722026/11/12 11:46:14 INFO Vacuumed table table=objects40732026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=040742026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=040752026/11/12 11:46:14 INFO Vacuumed table table=pending_closures40762026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts40772026/11/12 11:46:14 INFO Vacuumed table table=pending_objects40782026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads40792026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040802026/11/12 11:46:14 INFO Vacuumed table table=closures40812026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=040822026/11/12 11:46:14 INFO Received push request method=POST path=/40832026/11/12 11:46:14 INFO Vacuumed table table=pending_closures40842026/11/12 11:46:14 INFO Received push request method=POST path=/40852026/11/12 11:46:14 INFO Vacuumed table table=objects40862026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period40872026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts40882026/11/12 11:46:14 INFO Received uploads request method=POST path=/40892026/11/12 11:46:14 INFO Vacuumed table table=pending_objects40902026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040912026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads40922026/11/12 11:46:14 INFO Vacuumed table table=closures40932026/11/12 11:46:14 INFO Vacuumed table table=pending_closures40942026/11/12 11:46:14 INFO Vacuumed table table=objects40952026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts40962026/11/12 11:46:14 INFO Received push request method=POST path=/40972026/11/12 11:46:14 INFO Vacuumed table table=pending_objects40982026/11/12 11:46:14 INFO Received push request method=POST path=/40992026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads41002026/11/12 11:46:14 INFO Vacuumed table table=closures41012026/11/12 11:46:14 INFO Vacuumed table table=objects41022026/11/12 11:46:14 INFO Received complete upload request method=POST path=/41032026/11/12 11:46:14 INFO Completed upload id=19 objects=6 duration=928.49µs41042026/11/12 11:46:14 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."41052026/11/12 11:46:14 INFO Received uploads request method=POST path=/41062026/11/12 11:46:14 INFO Received uploads request method=POST path=/41072026/11/12 11:46:14 INFO Received push request method=POST path=/41082026/11/12 11:46:14 INFO Received complete push request method=POST path=/41092026/11/12 11:46:14 INFO Received complete push request method=POST path=/41102026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=041112026/11/12 11:46:14 INFO Completed push id=26 objects=5 duration=2.08246ms41122026/11/12 11:46:14 INFO Completed push id=4 objects=6 duration=2.15232ms41132026/11/12 11:46:14 INFO Received uploads request method=POST path=/41142026/11/12 11:46:14 INFO Received push request method=POST path=/41152026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041162026/11/12 11:46:14 INFO Vacuumed table table=pending_closures41172026/11/12 11:46:14 INFO Received uploads request method=POST path=/41182026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts41192026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=041202026/11/12 11:46:14 INFO Vacuumed table table=pending_objects41212026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads41222026/11/12 11:46:14 INFO Received uploads request method=POST path=/41232026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period41242026/11/12 11:46:14 INFO Vacuumed table table=closures41252026/11/12 11:46:14 INFO Vacuumed table table=objects41262026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041272026/11/12 11:46:14 INFO Vacuumed table table=pending_closures41282026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts41292026/11/12 11:46:14 INFO Vacuumed table table=pending_objects41302026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads41312026/11/12 11:46:14 INFO Vacuumed table table=closures41322026/11/12 11:46:14 INFO Received complete upload request method=POST path=/41332026/11/12 11:46:14 INFO Vacuumed table table=objects41342026/11/12 11:46:14 INFO Completed upload id=22 objects=7 duration=1.19196ms41352026/11/12 11:46:14 INFO Received uploads request method=POST path=/41362026/11/12 11:46:14 INFO Received push request method=POST path=/41372026/11/12 11:46:14 INFO Received uploads request method=POST path=/41382026/11/12 11:46:14 INFO Received complete upload request method=POST path=/41392026/11/12 11:46:14 INFO Completed upload id=24 objects=14 duration=1.44419ms41402026/11/12 11:46:14 INFO Received complete upload request method=POST path=/41412026/11/12 11:46:14 INFO Received uploads request method=POST path=/41422026/11/12 11:46:14 INFO Received push request method=POST path=/41432026/11/12 11:46:14 INFO Completed upload id=12 objects=26 duration=4.39285ms41442026/11/12 11:46:14 INFO Received push request method=POST path=/41452026/11/12 11:46:14 INFO Received complete push request method=POST path=/41462026/11/12 11:46:14 INFO Completed push id=14 objects=3 duration=1.4888ms41472026/11/12 11:46:14 INFO Received push request method=POST path=/41482026/11/12 11:46:14 INFO Received complete upload request method=POST path=/41492026/11/12 11:46:14 INFO Received push request method=POST path=/41502026/11/12 11:46:14 INFO Received complete push request method=POST path=/41512026/11/12 11:46:14 INFO Completed upload id=26 objects=6 duration=1.4903ms41522026/11/12 11:46:14 INFO Completed push id=19 objects=3 duration=1.67764ms41532026/11/12 11:46:14 INFO Received uploads request method=POST path=/41542026/11/12 11:46:14 INFO Received complete upload request method=POST path=/41552026/11/12 11:46:14 INFO Completed upload id=16 objects=17 duration=2.85814ms41562026/11/12 11:46:14 INFO Received push request method=POST path=/41572026/11/12 11:46:14 INFO Received uploads request method=POST path=/41582026/11/12 11:46:14 INFO Received uploads request method=POST path=/41592026/11/12 11:46:14 INFO Received complete push request method=POST path=/41602026/11/12 11:46:14 INFO Completed push id=22 objects=3 duration=3.15219ms41612026/11/12 11:46:14 INFO Received push request method=POST path=/41622026/11/12 11:46:14 INFO Received uploads request method=POST path=/41632026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=041642026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=041652026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period41662026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period41672026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=041682026/11/12 11:46:14 INFO Received uploads request method=POST path=/41692026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041702026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period41712026/11/12 11:46:14 INFO Received push request method=POST path=/41722026/11/12 11:46:14 INFO Vacuumed table table=pending_closures41732026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041742026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts41752026/11/12 11:46:14 INFO Vacuumed table table=pending_closures41762026/11/12 11:46:14 INFO Received uploads request method=POST path=/41772026/11/12 11:46:14 INFO Vacuumed table table=pending_objects41782026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts41792026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads41802026/11/12 11:46:14 INFO Received complete push request method=POST path=/41812026/11/12 11:46:14 INFO Vacuumed table table=closures41822026/11/12 11:46:14 INFO Vacuumed table table=pending_objects41832026/11/12 11:46:14 INFO Received complete upload request method=POST path=/41842026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads41852026/11/12 11:46:14 INFO Completed push id=24 objects=9 duration=1.49825ms41862026/11/12 11:46:14 INFO Vacuumed table table=objects41872026/11/12 11:46:14 INFO Vacuumed table table=closures41882026/11/12 11:46:14 INFO Completed upload id=13 objects=8 duration=1.59054ms41892026/11/12 11:46:14 INFO Vacuumed table table=objects41902026/11/12 11:46:14 INFO Received push request method=POST path=/41912026/11/12 11:46:14 INFO Received complete upload request method=POST path=/41922026/11/12 11:46:14 INFO Completed upload id=29 objects=5 duration=1.81613ms41932026/11/12 11:46:14 INFO Received uploads request method=POST path=/41942026/11/12 11:46:14 INFO Received complete upload request method=POST path=/41952026/11/12 11:46:14 INFO Received uploads request method=POST path=/41962026/11/12 11:46:14 INFO Completed upload id=20 objects=4 duration=1.20491ms41972026/11/12 11:46:14 INFO Received uploads request method=POST path=/41982026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=041992026/11/12 11:46:14 INFO Received complete push request method=POST path=/42002026/11/12 11:46:14 INFO Received push request method=POST path=/42012026/11/12 11:46:14 INFO Completed push id=16 objects=8 duration=2.40043ms42022026/11/12 11:46:14 INFO Received uploads request method=POST path=/42032026/11/12 11:46:14 INFO Received uploads request method=POST path=/42042026/11/12 11:46:14 INFO Received push request method=POST path=/42052026/11/12 11:46:14 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."42062026/11/12 11:46:14 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."42072026/11/12 11:46:14 INFO Received complete upload request method=POST path=/42082026/11/12 11:46:14 INFO Completed upload id=22 objects=6 duration=1.34253ms42092026/11/12 11:46:14 INFO Received push request method=POST path=/42102026/11/12 11:46:14 INFO Received push request method=POST path=/42112026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=042122026/11/12 11:46:14 INFO Received complete push request method=POST path=/42132026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period42142026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042152026/11/12 11:46:14 INFO Completed push id=7 objects=15 duration=3.22731ms42162026/11/12 11:46:14 INFO Vacuumed table table=pending_closures42172026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts42182026/11/12 11:46:14 INFO Vacuumed table table=pending_objects42192026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads42202026/11/12 11:46:14 INFO Vacuumed table table=closures42212026/11/12 11:46:14 INFO Received uploads request method=POST path=/42222026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=042232026/11/12 11:46:14 INFO Received push request method=POST path=/42242026/11/12 11:46:14 INFO Vacuumed table table=objects42252026/11/12 11:46:14 INFO Received push request method=POST path=/42262026/11/12 11:46:14 INFO Received uploads request method=POST path=/42272026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period42282026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=042292026/11/12 11:46:14 INFO Received complete push request method=POST path=/42302026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period42312026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042322026/11/12 11:46:14 INFO Completed push id=16 objects=5 duration=1.68458ms42332026/11/12 11:46:14 INFO Received uploads request method=POST path=/42342026/11/12 11:46:14 INFO Vacuumed table table=pending_closures42352026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042362026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts42372026/11/12 11:46:14 INFO Vacuumed table table=pending_closures42382026/11/12 11:46:14 INFO Vacuumed table table=pending_objects42392026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads42402026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts42412026/11/12 11:46:14 INFO Received complete push request method=POST path=/42422026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=042432026/11/12 11:46:14 INFO Vacuumed table table=closures42442026/11/12 11:46:14 INFO Vacuumed table table=pending_objects42452026/11/12 11:46:14 INFO Vacuumed table table=objects42462026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads42472026/11/12 11:46:14 INFO Vacuumed table table=closures42482026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period42492026/11/12 11:46:14 INFO Completed push id=18 objects=6 duration=3.78242ms42502026/11/12 11:46:14 INFO Vacuumed table table=objects42512026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042522026/11/12 11:46:14 INFO Received complete push request method=POST path=/42532026/11/12 11:46:14 INFO Vacuumed table table=pending_closures42542026/11/12 11:46:14 INFO Completed push id=24 objects=3 duration=1.62143ms42552026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts42562026/11/12 11:46:14 INFO Vacuumed table table=pending_objects42572026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads42582026/11/12 11:46:14 INFO Vacuumed table table=closures42592026/11/12 11:46:14 INFO Vacuumed table table=objects42602026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=042612026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period42622026/11/12 11:46:14 INFO Received uploads request method=POST path=/42632026/11/12 11:46:14 INFO Received push request method=POST path=/42642026/11/12 11:46:14 INFO Received push request method=POST path=/42652026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=042662026/11/12 11:46:14 INFO Received complete upload request method=POST path=/42672026/11/12 11:46:14 INFO Completed upload id=25 objects=4 duration=1.09382ms42682026/11/12 11:46:14 INFO Vacuumed table table=pending_closures42692026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts42702026/11/12 11:46:14 INFO Received complete push request method=POST path=/42712026/11/12 11:46:14 INFO Vacuumed table table=pending_objects42722026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads42732026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=042742026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=042752026/11/12 11:46:14 INFO Vacuumed table table=closures42762026/11/12 11:46:14 INFO Vacuumed table table=pending_closures42772026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period42782026/11/12 11:46:14 INFO Vacuumed table table=objects42792026/11/12 11:46:14 INFO Received push request method=POST path=/42802026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts42812026/11/12 11:46:14 INFO Completed push id=11 objects=6 duration=4.04521ms42822026/11/12 11:46:14 INFO Received push request method=POST path=/42832026/11/12 11:46:14 INFO Vacuumed table table=pending_objects42842026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042852026/11/12 11:46:14 INFO Received complete upload request method=POST path=/42862026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads42872026/11/12 11:46:14 INFO Completed upload id=23 objects=9 duration=1.222779ms42882026/11/12 11:46:14 INFO Vacuumed table table=closures42892026/11/12 11:46:14 INFO Vacuumed table table=pending_closures42902026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts42912026/11/12 11:46:14 INFO Vacuumed table table=objects42922026/11/12 11:46:14 INFO Vacuumed table table=pending_objects42932026/11/12 11:46:14 INFO Received push request method=POST path=/42942026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads42952026/11/12 11:46:14 INFO Vacuumed table table=closures42962026/11/12 11:46:14 INFO Vacuumed table table=objects42972026/11/12 11:46:14 INFO Received uploads request method=POST path=/42982026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=042992026/11/12 11:46:14 INFO Received uploads request method=POST path=/43002026/11/12 11:46:14 INFO Received complete push request method=POST path=/43012026/11/12 11:46:14 INFO Received push request method=POST path=/43022026/11/12 11:46:14 INFO Completed push id=31 objects=14 duration=3.2863ms43032026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=043042026/11/12 11:46:14 INFO Vacuumed table table=pending_closures43052026/11/12 11:46:14 INFO Received push request method=POST path=/43062026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts43072026/11/12 11:46:14 INFO Vacuumed table table=pending_objects43082026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads43092026/11/12 11:46:14 INFO Vacuumed table table=closures43102026/11/12 11:46:14 INFO Received uploads request method=POST path=/43112026/11/12 11:46:14 INFO Vacuumed table table=objects43122026/11/12 11:46:14 INFO Received complete upload request method=POST path=/43132026/11/12 11:46:14 INFO Completed upload id=21 objects=7 duration=2.68209ms43142026/11/12 11:46:14 INFO Received complete push request method=POST path=/43152026/11/12 11:46:14 INFO Received push request method=POST path=/43162026/11/12 11:46:14 INFO Completed push id=18 objects=6 duration=1.43766ms43172026/11/12 11:46:14 INFO Received push request method=POST path=/43182026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=043192026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period43202026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043212026/11/12 11:46:14 INFO Vacuumed table table=pending_closures43222026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts43232026/11/12 11:46:14 INFO Received complete upload request method=POST path=/43242026/11/12 11:46:14 INFO Vacuumed table table=pending_objects43252026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=043262026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads43272026/11/12 11:46:14 INFO Vacuumed table table=closures43282026/11/12 11:46:14 INFO Received uploads request method=POST path=/43292026/11/12 11:46:14 INFO Completed upload id=17 objects=25 duration=2.12287ms43302026/11/12 11:46:14 INFO Vacuumed table table=objects43312026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period43322026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=043332026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043342026/11/12 11:46:14 INFO Received complete push request method=POST path=/43352026/11/12 11:46:14 INFO Vacuumed table table=pending_closures43362026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043372026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts43382026/11/12 11:46:14 INFO Vacuumed table table=pending_closures43392026/11/12 11:46:14 INFO Received push request method=POST path=/43402026/11/12 11:46:14 INFO Completed push id=5 objects=28 duration=3.24479ms43412026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts43422026/11/12 11:46:14 INFO Vacuumed table table=pending_objects43432026/11/12 11:46:14 INFO Received complete push request method=POST path=/43442026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads43452026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=043462026/11/12 11:46:14 INFO Vacuumed table table=pending_objects43472026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads43482026/11/12 11:46:14 INFO Vacuumed table table=closures43492026/11/12 11:46:14 INFO Vacuumed table table=closures43502026/11/12 11:46:14 INFO Vacuumed table table=objects43512026/11/12 11:46:14 INFO Vacuumed table table=objects43522026/11/12 11:46:14 INFO Received push request method=POST path=/43532026/11/12 11:46:14 INFO Completed push id=27 objects=25 duration=3.49933ms43542026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043552026/11/12 11:46:14 INFO Vacuumed table table=pending_closures43562026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts43572026/11/12 11:46:14 INFO Vacuumed table table=pending_objects43582026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads43592026/11/12 11:46:14 INFO Vacuumed table table=closures43602026/11/12 11:46:14 INFO Vacuumed table table=objects43612026/11/12 11:46:14 INFO Received complete push request method=POST path=/43622026/11/12 11:46:14 INFO Received complete upload request method=POST path=/43632026/11/12 11:46:14 INFO Completed push id=20 objects=8 duration=1.34897ms43642026/11/12 11:46:14 INFO Received uploads request method=POST path=/43652026/11/12 11:46:14 INFO Completed upload id=15 objects=3 duration=2.06819ms43662026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=043672026/11/12 11:46:14 INFO Received push request method=POST path=/43682026/11/12 11:46:14 INFO Received push request method=POST path=/43692026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043702026/11/12 11:46:14 INFO Vacuumed table table=pending_closures43712026/11/12 11:46:14 INFO Received complete push request method=POST path=/43722026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts43732026/11/12 11:46:14 INFO Completed push id=10 objects=2 duration=1.91313ms43742026/11/12 11:46:14 INFO Received uploads request method=POST path=/43752026/11/12 11:46:14 INFO Vacuumed table table=pending_objects43762026/11/12 11:46:14 INFO Received uploads request method=POST path=/43772026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads43782026/11/12 11:46:14 INFO Vacuumed table table=closures43792026/11/12 11:46:14 INFO Vacuumed table table=objects43802026/11/12 11:46:14 INFO Received uploads request method=POST path=/43812026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=043822026/11/12 11:46:14 INFO Received complete upload request method=POST path=/43832026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043842026/11/12 11:46:14 INFO Received uploads request method=POST path=/43852026/11/12 11:46:14 INFO Vacuumed table table=pending_closures43862026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts43872026/11/12 11:46:14 INFO Completed upload id=20 objects=3 duration=3.15746ms43882026/11/12 11:46:14 INFO Received complete upload request method=POST path=/43892026/11/12 11:46:14 INFO Vacuumed table table=pending_objects43902026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads43912026/11/12 11:46:14 INFO Vacuumed table table=closures43922026/11/12 11:46:14 INFO Received complete upload request method=POST path=/43932026/11/12 11:46:14 INFO Completed upload id=31 objects=3 duration=2.5961ms43942026/11/12 11:46:14 INFO Vacuumed table table=objects43952026/11/12 11:46:14 INFO Completed upload id=21 objects=2 duration=1.52969ms43962026/11/12 11:46:14 INFO Received uploads request method=POST path=/43972026/11/12 11:46:14 INFO Received push request method=POST path=/43982026/11/12 11:46:14 INFO Received uploads request method=POST path=/43992026/11/12 11:46:14 INFO Received complete push request method=POST path=/44002026/11/12 11:46:14 INFO Completed push id=21 objects=3 duration=953.34µs44012026/11/12 11:46:14 INFO Received complete upload request method=POST path=/44022026/11/12 11:46:14 INFO Completed upload id=17 objects=5 duration=1.357639ms44032026/11/12 11:46:14 INFO Received push request method=POST path=/44042026/11/12 11:46:14 INFO Received complete upload request method=POST path=/44052026/11/12 11:46:14 INFO Received push request method=POST path=/44062026/11/12 11:46:14 INFO Completed upload id=22 objects=2 duration=1.57252ms44072026/11/12 11:46:14 INFO Received complete push request method=POST path=/44082026/11/12 11:46:14 INFO Received uploads request method=POST path=/44092026/11/12 11:46:14 INFO Completed push id=24 objects=5 duration=3.24558ms44102026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=044112026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period44122026/11/12 11:46:14 INFO Received push request method=POST path=/44132026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044142026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=044152026/11/12 11:46:14 INFO Vacuumed table table=pending_closures44162026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts44172026/11/12 11:46:14 INFO Received complete push request method=POST path=/44182026/11/12 11:46:14 INFO Received complete upload request method=POST path=/44192026/11/12 11:46:14 INFO Vacuumed table table=pending_objects44202026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads44212026/11/12 11:46:14 INFO Completed push id=7 objects=7 duration=1.99074ms44222026/11/12 11:46:14 INFO Vacuumed table table=closures44232026/11/12 11:46:14 INFO Received uploads request method=POST path=/44242026/11/12 11:46:14 INFO Completed upload id=33 objects=5 duration=2.87023ms44252026/11/12 11:46:14 INFO Vacuumed table table=objects44262026/11/12 11:46:14 INFO Received push request method=POST path=/44272026/11/12 11:46:14 INFO Received uploads request method=POST path=/44282026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=044292026/11/12 11:46:14 INFO Vacuumed table table=pending_closures44302026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts44312026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=044322026/11/12 11:46:14 INFO Vacuumed table table=pending_objects44332026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads44342026/11/12 11:46:14 INFO Vacuumed table table=closures44352026/11/12 11:46:14 INFO Received complete push request method=POST path=/44362026/11/12 11:46:14 INFO Vacuumed table table=objects44372026/11/12 11:46:14 INFO Received uploads request method=POST path=/44382026/11/12 11:46:14 INFO Completed push id=19 objects=3 duration=1.44014ms44392026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period44402026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=044412026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044422026/11/12 11:46:14 INFO Vacuumed table table=pending_closures44432026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044442026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts44452026/11/12 11:46:14 INFO Vacuumed table table=pending_closures44462026/11/12 11:46:14 INFO Vacuumed table table=pending_objects44472026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts44482026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads44492026/11/12 11:46:14 INFO Received complete push request method=POST path=/44502026/11/12 11:46:14 INFO Vacuumed table table=closures44512026/11/12 11:46:14 INFO Received uploads request method=POST path=/44522026/11/12 11:46:14 INFO Vacuumed table table=pending_objects44532026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads44542026/11/12 11:46:14 INFO Vacuumed table table=objects44552026/11/12 11:46:14 INFO Received complete push request method=POST path=/44562026/11/12 11:46:14 INFO Completed push id=12 objects=6 duration=1.88972ms44572026/11/12 11:46:14 INFO Vacuumed table table=closures44582026/11/12 11:46:14 INFO Vacuumed table table=objects44592026/11/12 11:46:14 INFO Completed push id=19 objects=14 duration=2.72495ms44602026/11/12 11:46:14 INFO Received complete upload request method=POST path=/44612026/11/12 11:46:14 INFO Completed upload id=24 objects=7 duration=1.18783ms44622026/11/12 11:46:14 INFO Received uploads request method=POST path=/44632026/11/12 11:46:14 INFO Received complete push request method=POST path=/44642026/11/12 11:46:14 INFO Received complete upload request method=POST path=/44652026/11/12 11:46:14 INFO Completed push id=27 objects=11 duration=1.74045ms44662026/11/12 11:46:14 INFO Completed upload id=27 objects=4 duration=1.56111ms44672026/11/12 11:46:14 INFO Received complete push request method=POST path=/44682026/11/12 11:46:14 INFO Received complete upload request method=POST path=/44692026/11/12 11:46:14 INFO Completed push id=17 objects=22 duration=2.308609ms44702026/11/12 11:46:14 INFO Completed upload id=18 objects=2 duration=2.71942ms44712026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=044722026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044732026/11/12 11:46:14 INFO Vacuumed table table=pending_closures44742026/11/12 11:46:14 INFO Received uploads request method=POST path=/44752026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts44762026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=044772026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=044782026/11/12 11:46:14 INFO Vacuumed table table=pending_objects44792026/11/12 11:46:14 INFO Received complete push request method=POST path=/44802026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads44812026/11/12 11:46:14 INFO Received complete push request method=POST path=/44822026/11/12 11:46:14 INFO Received uploads request method=POST path=/44832026/11/12 11:46:14 INFO Vacuumed table table=closures44842026/11/12 11:46:14 INFO Completed push id=20 objects=4 duration=1.61376ms44852026/11/12 11:46:14 INFO Completed push id=16 objects=26 duration=2.00616ms44862026/11/12 11:46:14 INFO Vacuumed table table=objects44872026/11/12 11:46:14 INFO Aborted multipart uploads count=0 kept=044882026/11/12 11:46:14 WARN Force mode enabled - objects will be deleted immediately without grace period44892026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=3 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044902026/11/12 11:46:14 INFO Received complete push request method=POST path=/44912026/11/12 11:46:14 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044922026/11/12 11:46:14 INFO Vacuumed table table=pending_closures44932026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts44942026/11/12 11:46:14 INFO Vacuumed table table=pending_closures44952026/11/12 11:46:14 INFO Received complete upload request method=POST path=/44962026/11/12 11:46:14 INFO Completed push id=13 objects=11 duration=2.04231ms44972026/11/12 11:46:14 INFO Vacuumed table table=commit_receipts44982026/11/12 11:46:14 INFO Completed upload id=29 objects=26 duration=1.60792ms44992026/11/12 11:46:14 INFO Received complete upload request method=POST path=/45002026/11/12 11:46:14 INFO Vacuumed table table=pending_objects45012026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads45022026/11/12 11:46:14 INFO Vacuumed table table=pending_objects45032026/11/12 11:46:14 INFO Received uploads request method=POST path=/45042026/11/12 11:46:14 INFO Received push request method=POST path=/45052026/11/12 11:46:14 INFO Vacuumed table table=multipart_uploads45062026/11/12 11:46:14 INFO Vacuumed table table=closures45072026/11/12 11:46:14 INFO Completed upload id=21 objects=25 duration=2.07067ms45082026/11/12 11:46:14 INFO Vacuumed table table=closures45092026/11/12 11:46:14 INFO Vacuumed table table=objects45102026/11/12 11:46:14 INFO Vacuumed table table=objects45112026/11/12 11:46:14 INFO Received push request method=POST path=/45122026/11/12 11:46:15 INFO Received complete push request method=POST path=/45132026/11/12 11:46:15 INFO Received complete upload request method=POST path=/45142026/11/12 11:46:15 INFO Received complete push request method=POST path=/45152026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=045162026/11/12 11:46:15 INFO Completed push id=21 objects=26 duration=1.7905ms45172026/11/12 11:46:15 INFO Completed upload id=26 objects=2 duration=2.09728ms45182026/11/12 11:46:15 INFO Received uploads request method=POST path=/45192026/11/12 11:46:15 INFO Completed push id=7 objects=24 duration=3.75655ms45202026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period45212026/11/12 11:46:15 INFO Received complete push request method=POST path=/45222026/11/12 11:46:15 INFO Received complete push request method=POST path=/45232026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045242026/11/12 11:46:15 INFO Completed push id=30 objects=8 duration=1.70947ms45252026/11/12 11:46:15 INFO Vacuumed table table=pending_closures45262026/11/12 11:46:15 INFO Completed push id=13 objects=22 duration=2.12773ms45272026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts45282026/11/12 11:46:15 INFO Vacuumed table table=pending_objects45292026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads45302026/11/12 11:46:15 INFO Vacuumed table table=closures45312026/11/12 11:46:15 INFO Received complete upload request method=POST path=/45322026/11/12 11:46:15 INFO Received uploads request method=POST path=/45332026/11/12 11:46:15 INFO Vacuumed table table=objects45342026/11/12 11:46:15 INFO Completed upload id=22 objects=8 duration=1.47099ms45352026/11/12 11:46:15 INFO Received push request method=POST path=/45362026/11/12 11:46:15 INFO Received uploads request method=POST path=/45372026/11/12 11:46:15 INFO Received push request method=POST path=/45382026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=045392026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045402026/11/12 11:46:15 INFO Received uploads request method=POST path=/45412026/11/12 11:46:15 INFO Vacuumed table table=pending_closures45422026/11/12 11:46:15 INFO Received complete upload request method=POST path=/45432026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts45442026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=045452026/11/12 11:46:15 INFO Completed upload id=13 objects=7 duration=2.01454ms45462026/11/12 11:46:15 INFO Received complete upload request method=POST path=/45472026/11/12 11:46:15 INFO Received complete upload request method=POST path=/45482026/11/12 11:46:15 INFO Vacuumed table table=pending_closures45492026/11/12 11:46:15 INFO Vacuumed table table=pending_objects45502026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads45512026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts45522026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=045532026/11/12 11:46:15 INFO Completed upload id=28 objects=17 duration=1.39975ms45542026/11/12 11:46:15 INFO Received complete push request method=POST path=/45552026/11/12 11:46:15 INFO Completed upload id=31 objects=5 duration=1.2932ms45562026/11/12 11:46:15 INFO Vacuumed table table=closures45572026/11/12 11:46:15 INFO Completed push id=21 objects=6 duration=1.68515ms45582026/11/12 11:46:15 INFO Vacuumed table table=objects45592026/11/12 11:46:15 INFO Vacuumed table table=pending_objects45602026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads45612026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period45622026/11/12 11:46:15 INFO Received complete push request method=POST path=/45632026/11/12 11:46:15 INFO Vacuumed table table=closures45642026/11/12 11:46:15 INFO Vacuumed table table=objects45652026/11/12 11:46:15 INFO Received complete push request method=POST path=/45662026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045672026/11/12 11:46:15 INFO Completed push id=23 objects=5 duration=1.17415ms45682026/11/12 11:46:15 INFO Completed push id=27 objects=6 duration=3.44631ms45692026/11/12 11:46:15 INFO Vacuumed table table=pending_closures45702026/11/12 11:46:15 INFO Received push request method=POST path=/45712026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts45722026/11/12 11:46:15 INFO Received push request method=POST path=/45732026/11/12 11:46:15 INFO Vacuumed table table=pending_objects45742026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads45752026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=045762026/11/12 11:46:15 INFO Vacuumed table table=closures45772026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period45782026/11/12 11:46:15 INFO Vacuumed table table=objects45792026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045802026/11/12 11:46:15 INFO Vacuumed table table=pending_closures45812026/11/12 11:46:15 INFO Received complete upload request method=POST path=/45822026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts45832026/11/12 11:46:15 INFO Completed upload id=20 objects=4 duration=1.26209ms45842026/11/12 11:46:15 INFO Vacuumed table table=pending_objects45852026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads45862026/11/12 11:46:15 INFO Received uploads request method=POST path=/45872026/11/12 11:46:15 INFO Vacuumed table table=closures45882026/11/12 11:46:15 INFO Received complete push request method=POST path=/45892026/11/12 11:46:15 INFO Vacuumed table table=objects45902026/11/12 11:46:15 INFO Completed push id=29 objects=2 duration=1.37877ms45912026/11/12 11:46:15 INFO Received complete push request method=POST path=/45922026/11/12 11:46:15 INFO Received complete upload request method=POST path=/45932026/11/12 11:46:15 INFO Completed push id=32 objects=3 duration=1.32914ms45942026/11/12 11:46:15 INFO Received push request method=POST path=/45952026/11/12 11:46:15 INFO Completed upload id=22 objects=7 duration=1.82788ms45962026/11/12 11:46:15 INFO Received complete upload request method=POST path=/45972026/11/12 11:46:15 INFO Received complete upload request method=POST path=/45982026/11/12 11:46:15 INFO Received complete push request method=POST path=/45992026/11/12 11:46:15 INFO Completed upload id=12 objects=6 duration=1.57367ms46002026/11/12 11:46:15 INFO Received complete push request method=POST path=/46012026/11/12 11:46:15 INFO Completed upload id=24 objects=8 duration=1.67178ms46022026/11/12 11:46:15 INFO Received uploads request method=POST path=/46032026/11/12 11:46:15 INFO Completed push id=16 objects=7 duration=2.12724ms46042026/11/12 11:46:15 INFO Completed push id=30 objects=4 duration=2.18347ms46052026/11/12 11:46:15 INFO Received complete upload request method=POST path=/46062026/11/12 11:46:15 INFO Completed upload id=23 objects=7 duration=1.08991ms46072026/11/12 11:46:15 INFO Received push request method=POST path=/46082026/11/12 11:46:15 INFO Received uploads request method=POST path=/46092026/11/12 11:46:15 INFO Received complete push request method=POST path=/46102026/11/12 11:46:15 INFO Received push request method=POST path=/46112026/11/12 11:46:15 INFO Received complete push request method=POST path=/46122026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=046132026/11/12 11:46:15 INFO Completed push id=31 objects=6 duration=2.28668ms46142026/11/12 11:46:15 INFO Completed push id=24 objects=4 duration=1.74676ms46152026/11/12 11:46:15 INFO Received complete upload request method=POST path=/46162026/11/12 11:46:15 INFO Received uploads request method=POST path=/46172026/11/12 11:46:15 INFO Received complete push request method=POST path=/46182026/11/12 11:46:15 INFO Completed upload id=15 objects=2 duration=1.44319ms46192026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period46202026/11/12 11:46:15 INFO Received complete push request method=POST path=/46212026/11/12 11:46:15 INFO Received complete upload request method=POST path=/46222026/11/12 11:46:15 INFO Received push request method=POST path=/46232026/11/12 11:46:15 INFO Received uploads request method=POST path=/46242026/11/12 11:46:15 INFO Completed push id=17 objects=3 duration=2.11827ms46252026/11/12 11:46:15 INFO Received uploads request method=POST path=/46262026/11/12 11:46:15 INFO Completed upload id=28 objects=5 duration=1.63696ms46272026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046282026/11/12 11:46:15 INFO Completed push id=23 objects=28 duration=3.6926ms46292026/11/12 11:46:15 INFO Vacuumed table table=pending_closures46302026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts46312026/11/12 11:46:15 INFO Vacuumed table table=pending_objects46322026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads46332026/11/12 11:46:15 INFO Vacuumed table table=closures46342026/11/12 11:46:15 INFO Vacuumed table table=objects46352026/11/12 11:46:15 INFO Received complete push request method=POST path=/46362026/11/12 11:46:15 INFO Completed push id=23 objects=6 duration=1.74689ms46372026/11/12 11:46:15 INFO Received complete push request method=POST path=/46382026/11/12 11:46:15 INFO Completed push id=19 objects=14 duration=1.73079ms46392026/11/12 11:46:15 INFO Received complete push request method=POST path=/46402026/11/12 11:46:15 INFO Completed push id=32 objects=9 duration=1.94737ms46412026/11/12 11:46:15 INFO Received push request method=POST path=/46422026/11/12 11:46:15 INFO Received push request method=POST path=/46432026/11/12 11:46:15 INFO Received complete push request method=POST path=/46442026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=046452026/11/12 11:46:15 INFO Completed push id=25 objects=9 duration=1.53599ms46462026/11/12 11:46:15 INFO Received complete push request method=POST path=/46472026/11/12 11:46:15 INFO Received complete push request method=POST path=/46482026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046492026/11/12 11:46:15 INFO Received complete upload request method=POST path=/46502026/11/12 11:46:15 INFO Completed push id=23 objects=5 duration=1.99468ms46512026/11/12 11:46:15 INFO Vacuumed table table=pending_closures46522026/11/12 11:46:15 INFO Completed upload id=30 objects=5 duration=1.44176ms46532026/11/12 11:46:15 INFO Completed push id=16 objects=14 duration=3.13339ms46542026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts46552026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=046562026/11/12 11:46:15 INFO Vacuumed table table=pending_objects46572026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads46582026/11/12 11:46:15 INFO Vacuumed table table=closures46592026/11/12 11:46:15 INFO Received complete push request method=POST path=/46602026/11/12 11:46:15 INFO Vacuumed table table=objects46612026/11/12 11:46:15 INFO Completed push id=24 objects=6 duration=1.2363ms46622026/11/12 11:46:15 INFO Received uploads request method=POST path=/46632026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=046642026/11/12 11:46:15 INFO Received complete upload request method=POST path=/46652026/11/12 11:46:15 INFO Completed upload id=24 objects=18 duration=1.33413ms46662026/11/12 11:46:15 INFO Received uploads request method=POST path=/46672026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046682026/11/12 11:46:15 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."46692026/11/12 11:46:15 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."46702026/11/12 11:46:15 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."46712026/11/12 11:46:15 INFO Vacuumed table table=pending_closures46722026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts46732026/11/12 11:46:15 INFO Received complete upload request method=POST path=/46742026/11/12 11:46:15 INFO Received push request method=POST path=/46752026/11/12 11:46:15 INFO Vacuumed table table=pending_objects46762026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads46772026/11/12 11:46:15 INFO Completed upload id=26 objects=3 duration=1.27097ms46782026/11/12 11:46:15 INFO Received complete upload request method=POST path=/46792026/11/12 11:46:15 INFO Vacuumed table table=closures46802026/11/12 11:46:15 INFO Completed upload id=27 objects=8 duration=1.40031ms46812026/11/12 11:46:15 INFO Received complete upload request method=POST path=/46822026/11/12 11:46:15 INFO Vacuumed table table=objects46832026/11/12 11:46:15 INFO Completed upload id=33 objects=9 duration=1.31819ms46842026/11/12 11:46:15 INFO Received complete upload request method=POST path=/46852026/11/12 11:46:15 INFO Completed upload id=24 objects=3 duration=1.07422ms46862026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=046872026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period46882026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046892026/11/12 11:46:15 INFO Received complete push request method=POST path=/46902026/11/12 11:46:15 INFO Vacuumed table table=pending_closures46912026/11/12 11:46:15 INFO Completed push id=31 objects=3 duration=1.22247ms46922026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts46932026/11/12 11:46:15 INFO Vacuumed table table=pending_objects46942026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads46952026/11/12 11:46:15 INFO Received complete push request method=POST path=/46962026/11/12 11:46:15 INFO Vacuumed table table=closures46972026/11/12 11:46:15 INFO Received push request method=POST path=/46982026/11/12 11:46:15 INFO Vacuumed table table=objects46992026/11/12 11:46:15 INFO Received complete push request method=POST path=/47002026/11/12 11:46:15 INFO Completed push id=25 objects=2 duration=2.18091ms47012026/11/12 11:46:15 INFO Completed push id=13 objects=2 duration=1.55675ms47022026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=047032026/11/12 11:46:15 INFO Received complete push request method=POST path=/47042026/11/12 11:46:15 INFO Received complete upload request method=POST path=/47052026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period47062026/11/12 11:46:15 INFO Received complete push request method=POST path=/47072026/11/12 11:46:15 INFO Received push request method=POST path=/47082026/11/12 11:46:15 INFO Completed upload id=20 objects=6 duration=1.352ms47092026/11/12 11:46:15 INFO Completed push id=32 objects=7 duration=1.27459ms47102026/11/12 11:46:15 INFO Completed push id=25 objects=7 duration=2.07844ms47112026/11/12 11:46:15 INFO Received complete push request method=POST path=/47122026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047132026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=047142026/11/12 11:46:15 INFO Vacuumed table table=pending_closures47152026/11/12 11:46:15 INFO Completed push id=25 objects=4 duration=1.74351ms47162026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts47172026/11/12 11:46:15 INFO Received complete upload request method=POST path=/47182026/11/12 11:46:15 INFO Vacuumed table table=pending_objects47192026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period47202026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads47212026/11/12 11:46:15 INFO Completed upload id=27 objects=7 duration=1.37396ms47222026/11/12 11:46:15 INFO Vacuumed table table=closures47232026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047242026/11/12 11:46:15 INFO Vacuumed table table=objects47252026/11/12 11:46:15 INFO Vacuumed table table=pending_closures47262026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts47272026/11/12 11:46:15 INFO Vacuumed table table=pending_objects47282026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads47292026/11/12 11:46:15 INFO Vacuumed table table=closures47302026/11/12 11:46:15 INFO Received push request method=POST path=/47312026/11/12 11:46:15 INFO Vacuumed table table=objects47322026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=047332026/11/12 11:46:15 INFO Received complete push request method=POST path=/47342026/11/12 11:46:15 INFO Received complete upload request method=POST path=/47352026/11/12 11:46:15 INFO Completed upload id=27 objects=13 duration=904.94µs47362026/11/12 11:46:15 INFO Completed push id=27 objects=4 duration=1.64584ms47372026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047382026/11/12 11:46:15 INFO Vacuumed table table=pending_closures47392026/11/12 11:46:15 INFO Received complete push request method=POST path=/47402026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts47412026/11/12 11:46:15 INFO Completed push id=26 objects=4 duration=998.11µs47422026/11/12 11:46:15 INFO Vacuumed table table=pending_objects47432026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads47442026/11/12 11:46:15 INFO Vacuumed table table=closures47452026/11/12 11:46:15 INFO Vacuumed table table=objects47462026/11/12 11:46:15 INFO Received complete push request method=POST path=/47472026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=047482026/11/12 11:46:15 INFO Completed push id=32 objects=6 duration=1.40081ms47492026/11/12 11:46:15 INFO Received complete upload request method=POST path=/47502026/11/12 11:46:15 INFO Received complete upload request method=POST path=/47512026/11/12 11:46:15 INFO Completed upload id=28 objects=3 duration=960.62µs47522026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047532026/11/12 11:46:15 INFO Received push request method=POST path=/47542026/11/12 11:46:15 INFO Completed upload id=19 objects=2 duration=3.16955ms47552026/11/12 11:46:15 INFO Vacuumed table table=pending_closures47562026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts47572026/11/12 11:46:15 INFO Vacuumed table table=pending_objects47582026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads47592026/11/12 11:46:15 INFO Vacuumed table table=closures47602026/11/12 11:46:15 INFO Received uploads request method=POST path=/47612026/11/12 11:46:15 INFO Vacuumed table table=objects47622026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=047632026/11/12 11:46:15 INFO Received uploads request method=POST path=/47642026/11/12 11:46:15 INFO Received complete upload request method=POST path=/47652026/11/12 11:46:15 INFO Completed upload id=28 objects=13 duration=836.66µs47662026/11/12 11:46:15 INFO Received push request method=POST path=/47672026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047682026/11/12 11:46:15 INFO Received complete upload request method=POST path=/47692026/11/12 11:46:15 INFO Vacuumed table table=pending_closures47702026/11/12 11:46:15 INFO Received complete upload request method=POST path=/47712026/11/12 11:46:15 INFO Received complete upload request method=POST path=/47722026/11/12 11:46:15 INFO Completed upload id=26 objects=24 duration=1.516ms47732026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts47742026/11/12 11:46:15 INFO Completed upload id=33 objects=7 duration=1.06646ms47752026/11/12 11:46:15 INFO Vacuumed table table=pending_objects47762026/11/12 11:46:15 INFO Received uploads request method=POST path=/47772026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads47782026/11/12 11:46:15 INFO Vacuumed table table=closures47792026/11/12 11:46:15 INFO Completed upload id=13 objects=14 duration=3.89319ms47802026/11/12 11:46:15 INFO Vacuumed table table=objects47812026/11/12 11:46:15 INFO Received complete push request method=POST path=/47822026/11/12 11:46:15 INFO Received complete push request method=POST path=/47832026/11/12 11:46:15 INFO Completed push id=28 objects=11 duration=2.64052ms47842026/11/12 11:46:15 INFO Completed push id=28 objects=26 duration=2.03809ms47852026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=047862026/11/12 11:46:15 INFO Received complete push request method=POST path=/47872026/11/12 11:46:15 INFO Received uploads request method=POST path=/47882026/11/12 11:46:15 INFO Completed push id=29 objects=23 duration=1.67165ms47892026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047902026/11/12 11:46:15 INFO Vacuumed table table=pending_closures47912026/11/12 11:46:15 INFO Received complete push request method=POST path=/47922026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts47932026/11/12 11:46:15 INFO Completed push id=29 objects=3 duration=816.6µs47942026/11/12 11:46:15 INFO Vacuumed table table=pending_objects47952026/11/12 11:46:15 INFO Received complete push request method=POST path=/47962026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads47972026/11/12 11:46:15 INFO Vacuumed table table=closures47982026/11/12 11:46:15 INFO Received uploads request method=POST path=/47992026/11/12 11:46:15 INFO Completed push id=26 objects=23 duration=1.48649ms48002026/11/12 11:46:15 INFO Vacuumed table table=objects48012026/11/12 11:46:15 INFO Received complete upload request method=POST path=/48022026/11/12 11:46:15 INFO Completed upload id=34 objects=3 duration=1.47068ms48032026/11/12 11:46:15 INFO Received complete push request method=POST path=/48042026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=048052026/11/12 11:46:15 INFO Received uploads request method=POST path=/48062026/11/12 11:46:15 INFO Completed push id=33 objects=7 duration=2.10292ms48072026/11/12 11:46:15 INFO Received complete push request method=POST path=/48082026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048092026/11/12 11:46:15 INFO Completed push id=30 objects=3 duration=1.10516ms48102026/11/12 11:46:15 INFO Vacuumed table table=pending_closures48112026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts48122026/11/12 11:46:15 INFO Vacuumed table table=pending_objects48132026/11/12 11:46:15 INFO Received complete push request method=POST path=/48142026/11/12 11:46:15 INFO Received complete upload request method=POST path=/48152026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads48162026/11/12 11:46:15 INFO Vacuumed table table=closures48172026/11/12 11:46:15 INFO Received complete upload request method=POST path=/48182026/11/12 11:46:15 INFO Completed push id=30 objects=6 duration=1.46301ms48192026/11/12 11:46:15 INFO Completed upload id=21 objects=2 duration=1.4826ms48202026/11/12 11:46:15 INFO Vacuumed table table=objects48212026/11/12 11:46:15 INFO Completed upload id=26 objects=8 duration=1.40256ms48222026/11/12 11:46:15 INFO Received complete upload request method=POST path=/48232026/11/12 11:46:15 INFO Received complete upload request method=POST path=/48242026/11/12 11:46:15 INFO Received complete upload request method=POST path=/48252026/11/12 11:46:15 INFO Received complete push request method=POST path=/48262026/11/12 11:46:15 INFO Received complete upload request method=POST path=/48272026/11/12 11:46:15 INFO Received push request method=POST path=/48282026/11/12 11:46:15 INFO Received complete push request method=POST path=/48292026/11/12 11:46:15 INFO Received uploads request method=POST path=/48302026/11/12 11:46:15 INFO Received complete push request method=POST path=/48312026/11/12 11:46:15 INFO Completed upload id=31 objects=2 duration=58.883048ms48322026/11/12 11:46:15 INFO Completed push id=32 objects=19 duration=48.416899ms48332026/11/12 11:46:15 INFO Completed upload id=25 objects=6 duration=56.74824ms48342026/11/12 11:46:15 INFO Completed push id=35 objects=11 duration=52.47511ms48352026/11/12 11:46:15 INFO Completed upload id=26 objects=3 duration=52.37145ms48362026/11/12 11:46:15 INFO Completed push id=28 objects=14 duration=41.73482ms48372026/11/12 11:46:15 INFO Completed upload id=25 objects=16 duration=56.02099ms48382026/11/12 11:46:15 INFO Received complete push request method=POST path=/48392026/11/12 11:46:15 INFO Received complete push request method=POST path=/48402026/11/12 11:46:15 INFO Completed push id=33 objects=3 duration=1.5374ms48412026/11/12 11:46:15 INFO Completed push id=31 objects=6 duration=3.51344ms48422026/11/12 11:46:15 INFO Received complete upload request method=POST path=/48432026/11/12 11:46:15 INFO Completed upload id=27 objects=26 duration=1.66307ms48442026/11/12 11:46:15 INFO Received complete push request method=POST path=/48452026/11/12 11:46:15 INFO Completed push id=27 objects=15 duration=1.76143ms48462026/11/12 11:46:15 INFO Received complete push request method=POST path=/48472026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=048482026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period48492026/11/12 11:46:15 INFO Completed push id=29 objects=24 duration=1.87333ms48502026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048512026/11/12 11:46:15 INFO Vacuumed table table=pending_closures48522026/11/12 11:46:15 INFO Received complete push request method=POST path=/48532026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts48542026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=048552026/11/12 11:46:15 INFO Completed push id=33 objects=6 duration=1.51269ms48562026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=048572026/11/12 11:46:15 INFO Vacuumed table table=pending_objects48582026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads48592026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=048602026/11/12 11:46:15 INFO Received push request method=POST path=/48612026/11/12 11:46:15 INFO Received push request method=POST path=/48622026/11/12 11:46:15 INFO Received complete push request method=POST path=/48632026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period48642026/11/12 11:46:15 INFO Vacuumed table table=closures48652026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=048662026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period48672026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period48682026/11/12 11:46:15 INFO Completed push id=34 objects=6 duration=1.4697ms48692026/11/12 11:46:15 INFO Vacuumed table table=objects48702026/11/12 11:46:15 INFO Received uploads request method=POST path=/48712026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048722026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period48732026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048742026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048752026/11/12 11:46:15 INFO Vacuumed table table=pending_closures48762026/11/12 11:46:15 INFO Vacuumed table table=pending_closures48772026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts48782026/11/12 11:46:15 INFO Vacuumed table table=pending_closures48792026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=6 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048802026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts48812026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts48822026/11/12 11:46:15 INFO Vacuumed table table=pending_objects48832026/11/12 11:46:15 INFO Received complete push request method=POST path=/48842026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads48852026/11/12 11:46:15 INFO Vacuumed table table=pending_closures48862026/11/12 11:46:15 INFO Vacuumed table table=pending_objects48872026/11/12 11:46:15 INFO Vacuumed table table=pending_objects48882026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads48892026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads48902026/11/12 11:46:15 INFO Vacuumed table table=closures48912026/11/12 11:46:15 INFO Completed push id=28 objects=6 duration=1.73425ms48922026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts48932026/11/12 11:46:15 INFO Vacuumed table table=closures48942026/11/12 11:46:15 INFO Vacuumed table table=closures48952026/11/12 11:46:15 INFO Vacuumed table table=objects48962026/11/12 11:46:15 INFO Vacuumed table table=objects48972026/11/12 11:46:15 INFO Vacuumed table table=objects48982026/11/12 11:46:15 INFO Vacuumed table table=pending_objects48992026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads49002026/11/12 11:46:15 INFO Vacuumed table table=closures49012026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=049022026/11/12 11:46:15 INFO Vacuumed table table=objects49032026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period49042026/11/12 11:46:15 INFO Received push request method=POST path=/49052026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=049062026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period49072026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=049082026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049092026/11/12 11:46:15 INFO Vacuumed table table=pending_closures49102026/11/12 11:46:15 INFO Vacuumed table table=pending_closures49112026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts49122026/11/12 11:46:15 INFO Received complete push request method=POST path=/49132026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts49142026/11/12 11:46:15 INFO Received uploads request method=POST path=/49152026/11/12 11:46:15 INFO Received uploads request method=POST path=/49162026/11/12 11:46:15 INFO Completed push id=35 objects=8 duration=1.49802ms49172026/11/12 11:46:15 INFO Vacuumed table table=pending_objects49182026/11/12 11:46:15 INFO Vacuumed table table=pending_objects49192026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads49202026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads49212026/11/12 11:46:15 INFO Vacuumed table table=closures49222026/11/12 11:46:15 INFO Vacuumed table table=closures49232026/11/12 11:46:15 INFO Vacuumed table table=objects49242026/11/12 11:46:15 INFO Vacuumed table table=objects49252026/11/12 11:46:15 INFO Received complete push request method=POST path=/49262026/11/12 11:46:15 INFO Completed push id=29 objects=6 duration=1.287409ms49272026/11/12 11:46:15 INFO Received push request method=POST path=/49282026/11/12 11:46:15 INFO Received uploads request method=POST path=/49292026/11/12 11:46:15 INFO Received complete push request method=POST path=/49302026/11/12 11:46:15 INFO Completed push id=28 objects=19 duration=913.4µs49312026/11/12 11:46:15 INFO Received uploads request method=POST path=/49322026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=049332026/11/12 11:46:15 INFO Received complete upload request method=POST path=/49342026/11/12 11:46:15 INFO Completed upload id=36 objects=9 duration=1.20616ms49352026/11/12 11:46:15 INFO Received uploads request method=POST path=/49362026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period49372026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049382026/11/12 11:46:15 INFO Received push request method=POST path=/49392026/11/12 11:46:15 INFO Vacuumed table table=pending_closures49402026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts49412026/11/12 11:46:15 INFO Received complete push request method=POST path=/49422026/11/12 11:46:15 INFO Vacuumed table table=pending_objects49432026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads49442026/11/12 11:46:15 INFO Vacuumed table table=closures49452026/11/12 11:46:15 INFO Completed push id=11 objects=9 duration=1.96255ms49462026/11/12 11:46:15 INFO Vacuumed table table=objects49472026/11/12 11:46:15 INFO Received complete push request method=POST path=/49482026/11/12 11:46:15 INFO Received complete upload request method=POST path=/49492026/11/12 11:46:15 INFO Completed push id=9 objects=9 duration=3.28467ms49502026/11/12 11:46:15 INFO Completed upload id=29 objects=4 duration=2.03715ms49512026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=049522026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period49532026/11/12 11:46:15 INFO Received complete upload request method=POST path=/49542026/11/12 11:46:15 INFO Received push request method=POST path=/49552026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049562026/11/12 11:46:15 INFO Completed upload id=30 objects=8 duration=1.28376ms49572026/11/12 11:46:15 INFO Received complete push request method=POST path=/49582026/11/12 11:46:15 INFO Completed push id=29 objects=10 duration=1.00446ms49592026/11/12 11:46:15 INFO Vacuumed table table=pending_closures49602026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts49612026/11/12 11:46:15 INFO Vacuumed table table=pending_objects49622026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads49632026/11/12 11:46:15 INFO Received uploads request method=POST path=/49642026/11/12 11:46:15 INFO Vacuumed table table=closures49652026/11/12 11:46:15 INFO Vacuumed table table=objects49662026/11/12 11:46:15 INFO Received complete push request method=POST path=/49672026/11/12 11:46:15 INFO Received complete push request method=POST path=/49682026/11/12 11:46:15 INFO Completed push id=1 objects=9 duration=1.47455ms49692026/11/12 11:46:15 INFO Received uploads request method=POST path=/49702026/11/12 11:46:15 INFO Completed push id=15 objects=24 duration=1.65525ms49712026/11/12 11:46:15 INFO Received complete upload request method=POST path=/49722026/11/12 11:46:15 INFO Completed upload id=21 objects=8 duration=1.196789ms49732026/11/12 11:46:15 INFO Received complete push request method=POST path=/49742026/11/12 11:46:15 INFO Completed push id=24 objects=24 duration=3.480609ms49752026/11/12 11:46:15 INFO Received complete upload request method=POST path=/49762026/11/12 11:46:15 INFO Completed upload id=31 objects=6 duration=1.33994ms49772026/11/12 11:46:15 INFO Received complete upload request method=POST path=/49782026/11/12 11:46:15 INFO Received complete upload request method=POST path=/49792026/11/12 11:46:15 INFO Completed upload id=30 objects=6 duration=739.85µs49802026/11/12 11:46:15 INFO Completed upload id=21 objects=9 duration=2.20799ms49812026/11/12 11:46:15 INFO Received complete upload request method=POST path=/49822026/11/12 11:46:15 INFO Received complete push request method=POST path=/49832026/11/12 11:46:15 INFO Completed upload id=19 objects=2 duration=1.1349ms49842026/11/12 11:46:15 INFO Received uploads request method=POST path=/49852026/11/12 11:46:15 INFO Completed push id=22 objects=6 duration=1.60065ms49862026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=049872026/11/12 11:46:15 INFO Received uploads request method=POST path=/49882026/11/12 11:46:15 INFO Received complete upload request method=POST path=/49892026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period49902026/11/12 11:46:15 INFO Completed upload id=32 objects=4 duration=972.5µs49912026/11/12 11:46:15 INFO Received complete upload request method=POST path=/49922026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049932026/11/12 11:46:15 INFO Completed upload id=31 objects=5 duration=628.35µs49942026/11/12 11:46:15 INFO Vacuumed table table=pending_closures49952026/11/12 11:46:15 INFO Received complete push request method=POST path=/49962026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts49972026/11/12 11:46:15 INFO Received complete push request method=POST path=/49982026/11/12 11:46:15 INFO Completed push id=24 objects=8 duration=1.34152ms49992026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50002026/11/12 11:46:15 INFO Completed push id=25 objects=10 duration=2.45766ms50012026/11/12 11:46:15 INFO Vacuumed table table=pending_objects50022026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads50032026/11/12 11:46:15 INFO Completed upload id=24 objects=7 duration=1.13541ms50042026/11/12 11:46:15 INFO Vacuumed table table=closures50052026/11/12 11:46:15 INFO Vacuumed table table=objects50062026/11/12 11:46:15 INFO Received complete push request method=POST path=/50072026/11/12 11:46:15 INFO Completed push id=11 objects=6 duration=1.8876ms50082026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50092026/11/12 11:46:15 INFO Completed upload id=22 objects=5 duration=1.95671ms50102026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50112026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50122026/11/12 11:46:15 INFO Completed upload id=25 objects=9 duration=1.24108ms50132026/11/12 11:46:15 INFO Completed upload id=34 objects=3 duration=1.39591ms50142026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50152026/11/12 11:46:15 INFO Completed upload id=29 objects=5 duration=1.14851ms50162026/11/12 11:46:15 INFO Received complete push request method=POST path=/50172026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50182026/11/12 11:46:15 INFO Completed push id=26 objects=3 duration=1.13414ms50192026/11/12 11:46:15 INFO Received uploads request method=POST path=/50202026/11/12 11:46:15 INFO Completed upload id=33 objects=10 duration=1.24904ms50212026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50222026/11/12 11:46:15 INFO Completed upload id=2 objects=7 duration=1.23292ms50232026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50242026/11/12 11:46:15 INFO Received complete push request method=POST path=/50252026/11/12 11:46:15 INFO Received complete push request method=POST path=/50262026/11/12 11:46:15 INFO Completed upload id=27 objects=3 duration=1.094309ms50272026/11/12 11:46:15 INFO Completed push id=28 objects=3 duration=1.915699ms50282026/11/12 11:46:15 INFO Completed push id=20 objects=13 duration=1.56599ms50292026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=050302026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50312026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50322026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period50332026/11/12 11:46:15 INFO Completed upload id=30 objects=2 duration=1.16855ms50342026/11/12 11:46:15 INFO Completed upload id=28 objects=8 duration=1.48046ms50352026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050362026/11/12 11:46:15 INFO Vacuumed table table=pending_closures50372026/11/12 11:46:15 INFO Received uploads request method=POST path=/50382026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts50392026/11/12 11:46:15 INFO Vacuumed table table=pending_objects50402026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads50412026/11/12 11:46:15 INFO Vacuumed table table=closures50422026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50432026/11/12 11:46:15 INFO Vacuumed table table=objects50442026/11/12 11:46:15 INFO Completed upload id=28 objects=6 duration=1.15219ms50452026/11/12 11:46:15 INFO Received complete push request method=POST path=/50462026/11/12 11:46:15 INFO Received complete push request method=POST path=/50472026/11/12 11:46:15 INFO Completed push id=29 objects=15 duration=1.41043ms50482026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50492026/11/12 11:46:15 INFO Completed push id=22 objects=4 duration=1.27606ms50502026/11/12 11:46:15 INFO Completed upload id=34 objects=23 duration=1.36821ms50512026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50522026/11/12 11:46:15 INFO Received complete push request method=POST path=/50532026/11/12 11:46:15 INFO Completed upload id=29 objects=10 duration=1.15261ms50542026/11/12 11:46:15 INFO Completed push id=31 objects=20 duration=1.42897ms50552026/11/12 11:46:15 INFO Received complete push request method=POST path=/50562026/11/12 11:46:15 INFO Completed push id=30 objects=4 duration=1.11967ms50572026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50582026/11/12 11:46:15 INFO Completed upload id=8 objects=6 duration=1.29827ms50592026/11/12 11:46:15 INFO Received complete push request method=POST path=/50602026/11/12 11:46:15 INFO Completed push id=30 objects=6 duration=1.35913ms50612026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50622026/11/12 11:46:15 INFO Completed upload id=30 objects=20 duration=1.13603ms50632026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50642026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50652026/11/12 11:46:15 INFO Completed upload id=32 objects=5 duration=1.00332ms50662026/11/12 11:46:15 INFO Received complete push request method=POST path=/50672026/11/12 11:46:15 INFO Completed upload id=23 objects=7 duration=1.15912ms50682026/11/12 11:46:15 INFO Received push request method=POST path=/50692026/11/12 11:46:15 INFO Completed push id=35 objects=6 duration=1.41205ms50702026/11/12 11:46:15 INFO Received complete push request method=POST path=/50712026/11/12 11:46:15 INFO Completed push id=31 objects=7 duration=1.2245ms50722026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50732026/11/12 11:46:15 INFO Completed upload id=31 objects=3 duration=1.19131ms50742026/11/12 11:46:15 INFO Received complete push request method=POST path=/50752026/11/12 11:46:15 INFO Completed push id=31 objects=11 duration=1.26579ms50762026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50772026/11/12 11:46:15 INFO Received complete push request method=POST path=/50782026/11/12 11:46:15 INFO Completed upload id=33 objects=3 duration=939.01µs50792026/11/12 11:46:15 INFO Completed push id=24 objects=5 duration=1.30131ms50802026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=050812026/11/12 11:46:15 INFO Received uploads request method=POST path=/50822026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50832026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period50842026/11/12 11:46:15 INFO Completed upload id=32 objects=5 duration=1.46045ms50852026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=050862026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=050872026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50882026/11/12 11:46:15 INFO Vacuumed table table=pending_closures50892026/11/12 11:46:15 INFO Completed upload id=32 objects=2 duration=955.03µs50902026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period50912026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts50922026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=050932026/11/12 11:46:15 INFO Received complete upload request method=POST path=/50942026/11/12 11:46:15 INFO Vacuumed table table=pending_objects50952026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050962026/11/12 11:46:15 INFO Completed upload id=25 objects=2 duration=986.2µs50972026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads50982026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period50992026/11/12 11:46:15 INFO Vacuumed table table=closures51002026/11/12 11:46:15 INFO Vacuumed table table=pending_closures51012026/11/12 11:46:15 INFO Vacuumed table table=objects51022026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051032026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts51042026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=051052026/11/12 11:46:15 INFO Vacuumed table table=pending_closures51062026/11/12 11:46:15 INFO Vacuumed table table=pending_objects51072026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads51082026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts51092026/11/12 11:46:15 INFO Vacuumed table table=closures51102026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period51112026/11/12 11:46:15 INFO Vacuumed table table=pending_objects51122026/11/12 11:46:15 INFO Vacuumed table table=objects51132026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51142026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads51152026/11/12 11:46:15 INFO Vacuumed table table=closures51162026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051172026/11/12 11:46:15 INFO Completed upload id=12 objects=9 duration=1.34655ms51182026/11/12 11:46:15 INFO Vacuumed table table=objects51192026/11/12 11:46:15 INFO Vacuumed table table=pending_closures51202026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts51212026/11/12 11:46:15 INFO Vacuumed table table=pending_objects51222026/11/12 11:46:15 INFO Received complete push request method=POST path=/51232026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads51242026/11/12 11:46:15 INFO Completed push id=33 objects=3 duration=1.1365ms51252026/11/12 11:46:15 INFO Vacuumed table table=closures51262026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51272026/11/12 11:46:15 INFO Vacuumed table table=objects51282026/11/12 11:46:15 INFO Completed upload id=26 objects=20 duration=929.21µs51292026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51302026/11/12 11:46:15 INFO Completed upload id=34 objects=3 duration=1.01781ms51312026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51322026/11/12 11:46:15 INFO Received complete push request method=POST path=/51332026/11/12 11:46:15 INFO Completed upload id=34 objects=24 duration=1.54566ms51342026/11/12 11:46:15 INFO Completed push id=28 objects=7 duration=1.168549ms51352026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=051362026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period51372026/11/12 11:46:15 INFO Received push request method=POST path=/51382026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051392026/11/12 11:46:15 INFO Vacuumed table table=pending_closures51402026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts51412026/11/12 11:46:15 INFO Vacuumed table table=pending_objects51422026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=051432026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads51442026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period51452026/11/12 11:46:15 INFO Vacuumed table table=closures51462026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051472026/11/12 11:46:15 INFO Vacuumed table table=objects51482026/11/12 11:46:15 INFO Vacuumed table table=pending_closures51492026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts51502026/11/12 11:46:15 INFO Vacuumed table table=pending_objects51512026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads51522026/11/12 11:46:15 INFO Vacuumed table table=closures51532026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51542026/11/12 11:46:15 INFO Vacuumed table table=objects51552026/11/12 11:46:15 INFO Completed upload id=30 objects=6 duration=1.31913ms51562026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51572026/11/12 11:46:15 INFO Completed upload id=23 objects=7 duration=2.78617ms51582026/11/12 11:46:15 INFO Received complete push request method=POST path=/51592026/11/12 11:46:15 INFO Completed push id=31 objects=5 duration=1.2589ms51602026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51612026/11/12 11:46:15 INFO Completed upload id=26 objects=5 duration=1.04449ms51622026/11/12 11:46:15 INFO Received complete push request method=POST path=/51632026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51642026/11/12 11:46:15 INFO Received complete push request method=POST path=/51652026/11/12 11:46:15 INFO Completed push id=32 objects=26 duration=1.57358ms51662026/11/12 11:46:15 INFO Completed upload id=18 objects=22 duration=1.80103ms51672026/11/12 11:46:15 INFO Completed push id=27 objects=3 duration=1.89484ms51682026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51692026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51702026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51712026/11/12 11:46:15 INFO Completed upload id=33 objects=2 duration=1.10706ms51722026/11/12 11:46:15 INFO Completed upload id=28 objects=5 duration=1.41814ms51732026/11/12 11:46:15 INFO Completed upload id=21 objects=8 duration=1.51835ms51742026/11/12 11:46:15 INFO Received complete push request method=POST path=/51752026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51762026/11/12 11:46:15 INFO Completed push id=29 objects=17 duration=1.61657ms51772026/11/12 11:46:15 INFO Completed upload id=34 objects=5 duration=1.17936ms51782026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51792026/11/12 11:46:15 INFO Completed upload id=22 objects=7 duration=997.26µs51802026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=051812026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51822026/11/12 11:46:15 INFO Completed upload id=35 objects=11 duration=1.14122ms51832026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period51842026/11/12 11:46:15 INFO Received complete push request method=POST path=/51852026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051862026/11/12 11:46:15 INFO Completed push id=24 objects=5 duration=1.16822ms51872026/11/12 11:46:15 INFO Vacuumed table table=pending_closures51882026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts51892026/11/12 11:46:15 INFO Vacuumed table table=pending_objects51902026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads51912026/11/12 11:46:15 INFO Vacuumed table table=closures51922026/11/12 11:46:15 INFO Vacuumed table table=objects51932026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51942026/11/12 11:46:15 INFO Completed upload id=36 objects=3 duration=1.11355ms51952026/11/12 11:46:15 INFO Received complete upload request method=POST path=/51962026/11/12 11:46:15 INFO Completed upload id=25 objects=8 duration=1.16361ms51972026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=051982026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period51992026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052002026/11/12 11:46:15 INFO Vacuumed table table=pending_closures52012026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts52022026/11/12 11:46:15 INFO Vacuumed table table=pending_objects52032026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads52042026/11/12 11:46:15 INFO Received complete push request method=POST path=/52052026/11/12 11:46:15 INFO Vacuumed table table=closures52062026/11/12 11:46:15 INFO Completed push id=26 objects=4 duration=1.26129ms52072026/11/12 11:46:15 INFO Vacuumed table table=objects52082026/11/12 11:46:15 INFO Received complete push request method=POST path=/52092026/11/12 11:46:15 INFO Completed push id=30 objects=2 duration=1.12988ms52102026/11/12 11:46:15 INFO Received complete push request method=POST path=/52112026/11/12 11:46:15 INFO Completed push id=32 objects=7 duration=1.40775ms52122026/11/12 11:46:15 INFO Received complete upload request method=POST path=/52132026/11/12 11:46:15 INFO Completed upload id=33 objects=6 duration=1.08763ms52142026/11/12 11:46:15 INFO Received complete upload request method=POST path=/52152026/11/12 11:46:15 INFO Completed upload id=34 objects=3 duration=987.24µs52162026/11/12 11:46:15 INFO Aborted multipart uploads count=0 kept=052172026/11/12 11:46:15 WARN Force mode enabled - objects will be deleted immediately without grace period52182026/11/12 11:46:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052192026/11/12 11:46:15 INFO Vacuumed table table=pending_closures52202026/11/12 11:46:15 INFO Vacuumed table table=commit_receipts52212026/11/12 11:46:15 INFO Vacuumed table table=pending_objects52222026/11/12 11:46:15 INFO Vacuumed table table=multipart_uploads52232026/11/12 11:46:15 INFO Vacuumed table table=closures52242026/11/12 11:46:15 INFO Vacuumed table table=objects5225--- PASS: FuzzPushGC (0.00s)5226 --- PASS: FuzzPushGC/seed#3 (2.94s)5227 --- PASS: FuzzPushGC/seed#0 (3.01s)5228 --- PASS: FuzzPushGC/seed#7 (3.19s)5229 --- PASS: FuzzPushGC/seed#8 (3.19s)5230 --- PASS: FuzzPushGC/seed#14 (3.20s)5231 --- PASS: FuzzPushGC/seed#2 (3.20s)5232 --- PASS: FuzzPushGC/seed#4 (3.25s)5233 --- PASS: FuzzPushGC/seed#13 (3.33s)5234 --- PASS: FuzzPushGC/seed#12 (3.39s)5235 --- PASS: FuzzPushGC/seed#15 (3.40s)5236 --- PASS: FuzzPushGC/seed#6 (3.40s)5237 --- PASS: FuzzPushGC/seed#5 (3.44s)5238 --- PASS: FuzzPushGC/seed#9 (3.45s)5239 --- PASS: FuzzPushGC/seed#1 (3.57s)5240 --- PASS: FuzzPushGC/seed#11 (3.60s)5241 --- PASS: FuzzPushGC/seed#10 (3.75s)5242PASS5243{"timestamp":"2026-11-12T11:46:15.952335187Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:32910","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(399)"}5244{"timestamp":"2026-11-12T11:46:15.952483227Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:34410","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(389)"}5245{"timestamp":"2026-11-12T11:46:15.952664527Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:34356","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(392)"}5246{"timestamp":"2026-11-12T11:46:15.952889297Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:33586","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(395)"}5247{"timestamp":"2026-11-12T11:46:15.952981017Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:34350","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(399)"}52482026-11-12 11:46:16.386 UTC [131] LOG: received smart shutdown request52492026-11-12 11:46:16.390 UTC [131] LOG: background worker "logical replication launcher" (PID 141) exited with exit code 152502026-11-12 11:46:17.289 UTC [136] LOG: checkpoint complete: wrote 3770 buffers (23.0%), wrote 5 SLRU buffers; 0 WAL file(s) added, 0 removed, 33 recycled; write=16.381 s, sync=0.741 s, total=17.213 s; sync files=43130, longest=0.001 s, average=0.001 s; distance=533612 kB, estimate=533612 kB; lsn=0/275412F0, redo lsn=0/22080F4052512026-11-12 11:46:17.292 UTC [136] LOG: shutting down52522026-11-12 11:46:17.292 UTC [136] LOG: checkpoint starting: shutdown immediate52532026-11-12 11:46:18.002 UTC [136] LOG: checkpoint complete: wrote 9981 buffers (60.9%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 5 recycled; write=0.180 s, sync=0.513 s, total=0.711 s; sync files=3116, longest=0.014 s, average=0.001 s; distance=86785 kB, estimate=488929 kB; lsn=0/27541368, redo lsn=0/2754136852542026-11-12 11:46:18.090 UTC [131] LOG: database system is shut down5255Running OIDC tests...5256=== RUN TestAudienceForIssuer5257=== PAUSE TestAudienceForIssuer5258=== RUN TestProviderClientGivesUpOnSilentIssuer5259=== PAUSE TestProviderClientGivesUpOnSilentIssuer5260=== RUN TestGlobMatch5261=== PAUSE TestGlobMatch5262=== RUN TestValidateToken_ValidToken5263=== PAUSE TestValidateToken_ValidToken5264=== RUN TestValidateToken_WrongAudience5265=== PAUSE TestValidateToken_WrongAudience5266=== RUN TestValidateToken_Expired5267=== PAUSE TestValidateToken_Expired5268=== RUN TestValidateToken_BoundClaimsMismatch5269=== PAUSE TestValidateToken_BoundClaimsMismatch5270=== RUN TestValidateToken_BoundSubjectMismatch5271=== PAUSE TestValidateToken_BoundSubjectMismatch5272=== RUN TestValidateToken_MultipleProviders5273=== PAUSE TestValidateToken_MultipleProviders5274=== RUN TestValidateToken_UnknownIssuerFetchesNoKeys5275=== PAUSE TestValidateToken_UnknownIssuerFetchesNoKeys5276=== RUN TestValidateToken_GoogleSchemelessIssuer5277=== PAUSE TestValidateToken_GoogleSchemelessIssuer5278=== RUN TestValidateToken_KubernetesServiceAccount5279=== PAUSE TestValidateToken_KubernetesServiceAccount5280=== RUN TestNewValidator_KubernetesRequiresCA5281=== PAUSE TestNewValidator_KubernetesRequiresCA5282=== RUN TestValidateToken_KubernetesIssuerFromOwnToken5283=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken5284=== RUN TestPins_ReservedForMatchingRule5285=== PAUSE TestPins_ReservedForMatchingRule5286=== RUN TestPins_TopLevelShorthand5287=== PAUSE TestPins_TopLevelShorthand5288=== RUN TestPins_ConfigValidation5289=== PAUSE TestPins_ConfigValidation5290=== RUN TestScopes_LegacyProviderDefaultsToWrite5291=== PAUSE TestScopes_LegacyProviderDefaultsToWrite5292=== RUN TestScopes_Rules5293=== PAUSE TestScopes_Rules5294=== RUN TestScopes_ConfigValidation5295=== PAUSE TestScopes_ConfigValidation5296=== CONT TestAudienceForIssuer5297=== CONT TestPins_TopLevelShorthand5298--- PASS: TestAudienceForIssuer (0.00s)5299=== CONT TestValidateToken_WrongAudience5300=== CONT TestValidateToken_ValidToken5301=== CONT TestGlobMatch5302=== RUN TestGlobMatch/foo_foo5303=== PAUSE TestGlobMatch/foo_foo5304=== RUN TestGlobMatch/foo_bar5305=== PAUSE TestGlobMatch/foo_bar5306=== RUN TestGlobMatch/*_5307=== CONT TestProviderClientGivesUpOnSilentIssuer5308=== CONT TestValidateToken_MultipleProviders5309=== CONT TestValidateToken_GoogleSchemelessIssuer5310=== CONT TestValidateToken_UnknownIssuerFetchesNoKeys5311=== CONT TestScopes_Rules5312=== CONT TestPins_ReservedForMatchingRule5313=== CONT TestScopes_ConfigValidation5314=== CONT TestValidateToken_KubernetesIssuerFromOwnToken5315=== CONT TestNewValidator_KubernetesRequiresCA5316=== CONT TestValidateToken_BoundSubjectMismatch5317=== CONT TestValidateToken_KubernetesServiceAccount5318=== CONT TestValidateToken_BoundClaimsMismatch5319=== CONT TestScopes_LegacyProviderDefaultsToWrite5320=== CONT TestPins_ConfigValidation5321=== CONT TestValidateToken_Expired5322=== PAUSE TestGlobMatch/*_5323=== RUN TestGlobMatch/*_anything5324=== PAUSE TestGlobMatch/*_anything5325=== RUN TestGlobMatch/foo*_foo5326=== PAUSE TestGlobMatch/foo*_foo53272026/11/12 11:46:20 INFO OIDC provider initialized name=known issuer=https://known.example5328=== RUN TestGlobMatch/foo*_foobar5329=== PAUSE TestGlobMatch/foo*_foobar5330=== RUN TestGlobMatch/foo*_bar5331=== PAUSE TestGlobMatch/foo*_bar5332=== RUN TestGlobMatch/*bar_bar5333=== PAUSE TestGlobMatch/*bar_bar5334=== RUN TestGlobMatch/*bar_foobar5335=== PAUSE TestGlobMatch/*bar_foobar5336=== RUN TestGlobMatch/*bar_foo5337=== PAUSE TestGlobMatch/*bar_foo5338=== RUN TestGlobMatch/foo*bar_foobar5339=== PAUSE TestGlobMatch/foo*bar_foobar5340=== RUN TestGlobMatch/foo*bar_foo123bar5341=== PAUSE TestGlobMatch/foo*bar_foo123bar5342=== RUN TestGlobMatch/foo*bar_foobarbaz5343=== PAUSE TestGlobMatch/foo*bar_foobarbaz5344=== RUN TestGlobMatch/*/*_foo/bar5345=== PAUSE TestGlobMatch/*/*_foo/bar5346=== RUN TestGlobMatch/*/*_foo5347=== PAUSE TestGlobMatch/*/*_foo5348=== RUN TestGlobMatch/refs/heads/*_refs/heads/main5349=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main5350=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.05351=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.05352=== RUN TestGlobMatch/refs/*/main_refs/heads/main5353=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main5354=== RUN TestGlobMatch/fo?_foo5355=== PAUSE TestGlobMatch/fo?_foo5356=== RUN TestGlobMatch/fo?_fo5357=== PAUSE TestGlobMatch/fo?_fo5358=== RUN TestGlobMatch/fo?_fooo5359=== PAUSE TestGlobMatch/fo?_fooo5360=== RUN TestGlobMatch/?oo_foo5361=== PAUSE TestGlobMatch/?oo_foo5362=== RUN TestGlobMatch/?oo_boo5363=== PAUSE TestGlobMatch/?oo_boo5364=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5365=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5366=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5367=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5368=== CONT TestGlobMatch/foo_foo5369=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5370=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5371=== CONT TestGlobMatch/?oo_boo5372=== CONT TestGlobMatch/?oo_foo5373=== CONT TestGlobMatch/fo?_fooo5374=== CONT TestGlobMatch/fo?_fo5375=== CONT TestGlobMatch/fo?_foo5376=== CONT TestGlobMatch/refs/*/main_refs/heads/main5377=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.05378=== CONT TestGlobMatch/refs/heads/*_refs/heads/main5379=== CONT TestGlobMatch/*/*_foo5380=== CONT TestGlobMatch/*/*_foo/bar5381=== CONT TestGlobMatch/foo*bar_foobarbaz5382=== CONT TestGlobMatch/foo*bar_foo123bar5383=== CONT TestGlobMatch/foo*bar_foobar5384=== CONT TestGlobMatch/*bar_foo5385=== CONT TestGlobMatch/*bar_foobar5386=== CONT TestGlobMatch/*bar_bar5387=== CONT TestGlobMatch/foo*_bar5388--- PASS: TestProviderClientGivesUpOnSilentIssuer (0.00s)5389=== CONT TestGlobMatch/*_5390=== CONT TestGlobMatch/foo*_foo5391=== CONT TestGlobMatch/*_anything5392=== CONT TestGlobMatch/foo*_foobar5393=== CONT TestGlobMatch/foo_bar5394--- PASS: TestScopes_ConfigValidation (0.00s)5395--- PASS: TestPins_ConfigValidation (0.00s)5396--- PASS: TestGlobMatch (0.01s)5397 --- PASS: TestGlobMatch/foo_foo (0.00s)5398 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)5399 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)5400 --- PASS: TestGlobMatch/?oo_boo (0.00s)5401 --- PASS: TestGlobMatch/?oo_foo (0.00s)5402 --- PASS: TestGlobMatch/fo?_fooo (0.00s)5403 --- PASS: TestGlobMatch/fo?_fo (0.00s)5404 --- PASS: TestGlobMatch/fo?_foo (0.00s)5405 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)5406 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)5407 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)5408 --- PASS: TestGlobMatch/*/*_foo (0.00s)5409 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)5410 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)5411 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)5412 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)5413 --- PASS: TestGlobMatch/*bar_foo (0.00s)5414 --- PASS: TestGlobMatch/*bar_foobar (0.00s)5415 --- PASS: TestGlobMatch/*bar_bar (0.00s)5416 --- PASS: TestGlobMatch/*_ (0.00s)5417 --- PASS: TestGlobMatch/foo*_bar (0.00s)5418 --- PASS: TestGlobMatch/foo*_foo (0.00s)5419 --- PASS: TestGlobMatch/*_anything (0.00s)5420 --- PASS: TestGlobMatch/foo*_foobar (0.00s)5421 --- PASS: TestGlobMatch/foo_bar (0.00s)54222026/11/12 11:46:20 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:43983/oidc54232026/11/12 11:46:20 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:33199/oidc5424--- PASS: TestValidateToken_ValidToken (0.03s)5425--- PASS: TestPins_TopLevelShorthand (0.03s)54262026/11/12 11:46:20 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:42069/oidc54272026/11/12 11:46:20 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:42747/oidc54282026/11/12 11:46:20 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:37949/oidc5429--- PASS: TestValidateToken_BoundClaimsMismatch (0.04s)5430--- PASS: TestValidateToken_WrongAudience (0.05s)5431--- PASS: TestPins_ReservedForMatchingRule (0.05s)54322026/11/12 11:46:20 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:36943/oidc54332026/11/12 11:46:20 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:32969/oidc54342026/11/12 11:46:20 http: TLS handshake error from 127.0.0.1:54936: remote error: tls: bad certificate5435--- PASS: TestNewValidator_KubernetesRequiresCA (0.05s)5436--- PASS: TestValidateToken_BoundSubjectMismatch (0.05s)54372026/11/12 11:46:20 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC1235438--- PASS: TestScopes_Rules (0.06s)5439--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (0.06s)5440--- PASS: TestValidateToken_UnknownIssuerFetchesNoKeys (0.06s)54412026/11/12 11:46:20 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:42623/oidc54422026/11/12 11:46:20 INFO OIDC provider initialized name=google issuer=https://accounts.google.com5443--- PASS: TestValidateToken_Expired (0.08s)5444--- PASS: TestValidateToken_GoogleSchemelessIssuer (0.08s)54452026/11/12 11:46:20 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:38727/oidc54462026/11/12 11:46:20 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:46515/oidc5447--- PASS: TestValidateToken_MultipleProviders (0.10s)54482026/11/12 11:46:20 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:394275449--- PASS: TestValidateToken_KubernetesServiceAccount (0.12s)54502026/11/12 11:46:21 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:39879/oidc5451--- PASS: TestScopes_LegacyProviderDefaultsToWrite (0.24s)5452PASS5453Running hook tests...5454=== RUN TestSendPathsEmpty5455=== PAUSE TestSendPathsEmpty5456=== RUN TestQueueEnqueueAndFetch5457=== PAUSE TestQueueEnqueueAndFetch5458=== RUN TestQueueDeduplication5459=== PAUSE TestQueueDeduplication5460=== RUN TestQueueRemove5461=== PAUSE TestQueueRemove5462=== RUN TestQueueFetchBatchLimit5463=== PAUSE TestQueueFetchBatchLimit5464=== RUN TestQueueRetryMovesToBack5465=== PAUSE TestQueueRetryMovesToBack5466=== RUN TestQueueFetchRemoveLifecycle5467=== PAUSE TestQueueFetchRemoveLifecycle5468=== RUN TestQueueConcurrentWriters5469=== PAUSE TestQueueConcurrentWriters5470=== RUN TestQueueRemoveLargeClosure5471=== PAUSE TestQueueRemoveLargeClosure5472=== RUN TestServerClientIntegration5473--- PASS: TestServerClientIntegration (0.00s)5474=== RUN TestServerQueueError54752026/11/12 11:46:21 ERROR Failed to queue paths error="permission denied" count=15476--- PASS: TestServerQueueError (0.00s)5477=== RUN TestGetListenerSocketActivation5478 server_test.go:220: === RUN TestGetListenerSocketActivation5479 --- PASS: TestGetListenerSocketActivation (0.00s)5480 PASS5481 5482--- PASS: TestGetListenerSocketActivation (0.01s)5483=== RUN TestServerStalledClientDoesNotBlockShutdown5484=== PAUSE TestServerStalledClientDoesNotBlockShutdown5485=== RUN TestServerBacksOffOnAcceptErrors5486=== PAUSE TestServerBacksOffOnAcceptErrors5487=== RUN TestDrainIsolatesPoisonPath5488=== PAUSE TestDrainIsolatesPoisonPath5489=== RUN TestRunNotBlockedByPoisonHead5490=== PAUSE TestRunNotBlockedByPoisonHead5491=== RUN TestDrainGivesUpWhenServerDown5492=== PAUSE TestDrainGivesUpWhenServerDown5493=== RUN TestFailedPathPrunedByLaterClosure5494=== PAUSE TestFailedPathPrunedByLaterClosure5495=== RUN TestWorkerUploadsAndRemoves5496=== PAUSE TestWorkerUploadsAndRemoves5497=== RUN TestWorkerSkipsGCdPaths5498=== PAUSE TestWorkerSkipsGCdPaths5499=== RUN TestWorkerPrunesClosureDeps5500=== PAUSE TestWorkerPrunesClosureDeps5501=== RUN TestDrainTimeout5502=== PAUSE TestDrainTimeout5503=== RUN TestWorkerRemoveFailureIsNotProgress5504=== PAUSE TestWorkerRemoveFailureIsNotProgress5505=== RUN TestWorkerKeepsPathItCannotStat5506=== PAUSE TestWorkerKeepsPathItCannotStat5507=== CONT TestSendPathsEmpty5508=== CONT TestDrainTimeout5509=== CONT TestWorkerPrunesClosureDeps5510=== CONT TestWorkerKeepsPathItCannotStat5511=== CONT TestWorkerRemoveFailureIsNotProgress5512--- PASS: TestSendPathsEmpty (0.00s)5513=== CONT TestQueueRemoveLargeClosure5514=== CONT TestQueueConcurrentWriters5515=== CONT TestServerStalledClientDoesNotBlockShutdown5516=== CONT TestQueueFetchRemoveLifecycle5517=== CONT TestWorkerSkipsGCdPaths5518=== CONT TestQueueRetryMovesToBack5519=== CONT TestWorkerUploadsAndRemoves5520=== CONT TestQueueFetchBatchLimit5521=== CONT TestFailedPathPrunedByLaterClosure5522=== CONT TestQueueRemove5523=== CONT TestDrainIsolatesPoisonPath5524=== CONT TestQueueDeduplication5525=== CONT TestServerBacksOffOnAcceptErrors5526=== CONT TestRunNotBlockedByPoisonHead55272000/01/01 00:00:00 ERROR Accept failed error="too many open files"55282000/01/01 00:00:00 ERROR Accept failed error="too many open files"55292000/01/01 00:00:00 ERROR Accept failed error="too many open files"55302000/01/01 00:00:00 ERROR Accept failed error="too many open files"55312000/01/01 00:00:00 ERROR Accept failed error="too many open files"55322000/01/01 00:00:00 ERROR Accept failed error="too many open files"55332000/01/01 00:00:00 ERROR Accept failed error="too many open files"55342000/01/01 00:00:00 ERROR Accept failed error="too many open files"5535--- PASS: TestServerBacksOffOnAcceptErrors (0.00s)5536=== CONT TestQueueEnqueueAndFetch5537=== CONT TestDrainGivesUpWhenServerDown55382026/11/12 11:46:21 INFO Upload queue status pending=255392026/11/12 11:46:21 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat3045748728/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat3045748728/002/locked/aaa: permission denied"55402026/11/12 11:46:21 INFO Uploading batch count=455412026/11/12 11:46:21 ERROR Upload failed error="upload failed" count=455422026/11/12 11:46:21 INFO Uploading batch count=155432026/11/12 11:46:21 INFO Upload queue status pending=255442026/11/12 11:46:21 INFO Uploading batch count=155452026/11/12 11:46:21 ERROR Upload failed error="upload failed" count=155462026/11/12 11:46:21 INFO Upload queue status pending=355472026/11/12 11:46:21 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerSkipsGCdPaths247874386/002/nonexistent55482026/11/12 11:46:21 INFO Upload queue status pending=255492026/11/12 11:46:21 INFO Uploading batch count=15550--- PASS: TestQueueRetryMovesToBack (0.02s)55512026/11/12 11:46:21 ERROR Upload failed error="upload failed" count=15552--- PASS: TestQueueEnqueueAndFetch (0.01s)55532026/11/12 11:46:21 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainIsolatesPoisonPath1818522375/002/bbb55542026/11/12 11:46:21 INFO Uploading batch count=255552026/11/12 11:46:21 INFO Uploading batch count=15556--- PASS: TestQueueFetchBatchLimit (0.02s)55572000/01/01 00:00:00 INFO Uploading batch count=255582026/11/12 11:46:21 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat3045748728/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat3045748728/002/locked/aaa: permission denied"55592026/11/12 11:46:21 INFO Uploading batch count=255602026/11/12 11:46:21 ERROR Upload failed error="upload failed" count=255612000/01/01 00:00:00 ERROR Upload failed error="context deadline exceeded" count=255622026/11/12 11:46:21 INFO Uploading batch count=155632026/11/12 11:46:21 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1808722695/002/a55642000/01/01 00:00:00 ERROR Drain finished with paths left in queue remaining=45565--- PASS: TestQueueRemove (0.02s)5566--- PASS: TestQueueFetchRemoveLifecycle (0.02s)55672026/11/12 11:46:21 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress3096337747/002/nonexistent55682026/11/12 11:46:21 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=155692026/11/12 11:46:21 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat3045748728/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat3045748728/002/locked/aaa: permission denied"5570--- PASS: TestQueueDeduplication (0.02s)55712026/11/12 11:46:21 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress3096337747/002/nonexistent55722026/11/12 11:46:21 INFO Uploading batch count=155732026/11/12 11:46:21 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1808722695/002/b55742026/11/12 11:46:21 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=155752026/11/12 11:46:21 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress3096337747/002/nonexistent55762026/11/12 11:46:21 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=155772026/11/12 11:46:21 ERROR Drain finished with paths left in queue remaining=155782026/11/12 11:46:21 ERROR Drain finished with paths left in queue remaining=155792026/11/12 11:46:21 INFO Uploading batch count=155802026/11/12 11:46:21 ERROR Upload failed error="upload failed" count=155812026/11/12 11:46:21 INFO Uploading batch count=255822026/11/12 11:46:21 ERROR Upload failed error="upload failed" count=255832026/11/12 11:46:21 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1808722695/002/c5584--- PASS: TestWorkerRemoveFailureIsNotProgress (0.02s)5585--- PASS: TestDrainTimeout (0.02s)55862026/11/12 11:46:21 INFO Uploading batch count=155872026/11/12 11:46:21 ERROR Upload failed error="upload failed" count=155882026/11/12 11:46:21 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1808722695/002/d5589--- PASS: TestWorkerKeepsPathItCannotStat (0.02s)55902026/11/12 11:46:21 INFO Uploading batch count=25591--- PASS: TestFailedPathPrunedByLaterClosure (0.02s)55922026/11/12 11:46:21 ERROR Upload failed error="upload failed" count=255932026/11/12 11:46:21 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1808722695/002/e55942026/11/12 11:46:21 INFO Uploading batch count=155952026/11/12 11:46:21 ERROR Upload failed error="upload failed" count=155962026/11/12 11:46:21 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1808722695/002/f55972026/11/12 11:46:21 ERROR Drain finished with paths left in queue remaining=155982026/11/12 11:46:21 ERROR Drain finished with paths left in queue remaining=105599--- PASS: TestDrainIsolatesPoisonPath (0.03s)5600--- PASS: TestDrainGivesUpWhenServerDown (0.02s)5601--- PASS: TestWorkerPrunesClosureDeps (0.04s)5602--- PASS: TestWorkerSkipsGCdPaths (0.04s)5603--- PASS: TestWorkerUploadsAndRemoves (0.04s)5604--- PASS: TestQueueRemoveLargeClosure (0.06s)56052026/11/12 11:46:21 ERROR Failed to decode request error="read unix /build/hook3688401226/test.sock->@: i/o timeout"56062026/11/12 11:46:21 ERROR Failed to write response error="write unix /build/hook3688401226/test.sock->@: i/o timeout"5607--- PASS: TestServerStalledClientDoesNotBlockShutdown (0.20s)5608--- PASS: TestQueueConcurrentWriters (0.22s)56092026/11/12 11:46:22 INFO Uploading batch count=156102026/11/12 11:46:22 INFO Uploading batch count=156112026/11/12 11:46:22 INFO Uploading batch count=156122026/11/12 11:46:22 ERROR Upload failed error="upload failed" count=156132026/11/12 11:46:22 INFO Uploading batch count=156142026/11/12 11:46:22 ERROR Upload failed error="upload failed" count=156152026/11/12 11:46:22 INFO Uploading batch count=156162026/11/12 11:46:22 ERROR Upload failed error="upload failed" count=156172026/11/12 11:46:22 INFO Uploading batch count=156182026/11/12 11:46:22 ERROR Upload failed error="upload failed" count=156192026/11/12 11:46:22 ERROR Drain finished with paths left in queue remaining=15620--- PASS: TestRunNotBlockedByPoisonHead (1.04s)5621PASS5622Running signing tests...5623=== RUN TestExternalSigner5624=== PAUSE TestExternalSigner5625=== RUN TestExternalSignerWrongPublicKey5626=== PAUSE TestExternalSignerWrongPublicKey5627=== RUN TestExternalSignerErrors5628=== PAUSE TestExternalSignerErrors5629=== RUN TestExternalSignerRestartsAfterCrash5630=== PAUSE TestExternalSignerRestartsAfterCrash5631=== RUN TestExternalSignerProgramErrorKeepsProcess5632=== PAUSE TestExternalSignerProgramErrorKeepsProcess5633=== RUN TestExternalSignerClose5634=== PAUSE TestExternalSignerClose5635=== RUN TestExternalSignerRejectsInvalidPublicKey5636=== PAUSE TestExternalSignerRejectsInvalidPublicKey5637=== RUN TestExternalSignerCancellation5638=== PAUSE TestExternalSignerCancellation5639=== RUN TestGenerateFingerprint5640=== PAUSE TestGenerateFingerprint5641=== RUN TestParseSigningKey5642=== PAUSE TestParseSigningKey5643=== RUN TestSign5644=== PAUSE TestSign5645=== RUN TestSignBatch5646=== PAUSE TestSignBatch5647=== RUN TestNilKeyPublicKey5648=== PAUSE TestNilKeyPublicKey5649=== RUN TestSignErrors5650=== PAUSE TestSignErrors5651=== CONT TestExternalSigner5652=== CONT TestExternalSignerCancellation5653=== CONT TestSignErrors5654=== CONT TestSign5655=== CONT TestExternalSignerRejectsInvalidPublicKey5656=== CONT TestExternalSignerClose5657=== CONT TestExternalSignerRestartsAfterCrash5658=== CONT TestSignBatch5659=== CONT TestParseSigningKey5660=== RUN TestParseSigningKey/valid_32-byte_key5661=== PAUSE TestParseSigningKey/valid_32-byte_key5662=== RUN TestParseSigningKey/valid_32-byte_key_with_different_name5663=== PAUSE TestParseSigningKey/valid_32-byte_key_with_different_name5664=== RUN TestParseSigningKey/no_colon5665=== PAUSE TestParseSigningKey/no_colon5666=== CONT TestGenerateFingerprint5667=== RUN TestGenerateFingerprint/basic_with_references5668=== CONT TestNilKeyPublicKey5669=== CONT TestExternalSignerWrongPublicKey5670=== CONT TestExternalSignerErrors5671=== CONT TestExternalSignerProgramErrorKeepsProcess5672=== RUN TestExternalSignerRejectsInvalidPublicKey/#005673=== PAUSE TestExternalSignerRejectsInvalidPublicKey/#005674=== RUN TestSignErrors/nil_key5675--- PASS: TestSign (0.00s)5676=== RUN TestParseSigningKey/empty_name5677=== PAUSE TestGenerateFingerprint/basic_with_references5678=== RUN TestExternalSignerRejectsInvalidPublicKey/no-colon5679=== RUN TestExternalSignerErrors/exit_at_startup5680=== PAUSE TestExternalSignerErrors/exit_at_startup5681--- PASS: TestSignBatch (0.00s)5682--- PASS: TestNilKeyPublicKey (0.00s)5683=== PAUSE TestParseSigningKey/empty_name5684=== RUN TestParseSigningKey/invalid_base645685=== PAUSE TestParseSigningKey/invalid_base645686=== RUN TestParseSigningKey/wrong_length5687=== PAUSE TestParseSigningKey/wrong_length5688=== CONT TestParseSigningKey/valid_32-byte_key5689=== CONT TestParseSigningKey/empty_name5690=== CONT TestParseSigningKey/no_colon5691=== RUN TestGenerateFingerprint/no_references5692=== PAUSE TestGenerateFingerprint/no_references5693=== CONT TestParseSigningKey/invalid_base645694=== PAUSE TestSignErrors/nil_key5695=== PAUSE TestExternalSignerRejectsInvalidPublicKey/no-colon5696=== RUN TestExternalSignerErrors/exit_on_request5697=== PAUSE TestExternalSignerErrors/exit_on_request5698=== CONT TestParseSigningKey/wrong_length5699=== CONT TestParseSigningKey/valid_32-byte_key_with_different_name5700=== RUN TestExternalSignerErrors/invalid_JSON5701--- PASS: TestParseSigningKey (0.00s)5702 --- PASS: TestParseSigningKey/valid_32-byte_key (0.00s)5703 --- PASS: TestParseSigningKey/empty_name (0.00s)5704 --- PASS: TestParseSigningKey/no_colon (0.00s)5705 --- PASS: TestParseSigningKey/invalid_base64 (0.00s)5706 --- PASS: TestParseSigningKey/wrong_length (0.00s)5707 --- PASS: TestParseSigningKey/valid_32-byte_key_with_different_name (0.00s)5708=== PAUSE TestExternalSignerErrors/invalid_JSON5709=== RUN TestExternalSignerErrors/wrong_shape5710=== PAUSE TestExternalSignerErrors/wrong_shape5711=== RUN TestExternalSignerErrors/wrong_id5712=== PAUSE TestExternalSignerErrors/wrong_id5713=== RUN TestSignErrors/nil_narinfo5714=== RUN TestGenerateFingerprint/unsorted_references_get_sorted5715=== PAUSE TestGenerateFingerprint/unsorted_references_get_sorted5716=== RUN TestExternalSignerErrors/invalid_signature_base645717=== PAUSE TestExternalSignerErrors/invalid_signature_base645718=== RUN TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5719=== PAUSE TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5720--- PASS: TestExternalSignerClose (0.00s)5721=== PAUSE TestSignErrors/nil_narinfo5722=== RUN TestGenerateFingerprint/invalid_nar_hash_prefix5723=== PAUSE TestGenerateFingerprint/invalid_nar_hash_prefix5724=== RUN TestExternalSignerErrors/wrong_signatures_shape5725=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5726=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5727=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5728=== RUN TestSignErrors/invalid_narinfo5729=== PAUSE TestSignErrors/invalid_narinfo5730=== RUN TestSignErrors/canceled5731=== PAUSE TestSignErrors/canceled5732=== CONT TestSignErrors/nil_key5733=== CONT TestSignErrors/nil_narinfo5734=== RUN TestGenerateFingerprint/invalid_nar_hash_length5735=== PAUSE TestGenerateFingerprint/invalid_nar_hash_length5736=== CONT TestSignErrors/canceled5737=== PAUSE TestExternalSignerErrors/wrong_signatures_shape5738=== RUN TestExternalSignerErrors/too_few_signatures5739=== PAUSE TestExternalSignerErrors/too_few_signatures5740=== RUN TestExternalSignerErrors/missing_signature_name5741=== PAUSE TestExternalSignerErrors/missing_signature_name5742=== RUN TestExternalSignerErrors/wrong_signature_name5743=== PAUSE TestExternalSignerErrors/wrong_signature_name5744=== RUN TestExternalSignerErrors/wrong_signature_length5745=== PAUSE TestExternalSignerErrors/wrong_signature_length5746=== RUN TestExternalSignerErrors/program_error5747=== PAUSE TestExternalSignerErrors/program_error5748=== RUN TestExternalSignerErrors/missing_signatures5749=== PAUSE TestExternalSignerErrors/missing_signatures5750=== RUN TestExternalSignerErrors/null_signatures5751=== PAUSE TestExternalSignerErrors/null_signatures5752=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5753=== CONT TestExternalSignerRejectsInvalidPublicKey/#005754=== CONT TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5755=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5756=== CONT TestExternalSignerRejectsInvalidPublicKey/no-colon5757=== RUN TestGenerateFingerprint/invalid_store_path_prefix5758=== PAUSE TestGenerateFingerprint/invalid_store_path_prefix5759=== RUN TestExternalSignerErrors/null5760=== PAUSE TestExternalSignerErrors/null5761--- PASS: TestExternalSignerProgramErrorKeepsProcess (0.00s)5762=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5763--- PASS: TestExternalSignerRejectsInvalidPublicKey (0.01s)5764 --- PASS: TestExternalSignerRejectsInvalidPublicKey/#00 (0.00s)5765 --- PASS: TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= (0.00s)5766 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:invalid! (0.00s)5767 --- PASS: TestExternalSignerRejectsInvalidPublicKey/no-colon (0.00s)5768 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:YQ== (0.00s)5769=== RUN TestGenerateFingerprint/invalid_reference_prefix5770=== PAUSE TestGenerateFingerprint/invalid_reference_prefix5771=== CONT TestGenerateFingerprint/basic_with_references5772=== CONT TestGenerateFingerprint/invalid_store_path_prefix5773=== CONT TestSignErrors/invalid_narinfo5774--- PASS: TestSignErrors (0.01s)5775 --- PASS: TestSignErrors/nil_key (0.00s)5776 --- PASS: TestSignErrors/nil_narinfo (0.00s)5777 --- PASS: TestSignErrors/canceled (0.00s)5778 --- PASS: TestSignErrors/invalid_narinfo (0.00s)5779=== CONT TestGenerateFingerprint/invalid_nar_hash_prefix5780=== RUN TestExternalSignerErrors/unrequested_key5781=== PAUSE TestExternalSignerErrors/unrequested_key5782=== CONT TestGenerateFingerprint/invalid_nar_hash_length5783=== CONT TestGenerateFingerprint/invalid_reference_prefix5784=== CONT TestGenerateFingerprint/unsorted_references_get_sorted5785=== CONT TestGenerateFingerprint/no_references5786--- PASS: TestExternalSignerWrongPublicKey (0.00s)5787--- PASS: TestGenerateFingerprint (0.00s)5788 --- PASS: TestGenerateFingerprint/basic_with_references (0.00s)5789 --- PASS: TestGenerateFingerprint/invalid_store_path_prefix (0.00s)5790 --- PASS: TestGenerateFingerprint/invalid_nar_hash_prefix (0.00s)5791 --- PASS: TestGenerateFingerprint/invalid_nar_hash_length (0.00s)5792 --- PASS: TestGenerateFingerprint/invalid_reference_prefix (0.00s)5793 --- PASS: TestGenerateFingerprint/unsorted_references_get_sorted (0.00s)5794 --- PASS: TestGenerateFingerprint/no_references (0.00s)5795=== CONT TestExternalSignerErrors/exit_at_startup5796=== CONT TestExternalSignerErrors/unrequested_key5797=== CONT TestExternalSignerErrors/null5798=== CONT TestExternalSignerErrors/wrong_signatures_shape5799=== CONT TestExternalSignerErrors/invalid_signature_base645800=== CONT TestExternalSignerErrors/too_few_signatures5801=== CONT TestExternalSignerErrors/wrong_id5802=== CONT TestExternalSignerErrors/null_signatures5803=== CONT TestExternalSignerErrors/wrong_shape5804=== CONT TestExternalSignerErrors/wrong_signature_name5805=== CONT TestExternalSignerErrors/invalid_JSON5806=== CONT TestExternalSignerErrors/wrong_signature_length5807=== CONT TestExternalSignerErrors/missing_signatures5808=== CONT TestExternalSignerErrors/missing_signature_name5809=== CONT TestExternalSignerErrors/exit_on_request5810=== CONT TestExternalSignerErrors/program_error5811--- PASS: TestExternalSigner (0.01s)5812--- PASS: TestExternalSignerRestartsAfterCrash (0.01s)5813--- PASS: TestExternalSignerErrors (0.00s)5814 --- PASS: TestExternalSignerErrors/exit_at_startup (0.00s)5815 --- PASS: TestExternalSignerErrors/null (0.00s)5816 --- PASS: TestExternalSignerErrors/unrequested_key (0.01s)5817 --- PASS: TestExternalSignerErrors/invalid_signature_base64 (0.01s)5818 --- PASS: TestExternalSignerErrors/too_few_signatures (0.01s)5819 --- PASS: TestExternalSignerErrors/wrong_signatures_shape (0.01s)5820 --- PASS: TestExternalSignerErrors/null_signatures (0.01s)5821 --- PASS: TestExternalSignerErrors/wrong_shape (0.01s)5822 --- PASS: TestExternalSignerErrors/wrong_id (0.01s)5823 --- PASS: TestExternalSignerErrors/program_error (0.01s)5824 --- PASS: TestExternalSignerErrors/missing_signature_name (0.01s)5825 --- PASS: TestExternalSignerErrors/missing_signatures (0.01s)5826 --- PASS: TestExternalSignerErrors/wrong_signature_name (0.01s)5827 --- PASS: TestExternalSignerErrors/exit_on_request (0.01s)5828 --- PASS: TestExternalSignerErrors/wrong_signature_length (0.01s)5829 --- PASS: TestExternalSignerErrors/invalid_JSON (0.01s)5830--- PASS: TestExternalSignerCancellation (0.02s)5831PASS5832Running OpenBao signer tests...5833=== RUN TestSign5834=== PAUSE TestSign5835=== RUN TestSignErrors5836=== PAUSE TestSignErrors5837=== RUN TestServe5838=== PAUSE TestServe5839=== RUN TestServeRejectsGarbage5840=== PAUSE TestServeRejectsGarbage5841=== CONT TestSign5842=== CONT TestServe5843=== CONT TestSignErrors5844=== CONT TestServeRejectsGarbage5845=== RUN TestSignErrors/http_error5846=== PAUSE TestSignErrors/http_error5847=== RUN TestSignErrors/invalid_json5848=== PAUSE TestSignErrors/invalid_json5849=== RUN TestSignErrors/no_prefix5850=== PAUSE TestSignErrors/no_prefix5851=== RUN TestSignErrors/invalid_base645852=== PAUSE TestSignErrors/invalid_base645853=== CONT TestSignErrors/http_error5854--- PASS: TestServeRejectsGarbage (0.00s)5855=== CONT TestSignErrors/no_prefix5856--- PASS: TestServe (0.00s)5857=== CONT TestSignErrors/invalid_json5858=== CONT TestSignErrors/invalid_base645859--- PASS: TestSign (0.00s)5860--- PASS: TestSignErrors (0.00s)5861 --- PASS: TestSignErrors/no_prefix (0.00s)5862 --- PASS: TestSignErrors/invalid_base64 (0.00s)5863 --- PASS: TestSignErrors/invalid_json (0.00s)5864 --- PASS: TestSignErrors/http_error (0.00s)5865PASS