niks3-go-unit-tests
checks.x86_64-linux.go-unit-tests
· build #502
· raw
1tribuchet: building on jamie2Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestUploadBuildLog_FileBodyReplayedOnRetry6=== PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry7=== RUN TestRegisterUploadedObjectReusesConnections8=== PAUSE TestRegisterUploadedObjectReusesConnections9=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica10=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica11=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun12=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun13=== RUN TestCaseHackSuffix14=== PAUSE TestCaseHackSuffix15=== RUN TestFilterOversizedClosures16=== PAUSE TestFilterOversizedClosures17=== RUN TestUploadMultipart_PartsInParallel18=== PAUSE TestUploadMultipart_PartsInParallel19=== RUN TestUploadMultipart_ReturnsAfterDroppedBody20=== PAUSE TestUploadMultipart_ReturnsAfterDroppedBody21=== RUN TestUploadMultipart_FailedProducerIsNotCompleted22=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted23=== RUN TestPartSizeForNAR24=== PAUSE TestPartSizeForNAR25=== RUN TestDumpPathCaseHackMatchesNix26--- PASS: TestDumpPathCaseHackMatchesNix (0.03s)27=== RUN TestDumpPathCaseHackCollision28--- PASS: TestDumpPathCaseHackCollision (0.00s)29=== RUN TestDumpPathMatchesNix30=== PAUSE TestDumpPathMatchesNix31=== RUN TestDumpPathSingleFile32=== PAUSE TestDumpPathSingleFile33=== RUN TestDumpPathWriterError34=== PAUSE TestDumpPathWriterError35=== RUN TestEncodeNixBase3236=== PAUSE TestEncodeNixBase3237=== RUN TestEncodeNixBase32WithRealHash38=== PAUSE TestEncodeNixBase32WithRealHash39=== RUN TestConvertHashToNix3240=== PAUSE TestConvertHashToNix3241=== RUN TestGetStorePathHash42=== PAUSE TestGetStorePathHash43=== RUN TestPathInfoHashCompatibility44=== PAUSE TestPathInfoHashCompatibility45=== RUN TestParsePathInfoJSON46=== PAUSE TestParsePathInfoJSON47=== RUN TestParsePathInfoJSONMultiplePaths48=== PAUSE TestParsePathInfoJSONMultiplePaths49=== RUN TestPathInfoCACompatibility50=== PAUSE TestPathInfoCACompatibility51=== RUN TestRateLimiterFeedback52=== PAUSE TestRateLimiterFeedback53=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess54=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess55=== RUN TestRegisterUploadedObject_BoundedAgainstSilentServer56=== PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer57=== RUN TestRegisterUploadedObject_OutlivesPush58=== PAUSE TestRegisterUploadedObject_OutlivesPush59=== RUN TestResolveStorePath60=== PAUSE TestResolveStorePath61=== RUN TestDoWithRetry_BodyReplayedViaGetBody62=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody63=== RUN TestRetryTimeoutOutlastsMaxRetries64=== PAUSE TestRetryTimeoutOutlastsMaxRetries65=== RUN TestDoWithRetry_FinalResponseBodyReadable66=== PAUSE TestDoWithRetry_FinalResponseBodyReadable67=== RUN TestShellSplit68=== PAUSE TestShellSplit69=== RUN TestShellSplitErrors70=== PAUSE TestShellSplitErrors71=== RUN TestStreamPushReportsEveryPath72=== PAUSE TestStreamPushReportsEveryPath73=== RUN TestStreamPushBatchesUnderLoad74=== PAUSE TestStreamPushBatchesUnderLoad75=== RUN TestStreamPushIsolatesFailures76=== PAUSE TestStreamPushIsolatesFailures77=== RUN TestStreamPushGivesUpOnDeadServer78=== PAUSE TestStreamPushGivesUpOnDeadServer79=== RUN TestStreamPushRequestLine80=== PAUSE TestStreamPushRequestLine81=== RUN TestStreamPushReportsSignatures82=== PAUSE TestStreamPushReportsSignatures83=== RUN TestStreamPushReportsSkippedPaths84=== PAUSE TestStreamPushReportsSkippedPaths85=== RUN TestClientSignaturesByStorePath86=== PAUSE TestClientSignaturesByStorePath87=== RUN TestStreamPushAnswersEveryLineWhenCanceled88=== PAUSE TestStreamPushAnswersEveryLineWhenCanceled89=== RUN TestStreamPushStopsWhenCanceled90=== PAUSE TestStreamPushStopsWhenCanceled91=== RUN TestSetClientTLS92=== PAUSE TestSetClientTLS93=== RUN TestSetClientTLSDoesNotMutateDefaultTransport94=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport95=== RUN TestSetClientTLSErrors96=== PAUSE TestSetClientTLSErrors97=== RUN TestStaticToken98=== PAUSE TestStaticToken99=== RUN TestFileTokenReadsAndCaches100=== PAUSE TestFileTokenReadsAndCaches101=== RUN TestFileTokenMissing102=== PAUSE TestFileTokenMissing103=== RUN TestFileTokenEmpty104=== PAUSE TestFileTokenEmpty105=== RUN TestScriptTokenNoExpiryRerunsEveryCall106=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall107=== RUN TestScriptTokenCachesUntilRefresh108=== PAUSE TestScriptTokenCachesUntilRefresh109=== RUN TestScriptTokenEmptyToken110=== PAUSE TestScriptTokenEmptyToken111=== RUN TestScriptTokenBadJSON112=== PAUSE TestScriptTokenBadJSON113=== RUN TestScriptTokenScriptFails114=== PAUSE TestScriptTokenScriptFails115=== RUN TestScriptTokenEmptyCommand116=== PAUSE TestScriptTokenEmptyCommand117=== CONT TestDoServerRequestAttachesToken118=== CONT TestStreamPushGivesUpOnDeadServer119=== CONT TestStaticToken120--- PASS: TestStaticToken (0.00s)121=== CONT TestGetStorePathHash122=== CONT TestScriptTokenEmptyCommand123--- PASS: TestScriptTokenEmptyCommand (0.00s)124=== RUN TestGetStorePathHash/valid_store_path125=== PAUSE TestGetStorePathHash/valid_store_path126=== CONT TestScriptTokenBadJSON127=== CONT TestScriptTokenEmptyToken128=== CONT TestScriptTokenCachesUntilRefresh129=== CONT TestScriptTokenNoExpiryRerunsEveryCall130=== CONT TestFileTokenEmpty131=== CONT TestFileTokenMissing1322026/11/12 00:32:46 ERROR Upload failed error="connection refused" count=20133=== CONT TestFileTokenReadsAndCaches1342026/11/12 00:32:46 ERROR Server seems unavailable, giving up on batch untried=17135=== CONT TestPathInfoHashCompatibility136=== CONT TestStreamPushAnswersEveryLineWhenCanceled137=== CONT TestSetClientTLSErrors138=== CONT TestStreamPushIsolatesFailures139=== CONT TestSetClientTLSDoesNotMutateDefaultTransport140=== CONT TestSetClientTLS141=== CONT TestStreamPushBatchesUnderLoad142=== CONT TestStreamPushReportsEveryPath143=== CONT TestStreamPushStopsWhenCanceled144=== CONT TestShellSplitErrors145=== CONT TestEncodeNixBase32146=== CONT TestShellSplit147=== CONT TestScriptTokenScriptFails148=== CONT TestDoWithRetry_FinalResponseBodyReadable149=== RUN TestGetStorePathHash/basename_without_hyphen_should_error150=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)151--- PASS: TestFileTokenEmpty (0.00s)152--- PASS: TestFileTokenReadsAndCaches (0.00s)1532026/11/12 00:32:46 ERROR Upload failed error="bad path" count=3154=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=false155=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=false156=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=true157=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=true158--- PASS: TestStreamPushIsolatesFailures (0.00s)1592000/01/01 00:00:00 ERROR Upload failed error="context canceled" count=2160=== CONT TestEncodeNixBase32WithRealHash161--- PASS: TestStreamPushReportsEveryPath (0.00s)162=== CONT TestDumpPathSingleFile163--- PASS: TestEncodeNixBase32WithRealHash (0.00s)164=== CONT TestResolveStorePath165=== CONT TestDoWithRetry_BodyReplayedViaGetBody166=== CONT TestRegisterUploadedObject_OutlivesPush167=== CONT TestDumpPathWriterError168--- PASS: TestStreamPushStopsWhenCanceled (0.00s)169=== RUN TestEncodeNixBase32/test_string_hash170=== CONT TestRegisterUploadedObject_BoundedAgainstSilentServer171=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess172--- PASS: TestFileTokenMissing (0.00s)173--- PASS: TestStreamPushAnswersEveryLineWhenCanceled (0.00s)174=== CONT TestCaseHackSuffix175=== PAUSE TestEncodeNixBase32/test_string_hash176=== CONT TestUploadMultipart_ReturnsAfterDroppedBody1772026/11/12 00:32:46 WARN Rate limiter enabled after throttle name=server-test rate=51782026/11/12 00:32:46 WARN Rate limiter enabled after throttle name=server-test rate=5179=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error180=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)1812026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:364051822026/11/12 00:32:46 WARN Rate limiter enabled after throttle name=server-test rate=51832026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:342131842026/11/12 00:32:46 WARN Rate limiter backed off name=server-test rate=5185=== CONT TestConvertHashToNix32186=== RUN TestConvertHashToNix32/SRI_format_to_Nix32187=== CONT TestRetryTimeoutOutlastsMaxRetries188=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32189=== CONT TestRateLimiterFeedback190--- PASS: TestShellSplit (0.00s)191=== RUN TestRateLimiterFeedback/429_enables_limiter192--- PASS: TestShellSplitErrors (0.00s)193--- PASS: TestScriptTokenBadJSON (0.01s)194=== CONT TestPathInfoCACompatibility195=== RUN TestEncodeNixBase32/empty_input196=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon197=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error198=== CONT TestFilterOversizedClosures1992026/11/12 00:32:46 WARN Rate limiter backed off name=server-test rate=5200=== CONT TestParsePathInfoJSONMultiplePaths2012026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:36405202=== CONT TestParsePathInfoJSON203=== CONT TestUploadMultipart_PartsInParallel204=== RUN TestConvertHashToNix32/already_Nix32_format205--- PASS: TestScriptTokenEmptyToken (0.01s)206=== PAUSE TestRateLimiterFeedback/429_enables_limiter207=== PAUSE TestConvertHashToNix32/already_Nix32_format208=== RUN TestConvertHashToNix32/invalid_format209=== PAUSE TestConvertHashToNix32/invalid_format210--- PASS: TestStreamPushGivesUpOnDeadServer (0.01s)211--- PASS: TestResolveStorePath (0.00s)212=== RUN TestPathInfoCACompatibility/null_ca_field213=== CONT TestRegisterUploadedObjectReusesConnections214=== PAUSE TestEncodeNixBase32/empty_input215=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error216=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error217=== RUN TestFilterOversizedClosures/no_limit_keeps_everything218=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything2192026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:34951220=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon221=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths222=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped223=== RUN TestParsePathInfoJSON/Nix_format224=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica225=== RUN TestRateLimiterFeedback/503_enables_limiter226=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun227--- PASS: TestScriptTokenScriptFails (0.00s)228--- PASS: TestDoServerRequestAttachesToken (0.01s)229=== PAUSE TestPathInfoCACompatibility/null_ca_field230=== CONT TestStreamPushReportsSignatures231=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error232=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI233--- PASS: TestRegisterUploadedObject_OutlivesPush (0.01s)234--- PASS: TestDoWithRetry_FinalResponseBodyReadable (0.00s)235--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.01s)236=== RUN TestSetClientTLS/rejects_connection_without_client_cert237--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.01s)238--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.01s)239=== RUN TestSetClientTLSErrors/missing_cert_file240=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert2412026/11/12 00:32:46 ERROR Upload failed error=boom count=1242=== PAUSE TestRateLimiterFeedback/503_enables_limiter2432026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:34951244=== CONT TestStreamPushReportsSkippedPaths2452026/11/12 00:32:46 INFO Garbage collection started246=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter2472026/11/12 00:32:46 INFO Garbage collection started248=== CONT TestUploadMultipart_FailedProducerIsNotCompleted249=== CONT TestDumpPathMatchesNix250=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths251=== CONT TestStreamPushRequestLine252=== CONT TestUploadBuildLog_FileBodyReplayedOnRetry253=== PAUSE TestParsePathInfoJSON/Nix_format254=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped2552026/11/12 00:32:46 ERROR Upload failed error=boom count=1256=== RUN TestPathInfoCACompatibility/old_string_format_-_text257=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text258=== CONT TestPartSizeForNAR259=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI260=== RUN TestPartSizeForNAR/zero_stays_at_minimum261=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum262=== RUN TestPartSizeForNAR/small_stays_at_minimum263=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=false264--- PASS: TestScriptTokenCachesUntilRefresh (0.04s)265--- PASS: TestUploadMultipart_ReturnsAfterDroppedBody (0.03s)266=== PAUSE TestPartSizeForNAR/small_stays_at_minimum267=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum268=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum269=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts270=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts271=== RUN TestPartSizeForNAR/1_TiB272=== PAUSE TestPartSizeForNAR/1_TiB273=== PAUSE TestSetClientTLSErrors/missing_cert_file274=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA275=== CONT TestClientSignaturesByStorePath276=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter277=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter278=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter279=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA280=== RUN TestSetClientTLS/preserves_debug_logging_transport281=== PAUSE TestSetClientTLS/preserves_debug_logging_transport282=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part283=== RUN TestSetClientTLSErrors/missing_key_file2842026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:34951285=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths286=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths287=== RUN TestParsePathInfoJSON/Lix_format2882026/11/12 00:32:46 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=0289=== PAUSE TestParsePathInfoJSON/Lix_format290=== RUN TestParsePathInfoJSON/empty_input291=== PAUSE TestParsePathInfoJSON/empty_input292=== RUN TestParsePathInfoJSON/whitespace_only293=== PAUSE TestParsePathInfoJSON/whitespace_only294=== RUN TestParsePathInfoJSON/invalid_JSON295=== CONT TestEncodeNixBase32/empty_input296=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive297=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512298--- PASS: TestStreamPushReportsSignatures (0.03s)299--- PASS: TestStreamPushReportsSkippedPaths (0.00s)300--- PASS: TestDumpPathSingleFile (0.04s)301=== RUN TestPartSizeForNAR/5_TiB_S3_max_object302--- PASS: TestClientSignaturesByStorePath (0.00s)303=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object304=== CONT TestConvertHashToNix32/SRI_format_to_Nix32305=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error306=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part307=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary308=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary309=== CONT TestConvertHashToNix32/invalid_format310=== CONT TestRateLimiterFeedback/429_enables_limiter311=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter312=== PAUSE TestSetClientTLSErrors/missing_key_file313=== CONT TestConvertHashToNix32/already_Nix32_format314=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter315=== CONT TestEncodeNixBase32/test_string_hash316=== RUN TestFilterOversizedClosures/all_closures_skipped317=== PAUSE TestParsePathInfoJSON/invalid_JSON3182026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:34951319=== PAUSE TestFilterOversizedClosures/all_closures_skipped320=== CONT TestGetStorePathHash/valid_store_path321=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA3222026/11/12 00:32:46 WARN Garbage collection finished on another replica. Its statistics are not available here323=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive324=== RUN TestPathInfoCACompatibility/new_structured_format_-_text3252026/11/12 00:32:46 WARN Rate limiter enabled after throttle name=server-test rate=5326=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha5123272026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:40589328=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=true3292026/11/12 00:32:46 WARN Rate limiter enabled after throttle name=s3-test rate=5330=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error3312026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:35741332=== CONT TestParsePathInfoJSON/Nix_format3332026/11/12 00:32:46 WARN Rate limiter backed off name=server-test rate=5334--- PASS: TestCaseHackSuffix (0.04s)3352026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:34951336=== RUN TestPartSizeForNAR/capped_at_5_GiB337=== CONT TestGetStorePathHash/basename_without_hyphen_should_error338=== RUN TestSetClientTLSErrors/missing_ca_file339=== CONT TestRateLimiterFeedback/503_enables_limiter340=== CONT TestSetClientTLS/rejects_connection_without_client_cert341=== CONT TestSetClientTLS/preserves_debug_logging_transport342=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths343=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part344=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths345=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text346=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary347=== CONT TestParsePathInfoJSON/invalid_JSON348=== CONT TestParsePathInfoJSON/Lix_format349=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method350=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method351=== CONT TestFilterOversizedClosures/no_limit_keeps_everything352=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512353--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (0.00s)3542026/11/12 00:32:46 WARN Rate limiter enabled after throttle name=server-test rate=5355=== PAUSE TestPartSizeForNAR/capped_at_5_GiB3562026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:36357357=== PAUSE TestSetClientTLSErrors/missing_ca_file3582026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:34951359=== CONT TestPathInfoCACompatibility/new_structured_format_-_text3602026/11/12 00:32:46 WARN Rate limiter backed off name=server-test rate=5361=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive362=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method363=== CONT TestFilterOversizedClosures/all_closures_skipped364=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped365=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI366=== CONT TestParsePathInfoJSON/empty_input367=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)368=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon369--- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (0.01s)370=== CONT TestPathInfoCACompatibility/null_ca_field371=== RUN TestSetClientTLSErrors/invalid_ca_file372=== PAUSE TestSetClientTLSErrors/invalid_ca_file3732026/11/12 00:32:46 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000374=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts3752026/11/12 00:32:46 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=50376=== CONT TestSetClientTLSErrors/missing_ca_file3772026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:34951378=== CONT TestPartSizeForNAR/small_stays_at_minimum379=== CONT TestSetClientTLSErrors/invalid_ca_file380=== CONT TestPathInfoCACompatibility/old_string_format_-_text381=== CONT TestPartSizeForNAR/zero_stays_at_minimum382=== CONT TestPartSizeForNAR/1_TiB383=== CONT TestParsePathInfoJSON/whitespace_only384=== CONT TestPartSizeForNAR/capped_at_5_GiB385=== CONT TestPartSizeForNAR/5_TiB_S3_max_object386=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum387=== CONT TestSetClientTLSErrors/missing_cert_file388=== CONT TestSetClientTLSErrors/missing_key_file389--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (0.00s)390--- PASS: TestConvertHashToNix32 (0.00s)391 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)392 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)393 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)394--- PASS: TestEncodeNixBase32 (0.01s)395 --- PASS: TestEncodeNixBase32/empty_input (0.00s)396 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)397--- PASS: TestGetStorePathHash (0.01s)398 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)399 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)400 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)401 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)402--- PASS: TestPathInfoHashCompatibility (0.04s)403 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)404 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)405 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)406 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)407--- PASS: TestPathInfoCACompatibility (0.04s)408 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)409 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)410 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)411 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)412 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)413--- PASS: TestParsePathInfoJSON (0.03s)414 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)415 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)416 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)417 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)418 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)419--- PASS: TestParsePathInfoJSONMultiplePaths (0.03s)420 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)421 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)4222026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:34951423--- PASS: TestUploadMultipart_FailedProducerIsNotCompleted (0.00s)424 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part (0.00s)425 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary (0.00s)426--- PASS: TestRateLimiterFeedback (0.03s)427 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s)428 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.00s)429 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.00s)430 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s)431--- PASS: TestFilterOversizedClosures (0.03s)432 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)433 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)434 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)435--- PASS: TestStreamPushBatchesUnderLoad (0.00s)436 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=false (0.00s)437 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=true (0.00s)438--- PASS: TestPartSizeForNAR (0.01s)439 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)440 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)441 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)442 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)443 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)444 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)445 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)4462026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:34951447--- PASS: TestSetClientTLSErrors (0.05s)448 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)449 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)450 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)451 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)4522026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:349514532026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:349514542026/11/12 00:32:46 http: TLS handshake error from 127.0.0.1:43886: remote error: tls: bad certificate4552026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:34951456--- PASS: TestStreamPushRequestLine (0.02s)457--- PASS: TestSetClientTLS (0.04s)458 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.00s)459 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.00s)460 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.02s)4612026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:349514622026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:349514632026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:34951464--- PASS: TestRegisterUploadedObjectReusesConnections (0.06s)4652026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:349514662026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:349514672026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:349514682026/11/12 00:32:46 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:34951469--- PASS: TestRetryTimeoutOutlastsMaxRetries (0.07s)470--- PASS: TestDumpPathWriterError (0.08s)4712026/11/12 00:32:46 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:42491/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:42491/api/uploads/complete4722026/11/12 00:32:46 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4732026/11/12 00:32:46 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:42491/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:42491/api/uploads/complete4742026/11/12 00:32:46 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4752026/11/12 00:32:46 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:42491/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:42491/api/uploads/complete4762026/11/12 00:32:46 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"477--- PASS: TestRegisterUploadedObject_BoundedAgainstSilentServer (0.10s)478--- PASS: TestDumpPathMatchesNix (0.09s)479--- PASS: TestUploadMultipart_PartsInParallel (0.65s)480--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)481PASS482Running server tests...483The files belonging to this database system will be owned by user "nixbld".484This user must also own the server process.485486The database cluster will be initialized with locale "C".487The default database encoding has accordingly been set to "SQL_ASCII".488The default text search configuration will be set to "english".489490Data page checksums are enabled.491492creating directory /build/postgres1776330608/data ... ok493creating subdirectories ... ok494selecting dynamic shared memory implementation ... posix495selecting default "max_connections" ... 100496selecting default "shared_buffers" ... 128MB497selecting default time zone ... UTC498creating configuration files ... ok499running bootstrap script ... ok500performing post-bootstrap initialization ... ok501syncing data to disk ... ok502503initdb: warning: enabling "trust" authentication for local connections504initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.505506Success. You can now start the database server using:507508 pg_ctl -D /build/postgres1776330608/data -l logfile start509510/build/postgres1776330608:5432 - no response5112026-11-12 00:32:48.317 UTC [128] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit5122026-11-12 00:32:48.317 UTC [128] LOG: listening on Unix socket "/build/postgres1776330608/.s.PGSQL.5432"5132026-11-12 00:32:48.322 UTC [135] LOG: database system was shut down at 2026-11-12 00:32:48 UTC5142026-11-12 00:32:48.325 UTC [128] LOG: database system is ready to accept connections515/build/postgres1776330608:5432 - accepting connections516{"timestamp":"2026-11-12T00:32:48.619465093Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0d0a2fdf-1275-41d2-814b-142c4988c3b3","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(392)"}517=== RUN TestAbortSucceeded518=== PAUSE TestAbortSucceeded519=== RUN TestService_AuthMiddleware520=== PAUSE TestService_AuthMiddleware521=== RUN TestService_AuthMiddleware_MTLSProxyHeader522=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader523=== RUN TestService_AuthMiddleware_MTLSBoundSubjects524=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects525=== RUN TestService_ReadAuthMiddleware526=== PAUSE TestService_ReadAuthMiddleware527=== RUN TestService_AuthMiddleware_OIDC528=== PAUSE TestService_AuthMiddleware_OIDC529=== RUN TestService_RequireScope_OIDC530=== PAUSE TestService_RequireScope_OIDC531=== RUN TestService_ReadScope_PublicByDefault532=== PAUSE TestService_ReadScope_PublicByDefault533=== RUN TestCacheConfigHandler534=== PAUSE TestCacheConfigHandler535=== RUN TestCacheStatsHandler536=== PAUSE TestCacheStatsHandler537=== RUN TestClientCADerivations538=== PAUSE TestClientCADerivations539=== RUN TestClientErrorHandling540=== PAUSE TestClientErrorHandling541=== RUN TestClientIntegration542=== PAUSE TestClientIntegration543=== RUN TestClientMultipleUploads544=== PAUSE TestClientMultipleUploads545=== RUN TestClientWithDependencies546=== PAUSE TestClientWithDependencies547=== RUN TestClientSharedPathCommittedMidPush548=== PAUSE TestClientSharedPathCommittedMidPush549=== RUN TestPinProtectsFromGC550=== PAUSE TestPinProtectsFromGC551=== RUN TestClientPushesUseOnePush552=== PAUSE TestClientPushesUseOnePush553=== RUN TestClientFallsBackToClosures554=== PAUSE TestClientFallsBackToClosures555=== RUN TestDeleteClosuresRacingPins556=== PAUSE TestDeleteClosuresRacingPins557=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock558=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock559=== RUN TestResolveDBConnectionString560=== PAUSE TestResolveDBConnectionString561=== RUN TestConnectWaitsForAPeerMigration562=== PAUSE TestConnectWaitsForAPeerMigration563=== RUN TestLeadElectsOneAndHandsOver564=== PAUSE TestLeadElectsOneAndHandsOver565=== RUN TestLeadIncumbentWinsAfterRestart5662026/11/12 00:32:48 INFO lead: acquired remote=192.0.2.1:1234 generation=15672026/11/12 00:32:49 INFO lead: released remote=192.0.2.1:1234 generation=15682026/11/12 00:32:49 INFO lead: acquired remote=192.0.2.1:1234 generation=25692026/11/12 00:32:50 INFO lead: released remote=192.0.2.1:1234 generation=2570--- PASS: TestLeadIncumbentWinsAfterRestart (1.89s)571=== RUN TestLeadUnannouncedStandbyYieldsToIncumbent572=== PAUSE TestLeadUnannouncedStandbyYieldsToIncumbent573=== RUN TestLeadIncumbentAnnouncesAtOnce574=== PAUSE TestLeadIncumbentAnnouncesAtOnce575=== RUN TestLeadGenerationGrowsWithEveryGrant576=== PAUSE TestLeadGenerationGrowsWithEveryGrant577=== RUN TestLeadEndsOnShutdown578=== PAUSE TestLeadEndsOnShutdown579=== RUN TestLeadEndsWhenItsConnectionHangs5802026/11/12 00:32:50 INFO lead: acquired remote=192.0.2.1:1234 generation=15812026-11-12 00:32:52.126 UTC [575] FATAL: terminating connection due to administrator command5822026/11/12 00:32:52 INFO lead: acquired remote=192.0.2.1:1234 generation=25832026/11/12 00:32:53 WARN lead: lock connection lost error="timeout: context deadline exceeded"5842026/11/12 00:32:53 INFO lead: released remote=192.0.2.1:1234 generation=15852026/11/12 00:32:53 INFO lead: released remote=192.0.2.1:1234 generation=2586--- PASS: TestLeadEndsWhenItsConnectionHangs (3.12s)587=== RUN TestGCAdvisoryLockBlocksConcurrentRun588=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun589=== RUN TestGCBugBareHashReferences590=== PAUSE TestGCBugBareHashReferences591=== RUN TestGCMetrics592=== PAUSE TestGCMetrics593=== RUN TestPushDedupSurvivesConcurrentGC594=== PAUSE TestPushDedupSurvivesConcurrentGC595=== RUN TestDeduplicatedObjectsRecordedAsPending596=== PAUSE TestDeduplicatedObjectsRecordedAsPending597=== RUN TestGCSweepSkipsPendingObjects598=== PAUSE TestGCSweepSkipsPendingObjects599=== RUN TestTombstonedObjectOfferedWithoutWaiting600=== PAUSE TestTombstonedObjectOfferedWithoutWaiting601=== RUN TestGCSweepDeliversEachKeyOnce602=== PAUSE TestGCSweepDeliversEachKeyOnce603=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection604=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection605=== RUN TestForceGCDuringPushOffersSweptObject606=== PAUSE TestForceGCDuringPushOffersSweptObject607=== RUN TestFailedS3DeleteKeepsTombstone608=== PAUSE TestFailedS3DeleteKeepsTombstone609=== RUN TestSweepSkipsRowsLockedByPush610=== PAUSE TestSweepSkipsRowsLockedByPush611=== RUN TestRegisterCompletedObjectNeedsPendingRow612=== PAUSE TestRegisterCompletedObjectNeedsPendingRow613=== RUN TestGetExistingObjectsLocksRows614=== PAUSE TestGetExistingObjectsLocksRows615=== RUN TestSweepRechecksPendingAfterLocking616=== PAUSE TestSweepRechecksPendingAfterLocking617=== RUN TestPushWaitsForSweepHoldingRow618=== PAUSE TestPushWaitsForSweepHoldingRow619=== RUN TestCommitRacingPendingCleanupKeepsObjects620=== PAUSE TestCommitRacingPendingCleanupKeepsObjects621=== RUN TestGCEndsOnShutdown622=== PAUSE TestGCEndsOnShutdown623=== RUN TestGCTaskStore_StartNew624=== PAUSE TestGCTaskStore_StartNew625=== RUN TestGCTaskStore_DeduplicateSameParams626=== PAUSE TestGCTaskStore_DeduplicateSameParams627=== RUN TestGCTaskStore_ConflictDifferentParams628=== PAUSE TestGCTaskStore_ConflictDifferentParams629=== RUN TestGCTaskStore_GetEmpty630=== PAUSE TestGCTaskStore_GetEmpty631=== RUN TestGCTaskStore_GetReturnsLatest632=== PAUSE TestGCTaskStore_GetReturnsLatest633=== RUN TestGCTaskStore_CompletedAllowsNewTask634=== PAUSE TestGCTaskStore_CompletedAllowsNewTask635=== RUN TestGCTaskStore_PhaseUpdates636=== PAUSE TestGCTaskStore_PhaseUpdates637=== RUN TestGCTaskStore_Fail638=== PAUSE TestGCTaskStore_Fail639=== RUN TestGracefulShutdownDrainsInflight640=== PAUSE TestGracefulShutdownDrainsInflight641=== RUN TestService_healthCheckHandler642=== PAUSE TestService_healthCheckHandler643=== RUN TestService_readinessHandler644=== PAUSE TestService_readinessHandler645=== RUN TestGenerateLandingPage646=== PAUSE TestGenerateLandingPage647=== RUN TestEnsureAbortUploadsRule648=== PAUSE TestEnsureAbortUploadsRule649=== RUN TestCacheConfigHandlerMaxNarSize650=== PAUSE TestCacheConfigHandlerMaxNarSize651=== RUN TestCreatePendingClosureRejectsOversizedNAR652=== PAUSE TestCreatePendingClosureRejectsOversizedNAR653=== RUN TestNARDeduplicationMetadataUploadBug654=== PAUSE TestNARDeduplicationMetadataUploadBug655=== RUN TestMetricsInventory656=== PAUSE TestMetricsInventory657=== RUN TestMetricsCommit658=== PAUSE TestMetricsCommit659=== RUN TestService_NativeMTLS660=== PAUSE TestService_NativeMTLS661=== RUN TestServerTLSConfig662=== PAUSE TestServerTLSConfig663=== RUN TestMultipartCleanup664=== PAUSE TestMultipartCleanup665=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded666=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded667=== RUN TestPendingCleanupUsesOneCutoff668=== PAUSE TestPendingCleanupUsesOneCutoff669=== RUN TestObjectStatsTrigger670=== PAUSE TestObjectStatsTrigger671=== RUN TestObjectStatsUpdatedOncePerCommit672=== PAUSE TestObjectStatsUpdatedOncePerCommit673=== RUN TestOrphanedObjectsGC674=== PAUSE TestOrphanedObjectsGC675=== RUN TestOrphanedObjectsGCStressTest676=== PAUSE TestOrphanedObjectsGCStressTest677=== RUN TestCreatePin_ReservedPins678=== PAUSE TestCreatePin_ReservedPins679=== RUN TestPresentReportsOnlyClosureRoots680=== PAUSE TestPresentReportsOnlyClosureRoots681=== RUN TestPresentRefreshesOnlyStaleClosures682=== PAUSE TestPresentRefreshesOnlyStaleClosures683=== RUN TestPresentNotReportedWhileGCDeletesClosure684=== PAUSE TestPresentNotReportedWhileGCDeletesClosure685=== RUN TestParseSingleRange686=== PAUSE TestParseSingleRange687=== RUN TestProxyHeadersOnlyTrustedOnSocket6882026/11/12 00:32:53 INFO Starting HTTP server address=127.0.0.1:361476892026/11/12 00:32:53 INFO Starting HTTP server address=proxy.sock6902026/11/12 00:32:53 WARN mTLS auth: subject not in bound subjects subject="CN=someone"6912026/11/12 00:32:53 INFO Shutdown signal received, draining in-flight requests timeout=10s692--- PASS: TestProxyHeadersOnlyTrustedOnSocket (0.10s)693=== RUN TestIsValidCachePath694=== PAUSE TestIsValidCachePath695=== RUN TestReadProxyNarinfo696=== PAUSE TestReadProxyNarinfo697=== RUN TestReadProxyNarinfoAlreadyDecompressed698=== PAUSE TestReadProxyNarinfoAlreadyDecompressed699=== RUN TestReadProxyContentNegotiation700=== PAUSE TestReadProxyContentNegotiation701=== RUN TestReadProxyNarStreaming702=== PAUSE TestReadProxyNarStreaming703=== RUN TestReadProxy404704=== PAUSE TestReadProxy404705=== RUN TestReadProxyInvalidPath706=== PAUSE TestReadProxyInvalidPath707=== RUN TestReadProxyHead708=== PAUSE TestReadProxyHead709=== RUN TestReadProxyOutlastsServerWriteTimeout710=== PAUSE TestReadProxyOutlastsServerWriteTimeout711=== RUN TestReadProxyConditionalGet712=== PAUSE TestReadProxyConditionalGet713=== RUN TestReadProxyRootRedirectsToIndexHTML714=== PAUSE TestReadProxyRootRedirectsToIndexHTML715=== RUN TestReadProxyDisabled716=== PAUSE TestReadProxyDisabled717=== RUN TestReadRedirectNar718=== PAUSE TestReadRedirectNar719=== RUN TestReadRedirectKeepsNarinfoProxied720=== PAUSE TestReadRedirectKeepsNarinfoProxied721=== RUN TestReadProxyRangeRequest722=== PAUSE TestReadProxyRangeRequest723=== RUN TestReadRedirectUsesPublicS3URL724=== PAUSE TestReadRedirectUsesPublicS3URL725=== RUN TestPush_OverlappingRootsStoreOneRowPerKey726=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey727=== RUN TestPush_CompleteCommitsEveryRoot728=== PAUSE TestPush_CompleteCommitsEveryRoot729=== RUN TestPush_RepeatedRootCommitsOnce730=== PAUSE TestPush_RepeatedRootCommitsOnce731=== RUN TestPush_CompleteWaitsForCleanupOfThePush732=== PAUSE TestPush_CompleteWaitsForCleanupOfThePush733=== RUN TestPush_RootsAreLockedInByteOrder734=== PAUSE TestPush_RootsAreLockedInByteOrder735=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit736=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit737=== RUN TestPush_RejectsBadRequests738=== PAUSE TestPush_RejectsBadRequests739=== RUN TestPush_SignsNarinfosOfItsPendingObjects740=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects741=== RUN TestSharedNarBothUploadsComplete742=== PAUSE TestSharedNarBothUploadsComplete743=== RUN TestCompleteMultipartUpload_ErrorButObjectExists744=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists745=== RUN TestCompletedNarNotReofferedAcrossClosures746=== PAUSE TestCompletedNarNotReofferedAcrossClosures747=== RUN TestPresignedUploadRegisteredBeforeCommit748=== PAUSE TestPresignedUploadRegisteredBeforeCommit749=== RUN TestService_Rustfstest750=== PAUSE TestService_Rustfstest751=== RUN TestParseSize752=== PAUSE TestParseSize753=== RUN TestSkippedUploadsHandler754=== PAUSE TestSkippedUploadsHandler755=== RUN TestSystemdListenerNotActivated756--- PASS: TestSystemdListenerNotActivated (0.00s)757=== RUN TestWatchdogBeatsWhenHealthy758--- PASS: TestWatchdogBeatsWhenHealthy (0.02s)759=== RUN TestWatchdogSkipsWhenUnhealthy7602026/11/12 00:32:53 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7612026/11/12 00:32:53 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7622026/11/12 00:32:53 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7632026/11/12 00:32:53 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7642026/11/12 00:32:53 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7652026/11/12 00:32:53 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7662026/11/12 00:32:53 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7672026/11/12 00:32:54 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7682026/11/12 00:32:54 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7692026/11/12 00:32:54 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"770--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)771=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle772=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle773=== RUN TestProxyWriteTimeout774=== PAUSE TestProxyWriteTimeout775=== RUN TestPendingClosureWriteTimeout776=== PAUSE TestPendingClosureWriteTimeout777=== RUN TestIsValidUploadKey778=== PAUSE TestIsValidUploadKey779=== RUN TestUploadHandlersRejectInvalidKeys780=== PAUSE TestUploadHandlersRejectInvalidKeys781=== RUN TestUploadHandlersRejectOversizedBody782=== PAUSE TestUploadHandlersRejectOversizedBody783=== RUN TestService_cleanupPendingClosuresHandler784=== PAUSE TestService_cleanupPendingClosuresHandler785=== RUN TestService_createPendingClosureHandler786=== PAUSE TestService_createPendingClosureHandler787=== RUN TestService_verifyS3Integrity788=== PAUSE TestService_verifyS3Integrity789=== RUN TestService_verifyS3ReuploadsStaleNarinfo790=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo791=== RUN TestCompleteMultipartUnregistered792=== PAUSE TestCompleteMultipartUnregistered793=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT794=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT795=== CONT TestAbortSucceeded796--- PASS: TestAbortSucceeded (0.00s)797=== CONT TestOrphanedObjectsGCStressTest798=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT799=== CONT TestCompleteMultipartUnregistered800=== CONT TestService_verifyS3ReuploadsStaleNarinfo801=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed802=== CONT TestService_verifyS3Integrity803=== CONT TestService_createPendingClosureHandler804=== CONT TestService_cleanupPendingClosuresHandler805=== CONT TestUploadHandlersRejectOversizedBody806=== CONT TestUploadHandlersRejectInvalidKeys807=== CONT TestIsValidUploadKey808=== CONT TestPendingClosureWriteTimeout809=== CONT TestProxyWriteTimeout810=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle811=== CONT TestSkippedUploadsHandler812=== CONT TestParseSize813=== CONT TestService_Rustfstest814=== CONT TestPresignedUploadRegisteredBeforeCommit815=== CONT TestCompletedNarNotReofferedAcrossClosures816=== CONT TestCompleteMultipartUpload_ErrorButObjectExists817=== CONT TestSharedNarBothUploadsComplete818=== CONT TestPush_SignsNarinfosOfItsPendingObjects819=== CONT TestPush_RejectsBadRequests820=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit821=== CONT TestPush_RootsAreLockedInByteOrder822=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed823=== RUN TestPendingClosureWriteTimeout/empty824=== RUN TestIsValidUploadKey/narinfo825=== RUN TestProxyWriteTimeout/narinfo826--- PASS: TestParseSize (0.00s)827=== CONT TestPush_CompleteWaitsForCleanupOfThePush828=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info8292026/11/12 00:32:54 INFO Client skipped oversized paths paths=3 nar_bytes=5000000000830=== RUN TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed831=== PAUSE TestPendingClosureWriteTimeout/empty832=== RUN TestPendingClosureWriteTimeout/400_objects833=== PAUSE TestPendingClosureWriteTimeout/400_objects834=== RUN TestPendingClosureWriteTimeout/670k_objects835=== PAUSE TestPendingClosureWriteTimeout/670k_objects836=== PAUSE TestIsValidUploadKey/narinfo837=== PAUSE TestProxyWriteTimeout/narinfo838=== RUN TestProxyWriteTimeout/1_GiB_nar839=== PAUSE TestProxyWriteTimeout/1_GiB_nar840=== RUN TestProxyWriteTimeout/10_GiB_nar841=== PAUSE TestProxyWriteTimeout/10_GiB_nar842=== RUN TestProxyWriteTimeout/unknown_size843=== PAUSE TestProxyWriteTimeout/unknown_size844=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info845=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal846=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal847=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key848=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key849=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key850=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key851=== CONT TestPush_RepeatedRootCommitsOnce852=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed853=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost854=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost855=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed856=== CONT TestPush_CompleteCommitsEveryRoot857=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed858=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed859=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline860=== RUN TestIsValidUploadKey/nar_zst861=== PAUSE TestIsValidUploadKey/nar_zst862=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed863=== RUN TestIsValidUploadKey/nar_xz864=== PAUSE TestIsValidUploadKey/nar_xz865=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline866=== CONT TestPush_OverlappingRootsStoreOneRowPerKey867=== RUN TestIsValidUploadKey/nar_plain868=== PAUSE TestIsValidUploadKey/nar_plain869=== RUN TestIsValidUploadKey/listing870=== PAUSE TestIsValidUploadKey/listing871=== RUN TestIsValidUploadKey/build_log872=== PAUSE TestIsValidUploadKey/build_log873=== CONT TestReadRedirectUsesPublicS3URL874=== RUN TestIsValidUploadKey/build_log_home-manager_file875=== PAUSE TestIsValidUploadKey/build_log_home-manager_file876=== RUN TestIsValidUploadKey/build_log_plus_in_name877=== PAUSE TestIsValidUploadKey/build_log_plus_in_name878=== RUN TestIsValidUploadKey/build_log_question_mark879=== PAUSE TestIsValidUploadKey/build_log_question_mark880=== RUN TestIsValidUploadKey/build_log_equals881=== PAUSE TestIsValidUploadKey/build_log_equals882=== RUN TestIsValidUploadKey/realisation883=== PAUSE TestIsValidUploadKey/realisation884=== RUN TestIsValidUploadKey/realisation_plus_in_output885=== PAUSE TestIsValidUploadKey/realisation_plus_in_output886=== RUN TestIsValidUploadKey/nix-cache-info887=== PAUSE TestIsValidUploadKey/nix-cache-info888=== RUN TestIsValidUploadKey/index.html889=== PAUSE TestIsValidUploadKey/index.html890=== RUN TestIsValidUploadKey/narinfo_key,_nar_type891=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type892=== RUN TestIsValidUploadKey/nar_key,_narinfo_type893=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type894=== RUN TestIsValidUploadKey/listing_key,_narinfo_type895=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type896=== RUN TestIsValidUploadKey/traversal897=== PAUSE TestIsValidUploadKey/traversal898=== RUN TestIsValidUploadKey/traversal_nar899=== PAUSE TestIsValidUploadKey/traversal_nar900=== RUN TestIsValidUploadKey/absolute901=== PAUSE TestIsValidUploadKey/absolute902=== RUN TestIsValidUploadKey/empty_key903=== PAUSE TestIsValidUploadKey/empty_key904=== RUN TestIsValidUploadKey/unknown_type905=== PAUSE TestIsValidUploadKey/unknown_type906=== CONT TestReadProxyRangeRequest907--- PASS: TestSkippedUploadsHandler (0.10s)908=== CONT TestReadRedirectKeepsNarinfoProxied909=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure910=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure911=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart912=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart913=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts914=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts915=== CONT TestReadRedirectNar9162026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures9172026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures9182026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures9192026/11/12 00:32:54 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9202026/11/12 00:32:54 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst921--- PASS: TestCompleteMultipartUnregistered (0.44s)922=== CONT TestReadProxyDisabled9232026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures9242026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures925--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (0.49s)926=== CONT TestReadProxyRootRedirectsToIndexHTML9272026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures9282026/11/12 00:32:54 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst9292026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures9302026/11/12 00:32:54 INFO Received push request method=POST path=/api/pushes931--- PASS: TestPresignedUploadRegisteredBeforeCommit (0.52s)932=== CONT TestReadProxyConditionalGet9332026/11/12 00:32:54 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9342026/11/12 00:32:54 INFO Signed narinfos id=1 count=19352026/11/12 00:32:54 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9362026/11/12 00:32:54 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9372026/11/12 00:32:54 ERROR Failed to sign narinfos signer="index 1" error="signing key cannot be nil"9382026/11/12 00:32:54 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9392026/11/12 00:32:54 ERROR Failed to sign narinfos signer=niks3-test-1 error="signer failed"9402026/11/12 00:32:54 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9412026/11/12 00:32:54 ERROR Signer returned unexpected signature count signer=niks3-test-1 got=0 want=19422026/11/12 00:32:54 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9432026/11/12 00:32:54 ERROR Signer returned unexpected object key signer=niks3-test-1 object_key=unexpected.narinfo944--- PASS: TestPush_SignsNarinfosOfItsPendingObjects (0.53s)945=== CONT TestReadProxyOutlastsServerWriteTimeout946--- PASS: TestService_Rustfstest (0.54s)947=== CONT TestReadProxyHead948=== RUN TestPush_RejectsBadRequests/no_roots949=== PAUSE TestPush_RejectsBadRequests/no_roots950=== RUN TestPush_RejectsBadRequests/no_objects951=== PAUSE TestPush_RejectsBadRequests/no_objects952=== RUN TestPush_RejectsBadRequests/bad_root953=== PAUSE TestPush_RejectsBadRequests/bad_root954=== RUN TestPush_RejectsBadRequests/root_not_in_objects955=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects956=== CONT TestReadProxyInvalidPath9572026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures9582026/11/12 00:32:54 INFO Received push request method=POST path=/api/pushes9592026/11/12 00:32:54 INFO Received cleanup request method=DELETE path=/api/pending_closures9602026/11/12 00:32:54 INFO Aborted multipart uploads count=0 kept=09612026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures9622026/11/12 00:32:54 INFO Received cleanup request method=DELETE path=/api/pending_closures9632026/11/12 00:32:54 INFO Aborted multipart uploads count=1 kept=09642026/11/12 00:32:54 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete9652026-11-12 00:32:54.706 UTC [660] ERROR: Closure does not exist: id=19662026-11-12 00:32:54.706 UTC [660] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 13 at RAISE9672026-11-12 00:32:54.706 UTC [660] STATEMENT: -- name: CommitPendingClosure :one968 SELECT commit_pending_closure($1::bigint)::bigint969 970--- PASS: TestService_cleanupPendingClosuresHandler (0.65s)971=== CONT TestReadProxy4049722026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures9732026/11/12 00:32:54 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9742026/11/12 00:32:54 INFO Received push request method=POST path=/api/pushes9752026/11/12 00:32:54 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9762026/11/12 00:32:54 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MzUxMzI5ZTgtMjdjNC00NjY1LWJjOTAtZjMxYTMyYzBkZjg0LmRkOTIxMzgxLWM0NTUtNDcwYi1hOGFjLWNmNDRjODRhMjdlZHgxNzk0NDQzNTc0NzE3MzQyOTM39772026/11/12 00:32:54 INFO Received complete push request method=POST path=/api/pushes/1/complete9782026/11/12 00:32:54 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MzUxMzI5ZTgtMjdjNC00NjY1LWJjOTAtZjMxYTMyYzBkZjg0LmRkOTIxMzgxLWM0NTUtNDcwYi1hOGFjLWNmNDRjODRhMjdlZHgxNzk0NDQzNTc0NzE3MzQyOTM3 parts=19792026/11/12 00:32:54 INFO Received push request method=POST path=/api/pushes9802026/11/12 00:32:54 INFO Completed push id=1 objects=4 duration=7.429313ms9812026/11/12 00:32:54 INFO Received push request method=POST path=/api/pushes982--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (0.70s)983=== CONT TestReadProxyNarStreaming9842026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures9852026/11/12 00:32:54 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9862026-11-12 00:32:54.799 UTC [700] ERROR: could not obtain lock on row in relation "closures"9872026-11-12 00:32:54.799 UTC [700] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9882026-11-12 00:32:54.800 UTC [700] ERROR: could not obtain lock on row in relation "closures"9892026-11-12 00:32:54.800 UTC [700] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9902026-11-12 00:32:54.800 UTC [700] ERROR: could not obtain lock on row in relation "closures"9912026-11-12 00:32:54.800 UTC [700] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9922026-11-12 00:32:54.800 UTC [700] ERROR: could not obtain lock on row in relation "closures"9932026-11-12 00:32:54.800 UTC [700] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9942026-11-12 00:32:54.801 UTC [700] ERROR: could not obtain lock on row in relation "closures"9952026-11-12 00:32:54.801 UTC [700] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT996--- PASS: TestPush_RootsAreLockedInByteOrder (0.66s)997=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection9982026/11/12 00:32:54 INFO Received push request method=POST path=/api/pushes9992026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures10002026/11/12 00:32:54 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=MzUxMzI5ZTgtMjdjNC00NjY1LWJjOTAtZjMxYTMyYzBkZjg0LjY5OWJkNzQzLWJlNjgtNDg0NC04N2IyLTBlNzdmNDAyYjA2NHgxNzk0NDQzNTc0NDU0ODg4NTYw parts=1010012026/11/12 00:32:54 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10022026/11/12 00:32:54 INFO Completed upload id=1 objects=2 duration=4.879872ms10032026/11/12 00:32:54 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000010042026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures10052026/11/12 00:32:54 INFO Aborted multipart uploads count=0 kept=010062026/11/12 00:32:54 INFO Starting cleanup of old closures method=DELETE path=/api/closures10072026/11/12 00:32:54 WARN Force mode enabled - objects will be deleted immediately without grace period10082026/11/12 00:32:54 INFO Aborted multipart uploads count=0 kept=01009--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (0.68s)1010=== CONT TestOrphanedObjectsGC10112026/11/12 00:32:54 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=01012--- PASS: TestReadRedirectNar (0.60s)1013=== CONT TestReadProxyContentNegotiation10142026/11/12 00:32:54 INFO Vacuumed table table=pending_closures10152026/11/12 00:32:54 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=010162026/11/12 00:32:54 INFO Vacuumed table table=pending_objects10172026/11/12 00:32:54 INFO Vacuumed table table=multipart_uploads10182026/11/12 00:32:54 INFO Vacuumed table table=pending_closures10192026/11/12 00:32:54 INFO Vacuumed table table=closures10202026/11/12 00:32:54 INFO Vacuumed table table=pending_objects10212026/11/12 00:32:54 INFO Vacuumed table table=objects10222026/11/12 00:32:54 INFO Vacuumed table table=multipart_uploads10232026/11/12 00:32:54 INFO Vacuumed table table=closures10242026/11/12 00:32:54 INFO Vacuumed table table=objects10252026/11/12 00:32:54 INFO Received complete push request method=POST path=/api/pushes/2/complete1026--- PASS: TestReadRedirectKeepsNarinfoProxied (0.71s)1027=== CONT TestObjectStatsUpdatedOncePerCommit10282026/11/12 00:32:54 INFO Completed push id=2 objects=4 duration=4.509702ms1029--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (0.72s)1030=== CONT TestReadProxyNarinfoAlreadyDecompressed10312026/11/12 00:32:54 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10322026/11/12 00:32:54 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001033--- PASS: TestService_createPendingClosureHandler (0.82s)1034=== CONT TestObjectStatsTrigger1035--- PASS: TestReadRedirectUsesPublicS3URL (0.75s)1036=== CONT TestReadProxyNarinfo10372026/11/12 00:32:54 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=MzUxMzI5ZTgtMjdjNC00NjY1LWJjOTAtZjMxYTMyYzBkZjg0LjQ2Yjk5ODczLTI3YWUtNDExNi1hNzc3LTZjOWViY2QyODIxY3gxNzk0NDQzNTc0NTI3Mzk4NDEw parts=1010382026/11/12 00:32:54 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10392026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures10402026/11/12 00:32:54 INFO Completed upload id=1 objects=2 duration=5.361112ms10412026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures10422026/11/12 00:32:54 INFO Received uploads request method=POST path=/api/pending_closures10432026/11/12 00:32:54 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo10442026/11/12 00:32:54 WARN Found objects in DB but missing from S3, will re-upload count=11045--- PASS: TestService_verifyS3Integrity (0.87s)1046=== CONT TestPendingCleanupUsesOneCutoff10472026/11/12 00:32:54 INFO Received push request method=POST path=/api/pushes10482026/11/12 00:32:54 INFO Received push request method=POST path=/api/pushes10492026/11/12 00:32:54 INFO Received complete push request method=POST path=/api/pushes/1/complete10502026-11-12 00:32:54.979 UTC [691] ERROR: Push does not exist: id=110512026-11-12 00:32:54.979 UTC [691] CONTEXT: PL/pgSQL function commit_push(bigint) line 11 at RAISE10522026-11-12 00:32:54.979 UTC [691] STATEMENT: SELECT commit_push($1)1053--- PASS: TestPush_CompleteWaitsForCleanupOfThePush (0.92s)1054=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded10552026/11/12 00:32:54 INFO Completed push id=1 objects=6 duration=10.053444ms1056--- PASS: TestPush_CompleteCommitsEveryRoot (0.84s)1057=== CONT TestMultipartCleanup10582026/11/12 00:32:55 INFO Received complete push request method=POST path=/api/pushes/1/complete10592026/11/12 00:32:55 INFO Completed push id=1 objects=4 duration=9.101644ms1060--- PASS: TestPush_RepeatedRootCommitsOnce (0.86s)1061=== CONT TestServerTLSConfig1062=== RUN TestServerTLSConfig/no_client_CA1063=== PAUSE TestServerTLSConfig/no_client_CA1064=== RUN TestServerTLSConfig/missing_CA_file1065=== PAUSE TestServerTLSConfig/missing_CA_file1066=== RUN TestServerTLSConfig/not_a_PEM_file1067=== PAUSE TestServerTLSConfig/not_a_PEM_file1068=== CONT TestIsValidCachePath1069=== RUN TestIsValidCachePath/narinfo1070=== PAUSE TestIsValidCachePath/narinfo1071=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars1072=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars1073=== RUN TestIsValidCachePath/nar_zst1074=== PAUSE TestIsValidCachePath/nar_zst1075=== RUN TestIsValidCachePath/nar_xz1076=== PAUSE TestIsValidCachePath/nar_xz1077=== RUN TestIsValidCachePath/nar_bz21078=== PAUSE TestIsValidCachePath/nar_bz21079=== RUN TestIsValidCachePath/nar_uncompressed1080=== PAUSE TestIsValidCachePath/nar_uncompressed1081=== RUN TestIsValidCachePath/ls1082=== PAUSE TestIsValidCachePath/ls1083=== RUN TestIsValidCachePath/log1084=== PAUSE TestIsValidCachePath/log1085=== RUN TestIsValidCachePath/realisation1086=== PAUSE TestIsValidCachePath/realisation1087=== RUN TestIsValidCachePath/nix-cache-info1088=== PAUSE TestIsValidCachePath/nix-cache-info1089=== RUN TestIsValidCachePath/index.html1090=== PAUSE TestIsValidCachePath/index.html1091=== RUN TestIsValidCachePath/traversal_parent1092=== PAUSE TestIsValidCachePath/traversal_parent1093=== RUN TestIsValidCachePath/traversal_in_middle1094=== PAUSE TestIsValidCachePath/traversal_in_middle1095=== RUN TestIsValidCachePath/invalid_char_e1096=== PAUSE TestIsValidCachePath/invalid_char_e1097=== RUN TestIsValidCachePath/invalid_char_u1098=== PAUSE TestIsValidCachePath/invalid_char_u1099=== RUN TestIsValidCachePath/random_path1100=== PAUSE TestIsValidCachePath/random_path1101=== RUN TestIsValidCachePath/empty1102=== PAUSE TestIsValidCachePath/empty1103=== RUN TestIsValidCachePath/leading_slash1104=== PAUSE TestIsValidCachePath/leading_slash1105=== RUN TestIsValidCachePath/wrong_extension1106=== PAUSE TestIsValidCachePath/wrong_extension1107=== RUN TestIsValidCachePath/short_hash1108=== PAUSE TestIsValidCachePath/short_hash1109=== CONT TestService_NativeMTLS1110--- PASS: TestReadProxyRangeRequest (0.86s)1111=== CONT TestMetricsCommit1112--- PASS: TestReadProxyDisabled (0.54s)1113=== CONT TestParseSingleRange1114=== RUN TestParseSingleRange/none1115=== PAUSE TestParseSingleRange/none1116=== RUN TestParseSingleRange/unknown_unit1117=== PAUSE TestParseSingleRange/unknown_unit1118=== RUN TestParseSingleRange/multi-range_ignored1119=== PAUSE TestParseSingleRange/multi-range_ignored1120=== RUN TestParseSingleRange/malformed_no_dash1121=== PAUSE TestParseSingleRange/malformed_no_dash1122=== RUN TestParseSingleRange/malformed_both_empty1123=== PAUSE TestParseSingleRange/malformed_both_empty1124=== RUN TestParseSingleRange/malformed_end_before_start1125=== PAUSE TestParseSingleRange/malformed_end_before_start1126=== RUN TestParseSingleRange/closed1127=== PAUSE TestParseSingleRange/closed1128=== RUN TestParseSingleRange/open-ended1129=== PAUSE TestParseSingleRange/open-ended1130=== RUN TestParseSingleRange/end_clamped_to_size1131=== PAUSE TestParseSingleRange/end_clamped_to_size1132=== RUN TestParseSingleRange/suffix1133=== PAUSE TestParseSingleRange/suffix1134=== RUN TestParseSingleRange/suffix_exceeds_size1135=== PAUSE TestParseSingleRange/suffix_exceeds_size1136=== RUN TestParseSingleRange/single_byte1137=== PAUSE TestParseSingleRange/single_byte1138=== RUN TestParseSingleRange/start_past_EOF1139=== PAUSE TestParseSingleRange/start_past_EOF1140=== RUN TestParseSingleRange/start_far_past_EOF1141=== PAUSE TestParseSingleRange/start_far_past_EOF1142=== CONT TestMetricsInventory1143--- PASS: TestReadProxyRootRedirectsToIndexHTML (0.53s)1144=== CONT TestPresentNotReportedWhileGCDeletesClosure1145=== RUN TestPresentNotReportedWhileGCDeletesClosure/stale1146=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/stale1147=== RUN TestPresentNotReportedWhileGCDeletesClosure/fresh1148=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/fresh1149=== CONT TestNARDeduplicationMetadataUploadBug1150--- PASS: TestReadProxyConditionalGet (0.54s)1151=== CONT TestPresentRefreshesOnlyStaleClosures1152--- PASS: TestReadProxyHead (0.59s)1153--- PASS: TestReadProxyInvalidPath (0.57s)1154=== CONT TestPresentReportsOnlyClosureRoots1155=== CONT TestCreatePendingClosureRejectsOversizedNAR11562026/11/12 00:32:55 INFO Received uploads request method=POST path=/api/pending_closures1157--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.00s)1158=== CONT TestCacheConfigHandlerMaxNarSize1159--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)1160=== CONT TestCreatePin_ReservedPins1161--- PASS: TestReadProxyNarStreaming (0.51s)1162=== CONT TestEnsureAbortUploadsRule11632026/11/12 00:32:55 INFO Received uploads request method=POST path=/api/pending_closures1164--- PASS: TestReadProxy404 (0.59s)1165=== CONT TestGCTaskStore_StartNew1166--- PASS: TestGCTaskStore_StartNew (0.00s)1167=== CONT TestGCTaskStore_DeduplicateSameParams1168--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)1169=== CONT TestGCEndsOnShutdown1170=== RUN TestGCEndsOnShutdown/before_the_run1171=== PAUSE TestGCEndsOnShutdown/before_the_run1172=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1173=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1174=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1175=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1176=== CONT TestGenerateLandingPage1177--- PASS: TestGenerateLandingPage (0.00s)1178=== CONT TestCommitRacingPendingCleanupKeepsObjects11792026/11/12 00:32:55 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:39983/oidc11802026/11/12 00:32:55 INFO Received complete multipart upload request method=POST path=/api/multipart/complete11812026/11/12 00:32:55 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=MzUxMzI5ZTgtMjdjNC00NjY1LWJjOTAtZjMxYTMyYzBkZjg0LmMyMTYyYWExLWMzMDUtNDA4MC04ZjU3LWNlYThlNmM5NGNlMXgxNzk0NDQzNTc0ODA1MzY3Mzgz parts=121182--- PASS: TestObjectStatsTrigger (0.51s)1183=== CONT TestService_readinessHandler1184--- PASS: TestReadProxyContentNegotiation (0.56s)1185=== CONT TestPushWaitsForSweepHoldingRow1186--- PASS: TestReadProxyNarinfoAlreadyDecompressed (0.56s)1187=== CONT TestSweepRechecksPendingAfterLocking11882026/11/12 00:32:55 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1189--- PASS: TestObjectStatsUpdatedOncePerCommit (0.58s)1190=== CONT TestService_healthCheckHandler11912026/11/12 00:32:55 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=MzUxMzI5ZTgtMjdjNC00NjY1LWJjOTAtZjMxYTMyYzBkZjg0LjZlYThkYTg3LTg4OGItNDIzMC1hNjAwLThlNGJjYzhjNzkxNHgxNzk0NDQzNTc0OTIzMzMyNjkx parts=1211922026/11/12 00:32:55 INFO Received uploads request method=POST path=/api/pending_closures1193--- PASS: TestCompletedNarNotReofferedAcrossClosures (1.42s)1194=== CONT TestGetExistingObjectsLocksRows11952026/11/12 00:32:55 INFO Received uploads request method=POST path=/api/pending_closures11962026/11/12 00:32:55 INFO Received uploads request method=POST path=/api/pending_closures1197--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (0.70s)1198=== CONT TestGracefulShutdownDrainsInflight11992026/11/12 00:32:55 INFO Starting HTTP server address=127.0.0.1:3570512002026/11/12 00:32:55 INFO Shutdown signal received, draining in-flight requests timeout=10s12012026/11/12 00:32:55 INFO Received cleanup request method=DELETE path=/api/pending_closures12022026/11/12 00:32:55 INFO Received uploads request method=POST path=/api/pending_closures1203--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (0.56s)1204=== CONT TestRegisterCompletedObjectNeedsPendingRow12052026/11/12 00:32:55 INFO Received uploads request method=POST path=/api/pending_closures12062026/11/12 00:32:55 ERROR Failed to decode narinfo key=3hcdxyjf9yiq7qf3i4548drb6sjmwa1v.narinfo limit=16777216 error=<nil>1207--- PASS: TestGracefulShutdownDrainsInflight (0.07s)1208=== CONT TestGCTaskStore_Fail1209--- PASS: TestGCTaskStore_Fail (0.00s)1210=== CONT TestSweepSkipsRowsLockedByPush1211--- PASS: TestReadProxyNarinfo (0.68s)1212=== CONT TestGCTaskStore_PhaseUpdates1213--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)1214=== CONT TestFailedS3DeleteKeepsTombstone12152026/11/12 00:32:55 WARN mTLS auth: subject not in bound subjects subject="CN=reader"12162026/11/12 00:32:55 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1217--- PASS: TestService_NativeMTLS (0.57s)1218=== CONT TestGCTaskStore_CompletedAllowsNewTask1219--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)1220=== CONT TestGCTaskStore_GetReturnsLatest1221--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)1222=== CONT TestForceGCDuringPushOffersSweptObject1223=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows1224=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows1225=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check1226=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check1227=== CONT TestGCTaskStore_GetEmpty1228--- PASS: TestGCTaskStore_GetEmpty (0.00s)1229=== CONT TestGCTaskStore_ConflictDifferentParams1230--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)1231=== CONT TestClientFallsBackToClosures12322026/11/12 00:32:55 INFO Received push request method=POST path=/api/pushes12332026/11/12 00:32:55 INFO Received complete push request method=POST path=/api/pushes/1/complete12342026/11/12 00:32:55 INFO Completed push id=1 objects=4 duration=7.996864ms12352026/11/12 00:32:55 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete12362026/11/12 00:32:55 INFO Completed upload id=2 objects=2 duration=1.698811ms1237--- PASS: TestMetricsCommit (0.64s)1238=== CONT TestGCSweepDeliversEachKeyOnce12392026/11/12 00:32:55 INFO Received cleanup request method=DELETE path=/api/pending_closures12402026/11/12 00:32:55 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket45/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""12412026/11/12 00:32:55 INFO Aborted multipart uploads count=0 kept=11242--- PASS: TestMetricsInventory (0.63s)1243=== CONT TestTombstonedObjectOfferedWithoutWaiting12442026/11/12 00:32:55 INFO Received cleanup request method=DELETE path=/api/pending_closures12452026/11/12 00:32:55 INFO Aborted multipart uploads count=1 kept=01246--- PASS: TestMultipartCleanup (0.69s)1247=== CONT TestGCSweepSkipsPendingObjects1248--- PASS: TestPresentRefreshesOnlyStaleClosures (0.59s)1249=== CONT TestDeduplicatedObjectsRecordedAsPending1250=== NAME TestNARDeduplicationMetadataUploadBug1251 metadata_upload_test.go:48: First store path: /build/TestNARDeduplicationMetadataUploadBug1356386607/001/store/kpjz0a5l82m9n7rjg0jhazcn2dmv4jad-file1.txt1252=== NAME TestOrphanedObjectsGC1253 orphaned_objects_gc_test.go:287: GC Test Summary:1254 orphaned_objects_gc_test.go:288: - Kept: 2 objects from closure A1255 orphaned_objects_gc_test.go:289: - Deleted: 2 objects from closure B1256 orphaned_objects_gc_test.go:290: - Deleted: 6 orphaned chain objects (X1->X2->X3)1257 orphaned_objects_gc_test.go:291: - Deleted: 2 orphaned single objects (Y)1258 orphaned_objects_gc_test.go:292: - Total deleted: 10 objects1259--- PASS: TestOrphanedObjectsGC (0.91s)1260=== CONT TestPushDedupSurvivesConcurrentGC1261--- PASS: TestPresentReportsOnlyClosureRoots (0.54s)1262=== CONT TestGCMetrics12632026/11/12 00:32:55 INFO Installed lifecycle rule aborting incomplete NAR uploads bucket=bucket52 days=21264--- PASS: TestEnsureAbortUploadsRule (0.49s)1265=== CONT TestGCBugBareHashReferences12662026/11/12 00:32:55 INFO Received push request method=POST path=/api/pushes12672026/11/12 00:32:55 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)12682026/11/12 00:32:55 INFO Uploading kpjz0a5l82m9n7rjg0jhazcn2dmv4jad-file1.txt (160B)12692026/11/12 00:32:55 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux12702026/11/12 00:32:55 WARN Refused reserved pin name=worker-x86_64-linux12712026/11/12 00:32:55 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux12722026/11/12 00:32:55 INFO Received create pin request method=POST path=/api/pins/my-app12732026/11/12 00:32:55 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux1274--- PASS: TestCreatePin_ReservedPins (0.63s)1275=== CONT TestGCAdvisoryLockBlocksConcurrentRun12762026/11/12 00:32:55 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"12772026/11/12 00:32:55 WARN Failed to register uploaded object key=kpjz0a5l82m9n7rjg0jhazcn2dmv4jad.ls error="server returned 404: 404 page not found\n"12782026/11/12 00:32:55 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12792026/11/12 00:32:55 INFO Signed narinfos id=1 count=112802026/11/12 00:32:55 INFO Uploading 1 narinfos12812026/11/12 00:32:55 INFO Received complete push request method=POST path=/api/pushes/1/complete12822026/11/12 00:32:55 WARN Failed to register uploaded object key=kpjz0a5l82m9n7rjg0jhazcn2dmv4jad.narinfo error="server returned 404: 404 page not found\n"12832026/11/12 00:32:55 INFO Aborted multipart uploads count=0 kept=012842026/11/12 00:32:55 WARN readiness check failed error="closed pool"1285--- PASS: TestService_readinessHandler (0.45s)1286=== CONT TestConnectWaitsForAPeerMigration12872026/11/12 00:32:55 WARN Force mode enabled - objects will be deleted immediately without grace period12882026/11/12 00:32:55 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012892026/11/12 00:32:55 INFO Completed push id=1 objects=3 duration=17.418317ms12902026/11/12 00:32:55 INFO Upload complete. (80ms)12912026/11/12 00:32:55 INFO Vacuumed table table=pending_closures1292=== NAME TestNARDeduplicationMetadataUploadBug1293 metadata_upload_test.go:54: Retrieved narinfo from S3:1294 StorePath: /build/TestNARDeduplicationMetadataUploadBug1356386607/001/store/kpjz0a5l82m9n7rjg0jhazcn2dmv4jad-file1.txt1295 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1296 Compression: zstd1297 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1298 NarSize: 1601299 References: 1300 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf13012026/11/12 00:32:55 INFO Vacuumed table table=pending_objects13022026/11/12 00:32:55 INFO Vacuumed table table=multipart_uploads1303 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)1304 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):1305 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}13062026/11/12 00:32:55 INFO Vacuumed table table=closures13072026/11/12 00:32:55 INFO Vacuumed table table=objects1308--- PASS: TestCommitRacingPendingCleanupKeepsObjects (0.56s)1309=== CONT TestClientPushesUseOnePush13102026/11/12 00:32:55 INFO Received complete multipart upload request method=POST path=/api/multipart/complete13112026/11/12 00:32:55 INFO Received uploads request method=POST path=/api/pending_closures1312=== NAME TestNARDeduplicationMetadataUploadBug1313 metadata_upload_test.go:64: Second store path (same content): /build/TestNARDeduplicationMetadataUploadBug1356386607/001/store/ln9dnwg92i7c1mc01lqd20rfz2w21xnd-file2.txt13142026/11/12 00:32:55 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=MzUxMzI5ZTgtMjdjNC00NjY1LWJjOTAtZjMxYTMyYzBkZjg0LjhiMzBiZDg0LWIwNWUtNDEwOS05NTliLTE4MDZhNjg0NTY4ZHgxNzk0NDQzNTc0ODEyNzcwMTE2 parts=121315--- PASS: TestSharedNarBothUploadsComplete (1.83s)1316=== CONT TestResolveDBConnectionString1317=== RUN TestResolveDBConnectionString/flag_wins1318=== PAUSE TestResolveDBConnectionString/flag_wins1319=== RUN TestResolveDBConnectionString/file_when_flag_empty1320=== PAUSE TestResolveDBConnectionString/file_when_flag_empty1321=== RUN TestResolveDBConnectionString/missing_file_is_an_error1322=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error1323=== RUN TestResolveDBConnectionString/PGHOST_allows_empty1324=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty1325=== RUN TestResolveDBConnectionString/nothing_configured1326=== PAUSE TestResolveDBConnectionString/nothing_configured1327=== CONT TestPinProtectsFromGC13282026/11/12 00:32:55 INFO Aborted multipart uploads count=0 kept=01329--- PASS: TestService_healthCheckHandler (0.47s)1330=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock13312026/11/12 00:32:55 WARN Force mode enabled - objects will be deleted immediately without grace period13322026/11/12 00:32:55 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013332026/11/12 00:32:55 INFO Vacuumed table table=pending_closures13342026/11/12 00:32:55 INFO Received push request method=POST path=/api/pushes13352026/11/12 00:32:55 INFO Vacuumed table table=pending_objects13362026/11/12 00:32:55 INFO Vacuumed table table=multipart_uploads13372026/11/12 00:32:55 INFO Vacuumed table table=closures13382026/11/12 00:32:55 INFO Vacuumed table table=objects13392026/11/12 00:32:55 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)1340--- PASS: TestSweepRechecksPendingAfterLocking (0.53s)1341=== CONT TestClientSharedPathCommittedMidPush13422026/11/12 00:32:55 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign13432026/11/12 00:32:55 WARN Failed to register uploaded object key=ln9dnwg92i7c1mc01lqd20rfz2w21xnd.ls error="server returned 404: 404 page not found\n"13442026/11/12 00:32:55 INFO Signed narinfos id=2 count=113452026/11/12 00:32:55 INFO Uploading 1 narinfos1346--- PASS: TestGetExistingObjectsLocksRows (0.49s)1347=== CONT TestClientWithDependencies13482026/11/12 00:32:55 INFO Received complete push request method=POST path=/api/pushes/2/complete13492026/11/12 00:32:55 WARN Failed to register uploaded object key=ln9dnwg92i7c1mc01lqd20rfz2w21xnd.narinfo error="server returned 404: 404 page not found\n"13502026/11/12 00:32:55 INFO Completed push id=2 objects=3 duration=2.664911ms13512026/11/12 00:32:55 INFO Upload complete. (52ms)1352=== NAME TestNARDeduplicationMetadataUploadBug1353 metadata_upload_test.go:76: Retrieved narinfo from S3:1354 StorePath: /build/TestNARDeduplicationMetadataUploadBug1356386607/001/store/ln9dnwg92i7c1mc01lqd20rfz2w21xnd-file2.txt1355 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1356 Compression: zstd1357 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1358 NarSize: 1601359 References: 1360 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1361 metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes)1362 metadata_upload_test.go:77: Decompressed .ls content (49 bytes):1363 {"version":1,"root":{"type":"regular","size":44}}1364--- PASS: TestRegisterCompletedObjectNeedsPendingRow (0.43s)1365=== CONT TestClientMultipleUploads1366--- PASS: TestNARDeduplicationMetadataUploadBug (0.90s)1367=== CONT TestDeleteClosuresRacingPins13682026/11/12 00:32:56 INFO Received uploads request method=POST path=/api/pending_closures13692026/11/12 00:32:56 INFO Aborted multipart uploads count=0 kept=013702026/11/12 00:32:56 WARN Force mode enabled - objects will be deleted immediately without grace period13712026/11/12 00:32:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013722026/11/12 00:32:56 INFO Vacuumed table table=pending_closures13732026/11/12 00:32:56 INFO Vacuumed table table=pending_objects13742026/11/12 00:32:56 INFO Vacuumed table table=multipart_uploads13752026/11/12 00:32:56 INFO Vacuumed table table=closures13762026/11/12 00:32:56 INFO Vacuumed table table=objects13772026/11/12 00:32:56 INFO Aborted multipart uploads count=0 kept=013782026/11/12 00:32:56 WARN Force mode enabled - objects will be deleted immediately without grace period13792026/11/12 00:32:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=013802026/11/12 00:32:56 INFO Vacuumed table table=pending_closures13812026/11/12 00:32:56 INFO Vacuumed table table=pending_objects13822026/11/12 00:32:56 INFO Vacuumed table table=multipart_uploads13832026/11/12 00:32:56 INFO Vacuumed table table=closures13842026/11/12 00:32:56 INFO Vacuumed table table=objects1385--- PASS: TestSweepSkipsRowsLockedByPush (0.51s)1386=== CONT TestClientIntegration13872026/11/12 00:32:56 INFO Aborted multipart uploads count=0 kept=013882026/11/12 00:32:56 INFO Received uploads request method=POST path=/api/pending_closures13892026/11/12 00:32:56 WARN Force mode enabled - objects will be deleted immediately without grace period13902026/11/12 00:32:56 INFO Aborted multipart uploads count=0 kept=01391--- PASS: TestTombstonedObjectOfferedWithoutWaiting (0.44s)1392=== CONT TestLeadElectsOneAndHandsOver13932026/11/12 00:32:56 WARN Force mode enabled - objects will be deleted immediately without grace period13942026/11/12 00:32:56 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."13952026/11/12 00:32:56 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."13962026/11/12 00:32:56 INFO Received uploads request method=POST path=/api/pending_closures1397--- PASS: TestFailedS3DeleteKeepsTombstone (0.55s)1398=== CONT TestClientErrorHandling1399=== RUN TestClientErrorHandling/InvalidStorePath1400=== PAUSE TestClientErrorHandling/InvalidStorePath1401=== RUN TestClientErrorHandling/InvalidAuthToken1402=== PAUSE TestClientErrorHandling/InvalidAuthToken1403=== RUN TestClientErrorHandling/ServerNotAvailable1404=== PAUSE TestClientErrorHandling/ServerNotAvailable1405=== CONT TestService_AuthMiddleware_OIDC14062026/11/12 00:32:56 INFO Aborted multipart uploads count=0 kept=014072026/11/12 00:32:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=014082026/11/12 00:32:56 INFO Vacuumed table table=pending_closures14092026/11/12 00:32:56 INFO Vacuumed table table=pending_objects14102026/11/12 00:32:56 INFO Vacuumed table table=multipart_uploads14112026/11/12 00:32:56 INFO Vacuumed table table=closures14122026/11/12 00:32:56 WARN Force mode enabled - objects will be deleted immediately without grace period14132026/11/12 00:32:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=014142026/11/12 00:32:56 INFO Vacuumed table table=objects14152026/11/12 00:32:56 INFO Received uploads request method=POST path=/api/pending_closures1416--- PASS: TestGCSweepDeliversEachKeyOnce (0.50s)1417=== CONT TestService_ReadAuthMiddleware14182026/11/12 00:32:56 INFO Vacuumed table table=pending_closures14192026/11/12 00:32:56 INFO Vacuumed table table=pending_objects14202026/11/12 00:32:56 INFO Vacuumed table table=multipart_uploads1421--- PASS: TestDeduplicatedObjectsRecordedAsPending (0.45s)14222026/11/12 00:32:56 INFO Vacuumed table table=closures1423=== CONT TestLeadEndsOnShutdown14242026/11/12 00:32:56 INFO Vacuumed table table=objects14252026/11/12 00:32:56 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:43685/oidc1426--- PASS: TestGCSweepSkipsPendingObjects (0.49s)1427=== CONT TestClientCADerivations14282026/11/12 00:32:56 INFO Aborted multipart uploads count=0 kept=014292026/11/12 00:32:56 WARN Force mode enabled - objects will be deleted immediately without grace period14302026/11/12 00:32:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=014312026/11/12 00:32:56 INFO Vacuumed table table=pending_closures14322026/11/12 00:32:56 INFO Vacuumed table table=pending_objects14332026/11/12 00:32:56 INFO Vacuumed table table=multipart_uploads14342026/11/12 00:32:56 INFO Vacuumed table table=closures14352026/11/12 00:32:56 INFO Vacuumed table table=objects1436--- PASS: TestGCMetrics (0.46s)1437=== CONT TestLeadGenerationGrowsWithEveryGrant14382026/11/12 00:32:56 INFO Received uploads request method=POST path=/api/pending_closures14392026/11/12 00:32:56 INFO Received uploads request method=POST path=/api/pending_closures14402026/11/12 00:32:56 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=nixbld database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"1441--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (0.43s)1442=== CONT TestCacheStatsHandler14432026/11/12 00:32:56 INFO Received uploads request method=POST path=/api/pending_closures14442026/11/12 00:32:56 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)14452026/11/12 00:32:56 INFO Uploading v4rsvzakrx7dlxidww3z2jmcj9a64z1i-a (216B)14462026/11/12 00:32:56 INFO Uploading zlrf7ihnvblrwszgafz5iwdwi2akslgg-shared-dep (136B)14472026/11/12 00:32:56 WARN Failed to register uploaded object key=aqqkj1z0sy4g4j8jbmk35hrm2nqbly79.ls error="server returned 404: 404 page not found\n"14482026/11/12 00:32:56 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"14492026/11/12 00:32:56 WARN Failed to register uploaded object key=zlrf7ihnvblrwszgafz5iwdwi2akslgg.ls error="server returned 404: 404 page not found\n"14502026/11/12 00:32:56 WARN Failed to register uploaded object key=nar/0kz70sqwmhvbh9w13nyfqv2yx6p0jp6zrb9jmrzbs3flv99l69n2.nar.zst error="server returned 404: 404 page not found\n"14512026/11/12 00:32:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign14522026/11/12 00:32:56 WARN Failed to register uploaded object key=v4rsvzakrx7dlxidww3z2jmcj9a64z1i.ls error="server returned 404: 404 page not found\n"14532026/11/12 00:32:56 INFO Signed narinfos id=1 count=214542026/11/12 00:32:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign14552026/11/12 00:32:56 INFO Signed narinfos id=2 count=214562026/11/12 00:32:56 INFO Uploading 4 narinfos14572026/11/12 00:32:56 WARN Failed to register uploaded object key=zlrf7ihnvblrwszgafz5iwdwi2akslgg.narinfo error="server returned 404: 404 page not found\n"14582026/11/12 00:32:56 WARN Failed to register uploaded object key=v4rsvzakrx7dlxidww3z2jmcj9a64z1i.narinfo error="server returned 404: 404 page not found\n"14592026/11/12 00:32:56 WARN Failed to register uploaded object key=aqqkj1z0sy4g4j8jbmk35hrm2nqbly79.narinfo error="server returned 404: 404 page not found\n"14602026/11/12 00:32:56 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete14612026/11/12 00:32:56 WARN Failed to register uploaded object key=zlrf7ihnvblrwszgafz5iwdwi2akslgg.narinfo error="server returned 404: 404 page not found\n"14622026/11/12 00:32:56 INFO Aborted multipart uploads count=0 kept=014632026/11/12 00:32:56 INFO Completed upload id=1 objects=6 duration=11.518805ms14642026/11/12 00:32:56 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete14652026/11/12 00:32:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=014662026/11/12 00:32:56 INFO Completed upload id=2 objects=6 duration=2.759361ms14672026/11/12 00:32:56 INFO Upload complete. (101ms)1468=== NAME TestClientFallsBackToClosures1469 client_pushes_test.go:112: Retrieved narinfo from S3:1470 StorePath: /build/TestClientFallsBackToClosures1980590016/001/store/zlrf7ihnvblrwszgafz5iwdwi2akslgg-shared-dep1471 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1472 Compression: zstd1473 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821474 NarSize: 1361475 References: 1476 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1477 client_pushes_test.go:112: Retrieved narinfo from S3:1478 StorePath: /build/TestClientFallsBackToClosures1980590016/001/store/v4rsvzakrx7dlxidww3z2jmcj9a64z1i-a1479 URL: nar/0kz70sqwmhvbh9w13nyfqv2yx6p0jp6zrb9jmrzbs3flv99l69n2.nar.zst1480 Compression: zstd1481 NarHash: sha256:0kz70sqwmhvbh9w13nyfqv2yx6p0jp6zrb9jmrzbs3flv99l69n21482 NarSize: 2161483 References: /build/TestClientFallsBackToClosures1980590016/001/store/zlrf7ihnvblrwszgafz5iwdwi2akslgg-shared-dep1484 CA: text:sha256:0443rr5697hhy7ixafadkjnbhk414x3sfjzsqhb8vdylyy2z768f14852026/11/12 00:32:56 INFO Vacuumed table table=pending_closures1486 client_pushes_test.go:112: Retrieved narinfo from S3:1487 StorePath: /build/TestClientFallsBackToClosures1980590016/001/store/aqqkj1z0sy4g4j8jbmk35hrm2nqbly79-b1488 URL: nar/0kz70sqwmhvbh9w13nyfqv2yx6p0jp6zrb9jmrzbs3flv99l69n2.nar.zst1489 Compression: zstd1490 NarHash: sha256:0kz70sqwmhvbh9w13nyfqv2yx6p0jp6zrb9jmrzbs3flv99l69n21491 NarSize: 2161492 References: /build/TestClientFallsBackToClosures1980590016/001/store/zlrf7ihnvblrwszgafz5iwdwi2akslgg-shared-dep1493 CA: text:sha256:0443rr5697hhy7ixafadkjnbhk414x3sfjzsqhb8vdylyy2z768f14942026/11/12 00:32:56 INFO Vacuumed table table=pending_objects14952026/11/12 00:32:56 INFO Vacuumed table table=multipart_uploads14962026/11/12 00:32:56 INFO Vacuumed table table=closures14972026/11/12 00:32:56 INFO Vacuumed table table=objects1498--- PASS: TestClientFallsBackToClosures (0.74s)1499=== CONT TestLeadIncumbentAnnouncesAtOnce15002026/11/12 00:32:56 INFO Aborted multipart uploads count=0 kept=015012026/11/12 00:32:56 WARN Force mode enabled - objects will be deleted immediately without grace period15022026/11/12 00:32:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=015032026/11/12 00:32:56 INFO Vacuumed table table=pending_closures15042026/11/12 00:32:56 INFO Vacuumed table table=pending_objects15052026/11/12 00:32:56 INFO Vacuumed table table=multipart_uploads15062026/11/12 00:32:56 INFO Vacuumed table table=closures15072026/11/12 00:32:56 INFO Vacuumed table table=objects1508--- PASS: TestPushDedupSurvivesConcurrentGC (0.60s)1509=== CONT TestService_AuthMiddleware_MTLSBoundSubjects1510--- PASS: TestPushWaitsForSweepHoldingRow (0.98s)1511=== CONT TestService_AuthMiddleware1512=== NAME TestPinProtectsFromGC1513 client_integration_test.go:791: Pinned store path: /build/TestPinProtectsFromGC279485983/001/store/03hzryskh0yvlr5y0x1gdbjwbbdi9i4r-pinned-file.txt1514 client_integration_test.go:792: Unpinned store path: /build/TestPinProtectsFromGC279485983/001/store/yn7fq4ki9i2g1yc5hhjfs5rbp85b0zcn-unpinned-file.txt1515--- PASS: TestGCBugBareHashReferences (0.69s)1516=== CONT TestService_AuthMiddleware_MTLSProxyHeader15172026/11/12 00:32:56 INFO Received push request method=POST path=/api/pushes15182026/11/12 00:32:56 INFO Received push request method=POST path=/api/pushes15192026/11/12 00:32:56 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)15202026/11/12 00:32:56 INFO Uploading 3dwrjpqssx03njip4qvb3kvnqjliim97-shared-dep (136B)15212026/11/12 00:32:56 INFO Uploading wf8nm72sxm5hfwik5i1hrxyzn69x0yrs-b (216B)15222026/11/12 00:32:56 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)15232026/11/12 00:32:56 INFO Uploading 03hzryskh0yvlr5y0x1gdbjwbbdi9i4r-pinned-file.txt (128B)1524=== NAME TestClientWithDependencies1525 client_integration_test.go:661: Built derivation: /build/TestClientWithDependencies431493837/001/store/yd36cfixd15xnkfsyr8kqg089yjlqw8x-test-script15262026/11/12 00:32:56 WARN Failed to register uploaded object key=sjk2wipi00faj0a70dqyrfi9i5r9b7mn.ls error="server returned 404: 404 page not found\n"15272026/11/12 00:32:56 WARN Failed to register uploaded object key=03hzryskh0yvlr5y0x1gdbjwbbdi9i4r.ls error="server returned 404: 404 page not found\n"15282026/11/12 00:32:56 WARN Failed to register uploaded object key=3dwrjpqssx03njip4qvb3kvnqjliim97.ls error="server returned 404: 404 page not found\n"15292026/11/12 00:32:56 WARN Failed to register uploaded object key=wf8nm72sxm5hfwik5i1hrxyzn69x0yrs.ls error="server returned 404: 404 page not found\n"15302026/11/12 00:32:56 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"15312026/11/12 00:32:56 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15322026/11/12 00:32:56 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"15332026/11/12 00:32:56 WARN Failed to register uploaded object key=nar/11b5pzhk26z46pgna3s15yxmk2mvp112sg3ilzan864zjhr3smp9.nar.zst error="server returned 404: 404 page not found\n"15342026/11/12 00:32:56 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15352026/11/12 00:32:56 INFO Signed narinfos id=1 count=315362026/11/12 00:32:56 INFO Signed narinfos id=1 count=115372026/11/12 00:32:56 INFO Uploading 3 narinfos15382026/11/12 00:32:56 INFO Uploading 1 narinfos1539=== NAME TestClientMultipleUploads1540 client_integration_test.go:406: Created store path 0: /build/TestClientMultipleUploads2973492381/001/store/knnvc6dgmad0yi9ybpcwdjbl3gnns52i-test-file-0.txt15412026/11/12 00:32:56 WARN Failed to register uploaded object key=3dwrjpqssx03njip4qvb3kvnqjliim97.narinfo error="server returned 404: 404 page not found\n"15422026/11/12 00:32:56 WARN Failed to register uploaded object key=sjk2wipi00faj0a70dqyrfi9i5r9b7mn.narinfo error="server returned 404: 404 page not found\n"15432026/11/12 00:32:56 INFO Received complete push request method=POST path=/api/pushes/1/complete15442026/11/12 00:32:56 INFO Received complete push request method=POST path=/api/pushes/1/complete15452026/11/12 00:32:56 WARN Failed to register uploaded object key=wf8nm72sxm5hfwik5i1hrxyzn69x0yrs.narinfo error="server returned 404: 404 page not found\n"15462026/11/12 00:32:56 INFO lead: acquired remote=192.0.2.1:1234 generation=115472026/11/12 00:32:56 WARN Failed to register uploaded object key=03hzryskh0yvlr5y0x1gdbjwbbdi9i4r.narinfo error="server returned 404: 404 page not found\n"15482026/11/12 00:32:56 INFO Completed push id=1 objects=3 duration=6.127372ms15492026/11/12 00:32:56 INFO Upload complete. (64ms)15502026/11/12 00:32:56 INFO Completed push id=1 objects=8 duration=10.439194ms15512026/11/12 00:32:56 INFO Upload complete. (69ms)1552=== NAME TestClientPushesUseOnePush1553 client_pushes_test.go:97: Retrieved narinfo from S3:1554 StorePath: /build/TestClientPushesUseOnePush2588166740/001/store/3dwrjpqssx03njip4qvb3kvnqjliim97-shared-dep1555 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1556 Compression: zstd1557 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821558 NarSize: 1361559 References: 1560 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1561 client_pushes_test.go:97: Retrieved narinfo from S3:1562 StorePath: /build/TestClientPushesUseOnePush2588166740/001/store/sjk2wipi00faj0a70dqyrfi9i5r9b7mn-a1563 URL: nar/11b5pzhk26z46pgna3s15yxmk2mvp112sg3ilzan864zjhr3smp9.nar.zst1564 Compression: zstd1565 NarHash: sha256:11b5pzhk26z46pgna3s15yxmk2mvp112sg3ilzan864zjhr3smp91566 NarSize: 2161567 References: /build/TestClientPushesUseOnePush2588166740/001/store/3dwrjpqssx03njip4qvb3kvnqjliim97-shared-dep1568 CA: text:sha256:12k4dw87ilr8ikzpc1znwi2qp3dv2spqlaqm0mymg8y8bafsc5l11569=== NAME TestOrphanedObjectsGCStressTest1570 orphaned_objects_gc_test.go:422: Created 10 active closures, 5 to-delete closures, 20 orphaned chains1571=== NAME TestClientPushesUseOnePush1572 client_pushes_test.go:97: Retrieved narinfo from S3:1573 StorePath: /build/TestClientPushesUseOnePush2588166740/001/store/wf8nm72sxm5hfwik5i1hrxyzn69x0yrs-b1574 URL: nar/11b5pzhk26z46pgna3s15yxmk2mvp112sg3ilzan864zjhr3smp9.nar.zst1575 Compression: zstd1576 NarHash: sha256:11b5pzhk26z46pgna3s15yxmk2mvp112sg3ilzan864zjhr3smp91577 NarSize: 2161578 References: /build/TestClientPushesUseOnePush2588166740/001/store/3dwrjpqssx03njip4qvb3kvnqjliim97-shared-dep1579 CA: text:sha256:12k4dw87ilr8ikzpc1znwi2qp3dv2spqlaqm0mymg8y8bafsc5l11580=== NAME TestClientIntegration1581 client_integration_test.go:292: Created store path: /build/TestClientIntegration83687626/002/store/rz8pygf5jf9xxnfvgxg6b8f8b35xfmif-test-file.txt1582--- PASS: TestService_ReadAuthMiddleware (0.35s)1583=== CONT TestCacheConfigHandler1584=== RUN TestCacheConfigHandler/full_config,_no_issuer1585=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1586=== RUN TestCacheConfigHandler/no_cache_url_configured1587=== PAUSE TestCacheConfigHandler/no_cache_url_configured1588=== RUN TestCacheConfigHandler/no_signing_keys1589=== PAUSE TestCacheConfigHandler/no_signing_keys1590=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1591=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1592=== CONT TestService_ReadScope_PublicByDefault1593=== NAME TestClientWithDependencies1594 client_integration_test.go:663: Found 1 dependencies (including self)1595--- PASS: TestClientPushesUseOnePush (0.64s)1596=== CONT TestService_RequireScope_OIDC1597=== NAME TestClientMultipleUploads1598 client_integration_test.go:406: Created store path 1: /build/TestClientMultipleUploads2973492381/001/store/ikrzhi0rj771myfl1ll4i1rd02saf3f4-test-file-1.txt15992026/11/12 00:32:56 INFO Received push request method=POST path=/api/pushes1600=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token1601=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token1602=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1603=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1604=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1605=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1606=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1607=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1608=== CONT TestLeadUnannouncedStandbyYieldsToIncumbent16092026/11/12 00:32:56 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)16102026/11/12 00:32:56 INFO Uploading a1lmxy39z8lbjir4zcvad3rjbgg14flb-top (224B)16112026/11/12 00:32:56 INFO Uploading x5dqagm77gmkvqpl0h0i5br4lrw80456-shared-dep (136B)1612=== NAME TestOrphanedObjectsGCStressTest1613 orphaned_objects_gc_test.go:440: Marked 210 objects for deletion1614--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (0.61s)1615=== CONT TestProxyWriteTimeout/narinfo1616=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info16172026/11/12 00:32:56 INFO Received uploads request method=POST path=/1618=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key16192026/11/12 00:32:56 INFO Received request for more parts method=POST path=/1620=== CONT TestProxyWriteTimeout/1_GiB_nar1621=== CONT TestProxyWriteTimeout/10_GiB_nar1622=== CONT TestProxyWriteTimeout/unknown_size1623=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key16242026/11/12 00:32:56 INFO Received complete multipart upload request method=POST path=/1625--- PASS: TestProxyWriteTimeout (0.09s)1626 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1627 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)1628 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1629 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1630=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal16312026/11/12 00:32:56 INFO Received uploads request method=POST path=/1632--- PASS: TestUploadHandlersRejectInvalidKeys (0.09s)1633 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)1634 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1635 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1636 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s)1637=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed16382026/11/12 00:32:56 WARN Failed to register uploaded object key=nar/0qj3vpnnbm3ca82dmfwp27nxndbv13z2csc90pvf2iczpa2ijsn8.nar.zst error="server returned 404: 404 page not found\n"16392026/11/12 00:32:56 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"16402026/11/12 00:32:56 WARN Failed to register uploaded object key=a1lmxy39z8lbjir4zcvad3rjbgg14flb.ls error="server returned 404: 404 page not found\n"16412026/11/12 00:32:56 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16422026/11/12 00:32:56 WARN Failed to register uploaded object key=x5dqagm77gmkvqpl0h0i5br4lrw80456.ls error="server returned 404: 404 page not found\n"16432026/11/12 00:32:56 INFO Signed narinfos id=1 count=216442026/11/12 00:32:56 INFO Uploading 2 narinfos16452026/11/12 00:32:56 WARN Failed to register uploaded object key=x5dqagm77gmkvqpl0h0i5br4lrw80456.narinfo error="server returned 404: 404 page not found\n"16462026/11/12 00:32:56 INFO Received complete push request method=POST path=/api/pushes/1/complete16472026/11/12 00:32:56 WARN Failed to register uploaded object key=a1lmxy39z8lbjir4zcvad3rjbgg14flb.narinfo error="server returned 404: 404 page not found\n"1648=== NAME TestClientMultipleUploads1649 client_integration_test.go:406: Created store path 2: /build/TestClientMultipleUploads2973492381/001/store/gq8yk95hakk1662z4ma0c4w9pdsdpaxc-test-file-2.txt16502026/11/12 00:32:56 INFO Completed push id=1 objects=6 duration=10.513924ms16512026/11/12 00:32:56 INFO Upload complete. (73ms)1652=== NAME TestClientSharedPathCommittedMidPush1653 client_integration_test.go:740: Retrieved narinfo from S3:1654 StorePath: /build/TestClientSharedPathCommittedMidPush2993472966/001/store/x5dqagm77gmkvqpl0h0i5br4lrw80456-shared-dep1655 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1656 Compression: zstd1657 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821658 NarSize: 1361659 References: 1660 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n16612026/11/12 00:32:56 INFO Received push request method=POST path=/api/pushes1662 client_integration_test.go:740: Retrieved narinfo from S3:1663 StorePath: /build/TestClientSharedPathCommittedMidPush2993472966/001/store/a1lmxy39z8lbjir4zcvad3rjbgg14flb-top1664 URL: nar/0qj3vpnnbm3ca82dmfwp27nxndbv13z2csc90pvf2iczpa2ijsn8.nar.zst1665 Compression: zstd1666 NarHash: sha256:0qj3vpnnbm3ca82dmfwp27nxndbv13z2csc90pvf2iczpa2ijsn81667 NarSize: 2241668 References: /build/TestClientSharedPathCommittedMidPush2993472966/001/store/x5dqagm77gmkvqpl0h0i5br4lrw80456-shared-dep1669 CA: text:sha256:15w156asf10y5adl050h4grjxk4lb8s56dy636m25y71ra6lj1wx1670--- PASS: TestClientSharedPathCommittedMidPush (0.60s)1671=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed16722026/11/12 00:32:56 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)16732026/11/12 00:32:56 INFO Uploading yn7fq4ki9i2g1yc5hhjfs5rbp85b0zcn-unpinned-file.txt (128B)16742026/11/12 00:32:56 WARN Failed to register uploaded object key=yn7fq4ki9i2g1yc5hhjfs5rbp85b0zcn.ls error="server returned 404: 404 page not found\n"16752026/11/12 00:32:56 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign16762026/11/12 00:32:56 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"16772026/11/12 00:32:56 INFO Signed narinfos id=2 count=116782026/11/12 00:32:56 INFO Uploading 1 narinfos16792026/11/12 00:32:56 INFO Received push request method=POST path=/api/pushes16802026/11/12 00:32:56 INFO lead: acquired remote=192.0.2.1:1234 generation=116812026/11/12 00:32:56 INFO Received complete push request method=POST path=/api/pushes/2/complete16822026/11/12 00:32:56 WARN Failed to register uploaded object key=yn7fq4ki9i2g1yc5hhjfs5rbp85b0zcn.narinfo error="server returned 404: 404 page not found\n"16832026/11/12 00:32:56 INFO Received push request method=POST path=/api/pushes16842026/11/12 00:32:56 INFO Completed push id=2 objects=3 duration=3.443741ms16852026/11/12 00:32:56 INFO Upload complete. (62ms)16862026/11/12 00:32:56 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)16872026/11/12 00:32:56 INFO Uploading rz8pygf5jf9xxnfvgxg6b8f8b35xfmif-test-file.txt (152B)16882026/11/12 00:32:56 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:40823/oidc16892026/11/12 00:32:56 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)16902026/11/12 00:32:56 INFO Uploading yd36cfixd15xnkfsyr8kqg089yjlqw8x-test-script (136B)16912026/11/12 00:32:56 WARN Failed to register uploaded object key=rz8pygf5jf9xxnfvgxg6b8f8b35xfmif.ls error="server returned 404: 404 page not found\n"16922026/11/12 00:32:56 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"16932026/11/12 00:32:56 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16942026/11/12 00:32:56 INFO Signed narinfos id=1 count=116952026/11/12 00:32:56 INFO Uploading 1 narinfos16962026/11/12 00:32:56 WARN Failed to register uploaded object key=log/zlpcmznvrya2nm5ckmwcfsgsdjns05c3-test-script.drv error="server returned 404: 404 page not found\n"16972026/11/12 00:32:56 WARN Failed to register uploaded object key=yd36cfixd15xnkfsyr8kqg089yjlqw8x.ls error="server returned 404: 404 page not found\n"16982026/11/12 00:32:56 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16992026/11/12 00:32:56 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"17002026/11/12 00:32:56 INFO Signed narinfos id=1 count=117012026/11/12 00:32:56 INFO Uploading 1 narinfos17022026/11/12 00:32:56 INFO Received complete push request method=POST path=/api/pushes/1/complete17032026/11/12 00:32:56 WARN Failed to register uploaded object key=rz8pygf5jf9xxnfvgxg6b8f8b35xfmif.narinfo error="server returned 404: 404 page not found\n"17042026/11/12 00:32:56 INFO Received complete push request method=POST path=/api/pushes/1/complete17052026/11/12 00:32:56 WARN Failed to register uploaded object key=yd36cfixd15xnkfsyr8kqg089yjlqw8x.narinfo error="server returned 404: 404 page not found\n"17062026/11/12 00:32:56 INFO Completed push id=1 objects=4 duration=5.882533ms17072026/11/12 00:32:56 INFO Upload complete. (67ms)17082026/11/12 00:32:56 INFO Completed push id=1 objects=3 duration=10.513425ms17092026/11/12 00:32:56 INFO Upload complete. (71ms)17102026/11/12 00:32:56 INFO lead: acquired remote=192.0.2.1:1234 generation=117112026/11/12 00:32:56 INFO Aborted multipart uploads count=0 kept=017122026/11/12 00:32:56 INFO Starting cleanup of old closures method=DELETE path=/api/closures17132026/11/12 00:32:56 INFO Garbage collection started17142026/11/12 00:32:56 WARN Force mode enabled - objects will be deleted immediately without grace period17152026/11/12 00:32:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=017162026/11/12 00:32:56 INFO Received push request method=POST path=/api/pushes17172026/11/12 00:32:56 INFO Vacuumed table table=pending_closures17182026/11/12 00:32:56 INFO Aborted multipart uploads count=0 kept=017192026/11/12 00:32:56 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)17202026/11/12 00:32:56 INFO Uploading ikrzhi0rj771myfl1ll4i1rd02saf3f4-test-file-1.txt (160B)17212026/11/12 00:32:56 INFO Uploading knnvc6dgmad0yi9ybpcwdjbl3gnns52i-test-file-0.txt (160B)17222026/11/12 00:32:56 INFO Uploading gq8yk95hakk1662z4ma0c4w9pdsdpaxc-test-file-2.txt (160B)17232026/11/12 00:32:56 INFO Vacuumed table table=pending_objects17242026/11/12 00:32:56 INFO Vacuumed table table=multipart_uploads17252026/11/12 00:32:56 INFO Vacuumed table table=closures17262026/11/12 00:32:56 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"17272026/11/12 00:32:56 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"17282026/11/12 00:32:56 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign17292026/11/12 00:32:56 WARN Failed to register uploaded object key=knnvc6dgmad0yi9ybpcwdjbl3gnns52i.ls error="server returned 404: 404 page not found\n"17302026/11/12 00:32:56 WARN Failed to register uploaded object key=gq8yk95hakk1662z4ma0c4w9pdsdpaxc.ls error="server returned 404: 404 page not found\n"17312026/11/12 00:32:56 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"17322026/11/12 00:32:56 WARN Failed to register uploaded object key=ikrzhi0rj771myfl1ll4i1rd02saf3f4.ls error="server returned 404: 404 page not found\n"17332026/11/12 00:32:56 INFO Signed narinfos id=1 count=317342026/11/12 00:32:56 INFO Uploading 3 narinfos17352026/11/12 00:32:56 WARN Force mode enabled - objects will be deleted immediately without grace period17362026/11/12 00:32:56 WARN Failed to register uploaded object key=gq8yk95hakk1662z4ma0c4w9pdsdpaxc.narinfo error="server returned 404: 404 page not found\n"17372026/11/12 00:32:56 WARN Failed to register uploaded object key=ikrzhi0rj771myfl1ll4i1rd02saf3f4.narinfo error="server returned 404: 404 page not found\n"17382026/11/12 00:32:56 INFO Received complete push request method=POST path=/api/pushes/1/complete17392026/11/12 00:32:56 WARN Failed to register uploaded object key=knnvc6dgmad0yi9ybpcwdjbl3gnns52i.narinfo error="server returned 404: 404 page not found\n"17402026/11/12 00:32:56 INFO Vacuumed table table=objects17412026/11/12 00:32:56 INFO All 1 paths already cached1742--- PASS: TestCacheStatsHandler (0.39s)1743=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1744=== NAME TestClientIntegration1745 client_integration_test.go:318: Retrieved narinfo from S3:1746 StorePath: /build/TestClientIntegration83687626/002/store/rz8pygf5jf9xxnfvgxg6b8f8b35xfmif-test-file.txt1747 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1748 Compression: zstd1749 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11750 NarSize: 1521751 References: 1752 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11753=== NAME TestClientWithDependencies1754 client_integration_test.go:677: Skipping nix copy test - isolated store (/build/TestClientWithDependencies431493837/001/store) requires matching store prefix1755=== NAME TestClientCADerivations1756 client_ca_test.go:136: Built CA derivation: /build/TestClientCADerivations3584074901/001/store/g5s9q19s17w3vm8hc8xblrsjm1qhwssl-ca-test1757=== NAME TestClientIntegration1758 client_integration_test.go:319: Retrieved .ls file from S3 (compressed size: 77 bytes)1759 client_integration_test.go:319: Decompressed .ls content (64 bytes):1760 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}1761--- PASS: TestClientWithDependencies (0.69s)1762=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost17632026/11/12 00:32:56 INFO Completed push id=1 objects=9 duration=9.892634ms17642026/11/12 00:32:56 INFO Upload complete. (74ms)1765=== NAME TestClientMultipleUploads1766 client_integration_test.go:417: Uploaded 3 paths in 109.115044ms1767--- PASS: TestClientMultipleUploads (0.69s)1768=== CONT TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed17692026/11/12 00:32:56 INFO lead: acquired remote=192.0.2.1:1234 generation=117702026/11/12 00:32:56 INFO lead: released remote=192.0.2.1:1234 generation=11771--- PASS: TestLeadIncumbentAnnouncesAtOnce (0.34s)1772=== CONT TestPendingClosureWriteTimeout/empty1773=== CONT TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline17742026/11/12 00:32:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=01775=== NAME TestClientCADerivations1776 client_ca_test.go:139: Found 1 dependencies (including self)17772026/11/12 00:32:56 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"17782026/11/12 00:32:56 WARN mTLS auth: bound subjects configured but subject DN unavailable17792026/11/12 00:32:56 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1780--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (0.35s)1781=== CONT TestPendingClosureWriteTimeout/670k_objects1782=== CONT TestPendingClosureWriteTimeout/400_objects1783=== CONT TestIsValidUploadKey/narinfo1784=== CONT TestIsValidUploadKey/realisation_plus_in_output1785=== CONT TestIsValidUploadKey/unknown_type1786=== CONT TestIsValidUploadKey/empty_key1787=== CONT TestIsValidUploadKey/absolute1788=== CONT TestIsValidUploadKey/traversal_nar1789=== CONT TestIsValidUploadKey/traversal1790=== CONT TestIsValidUploadKey/listing_key,_narinfo_type1791=== CONT TestIsValidUploadKey/index.html1792=== CONT TestIsValidUploadKey/nix-cache-info1793=== CONT TestIsValidUploadKey/nar_key,_narinfo_type1794=== CONT TestIsValidUploadKey/build_log_home-manager_file1795=== CONT TestIsValidUploadKey/realisation1796=== CONT TestIsValidUploadKey/build_log_equals1797=== CONT TestIsValidUploadKey/build_log_question_mark1798=== CONT TestIsValidUploadKey/build_log_plus_in_name1799=== CONT TestIsValidUploadKey/nar_plain1800=== CONT TestIsValidUploadKey/build_log1801=== CONT TestIsValidUploadKey/listing1802=== CONT TestIsValidUploadKey/nar_xz1803=== CONT TestIsValidUploadKey/nar_zst1804=== CONT TestIsValidUploadKey/narinfo_key,_nar_type1805--- PASS: TestIsValidUploadKey (0.09s)1806 --- PASS: TestIsValidUploadKey/narinfo (0.00s)1807 --- PASS: TestIsValidUploadKey/realisation_plus_in_output (0.00s)1808 --- PASS: TestIsValidUploadKey/unknown_type (0.00s)1809 --- PASS: TestIsValidUploadKey/empty_key (0.00s)1810 --- PASS: TestIsValidUploadKey/absolute (0.00s)1811 --- PASS: TestIsValidUploadKey/traversal_nar (0.00s)1812 --- PASS: TestIsValidUploadKey/traversal (0.00s)1813 --- PASS: TestIsValidUploadKey/listing_key,_narinfo_type (0.00s)1814 --- PASS: TestIsValidUploadKey/index.html (0.00s)1815 --- PASS: TestIsValidUploadKey/nix-cache-info (0.00s)1816 --- PASS: TestIsValidUploadKey/nar_key,_narinfo_type (0.00s)1817 --- PASS: TestIsValidUploadKey/build_log_home-manager_file (0.00s)1818 --- PASS: TestIsValidUploadKey/realisation (0.00s)1819 --- PASS: TestIsValidUploadKey/build_log_equals (0.00s)1820 --- PASS: TestIsValidUploadKey/build_log_question_mark (0.00s)1821 --- PASS: TestIsValidUploadKey/build_log_plus_in_name (0.00s)1822 --- PASS: TestIsValidUploadKey/nar_plain (0.00s)1823 --- PASS: TestIsValidUploadKey/build_log (0.00s)1824 --- PASS: TestIsValidUploadKey/listing (0.00s)1825 --- PASS: TestIsValidUploadKey/nar_xz (0.00s)1826 --- PASS: TestIsValidUploadKey/nar_zst (0.00s)1827 --- PASS: TestIsValidUploadKey/narinfo_key,_nar_type (0.00s)1828=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure18292026/11/12 00:32:56 INFO Received uploads request method=POST path=/18302026/11/12 00:32:56 INFO Vacuumed table table=pending_closures18312026/11/12 00:32:56 INFO Vacuumed table table=pending_objects18322026/11/12 00:32:56 INFO Vacuumed table table=multipart_uploads18332026/11/12 00:32:56 INFO Vacuumed table table=closures18342026/11/12 00:32:56 INFO Vacuumed table table=objects18352026/11/12 00:32:56 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"1836--- PASS: TestService_AuthMiddleware (0.34s)1837=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts18382026/11/12 00:32:56 INFO Received request for more parts method=POST path=/1839--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (0.28s)1840=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart18412026/11/12 00:32:56 INFO Received complete multipart upload request method=POST path=/18422026/11/12 00:32:56 INFO Received push request method=POST path=/api/pushes18432026/11/12 00:32:56 INFO Object in database but missing from S3 key=rz8pygf5jf9xxnfvgxg6b8f8b35xfmif.ls18442026/11/12 00:32:56 WARN Found objects in DB but missing from S3, will re-upload count=118452026/11/12 00:32:56 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)1846--- PASS: TestService_ReadScope_PublicByDefault (0.25s)1847=== CONT TestPush_RejectsBadRequests/no_roots18482026/11/12 00:32:56 INFO Received push request method=POST path=/api/pushes1849=== CONT TestPush_RejectsBadRequests/bad_root18502026/11/12 00:32:56 INFO Received push request method=POST path=/api/pushes1851=== CONT TestPush_RejectsBadRequests/no_objects18522026/11/12 00:32:56 INFO Received push request method=POST path=/api/pushes1853=== CONT TestPush_RejectsBadRequests/root_not_in_objects18542026/11/12 00:32:56 INFO Received push request method=POST path=/api/pushes1855=== CONT TestServerTLSConfig/no_client_CA1856=== CONT TestServerTLSConfig/not_a_PEM_file1857--- PASS: TestPush_RejectsBadRequests (0.56s)1858 --- PASS: TestPush_RejectsBadRequests/no_roots (0.00s)1859 --- PASS: TestPush_RejectsBadRequests/bad_root (0.00s)1860 --- PASS: TestPush_RejectsBadRequests/no_objects (0.00s)1861 --- PASS: TestPush_RejectsBadRequests/root_not_in_objects (0.00s)1862=== CONT TestServerTLSConfig/missing_CA_file1863=== CONT TestIsValidCachePath/narinfo1864=== CONT TestIsValidCachePath/index.html1865=== CONT TestIsValidCachePath/short_hash1866=== CONT TestIsValidCachePath/wrong_extension1867=== CONT TestIsValidCachePath/leading_slash1868=== CONT TestIsValidCachePath/empty1869=== CONT TestIsValidCachePath/random_path1870=== CONT TestIsValidCachePath/invalid_char_u1871=== CONT TestIsValidCachePath/invalid_char_e1872=== CONT TestIsValidCachePath/traversal_in_middle1873=== CONT TestIsValidCachePath/nar_uncompressed1874=== CONT TestIsValidCachePath/nix-cache-info1875=== CONT TestIsValidCachePath/realisation1876=== CONT TestIsValidCachePath/traversal_parent1877=== CONT TestIsValidCachePath/log1878=== CONT TestIsValidCachePath/ls1879=== CONT TestIsValidCachePath/nar_xz1880=== CONT TestIsValidCachePath/nar_bz21881=== CONT TestIsValidCachePath/nar_zst1882=== CONT TestParseSingleRange/none1883=== CONT TestParseSingleRange/open-ended1884=== CONT TestParseSingleRange/start_far_past_EOF1885=== CONT TestParseSingleRange/start_past_EOF1886=== CONT TestParseSingleRange/single_byte1887=== CONT TestParseSingleRange/suffix_exceeds_size1888=== CONT TestParseSingleRange/end_clamped_to_size1889=== CONT TestParseSingleRange/suffix18902026/11/12 00:32:56 INFO Received complete push request method=POST path=/api/pushes/2/complete1891=== CONT TestParseSingleRange/closed1892--- PASS: TestServerTLSConfig (0.00s)1893 --- PASS: TestServerTLSConfig/no_client_CA (0.00s)1894 --- PASS: TestServerTLSConfig/missing_CA_file (0.00s)1895 --- PASS: TestServerTLSConfig/not_a_PEM_file (0.00s)1896=== CONT TestIsValidCachePath/narinfo_all_nix_base32_chars1897=== CONT TestParseSingleRange/malformed_end_before_start18982026/11/12 00:32:56 WARN Failed to register uploaded object key=rz8pygf5jf9xxnfvgxg6b8f8b35xfmif.ls error="server returned 404: 404 page not found\n"1899=== CONT TestParseSingleRange/multi-range_ignored1900=== CONT TestParseSingleRange/malformed_no_dash1901=== CONT TestParseSingleRange/unknown_unit1902=== CONT TestPresentNotReportedWhileGCDeletesClosure/stale1903--- PASS: TestIsValidCachePath (0.00s)1904 --- PASS: TestIsValidCachePath/narinfo (0.00s)1905 --- PASS: TestIsValidCachePath/index.html (0.00s)1906 --- PASS: TestIsValidCachePath/short_hash (0.00s)1907 --- PASS: TestIsValidCachePath/wrong_extension (0.00s)1908 --- PASS: TestIsValidCachePath/leading_slash (0.00s)1909 --- PASS: TestIsValidCachePath/empty (0.00s)1910 --- PASS: TestIsValidCachePath/random_path (0.00s)1911 --- PASS: TestIsValidCachePath/invalid_char_u (0.00s)1912 --- PASS: TestIsValidCachePath/invalid_char_e (0.00s)1913 --- PASS: TestIsValidCachePath/traversal_in_middle (0.00s)1914 --- PASS: TestIsValidCachePath/nar_uncompressed (0.00s)1915 --- PASS: TestIsValidCachePath/nix-cache-info (0.00s)1916 --- PASS: TestIsValidCachePath/realisation (0.00s)1917 --- PASS: TestIsValidCachePath/traversal_parent (0.00s)1918 --- PASS: TestIsValidCachePath/log (0.00s)1919 --- PASS: TestIsValidCachePath/ls (0.00s)1920 --- PASS: TestIsValidCachePath/nar_xz (0.00s)1921 --- PASS: TestIsValidCachePath/nar_bz2 (0.00s)1922 --- PASS: TestIsValidCachePath/nar_zst (0.00s)1923 --- PASS: TestIsValidCachePath/narinfo_all_nix_base32_chars (0.00s)1924=== CONT TestParseSingleRange/malformed_both_empty1925--- PASS: TestParseSingleRange (0.00s)1926 --- PASS: TestParseSingleRange/none (0.00s)1927 --- PASS: TestParseSingleRange/open-ended (0.00s)1928 --- PASS: TestParseSingleRange/start_far_past_EOF (0.00s)1929 --- PASS: TestParseSingleRange/start_past_EOF (0.00s)1930 --- PASS: TestParseSingleRange/single_byte (0.00s)1931 --- PASS: TestParseSingleRange/suffix_exceeds_size (0.00s)1932 --- PASS: TestParseSingleRange/end_clamped_to_size (0.00s)1933 --- PASS: TestParseSingleRange/suffix (0.00s)1934 --- PASS: TestParseSingleRange/closed (0.00s)1935 --- PASS: TestParseSingleRange/malformed_end_before_start (0.00s)1936 --- PASS: TestParseSingleRange/multi-range_ignored (0.00s)1937 --- PASS: TestParseSingleRange/malformed_no_dash (0.00s)1938 --- PASS: TestParseSingleRange/unknown_unit (0.00s)1939 --- PASS: TestParseSingleRange/malformed_both_empty (0.00s)1940=== CONT TestPresentNotReportedWhileGCDeletesClosure/fresh19412026/11/12 00:32:56 INFO Completed push id=2 objects=3 duration=3.784782ms19422026/11/12 00:32:56 INFO Upload complete. (56ms)1943=== NAME TestClientIntegration1944 client_integration_test.go:336: Retrieved .ls file from S3 (compressed size: 62 bytes)1945 client_integration_test.go:336: Decompressed .ls content (49 bytes):1946 {"version":1,"root":{"type":"regular","size":39}}1947=== CONT TestGCEndsOnShutdown/before_the_run19482026/11/12 00:32:56 INFO lead: acquired remote=192.0.2.1:1234 generation=119492026/11/12 00:32:56 INFO Received push request method=POST path=/api/pushes19502026/11/12 00:32:56 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)19512026/11/12 00:32:56 INFO Uploading g5s9q19s17w3vm8hc8xblrsjm1qhwssl-ca-test (144B)19522026/11/12 00:32:56 WARN Failed to register uploaded object key=g5s9q19s17w3vm8hc8xblrsjm1qhwssl.ls error="server returned 404: 404 page not found\n"19532026/11/12 00:32:56 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"19542026/11/12 00:32:56 WARN Failed to register uploaded object key=log/861rp1lry9f60zziir6048kx3akbrijn-ca-test.drv error="server returned 404: 404 page not found\n"19552026/11/12 00:32:56 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign19562026/11/12 00:32:56 INFO Signed narinfos id=1 count=119572026/11/12 00:32:56 INFO Uploading 1 narinfos19582026/11/12 00:32:56 INFO Received uploads request method=POST path=/api/pending_closures19592026/11/12 00:32:56 INFO Received complete push request method=POST path=/api/pushes/1/complete19602026/11/12 00:32:56 WARN Failed to register uploaded object key=g5s9q19s17w3vm8hc8xblrsjm1qhwssl.narinfo error="server returned 404: 404 page not found\n"19612026/11/12 00:32:56 INFO Completed push id=1 objects=4 duration=16.190467ms19622026/11/12 00:32:56 INFO Upload complete. (109ms)1963=== NAME TestClientCADerivations1964 client_ca_test.go:180: Narinfo contains CA field: StorePath: /build/TestClientCADerivations3584074901/001/store/g5s9q19s17w3vm8hc8xblrsjm1qhwssl-ca-test1965 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1966 Compression: zstd1967 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1968 NarSize: 1441969 References: 1970 Deriver: /build/TestClientCADerivations3584074901/001/store/861rp1lry9f60zziir6048kx3akbrijn-ca-test.drv1971 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1972 client_ca_test.go:185: Checking for realisation files in S3...1973 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1974 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache19752026/11/12 00:32:56 INFO Received uploads request method=POST path=/api/pending_closures19762026/11/12 00:32:56 INFO Received push request method=POST path=/api/pushes19772026/11/12 00:32:56 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)1978=== RUN TestService_RequireScope_OIDC/builder_may_write1979=== PAUSE TestService_RequireScope_OIDC/builder_may_write1980=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1981=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1982=== RUN TestService_RequireScope_OIDC/ops_may_admin1983=== PAUSE TestService_RequireScope_OIDC/ops_may_admin19842026/11/12 00:32:56 INFO Received complete push request method=POST path=/api/pushes/3/complete1985=== RUN TestService_RequireScope_OIDC/ops_may_not_write19862026/11/12 00:32:56 WARN Failed to register uploaded object key=rz8pygf5jf9xxnfvgxg6b8f8b35xfmif.ls error="server returned 404: 404 page not found\n"1987=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1988=== RUN TestService_RequireScope_OIDC/reader_may_not_write1989=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1990=== RUN TestService_RequireScope_OIDC/static_token_may_admin1991=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1992=== RUN TestService_RequireScope_OIDC/static_token_may_write1993=== PAUSE TestService_RequireScope_OIDC/static_token_may_write1994=== RUN TestService_RequireScope_OIDC/reader_may_read1995=== PAUSE TestService_RequireScope_OIDC/reader_may_read1996=== RUN TestService_RequireScope_OIDC/writer_implies_read1997=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1998=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1999=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read2000=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc
loading 656 hidden lines…
26572026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=026582026/11/12 00:33:10 INFO Received uploads request method=POST path=/26592026/11/12 00:33:10 WARN Force mode enabled - objects will be deleted immediately without grace period26602026/11/12 00:33:10 INFO Received complete push request method=POST path=/26612026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."26622026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=026632026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=026642026/11/12 00:33:10 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=026652026/11/12 00:33:10 INFO Received push request method=POST path=/26662026/11/12 00:33:10 WARN Force mode enabled - objects will be deleted immediately without grace period26672026/11/12 00:33:10 INFO Vacuumed table table=pending_closures26682026/11/12 00:33:10 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=026692026/11/12 00:33:10 INFO Vacuumed table table=pending_objects26702026/11/12 00:33:10 INFO Vacuumed table table=multipart_uploads26712026/11/12 00:33:10 INFO Vacuumed table table=pending_closures26722026/11/12 00:33:10 INFO Completed push id=3 objects=8 duration=6.741033ms26732026/11/12 00:33:10 INFO Received push request method=POST path=/26742026/11/12 00:33:10 INFO Vacuumed table table=closures26752026/11/12 00:33:10 INFO Vacuumed table table=pending_objects26762026/11/12 00:33:10 INFO Vacuumed table table=multipart_uploads26772026/11/12 00:33:10 INFO Vacuumed table table=objects26782026/11/12 00:33:10 INFO Vacuumed table table=closures26792026/11/12 00:33:10 INFO Vacuumed table table=objects26802026/11/12 00:33:10 INFO Received complete push request method=POST path=/26812026/11/12 00:33:10 INFO Completed push id=6 objects=6 duration=2.216041ms26822026/11/12 00:33:10 INFO Received uploads request method=POST path=/26832026/11/12 00:33:10 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."26842026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."26852026/11/12 00:33:10 INFO Received push request method=POST path=/26862026/11/12 00:33:10 INFO Received complete upload request method=POST path=/26872026/11/12 00:33:10 INFO Completed upload id=3 objects=5 duration=1.373231ms26882026/11/12 00:33:10 INFO Received complete upload request method=POST path=/26892026/11/12 00:33:10 INFO Received push request method=POST path=/26902026/11/12 00:33:10 INFO Completed upload id=3 objects=24 duration=2.129201ms26912026/11/12 00:33:10 INFO Received uploads request method=POST path=/26922026/11/12 00:33:10 INFO Received complete push request method=POST path=/26932026/11/12 00:33:10 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo error="Access Denied."26942026/11/12 00:33:10 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."26952026/11/12 00:33:10 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."26962026/11/12 00:33:10 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."26972026/11/12 00:33:10 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."26982026/11/12 00:33:10 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."26992026/11/12 00:33:10 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."27002026/11/12 00:33:10 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."27012026/11/12 00:33:10 ERROR failed to remove object object=nar/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.nar.zst error="Access Denied."27022026/11/12 00:33:10 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."27032026/11/12 00:33:10 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."27042026/11/12 00:33:10 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."27052026/11/12 00:33:10 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."27062026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."27072026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."27082026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."27092026/11/12 00:33:10 INFO Received uploads request method=POST path=/27102026/11/12 00:33:10 INFO Completed push id=6 objects=5 duration=3.704811ms27112026/11/12 00:33:10 INFO Received push request method=POST path=/27122026/11/12 00:33:10 INFO Received complete upload request method=POST path=/27132026/11/12 00:33:10 INFO Received push request method=POST path=/27142026/11/12 00:33:10 INFO Received uploads request method=POST path=/27152026/11/12 00:33:10 INFO Received push request method=POST path=/27162026/11/12 00:33:10 INFO Completed upload id=5 objects=5 duration=1.92649ms27172026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=027182026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=027192026/11/12 00:33:10 INFO Received push request method=POST path=/27202026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=027212026/11/12 00:33:10 INFO Received push request method=POST path=/27222026/11/12 00:33:10 INFO Received uploads request method=POST path=/27232026/11/12 00:33:10 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=027242026/11/12 00:33:10 INFO Received complete upload request method=POST path=/27252026/11/12 00:33:10 INFO Received uploads request method=POST path=/27262026/11/12 00:33:10 INFO Vacuumed table table=pending_closures27272026/11/12 00:33:10 INFO Received complete upload request method=POST path=/27282026/11/12 00:33:10 INFO Completed upload id=3 objects=8 duration=3.009221ms27292026/11/12 00:33:10 INFO Vacuumed table table=pending_objects27302026/11/12 00:33:10 INFO Vacuumed table table=multipart_uploads27312026/11/12 00:33:10 INFO Vacuumed table table=closures27322026/11/12 00:33:10 INFO Completed upload id=9 objects=7 duration=2.732811ms27332026/11/12 00:33:10 INFO Vacuumed table table=objects27342026/11/12 00:33:10 INFO Received push request method=POST path=/27352026/11/12 00:33:10 INFO Received push request method=POST path=/27362026/11/12 00:33:10 INFO Received complete push request method=POST path=/27372026/11/12 00:33:10 INFO Completed push id=4 objects=7 duration=2.013361ms27382026/11/12 00:33:10 INFO Received complete push request method=POST path=/27392026/11/12 00:33:10 INFO Completed push id=4 objects=4 duration=2.207771ms27402026/11/12 00:33:10 INFO Received push request method=POST path=/27412026/11/12 00:33:10 INFO Received uploads request method=POST path=/27422026/11/12 00:33:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=027432026/11/12 00:33:10 INFO Vacuumed table table=pending_closures27442026/11/12 00:33:10 INFO Vacuumed table table=pending_objects27452026/11/12 00:33:10 INFO Vacuumed table table=multipart_uploads27462026/11/12 00:33:10 INFO Vacuumed table table=closures27472026/11/12 00:33:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=027482026/11/12 00:33:10 INFO Vacuumed table table=objects27492026/11/12 00:33:10 INFO Vacuumed table table=pending_closures27502026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=027512026/11/12 00:33:10 INFO Received complete upload request method=POST path=/27522026/11/12 00:33:10 INFO Received push request method=POST path=/27532026/11/12 00:33:10 INFO Vacuumed table table=pending_objects27542026/11/12 00:33:10 INFO Vacuumed table table=multipart_uploads27552026/11/12 00:33:10 INFO Received complete push request method=POST path=/27562026/11/12 00:33:10 INFO Completed upload id=11 objects=14 duration=2.113931ms27572026/11/12 00:33:10 INFO Received uploads request method=POST path=/27582026/11/12 00:33:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=027592026/11/12 00:33:10 INFO Completed push id=7 objects=27 duration=2.386691ms27602026/11/12 00:33:10 INFO Vacuumed table table=closures27612026/11/12 00:33:10 INFO Vacuumed table table=pending_closures27622026/11/12 00:33:10 INFO Received uploads request method=POST path=/27632026/11/12 00:33:10 INFO Vacuumed table table=pending_objects27642026/11/12 00:33:10 INFO Vacuumed table table=multipart_uploads27652026/11/12 00:33:10 INFO Vacuumed table table=objects27662026/11/12 00:33:10 INFO Received uploads request method=POST path=/27672026/11/12 00:33:10 INFO Vacuumed table table=closures27682026/11/12 00:33:10 INFO Vacuumed table table=objects27692026/11/12 00:33:10 INFO Received uploads request method=POST path=/27702026/11/12 00:33:10 INFO Received push request method=POST path=/27712026/11/12 00:33:10 INFO Received push request method=POST path=/27722026/11/12 00:33:10 INFO Received complete push request method=POST path=/27732026/11/12 00:33:10 INFO Completed push id=2 objects=10 duration=3.442452ms27742026/11/12 00:33:10 INFO Received complete push request method=POST path=/27752026/11/12 00:33:10 INFO Received push request method=POST path=/27762026/11/12 00:33:10 INFO Received complete push request method=POST path=/27772026/11/12 00:33:10 INFO Completed push id=7 objects=20 duration=2.599441ms27782026/11/12 00:33:10 INFO Received uploads request method=POST path=/27792026/11/12 00:33:10 INFO Completed push id=3 objects=5 duration=7.029163ms27802026/11/12 00:33:10 INFO Received uploads request method=POST path=/27812026/11/12 00:33:10 INFO Received complete upload request method=POST path=/27822026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=027832026/11/12 00:33:10 INFO Completed upload id=3 objects=9 duration=1.529581ms27842026/11/12 00:33:10 INFO Received push request method=POST path=/27852026/11/12 00:33:10 WARN Force mode enabled - objects will be deleted immediately without grace period27862026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=027872026/11/12 00:33:10 WARN Force mode enabled - objects will be deleted immediately without grace period27882026/11/12 00:33:10 INFO Received uploads request method=POST path=/27892026/11/12 00:33:10 INFO Received push request method=POST path=/27902026/11/12 00:33:10 INFO Received complete push request method=POST path=/27912026/11/12 00:33:10 INFO Received push request method=POST path=/27922026/11/12 00:33:10 INFO Received complete push request method=POST path=/27932026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=027942026/11/12 00:33:10 INFO Completed push id=2 objects=7 duration=4.966232ms27952026/11/12 00:33:10 INFO Received push request method=POST path=/27962026/11/12 00:33:10 INFO Completed push id=3 objects=22 duration=6.182432ms27972026/11/12 00:33:10 INFO Received push request method=POST path=/27982026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=027992026/11/12 00:33:10 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."28002026/11/12 00:33:10 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."28012026/11/12 00:33:10 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."28022026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=028032026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=028042026/11/12 00:33:10 INFO Received push request method=POST path=/28052026/11/12 00:33:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028062026/11/12 00:33:10 INFO Vacuumed table table=pending_closures28072026/11/12 00:33:10 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028082026/11/12 00:33:10 INFO Vacuumed table table=pending_objects28092026/11/12 00:33:10 INFO Vacuumed table table=pending_closures28102026/11/12 00:33:10 INFO Vacuumed table table=multipart_uploads28112026/11/12 00:33:10 INFO Received push request method=POST path=/28122026/11/12 00:33:10 INFO Received complete push request method=POST path=/28132026/11/12 00:33:10 INFO Vacuumed table table=closures28142026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=028152026/11/12 00:33:10 INFO Received push request method=POST path=/28162026/11/12 00:33:10 WARN Force mode enabled - objects will be deleted immediately without grace period28172026/11/12 00:33:10 INFO Vacuumed table table=objects28182026/11/12 00:33:10 INFO Vacuumed table table=pending_objects28192026/11/12 00:33:10 INFO Vacuumed table table=multipart_uploads28202026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=028212026/11/12 00:33:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028222026/11/12 00:33:10 INFO Received complete push request method=POST path=/28232026/11/12 00:33:10 INFO Completed push id=5 objects=9 duration=5.589913ms28242026/11/12 00:33:10 INFO Vacuumed table table=closures28252026/11/12 00:33:10 INFO Received complete push request method=POST path=/28262026/11/12 00:33:10 INFO Vacuumed table table=pending_closures28272026/11/12 00:33:10 INFO Vacuumed table table=objects28282026/11/12 00:33:10 INFO Completed push id=8 objects=7 duration=2.19981ms28292026/11/12 00:33:10 INFO Completed push id=5 objects=6 duration=1.836121ms28302026/11/12 00:33:10 INFO Received complete upload request method=POST path=/28312026/11/12 00:33:10 INFO Vacuumed table table=pending_objects28322026/11/12 00:33:10 INFO Vacuumed table table=multipart_uploads28332026/11/12 00:33:10 INFO Vacuumed table table=closures28342026/11/12 00:33:10 INFO Completed upload id=4 objects=7 duration=2.426891ms28352026/11/12 00:33:10 INFO Vacuumed table table=objects28362026/11/12 00:33:10 INFO Received complete push request method=POST path=/28372026/11/12 00:33:10 INFO Completed push id=5 objects=3 duration=2.119771ms28382026/11/12 00:33:10 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=028392026/11/12 00:33:10 INFO Vacuumed table table=pending_closures28402026/11/12 00:33:10 INFO Received uploads request method=POST path=/28412026/11/12 00:33:10 INFO Vacuumed table table=pending_objects28422026/11/12 00:33:10 INFO Vacuumed table table=multipart_uploads28432026/11/12 00:33:10 INFO Vacuumed table table=closures28442026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=028452026/11/12 00:33:10 INFO Received push request method=POST path=/28462026/11/12 00:33:10 INFO Vacuumed table table=objects28472026/11/12 00:33:10 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."28482026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."28492026/11/12 00:33:10 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."28502026/11/12 00:33:10 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."28512026/11/12 00:33:10 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."28522026/11/12 00:33:10 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."28532026/11/12 00:33:10 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."28542026/11/12 00:33:10 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."28552026/11/12 00:33:10 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."28562026/11/12 00:33:10 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."28572026/11/12 00:33:10 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."28582026/11/12 00:33:10 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."28592026/11/12 00:33:10 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."28602026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:aaaaaaaa!out.doi error="Access Denied."28612026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."28622026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."28632026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."28642026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."28652026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."28662026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."28672026/11/12 00:33:10 INFO Received push request method=POST path=/28682026/11/12 00:33:10 WARN Force mode enabled - objects will be deleted immediately without grace period28692026/11/12 00:33:10 INFO Received uploads request method=POST path=/28702026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=028712026/11/12 00:33:10 INFO Received uploads request method=POST path=/28722026/11/12 00:33:10 WARN Force mode enabled - objects will be deleted immediately without grace period28732026/11/12 00:33:10 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."28742026/11/12 00:33:10 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."28752026/11/12 00:33:10 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."28762026/11/12 00:33:10 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."28772026/11/12 00:33:10 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."28782026/11/12 00:33:10 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."28792026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."28802026/11/12 00:33:10 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."28812026/11/12 00:33:10 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."28822026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."28832026/11/12 00:33:10 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."28842026/11/12 00:33:10 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."28852026/11/12 00:33:10 INFO Received complete push request method=POST path=/28862026/11/12 00:33:10 INFO Aborted multipart uploads count=0 kept=028872026/11/12 00:33:10 INFO Received push request method=POST path=/28882026/11/12 00:33:10 INFO Completed push id=5 objects=3 duration=3.502421ms28892026/11/12 00:33:10 WARN Force mode enabled - objects will be deleted immediately without grace period28902026/11/12 00:33:10 INFO Received push request method=POST path=/28912026/11/12 00:33:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028922026/11/12 00:33:10 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."28932026/11/12 00:33:10 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."28942026/11/12 00:33:10 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."28952026/11/12 00:33:10 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."28962026/11/12 00:33:10 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."28972026/11/12 00:33:10 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."28982026/11/12 00:33:10 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."28992026/11/12 00:33:10 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."29002026/11/12 00:33:10 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."29012026/11/12 00:33:10 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."29022026/11/12 00:33:10 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."29032026/11/12 00:33:10 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."29042026/11/12 00:33:10 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."29052026/11/12 00:33:10 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."29062026/11/12 00:33:10 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."29072026/11/12 00:33:10 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."29082026/11/12 00:33:10 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."29092026/11/12 00:33:10 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."29102026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."29112026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."29122026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."29132026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."29142026/11/12 00:33:10 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."29152026/11/12 00:33:10 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."29162026/11/12 00:33:10 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."29172026/11/12 00:33:10 INFO Vacuumed table table=pending_closures29182026/11/12 00:33:10 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."29192026/11/12 00:33:10 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."29202026/11/12 00:33:10 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."29212026/11/12 00:33:10 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."29222026/11/12 00:33:10 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."29232026/11/12 00:33:10 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."29242026/11/12 00:33:10 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."29252026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."29262026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."29272026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."29282026/11/12 00:33:10 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."29292026/11/12 00:33:10 INFO Vacuumed table table=pending_objects29302026/11/12 00:33:10 INFO Vacuumed table table=multipart_uploads29312026/11/12 00:33:10 INFO Vacuumed table table=closures29322026/11/12 00:33:10 INFO Vacuumed table table=objects29332026/11/12 00:33:10 INFO Received uploads request method=POST path=/29342026/11/12 00:33:10 INFO Received uploads request method=POST path=/29352026/11/12 00:33:10 INFO Received uploads request method=POST path=/29362026/11/12 00:33:10 INFO Received uploads request method=POST path=/29372026/11/12 00:33:11 INFO Received uploads request method=POST path=/29382026/11/12 00:33:11 INFO Received complete push request method=POST path=/29392026/11/12 00:33:11 INFO Received push request method=POST path=/29402026/11/12 00:33:11 INFO Completed push id=10 objects=2 duration=2.592971ms29412026/11/12 00:33:11 INFO Received push request method=POST path=/29422026/11/12 00:33:11 INFO Received complete upload request method=POST path=/29432026/11/12 00:33:11 INFO Received push request method=POST path=/29442026/11/12 00:33:11 INFO Completed upload id=9 objects=16 duration=1.476231ms29452026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=029462026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=4 objects-deleted-after-grace-period=0 objects-failed-to-delete=029472026/11/12 00:33:11 INFO Received push request method=POST path=/29482026/11/12 00:33:11 INFO Received complete push request method=POST path=/29492026/11/12 00:33:11 INFO Vacuumed table table=pending_closures29502026/11/12 00:33:11 INFO Received uploads request method=POST path=/29512026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=029522026/11/12 00:33:11 INFO Vacuumed table table=pending_objects29532026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads29542026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period29552026/11/12 00:33:11 INFO Completed push id=14 objects=3 duration=3.709261ms29562026/11/12 00:33:11 INFO Vacuumed table table=closures29572026/11/12 00:33:11 INFO Received uploads request method=POST path=/29582026/11/12 00:33:11 INFO Received push request method=POST path=/29592026/11/12 00:33:11 INFO Vacuumed table table=objects29602026/11/12 00:33:11 INFO Received push request method=POST path=/29612026/11/12 00:33:11 INFO Received complete upload request method=POST path=/29622026/11/12 00:33:11 INFO Completed upload id=12 objects=21 duration=1.51011ms29632026/11/12 00:33:11 INFO Received uploads request method=POST path=/29642026/11/12 00:33:11 INFO Received complete push request method=POST path=/29652026/11/12 00:33:11 INFO Completed push id=8 objects=2 duration=1.894011ms29662026/11/12 00:33:11 INFO Received push request method=POST path=/29672026/11/12 00:33:11 INFO Received uploads request method=POST path=/29682026/11/12 00:33:11 INFO Received complete upload request method=POST path=/29692026/11/12 00:33:11 INFO Completed upload id=2 objects=2 duration=1.65289ms29702026/11/12 00:33:11 INFO Received push request method=POST path=/29712026/11/12 00:33:11 INFO Received push request method=POST path=/29722026/11/12 00:33:11 INFO Received push request method=POST path=/29732026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=029742026/11/12 00:33:11 INFO Received uploads request method=POST path=/29752026/11/12 00:33:11 INFO Received push request method=POST path=/29762026/11/12 00:33:11 INFO Vacuumed table table=pending_closures29772026/11/12 00:33:11 INFO Received push request method=POST path=/29782026/11/12 00:33:11 INFO Vacuumed table table=pending_objects29792026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads29802026/11/12 00:33:11 INFO Vacuumed table table=closures29812026/11/12 00:33:11 INFO Received push request method=POST path=/29822026/11/12 00:33:11 INFO Vacuumed table table=objects29832026/11/12 00:33:11 INFO Received push request method=POST path=/29842026/11/12 00:33:11 INFO Received push request method=POST path=/29852026/11/12 00:33:11 INFO Received uploads request method=POST path=/29862026/11/12 00:33:11 INFO Received push request method=POST path=/29872026/11/12 00:33:11 INFO Received complete push request method=POST path=/29882026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=029892026/11/12 00:33:11 INFO Completed push id=5 objects=6 duration=1.91601ms29902026/11/12 00:33:11 INFO Received complete push request method=POST path=/29912026/11/12 00:33:11 INFO Vacuumed table table=pending_closures29922026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=029932026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=029942026/11/12 00:33:11 INFO Completed push id=6 objects=5 duration=2.081191ms29952026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period29962026/11/12 00:33:11 INFO Received complete push request method=POST path=/29972026/11/12 00:33:11 INFO Vacuumed table table=pending_objects29982026/11/12 00:33:11 INFO Received push request method=POST path=/29992026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads30002026/11/12 00:33:11 INFO Received push request method=POST path=/30012026/11/12 00:33:11 INFO Vacuumed table table=closures30022026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030032026/11/12 00:33:11 INFO Completed push id=8 objects=6 duration=1.924571ms30042026/11/12 00:33:11 INFO Vacuumed table table=objects30052026/11/12 00:33:11 INFO Received push request method=POST path=/30062026/11/12 00:33:11 INFO Vacuumed table table=pending_closures30072026/11/12 00:33:11 INFO Vacuumed table table=pending_objects30082026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads30092026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=030102026/11/12 00:33:11 INFO Vacuumed table table=closures30112026/11/12 00:33:11 INFO Received complete push request method=POST path=/30122026/11/12 00:33:11 INFO Vacuumed table table=pending_closures30132026/11/12 00:33:11 INFO Vacuumed table table=objects30142026/11/12 00:33:11 INFO Completed push id=10 objects=3 duration=1.699821ms30152026/11/12 00:33:11 INFO Vacuumed table table=pending_objects30162026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads30172026/11/12 00:33:11 INFO Vacuumed table table=closures30182026/11/12 00:33:11 INFO Received complete push request method=POST path=/30192026/11/12 00:33:11 INFO Vacuumed table table=objects30202026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=030212026/11/12 00:33:11 INFO Completed push id=13 objects=23 duration=2.208361ms30222026/11/12 00:33:11 INFO Received push request method=POST path=/30232026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period30242026/11/12 00:33:11 INFO Received push request method=POST path=/30252026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030262026/11/12 00:33:11 INFO Vacuumed table table=pending_closures30272026/11/12 00:33:11 INFO Received complete upload request method=POST path=/30282026/11/12 00:33:11 INFO Vacuumed table table=pending_objects30292026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads30302026/11/12 00:33:11 INFO Vacuumed table table=closures30312026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=030322026/11/12 00:33:11 INFO Received complete upload request method=POST path=/30332026/11/12 00:33:11 INFO Received uploads request method=POST path=/30342026/11/12 00:33:11 INFO Completed upload id=4 objects=14 duration=3.558281ms30352026/11/12 00:33:11 INFO Received uploads request method=POST path=/30362026/11/12 00:33:11 INFO Vacuumed table table=objects30372026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period30382026/11/12 00:33:11 INFO Completed upload id=6 objects=24 duration=3.932291ms30392026/11/12 00:33:11 INFO Received push request method=POST path=/30402026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030412026/11/12 00:33:11 INFO Received uploads request method=POST path=/30422026/11/12 00:33:11 INFO Vacuumed table table=pending_closures30432026/11/12 00:33:11 INFO Vacuumed table table=pending_objects30442026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads30452026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=030462026/11/12 00:33:11 INFO Vacuumed table table=closures30472026/11/12 00:33:11 INFO Vacuumed table table=objects30482026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period30492026/11/12 00:33:11 INFO Received push request method=POST path=/30502026/11/12 00:33:11 INFO Received push request method=POST path=/30512026/11/12 00:33:11 INFO Received uploads request method=POST path=/30522026/11/12 00:33:11 INFO Received uploads request method=POST path=/30532026/11/12 00:33:11 INFO Received push request method=POST path=/30542026/11/12 00:33:11 INFO Received push request method=POST path=/30552026/11/12 00:33:11 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."30562026/11/12 00:33:11 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."30572026/11/12 00:33:11 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."30582026/11/12 00:33:11 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."30592026/11/12 00:33:11 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."30602026/11/12 00:33:11 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."30612026/11/12 00:33:11 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."30622026/11/12 00:33:11 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."30632026/11/12 00:33:11 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."30642026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=030652026/11/12 00:33:11 INFO Received push request method=POST path=/30662026/11/12 00:33:11 INFO Received uploads request method=POST path=/30672026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period30682026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030692026/11/12 00:33:11 INFO Vacuumed table table=pending_closures30702026/11/12 00:33:11 INFO Vacuumed table table=pending_objects30712026/11/12 00:33:11 INFO Received complete upload request method=POST path=/30722026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads30732026/11/12 00:33:11 INFO Received push request method=POST path=/30742026/11/12 00:33:11 INFO Vacuumed table table=closures30752026/11/12 00:33:11 INFO Vacuumed table table=objects30762026/11/12 00:33:11 INFO Received uploads request method=POST path=/30772026/11/12 00:33:11 INFO Completed upload id=10 objects=2 duration=2.853141ms30782026/11/12 00:33:11 INFO Received uploads request method=POST path=/30792026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=030802026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period30812026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=030822026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030832026/11/12 00:33:11 INFO Received push request method=POST path=/30842026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period30852026/11/12 00:33:11 INFO Vacuumed table table=pending_closures30862026/11/12 00:33:11 INFO Received uploads request method=POST path=/30872026/11/12 00:33:11 INFO Received push request method=POST path=/30882026/11/12 00:33:11 INFO Vacuumed table table=pending_objects30892026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads30902026/11/12 00:33:11 INFO Vacuumed table table=closures30912026/11/12 00:33:11 INFO Vacuumed table table=objects30922026/11/12 00:33:11 INFO Received complete push request method=POST path=/30932026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=030942026/11/12 00:33:11 INFO Completed push id=8 objects=6 duration=2.265761ms30952026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=030962026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period30972026/11/12 00:33:11 INFO Received push request method=POST path=/30982026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=030992026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031002026/11/12 00:33:11 INFO Vacuumed table table=pending_closures31012026/11/12 00:33:11 INFO Received push request method=POST path=/31022026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031032026/11/12 00:33:11 INFO Vacuumed table table=pending_closures31042026/11/12 00:33:11 INFO Vacuumed table table=pending_objects31052026/11/12 00:33:11 INFO Vacuumed table table=pending_objects31062026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads31072026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads31082026/11/12 00:33:11 INFO Received uploads request method=POST path=/31092026/11/12 00:33:11 INFO Vacuumed table table=closures31102026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=031112026/11/12 00:33:11 INFO Vacuumed table table=closures31122026/11/12 00:33:11 INFO Vacuumed table table=objects31132026/11/12 00:33:11 INFO Received push request method=POST path=/31142026/11/12 00:33:11 INFO Vacuumed table table=objects31152026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=9 objects-failed-to-delete=031162026/11/12 00:33:11 INFO Vacuumed table table=pending_closures31172026/11/12 00:33:11 INFO Vacuumed table table=pending_objects31182026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads31192026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=031202026/11/12 00:33:11 INFO Vacuumed table table=closures31212026/11/12 00:33:11 INFO Vacuumed table table=objects31222026/11/12 00:33:11 INFO Vacuumed table table=pending_closures31232026/11/12 00:33:11 INFO Received uploads request method=POST path=/31242026/11/12 00:33:11 INFO Vacuumed table table=pending_objects31252026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads31262026/11/12 00:33:11 INFO Vacuumed table table=closures31272026/11/12 00:33:11 INFO Vacuumed table table=objects31282026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=031292026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=031302026/11/12 00:33:11 INFO Vacuumed table table=pending_closures31312026/11/12 00:33:11 INFO Received complete upload request method=POST path=/31322026/11/12 00:33:11 INFO Vacuumed table table=pending_objects31332026/11/12 00:33:11 INFO Received push request method=POST path=/31342026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period31352026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads31362026/11/12 00:33:11 INFO Completed upload id=9 objects=3 duration=1.73136ms31372026/11/12 00:33:11 INFO Received push request method=POST path=/31382026/11/12 00:33:11 INFO Received push request method=POST path=/31392026/11/12 00:33:11 INFO Vacuumed table table=closures31402026/11/12 00:33:11 INFO Vacuumed table table=objects31412026/11/12 00:33:11 INFO Received uploads request method=POST path=/31422026/11/12 00:33:11 INFO Received push request method=POST path=/31432026/11/12 00:33:11 INFO Received push request method=POST path=/31442026/11/12 00:33:11 INFO Received uploads request method=POST path=/31452026/11/12 00:33:11 INFO Received complete push request method=POST path=/31462026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=4 objects-failed-to-delete=031472026/11/12 00:33:11 INFO Vacuumed table table=pending_closures31482026/11/12 00:33:11 INFO Completed push id=11 objects=11 duration=3.829591ms31492026/11/12 00:33:11 INFO Vacuumed table table=pending_objects31502026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads31512026/11/12 00:33:11 INFO Vacuumed table table=closures31522026/11/12 00:33:11 INFO Received push request method=POST path=/31532026/11/12 00:33:11 INFO Received uploads request method=POST path=/31542026/11/12 00:33:11 INFO Vacuumed table table=objects31552026/11/12 00:33:11 INFO Received push request method=POST path=/31562026/11/12 00:33:11 INFO Received push request method=POST path=/31572026/11/12 00:33:11 INFO Received complete push request method=POST path=/31582026/11/12 00:33:11 INFO Received push request method=POST path=/31592026/11/12 00:33:11 INFO Completed push id=8 objects=6 duration=3.354061ms31602026/11/12 00:33:11 INFO Received push request method=POST path=/31612026/11/12 00:33:11 INFO Received push request method=POST path=/31622026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=031632026/11/12 00:33:11 INFO Received complete upload request method=POST path=/31642026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031652026/11/12 00:33:11 INFO Received push request method=POST path=/31662026/11/12 00:33:11 INFO Received complete upload request method=POST path=/31672026/11/12 00:33:11 INFO Completed upload id=10 objects=6 duration=3.022051ms31682026/11/12 00:33:11 INFO Vacuumed table table=pending_closures31692026/11/12 00:33:11 INFO Completed upload id=10 objects=17 duration=1.764851ms31702026/11/12 00:33:11 INFO Vacuumed table table=pending_objects31712026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads31722026/11/12 00:33:11 INFO Vacuumed table table=closures31732026/11/12 00:33:11 INFO Received uploads request method=POST path=/31742026/11/12 00:33:11 INFO Vacuumed table table=objects31752026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=031762026/11/12 00:33:11 INFO Received uploads request method=POST path=/31772026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=031782026/11/12 00:33:11 INFO Received uploads request method=POST path=/31792026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period31802026/11/12 00:33:11 INFO Received uploads request method=POST path=/31812026/11/12 00:33:11 INFO Received push request method=POST path=/31822026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031832026/11/12 00:33:11 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."31842026/11/12 00:33:11 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."31852026/11/12 00:33:11 INFO Vacuumed table table=pending_closures31862026/11/12 00:33:11 INFO Vacuumed table table=pending_objects31872026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads31882026/11/12 00:33:11 INFO Vacuumed table table=closures31892026/11/12 00:33:11 INFO Received complete upload request method=POST path=/31902026/11/12 00:33:11 INFO Vacuumed table table=objects31912026/11/12 00:33:11 INFO Received complete push request method=POST path=/31922026/11/12 00:33:11 INFO Completed upload id=10 objects=23 duration=2.269791ms31932026/11/12 00:33:11 INFO Received uploads request method=POST path=/31942026/11/12 00:33:11 INFO Received uploads request method=POST path=/31952026/11/12 00:33:11 INFO Received complete push request method=POST path=/31962026/11/12 00:33:11 INFO Completed push id=6 objects=26 duration=4.495212ms31972026/11/12 00:33:11 INFO Received push request method=POST path=/31982026/11/12 00:33:11 INFO Completed push id=5 objects=8 duration=3.751281ms31992026/11/12 00:33:11 INFO Received complete push request method=POST path=/32002026/11/12 00:33:11 INFO Received uploads request method=POST path=/32012026/11/12 00:33:11 INFO Completed push id=4 objects=6 duration=2.442421ms32022026/11/12 00:33:11 INFO Received push request method=POST path=/32032026/11/12 00:33:11 INFO Received complete push request method=POST path=/32042026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=032052026/11/12 00:33:11 INFO Received uploads request method=POST path=/32062026/11/12 00:33:11 INFO Completed push id=11 objects=8 duration=3.299221ms32072026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032082026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=032092026/11/12 00:33:11 INFO Vacuumed table table=pending_closures32102026/11/12 00:33:11 INFO Received push request method=POST path=/32112026/11/12 00:33:11 INFO Received uploads request method=POST path=/32122026/11/12 00:33:11 INFO Vacuumed table table=pending_objects32132026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads32142026/11/12 00:33:11 INFO Vacuumed table table=closures32152026/11/12 00:33:11 INFO Received push request method=POST path=/32162026/11/12 00:33:11 INFO Vacuumed table table=objects32172026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032182026/11/12 00:33:11 INFO Vacuumed table table=pending_closures32192026/11/12 00:33:11 INFO Vacuumed table table=pending_objects32202026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads32212026/11/12 00:33:11 INFO Vacuumed table table=closures32222026/11/12 00:33:11 INFO Received push request method=POST path=/32232026/11/12 00:33:11 INFO Vacuumed table table=objects32242026/11/12 00:33:11 INFO Received uploads request method=POST path=/32252026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=032262026/11/12 00:33:11 INFO Received push request method=POST path=/32272026/11/12 00:33:11 INFO Received uploads request method=POST path=/32282026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032292026/11/12 00:33:11 INFO Vacuumed table table=pending_closures32302026/11/12 00:33:11 INFO Received uploads request method=POST path=/32312026/11/12 00:33:11 INFO Received push request method=POST path=/32322026/11/12 00:33:11 INFO Vacuumed table table=pending_objects32332026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads32342026/11/12 00:33:11 INFO Received uploads request method=POST path=/32352026/11/12 00:33:11 INFO Vacuumed table table=closures32362026/11/12 00:33:11 INFO Received uploads request method=POST path=/32372026/11/12 00:33:11 INFO Vacuumed table table=objects32382026/11/12 00:33:11 INFO Received push request method=POST path=/32392026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=032402026/11/12 00:33:11 INFO Received uploads request method=POST path=/32412026/11/12 00:33:11 INFO Received complete upload request method=POST path=/32422026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=032432026/11/12 00:33:11 INFO Completed upload id=15 objects=4 duration=2.47916ms32442026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period32452026/11/12 00:33:11 INFO Received push request method=POST path=/32462026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=032472026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032482026/11/12 00:33:11 INFO Vacuumed table table=pending_closures32492026/11/12 00:33:11 INFO Vacuumed table table=pending_closures32502026/11/12 00:33:11 INFO Vacuumed table table=pending_objects32512026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads32522026/11/12 00:33:11 INFO Vacuumed table table=closures32532026/11/12 00:33:11 INFO Vacuumed table table=pending_objects32542026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads32552026/11/12 00:33:11 INFO Vacuumed table table=objects32562026/11/12 00:33:11 INFO Received complete push request method=POST path=/32572026/11/12 00:33:11 INFO Vacuumed table table=closures32582026/11/12 00:33:11 INFO Received complete upload request method=POST path=/32592026/11/12 00:33:11 INFO Vacuumed table table=objects32602026/11/12 00:33:11 INFO Completed push id=12 objects=3 duration=2.149481ms32612026/11/12 00:33:11 INFO Received complete upload request method=POST path=/32622026/11/12 00:33:11 INFO Completed upload id=9 objects=2 duration=1.72556ms32632026/11/12 00:33:11 INFO Completed upload id=11 objects=6 duration=1.603291ms32642026/11/12 00:33:11 INFO Received complete push request method=POST path=/32652026/11/12 00:33:11 INFO Received uploads request method=POST path=/32662026/11/12 00:33:11 INFO Received push request method=POST path=/32672026/11/12 00:33:11 INFO Completed push id=14 objects=6 duration=1.830071ms32682026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=032692026/11/12 00:33:11 INFO Received complete push request method=POST path=/32702026/11/12 00:33:11 INFO Received push request method=POST path=/32712026/11/12 00:33:11 INFO Received push request method=POST path=/32722026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032732026/11/12 00:33:11 INFO Completed push id=13 objects=6 duration=3.413122ms32742026/11/12 00:33:11 INFO Vacuumed table table=pending_closures32752026/11/12 00:33:11 INFO Received complete push request method=POST path=/32762026/11/12 00:33:11 INFO Received uploads request method=POST path=/32772026/11/12 00:33:11 INFO Vacuumed table table=pending_objects32782026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=032792026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads32802026/11/12 00:33:11 INFO Completed push id=9 objects=24 duration=1.950331ms32812026/11/12 00:33:11 INFO Vacuumed table table=closures32822026/11/12 00:33:11 INFO Received complete push request method=POST path=/32832026/11/12 00:33:11 INFO Vacuumed table table=objects32842026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period32852026/11/12 00:33:11 INFO Completed push id=8 objects=3 duration=1.965081ms32862026/11/12 00:33:11 INFO Received push request method=POST path=/32872026/11/12 00:33:11 INFO Received uploads request method=POST path=/32882026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032892026/11/12 00:33:11 INFO Received uploads request method=POST path=/32902026/11/12 00:33:11 INFO Received uploads request method=POST path=/32912026/11/12 00:33:11 INFO Received push request method=POST path=/32922026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=032932026/11/12 00:33:11 INFO Received push request method=POST path=/32942026/11/12 00:33:11 INFO Received push request method=POST path=/32952026/11/12 00:33:11 INFO Received uploads request method=POST path=/32962026/11/12 00:33:11 INFO Vacuumed table table=pending_closures32972026/11/12 00:33:11 INFO Received push request method=POST path=/32982026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period32992026/11/12 00:33:11 INFO Vacuumed table table=pending_objects33002026/11/12 00:33:11 INFO Received complete push request method=POST path=/33012026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads33022026/11/12 00:33:11 INFO Received uploads request method=POST path=/33032026/11/12 00:33:11 INFO Vacuumed table table=closures33042026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033052026/11/12 00:33:11 INFO Vacuumed table table=objects33062026/11/12 00:33:11 INFO Vacuumed table table=pending_closures33072026/11/12 00:33:11 INFO Completed push id=13 objects=12 duration=3.822141ms33082026/11/12 00:33:11 INFO Received complete upload request method=POST path=/33092026/11/12 00:33:11 INFO Vacuumed table table=pending_objects33102026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads33112026/11/12 00:33:11 INFO Vacuumed table table=closures33122026/11/12 00:33:11 INFO Vacuumed table table=objects33132026/11/12 00:33:11 INFO Completed upload id=10 objects=6 duration=3.043471ms33142026/11/12 00:33:11 INFO Received push request method=POST path=/33152026/11/12 00:33:11 INFO Received uploads request method=POST path=/33162026/11/12 00:33:11 INFO Received push request method=POST path=/33172026/11/12 00:33:11 INFO Received uploads request method=POST path=/33182026/11/12 00:33:11 INFO Received complete upload request method=POST path=/33192026/11/12 00:33:11 INFO Received uploads request method=POST path=/33202026/11/12 00:33:11 INFO Received uploads request method=POST path=/33212026/11/12 00:33:11 INFO Completed upload id=16 objects=3 duration=1.18276ms33222026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=033232026/11/12 00:33:11 INFO Received complete upload request method=POST path=/33242026/11/12 00:33:11 INFO Received uploads request method=POST path=/33252026/11/12 00:33:11 INFO Received push request method=POST path=/33262026/11/12 00:33:11 INFO Received uploads request method=POST path=/33272026/11/12 00:33:11 INFO Completed upload id=17 objects=6 duration=2.969471ms33282026/11/12 00:33:11 INFO Received complete push request method=POST path=/33292026/11/12 00:33:11 INFO Received complete upload request method=POST path=/33302026/11/12 00:33:11 INFO Completed push id=12 objects=15 duration=2.360311ms33312026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=033322026/11/12 00:33:11 INFO Completed upload id=11 objects=6 duration=2.631681ms33332026/11/12 00:33:11 INFO Vacuumed table table=pending_closures33342026/11/12 00:33:11 INFO Vacuumed table table=pending_objects33352026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads33362026/11/12 00:33:11 INFO Vacuumed table table=closures33372026/11/12 00:33:11 INFO Vacuumed table table=objects33382026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=033392026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=033402026/11/12 00:33:11 INFO Received complete upload request method=POST path=/33412026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=033422026/11/12 00:33:11 INFO Received uploads request method=POST path=/33432026/11/12 00:33:11 INFO Completed upload id=13 objects=10 duration=1.608301ms33442026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=033452026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033462026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=033472026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period33482026/11/12 00:33:11 INFO Vacuumed table table=pending_closures33492026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period33502026/11/12 00:33:11 INFO Vacuumed table table=pending_closures33512026/11/12 00:33:11 INFO Received complete upload request method=POST path=/33522026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=033532026/11/12 00:33:11 INFO Completed upload id=10 objects=10 duration=1.51842ms33542026/11/12 00:33:11 INFO Vacuumed table table=pending_objects33552026/11/12 00:33:11 INFO Vacuumed table table=pending_objects33562026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033572026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads33582026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads33592026/11/12 00:33:11 INFO Vacuumed table table=pending_closures33602026/11/12 00:33:11 INFO Received uploads request method=POST path=/33612026/11/12 00:33:11 INFO Vacuumed table table=closures33622026/11/12 00:33:11 INFO Vacuumed table table=closures33632026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033642026/11/12 00:33:11 INFO Vacuumed table table=objects33652026/11/12 00:33:11 INFO Vacuumed table table=objects33662026/11/12 00:33:11 INFO Vacuumed table table=pending_objects33672026/11/12 00:33:11 INFO Received push request method=POST path=/33682026/11/12 00:33:11 INFO Vacuumed table table=pending_closures33692026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads33702026/11/12 00:33:11 INFO Vacuumed table table=closures33712026/11/12 00:33:11 INFO Vacuumed table table=objects33722026/11/12 00:33:11 INFO Vacuumed table table=pending_objects33732026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads33742026/11/12 00:33:11 INFO Vacuumed table table=closures33752026/11/12 00:33:11 INFO Vacuumed table table=objects33762026/11/12 00:33:11 INFO Received complete push request method=POST path=/33772026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=033782026/11/12 00:33:11 INFO Received push request method=POST path=/33792026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period33802026/11/12 00:33:11 INFO Completed push id=17 objects=6 duration=2.894741ms33812026/11/12 00:33:11 INFO Received push request method=POST path=/33822026/11/12 00:33:11 INFO Received uploads request method=POST path=/33832026/11/12 00:33:11 INFO Received push request method=POST path=/33842026/11/12 00:33:11 INFO Received push request method=POST path=/33852026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=033862026/11/12 00:33:11 INFO Received uploads request method=POST path=/33872026/11/12 00:33:11 INFO Received push request method=POST path=/33882026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period33892026/11/12 00:33:11 INFO Received push request method=POST path=/33902026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033912026/11/12 00:33:11 INFO Vacuumed table table=pending_closures33922026/11/12 00:33:11 INFO Vacuumed table table=pending_objects33932026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads33942026/11/12 00:33:11 INFO Vacuumed table table=closures33952026/11/12 00:33:11 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."33962026/11/12 00:33:11 INFO Vacuumed table table=objects33972026/11/12 00:33:11 INFO Received complete push request method=POST path=/33982026/11/12 00:33:11 INFO Completed push id=8 objects=6 duration=1.95196ms33992026/11/12 00:33:11 INFO Received complete upload request method=POST path=/34002026/11/12 00:33:11 INFO Completed upload id=8 objects=3 duration=2.40832ms34012026/11/12 00:33:11 INFO Received uploads request method=POST path=/34022026/11/12 00:33:11 INFO Received complete upload request method=POST path=/34032026/11/12 00:33:11 INFO Completed upload id=16 objects=23 duration=1.446951ms34042026/11/12 00:33:11 INFO Received push request method=POST path=/34052026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=034062026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=034072026/11/12 00:33:11 INFO Received push request method=POST path=/34082026/11/12 00:33:11 INFO Received complete push request method=POST path=/34092026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034102026/11/12 00:33:11 INFO Vacuumed table table=pending_closures34112026/11/12 00:33:11 INFO Completed push id=4 objects=7 duration=2.311881ms34122026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034132026/11/12 00:33:11 INFO Vacuumed table table=pending_objects34142026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads34152026/11/12 00:33:11 INFO Vacuumed table table=pending_closures34162026/11/12 00:33:11 INFO Vacuumed table table=closures34172026/11/12 00:33:11 INFO Vacuumed table table=pending_objects34182026/11/12 00:33:11 INFO Vacuumed table table=objects34192026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads34202026/11/12 00:33:11 INFO Received uploads request method=POST path=/34212026/11/12 00:33:11 INFO Received complete push request method=POST path=/34222026/11/12 00:33:11 INFO Vacuumed table table=closures34232026/11/12 00:33:11 INFO Vacuumed table table=objects34242026/11/12 00:33:11 INFO Completed push id=12 objects=7 duration=2.013011ms34252026/11/12 00:33:11 INFO Received complete upload request method=POST path=/34262026/11/12 00:33:11 INFO Received push request method=POST path=/34272026/11/12 00:33:11 INFO Completed upload id=15 objects=6 duration=1.433271ms34282026/11/12 00:33:11 INFO Received complete push request method=POST path=/34292026/11/12 00:33:11 INFO Received complete upload request method=POST path=/34302026/11/12 00:33:11 INFO Received uploads request method=POST path=/34312026/11/12 00:33:11 INFO Received push request method=POST path=/34322026/11/12 00:33:11 INFO Received uploads request method=POST path=/34332026/11/12 00:33:11 INFO Completed upload id=18 objects=4 duration=2.625011ms34342026/11/12 00:33:11 INFO Completed push id=9 objects=5 duration=2.956811ms34352026/11/12 00:33:11 INFO Received complete push request method=POST path=/34362026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=11 objects-failed-to-delete=034372026/11/12 00:33:11 INFO Received push request method=POST path=/34382026/11/12 00:33:11 INFO Vacuumed table table=pending_closures34392026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=034402026/11/12 00:33:11 INFO Vacuumed table table=pending_objects34412026/11/12 00:33:11 INFO Completed push id=19 objects=6 duration=4.020482ms34422026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads34432026/11/12 00:33:11 INFO Vacuumed table table=closures34442026/11/12 00:33:11 INFO Vacuumed table table=objects34452026/11/12 00:33:11 INFO Received uploads request method=POST path=/34462026/11/12 00:33:11 INFO Received push request method=POST path=/34472026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=034482026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period34492026/11/12 00:33:11 INFO Received uploads request method=POST path=/34502026/11/12 00:33:11 INFO Received push request method=POST path=/34512026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034522026/11/12 00:33:11 INFO Vacuumed table table=pending_closures34532026/11/12 00:33:11 INFO Received complete upload request method=POST path=/34542026/11/12 00:33:11 INFO Received uploads request method=POST path=/34552026/11/12 00:33:11 INFO Vacuumed table table=pending_objects34562026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads34572026/11/12 00:33:11 INFO Vacuumed table table=closures34582026/11/12 00:33:11 INFO Completed upload id=19 objects=25 duration=2.933191ms34592026/11/12 00:33:11 INFO Vacuumed table table=objects34602026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=034612026/11/12 00:33:11 INFO Received uploads request method=POST path=/34622026/11/12 00:33:11 INFO Received push request method=POST path=/34632026/11/12 00:33:11 INFO Vacuumed table table=pending_closures34642026/11/12 00:33:11 INFO Vacuumed table table=pending_objects34652026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads34662026/11/12 00:33:11 INFO Vacuumed table table=closures34672026/11/12 00:33:11 INFO Received uploads request method=POST path=/34682026/11/12 00:33:11 INFO Vacuumed table table=objects34692026/11/12 00:33:11 INFO Received complete push request method=POST path=/34702026/11/12 00:33:11 INFO Received complete upload request method=POST path=/34712026/11/12 00:33:11 INFO Completed push id=3 objects=21 duration=4.467311ms34722026/11/12 00:33:11 INFO Received complete push request method=POST path=/34732026/11/12 00:33:11 INFO Completed upload id=17 objects=7 duration=1.220131ms34742026/11/12 00:33:11 INFO Received complete push request method=POST path=/34752026/11/12 00:33:11 INFO Completed push id=6 objects=6 duration=2.16802ms34762026/11/12 00:33:11 INFO Completed push id=17 objects=6 duration=2.073911ms34772026/11/12 00:33:11 INFO Received complete push request method=POST path=/34782026/11/12 00:33:11 INFO Received complete push request method=POST path=/34792026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=034802026/11/12 00:33:11 INFO Received push request method=POST path=/34812026/11/12 00:33:11 INFO Completed push id=12 objects=2 duration=3.165301ms34822026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period34832026/11/12 00:33:11 INFO Completed push id=9 objects=4 duration=3.561411ms34842026/11/12 00:33:11 INFO Received uploads request method=POST path=/34852026/11/12 00:33:11 INFO Received push request method=POST path=/34862026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=034872026/11/12 00:33:11 INFO Vacuumed table table=pending_closures34882026/11/12 00:33:11 INFO Received push request method=POST path=/34892026/11/12 00:33:11 INFO Vacuumed table table=pending_objects34902026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads34912026/11/12 00:33:11 INFO Vacuumed table table=closures34922026/11/12 00:33:11 INFO Vacuumed table table=objects34932026/11/12 00:33:11 INFO Received push request method=POST path=/34942026/11/12 00:33:11 INFO Received uploads request method=POST path=/34952026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=034962026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=034972026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period34982026/11/12 00:33:11 INFO Received uploads request method=POST path=/34992026/11/12 00:33:11 INFO Received uploads request method=POST path=/35002026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=035012026/11/12 00:33:11 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."35022026/11/12 00:33:11 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."35032026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035042026/11/12 00:33:11 INFO Received complete push request method=POST path=/35052026/11/12 00:33:11 INFO Received uploads request method=POST path=/35062026/11/12 00:33:11 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."35072026/11/12 00:33:11 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."35082026/11/12 00:33:11 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."35092026/11/12 00:33:11 INFO Vacuumed table table=pending_closures35102026/11/12 00:33:11 INFO Vacuumed table table=pending_objects35112026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads35122026/11/12 00:33:11 INFO Completed push id=14 objects=4 duration=3.517641ms35132026/11/12 00:33:11 INFO Vacuumed table table=closures35142026/11/12 00:33:11 INFO Received uploads request method=POST path=/35152026/11/12 00:33:11 INFO Vacuumed table table=objects35162026/11/12 00:33:11 INFO Received push request method=POST path=/35172026/11/12 00:33:11 INFO Received complete push request method=POST path=/35182026/11/12 00:33:11 INFO Received push request method=POST path=/35192026/11/12 00:33:11 INFO Completed push id=7 objects=7 duration=2.027081ms35202026/11/12 00:33:11 INFO Received uploads request method=POST path=/35212026/11/12 00:33:11 INFO Received complete push request method=POST path=/35222026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=035232026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period35242026/11/12 00:33:11 INFO Completed push id=14 objects=3 duration=1.749001ms35252026/11/12 00:33:11 INFO Received complete upload request method=POST path=/35262026/11/12 00:33:11 INFO Completed upload id=15 objects=9 duration=1.457661ms35272026/11/12 00:33:11 INFO Received push request method=POST path=/35282026/11/12 00:33:11 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."35292026/11/12 00:33:11 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."35302026/11/12 00:33:11 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."35312026/11/12 00:33:11 INFO Received uploads request method=POST path=/35322026/11/12 00:33:11 INFO Received uploads request method=POST path=/35332026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=035342026/11/12 00:33:11 INFO Received complete push request method=POST path=/35352026/11/12 00:33:11 INFO Received complete push request method=POST path=/35362026/11/12 00:33:11 INFO Received push request method=POST path=/35372026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035382026/11/12 00:33:11 INFO Completed push id=10 objects=15 duration=2.578551ms35392026/11/12 00:33:11 INFO Received uploads request method=POST path=/35402026/11/12 00:33:11 INFO Vacuumed table table=pending_closures35412026/11/12 00:33:11 INFO Completed push id=9 objects=7 duration=4.032772ms35422026/11/12 00:33:11 INFO Vacuumed table table=pending_objects35432026/11/12 00:33:11 INFO Received complete upload request method=POST path=/35442026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads35452026/11/12 00:33:11 INFO Received uploads request method=POST path=/35462026/11/12 00:33:11 INFO Vacuumed table table=closures35472026/11/12 00:33:11 INFO Completed upload id=14 objects=6 duration=1.272051ms35482026/11/12 00:33:11 INFO Received push request method=POST path=/35492026/11/12 00:33:11 INFO Vacuumed table table=objects35502026/11/12 00:33:11 INFO Received uploads request method=POST path=/35512026/11/12 00:33:11 INFO Received uploads request method=POST path=/35522026/11/12 00:33:11 INFO Received uploads request method=POST path=/35532026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=035542026/11/12 00:33:11 INFO Received push request method=POST path=/35552026/11/12 00:33:11 INFO Received uploads request method=POST path=/35562026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035572026/11/12 00:33:11 INFO Vacuumed table table=pending_closures35582026/11/12 00:33:11 INFO Received complete upload request method=POST path=/35592026/11/12 00:33:11 INFO Received complete upload request method=POST path=/35602026/11/12 00:33:11 INFO Vacuumed table table=pending_objects35612026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads35622026/11/12 00:33:11 INFO Completed upload id=10 objects=7 duration=1.715421ms35632026/11/12 00:33:11 INFO Completed upload id=18 objects=11 duration=1.85449ms35642026/11/12 00:33:11 INFO Received complete push request method=POST path=/35652026/11/12 00:33:11 INFO Vacuumed table table=closures35662026/11/12 00:33:11 INFO Vacuumed table table=objects35672026/11/12 00:33:11 INFO Completed push id=21 objects=17 duration=2.30623ms35682026/11/12 00:33:11 INFO Received complete upload request method=POST path=/35692026/11/12 00:33:11 INFO Completed upload id=15 objects=6 duration=1.15688ms35702026/11/12 00:33:11 INFO Received uploads request method=POST path=/35712026/11/12 00:33:11 INFO Received push request method=POST path=/35722026/11/12 00:33:11 INFO Received uploads request method=POST path=/35732026/11/12 00:33:11 INFO Received complete upload request method=POST path=/35742026/11/12 00:33:11 INFO Received uploads request method=POST path=/35752026/11/12 00:33:11 INFO Completed upload id=7 objects=3 duration=1.184271ms35762026/11/12 00:33:11 INFO Received complete push request method=POST path=/35772026/11/12 00:33:11 INFO Received uploads request method=POST path=/35782026/11/12 00:33:11 INFO Completed push id=15 objects=7 duration=2.448431ms35792026/11/12 00:33:11 INFO Received push request method=POST path=/35802026/11/12 00:33:11 INFO Received complete push request method=POST path=/35812026/11/12 00:33:11 INFO Completed push id=19 objects=7 duration=1.76645ms35822026/11/12 00:33:11 INFO Received complete push request method=POST path=/35832026/11/12 00:33:11 INFO Completed push id=15 objects=5 duration=2.004601ms35842026/11/12 00:33:11 INFO Received push request method=POST path=/35852026/11/12 00:33:11 INFO Received push request method=POST path=/35862026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=035872026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=035882026/11/12 00:33:11 INFO Received uploads request method=POST path=/35892026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period35902026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period35912026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035922026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035932026/11/12 00:33:11 INFO Vacuumed table table=pending_closures35942026/11/12 00:33:11 INFO Received push request method=POST path=/35952026/11/12 00:33:11 INFO Received uploads request method=POST path=/35962026/11/12 00:33:11 INFO Vacuumed table table=pending_closures35972026/11/12 00:33:11 INFO Vacuumed table table=pending_objects35982026/11/12 00:33:11 INFO Received push request method=POST path=/35992026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads36002026/11/12 00:33:11 INFO Vacuumed table table=pending_objects36012026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads36022026/11/12 00:33:11 INFO Vacuumed table table=closures36032026/11/12 00:33:11 INFO Vacuumed table table=closures36042026/11/12 00:33:11 INFO Received push request method=POST path=/36052026/11/12 00:33:11 INFO Received complete push request method=POST path=/36062026/11/12 00:33:11 INFO Vacuumed table table=objects36072026/11/12 00:33:11 INFO Received uploads request method=POST path=/36082026/11/12 00:33:11 INFO Vacuumed table table=objects36092026/11/12 00:33:11 INFO Received complete push request method=POST path=/36102026/11/12 00:33:11 INFO Completed push id=17 objects=22 duration=3.061371ms36112026/11/12 00:33:11 INFO Completed push id=18 objects=23 duration=2.16725ms36122026/11/12 00:33:11 INFO Received uploads request method=POST path=/36132026/11/12 00:33:11 INFO Received uploads request method=POST path=/36142026/11/12 00:33:11 INFO Received complete push request method=POST path=/36152026/11/12 00:33:11 INFO Received uploads request method=POST path=/36162026/11/12 00:33:11 INFO Completed push id=15 objects=24 duration=4.436991ms36172026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=036182026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=036192026/11/12 00:33:11 INFO Received push request method=POST path=/36202026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period36212026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036222026/11/12 00:33:11 INFO Vacuumed table table=pending_closures36232026/11/12 00:33:11 INFO Received push request method=POST path=/36242026/11/12 00:33:11 INFO Vacuumed table table=pending_objects36252026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads36262026/11/12 00:33:11 INFO Vacuumed table table=closures36272026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=036282026/11/12 00:33:11 INFO Vacuumed table table=objects36292026/11/12 00:33:11 INFO Received push request method=POST path=/36302026/11/12 00:33:11 INFO Received uploads request method=POST path=/36312026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period36322026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=036332026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=036342026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036352026/11/12 00:33:11 INFO Vacuumed table table=pending_closures36362026/11/12 00:33:11 INFO Vacuumed table table=pending_closures36372026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period36382026/11/12 00:33:11 INFO Vacuumed table table=pending_objects36392026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads36402026/11/12 00:33:11 INFO Vacuumed table table=pending_objects36412026/11/12 00:33:11 INFO Vacuumed table table=closures36422026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads36432026/11/12 00:33:11 INFO Vacuumed table table=closures36442026/11/12 00:33:11 INFO Received complete upload request method=POST path=/36452026/11/12 00:33:11 INFO Vacuumed table table=objects36462026/11/12 00:33:11 INFO Received uploads request method=POST path=/36472026/11/12 00:33:11 INFO Vacuumed table table=objects36482026/11/12 00:33:11 INFO Completed upload id=18 objects=4 duration=1.673991ms36492026/11/12 00:33:11 INFO Received complete push request method=POST path=/36502026/11/12 00:33:11 INFO Completed push id=22 objects=3 duration=2.695491ms36512026/11/12 00:33:11 INFO Received push request method=POST path=/36522026/11/12 00:33:11 INFO Received uploads request method=POST path=/36532026/11/12 00:33:11 INFO Received complete push request method=POST path=/36542026/11/12 00:33:11 INFO Received push request method=POST path=/36552026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=036562026/11/12 00:33:11 INFO Completed push id=16 objects=27 duration=2.107461ms36572026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period36582026/11/12 00:33:11 INFO Received push request method=POST path=/36592026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036602026/11/12 00:33:11 INFO Vacuumed table table=pending_closures36612026/11/12 00:33:11 INFO Received complete push request method=POST path=/36622026/11/12 00:33:11 INFO Received complete push request method=POST path=/36632026/11/12 00:33:11 INFO Vacuumed table table=pending_objects36642026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads36652026/11/12 00:33:11 INFO Completed push id=11 objects=9 duration=2.019761ms36662026/11/12 00:33:11 INFO Vacuumed table table=closures36672026/11/12 00:33:11 INFO Received uploads request method=POST path=/36682026/11/12 00:33:11 INFO Completed push id=16 objects=16 duration=3.901251ms36692026/11/12 00:33:11 INFO Received uploads request method=POST path=/36702026/11/12 00:33:11 INFO Received uploads request method=POST path=/36712026/11/12 00:33:11 INFO Vacuumed table table=objects36722026/11/12 00:33:11 INFO Received push request method=POST path=/36732026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=036742026/11/12 00:33:11 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."36752026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period36762026/11/12 00:33:11 INFO Received uploads request method=POST path=/36772026/11/12 00:33:11 INFO Received uploads request method=POST path=/36782026/11/12 00:33:11 INFO Received push request method=POST path=/36792026/11/12 00:33:11 INFO Received complete push request method=POST path=/36802026/11/12 00:33:11 INFO Received complete upload request method=POST path=/36812026/11/12 00:33:11 INFO Completed upload id=22 objects=2 duration=1.39168ms36822026/11/12 00:33:11 INFO Received uploads request method=POST path=/36832026/11/12 00:33:11 INFO Completed push id=16 objects=4 duration=3.200291ms36842026/11/12 00:33:11 INFO Received complete upload request method=POST path=/36852026/11/12 00:33:11 INFO Received push request method=POST path=/36862026/11/12 00:33:11 INFO Completed upload id=7 objects=10 duration=2.172391ms36872026/11/12 00:33:11 INFO Received push request method=POST path=/36882026/11/12 00:33:11 INFO Received complete push request method=POST path=/36892026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=036902026/11/12 00:33:11 INFO Received uploads request method=POST path=/36912026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period36922026/11/12 00:33:11 INFO Completed push id=19 objects=8 duration=4.196382ms36932026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036942026/11/12 00:33:11 INFO Received complete upload request method=POST path=/36952026/11/12 00:33:11 INFO Vacuumed table table=pending_closures36962026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=036972026/11/12 00:33:11 INFO Vacuumed table table=pending_objects36982026/11/12 00:33:11 INFO Received push request method=POST path=/36992026/11/12 00:33:11 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."37002026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads37012026/11/12 00:33:11 INFO Completed upload id=12 objects=26 duration=3.011061ms37022026/11/12 00:33:11 INFO Vacuumed table table=closures37032026/11/12 00:33:11 INFO Received uploads request method=POST path=/37042026/11/12 00:33:11 INFO Vacuumed table table=objects37052026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=037062026/11/12 00:33:11 INFO Received uploads request method=POST path=/37072026/11/12 00:33:11 INFO Received uploads request method=POST path=/37082026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=037092026/11/12 00:33:11 INFO Received push request method=POST path=/37102026/11/12 00:33:11 INFO Received complete push request method=POST path=/37112026/11/12 00:33:11 INFO Vacuumed table table=pending_closures37122026/11/12 00:33:11 INFO Completed push id=15 objects=4 duration=1.448151ms37132026/11/12 00:33:11 INFO Vacuumed table table=pending_objects37142026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads37152026/11/12 00:33:11 INFO Vacuumed table table=closures37162026/11/12 00:33:11 INFO Received uploads request method=POST path=/37172026/11/12 00:33:11 INFO Vacuumed table table=objects37182026/11/12 00:33:11 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."37192026/11/12 00:33:11 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."37202026/11/12 00:33:11 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."37212026/11/12 00:33:11 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."37222026/11/12 00:33:11 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."37232026/11/12 00:33:11 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."37242026/11/12 00:33:11 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."37252026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=037262026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period37272026/11/12 00:33:11 INFO Received uploads request method=POST path=/37282026/11/12 00:33:11 INFO Received uploads request method=POST path=/37292026/11/12 00:33:11 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."37302026/11/12 00:33:11 INFO Received complete push request method=POST path=/37312026/11/12 00:33:11 INFO Completed push id=18 objects=10 duration=2.014621ms37322026/11/12 00:33:11 INFO Received complete upload request method=POST path=/37332026/11/12 00:33:11 INFO Received complete upload request method=POST path=/37342026/11/12 00:33:11 INFO Received push request method=POST path=/37352026/11/12 00:33:11 INFO Completed upload id=20 objects=5 duration=1.25704ms37362026/11/12 00:33:11 INFO Completed upload id=6 objects=6 duration=1.60209ms37372026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=037382026/11/12 00:33:11 INFO Received uploads request method=POST path=/37392026/11/12 00:33:11 INFO Received uploads request method=POST path=/37402026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037412026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=037422026/11/12 00:33:11 INFO Vacuumed table table=pending_closures37432026/11/12 00:33:11 INFO Received uploads request method=POST path=/37442026/11/12 00:33:11 INFO Vacuumed table table=pending_objects37452026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads37462026/11/12 00:33:11 INFO Vacuumed table table=closures37472026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037482026/11/12 00:33:11 INFO Vacuumed table table=objects37492026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=037502026/11/12 00:33:11 INFO Vacuumed table table=pending_closures37512026/11/12 00:33:11 INFO Vacuumed table table=pending_objects37522026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period37532026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads37542026/11/12 00:33:11 INFO Vacuumed table table=closures37552026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037562026/11/12 00:33:11 INFO Vacuumed table table=objects37572026/11/12 00:33:11 INFO Vacuumed table table=pending_closures37582026/11/12 00:33:11 INFO Received uploads request method=POST path=/37592026/11/12 00:33:11 INFO Vacuumed table table=pending_objects37602026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads37612026/11/12 00:33:11 INFO Vacuumed table table=closures37622026/11/12 00:33:11 INFO Received uploads request method=POST path=/37632026/11/12 00:33:11 INFO Vacuumed table table=objects37642026/11/12 00:33:11 INFO Received uploads request method=POST path=/37652026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=037662026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period37672026/11/12 00:33:11 INFO Received uploads request method=POST path=/37682026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037692026/11/12 00:33:11 INFO Received uploads request method=POST path=/37702026/11/12 00:33:11 INFO Vacuumed table table=pending_closures37712026/11/12 00:33:11 INFO Vacuumed table table=pending_objects37722026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads37732026/11/12 00:33:11 INFO Vacuumed table table=closures37742026/11/12 00:33:11 INFO Received complete push request method=POST path=/37752026/11/12 00:33:11 INFO Vacuumed table table=objects37762026/11/12 00:33:11 INFO Completed push id=16 objects=8 duration=1.25081ms37772026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=037782026/11/12 00:33:11 INFO Received push request method=POST path=/37792026/11/12 00:33:11 INFO Received complete push request method=POST path=/37802026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period37812026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037822026/11/12 00:33:11 INFO Completed push id=1 objects=11 duration=2.714731ms37832026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=037842026/11/12 00:33:11 INFO Vacuumed table table=pending_closures37852026/11/12 00:33:11 INFO Vacuumed table table=pending_objects37862026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads37872026/11/12 00:33:11 INFO Vacuumed table table=closures37882026/11/12 00:33:11 INFO Received uploads request method=POST path=/37892026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period37902026/11/12 00:33:11 INFO Received complete upload request method=POST path=/37912026/11/12 00:33:11 INFO Vacuumed table table=objects37922026/11/12 00:33:11 INFO Received push request method=POST path=/37932026/11/12 00:33:11 INFO Received uploads request method=POST path=/37942026/11/12 00:33:11 INFO Received push request method=POST path=/37952026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037962026/11/12 00:33:11 INFO Completed upload id=14 objects=8 duration=2.347581ms37972026/11/12 00:33:11 INFO Vacuumed table table=pending_closures37982026/11/12 00:33:11 INFO Received push request method=POST path=/37992026/11/12 00:33:11 INFO Vacuumed table table=pending_objects38002026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads38012026/11/12 00:33:11 INFO Vacuumed table table=closures38022026/11/12 00:33:11 INFO Vacuumed table table=objects38032026/11/12 00:33:11 INFO Received push request method=POST path=/38042026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=038052026/11/12 00:33:11 INFO Received uploads request method=POST path=/38062026/11/12 00:33:11 INFO Received complete push request method=POST path=/38072026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period38082026/11/12 00:33:11 INFO Completed push id=14 objects=3 duration=1.223611ms38092026/11/12 00:33:11 INFO Received push request method=POST path=/38102026/11/12 00:33:11 INFO Received complete upload request method=POST path=/38112026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038122026/11/12 00:33:11 INFO Vacuumed table table=pending_closures38132026/11/12 00:33:11 INFO Received push request method=POST path=/38142026/11/12 00:33:11 INFO Completed upload id=11 objects=7 duration=1.707221ms38152026/11/12 00:33:11 INFO Vacuumed table table=pending_objects38162026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=038172026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads38182026/11/12 00:33:11 INFO Received uploads request method=POST path=/38192026/11/12 00:33:11 INFO Vacuumed table table=closures38202026/11/12 00:33:11 INFO Received complete upload request method=POST path=/38212026/11/12 00:33:11 INFO Vacuumed table table=objects38222026/11/12 00:33:11 INFO Completed upload id=16 objects=3 duration=1.28638ms38232026/11/12 00:33:11 WARN Force mode enabled - objects will be deleted immediately without grace period38242026/11/12 00:33:11 INFO Received push request method=POST path=/38252026/11/12 00:33:11 INFO Aborted multipart uploads count=0 kept=038262026/11/12 00:33:11 INFO Garbage collection completed failed-uploads-deleted=5 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038272026/11/12 00:33:11 INFO Vacuumed table table=pending_closures38282026/11/12 00:33:11 INFO Received uploads request method=POST path=/38292026/11/12 00:33:11 INFO Vacuumed table table=pending_objects38302026/11/12 00:33:11 INFO Vacuumed table table=multipart_uploads38312026/11/12 00:33:11 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."38322026/11/12 00:33:11 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."38332026/11/12 00:33:11 INFO Vacuumed table table=closures38342026/11/12 00:33:11 INFO Received push request method=POST path=/38352026/11/12 00:33:11 INFO Vacuumed table table=objects38362026/11/12 00:33:11 INFO Received uploads request method=POST path=/38372026/11/12 00:33:11 INFO Received push request method=POST path=/38382026/11/12 00:33:11 INFO Received uploads request method=POST path=/38392026/11/12 00:33:11 INFO Received push request method=POST path=/38402026/11/12 00:33:11 INFO Received uploads request method=POST path=/38412026/11/12 00:33:11 INFO Received push request method=POST path=/38422026/11/12 00:33:11 INFO Received push request method=POST path=/38432026/11/12 00:33:11 INFO Received uploads request method=POST path=/38442026/11/12 00:33:11 INFO Received complete push request method=POST path=/38452026/11/12 00:33:11 INFO Completed push id=11 objects=6 duration=1.695481ms38462026/11/12 00:33:11 INFO Received uploads request method=POST path=/38472026/11/12 00:33:11 INFO Received complete push request method=POST path=/38482026/11/12 00:33:11 INFO Received uploads request method=POST path=/38492026/11/12 00:33:11 INFO Received uploads request method=POST path=/38502026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=038512026/11/12 00:33:12 INFO Completed push id=26 objects=14 duration=3.037712ms38522026/11/12 00:33:12 INFO Received push request method=POST path=/38532026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038542026/11/12 00:33:12 INFO Received uploads request method=POST path=/38552026/11/12 00:33:12 INFO Vacuumed table table=pending_closures38562026/11/12 00:33:12 INFO Vacuumed table table=pending_objects38572026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads38582026/11/12 00:33:12 INFO Vacuumed table table=closures38592026/11/12 00:33:12 INFO Vacuumed table table=objects38602026/11/12 00:33:12 INFO Received complete upload request method=POST path=/38612026/11/12 00:33:12 INFO Completed upload id=23 objects=15 duration=2.879381ms38622026/11/12 00:33:12 INFO Received complete upload request method=POST path=/38632026/11/12 00:33:12 INFO Completed upload id=20 objects=4 duration=1.07714ms38642026/11/12 00:33:12 INFO Received uploads request method=POST path=/38652026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=038662026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period38672026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=038682026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period38692026/11/12 00:33:12 INFO Received complete push request method=POST path=/38702026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038712026/11/12 00:33:12 INFO Received complete push request method=POST path=/38722026/11/12 00:33:12 INFO Received push request method=POST path=/38732026/11/12 00:33:12 INFO Vacuumed table table=pending_closures38742026/11/12 00:33:12 INFO Completed push id=18 objects=6 duration=1.378911ms38752026/11/12 00:33:12 INFO Completed push id=14 objects=3 duration=3.749871ms38762026/11/12 00:33:12 INFO Vacuumed table table=pending_objects38772026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads38782026/11/12 00:33:12 INFO Received push request method=POST path=/38792026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=038802026/11/12 00:33:12 INFO Vacuumed table table=closures38812026/11/12 00:33:12 INFO Vacuumed table table=objects38822026/11/12 00:33:12 INFO Vacuumed table table=pending_closures38832026/11/12 00:33:12 INFO Vacuumed table table=pending_objects38842026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads38852026/11/12 00:33:12 INFO Vacuumed table table=closures38862026/11/12 00:33:12 INFO Vacuumed table table=objects38872026/11/12 00:33:12 INFO Received push request method=POST path=/38882026/11/12 00:33:12 INFO Received push request method=POST path=/38892026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=038902026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period38912026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=038922026/11/12 00:33:12 INFO Received uploads request method=POST path=/38932026/11/12 00:33:12 INFO Received push request method=POST path=/38942026/11/12 00:33:12 INFO Received push request method=POST path=/38952026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=038962026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038972026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=038982026/11/12 00:33:12 INFO Received complete push request method=POST path=/38992026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period39002026/11/12 00:33:12 INFO Vacuumed table table=pending_closures39012026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=039022026/11/12 00:33:12 INFO Received complete push request method=POST path=/39032026/11/12 00:33:12 INFO Vacuumed table table=pending_closures39042026/11/12 00:33:12 INFO Received complete upload request method=POST path=/39052026/11/12 00:33:12 INFO Vacuumed table table=pending_objects39062026/11/12 00:33:12 INFO Received complete push request method=POST path=/39072026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads39082026/11/12 00:33:12 INFO Vacuumed table table=pending_objects39092026/11/12 00:33:12 INFO Completed push id=21 objects=9 duration=2.702791ms39102026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads39112026/11/12 00:33:12 INFO Completed push id=16 objects=5 duration=1.945581ms39122026/11/12 00:33:12 INFO Completed push id=18 objects=6 duration=3.840561ms39132026/11/12 00:33:12 INFO Vacuumed table table=closures39142026/11/12 00:33:12 INFO Completed upload id=21 objects=7 duration=2.720361ms39152026/11/12 00:33:12 INFO Vacuumed table table=closures39162026/11/12 00:33:12 INFO Received uploads request method=POST path=/39172026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period39182026/11/12 00:33:12 INFO Vacuumed table table=objects39192026/11/12 00:33:12 INFO Vacuumed table table=objects39202026/11/12 00:33:12 INFO Received complete upload request method=POST path=/39212026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039222026/11/12 00:33:12 INFO Completed upload id=19 objects=6 duration=1.408951ms39232026/11/12 00:33:12 INFO Vacuumed table table=pending_closures39242026/11/12 00:33:12 INFO Received push request method=POST path=/39252026/11/12 00:33:12 INFO Vacuumed table table=pending_objects39262026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads39272026/11/12 00:33:12 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."39282026/11/12 00:33:12 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."39292026/11/12 00:33:12 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."39302026/11/12 00:33:12 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."39312026/11/12 00:33:12 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."39322026/11/12 00:33:12 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."39332026/11/12 00:33:12 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."39342026/11/12 00:33:12 INFO Vacuumed table table=closures39352026/11/12 00:33:12 INFO Vacuumed table table=objects39362026/11/12 00:33:12 INFO Received uploads request method=POST path=/39372026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=039382026/11/12 00:33:12 INFO Received uploads request method=POST path=/39392026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039402026/11/12 00:33:12 INFO Vacuumed table table=pending_closures39412026/11/12 00:33:12 INFO Vacuumed table table=pending_objects39422026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads39432026/11/12 00:33:12 INFO Received complete upload request method=POST path=/39442026/11/12 00:33:12 INFO Vacuumed table table=closures39452026/11/12 00:33:12 INFO Completed upload id=23 objects=9 duration=1.11023ms39462026/11/12 00:33:12 INFO Vacuumed table table=objects39472026/11/12 00:33:12 INFO Received complete upload request method=POST path=/39482026/11/12 00:33:12 INFO Completed upload id=20 objects=3 duration=1.402891ms39492026/11/12 00:33:12 INFO Received push request method=POST path=/39502026/11/12 00:33:12 INFO Received uploads request method=POST path=/39512026/11/12 00:33:12 INFO Received complete upload request method=POST path=/39522026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=039532026/11/12 00:33:12 INFO Completed upload id=22 objects=7 duration=1.43803ms39542026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=039552026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039562026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=039572026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period39582026/11/12 00:33:12 INFO Vacuumed table table=pending_closures39592026/11/12 00:33:12 INFO Received uploads request method=POST path=/39602026/11/12 00:33:12 INFO Vacuumed table table=pending_objects39612026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads39622026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039632026/11/12 00:33:12 INFO Vacuumed table table=closures39642026/11/12 00:33:12 INFO Received push request method=POST path=/39652026/11/12 00:33:12 INFO Vacuumed table table=pending_closures39662026/11/12 00:33:12 INFO Vacuumed table table=objects39672026/11/12 00:33:12 INFO Vacuumed table table=pending_objects39682026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads39692026/11/12 00:33:12 INFO Received uploads request method=POST path=/39702026/11/12 00:33:12 INFO Vacuumed table table=closures39712026/11/12 00:33:12 INFO Vacuumed table table=objects39722026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=039732026/11/12 00:33:12 INFO Received push request method=POST path=/39742026/11/12 00:33:12 INFO Vacuumed table table=pending_closures39752026/11/12 00:33:12 INFO Received complete upload request method=POST path=/39762026/11/12 00:33:12 INFO Vacuumed table table=pending_objects39772026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads39782026/11/12 00:33:12 INFO Completed upload id=21 objects=3 duration=1.17015ms39792026/11/12 00:33:12 INFO Vacuumed table table=closures39802026/11/12 00:33:12 INFO Vacuumed table table=objects39812026/11/12 00:33:12 INFO Received complete push request method=POST path=/39822026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=039832026/11/12 00:33:12 INFO Completed push id=4 objects=6 duration=1.95712ms39842026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039852026/11/12 00:33:12 INFO Vacuumed table table=pending_closures39862026/11/12 00:33:12 INFO Vacuumed table table=pending_objects39872026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads39882026/11/12 00:33:12 INFO Received push request method=POST path=/39892026/11/12 00:33:12 INFO Vacuumed table table=closures39902026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=039912026/11/12 00:33:12 INFO Vacuumed table table=objects39922026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period39932026/11/12 00:33:12 INFO Received push request method=POST path=/39942026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039952026/11/12 00:33:12 INFO Received push request method=POST path=/39962026/11/12 00:33:12 INFO Received complete push request method=POST path=/39972026/11/12 00:33:12 INFO Vacuumed table table=pending_closures39982026/11/12 00:33:12 INFO Received uploads request method=POST path=/39992026/11/12 00:33:12 INFO Vacuumed table table=pending_objects40002026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads40012026/11/12 00:33:12 INFO Completed push id=13 objects=15 duration=2.761212ms40022026/11/12 00:33:12 INFO Vacuumed table table=closures40032026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=040042026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=040052026/11/12 00:33:12 INFO Vacuumed table table=objects40062026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period40072026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040082026/11/12 00:33:12 INFO Received complete upload request method=POST path=/40092026/11/12 00:33:12 INFO Received uploads request method=POST path=/40102026/11/12 00:33:12 INFO Vacuumed table table=pending_closures40112026/11/12 00:33:12 INFO Vacuumed table table=pending_objects40122026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads40132026/11/12 00:33:12 INFO Completed upload id=16 objects=17 duration=3.009881ms40142026/11/12 00:33:12 INFO Vacuumed table table=closures40152026/11/12 00:33:12 INFO Vacuumed table table=objects40162026/11/12 00:33:12 INFO Received uploads request method=POST path=/40172026/11/12 00:33:12 INFO Received uploads request method=POST path=/40182026/11/12 00:33:12 INFO Received push request method=POST path=/40192026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=7 objects-failed-to-delete=040202026/11/12 00:33:12 INFO Received push request method=POST path=/40212026/11/12 00:33:12 INFO Received complete upload request method=POST path=/40222026/11/12 00:33:12 INFO Vacuumed table table=pending_closures40232026/11/12 00:33:12 INFO Received uploads request method=POST path=/40242026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=040252026/11/12 00:33:12 INFO Completed upload id=14 objects=16 duration=2.420991ms40262026/11/12 00:33:12 INFO Vacuumed table table=pending_objects40272026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads40282026/11/12 00:33:12 INFO Received complete upload request method=POST path=/40292026/11/12 00:33:12 INFO Received push request method=POST path=/40302026/11/12 00:33:12 INFO Vacuumed table table=closures40312026/11/12 00:33:12 INFO Completed upload id=24 objects=7 duration=1.02011ms40322026/11/12 00:33:12 INFO Vacuumed table table=objects40332026/11/12 00:33:12 INFO Received uploads request method=POST path=/40342026/11/12 00:33:12 INFO Received push request method=POST path=/40352026/11/12 00:33:12 INFO Received complete push request method=POST path=/40362026/11/12 00:33:12 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."40372026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=040382026/11/12 00:33:12 INFO Completed push id=24 objects=9 duration=2.248131ms40392026/11/12 00:33:12 INFO Received uploads request method=POST path=/40402026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period40412026/11/12 00:33:12 INFO Received push request method=POST path=/40422026/11/12 00:33:12 INFO Received push request method=POST path=/40432026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=040442026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period40452026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040462026/11/12 00:33:12 INFO Received push request method=POST path=/40472026/11/12 00:33:12 INFO Vacuumed table table=pending_closures40482026/11/12 00:33:12 INFO Vacuumed table table=pending_objects40492026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads40502026/11/12 00:33:12 INFO Vacuumed table table=closures40512026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=040522026/11/12 00:33:12 INFO Vacuumed table table=objects40532026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040542026/11/12 00:33:12 INFO Received uploads request method=POST path=/40552026/11/12 00:33:12 INFO Vacuumed table table=pending_closures40562026/11/12 00:33:12 INFO Vacuumed table table=pending_objects40572026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads40582026/11/12 00:33:12 INFO Vacuumed table table=closures40592026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=040602026/11/12 00:33:12 INFO Received push request method=POST path=/40612026/11/12 00:33:12 INFO Vacuumed table table=objects40622026/11/12 00:33:12 INFO Received complete upload request method=POST path=/40632026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period40642026/11/12 00:33:12 INFO Completed upload id=27 objects=4 duration=1.27653ms40652026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040662026/11/12 00:33:12 INFO Received push request method=POST path=/40672026/11/12 00:33:12 INFO Vacuumed table table=pending_closures40682026/11/12 00:33:12 INFO Vacuumed table table=pending_objects40692026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads40702026/11/12 00:33:12 INFO Vacuumed table table=closures40712026/11/12 00:33:12 INFO Received uploads request method=POST path=/40722026/11/12 00:33:12 INFO Vacuumed table table=objects40732026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=040742026/11/12 00:33:12 INFO Received push request method=POST path=/40752026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period40762026/11/12 00:33:12 INFO Received push request method=POST path=/40772026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040782026/11/12 00:33:12 INFO Vacuumed table table=pending_closures40792026/11/12 00:33:12 INFO Vacuumed table table=pending_objects40802026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads40812026/11/12 00:33:12 INFO Vacuumed table table=closures40822026/11/12 00:33:12 INFO Vacuumed table table=objects40832026/11/12 00:33:12 INFO Received complete push request method=POST path=/40842026/11/12 00:33:12 INFO Received uploads request method=POST path=/40852026/11/12 00:33:12 INFO Completed push id=20 objects=3 duration=2.12447ms40862026/11/12 00:33:12 INFO Received push request method=POST path=/40872026/11/12 00:33:12 INFO Received complete push request method=POST path=/40882026/11/12 00:33:12 INFO Completed push id=13 objects=11 duration=1.689421ms40892026/11/12 00:33:12 INFO Received complete push request method=POST path=/40902026/11/12 00:33:12 INFO Completed push id=17 objects=11 duration=1.934231ms40912026/11/12 00:33:12 INFO Received uploads request method=POST path=/40922026/11/12 00:33:12 INFO Received uploads request method=POST path=/40932026/11/12 00:33:12 INFO Received complete upload request method=POST path=/40942026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=040952026/11/12 00:33:12 INFO Completed upload id=23 objects=16 duration=3.025321ms40962026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period40972026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040982026/11/12 00:33:12 INFO Vacuumed table table=pending_closures40992026/11/12 00:33:12 INFO Vacuumed table table=pending_objects41002026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=041012026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads41022026/11/12 00:33:12 INFO Vacuumed table table=closures41032026/11/12 00:33:12 INFO Vacuumed table table=objects41042026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041052026/11/12 00:33:12 INFO Vacuumed table table=pending_closures41062026/11/12 00:33:12 INFO Received push request method=POST path=/41072026/11/12 00:33:12 INFO Vacuumed table table=pending_objects41082026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads41092026/11/12 00:33:12 INFO Vacuumed table table=closures41102026/11/12 00:33:12 INFO Vacuumed table table=objects41112026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=041122026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period41132026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041142026/11/12 00:33:12 INFO Received uploads request method=POST path=/41152026/11/12 00:33:12 INFO Vacuumed table table=pending_closures41162026/11/12 00:33:12 INFO Received push request method=POST path=/41172026/11/12 00:33:12 INFO Vacuumed table table=pending_objects41182026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads41192026/11/12 00:33:12 INFO Vacuumed table table=closures41202026/11/12 00:33:12 INFO Vacuumed table table=objects41212026/11/12 00:33:12 INFO Received uploads request method=POST path=/41222026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=041232026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period41242026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=041252026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period41262026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041272026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041282026/11/12 00:33:12 INFO Vacuumed table table=pending_closures41292026/11/12 00:33:12 INFO Vacuumed table table=pending_closures41302026/11/12 00:33:12 INFO Received complete push request method=POST path=/41312026/11/12 00:33:12 INFO Vacuumed table table=pending_objects41322026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads41332026/11/12 00:33:12 INFO Vacuumed table table=pending_objects41342026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads41352026/11/12 00:33:12 INFO Vacuumed table table=closures41362026/11/12 00:33:12 INFO Completed push id=16 objects=26 duration=1.55362ms41372026/11/12 00:33:12 INFO Vacuumed table table=closures41382026/11/12 00:33:12 INFO Vacuumed table table=objects41392026/11/12 00:33:12 INFO Vacuumed table table=objects41402026/11/12 00:33:12 INFO Received complete push request method=POST path=/41412026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=041422026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=041432026/11/12 00:33:12 INFO Vacuumed table table=pending_closures41442026/11/12 00:33:12 INFO Completed push id=22 objects=3 duration=3.298781ms41452026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041462026/11/12 00:33:12 INFO Vacuumed table table=pending_objects41472026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads41482026/11/12 00:33:12 INFO Vacuumed table table=pending_closures41492026/11/12 00:33:12 INFO Vacuumed table table=closures41502026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=041512026/11/12 00:33:12 INFO Vacuumed table table=pending_objects41522026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads41532026/11/12 00:33:12 INFO Vacuumed table table=objects41542026/11/12 00:33:12 INFO Vacuumed table table=closures41552026/11/12 00:33:12 INFO Vacuumed table table=objects41562026/11/12 00:33:12 INFO Received uploads request method=POST path=/41572026/11/12 00:33:12 INFO Received push request method=POST path=/41582026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041592026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=041602026/11/12 00:33:12 INFO Vacuumed table table=pending_closures41612026/11/12 00:33:12 INFO Received push request method=POST path=/41622026/11/12 00:33:12 INFO Vacuumed table table=pending_objects41632026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads41642026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period41652026/11/12 00:33:12 INFO Vacuumed table table=closures41662026/11/12 00:33:12 INFO Received complete upload request method=POST path=/41672026/11/12 00:33:12 INFO Vacuumed table table=objects41682026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041692026/11/12 00:33:12 INFO Completed upload id=23 objects=18 duration=2.411391ms41702026/11/12 00:33:12 INFO Vacuumed table table=pending_closures41712026/11/12 00:33:12 INFO Vacuumed table table=pending_objects41722026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads41732026/11/12 00:33:12 INFO Received push request method=POST path=/41742026/11/12 00:33:12 INFO Vacuumed table table=closures41752026/11/12 00:33:12 INFO Received complete push request method=POST path=/41762026/11/12 00:33:12 INFO Vacuumed table table=objects41772026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=041782026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=041792026/11/12 00:33:12 INFO Completed push id=23 objects=28 duration=2.186611ms41802026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period41812026/11/12 00:33:12 INFO Received uploads request method=POST path=/41822026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041832026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period41842026/11/12 00:33:12 INFO Vacuumed table table=pending_closures41852026/11/12 00:33:12 INFO Received uploads request method=POST path=/41862026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041872026/11/12 00:33:12 INFO Vacuumed table table=pending_objects41882026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads41892026/11/12 00:33:12 INFO Vacuumed table table=pending_closures41902026/11/12 00:33:12 INFO Vacuumed table table=closures41912026/11/12 00:33:12 INFO Vacuumed table table=pending_objects41922026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads41932026/11/12 00:33:12 INFO Vacuumed table table=objects41942026/11/12 00:33:12 INFO Vacuumed table table=closures41952026/11/12 00:33:12 INFO Vacuumed table table=objects41962026/11/12 00:33:12 INFO Received uploads request method=POST path=/41972026/11/12 00:33:12 INFO Received complete push request method=POST path=/41982026/11/12 00:33:12 INFO Completed push id=27 objects=25 duration=3.195171ms41992026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=042002026/11/12 00:33:12 INFO Received push request method=POST path=/42012026/11/12 00:33:12 INFO Received push request method=POST path=/42022026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period42032026/11/12 00:33:12 INFO Received push request method=POST path=/42042026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042052026/11/12 00:33:12 INFO Received complete upload request method=POST path=/42062026/11/12 00:33:12 INFO Received complete push request method=POST path=/42072026/11/12 00:33:12 INFO Received push request method=POST path=/42082026/11/12 00:33:12 INFO Received uploads request method=POST path=/42092026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=042102026/11/12 00:33:12 INFO Received complete upload request method=POST path=/42112026/11/12 00:33:12 INFO Received complete upload request method=POST path=/42122026/11/12 00:33:12 INFO Completed upload id=27 objects=8 duration=145.838136ms42132026/11/12 00:33:12 INFO Completed upload id=13 objects=8 duration=139.996924ms42142026/11/12 00:33:12 INFO Completed push id=27 objects=6 duration=155.69747ms42152026/11/12 00:33:12 INFO Completed upload id=26 objects=6 duration=156.43354ms42162026/11/12 00:33:12 INFO Vacuumed table table=pending_closures42172026/11/12 00:33:12 INFO Received complete upload request method=POST path=/42182026/11/12 00:33:12 INFO Vacuumed table table=pending_objects42192026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads42202026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042212026/11/12 00:33:12 INFO Vacuumed table table=closures42222026/11/12 00:33:12 INFO Vacuumed table table=pending_closures42232026/11/12 00:33:12 INFO Completed upload id=33 objects=5 duration=3.307792ms42242026/11/12 00:33:12 INFO Vacuumed table table=objects42252026/11/12 00:33:12 INFO Vacuumed table table=pending_objects42262026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads42272026/11/12 00:33:12 INFO Vacuumed table table=closures42282026/11/12 00:33:12 INFO Vacuumed table table=objects42292026/11/12 00:33:12 INFO Received complete upload request method=POST path=/42302026/11/12 00:33:12 INFO Completed upload id=17 objects=25 duration=3.336571ms42312026/11/12 00:33:12 INFO Received push request method=POST path=/42322026/11/12 00:33:12 INFO Received push request method=POST path=/42332026/11/12 00:33:12 INFO Received complete push request method=POST path=/42342026/11/12 00:33:12 INFO Received push request method=POST path=/42352026/11/12 00:33:12 INFO Received push request method=POST path=/42362026/11/12 00:33:12 INFO Received uploads request method=POST path=/42372026/11/12 00:33:12 INFO Received complete upload request method=POST path=/42382026/11/12 00:33:12 INFO Received uploads request method=POST path=/42392026/11/12 00:33:12 INFO Received complete upload request method=POST path=/42402026/11/12 00:33:12 INFO Completed push id=31 objects=14 duration=3.189581ms42412026/11/12 00:33:12 INFO Completed upload id=22 objects=22 duration=1.727471ms42422026/11/12 00:33:12 INFO Completed upload id=15 objects=3 duration=2.109251ms42432026/11/12 00:33:12 INFO Received complete push request method=POST path=/42442026/11/12 00:33:12 INFO Completed push id=16 objects=14 duration=1.596191ms42452026/11/12 00:33:12 INFO Received complete push request method=POST path=/42462026/11/12 00:33:12 INFO Completed push id=24 objects=5 duration=2.989531ms42472026/11/12 00:33:12 INFO Received push request method=POST path=/42482026/11/12 00:33:12 INFO Received uploads request method=POST path=/42492026/11/12 00:33:12 INFO Received complete push request method=POST path=/42502026/11/12 00:33:12 INFO Completed push id=24 objects=3 duration=1.562591ms42512026/11/12 00:33:12 INFO Received complete upload request method=POST path=/42522026/11/12 00:33:12 INFO Completed upload id=15 objects=2 duration=1.41503ms42532026/11/12 00:33:12 INFO Received push request method=POST path=/42542026/11/12 00:33:12 INFO Received complete upload request method=POST path=/42552026/11/12 00:33:12 INFO Received complete push request method=POST path=/42562026/11/12 00:33:12 INFO Received uploads request method=POST path=/42572026/11/12 00:33:12 INFO Received push request method=POST path=/42582026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=042592026/11/12 00:33:12 INFO Completed upload id=29 objects=5 duration=1.415421ms42602026/11/12 00:33:12 INFO Completed push id=21 objects=3 duration=1.780831ms42612026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=042622026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042632026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period42642026/11/12 00:33:12 INFO Vacuumed table table=pending_closures42652026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042662026/11/12 00:33:12 INFO Vacuumed table table=pending_objects42672026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads42682026/11/12 00:33:12 INFO Vacuumed table table=pending_closures42692026/11/12 00:33:12 INFO Vacuumed table table=closures42702026/11/12 00:33:12 INFO Vacuumed table table=pending_objects42712026/11/12 00:33:12 INFO Vacuumed table table=objects42722026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads42732026/11/12 00:33:12 INFO Vacuumed table table=closures42742026/11/12 00:33:12 INFO Vacuumed table table=objects42752026/11/12 00:33:12 INFO Received push request method=POST path=/42762026/11/12 00:33:12 INFO Received complete upload request method=POST path=/42772026/11/12 00:33:12 INFO Completed upload id=21 objects=2 duration=1.26282ms42782026/11/12 00:33:12 INFO Received push request method=POST path=/42792026/11/12 00:33:12 INFO Received push request method=POST path=/42802026/11/12 00:33:12 INFO Received complete upload request method=POST path=/42812026/11/12 00:33:12 INFO Received uploads request method=POST path=/42822026/11/12 00:33:12 INFO Completed upload id=21 objects=25 duration=1.771451ms42832026/11/12 00:33:12 INFO Received complete upload request method=POST path=/42842026/11/12 00:33:12 INFO Completed upload id=22 objects=6 duration=1.225431ms42852026/11/12 00:33:12 INFO Received push request method=POST path=/42862026/11/12 00:33:12 INFO Received uploads request method=POST path=/42872026/11/12 00:33:12 INFO Received complete push request method=POST path=/42882026/11/12 00:33:12 INFO Completed push id=26 objects=5 duration=2.041591ms42892026/11/12 00:33:12 INFO Received uploads request method=POST path=/42902026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=042912026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042922026/11/12 00:33:12 INFO Vacuumed table table=pending_closures42932026/11/12 00:33:12 INFO Vacuumed table table=pending_objects42942026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=042952026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads42962026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=042972026/11/12 00:33:12 INFO Vacuumed table table=closures42982026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period42992026/11/12 00:33:12 INFO Vacuumed table table=objects43002026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043012026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043022026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=043032026/11/12 00:33:12 INFO Received uploads request method=POST path=/43042026/11/12 00:33:12 INFO Received uploads request method=POST path=/43052026/11/12 00:33:12 INFO Vacuumed table table=pending_closures43062026/11/12 00:33:12 INFO Vacuumed table table=pending_closures43072026/11/12 00:33:12 INFO Vacuumed table table=pending_objects43082026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period43092026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads43102026/11/12 00:33:12 INFO Vacuumed table table=pending_objects43112026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads43122026/11/12 00:33:12 INFO Vacuumed table table=closures43132026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043142026/11/12 00:33:12 INFO Vacuumed table table=closures43152026/11/12 00:33:12 INFO Vacuumed table table=objects43162026/11/12 00:33:12 INFO Received complete upload request method=POST path=/43172026/11/12 00:33:12 INFO Vacuumed table table=objects43182026/11/12 00:33:12 INFO Vacuumed table table=pending_closures43192026/11/12 00:33:12 INFO Completed upload id=24 objects=14 duration=1.314801ms43202026/11/12 00:33:12 INFO Vacuumed table table=pending_objects43212026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads43222026/11/12 00:33:12 INFO Vacuumed table table=closures43232026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=043242026/11/12 00:33:12 INFO Received complete push request method=POST path=/43252026/11/12 00:33:12 INFO Vacuumed table table=objects43262026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043272026/11/12 00:33:12 INFO Completed push id=7 objects=15 duration=2.893181ms43282026/11/12 00:33:12 INFO Vacuumed table table=pending_closures43292026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=043302026/11/12 00:33:12 INFO Vacuumed table table=pending_objects43312026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads43322026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period43332026/11/12 00:33:12 INFO Vacuumed table table=closures43342026/11/12 00:33:12 INFO Received complete upload request method=POST path=/43352026/11/12 00:33:12 INFO Vacuumed table table=objects43362026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043372026/11/12 00:33:12 INFO Completed upload id=25 objects=16 duration=2.786521ms43382026/11/12 00:33:12 INFO Vacuumed table table=pending_closures43392026/11/12 00:33:12 INFO Vacuumed table table=pending_objects43402026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads43412026/11/12 00:33:12 INFO Received push request method=POST path=/43422026/11/12 00:33:12 INFO Vacuumed table table=closures43432026/11/12 00:33:12 INFO Vacuumed table table=objects43442026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=043452026/11/12 00:33:12 INFO Received complete push request method=POST path=/43462026/11/12 00:33:12 INFO Completed push id=19 objects=3 duration=1.16173ms43472026/11/12 00:33:12 INFO Received push request method=POST path=/43482026/11/12 00:33:12 INFO Received complete push request method=POST path=/43492026/11/12 00:33:12 INFO Received complete push request method=POST path=/43502026/11/12 00:33:12 INFO Completed push id=21 objects=26 duration=1.58132ms43512026/11/12 00:33:12 INFO Completed push id=7 objects=7 duration=2.039671ms43522026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=3 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043532026/11/12 00:33:12 INFO Vacuumed table table=pending_closures43542026/11/12 00:33:12 INFO Received complete push request method=POST path=/43552026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=043562026/11/12 00:33:12 INFO Received complete push request method=POST path=/43572026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period43582026/11/12 00:33:12 INFO Vacuumed table table=pending_objects43592026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads43602026/11/12 00:33:12 INFO Completed push id=5 objects=28 duration=2.180211ms43612026/11/12 00:33:12 INFO Completed push id=17 objects=22 duration=2.483631ms43622026/11/12 00:33:12 INFO Vacuumed table table=closures43632026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043642026/11/12 00:33:12 INFO Vacuumed table table=pending_closures43652026/11/12 00:33:12 INFO Vacuumed table table=objects43662026/11/12 00:33:12 INFO Vacuumed table table=pending_objects43672026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=043682026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads43692026/11/12 00:33:12 INFO Vacuumed table table=closures43702026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period43712026/11/12 00:33:12 INFO Received push request method=POST path=/43722026/11/12 00:33:12 INFO Vacuumed table table=objects43732026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043742026/11/12 00:33:12 INFO Received uploads request method=POST path=/43752026/11/12 00:33:12 INFO Vacuumed table table=pending_closures43762026/11/12 00:33:12 INFO Vacuumed table table=pending_objects43772026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads43782026/11/12 00:33:12 INFO Received uploads request method=POST path=/43792026/11/12 00:33:12 INFO Vacuumed table table=closures43802026/11/12 00:33:12 INFO Vacuumed table table=objects43812026/11/12 00:33:12 INFO Received complete upload request method=POST path=/43822026/11/12 00:33:12 INFO Completed upload id=12 objects=6 duration=1.36314ms43832026/11/12 00:33:12 INFO Received push request method=POST path=/43842026/11/12 00:33:12 INFO Received complete push request method=POST path=/43852026/11/12 00:33:12 INFO Completed push id=12 objects=6 duration=1.464231ms43862026/11/12 00:33:12 INFO Received complete upload request method=POST path=/43872026/11/12 00:33:12 INFO Received uploads request method=POST path=/43882026/11/12 00:33:12 INFO Completed upload id=28 objects=17 duration=1.198131ms43892026/11/12 00:33:12 INFO Received complete push request method=POST path=/43902026/11/12 00:33:12 INFO Received complete upload request method=POST path=/43912026/11/12 00:33:12 INFO Received push request method=POST path=/43922026/11/12 00:33:12 INFO Completed push id=10 objects=2 duration=1.80578ms43932026/11/12 00:33:12 INFO Completed upload id=19 objects=2 duration=2.326171ms43942026/11/12 00:33:12 INFO Received uploads request method=POST path=/43952026/11/12 00:33:12 INFO Received push request method=POST path=/43962026/11/12 00:33:12 INFO Received complete upload request method=POST path=/43972026/11/12 00:33:12 INFO Completed upload id=25 objects=6 duration=1.72427ms43982026/11/12 00:33:12 INFO Received uploads request method=POST path=/43992026/11/12 00:33:12 INFO Received push request method=POST path=/44002026/11/12 00:33:12 INFO Received complete push request method=POST path=/44012026/11/12 00:33:12 INFO Received complete push request method=POST path=/44022026/11/12 00:33:12 INFO Completed push id=17 objects=3 duration=1.929721ms44032026/11/12 00:33:12 INFO Received push request method=POST path=/44042026/11/12 00:33:12 INFO Received uploads request method=POST path=/44052026/11/12 00:33:12 INFO Completed push id=7 objects=24 duration=3.731761ms44062026/11/12 00:33:12 INFO Received complete push request method=POST path=/44072026/11/12 00:33:12 INFO Received uploads request method=POST path=/44082026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=044092026/11/12 00:33:12 INFO Completed push id=29 objects=2 duration=1.330211ms44102026/11/12 00:33:12 INFO Received push request method=POST path=/44112026/11/12 00:33:12 INFO Received uploads request method=POST path=/44122026/11/12 00:33:12 INFO Received push request method=POST path=/44132026/11/12 00:33:12 INFO Received complete upload request method=POST path=/44142026/11/12 00:33:12 INFO Completed upload id=17 objects=5 duration=1.63228ms44152026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=044162026/11/12 00:33:12 INFO Vacuumed table table=pending_closures44172026/11/12 00:33:12 INFO Received uploads request method=POST path=/44182026/11/12 00:33:12 INFO Received push request method=POST path=/44192026/11/12 00:33:12 INFO Vacuumed table table=pending_objects44202026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads44212026/11/12 00:33:12 INFO Vacuumed table table=closures44222026/11/12 00:33:12 INFO Vacuumed table table=objects44232026/11/12 00:33:12 INFO Received complete push request method=POST path=/44242026/11/12 00:33:12 INFO Completed push id=23 objects=5 duration=1.315271ms44252026/11/12 00:33:12 INFO Received complete push request method=POST path=/44262026/11/12 00:33:12 INFO Received complete push request method=POST path=/44272026/11/12 00:33:12 INFO Completed push id=33 objects=7 duration=2.004931ms44282026/11/12 00:33:12 INFO Completed push id=31 objects=6 duration=1.52967ms44292026/11/12 00:33:12 INFO Received complete push request method=POST path=/44302026/11/12 00:33:12 INFO Received complete push request method=POST path=/44312026/11/12 00:33:12 INFO Received push request method=POST path=/44322026/11/12 00:33:12 INFO Received uploads request method=POST path=/44332026/11/12 00:33:12 INFO Completed push id=13 objects=22 duration=1.633851ms44342026/11/12 00:33:12 INFO Received complete upload request method=POST path=/44352026/11/12 00:33:12 INFO Completed push id=30 objects=4 duration=3.710722ms44362026/11/12 00:33:12 INFO Completed upload id=26 objects=8 duration=1.34227ms44372026/11/12 00:33:12 INFO Received uploads request method=POST path=/44382026/11/12 00:33:12 INFO Received complete upload request method=POST path=/44392026/11/12 00:33:12 INFO Completed upload id=31 objects=3 duration=2.440591ms44402026/11/12 00:33:12 INFO Received complete push request method=POST path=/44412026/11/12 00:33:12 INFO Received uploads request method=POST path=/44422026/11/12 00:33:12 INFO Completed push id=19 objects=3 duration=1.321341ms44432026/11/12 00:33:12 INFO Received complete upload request method=POST path=/44442026/11/12 00:33:12 INFO Completed upload id=24 objects=3 duration=1.0369ms44452026/11/12 00:33:12 INFO Received complete upload request method=POST path=/44462026/11/12 00:33:12 INFO Completed upload id=26 objects=3 duration=1.131161ms44472026/11/12 00:33:12 INFO Received complete push request method=POST path=/44482026/11/12 00:33:12 INFO Completed push id=32 objects=9 duration=1.222281ms44492026/11/12 00:33:12 INFO Received complete push request method=POST path=/44502026/11/12 00:33:12 INFO Received uploads request method=POST path=/44512026/11/12 00:33:12 INFO Completed push id=21 objects=6 duration=1.43491ms44522026/11/12 00:33:12 INFO Received uploads request method=POST path=/44532026/11/12 00:33:12 INFO Received complete push request method=POST path=/44542026/11/12 00:33:12 INFO Received uploads request method=POST path=/44552026/11/12 00:33:12 INFO Completed push id=28 objects=14 duration=1.735411ms44562026/11/12 00:33:12 INFO Received uploads request method=POST path=/44572026/11/12 00:33:12 INFO Received complete upload request method=POST path=/44582026/11/12 00:33:12 INFO Received complete upload request method=POST path=/44592026/11/12 00:33:12 INFO Completed upload id=25 objects=4 duration=1.185301ms44602026/11/12 00:33:12 INFO Received push request method=POST path=/44612026/11/12 00:33:12 INFO Completed upload id=22 objects=2 duration=1.44108ms44622026/11/12 00:33:12 INFO Received uploads request method=POST path=/44632026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=044642026/11/12 00:33:12 INFO Received complete push request method=POST path=/44652026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period44662026/11/12 00:33:12 INFO Completed push id=25 objects=4 duration=1.223321ms44672026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044682026/11/12 00:33:12 INFO Vacuumed table table=pending_closures44692026/11/12 00:33:12 INFO Vacuumed table table=pending_objects44702026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads44712026/11/12 00:33:12 INFO Vacuumed table table=closures44722026/11/12 00:33:12 INFO Vacuumed table table=objects44732026/11/12 00:33:12 INFO Received complete push request method=POST path=/44742026/11/12 00:33:12 INFO Received complete upload request method=POST path=/44752026/11/12 00:33:12 INFO Received complete push request method=POST path=/44762026/11/12 00:33:12 INFO Completed upload id=33 objects=9 duration=1.039751ms44772026/11/12 00:33:12 INFO Completed push id=20 objects=4 duration=1.322041ms44782026/11/12 00:33:12 INFO Completed push id=11 objects=9 duration=1.747961ms44792026/11/12 00:33:12 INFO Received uploads request method=POST path=/44802026/11/12 00:33:12 INFO Received complete upload request method=POST path=/44812026/11/12 00:33:12 INFO Completed upload id=22 objects=7 duration=1.386591ms44822026/11/12 00:33:12 INFO Received push request method=POST path=/44832026/11/12 00:33:12 INFO Received push request method=POST path=/44842026/11/12 00:33:12 INFO Received complete upload request method=POST path=/44852026/11/12 00:33:12 INFO Received complete push request method=POST path=/44862026/11/12 00:33:12 INFO Received push request method=POST path=/44872026/11/12 00:33:12 INFO Completed upload id=26 objects=2 duration=2.040321ms44882026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=044892026/11/12 00:33:12 INFO Received uploads request method=POST path=/44902026/11/12 00:33:12 INFO Completed push id=19 objects=14 duration=1.931901ms44912026/11/12 00:33:12 INFO Received push request method=POST path=/44922026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period44932026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044942026/11/12 00:33:12 INFO Vacuumed table table=pending_closures44952026/11/12 00:33:12 INFO Vacuumed table table=pending_objects44962026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads44972026/11/12 00:33:12 INFO Vacuumed table table=closures44982026/11/12 00:33:12 INFO Vacuumed table table=objects44992026/11/12 00:33:12 INFO Received uploads request method=POST path=/45002026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=045012026/11/12 00:33:12 INFO Received complete push request method=POST path=/45022026/11/12 00:33:12 INFO Received uploads request method=POST path=/45032026/11/12 00:33:12 INFO Received push request method=POST path=/45042026/11/12 00:33:12 INFO Completed push id=24 objects=4 duration=1.344081ms45052026/11/12 00:33:12 INFO Received uploads request method=POST path=/45062026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period45072026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=045082026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045092026/11/12 00:33:12 INFO Received complete upload request method=POST path=/45102026/11/12 00:33:12 INFO Received complete push request method=POST path=/45112026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045122026/11/12 00:33:12 INFO Vacuumed table table=pending_closures45132026/11/12 00:33:12 INFO Completed upload id=22 objects=5 duration=1.885091ms45142026/11/12 00:33:12 INFO Received uploads request method=POST path=/45152026/11/12 00:33:12 INFO Completed push id=15 objects=24 duration=1.971131ms45162026/11/12 00:33:12 INFO Received complete push request method=POST path=/45172026/11/12 00:33:12 INFO Vacuumed table table=pending_closures45182026/11/12 00:33:12 INFO Vacuumed table table=pending_objects45192026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads45202026/11/12 00:33:12 INFO Vacuumed table table=closures45212026/11/12 00:33:12 INFO Completed push id=19 objects=14 duration=2.155811ms45222026/11/12 00:33:12 INFO Vacuumed table table=pending_objects45232026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads45242026/11/12 00:33:12 INFO Vacuumed table table=objects45252026/11/12 00:33:12 INFO Vacuumed table table=closures45262026/11/12 00:33:12 INFO Received complete upload request method=POST path=/45272026/11/12 00:33:12 INFO Received complete upload request method=POST path=/45282026/11/12 00:33:12 INFO Vacuumed table table=objects45292026/11/12 00:33:12 INFO Completed upload id=22 objects=8 duration=1.27066ms45302026/11/12 00:33:12 INFO Completed upload id=18 objects=2 duration=2.294201ms45312026/11/12 00:33:12 INFO Received uploads request method=POST path=/45322026/11/12 00:33:12 INFO Received complete push request method=POST path=/45332026/11/12 00:33:12 INFO Completed push id=25 objects=9 duration=1.15423ms45342026/11/12 00:33:12 INFO Received complete push request method=POST path=/45352026/11/12 00:33:12 INFO Completed push id=20 objects=8 duration=1.272441ms45362026/11/12 00:33:12 INFO Received complete push request method=POST path=/45372026/11/12 00:33:12 INFO Completed push id=32 objects=7 duration=1.362271ms45382026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=045392026/11/12 00:33:12 INFO Received complete push request method=POST path=/45402026/11/12 00:33:12 INFO Completed push id=26 objects=23 duration=1.68099ms45412026/11/12 00:33:12 INFO Received complete upload request method=POST path=/45422026/11/12 00:33:12 INFO Received complete push request method=POST path=/45432026/11/12 00:33:12 INFO Completed upload id=19 objects=2 duration=1.122761ms45442026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period45452026/11/12 00:33:12 INFO Completed push id=28 objects=3 duration=1.93642ms45462026/11/12 00:33:12 INFO Received complete push request method=POST path=/45472026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045482026/11/12 00:33:12 INFO Completed push id=13 objects=2 duration=1.38874ms45492026/11/12 00:33:12 INFO Vacuumed table table=pending_closures45502026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=045512026/11/12 00:33:12 INFO Vacuumed table table=pending_objects45522026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads45532026/11/12 00:33:12 INFO Vacuumed table table=closures45542026/11/12 00:33:12 INFO Vacuumed table table=objects45552026/11/12 00:33:12 INFO Received complete push request method=POST path=/45562026/11/12 00:33:12 INFO Received complete upload request method=POST path=/45572026/11/12 00:33:12 INFO Received push request method=POST path=/45582026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period45592026/11/12 00:33:12 INFO Completed push id=23 objects=5 duration=1.30382ms45602026/11/12 00:33:12 INFO Received complete push request method=POST path=/45612026/11/12 00:33:12 INFO Completed upload id=20 objects=6 duration=1.26719ms45622026/11/12 00:33:12 INFO Received push request method=POST path=/45632026/11/12 00:33:12 INFO Completed push id=9 objects=9 duration=2.867451ms45642026/11/12 00:33:12 INFO Received complete upload request method=POST path=/45652026/11/12 00:33:12 INFO Completed upload id=26 objects=3 duration=1.110901ms45662026/11/12 00:33:12 INFO Received uploads request method=POST path=/45672026/11/12 00:33:12 INFO Received complete push request method=POST path=/45682026/11/12 00:33:12 INFO Completed push id=26 objects=4 duration=1.289261ms45692026/11/12 00:33:12 INFO Received complete upload request method=POST path=/45702026/11/12 00:33:12 INFO Completed upload id=24 objects=7 duration=1.070571ms45712026/11/12 00:33:12 INFO Received push request method=POST path=/45722026/11/12 00:33:12 INFO Received complete push request method=POST path=/45732026/11/12 00:33:12 INFO Received complete upload request method=POST path=/45742026/11/12 00:33:12 INFO Completed push id=29 objects=15 duration=1.62026ms45752026/11/12 00:33:12 INFO Completed upload id=27 objects=13 duration=1.319381ms45762026/11/12 00:33:12 INFO Received uploads request method=POST path=/45772026/11/12 00:33:12 INFO Received complete upload request method=POST path=/45782026/11/12 00:33:12 INFO Completed upload id=24 objects=8 duration=1.067241ms45792026/11/12 00:33:12 INFO Received complete upload request method=POST path=/45802026/11/12 00:33:12 INFO Completed upload id=21 objects=8 duration=1.26925ms45812026/11/12 00:33:12 INFO Received push request method=POST path=/45822026/11/12 00:33:12 INFO Received complete push request method=POST path=/45832026/11/12 00:33:12 INFO Received complete upload request method=POST path=/45842026/11/12 00:33:12 INFO Completed push id=27 objects=15 duration=1.48671ms45852026/11/12 00:33:12 INFO Received complete push request method=POST path=/45862026/11/12 00:33:12 INFO Completed upload id=27 objects=7 duration=1.084051ms45872026/11/12 00:33:12 INFO Received complete upload request method=POST path=/45882026/11/12 00:33:12 INFO Received complete upload request method=POST path=/45892026/11/12 00:33:12 INFO Completed push id=26 objects=3 duration=1.19449ms45902026/11/12 00:33:12 INFO Completed upload id=28 objects=13 duration=1.33136ms45912026/11/12 00:33:12 INFO Completed upload id=13 objects=7 duration=1.290621ms45922026/11/12 00:33:12 INFO Received complete push request method=POST path=/45932026/11/12 00:33:12 INFO Completed push id=30 objects=6 duration=1.08965ms45942026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=045952026/11/12 00:33:12 INFO Received complete upload request method=POST path=/45962026/11/12 00:33:12 INFO Received uploads request method=POST path=/45972026/11/12 00:33:12 INFO Completed upload id=21 objects=9 duration=1.997001ms45982026/11/12 00:33:12 INFO Received complete upload request method=POST path=/45992026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046002026/11/12 00:33:12 INFO Completed upload id=28 objects=5 duration=977.08µs46012026/11/12 00:33:12 INFO Vacuumed table table=pending_closures46022026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=046032026/11/12 00:33:12 INFO Received complete push request method=POST path=/46042026/11/12 00:33:12 INFO Vacuumed table table=pending_objects46052026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads46062026/11/12 00:33:12 INFO Vacuumed table table=pending_closures46072026/11/12 00:33:12 INFO Completed push id=22 objects=6 duration=1.441881ms46082026/11/12 00:33:12 INFO Vacuumed table table=closures46092026/11/12 00:33:12 INFO Vacuumed table table=objects46102026/11/12 00:33:12 INFO Vacuumed table table=pending_objects46112026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads46122026/11/12 00:33:12 INFO Vacuumed table table=closures46132026/11/12 00:33:12 INFO Vacuumed table table=objects46142026/11/12 00:33:12 INFO Received complete upload request method=POST path=/46152026/11/12 00:33:12 INFO Received complete upload request method=POST path=/46162026/11/12 00:33:12 INFO Completed upload id=28 objects=8 duration=942.1µs46172026/11/12 00:33:12 INFO Completed upload id=20 objects=4 duration=1.02666ms46182026/11/12 00:33:12 INFO Received complete upload request method=POST path=/46192026/11/12 00:33:12 INFO Completed upload id=28 objects=3 duration=1.016551ms46202026/11/12 00:33:12 INFO Received complete push request method=POST path=/46212026/11/12 00:33:12 INFO Completed push id=29 objects=3 duration=1.426311ms46222026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=046232026/11/12 00:33:12 INFO Received complete upload request method=POST path=/46242026/11/12 00:33:12 INFO Completed upload id=31 objects=3 duration=967.16µs46252026/11/12 00:33:12 INFO Received complete push request method=POST path=/46262026/11/12 00:33:12 INFO Completed push id=25 objects=10 duration=2.058911ms46272026/11/12 00:33:12 INFO Received complete push request method=POST path=/46282026/11/12 00:33:12 INFO Received complete push request method=POST path=/46292026/11/12 00:33:12 INFO Completed push id=24 objects=8 duration=1.502971ms46302026/11/12 00:33:12 INFO Received uploads request method=POST path=/46312026/11/12 00:33:12 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."46322026/11/12 00:33:12 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."46332026/11/12 00:33:12 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."46342026/11/12 00:33:12 INFO Completed push id=16 objects=7 duration=1.85826ms46352026/11/12 00:33:12 INFO Received complete upload request method=POST path=/46362026/11/12 00:33:12 INFO Completed upload id=30 objects=5 duration=1.07907ms46372026/11/12 00:33:12 INFO Received push request method=POST path=/46382026/11/12 00:33:12 INFO Received complete push request method=POST path=/46392026/11/12 00:33:12 INFO Completed push id=27 objects=11 duration=1.419131ms46402026/11/12 00:33:12 INFO Received complete upload request method=POST path=/46412026/11/12 00:33:12 INFO Received complete upload request method=POST path=/46422026/11/12 00:33:12 INFO Completed upload id=23 objects=7 duration=1.29834ms46432026/11/12 00:33:12 INFO Completed upload id=29 objects=10 duration=943.641µs46442026/11/12 00:33:12 INFO Received complete upload request method=POST path=/46452026/11/12 00:33:12 INFO Completed upload id=31 objects=2 duration=947.44µs46462026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=046472026/11/12 00:33:12 INFO Received complete upload request method=POST path=/46482026/11/12 00:33:12 INFO Received complete push request method=POST path=/46492026/11/12 00:33:12 INFO Completed upload id=21 objects=2 duration=1.07654ms46502026/11/12 00:33:12 WARN Force mode enabled - objects will be deleted immediately without grace period46512026/11/12 00:33:12 INFO Completed push id=29 objects=23 duration=1.366801ms46522026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046532026/11/12 00:33:12 INFO Received complete upload request method=POST path=/46542026/11/12 00:33:12 INFO Received complete upload request method=POST path=/46552026/11/12 00:33:12 INFO Completed upload id=25 objects=9 duration=1.233981ms46562026/11/12 00:33:12 INFO Vacuumed table table=pending_closures46572026/11/12 00:33:12 INFO Completed upload id=29 objects=5 duration=1.13547ms46582026/11/12 00:33:12 INFO Received complete push request method=POST path=/46592026/11/12 00:33:12 INFO Vacuumed table table=pending_objects46602026/11/12 00:33:12 INFO Received push request method=POST path=/46612026/11/12 00:33:12 INFO Vacuumed table table=multipart_uploads46622026/11/12 00:33:12 INFO Completed push id=28 objects=19 duration=1.501101ms46632026/11/12 00:33:12 INFO Vacuumed table table=closures46642026/11/12 00:33:12 INFO Vacuumed table table=objects46652026/11/12 00:33:12 INFO Received push request method=POST path=/46662026/11/12 00:33:12 INFO Received uploads request method=POST path=/46672026/11/12 00:33:12 INFO Received complete upload request method=POST path=/46682026/11/12 00:33:12 INFO Received complete upload request method=POST path=/46692026/11/12 00:33:12 INFO Completed upload id=30 objects=20 duration=1.10081ms46702026/11/12 00:33:12 INFO Completed upload id=29 objects=26 duration=1.71581ms46712026/11/12 00:33:12 INFO Received complete push request method=POST path=/46722026/11/12 00:33:12 INFO Received complete push request method=POST path=/46732026/11/12 00:33:12 INFO Completed push id=33 objects=6 duration=1.21018ms46742026/11/12 00:33:12 INFO Completed push id=31 objects=3 duration=1.84019ms46752026/11/12 00:33:12 INFO Received uploads request method=POST path=/46762026/11/12 00:33:12 INFO Received complete push request method=POST path=/46772026/11/12 00:33:12 INFO Completed push id=23 objects=6 duration=1.18374ms46782026/11/12 00:33:12 INFO Received complete upload request method=POST path=/46792026/11/12 00:33:12 INFO Received complete push request method=POST path=/46802026/11/12 00:33:12 INFO Completed upload id=27 objects=3 duration=1.14786ms46812026/11/12 00:33:12 INFO Completed push id=30 objects=6 duration=1.13968ms46822026/11/12 00:33:12 INFO Received complete upload request method=POST path=/46832026/11/12 00:33:12 INFO Completed upload id=30 objects=2 duration=1.23444ms46842026/11/12 00:33:12 INFO Received push request method=POST path=/46852026/11/12 00:33:12 INFO Aborted multipart uploads count=0 kept=046862026/11/12 00:33:12 INFO Received complete push request method=POST path=/46872026/11/12 00:33:12 INFO Received complete push request method=POST path=/46882026/11/12 00:33:12 INFO Completed push id=30 objects=8 duration=1.21727ms46892026/11/12 00:33:12 INFO Completed push id=31 objects=11 duration=1.204111ms46902026/11/12 00:33:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046912026/11/12 00:33:12 INFO Vacuumed table table=pending_closures46922026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=046932026/11/12 00:33:13 INFO Received complete push request method=POST path=/46942026/11/12 00:33:13 INFO Vacuumed table table=pending_objects46952026/11/12 00:33:13 INFO Received complete push request method=POST path=/46962026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads46972026/11/12 00:33:13 INFO Vacuumed table table=closures46982026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period46992026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47002026/11/12 00:33:13 INFO Completed push id=29 objects=10 duration=1.677651ms47012026/11/12 00:33:13 INFO Completed push id=24 objects=6 duration=1.276311ms47022026/11/12 00:33:13 INFO Vacuumed table table=objects47032026/11/12 00:33:13 INFO Completed upload id=28 objects=6 duration=1.00871ms47042026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=047052026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047062026/11/12 00:33:13 INFO Received push request method=POST path=/47072026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47082026/11/12 00:33:13 INFO Vacuumed table table=pending_closures47092026/11/12 00:33:13 INFO Received complete push request method=POST path=/47102026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period47112026/11/12 00:33:13 INFO Completed upload id=24 objects=18 duration=1.572751ms47122026/11/12 00:33:13 INFO Vacuumed table table=pending_objects47132026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads47142026/11/12 00:33:13 INFO Completed push id=31 objects=20 duration=1.250171ms47152026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047162026/11/12 00:33:13 INFO Vacuumed table table=closures47172026/11/12 00:33:13 INFO Vacuumed table table=pending_closures47182026/11/12 00:33:13 INFO Vacuumed table table=objects47192026/11/12 00:33:13 INFO Vacuumed table table=pending_objects47202026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads47212026/11/12 00:33:13 INFO Received complete push request method=POST path=/47222026/11/12 00:33:13 INFO Received complete push request method=POST path=/47232026/11/12 00:33:13 INFO Vacuumed table table=closures47242026/11/12 00:33:13 INFO Received uploads request method=POST path=/47252026/11/12 00:33:13 INFO Completed push id=33 objects=3 duration=1.34525ms47262026/11/12 00:33:13 INFO Vacuumed table table=objects47272026/11/12 00:33:13 INFO Completed push id=32 objects=6 duration=1.8541ms47282026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47292026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47302026/11/12 00:33:13 INFO Completed upload id=32 objects=2 duration=871.31µs47312026/11/12 00:33:13 INFO Completed upload id=31 objects=5 duration=967.66µs47322026/11/12 00:33:13 INFO Received complete push request method=POST path=/47332026/11/12 00:33:13 INFO Completed push id=30 objects=4 duration=1.108041ms47342026/11/12 00:33:13 INFO Received complete push request method=POST path=/47352026/11/12 00:33:13 INFO Received complete push request method=POST path=/47362026/11/12 00:33:13 INFO Completed push id=28 objects=11 duration=2.656531ms47372026/11/12 00:33:13 INFO Completed push id=25 objects=2 duration=2.172411ms47382026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47392026/11/12 00:33:13 INFO Completed upload id=32 objects=5 duration=972.05µs47402026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47412026/11/12 00:33:13 INFO Completed upload id=30 objects=6 duration=989.94µs47422026/11/12 00:33:13 INFO Received complete push request method=POST path=/47432026/11/12 00:33:13 INFO Completed push id=25 objects=7 duration=1.18171ms47442026/11/12 00:33:13 INFO Received uploads request method=POST path=/47452026/11/12 00:33:13 INFO Received complete push request method=POST path=/47462026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47472026/11/12 00:33:13 INFO Completed push id=34 objects=6 duration=1.17421ms47482026/11/12 00:33:13 INFO Completed upload id=33 objects=7 duration=1.23397ms47492026/11/12 00:33:13 INFO Received complete push request method=POST path=/47502026/11/12 00:33:13 INFO Received complete push request method=POST path=/47512026/11/12 00:33:13 INFO Received complete push request method=POST path=/47522026/11/12 00:33:13 INFO Completed push id=33 objects=3 duration=1.073081ms47532026/11/12 00:33:13 INFO Completed push id=32 objects=3 duration=1.191561ms47542026/11/12 00:33:13 INFO Completed push id=31 objects=7 duration=1.394801ms47552026/11/12 00:33:13 INFO Received complete push request method=POST path=/47562026/11/12 00:33:13 INFO Completed push id=27 objects=4 duration=1.131201ms47572026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47582026/11/12 00:33:13 INFO Completed upload id=33 objects=3 duration=1.10857ms47592026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47602026/11/12 00:33:13 INFO Received uploads request method=POST path=/47612026/11/12 00:33:13 INFO Completed upload id=31 objects=5 duration=1.2407ms47622026/11/12 00:33:13 INFO Received complete push request method=POST path=/47632026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47642026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47652026/11/12 00:33:13 INFO Completed push id=35 objects=8 duration=1.25539ms47662026/11/12 00:33:13 INFO Completed upload id=34 objects=3 duration=1.07571ms47672026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47682026/11/12 00:33:13 INFO Completed upload id=29 objects=4 duration=1.956741ms47692026/11/12 00:33:13 INFO Completed upload id=32 objects=5 duration=1.017551ms47702026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47712026/11/12 00:33:13 INFO Completed upload id=34 objects=3 duration=848.59µs47722026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=047732026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period47742026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047752026/11/12 00:33:13 INFO Vacuumed table table=pending_closures47762026/11/12 00:33:13 INFO Vacuumed table table=pending_objects47772026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads47782026/11/12 00:33:13 INFO Vacuumed table table=closures47792026/11/12 00:33:13 INFO Received complete push request method=POST path=/47802026/11/12 00:33:13 INFO Vacuumed table table=objects47812026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=047822026/11/12 00:33:13 INFO Completed push id=29 objects=24 duration=1.87699ms47832026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period47842026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47852026/11/12 00:33:13 INFO Completed upload id=34 objects=3 duration=1.11267ms47862026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047872026/11/12 00:33:13 INFO Received push request method=POST path=/47882026/11/12 00:33:13 INFO Vacuumed table table=pending_closures47892026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47902026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47912026/11/12 00:33:13 INFO Vacuumed table table=pending_objects47922026/11/12 00:33:13 INFO Completed upload id=26 objects=24 duration=1.363211ms47932026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads47942026/11/12 00:33:13 INFO Received complete push request method=POST path=/47952026/11/12 00:33:13 INFO Received complete upload request method=POST path=/47962026/11/12 00:33:13 INFO Received uploads request method=POST path=/47972026/11/12 00:33:13 INFO Completed upload id=36 objects=9 duration=1.277ms47982026/11/12 00:33:13 INFO Vacuumed table table=closures47992026/11/12 00:33:13 INFO Completed push id=35 objects=11 duration=1.209771ms48002026/11/12 00:33:13 INFO Received complete push request method=POST path=/48012026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=048022026/11/12 00:33:13 INFO Vacuumed table table=objects48032026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period48042026/11/12 00:33:13 INFO Completed push id=28 objects=26 duration=1.81198ms48052026/11/12 00:33:13 INFO Completed upload id=13 objects=14 duration=3.342572ms48062026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048072026/11/12 00:33:13 INFO Vacuumed table table=pending_closures48082026/11/12 00:33:13 INFO Vacuumed table table=pending_objects48092026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads48102026/11/12 00:33:13 INFO Vacuumed table table=closures48112026/11/12 00:33:13 INFO Vacuumed table table=objects48122026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=048132026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=048142026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period48152026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period48162026/11/12 00:33:13 INFO Received complete push request method=POST path=/48172026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048182026/11/12 00:33:13 INFO Vacuumed table table=pending_closures48192026/11/12 00:33:13 INFO Completed push id=24 objects=24 duration=3.419731ms48202026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=048212026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=048222026/11/12 00:33:13 INFO Vacuumed table table=pending_objects48232026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads48242026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=048252026/11/12 00:33:13 INFO Received uploads request method=POST path=/48262026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period48272026/11/12 00:33:13 INFO Vacuumed table table=closures48282026/11/12 00:33:13 INFO Received complete push request method=POST path=/48292026/11/12 00:33:13 INFO Vacuumed table table=objects48302026/11/12 00:33:13 INFO Vacuumed table table=pending_closures48312026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048322026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period48332026/11/12 00:33:13 INFO Completed push id=30 objects=3 duration=1.270441ms48342026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=048352026/11/12 00:33:13 INFO Vacuumed table table=pending_objects48362026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads48372026/11/12 00:33:13 INFO Vacuumed table table=pending_closures48382026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=6 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048392026/11/12 00:33:13 INFO Vacuumed table table=closures48402026/11/12 00:33:13 INFO Vacuumed table table=objects48412026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048422026/11/12 00:33:13 INFO Vacuumed table table=pending_objects48432026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads48442026/11/12 00:33:13 INFO Vacuumed table table=pending_closures48452026/11/12 00:33:13 INFO Vacuumed table table=pending_closures48462026/11/12 00:33:13 INFO Vacuumed table table=closures48472026/11/12 00:33:13 INFO Vacuumed table table=pending_objects48482026/11/12 00:33:13 INFO Vacuumed table table=objects48492026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads48502026/11/12 00:33:13 INFO Vacuumed table table=pending_objects48512026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads48522026/11/12 00:33:13 INFO Received complete upload request method=POST path=/48532026/11/12 00:33:13 INFO Vacuumed table table=closures48542026/11/12 00:33:13 INFO Vacuumed table table=closures48552026/11/12 00:33:13 INFO Vacuumed table table=objects48562026/11/12 00:33:13 INFO Vacuumed table table=objects48572026/11/12 00:33:13 INFO Completed upload id=34 objects=24 duration=1.68377ms48582026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=048592026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period48602026/11/12 00:33:13 INFO Received complete push request method=POST path=/48612026/11/12 00:33:13 INFO Received complete upload request method=POST path=/48622026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048632026/11/12 00:33:13 INFO Completed upload id=27 objects=26 duration=1.16999ms48642026/11/12 00:33:13 INFO Completed push id=32 objects=19 duration=1.73291ms48652026/11/12 00:33:13 INFO Vacuumed table table=pending_closures48662026/11/12 00:33:13 INFO Vacuumed table table=pending_objects48672026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads48682026/11/12 00:33:13 INFO Received complete push request method=POST path=/48692026/11/12 00:33:13 INFO Vacuumed table table=closures48702026/11/12 00:33:13 INFO Vacuumed table table=objects48712026/11/12 00:33:13 INFO Completed push id=11 objects=6 duration=2.004021ms48722026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=048732026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period48742026/11/12 00:33:13 INFO Received complete push request method=POST path=/48752026/11/12 00:33:13 INFO Received complete push request method=POST path=/48762026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048772026/11/12 00:33:13 INFO Completed push id=28 objects=6 duration=797.701µs48782026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=048792026/11/12 00:33:13 INFO Vacuumed table table=pending_closures48802026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period48812026/11/12 00:33:13 INFO Completed push id=31 objects=6 duration=3.082361ms48822026/11/12 00:33:13 INFO Vacuumed table table=pending_objects48832026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads48842026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048852026/11/12 00:33:13 INFO Vacuumed table table=closures48862026/11/12 00:33:13 INFO Received uploads request method=POST path=/48872026/11/12 00:33:13 INFO Vacuumed table table=objects48882026/11/12 00:33:13 INFO Vacuumed table table=pending_closures48892026/11/12 00:33:13 INFO Vacuumed table table=pending_objects48902026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads48912026/11/12 00:33:13 INFO Vacuumed table table=closures48922026/11/12 00:33:13 INFO Vacuumed table table=objects48932026/11/12 00:33:13 INFO Received complete push request method=POST path=/48942026/11/12 00:33:13 INFO Completed push id=20 objects=13 duration=1.472091ms48952026/11/12 00:33:13 INFO Received complete push request method=POST path=/48962026/11/12 00:33:13 INFO Completed push id=29 objects=6 duration=669.06µs48972026/11/12 00:33:13 INFO Received uploads request method=POST path=/48982026/11/12 00:33:13 INFO Received uploads request method=POST path=/48992026/11/12 00:33:13 INFO Received complete push request method=POST path=/49002026/11/12 00:33:13 INFO Completed push id=22 objects=4 duration=1.352171ms49012026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49022026/11/12 00:33:13 INFO Completed upload id=30 objects=8 duration=810.261µs49032026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49042026/11/12 00:33:13 INFO Completed upload id=23 objects=7 duration=1.128031ms49052026/11/12 00:33:13 INFO Received push request method=POST path=/49062026/11/12 00:33:13 INFO Received complete push request method=POST path=/49072026/11/12 00:33:13 INFO Completed push id=24 objects=5 duration=1.338841ms49082026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49092026/11/12 00:33:13 INFO Received uploads request method=POST path=/49102026/11/12 00:33:13 INFO Completed upload id=31 objects=6 duration=1.08494ms49112026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49122026/11/12 00:33:13 INFO Completed upload id=25 objects=2 duration=1.063641ms49132026/11/12 00:33:13 INFO Received complete push request method=POST path=/49142026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49152026/11/12 00:33:13 INFO Completed push id=1 objects=9 duration=1.793071ms49162026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=049172026/11/12 00:33:13 INFO Completed upload id=32 objects=4 duration=1.043ms49182026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period49192026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049202026/11/12 00:33:13 INFO Vacuumed table table=pending_closures49212026/11/12 00:33:13 INFO Vacuumed table table=pending_objects49222026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads49232026/11/12 00:33:13 INFO Vacuumed table table=closures49242026/11/12 00:33:13 INFO Vacuumed table table=objects49252026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49262026/11/12 00:33:13 INFO Completed upload id=26 objects=20 duration=1.2377ms49272026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49282026/11/12 00:33:13 INFO Completed upload id=33 objects=10 duration=1.60549ms49292026/11/12 00:33:13 INFO Received complete push request method=POST path=/49302026/11/12 00:33:13 INFO Completed push id=28 objects=7 duration=1.350241ms49312026/11/12 00:33:13 INFO Received push request method=POST path=/49322026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49332026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49342026/11/12 00:33:13 INFO Completed upload id=30 objects=6 duration=1.198ms49352026/11/12 00:33:13 INFO Completed upload id=34 objects=23 duration=1.68235ms49362026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49372026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49382026/11/12 00:33:13 INFO Completed upload id=2 objects=7 duration=1.377691ms49392026/11/12 00:33:13 INFO Completed upload id=23 objects=7 duration=2.658581ms49402026/11/12 00:33:13 INFO Received complete push request method=POST path=/49412026/11/12 00:33:13 INFO Completed push id=31 objects=5 duration=1.22683ms49422026/11/12 00:33:13 INFO Received complete push request method=POST path=/49432026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49442026/11/12 00:33:13 INFO Completed push id=35 objects=6 duration=1.330951ms49452026/11/12 00:33:13 INFO Completed upload id=26 objects=5 duration=1.4447ms49462026/11/12 00:33:13 INFO Received complete push request method=POST path=/49472026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=049482026/11/12 00:33:13 INFO Completed push id=27 objects=3 duration=2.223021ms49492026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period49502026/11/12 00:33:13 INFO Received complete push request method=POST path=/49512026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49522026/11/12 00:33:13 INFO Completed upload id=8 objects=6 duration=1.15612ms49532026/11/12 00:33:13 INFO Completed push id=32 objects=26 duration=1.974551ms49542026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=049552026/11/12 00:33:13 INFO Vacuumed table table=pending_closures49562026/11/12 00:33:13 INFO Vacuumed table table=pending_objects49572026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads49582026/11/12 00:33:13 INFO Vacuumed table table=closures49592026/11/12 00:33:13 INFO Vacuumed table table=objects49602026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49612026/11/12 00:33:13 INFO Completed upload id=28 objects=5 duration=926.841µs49622026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49632026/11/12 00:33:13 INFO Completed upload id=33 objects=2 duration=1.0779ms49642026/11/12 00:33:13 INFO Received complete push request method=POST path=/49652026/11/12 00:33:13 INFO Completed push id=29 objects=17 duration=1.42881ms49662026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49672026/11/12 00:33:13 INFO Completed upload id=34 objects=5 duration=1.12216ms49682026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=049692026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period49702026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049712026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49722026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49732026/11/12 00:33:13 INFO Completed upload id=35 objects=11 duration=1.05913ms49742026/11/12 00:33:13 INFO Completed upload id=12 objects=9 duration=1.22377ms49752026/11/12 00:33:13 INFO Vacuumed table table=pending_closures49762026/11/12 00:33:13 INFO Vacuumed table table=pending_objects49772026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads49782026/11/12 00:33:13 INFO Vacuumed table table=closures49792026/11/12 00:33:13 INFO Vacuumed table table=objects49802026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49812026/11/12 00:33:13 INFO Completed upload id=36 objects=3 duration=1.14963ms49822026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=049832026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period49842026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049852026/11/12 00:33:13 INFO Vacuumed table table=pending_closures49862026/11/12 00:33:13 INFO Vacuumed table table=pending_objects49872026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads49882026/11/12 00:33:13 INFO Vacuumed table table=closures49892026/11/12 00:33:13 INFO Vacuumed table table=objects49902026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49912026/11/12 00:33:13 INFO Completed upload id=18 objects=22 duration=1.608351ms49922026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49932026/11/12 00:33:13 INFO Completed upload id=21 objects=8 duration=948.681µs49942026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49952026/11/12 00:33:13 INFO Completed upload id=22 objects=7 duration=978.92µs49962026/11/12 00:33:13 INFO Received complete push request method=POST path=/49972026/11/12 00:33:13 INFO Completed push id=24 objects=5 duration=1.131981ms49982026/11/12 00:33:13 INFO Received complete upload request method=POST path=/49992026/11/12 00:33:13 INFO Completed upload id=25 objects=8 duration=1.151001ms50002026/11/12 00:33:13 INFO Received complete push request method=POST path=/50012026/11/12 00:33:13 INFO Completed push id=26 objects=4 duration=1.16277ms50022026/11/12 00:33:13 INFO Received complete push request method=POST path=/50032026/11/12 00:33:13 INFO Completed push id=30 objects=2 duration=1.06042ms50042026/11/12 00:33:13 INFO Received complete push request method=POST path=/50052026/11/12 00:33:13 INFO Completed push id=32 objects=7 duration=1.412101ms50062026/11/12 00:33:13 INFO Received complete upload request method=POST path=/50072026/11/12 00:33:13 INFO Completed upload id=33 objects=6 duration=1.12782ms50082026/11/12 00:33:13 INFO Received complete upload request method=POST path=/50092026/11/12 00:33:13 INFO Completed upload id=34 objects=3 duration=1.03623ms50102026/11/12 00:33:13 INFO Aborted multipart uploads count=0 kept=050112026/11/12 00:33:13 WARN Force mode enabled - objects will be deleted immediately without grace period50122026/11/12 00:33:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050132026/11/12 00:33:13 INFO Vacuumed table table=pending_closures50142026/11/12 00:33:13 INFO Vacuumed table table=pending_objects50152026/11/12 00:33:13 INFO Vacuumed table table=multipart_uploads50162026/11/12 00:33:13 INFO Vacuumed table table=closures50172026/11/12 00:33:13 INFO Vacuumed table table=objects5018--- PASS: FuzzPushGC (0.00s)5019 --- PASS: FuzzPushGC/seed#0 (2.93s)5020 --- PASS: FuzzPushGC/seed#15 (3.06s)5021 --- PASS: FuzzPushGC/seed#14 (3.10s)5022 --- PASS: FuzzPushGC/seed#3 (3.17s)5023 --- PASS: FuzzPushGC/seed#6 (3.18s)5024 --- PASS: FuzzPushGC/seed#5 (3.19s)5025 --- PASS: FuzzPushGC/seed#13 (3.20s)5026 --- PASS: FuzzPushGC/seed#2 (3.20s)5027 --- PASS: FuzzPushGC/seed#4 (3.21s)5028 --- PASS: FuzzPushGC/seed#8 (3.21s)5029 --- PASS: FuzzPushGC/seed#9 (3.23s)5030 --- PASS: FuzzPushGC/seed#7 (3.24s)5031 --- PASS: FuzzPushGC/seed#12 (3.45s)5032 --- PASS: FuzzPushGC/seed#1 (3.51s)5033 --- PASS: FuzzPushGC/seed#11 (3.55s)5034 --- PASS: FuzzPushGC/seed#10 (3.90s)5035PASS5036{"timestamp":"2026-11-12T00:33:13.835809138Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:40804","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(396)"}5037{"timestamp":"2026-11-12T00:33:13.836603349Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:41888","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(396)"}5038{"timestamp":"2026-11-12T00:33:13.836792879Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:42246","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(400)"}50392026-11-12 00:33:14.248 UTC [128] LOG: received smart shutdown request50402026-11-12 00:33:14.251 UTC [128] LOG: background worker "logical replication launcher" (PID 138) exited with exit code 150412026-11-12 00:33:15.077 UTC [133] LOG: checkpoint complete: wrote 4158 buffers (25.4%), wrote 5 SLRU buffers; 0 WAL file(s) added, 0 removed, 33 recycled; write=4.395 s, sync=0.655 s, total=5.127 s; sync files=42371, longest=0.013 s, average=0.001 s; distance=534899 kB, estimate=534899 kB; lsn=0/26D79AE8, redo lsn=0/221C28F850422026-11-12 00:33:15.080 UTC [133] LOG: shutting down50432026-11-12 00:33:15.081 UTC [133] LOG: checkpoint starting: shutdown immediate50442026-11-12 00:33:16.056 UTC [133] LOG: checkpoint complete: wrote 9277 buffers (56.6%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 4 recycled; write=0.175 s, sync=0.792 s, total=0.976 s; sync files=2812, longest=0.002 s, average=0.001 s; distance=77532 kB, estimate=489162 kB; lsn=0/26D79B60, redo lsn=0/26D79B6050452026-11-12 00:33:16.140 UTC [128] LOG: database system is shut down5046Running OIDC tests...5047=== RUN TestAudienceForIssuer5048=== PAUSE TestAudienceForIssuer5049=== RUN TestProviderClientGivesUpOnSilentIssuer5050=== PAUSE TestProviderClientGivesUpOnSilentIssuer5051=== RUN TestGlobMatch5052=== PAUSE TestGlobMatch5053=== RUN TestValidateToken_ValidToken5054=== PAUSE TestValidateToken_ValidToken5055=== RUN TestValidateToken_WrongAudience5056=== PAUSE TestValidateToken_WrongAudience5057=== RUN TestValidateToken_Expired5058=== PAUSE TestValidateToken_Expired5059=== RUN TestValidateToken_BoundClaimsMismatch5060=== PAUSE TestValidateToken_BoundClaimsMismatch5061=== RUN TestValidateToken_BoundSubjectMismatch5062=== PAUSE TestValidateToken_BoundSubjectMismatch5063=== RUN TestValidateToken_MultipleProviders5064=== PAUSE TestValidateToken_MultipleProviders5065=== RUN TestValidateToken_UnknownIssuerFetchesNoKeys5066=== PAUSE TestValidateToken_UnknownIssuerFetchesNoKeys5067=== RUN TestValidateToken_GoogleSchemelessIssuer5068=== PAUSE TestValidateToken_GoogleSchemelessIssuer5069=== RUN TestValidateToken_KubernetesServiceAccount5070=== PAUSE TestValidateToken_KubernetesServiceAccount5071=== RUN TestNewValidator_KubernetesRequiresCA5072=== PAUSE TestNewValidator_KubernetesRequiresCA5073=== RUN TestValidateToken_KubernetesIssuerFromOwnToken5074=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken5075=== RUN TestPins_ReservedForMatchingRule5076=== PAUSE TestPins_ReservedForMatchingRule5077=== RUN TestPins_TopLevelShorthand5078=== PAUSE TestPins_TopLevelShorthand5079=== RUN TestPins_ConfigValidation5080=== PAUSE TestPins_ConfigValidation5081=== RUN TestScopes_LegacyProviderDefaultsToWrite5082=== PAUSE TestScopes_LegacyProviderDefaultsToWrite5083=== RUN TestScopes_Rules5084=== PAUSE TestScopes_Rules5085=== RUN TestScopes_ConfigValidation5086=== PAUSE TestScopes_ConfigValidation5087=== CONT TestAudienceForIssuer5088=== CONT TestValidateToken_GoogleSchemelessIssuer5089=== CONT TestScopes_Rules5090--- PASS: TestAudienceForIssuer (0.00s)5091=== CONT TestValidateToken_UnknownIssuerFetchesNoKeys5092=== CONT TestValidateToken_MultipleProviders5093=== CONT TestValidateToken_BoundSubjectMismatch5094=== CONT TestValidateToken_BoundClaimsMismatch50952026/11/12 00:33:18 INFO OIDC provider initialized name=known issuer=https://known.example5096=== CONT TestValidateToken_Expired5097=== CONT TestValidateToken_WrongAudience5098=== CONT TestValidateToken_ValidToken5099=== CONT TestGlobMatch5100=== RUN TestGlobMatch/foo_foo5101=== CONT TestProviderClientGivesUpOnSilentIssuer5102=== CONT TestScopes_ConfigValidation5103--- PASS: TestProviderClientGivesUpOnSilentIssuer (0.00s)5104=== CONT TestValidateToken_KubernetesIssuerFromOwnToken5105=== CONT TestPins_ReservedForMatchingRule5106=== CONT TestScopes_LegacyProviderDefaultsToWrite5107--- PASS: TestScopes_ConfigValidation (0.00s)5108=== CONT TestNewValidator_KubernetesRequiresCA5109=== CONT TestValidateToken_KubernetesServiceAccount5110=== CONT TestPins_ConfigValidation5111=== CONT TestPins_TopLevelShorthand5112=== PAUSE TestGlobMatch/foo_foo5113=== RUN TestGlobMatch/foo_bar5114=== PAUSE TestGlobMatch/foo_bar5115=== RUN TestGlobMatch/*_5116=== PAUSE TestGlobMatch/*_5117=== RUN TestGlobMatch/*_anything5118=== PAUSE TestGlobMatch/*_anything5119=== RUN TestGlobMatch/foo*_foo5120=== PAUSE TestGlobMatch/foo*_foo5121=== RUN TestGlobMatch/foo*_foobar5122=== PAUSE TestGlobMatch/foo*_foobar5123=== RUN TestGlobMatch/foo*_bar5124=== PAUSE TestGlobMatch/foo*_bar5125=== RUN TestGlobMatch/*bar_bar5126=== PAUSE TestGlobMatch/*bar_bar5127=== RUN TestGlobMatch/*bar_foobar5128=== PAUSE TestGlobMatch/*bar_foobar5129=== RUN TestGlobMatch/*bar_foo5130=== PAUSE TestGlobMatch/*bar_foo5131=== RUN TestGlobMatch/foo*bar_foobar5132=== PAUSE TestGlobMatch/foo*bar_foobar5133=== RUN TestGlobMatch/foo*bar_foo123bar5134=== PAUSE TestGlobMatch/foo*bar_foo123bar5135=== RUN TestGlobMatch/foo*bar_foobarbaz5136=== PAUSE TestGlobMatch/foo*bar_foobarbaz5137=== RUN TestGlobMatch/*/*_foo/bar5138=== PAUSE TestGlobMatch/*/*_foo/bar5139=== RUN TestGlobMatch/*/*_foo5140=== PAUSE TestGlobMatch/*/*_foo5141=== RUN TestGlobMatch/refs/heads/*_refs/heads/main5142=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main5143=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.05144=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.05145=== RUN TestGlobMatch/refs/*/main_refs/heads/main5146=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main5147=== RUN TestGlobMatch/fo?_foo5148=== PAUSE TestGlobMatch/fo?_foo5149=== RUN TestGlobMatch/fo?_fo5150=== PAUSE TestGlobMatch/fo?_fo5151=== RUN TestGlobMatch/fo?_fooo5152=== PAUSE TestGlobMatch/fo?_fooo5153=== RUN TestGlobMatch/?oo_foo5154=== PAUSE TestGlobMatch/?oo_foo5155=== RUN TestGlobMatch/?oo_boo5156=== PAUSE TestGlobMatch/?oo_boo5157=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5158=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5159=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5160=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5161=== CONT TestGlobMatch/foo_foo5162=== CONT TestGlobMatch/?oo_boo5163=== CONT TestGlobMatch/*bar_bar5164=== CONT TestGlobMatch/foo*_bar5165=== CONT TestGlobMatch/foo*_foobar5166=== CONT TestGlobMatch/foo*_foo5167=== CONT TestGlobMatch/*_anything5168=== CONT TestGlobMatch/*_5169=== CONT TestGlobMatch/foo_bar5170=== CONT TestGlobMatch/foo*bar_foo123bar5171=== CONT TestGlobMatch/?oo_foo5172=== CONT TestGlobMatch/fo?_fooo5173=== CONT TestGlobMatch/refs/*/main_refs/heads/main5174=== CONT TestGlobMatch/fo?_foo5175=== CONT TestGlobMatch/*/*_foo5176=== CONT TestGlobMatch/*/*_foo/bar5177=== CONT TestGlobMatch/foo*bar_foobar5178=== CONT TestGlobMatch/*bar_foo5179=== CONT TestGlobMatch/*bar_foobar5180=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5181=== CONT TestGlobMatch/fo?_fo5182=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5183--- PASS: TestPins_ConfigValidation (0.00s)5184=== CONT TestGlobMatch/refs/heads/*_refs/heads/main5185=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.05186=== CONT TestGlobMatch/foo*bar_foobarbaz5187--- PASS: TestGlobMatch (0.01s)5188 --- PASS: TestGlobMatch/foo_foo (0.00s)5189 --- PASS: TestGlobMatch/?oo_boo (0.00s)5190 --- PASS: TestGlobMatch/*bar_bar (0.00s)5191 --- PASS: TestGlobMatch/foo*_bar (0.00s)5192 --- PASS: TestGlobMatch/foo*_foobar (0.00s)5193 --- PASS: TestGlobMatch/foo*_foo (0.00s)5194 --- PASS: TestGlobMatch/*_anything (0.00s)5195 --- PASS: TestGlobMatch/*_ (0.00s)5196 --- PASS: TestGlobMatch/foo_bar (0.00s)5197 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)5198 --- PASS: TestGlobMatch/?oo_foo (0.00s)5199 --- PASS: TestGlobMatch/fo?_fooo (0.00s)5200 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)5201 --- PASS: TestGlobMatch/fo?_foo (0.00s)5202 --- PASS: TestGlobMatch/*/*_foo (0.00s)5203 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)5204 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)5205 --- PASS: TestGlobMatch/*bar_foo (0.00s)5206 --- PASS: TestGlobMatch/*bar_foobar (0.00s)5207 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)5208 --- PASS: TestGlobMatch/fo?_fo (0.00s)5209 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)5210 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)5211 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)5212 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)52132026/11/12 00:33:18 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC12352142026/11/12 00:33:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:38723/oidc52152026/11/12 00:33:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:44639/oidc52162026/11/12 00:33:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:39437/oidc5217--- PASS: TestValidateToken_WrongAudience (0.04s)5218--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (0.04s)5219--- PASS: TestValidateToken_BoundClaimsMismatch (0.05s)5220--- PASS: TestValidateToken_Expired (0.05s)52212026/11/12 00:33:18 http: TLS handshake error from 127.0.0.1:37996: remote error: tls: bad certificate5222--- PASS: TestNewValidator_KubernetesRequiresCA (0.04s)5223--- PASS: TestValidateToken_UnknownIssuerFetchesNoKeys (0.06s)52242026/11/12 00:33:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:38965/oidc5225--- PASS: TestScopes_Rules (0.06s)52262026/11/12 00:33:18 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:44503/oidc52272026/11/12 00:33:18 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:36829/oidc52282026/11/12 00:33:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:40537/oidc5229--- PASS: TestValidateToken_MultipleProviders (0.07s)5230--- PASS: TestValidateToken_BoundSubjectMismatch (0.07s)52312026/11/12 00:33:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:34991/oidc5232--- PASS: TestValidateToken_ValidToken (0.07s)52332026/11/12 00:33:18 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:3446552342026/11/12 00:33:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:44281/oidc5235--- PASS: TestPins_TopLevelShorthand (0.08s)5236--- PASS: TestValidateToken_KubernetesServiceAccount (0.08s)52372026/11/12 00:33:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:43849/oidc5238--- PASS: TestPins_ReservedForMatchingRule (0.11s)52392026/11/12 00:33:18 INFO OIDC provider initialized name=google issuer=https://accounts.google.com5240--- PASS: TestValidateToken_GoogleSchemelessIssuer (0.14s)52412026/11/12 00:33:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:42745/oidc5242--- PASS: TestScopes_LegacyProviderDefaultsToWrite (0.17s)5243PASS5244Running hook tests...5245=== RUN TestSendPathsEmpty5246=== PAUSE TestSendPathsEmpty5247=== RUN TestQueueEnqueueAndFetch5248=== PAUSE TestQueueEnqueueAndFetch5249=== RUN TestQueueDeduplication5250=== PAUSE TestQueueDeduplication5251=== RUN TestQueueRemove5252=== PAUSE TestQueueRemove5253=== RUN TestQueueFetchBatchLimit5254=== PAUSE TestQueueFetchBatchLimit5255=== RUN TestQueueRetryMovesToBack5256=== PAUSE TestQueueRetryMovesToBack5257=== RUN TestQueueFetchRemoveLifecycle5258=== PAUSE TestQueueFetchRemoveLifecycle5259=== RUN TestQueueConcurrentWriters5260=== PAUSE TestQueueConcurrentWriters5261=== RUN TestQueueRemoveLargeClosure5262=== PAUSE TestQueueRemoveLargeClosure5263=== RUN TestServerClientIntegration5264--- PASS: TestServerClientIntegration (0.00s)5265=== RUN TestServerQueueError52662026/11/12 00:33:18 ERROR Failed to queue paths error="permission denied" count=15267--- PASS: TestServerQueueError (0.00s)5268=== RUN TestGetListenerSocketActivation5269 server_test.go:220: === RUN TestGetListenerSocketActivation5270 --- PASS: TestGetListenerSocketActivation (0.00s)5271 PASS5272 5273--- PASS: TestGetListenerSocketActivation (0.01s)5274=== RUN TestServerStalledClientDoesNotBlockShutdown5275=== PAUSE TestServerStalledClientDoesNotBlockShutdown5276=== RUN TestServerBacksOffOnAcceptErrors5277=== PAUSE TestServerBacksOffOnAcceptErrors5278=== RUN TestDrainIsolatesPoisonPath5279=== PAUSE TestDrainIsolatesPoisonPath5280=== RUN TestRunNotBlockedByPoisonHead5281=== PAUSE TestRunNotBlockedByPoisonHead5282=== RUN TestDrainGivesUpWhenServerDown5283=== PAUSE TestDrainGivesUpWhenServerDown5284=== RUN TestFailedPathPrunedByLaterClosure5285=== PAUSE TestFailedPathPrunedByLaterClosure5286=== RUN TestWorkerUploadsAndRemoves5287=== PAUSE TestWorkerUploadsAndRemoves5288=== RUN TestWorkerSkipsGCdPaths5289=== PAUSE TestWorkerSkipsGCdPaths5290=== RUN TestWorkerPrunesClosureDeps5291=== PAUSE TestWorkerPrunesClosureDeps5292=== RUN TestDrainTimeout5293=== PAUSE TestDrainTimeout5294=== RUN TestWorkerRemoveFailureIsNotProgress5295=== PAUSE TestWorkerRemoveFailureIsNotProgress5296=== RUN TestWorkerKeepsPathItCannotStat5297=== PAUSE TestWorkerKeepsPathItCannotStat5298=== CONT TestSendPathsEmpty5299=== CONT TestWorkerPrunesClosureDeps5300--- PASS: TestSendPathsEmpty (0.00s)5301=== CONT TestWorkerKeepsPathItCannotStat5302=== CONT TestWorkerRemoveFailureIsNotProgress5303=== CONT TestDrainTimeout5304=== CONT TestDrainIsolatesPoisonPath5305=== CONT TestQueueRetryMovesToBack5306=== CONT TestQueueFetchRemoveLifecycle5307=== CONT TestQueueFetchBatchLimit5308=== CONT TestQueueRemove5309=== CONT TestServerBacksOffOnAcceptErrors5310=== CONT TestQueueDeduplication53112000/01/01 00:00:00 ERROR Accept failed error="too many open files"53122000/01/01 00:00:00 ERROR Accept failed error="too many open files"53132000/01/01 00:00:00 ERROR Accept failed error="too many open files"53142000/01/01 00:00:00 ERROR Accept failed error="too many open files"53152000/01/01 00:00:00 ERROR Accept failed error="too many open files"53162000/01/01 00:00:00 ERROR Accept failed error="too many open files"53172000/01/01 00:00:00 ERROR Accept failed error="too many open files"53182000/01/01 00:00:00 ERROR Accept failed error="too many open files"5319--- PASS: TestServerBacksOffOnAcceptErrors (0.00s)5320=== CONT TestServerStalledClientDoesNotBlockShutdown5321=== CONT TestQueueRemoveLargeClosure5322=== CONT TestQueueEnqueueAndFetch5323=== CONT TestQueueConcurrentWriters5324=== CONT TestFailedPathPrunedByLaterClosure5325=== CONT TestWorkerUploadsAndRemoves5326=== CONT TestWorkerSkipsGCdPaths5327=== CONT TestRunNotBlockedByPoisonHead5328=== CONT TestDrainGivesUpWhenServerDown53292026/11/12 00:33:18 INFO Upload queue status pending=253302026/11/12 00:33:18 INFO Upload queue status pending=253312026/11/12 00:33:18 INFO Upload queue status pending=353322026/11/12 00:33:18 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerSkipsGCdPaths1733707524/002/nonexistent53332026/11/12 00:33:18 INFO Uploading batch count=253342026/11/12 00:33:18 ERROR Upload failed error="upload failed" count=253352026/11/12 00:33:18 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown4062691165/002/a53362026/11/12 00:33:18 INFO Uploading batch count=153372026/11/12 00:33:18 ERROR Upload failed error="upload failed" count=153382026/11/12 00:33:18 INFO Uploading batch count=153392026/11/12 00:33:18 INFO Uploading batch count=153402026/11/12 00:33:18 ERROR Upload failed error="upload failed" count=153412026/11/12 00:33:18 INFO Uploading batch count=453422026/11/12 00:33:18 ERROR Upload failed error="upload failed" count=453432026/11/12 00:33:18 INFO Uploading batch count=15344--- PASS: TestQueueEnqueueAndFetch (0.01s)53452026/11/12 00:33:18 INFO Uploading batch count=153462026/11/12 00:33:18 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainIsolatesPoisonPath4217601020/002/bbb53472026/11/12 00:33:18 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown4062691165/002/b53482026/11/12 00:33:18 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat735737890/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat735737890/002/locked/aaa: permission denied"53492026/11/12 00:33:18 INFO Upload queue status pending=25350--- PASS: TestQueueFetchBatchLimit (0.02s)53512026/11/12 00:33:18 INFO Uploading batch count=253522000/01/01 00:00:00 INFO Uploading batch count=25353--- PASS: TestQueueFetchRemoveLifecycle (0.02s)53542000/01/01 00:00:00 ERROR Upload failed error="context deadline exceeded" count=253552000/01/01 00:00:00 ERROR Drain finished with paths left in queue remaining=453562026/11/12 00:33:18 INFO Uploading batch count=253572026/11/12 00:33:18 ERROR Upload failed error="upload failed" count=253582026/11/12 00:33:18 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown4062691165/002/c53592026/11/12 00:33:18 INFO Uploading batch count=15360--- PASS: TestQueueDeduplication (0.02s)53612026/11/12 00:33:18 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat735737890/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat735737890/002/locked/aaa: permission denied"5362--- PASS: TestQueueRetryMovesToBack (0.02s)53632026/11/12 00:33:18 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown4062691165/002/d53642026/11/12 00:33:18 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat735737890/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat735737890/002/locked/aaa: permission denied"5365--- PASS: TestQueueRemove (0.02s)53662026/11/12 00:33:18 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress1160068844/002/nonexistent53672026/11/12 00:33:18 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=153682026/11/12 00:33:18 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress1160068844/002/nonexistent53692026/11/12 00:33:18 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=153702026/11/12 00:33:18 INFO Uploading batch count=153712026/11/12 00:33:18 ERROR Drain finished with paths left in queue remaining=153722026/11/12 00:33:18 ERROR Upload failed error="upload failed" count=153732026/11/12 00:33:18 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress1160068844/002/nonexistent53742026/11/12 00:33:18 INFO Uploading batch count=253752026/11/12 00:33:18 ERROR Upload failed error="upload failed" count=253762026/11/12 00:33:18 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown4062691165/002/e53772026/11/12 00:33:18 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=15378--- PASS: TestDrainTimeout (0.02s)53792026/11/12 00:33:18 ERROR Drain finished with paths left in queue remaining=153802026/11/12 00:33:18 INFO Uploading batch count=153812026/11/12 00:33:18 ERROR Upload failed error="upload failed" count=153822026/11/12 00:33:18 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown4062691165/002/f5383--- PASS: TestWorkerRemoveFailureIsNotProgress (0.02s)5384--- PASS: TestWorkerKeepsPathItCannotStat (0.02s)53852026/11/12 00:33:18 INFO Uploading batch count=153862026/11/12 00:33:18 ERROR Upload failed error="upload failed" count=15387--- PASS: TestFailedPathPrunedByLaterClosure (0.02s)53882026/11/12 00:33:18 ERROR Drain finished with paths left in queue remaining=1053892026/11/12 00:33:18 ERROR Drain finished with paths left in queue remaining=15390--- PASS: TestDrainGivesUpWhenServerDown (0.02s)5391--- PASS: TestDrainIsolatesPoisonPath (0.03s)5392--- PASS: TestWorkerSkipsGCdPaths (0.03s)5393--- PASS: TestWorkerPrunesClosureDeps (0.04s)5394--- PASS: TestWorkerUploadsAndRemoves (0.04s)5395--- PASS: TestQueueRemoveLargeClosure (0.06s)53962026/11/12 00:33:19 ERROR Failed to decode request error="read unix /build/hook2302813077/test.sock->@: i/o timeout"53972026/11/12 00:33:19 ERROR Failed to write response error="write unix /build/hook2302813077/test.sock->@: i/o timeout"5398--- PASS: TestServerStalledClientDoesNotBlockShutdown (0.20s)5399--- PASS: TestQueueConcurrentWriters (0.21s)54002026/11/12 00:33:19 INFO Uploading batch count=154012026/11/12 00:33:19 INFO Uploading batch count=154022026/11/12 00:33:19 INFO Uploading batch count=154032026/11/12 00:33:19 ERROR Upload failed error="upload failed" count=154042026/11/12 00:33:19 INFO Uploading batch count=154052026/11/12 00:33:19 ERROR Upload failed error="upload failed" count=154062026/11/12 00:33:19 INFO Uploading batch count=154072026/11/12 00:33:19 ERROR Upload failed error="upload failed" count=154082026/11/12 00:33:19 INFO Uploading batch count=154092026/11/12 00:33:19 ERROR Upload failed error="upload failed" count=154102026/11/12 00:33:19 ERROR Drain finished with paths left in queue remaining=15411--- PASS: TestRunNotBlockedByPoisonHead (1.03s)5412PASS5413Running signing tests...5414=== RUN TestExternalSigner5415=== PAUSE TestExternalSigner5416=== RUN TestExternalSignerWrongPublicKey5417=== PAUSE TestExternalSignerWrongPublicKey5418=== RUN TestExternalSignerErrors5419=== PAUSE TestExternalSignerErrors5420=== RUN TestExternalSignerRestartsAfterCrash5421=== PAUSE TestExternalSignerRestartsAfterCrash5422=== RUN TestExternalSignerProgramErrorKeepsProcess5423=== PAUSE TestExternalSignerProgramErrorKeepsProcess5424=== RUN TestExternalSignerClose5425=== PAUSE TestExternalSignerClose5426=== RUN TestExternalSignerRejectsInvalidPublicKey5427=== PAUSE TestExternalSignerRejectsInvalidPublicKey5428=== RUN TestExternalSignerCancellation5429=== PAUSE TestExternalSignerCancellation5430=== RUN TestGenerateFingerprint5431=== PAUSE TestGenerateFingerprint5432=== RUN TestParseSigningKey5433=== PAUSE TestParseSigningKey5434=== RUN TestSign5435=== PAUSE TestSign5436=== RUN TestSignBatch5437=== PAUSE TestSignBatch5438=== RUN TestNilKeyPublicKey5439=== PAUSE TestNilKeyPublicKey5440=== RUN TestSignErrors5441=== PAUSE TestSignErrors5442=== CONT TestExternalSigner5443=== CONT TestExternalSignerCancellation5444=== CONT TestSignBatch5445=== CONT TestSignErrors5446=== CONT TestNilKeyPublicKey5447--- PASS: TestNilKeyPublicKey (0.00s)5448=== CONT TestSign5449=== CONT TestExternalSignerProgramErrorKeepsProcess5450=== CONT TestExternalSignerRejectsInvalidPublicKey5451=== RUN TestExternalSignerRejectsInvalidPublicKey/#005452=== PAUSE TestExternalSignerRejectsInvalidPublicKey/#005453=== CONT TestExternalSignerClose5454=== CONT TestExternalSignerErrors5455=== CONT TestExternalSignerRestartsAfterCrash5456=== CONT TestExternalSignerWrongPublicKey5457=== CONT TestGenerateFingerprint5458=== CONT TestParseSigningKey5459=== RUN TestParseSigningKey/valid_32-byte_key5460=== PAUSE TestParseSigningKey/valid_32-byte_key5461=== RUN TestParseSigningKey/valid_32-byte_key_with_different_name5462=== PAUSE TestParseSigningKey/valid_32-byte_key_with_different_name5463=== RUN TestParseSigningKey/no_colon5464=== PAUSE TestParseSigningKey/no_colon5465=== RUN TestParseSigningKey/empty_name5466=== PAUSE TestParseSigningKey/empty_name5467=== RUN TestParseSigningKey/invalid_base645468=== PAUSE TestParseSigningKey/invalid_base645469=== RUN TestParseSigningKey/wrong_length5470=== PAUSE TestParseSigningKey/wrong_length5471=== CONT TestParseSigningKey/valid_32-byte_key5472=== CONT TestParseSigningKey/wrong_length5473=== CONT TestParseSigningKey/invalid_base645474=== RUN TestGenerateFingerprint/basic_with_references5475=== CONT TestParseSigningKey/empty_name5476=== CONT TestParseSigningKey/no_colon5477=== CONT TestParseSigningKey/valid_32-byte_key_with_different_name5478=== RUN TestExternalSignerErrors/wrong_id5479=== PAUSE TestExternalSignerErrors/wrong_id5480=== RUN TestExternalSignerRejectsInvalidPublicKey/no-colon5481=== RUN TestSignErrors/nil_key5482=== PAUSE TestSignErrors/nil_key5483=== RUN TestSignErrors/nil_narinfo5484=== PAUSE TestSignErrors/nil_narinfo5485=== RUN TestSignErrors/invalid_narinfo5486=== PAUSE TestSignErrors/invalid_narinfo5487=== RUN TestSignErrors/canceled5488=== PAUSE TestSignErrors/canceled5489=== CONT TestSignErrors/nil_key5490--- PASS: TestSign (0.00s)5491--- PASS: TestSignBatch (0.00s)5492--- PASS: TestParseSigningKey (0.00s)5493 --- PASS: TestParseSigningKey/valid_32-byte_key (0.00s)5494 --- PASS: TestParseSigningKey/wrong_length (0.00s)5495 --- PASS: TestParseSigningKey/invalid_base64 (0.00s)5496 --- PASS: TestParseSigningKey/empty_name (0.00s)5497 --- PASS: TestParseSigningKey/no_colon (0.00s)5498 --- PASS: TestParseSigningKey/valid_32-byte_key_with_different_name (0.00s)5499=== CONT TestSignErrors/canceled5500=== PAUSE TestGenerateFingerprint/basic_with_references5501=== RUN TestGenerateFingerprint/no_references5502=== PAUSE TestGenerateFingerprint/no_references5503=== RUN TestExternalSignerErrors/wrong_signature_name5504=== PAUSE TestExternalSignerErrors/wrong_signature_name5505=== PAUSE TestExternalSignerRejectsInvalidPublicKey/no-colon5506=== CONT TestSignErrors/invalid_narinfo5507=== CONT TestSignErrors/nil_narinfo5508--- PASS: TestSignErrors (0.00s)5509 --- PASS: TestSignErrors/nil_key (0.00s)5510 --- PASS: TestSignErrors/canceled (0.00s)5511 --- PASS: TestSignErrors/invalid_narinfo (0.00s)5512 --- PASS: TestSignErrors/nil_narinfo (0.00s)5513=== RUN TestGenerateFingerprint/unsorted_references_get_sorted5514=== PAUSE TestGenerateFingerprint/unsorted_references_get_sorted5515=== RUN TestGenerateFingerprint/invalid_nar_hash_prefix5516=== PAUSE TestGenerateFingerprint/invalid_nar_hash_prefix5517=== RUN TestGenerateFingerprint/invalid_nar_hash_length5518=== RUN TestExternalSignerErrors/exit_on_request5519=== RUN TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5520=== PAUSE TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5521=== PAUSE TestGenerateFingerprint/invalid_nar_hash_length5522=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5523=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5524=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5525=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5526=== CONT TestExternalSignerRejectsInvalidPublicKey/#005527=== PAUSE TestExternalSignerErrors/exit_on_request5528=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5529=== RUN TestGenerateFingerprint/invalid_store_path_prefix5530=== PAUSE TestGenerateFingerprint/invalid_store_path_prefix5531=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5532=== CONT TestExternalSignerRejectsInvalidPublicKey/no-colon5533=== CONT TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5534--- PASS: TestExternalSignerRejectsInvalidPublicKey (0.00s)5535 --- PASS: TestExternalSignerRejectsInvalidPublicKey/#00 (0.00s)5536 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:invalid! (0.00s)5537 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:YQ== (0.00s)5538 --- PASS: TestExternalSignerRejectsInvalidPublicKey/no-colon (0.00s)5539 --- PASS: TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= (0.00s)5540=== RUN TestExternalSignerErrors/wrong_shape5541=== PAUSE TestExternalSignerErrors/wrong_shape5542=== RUN TestExternalSignerErrors/null_signatures5543=== PAUSE TestExternalSignerErrors/null_signatures5544=== RUN TestExternalSignerErrors/wrong_signatures_shape5545=== PAUSE TestExternalSignerErrors/wrong_signatures_shape5546=== RUN TestGenerateFingerprint/invalid_reference_prefix5547=== PAUSE TestGenerateFingerprint/invalid_reference_prefix5548=== CONT TestGenerateFingerprint/basic_with_references5549=== RUN TestExternalSignerErrors/too_few_signatures5550=== PAUSE TestExternalSignerErrors/too_few_signatures5551=== CONT TestGenerateFingerprint/invalid_nar_hash_prefix5552=== CONT TestGenerateFingerprint/invalid_nar_hash_length5553=== CONT TestGenerateFingerprint/no_references5554=== CONT TestGenerateFingerprint/invalid_reference_prefix5555=== CONT TestGenerateFingerprint/invalid_store_path_prefix5556=== CONT TestGenerateFingerprint/unsorted_references_get_sorted5557--- PASS: TestGenerateFingerprint (0.00s)5558 --- PASS: TestGenerateFingerprint/basic_with_references (0.00s)5559 --- PASS: TestGenerateFingerprint/invalid_nar_hash_prefix (0.00s)5560 --- PASS: TestGenerateFingerprint/invalid_nar_hash_length (0.00s)5561 --- PASS: TestGenerateFingerprint/no_references (0.00s)5562 --- PASS: TestGenerateFingerprint/invalid_reference_prefix (0.00s)5563 --- PASS: TestGenerateFingerprint/invalid_store_path_prefix (0.00s)5564 --- PASS: TestGenerateFingerprint/unsorted_references_get_sorted (0.00s)5565=== RUN TestExternalSignerErrors/unrequested_key5566=== PAUSE TestExternalSignerErrors/unrequested_key5567=== RUN TestExternalSignerErrors/program_error5568=== PAUSE TestExternalSignerErrors/program_error5569=== RUN TestExternalSignerErrors/invalid_JSON5570=== PAUSE TestExternalSignerErrors/invalid_JSON5571=== RUN TestExternalSignerErrors/null5572=== PAUSE TestExternalSignerErrors/null5573=== RUN TestExternalSignerErrors/missing_signature_name5574=== PAUSE TestExternalSignerErrors/missing_signature_name5575=== RUN TestExternalSignerErrors/invalid_signature_base645576=== PAUSE TestExternalSignerErrors/invalid_signature_base645577=== RUN TestExternalSignerErrors/exit_at_startup5578--- PASS: TestExternalSignerClose (0.00s)5579=== PAUSE TestExternalSignerErrors/exit_at_startup5580=== RUN TestExternalSignerErrors/missing_signatures5581=== PAUSE TestExternalSignerErrors/missing_signatures5582=== RUN TestExternalSignerErrors/wrong_signature_length5583=== PAUSE TestExternalSignerErrors/wrong_signature_length5584=== CONT TestExternalSignerErrors/wrong_id5585=== CONT TestExternalSignerErrors/too_few_signatures5586--- PASS: TestExternalSignerProgramErrorKeepsProcess (0.00s)5587--- PASS: TestExternalSignerWrongPublicKey (0.00s)5588=== CONT TestExternalSignerErrors/wrong_shape5589=== CONT TestExternalSignerErrors/missing_signature_name5590=== CONT TestExternalSignerErrors/wrong_signature_length5591=== CONT TestExternalSignerErrors/missing_signatures5592=== CONT TestExternalSignerErrors/exit_at_startup5593=== CONT TestExternalSignerErrors/invalid_signature_base645594=== CONT TestExternalSignerErrors/wrong_signatures_shape5595=== CONT TestExternalSignerErrors/null_signatures5596=== CONT TestExternalSignerErrors/invalid_JSON5597=== CONT TestExternalSignerErrors/null5598=== CONT TestExternalSignerErrors/exit_on_request5599=== CONT TestExternalSignerErrors/wrong_signature_name5600=== CONT TestExternalSignerErrors/program_error5601=== CONT TestExternalSignerErrors/unrequested_key5602--- PASS: TestExternalSigner (0.01s)5603--- PASS: TestExternalSignerErrors (0.00s)5604 --- PASS: TestExternalSignerErrors/wrong_id (0.00s)5605 --- PASS: TestExternalSignerErrors/wrong_signature_length (0.00s)5606 --- PASS: TestExternalSignerErrors/too_few_signatures (0.00s)5607 --- PASS: TestExternalSignerErrors/missing_signature_name (0.00s)5608 --- PASS: TestExternalSignerErrors/wrong_signatures_shape (0.01s)5609 --- PASS: TestExternalSignerErrors/exit_at_startup (0.01s)5610 --- PASS: TestExternalSignerErrors/invalid_JSON (0.01s)5611 --- PASS: TestExternalSignerErrors/invalid_signature_base64 (0.01s)5612 --- PASS: TestExternalSignerErrors/wrong_shape (0.01s)5613 --- PASS: TestExternalSignerErrors/wrong_signature_name (0.01s)5614 --- PASS: TestExternalSignerErrors/missing_signatures (0.01s)5615 --- PASS: TestExternalSignerErrors/exit_on_request (0.01s)5616 --- PASS: TestExternalSignerErrors/null_signatures (0.01s)5617 --- PASS: TestExternalSignerErrors/program_error (0.01s)5618 --- PASS: TestExternalSignerErrors/unrequested_key (0.01s)5619 --- PASS: TestExternalSignerErrors/null (0.01s)5620--- PASS: TestExternalSignerRestartsAfterCrash (0.01s)5621--- PASS: TestExternalSignerCancellation (0.02s)5622PASS5623Running OpenBao signer tests...5624=== RUN TestSign5625=== PAUSE TestSign5626=== RUN TestSignErrors5627=== PAUSE TestSignErrors5628=== RUN TestServe5629=== PAUSE TestServe5630=== RUN TestServeRejectsGarbage5631=== PAUSE TestServeRejectsGarbage5632=== CONT TestSign5633=== CONT TestServe5634=== CONT TestSignErrors5635=== RUN TestSignErrors/no_prefix5636=== PAUSE TestSignErrors/no_prefix5637=== CONT TestServeRejectsGarbage5638=== RUN TestSignErrors/invalid_base645639=== PAUSE TestSignErrors/invalid_base645640--- PASS: TestServe (0.00s)5641=== RUN TestSignErrors/http_error5642=== PAUSE TestSignErrors/http_error5643=== RUN TestSignErrors/invalid_json5644=== PAUSE TestSignErrors/invalid_json5645=== CONT TestSignErrors/no_prefix5646=== CONT TestSignErrors/invalid_base645647--- PASS: TestServeRejectsGarbage (0.00s)5648=== CONT TestSignErrors/invalid_json5649=== CONT TestSignErrors/http_error5650--- PASS: TestSign (0.00s)5651--- PASS: TestSignErrors (0.00s)5652 --- PASS: TestSignErrors/invalid_base64 (0.00s)5653 --- PASS: TestSignErrors/no_prefix (0.00s)5654 --- PASS: TestSignErrors/http_error (0.00s)5655 --- PASS: TestSignErrors/invalid_json (0.00s)5656PASS