niks3-go-unit-tests
checks.aarch64-darwin.go-unit-tests
· build #499
· raw
1tribuchet: building on mac022Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestUploadBuildLog_FileBodyReplayedOnRetry6=== PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry7=== RUN TestRegisterUploadedObjectReusesConnections8=== PAUSE TestRegisterUploadedObjectReusesConnections9=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica10=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica11=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun12=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun13=== RUN TestCaseHackSuffix14=== PAUSE TestCaseHackSuffix15=== RUN TestFilterOversizedClosures16=== PAUSE TestFilterOversizedClosures17=== RUN TestUploadMultipart_PartsInParallel18=== PAUSE TestUploadMultipart_PartsInParallel19=== RUN TestUploadMultipart_ReturnsAfterDroppedBody20=== PAUSE TestUploadMultipart_ReturnsAfterDroppedBody21=== RUN TestPartSizeForNAR22=== PAUSE TestPartSizeForNAR23=== RUN TestDumpPathCaseHackMatchesNix24--- PASS: TestDumpPathCaseHackMatchesNix (0.10s)25=== RUN TestDumpPathCaseHackCollision26--- PASS: TestDumpPathCaseHackCollision (0.00s)27=== RUN TestDumpPathMatchesNix28=== PAUSE TestDumpPathMatchesNix29=== RUN TestDumpPathSingleFile30=== PAUSE TestDumpPathSingleFile31=== RUN TestDumpPathWriterError32=== PAUSE TestDumpPathWriterError33=== RUN TestEncodeNixBase3234=== PAUSE TestEncodeNixBase3235=== RUN TestEncodeNixBase32WithRealHash36=== PAUSE TestEncodeNixBase32WithRealHash37=== RUN TestConvertHashToNix3238=== PAUSE TestConvertHashToNix3239=== RUN TestGetStorePathHash40=== PAUSE TestGetStorePathHash41=== RUN TestPathInfoHashCompatibility42=== PAUSE TestPathInfoHashCompatibility43=== RUN TestParsePathInfoJSON44=== PAUSE TestParsePathInfoJSON45=== RUN TestParsePathInfoJSONMultiplePaths46=== PAUSE TestParsePathInfoJSONMultiplePaths47=== RUN TestPathInfoCACompatibility48=== PAUSE TestPathInfoCACompatibility49=== RUN TestRateLimiterFeedback50=== PAUSE TestRateLimiterFeedback51=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess52=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess53=== RUN TestRegisterUploadedObject_BoundedAgainstSilentServer54=== PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer55=== RUN TestRegisterUploadedObject_OutlivesPush56=== PAUSE TestRegisterUploadedObject_OutlivesPush57=== RUN TestResolveStorePath58=== PAUSE TestResolveStorePath59=== RUN TestDoWithRetry_BodyReplayedViaGetBody60=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody61=== RUN TestRetryTimeoutOutlastsMaxRetries62=== PAUSE TestRetryTimeoutOutlastsMaxRetries63=== RUN TestDoWithRetry_FinalResponseBodyReadable64=== PAUSE TestDoWithRetry_FinalResponseBodyReadable65=== RUN TestShellSplit66=== PAUSE TestShellSplit67=== RUN TestShellSplitErrors68=== PAUSE TestShellSplitErrors69=== RUN TestStreamPushReportsEveryPath70=== PAUSE TestStreamPushReportsEveryPath71=== RUN TestStreamPushBatchesUnderLoad72=== PAUSE TestStreamPushBatchesUnderLoad73=== RUN TestStreamPushIsolatesFailures74=== PAUSE TestStreamPushIsolatesFailures75=== RUN TestStreamPushGivesUpOnDeadServer76=== PAUSE TestStreamPushGivesUpOnDeadServer77=== RUN TestStreamPushRequestLine78=== PAUSE TestStreamPushRequestLine79=== RUN TestStreamPushReportsSignatures80=== PAUSE TestStreamPushReportsSignatures81=== RUN TestStreamPushReportsSkippedPaths82=== PAUSE TestStreamPushReportsSkippedPaths83=== RUN TestClientSignaturesByStorePath84=== PAUSE TestClientSignaturesByStorePath85=== RUN TestStreamPushAnswersEveryLineWhenCanceled86=== PAUSE TestStreamPushAnswersEveryLineWhenCanceled87=== RUN TestStreamPushStopsWhenCanceled88=== PAUSE TestStreamPushStopsWhenCanceled89=== RUN TestSetClientTLS90=== PAUSE TestSetClientTLS91=== RUN TestSetClientTLSDoesNotMutateDefaultTransport92=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport93=== RUN TestSetClientTLSErrors94=== PAUSE TestSetClientTLSErrors95=== RUN TestStaticToken96=== PAUSE TestStaticToken97=== RUN TestFileTokenReadsAndCaches98=== PAUSE TestFileTokenReadsAndCaches99=== RUN TestFileTokenMissing100=== PAUSE TestFileTokenMissing101=== RUN TestFileTokenEmpty102=== PAUSE TestFileTokenEmpty103=== RUN TestScriptTokenNoExpiryRerunsEveryCall104=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall105=== RUN TestScriptTokenCachesUntilRefresh106=== PAUSE TestScriptTokenCachesUntilRefresh107=== RUN TestScriptTokenEmptyToken108=== PAUSE TestScriptTokenEmptyToken109=== RUN TestScriptTokenBadJSON110=== PAUSE TestScriptTokenBadJSON111=== RUN TestScriptTokenScriptFails112=== PAUSE TestScriptTokenScriptFails113=== RUN TestScriptTokenEmptyCommand114=== PAUSE TestScriptTokenEmptyCommand115=== CONT TestDoServerRequestAttachesToken116=== CONT TestFileTokenEmpty117=== CONT TestRegisterUploadedObject_OutlivesPush118=== CONT TestScriptTokenEmptyCommand119--- PASS: TestScriptTokenEmptyCommand (0.00s)120=== CONT TestPathInfoCACompatibility121=== RUN TestPathInfoCACompatibility/null_ca_field122=== PAUSE TestPathInfoCACompatibility/null_ca_field123=== RUN TestPathInfoCACompatibility/old_string_format_-_text124=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text125=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive126=== CONT TestScriptTokenNoExpiryRerunsEveryCall127=== CONT TestScriptTokenScriptFails128--- PASS: TestFileTokenEmpty (0.00s)129=== CONT TestParsePathInfoJSONMultiplePaths130=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths131=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths132=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths133=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths134=== CONT TestParsePathInfoJSON135=== RUN TestParsePathInfoJSON/Nix_format136=== PAUSE TestParsePathInfoJSON/Nix_format137=== RUN TestParsePathInfoJSON/Lix_format138=== PAUSE TestParsePathInfoJSON/Lix_format139=== RUN TestParsePathInfoJSON/empty_input140=== PAUSE TestParsePathInfoJSON/empty_input141=== CONT TestScriptTokenBadJSON142=== CONT TestScriptTokenEmptyToken143=== CONT TestScriptTokenCachesUntilRefresh144=== CONT TestStreamPushReportsSignatures145=== CONT TestFileTokenMissing146=== CONT TestFileTokenReadsAndCaches1472026/11/12 00:27:34 ERROR Upload failed error=boom count=1148=== CONT TestStaticToken149=== CONT TestGetStorePathHash150=== RUN TestGetStorePathHash/valid_store_path151=== CONT TestSetClientTLSErrors152=== CONT TestSetClientTLSDoesNotMutateDefaultTransport153=== CONT TestSetClientTLS154=== CONT TestStreamPushStopsWhenCanceled155=== CONT TestStreamPushAnswersEveryLineWhenCanceled156=== CONT TestClientSignaturesByStorePath157=== CONT TestEncodeNixBase32WithRealHash158=== CONT TestEncodeNixBase32159=== RUN TestEncodeNixBase32/test_string_hash160=== PAUSE TestEncodeNixBase32/test_string_hash161=== RUN TestEncodeNixBase32/empty_input162=== CONT TestStreamPushReportsSkippedPaths163=== CONT TestDumpPathWriterError164=== CONT TestFilterOversizedClosures165=== RUN TestFilterOversizedClosures/no_limit_keeps_everything166=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything167=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped168=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped169=== RUN TestFilterOversizedClosures/all_closures_skipped170=== PAUSE TestFilterOversizedClosures/all_closures_skipped171=== CONT TestDumpPathSingleFile172=== CONT TestDumpPathMatchesNix1732000/01/01 00:00:00 ERROR Upload failed error="context canceled" count=2174=== CONT TestRegisterUploadedObject_BoundedAgainstSilentServer175=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess1762026/11/12 00:27:34 WARN Rate limiter enabled after throttle name=server-test rate=5177=== CONT TestRateLimiterFeedback178=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive179=== RUN TestParsePathInfoJSON/whitespace_only180=== CONT TestUploadMultipart_ReturnsAfterDroppedBody181--- PASS: TestStreamPushReportsSignatures (0.00s)182=== PAUSE TestParsePathInfoJSON/whitespace_only183=== RUN TestParsePathInfoJSON/invalid_JSON184--- PASS: TestStaticToken (0.00s)185--- PASS: TestStreamPushStopsWhenCanceled (0.00s)186--- PASS: TestClientSignaturesByStorePath (0.00s)187--- PASS: TestEncodeNixBase32WithRealHash (0.00s)188--- PASS: TestStreamPushReportsSkippedPaths (0.00s)189--- PASS: TestFileTokenMissing (0.00s)190--- PASS: TestFileTokenReadsAndCaches (0.00s)191--- PASS: TestScriptTokenScriptFails (0.01s)192=== PAUSE TestParsePathInfoJSON/invalid_JSON193=== PAUSE TestGetStorePathHash/valid_store_path194=== RUN TestGetStorePathHash/basename_without_hyphen_should_error195=== CONT TestUploadMultipart_PartsInParallel196=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error197=== PAUSE TestEncodeNixBase32/empty_input198=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error199=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error200=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error201=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error202=== RUN TestSetClientTLSErrors/missing_cert_file203=== PAUSE TestSetClientTLSErrors/missing_cert_file204=== RUN TestSetClientTLSErrors/missing_key_file205=== PAUSE TestSetClientTLSErrors/missing_key_file206=== RUN TestSetClientTLSErrors/missing_ca_file207=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica208=== RUN TestRateLimiterFeedback/429_enables_limiter209=== PAUSE TestRateLimiterFeedback/429_enables_limiter210=== RUN TestRateLimiterFeedback/503_enables_limiter211=== PAUSE TestRateLimiterFeedback/503_enables_limiter212=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter213=== CONT TestCaseHackSuffix214=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter215=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter216=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter217=== RUN TestPathInfoCACompatibility/new_structured_format_-_text218=== CONT TestRegisterUploadedObjectReusesConnections219=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text220=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method221=== PAUSE TestSetClientTLSErrors/missing_ca_file222=== RUN TestSetClientTLSErrors/invalid_ca_file223=== PAUSE TestSetClientTLSErrors/invalid_ca_file224=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method225=== RUN TestSetClientTLS/rejects_connection_without_client_cert226=== CONT TestStreamPushRequestLine227=== CONT TestShellSplitErrors228=== CONT TestStreamPushGivesUpOnDeadServer229=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert230=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA231=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA232=== RUN TestSetClientTLS/preserves_debug_logging_transport233=== PAUSE TestSetClientTLS/preserves_debug_logging_transport234--- PASS: TestStreamPushAnswersEveryLineWhenCanceled (0.01s)235--- PASS: TestShellSplitErrors (0.00s)236=== CONT TestStreamPushIsolatesFailures2372026/11/12 00:27:34 ERROR Upload failed error="connection refused" count=202382026/11/12 00:27:34 ERROR Server seems unavailable, giving up on batch untried=17239--- PASS: TestDoServerRequestAttachesToken (0.01s)240=== CONT TestStreamPushBatchesUnderLoad241--- PASS: TestStreamPushGivesUpOnDeadServer (0.00s)242=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=false243=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=false244=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=true245=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=true2462026/11/12 00:27:34 ERROR Upload failed error=boom count=1247=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun248=== CONT TestStreamPushReportsEveryPath249=== CONT TestPathInfoHashCompatibility250=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)251=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)252=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon253=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon254=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI255=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI256=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512257=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512258=== CONT TestUploadBuildLog_FileBodyReplayedOnRetry259=== CONT TestConvertHashToNix32260=== RUN TestConvertHashToNix32/SRI_format_to_Nix32261=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32262=== RUN TestConvertHashToNix32/already_Nix32_format263=== PAUSE TestConvertHashToNix32/already_Nix32_format264=== RUN TestConvertHashToNix32/invalid_format265=== PAUSE TestConvertHashToNix32/invalid_format266=== CONT TestShellSplit267--- PASS: TestShellSplit (0.00s)268=== CONT TestDoWithRetry_FinalResponseBodyReadable269=== CONT TestRetryTimeoutOutlastsMaxRetries270--- PASS: TestStreamPushReportsEveryPath (0.00s)271=== CONT TestDoWithRetry_BodyReplayedViaGetBody272--- PASS: TestRegisterUploadedObject_OutlivesPush (0.01s)273=== CONT TestResolveStorePath274--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.01s)275=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths2762026/11/12 00:27:34 INFO Garbage collection started277=== CONT TestPartSizeForNAR278=== RUN TestPartSizeForNAR/zero_stays_at_minimum279=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum280=== RUN TestPartSizeForNAR/small_stays_at_minimum281=== PAUSE TestPartSizeForNAR/small_stays_at_minimum282=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum283=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum284=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts285=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts286=== RUN TestPartSizeForNAR/1_TiB287=== PAUSE TestPartSizeForNAR/1_TiB288=== RUN TestPartSizeForNAR/5_TiB_S3_max_object289=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object290=== RUN TestPartSizeForNAR/capped_at_5_GiB291=== PAUSE TestPartSizeForNAR/capped_at_5_GiB292--- PASS: TestScriptTokenEmptyToken (0.01s)293=== CONT TestFilterOversizedClosures/no_limit_keeps_everything2942026/11/12 00:27:34 ERROR Upload failed error="bad path" count=3295=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped2962026/11/12 00:27:34 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000297--- PASS: TestStreamPushIsolatesFailures (0.00s)298=== CONT TestParsePathInfoJSON/Nix_format299=== CONT TestParsePathInfoJSON/whitespace_only300=== CONT TestParsePathInfoJSON/Lix_format301=== CONT TestParsePathInfoJSON/empty_input302=== CONT TestParsePathInfoJSON/invalid_JSON303=== CONT TestGetStorePathHash/valid_store_path304=== CONT TestFilterOversizedClosures/all_closures_skipped3052026/11/12 00:27:34 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50306--- PASS: TestParsePathInfoJSON (0.01s)307 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)308 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)309 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)310 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)311 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)312=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error313=== CONT TestGetStorePathHash/basename_without_hyphen_should_error314=== CONT TestEncodeNixBase32/empty_input315=== CONT TestRateLimiterFeedback/429_enables_limiter316=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error317=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter318=== CONT TestEncodeNixBase32/test_string_hash319=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter320=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths321=== CONT TestRateLimiterFeedback/503_enables_limiter322--- PASS: TestFilterOversizedClosures (0.00s)323 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)324 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)325 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)326--- PASS: TestGetStorePathHash (0.00s)327 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)328 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)329 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)330 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)331--- PASS: TestEncodeNixBase32 (0.00s)332 --- PASS: TestEncodeNixBase32/empty_input (0.00s)333 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)334--- PASS: TestParsePathInfoJSONMultiplePaths (0.00s)335 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)336 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)337--- PASS: TestResolveStorePath (0.00s)338=== CONT TestSetClientTLSErrors/missing_cert_file3392026/11/12 00:27:34 WARN Rate limiter enabled after throttle name=server-test rate=53402026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:517973412026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:518083422026/11/12 00:27:34 INFO Garbage collection started3432026/11/12 00:27:34 WARN Rate limiter backed off name=server-test rate=5344--- PASS: TestDoWithRetry_FinalResponseBodyReadable (0.01s)345=== CONT TestPathInfoCACompatibility/null_ca_field346=== CONT TestSetClientTLSErrors/invalid_ca_file347=== CONT TestSetClientTLSErrors/missing_ca_file348=== CONT TestSetClientTLSErrors/missing_key_file349=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method3502026/11/12 00:27:34 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=0351=== CONT TestPathInfoCACompatibility/new_structured_format_-_text352=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive353=== CONT TestPathInfoCACompatibility/old_string_format_-_text354--- PASS: TestPathInfoCACompatibility (0.01s)355 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)356 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)357 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)358 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)359 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)360=== CONT TestSetClientTLS/rejects_connection_without_client_cert361=== CONT TestSetClientTLS/preserves_debug_logging_transport3622026/11/12 00:27:34 WARN Rate limiter enabled after throttle name=server-test rate=53632026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:518493642026/11/12 00:27:34 WARN Rate limiter enabled after throttle name=server-test rate=53652026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:518203662026/11/12 00:27:34 WARN Rate limiter backed off name=server-test rate=53672026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:518203682026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:518083692026/11/12 00:27:34 WARN Rate limiter enabled after throttle name=server-test rate=53702026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:51844371--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (0.01s)372=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA3732026/11/12 00:27:34 WARN Rate limiter backed off name=server-test rate=5374--- PASS: TestUploadMultipart_ReturnsAfterDroppedBody (0.02s)375=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=false376=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)377=== CONT TestConvertHashToNix32/SRI_format_to_Nix32378=== CONT TestConvertHashToNix32/invalid_format379=== CONT TestConvertHashToNix32/already_Nix32_format380--- PASS: TestConvertHashToNix32 (0.00s)381 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)382 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)383 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)384=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512385=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI386=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon387--- PASS: TestPathInfoHashCompatibility (0.00s)388 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)389 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)390 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)391 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)392=== CONT TestPartSizeForNAR/zero_stays_at_minimum393=== CONT TestPartSizeForNAR/1_TiB394--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.01s)395=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=true396=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum397=== CONT TestPartSizeForNAR/small_stays_at_minimum398=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts399=== CONT TestPartSizeForNAR/capped_at_5_GiB400=== CONT TestPartSizeForNAR/5_TiB_S3_max_object401--- PASS: TestPartSizeForNAR (0.00s)402 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)403 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)404 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)405 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)406 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)407 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)408 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)409--- PASS: TestStreamPushBatchesUnderLoad (0.00s)410 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=false (0.00s)411 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=true (0.00s)4122026/11/12 00:27:34 WARN Rate limiter backed off name=server-test rate=5413--- PASS: TestRateLimiterFeedback (0.01s)414 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.01s)415 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.01s)416 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.01s)417 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.01s)418--- PASS: TestSetClientTLSErrors (0.01s)419 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)420 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)421 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)422 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)4232026/11/12 00:27:34 WARN Rate limiter enabled after throttle name=s3-test rate=54242026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:51829425--- PASS: TestScriptTokenBadJSON (0.02s)426--- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (0.01s)4272026/11/12 00:27:34 WARN Garbage collection finished on another replica. Its statistics are not available here428--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (0.02s)4292026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:518084302026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:51808431--- PASS: TestStreamPushRequestLine (0.02s)4322026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:518084332026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:518084342026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:518084352026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:518084362026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:518084372026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:518084382026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:518084392026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:51808440--- PASS: TestRegisterUploadedObjectReusesConnections (0.03s)4412026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:518084422026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:518084432026/11/12 00:27:34 http: TLS handshake error from 127.0.0.1:51864: remote error: tls: bad certificate444--- PASS: TestSetClientTLS (0.01s)445 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.00s)446 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.00s)447 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.03s)4482026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:518084492026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:518084502026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:518084512026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:518084522026/11/12 00:27:34 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:51808453--- PASS: TestScriptTokenCachesUntilRefresh (0.06s)454--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.06s)455--- PASS: TestRetryTimeoutOutlastsMaxRetries (0.05s)456--- PASS: TestDumpPathWriterError (0.08s)457--- PASS: TestDumpPathSingleFile (0.09s)458--- PASS: TestCaseHackSuffix (0.09s)4592026/11/12 00:27:34 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:51766/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:51766/api/uploads/complete4602026/11/12 00:27:34 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4612026/11/12 00:27:34 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:51766/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:51766/api/uploads/complete4622026/11/12 00:27:34 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4632026/11/12 00:27:34 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:51766/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:51766/api/uploads/complete4642026/11/12 00:27:34 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"465--- PASS: TestRegisterUploadedObject_BoundedAgainstSilentServer (0.10s)466--- PASS: TestDumpPathMatchesNix (0.12s)467--- PASS: TestUploadMultipart_PartsInParallel (0.63s)468--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)469PASS470Running server tests...471The files belonging to this database system will be owned by user "_tribuchetbld3".472This user must also own the server process.473474The database cluster will be initialized with locale "C".475The default database encoding has accordingly been set to "SQL_ASCII".476The default text search configuration will be set to "english".477478Data page checksums are enabled.479480creating directory /private/var/tb/3/scratch/81a37589b1b6aa8c/build/postgres2107914087/data ... ok481creating subdirectories ... ok482selecting dynamic shared memory implementation ... posix483selecting default "max_connections" ... 100484selecting default "shared_buffers" ... 128MB485selecting default time zone ... UTC486creating configuration files ... ok487running bootstrap script ... ok488performing post-bootstrap initialization ... ok489syncing data to disk ... ok490491initdb: warning: enabling "trust" authentication for local connections492initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.493494Success. You can now start the database server using:495496 pg_ctl -D /private/var/tb/3/scratch/81a37589b1b6aa8c/build/postgres2107914087/data -l logfile start497498/private/var/tb/3/scratch/81a37589b1b6aa8c/build/postgres2107914087:5432 - no response4992026-11-12 00:27:37.016 UTC [88001] LOG: starting PostgreSQL 18.6 on aarch64-apple-darwin25.6.0, compiled by clang version 21.1.8, 64-bit5002026-11-12 00:27:37.017 UTC [88001] LOG: listening on Unix socket "/private/var/tb/3/scratch/81a37589b1b6aa8c/build/postgres2107914087/.s.PGSQL.5432"5012026-11-12 00:27:37.022 UTC [88010] LOG: database system was shut down at 2026-11-12 00:27:36 UTC5022026-11-12 00:27:37.024 UTC [88001] LOG: database system is ready to accept connections503/private/var/tb/3/scratch/81a37589b1b6aa8c/build/postgres2107914087:5432 - accepting connections504{"timestamp":"2026-11-12T00:27:37.30051Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"05bf5f06-38c6-4192-a839-27d6a0574393","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":15,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(8)"}505{"timestamp":"2026-11-12T00:27:37.408747Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"fa5538dc-73a7-465a-9c73-29b2f3388157","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(8)"}506{"timestamp":"2026-11-12T00:27:37.510339Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"842ec9c2-3aac-4ea8-81ed-dce24d42d745","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(9)"}507=== RUN TestAbortSucceeded508=== PAUSE TestAbortSucceeded509=== RUN TestService_AuthMiddleware510=== PAUSE TestService_AuthMiddleware511=== RUN TestService_AuthMiddleware_MTLSProxyHeader512=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader513=== RUN TestService_AuthMiddleware_MTLSBoundSubjects514=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects515=== RUN TestService_ReadAuthMiddleware516=== PAUSE TestService_ReadAuthMiddleware517=== RUN TestService_AuthMiddleware_OIDC518=== PAUSE TestService_AuthMiddleware_OIDC519=== RUN TestService_RequireScope_OIDC520=== PAUSE TestService_RequireScope_OIDC521=== RUN TestService_ReadScope_PublicByDefault522=== PAUSE TestService_ReadScope_PublicByDefault523=== RUN TestCacheConfigHandler524=== PAUSE TestCacheConfigHandler525=== RUN TestCacheStatsHandler526=== PAUSE TestCacheStatsHandler527=== RUN TestClientCADerivations528=== PAUSE TestClientCADerivations529=== RUN TestClientErrorHandling530=== PAUSE TestClientErrorHandling531=== RUN TestClientIntegration532=== PAUSE TestClientIntegration533=== RUN TestClientMultipleUploads534=== PAUSE TestClientMultipleUploads535=== RUN TestClientWithDependencies536=== PAUSE TestClientWithDependencies537=== RUN TestClientSharedPathCommittedMidPush538=== PAUSE TestClientSharedPathCommittedMidPush539=== RUN TestPinProtectsFromGC540=== PAUSE TestPinProtectsFromGC541=== RUN TestClientPushesUseOnePush542=== PAUSE TestClientPushesUseOnePush543=== RUN TestClientFallsBackToClosures544=== PAUSE TestClientFallsBackToClosures545=== RUN TestDeleteClosuresRacingPins546=== PAUSE TestDeleteClosuresRacingPins547=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock548=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock549=== RUN TestResolveDBConnectionString550=== PAUSE TestResolveDBConnectionString551=== RUN TestConnectWaitsForAPeerMigration552=== PAUSE TestConnectWaitsForAPeerMigration553=== RUN TestLeadElectsOneAndHandsOver554=== PAUSE TestLeadElectsOneAndHandsOver555=== RUN TestLeadIncumbentWinsAfterRestart5562026/11/12 00:27:37 INFO lead: acquired remote=192.0.2.1:1234 generation=15572026/11/12 00:27:38 INFO lead: released remote=192.0.2.1:1234 generation=15582026/11/12 00:27:38 INFO lead: acquired remote=192.0.2.1:1234 generation=25592026/11/12 00:27:39 INFO lead: released remote=192.0.2.1:1234 generation=2560--- PASS: TestLeadIncumbentWinsAfterRestart (2.12s)561=== RUN TestLeadUnannouncedStandbyYieldsToIncumbent562=== PAUSE TestLeadUnannouncedStandbyYieldsToIncumbent563=== RUN TestLeadIncumbentAnnouncesAtOnce564=== PAUSE TestLeadIncumbentAnnouncesAtOnce565=== RUN TestLeadGenerationGrowsWithEveryGrant566=== PAUSE TestLeadGenerationGrowsWithEveryGrant567=== RUN TestLeadEndsOnShutdown568=== PAUSE TestLeadEndsOnShutdown569=== RUN TestLeadEndsWhenItsConnectionHangs5702026/11/12 00:27:39 INFO lead: acquired remote=192.0.2.1:1234 generation=15712026-11-12 00:27:41.373 UTC [88516] FATAL: terminating connection due to administrator command5722026/11/12 00:27:41 INFO lead: acquired remote=192.0.2.1:1234 generation=25732026/11/12 00:27:42 WARN lead: lock connection lost error="timeout: context deadline exceeded"5742026/11/12 00:27:42 INFO lead: released remote=192.0.2.1:1234 generation=15752026/11/12 00:27:42 INFO lead: released remote=192.0.2.1:1234 generation=2576--- PASS: TestLeadEndsWhenItsConnectionHangs (3.04s)577=== RUN TestGCAdvisoryLockBlocksConcurrentRun578=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun579=== RUN TestGCBugBareHashReferences580=== PAUSE TestGCBugBareHashReferences581=== RUN TestGCMetrics582=== PAUSE TestGCMetrics583=== RUN TestPushDedupSurvivesConcurrentGC584=== PAUSE TestPushDedupSurvivesConcurrentGC585=== RUN TestDeduplicatedObjectsRecordedAsPending586=== PAUSE TestDeduplicatedObjectsRecordedAsPending587=== RUN TestGCSweepSkipsPendingObjects588=== PAUSE TestGCSweepSkipsPendingObjects589=== RUN TestTombstonedObjectOfferedWithoutWaiting590=== PAUSE TestTombstonedObjectOfferedWithoutWaiting591=== RUN TestGCSweepDeliversEachKeyOnce592=== PAUSE TestGCSweepDeliversEachKeyOnce593=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection594=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection595=== RUN TestForceGCDuringPushOffersSweptObject596=== PAUSE TestForceGCDuringPushOffersSweptObject597=== RUN TestFailedS3DeleteKeepsTombstone598=== PAUSE TestFailedS3DeleteKeepsTombstone599=== RUN TestSweepSkipsRowsLockedByPush600=== PAUSE TestSweepSkipsRowsLockedByPush601=== RUN TestRegisterCompletedObjectNeedsPendingRow602=== PAUSE TestRegisterCompletedObjectNeedsPendingRow603=== RUN TestGetExistingObjectsLocksRows604=== PAUSE TestGetExistingObjectsLocksRows605=== RUN TestSweepRechecksPendingAfterLocking606=== PAUSE TestSweepRechecksPendingAfterLocking607=== RUN TestPushWaitsForSweepHoldingRow608=== PAUSE TestPushWaitsForSweepHoldingRow609=== RUN TestCommitRacingPendingCleanupKeepsObjects610=== PAUSE TestCommitRacingPendingCleanupKeepsObjects611=== RUN TestGCEndsOnShutdown612=== PAUSE TestGCEndsOnShutdown613=== RUN TestGCTaskStore_StartNew614=== PAUSE TestGCTaskStore_StartNew615=== RUN TestGCTaskStore_DeduplicateSameParams616=== PAUSE TestGCTaskStore_DeduplicateSameParams617=== RUN TestGCTaskStore_ConflictDifferentParams618=== PAUSE TestGCTaskStore_ConflictDifferentParams619=== RUN TestGCTaskStore_GetEmpty620=== PAUSE TestGCTaskStore_GetEmpty621=== RUN TestGCTaskStore_GetReturnsLatest622=== PAUSE TestGCTaskStore_GetReturnsLatest623=== RUN TestGCTaskStore_CompletedAllowsNewTask624=== PAUSE TestGCTaskStore_CompletedAllowsNewTask625=== RUN TestGCTaskStore_PhaseUpdates626=== PAUSE TestGCTaskStore_PhaseUpdates627=== RUN TestGCTaskStore_Fail628=== PAUSE TestGCTaskStore_Fail629=== RUN TestGracefulShutdownDrainsInflight630=== PAUSE TestGracefulShutdownDrainsInflight631=== RUN TestService_healthCheckHandler632=== PAUSE TestService_healthCheckHandler633=== RUN TestService_readinessHandler634=== PAUSE TestService_readinessHandler635=== RUN TestGenerateLandingPage636=== PAUSE TestGenerateLandingPage637=== RUN TestEnsureAbortUploadsRule638=== PAUSE TestEnsureAbortUploadsRule639=== RUN TestCacheConfigHandlerMaxNarSize640=== PAUSE TestCacheConfigHandlerMaxNarSize641=== RUN TestCreatePendingClosureRejectsOversizedNAR642=== PAUSE TestCreatePendingClosureRejectsOversizedNAR643=== RUN TestNARDeduplicationMetadataUploadBug644=== PAUSE TestNARDeduplicationMetadataUploadBug645=== RUN TestMetricsInventory646=== PAUSE TestMetricsInventory647=== RUN TestMetricsCommit648=== PAUSE TestMetricsCommit649=== RUN TestService_NativeMTLS650=== PAUSE TestService_NativeMTLS651=== RUN TestServerTLSConfig652=== PAUSE TestServerTLSConfig653=== RUN TestMultipartCleanup654=== PAUSE TestMultipartCleanup655=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded656=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded657=== RUN TestPendingCleanupUsesOneCutoff658=== PAUSE TestPendingCleanupUsesOneCutoff659=== RUN TestObjectStatsTrigger660=== PAUSE TestObjectStatsTrigger661=== RUN TestObjectStatsUpdatedOncePerCommit662=== PAUSE TestObjectStatsUpdatedOncePerCommit663=== RUN TestOrphanedObjectsGC664=== PAUSE TestOrphanedObjectsGC665=== RUN TestOrphanedObjectsGCStressTest666=== PAUSE TestOrphanedObjectsGCStressTest667=== RUN TestCreatePin_ReservedPins668=== PAUSE TestCreatePin_ReservedPins669=== RUN TestPresentReportsOnlyClosureRoots670=== PAUSE TestPresentReportsOnlyClosureRoots671=== RUN TestPresentRefreshesOnlyStaleClosures672=== PAUSE TestPresentRefreshesOnlyStaleClosures673=== RUN TestPresentNotReportedWhileGCDeletesClosure674=== PAUSE TestPresentNotReportedWhileGCDeletesClosure675=== RUN TestParseSingleRange676=== PAUSE TestParseSingleRange677=== RUN TestProxyHeadersOnlyTrustedOnSocket6782026/11/12 00:27:42 INFO Starting HTTP server address=127.0.0.1:518876792026/11/12 00:27:42 INFO Starting HTTP server address=proxy.sock6802026/11/12 00:27:42 WARN mTLS auth: subject not in bound subjects subject="CN=someone"6812026/11/12 00:27:42 INFO Shutdown signal received, draining in-flight requests timeout=10s682--- PASS: TestProxyHeadersOnlyTrustedOnSocket (0.23s)683=== RUN TestIsValidCachePath684=== PAUSE TestIsValidCachePath685=== RUN TestReadProxyNarinfo686=== PAUSE TestReadProxyNarinfo687=== RUN TestReadProxyNarinfoAlreadyDecompressed688=== PAUSE TestReadProxyNarinfoAlreadyDecompressed689=== RUN TestReadProxyContentNegotiation690=== PAUSE TestReadProxyContentNegotiation691=== RUN TestReadProxyNarStreaming692=== PAUSE TestReadProxyNarStreaming693=== RUN TestReadProxy404694=== PAUSE TestReadProxy404695=== RUN TestReadProxyInvalidPath696=== PAUSE TestReadProxyInvalidPath697=== RUN TestReadProxyHead698=== PAUSE TestReadProxyHead699=== RUN TestReadProxyOutlastsServerWriteTimeout700=== PAUSE TestReadProxyOutlastsServerWriteTimeout701=== RUN TestReadProxyConditionalGet702=== PAUSE TestReadProxyConditionalGet703=== RUN TestReadProxyRootRedirectsToIndexHTML704=== PAUSE TestReadProxyRootRedirectsToIndexHTML705=== RUN TestReadProxyDisabled706=== PAUSE TestReadProxyDisabled707=== RUN TestReadRedirectNar708=== PAUSE TestReadRedirectNar709=== RUN TestReadRedirectKeepsNarinfoProxied710=== PAUSE TestReadRedirectKeepsNarinfoProxied711=== RUN TestReadProxyRangeRequest712=== PAUSE TestReadProxyRangeRequest713=== RUN TestReadRedirectUsesPublicS3URL714=== PAUSE TestReadRedirectUsesPublicS3URL715=== RUN TestPush_OverlappingRootsStoreOneRowPerKey716=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey717=== RUN TestPush_CompleteCommitsEveryRoot718=== PAUSE TestPush_CompleteCommitsEveryRoot719=== RUN TestPush_RepeatedRootCommitsOnce720=== PAUSE TestPush_RepeatedRootCommitsOnce721=== RUN TestPush_CompleteWaitsForCleanupOfThePush722=== PAUSE TestPush_CompleteWaitsForCleanupOfThePush723=== RUN TestPush_RootsAreLockedInByteOrder724=== PAUSE TestPush_RootsAreLockedInByteOrder725=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit726=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit727=== RUN TestPush_RejectsBadRequests728=== PAUSE TestPush_RejectsBadRequests729=== RUN TestPush_SignsNarinfosOfItsPendingObjects730=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects731=== RUN TestCommitCanBeRepeatedUntilCleanup732=== PAUSE TestCommitCanBeRepeatedUntilCleanup733=== RUN TestSharedNarBothUploadsComplete734=== PAUSE TestSharedNarBothUploadsComplete735=== RUN TestCompleteMultipartUpload_ErrorButObjectExists736=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists737=== RUN TestCompletedNarNotReofferedAcrossClosures738=== PAUSE TestCompletedNarNotReofferedAcrossClosures739=== RUN TestPresignedUploadRegisteredBeforeCommit740=== PAUSE TestPresignedUploadRegisteredBeforeCommit741=== RUN TestService_Rustfstest742=== PAUSE TestService_Rustfstest743=== RUN TestParseSize744=== PAUSE TestParseSize745=== RUN TestSkippedUploadsHandler746=== PAUSE TestSkippedUploadsHandler747=== RUN TestSystemdListenerNotActivated748--- PASS: TestSystemdListenerNotActivated (0.00s)749=== RUN TestWatchdogBeatsWhenHealthy750--- PASS: TestWatchdogBeatsWhenHealthy (0.03s)751=== RUN TestWatchdogSkipsWhenUnhealthy7522026/11/12 00:27:43 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7532026/11/12 00:27:43 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7542026/11/12 00:27:43 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7552026/11/12 00:27:43 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7562026/11/12 00:27:43 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7572026/11/12 00:27:43 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7582026/11/12 00:27:43 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7592026/11/12 00:27:43 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7602026/11/12 00:27:43 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7612026/11/12 00:27:43 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"762--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)763=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle764=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle765=== RUN TestProxyWriteTimeout766=== PAUSE TestProxyWriteTimeout767=== RUN TestPendingClosureWriteTimeout768=== PAUSE TestPendingClosureWriteTimeout769=== RUN TestIsValidUploadKey770=== PAUSE TestIsValidUploadKey771=== RUN TestUploadHandlersRejectInvalidKeys772=== PAUSE TestUploadHandlersRejectInvalidKeys773=== RUN TestUploadHandlersRejectOversizedBody774=== PAUSE TestUploadHandlersRejectOversizedBody775=== RUN TestService_cleanupPendingClosuresHandler776=== PAUSE TestService_cleanupPendingClosuresHandler777=== RUN TestService_createPendingClosureHandler778=== PAUSE TestService_createPendingClosureHandler779=== RUN TestService_verifyS3Integrity780=== PAUSE TestService_verifyS3Integrity781=== RUN TestService_verifyS3ReuploadsStaleNarinfo782=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo783=== RUN TestCompleteMultipartUnregistered784=== PAUSE TestCompleteMultipartUnregistered785=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT786=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT787=== CONT TestPush_RejectsBadRequests788=== CONT TestPush_CompleteCommitsEveryRoot789=== CONT TestPresignedUploadRegisteredBeforeCommit790=== CONT TestPresentReportsOnlyClosureRoots791=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT792=== CONT TestCompleteMultipartUnregistered793=== CONT TestService_verifyS3ReuploadsStaleNarinfo794=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed795=== CONT TestService_verifyS3Integrity796=== CONT TestService_createPendingClosureHandler797=== CONT TestService_cleanupPendingClosuresHandler798=== CONT TestUploadHandlersRejectOversizedBody799=== CONT TestUploadHandlersRejectInvalidKeys800=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info801=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info802=== CONT TestIsValidUploadKey803=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal804=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal805=== CONT TestPendingClosureWriteTimeout806=== RUN TestIsValidUploadKey/narinfo807=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key808=== CONT TestPresentRefreshesOnlyStaleClosures809=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key810=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key811=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key812=== CONT TestOrphanedObjectsGCStressTest813=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit814=== CONT TestPush_RootsAreLockedInByteOrder815=== CONT TestPush_CompleteWaitsForCleanupOfThePush816=== CONT TestGCTaskStore_GetEmpty817--- PASS: TestGCTaskStore_GetEmpty (0.00s)818=== CONT TestOrphanedObjectsGC819=== CONT TestPush_RepeatedRootCommitsOnce820=== CONT TestGCTaskStore_GetReturnsLatest821=== CONT TestCreatePin_ReservedPins822=== CONT TestProxyWriteTimeout823=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed824=== RUN TestPendingClosureWriteTimeout/empty825=== PAUSE TestPendingClosureWriteTimeout/empty826=== RUN TestPendingClosureWriteTimeout/400_objects827=== PAUSE TestPendingClosureWriteTimeout/400_objects828=== RUN TestPendingClosureWriteTimeout/670k_objects829=== PAUSE TestPendingClosureWriteTimeout/670k_objects830=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline831=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline832--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)833=== CONT TestObjectStatsUpdatedOncePerCommit834=== CONT TestAbortSucceeded835--- PASS: TestAbortSucceeded (0.00s)836=== CONT TestGCTaskStore_DeduplicateSameParams837--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)838=== CONT TestGCTaskStore_StartNew839--- PASS: TestGCTaskStore_StartNew (0.00s)840=== CONT TestGCEndsOnShutdown841=== PAUSE TestIsValidUploadKey/narinfo842=== CONT TestGCTaskStore_ConflictDifferentParams843--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)844=== CONT TestObjectStatsTrigger845=== RUN TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed846=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed847=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost848=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost849=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed850=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed851=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed852=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed853=== CONT TestPendingCleanupUsesOneCutoff854=== RUN TestProxyWriteTimeout/narinfo855=== PAUSE TestProxyWriteTimeout/narinfo856=== RUN TestProxyWriteTimeout/1_GiB_nar857=== PAUSE TestProxyWriteTimeout/1_GiB_nar858=== RUN TestProxyWriteTimeout/10_GiB_nar859=== PAUSE TestProxyWriteTimeout/10_GiB_nar860=== RUN TestProxyWriteTimeout/unknown_size861=== PAUSE TestProxyWriteTimeout/unknown_size862=== RUN TestGCEndsOnShutdown/before_the_run863=== PAUSE TestGCEndsOnShutdown/before_the_run864=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete865=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete866=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark867=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark868=== CONT TestCommitRacingPendingCleanupKeepsObjects869=== RUN TestIsValidUploadKey/nar_zst870=== PAUSE TestIsValidUploadKey/nar_zst871=== RUN TestIsValidUploadKey/nar_xz872=== PAUSE TestIsValidUploadKey/nar_xz873=== RUN TestIsValidUploadKey/nar_plain874=== PAUSE TestIsValidUploadKey/nar_plain875=== RUN TestIsValidUploadKey/listing876=== PAUSE TestIsValidUploadKey/listing877=== RUN TestIsValidUploadKey/build_log878=== PAUSE TestIsValidUploadKey/build_log879=== RUN TestIsValidUploadKey/build_log_home-manager_file880=== PAUSE TestIsValidUploadKey/build_log_home-manager_file881=== RUN TestIsValidUploadKey/build_log_plus_in_name882=== PAUSE TestIsValidUploadKey/build_log_plus_in_name883=== RUN TestIsValidUploadKey/build_log_question_mark884=== PAUSE TestIsValidUploadKey/build_log_question_mark885=== RUN TestIsValidUploadKey/build_log_equals886=== PAUSE TestIsValidUploadKey/build_log_equals887=== RUN TestIsValidUploadKey/realisation888=== PAUSE TestIsValidUploadKey/realisation889=== RUN TestIsValidUploadKey/realisation_plus_in_output890=== PAUSE TestIsValidUploadKey/realisation_plus_in_output891=== RUN TestIsValidUploadKey/nix-cache-info892=== PAUSE TestIsValidUploadKey/nix-cache-info893=== RUN TestIsValidUploadKey/index.html894=== PAUSE TestIsValidUploadKey/index.html895=== RUN TestIsValidUploadKey/narinfo_key,_nar_type896=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type897=== RUN TestIsValidUploadKey/nar_key,_narinfo_type898=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type899=== RUN TestIsValidUploadKey/listing_key,_narinfo_type900=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type901=== RUN TestIsValidUploadKey/traversal902=== PAUSE TestIsValidUploadKey/traversal903=== RUN TestIsValidUploadKey/traversal_nar904=== PAUSE TestIsValidUploadKey/traversal_nar905=== RUN TestIsValidUploadKey/absolute906=== PAUSE TestIsValidUploadKey/absolute907=== RUN TestIsValidUploadKey/empty_key908=== PAUSE TestIsValidUploadKey/empty_key909=== RUN TestIsValidUploadKey/unknown_type910=== PAUSE TestIsValidUploadKey/unknown_type911=== CONT TestPushWaitsForSweepHoldingRow912=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded913=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure914=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure915=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart916=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart917=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts918=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts919=== CONT TestMultipartCleanup9202026/11/12 00:27:43 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:51889/oidc921--- PASS: TestPresentReportsOnlyClosureRoots (1.47s)922=== CONT TestSweepRechecksPendingAfterLocking923=== RUN TestPush_RejectsBadRequests/no_roots924=== PAUSE TestPush_RejectsBadRequests/no_roots925=== RUN TestPush_RejectsBadRequests/no_objects926=== PAUSE TestPush_RejectsBadRequests/no_objects927=== RUN TestPush_RejectsBadRequests/bad_root928=== PAUSE TestPush_RejectsBadRequests/bad_root929=== RUN TestPush_RejectsBadRequests/root_not_in_objects930=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects931=== CONT TestGetExistingObjectsLocksRows9322026/11/12 00:27:45 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9332026/11/12 00:27:45 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst934--- PASS: TestCompleteMultipartUnregistered (2.19s)935=== CONT TestServerTLSConfig936=== RUN TestServerTLSConfig/no_client_CA937=== PAUSE TestServerTLSConfig/no_client_CA938=== RUN TestServerTLSConfig/missing_CA_file939=== PAUSE TestServerTLSConfig/missing_CA_file940=== RUN TestServerTLSConfig/not_a_PEM_file941=== PAUSE TestServerTLSConfig/not_a_PEM_file942=== CONT TestRegisterCompletedObjectNeedsPendingRow943--- PASS: TestObjectStatsTrigger (2.52s)944=== CONT TestService_NativeMTLS9452026/11/12 00:27:46 INFO Received push request method=POST path=/api/pushes9462026/11/12 00:27:46 INFO Received complete push request method=POST path=/api/pushes/1/complete9472026/11/12 00:27:46 INFO Completed push id=1 objects=4 duration=73.792167ms9482026/11/12 00:27:46 INFO Received push request method=POST path=/api/pushes9492026/11/12 00:27:46 INFO Aborted multipart uploads count=0 kept=09502026/11/12 00:27:46 WARN Force mode enabled - objects will be deleted immediately without grace period9512026/11/12 00:27:46 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=09522026/11/12 00:27:46 INFO Received push request method=POST path=/api/pushes9532026/11/12 00:27:46 INFO Vacuumed table table=pending_closures9542026/11/12 00:27:46 INFO Vacuumed table table=commit_receipts9552026/11/12 00:27:46 INFO Vacuumed table table=pending_objects9562026/11/12 00:27:46 INFO Vacuumed table table=multipart_uploads9572026/11/12 00:27:46 INFO Received complete push request method=POST path=/api/pushes/1/complete9582026/11/12 00:27:46 INFO Vacuumed table table=closures9592026/11/12 00:27:46 INFO Vacuumed table table=objects9602026/11/12 00:27:46 INFO Completed push id=1 objects=6 duration=41.021167ms961--- PASS: TestPush_CompleteCommitsEveryRoot (3.29s)962=== CONT TestSweepSkipsRowsLockedByPush9632026/11/12 00:27:46 INFO Received complete push request method=POST path=/api/pushes/2/complete9642026/11/12 00:27:46 INFO Completed push id=2 objects=4 duration=18.198ms965--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (3.33s)966=== CONT TestMetricsCommit9672026/11/12 00:27:46 INFO Received uploads request method=POST path=/api/pending_closures9682026/11/12 00:27:46 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst9692026/11/12 00:27:46 INFO Received uploads request method=POST path=/api/pending_closures970--- PASS: TestPresignedUploadRegisteredBeforeCommit (3.52s)971=== CONT TestFailedS3DeleteKeepsTombstone9722026/11/12 00:27:46 INFO Received uploads request method=POST path=/api/pending_closures973--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (3.77s)974=== CONT TestMetricsInventory9752026/11/12 00:27:47 INFO Received cleanup request method=DELETE path=/api/pending_closures9762026/11/12 00:27:47 INFO Aborted multipart uploads count=0 kept=09772026/11/12 00:27:47 INFO Received uploads request method=POST path=/api/pending_closures9782026/11/12 00:27:47 INFO Received cleanup request method=DELETE path=/api/pending_closures9792026/11/12 00:27:47 INFO Aborted multipart uploads count=1 kept=09802026/11/12 00:27:47 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete9812026-11-12 00:27:47.361 UTC [89360] ERROR: Closure does not exist: id=19822026-11-12 00:27:47.361 UTC [89360] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 17 at RAISE9832026-11-12 00:27:47.361 UTC [89360] STATEMENT: -- name: CommitPendingClosure :one984 SELECT commit_pending_closure($1::bigint)::bigint985 986--- PASS: TestService_cleanupPendingClosuresHandler (4.11s)987=== CONT TestForceGCDuringPushOffersSweptObject988=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows989=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows990=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check991=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check992=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection9932026/11/12 00:27:47 INFO Received uploads request method=POST path=/api/pending_closures9942026/11/12 00:27:47 INFO Received uploads request method=POST path=/api/pending_closures9952026/11/12 00:27:47 INFO Received uploads request method=POST path=/api/pending_closures996--- PASS: TestObjectStatsUpdatedOncePerCommit (4.87s)997=== CONT TestGCSweepDeliversEachKeyOnce9982026/11/12 00:27:48 INFO Received push request method=POST path=/api/pushes9992026/11/12 00:27:48 INFO Received complete push request method=POST path=/api/pushes/1/complete10002026/11/12 00:27:48 INFO Completed push id=1 objects=4 duration=54.915625ms1001--- PASS: TestPush_RepeatedRootCommitsOnce (5.10s)1002=== CONT TestTombstonedObjectOfferedWithoutWaiting1003--- PASS: TestPresentRefreshesOnlyStaleClosures (5.33s)1004=== CONT TestGCSweepSkipsPendingObjects10052026/11/12 00:27:48 INFO Received push request method=POST path=/api/pushes10062026/11/12 00:27:49 INFO Received uploads request method=POST path=/api/pending_closures10072026/11/12 00:27:49 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10082026-11-12 00:27:49.235 UTC [89887] ERROR: Push does not exist: id=110092026-11-12 00:27:49.235 UTC [89887] CONTEXT: PL/pgSQL function commit_push(bigint) line 15 at RAISE10102026-11-12 00:27:49.235 UTC [89887] STATEMENT: SELECT commit_push($1)1011--- PASS: TestPush_CompleteWaitsForCleanupOfThePush (6.01s)1012=== CONT TestNARDeduplicationMetadataUploadBug10132026/11/12 00:27:49 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=OWJkOTYxOGEtMzNiNS00Y2ZmLTlkNWItZWE0ZDE1MzM3N2FlLjgzM2VhMzlhLWI5OGItNDQ4Ny1hMGRjLWQ1ZjM3Mzk5OTdmZngxNzk0NDQzMjY3NTg2MDgwMDAw parts=1010142026/11/12 00:27:49 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10152026/11/12 00:27:49 INFO Completed upload id=1 objects=2 duration=36.597542ms10162026/11/12 00:27:49 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000010172026/11/12 00:27:49 INFO Received uploads request method=POST path=/api/pending_closures10182026/11/12 00:27:49 INFO Starting cleanup of old closures method=DELETE path=/api/closures10192026/11/12 00:27:49 INFO Aborted multipart uploads count=0 kept=010202026/11/12 00:27:49 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010212026/11/12 00:27:49 INFO Vacuumed table table=pending_closures10222026/11/12 00:27:49 INFO Vacuumed table table=commit_receipts10232026/11/12 00:27:49 INFO Vacuumed table table=pending_objects10242026/11/12 00:27:49 INFO Vacuumed table table=multipart_uploads10252026/11/12 00:27:49 INFO Vacuumed table table=closures10262026/11/12 00:27:49 INFO Vacuumed table table=objects10272026/11/12 00:27:49 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001028--- PASS: TestService_createPendingClosureHandler (6.41s)1029=== CONT TestCreatePendingClosureRejectsOversizedNAR10302026/11/12 00:27:49 INFO Received uploads request method=POST path=/api/pending_closures1031--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.01s)1032=== CONT TestCacheConfigHandlerMaxNarSize1033--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)1034=== CONT TestEnsureAbortUploadsRule10352026/11/12 00:27:49 INFO Received uploads request method=POST path=/api/pending_closures10362026/11/12 00:27:50 INFO Received cleanup request method=DELETE path=/api/pending_closures10372026/11/12 00:27:50 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket21/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""10382026/11/12 00:27:50 INFO Aborted multipart uploads count=0 kept=110392026/11/12 00:27:50 INFO Received cleanup request method=DELETE path=/api/pending_closures10402026/11/12 00:27:50 INFO Aborted multipart uploads count=1 kept=01041--- PASS: TestMultipartCleanup (6.86s)1042=== CONT TestGenerateLandingPage1043--- PASS: TestGenerateLandingPage (0.00s)1044=== CONT TestDeduplicatedObjectsRecordedAsPending10452026/11/12 00:27:50 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux10462026/11/12 00:27:50 WARN Refused reserved pin name=worker-x86_64-linux10472026/11/12 00:27:50 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux10482026/11/12 00:27:50 INFO Received create pin request method=POST path=/api/pins/my-app10492026/11/12 00:27:50 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux1050--- PASS: TestCreatePin_ReservedPins (7.05s)1051=== CONT TestService_readinessHandler10522026/11/12 00:27:50 INFO Received push request method=POST path=/api/pushes10532026-11-12 00:27:51.000 UTC [89553] ERROR: could not obtain lock on row in relation "closures"10542026-11-12 00:27:51.000 UTC [89553] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT10552026-11-12 00:27:51.001 UTC [89553] ERROR: could not obtain lock on row in relation "closures"10562026-11-12 00:27:51.001 UTC [89553] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT10572026-11-12 00:27:51.008 UTC [89553] ERROR: could not obtain lock on row in relation "closures"10582026-11-12 00:27:51.008 UTC [89553] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT10592026-11-12 00:27:51.010 UTC [89553] ERROR: could not obtain lock on row in relation "closures"10602026-11-12 00:27:51.010 UTC [89553] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT10612026-11-12 00:27:51.015 UTC [89553] ERROR: could not obtain lock on row in relation "closures"10622026-11-12 00:27:51.015 UTC [89553] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT1063--- PASS: TestPush_RootsAreLockedInByteOrder (7.84s)1064=== CONT TestService_healthCheckHandler10652026/11/12 00:27:51 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10662026/11/12 00:27:51 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=OWJkOTYxOGEtMzNiNS00Y2ZmLTlkNWItZWE0ZDE1MzM3N2FlLjAyNjdiZWYyLTdjZTgtNGE0OS05YmNlLTM1NmZiMTY0NWI4ZXgxNzk0NDQzMjY5MjUzOTc1MDAw parts=1010672026/11/12 00:27:51 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10682026/11/12 00:27:51 INFO Received uploads request method=POST path=/api/pending_closures10692026/11/12 00:27:51 INFO Completed upload id=1 objects=2 duration=35.067958ms10702026/11/12 00:27:51 INFO Received uploads request method=POST path=/api/pending_closures10712026/11/12 00:27:51 INFO Received uploads request method=POST path=/api/pending_closures10722026/11/12 00:27:51 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo10732026/11/12 00:27:51 WARN Found objects in DB but missing from S3, will re-upload count=11074--- PASS: TestService_verifyS3Integrity (8.10s)1075=== CONT TestGracefulShutdownDrainsInflight10762026/11/12 00:27:51 INFO Starting HTTP server address=127.0.0.1:5192610772026/11/12 00:27:51 INFO Shutdown signal received, draining in-flight requests timeout=10s1078--- PASS: TestGracefulShutdownDrainsInflight (0.08s)1079=== CONT TestGCTaskStore_Fail1080--- PASS: TestGCTaskStore_Fail (0.00s)1081=== CONT TestGCTaskStore_PhaseUpdates1082--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)1083=== CONT TestPushDedupSurvivesConcurrentGC1084--- PASS: TestPushWaitsForSweepHoldingRow (8.49s)1085=== CONT TestGCTaskStore_CompletedAllowsNewTask1086--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)1087=== CONT TestGCMetrics1088=== NAME TestOrphanedObjectsGC1089 orphaned_objects_gc_test.go:287: GC Test Summary:1090 orphaned_objects_gc_test.go:288: - Kept: 2 objects from closure A1091 orphaned_objects_gc_test.go:289: - Deleted: 2 objects from closure B1092 orphaned_objects_gc_test.go:290: - Deleted: 6 orphaned chain objects (X1->X2->X3)1093 orphaned_objects_gc_test.go:291: - Deleted: 2 orphaned single objects (Y)1094 orphaned_objects_gc_test.go:292: - Total deleted: 10 objects1095--- PASS: TestOrphanedObjectsGC (8.85s)1096=== CONT TestSkippedUploadsHandler10972026/11/12 00:27:52 INFO Client skipped oversized paths paths=3 nar_bytes=50000000001098--- PASS: TestSkippedUploadsHandler (0.00s)1099=== CONT TestGCBugBareHashReferences11002026/11/12 00:27:52 INFO Aborted multipart uploads count=0 kept=011012026/11/12 00:27:52 WARN Force mode enabled - objects will be deleted immediately without grace period11022026/11/12 00:27:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011032026/11/12 00:27:52 INFO Vacuumed table table=pending_closures11042026/11/12 00:27:52 INFO Vacuumed table table=commit_receipts11052026/11/12 00:27:52 INFO Vacuumed table table=pending_objects11062026/11/12 00:27:52 INFO Vacuumed table table=multipart_uploads11072026/11/12 00:27:52 INFO Vacuumed table table=closures11082026/11/12 00:27:52 INFO Received uploads request method=POST path=/api/pending_closures11092026/11/12 00:27:52 INFO Vacuumed table table=objects1110--- PASS: TestCommitRacingPendingCleanupKeepsObjects (9.15s)1111=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle11122026/11/12 00:27:52 INFO Received uploads request method=POST path=/api/pending_closures11132026/11/12 00:27:52 INFO Received cleanup request method=DELETE path=/api/pending_closures11142026/11/12 00:27:52 INFO Received uploads request method=POST path=/api/pending_closures1115--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (9.62s)1116=== CONT TestParseSize1117--- PASS: TestParseSize (0.00s)1118=== CONT TestGCAdvisoryLockBlocksConcurrentRun11192026/11/12 00:27:53 INFO Aborted multipart uploads count=0 kept=011202026/11/12 00:27:53 WARN Force mode enabled - objects will be deleted immediately without grace period11212026/11/12 00:27:53 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011222026/11/12 00:27:53 INFO Vacuumed table table=pending_closures11232026/11/12 00:27:53 INFO Vacuumed table table=commit_receipts11242026/11/12 00:27:53 INFO Vacuumed table table=pending_objects11252026/11/12 00:27:53 INFO Vacuumed table table=multipart_uploads11262026/11/12 00:27:53 INFO Vacuumed table table=closures11272026/11/12 00:27:53 INFO Vacuumed table table=objects1128--- PASS: TestSweepRechecksPendingAfterLocking (8.61s)1129=== CONT TestClientWithDependencies1130--- PASS: TestGetExistingObjectsLocksRows (8.49s)1131=== CONT TestClientMultipleUploads1132--- PASS: TestRegisterCompletedObjectNeedsPendingRow (8.27s)1133=== CONT TestLeadEndsOnShutdown11342026/11/12 00:27:54 WARN mTLS auth: subject not in bound subjects subject="CN=reader"11352026/11/12 00:27:54 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1136--- PASS: TestService_NativeMTLS (8.21s)1137=== CONT TestLeadGenerationGrowsWithEveryGrant11382026/11/12 00:27:54 INFO Aborted multipart uploads count=0 kept=011392026/11/12 00:27:54 WARN Force mode enabled - objects will be deleted immediately without grace period11402026/11/12 00:27:54 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011412026/11/12 00:27:54 INFO Vacuumed table table=pending_closures11422026/11/12 00:27:54 INFO Vacuumed table table=commit_receipts11432026/11/12 00:27:54 INFO Vacuumed table table=pending_objects11442026/11/12 00:27:54 INFO Vacuumed table table=multipart_uploads11452026/11/12 00:27:54 INFO Vacuumed table table=closures11462026/11/12 00:27:54 INFO Vacuumed table table=objects11472026/11/12 00:27:54 INFO Aborted multipart uploads count=0 kept=011482026/11/12 00:27:54 WARN Force mode enabled - objects will be deleted immediately without grace period11492026/11/12 00:27:54 INFO Received push request method=POST path=/api/pushes11502026/11/12 00:27:54 INFO Received complete push request method=POST path=/api/pushes/1/complete11512026/11/12 00:27:54 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=011522026/11/12 00:27:54 INFO Vacuumed table table=pending_closures11532026/11/12 00:27:54 INFO Vacuumed table table=commit_receipts11542026/11/12 00:27:54 INFO Vacuumed table table=pending_objects11552026/11/12 00:27:54 INFO Vacuumed table table=multipart_uploads11562026/11/12 00:27:54 INFO Vacuumed table table=closures11572026/11/12 00:27:54 INFO Vacuumed table table=objects11582026/11/12 00:27:54 INFO Completed push id=1 objects=4 duration=94.291875ms1159--- PASS: TestSweepSkipsRowsLockedByPush (8.30s)1160=== CONT TestLeadIncumbentAnnouncesAtOnce11612026/11/12 00:27:54 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete11622026/11/12 00:27:54 INFO Completed upload id=2 objects=2 duration=2.166875ms1163--- PASS: TestMetricsCommit (8.28s)1164=== CONT TestLeadUnannouncedStandbyYieldsToIncumbent11652026/11/12 00:27:54 INFO Received uploads request method=POST path=/api/pending_closures11662026/11/12 00:27:55 INFO Aborted multipart uploads count=0 kept=011672026/11/12 00:27:55 WARN Force mode enabled - objects will be deleted immediately without grace period11682026/11/12 00:27:55 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."11692026/11/12 00:27:55 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."1170--- PASS: TestFailedS3DeleteKeepsTombstone (8.51s)1171=== CONT TestLeadElectsOneAndHandsOver1172--- PASS: TestMetricsInventory (8.34s)1173=== CONT TestConnectWaitsForAPeerMigration11742026/11/12 00:27:55 INFO Received uploads request method=POST path=/api/pending_closures1175--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (8.46s)1176=== CONT TestResolveDBConnectionString1177=== RUN TestResolveDBConnectionString/flag_wins1178=== PAUSE TestResolveDBConnectionString/flag_wins1179=== RUN TestResolveDBConnectionString/file_when_flag_empty1180=== PAUSE TestResolveDBConnectionString/file_when_flag_empty1181=== RUN TestResolveDBConnectionString/missing_file_is_an_error1182=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error1183=== RUN TestResolveDBConnectionString/PGHOST_allows_empty1184=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty1185=== RUN TestResolveDBConnectionString/nothing_configured1186=== PAUSE TestResolveDBConnectionString/nothing_configured1187=== CONT TestClientIntegration11882026/11/12 00:27:55 INFO Received uploads request method=POST path=/api/pending_closures1189--- PASS: TestTombstonedObjectOfferedWithoutWaiting (7.53s)1190=== CONT TestClientErrorHandling1191=== RUN TestClientErrorHandling/InvalidStorePath1192=== PAUSE TestClientErrorHandling/InvalidStorePath1193=== RUN TestClientErrorHandling/InvalidAuthToken1194=== PAUSE TestClientErrorHandling/InvalidAuthToken1195=== RUN TestClientErrorHandling/ServerNotAvailable1196=== PAUSE TestClientErrorHandling/ServerNotAvailable1197=== CONT TestClientCADerivations11982026/11/12 00:27:56 INFO Aborted multipart uploads count=0 kept=011992026/11/12 00:27:56 WARN Force mode enabled - objects will be deleted immediately without grace period12002026/11/12 00:27:56 INFO Received uploads request method=POST path=/api/pending_closures12012026/11/12 00:27:56 INFO Aborted multipart uploads count=0 kept=012022026/11/12 00:27:56 WARN Force mode enabled - objects will be deleted immediately without grace period12032026/11/12 00:27:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012042026/11/12 00:27:56 INFO Vacuumed table table=pending_closures12052026/11/12 00:27:56 INFO Vacuumed table table=commit_receipts12062026/11/12 00:27:56 INFO Vacuumed table table=pending_objects12072026/11/12 00:27:56 INFO Vacuumed table table=multipart_uploads12082026/11/12 00:27:56 INFO Vacuumed table table=closures12092026/11/12 00:27:56 INFO Vacuumed table table=objects12102026/11/12 00:27:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=012112026/11/12 00:27:56 INFO Vacuumed table table=pending_closures12122026/11/12 00:27:56 INFO Vacuumed table table=commit_receipts1213--- PASS: TestGCSweepSkipsPendingObjects (8.23s)1214=== CONT TestCacheStatsHandler12152026/11/12 00:27:56 INFO Vacuumed table table=pending_objects12162026/11/12 00:27:56 INFO Vacuumed table table=multipart_uploads12172026/11/12 00:27:56 INFO Vacuumed table table=closures12182026/11/12 00:27:56 INFO Vacuumed table table=objects1219--- PASS: TestGCSweepDeliversEachKeyOnce (8.71s)1220=== CONT TestCacheConfigHandler1221=== RUN TestCacheConfigHandler/full_config,_no_issuer1222=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1223=== RUN TestCacheConfigHandler/no_cache_url_configured1224=== PAUSE TestCacheConfigHandler/no_cache_url_configured1225=== RUN TestCacheConfigHandler/no_signing_keys1226=== PAUSE TestCacheConfigHandler/no_signing_keys1227=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1228=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1229=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock1230=== NAME TestNARDeduplicationMetadataUploadBug1231 metadata_upload_test.go:48: First store path: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestNARDeduplicationMetadataUploadBug2455598691/001/store/r5ymzhs3cw49cwc6p713bp3v01d8crbd-file1.txt12322026/11/12 00:27:57 INFO Installed lifecycle rule aborting incomplete NAR uploads bucket=bucket41 days=21233--- PASS: TestEnsureAbortUploadsRule (7.61s)1234=== CONT TestDeleteClosuresRacingPins12352026/11/12 00:27:57 INFO Received push request method=POST path=/api/pushes12362026/11/12 00:27:57 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)12372026/11/12 00:27:57 INFO Uploading r5ymzhs3cw49cwc6p713bp3v01d8crbd-file1.txt (160B)1238=== NAME TestOrphanedObjectsGCStressTest1239 orphaned_objects_gc_test.go:422: Created 10 active closures, 5 to-delete closures, 20 orphaned chains12402026/11/12 00:27:57 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"12412026/11/12 00:27:57 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12422026/11/12 00:27:57 WARN Failed to register uploaded object key=r5ymzhs3cw49cwc6p713bp3v01d8crbd.ls error="server returned 404: 404 page not found\n"12432026/11/12 00:27:57 INFO Signed narinfos id=1 count=112442026/11/12 00:27:57 INFO Uploading 1 narinfos12452026/11/12 00:27:57 WARN readiness check failed error="closed pool"1246--- PASS: TestService_readinessHandler (7.16s)1247=== CONT TestClientFallsBackToClosures1248=== NAME TestOrphanedObjectsGCStressTest1249 orphaned_objects_gc_test.go:440: Marked 210 objects for deletion12502026/11/12 00:27:57 INFO Received complete push request method=POST path=/api/pushes/1/complete12512026/11/12 00:27:57 WARN Failed to register uploaded object key=r5ymzhs3cw49cwc6p713bp3v01d8crbd.narinfo error="server returned 404: 404 page not found\n"12522026/11/12 00:27:57 INFO Completed push id=1 objects=3 duration=24.608542ms12532026/11/12 00:27:57 INFO Upload complete. (321ms)1254=== NAME TestNARDeduplicationMetadataUploadBug1255 metadata_upload_test.go:54: Retrieved narinfo from S3:1256 StorePath: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestNARDeduplicationMetadataUploadBug2455598691/001/store/r5ymzhs3cw49cwc6p713bp3v01d8crbd-file1.txt1257 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1258 Compression: zstd1259 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1260 NarSize: 1601261 References: 1262 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1263 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)1264 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):1265 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}1266 metadata_upload_test.go:64: Second store path (same content): /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestNARDeduplicationMetadataUploadBug2455598691/001/store/vl916mvinc8x0bxyyzavx66li5k6v758-file2.txt12672026/11/12 00:27:57 INFO Received uploads request method=POST path=/api/pending_closures1268--- PASS: TestDeduplicatedObjectsRecordedAsPending (7.52s)1269=== CONT TestClientPushesUseOnePush12702026/11/12 00:27:57 INFO Received push request method=POST path=/api/pushes12712026/11/12 00:27:57 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)12722026/11/12 00:27:57 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign12732026/11/12 00:27:57 INFO Signed narinfos id=2 count=112742026/11/12 00:27:57 WARN Failed to register uploaded object key=vl916mvinc8x0bxyyzavx66li5k6v758.ls error="server returned 404: 404 page not found\n"12752026/11/12 00:27:57 INFO Uploading 1 narinfos12762026/11/12 00:27:57 INFO Received complete push request method=POST path=/api/pushes/2/complete12772026/11/12 00:27:57 WARN Failed to register uploaded object key=vl916mvinc8x0bxyyzavx66li5k6v758.narinfo error="server returned 404: 404 page not found\n"12782026/11/12 00:27:57 INFO Completed push id=2 objects=3 duration=1.617208ms12792026/11/12 00:27:57 INFO Upload complete. (209ms)1280=== NAME TestNARDeduplicationMetadataUploadBug1281 metadata_upload_test.go:76: Retrieved narinfo from S3:1282 StorePath: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestNARDeduplicationMetadataUploadBug2455598691/001/store/vl916mvinc8x0bxyyzavx66li5k6v758-file2.txt1283 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1284 Compression: zstd1285 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1286 NarSize: 1601287 References: 1288 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1289 metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes)1290 metadata_upload_test.go:77: Decompressed .ls content (49 bytes):1291 {"version":1,"root":{"type":"regular","size":44}}1292--- PASS: TestService_healthCheckHandler (6.91s)1293=== CONT TestPinProtectsFromGC1294--- PASS: TestNARDeduplicationMetadataUploadBug (8.76s)1295=== CONT TestClientSharedPathCommittedMidPush12962026/11/12 00:27:58 INFO Received uploads request method=POST path=/api/pending_closures12972026/11/12 00:27:58 INFO Aborted multipart uploads count=1 kept=012982026/11/12 00:27:58 INFO Aborted multipart uploads count=0 kept=012992026/11/12 00:27:58 INFO Aborted multipart uploads count=0 kept=013002026/11/12 00:27:58 WARN Force mode enabled - objects will be deleted immediately without grace period13012026/11/12 00:27:58 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013022026/11/12 00:27:58 INFO Received cleanup request method=DELETE path=/api/pending_closures13032026/11/12 00:27:58 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013042026/11/12 00:27:58 INFO Vacuumed table table=pending_closures13052026/11/12 00:27:58 INFO Aborted multipart uploads count=1 kept=013062026/11/12 00:27:58 INFO Vacuumed table table=commit_receipts13072026/11/12 00:27:58 INFO Vacuumed table table=pending_objects13082026/11/12 00:27:58 INFO Vacuumed table table=multipart_uploads13092026/11/12 00:27:58 INFO Vacuumed table table=closures13102026/11/12 00:27:58 INFO Vacuumed table table=objects1311--- PASS: TestPendingCleanupUsesOneCutoff (15.39s)1312=== CONT TestService_ReadScope_PublicByDefault1313--- PASS: TestGCMetrics (6.88s)1314=== CONT TestService_RequireScope_OIDC13152026/11/12 00:27:58 INFO Vacuumed table table=pending_closures13162026/11/12 00:27:58 INFO Vacuumed table table=commit_receipts13172026/11/12 00:27:58 INFO Vacuumed table table=pending_objects13182026/11/12 00:27:58 INFO Vacuumed table table=multipart_uploads13192026/11/12 00:27:58 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:51967/oidc13202026/11/12 00:27:58 INFO Vacuumed table table=closures13212026/11/12 00:27:58 INFO Vacuumed table table=objects13222026/11/12 00:27:58 INFO Aborted multipart uploads count=0 kept=013232026/11/12 00:27:58 WARN Force mode enabled - objects will be deleted immediately without grace period13242026/11/12 00:27:58 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013252026/11/12 00:27:58 INFO Vacuumed table table=pending_closures13262026/11/12 00:27:58 INFO Vacuumed table table=commit_receipts13272026/11/12 00:27:58 INFO Vacuumed table table=pending_objects13282026/11/12 00:27:58 INFO Vacuumed table table=multipart_uploads13292026/11/12 00:27:58 INFO Vacuumed table table=closures13302026/11/12 00:27:58 INFO Vacuumed table table=objects1331--- PASS: TestPushDedupSurvivesConcurrentGC (7.45s)1332=== CONT TestService_AuthMiddleware_OIDC13332026/11/12 00:27:59 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:51970/oidc13342026/11/12 00:27:59 INFO Received uploads request method=POST path=/api/pending_closures1335--- PASS: TestGCBugBareHashReferences (7.17s)1336=== CONT TestReadProxyHead13372026/11/12 00:27:59 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=_tribuchetbld3 database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"1338--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (6.72s)1339=== CONT TestService_ReadAuthMiddleware13402026/11/12 00:27:59 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1341=== NAME TestClientMultipleUploads1342 client_integration_test.go:406: Created store path 0: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientMultipleUploads2737556343/001/store/rl4g9qcyfvnlkimy5rjhrljgxqrmrjri-test-file-0.txt13432026/11/12 00:28:00 INFO lead: acquired remote=192.0.2.1:1234 generation=11344 client_integration_test.go:406: Created store path 1: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientMultipleUploads2737556343/001/store/9dm3b144mcv3lm8456bxb2w1hq5rz878-test-file-1.txt1345=== NAME TestClientWithDependencies1346 client_integration_test.go:661: Built derivation: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientWithDependencies2554924223/001/store/wsq60pfs0h2a0pg2jadg8fylbjgdhf3z-test-script1347 client_integration_test.go:663: Found 1 dependencies (including self)1348=== NAME TestClientMultipleUploads1349 client_integration_test.go:406: Created store path 2: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientMultipleUploads2737556343/001/store/5s808q7mmpixpckc8qpp4093llg9lxyr-test-file-2.txt13502026/11/12 00:28:01 INFO lead: acquired remote=192.0.2.1:1234 generation=113512026/11/12 00:28:01 INFO Received push request method=POST path=/api/pushes13522026/11/12 00:28:01 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)13532026/11/12 00:28:01 INFO Uploading wsq60pfs0h2a0pg2jadg8fylbjgdhf3z-test-script (136B)13542026/11/12 00:28:01 INFO Received push request method=POST path=/api/pushes13552026/11/12 00:28:01 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)13562026/11/12 00:28:01 INFO Uploading rl4g9qcyfvnlkimy5rjhrljgxqrmrjri-test-file-0.txt (160B)13572026/11/12 00:28:01 INFO Uploading 5s808q7mmpixpckc8qpp4093llg9lxyr-test-file-2.txt (160B)13582026/11/12 00:28:01 INFO Uploading 9dm3b144mcv3lm8456bxb2w1hq5rz878-test-file-1.txt (160B)13592026/11/12 00:28:01 WARN Failed to register uploaded object key=wsq60pfs0h2a0pg2jadg8fylbjgdhf3z.ls error="server returned 404: 404 page not found\n"13602026/11/12 00:28:01 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"13612026/11/12 00:28:01 WARN Failed to register uploaded object key=log/k4p0m36gfrdic5nv3wwqach2blxivcyc-test-script.drv error="server returned 404: 404 page not found\n"13622026/11/12 00:28:01 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13632026/11/12 00:28:01 INFO Signed narinfos id=1 count=113642026/11/12 00:28:01 INFO Uploading 1 narinfos13652026/11/12 00:28:01 WARN Failed to register uploaded object key=5s808q7mmpixpckc8qpp4093llg9lxyr.ls error="server returned 404: 404 page not found\n"13662026/11/12 00:28:01 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"13672026/11/12 00:28:01 WARN Failed to register uploaded object key=rl4g9qcyfvnlkimy5rjhrljgxqrmrjri.ls error="server returned 404: 404 page not found\n"13682026/11/12 00:28:01 WARN Failed to register uploaded object key=9dm3b144mcv3lm8456bxb2w1hq5rz878.ls error="server returned 404: 404 page not found\n"13692026/11/12 00:28:01 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"13702026/11/12 00:28:01 INFO Received complete push request method=POST path=/api/pushes/1/complete13712026/11/12 00:28:01 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13722026/11/12 00:28:01 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"13732026/11/12 00:28:01 WARN Failed to register uploaded object key=wsq60pfs0h2a0pg2jadg8fylbjgdhf3z.narinfo error="server returned 404: 404 page not found\n"13742026/11/12 00:28:01 INFO Signed narinfos id=1 count=313752026/11/12 00:28:01 INFO Uploading 3 narinfos13762026/11/12 00:28:01 INFO Completed push id=1 objects=4 duration=32.402584ms13772026/11/12 00:28:01 INFO Upload complete. (411ms)1378=== NAME TestClientWithDependencies1379 client_integration_test.go:665: Skipping nix copy test - isolated store (/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientWithDependencies2554924223/001/store) requires matching store prefix13802026/11/12 00:28:01 WARN Failed to register uploaded object key=5s808q7mmpixpckc8qpp4093llg9lxyr.narinfo error="server returned 404: 404 page not found\n"13812026/11/12 00:28:01 WARN Failed to register uploaded object key=rl4g9qcyfvnlkimy5rjhrljgxqrmrjri.narinfo error="server returned 404: 404 page not found\n"13822026/11/12 00:28:01 INFO Received complete push request method=POST path=/api/pushes/1/complete13832026/11/12 00:28:01 WARN Failed to register uploaded object key=9dm3b144mcv3lm8456bxb2w1hq5rz878.narinfo error="server returned 404: 404 page not found\n"13842026/11/12 00:28:01 INFO Completed push id=1 objects=9 duration=19.592667ms13852026/11/12 00:28:01 INFO Upload complete. (410ms)1386=== NAME TestClientMultipleUploads1387 client_integration_test.go:417: Uploaded 3 paths in 507.227458ms13882026/11/12 00:28:01 INFO lead: acquired remote=192.0.2.1:1234 generation=113892026/11/12 00:28:01 INFO lead: released remote=192.0.2.1:1234 generation=11390--- PASS: TestLeadIncumbentAnnouncesAtOnce (6.55s)1391--- PASS: TestClientWithDependencies (8.08s)1392=== CONT TestPush_OverlappingRootsStoreOneRowPerKey1393=== CONT TestService_AuthMiddleware_MTLSBoundSubjects1394--- PASS: TestClientMultipleUploads (8.01s)1395=== CONT TestService_AuthMiddleware_MTLSProxyHeader13962026/11/12 00:28:01 INFO lead: acquired remote=192.0.2.1:1234 generation=113972026/11/12 00:28:01 INFO lead: released remote=192.0.2.1:1234 generation=11398--- PASS: TestLeadEndsOnShutdown (8.04s)1399=== CONT TestReadRedirectUsesPublicS3URL14002026/11/12 00:28:01 INFO lead: acquired remote=192.0.2.1:1234 generation=114012026/11/12 00:28:02 INFO lead: released remote=192.0.2.1:1234 generation=114022026/11/12 00:28:02 INFO lead: acquired remote=192.0.2.1:1234 generation=21403=== NAME TestClientIntegration1404 client_integration_test.go:292: Created store path: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientIntegration242184035/002/store/wqmjh2kg4l1y4ngf24dp7ibqk2n8ld03-test-file.txt14052026/11/12 00:28:02 INFO lead: released remote=192.0.2.1:1234 generation=11406--- PASS: TestCacheStatsHandler (6.04s)14072026/11/12 00:28:02 INFO lead: acquired remote=192.0.2.1:1234 generation=21408=== CONT TestService_AuthMiddleware1409=== NAME TestClientCADerivations1410 client_ca_test.go:136: Built CA derivation: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientCADerivations3877557280/001/store/hfz3i0066fkhkivcxdk3jw9h6v3pi0va-ca-test14112026/11/12 00:28:02 INFO lead: given back to the incumbent remote=192.0.2.1:1234 generation=214122026/11/12 00:28:02 INFO lead: acquired remote=192.0.2.1:1234 generation=314132026/11/12 00:28:02 INFO Received push request method=POST path=/api/pushes1414--- PASS: TestConnectWaitsForAPeerMigration (7.59s)1415=== CONT TestReadProxyRangeRequest1416=== NAME TestClientCADerivations1417 client_ca_test.go:139: Found 1 dependencies (including self)14182026/11/12 00:28:02 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)14192026/11/12 00:28:02 INFO Uploading wqmjh2kg4l1y4ngf24dp7ibqk2n8ld03-test-file.txt (152B)14202026/11/12 00:28:03 WARN Failed to register uploaded object key=wqmjh2kg4l1y4ngf24dp7ibqk2n8ld03.ls error="server returned 404: 404 page not found\n"14212026/11/12 00:28:03 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign14222026/11/12 00:28:03 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"14232026/11/12 00:28:03 INFO Signed narinfos id=1 count=114242026/11/12 00:28:03 INFO Uploading 1 narinfos14252026/11/12 00:28:03 INFO Received complete push request method=POST path=/api/pushes/1/complete14262026/11/12 00:28:03 WARN Failed to register uploaded object key=wqmjh2kg4l1y4ngf24dp7ibqk2n8ld03.narinfo error="server returned 404: 404 page not found\n"14272026/11/12 00:28:03 INFO lead: released remote=192.0.2.1:1234 generation=114282026/11/12 00:28:03 INFO Completed push id=1 objects=3 duration=86.10275ms14292026/11/12 00:28:03 INFO Upload complete. (432ms)14302026/11/12 00:28:03 INFO lead: released remote=192.0.2.1:1234 generation=214312026/11/12 00:28:03 INFO lead: acquired remote=192.0.2.1:1234 generation=214322026/11/12 00:28:03 INFO Received push request method=POST path=/api/pushes14332026/11/12 00:28:03 INFO lead: acquired remote=192.0.2.1:1234 generation=314342026/11/12 00:28:03 INFO lead: released remote=192.0.2.1:1234 generation=31435--- PASS: TestLeadGenerationGrowsWithEveryGrant (9.24s)1436=== CONT TestReadRedirectKeepsNarinfoProxied14372026/11/12 00:28:03 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)14382026/11/12 00:28:03 INFO Uploading hfz3i0066fkhkivcxdk3jw9h6v3pi0va-ca-test (144B)14392026/11/12 00:28:03 INFO All 1 paths already cached1440=== NAME TestClientIntegration1441 client_integration_test.go:318: Retrieved narinfo from S3:1442 StorePath: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientIntegration242184035/002/store/wqmjh2kg4l1y4ngf24dp7ibqk2n8ld03-test-file.txt1443 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1444 Compression: zstd1445 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11446 NarSize: 1521447 References: 1448 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11449 client_integration_test.go:319: Retrieved .ls file from S3 (compressed size: 77 bytes)1450 client_integration_test.go:319: Decompressed .ls content (64 bytes):1451 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}14522026/11/12 00:28:03 WARN Failed to register uploaded object key=hfz3i0066fkhkivcxdk3jw9h6v3pi0va.ls error="server returned 404: 404 page not found\n"14532026/11/12 00:28:03 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"14542026/11/12 00:28:03 WARN Failed to register uploaded object key=log/mic2sxmpvl94vpxd3rkypksk60d3bpmh-ca-test.drv error="server returned 404: 404 page not found\n"14552026/11/12 00:28:03 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign14562026/11/12 00:28:03 INFO Signed narinfos id=1 count=114572026/11/12 00:28:03 INFO Uploading 1 narinfos14582026/11/12 00:28:03 WARN Rate limiter enabled after throttle name=s3-test rate=514592026/11/12 00:28:03 WARN S3 rate limit hit operation="complete multipart upload" error="Please reduce your request rate."1460=== NAME TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle1461 throttle_test.go:213: Proxy stats: total=13, throttled=10, completeMultipart=101462 throttle_test.go:215: Rate limiter: enabled=true, rate=5.001463--- PASS: TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle (10.98s)1464=== CONT TestReadRedirectNar14652026/11/12 00:28:03 INFO Received complete push request method=POST path=/api/pushes/1/complete14662026/11/12 00:28:03 WARN Failed to register uploaded object key=hfz3i0066fkhkivcxdk3jw9h6v3pi0va.narinfo error="server returned 404: 404 page not found\n"14672026/11/12 00:28:03 INFO Completed push id=1 objects=4 duration=62.018625ms14682026/11/12 00:28:03 INFO Upload complete. (528ms)1469=== NAME TestClientCADerivations1470 client_ca_test.go:180: Narinfo contains CA field: StorePath: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientCADerivations3877557280/001/store/hfz3i0066fkhkivcxdk3jw9h6v3pi0va-ca-test1471 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1472 Compression: zstd1473 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1474 NarSize: 1441475 References: 1476 Deriver: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientCADerivations3877557280/001/store/mic2sxmpvl94vpxd3rkypksk60d3bpmh-ca-test.drv1477 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1478 client_ca_test.go:185: Checking for realisation files in S3...1479 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1480 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache1481 client_ca_test.go:258: nix copy output: error: binary cache 's3://bucket57?endpoint=http://localhost:51870®ion=eu-west-1' is for Nix stores with prefix '/nix/store', not '/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientCADerivations3877557280/001/store'1482 client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 114832026/11/12 00:28:03 INFO Received push request method=POST path=/api/pushes14842026/11/12 00:28:03 INFO Object in database but missing from S3 key=wqmjh2kg4l1y4ngf24dp7ibqk2n8ld03.ls14852026/11/12 00:28:03 WARN Found objects in DB but missing from S3, will re-upload count=114862026/11/12 00:28:03 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)1487--- PASS: TestClientCADerivations (7.98s)1488=== CONT TestReadProxyNarinfoAlreadyDecompressed14892026/11/12 00:28:03 WARN Failed to register uploaded object key=wqmjh2kg4l1y4ngf24dp7ibqk2n8ld03.ls error="server returned 404: 404 page not found\n"14902026/11/12 00:28:03 INFO Received complete push request method=POST path=/api/pushes/2/complete14912026/11/12 00:28:03 INFO Completed push id=2 objects=3 duration=2.065292ms14922026/11/12 00:28:03 INFO Upload complete. (174ms)1493=== NAME TestClientIntegration1494 client_integration_test.go:336: Retrieved .ls file from S3 (compressed size: 62 bytes)1495 client_integration_test.go:336: Decompressed .ls content (49 bytes):1496 {"version":1,"root":{"type":"regular","size":39}}14972026/11/12 00:28:04 INFO Received push request method=POST path=/api/pushes14982026/11/12 00:28:04 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)14992026/11/12 00:28:04 INFO lead: released remote=192.0.2.1:1234 generation=21500--- PASS: TestLeadElectsOneAndHandsOver (9.07s)1501=== CONT TestReadProxyDisabled15022026/11/12 00:28:04 INFO Received complete push request method=POST path=/api/pushes/3/complete15032026/11/12 00:28:04 WARN Failed to register uploaded object key=wqmjh2kg4l1y4ngf24dp7ibqk2n8ld03.ls error="server returned 404: 404 page not found\n"15042026/11/12 00:28:04 INFO Completed push id=3 objects=3 duration=1.573541ms15052026/11/12 00:28:04 INFO Upload complete. (173ms)1506=== NAME TestClientIntegration1507 client_integration_test.go:353: Retrieved .ls file from S3 (compressed size: 62 bytes)1508 client_integration_test.go:353: Decompressed .ls content (49 bytes):1509 {"version":1,"root":{"type":"regular","size":39}}1510 client_integration_test.go:364: Testing garbage collection...15112026/11/12 00:28:04 INFO Starting cleanup of old closures method=DELETE path=/api/closures15122026/11/12 00:28:04 INFO Garbage collection started15132026/11/12 00:28:04 INFO Aborted multipart uploads count=0 kept=015142026/11/12 00:28:04 WARN Force mode enabled - objects will be deleted immediately without grace period1515--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (7.87s)1516=== CONT TestReadProxyInvalidPath15172026/11/12 00:28:04 INFO Received uploads request method=POST path=/api/pending_closures15182026/11/12 00:28:04 INFO lead: released remote=192.0.2.1:1234 generation=31519--- PASS: TestLeadUnannouncedStandbyYieldsToIncumbent (9.92s)1520=== CONT TestReadProxyRootRedirectsToIndexHTML15212026/11/12 00:28:04 INFO Received uploads request method=POST path=/api/pending_closures15222026/11/12 00:28:04 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)15232026/11/12 00:28:04 INFO Uploading khl89xa82d87sz2r1n1qah9ns08m0d2w-a (256B)15242026/11/12 00:28:04 INFO Uploading 6ia2i8ckhff94ymmy0ilwv48ywyjw645-shared-dep (136B)15252026/11/12 00:28:04 WARN Failed to register uploaded object key=9nrr2j17dfq5jn79g6zvs61krgz12bjz.ls error="server returned 404: 404 page not found\n"15262026/11/12 00:28:04 WARN Failed to register uploaded object key=6ia2i8ckhff94ymmy0ilwv48ywyjw645.ls error="server returned 404: 404 page not found\n"15272026/11/12 00:28:04 WARN Failed to register uploaded object key=khl89xa82d87sz2r1n1qah9ns08m0d2w.ls error="server returned 404: 404 page not found\n"15282026/11/12 00:28:04 WARN Failed to register uploaded object key=nar/1jmlfkawn0vz52qdqgx07x7zr82jz0qypcr11hxa1h1syx9fm6cm.nar.zst error="server returned 404: 404 page not found\n"15292026/11/12 00:28:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign15302026/11/12 00:28:04 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"15312026/11/12 00:28:04 INFO Signed narinfos id=2 count=215322026/11/12 00:28:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign15332026/11/12 00:28:05 INFO Signed narinfos id=1 count=215342026/11/12 00:28:05 INFO Uploading 4 narinfos15352026/11/12 00:28:05 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=015362026/11/12 00:28:05 WARN Failed to register uploaded object key=6ia2i8ckhff94ymmy0ilwv48ywyjw645.narinfo error="server returned 404: 404 page not found\n"15372026/11/12 00:28:05 WARN Failed to register uploaded object key=9nrr2j17dfq5jn79g6zvs61krgz12bjz.narinfo error="server returned 404: 404 page not found\n"15382026/11/12 00:28:05 WARN Failed to register uploaded object key=khl89xa82d87sz2r1n1qah9ns08m0d2w.narinfo error="server returned 404: 404 page not found\n"15392026/11/12 00:28:05 INFO Vacuumed table table=pending_closures15402026/11/12 00:28:05 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete15412026/11/12 00:28:05 WARN Failed to register uploaded object key=6ia2i8ckhff94ymmy0ilwv48ywyjw645.narinfo error="server returned 404: 404 page not found\n"15422026/11/12 00:28:05 INFO Vacuumed table table=commit_receipts15432026/11/12 00:28:05 INFO Completed upload id=1 objects=6 duration=77.049292ms15442026/11/12 00:28:05 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete15452026/11/12 00:28:05 INFO Completed upload id=2 objects=6 duration=1.417042ms15462026/11/12 00:28:05 INFO Vacuumed table table=pending_objects15472026/11/12 00:28:05 INFO Upload complete. (651ms)15482026/11/12 00:28:05 INFO Vacuumed table table=multipart_uploads1549=== NAME TestClientFallsBackToClosures1550 client_pushes_test.go:112: Retrieved narinfo from S3:1551 StorePath: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientFallsBackToClosures855987840/001/store/6ia2i8ckhff94ymmy0ilwv48ywyjw645-shared-dep1552 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1553 Compression: zstd1554 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821555 NarSize: 1361556 References: 1557 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1558 client_pushes_test.go:112: Retrieved narinfo from S3:1559 StorePath: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientFallsBackToClosures855987840/001/store/khl89xa82d87sz2r1n1qah9ns08m0d2w-a1560 URL: nar/1jmlfkawn0vz52qdqgx07x7zr82jz0qypcr11hxa1h1syx9fm6cm.nar.zst1561 Compression: zstd1562 NarHash: sha256:1jmlfkawn0vz52qdqgx07x7zr82jz0qypcr11hxa1h1syx9fm6cm1563 NarSize: 2561564 References: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientFallsBackToClosures855987840/001/store/6ia2i8ckhff94ymmy0ilwv48ywyjw645-shared-dep15652026/11/12 00:28:05 INFO Received push request method=POST path=/api/pushes1566 CA: text:sha256:1vdhfnywkbnj2z7wjv25pi3sa4rpzlfi9gj68a8k6rrilgfvb40115672026/11/12 00:28:05 INFO Vacuumed table table=closures1568 client_pushes_test.go:112: Retrieved narinfo from S3:1569 StorePath: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientFallsBackToClosures855987840/001/store/9nrr2j17dfq5jn79g6zvs61krgz12bjz-b1570 URL: nar/1jmlfkawn0vz52qdqgx07x7zr82jz0qypcr11hxa1h1syx9fm6cm.nar.zst1571 Compression: zstd1572 NarHash: sha256:1jmlfkawn0vz52qdqgx07x7zr82jz0qypcr11hxa1h1syx9fm6cm1573 NarSize: 2561574 References: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientFallsBackToClosures855987840/001/store/6ia2i8ckhff94ymmy0ilwv48ywyjw645-shared-dep1575 CA: text:sha256:1vdhfnywkbnj2z7wjv25pi3sa4rpzlfi9gj68a8k6rrilgfvb40115762026/11/12 00:28:05 INFO Vacuumed table table=objects15772026/11/12 00:28:05 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)15782026/11/12 00:28:05 INFO Uploading 7adlf7fq09i68w17q08ci071rgi3vfdg-shared-dep (136B)15792026/11/12 00:28:05 INFO Uploading xjssfw2jzhcgk5rkva1wyy7by7y0hrjg-a (256B)1580--- PASS: TestClientFallsBackToClosures (7.90s)1581=== CONT TestReadProxy4041582=== NAME TestPinProtectsFromGC1583 client_integration_test.go:779: Pinned store path: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestPinProtectsFromGC358702225/001/store/pyarjh3wzch5f1zl52db13m8c75ynvqv-pinned-file.txt1584 client_integration_test.go:780: Unpinned store path: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestPinProtectsFromGC358702225/001/store/x1vrpzq7jhya5ssx6d2dz2akgr265yma-unpinned-file.txt15852026/11/12 00:28:05 WARN Failed to register uploaded object key=xjssfw2jzhcgk5rkva1wyy7by7y0hrjg.ls error="server returned 404: 404 page not found\n"15862026/11/12 00:28:05 WARN Failed to register uploaded object key=sj7xxhqlkibsc0d74p7hz416kvi9ai17.ls error="server returned 404: 404 page not found\n"15872026/11/12 00:28:05 WARN Failed to register uploaded object key=7adlf7fq09i68w17q08ci071rgi3vfdg.ls error="server returned 404: 404 page not found\n"15882026/11/12 00:28:05 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"15892026/11/12 00:28:05 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15902026/11/12 00:28:05 WARN Failed to register uploaded object key=nar/16rg8xhwz2l6slkr444rqk03dbzl4wqfpg729mr0ysji3qrsxjx6.nar.zst error="server returned 404: 404 page not found\n"15912026/11/12 00:28:05 INFO Signed narinfos id=1 count=315922026/11/12 00:28:05 INFO Uploading 3 narinfos1593--- PASS: TestService_ReadScope_PublicByDefault (6.87s)15942026/11/12 00:28:05 WARN Failed to register uploaded object key=xjssfw2jzhcgk5rkva1wyy7by7y0hrjg.narinfo error="server returned 404: 404 page not found\n"1595=== CONT TestReadProxyConditionalGet15962026/11/12 00:28:05 WARN Failed to register uploaded object key=7adlf7fq09i68w17q08ci071rgi3vfdg.narinfo error="server returned 404: 404 page not found\n"15972026/11/12 00:28:05 INFO Received complete push request method=POST path=/api/pushes/1/complete15982026/11/12 00:28:05 WARN Failed to register uploaded object key=sj7xxhqlkibsc0d74p7hz416kvi9ai17.narinfo error="server returned 404: 404 page not found\n"15992026/11/12 00:28:05 INFO Received push request method=POST path=/api/pushes16002026/11/12 00:28:05 INFO Completed push id=1 objects=8 duration=53.2995ms16012026/11/12 00:28:05 INFO Upload complete. (475ms)16022026/11/12 00:28:05 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)16032026/11/12 00:28:05 INFO Uploading pyarjh3wzch5f1zl52db13m8c75ynvqv-pinned-file.txt (128B)1604=== NAME TestClientPushesUseOnePush1605 client_pushes_test.go:97: Retrieved narinfo from S3:1606 StorePath: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientPushesUseOnePush2858862339/001/store/7adlf7fq09i68w17q08ci071rgi3vfdg-shared-dep1607 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1608 Compression: zstd1609 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821610 NarSize: 1361611 References: 1612 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1613 client_pushes_test.go:97: Retrieved narinfo from S3:1614 StorePath: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientPushesUseOnePush2858862339/001/store/xjssfw2jzhcgk5rkva1wyy7by7y0hrjg-a1615 URL: nar/16rg8xhwz2l6slkr444rqk03dbzl4wqfpg729mr0ysji3qrsxjx6.nar.zst1616 Compression: zstd1617 NarHash: sha256:16rg8xhwz2l6slkr444rqk03dbzl4wqfpg729mr0ysji3qrsxjx61618 NarSize: 2561619 References: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientPushesUseOnePush2858862339/001/store/7adlf7fq09i68w17q08ci071rgi3vfdg-shared-dep1620 CA: text:sha256:01bv14bmx49zgc6s6maplyhhx7p7z5dkzpgdkg13f75kacjdpg5r1621 client_pushes_test.go:97: Retrieved narinfo from S3:1622 StorePath: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientPushesUseOnePush2858862339/001/store/sj7xxhqlkibsc0d74p7hz416kvi9ai17-b1623 URL: nar/16rg8xhwz2l6slkr444rqk03dbzl4wqfpg729mr0ysji3qrsxjx6.nar.zst1624 Compression: zstd1625 NarHash: sha256:16rg8xhwz2l6slkr444rqk03dbzl4wqfpg729mr0ysji3qrsxjx61626 NarSize: 2561627 References: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientPushesUseOnePush2858862339/001/store/7adlf7fq09i68w17q08ci071rgi3vfdg-shared-dep1628 CA: text:sha256:01bv14bmx49zgc6s6maplyhhx7p7z5dkzpgdkg13f75kacjdpg5r16292026/11/12 00:28:05 WARN Failed to register uploaded object key=pyarjh3wzch5f1zl52db13m8c75ynvqv.ls error="server returned 404: 404 page not found\n"16302026/11/12 00:28:05 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16312026/11/12 00:28:05 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"16322026/11/12 00:28:05 INFO Signed narinfos id=1 count=116332026/11/12 00:28:05 INFO Uploading 1 narinfos1634--- PASS: TestClientPushesUseOnePush (8.03s)1635=== CONT TestReadProxyOutlastsServerWriteTimeout16362026/11/12 00:28:05 INFO Received complete push request method=POST path=/api/pushes/1/complete16372026/11/12 00:28:05 WARN Failed to register uploaded object key=pyarjh3wzch5f1zl52db13m8c75ynvqv.narinfo error="server returned 404: 404 page not found\n"16382026/11/12 00:28:05 INFO Received push request method=POST path=/api/pushes16392026/11/12 00:28:05 INFO Completed push id=1 objects=3 duration=40.767708ms16402026/11/12 00:28:05 INFO Upload complete. (455ms)16412026/11/12 00:28:05 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)16422026/11/12 00:28:05 INFO Uploading x2fybgsmk1nd6d4dsbf6yqp5y455l9id-top (264B)16432026/11/12 00:28:05 INFO Uploading cawzzz6klxzi013xp51habl2lvlr62f7-shared-dep (136B)1644=== RUN TestService_RequireScope_OIDC/builder_may_write1645=== PAUSE TestService_RequireScope_OIDC/builder_may_write1646=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1647=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1648=== RUN TestService_RequireScope_OIDC/ops_may_admin1649=== PAUSE TestService_RequireScope_OIDC/ops_may_admin1650=== RUN TestService_RequireScope_OIDC/ops_may_not_write1651=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1652=== RUN TestService_RequireScope_OIDC/reader_may_not_write1653=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1654=== RUN TestService_RequireScope_OIDC/static_token_may_admin1655=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1656=== RUN TestService_RequireScope_OIDC/static_token_may_write1657=== PAUSE TestService_RequireScope_OIDC/static_token_may_write1658=== RUN TestService_RequireScope_OIDC/reader_may_read1659=== PAUSE TestService_RequireScope_OIDC/reader_may_read1660=== RUN TestService_RequireScope_OIDC/writer_implies_read1661=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1662=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1663=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read1664=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc1665=== PAUSE TestService_RequireScope_OIDC/reader_may_read_with_netrc1666=== RUN TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1667=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1668=== RUN TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1669=== PAUSE TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1670=== RUN TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1671=== PAUSE TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1672=== CONT TestReadProxyNarStreaming16732026/11/12 00:28:06 WARN Failed to register uploaded object key=cawzzz6klxzi013xp51habl2lvlr62f7.ls error="server returned 404: 404 page not found\n"16742026/11/12 00:28:06 WARN Failed to register uploaded object key=x2fybgsmk1nd6d4dsbf6yqp5y455l9id.ls error="server returned 404: 404 page not found\n"16752026/11/12 00:28:06 WARN Failed to register uploaded object key=nar/15b4b1w8ia4zfyaiqvw2i30wfqdb1wgm9zlzn7h1z9a5165qsba2.nar.zst error="server returned 404: 404 page not found\n"16762026/11/12 00:28:06 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16772026/11/12 00:28:06 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"16782026/11/12 00:28:06 INFO Signed narinfos id=1 count=216792026/11/12 00:28:06 INFO Uploading 2 narinfos16802026/11/12 00:28:06 INFO Received push request method=POST path=/api/pushes16812026/11/12 00:28:06 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)16822026/11/12 00:28:06 INFO Uploading x1vrpzq7jhya5ssx6d2dz2akgr265yma-unpinned-file.txt (128B)16832026/11/12 00:28:06 WARN Failed to register uploaded object key=x2fybgsmk1nd6d4dsbf6yqp5y455l9id.narinfo error="server returned 404: 404 page not found\n"16842026/11/12 00:28:06 INFO Received complete push request method=POST path=/api/pushes/1/complete16852026/11/12 00:28:06 WARN Failed to register uploaded object key=cawzzz6klxzi013xp51habl2lvlr62f7.narinfo error="server returned 404: 404 page not found\n"16862026/11/12 00:28:06 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"16872026/11/12 00:28:06 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign16882026/11/12 00:28:06 WARN Failed to register uploaded object key=x1vrpzq7jhya5ssx6d2dz2akgr265yma.ls error="server returned 404: 404 page not found\n"16892026/11/12 00:28:06 INFO Signed narinfos id=2 count=116902026/11/12 00:28:06 INFO Uploading 1 narinfos16912026/11/12 00:28:06 INFO Received complete push request method=POST path=/api/pushes/2/complete16922026/11/12 00:28:06 WARN Failed to register uploaded object key=x1vrpzq7jhya5ssx6d2dz2akgr265yma.narinfo error="server returned 404: 404 page not found\n"16932026/11/12 00:28:06 INFO Completed push id=2 objects=3 duration=1.375125ms16942026/11/12 00:28:06 INFO Completed push id=1 objects=6 duration=120.765125ms16952026/11/12 00:28:06 INFO Upload complete. (303ms)16962026/11/12 00:28:06 INFO Upload complete. (544ms)1697=== NAME TestClientSharedPathCommittedMidPush1698 client_integration_test.go:728: Retrieved narinfo from S3:1699 StorePath: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientSharedPathCommittedMidPush1414612351/001/store/cawzzz6klxzi013xp51habl2lvlr62f7-shared-dep1700 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1701 Compression: zstd1702 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821703 NarSize: 1361704 References: 1705 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1706 client_integration_test.go:728: Retrieved narinfo from S3:1707 StorePath: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientSharedPathCommittedMidPush1414612351/001/store/x2fybgsmk1nd6d4dsbf6yqp5y455l9id-top1708 URL: nar/15b4b1w8ia4zfyaiqvw2i30wfqdb1wgm9zlzn7h1z9a5165qsba2.nar.zst1709 Compression: zstd1710 NarHash: sha256:15b4b1w8ia4zfyaiqvw2i30wfqdb1wgm9zlzn7h1z9a5165qsba21711 NarSize: 2641712 References: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestClientSharedPathCommittedMidPush1414612351/001/store/cawzzz6klxzi013xp51habl2lvlr62f7-shared-dep1713 CA: text:sha256:1fl2425pbpxqxf8scpw00yqfp6gf7g4138815f5v5p1cr5xlvplj17142026/11/12 00:28:06 INFO Starting cleanup of old closures method=DELETE path=/api/closures17152026/11/12 00:28:06 INFO Garbage collection started17162026/11/12 00:28:06 INFO Aborted multipart uploads count=0 kept=01717--- PASS: TestClientSharedPathCommittedMidPush (8.37s)1718=== CONT TestReadProxyContentNegotiation17192026/11/12 00:28:06 WARN Force mode enabled - objects will be deleted immediately without grace period1720=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token1721=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token1722=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1723=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1724=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1725=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1726=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1727=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1728=== CONT TestService_Rustfstest17292026/11/12 00:28:06 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=01730=== NAME TestClientIntegration1731 client_integration_test.go:371: Objects in database after GC:1732 client_integration_test.go:371: Successfully deleted all objects with GC --force1733--- PASS: TestClientIntegration (10.91s)1734=== CONT TestIsValidCachePath1735=== RUN TestIsValidCachePath/narinfo1736=== PAUSE TestIsValidCachePath/narinfo1737=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars1738=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars1739=== RUN TestIsValidCachePath/nar_zst1740=== PAUSE TestIsValidCachePath/nar_zst1741=== RUN TestIsValidCachePath/nar_xz1742=== PAUSE TestIsValidCachePath/nar_xz1743=== RUN TestIsValidCachePath/nar_bz21744=== PAUSE TestIsValidCachePath/nar_bz21745=== RUN TestIsValidCachePath/nar_uncompressed1746=== PAUSE TestIsValidCachePath/nar_uncompressed1747=== RUN TestIsValidCachePath/ls1748=== PAUSE TestIsValidCachePath/ls1749=== RUN TestIsValidCachePath/log1750=== PAUSE TestIsValidCachePath/log1751=== RUN TestIsValidCachePath/realisation1752=== PAUSE TestIsValidCachePath/realisation1753=== RUN TestIsValidCachePath/nix-cache-info1754=== PAUSE TestIsValidCachePath/nix-cache-info1755=== RUN TestIsValidCachePath/index.html1756=== PAUSE TestIsValidCachePath/index.html1757=== RUN TestIsValidCachePath/traversal_parent1758=== PAUSE TestIsValidCachePath/traversal_parent1759=== RUN TestIsValidCachePath/traversal_in_middle1760=== PAUSE TestIsValidCachePath/traversal_in_middle1761=== RUN TestIsValidCachePath/invalid_char_e1762=== PAUSE TestIsValidCachePath/invalid_char_e1763=== RUN TestIsValidCachePath/invalid_char_u1764=== PAUSE TestIsValidCachePath/invalid_char_u1765=== RUN TestIsValidCachePath/random_path1766=== PAUSE TestIsValidCachePath/random_path1767=== RUN TestIsValidCachePath/empty1768=== PAUSE TestIsValidCachePath/empty1769=== RUN TestIsValidCachePath/leading_slash1770=== PAUSE TestIsValidCachePath/leading_slash1771=== RUN TestIsValidCachePath/wrong_extension1772=== PAUSE TestIsValidCachePath/wrong_extension1773=== RUN TestIsValidCachePath/short_hash1774=== PAUSE TestIsValidCachePath/short_hash1775=== CONT TestParseSingleRange1776=== RUN TestParseSingleRange/none1777=== PAUSE TestParseSingleRange/none1778=== RUN TestParseSingleRange/unknown_unit1779=== PAUSE TestParseSingleRange/unknown_unit1780=== RUN TestParseSingleRange/multi-range_ignored1781=== PAUSE TestParseSingleRange/multi-range_ignored1782=== RUN TestParseSingleRange/malformed_no_dash1783=== PAUSE TestParseSingleRange/malformed_no_dash1784=== RUN TestParseSingleRange/malformed_both_empty1785=== PAUSE TestParseSingleRange/malformed_both_empty1786=== RUN TestParseSingleRange/malformed_end_before_start1787=== PAUSE TestParseSingleRange/malformed_end_before_start1788=== RUN TestParseSingleRange/closed1789=== PAUSE TestParseSingleRange/closed1790=== RUN TestParseSingleRange/open-ended1791=== PAUSE TestParseSingleRange/open-ended1792=== RUN TestParseSingleRange/end_clamped_to_size1793=== PAUSE TestParseSingleRange/end_clamped_to_size1794=== RUN TestParseSingleRange/suffix1795=== PAUSE TestParseSingleRange/suffix1796=== RUN TestParseSingleRange/suffix_exceeds_size1797=== PAUSE TestParseSingleRange/suffix_exceeds_size1798=== RUN TestParseSingleRange/single_byte1799=== PAUSE TestParseSingleRange/single_byte1800=== RUN TestParseSingleRange/start_past_EOF1801=== PAUSE TestParseSingleRange/start_past_EOF1802=== RUN TestParseSingleRange/start_far_past_EOF1803=== PAUSE TestParseSingleRange/start_far_past_EOF1804=== CONT TestReadProxyNarinfo18052026/11/12 00:28:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=01806--- PASS: TestReadProxyHead (7.72s)1807=== CONT TestSharedNarBothUploadsComplete18082026/11/12 00:28:07 INFO Vacuumed table table=pending_closures18092026/11/12 00:28:07 INFO Vacuumed table table=commit_receipts18102026/11/12 00:28:07 INFO Vacuumed table table=pending_objects18112026/11/12 00:28:07 INFO Vacuumed table table=multipart_uploads18122026/11/12 00:28:07 INFO Vacuumed table table=closures18132026/11/12 00:28:07 INFO Vacuumed table table=objects1814--- PASS: TestService_ReadAuthMiddleware (7.67s)1815=== CONT TestCommitCanBeRepeatedUntilCleanup1816=== RUN TestCommitCanBeRepeatedUntilCleanup/push1817=== PAUSE TestCommitCanBeRepeatedUntilCleanup/push1818=== RUN TestCommitCanBeRepeatedUntilCleanup/pending_closure1819=== PAUSE TestCommitCanBeRepeatedUntilCleanup/pending_closure1820=== CONT TestCompletedNarNotReofferedAcrossClosures18212026/11/12 00:28:07 INFO Received push request method=POST path=/api/pushes1822--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (6.49s)1823=== CONT TestPresentNotReportedWhileGCDeletesClosure1824=== RUN TestPresentNotReportedWhileGCDeletesClosure/stale1825=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/stale1826=== RUN TestPresentNotReportedWhileGCDeletesClosure/fresh1827=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/fresh1828=== CONT TestCompleteMultipartUpload_ErrorButObjectExists18292026/11/12 00:28:08 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"18302026/11/12 00:28:08 WARN mTLS auth: bound subjects configured but subject DN unavailable18312026/11/12 00:28:08 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1832--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (6.76s)1833=== CONT TestPush_SignsNarinfosOfItsPendingObjects18342026/11/12 00:28:08 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=018352026/11/12 00:28:08 INFO Received create pin request method=POST path=/api/pins/myapp18362026/11/12 00:28:08 INFO Created/updated pin name=myapp store_path=/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestPinProtectsFromGC358702225/001/store/pyarjh3wzch5f1zl52db13m8c75ynvqv-pinned-file.txt narinfo_key=pyarjh3wzch5f1zl52db13m8c75ynvqv.narinfo18372026/11/12 00:28:08 INFO Starting cleanup of old closures method=DELETE path=/api/closures18382026/11/12 00:28:08 INFO Garbage collection started18392026/11/12 00:28:08 INFO Aborted multipart uploads count=0 kept=018402026/11/12 00:28:08 WARN Force mode enabled - objects will be deleted immediately without grace period18412026/11/12 00:28:08 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=018422026/11/12 00:28:08 INFO Vacuumed table table=pending_closures18432026/11/12 00:28:08 INFO Vacuumed table table=commit_receipts18442026/11/12 00:28:08 INFO Vacuumed table table=pending_objects18452026/11/12 00:28:08 INFO Vacuumed table table=multipart_uploads18462026/11/12 00:28:08 INFO Vacuumed table table=closures18472026/11/12 00:28:08 INFO Vacuumed table table=objects1848--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (7.08s)1849=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info18502026/11/12 00:28:08 INFO Received uploads request method=POST path=/1851=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key18522026/11/12 00:28:08 INFO Received request for more parts method=POST path=/1853=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key18542026/11/12 00:28:08 INFO Received complete multipart upload request method=POST path=/1855=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal18562026/11/12 00:28:08 INFO Received uploads request method=POST path=/1857--- PASS: TestUploadHandlersRejectInvalidKeys (0.00s)1858 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)1859 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1860 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1861 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s)1862=== CONT TestPendingClosureWriteTimeout/empty1863=== CONT TestPendingClosureWriteTimeout/670k_objects1864=== CONT TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1865--- PASS: TestReadRedirectUsesPublicS3URL (7.24s)1866=== CONT TestPendingClosureWriteTimeout/400_objects1867=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed18682026/11/12 00:28:09 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"1869--- PASS: TestService_AuthMiddleware (6.43s)1870=== CONT TestProxyWriteTimeout/narinfo1871=== CONT TestGCEndsOnShutdown/before_the_run1872--- PASS: TestReadProxyRangeRequest (6.72s)1873=== CONT TestIsValidUploadKey/narinfo1874=== CONT TestProxyWriteTimeout/unknown_size1875=== CONT TestProxyWriteTimeout/10_GiB_nar1876=== CONT TestProxyWriteTimeout/1_GiB_nar1877--- PASS: TestProxyWriteTimeout (0.02s)1878 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1879 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1880 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1881 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)1882=== CONT TestIsValidUploadKey/realisation_plus_in_output1883=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1884--- PASS: TestReadRedirectKeepsNarinfoProxied (6.76s)1885=== CONT TestIsValidUploadKey/unknown_type1886=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1887--- PASS: TestReadRedirectNar (6.99s)1888=== CONT TestIsValidUploadKey/listing_key,_narinfo_type1889=== CONT TestIsValidUploadKey/empty_key1890=== CONT TestIsValidUploadKey/nar_key,_narinfo_type1891=== CONT TestIsValidUploadKey/narinfo_key,_nar_type1892=== CONT TestIsValidUploadKey/index.html1893=== CONT TestIsValidUploadKey/absolute1894=== CONT TestIsValidUploadKey/traversal_nar1895=== CONT TestIsValidUploadKey/traversal1896=== CONT TestIsValidUploadKey/nix-cache-info1897=== CONT TestIsValidUploadKey/build_log_equals1898=== CONT TestIsValidUploadKey/build_log_home-manager_file1899=== CONT TestIsValidUploadKey/build_log_question_mark1900=== CONT TestIsValidUploadKey/build_log_plus_in_name1901=== CONT TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1902=== NAME TestPinProtectsFromGC1903 client_integration_test.go:888: Pin successfully protected closure from garbage collection1904--- PASS: TestDeleteClosuresRacingPins (13.25s)1905=== CONT TestIsValidUploadKey/realisation1906=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1907--- PASS: TestPinProtectsFromGC (12.57s)1908=== CONT TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1909--- PASS: TestReadProxyNarinfoAlreadyDecompressed (6.94s)1910=== CONT TestIsValidUploadKey/nar_plain1911=== CONT TestIsValidUploadKey/build_log1912=== CONT TestIsValidUploadKey/listing1913=== CONT TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1914--- PASS: TestReadProxyDisabled (6.76s)1915=== CONT TestIsValidUploadKey/nar_xz1916=== CONT TestIsValidUploadKey/nar_zst1917--- PASS: TestIsValidUploadKey (0.07s)1918 --- PASS: TestIsValidUploadKey/narinfo (0.00s)1919 --- PASS: TestIsValidUploadKey/realisation_plus_in_output (0.00s)1920 --- PASS: TestIsValidUploadKey/unknown_type (0.00s)1921 --- PASS: TestIsValidUploadKey/listing_key,_narinfo_type (0.00s)1922 --- PASS: TestIsValidUploadKey/empty_key (0.00s)1923 --- PASS: TestIsValidUploadKey/nar_key,_narinfo_type (0.00s)1924 --- PASS: TestIsValidUploadKey/narinfo_key,_nar_type (0.00s)1925 --- PASS: TestIsValidUploadKey/index.html (0.00s)1926 --- PASS: TestIsValidUploadKey/absolute (0.00s)1927 --- PASS: TestIsValidUploadKey/traversal_nar (0.00s)1928 --- PASS: TestIsValidUploadKey/traversal (0.00s)1929 --- PASS: TestIsValidUploadKey/nix-cache-info (0.00s)1930 --- PASS: TestIsValidUploadKey/build_log_equals (0.00s)1931 --- PASS: TestIsValidUploadKey/build_log_home-manager_file (0.00s)1932 --- PASS: TestIsValidUploadKey/build_log_question_mark (0.00s)1933 --- PASS: TestIsValidUploadKey/build_log_plus_in_name (0.00s)1934 --- PASS: TestIsValidUploadKey/realisation (0.00s)1935 --- PASS: TestIsValidUploadKey/nar_plain (0.00s)1936 --- PASS: TestIsValidUploadKey/build_log (0.00s)1937 --- PASS: TestIsValidUploadKey/listing (0.00s)1938 --- PASS: TestIsValidUploadKey/nar_xz (0.00s)1939 --- PASS: TestIsValidUploadKey/nar_zst (0.00s)1940=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure19412026/11/12 00:28:11 INFO Received uploads request method=POST path=/1942--- PASS: TestReadProxyRootRedirectsToIndexHTML (6.72s)1943=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts19442026/11/12 00:28:11 INFO Received request for more parts method=POST path=/1945=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart19462026/11/12 00:28:11 INFO Received complete multipart upload request method=POST path=/1947=== CONT TestPush_RejectsBadRequests/no_roots19482026/11/12 00:28:11 INFO Received push request method=POST path=/api/pushes1949=== CONT TestPush_RejectsBadRequests/bad_root19502026/11/12 00:28:11 INFO Received push request method=POST path=/api/pushes1951=== CONT TestPush_RejectsBadRequests/root_not_in_objects19522026/11/12 00:28:11 INFO Received push request method=POST path=/api/pushes1953=== CONT TestPush_RejectsBadRequests/no_objects19542026/11/12 00:28:11 INFO Received push request method=POST path=/api/pushes1955--- PASS: TestPush_RejectsBadRequests (1.73s)1956 --- PASS: TestPush_RejectsBadRequests/no_roots (0.01s)1957 --- PASS: TestPush_RejectsBadRequests/bad_root (0.00s)1958 --- PASS: TestPush_RejectsBadRequests/root_not_in_objects (0.00s)1959 --- PASS: TestPush_RejectsBadRequests/no_objects (0.00s)1960=== CONT TestServerTLSConfig/no_client_CA1961=== CONT TestServerTLSConfig/not_a_PEM_file1962=== CONT TestServerTLSConfig/missing_CA_file1963--- PASS: TestServerTLSConfig (0.00s)1964 --- PASS: TestServerTLSConfig/no_client_CA (0.00s)1965 --- PASS: TestServerTLSConfig/not_a_PEM_file (0.00s)1966 --- PASS: TestServerTLSConfig/missing_CA_file (0.00s)1967=== CONT TestForceGCDuringPushOffersSweptObject/before_pending_rows1968--- PASS: TestUploadHandlersRejectOversizedBody (0.09s)1969 --- PASS: TestUploadHandlersRejectOversizedBody/request_more_parts (0.04s)1970 --- PASS: TestUploadHandlersRejectOversizedBody/complete_multipart (0.04s)1971 --- PASS: TestUploadHandlersRejectOversizedBody/create_pending_closure (0.55s)1972=== CONT TestForceGCDuringPushOffersSweptObject/after_presence_check1973--- PASS: TestReadProxyInvalidPath (7.11s)1974=== CONT TestResolveDBConnectionString/flag_wins1975=== CONT TestResolveDBConnectionString/PGHOST_allows_empty1976=== CONT TestResolveDBConnectionString/nothing_configured1977=== CONT TestResolveDBConnectionString/missing_file_is_an_error1978=== CONT TestResolveDBConnectionString/file_when_flag_empty1979=== CONT TestClientErrorHandling/InvalidStorePath1980--- PASS: TestResolveDBConnectionString (0.02s)1981 --- PASS: TestResolveDBConnectionString/flag_wins (0.00s)1982 --- PASS: TestResolveDBConnectionString/PGHOST_allows_empty (0.00s)1983 --- PASS: TestResolveDBConnectionString/nothing_configured (0.00s)1984 --- PASS: TestResolveDBConnectionString/missing_file_is_an_error (0.00s)1985 --- PASS: TestResolveDBConnectionString/file_when_flag_empty (0.00s)1986--- PASS: TestReadProxyConditionalGet (6.62s)1987=== CONT TestClientErrorHandling/ServerNotAvailable19882026/11/12 00:28:12 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/objects/present19892026/11/12 00:28:12 WARN Request failed, retrying attempt=2 backoff=204.827948ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/objects/present19902026/11/12 00:28:12 WARN Request failed, retrying attempt=3 backoff=414.218444ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/objects/present1991--- PASS: TestReadProxy404 (7.74s)1992=== CONT TestClientErrorHandling/InvalidAuthToken1993--- PASS: TestReadProxyNarStreaming (7.26s)1994=== CONT TestCacheConfigHandler/full_config,_no_issuer1995=== CONT TestCacheConfigHandler/no_cache_url_configured1996=== CONT TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1997=== CONT TestCacheConfigHandler/no_signing_keys1998--- PASS: TestCacheConfigHandler (0.01s)1999 --- PASS: TestCacheConfigHandler/full_config,_no_issuer (0.00s)2000 --- PASS: TestCacheConfigHandler/no_cache_url_configured (0.00s)
loading 873 hidden lines…
28742026/11/12 00:28:31 INFO Completed upload id=15 objects=4 duration=8.027834ms28752026/11/12 00:28:31 INFO Received complete upload request method=POST path=/28762026/11/12 00:28:31 INFO Received push request method=POST path=/28772026/11/12 00:28:31 INFO Aborted multipart uploads count=0 kept=028782026/11/12 00:28:32 INFO Completed upload id=1 objects=9 duration=33.927584ms28792026/11/12 00:28:32 WARN Force mode enabled - objects will be deleted immediately without grace period28802026/11/12 00:28:32 INFO Received complete upload request method=POST path=/28812026/11/12 00:28:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028822026/11/12 00:28:32 INFO Completed upload id=9 objects=2 duration=1.322583ms28832026/11/12 00:28:32 INFO Aborted multipart uploads count=0 kept=028842026/11/12 00:28:32 INFO Vacuumed table table=pending_closures28852026/11/12 00:28:32 INFO Vacuumed table table=commit_receipts28862026/11/12 00:28:32 INFO Vacuumed table table=pending_objects28872026/11/12 00:28:32 INFO Vacuumed table table=multipart_uploads28882026/11/12 00:28:32 INFO Vacuumed table table=closures28892026/11/12 00:28:32 INFO Vacuumed table table=objects28902026/11/12 00:28:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028912026/11/12 00:28:32 INFO Received uploads request method=POST path=/28922026/11/12 00:28:32 INFO Vacuumed table table=pending_closures28932026/11/12 00:28:32 INFO Received push request method=POST path=/28942026/11/12 00:28:32 INFO Aborted multipart uploads count=0 kept=028952026/11/12 00:28:32 INFO Vacuumed table table=commit_receipts28962026/11/12 00:28:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028972026/11/12 00:28:32 INFO Vacuumed table table=pending_closures28982026/11/12 00:28:32 INFO Vacuumed table table=pending_objects28992026/11/12 00:28:32 INFO Received push request method=POST path=/29002026/11/12 00:28:32 INFO Received push request method=POST path=/29012026/11/12 00:28:32 INFO Vacuumed table table=multipart_uploads29022026/11/12 00:28:32 INFO Vacuumed table table=commit_receipts29032026/11/12 00:28:32 INFO Vacuumed table table=closures29042026/11/12 00:28:32 INFO Received complete push request method=POST path=/29052026/11/12 00:28:32 INFO Received push request method=POST path=/29062026/11/12 00:28:32 INFO Vacuumed table table=pending_objects29072026/11/12 00:28:32 INFO Completed push id=8 objects=3 duration=6.540292ms29082026/11/12 00:28:32 INFO Received push request method=POST path=/29092026/11/12 00:28:32 INFO Vacuumed table table=multipart_uploads29102026/11/12 00:28:32 INFO Vacuumed table table=closures29112026/11/12 00:28:32 INFO Vacuumed table table=objects29122026/11/12 00:28:32 INFO Received push request method=POST path=/29132026/11/12 00:28:32 INFO Aborted multipart uploads count=0 kept=029142026/11/12 00:28:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029152026/11/12 00:28:32 INFO Vacuumed table table=objects29162026/11/12 00:28:32 INFO Vacuumed table table=pending_closures29172026/11/12 00:28:32 INFO Received uploads request method=POST path=/29182026/11/12 00:28:32 INFO Received uploads request method=POST path=/29192026/11/12 00:28:32 INFO Vacuumed table table=commit_receipts29202026/11/12 00:28:32 INFO Vacuumed table table=pending_objects29212026/11/12 00:28:32 INFO Aborted multipart uploads count=0 kept=029222026/11/12 00:28:32 INFO Received push request method=POST path=/29232026/11/12 00:28:32 INFO Vacuumed table table=multipart_uploads29242026/11/12 00:28:32 INFO Received uploads request method=POST path=/29252026/11/12 00:28:32 INFO Vacuumed table table=closures29262026/11/12 00:28:32 WARN Force mode enabled - objects will be deleted immediately without grace period29272026/11/12 00:28:32 INFO Vacuumed table table=objects29282026/11/12 00:28:32 INFO Received complete push request method=POST path=/29292026/11/12 00:28:32 INFO Aborted multipart uploads count=0 kept=029302026/11/12 00:28:32 INFO Received push request method=POST path=/29312026/11/12 00:28:32 INFO Completed push id=3 objects=21 duration=6.986292ms29322026/11/12 00:28:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029332026/11/12 00:28:32 INFO Vacuumed table table=pending_closures29342026/11/12 00:28:32 INFO Received push request method=POST path=/29352026/11/12 00:28:32 INFO Received complete upload request method=POST path=/29362026/11/12 00:28:32 INFO Vacuumed table table=commit_receipts29372026/11/12 00:28:32 INFO Completed upload id=16 objects=3 duration=2.724417ms29382026/11/12 00:28:32 INFO Vacuumed table table=pending_objects29392026/11/12 00:28:32 INFO Vacuumed table table=multipart_uploads29402026/11/12 00:28:32 INFO Vacuumed table table=closures29412026/11/12 00:28:32 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."29422026/11/12 00:28:32 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."29432026/11/12 00:28:32 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."29442026/11/12 00:28:32 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."29452026/11/12 00:28:32 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."29462026/11/12 00:28:32 INFO Received uploads request method=POST path=/29472026/11/12 00:28:32 INFO Received uploads request method=POST path=/29482026/11/12 00:28:32 INFO Received complete push request method=POST path=/29492026/11/12 00:28:32 INFO Vacuumed table table=objects29502026/11/12 00:28:32 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."29512026/11/12 00:28:32 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."29522026/11/12 00:28:32 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."29532026/11/12 00:28:32 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."29542026/11/12 00:28:32 INFO Received uploads request method=POST path=/29552026/11/12 00:28:32 INFO Completed push id=1 objects=7 duration=14.248792ms29562026/11/12 00:28:32 INFO Aborted multipart uploads count=0 kept=029572026/11/12 00:28:32 INFO Aborted multipart uploads count=0 kept=029582026/11/12 00:28:32 INFO Received uploads request method=POST path=/29592026/11/12 00:28:32 INFO Aborted multipart uploads count=0 kept=029602026/11/12 00:28:32 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029612026/11/12 00:28:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029622026/11/12 00:28:32 WARN Force mode enabled - objects will be deleted immediately without grace period29632026/11/12 00:28:32 INFO Received uploads request method=POST path=/29642026/11/12 00:28:32 INFO Vacuumed table table=pending_closures29652026/11/12 00:28:32 INFO Received uploads request method=POST path=/29662026/11/12 00:28:32 INFO Vacuumed table table=commit_receipts29672026/11/12 00:28:32 INFO Vacuumed table table=pending_closures29682026/11/12 00:28:32 INFO Vacuumed table table=commit_receipts29692026/11/12 00:28:32 INFO Received push request method=POST path=/29702026/11/12 00:28:32 INFO Received complete upload request method=POST path=/29712026/11/12 00:28:32 INFO Vacuumed table table=pending_objects29722026/11/12 00:28:32 INFO Vacuumed table table=pending_objects29732026/11/12 00:28:32 INFO Completed upload id=13 objects=10 duration=943.708µs29742026/11/12 00:28:32 INFO Vacuumed table table=multipart_uploads29752026/11/12 00:28:32 INFO Vacuumed table table=multipart_uploads29762026/11/12 00:28:32 INFO Vacuumed table table=closures29772026/11/12 00:28:32 INFO Vacuumed table table=objects29782026/11/12 00:28:32 INFO Vacuumed table table=closures29792026/11/12 00:28:32 INFO Vacuumed table table=objects29802026/11/12 00:28:32 INFO Received push request method=POST path=/29812026/11/12 00:28:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=9 objects-failed-to-delete=029822026/11/12 00:28:32 INFO Received push request method=POST path=/29832026/11/12 00:28:32 INFO Aborted multipart uploads count=0 kept=029842026/11/12 00:28:32 INFO Received uploads request method=POST path=/29852026/11/12 00:28:32 INFO Received push request method=POST path=/29862026/11/12 00:28:32 WARN Force mode enabled - objects will be deleted immediately without grace period29872026/11/12 00:28:32 INFO Received uploads request method=POST path=/29882026/11/12 00:28:32 INFO Aborted multipart uploads count=0 kept=029892026/11/12 00:28:32 INFO Vacuumed table table=pending_closures29902026/11/12 00:28:32 INFO Received uploads request method=POST path=/29912026/11/12 00:28:32 WARN Force mode enabled - objects will be deleted immediately without grace period29922026/11/12 00:28:32 INFO Vacuumed table table=commit_receipts29932026/11/12 00:28:32 INFO Received uploads request method=POST path=/29942026/11/12 00:28:32 INFO Vacuumed table table=pending_objects29952026/11/12 00:28:32 INFO Received uploads request method=POST path=/29962026/11/12 00:28:32 INFO Received push request method=POST path=/29972026/11/12 00:28:32 INFO Vacuumed table table=multipart_uploads29982026/11/12 00:28:32 INFO Received push request method=POST path=/29992026/11/12 00:28:32 INFO Vacuumed table table=closures30002026/11/12 00:28:32 INFO Received complete upload request method=POST path=/30012026/11/12 00:28:32 INFO Vacuumed table table=objects30022026/11/12 00:28:32 INFO Completed upload id=18 objects=4 duration=4.459291ms30032026/11/12 00:28:32 INFO Received uploads request method=POST path=/30042026/11/12 00:28:32 INFO Aborted multipart uploads count=0 kept=030052026/11/12 00:28:32 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030062026/11/12 00:28:32 INFO Vacuumed table table=pending_closures30072026/11/12 00:28:32 INFO Received push request method=POST path=/30082026/11/12 00:28:32 INFO Vacuumed table table=commit_receipts30092026/11/12 00:28:32 INFO Received complete push request method=POST path=/30102026/11/12 00:28:32 INFO Vacuumed table table=pending_objects30112026/11/12 00:28:32 INFO Vacuumed table table=multipart_uploads30122026/11/12 00:28:32 INFO Completed push id=15 objects=5 duration=5.027916ms30132026/11/12 00:28:32 INFO Vacuumed table table=closures30142026/11/12 00:28:32 INFO Vacuumed table table=objects30152026/11/12 00:28:32 INFO Received complete upload request method=POST path=/30162026/11/12 00:28:32 INFO Completed upload id=6 objects=24 duration=4.438ms30172026/11/12 00:28:32 INFO Received uploads request method=POST path=/30182026/11/12 00:28:32 INFO Received complete push request method=POST path=/30192026/11/12 00:28:32 INFO Received uploads request method=POST path=/30202026/11/12 00:28:32 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."30212026/11/12 00:28:32 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."30222026/11/12 00:28:32 INFO Completed push id=17 objects=6 duration=9.456083ms30232026/11/12 00:28:32 INFO Received push request method=POST path=/30242026/11/12 00:28:32 INFO Received uploads request method=POST path=/30252026/11/12 00:28:32 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."30262026/11/12 00:28:32 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."30272026/11/12 00:28:32 INFO Received push request method=POST path=/30282026/11/12 00:28:32 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."30292026/11/12 00:28:32 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."30302026/11/12 00:28:32 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."30312026/11/12 00:28:32 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."30322026/11/12 00:28:32 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."30332026/11/12 00:28:32 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."30342026/11/12 00:28:32 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."30352026/11/12 00:28:32 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."30362026/11/12 00:28:32 INFO Received complete push request method=POST path=/30372026/11/12 00:28:32 INFO Received uploads request method=POST path=/30382026/11/12 00:28:32 INFO Received push request method=POST path=/30392026/11/12 00:28:32 INFO Received uploads request method=POST path=/30402026/11/12 00:28:32 INFO Received push request method=POST path=/30412026/11/12 00:28:33 INFO Completed push id=3 objects=5 duration=75.334334ms30422026/11/12 00:28:33 INFO Received push request method=POST path=/30432026/11/12 00:28:33 INFO Received complete upload request method=POST path=/30442026/11/12 00:28:33 INFO Received push request method=POST path=/30452026/11/12 00:28:33 INFO Completed upload id=2 objects=10 duration=3.048708ms30462026/11/12 00:28:33 INFO Aborted multipart uploads count=0 kept=030472026/11/12 00:28:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030482026/11/12 00:28:33 INFO Vacuumed table table=pending_closures30492026/11/12 00:28:33 INFO Received complete push request method=POST path=/30502026/11/12 00:28:33 INFO Vacuumed table table=commit_receipts30512026/11/12 00:28:33 INFO Received complete push request method=POST path=/30522026/11/12 00:28:33 INFO Completed push id=8 objects=6 duration=2.723375ms30532026/11/12 00:28:33 INFO Completed push id=9 objects=7 duration=2.378667ms30542026/11/12 00:28:33 INFO Vacuumed table table=pending_objects30552026/11/12 00:28:33 INFO Vacuumed table table=multipart_uploads30562026/11/12 00:28:33 INFO Vacuumed table table=closures30572026/11/12 00:28:33 INFO Received complete upload request method=POST path=/30582026/11/12 00:28:33 INFO Received uploads request method=POST path=/30592026/11/12 00:28:33 INFO Vacuumed table table=objects30602026/11/12 00:28:33 INFO Completed upload id=5 objects=7 duration=2.813667ms30612026/11/12 00:28:33 INFO Received push request method=POST path=/30622026/11/12 00:28:33 INFO Received push request method=POST path=/30632026/11/12 00:28:33 INFO Received push request method=POST path=/30642026/11/12 00:28:33 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."30652026/11/12 00:28:33 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."30662026/11/12 00:28:33 INFO Received push request method=POST path=/30672026/11/12 00:28:33 INFO Received uploads request method=POST path=/30682026/11/12 00:28:33 INFO Received complete upload request method=POST path=/30692026/11/12 00:28:33 INFO Aborted multipart uploads count=0 kept=030702026/11/12 00:28:33 INFO Completed upload id=6 objects=9 duration=1.667917ms30712026/11/12 00:28:33 INFO Received complete upload request method=POST path=/30722026/11/12 00:28:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=030732026/11/12 00:28:33 INFO Completed upload id=9 objects=3 duration=11.06625ms30742026/11/12 00:28:33 INFO Vacuumed table table=pending_closures30752026/11/12 00:28:33 INFO Vacuumed table table=commit_receipts30762026/11/12 00:28:33 INFO Vacuumed table table=pending_objects30772026/11/12 00:28:33 INFO Vacuumed table table=multipart_uploads30782026/11/12 00:28:33 INFO Received push request method=POST path=/30792026/11/12 00:28:33 INFO Received complete upload request method=POST path=/30802026/11/12 00:28:33 INFO Completed upload id=10 objects=7 duration=3.73ms30812026/11/12 00:28:33 INFO Received uploads request method=POST path=/30822026/11/12 00:28:33 INFO Aborted multipart uploads count=0 kept=030832026/11/12 00:28:33 INFO Vacuumed table table=closures30842026/11/12 00:28:33 WARN Force mode enabled - objects will be deleted immediately without grace period30852026/11/12 00:28:33 INFO Vacuumed table table=objects30862026/11/12 00:28:33 INFO Received uploads request method=POST path=/30872026/11/12 00:28:33 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030882026/11/12 00:28:33 INFO Aborted multipart uploads count=0 kept=030892026/11/12 00:28:33 INFO Vacuumed table table=pending_closures30902026/11/12 00:28:33 INFO Vacuumed table table=commit_receipts30912026/11/12 00:28:33 INFO Received complete upload request method=POST path=/30922026/11/12 00:28:33 INFO Received push request method=POST path=/30932026/11/12 00:28:33 INFO Vacuumed table table=pending_objects30942026/11/12 00:28:33 INFO Received push request method=POST path=/30952026/11/12 00:28:33 INFO Completed upload id=3 objects=5 duration=5.377083ms30962026/11/12 00:28:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=11 objects-failed-to-delete=030972026/11/12 00:28:33 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."30982026/11/12 00:28:33 INFO Vacuumed table table=multipart_uploads30992026/11/12 00:28:33 INFO Vacuumed table table=pending_closures31002026/11/12 00:28:33 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."31012026/11/12 00:28:33 INFO Vacuumed table table=commit_receipts31022026/11/12 00:28:33 INFO Vacuumed table table=closures31032026/11/12 00:28:33 INFO Vacuumed table table=pending_objects31042026/11/12 00:28:33 INFO Vacuumed table table=objects31052026/11/12 00:28:33 INFO Received complete upload request method=POST path=/31062026/11/12 00:28:33 INFO Vacuumed table table=multipart_uploads31072026/11/12 00:28:33 INFO Received complete push request method=POST path=/31082026/11/12 00:28:33 INFO Vacuumed table table=closures31092026/11/12 00:28:33 INFO Aborted multipart uploads count=0 kept=031102026/11/12 00:28:33 INFO Completed upload id=10 objects=2 duration=16.183208ms31112026/11/12 00:28:33 INFO Vacuumed table table=objects31122026/11/12 00:28:33 INFO Completed push id=3 objects=8 duration=33.28175ms31132026/11/12 00:28:33 INFO Received uploads request method=POST path=/31142026/11/12 00:28:33 INFO Received uploads request method=POST path=/31152026/11/12 00:28:33 INFO Received push request method=POST path=/31162026/11/12 00:28:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=031172026/11/12 00:28:33 INFO Received push request method=POST path=/31182026/11/12 00:28:33 INFO Received complete upload request method=POST path=/31192026/11/12 00:28:33 INFO Vacuumed table table=pending_closures31202026/11/12 00:28:33 INFO Completed upload id=3 objects=8 duration=8.450541ms31212026/11/12 00:28:33 INFO Received push request method=POST path=/31222026/11/12 00:28:33 INFO Vacuumed table table=commit_receipts31232026/11/12 00:28:33 INFO Received complete upload request method=POST path=/31242026/11/12 00:28:33 INFO Aborted multipart uploads count=0 kept=031252026/11/12 00:28:33 INFO Completed upload id=18 objects=4 duration=1.632042ms31262026/11/12 00:28:33 INFO Received uploads request method=POST path=/31272026/11/12 00:28:33 INFO Received push request method=POST path=/31282026/11/12 00:28:33 INFO Received uploads request method=POST path=/31292026/11/12 00:28:33 INFO Vacuumed table table=pending_objects31302026/11/12 00:28:33 INFO Received push request method=POST path=/31312026/11/12 00:28:33 INFO Received push request method=POST path=/31322026/11/12 00:28:33 INFO Received complete push request method=POST path=/31332026/11/12 00:28:33 INFO Vacuumed table table=multipart_uploads31342026/11/12 00:28:33 INFO Received push request method=POST path=/31352026/11/12 00:28:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=031362026/11/12 00:28:33 INFO Received uploads request method=POST path=/31372026/11/12 00:28:33 INFO Completed push id=6 objects=26 duration=3.907417ms31382026/11/12 00:28:33 INFO Received push request method=POST path=/31392026/11/12 00:28:33 INFO Received complete push request method=POST path=/31402026/11/12 00:28:33 INFO Vacuumed table table=closures31412026/11/12 00:28:33 INFO Vacuumed table table=pending_closures31422026/11/12 00:28:33 INFO Received complete push request method=POST path=/31432026/11/12 00:28:33 INFO Received complete push request method=POST path=/31442026/11/12 00:28:33 INFO Received complete push request method=POST path=/31452026/11/12 00:28:33 INFO Completed push id=4 objects=7 duration=3.511458ms31462026/11/12 00:28:33 INFO Vacuumed table table=commit_receipts31472026/11/12 00:28:33 INFO Vacuumed table table=objects31482026/11/12 00:28:33 INFO Completed push id=12 objects=3 duration=1.0265ms31492026/11/12 00:28:33 INFO Aborted multipart uploads count=0 kept=031502026/11/12 00:28:33 INFO Completed push id=3 objects=5 duration=4.907708ms31512026/11/12 00:28:33 INFO Vacuumed table table=pending_objects31522026/11/12 00:28:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031532026/11/12 00:28:33 INFO Vacuumed table table=multipart_uploads31542026/11/12 00:28:33 INFO Vacuumed table table=pending_closures31552026/11/12 00:28:33 INFO Vacuumed table table=commit_receipts31562026/11/12 00:28:33 INFO Completed push id=1 objects=5 duration=48.523916ms31572026/11/12 00:28:33 INFO Vacuumed table table=closures31582026/11/12 00:28:33 INFO Received uploads request method=POST path=/31592026/11/12 00:28:33 INFO Vacuumed table table=pending_objects31602026/11/12 00:28:33 INFO Received uploads request method=POST path=/31612026/11/12 00:28:33 INFO Vacuumed table table=objects31622026/11/12 00:28:33 INFO Vacuumed table table=multipart_uploads31632026/11/12 00:28:33 INFO Aborted multipart uploads count=0 kept=031642026/11/12 00:28:33 INFO Received complete push request method=POST path=/31652026/11/12 00:28:33 INFO Received complete push request method=POST path=/31662026/11/12 00:28:33 INFO Aborted multipart uploads count=0 kept=031672026/11/12 00:28:33 INFO Received push request method=POST path=/31682026/11/12 00:28:33 INFO Vacuumed table table=closures31692026/11/12 00:28:33 INFO Aborted multipart uploads count=0 kept=031702026/11/12 00:28:33 INFO Completed push id=14 objects=3 duration=2.967583ms31712026/11/12 00:28:33 WARN Force mode enabled - objects will be deleted immediately without grace period31722026/11/12 00:28:33 INFO Completed push id=14 objects=4 duration=6.633333ms31732026/11/12 00:28:33 INFO Vacuumed table table=objects31742026/11/12 00:28:33 INFO Received complete upload request method=POST path=/31752026/11/12 00:28:33 INFO Received uploads request method=POST path=/31762026/11/12 00:28:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031772026/11/12 00:28:33 INFO Completed upload id=4 objects=7 duration=5.053542ms31782026/11/12 00:28:33 INFO Vacuumed table table=pending_closures31792026/11/12 00:28:33 INFO Vacuumed table table=commit_receipts31802026/11/12 00:28:33 INFO Received push request method=POST path=/31812026/11/12 00:28:33 INFO Vacuumed table table=pending_objects31822026/11/12 00:28:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031832026/11/12 00:28:34 INFO Received push request method=POST path=/31842026/11/12 00:28:34 INFO Vacuumed table table=multipart_uploads31852026/11/12 00:28:34 INFO Received push request method=POST path=/31862026/11/12 00:28:34 INFO Received uploads request method=POST path=/31872026/11/12 00:28:34 INFO Vacuumed table table=closures31882026/11/12 00:28:34 INFO Received push request method=POST path=/31892026/11/12 00:28:34 INFO Vacuumed table table=objects31902026/11/12 00:28:34 INFO Vacuumed table table=pending_closures31912026/11/12 00:28:34 INFO Received push request method=POST path=/31922026/11/12 00:28:34 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."31932026/11/12 00:28:34 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."31942026/11/12 00:28:34 INFO Vacuumed table table=commit_receipts31952026/11/12 00:28:34 INFO Received uploads request method=POST path=/31962026/11/12 00:28:34 INFO Received complete push request method=POST path=/31972026/11/12 00:28:34 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."31982026/11/12 00:28:34 INFO Received complete push request method=POST path=/31992026/11/12 00:28:34 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."32002026/11/12 00:28:34 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."32012026/11/12 00:28:34 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."32022026/11/12 00:28:34 INFO Completed push id=2 objects=25 duration=4.741834ms32032026/11/12 00:28:34 INFO Completed push id=2 objects=10 duration=4.837666ms32042026/11/12 00:28:34 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."32052026/11/12 00:28:34 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."32062026/11/12 00:28:34 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."32072026/11/12 00:28:34 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."32082026/11/12 00:28:34 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."32092026/11/12 00:28:34 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."32102026/11/12 00:28:34 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."32112026/11/12 00:28:34 ERROR failed to remove object object=realisations/sha256:aaaaaaaa!out.doi error="Access Denied."32122026/11/12 00:28:34 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."32132026/11/12 00:28:34 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."32142026/11/12 00:28:34 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."32152026/11/12 00:28:34 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."32162026/11/12 00:28:34 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."32172026/11/12 00:28:34 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."32182026/11/12 00:28:34 INFO Vacuumed table table=pending_objects32192026/11/12 00:28:34 INFO Received complete push request method=POST path=/32202026/11/12 00:28:34 INFO Vacuumed table table=multipart_uploads32212026/11/12 00:28:34 INFO Completed push id=21 objects=9 duration=2.436333ms32222026/11/12 00:28:34 INFO Vacuumed table table=closures32232026/11/12 00:28:34 INFO Vacuumed table table=objects32242026/11/12 00:28:34 INFO Received uploads request method=POST path=/32252026/11/12 00:28:34 INFO Received uploads request method=POST path=/32262026/11/12 00:28:34 INFO Received uploads request method=POST path=/32272026/11/12 00:28:34 INFO Received push request method=POST path=/32282026/11/12 00:28:34 INFO Received push request method=POST path=/32292026/11/12 00:28:34 INFO Received push request method=POST path=/32302026/11/12 00:28:34 INFO Received complete push request method=POST path=/32312026/11/12 00:28:34 INFO Completed push id=4 objects=4 duration=1.1615ms32322026/11/12 00:28:34 INFO Aborted multipart uploads count=0 kept=032332026/11/12 00:28:34 INFO Received push request method=POST path=/32342026/11/12 00:28:34 INFO Received complete push request method=POST path=/32352026/11/12 00:28:34 INFO Completed push id=13 objects=12 duration=3.110042ms32362026/11/12 00:28:34 WARN Force mode enabled - objects will be deleted immediately without grace period32372026/11/12 00:28:34 INFO Received uploads request method=POST path=/32382026/11/12 00:28:34 INFO Received push request method=POST path=/32392026/11/12 00:28:34 INFO Received push request method=POST path=/32402026/11/12 00:28:34 INFO Received complete upload request method=POST path=/32412026/11/12 00:28:34 INFO Received push request method=POST path=/32422026/11/12 00:28:34 INFO Completed upload id=16 objects=23 duration=811.208µs32432026/11/12 00:28:34 INFO Received uploads request method=POST path=/32442026/11/12 00:28:34 INFO Aborted multipart uploads count=0 kept=032452026/11/12 00:28:34 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."32462026/11/12 00:28:34 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."32472026/11/12 00:28:34 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."32482026/11/12 00:28:34 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."32492026/11/12 00:28:34 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."32502026/11/12 00:28:34 INFO Received uploads request method=POST path=/32512026/11/12 00:28:34 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."32522026/11/12 00:28:34 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."32532026/11/12 00:28:34 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."32542026/11/12 00:28:34 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."32552026/11/12 00:28:34 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."32562026/11/12 00:28:34 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."32572026/11/12 00:28:34 INFO Received complete push request method=POST path=/32582026/11/12 00:28:34 INFO Received complete push request method=POST path=/32592026/11/12 00:28:34 INFO Received complete upload request method=POST path=/32602026/11/12 00:28:34 INFO Completed push id=2 objects=4 duration=1.001042ms32612026/11/12 00:28:34 INFO Completed push id=26 objects=14 duration=2.2335ms32622026/11/12 00:28:34 INFO Aborted multipart uploads count=0 kept=032632026/11/12 00:28:34 INFO Received complete push request method=POST path=/32642026/11/12 00:28:34 INFO Completed upload id=15 objects=6 duration=641.625µs32652026/11/12 00:28:34 INFO Completed push id=8 objects=7 duration=1.357792ms32662026/11/12 00:28:34 INFO Aborted multipart uploads count=0 kept=032672026/11/12 00:28:34 WARN Force mode enabled - objects will be deleted immediately without grace period32682026/11/12 00:28:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032692026/11/12 00:28:34 INFO Received uploads request method=POST path=/32702026/11/12 00:28:34 INFO Received push request method=POST path=/32712026/11/12 00:28:34 INFO Vacuumed table table=pending_closures32722026/11/12 00:28:34 INFO Aborted multipart uploads count=0 kept=032732026/11/12 00:28:34 INFO Vacuumed table table=commit_receipts32742026/11/12 00:28:34 WARN Force mode enabled - objects will be deleted immediately without grace period32752026/11/12 00:28:34 INFO Vacuumed table table=pending_objects32762026/11/12 00:28:34 INFO Vacuumed table table=multipart_uploads32772026/11/12 00:28:34 INFO Vacuumed table table=closures32782026/11/12 00:28:34 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=032792026/11/12 00:28:34 INFO Vacuumed table table=pending_closures32802026/11/12 00:28:34 INFO Vacuumed table table=objects32812026/11/12 00:28:34 INFO Vacuumed table table=commit_receipts32822026/11/12 00:28:34 INFO Received complete push request method=POST path=/32832026/11/12 00:28:34 INFO Vacuumed table table=pending_objects32842026/11/12 00:28:34 INFO Completed push id=11 objects=8 duration=10.208375ms32852026/11/12 00:28:34 INFO Vacuumed table table=multipart_uploads32862026/11/12 00:28:34 INFO Vacuumed table table=closures32872026/11/12 00:28:34 INFO Aborted multipart uploads count=0 kept=032882026/11/12 00:28:34 INFO Received uploads request method=POST path=/32892026/11/12 00:28:34 INFO Aborted multipart uploads count=0 kept=032902026/11/12 00:28:34 INFO Received uploads request method=POST path=/32912026/11/12 00:28:34 WARN Force mode enabled - objects will be deleted immediately without grace period32922026/11/12 00:28:34 WARN Force mode enabled - objects will be deleted immediately without grace period32932026/11/12 00:28:34 INFO Vacuumed table table=objects32942026/11/12 00:28:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032952026/11/12 00:28:34 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032962026/11/12 00:28:34 INFO Aborted multipart uploads count=0 kept=032972026/11/12 00:28:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=032982026/11/12 00:28:34 INFO Vacuumed table table=pending_closures32992026/11/12 00:28:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033002026/11/12 00:28:34 INFO Vacuumed table table=commit_receipts33012026/11/12 00:28:34 INFO Vacuumed table table=pending_closures33022026/11/12 00:28:34 INFO Received push request method=POST path=/33032026/11/12 00:28:34 INFO Received push request method=POST path=/33042026/11/12 00:28:34 INFO Vacuumed table table=pending_closures33052026/11/12 00:28:34 INFO Vacuumed table table=pending_closures33062026/11/12 00:28:34 INFO Vacuumed table table=commit_receipts33072026/11/12 00:28:34 INFO Vacuumed table table=commit_receipts33082026/11/12 00:28:34 INFO Vacuumed table table=pending_objects33092026/11/12 00:28:34 INFO Vacuumed table table=commit_receipts33102026/11/12 00:28:34 INFO Vacuumed table table=multipart_uploads33112026/11/12 00:28:34 INFO Vacuumed table table=pending_objects33122026/11/12 00:28:34 INFO Vacuumed table table=multipart_uploads33132026/11/12 00:28:34 INFO Vacuumed table table=closures33142026/11/12 00:28:34 INFO Vacuumed table table=pending_objects33152026/11/12 00:28:34 INFO Vacuumed table table=pending_objects33162026/11/12 00:28:34 INFO Vacuumed table table=closures33172026/11/12 00:28:34 INFO Vacuumed table table=objects33182026/11/12 00:28:34 INFO Vacuumed table table=multipart_uploads33192026/11/12 00:28:34 INFO Vacuumed table table=multipart_uploads33202026/11/12 00:28:34 INFO Vacuumed table table=objects33212026/11/12 00:28:34 INFO Received push request method=POST path=/33222026/11/12 00:28:34 INFO Vacuumed table table=closures33232026/11/12 00:28:34 INFO Vacuumed table table=objects33242026/11/12 00:28:34 INFO Received push request method=POST path=/33252026/11/12 00:28:34 INFO Vacuumed table table=closures33262026/11/12 00:28:34 INFO Received push request method=POST path=/33272026/11/12 00:28:34 INFO Received push request method=POST path=/33282026/11/12 00:28:34 INFO Received push request method=POST path=/33292026/11/12 00:28:34 INFO Vacuumed table table=objects33302026/11/12 00:28:34 INFO Received complete upload request method=POST path=/33312026/11/12 00:28:34 INFO Received uploads request method=POST path=/33322026/11/12 00:28:34 INFO Completed upload id=17 objects=7 duration=693.666µs33332026/11/12 00:28:34 INFO Received complete push request method=POST path=/33342026/11/12 00:28:34 INFO Completed push id=8 objects=2 duration=1.297667ms33352026/11/12 00:28:34 INFO Received uploads request method=POST path=/33362026/11/12 00:28:34 INFO Received complete push request method=POST path=/33372026/11/12 00:28:34 INFO Completed push id=5 objects=3 duration=3.274791ms33382026/11/12 00:28:34 INFO Received complete upload request method=POST path=/33392026/11/12 00:28:34 INFO Completed upload id=9 objects=16 duration=1.30125ms33402026/11/12 00:28:34 INFO Received push request method=POST path=/33412026/11/12 00:28:34 INFO Received uploads request method=POST path=/33422026/11/12 00:28:34 INFO Received push request method=POST path=/33432026/11/12 00:28:34 INFO Aborted multipart uploads count=0 kept=033442026/11/12 00:28:34 INFO Received push request method=POST path=/33452026/11/12 00:28:34 INFO Aborted multipart uploads count=0 kept=033462026/11/12 00:28:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033472026/11/12 00:28:34 INFO Received push request method=POST path=/33482026/11/12 00:28:34 INFO Received uploads request method=POST path=/33492026/11/12 00:28:34 INFO Aborted multipart uploads count=0 kept=033502026/11/12 00:28:34 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."33512026/11/12 00:28:34 INFO Received complete push request method=POST path=/33522026/11/12 00:28:34 INFO Vacuumed table table=pending_closures33532026/11/12 00:28:34 INFO Completed push id=13 objects=6 duration=9.777041ms33542026/11/12 00:28:34 INFO Vacuumed table table=commit_receipts33552026/11/12 00:28:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033562026/11/12 00:28:34 INFO Aborted multipart uploads count=0 kept=033572026/11/12 00:28:34 INFO Vacuumed table table=pending_objects33582026/11/12 00:28:34 INFO Vacuumed table table=multipart_uploads33592026/11/12 00:28:34 INFO Vacuumed table table=pending_closures33602026/11/12 00:28:34 INFO Vacuumed table table=closures33612026/11/12 00:28:34 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."33622026/11/12 00:28:34 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."33632026/11/12 00:28:34 INFO Vacuumed table table=commit_receipts33642026/11/12 00:28:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=4 objects-deleted-after-grace-period=0 objects-failed-to-delete=033652026/11/12 00:28:34 INFO Received push request method=POST path=/33662026/11/12 00:28:34 INFO Received uploads request method=POST path=/33672026/11/12 00:28:34 INFO Received uploads request method=POST path=/33682026/11/12 00:28:34 INFO Vacuumed table table=objects33692026/11/12 00:28:34 INFO Received push request method=POST path=/33702026/11/12 00:28:34 INFO Vacuumed table table=pending_objects33712026/11/12 00:28:34 INFO Received push request method=POST path=/33722026/11/12 00:28:34 INFO Vacuumed table table=multipart_uploads33732026/11/12 00:28:34 INFO Vacuumed table table=closures33742026/11/12 00:28:34 INFO Vacuumed table table=objects33752026/11/12 00:28:34 INFO Vacuumed table table=pending_closures33762026/11/12 00:28:34 INFO Received uploads request method=POST path=/33772026/11/12 00:28:34 INFO Received push request method=POST path=/33782026/11/12 00:28:34 INFO Received uploads request method=POST path=/33792026/11/12 00:28:34 INFO Aborted multipart uploads count=0 kept=033802026/11/12 00:28:34 INFO Vacuumed table table=commit_receipts33812026/11/12 00:28:34 WARN Force mode enabled - objects will be deleted immediately without grace period33822026/11/12 00:28:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033832026/11/12 00:28:34 INFO Received complete push request method=POST path=/33842026/11/12 00:28:34 INFO Completed push id=10 objects=3 duration=2.971541ms33852026/11/12 00:28:34 INFO Received complete upload request method=POST path=/33862026/11/12 00:28:34 INFO Vacuumed table table=pending_objects33872026/11/12 00:28:34 INFO Received push request method=POST path=/33882026/11/12 00:28:34 INFO Received complete push request method=POST path=/33892026/11/12 00:28:34 INFO Received complete push request method=POST path=/33902026/11/12 00:28:34 INFO Vacuumed table table=pending_closures33912026/11/12 00:28:34 INFO Vacuumed table table=multipart_uploads33922026/11/12 00:28:34 INFO Received uploads request method=POST path=/33932026/11/12 00:28:34 INFO Completed push id=4 objects=4 duration=4.210208ms33942026/11/12 00:28:34 INFO Received uploads request method=POST path=/33952026/11/12 00:28:34 INFO Received push request method=POST path=/33962026/11/12 00:28:34 INFO Completed push id=1 objects=11 duration=2.139667ms33972026/11/12 00:28:34 INFO Received push request method=POST path=/33982026/11/12 00:28:34 INFO Completed upload id=3 objects=6 duration=53.144375ms33992026/11/12 00:28:34 INFO Received uploads request method=POST path=/34002026/11/12 00:28:34 INFO Vacuumed table table=closures34012026/11/12 00:28:34 INFO Received complete push request method=POST path=/34022026/11/12 00:28:34 INFO Vacuumed table table=commit_receipts34032026/11/12 00:28:34 INFO Aborted multipart uploads count=0 kept=034042026/11/12 00:28:34 INFO Completed push id=16 objects=16 duration=4.526375ms34052026/11/12 00:28:34 INFO Vacuumed table table=pending_objects34062026/11/12 00:28:34 INFO Received uploads request method=POST path=/34072026/11/12 00:28:34 INFO Received push request method=POST path=/34082026/11/12 00:28:34 INFO Vacuumed table table=multipart_uploads34092026/11/12 00:28:34 INFO Aborted multipart uploads count=0 kept=034102026/11/12 00:28:34 WARN Force mode enabled - objects will be deleted immediately without grace period34112026/11/12 00:28:34 INFO Received complete upload request method=POST path=/34122026/11/12 00:28:34 INFO Vacuumed table table=objects34132026/11/12 00:28:34 INFO Completed upload id=14 objects=6 duration=1.517125ms34142026/11/12 00:28:34 INFO Received push request method=POST path=/34152026/11/12 00:28:35 INFO Received push request method=POST path=/34162026/11/12 00:28:35 INFO Vacuumed table table=closures34172026/11/12 00:28:35 INFO Received complete push request method=POST path=/34182026/11/12 00:28:35 INFO Completed push id=19 objects=6 duration=9.036291ms34192026/11/12 00:28:35 INFO Received push request method=POST path=/34202026/11/12 00:28:35 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=4 objects-failed-to-delete=034212026/11/12 00:28:35 INFO Vacuumed table table=objects34222026/11/12 00:28:35 INFO Received complete upload request method=POST path=/34232026/11/12 00:28:35 INFO Received complete upload request method=POST path=/34242026/11/12 00:28:35 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo error="Access Denied."34252026/11/12 00:28:35 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."34262026/11/12 00:28:35 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."34272026/11/12 00:28:35 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."34282026/11/12 00:28:35 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."34292026/11/12 00:28:35 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."34302026/11/12 00:28:35 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."34312026/11/12 00:28:35 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."34322026/11/12 00:28:35 INFO Vacuumed table table=pending_closures34332026/11/12 00:28:35 INFO Completed upload id=16 objects=3 duration=513.541µs34342026/11/12 00:28:35 INFO Completed upload id=13 objects=8 duration=814.75µs34352026/11/12 00:28:35 INFO Received push request method=POST path=/34362026/11/12 00:28:35 INFO Aborted multipart uploads count=0 kept=034372026/11/12 00:28:35 ERROR failed to remove object object=nar/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.nar.zst error="Access Denied."34382026/11/12 00:28:35 INFO Aborted multipart uploads count=0 kept=034392026/11/12 00:28:35 INFO Vacuumed table table=commit_receipts34402026/11/12 00:28:35 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."34412026/11/12 00:28:35 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."34422026/11/12 00:28:35 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."34432026/11/12 00:28:35 WARN Force mode enabled - objects will be deleted immediately without grace period34442026/11/12 00:28:35 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."34452026/11/12 00:28:35 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."34462026/11/12 00:28:35 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."34472026/11/12 00:28:35 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."34482026/11/12 00:28:35 INFO Vacuumed table table=pending_objects34492026/11/12 00:28:35 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034502026/11/12 00:28:35 INFO Vacuumed table table=multipart_uploads34512026/11/12 00:28:35 INFO Vacuumed table table=closures34522026/11/12 00:28:35 INFO Vacuumed table table=pending_closures34532026/11/12 00:28:35 INFO Vacuumed table table=objects34542026/11/12 00:28:35 INFO Vacuumed table table=commit_receipts34552026/11/12 00:28:35 INFO Received push request method=POST path=/34562026/11/12 00:28:35 INFO Aborted multipart uploads count=0 kept=034572026/11/12 00:28:35 INFO Vacuumed table table=pending_objects34582026/11/12 00:28:35 INFO Received complete push request method=POST path=/34592026/11/12 00:28:35 INFO Vacuumed table table=multipart_uploads34602026/11/12 00:28:35 INFO Completed push id=6 objects=6 duration=2.085625ms34612026/11/12 00:28:35 INFO Aborted multipart uploads count=0 kept=034622026/11/12 00:28:35 INFO Vacuumed table table=closures34632026/11/12 00:28:35 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."34642026/11/12 00:28:35 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."34652026/11/12 00:28:35 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."34662026/11/12 00:28:35 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."34672026/11/12 00:28:35 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."34682026/11/12 00:28:35 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."34692026/11/12 00:28:35 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."34702026/11/12 00:28:35 INFO Received complete upload request method=POST path=/34712026/11/12 00:28:35 INFO Received push request method=POST path=/34722026/11/12 00:28:35 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=034732026/11/12 00:28:35 INFO Completed upload id=10 objects=6 duration=2.68925ms34742026/11/12 00:28:35 INFO Vacuumed table table=objects34752026/11/12 00:28:35 INFO Received uploads request method=POST path=/34762026/11/12 00:28:35 INFO Vacuumed table table=pending_closures34772026/11/12 00:28:35 INFO Received push request method=POST path=/34782026/11/12 00:28:35 INFO Vacuumed table table=commit_receipts34792026/11/12 00:28:35 INFO Vacuumed table table=pending_objects34802026/11/12 00:28:35 INFO Vacuumed table table=multipart_uploads34812026/11/12 00:28:35 INFO Vacuumed table table=closures34822026/11/12 00:28:35 INFO Vacuumed table table=objects34832026/11/12 00:28:35 INFO Received push request method=POST path=/34842026/11/12 00:28:35 INFO Received uploads request method=POST path=/34852026/11/12 00:28:35 INFO Received uploads request method=POST path=/34862026/11/12 00:28:35 INFO Aborted multipart uploads count=0 kept=034872026/11/12 00:28:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=034882026/11/12 00:28:35 INFO Aborted multipart uploads count=0 kept=034892026/11/12 00:28:35 INFO Received uploads request method=POST path=/34902026/11/12 00:28:35 INFO Received uploads request method=POST path=/34912026/11/12 00:28:35 INFO Vacuumed table table=pending_closures34922026/11/12 00:28:35 INFO Received push request method=POST path=/34932026/11/12 00:28:35 WARN Force mode enabled - objects will be deleted immediately without grace period34942026/11/12 00:28:35 INFO Received push request method=POST path=/34952026/11/12 00:28:35 WARN Force mode enabled - objects will be deleted immediately without grace period34962026/11/12 00:28:35 INFO Received push request method=POST path=/34972026/11/12 00:28:35 INFO Received complete upload request method=POST path=/34982026/11/12 00:28:35 INFO Completed upload id=7 objects=3 duration=500.375µs34992026/11/12 00:28:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035002026/11/12 00:28:35 INFO Received uploads request method=POST path=/35012026/11/12 00:28:35 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."35022026/11/12 00:28:35 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."35032026/11/12 00:28:35 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."35042026/11/12 00:28:35 INFO Received push request method=POST path=/35052026/11/12 00:28:35 INFO Vacuumed table table=commit_receipts35062026/11/12 00:28:35 INFO Vacuumed table table=pending_closures35072026/11/12 00:28:35 INFO Vacuumed table table=pending_objects35082026/11/12 00:28:35 INFO Vacuumed table table=commit_receipts35092026/11/12 00:28:35 INFO Vacuumed table table=multipart_uploads35102026/11/12 00:28:35 INFO Received complete upload request method=POST path=/35112026/11/12 00:28:35 INFO Received push request method=POST path=/35122026/11/12 00:28:35 INFO Vacuumed table table=closures35132026/11/12 00:28:35 INFO Vacuumed table table=pending_objects35142026/11/12 00:28:35 INFO Completed upload id=19 objects=25 duration=3.624416ms35152026/11/12 00:28:35 INFO Vacuumed table table=multipart_uploads35162026/11/12 00:28:35 INFO Received uploads request method=POST path=/35172026/11/12 00:28:35 INFO Vacuumed table table=closures35182026/11/12 00:28:35 INFO Vacuumed table table=objects35192026/11/12 00:28:35 INFO Vacuumed table table=objects35202026/11/12 00:28:35 INFO Received uploads request method=POST path=/35212026/11/12 00:28:35 INFO Received uploads request method=POST path=/35222026/11/12 00:28:35 INFO Received complete push request method=POST path=/35232026/11/12 00:28:35 INFO Received push request method=POST path=/35242026/11/12 00:28:35 INFO Completed push id=17 objects=6 duration=1.499292ms35252026/11/12 00:28:35 INFO Aborted multipart uploads count=0 kept=035262026/11/12 00:28:35 INFO Aborted multipart uploads count=0 kept=035272026/11/12 00:28:35 INFO Aborted multipart uploads count=0 kept=035282026/11/12 00:28:35 WARN Force mode enabled - objects will be deleted immediately without grace period35292026/11/12 00:28:35 WARN Force mode enabled - objects will be deleted immediately without grace period35302026/11/12 00:28:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035312026/11/12 00:28:35 INFO Vacuumed table table=pending_closures35322026/11/12 00:28:35 INFO Received complete push request method=POST path=/35332026/11/12 00:28:35 INFO Vacuumed table table=commit_receipts35342026/11/12 00:28:35 INFO Vacuumed table table=pending_objects35352026/11/12 00:28:35 INFO Completed push id=27 objects=25 duration=7.369792ms35362026/11/12 00:28:35 INFO Received uploads request method=POST path=/35372026/11/12 00:28:35 INFO Vacuumed table table=multipart_uploads35382026/11/12 00:28:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=035392026/11/12 00:28:35 INFO Vacuumed table table=pending_closures35402026/11/12 00:28:35 INFO Vacuumed table table=closures35412026/11/12 00:28:35 INFO Vacuumed table table=commit_receipts35422026/11/12 00:28:35 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."35432026/11/12 00:28:35 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."35442026/11/12 00:28:35 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."35452026/11/12 00:28:35 INFO Vacuumed table table=objects35462026/11/12 00:28:35 INFO Vacuumed table table=pending_objects35472026/11/12 00:28:35 INFO Vacuumed table table=multipart_uploads35482026/11/12 00:28:35 INFO Vacuumed table table=closures35492026/11/12 00:28:35 INFO Vacuumed table table=objects35502026/11/12 00:28:35 INFO Received push request method=POST path=/35512026/11/12 00:28:35 INFO Aborted multipart uploads count=0 kept=035522026/11/12 00:28:35 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035532026/11/12 00:28:35 INFO Vacuumed table table=pending_closures35542026/11/12 00:28:35 INFO Vacuumed table table=commit_receipts35552026/11/12 00:28:35 INFO Received uploads request method=POST path=/35562026/11/12 00:28:35 INFO Received uploads request method=POST path=/35572026/11/12 00:28:35 INFO Vacuumed table table=pending_objects35582026/11/12 00:28:35 INFO Received complete push request method=POST path=/35592026/11/12 00:28:35 INFO Vacuumed table table=multipart_uploads35602026/11/12 00:28:35 INFO Aborted multipart uploads count=0 kept=035612026/11/12 00:28:35 INFO Completed push id=18 objects=23 duration=3.9095ms35622026/11/12 00:28:35 INFO Vacuumed table table=closures35632026/11/12 00:28:35 WARN Force mode enabled - objects will be deleted immediately without grace period35642026/11/12 00:28:35 INFO Vacuumed table table=objects35652026/11/12 00:28:35 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035662026/11/12 00:28:35 INFO Vacuumed table table=pending_closures35672026/11/12 00:28:35 INFO Vacuumed table table=commit_receipts35682026/11/12 00:28:35 INFO Received complete push request method=POST path=/35692026/11/12 00:28:35 INFO Received push request method=POST path=/35702026/11/12 00:28:35 INFO Received uploads request method=POST path=/35712026/11/12 00:28:35 INFO Received push request method=POST path=/35722026/11/12 00:28:35 INFO Received uploads request method=POST path=/35732026/11/12 00:28:35 INFO Vacuumed table table=pending_objects35742026/11/12 00:28:35 INFO Completed push id=7 objects=27 duration=1.097417ms35752026/11/12 00:28:35 INFO Received push request method=POST path=/35762026/11/12 00:28:35 INFO Received complete push request method=POST path=/35772026/11/12 00:28:35 INFO Vacuumed table table=multipart_uploads35782026/11/12 00:28:35 INFO Received complete upload request method=POST path=/35792026/11/12 00:28:35 INFO Vacuumed table table=closures35802026/11/12 00:28:35 INFO Completed push id=24 objects=5 duration=8.062084ms35812026/11/12 00:28:35 INFO Completed upload id=6 objects=7 duration=3.03275ms35822026/11/12 00:28:35 INFO Received uploads request method=POST path=/35832026/11/12 00:28:35 INFO Vacuumed table table=objects35842026/11/12 00:28:35 INFO Aborted multipart uploads count=0 kept=035852026/11/12 00:28:35 WARN Force mode enabled - objects will be deleted immediately without grace period35862026/11/12 00:28:35 INFO Received uploads request method=POST path=/35872026/11/12 00:28:35 INFO Received uploads request method=POST path=/35882026/11/12 00:28:35 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."35892026/11/12 00:28:35 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."35902026/11/12 00:28:35 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."35912026/11/12 00:28:35 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."35922026/11/12 00:28:35 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."35932026/11/12 00:28:35 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."35942026/11/12 00:28:35 INFO Received uploads request method=POST path=/35952026/11/12 00:28:35 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."35962026/11/12 00:28:35 INFO Received push request method=POST path=/35972026/11/12 00:28:35 INFO Received complete push request method=POST path=/35982026/11/12 00:28:35 INFO Aborted multipart uploads count=0 kept=035992026/11/12 00:28:35 INFO Completed push id=4 objects=7 duration=1.192292ms36002026/11/12 00:28:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036012026/11/12 00:28:35 INFO Vacuumed table table=pending_closures36022026/11/12 00:28:35 INFO Vacuumed table table=commit_receipts36032026/11/12 00:28:35 INFO Received complete push request method=POST path=/36042026/11/12 00:28:35 INFO Completed push id=11 objects=9 duration=3.234333ms36052026/11/12 00:28:35 INFO Vacuumed table table=pending_objects36062026/11/12 00:28:35 INFO Vacuumed table table=multipart_uploads36072026/11/12 00:28:35 INFO Received uploads request method=POST path=/36082026/11/12 00:28:35 INFO Vacuumed table table=closures36092026/11/12 00:28:35 INFO Received uploads request method=POST path=/36102026/11/12 00:28:35 INFO Received uploads request method=POST path=/36112026/11/12 00:28:35 INFO Vacuumed table table=objects36122026/11/12 00:28:35 INFO Received uploads request method=POST path=/36132026/11/12 00:28:35 INFO Received push request method=POST path=/36142026/11/12 00:28:35 INFO Received complete upload request method=POST path=/36152026/11/12 00:28:35 INFO Completed upload id=17 objects=6 duration=2.126625ms36162026/11/12 00:28:35 INFO Received complete upload request method=POST path=/36172026/11/12 00:28:35 INFO Received push request method=POST path=/36182026/11/12 00:28:35 INFO Completed upload id=21 objects=3 duration=3.409833ms36192026/11/12 00:28:35 INFO Received complete push request method=POST path=/36202026/11/12 00:28:35 INFO Received complete upload request method=POST path=/36212026/11/12 00:28:35 INFO Received push request method=POST path=/36222026/11/12 00:28:35 INFO Received uploads request method=POST path=/36232026/11/12 00:28:35 INFO Completed push id=5 objects=28 duration=3.472084ms36242026/11/12 00:28:35 INFO Completed upload id=3 objects=9 duration=846.167µs36252026/11/12 00:28:35 INFO Received push request method=POST path=/36262026/11/12 00:28:35 INFO Aborted multipart uploads count=0 kept=036272026/11/12 00:28:35 WARN Force mode enabled - objects will be deleted immediately without grace period36282026/11/12 00:28:35 INFO Received push request method=POST path=/36292026/11/12 00:28:35 INFO Received push request method=POST path=/36302026/11/12 00:28:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=7 objects-failed-to-delete=036312026/11/12 00:28:35 INFO Received complete push request method=POST path=/36322026/11/12 00:28:35 INFO Received uploads request method=POST path=/36332026/11/12 00:28:35 INFO Completed push id=10 objects=2 duration=1.6635ms36342026/11/12 00:28:36 INFO Vacuumed table table=pending_closures36352026/11/12 00:28:36 INFO Vacuumed table table=commit_receipts36362026/11/12 00:28:36 INFO Vacuumed table table=pending_objects36372026/11/12 00:28:36 INFO Vacuumed table table=multipart_uploads36382026/11/12 00:28:36 INFO Vacuumed table table=closures36392026/11/12 00:28:36 INFO Vacuumed table table=objects36402026/11/12 00:28:36 INFO Received complete upload request method=POST path=/36412026/11/12 00:28:36 INFO Completed upload id=11 objects=6 duration=3.422208ms36422026/11/12 00:28:36 INFO Aborted multipart uploads count=0 kept=036432026/11/12 00:28:36 WARN Force mode enabled - objects will be deleted immediately without grace period36442026/11/12 00:28:36 INFO Received uploads request method=POST path=/36452026/11/12 00:28:36 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036462026/11/12 00:28:36 INFO Vacuumed table table=pending_closures36472026/11/12 00:28:36 INFO Vacuumed table table=commit_receipts36482026/11/12 00:28:36 INFO Received uploads request method=POST path=/36492026/11/12 00:28:36 INFO Received complete push request method=POST path=/36502026/11/12 00:28:36 INFO Vacuumed table table=pending_objects36512026/11/12 00:28:36 INFO Completed push id=4 objects=6 duration=1.990916ms36522026/11/12 00:28:36 INFO Vacuumed table table=multipart_uploads36532026/11/12 00:28:36 INFO Vacuumed table table=closures36542026/11/12 00:28:36 INFO Vacuumed table table=objects36552026/11/12 00:28:36 INFO Received push request method=POST path=/36562026/11/12 00:28:36 INFO Received complete upload request method=POST path=/36572026/11/12 00:28:36 INFO Received complete push request method=POST path=/36582026/11/12 00:28:36 INFO Received complete push request method=POST path=/36592026/11/12 00:28:36 INFO Completed upload id=17 objects=5 duration=711.208µs36602026/11/12 00:28:36 INFO Received uploads request method=POST path=/36612026/11/12 00:28:36 INFO Received complete push request method=POST path=/36622026/11/12 00:28:36 INFO Completed push id=21 objects=26 duration=1.803875ms36632026/11/12 00:28:36 INFO Aborted multipart uploads count=0 kept=036642026/11/12 00:28:36 INFO Received uploads request method=POST path=/36652026/11/12 00:28:36 INFO Completed push id=16 objects=4 duration=3.277417ms36662026/11/12 00:28:36 INFO Received uploads request method=POST path=/36672026/11/12 00:28:36 INFO Completed push id=5 objects=6 duration=2.919542ms36682026/11/12 00:28:36 INFO Received complete push request method=POST path=/36692026/11/12 00:28:36 INFO Received complete upload request method=POST path=/36702026/11/12 00:28:36 WARN Force mode enabled - objects will be deleted immediately without grace period36712026/11/12 00:28:36 INFO Completed push id=2 objects=7 duration=4.860625ms36722026/11/12 00:28:36 INFO Completed upload id=10 objects=23 duration=15.287042ms36732026/11/12 00:28:36 INFO Received uploads request method=POST path=/36742026/11/12 00:28:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036752026/11/12 00:28:36 INFO Vacuumed table table=pending_closures36762026/11/12 00:28:36 INFO Vacuumed table table=commit_receipts36772026/11/12 00:28:36 INFO Vacuumed table table=pending_objects36782026/11/12 00:28:36 INFO Vacuumed table table=multipart_uploads36792026/11/12 00:28:36 INFO Vacuumed table table=closures36802026/11/12 00:28:36 INFO Vacuumed table table=objects36812026/11/12 00:28:36 INFO Received uploads request method=POST path=/36822026/11/12 00:28:36 INFO Aborted multipart uploads count=0 kept=036832026/11/12 00:28:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036842026/11/12 00:28:36 INFO Received push request method=POST path=/36852026/11/12 00:28:36 INFO Aborted multipart uploads count=0 kept=036862026/11/12 00:28:36 INFO Received push request method=POST path=/36872026/11/12 00:28:36 INFO Received complete upload request method=POST path=/36882026/11/12 00:28:36 INFO Received push request method=POST path=/36892026/11/12 00:28:36 WARN Force mode enabled - objects will be deleted immediately without grace period36902026/11/12 00:28:36 INFO Received push request method=POST path=/36912026/11/12 00:28:36 INFO Completed upload id=10 objects=6 duration=2.827709ms36922026/11/12 00:28:36 INFO Received complete push request method=POST path=/36932026/11/12 00:28:36 INFO Vacuumed table table=pending_closures36942026/11/12 00:28:36 INFO Completed push id=15 objects=7 duration=1.18025ms36952026/11/12 00:28:36 INFO Vacuumed table table=commit_receipts36962026/11/12 00:28:36 INFO Received complete push request method=POST path=/36972026/11/12 00:28:36 INFO Received complete upload request method=POST path=/36982026/11/12 00:28:36 INFO Completed upload id=20 objects=5 duration=2.109416ms36992026/11/12 00:28:36 INFO Completed push id=9 objects=4 duration=2.159875ms37002026/11/12 00:28:36 INFO Received push request method=POST path=/37012026/11/12 00:28:36 INFO Vacuumed table table=pending_objects37022026/11/12 00:28:36 INFO Vacuumed table table=multipart_uploads37032026/11/12 00:28:36 INFO Vacuumed table table=closures37042026/11/12 00:28:36 INFO Received complete push request method=POST path=/37052026/11/12 00:28:36 INFO Vacuumed table table=objects37062026/11/12 00:28:36 INFO Completed push id=19 objects=3 duration=5.181709ms37072026/11/12 00:28:36 INFO Received complete push request method=POST path=/37082026/11/12 00:28:36 INFO Completed push id=9 objects=5 duration=2.406875ms37092026/11/12 00:28:36 INFO Received complete upload request method=POST path=/37102026/11/12 00:28:36 INFO Received uploads request method=POST path=/37112026/11/12 00:28:36 INFO Completed upload id=28 objects=17 duration=736.291µs37122026/11/12 00:28:36 INFO Received complete push request method=POST path=/37132026/11/12 00:28:36 INFO Received complete upload request method=POST path=/37142026/11/12 00:28:36 INFO Aborted multipart uploads count=0 kept=037152026/11/12 00:28:36 INFO Received uploads request method=POST path=/37162026/11/12 00:28:36 INFO Received uploads request method=POST path=/37172026/11/12 00:28:36 INFO Completed upload id=23 objects=15 duration=4.447083ms37182026/11/12 00:28:36 INFO Completed push id=22 objects=3 duration=6.507291ms37192026/11/12 00:28:36 WARN Force mode enabled - objects will be deleted immediately without grace period37202026/11/12 00:28:36 INFO Received uploads request method=POST path=/37212026/11/12 00:28:36 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037222026/11/12 00:28:36 INFO Aborted multipart uploads count=0 kept=037232026/11/12 00:28:36 INFO Vacuumed table table=pending_closures37242026/11/12 00:28:36 INFO Received uploads request method=POST path=/37252026/11/12 00:28:36 INFO Vacuumed table table=commit_receipts37262026/11/12 00:28:36 WARN Force mode enabled - objects will be deleted immediately without grace period37272026/11/12 00:28:36 INFO Vacuumed table table=pending_objects37282026/11/12 00:28:36 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037292026/11/12 00:28:36 INFO Vacuumed table table=multipart_uploads37302026/11/12 00:28:36 INFO Vacuumed table table=pending_closures37312026/11/12 00:28:36 INFO Vacuumed table table=closures37322026/11/12 00:28:36 INFO Vacuumed table table=commit_receipts37332026/11/12 00:28:36 INFO Vacuumed table table=objects37342026/11/12 00:28:36 INFO Vacuumed table table=pending_objects37352026/11/12 00:28:36 INFO Vacuumed table table=multipart_uploads37362026/11/12 00:28:36 INFO Vacuumed table table=closures37372026/11/12 00:28:36 INFO Aborted multipart uploads count=0 kept=037382026/11/12 00:28:36 INFO Vacuumed table table=objects37392026/11/12 00:28:36 WARN Force mode enabled - objects will be deleted immediately without grace period37402026/11/12 00:28:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037412026/11/12 00:28:36 INFO Received uploads request method=POST path=/37422026/11/12 00:28:36 INFO Received push request method=POST path=/37432026/11/12 00:28:36 INFO Received complete push request method=POST path=/37442026/11/12 00:28:36 INFO Received uploads request method=POST path=/37452026/11/12 00:28:36 INFO Received complete upload request method=POST path=/37462026/11/12 00:28:36 INFO Vacuumed table table=pending_closures37472026/11/12 00:28:36 INFO Completed push id=29 objects=2 duration=1.056208ms37482026/11/12 00:28:36 INFO Completed upload id=15 objects=9 duration=5.624542ms37492026/11/12 00:28:36 INFO Received uploads request method=POST path=/37502026/11/12 00:28:36 INFO Received complete upload request method=POST path=/37512026/11/12 00:28:36 INFO Received push request method=POST path=/37522026/11/12 00:28:36 INFO Vacuumed table table=commit_receipts37532026/11/12 00:28:36 INFO Received complete push request method=POST path=/37542026/11/12 00:28:36 INFO Completed push id=8 objects=6 duration=1.810708ms37552026/11/12 00:28:36 INFO Completed upload id=11 objects=6 duration=1.809042ms37562026/11/12 00:28:36 INFO Vacuumed table table=pending_objects37572026/11/12 00:28:36 INFO Received push request method=POST path=/37582026/11/12 00:28:36 INFO Vacuumed table table=multipart_uploads37592026/11/12 00:28:36 INFO Vacuumed table table=closures37602026/11/12 00:28:36 INFO Vacuumed table table=objects37612026/11/12 00:28:36 INFO Received complete push request method=POST path=/37622026/11/12 00:28:36 INFO Received push request method=POST path=/37632026/11/12 00:28:36 INFO Completed push id=31 objects=6 duration=4.001292ms37642026/11/12 00:28:36 INFO Received push request method=POST path=/37652026/11/12 00:28:36 INFO Received push request method=POST path=/37662026/11/12 00:28:36 INFO Aborted multipart uploads count=0 kept=037672026/11/12 00:28:36 INFO Received push request method=POST path=/37682026/11/12 00:28:36 INFO Received complete upload request method=POST path=/37692026/11/12 00:28:36 WARN Force mode enabled - objects will be deleted immediately without grace period37702026/11/12 00:28:36 INFO Received complete push request method=POST path=/37712026/11/12 00:28:36 INFO Completed upload id=19 objects=6 duration=4.576083ms37722026/11/12 00:28:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037732026/11/12 00:28:36 INFO Received complete push request method=POST path=/37742026/11/12 00:28:36 INFO Completed push id=20 objects=4 duration=6.87475ms37752026/11/12 00:28:36 INFO Vacuumed table table=pending_closures37762026/11/12 00:28:36 INFO Completed push id=32 objects=9 duration=3.42575ms37772026/11/12 00:28:36 INFO Vacuumed table table=commit_receipts37782026/11/12 00:28:36 INFO Received complete upload request method=POST path=/37792026/11/12 00:28:36 INFO Received push request method=POST path=/37802026/11/12 00:28:36 INFO Vacuumed table table=pending_objects37812026/11/12 00:28:36 INFO Completed upload id=22 objects=2 duration=1.932709ms37822026/11/12 00:28:36 INFO Received complete push request method=POST path=/37832026/11/12 00:28:36 INFO Vacuumed table table=multipart_uploads37842026/11/12 00:28:36 INFO Completed push id=24 objects=3 duration=864.709µs37852026/11/12 00:28:36 INFO Vacuumed table table=closures37862026/11/12 00:28:36 INFO Received complete push request method=POST path=/37872026/11/12 00:28:36 INFO Completed push id=12 objects=7 duration=1.7925ms37882026/11/12 00:28:36 INFO Vacuumed table table=objects37892026/11/12 00:28:36 INFO Received complete push request method=POST path=/37902026/11/12 00:28:36 INFO Completed push id=21 objects=17 duration=5.798667ms37912026/11/12 00:28:36 INFO Aborted multipart uploads count=0 kept=037922026/11/12 00:28:36 INFO Received complete push request method=POST path=/37932026/11/12 00:28:36 INFO Completed push id=7 objects=7 duration=5.07575ms37942026/11/12 00:28:36 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=037952026/11/12 00:28:36 INFO Received push request method=POST path=/37962026/11/12 00:28:36 INFO Aborted multipart uploads count=0 kept=037972026/11/12 00:28:36 INFO Vacuumed table table=pending_closures37982026/11/12 00:28:36 INFO Vacuumed table table=commit_receipts37992026/11/12 00:28:36 INFO Received complete upload request method=POST path=/38002026/11/12 00:28:36 INFO Received complete push request method=POST path=/38012026/11/12 00:28:36 INFO Vacuumed table table=pending_objects38022026/11/12 00:28:36 INFO Aborted multipart uploads count=0 kept=038032026/11/12 00:28:36 INFO Vacuumed table table=multipart_uploads38042026/11/12 00:28:36 INFO Completed upload id=22 objects=7 duration=1.06875ms38052026/11/12 00:28:36 WARN Force mode enabled - objects will be deleted immediately without grace period38062026/11/12 00:28:36 INFO Completed push id=19 objects=7 duration=1.034334ms38072026/11/12 00:28:36 INFO Vacuumed table table=closures38082026/11/12 00:28:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038092026/11/12 00:28:36 INFO Vacuumed table table=objects38102026/11/12 00:28:36 INFO Vacuumed table table=pending_closures38112026/11/12 00:28:36 INFO Vacuumed table table=commit_receipts38122026/11/12 00:28:36 INFO Vacuumed table table=pending_objects38132026/11/12 00:28:36 INFO Received uploads request method=POST path=/38142026/11/12 00:28:36 INFO Vacuumed table table=multipart_uploads38152026/11/12 00:28:36 INFO Vacuumed table table=closures38162026/11/12 00:28:36 INFO Received complete upload request method=POST path=/38172026/11/12 00:28:36 INFO Vacuumed table table=objects38182026/11/12 00:28:36 INFO Received complete push request method=POST path=/38192026/11/12 00:28:36 INFO Completed upload id=33 objects=9 duration=1.124583ms38202026/11/12 00:28:36 INFO Completed push id=12 objects=15 duration=3.976792ms38212026/11/12 00:28:36 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=038222026/11/12 00:28:36 INFO Vacuumed table table=pending_closures38232026/11/12 00:28:36 INFO Received uploads request method=POST path=/38242026/11/12 00:28:36 INFO Vacuumed table table=commit_receipts38252026/11/12 00:28:36 INFO Received push request method=POST path=/38262026/11/12 00:28:36 INFO Aborted multipart uploads count=0 kept=038272026/11/12 00:28:36 INFO Aborted multipart uploads count=0 kept=038282026/11/12 00:28:37 WARN Force mode enabled - objects will be deleted immediately without grace period38292026/11/12 00:28:37 INFO Received push request method=POST path=/38302026/11/12 00:28:37 WARN Force mode enabled - objects will be deleted immediately without grace period38312026/11/12 00:28:37 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038322026/11/12 00:28:37 INFO Aborted multipart uploads count=0 kept=038332026/11/12 00:28:37 INFO Vacuumed table table=pending_closures38342026/11/12 00:28:37 WARN Force mode enabled - objects will be deleted immediately without grace period38352026/11/12 00:28:37 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038362026/11/12 00:28:37 INFO Vacuumed table table=commit_receipts38372026/11/12 00:28:37 INFO Vacuumed table table=pending_objects38382026/11/12 00:28:37 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038392026/11/12 00:28:37 INFO Vacuumed table table=pending_objects38402026/11/12 00:28:37 INFO Vacuumed table table=multipart_uploads38412026/11/12 00:28:37 INFO Vacuumed table table=pending_closures38422026/11/12 00:28:37 INFO Aborted multipart uploads count=0 kept=038432026/11/12 00:28:37 INFO Received complete push request method=POST path=/38442026/11/12 00:28:37 INFO Received push request method=POST path=/38452026/11/12 00:28:37 INFO Vacuumed table table=multipart_uploads38462026/11/12 00:28:37 INFO Vacuumed table table=commit_receipts38472026/11/12 00:28:37 INFO Completed push id=24 objects=9 duration=1.465917ms38482026/11/12 00:28:37 INFO Vacuumed table table=closures38492026/11/12 00:28:37 INFO Received complete upload request method=POST path=/38502026/11/12 00:28:37 WARN Force mode enabled - objects will be deleted immediately without grace period38512026/11/12 00:28:37 INFO Vacuumed table table=pending_closures38522026/11/12 00:28:37 INFO Vacuumed table table=closures38532026/11/12 00:28:37 INFO Vacuumed table table=objects38542026/11/12 00:28:37 INFO Vacuumed table table=pending_objects38552026/11/12 00:28:37 INFO Completed upload id=22 objects=8 duration=1.537167ms38562026/11/12 00:28:37 INFO Vacuumed table table=commit_receipts38572026/11/12 00:28:37 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038582026/11/12 00:28:37 INFO Vacuumed table table=objects38592026/11/12 00:28:37 INFO Received push request method=POST path=/38602026/11/12 00:28:37 INFO Vacuumed table table=multipart_uploads38612026/11/12 00:28:37 INFO Vacuumed table table=pending_objects38622026/11/12 00:28:37 INFO Vacuumed table table=pending_closures38632026/11/12 00:28:37 INFO Vacuumed table table=multipart_uploads38642026/11/12 00:28:37 INFO Vacuumed table table=commit_receipts38652026/11/12 00:28:37 INFO Vacuumed table table=closures38662026/11/12 00:28:37 INFO Received push request method=POST path=/38672026/11/12 00:28:37 INFO Vacuumed table table=closures38682026/11/12 00:28:37 INFO Vacuumed table table=pending_objects38692026/11/12 00:28:37 INFO Vacuumed table table=objects38702026/11/12 00:28:37 INFO Vacuumed table table=objects38712026/11/12 00:28:37 INFO Received complete push request method=POST path=/38722026/11/12 00:28:37 INFO Vacuumed table table=multipart_uploads38732026/11/12 00:28:37 INFO Received push request method=POST path=/38742026/11/12 00:28:37 INFO Completed push id=23 objects=5 duration=4.464084ms38752026/11/12 00:28:37 INFO Aborted multipart uploads count=0 kept=038762026/11/12 00:28:37 INFO Vacuumed table table=closures38772026/11/12 00:28:37 INFO Vacuumed table table=objects38782026/11/12 00:28:37 INFO Received complete upload request method=POST path=/38792026/11/12 00:28:37 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038802026/11/12 00:28:37 INFO Received push request method=POST path=/38812026/11/12 00:28:37 INFO Completed upload id=17 objects=25 duration=8.080292ms38822026/11/12 00:28:37 INFO Vacuumed table table=pending_closures38832026/11/12 00:28:37 INFO Vacuumed table table=commit_receipts38842026/11/12 00:28:37 INFO Vacuumed table table=pending_objects38852026/11/12 00:28:37 INFO Vacuumed table table=multipart_uploads38862026/11/12 00:28:37 INFO Vacuumed table table=closures38872026/11/12 00:28:37 INFO Received uploads request method=POST path=/38882026/11/12 00:28:37 INFO Received complete upload request method=POST path=/38892026/11/12 00:28:37 INFO Vacuumed table table=objects38902026/11/12 00:28:37 INFO Aborted multipart uploads count=0 kept=038912026/11/12 00:28:37 INFO Completed upload id=3 objects=24 duration=2.414625ms38922026/11/12 00:28:37 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038932026/11/12 00:28:37 INFO Received uploads request method=POST path=/38942026/11/12 00:28:37 INFO Received complete upload request method=POST path=/38952026/11/12 00:28:37 INFO Received uploads request method=POST path=/38962026/11/12 00:28:37 INFO Received push request method=POST path=/38972026/11/12 00:28:37 INFO Vacuumed table table=pending_closures38982026/11/12 00:28:37 INFO Completed upload id=24 objects=8 duration=4.933791ms38992026/11/12 00:28:37 INFO Received uploads request method=POST path=/39002026/11/12 00:28:37 INFO Received push request method=POST path=/39012026/11/12 00:28:37 INFO Vacuumed table table=commit_receipts39022026/11/12 00:28:37 INFO Received complete upload request method=POST path=/39032026/11/12 00:28:37 INFO Received uploads request method=POST path=/39042026/11/12 00:28:37 INFO Completed upload id=18 objects=11 duration=2.100166ms39052026/11/12 00:28:37 INFO Aborted multipart uploads count=0 kept=039062026/11/12 00:28:37 INFO Vacuumed table table=pending_objects39072026/11/12 00:28:37 INFO Vacuumed table table=multipart_uploads39082026/11/12 00:28:37 INFO Received complete upload request method=POST path=/39092026/11/12 00:28:37 INFO Vacuumed table table=closures39102026/11/12 00:28:37 INFO Received uploads request method=POST path=/39112026/11/12 00:28:37 INFO Received uploads request method=POST path=/39122026/11/12 00:28:37 INFO Completed upload id=28 objects=5 duration=2.058959ms39132026/11/12 00:28:37 INFO Vacuumed table table=objects39142026/11/12 00:28:37 INFO Received complete upload request method=POST path=/39152026/11/12 00:28:37 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=039162026/11/12 00:28:37 INFO Received uploads request method=POST path=/39172026/11/12 00:28:37 INFO Completed upload id=21 objects=2 duration=3.820667ms39182026/11/12 00:28:37 INFO Received push request method=POST path=/39192026/11/12 00:28:37 INFO Received uploads request method=POST path=/39202026/11/12 00:28:37 INFO Vacuumed table table=pending_closures39212026/11/12 00:28:37 INFO Received push request method=POST path=/39222026/11/12 00:28:37 INFO Vacuumed table table=commit_receipts39232026/11/12 00:28:37 INFO Received push request method=POST path=/39242026/11/12 00:28:37 INFO Aborted multipart uploads count=0 kept=039252026/11/12 00:28:37 WARN Force mode enabled - objects will be deleted immediately without grace period39262026/11/12 00:28:37 INFO Vacuumed table table=pending_objects39272026/11/12 00:28:37 INFO Received push request method=POST path=/39282026/11/12 00:28:37 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039292026/11/12 00:28:37 INFO Vacuumed table table=multipart_uploads39302026/11/12 00:28:37 INFO Vacuumed table table=pending_closures39312026/11/12 00:28:37 INFO Vacuumed table table=closures39322026/11/12 00:28:37 INFO Aborted multipart uploads count=0 kept=039332026/11/12 00:28:37 INFO Vacuumed table table=commit_receipts39342026/11/12 00:28:37 INFO Vacuumed table table=objects39352026/11/12 00:28:37 INFO Received complete push request method=POST path=/39362026/11/12 00:28:37 WARN Force mode enabled - objects will be deleted immediately without grace period39372026/11/12 00:28:37 INFO Received push request method=POST path=/39382026/11/12 00:28:37 INFO Vacuumed table table=pending_objects39392026/11/12 00:28:37 INFO Completed push id=21 objects=3 duration=1.417958ms39402026/11/12 00:28:37 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039412026/11/12 00:28:37 INFO Vacuumed table table=multipart_uploads39422026/11/12 00:28:37 INFO Vacuumed table table=pending_closures39432026/11/12 00:28:37 INFO Received uploads request method=POST path=/39442026/11/12 00:28:37 INFO Received complete upload request method=POST path=/39452026/11/12 00:28:37 INFO Completed upload id=15 objects=6 duration=852.75µs39462026/11/12 00:28:37 INFO Vacuumed table table=closures39472026/11/12 00:28:37 INFO Vacuumed table table=commit_receipts39482026/11/12 00:28:37 INFO Received complete upload request method=POST path=/39492026/11/12 00:28:37 INFO Vacuumed table table=objects39502026/11/12 00:28:37 INFO Completed upload id=30 objects=5 duration=697.333µs39512026/11/12 00:28:37 INFO Vacuumed table table=pending_objects39522026/11/12 00:28:37 INFO Vacuumed table table=multipart_uploads39532026/11/12 00:28:37 INFO Vacuumed table table=closures39542026/11/12 00:28:37 INFO Vacuumed table table=objects39552026/11/12 00:28:37 INFO Received uploads request method=POST path=/39562026/11/12 00:28:37 INFO Received complete upload request method=POST path=/39572026/11/12 00:28:37 INFO Aborted multipart uploads count=0 kept=039582026/11/12 00:28:37 INFO Completed upload id=12 objects=26 duration=1.204875ms39592026/11/12 00:28:37 WARN Force mode enabled - objects will be deleted immediately without grace period39602026/11/12 00:28:37 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=039612026/11/12 00:28:37 INFO Received push request method=POST path=/39622026/11/12 00:28:37 INFO Vacuumed table table=pending_closures39632026/11/12 00:28:37 INFO Vacuumed table table=commit_receipts39642026/11/12 00:28:37 INFO Vacuumed table table=pending_objects39652026/11/12 00:28:37 INFO Vacuumed table table=multipart_uploads39662026/11/12 00:28:37 INFO Vacuumed table table=closures39672026/11/12 00:28:37 INFO Vacuumed table table=objects39682026/11/12 00:28:37 INFO Aborted multipart uploads count=0 kept=039692026/11/12 00:28:37 INFO Received push request method=POST path=/39702026/11/12 00:28:37 WARN Force mode enabled - objects will be deleted immediately without grace period39712026/11/12 00:28:37 INFO Received uploads request method=POST path=/39722026/11/12 00:28:37 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039732026/11/12 00:28:37 INFO Vacuumed table table=pending_closures39742026/11/12 00:28:37 INFO Vacuumed table table=commit_receipts39752026/11/12 00:28:37 INFO Received uploads request method=POST path=/39762026/11/12 00:28:37 INFO Received push request method=POST path=/39772026/11/12 00:28:37 INFO Vacuumed table table=pending_objects39782026/11/12 00:28:37 INFO Received uploads request method=POST path=/39792026/11/12 00:28:37 INFO Vacuumed table table=multipart_uploads39802026/11/12 00:28:37 INFO Vacuumed table table=closures39812026/11/12 00:28:37 INFO Vacuumed table table=objects39822026/11/12 00:28:38 INFO Aborted multipart uploads count=0 kept=039832026/11/12 00:28:38 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=3 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039842026/11/12 00:28:38 INFO Vacuumed table table=pending_closures39852026/11/12 00:28:38 INFO Vacuumed table table=commit_receipts39862026/11/12 00:28:38 INFO Vacuumed table table=pending_objects39872026/11/12 00:28:38 INFO Vacuumed table table=multipart_uploads39882026/11/12 00:28:38 INFO Vacuumed table table=closures39892026/11/12 00:28:38 INFO Received uploads request method=POST path=/39902026/11/12 00:28:38 INFO Vacuumed table table=objects39912026/11/12 00:28:38 INFO Received complete push request method=POST path=/39922026/11/12 00:28:38 INFO Received push request method=POST path=/39932026/11/12 00:28:38 INFO Completed push id=16 objects=8 duration=978.084µs39942026/11/12 00:28:38 INFO Received complete upload request method=POST path=/39952026/11/12 00:28:38 INFO Completed upload id=26 objects=2 duration=966.875µs39962026/11/12 00:28:38 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."39972026/11/12 00:28:38 INFO Received complete push request method=POST path=/39982026/11/12 00:28:38 INFO Completed push id=31 objects=3 duration=1.857ms39992026/11/12 00:28:38 INFO Received push request method=POST path=/40002026/11/12 00:28:38 INFO Aborted multipart uploads count=0 kept=040012026/11/12 00:28:38 WARN Force mode enabled - objects will be deleted immediately without grace period40022026/11/12 00:28:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040032026/11/12 00:28:38 INFO Vacuumed table table=pending_closures40042026/11/12 00:28:38 INFO Vacuumed table table=commit_receipts40052026/11/12 00:28:38 INFO Vacuumed table table=pending_objects40062026/11/12 00:28:38 INFO Received complete upload request method=POST path=/40072026/11/12 00:28:38 INFO Received uploads request method=POST path=/40082026/11/12 00:28:38 INFO Vacuumed table table=multipart_uploads40092026/11/12 00:28:38 INFO Completed upload id=4 objects=14 duration=7.29125ms40102026/11/12 00:28:38 INFO Vacuumed table table=closures40112026/11/12 00:28:38 INFO Received complete push request method=POST path=/40122026/11/12 00:28:38 INFO Completed push id=31 objects=14 duration=6.286959ms40132026/11/12 00:28:38 INFO Aborted multipart uploads count=0 kept=040142026/11/12 00:28:38 INFO Vacuumed table table=objects40152026/11/12 00:28:38 WARN Force mode enabled - objects will be deleted immediately without grace period40162026/11/12 00:28:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040172026/11/12 00:28:38 INFO Vacuumed table table=pending_closures40182026/11/12 00:28:38 INFO Aborted multipart uploads count=0 kept=040192026/11/12 00:28:38 INFO Vacuumed table table=commit_receipts40202026/11/12 00:28:38 INFO Vacuumed table table=pending_objects40212026/11/12 00:28:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040222026/11/12 00:28:38 INFO Vacuumed table table=multipart_uploads40232026/11/12 00:28:38 INFO Vacuumed table table=closures40242026/11/12 00:28:38 INFO Received push request method=POST path=/40252026/11/12 00:28:38 INFO Vacuumed table table=objects40262026/11/12 00:28:38 INFO Aborted multipart uploads count=0 kept=040272026/11/12 00:28:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040282026/11/12 00:28:38 INFO Vacuumed table table=pending_closures40292026/11/12 00:28:38 INFO Vacuumed table table=commit_receipts40302026/11/12 00:28:38 INFO Vacuumed table table=pending_objects40312026/11/12 00:28:38 INFO Vacuumed table table=pending_closures40322026/11/12 00:28:38 INFO Vacuumed table table=multipart_uploads40332026/11/12 00:28:38 INFO Vacuumed table table=commit_receipts40342026/11/12 00:28:38 INFO Vacuumed table table=closures40352026/11/12 00:28:38 INFO Vacuumed table table=objects40362026/11/12 00:28:38 INFO Vacuumed table table=pending_objects40372026/11/12 00:28:38 INFO Vacuumed table table=multipart_uploads40382026/11/12 00:28:38 INFO Vacuumed table table=closures40392026/11/12 00:28:38 INFO Vacuumed table table=objects40402026/11/12 00:28:38 INFO Received complete push request method=POST path=/40412026/11/12 00:28:38 INFO Received complete push request method=POST path=/40422026/11/12 00:28:38 INFO Aborted multipart uploads count=0 kept=040432026/11/12 00:28:38 INFO Completed push id=19 objects=8 duration=3.869833ms40442026/11/12 00:28:38 INFO Completed push id=7 objects=24 duration=3.628416ms40452026/11/12 00:28:38 INFO Received push request method=POST path=/40462026/11/12 00:28:38 WARN Force mode enabled - objects will be deleted immediately without grace period40472026/11/12 00:28:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040482026/11/12 00:28:38 INFO Vacuumed table table=pending_closures40492026/11/12 00:28:38 INFO Vacuumed table table=commit_receipts40502026/11/12 00:28:38 INFO Received complete push request method=POST path=/40512026/11/12 00:28:38 INFO Vacuumed table table=pending_objects40522026/11/12 00:28:38 INFO Received push request method=POST path=/40532026/11/12 00:28:38 INFO Completed push id=16 objects=27 duration=1.404917ms40542026/11/12 00:28:38 INFO Received complete push request method=POST path=/40552026/11/12 00:28:38 INFO Vacuumed table table=multipart_uploads40562026/11/12 00:28:38 INFO Completed push id=32 objects=6 duration=10.187125ms40572026/11/12 00:28:38 INFO Received complete push request method=POST path=/40582026/11/12 00:28:38 INFO Vacuumed table table=closures40592026/11/12 00:28:38 INFO Vacuumed table table=objects40602026/11/12 00:28:38 INFO Completed push id=15 objects=4 duration=881.833µs40612026/11/12 00:28:38 INFO Received uploads request method=POST path=/40622026/11/12 00:28:38 INFO Received complete push request method=POST path=/40632026/11/12 00:28:38 INFO Completed push id=11 objects=6 duration=2.581875ms40642026/11/12 00:28:38 INFO Received push request method=POST path=/40652026/11/12 00:28:38 INFO Received uploads request method=POST path=/40662026/11/12 00:28:38 INFO Received complete push request method=POST path=/40672026/11/12 00:28:38 INFO Received complete push request method=POST path=/40682026/11/12 00:28:38 INFO Completed push id=8 objects=6 duration=2.519167ms40692026/11/12 00:28:38 INFO Completed push id=7 objects=20 duration=6.209291ms40702026/11/12 00:28:38 INFO Received complete push request method=POST path=/40712026/11/12 00:28:38 INFO Received uploads request method=POST path=/40722026/11/12 00:28:38 INFO Received uploads request method=POST path=/40732026/11/12 00:28:38 INFO Received complete upload request method=POST path=/40742026/11/12 00:28:38 INFO Received uploads request method=POST path=/40752026/11/12 00:28:38 INFO Completed upload id=33 objects=7 duration=3.156625ms40762026/11/12 00:28:38 INFO Aborted multipart uploads count=0 kept=040772026/11/12 00:28:38 INFO Completed push id=30 objects=4 duration=7.830166ms40782026/11/12 00:28:38 INFO Received push request method=POST path=/40792026/11/12 00:28:38 INFO Received complete push request method=POST path=/40802026/11/12 00:28:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040812026/11/12 00:28:38 INFO Vacuumed table table=pending_closures40822026/11/12 00:28:38 INFO Completed push id=15 objects=24 duration=8.456ms40832026/11/12 00:28:38 INFO Vacuumed table table=commit_receipts40842026/11/12 00:28:38 INFO Received complete push request method=POST path=/40852026/11/12 00:28:38 INFO Vacuumed table table=pending_objects40862026/11/12 00:28:38 INFO Vacuumed table table=multipart_uploads40872026/11/12 00:28:38 INFO Completed push id=17 objects=22 duration=24.053167ms40882026/11/12 00:28:38 INFO Received complete upload request method=POST path=/40892026/11/12 00:28:38 INFO Received push request method=POST path=/40902026/11/12 00:28:38 INFO Completed upload id=34 objects=3 duration=406.333µs40912026/11/12 00:28:38 INFO Vacuumed table table=closures40922026/11/12 00:28:38 INFO Vacuumed table table=objects40932026/11/12 00:28:38 INFO Received uploads request method=POST path=/40942026/11/12 00:28:38 INFO Received push request method=POST path=/40952026/11/12 00:28:38 INFO Received complete push request method=POST path=/40962026/11/12 00:28:38 INFO Received uploads request method=POST path=/40972026/11/12 00:28:38 INFO Completed push id=18 objects=6 duration=2.978167ms40982026/11/12 00:28:38 INFO Received uploads request method=POST path=/40992026/11/12 00:28:38 INFO Aborted multipart uploads count=0 kept=041002026/11/12 00:28:38 INFO Received push request method=POST path=/41012026/11/12 00:28:38 INFO Aborted multipart uploads count=0 kept=041022026/11/12 00:28:38 WARN Force mode enabled - objects will be deleted immediately without grace period41032026/11/12 00:28:38 INFO Received uploads request method=POST path=/41042026/11/12 00:28:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041052026/11/12 00:28:38 INFO Vacuumed table table=pending_closures41062026/11/12 00:28:38 INFO Vacuumed table table=commit_receipts41072026/11/12 00:28:38 INFO Aborted multipart uploads count=0 kept=041082026/11/12 00:28:38 INFO Vacuumed table table=pending_objects41092026/11/12 00:28:38 INFO Vacuumed table table=multipart_uploads41102026/11/12 00:28:38 INFO Vacuumed table table=closures41112026/11/12 00:28:38 INFO Vacuumed table table=objects41122026/11/12 00:28:38 INFO Aborted multipart uploads count=0 kept=041132026/11/12 00:28:38 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041142026/11/12 00:28:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041152026/11/12 00:28:38 INFO Vacuumed table table=pending_closures41162026/11/12 00:28:38 INFO Received complete push request method=POST path=/41172026/11/12 00:28:38 INFO Vacuumed table table=commit_receipts41182026/11/12 00:28:38 INFO Completed push id=35 objects=11 duration=2.416917ms41192026/11/12 00:28:38 INFO Vacuumed table table=pending_closures41202026/11/12 00:28:38 INFO Vacuumed table table=commit_receipts41212026/11/12 00:28:38 INFO Vacuumed table table=pending_objects41222026/11/12 00:28:38 INFO Vacuumed table table=multipart_uploads41232026/11/12 00:28:38 INFO Vacuumed table table=pending_objects41242026/11/12 00:28:38 INFO Vacuumed table table=multipart_uploads41252026/11/12 00:28:38 INFO Aborted multipart uploads count=0 kept=041262026/11/12 00:28:38 INFO Vacuumed table table=closures41272026/11/12 00:28:38 INFO Vacuumed table table=closures41282026/11/12 00:28:38 INFO Vacuumed table table=objects41292026/11/12 00:28:38 INFO Vacuumed table table=objects41302026/11/12 00:28:38 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."41312026/11/12 00:28:38 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."41322026/11/12 00:28:38 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."41332026/11/12 00:28:38 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041342026/11/12 00:28:38 INFO Vacuumed table table=pending_closures41352026/11/12 00:28:38 INFO Vacuumed table table=commit_receipts41362026/11/12 00:28:38 INFO Vacuumed table table=pending_objects41372026/11/12 00:28:38 INFO Vacuumed table table=multipart_uploads41382026/11/12 00:28:38 INFO Received uploads request method=POST path=/41392026/11/12 00:28:38 INFO Vacuumed table table=closures41402026/11/12 00:28:38 INFO Vacuumed table table=objects41412026/11/12 00:28:38 INFO Received push request method=POST path=/41422026/11/12 00:28:38 INFO Received push request method=POST path=/41432026/11/12 00:28:38 INFO Aborted multipart uploads count=0 kept=041442026/11/12 00:28:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041452026/11/12 00:28:38 INFO Vacuumed table table=pending_closures41462026/11/12 00:28:38 INFO Vacuumed table table=commit_receipts41472026/11/12 00:28:38 INFO Vacuumed table table=pending_objects41482026/11/12 00:28:38 INFO Vacuumed table table=multipart_uploads41492026/11/12 00:28:38 INFO Received push request method=POST path=/41502026/11/12 00:28:38 INFO Vacuumed table table=closures41512026/11/12 00:28:38 INFO Vacuumed table table=objects41522026/11/12 00:28:38 INFO Received uploads request method=POST path=/41532026/11/12 00:28:38 INFO Aborted multipart uploads count=0 kept=041542026/11/12 00:28:38 WARN Force mode enabled - objects will be deleted immediately without grace period41552026/11/12 00:28:38 INFO Garbage collection completed failed-uploads-deleted=6 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041562026/11/12 00:28:38 INFO Vacuumed table table=pending_closures41572026/11/12 00:28:38 INFO Received push request method=POST path=/41582026/11/12 00:28:38 INFO Vacuumed table table=commit_receipts41592026/11/12 00:28:38 INFO Received uploads request method=POST path=/41602026/11/12 00:28:38 INFO Received complete push request method=POST path=/41612026/11/12 00:28:38 INFO Received uploads request method=POST path=/41622026/11/12 00:28:38 INFO Received complete upload request method=POST path=/41632026/11/12 00:28:38 INFO Received push request method=POST path=/41642026/11/12 00:28:38 INFO Completed push id=5 objects=3 duration=2.984375ms41652026/11/12 00:28:38 INFO Vacuumed table table=pending_objects41662026/11/12 00:28:38 INFO Completed upload id=20 objects=3 duration=2.976375ms41672026/11/12 00:28:38 INFO Received complete push request method=POST path=/41682026/11/12 00:28:38 INFO Received complete push request method=POST path=/41692026/11/12 00:28:38 INFO Vacuumed table table=multipart_uploads41702026/11/12 00:28:38 INFO Received complete push request method=POST path=/41712026/11/12 00:28:39 INFO Completed push id=5 objects=8 duration=7.349416ms41722026/11/12 00:28:39 INFO Received push request method=POST path=/41732026/11/12 00:28:39 INFO Received uploads request method=POST path=/41742026/11/12 00:28:39 INFO Completed push id=7 objects=7 duration=10.393792ms41752026/11/12 00:28:39 INFO Vacuumed table table=closures41762026/11/12 00:28:39 INFO Received complete push request method=POST path=/41772026/11/12 00:28:39 INFO Completed push id=18 objects=10 duration=14.68875ms41782026/11/12 00:28:39 INFO Vacuumed table table=objects41792026/11/12 00:28:39 INFO Completed push id=32 objects=7 duration=1.564042ms41802026/11/12 00:28:39 INFO Received uploads request method=POST path=/41812026/11/12 00:28:39 INFO Received push request method=POST path=/41822026/11/12 00:28:39 INFO Received push request method=POST path=/41832026/11/12 00:28:39 INFO Received complete upload request method=POST path=/41842026/11/12 00:28:39 INFO Received complete push request method=POST path=/41852026/11/12 00:28:39 INFO Received complete push request method=POST path=/41862026/11/12 00:28:39 INFO Received uploads request method=POST path=/41872026/11/12 00:28:39 INFO Completed upload id=33 objects=5 duration=2.161333ms41882026/11/12 00:28:39 INFO Completed push id=12 objects=6 duration=735.25µs41892026/11/12 00:28:39 INFO Completed push id=28 objects=11 duration=1.84575ms41902026/11/12 00:28:39 INFO Received complete upload request method=POST path=/41912026/11/12 00:28:39 INFO Completed upload id=6 objects=6 duration=991.416µs41922026/11/12 00:28:39 INFO Received complete push request method=POST path=/41932026/11/12 00:28:39 INFO Completed push id=14 objects=3 duration=1.349709ms41942026/11/12 00:28:39 INFO Received complete push request method=POST path=/41952026/11/12 00:28:39 INFO Completed push id=26 objects=4 duration=1.491167ms41962026/11/12 00:28:39 INFO Received complete push request method=POST path=/41972026/11/12 00:28:39 INFO Completed push id=13 objects=2 duration=1.628125ms41982026/11/12 00:28:39 INFO Received uploads request method=POST path=/41992026/11/12 00:28:39 INFO Received complete upload request method=POST path=/42002026/11/12 00:28:39 INFO Completed upload id=28 objects=13 duration=815.042µs42012026/11/12 00:28:39 INFO Received push request method=POST path=/42022026/11/12 00:28:39 INFO Aborted multipart uploads count=0 kept=042032026/11/12 00:28:39 WARN Force mode enabled - objects will be deleted immediately without grace period42042026/11/12 00:28:39 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042052026/11/12 00:28:39 INFO Vacuumed table table=pending_closures42062026/11/12 00:28:39 INFO Vacuumed table table=commit_receipts42072026/11/12 00:28:39 INFO Vacuumed table table=pending_objects42082026/11/12 00:28:39 INFO Received push request method=POST path=/42092026/11/12 00:28:39 INFO Vacuumed table table=multipart_uploads42102026/11/12 00:28:39 INFO Aborted multipart uploads count=0 kept=042112026/11/12 00:28:39 INFO Received complete upload request method=POST path=/42122026/11/12 00:28:39 INFO Completed upload id=27 objects=13 duration=1.359417ms42132026/11/12 00:28:39 INFO Vacuumed table table=closures42142026/11/12 00:28:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042152026/11/12 00:28:39 INFO Vacuumed table table=objects42162026/11/12 00:28:39 INFO Vacuumed table table=pending_closures42172026/11/12 00:28:39 INFO Vacuumed table table=commit_receipts42182026/11/12 00:28:39 INFO Vacuumed table table=pending_objects42192026/11/12 00:28:39 INFO Vacuumed table table=multipart_uploads42202026/11/12 00:28:39 INFO Received complete upload request method=POST path=/42212026/11/12 00:28:39 INFO Received uploads request method=POST path=/42222026/11/12 00:28:39 INFO Vacuumed table table=closures42232026/11/12 00:28:39 INFO Received complete upload request method=POST path=/42242026/11/12 00:28:39 INFO Completed upload id=24 objects=7 duration=582.416µs42252026/11/12 00:28:39 INFO Vacuumed table table=objects42262026/11/12 00:28:39 INFO Completed upload id=20 objects=4 duration=2.473542ms42272026/11/12 00:28:39 INFO Received complete push request method=POST path=/42282026/11/12 00:28:39 INFO Completed push id=8 objects=6 duration=668.125µs42292026/11/12 00:28:39 INFO Aborted multipart uploads count=0 kept=042302026/11/12 00:28:39 INFO Received complete push request method=POST path=/42312026/11/12 00:28:39 INFO Completed push id=29 objects=3 duration=3.323791ms42322026/11/12 00:28:39 WARN Force mode enabled - objects will be deleted immediately without grace period42332026/11/12 00:28:39 INFO Aborted multipart uploads count=0 kept=042342026/11/12 00:28:39 INFO Garbage collection completed failed-uploads-deleted=5 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042352026/11/12 00:28:39 INFO Received push request method=POST path=/42362026/11/12 00:28:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042372026/11/12 00:28:39 INFO Vacuumed table table=pending_closures42382026/11/12 00:28:39 INFO Vacuumed table table=commit_receipts42392026/11/12 00:28:39 INFO Vacuumed table table=pending_objects42402026/11/12 00:28:39 INFO Vacuumed table table=multipart_uploads42412026/11/12 00:28:39 INFO Vacuumed table table=pending_closures42422026/11/12 00:28:39 INFO Vacuumed table table=closures42432026/11/12 00:28:39 INFO Aborted multipart uploads count=0 kept=042442026/11/12 00:28:39 INFO Vacuumed table table=objects42452026/11/12 00:28:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042462026/11/12 00:28:39 INFO Vacuumed table table=commit_receipts42472026/11/12 00:28:39 INFO Vacuumed table table=pending_closures42482026/11/12 00:28:39 INFO Vacuumed table table=commit_receipts42492026/11/12 00:28:39 INFO Vacuumed table table=pending_objects42502026/11/12 00:28:39 INFO Vacuumed table table=pending_objects42512026/11/12 00:28:39 INFO Vacuumed table table=multipart_uploads42522026/11/12 00:28:39 INFO Received complete upload request method=POST path=/42532026/11/12 00:28:39 INFO Completed upload id=31 objects=2 duration=7.059833ms42542026/11/12 00:28:39 INFO Vacuumed table table=closures42552026/11/12 00:28:39 INFO Vacuumed table table=multipart_uploads42562026/11/12 00:28:39 INFO Received uploads request method=POST path=/42572026/11/12 00:28:39 INFO Vacuumed table table=objects42582026/11/12 00:28:39 INFO Vacuumed table table=closures42592026/11/12 00:28:39 INFO Received uploads request method=POST path=/42602026/11/12 00:28:39 INFO Received uploads request method=POST path=/42612026/11/12 00:28:39 INFO Received push request method=POST path=/42622026/11/12 00:28:39 INFO Vacuumed table table=objects42632026/11/12 00:28:39 INFO Received complete push request method=POST path=/42642026/11/12 00:28:39 INFO Received complete push request method=POST path=/42652026/11/12 00:28:39 INFO Completed push id=29 objects=24 duration=6.048584ms42662026/11/12 00:28:39 INFO Received complete push request method=POST path=/42672026/11/12 00:28:39 INFO Completed push id=33 objects=6 duration=775.5µs42682026/11/12 00:28:39 INFO Completed push id=9 objects=24 duration=1.734084ms42692026/11/12 00:28:39 INFO Received uploads request method=POST path=/42702026/11/12 00:28:39 INFO Received uploads request method=POST path=/42712026/11/12 00:28:39 INFO Received uploads request method=POST path=/42722026/11/12 00:28:39 INFO Received push request method=POST path=/42732026/11/12 00:28:39 INFO Aborted multipart uploads count=0 kept=042742026/11/12 00:28:39 INFO Aborted multipart uploads count=0 kept=042752026/11/12 00:28:39 INFO Aborted multipart uploads count=0 kept=042762026/11/12 00:28:39 INFO Aborted multipart uploads count=0 kept=042772026/11/12 00:28:39 INFO Received uploads request method=POST path=/42782026/11/12 00:28:39 WARN Force mode enabled - objects will be deleted immediately without grace period42792026/11/12 00:28:39 WARN Force mode enabled - objects will be deleted immediately without grace period42802026/11/12 00:28:39 WARN Force mode enabled - objects will be deleted immediately without grace period42812026/11/12 00:28:39 WARN Force mode enabled - objects will be deleted immediately without grace period42822026/11/12 00:28:39 INFO Received push request method=POST path=/42832026/11/12 00:28:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042842026/11/12 00:28:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042852026/11/12 00:28:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042862026/11/12 00:28:39 INFO Vacuumed table table=pending_closures42872026/11/12 00:28:39 INFO Vacuumed table table=pending_closures42882026/11/12 00:28:39 INFO Vacuumed table table=commit_receipts42892026/11/12 00:28:39 INFO Received push request method=POST path=/42902026/11/12 00:28:39 INFO Vacuumed table table=commit_receipts42912026/11/12 00:28:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=042922026/11/12 00:28:39 INFO Vacuumed table table=pending_closures42932026/11/12 00:28:39 INFO Received push request method=POST path=/42942026/11/12 00:28:39 INFO Vacuumed table table=pending_objects42952026/11/12 00:28:39 INFO Received uploads request method=POST path=/42962026/11/12 00:28:39 INFO Received complete upload request method=POST path=/42972026/11/12 00:28:39 INFO Received complete push request method=POST path=/42982026/11/12 00:28:39 INFO Vacuumed table table=commit_receipts42992026/11/12 00:28:39 INFO Vacuumed table table=multipart_uploads43002026/11/12 00:28:39 INFO Vacuumed table table=pending_objects43012026/11/12 00:28:39 INFO Completed upload id=8 objects=3 duration=2.937459ms43022026/11/12 00:28:39 INFO Completed push id=16 objects=5 duration=2.672ms43032026/11/12 00:28:39 INFO Vacuumed table table=closures43042026/11/12 00:28:39 INFO Vacuumed table table=pending_closures43052026/11/12 00:28:39 INFO Vacuumed table table=pending_objects43062026/11/12 00:28:39 INFO Vacuumed table table=multipart_uploads43072026/11/12 00:28:39 INFO Vacuumed table table=commit_receipts43082026/11/12 00:28:39 INFO Vacuumed table table=multipart_uploads43092026/11/12 00:28:39 INFO Vacuumed table table=closures43102026/11/12 00:28:39 INFO Received complete push request method=POST path=/43112026/11/12 00:28:39 INFO Received push request method=POST path=/43122026/11/12 00:28:39 INFO Received push request method=POST path=/43132026/11/12 00:28:39 INFO Vacuumed table table=objects43142026/11/12 00:28:39 INFO Completed push id=13 objects=22 duration=4.779375ms43152026/11/12 00:28:39 INFO Vacuumed table table=objects43162026/11/12 00:28:39 INFO Vacuumed table table=closures43172026/11/12 00:28:39 INFO Received complete upload request method=POST path=/43182026/11/12 00:28:39 INFO Vacuumed table table=pending_objects43192026/11/12 00:28:39 INFO Received uploads request method=POST path=/43202026/11/12 00:28:39 INFO Received complete upload request method=POST path=/43212026/11/12 00:28:39 INFO Vacuumed table table=objects43222026/11/12 00:28:39 INFO Vacuumed table table=multipart_uploads43232026/11/12 00:28:39 INFO Completed upload id=10 objects=10 duration=1.534583ms43242026/11/12 00:28:39 INFO Completed upload id=7 objects=10 duration=2.728209ms43252026/11/12 00:28:39 INFO Received uploads request method=POST path=/43262026/11/12 00:28:39 INFO Received uploads request method=POST path=/43272026/11/12 00:28:39 INFO Received complete upload request method=POST path=/43282026/11/12 00:28:39 INFO Vacuumed table table=closures43292026/11/12 00:28:39 INFO Completed upload id=21 objects=25 duration=5.712667ms43302026/11/12 00:28:39 INFO Vacuumed table table=objects43312026/11/12 00:28:39 INFO Received complete push request method=POST path=/43322026/11/12 00:28:39 INFO Completed push id=21 objects=6 duration=907.208µs43332026/11/12 00:28:39 INFO Received push request method=POST path=/43342026/11/12 00:28:39 INFO Received push request method=POST path=/43352026/11/12 00:28:39 INFO Received uploads request method=POST path=/43362026/11/12 00:28:39 INFO Received uploads request method=POST path=/43372026/11/12 00:28:39 INFO Aborted multipart uploads count=0 kept=043382026/11/12 00:28:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=043392026/11/12 00:28:39 INFO Vacuumed table table=pending_closures43402026/11/12 00:28:39 INFO Vacuumed table table=commit_receipts43412026/11/12 00:28:39 INFO Vacuumed table table=pending_objects43422026/11/12 00:28:39 INFO Vacuumed table table=multipart_uploads43432026/11/12 00:28:39 INFO Vacuumed table table=closures43442026/11/12 00:28:39 INFO Vacuumed table table=objects43452026/11/12 00:28:39 INFO Received complete push request method=POST path=/43462026/11/12 00:28:39 INFO Completed push id=23 objects=28 duration=3.352042ms43472026/11/12 00:28:39 INFO Received uploads request method=POST path=/43482026/11/12 00:28:39 INFO Received push request method=POST path=/43492026/11/12 00:28:39 INFO Received complete upload request method=POST path=/43502026/11/12 00:28:39 INFO Completed upload id=22 objects=7 duration=1.294042ms43512026/11/12 00:28:39 INFO Received complete upload request method=POST path=/43522026/11/12 00:28:39 INFO Received uploads request method=POST path=/43532026/11/12 00:28:39 INFO Received complete upload request method=POST path=/43542026/11/12 00:28:39 INFO Completed upload id=29 objects=4 duration=1.073875ms43552026/11/12 00:28:39 INFO Completed upload id=10 objects=17 duration=621.542µs43562026/11/12 00:28:39 INFO Received push request method=POST path=/43572026/11/12 00:28:39 INFO Received complete push request method=POST path=/43582026/11/12 00:28:39 INFO Completed push id=24 objects=4 duration=4.036708ms43592026/11/12 00:28:39 INFO Received complete upload request method=POST path=/43602026/11/12 00:28:39 INFO Received complete push request method=POST path=/43612026/11/12 00:28:39 INFO Completed push id=6 objects=6 duration=2.542542ms43622026/11/12 00:28:39 INFO Completed upload id=14 objects=16 duration=3.249334ms43632026/11/12 00:28:39 INFO Aborted multipart uploads count=0 kept=043642026/11/12 00:28:39 INFO Aborted multipart uploads count=0 kept=043652026/11/12 00:28:39 INFO Aborted multipart uploads count=0 kept=043662026/11/12 00:28:39 INFO Received uploads request method=POST path=/43672026/11/12 00:28:39 INFO Received complete upload request method=POST path=/43682026/11/12 00:28:39 WARN Force mode enabled - objects will be deleted immediately without grace period43692026/11/12 00:28:39 INFO Completed upload id=27 objects=8 duration=669.25µs43702026/11/12 00:28:39 INFO Received push request method=POST path=/43712026/11/12 00:28:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043722026/11/12 00:28:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043732026/11/12 00:28:39 INFO Vacuumed table table=pending_closures43742026/11/12 00:28:39 INFO Vacuumed table table=pending_closures43752026/11/12 00:28:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043762026/11/12 00:28:39 INFO Vacuumed table table=commit_receipts43772026/11/12 00:28:39 INFO Vacuumed table table=commit_receipts43782026/11/12 00:28:39 INFO Vacuumed table table=pending_closures43792026/11/12 00:28:39 INFO Vacuumed table table=commit_receipts43802026/11/12 00:28:39 INFO Vacuumed table table=pending_objects43812026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads43822026/11/12 00:28:40 INFO Vacuumed table table=pending_objects43832026/11/12 00:28:40 INFO Vacuumed table table=pending_objects43842026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads43852026/11/12 00:28:40 INFO Received complete push request method=POST path=/43862026/11/12 00:28:40 INFO Vacuumed table table=closures43872026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads43882026/11/12 00:28:40 INFO Vacuumed table table=closures43892026/11/12 00:28:40 INFO Completed push id=25 objects=9 duration=999.583µs43902026/11/12 00:28:40 INFO Vacuumed table table=objects43912026/11/12 00:28:40 INFO Vacuumed table table=closures43922026/11/12 00:28:40 INFO Vacuumed table table=objects43932026/11/12 00:28:40 INFO Vacuumed table table=objects43942026/11/12 00:28:40 INFO Received push request method=POST path=/43952026/11/12 00:28:40 INFO Received push request method=POST path=/43962026/11/12 00:28:40 INFO Received push request method=POST path=/43972026/11/12 00:28:40 INFO Received uploads request method=POST path=/43982026/11/12 00:28:40 INFO Received complete upload request method=POST path=/43992026/11/12 00:28:40 INFO Received uploads request method=POST path=/44002026/11/12 00:28:40 INFO Completed upload id=26 objects=3 duration=1.088708ms44012026/11/12 00:28:40 INFO Aborted multipart uploads count=0 kept=044022026/11/12 00:28:40 INFO Received push request method=POST path=/44032026/11/12 00:28:40 INFO Received uploads request method=POST path=/44042026/11/12 00:28:40 WARN Force mode enabled - objects will be deleted immediately without grace period44052026/11/12 00:28:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044062026/11/12 00:28:40 INFO Received uploads request method=POST path=/44072026/11/12 00:28:40 INFO Vacuumed table table=pending_closures44082026/11/12 00:28:40 INFO Vacuumed table table=commit_receipts44092026/11/12 00:28:40 INFO Received uploads request method=POST path=/44102026/11/12 00:28:40 INFO Vacuumed table table=pending_objects44112026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads44122026/11/12 00:28:40 INFO Vacuumed table table=closures44132026/11/12 00:28:40 INFO Vacuumed table table=objects44142026/11/12 00:28:40 INFO Received complete upload request method=POST path=/44152026/11/12 00:28:40 INFO Completed upload id=12 objects=6 duration=1.04275ms44162026/11/12 00:28:40 INFO Received complete push request method=POST path=/44172026/11/12 00:28:40 INFO Completed push id=14 objects=6 duration=4.728083ms44182026/11/12 00:28:40 INFO Received push request method=POST path=/44192026/11/12 00:28:40 INFO Aborted multipart uploads count=0 kept=044202026/11/12 00:28:40 INFO Received complete push request method=POST path=/44212026/11/12 00:28:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044222026/11/12 00:28:40 INFO Vacuumed table table=pending_closures44232026/11/12 00:28:40 INFO Completed push id=11 objects=6 duration=5.068459ms44242026/11/12 00:28:40 INFO Vacuumed table table=commit_receipts44252026/11/12 00:28:40 INFO Received complete upload request method=POST path=/44262026/11/12 00:28:40 INFO Vacuumed table table=pending_objects44272026/11/12 00:28:40 INFO Completed upload id=27 objects=7 duration=535.833µs44282026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads44292026/11/12 00:28:40 INFO Vacuumed table table=closures44302026/11/12 00:28:40 INFO Vacuumed table table=objects44312026/11/12 00:28:40 INFO Received complete push request method=POST path=/44322026/11/12 00:28:40 INFO Completed push id=14 objects=3 duration=2.043209ms44332026/11/12 00:28:40 INFO Received complete push request method=POST path=/44342026/11/12 00:28:40 INFO Completed push id=17 objects=3 duration=984.5µs44352026/11/12 00:28:40 INFO Received push request method=POST path=/44362026/11/12 00:28:40 INFO Received push request method=POST path=/44372026/11/12 00:28:40 INFO Aborted multipart uploads count=0 kept=044382026/11/12 00:28:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044392026/11/12 00:28:40 INFO Received complete push request method=POST path=/44402026/11/12 00:28:40 INFO Vacuumed table table=pending_closures44412026/11/12 00:28:40 INFO Vacuumed table table=commit_receipts44422026/11/12 00:28:40 INFO Completed push id=10 objects=15 duration=3.078375ms44432026/11/12 00:28:40 INFO Vacuumed table table=pending_objects44442026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads44452026/11/12 00:28:40 INFO Received complete upload request method=POST path=/44462026/11/12 00:28:40 INFO Vacuumed table table=closures44472026/11/12 00:28:40 INFO Completed upload id=28 objects=3 duration=6.597667ms44482026/11/12 00:28:40 INFO Received uploads request method=POST path=/44492026/11/12 00:28:40 INFO Received complete push request method=POST path=/44502026/11/12 00:28:40 INFO Aborted multipart uploads count=0 kept=044512026/11/12 00:28:40 INFO Vacuumed table table=objects44522026/11/12 00:28:40 INFO Completed push id=17 objects=11 duration=1.065584ms44532026/11/12 00:28:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044542026/11/12 00:28:40 INFO Vacuumed table table=pending_closures44552026/11/12 00:28:40 INFO Vacuumed table table=commit_receipts44562026/11/12 00:28:40 INFO Received push request method=POST path=/44572026/11/12 00:28:40 INFO Vacuumed table table=pending_objects44582026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads44592026/11/12 00:28:40 INFO Vacuumed table table=closures44602026/11/12 00:28:40 INFO Vacuumed table table=objects44612026/11/12 00:28:40 INFO Received push request method=POST path=/44622026/11/12 00:28:40 INFO Received complete push request method=POST path=/44632026/11/12 00:28:40 INFO Completed push id=20 objects=13 duration=860.292µs44642026/11/12 00:28:40 INFO Received complete push request method=POST path=/44652026/11/12 00:28:40 INFO Aborted multipart uploads count=0 kept=044662026/11/12 00:28:40 INFO Completed push id=23 objects=5 duration=934.167µs44672026/11/12 00:28:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044682026/11/12 00:28:40 INFO Aborted multipart uploads count=0 kept=044692026/11/12 00:28:40 INFO Vacuumed table table=pending_closures44702026/11/12 00:28:40 INFO Vacuumed table table=commit_receipts44712026/11/12 00:28:40 INFO Received push request method=POST path=/44722026/11/12 00:28:40 INFO Vacuumed table table=pending_objects44732026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads44742026/11/12 00:28:40 INFO Vacuumed table table=closures44752026/11/12 00:28:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=044762026/11/12 00:28:40 INFO Vacuumed table table=objects44772026/11/12 00:28:40 INFO Vacuumed table table=pending_closures44782026/11/12 00:28:40 INFO Vacuumed table table=commit_receipts44792026/11/12 00:28:40 INFO Vacuumed table table=pending_objects44802026/11/12 00:28:40 INFO Received complete push request method=POST path=/44812026/11/12 00:28:40 INFO Received complete upload request method=POST path=/44822026/11/12 00:28:40 INFO Received complete upload request method=POST path=/44832026/11/12 00:28:40 INFO Completed push id=22 objects=4 duration=713.667µs44842026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads44852026/11/12 00:28:40 INFO Completed upload id=11 objects=7 duration=416.125µs44862026/11/12 00:28:40 INFO Completed upload id=25 objects=16 duration=1.886834ms44872026/11/12 00:28:40 INFO Vacuumed table table=closures44882026/11/12 00:28:40 INFO Vacuumed table table=objects44892026/11/12 00:28:40 INFO Received uploads request method=POST path=/44902026/11/12 00:28:40 INFO Received complete push request method=POST path=/44912026/11/12 00:28:40 INFO Completed push id=29 objects=23 duration=1.14975ms44922026/11/12 00:28:40 INFO Received complete upload request method=POST path=/44932026/11/12 00:28:40 INFO Completed upload id=24 objects=3 duration=794.084µs44942026/11/12 00:28:40 INFO Received push request method=POST path=/44952026/11/12 00:28:40 INFO Received uploads request method=POST path=/44962026/11/12 00:28:40 INFO Received push request method=POST path=/44972026/11/12 00:28:40 INFO Received complete push request method=POST path=/44982026/11/12 00:28:40 INFO Received complete upload request method=POST path=/44992026/11/12 00:28:40 INFO Completed push id=30 objects=6 duration=914.625µs45002026/11/12 00:28:40 INFO Aborted multipart uploads count=0 kept=045012026/11/12 00:28:40 WARN Force mode enabled - objects will be deleted immediately without grace period45022026/11/12 00:28:40 INFO Completed upload id=23 objects=18 duration=2.254ms45032026/11/12 00:28:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045042026/11/12 00:28:40 INFO Vacuumed table table=pending_closures45052026/11/12 00:28:40 INFO Received complete push request method=POST path=/45062026/11/12 00:28:40 INFO Vacuumed table table=commit_receipts45072026/11/12 00:28:40 INFO Vacuumed table table=pending_objects45082026/11/12 00:28:40 INFO Completed push id=25 objects=4 duration=4.971042ms45092026/11/12 00:28:40 INFO Received uploads request method=POST path=/45102026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads45112026/11/12 00:28:40 INFO Vacuumed table table=closures45122026/11/12 00:28:40 INFO Vacuumed table table=objects45132026/11/12 00:28:40 INFO Received complete upload request method=POST path=/45142026/11/12 00:28:40 INFO Completed upload id=23 objects=7 duration=1.266625ms45152026/11/12 00:28:40 INFO Received complete upload request method=POST path=/45162026/11/12 00:28:40 INFO Completed upload id=23 objects=16 duration=3.433125ms45172026/11/12 00:28:40 INFO Aborted multipart uploads count=0 kept=045182026/11/12 00:28:40 INFO Received push request method=POST path=/45192026/11/12 00:28:40 WARN Force mode enabled - objects will be deleted immediately without grace period45202026/11/12 00:28:40 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045212026/11/12 00:28:40 INFO Vacuumed table table=pending_closures45222026/11/12 00:28:40 INFO Vacuumed table table=commit_receipts45232026/11/12 00:28:40 INFO Vacuumed table table=pending_objects45242026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads45252026/11/12 00:28:40 INFO Vacuumed table table=closures45262026/11/12 00:28:40 INFO Vacuumed table table=objects45272026/11/12 00:28:40 INFO Received complete upload request method=POST path=/45282026/11/12 00:28:40 INFO Completed upload id=26 objects=6 duration=650µs45292026/11/12 00:28:40 INFO Received push request method=POST path=/45302026/11/12 00:28:40 INFO Received uploads request method=POST path=/45312026/11/12 00:28:40 INFO Received push request method=POST path=/45322026/11/12 00:28:40 INFO Aborted multipart uploads count=0 kept=045332026/11/12 00:28:40 WARN Force mode enabled - objects will be deleted immediately without grace period45342026/11/12 00:28:40 INFO Received push request method=POST path=/45352026/11/12 00:28:40 INFO Aborted multipart uploads count=0 kept=045362026/11/12 00:28:40 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=045372026/11/12 00:28:40 INFO Received complete push request method=POST path=/45382026/11/12 00:28:40 WARN Force mode enabled - objects will be deleted immediately without grace period45392026/11/12 00:28:40 INFO Completed push id=24 objects=5 duration=638.042µs45402026/11/12 00:28:40 INFO Received complete push request method=POST path=/45412026/11/12 00:28:40 INFO Vacuumed table table=pending_closures45422026/11/12 00:28:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045432026/11/12 00:28:40 INFO Received uploads request method=POST path=/45442026/11/12 00:28:40 INFO Completed push id=33 objects=3 duration=628.5µs45452026/11/12 00:28:40 INFO Vacuumed table table=commit_receipts45462026/11/12 00:28:40 INFO Vacuumed table table=pending_closures45472026/11/12 00:28:40 INFO Vacuumed table table=commit_receipts45482026/11/12 00:28:40 INFO Vacuumed table table=pending_objects45492026/11/12 00:28:40 INFO Vacuumed table table=pending_objects45502026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads45512026/11/12 00:28:40 INFO Received uploads request method=POST path=/45522026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads45532026/11/12 00:28:40 INFO Vacuumed table table=closures45542026/11/12 00:28:40 INFO Vacuumed table table=closures45552026/11/12 00:28:40 INFO Vacuumed table table=objects45562026/11/12 00:28:40 INFO Received push request method=POST path=/45572026/11/12 00:28:40 INFO Vacuumed table table=objects45582026/11/12 00:28:40 INFO Received push request method=POST path=/45592026/11/12 00:28:40 INFO Received push request method=POST path=/45602026/11/12 00:28:40 INFO Received complete upload request method=POST path=/45612026/11/12 00:28:40 INFO Completed upload id=29 objects=5 duration=538.333µs45622026/11/12 00:28:40 INFO Received complete push request method=POST path=/45632026/11/12 00:28:40 INFO Completed push id=34 objects=6 duration=3.671167ms45642026/11/12 00:28:40 INFO Received uploads request method=POST path=/45652026/11/12 00:28:40 INFO Received complete upload request method=POST path=/45662026/11/12 00:28:40 INFO Completed upload id=25 objects=2 duration=809µs45672026/11/12 00:28:40 INFO Received push request method=POST path=/45682026/11/12 00:28:40 INFO Received complete upload request method=POST path=/45692026/11/12 00:28:40 INFO Completed upload id=22 objects=22 duration=686.666µs45702026/11/12 00:28:40 INFO Received uploads request method=POST path=/45712026/11/12 00:28:40 INFO Received uploads request method=POST path=/45722026/11/12 00:28:40 INFO Received uploads request method=POST path=/45732026/11/12 00:28:40 INFO Received complete push request method=POST path=/45742026/11/12 00:28:40 INFO Completed push id=35 objects=8 duration=819.583µs45752026/11/12 00:28:40 INFO Received uploads request method=POST path=/45762026/11/12 00:28:40 INFO Received complete upload request method=POST path=/45772026/11/12 00:28:40 INFO Completed upload id=26 objects=20 duration=2.820417ms45782026/11/12 00:28:40 INFO Received push request method=POST path=/45792026/11/12 00:28:40 INFO Received push request method=POST path=/45802026/11/12 00:28:40 INFO Received complete push request method=POST path=/45812026/11/12 00:28:40 INFO Completed push id=12 objects=2 duration=2.527708ms45822026/11/12 00:28:40 INFO Received complete upload request method=POST path=/45832026/11/12 00:28:40 INFO Completed upload id=36 objects=9 duration=997.584µs45842026/11/12 00:28:40 INFO Aborted multipart uploads count=0 kept=045852026/11/12 00:28:40 WARN Force mode enabled - objects will be deleted immediately without grace period45862026/11/12 00:28:40 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045872026/11/12 00:28:40 INFO Vacuumed table table=pending_closures45882026/11/12 00:28:40 INFO Vacuumed table table=commit_receipts45892026/11/12 00:28:40 INFO Vacuumed table table=pending_objects45902026/11/12 00:28:40 INFO Aborted multipart uploads count=0 kept=045912026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads45922026/11/12 00:28:40 INFO Vacuumed table table=closures45932026/11/12 00:28:40 INFO Vacuumed table table=objects45942026/11/12 00:28:40 INFO Received uploads request method=POST path=/45952026/11/12 00:28:40 WARN Force mode enabled - objects will be deleted immediately without grace period45962026/11/12 00:28:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045972026/11/12 00:28:40 INFO Vacuumed table table=pending_closures45982026/11/12 00:28:40 INFO Vacuumed table table=commit_receipts45992026/11/12 00:28:40 INFO Received complete push request method=POST path=/46002026/11/12 00:28:40 INFO Completed push id=26 objects=5 duration=1.242833ms46012026/11/12 00:28:40 INFO Vacuumed table table=pending_objects46022026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads46032026/11/12 00:28:40 INFO Received uploads request method=POST path=/46042026/11/12 00:28:40 INFO Received complete push request method=POST path=/46052026/11/12 00:28:40 INFO Vacuumed table table=closures46062026/11/12 00:28:40 INFO Completed push id=26 objects=23 duration=3.959208ms46072026/11/12 00:28:40 INFO Vacuumed table table=objects46082026/11/12 00:28:40 INFO Received uploads request method=POST path=/46092026/11/12 00:28:40 INFO Aborted multipart uploads count=0 kept=046102026/11/12 00:28:40 WARN Force mode enabled - objects will be deleted immediately without grace period46112026/11/12 00:28:40 INFO Received complete push request method=POST path=/46122026/11/12 00:28:40 INFO Completed push id=28 objects=7 duration=762.125µs46132026/11/12 00:28:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046142026/11/12 00:28:40 INFO Vacuumed table table=pending_closures46152026/11/12 00:28:40 INFO Vacuumed table table=commit_receipts46162026/11/12 00:28:40 INFO Vacuumed table table=pending_objects46172026/11/12 00:28:40 INFO Vacuumed table table=multipart_uploads46182026/11/12 00:28:40 INFO Vacuumed table table=closures46192026/11/12 00:28:40 INFO Received push request method=POST path=/46202026/11/12 00:28:40 INFO Vacuumed table table=objects46212026/11/12 00:28:40 INFO Aborted multipart uploads count=0 kept=046222026/11/12 00:28:40 INFO Received complete upload request method=POST path=/46232026/11/12 00:28:40 INFO Aborted multipart uploads count=0 kept=046242026/11/12 00:28:40 INFO Received complete upload request method=POST path=/46252026/11/12 00:28:40 INFO Completed upload id=24 objects=14 duration=425.125µs46262026/11/12 00:28:40 WARN Force mode enabled - objects will be deleted immediately without grace period46272026/11/12 00:28:40 INFO Completed upload id=22 objects=5 duration=2.543708ms46282026/11/12 00:28:40 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046292026/11/12 00:28:40 INFO Vacuumed table table=pending_closures46302026/11/12 00:28:40 INFO Vacuumed table table=commit_receipts46312026/11/12 00:28:40 INFO Vacuumed table table=pending_objects46322026/11/12 00:28:41 INFO Vacuumed table table=multipart_uploads46332026/11/12 00:28:41 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."46342026/11/12 00:28:41 INFO Vacuumed table table=closures46352026/11/12 00:28:41 INFO Vacuumed table table=objects46362026/11/12 00:28:41 INFO Received uploads request method=POST path=/46372026/11/12 00:28:41 INFO Received uploads request method=POST path=/46382026/11/12 00:28:41 INFO Received complete push request method=POST path=/46392026/11/12 00:28:41 INFO Received complete push request method=POST path=/46402026/11/12 00:28:41 INFO Completed push id=19 objects=3 duration=1.562666ms46412026/11/12 00:28:41 INFO Received complete upload request method=POST path=/46422026/11/12 00:28:41 INFO Completed push id=28 objects=3 duration=2.284541ms46432026/11/12 00:28:41 INFO Completed upload id=30 objects=6 duration=1.008708ms46442026/11/12 00:28:41 INFO Received uploads request method=POST path=/46452026/11/12 00:28:41 INFO Aborted multipart uploads count=0 kept=046462026/11/12 00:28:41 WARN Force mode enabled - objects will be deleted immediately without grace period46472026/11/12 00:28:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046482026/11/12 00:28:41 INFO Vacuumed table table=pending_closures46492026/11/12 00:28:41 INFO Vacuumed table table=commit_receipts46502026/11/12 00:28:41 INFO Aborted multipart uploads count=0 kept=046512026/11/12 00:28:41 INFO Received uploads request method=POST path=/46522026/11/12 00:28:41 INFO Vacuumed table table=pending_objects46532026/11/12 00:28:41 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046542026/11/12 00:28:41 INFO Vacuumed table table=multipart_uploads46552026/11/12 00:28:41 INFO Vacuumed table table=pending_closures46562026/11/12 00:28:41 INFO Vacuumed table table=closures46572026/11/12 00:28:41 INFO Vacuumed table table=commit_receipts46582026/11/12 00:28:41 INFO Vacuumed table table=objects46592026/11/12 00:28:41 INFO Vacuumed table table=pending_objects46602026/11/12 00:28:41 INFO Vacuumed table table=multipart_uploads46612026/11/12 00:28:41 INFO Vacuumed table table=closures46622026/11/12 00:28:41 INFO Vacuumed table table=objects46632026/11/12 00:28:41 INFO Received complete push request method=POST path=/46642026/11/12 00:28:41 INFO Received complete push request method=POST path=/46652026/11/12 00:28:41 INFO Completed push id=29 objects=15 duration=765.958µs46662026/11/12 00:28:41 INFO Completed push id=31 objects=5 duration=943.75µs46672026/11/12 00:28:41 INFO Received complete upload request method=POST path=/46682026/11/12 00:28:41 INFO Received complete upload request method=POST path=/46692026/11/12 00:28:41 INFO Received push request method=POST path=/46702026/11/12 00:28:41 INFO Completed upload id=19 objects=2 duration=5.745416ms46712026/11/12 00:28:41 INFO Completed upload id=14 objects=8 duration=5.947417ms46722026/11/12 00:28:41 INFO Received uploads request method=POST path=/46732026/11/12 00:28:41 INFO Received complete upload request method=POST path=/46742026/11/12 00:28:41 INFO Received uploads request method=POST path=/46752026/11/12 00:28:41 INFO Completed upload id=31 objects=3 duration=4.062417ms46762026/11/12 00:28:41 INFO Received complete push request method=POST path=/46772026/11/12 00:28:41 INFO Completed push id=27 objects=15 duration=2.010542ms46782026/11/12 00:28:41 INFO Received complete push request method=POST path=/46792026/11/12 00:28:41 INFO Received uploads request method=POST path=/46802026/11/12 00:28:41 INFO Completed push id=30 objects=6 duration=825.042µs46812026/11/12 00:28:41 INFO Received uploads request method=POST path=/46822026/11/12 00:28:41 INFO Received complete push request method=POST path=/46832026/11/12 00:28:41 INFO Received complete upload request method=POST path=/46842026/11/12 00:28:41 INFO Completed push id=33 objects=7 duration=4.403541ms46852026/11/12 00:28:41 INFO Completed upload id=31 objects=3 duration=4.250291ms46862026/11/12 00:28:41 INFO Received complete upload request method=POST path=/46872026/11/12 00:28:41 INFO Aborted multipart uploads count=0 kept=046882026/11/12 00:28:41 INFO Completed upload id=22 objects=2 duration=2.634417ms46892026/11/12 00:28:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046902026/11/12 00:28:41 INFO Vacuumed table table=pending_closures46912026/11/12 00:28:41 INFO Vacuumed table table=commit_receipts46922026/11/12 00:28:41 INFO Vacuumed table table=pending_objects46932026/11/12 00:28:41 INFO Vacuumed table table=multipart_uploads46942026/11/12 00:28:41 INFO Vacuumed table table=closures46952026/11/12 00:28:41 INFO Received complete push request method=POST path=/46962026/11/12 00:28:41 INFO Vacuumed table table=objects46972026/11/12 00:28:41 INFO Completed push id=32 objects=26 duration=2.016834ms46982026/11/12 00:28:41 INFO Received complete upload request method=POST path=/46992026/11/12 00:28:41 INFO Completed upload id=26 objects=3 duration=1.914125ms47002026/11/12 00:28:41 INFO Received uploads request method=POST path=/47012026/11/12 00:28:41 INFO Received uploads request method=POST path=/47022026/11/12 00:28:41 INFO Aborted multipart uploads count=0 kept=047032026/11/12 00:28:41 INFO Aborted multipart uploads count=0 kept=047042026/11/12 00:28:41 WARN Force mode enabled - objects will be deleted immediately without grace period47052026/11/12 00:28:41 WARN Force mode enabled - objects will be deleted immediately without grace period47062026/11/12 00:28:41 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047072026/11/12 00:28:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047082026/11/12 00:28:41 INFO Vacuumed table table=pending_closures47092026/11/12 00:28:41 INFO Received complete push request method=POST path=/47102026/11/12 00:28:41 INFO Vacuumed table table=commit_receipts47112026/11/12 00:28:41 INFO Completed push id=22 objects=3 duration=5.60825ms47122026/11/12 00:28:41 INFO Vacuumed table table=pending_closures47132026/11/12 00:28:41 INFO Received complete upload request method=POST path=/47142026/11/12 00:28:41 INFO Vacuumed table table=pending_objects47152026/11/12 00:28:41 INFO Vacuumed table table=multipart_uploads47162026/11/12 00:28:41 INFO Completed upload id=33 objects=2 duration=882.583µs47172026/11/12 00:28:41 INFO Vacuumed table table=commit_receipts47182026/11/12 00:28:41 INFO Aborted multipart uploads count=0 kept=047192026/11/12 00:28:41 INFO Received uploads request method=POST path=/47202026/11/12 00:28:41 INFO Received complete upload request method=POST path=/47212026/11/12 00:28:41 INFO Vacuumed table table=closures47222026/11/12 00:28:41 INFO Completed upload id=25 objects=4 duration=1.067917ms47232026/11/12 00:28:41 WARN Force mode enabled - objects will be deleted immediately without grace period47242026/11/12 00:28:41 INFO Vacuumed table table=pending_objects47252026/11/12 00:28:41 INFO Vacuumed table table=objects47262026/11/12 00:28:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047272026/11/12 00:28:41 INFO Vacuumed table table=multipart_uploads47282026/11/12 00:28:41 INFO Vacuumed table table=pending_closures47292026/11/12 00:28:41 INFO Vacuumed table table=commit_receipts47302026/11/12 00:28:41 INFO Vacuumed table table=closures47312026/11/12 00:28:41 INFO Vacuumed table table=pending_objects47322026/11/12 00:28:41 INFO Vacuumed table table=objects47332026/11/12 00:28:41 INFO Vacuumed table table=multipart_uploads47342026/11/12 00:28:41 INFO Aborted multipart uploads count=0 kept=047352026/11/12 00:28:41 INFO Vacuumed table table=closures47362026/11/12 00:28:41 INFO Vacuumed table table=objects47372026/11/12 00:28:41 INFO Received complete upload request method=POST path=/47382026/11/12 00:28:41 WARN Force mode enabled - objects will be deleted immediately without grace period47392026/11/12 00:28:41 INFO Completed upload id=34 objects=5 duration=1.343917ms47402026/11/12 00:28:41 INFO Received complete upload request method=POST path=/47412026/11/12 00:28:41 INFO Received push request method=POST path=/47422026/11/12 00:28:41 INFO Received complete push request method=POST path=/47432026/11/12 00:28:41 INFO Completed upload id=18 objects=2 duration=2.655666ms47442026/11/12 00:28:41 INFO Completed push id=18 objects=6 duration=3.419167ms47452026/11/12 00:28:41 INFO Aborted multipart uploads count=0 kept=047462026/11/12 00:28:41 INFO Received complete upload request method=POST path=/47472026/11/12 00:28:41 WARN Force mode enabled - objects will be deleted immediately without grace period47482026/11/12 00:28:41 INFO Completed upload id=35 objects=11 duration=1.312416ms47492026/11/12 00:28:41 INFO Received push request method=POST path=/47502026/11/12 00:28:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=047512026/11/12 00:28:41 INFO Received complete upload request method=POST path=/47522026/11/12 00:28:41 INFO Completed upload id=23 objects=9 duration=1.205208ms47532026/11/12 00:28:41 INFO Vacuumed table table=pending_closures47542026/11/12 00:28:41 INFO Vacuumed table table=commit_receipts47552026/11/12 00:28:41 INFO Received complete upload request method=POST path=/47562026/11/12 00:28:41 INFO Completed upload id=36 objects=3 duration=2.609ms47572026/11/12 00:28:41 INFO Vacuumed table table=pending_objects47582026/11/12 00:28:41 INFO Vacuumed table table=multipart_uploads47592026/11/12 00:28:41 INFO Vacuumed table table=closures47602026/11/12 00:28:41 INFO Vacuumed table table=objects47612026/11/12 00:28:41 INFO Received complete push request method=POST path=/47622026/11/12 00:28:41 INFO Completed push id=20 objects=8 duration=1.501667ms47632026/11/12 00:28:41 INFO Received complete push request method=POST path=/47642026/11/12 00:28:41 INFO Completed push id=28 objects=19 duration=3.48175ms47652026/11/12 00:28:41 INFO Aborted multipart uploads count=0 kept=047662026/11/12 00:28:41 WARN Force mode enabled - objects will be deleted immediately without grace period47672026/11/12 00:28:41 INFO Received complete push request method=POST path=/47682026/11/12 00:28:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047692026/11/12 00:28:41 INFO Received uploads request method=POST path=/47702026/11/12 00:28:41 INFO Completed push id=7 objects=15 duration=3.772917ms47712026/11/12 00:28:41 INFO Received push request method=POST path=/47722026/11/12 00:28:41 INFO Vacuumed table table=pending_closures47732026/11/12 00:28:41 INFO Received complete push request method=POST path=/47742026/11/12 00:28:41 INFO Vacuumed table table=commit_receipts47752026/11/12 00:28:41 INFO Vacuumed table table=pending_objects47762026/11/12 00:28:41 INFO Completed push id=13 objects=15 duration=11.54725ms47772026/11/12 00:28:41 INFO Vacuumed table table=multipart_uploads47782026/11/12 00:28:41 INFO Vacuumed table table=closures47792026/11/12 00:28:41 INFO Received uploads request method=POST path=/47802026/11/12 00:28:41 INFO Vacuumed table table=objects47812026/11/12 00:28:41 INFO Received push request method=POST path=/47822026/11/12 00:28:41 INFO Received complete upload request method=POST path=/47832026/11/12 00:28:41 INFO Completed upload id=13 objects=7 duration=3.285458ms47842026/11/12 00:28:41 INFO Aborted multipart uploads count=0 kept=047852026/11/12 00:28:41 INFO Received uploads request method=POST path=/47862026/11/12 00:28:41 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."47872026/11/12 00:28:41 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."47882026/11/12 00:28:41 INFO Received complete upload request method=POST path=/47892026/11/12 00:28:41 INFO Received push request method=POST path=/47902026/11/12 00:28:41 INFO Received complete push request method=POST path=/47912026/11/12 00:28:41 INFO Completed upload id=20 objects=4 duration=743.583µs47922026/11/12 00:28:41 INFO Completed push id=29 objects=10 duration=1.065792ms47932026/11/12 00:28:41 INFO Received uploads request method=POST path=/47942026/11/12 00:28:41 INFO Received complete upload request method=POST path=/47952026/11/12 00:28:41 INFO Completed upload id=16 objects=17 duration=4.862542ms47962026/11/12 00:28:41 INFO Received push request method=POST path=/47972026/11/12 00:28:41 INFO Received push request method=POST path=/47982026/11/12 00:28:41 INFO Received complete upload request method=POST path=/47992026/11/12 00:28:41 INFO Completed upload id=23 objects=7 duration=836.709µs48002026/11/12 00:28:41 INFO Received complete push request method=POST path=/48012026/11/12 00:28:41 INFO Completed push id=27 objects=11 duration=498.959µs48022026/11/12 00:28:41 INFO Aborted multipart uploads count=0 kept=048032026/11/12 00:28:41 WARN Force mode enabled - objects will be deleted immediately without grace period48042026/11/12 00:28:42 INFO Received complete upload request method=POST path=/48052026/11/12 00:28:42 INFO Completed upload id=30 objects=6 duration=1.119667ms48062026/11/12 00:28:42 INFO Received complete push request method=POST path=/48072026/11/12 00:28:42 INFO Completed push id=20 objects=3 duration=1.762833ms48082026/11/12 00:28:42 INFO Received uploads request method=POST path=/48092026/11/12 00:28:42 INFO Received complete upload request method=POST path=/48102026/11/12 00:28:42 INFO Received complete push request method=POST path=/48112026/11/12 00:28:42 INFO Completed upload id=29 objects=26 duration=718.167µs48122026/11/12 00:28:42 INFO Completed push id=16 objects=26 duration=1.016125ms48132026/11/12 00:28:42 INFO Received complete push request method=POST path=/48142026/11/12 00:28:42 INFO Completed push id=9 objects=9 duration=4.6565ms48152026/11/12 00:28:42 INFO Received complete upload request method=POST path=/48162026/11/12 00:28:42 INFO Completed upload id=31 objects=5 duration=852.25µs48172026/11/12 00:28:42 INFO Received complete push request method=POST path=/48182026/11/12 00:28:42 INFO Completed push id=30 objects=8 duration=7.192042ms48192026/11/12 00:28:42 INFO Aborted multipart uploads count=0 kept=048202026/11/12 00:28:42 WARN Force mode enabled - objects will be deleted immediately without grace period48212026/11/12 00:28:42 INFO Received complete upload request method=POST path=/48222026/11/12 00:28:42 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048232026/11/12 00:28:42 INFO Completed upload id=34 objects=3 duration=5.309125ms48242026/11/12 00:28:42 INFO Vacuumed table table=pending_closures48252026/11/12 00:28:42 INFO Vacuumed table table=commit_receipts48262026/11/12 00:28:42 INFO Aborted multipart uploads count=0 kept=048272026/11/12 00:28:42 INFO Vacuumed table table=pending_objects48282026/11/12 00:28:42 INFO Vacuumed table table=multipart_uploads48292026/11/12 00:28:42 INFO Received complete upload request method=POST path=/48302026/11/12 00:28:42 INFO Vacuumed table table=closures48312026/11/12 00:28:42 INFO Received complete upload request method=POST path=/48322026/11/12 00:28:42 INFO Completed upload id=31 objects=5 duration=472.125µs48332026/11/12 00:28:42 INFO Vacuumed table table=objects48342026/11/12 00:28:42 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."48352026/11/12 00:28:42 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."48362026/11/12 00:28:42 INFO Completed upload id=21 objects=8 duration=1.9155ms48372026/11/12 00:28:42 INFO Aborted multipart uploads count=0 kept=048382026/11/12 00:28:42 WARN Force mode enabled - objects will be deleted immediately without grace period48392026/11/12 00:28:42 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048402026/11/12 00:28:42 INFO Received push request method=POST path=/48412026/11/12 00:28:42 INFO Vacuumed table table=pending_closures48422026/11/12 00:28:42 INFO Vacuumed table table=commit_receipts48432026/11/12 00:28:42 INFO Vacuumed table table=pending_objects48442026/11/12 00:28:42 INFO Received complete push request method=POST path=/48452026/11/12 00:28:42 INFO Vacuumed table table=multipart_uploads48462026/11/12 00:28:42 INFO Received complete push request method=POST path=/48472026/11/12 00:28:42 INFO Received complete push request method=POST path=/48482026/11/12 00:28:42 INFO Vacuumed table table=closures48492026/11/12 00:28:42 INFO Completed push id=27 objects=6 duration=7.635416ms48502026/11/12 00:28:42 INFO Completed push id=32 objects=3 duration=7.808209ms48512026/11/12 00:28:42 INFO Completed push id=22 objects=6 duration=7.767875ms48522026/11/12 00:28:42 INFO Vacuumed table table=objects48532026/11/12 00:28:42 INFO Received push request method=POST path=/48542026/11/12 00:28:42 INFO Aborted multipart uploads count=0 kept=048552026/11/12 00:28:42 WARN Force mode enabled - objects will be deleted immediately without grace period48562026/11/12 00:28:42 INFO Received complete push request method=POST path=/48572026/11/12 00:28:42 INFO Aborted multipart uploads count=0 kept=048582026/11/12 00:28:42 INFO Completed push id=24 objects=8 duration=2.758ms48592026/11/12 00:28:42 WARN Force mode enabled - objects will be deleted immediately without grace period48602026/11/12 00:28:42 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048612026/11/12 00:28:42 INFO Vacuumed table table=pending_closures48622026/11/12 00:28:42 INFO Vacuumed table table=commit_receipts48632026/11/12 00:28:42 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=048642026/11/12 00:28:42 INFO Received push request method=POST path=/48652026/11/12 00:28:42 INFO Vacuumed table table=pending_closures48662026/11/12 00:28:42 INFO Vacuumed table table=pending_objects48672026/11/12 00:28:42 INFO Vacuumed table table=multipart_uploads48682026/11/12 00:28:42 INFO Vacuumed table table=commit_receipts48692026/11/12 00:28:42 INFO Vacuumed table table=closures48702026/11/12 00:28:42 INFO Vacuumed table table=pending_objects48712026/11/12 00:28:42 INFO Vacuumed table table=multipart_uploads48722026/11/12 00:28:42 INFO Vacuumed table table=objects48732026/11/12 00:28:42 INFO Received complete upload request method=POST path=/48742026/11/12 00:28:42 INFO Vacuumed table table=closures48752026/11/12 00:28:42 INFO Completed upload id=25 objects=9 duration=925.584µs48762026/11/12 00:28:42 INFO Vacuumed table table=objects48772026/11/12 00:28:42 INFO Received complete upload request method=POST path=/48782026/11/12 00:28:42 INFO Completed upload id=15 objects=2 duration=1.313625ms48792026/11/12 00:28:42 INFO Received complete upload request method=POST path=/48802026/11/12 00:28:42 INFO Completed upload id=24 objects=18 duration=1.034ms48812026/11/12 00:28:42 INFO Received complete upload request method=POST path=/48822026/11/12 00:28:42 INFO Completed upload id=22 objects=6 duration=1.118875ms48832026/11/12 00:28:42 INFO Received complete push request method=POST path=/48842026/11/12 00:28:42 INFO Received complete upload request method=POST path=/48852026/11/12 00:28:42 INFO Received complete upload request method=POST path=/48862026/11/12 00:28:42 INFO Completed push id=25 objects=2 duration=5.893208ms48872026/11/12 00:28:42 INFO Completed upload id=27 objects=3 duration=2.334333ms48882026/11/12 00:28:42 INFO Received uploads request method=POST path=/48892026/11/12 00:28:42 INFO Completed upload id=21 objects=7 duration=3.406416ms48902026/11/12 00:28:42 INFO Received complete push request method=POST path=/48912026/11/12 00:28:42 INFO Completed push id=27 objects=4 duration=998.709µs48922026/11/12 00:28:42 INFO Received complete upload request method=POST path=/48932026/11/12 00:28:42 INFO Completed upload id=28 objects=6 duration=890.459µs48942026/11/12 00:28:42 INFO Received uploads request method=POST path=/48952026/11/12 00:28:42 INFO Received complete push request method=POST path=/48962026/11/12 00:28:42 INFO Completed push id=30 objects=4 duration=6.398375ms48972026/11/12 00:28:42 INFO Received push request method=POST path=/48982026/11/12 00:28:42 INFO Received push request method=POST path=/48992026/11/12 00:28:42 INFO Received complete push request method=POST path=/49002026/11/12 00:28:42 INFO Completed push id=19 objects=14 duration=1.540959ms49012026/11/12 00:28:42 INFO Received complete push request method=POST path=/49022026/11/12 00:28:42 INFO Completed push id=31 objects=7 duration=523.125µs49032026/11/12 00:28:42 INFO Received push request method=POST path=/49042026/11/12 00:28:42 INFO Received uploads request method=POST path=/49052026/11/12 00:28:42 INFO Received complete upload request method=POST path=/49062026/11/12 00:28:42 INFO Completed upload id=32 objects=5 duration=615.25µs49072026/11/12 00:28:42 INFO Aborted multipart uploads count=0 kept=049082026/11/12 00:28:42 INFO Received push request method=POST path=/49092026/11/12 00:28:42 WARN Force mode enabled - objects will be deleted immediately without grace period49102026/11/12 00:28:42 INFO Received uploads request method=POST path=/49112026/11/12 00:28:42 INFO Received uploads request method=POST path=/49122026/11/12 00:28:42 INFO Received uploads request method=POST path=/49132026/11/12 00:28:42 INFO Received complete push request method=POST path=/49142026/11/12 00:28:42 INFO Completed push id=28 objects=26 duration=2.253084ms49152026/11/12 00:28:42 INFO Received complete upload request method=POST path=/49162026/11/12 00:28:42 INFO Received uploads request method=POST path=/49172026/11/12 00:28:42 INFO Received uploads request method=POST path=/49182026/11/12 00:28:42 INFO Completed upload id=26 objects=8 duration=874.541µs49192026/11/12 00:28:42 INFO Received complete push request method=POST path=/49202026/11/12 00:28:42 INFO Completed push id=30 objects=3 duration=758.459µs49212026/11/12 00:28:42 INFO Received complete upload request method=POST path=/49222026/11/12 00:28:42 INFO Completed upload id=27 objects=4 duration=3.351291ms49232026/11/12 00:28:42 INFO Received uploads request method=POST path=/49242026/11/12 00:28:42 INFO Received complete push request method=POST path=/49252026/11/12 00:28:42 INFO Completed push id=28 objects=14 duration=1.51075ms49262026/11/12 00:28:42 INFO Received complete push request method=POST path=/49272026/11/12 00:28:42 INFO Completed push id=32 objects=19 duration=950.666µs49282026/11/12 00:28:42 INFO Received complete push request method=POST path=/49292026/11/12 00:28:42 INFO Completed push id=13 objects=11 duration=3.636416ms49302026/11/12 00:28:42 INFO Received uploads request method=POST path=/49312026/11/12 00:28:42 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=049322026/11/12 00:28:42 INFO Vacuumed table table=pending_closures49332026/11/12 00:28:42 INFO Vacuumed table table=commit_receipts49342026/11/12 00:28:42 INFO Vacuumed table table=pending_objects49352026/11/12 00:28:42 INFO Vacuumed table table=multipart_uploads49362026/11/12 00:28:42 INFO Received uploads request method=POST path=/49372026/11/12 00:28:42 INFO Vacuumed table table=closures49382026/11/12 00:28:42 INFO Aborted multipart uploads count=0 kept=049392026/11/12 00:28:42 INFO Received push request method=POST path=/49402026/11/12 00:28:42 INFO Vacuumed table table=objects49412026/11/12 00:28:42 WARN Force mode enabled - objects will be deleted immediately without grace period49422026/11/12 00:28:42 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049432026/11/12 00:28:42 INFO Vacuumed table table=pending_closures49442026/11/12 00:28:42 INFO Vacuumed table table=commit_receipts49452026/11/12 00:28:42 INFO Received uploads request method=POST path=/49462026/11/12 00:28:42 INFO Vacuumed table table=pending_objects49472026/11/12 00:28:42 INFO Vacuumed table table=multipart_uploads49482026/11/12 00:28:42 INFO Vacuumed table table=closures49492026/11/12 00:28:42 INFO Vacuumed table table=objects49502026/11/12 00:28:42 INFO Aborted multipart uploads count=0 kept=049512026/11/12 00:28:42 INFO Received uploads request method=POST path=/49522026/11/12 00:28:42 WARN Force mode enabled - objects will be deleted immediately without grace period49532026/11/12 00:28:43 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049542026/11/12 00:28:43 INFO Vacuumed table table=pending_closures49552026/11/12 00:28:43 INFO Received push request method=POST path=/49562026/11/12 00:28:43 INFO Vacuumed table table=commit_receipts49572026/11/12 00:28:43 INFO Vacuumed table table=pending_objects49582026/11/12 00:28:43 INFO Vacuumed table table=multipart_uploads49592026/11/12 00:28:43 INFO Vacuumed table table=closures49602026/11/12 00:28:43 INFO Aborted multipart uploads count=0 kept=049612026/11/12 00:28:43 INFO Vacuumed table table=objects49622026/11/12 00:28:43 WARN Force mode enabled - objects will be deleted immediately without grace period49632026/11/12 00:28:43 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049642026/11/12 00:28:43 INFO Vacuumed table table=pending_closures49652026/11/12 00:28:43 INFO Aborted multipart uploads count=0 kept=049662026/11/12 00:28:43 INFO Vacuumed table table=commit_receipts49672026/11/12 00:28:43 INFO Vacuumed table table=pending_objects49682026/11/12 00:28:43 WARN Force mode enabled - objects will be deleted immediately without grace period49692026/11/12 00:28:43 INFO Vacuumed table table=multipart_uploads49702026/11/12 00:28:43 INFO Vacuumed table table=closures49712026/11/12 00:28:43 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049722026/11/12 00:28:43 INFO Vacuumed table table=objects49732026/11/12 00:28:43 INFO Vacuumed table table=pending_closures49742026/11/12 00:28:43 INFO Vacuumed table table=commit_receipts49752026/11/12 00:28:43 INFO Vacuumed table table=pending_objects49762026/11/12 00:28:43 INFO Vacuumed table table=multipart_uploads49772026/11/12 00:28:43 INFO Vacuumed table table=closures49782026/11/12 00:28:43 INFO Received complete upload request method=POST path=/49792026/11/12 00:28:43 INFO Vacuumed table table=objects49802026/11/12 00:28:43 INFO Completed upload id=15 objects=3 duration=1.900791ms49812026/11/12 00:28:43 INFO Received uploads request method=POST path=/49822026/11/12 00:28:43 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=049832026/11/12 00:28:43 INFO Vacuumed table table=pending_closures49842026/11/12 00:28:43 INFO Vacuumed table table=commit_receipts49852026/11/12 00:28:43 INFO Received complete upload request method=POST path=/49862026/11/12 00:28:43 INFO Completed upload id=34 objects=24 duration=676.167µs49872026/11/12 00:28:43 INFO Vacuumed table table=pending_objects49882026/11/12 00:28:43 INFO Vacuumed table table=multipart_uploads49892026/11/12 00:28:43 INFO Vacuumed table table=closures49902026/11/12 00:28:43 INFO Vacuumed table table=objects49912026/11/12 00:28:43 INFO Received uploads request method=POST path=/49922026/11/12 00:28:43 INFO Received push request method=POST path=/49932026/11/12 00:28:43 INFO Aborted multipart uploads count=0 kept=049942026/11/12 00:28:43 INFO Received push request method=POST path=/49952026/11/12 00:28:43 WARN Force mode enabled - objects will be deleted immediately without grace period49962026/11/12 00:28:43 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049972026/11/12 00:28:43 INFO Vacuumed table table=pending_closures49982026/11/12 00:28:43 INFO Received complete push request method=POST path=/49992026/11/12 00:28:43 INFO Vacuumed table table=commit_receipts50002026/11/12 00:28:43 INFO Completed push id=16 objects=14 duration=2.340625ms50012026/11/12 00:28:43 INFO Vacuumed table table=pending_objects50022026/11/12 00:28:43 INFO Vacuumed table table=multipart_uploads50032026/11/12 00:28:43 INFO Received complete upload request method=POST path=/50042026/11/12 00:28:43 INFO Vacuumed table table=closures50052026/11/12 00:28:43 INFO Completed upload id=21 objects=9 duration=8.423625ms50062026/11/12 00:28:43 INFO Vacuumed table table=objects50072026/11/12 00:28:43 INFO Received uploads request method=POST path=/50082026/11/12 00:28:43 INFO Aborted multipart uploads count=0 kept=050092026/11/12 00:28:43 WARN Force mode enabled - objects will be deleted immediately without grace period50102026/11/12 00:28:43 INFO Received complete push request method=POST path=/50112026/11/12 00:28:43 INFO Completed push id=25 objects=10 duration=2.980959ms50122026/11/12 00:28:43 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050132026/11/12 00:28:43 INFO Vacuumed table table=pending_closures50142026/11/12 00:28:43 INFO Vacuumed table table=commit_receipts50152026/11/12 00:28:43 INFO Vacuumed table table=pending_objects50162026/11/12 00:28:43 INFO Vacuumed table table=multipart_uploads50172026/11/12 00:28:43 INFO Vacuumed table table=closures50182026/11/12 00:28:43 INFO Aborted multipart uploads count=0 kept=050192026/11/12 00:28:43 INFO Vacuumed table table=objects50202026/11/12 00:28:43 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050212026/11/12 00:28:43 INFO Vacuumed table table=pending_closures50222026/11/12 00:28:43 INFO Vacuumed table table=commit_receipts50232026/11/12 00:28:43 INFO Vacuumed table table=pending_objects50242026/11/12 00:28:43 INFO Received complete upload request method=POST path=/50252026/11/12 00:28:43 INFO Vacuumed table table=multipart_uploads50262026/11/12 00:28:43 INFO Completed upload id=29 objects=5 duration=1.03ms50272026/11/12 00:28:43 INFO Vacuumed table table=closures50282026/11/12 00:28:43 INFO Vacuumed table table=objects50292026/11/12 00:28:43 INFO Aborted multipart uploads count=0 kept=050302026/11/12 00:28:43 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050312026/11/12 00:28:43 INFO Vacuumed table table=pending_closures50322026/11/12 00:28:43 INFO Vacuumed table table=commit_receipts50332026/11/12 00:28:43 INFO Vacuumed table table=pending_objects50342026/11/12 00:28:43 INFO Vacuumed table table=multipart_uploads50352026/11/12 00:28:43 INFO Received complete upload request method=POST path=/50362026/11/12 00:28:43 INFO Vacuumed table table=closures50372026/11/12 00:28:43 INFO Completed upload id=30 objects=2 duration=594.958µs50382026/11/12 00:28:43 INFO Vacuumed table table=objects50392026/11/12 00:28:43 INFO Received complete push request method=POST path=/50402026/11/12 00:28:43 INFO Completed push id=16 objects=7 duration=1.50575ms50412026/11/12 00:28:43 INFO Received push request method=POST path=/50422026/11/12 00:28:43 INFO Received complete push request method=POST path=/50432026/11/12 00:28:43 INFO Completed push id=31 objects=20 duration=4.856ms50442026/11/12 00:28:43 INFO Received complete upload request method=POST path=/50452026/11/12 00:28:43 INFO Received uploads request method=POST path=/50462026/11/12 00:28:43 INFO Completed upload id=32 objects=5 duration=961.458µs50472026/11/12 00:28:43 INFO Received complete upload request method=POST path=/50482026/11/12 00:28:43 INFO Received complete upload request method=POST path=/50492026/11/12 00:28:43 INFO Completed upload id=25 objects=6 duration=1.033875ms50502026/11/12 00:28:43 INFO Completed upload id=33 objects=3 duration=6.734167ms50512026/11/12 00:28:43 INFO Received complete push request method=POST path=/50522026/11/12 00:28:43 INFO Completed push id=17 objects=22 duration=1.625958ms50532026/11/12 00:28:43 INFO Received complete push request method=POST path=/50542026/11/12 00:28:43 INFO Completed push id=23 objects=6 duration=1.069792ms50552026/11/12 00:28:43 INFO Aborted multipart uploads count=0 kept=050562026/11/12 00:28:43 WARN Force mode enabled - objects will be deleted immediately without grace period50572026/11/12 00:28:43 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050582026/11/12 00:28:43 INFO Received uploads request method=POST path=/50592026/11/12 00:28:43 INFO Vacuumed table table=pending_closures50602026/11/12 00:28:43 INFO Vacuumed table table=commit_receipts50612026/11/12 00:28:43 INFO Vacuumed table table=pending_objects50622026/11/12 00:28:43 INFO Received complete push request method=POST path=/50632026/11/12 00:28:43 INFO Vacuumed table table=multipart_uploads50642026/11/12 00:28:43 INFO Completed push id=24 objects=6 duration=982.375µs50652026/11/12 00:28:43 INFO Vacuumed table table=closures50662026/11/12 00:28:43 INFO Vacuumed table table=objects50672026/11/12 00:28:43 INFO Received push request method=POST path=/50682026/11/12 00:28:43 INFO Received uploads request method=POST path=/50692026/11/12 00:28:43 INFO Received complete push request method=POST path=/50702026/11/12 00:28:43 INFO Completed push id=11 objects=9 duration=2.337375ms50712026/11/12 00:28:43 INFO Received complete push request method=POST path=/50722026/11/12 00:28:43 INFO Completed push id=25 objects=7 duration=2.356125ms50732026/11/12 00:28:43 INFO Received push request method=POST path=/50742026/11/12 00:28:43 INFO Received complete push request method=POST path=/50752026/11/12 00:28:43 INFO Received complete push request method=POST path=/50762026/11/12 00:28:43 INFO Completed push id=19 objects=14 duration=2.989042ms50772026/11/12 00:28:43 INFO Completed push id=15 objects=24 duration=3.103041ms50782026/11/12 00:28:43 INFO Received complete upload request method=POST path=/50792026/11/12 00:28:43 INFO Completed upload id=13 objects=14 duration=2.520708ms50802026/11/12 00:28:43 INFO Received uploads request method=POST path=/50812026/11/12 00:28:43 INFO Received complete upload request method=POST path=/50822026/11/12 00:28:43 INFO Aborted multipart uploads count=0 kept=050832026/11/12 00:28:43 INFO Completed upload id=19 objects=2 duration=580.625µs50842026/11/12 00:28:43 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050852026/11/12 00:28:43 INFO Vacuumed table table=pending_closures50862026/11/12 00:28:43 INFO Vacuumed table table=commit_receipts50872026/11/12 00:28:43 INFO Vacuumed table table=pending_objects50882026/11/12 00:28:43 INFO Vacuumed table table=multipart_uploads50892026/11/12 00:28:43 INFO Vacuumed table table=closures50902026/11/12 00:28:43 INFO Vacuumed table table=objects50912026/11/12 00:28:43 INFO Received complete upload request method=POST path=/50922026/11/12 00:28:43 INFO Completed upload id=24 objects=7 duration=956.625µs50932026/11/12 00:28:43 INFO Received complete push request method=POST path=/50942026/11/12 00:28:43 INFO Received complete upload request method=POST path=/50952026/11/12 00:28:43 INFO Completed push id=26 objects=3 duration=1.219833ms50962026/11/12 00:28:43 INFO Completed upload id=26 objects=24 duration=1.352167ms50972026/11/12 00:28:44 INFO Received complete push request method=POST path=/50982026/11/12 00:28:44 INFO Received complete upload request method=POST path=/50992026/11/12 00:28:44 INFO Completed push id=31 objects=6 duration=3.892708ms51002026/11/12 00:28:44 INFO Completed upload id=28 objects=8 duration=833.584µs51012026/11/12 00:28:44 INFO Received complete upload request method=POST path=/51022026/11/12 00:28:44 INFO Completed upload id=20 objects=6 duration=1.976917ms51032026/11/12 00:28:44 INFO Received complete upload request method=POST path=/51042026/11/12 00:28:44 INFO Received uploads request method=POST path=/51052026/11/12 00:28:44 INFO Completed upload id=29 objects=10 duration=594.625µs51062026/11/12 00:28:44 INFO Received complete upload request method=POST path=/51072026/11/12 00:28:44 INFO Received uploads request method=POST path=/51082026/11/12 00:28:44 INFO Completed upload id=30 objects=20 duration=1.028042ms51092026/11/12 00:28:44 INFO Received complete push request method=POST path=/51102026/11/12 00:28:44 INFO Completed push id=31 objects=11 duration=614.083µs51112026/11/12 00:28:44 INFO Received complete upload request method=POST path=/51122026/11/12 00:28:44 INFO Completed upload id=21 objects=2 duration=678.625µs51132026/11/12 00:28:44 INFO Received complete upload request method=POST path=/51142026/11/12 00:28:44 INFO Received complete upload request method=POST path=/51152026/11/12 00:28:44 INFO Completed upload id=32 objects=2 duration=861.083µs51162026/11/12 00:28:44 INFO Completed upload id=27 objects=26 duration=976.833µs51172026/11/12 00:28:44 INFO Received complete push request method=POST path=/51182026/11/12 00:28:44 INFO Received uploads request method=POST path=/51192026/11/12 00:28:44 INFO Completed push id=33 objects=3 duration=1.423292ms51202026/11/12 00:28:44 INFO Received complete push request method=POST path=/51212026/11/12 00:28:44 INFO Completed push id=28 objects=6 duration=1.084708ms51222026/11/12 00:28:44 INFO Received complete upload request method=POST path=/51232026/11/12 00:28:44 INFO Completed upload id=34 objects=3 duration=2.115917ms51242026/11/12 00:28:44 INFO Aborted multipart uploads count=0 kept=051252026/11/12 00:28:44 INFO Received complete push request method=POST path=/51262026/11/12 00:28:44 INFO Completed push id=29 objects=6 duration=625.416µs51272026/11/12 00:28:44 WARN Force mode enabled - objects will be deleted immediately without grace period51282026/11/12 00:28:44 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051292026/11/12 00:28:44 INFO Vacuumed table table=pending_closures51302026/11/12 00:28:44 INFO Vacuumed table table=commit_receipts51312026/11/12 00:28:44 INFO Vacuumed table table=pending_objects51322026/11/12 00:28:44 INFO Vacuumed table table=multipart_uploads51332026/11/12 00:28:44 INFO Vacuumed table table=closures51342026/11/12 00:28:44 INFO Vacuumed table table=objects51352026/11/12 00:28:44 INFO Aborted multipart uploads count=0 kept=051362026/11/12 00:28:44 WARN Force mode enabled - objects will be deleted immediately without grace period51372026/11/12 00:28:44 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051382026/11/12 00:28:44 INFO Vacuumed table table=pending_closures51392026/11/12 00:28:44 INFO Vacuumed table table=commit_receipts51402026/11/12 00:28:44 INFO Vacuumed table table=pending_objects51412026/11/12 00:28:44 INFO Vacuumed table table=multipart_uploads51422026/11/12 00:28:44 INFO Vacuumed table table=closures51432026/11/12 00:28:44 INFO Vacuumed table table=objects51442026/11/12 00:28:44 INFO Received uploads request method=POST path=/51452026/11/12 00:28:44 INFO Received push request method=POST path=/51462026/11/12 00:28:44 INFO Received complete upload request method=POST path=/51472026/11/12 00:28:44 INFO Completed upload id=30 objects=8 duration=906.541µs51482026/11/12 00:28:44 INFO Received complete push request method=POST path=/51492026/11/12 00:28:44 INFO Completed push id=24 objects=24 duration=2.261ms51502026/11/12 00:28:44 INFO Aborted multipart uploads count=0 kept=051512026/11/12 00:28:44 INFO Received complete push request method=POST path=/51522026/11/12 00:28:44 INFO Completed push id=1 objects=9 duration=966.25µs51532026/11/12 00:28:44 WARN Force mode enabled - objects will be deleted immediately without grace period51542026/11/12 00:28:44 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051552026/11/12 00:28:44 INFO Vacuumed table table=pending_closures51562026/11/12 00:28:44 INFO Vacuumed table table=commit_receipts51572026/11/12 00:28:44 INFO Received complete upload request method=POST path=/51582026/11/12 00:28:44 INFO Vacuumed table table=pending_objects51592026/11/12 00:28:44 INFO Completed upload id=31 objects=6 duration=640.166µs51602026/11/12 00:28:44 INFO Vacuumed table table=multipart_uploads51612026/11/12 00:28:44 INFO Vacuumed table table=closures51622026/11/12 00:28:44 INFO Vacuumed table table=objects51632026/11/12 00:28:44 INFO Received complete upload request method=POST path=/51642026/11/12 00:28:44 INFO Completed upload id=32 objects=4 duration=3.771792ms51652026/11/12 00:28:44 INFO Received uploads request method=POST path=/51662026/11/12 00:28:44 INFO Received uploads request method=POST path=/51672026/11/12 00:28:44 INFO Received complete upload request method=POST path=/51682026/11/12 00:28:44 INFO Completed upload id=33 objects=10 duration=617.208µs51692026/11/12 00:28:44 INFO Received push request method=POST path=/51702026/11/12 00:28:44 INFO Received uploads request method=POST path=/51712026/11/12 00:28:44 INFO Received complete upload request method=POST path=/51722026/11/12 00:28:44 INFO Completed upload id=34 objects=23 duration=1.630792ms51732026/11/12 00:28:44 INFO Aborted multipart uploads count=0 kept=051742026/11/12 00:28:44 WARN Force mode enabled - objects will be deleted immediately without grace period51752026/11/12 00:28:44 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051762026/11/12 00:28:44 INFO Vacuumed table table=pending_closures51772026/11/12 00:28:44 INFO Vacuumed table table=commit_receipts51782026/11/12 00:28:44 INFO Vacuumed table table=pending_objects51792026/11/12 00:28:44 INFO Vacuumed table table=multipart_uploads51802026/11/12 00:28:44 INFO Vacuumed table table=closures51812026/11/12 00:28:44 INFO Vacuumed table table=objects51822026/11/12 00:28:44 INFO Received complete upload request method=POST path=/51832026/11/12 00:28:44 INFO Completed upload id=2 objects=7 duration=741.292µs51842026/11/12 00:28:44 INFO Received complete push request method=POST path=/51852026/11/12 00:28:44 INFO Completed push id=35 objects=6 duration=1.305208ms51862026/11/12 00:28:44 INFO Received push request method=POST path=/51872026/11/12 00:28:44 INFO Aborted multipart uploads count=0 kept=051882026/11/12 00:28:44 WARN Force mode enabled - objects will be deleted immediately without grace period51892026/11/12 00:28:44 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=051902026/11/12 00:28:44 INFO Vacuumed table table=pending_closures51912026/11/12 00:28:44 INFO Vacuumed table table=commit_receipts51922026/11/12 00:28:44 INFO Vacuumed table table=pending_objects51932026/11/12 00:28:44 INFO Vacuumed table table=multipart_uploads51942026/11/12 00:28:44 INFO Vacuumed table table=closures51952026/11/12 00:28:44 INFO Vacuumed table table=objects51962026/11/12 00:28:44 INFO Received complete upload request method=POST path=/51972026/11/12 00:28:44 INFO Completed upload id=23 objects=7 duration=1.960708ms51982026/11/12 00:28:45 INFO Received complete upload request method=POST path=/51992026/11/12 00:28:45 INFO Completed upload id=26 objects=5 duration=593.166µs52002026/11/12 00:28:45 INFO Received complete upload request method=POST path=/52012026/11/12 00:28:45 INFO Completed upload id=8 objects=6 duration=700.167µs52022026/11/12 00:28:45 INFO Received complete push request method=POST path=/52032026/11/12 00:28:45 INFO Completed push id=27 objects=3 duration=1.268334ms52042026/11/12 00:28:45 INFO Received complete upload request method=POST path=/52052026/11/12 00:28:45 INFO Completed upload id=28 objects=5 duration=1.049583ms52062026/11/12 00:28:45 INFO Received complete push request method=POST path=/52072026/11/12 00:28:45 INFO Completed push id=29 objects=17 duration=805.667µs52082026/11/12 00:28:45 INFO Aborted multipart uploads count=0 kept=052092026/11/12 00:28:45 WARN Force mode enabled - objects will be deleted immediately without grace period52102026/11/12 00:28:45 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052112026/11/12 00:28:45 INFO Vacuumed table table=pending_closures52122026/11/12 00:28:45 INFO Vacuumed table table=commit_receipts52132026/11/12 00:28:45 INFO Vacuumed table table=pending_objects52142026/11/12 00:28:45 INFO Vacuumed table table=multipart_uploads52152026/11/12 00:28:45 INFO Vacuumed table table=closures52162026/11/12 00:28:45 INFO Vacuumed table table=objects52172026/11/12 00:28:45 INFO Received complete upload request method=POST path=/52182026/11/12 00:28:45 INFO Completed upload id=12 objects=9 duration=672.208µs52192026/11/12 00:28:45 INFO Received complete upload request method=POST path=/52202026/11/12 00:28:45 INFO Completed upload id=18 objects=22 duration=669.417µs52212026/11/12 00:28:45 INFO Received complete upload request method=POST path=/52222026/11/12 00:28:45 INFO Completed upload id=21 objects=8 duration=509.458µs52232026/11/12 00:28:45 INFO Received complete upload request method=POST path=/52242026/11/12 00:28:45 INFO Completed upload id=22 objects=7 duration=574.834µs52252026/11/12 00:28:45 INFO Received complete push request method=POST path=/52262026/11/12 00:28:45 INFO Completed push id=24 objects=5 duration=593.333µs52272026/11/12 00:28:45 INFO Received complete upload request method=POST path=/52282026/11/12 00:28:45 INFO Completed upload id=25 objects=8 duration=549.167µs52292026/11/12 00:28:45 INFO Received complete push request method=POST path=/52302026/11/12 00:28:45 INFO Completed push id=26 objects=4 duration=590.083µs52312026/11/12 00:28:45 INFO Received complete push request method=POST path=/52322026/11/12 00:28:45 INFO Completed push id=30 objects=2 duration=613.5µs52332026/11/12 00:28:45 INFO Received complete push request method=POST path=/52342026/11/12 00:28:45 INFO Completed push id=32 objects=7 duration=748.625µs52352026/11/12 00:28:45 INFO Received complete upload request method=POST path=/52362026/11/12 00:28:45 INFO Completed upload id=33 objects=6 duration=574.708µs52372026/11/12 00:28:45 INFO Received complete upload request method=POST path=/52382026/11/12 00:28:45 INFO Completed upload id=34 objects=3 duration=462.209µs52392026/11/12 00:28:45 INFO Aborted multipart uploads count=0 kept=052402026/11/12 00:28:45 WARN Force mode enabled - objects will be deleted immediately without grace period52412026/11/12 00:28:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052422026/11/12 00:28:45 INFO Vacuumed table table=pending_closures52432026/11/12 00:28:45 INFO Vacuumed table table=commit_receipts52442026/11/12 00:28:45 INFO Vacuumed table table=pending_objects52452026/11/12 00:28:45 INFO Vacuumed table table=multipart_uploads52462026/11/12 00:28:45 INFO Vacuumed table table=closures52472026/11/12 00:28:45 INFO Vacuumed table table=objects5248--- PASS: FuzzPushGC (0.00s)5249 --- PASS: FuzzPushGC/seed#0 (11.85s)5250 --- PASS: FuzzPushGC/seed#8 (13.71s)5251 --- PASS: FuzzPushGC/seed#14 (14.39s)5252 --- PASS: FuzzPushGC/seed#2 (14.41s)5253 --- PASS: FuzzPushGC/seed#4 (15.62s)5254 --- PASS: FuzzPushGC/seed#15 (16.12s)5255 --- PASS: FuzzPushGC/seed#11 (16.40s)5256 --- PASS: FuzzPushGC/seed#13 (16.94s)5257 --- PASS: FuzzPushGC/seed#3 (17.05s)5258 --- PASS: FuzzPushGC/seed#7 (17.62s)5259 --- PASS: FuzzPushGC/seed#9 (17.95s)5260 --- PASS: FuzzPushGC/seed#6 (18.33s)5261 --- PASS: FuzzPushGC/seed#5 (19.01s)5262 --- PASS: FuzzPushGC/seed#12 (19.59s)5263 --- PASS: FuzzPushGC/seed#1 (19.81s)5264 --- PASS: FuzzPushGC/seed#10 (20.50s)5265PASS5266{"timestamp":"2026-11-12T00:28:45.908928Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:52020","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(10)"}52672026-11-12 00:28:46.165 UTC [88001] LOG: received smart shutdown request52682026-11-12 00:28:46.167 UTC [88001] LOG: background worker "logical replication launcher" (PID 88014) exited with exit code 152692026-11-12 00:28:47.681 UTC [88008] LOG: checkpoint complete: wrote 2954 buffers (18.0%), wrote 5 SLRU buffers; 0 WAL file(s) added, 0 removed, 33 recycled; write=31.034 s, sync=1.365 s, total=32.531 s; sync files=43463, longest=0.008 s, average=0.001 s; distance=533810 kB, estimate=533810 kB; lsn=0/28475328, redo lsn=0/220D99A852702026-11-12 00:28:47.685 UTC [88008] LOG: shutting down52712026-11-12 00:28:47.685 UTC [88008] LOG: checkpoint starting: shutdown immediate52722026-11-12 00:28:48.261 UTC [88008] LOG: checkpoint complete: wrote 9171 buffers (56.0%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 6 recycled; write=0.499 s, sync=0.072 s, total=0.577 s; sync files=3127, longest=0.001 s, average=0.001 s; distance=101998 kB, estimate=490629 kB; lsn=0/284753A0, redo lsn=0/284753A052732026-11-12 00:28:48.276 UTC [88001] LOG: database system is shut down5274Running OIDC tests...5275=== RUN TestAudienceForIssuer5276=== PAUSE TestAudienceForIssuer5277=== RUN TestGlobMatch5278=== PAUSE TestGlobMatch5279=== RUN TestValidateToken_ValidToken5280=== PAUSE TestValidateToken_ValidToken5281=== RUN TestValidateToken_WrongAudience5282=== PAUSE TestValidateToken_WrongAudience5283=== RUN TestValidateToken_Expired5284=== PAUSE TestValidateToken_Expired5285=== RUN TestValidateToken_BoundClaimsMismatch5286=== PAUSE TestValidateToken_BoundClaimsMismatch5287=== RUN TestValidateToken_BoundSubjectMismatch5288=== PAUSE TestValidateToken_BoundSubjectMismatch5289=== RUN TestValidateToken_MultipleProviders5290=== PAUSE TestValidateToken_MultipleProviders5291=== RUN TestValidateToken_NoMatchingProvider5292=== PAUSE TestValidateToken_NoMatchingProvider5293=== RUN TestValidateToken_KubernetesServiceAccount5294=== PAUSE TestValidateToken_KubernetesServiceAccount5295=== RUN TestNewValidator_KubernetesRequiresCA5296=== PAUSE TestNewValidator_KubernetesRequiresCA5297=== RUN TestValidateToken_KubernetesIssuerFromOwnToken5298=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken5299=== RUN TestPins_ReservedForMatchingRule5300=== PAUSE TestPins_ReservedForMatchingRule5301=== RUN TestPins_TopLevelShorthand5302=== PAUSE TestPins_TopLevelShorthand5303=== RUN TestPins_ConfigValidation5304=== PAUSE TestPins_ConfigValidation5305=== RUN TestScopes_LegacyProviderDefaultsToWrite5306=== PAUSE TestScopes_LegacyProviderDefaultsToWrite5307=== RUN TestScopes_Rules5308=== PAUSE TestScopes_Rules5309=== RUN TestScopes_ConfigValidation5310=== PAUSE TestScopes_ConfigValidation5311=== CONT TestAudienceForIssuer5312--- PASS: TestAudienceForIssuer (0.00s)5313=== CONT TestValidateToken_ValidToken5314=== CONT TestPins_ConfigValidation5315=== CONT TestValidateToken_KubernetesServiceAccount5316=== CONT TestValidateToken_WrongAudience5317=== CONT TestValidateToken_BoundClaimsMismatch5318=== CONT TestScopes_ConfigValidation5319=== CONT TestValidateToken_MultipleProviders5320=== CONT TestValidateToken_NoMatchingProvider5321=== CONT TestValidateToken_Expired5322--- PASS: TestPins_ConfigValidation (0.00s)5323--- PASS: TestScopes_ConfigValidation (0.01s)5324=== CONT TestValidateToken_BoundSubjectMismatch5325=== CONT TestGlobMatch5326=== RUN TestGlobMatch/foo_foo5327=== PAUSE TestGlobMatch/foo_foo5328=== RUN TestGlobMatch/foo_bar5329=== PAUSE TestGlobMatch/foo_bar5330=== RUN TestGlobMatch/*_5331=== PAUSE TestGlobMatch/*_5332=== RUN TestGlobMatch/*_anything5333=== PAUSE TestGlobMatch/*_anything5334=== RUN TestGlobMatch/foo*_foo5335=== PAUSE TestGlobMatch/foo*_foo5336=== RUN TestGlobMatch/foo*_foobar5337=== PAUSE TestGlobMatch/foo*_foobar5338=== RUN TestGlobMatch/foo*_bar5339=== PAUSE TestGlobMatch/foo*_bar5340=== RUN TestGlobMatch/*bar_bar5341=== PAUSE TestGlobMatch/*bar_bar5342=== RUN TestGlobMatch/*bar_foobar5343=== PAUSE TestGlobMatch/*bar_foobar5344=== RUN TestGlobMatch/*bar_foo5345=== PAUSE TestGlobMatch/*bar_foo5346=== RUN TestGlobMatch/foo*bar_foobar5347=== PAUSE TestGlobMatch/foo*bar_foobar5348=== RUN TestGlobMatch/foo*bar_foo123bar5349=== PAUSE TestGlobMatch/foo*bar_foo123bar5350=== RUN TestGlobMatch/foo*bar_foobarbaz5351=== PAUSE TestGlobMatch/foo*bar_foobarbaz5352=== RUN TestGlobMatch/*/*_foo/bar5353=== PAUSE TestGlobMatch/*/*_foo/bar5354=== RUN TestGlobMatch/*/*_foo5355=== PAUSE TestGlobMatch/*/*_foo5356=== RUN TestGlobMatch/refs/heads/*_refs/heads/main5357=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main5358=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.05359=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.05360=== RUN TestGlobMatch/refs/*/main_refs/heads/main5361=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main5362=== RUN TestGlobMatch/fo?_foo5363=== PAUSE TestGlobMatch/fo?_foo5364=== RUN TestGlobMatch/fo?_fo5365=== PAUSE TestGlobMatch/fo?_fo5366=== RUN TestGlobMatch/fo?_fooo5367=== PAUSE TestGlobMatch/fo?_fooo5368=== RUN TestGlobMatch/?oo_foo5369=== PAUSE TestGlobMatch/?oo_foo5370=== RUN TestGlobMatch/?oo_boo5371=== PAUSE TestGlobMatch/?oo_boo5372=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5373=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5374=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5375=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5376=== CONT TestGlobMatch/foo_foo5377=== CONT TestScopes_LegacyProviderDefaultsToWrite5378=== CONT TestGlobMatch/*/*_foo/bar5379=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5380=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5381=== CONT TestGlobMatch/?oo_boo5382=== CONT TestGlobMatch/?oo_foo5383=== CONT TestGlobMatch/fo?_fooo5384=== CONT TestGlobMatch/fo?_fo5385=== CONT TestGlobMatch/fo?_foo5386=== CONT TestGlobMatch/refs/*/main_refs/heads/main5387=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.05388=== CONT TestGlobMatch/refs/heads/*_refs/heads/main5389=== CONT TestGlobMatch/*/*_foo5390=== CONT TestGlobMatch/*bar_bar5391=== CONT TestGlobMatch/foo*bar_foobarbaz5392=== CONT TestGlobMatch/foo*bar_foo123bar5393=== CONT TestGlobMatch/foo*bar_foobar5394=== CONT TestGlobMatch/*bar_foo5395=== CONT TestGlobMatch/*bar_foobar5396=== CONT TestGlobMatch/foo*_foo5397=== CONT TestGlobMatch/foo*_bar5398=== CONT TestGlobMatch/foo*_foobar5399=== CONT TestGlobMatch/*_5400=== CONT TestGlobMatch/*_anything5401=== CONT TestGlobMatch/foo_bar5402--- PASS: TestGlobMatch (0.00s)5403 --- PASS: TestGlobMatch/foo_foo (0.00s)5404 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)5405 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)5406 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)5407 --- PASS: TestGlobMatch/?oo_boo (0.00s)5408 --- PASS: TestGlobMatch/?oo_foo (0.00s)5409 --- PASS: TestGlobMatch/fo?_fooo (0.00s)5410 --- PASS: TestGlobMatch/fo?_fo (0.00s)5411 --- PASS: TestGlobMatch/fo?_foo (0.00s)5412 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)5413 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)5414 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)5415 --- PASS: TestGlobMatch/*/*_foo (0.00s)5416 --- PASS: TestGlobMatch/*bar_bar (0.00s)5417 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)5418 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)5419 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)5420 --- PASS: TestGlobMatch/*bar_foo (0.00s)5421 --- PASS: TestGlobMatch/*bar_foobar (0.00s)5422 --- PASS: TestGlobMatch/foo*_foo (0.00s)5423 --- PASS: TestGlobMatch/foo*_bar (0.00s)5424 --- PASS: TestGlobMatch/foo*_foobar (0.00s)5425 --- PASS: TestGlobMatch/*_ (0.00s)5426 --- PASS: TestGlobMatch/*_anything (0.00s)5427 --- PASS: TestGlobMatch/foo_bar (0.00s)5428=== CONT TestPins_ReservedForMatchingRule5429=== CONT TestScopes_Rules5430=== CONT TestValidateToken_KubernetesIssuerFromOwnToken5431=== CONT TestNewValidator_KubernetesRequiresCA5432=== CONT TestPins_TopLevelShorthand54332026/11/12 00:28:51 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:53799/oidc54342026/11/12 00:28:51 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:53801/oidc54352026/11/12 00:28:51 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:53802/oidc5436--- PASS: TestValidateToken_WrongAudience (0.12s)5437--- PASS: TestValidateToken_ValidToken (0.13s)54382026/11/12 00:28:51 http: TLS handshake error from 127.0.0.1:53806: read tcp 127.0.0.1:53805->127.0.0.1:53806: use of closed network connection5439--- PASS: TestNewValidator_KubernetesRequiresCA (0.12s)5440--- PASS: TestPins_TopLevelShorthand (0.10s)54412026/11/12 00:28:51 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:53807/oidc5442--- PASS: TestValidateToken_BoundSubjectMismatch (0.14s)54432026/11/12 00:28:51 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC12354442026/11/12 00:28:51 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:53810/oidc54452026/11/12 00:28:51 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:53812/oidc5446--- PASS: TestValidateToken_Expired (0.17s)5447--- PASS: TestValidateToken_BoundClaimsMismatch (0.18s)5448--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (0.17s)54492026/11/12 00:28:51 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:53815/oidc54502026/11/12 00:28:51 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:53817/oidc5451--- PASS: TestScopes_LegacyProviderDefaultsToWrite (0.26s)5452--- PASS: TestValidateToken_NoMatchingProvider (0.27s)54532026/11/12 00:28:51 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:53822/oidc54542026/11/12 00:28:51 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:538205455--- PASS: TestValidateToken_KubernetesServiceAccount (0.31s)5456--- PASS: TestScopes_Rules (0.29s)54572026/11/12 00:28:51 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:53824/oidc5458--- PASS: TestPins_ReservedForMatchingRule (0.32s)54592026/11/12 00:28:51 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:53826/oidc54602026/11/12 00:28:51 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:53827/oidc5461--- PASS: TestValidateToken_MultipleProviders (0.41s)5462PASS5463Running hook tests...5464=== RUN TestSendPathsEmpty5465=== PAUSE TestSendPathsEmpty5466=== RUN TestQueueEnqueueAndFetch5467=== PAUSE TestQueueEnqueueAndFetch5468=== RUN TestQueueDeduplication5469=== PAUSE TestQueueDeduplication5470=== RUN TestQueueRemove5471=== PAUSE TestQueueRemove5472=== RUN TestQueueFetchBatchLimit5473=== PAUSE TestQueueFetchBatchLimit5474=== RUN TestQueueRetryMovesToBack5475=== PAUSE TestQueueRetryMovesToBack5476=== RUN TestQueueFetchRemoveLifecycle5477=== PAUSE TestQueueFetchRemoveLifecycle5478=== RUN TestQueueConcurrentWriters5479=== PAUSE TestQueueConcurrentWriters5480=== RUN TestQueueRemoveLargeClosure5481=== PAUSE TestQueueRemoveLargeClosure5482=== RUN TestServerClientIntegration5483--- PASS: TestServerClientIntegration (0.00s)5484=== RUN TestServerQueueError54852026/11/12 00:28:51 ERROR Failed to queue paths error="permission denied" count=15486--- PASS: TestServerQueueError (0.00s)5487=== RUN TestGetListenerSocketActivation5488 server_test.go:147: systemd socket activation does not exist on darwin5489--- SKIP: TestGetListenerSocketActivation (0.00s)5490=== RUN TestServerStalledClientDoesNotBlockShutdown5491=== PAUSE TestServerStalledClientDoesNotBlockShutdown5492=== RUN TestServerBacksOffOnAcceptErrors5493=== PAUSE TestServerBacksOffOnAcceptErrors5494=== RUN TestDrainIsolatesPoisonPath5495=== PAUSE TestDrainIsolatesPoisonPath5496=== RUN TestRunNotBlockedByPoisonHead5497=== PAUSE TestRunNotBlockedByPoisonHead5498=== RUN TestDrainGivesUpWhenServerDown5499=== PAUSE TestDrainGivesUpWhenServerDown5500=== RUN TestFailedPathPrunedByLaterClosure5501=== PAUSE TestFailedPathPrunedByLaterClosure5502=== RUN TestWorkerUploadsAndRemoves5503=== PAUSE TestWorkerUploadsAndRemoves5504=== RUN TestWorkerSkipsGCdPaths5505=== PAUSE TestWorkerSkipsGCdPaths5506=== RUN TestWorkerPrunesClosureDeps5507=== PAUSE TestWorkerPrunesClosureDeps5508=== RUN TestDrainTimeout5509=== PAUSE TestDrainTimeout5510=== RUN TestWorkerRemoveFailureIsNotProgress5511=== PAUSE TestWorkerRemoveFailureIsNotProgress5512=== RUN TestWorkerKeepsPathItCannotStat5513=== PAUSE TestWorkerKeepsPathItCannotStat5514=== CONT TestSendPathsEmpty5515=== CONT TestWorkerUploadsAndRemoves5516--- PASS: TestSendPathsEmpty (0.00s)5517=== CONT TestWorkerKeepsPathItCannotStat5518=== CONT TestWorkerRemoveFailureIsNotProgress5519=== CONT TestDrainTimeout5520=== CONT TestWorkerPrunesClosureDeps5521=== CONT TestWorkerSkipsGCdPaths5522=== CONT TestQueueFetchRemoveLifecycle5523=== CONT TestDrainIsolatesPoisonPath5524=== CONT TestServerBacksOffOnAcceptErrors5525=== CONT TestQueueRetryMovesToBack55262000/01/01 00:00:00 ERROR Accept failed error="too many open files"55272000/01/01 00:00:00 ERROR Accept failed error="too many open files"55282000/01/01 00:00:00 ERROR Accept failed error="too many open files"55292000/01/01 00:00:00 ERROR Accept failed error="too many open files"55302000/01/01 00:00:00 ERROR Accept failed error="too many open files"55312000/01/01 00:00:00 ERROR Accept failed error="too many open files"55322000/01/01 00:00:00 ERROR Accept failed error="too many open files"55332000/01/01 00:00:00 ERROR Accept failed error="too many open files"5534=== CONT TestServerStalledClientDoesNotBlockShutdown5535=== CONT TestQueueFetchBatchLimit5536=== CONT TestQueueRemoveLargeClosure5537=== CONT TestQueueRemove5538=== CONT TestQueueConcurrentWriters5539=== CONT TestQueueDeduplication5540=== CONT TestDrainGivesUpWhenServerDown5541=== CONT TestQueueEnqueueAndFetch5542=== CONT TestFailedPathPrunedByLaterClosure5543=== CONT TestRunNotBlockedByPoisonHead5544--- PASS: TestServerBacksOffOnAcceptErrors (0.00s)5545--- PASS: TestQueueEnqueueAndFetch (0.01s)55462026/11/12 00:28:51 WARN Cannot stat store path, will retry later path=/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestWorkerKeepsPathItCannotStat1533861753/002/locked/aaa error="stat /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestWorkerKeepsPathItCannotStat1533861753/002/locked/aaa: permission denied"55472026/11/12 00:28:51 WARN Cannot stat store path, will retry later path=/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestWorkerKeepsPathItCannotStat1533861753/002/locked/aaa error="stat /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestWorkerKeepsPathItCannotStat1533861753/002/locked/aaa: permission denied"55482026/11/12 00:28:51 WARN Cannot stat store path, will retry later path=/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestWorkerKeepsPathItCannotStat1533861753/002/locked/aaa error="stat /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestWorkerKeepsPathItCannotStat1533861753/002/locked/aaa: permission denied"55492026/11/12 00:28:51 INFO Upload queue status pending=255502026/11/12 00:28:51 INFO Upload queue status pending=355512026/11/12 00:28:51 INFO Uploading batch count=155522026/11/12 00:28:51 ERROR Upload failed error="upload failed" count=155532026/11/12 00:28:51 INFO Uploading batch count=255542026/11/12 00:28:51 ERROR Upload failed error="upload failed" count=255552026/11/12 00:28:51 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestDrainGivesUpWhenServerDown1308068088/002/a5556--- PASS: TestQueueDeduplication (0.01s)55572000/01/01 00:00:00 INFO Uploading batch count=255582026/11/12 00:28:51 INFO Uploading batch count=15559--- PASS: TestQueueRetryMovesToBack (0.01s)55602026/11/12 00:28:51 ERROR Upload failed error="upload failed" count=155612026/11/12 00:28:51 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestDrainGivesUpWhenServerDown1308068088/002/b55622026/11/12 00:28:51 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestWorkerSkipsGCdPaths1589255179/002/nonexistent55632000/01/01 00:00:00 ERROR Upload failed error="context deadline exceeded" count=255642026/11/12 00:28:51 INFO Upload queue status pending=255652026/11/12 00:28:51 INFO Uploading batch count=455662026/11/12 00:28:51 INFO Upload queue status pending=255672026/11/12 00:28:51 INFO Uploading batch count=255682026/11/12 00:28:51 INFO Uploading batch count=155692026/11/12 00:28:51 ERROR Drain finished with paths left in queue remaining=155702026/11/12 00:28:51 ERROR Upload failed error="upload failed" count=455712000/01/01 00:00:00 ERROR Drain finished with paths left in queue remaining=45572--- PASS: TestQueueFetchRemoveLifecycle (0.01s)55732026/11/12 00:28:51 INFO Uploading batch count=155742026/11/12 00:28:51 INFO Uploading batch count=255752026/11/12 00:28:51 ERROR Upload failed error="upload failed" count=255762026/11/12 00:28:51 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestDrainGivesUpWhenServerDown1308068088/002/c55772026/11/12 00:28:51 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestDrainIsolatesPoisonPath3355959936/002/bbb55782026/11/12 00:28:51 INFO Uploading batch count=15579--- PASS: TestQueueFetchBatchLimit (0.01s)55802026/11/12 00:28:51 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestDrainGivesUpWhenServerDown1308068088/002/d5581--- PASS: TestQueueRemove (0.01s)55822026/11/12 00:28:51 INFO Uploading batch count=155832026/11/12 00:28:51 INFO Uploading batch count=255842026/11/12 00:28:51 ERROR Upload failed error="upload failed" count=255852026/11/12 00:28:51 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestDrainGivesUpWhenServerDown1308068088/002/e55862026/11/12 00:28:51 INFO Uploading batch count=155872026/11/12 00:28:51 ERROR Upload failed error="upload failed" count=15588--- PASS: TestWorkerKeepsPathItCannotStat (0.02s)55892026/11/12 00:28:51 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestDrainGivesUpWhenServerDown1308068088/002/f55902026/11/12 00:28:51 INFO Uploading batch count=155912026/11/12 00:28:51 ERROR Upload failed error="upload failed" count=155922026/11/12 00:28:51 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestWorkerRemoveFailureIsNotProgress1522055824/002/nonexistent55932026/11/12 00:28:51 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=155942026/11/12 00:28:51 ERROR Drain finished with paths left in queue remaining=1055952026/11/12 00:28:51 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestWorkerRemoveFailureIsNotProgress1522055824/002/nonexistent55962026/11/12 00:28:51 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=155972026/11/12 00:28:51 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestWorkerRemoveFailureIsNotProgress1522055824/002/nonexistent55982026/11/12 00:28:51 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=155992026/11/12 00:28:51 INFO Uploading batch count=156002026/11/12 00:28:51 ERROR Upload failed error="upload failed" count=15601--- PASS: TestDrainTimeout (0.02s)56022026/11/12 00:28:51 ERROR Drain finished with paths left in queue remaining=156032026/11/12 00:28:51 ERROR Drain finished with paths left in queue remaining=15604--- PASS: TestFailedPathPrunedByLaterClosure (0.02s)5605--- PASS: TestWorkerRemoveFailureIsNotProgress (0.02s)5606--- PASS: TestDrainIsolatesPoisonPath (0.02s)5607--- PASS: TestDrainGivesUpWhenServerDown (0.02s)5608--- PASS: TestWorkerSkipsGCdPaths (0.03s)5609--- PASS: TestWorkerPrunesClosureDeps (0.04s)5610--- PASS: TestWorkerUploadsAndRemoves (0.04s)5611--- PASS: TestQueueRemoveLargeClosure (0.07s)5612--- PASS: TestQueueConcurrentWriters (0.07s)56132026/11/12 00:28:52 ERROR Failed to decode request error="read unix /private/var/tb/3/scratch/81a37589b1b6aa8c/build/hook2766991972/test.sock->: i/o timeout"56142026/11/12 00:28:52 ERROR Failed to write response error="write unix /private/var/tb/3/scratch/81a37589b1b6aa8c/build/hook2766991972/test.sock->: i/o timeout"5615--- PASS: TestServerStalledClientDoesNotBlockShutdown (0.20s)56162026/11/12 00:28:52 INFO Uploading batch count=156172026/11/12 00:28:52 INFO Uploading batch count=156182026/11/12 00:28:52 INFO Uploading batch count=156192026/11/12 00:28:52 ERROR Upload failed error="upload failed" count=156202026/11/12 00:28:52 INFO Uploading batch count=156212026/11/12 00:28:52 ERROR Upload failed error="upload failed" count=156222026/11/12 00:28:52 INFO Uploading batch count=156232026/11/12 00:28:52 ERROR Upload failed error="upload failed" count=156242026/11/12 00:28:52 INFO Uploading batch count=156252026/11/12 00:28:52 ERROR Upload failed error="upload failed" count=156262026/11/12 00:28:52 ERROR Drain finished with paths left in queue remaining=15627--- PASS: TestRunNotBlockedByPoisonHead (1.04s)5628PASS5629Running signing tests...5630=== RUN TestExternalSigner5631=== PAUSE TestExternalSigner5632=== RUN TestExternalSignerWrongPublicKey5633=== PAUSE TestExternalSignerWrongPublicKey5634=== RUN TestExternalSignerErrors5635=== PAUSE TestExternalSignerErrors5636=== RUN TestExternalSignerRestartsAfterCrash5637=== PAUSE TestExternalSignerRestartsAfterCrash5638=== RUN TestExternalSignerProgramErrorKeepsProcess5639=== PAUSE TestExternalSignerProgramErrorKeepsProcess5640=== RUN TestExternalSignerClose5641=== PAUSE TestExternalSignerClose5642=== RUN TestExternalSignerRejectsInvalidPublicKey5643=== PAUSE TestExternalSignerRejectsInvalidPublicKey5644=== RUN TestExternalSignerCancellation5645=== PAUSE TestExternalSignerCancellation5646=== RUN TestGenerateFingerprint5647=== PAUSE TestGenerateFingerprint5648=== RUN TestParseSigningKey5649=== PAUSE TestParseSigningKey5650=== RUN TestSign5651=== PAUSE TestSign5652=== RUN TestSignBatch5653=== PAUSE TestSignBatch5654=== RUN TestNilKeyPublicKey5655=== PAUSE TestNilKeyPublicKey5656=== RUN TestSignErrors5657=== PAUSE TestSignErrors5658=== CONT TestExternalSigner5659=== CONT TestSignBatch5660=== CONT TestExternalSignerWrongPublicKey5661=== CONT TestExternalSignerProgramErrorKeepsProcess5662=== CONT TestExternalSignerCancellation5663=== CONT TestSign5664=== CONT TestParseSigningKey5665=== RUN TestParseSigningKey/valid_32-byte_key5666=== PAUSE TestParseSigningKey/valid_32-byte_key5667=== RUN TestParseSigningKey/valid_32-byte_key_with_different_name5668=== PAUSE TestParseSigningKey/valid_32-byte_key_with_different_name5669=== RUN TestParseSigningKey/no_colon5670=== PAUSE TestParseSigningKey/no_colon5671=== RUN TestParseSigningKey/empty_name5672=== PAUSE TestParseSigningKey/empty_name5673=== RUN TestParseSigningKey/invalid_base645674=== PAUSE TestParseSigningKey/invalid_base645675=== RUN TestParseSigningKey/wrong_length5676=== PAUSE TestParseSigningKey/wrong_length5677=== CONT TestParseSigningKey/valid_32-byte_key5678=== CONT TestParseSigningKey/wrong_length5679=== CONT TestGenerateFingerprint5680=== RUN TestGenerateFingerprint/basic_with_references5681=== PAUSE TestGenerateFingerprint/basic_with_references5682=== RUN TestGenerateFingerprint/no_references5683=== PAUSE TestGenerateFingerprint/no_references5684=== RUN TestGenerateFingerprint/unsorted_references_get_sorted5685=== PAUSE TestGenerateFingerprint/unsorted_references_get_sorted5686=== RUN TestGenerateFingerprint/invalid_nar_hash_prefix5687=== PAUSE TestGenerateFingerprint/invalid_nar_hash_prefix5688=== RUN TestGenerateFingerprint/invalid_nar_hash_length5689=== PAUSE TestGenerateFingerprint/invalid_nar_hash_length5690=== CONT TestExternalSignerRejectsInvalidPublicKey5691=== RUN TestExternalSignerRejectsInvalidPublicKey/#005692=== PAUSE TestExternalSignerRejectsInvalidPublicKey/#005693=== RUN TestExternalSignerRejectsInvalidPublicKey/no-colon5694=== PAUSE TestExternalSignerRejectsInvalidPublicKey/no-colon5695=== CONT TestExternalSignerClose5696=== CONT TestSignErrors5697=== CONT TestNilKeyPublicKey5698--- PASS: TestNilKeyPublicKey (0.00s)5699=== CONT TestExternalSignerErrors5700=== RUN TestExternalSignerErrors/program_error5701=== CONT TestExternalSignerRestartsAfterCrash5702=== CONT TestParseSigningKey/invalid_base645703=== CONT TestParseSigningKey/empty_name5704=== CONT TestParseSigningKey/no_colon5705=== CONT TestParseSigningKey/valid_32-byte_key_with_different_name5706=== RUN TestGenerateFingerprint/invalid_store_path_prefix5707=== PAUSE TestGenerateFingerprint/invalid_store_path_prefix5708=== RUN TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5709=== RUN TestGenerateFingerprint/invalid_reference_prefix5710=== PAUSE TestGenerateFingerprint/invalid_reference_prefix5711=== PAUSE TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5712=== CONT TestGenerateFingerprint/basic_with_references5713=== CONT TestGenerateFingerprint/invalid_reference_prefix5714=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5715=== CONT TestGenerateFingerprint/invalid_nar_hash_length5716=== PAUSE TestExternalSignerErrors/program_error5717=== RUN TestExternalSignerErrors/wrong_signatures_shape5718=== CONT TestGenerateFingerprint/unsorted_references_get_sorted5719=== CONT TestGenerateFingerprint/invalid_store_path_prefix5720=== CONT TestGenerateFingerprint/invalid_nar_hash_prefix5721=== CONT TestGenerateFingerprint/no_references5722=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5723=== PAUSE TestExternalSignerErrors/wrong_signatures_shape5724--- PASS: TestGenerateFingerprint (0.00s)5725 --- PASS: TestGenerateFingerprint/basic_with_references (0.00s)5726 --- PASS: TestGenerateFingerprint/invalid_reference_prefix (0.00s)5727 --- PASS: TestGenerateFingerprint/invalid_nar_hash_length (0.00s)5728 --- PASS: TestGenerateFingerprint/unsorted_references_get_sorted (0.00s)5729 --- PASS: TestGenerateFingerprint/invalid_store_path_prefix (0.00s)5730 --- PASS: TestGenerateFingerprint/invalid_nar_hash_prefix (0.00s)5731 --- PASS: TestGenerateFingerprint/no_references (0.00s)5732=== RUN TestExternalSignerErrors/exit_at_startup5733=== PAUSE TestExternalSignerErrors/exit_at_startup5734=== RUN TestExternalSignerErrors/null5735=== PAUSE TestExternalSignerErrors/null5736=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5737=== RUN TestExternalSignerErrors/too_few_signatures5738=== PAUSE TestExternalSignerErrors/too_few_signatures5739=== RUN TestExternalSignerErrors/unrequested_key5740=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5741=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5742=== CONT TestExternalSignerRejectsInvalidPublicKey/#005743=== CONT TestExternalSignerRejectsInvalidPublicKey/no-colon5744=== CONT TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5745=== PAUSE TestExternalSignerErrors/unrequested_key5746=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5747--- PASS: TestExternalSignerRejectsInvalidPublicKey (0.00s)5748 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:invalid! (0.00s)5749 --- PASS: TestExternalSignerRejectsInvalidPublicKey/#00 (0.00s)5750 --- PASS: TestExternalSignerRejectsInvalidPublicKey/no-colon (0.00s)5751 --- PASS: TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= (0.00s)5752 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:YQ== (0.00s)5753=== RUN TestExternalSignerErrors/wrong_signature_name5754=== PAUSE TestExternalSignerErrors/wrong_signature_name5755=== RUN TestExternalSignerErrors/invalid_signature_base645756=== PAUSE TestExternalSignerErrors/invalid_signature_base645757=== RUN TestExternalSignerErrors/wrong_shape5758=== PAUSE TestExternalSignerErrors/wrong_shape5759=== RUN TestExternalSignerErrors/missing_signatures5760=== PAUSE TestExternalSignerErrors/missing_signatures5761=== RUN TestExternalSignerErrors/wrong_signature_length5762=== PAUSE TestExternalSignerErrors/wrong_signature_length5763=== RUN TestExternalSignerErrors/exit_on_request5764=== PAUSE TestExternalSignerErrors/exit_on_request5765=== RUN TestExternalSignerErrors/invalid_JSON5766=== PAUSE TestExternalSignerErrors/invalid_JSON5767=== RUN TestExternalSignerErrors/wrong_id5768=== PAUSE TestExternalSignerErrors/wrong_id5769=== RUN TestExternalSignerErrors/null_signatures5770=== PAUSE TestExternalSignerErrors/null_signatures5771=== RUN TestExternalSignerErrors/missing_signature_name5772=== PAUSE TestExternalSignerErrors/missing_signature_name5773=== CONT TestExternalSignerErrors/program_error5774=== CONT TestExternalSignerErrors/missing_signatures5775=== CONT TestExternalSignerErrors/wrong_shape5776--- PASS: TestSign (0.00s)5777=== CONT TestExternalSignerErrors/null5778=== RUN TestSignErrors/nil_key5779=== CONT TestExternalSignerErrors/exit_at_startup5780--- PASS: TestParseSigningKey (0.00s)5781 --- PASS: TestParseSigningKey/wrong_length (0.00s)5782 --- PASS: TestParseSigningKey/invalid_base64 (0.00s)5783 --- PASS: TestParseSigningKey/empty_name (0.00s)5784 --- PASS: TestParseSigningKey/no_colon (0.00s)5785 --- PASS: TestParseSigningKey/valid_32-byte_key (0.00s)5786 --- PASS: TestParseSigningKey/valid_32-byte_key_with_different_name (0.00s)5787=== CONT TestExternalSignerErrors/wrong_signatures_shape5788=== CONT TestExternalSignerErrors/too_few_signatures5789=== CONT TestExternalSignerErrors/wrong_id5790=== CONT TestExternalSignerErrors/invalid_JSON5791=== CONT TestExternalSignerErrors/exit_on_request5792=== CONT TestExternalSignerErrors/wrong_signature_length5793=== CONT TestExternalSignerErrors/invalid_signature_base645794--- PASS: TestSignBatch (0.00s)5795=== CONT TestExternalSignerErrors/wrong_signature_name5796=== CONT TestExternalSignerErrors/unrequested_key5797=== PAUSE TestSignErrors/nil_key5798=== CONT TestExternalSignerErrors/missing_signature_name5799=== CONT TestExternalSignerErrors/null_signatures5800=== RUN TestSignErrors/nil_narinfo5801=== PAUSE TestSignErrors/nil_narinfo5802=== RUN TestSignErrors/invalid_narinfo5803=== PAUSE TestSignErrors/invalid_narinfo5804=== RUN TestSignErrors/canceled5805=== PAUSE TestSignErrors/canceled5806=== CONT TestSignErrors/nil_key5807=== CONT TestSignErrors/nil_narinfo5808=== CONT TestSignErrors/canceled5809=== CONT TestSignErrors/invalid_narinfo5810--- PASS: TestSignErrors (0.00s)5811 --- PASS: TestSignErrors/nil_key (0.00s)5812 --- PASS: TestSignErrors/nil_narinfo (0.00s)5813 --- PASS: TestSignErrors/canceled (0.00s)5814 --- PASS: TestSignErrors/invalid_narinfo (0.00s)5815--- PASS: TestExternalSignerWrongPublicKey (0.22s)5816--- PASS: TestExternalSignerCancellation (0.66s)5817--- PASS: TestExternalSignerProgramErrorKeepsProcess (2.01s)5818--- PASS: TestExternalSigner (2.22s)5819--- PASS: TestExternalSignerClose (3.00s)5820--- PASS: TestExternalSignerErrors (0.00s)5821 --- PASS: TestExternalSignerErrors/exit_on_request (0.43s)5822 --- PASS: TestExternalSignerErrors/wrong_id (0.83s)5823 --- PASS: TestExternalSignerErrors/missing_signatures (1.02s)5824 --- PASS: TestExternalSignerErrors/too_few_signatures (1.22s)5825 --- PASS: TestExternalSignerErrors/wrong_signature_length (1.41s)5826 --- PASS: TestExternalSignerErrors/wrong_signatures_shape (1.61s)5827 --- PASS: TestExternalSignerErrors/exit_at_startup (1.81s)5828 --- PASS: TestExternalSignerErrors/program_error (2.40s)5829 --- PASS: TestExternalSignerErrors/wrong_shape (2.59s)5830 --- PASS: TestExternalSignerErrors/null (2.80s)5831 --- PASS: TestExternalSignerErrors/invalid_signature_base64 (3.19s)5832 --- PASS: TestExternalSignerErrors/wrong_signature_name (3.38s)5833 --- PASS: TestExternalSignerErrors/null_signatures (3.59s)5834 --- PASS: TestExternalSignerErrors/invalid_JSON (3.78s)5835 --- PASS: TestExternalSignerErrors/missing_signature_name (3.98s)5836 --- PASS: TestExternalSignerErrors/unrequested_key (4.18s)5837--- PASS: TestExternalSignerRestartsAfterCrash (4.41s)5838PASS5839Running OpenBao signer tests...5840/private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestExternalSignerRestartsAfterCrash1079263856/001/sign program: line 7: /private/var/tb/3/scratch/81a37589b1b6aa8c/build/TestExternalSignerRestartsAfterCrash1079263856/001/sign program.starts: No such file or directory5841=== RUN TestSign5842=== PAUSE TestSign5843=== RUN TestSignErrors5844=== PAUSE TestSignErrors5845=== RUN TestServe5846=== PAUSE TestServe5847=== RUN TestServeRejectsGarbage5848=== PAUSE TestServeRejectsGarbage5849=== CONT TestSign5850=== CONT TestServe5851=== CONT TestSignErrors5852=== RUN TestSignErrors/http_error5853=== CONT TestServeRejectsGarbage5854=== PAUSE TestSignErrors/http_error5855=== RUN TestSignErrors/invalid_json5856--- PASS: TestServeRejectsGarbage (0.00s)5857=== PAUSE TestSignErrors/invalid_json5858=== RUN TestSignErrors/no_prefix5859=== PAUSE TestSignErrors/no_prefix5860=== RUN TestSignErrors/invalid_base645861=== PAUSE TestSignErrors/invalid_base645862=== CONT TestSignErrors/http_error5863=== CONT TestSignErrors/no_prefix5864=== CONT TestSignErrors/invalid_base645865=== CONT TestSignErrors/invalid_json5866--- PASS: TestServe (0.00s)5867--- PASS: TestSign (0.00s)5868--- PASS: TestSignErrors (0.00s)5869 --- PASS: TestSignErrors/no_prefix (0.00s)5870 --- PASS: TestSignErrors/invalid_base64 (0.00s)5871 --- PASS: TestSignErrors/http_error (0.00s)5872 --- PASS: TestSignErrors/invalid_json (0.00s)5873PASS