niks3-go-unit-tests log

succeeded `niks3-go-unit-tests` `checks.aarch64-darwin.go-unit-tests` · build #491 · raw

1tribuchet: building on mac022Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestUploadBuildLog_FileBodyReplayedOnRetry6=== PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry7=== RUN TestRegisterUploadedObjectReusesConnections8=== PAUSE TestRegisterUploadedObjectReusesConnections9=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica10=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica11=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun12=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun13=== RUN TestCaseHackSuffix14=== PAUSE TestCaseHackSuffix15=== RUN TestFilterOversizedClosures16=== PAUSE TestFilterOversizedClosures17=== RUN TestUploadMultipart_PartsInParallel18=== PAUSE TestUploadMultipart_PartsInParallel19=== RUN TestUploadMultipart_ReturnsAfterDroppedBody20=== PAUSE TestUploadMultipart_ReturnsAfterDroppedBody21=== RUN TestPartSizeForNAR22=== PAUSE TestPartSizeForNAR23=== RUN TestDumpPathCaseHackMatchesNix24--- PASS: TestDumpPathCaseHackMatchesNix (0.05s)25=== RUN TestDumpPathCaseHackCollision26--- PASS: TestDumpPathCaseHackCollision (0.00s)27=== RUN TestDumpPathMatchesNix28=== PAUSE TestDumpPathMatchesNix29=== RUN TestDumpPathSingleFile30=== PAUSE TestDumpPathSingleFile31=== RUN TestDumpPathWriterError32=== PAUSE TestDumpPathWriterError33=== RUN TestEncodeNixBase3234=== PAUSE TestEncodeNixBase3235=== RUN TestEncodeNixBase32WithRealHash36=== PAUSE TestEncodeNixBase32WithRealHash37=== RUN TestConvertHashToNix3238=== PAUSE TestConvertHashToNix3239=== RUN TestGetStorePathHash40=== PAUSE TestGetStorePathHash41=== RUN TestPathInfoHashCompatibility42=== PAUSE TestPathInfoHashCompatibility43=== RUN TestParsePathInfoJSON44=== PAUSE TestParsePathInfoJSON45=== RUN TestParsePathInfoJSONMultiplePaths46=== PAUSE TestParsePathInfoJSONMultiplePaths47=== RUN TestPathInfoCACompatibility48=== PAUSE TestPathInfoCACompatibility49=== RUN TestRateLimiterFeedback50=== PAUSE TestRateLimiterFeedback51=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess52=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess53=== RUN TestRegisterUploadedObject_BoundedAgainstSilentServer54=== PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer55=== RUN TestRegisterUploadedObject_OutlivesPush56=== PAUSE TestRegisterUploadedObject_OutlivesPush57=== RUN TestResolveStorePath58=== PAUSE TestResolveStorePath59=== RUN TestDoWithRetry_BodyReplayedViaGetBody60=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody61=== RUN TestRetryTimeoutOutlastsMaxRetries62=== PAUSE TestRetryTimeoutOutlastsMaxRetries63=== RUN TestDoWithRetry_FinalResponseBodyReadable64=== PAUSE TestDoWithRetry_FinalResponseBodyReadable65=== RUN TestShellSplit66=== PAUSE TestShellSplit67=== RUN TestShellSplitErrors68=== PAUSE TestShellSplitErrors69=== RUN TestStreamPushReportsEveryPath70=== PAUSE TestStreamPushReportsEveryPath71=== RUN TestStreamPushBatchesUnderLoad72=== PAUSE TestStreamPushBatchesUnderLoad73=== RUN TestStreamPushIsolatesFailures74=== PAUSE TestStreamPushIsolatesFailures75=== RUN TestStreamPushGivesUpOnDeadServer76=== PAUSE TestStreamPushGivesUpOnDeadServer77=== RUN TestStreamPushRequestLine78=== PAUSE TestStreamPushRequestLine79=== RUN TestStreamPushReportsSignatures80=== PAUSE TestStreamPushReportsSignatures81=== RUN TestStreamPushReportsSkippedPaths82=== PAUSE TestStreamPushReportsSkippedPaths83=== RUN TestClientSignaturesByStorePath84=== PAUSE TestClientSignaturesByStorePath85=== RUN TestStreamPushStopsWhenCanceled86=== PAUSE TestStreamPushStopsWhenCanceled87=== RUN TestSetClientTLS88=== PAUSE TestSetClientTLS89=== RUN TestSetClientTLSDoesNotMutateDefaultTransport90=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport91=== RUN TestSetClientTLSErrors92=== PAUSE TestSetClientTLSErrors93=== RUN TestStaticToken94=== PAUSE TestStaticToken95=== RUN TestFileTokenReadsAndCaches96=== PAUSE TestFileTokenReadsAndCaches97=== RUN TestFileTokenMissing98=== PAUSE TestFileTokenMissing99=== RUN TestFileTokenEmpty100=== PAUSE TestFileTokenEmpty101=== RUN TestScriptTokenNoExpiryRerunsEveryCall102=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall103=== RUN TestScriptTokenCachesUntilRefresh104=== PAUSE TestScriptTokenCachesUntilRefresh105=== RUN TestScriptTokenEmptyToken106=== PAUSE TestScriptTokenEmptyToken107=== RUN TestScriptTokenBadJSON108=== PAUSE TestScriptTokenBadJSON109=== RUN TestScriptTokenScriptFails110=== PAUSE TestScriptTokenScriptFails111=== RUN TestScriptTokenEmptyCommand112=== PAUSE TestScriptTokenEmptyCommand113=== CONT TestDoServerRequestAttachesToken114=== CONT TestStreamPushReportsSignatures115=== CONT TestParsePathInfoJSON116=== RUN TestParsePathInfoJSON/Nix_format117=== PAUSE TestParsePathInfoJSON/Nix_format118=== RUN TestParsePathInfoJSON/Lix_format119=== PAUSE TestParsePathInfoJSON/Lix_format120=== CONT TestStreamPushRequestLine121=== CONT TestDoWithRetry_BodyReplayedViaGetBody122=== CONT TestResolveStorePath123=== CONT TestRegisterUploadedObject_OutlivesPush124=== CONT TestRegisterUploadedObject_BoundedAgainstSilentServer125=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess126=== CONT TestRateLimiterFeedback127=== CONT TestPathInfoCACompatibility128=== CONT TestParsePathInfoJSONMultiplePaths129=== CONT TestStreamPushReportsEveryPath130=== CONT TestStreamPushGivesUpOnDeadServer131=== CONT TestStreamPushIsolatesFailures132=== CONT TestStreamPushBatchesUnderLoad133=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=false134=== CONT TestShellSplit135=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=false136=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=true1372026/11/11 23:37:11 ERROR Upload failed error="connection refused" count=201382026/11/11 23:37:11 ERROR Upload failed error="bad path" count=31392026/11/11 23:37:11 ERROR Server seems unavailable, giving up on batch untried=171402026/11/11 23:37:11 ERROR Upload failed error=boom count=11412026/11/11 23:37:11 ERROR Upload failed error=boom count=11422026/11/11 23:37:11 WARN Rate limiter enabled after throttle name=server-test rate=5143=== CONT TestShellSplitErrors144=== CONT TestDoWithRetry_FinalResponseBodyReadable145=== CONT TestPartSizeForNAR146=== RUN TestPartSizeForNAR/zero_stays_at_minimum147=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum148=== RUN TestPartSizeForNAR/small_stays_at_minimum149=== PAUSE TestPartSizeForNAR/small_stays_at_minimum150=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum151=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum152=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts153=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts154=== RUN TestPartSizeForNAR/1_TiB155=== PAUSE TestPartSizeForNAR/1_TiB156=== RUN TestPartSizeForNAR/5_TiB_S3_max_object157=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object158=== RUN TestPartSizeForNAR/capped_at_5_GiB159=== PAUSE TestPartSizeForNAR/capped_at_5_GiB160=== CONT TestPathInfoHashCompatibility161=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)162=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)163=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon164=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon165=== CONT TestUploadMultipart_ReturnsAfterDroppedBody166=== CONT TestConvertHashToNix32167=== RUN TestConvertHashToNix32/SRI_format_to_Nix32168=== CONT TestEncodeNixBase32WithRealHash169=== CONT TestRetryTimeoutOutlastsMaxRetries170=== RUN TestParsePathInfoJSON/empty_input171--- PASS: TestShellSplit (0.00s)172--- PASS: TestStreamPushReportsEveryPath (0.00s)173--- PASS: TestStreamPushIsolatesFailures (0.00s)1742026/11/11 23:37:11 WARN Rate limiter enabled after throttle name=server-test rate=51752026/11/11 23:37:11 WARN Rate limiter enabled after throttle name=server-test rate=51762026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:593961772026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:59388178=== CONT TestDumpPathWriterError179=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32180=== RUN TestRateLimiterFeedback/429_enables_limiter181=== CONT TestDumpPathSingleFile182=== CONT TestDumpPathMatchesNix183=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=true184=== CONT TestCaseHackSuffix185=== CONT TestGetStorePathHash186=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI187=== CONT TestEncodeNixBase32188=== RUN TestEncodeNixBase32/test_string_hash189=== PAUSE TestEncodeNixBase32/test_string_hash190=== RUN TestEncodeNixBase32/empty_input191=== PAUSE TestEncodeNixBase32/empty_input192=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths193=== RUN TestGetStorePathHash/valid_store_path194=== PAUSE TestGetStorePathHash/valid_store_path195=== RUN TestGetStorePathHash/basename_without_hyphen_should_error196=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error197=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error198=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error199=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error200=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error201=== CONT TestFileTokenMissing202=== CONT TestScriptTokenEmptyToken203=== CONT TestScriptTokenEmptyCommand204=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica205=== CONT TestScriptTokenBadJSON2062026/11/11 23:37:11 WARN Rate limiter backed off name=server-test rate=5207=== RUN TestPathInfoCACompatibility/null_ca_field208=== PAUSE TestPathInfoCACompatibility/null_ca_field209=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths210=== PAUSE TestParsePathInfoJSON/empty_input211=== RUN TestParsePathInfoJSON/whitespace_only212=== PAUSE TestParsePathInfoJSON/whitespace_only213=== RUN TestParsePathInfoJSON/invalid_JSON214=== PAUSE TestParsePathInfoJSON/invalid_JSON215=== CONT TestFilterOversizedClosures216=== RUN TestFilterOversizedClosures/no_limit_keeps_everything217=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything218=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped219=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped220=== RUN TestFilterOversizedClosures/all_closures_skipped221=== PAUSE TestFilterOversizedClosures/all_closures_skipped222=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun2232026/11/11 23:37:11 WARN Rate limiter backed off name=server-test rate=52242026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:59388225=== RUN TestConvertHashToNix32/already_Nix32_format226=== PAUSE TestConvertHashToNix32/already_Nix32_format227=== CONT TestScriptTokenNoExpiryRerunsEveryCall228=== RUN TestConvertHashToNix32/invalid_format229=== PAUSE TestConvertHashToNix32/invalid_format230=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI231=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512232=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512233=== CONT TestFileTokenEmpty2342026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:59403235=== CONT TestFileTokenReadsAndCaches236=== CONT TestSetClientTLSDoesNotMutateDefaultTransport237=== CONT TestUploadMultipart_PartsInParallel238--- PASS: TestStreamPushGivesUpOnDeadServer (0.00s)239--- PASS: TestStreamPushReportsSignatures (0.00s)240--- PASS: TestShellSplitErrors (0.00s)241--- PASS: TestEncodeNixBase32WithRealHash (0.00s)242--- PASS: TestRegisterUploadedObject_OutlivesPush (0.00s)243--- PASS: TestDoServerRequestAttachesToken (0.00s)2442026/11/11 23:37:11 INFO Garbage collection started245=== CONT TestScriptTokenScriptFails246=== CONT TestScriptTokenCachesUntilRefresh247=== RUN TestPathInfoCACompatibility/old_string_format_-_text248=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths249=== PAUSE TestRateLimiterFeedback/429_enables_limiter250--- PASS: TestResolveStorePath (0.00s)251--- PASS: TestScriptTokenEmptyCommand (0.00s)252--- PASS: TestDoWithRetry_FinalResponseBodyReadable (0.00s)253--- PASS: TestUploadMultipart_ReturnsAfterDroppedBody (0.00s)254=== CONT TestRegisterUploadedObjectReusesConnections255--- PASS: TestFileTokenEmpty (0.00s)256--- PASS: TestFileTokenReadsAndCaches (0.00s)257--- PASS: TestFileTokenMissing (0.00s)258=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths259=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text260=== CONT TestStreamPushStopsWhenCanceled261=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive262--- PASS: TestStreamPushStopsWhenCanceled (0.00s)263=== CONT TestSetClientTLS264=== RUN TestRateLimiterFeedback/503_enables_limiter265=== PAUSE TestRateLimiterFeedback/503_enables_limiter266=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive267=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter268=== RUN TestPathInfoCACompatibility/new_structured_format_-_text269=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter270--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.01s)271=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text272=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter273=== CONT TestClientSignaturesByStorePath274=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter275--- PASS: TestClientSignaturesByStorePath (0.00s)276=== CONT TestUploadBuildLog_FileBodyReplayedOnRetry2772026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:594032782026/11/11 23:37:11 INFO Garbage collection started279=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method280=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method281=== CONT TestStreamPushReportsSkippedPaths2822026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:59403283=== CONT TestPartSizeForNAR/zero_stays_at_minimum284=== CONT TestSetClientTLSErrors285=== CONT TestPartSizeForNAR/1_TiB286=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum2872026/11/11 23:37:11 WARN Rate limiter enabled after throttle name=s3-test rate=52882026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:594192892026/11/11 23:37:11 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=0290=== RUN TestSetClientTLSErrors/missing_cert_file291=== PAUSE TestSetClientTLSErrors/missing_cert_file292=== RUN TestSetClientTLSErrors/missing_key_file293=== PAUSE TestSetClientTLSErrors/missing_key_file294=== RUN TestSetClientTLSErrors/missing_ca_file295=== PAUSE TestSetClientTLSErrors/missing_ca_file296=== RUN TestSetClientTLSErrors/invalid_ca_file297=== PAUSE TestSetClientTLSErrors/invalid_ca_file298=== CONT TestStaticToken299--- PASS: TestStaticToken (0.00s)300=== CONT TestPartSizeForNAR/small_stays_at_minimum301=== CONT TestPartSizeForNAR/5_TiB_S3_max_object302=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=false303=== CONT TestPartSizeForNAR/capped_at_5_GiB304=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=true305=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts3062026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:59403307=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error308=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error309=== CONT TestGetStorePathHash/basename_without_hyphen_should_error310=== CONT TestParsePathInfoJSON/Nix_format311=== CONT TestFilterOversizedClosures/no_limit_keeps_everything312=== CONT TestParsePathInfoJSON/empty_input313=== CONT TestParsePathInfoJSON/invalid_JSON314=== CONT TestParsePathInfoJSON/whitespace_only315=== CONT TestParsePathInfoJSON/Lix_format316=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped3172026/11/11 23:37:11 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000318=== CONT TestFilterOversizedClosures/all_closures_skipped319=== CONT TestConvertHashToNix32/SRI_format_to_Nix32320=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)321--- PASS: TestStreamPushReportsSkippedPaths (0.00s)322=== CONT TestConvertHashToNix32/already_Nix32_format323=== CONT TestEncodeNixBase32/test_string_hash324--- PASS: TestStreamPushBatchesUnderLoad (0.00s)325 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=false (0.00s)326 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=true (0.00s)327=== CONT TestEncodeNixBase32/empty_input328--- PASS: TestEncodeNixBase32 (0.00s)329 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)330 --- PASS: TestEncodeNixBase32/empty_input (0.00s)331=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha5123322026/11/11 23:37:11 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=503332026/11/11 23:37:11 WARN Garbage collection finished on another replica. Its statistics are not available here334--- PASS: TestPartSizeForNAR (0.00s)335 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)336 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)337 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)338 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)339 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)340 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)341 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)342--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (0.01s)343=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter344=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI345=== CONT TestRateLimiterFeedback/503_enables_limiter346=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon347=== CONT TestRateLimiterFeedback/429_enables_limiter348=== CONT TestGetStorePathHash/valid_store_path349=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths350=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths351=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter352=== CONT TestConvertHashToNix32/invalid_format353--- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (0.01s)354--- PASS: TestScriptTokenScriptFails (0.01s)355--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (0.01s)356=== CONT TestPathInfoCACompatibility/null_ca_field357--- PASS: TestParsePathInfoJSON (0.01s)358 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)359 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)360 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)361 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)362 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)363--- PASS: TestConvertHashToNix32 (0.00s)364 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)365 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)366 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)367--- PASS: TestScriptTokenEmptyToken (0.02s)368=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive369=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method370=== CONT TestPathInfoCACompatibility/old_string_format_-_text371=== CONT TestSetClientTLSErrors/missing_cert_file372=== CONT TestSetClientTLSErrors/invalid_ca_file3732026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:59403374--- PASS: TestGetStorePathHash (0.00s)375 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)376 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)377 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)378 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)379--- PASS: TestStreamPushRequestLine (0.02s)380--- PASS: TestPathInfoHashCompatibility (0.01s)381 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)382 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)383 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)384 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)385=== CONT TestSetClientTLSErrors/missing_key_file386=== CONT TestPathInfoCACompatibility/new_structured_format_-_text387=== CONT TestSetClientTLSErrors/missing_ca_file388--- PASS: TestPathInfoCACompatibility (0.01s)389 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)390 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)391 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)392 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)393 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)394--- PASS: TestFilterOversizedClosures (0.00s)395 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)396 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)397 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)398--- PASS: TestParsePathInfoJSONMultiplePaths (0.01s)399 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)400 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)401--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.02s)4022026/11/11 23:37:11 WARN Rate limiter enabled after throttle name=server-test rate=54032026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:59484404--- PASS: TestScriptTokenBadJSON (0.02s)405=== RUN TestSetClientTLS/rejects_connection_without_client_cert406=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert407=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA408=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA409=== RUN TestSetClientTLS/preserves_debug_logging_transport410=== PAUSE TestSetClientTLS/preserves_debug_logging_transport411=== CONT TestSetClientTLS/rejects_connection_without_client_cert412=== CONT TestSetClientTLS/preserves_debug_logging_transport413=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA4142026/11/11 23:37:11 WARN Rate limiter backed off name=server-test rate=54152026/11/11 23:37:11 WARN Rate limiter enabled after throttle name=server-test rate=54162026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:594774172026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:59403418--- PASS: TestSetClientTLSErrors (0.00s)419 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)420 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)421 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)422 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)4232026/11/11 23:37:11 WARN Rate limiter backed off name=server-test rate=5424--- PASS: TestRateLimiterFeedback (0.01s)425 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s)426 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.01s)427 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.01s)428 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.01s)4292026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:594034302026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:594034312026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:594034322026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:594034332026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:594034342026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:594034352026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:594034362026/11/11 23:37:11 http: TLS handshake error from 127.0.0.1:59490: remote error: tls: bad certificate4372026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:594034382026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:59403439--- PASS: TestSetClientTLS (0.02s)440 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.00s)441 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.00s)442 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.02s)4432026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:59403444--- PASS: TestRegisterUploadedObjectReusesConnections (0.04s)4452026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:594034462026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:594034472026/11/11 23:37:11 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:59403448--- PASS: TestRetryTimeoutOutlastsMaxRetries (0.05s)449--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.05s)450--- PASS: TestCaseHackSuffix (0.06s)451--- PASS: TestScriptTokenCachesUntilRefresh (0.05s)452--- PASS: TestDumpPathWriterError (0.06s)453--- PASS: TestDumpPathSingleFile (0.06s)454--- PASS: TestDumpPathMatchesNix (0.09s)4552026/11/11 23:37:11 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:59390/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:59390/api/uploads/complete4562026/11/11 23:37:11 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4572026/11/11 23:37:11 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:59390/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:59390/api/uploads/complete4582026/11/11 23:37:11 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4592026/11/11 23:37:11 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:59390/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:59390/api/uploads/complete4602026/11/11 23:37:11 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"461--- PASS: TestRegisterUploadedObject_BoundedAgainstSilentServer (0.10s)462--- PASS: TestUploadMultipart_PartsInParallel (0.63s)463--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)464PASS465Running server tests...466The files belonging to this database system will be owned by user "_tribuchetbld7".467This user must also own the server process.468469The database cluster will be initialized with locale "C".470The default database encoding has accordingly been set to "SQL_ASCII".471The default text search configuration will be set to "english".472473Data page checksums are enabled.474475creating directory /private/var/tb/7/scratch/064274e7a4e82b9c/build/postgres448363538/data ... ok476creating subdirectories ... ok477selecting dynamic shared memory implementation ... posix478selecting default "max_connections" ... 100479selecting default "shared_buffers" ... 128MB480selecting default time zone ... UTC481creating configuration files ... ok482running bootstrap script ... ok483performing post-bootstrap initialization ... ok484syncing data to disk ... ok485486initdb: warning: enabling "trust" authentication for local connections487initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.488489Success. You can now start the database server using:490491 pg_ctl -D /private/var/tb/7/scratch/064274e7a4e82b9c/build/postgres448363538/data -l logfile start492493/private/var/tb/7/scratch/064274e7a4e82b9c/build/postgres448363538:5432 - no response4942026-11-11 23:37:13.409 UTC [89673] LOG: starting PostgreSQL 18.6 on aarch64-apple-darwin25.6.0, compiled by clang version 21.1.8, 64-bit4952026-11-11 23:37:13.409 UTC [89673] LOG: listening on Unix socket "/private/var/tb/7/scratch/064274e7a4e82b9c/build/postgres448363538/.s.PGSQL.5432"4962026-11-11 23:37:13.411 UTC [89680] LOG: database system was shut down at 2026-11-11 23:37:13 UTC4972026-11-11 23:37:13.412 UTC [89673] LOG: database system is ready to accept connections498/private/var/tb/7/scratch/064274e7a4e82b9c/build/postgres448363538:5432 - accepting connections499{"timestamp":"2026-11-11T23:37:13.628754Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6b6e1db9-4514-4c37-bf13-75dd373421da","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(5)"}500=== RUN TestAbortSucceeded501=== PAUSE TestAbortSucceeded502=== RUN TestService_AuthMiddleware503=== PAUSE TestService_AuthMiddleware504=== RUN TestService_AuthMiddleware_MTLSProxyHeader505=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader506=== RUN TestService_AuthMiddleware_MTLSBoundSubjects507=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects508=== RUN TestService_ReadAuthMiddleware509=== PAUSE TestService_ReadAuthMiddleware510=== RUN TestService_AuthMiddleware_OIDC511=== PAUSE TestService_AuthMiddleware_OIDC512=== RUN TestService_RequireScope_OIDC513=== PAUSE TestService_RequireScope_OIDC514=== RUN TestService_ReadScope_PublicByDefault515=== PAUSE TestService_ReadScope_PublicByDefault516=== RUN TestCacheConfigHandler517=== PAUSE TestCacheConfigHandler518=== RUN TestCacheStatsHandler519=== PAUSE TestCacheStatsHandler520=== RUN TestClientCADerivations521=== PAUSE TestClientCADerivations522=== RUN TestClientErrorHandling523=== PAUSE TestClientErrorHandling524=== RUN TestClientIntegration525=== PAUSE TestClientIntegration526=== RUN TestClientMultipleUploads527=== PAUSE TestClientMultipleUploads528=== RUN TestClientWithDependencies529=== PAUSE TestClientWithDependencies530=== RUN TestClientSharedPathCommittedMidPush531=== PAUSE TestClientSharedPathCommittedMidPush532=== RUN TestPinProtectsFromGC533=== PAUSE TestPinProtectsFromGC534=== RUN TestClientPushesUseOnePush535=== PAUSE TestClientPushesUseOnePush536=== RUN TestClientFallsBackToClosures537=== PAUSE TestClientFallsBackToClosures538=== RUN TestDeleteClosuresRacingPins539=== PAUSE TestDeleteClosuresRacingPins540=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock541=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock542=== RUN TestResolveDBConnectionString543=== PAUSE TestResolveDBConnectionString544=== RUN TestConnectWaitsForAPeerMigration545=== PAUSE TestConnectWaitsForAPeerMigration546=== RUN TestLeadElectsOneAndHandsOver547=== PAUSE TestLeadElectsOneAndHandsOver548=== RUN TestLeadIncumbentWinsAfterRestart5492026/11/11 23:37:13 INFO lead: acquired remote=192.0.2.1:1234 generation=15502026/11/11 23:37:14 INFO lead: released remote=192.0.2.1:1234 generation=15512026/11/11 23:37:14 INFO lead: acquired remote=192.0.2.1:1234 generation=25522026/11/11 23:37:15 INFO lead: released remote=192.0.2.1:1234 generation=2553--- PASS: TestLeadIncumbentWinsAfterRestart (1.93s)554=== RUN TestLeadUnannouncedStandbyYieldsToIncumbent555=== PAUSE TestLeadUnannouncedStandbyYieldsToIncumbent556=== RUN TestLeadIncumbentAnnouncesAtOnce557=== PAUSE TestLeadIncumbentAnnouncesAtOnce558=== RUN TestLeadGenerationGrowsWithEveryGrant559=== PAUSE TestLeadGenerationGrowsWithEveryGrant560=== RUN TestLeadEndsOnShutdown561=== PAUSE TestLeadEndsOnShutdown562=== RUN TestLeadEndsWhenItsConnectionHangs5632026/11/11 23:37:15 INFO lead: acquired remote=192.0.2.1:1234 generation=15642026-11-11 23:37:17.231 UTC [89716] FATAL: terminating connection due to administrator command5652026/11/11 23:37:17 INFO lead: acquired remote=192.0.2.1:1234 generation=25662026/11/11 23:37:18 WARN lead: lock connection lost error="timeout: context deadline exceeded"5672026/11/11 23:37:18 INFO lead: released remote=192.0.2.1:1234 generation=15682026/11/11 23:37:18 INFO lead: released remote=192.0.2.1:1234 generation=2569--- PASS: TestLeadEndsWhenItsConnectionHangs (2.96s)570=== RUN TestGCAdvisoryLockBlocksConcurrentRun571=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun572=== RUN TestGCBugBareHashReferences573=== PAUSE TestGCBugBareHashReferences574=== RUN TestGCMetrics575=== PAUSE TestGCMetrics576=== RUN TestPushDedupSurvivesConcurrentGC577=== PAUSE TestPushDedupSurvivesConcurrentGC578=== RUN TestDeduplicatedObjectsRecordedAsPending579=== PAUSE TestDeduplicatedObjectsRecordedAsPending580=== RUN TestGCSweepSkipsPendingObjects581=== PAUSE TestGCSweepSkipsPendingObjects582=== RUN TestTombstonedObjectOfferedWithoutWaiting583=== PAUSE TestTombstonedObjectOfferedWithoutWaiting584=== RUN TestGCSweepDeliversEachKeyOnce585=== PAUSE TestGCSweepDeliversEachKeyOnce586=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection587=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection588=== RUN TestForceGCDuringPushOffersSweptObject589=== PAUSE TestForceGCDuringPushOffersSweptObject590=== RUN TestFailedS3DeleteKeepsTombstone591=== PAUSE TestFailedS3DeleteKeepsTombstone592=== RUN TestSweepSkipsRowsLockedByPush593=== PAUSE TestSweepSkipsRowsLockedByPush594=== RUN TestRegisterCompletedObjectNeedsPendingRow595=== PAUSE TestRegisterCompletedObjectNeedsPendingRow596=== RUN TestGetExistingObjectsLocksRows597=== PAUSE TestGetExistingObjectsLocksRows598=== RUN TestSweepRechecksPendingAfterLocking599=== PAUSE TestSweepRechecksPendingAfterLocking600=== RUN TestPushWaitsForSweepHoldingRow601=== PAUSE TestPushWaitsForSweepHoldingRow602=== RUN TestCommitRacingPendingCleanupKeepsObjects603=== PAUSE TestCommitRacingPendingCleanupKeepsObjects604=== RUN TestGCEndsOnShutdown605=== PAUSE TestGCEndsOnShutdown606=== RUN TestGCTaskStore_StartNew607=== PAUSE TestGCTaskStore_StartNew608=== RUN TestGCTaskStore_DeduplicateSameParams609=== PAUSE TestGCTaskStore_DeduplicateSameParams610=== RUN TestGCTaskStore_ConflictDifferentParams611=== PAUSE TestGCTaskStore_ConflictDifferentParams612=== RUN TestGCTaskStore_GetEmpty613=== PAUSE TestGCTaskStore_GetEmpty614=== RUN TestGCTaskStore_GetReturnsLatest615=== PAUSE TestGCTaskStore_GetReturnsLatest616=== RUN TestGCTaskStore_CompletedAllowsNewTask617=== PAUSE TestGCTaskStore_CompletedAllowsNewTask618=== RUN TestGCTaskStore_PhaseUpdates619=== PAUSE TestGCTaskStore_PhaseUpdates620=== RUN TestGCTaskStore_Fail621=== PAUSE TestGCTaskStore_Fail622=== RUN TestGracefulShutdownDrainsInflight623=== PAUSE TestGracefulShutdownDrainsInflight624=== RUN TestService_healthCheckHandler625=== PAUSE TestService_healthCheckHandler626=== RUN TestService_readinessHandler627=== PAUSE TestService_readinessHandler628=== RUN TestGenerateLandingPage629=== PAUSE TestGenerateLandingPage630=== RUN TestEnsureAbortUploadsRule631=== PAUSE TestEnsureAbortUploadsRule632=== RUN TestCacheConfigHandlerMaxNarSize633=== PAUSE TestCacheConfigHandlerMaxNarSize634=== RUN TestCreatePendingClosureRejectsOversizedNAR635=== PAUSE TestCreatePendingClosureRejectsOversizedNAR636=== RUN TestNARDeduplicationMetadataUploadBug637=== PAUSE TestNARDeduplicationMetadataUploadBug638=== RUN TestMetricsInventory639=== PAUSE TestMetricsInventory640=== RUN TestMetricsCommit641=== PAUSE TestMetricsCommit642=== RUN TestService_NativeMTLS643=== PAUSE TestService_NativeMTLS644=== RUN TestServerTLSConfig645=== PAUSE TestServerTLSConfig646=== RUN TestMultipartCleanup647=== PAUSE TestMultipartCleanup648=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded649=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded650=== RUN TestPendingCleanupUsesOneCutoff651=== PAUSE TestPendingCleanupUsesOneCutoff652=== RUN TestObjectStatsTrigger653=== PAUSE TestObjectStatsTrigger654=== RUN TestObjectStatsUpdatedOncePerCommit655=== PAUSE TestObjectStatsUpdatedOncePerCommit656=== RUN TestOrphanedObjectsGC657=== PAUSE TestOrphanedObjectsGC658=== RUN TestOrphanedObjectsGCStressTest659=== PAUSE TestOrphanedObjectsGCStressTest660=== RUN TestCreatePin_ReservedPins661=== PAUSE TestCreatePin_ReservedPins662=== RUN TestPresentReportsOnlyClosureRoots663=== PAUSE TestPresentReportsOnlyClosureRoots664=== RUN TestPresentRefreshesOnlyStaleClosures665=== PAUSE TestPresentRefreshesOnlyStaleClosures666=== RUN TestPresentNotReportedWhileGCDeletesClosure667=== PAUSE TestPresentNotReportedWhileGCDeletesClosure668=== RUN TestParseSingleRange669=== PAUSE TestParseSingleRange670=== RUN TestProxyHeadersOnlyTrustedOnSocket6712026/11/11 23:37:18 INFO Starting HTTP server address=127.0.0.1:595106722026/11/11 23:37:18 INFO Starting HTTP server address=proxy.sock6732026/11/11 23:37:18 WARN mTLS auth: subject not in bound subjects subject="CN=someone"6742026/11/11 23:37:18 INFO Shutdown signal received, draining in-flight requests timeout=10s675--- PASS: TestProxyHeadersOnlyTrustedOnSocket (0.16s)676=== RUN TestIsValidCachePath677=== PAUSE TestIsValidCachePath678=== RUN TestReadProxyNarinfo679=== PAUSE TestReadProxyNarinfo680=== RUN TestReadProxyNarinfoAlreadyDecompressed681=== PAUSE TestReadProxyNarinfoAlreadyDecompressed682=== RUN TestReadProxyContentNegotiation683=== PAUSE TestReadProxyContentNegotiation684=== RUN TestReadProxyNarStreaming685=== PAUSE TestReadProxyNarStreaming686=== RUN TestReadProxy404687=== PAUSE TestReadProxy404688=== RUN TestReadProxyInvalidPath689=== PAUSE TestReadProxyInvalidPath690=== RUN TestReadProxyHead691=== PAUSE TestReadProxyHead692=== RUN TestReadProxyOutlastsServerWriteTimeout693=== PAUSE TestReadProxyOutlastsServerWriteTimeout694=== RUN TestReadProxyConditionalGet695=== PAUSE TestReadProxyConditionalGet696=== RUN TestReadProxyRootRedirectsToIndexHTML697=== PAUSE TestReadProxyRootRedirectsToIndexHTML698=== RUN TestReadProxyDisabled699=== PAUSE TestReadProxyDisabled700=== RUN TestReadRedirectNar701=== PAUSE TestReadRedirectNar702=== RUN TestReadRedirectKeepsNarinfoProxied703=== PAUSE TestReadRedirectKeepsNarinfoProxied704=== RUN TestReadProxyRangeRequest705=== PAUSE TestReadProxyRangeRequest706=== RUN TestReadRedirectUsesPublicS3URL707=== PAUSE TestReadRedirectUsesPublicS3URL708=== RUN TestPush_OverlappingRootsStoreOneRowPerKey709=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey710=== RUN TestPush_CompleteCommitsEveryRoot711=== PAUSE TestPush_CompleteCommitsEveryRoot712=== RUN TestPush_RepeatedRootCommitsOnce713=== PAUSE TestPush_RepeatedRootCommitsOnce714=== RUN TestPush_CompleteWaitsForCleanupOfThePush715=== PAUSE TestPush_CompleteWaitsForCleanupOfThePush716=== RUN TestPush_RootsAreLockedInByteOrder717=== PAUSE TestPush_RootsAreLockedInByteOrder718=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit719=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit720=== RUN TestPush_RejectsBadRequests721=== PAUSE TestPush_RejectsBadRequests722=== RUN TestPush_SignsNarinfosOfItsPendingObjects723=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects724=== RUN TestSharedNarBothUploadsComplete725=== PAUSE TestSharedNarBothUploadsComplete726=== RUN TestCompleteMultipartUpload_ErrorButObjectExists727=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists728=== RUN TestCompletedNarNotReofferedAcrossClosures729=== PAUSE TestCompletedNarNotReofferedAcrossClosures730=== RUN TestPresignedUploadRegisteredBeforeCommit731=== PAUSE TestPresignedUploadRegisteredBeforeCommit732=== RUN TestService_Rustfstest733=== PAUSE TestService_Rustfstest734=== RUN TestParseSize735=== PAUSE TestParseSize736=== RUN TestSkippedUploadsHandler737=== PAUSE TestSkippedUploadsHandler738=== RUN TestSystemdListenerNotActivated739--- PASS: TestSystemdListenerNotActivated (0.00s)740=== RUN TestWatchdogBeatsWhenHealthy741--- PASS: TestWatchdogBeatsWhenHealthy (0.03s)742=== RUN TestWatchdogSkipsWhenUnhealthy7432026/11/11 23:37:18 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7442026/11/11 23:37:18 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7452026/11/11 23:37:18 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7462026/11/11 23:37:18 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7472026/11/11 23:37:18 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7482026/11/11 23:37:18 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7492026/11/11 23:37:18 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7502026/11/11 23:37:18 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7512026/11/11 23:37:19 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7522026/11/11 23:37:19 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"753--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)754=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle755=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle756=== RUN TestProxyWriteTimeout757=== PAUSE TestProxyWriteTimeout758=== RUN TestPendingClosureWriteTimeout759=== PAUSE TestPendingClosureWriteTimeout760=== RUN TestIsValidUploadKey761=== PAUSE TestIsValidUploadKey762=== RUN TestUploadHandlersRejectInvalidKeys763=== PAUSE TestUploadHandlersRejectInvalidKeys764=== RUN TestUploadHandlersRejectOversizedBody765=== PAUSE TestUploadHandlersRejectOversizedBody766=== RUN TestService_cleanupPendingClosuresHandler767=== PAUSE TestService_cleanupPendingClosuresHandler768=== RUN TestService_createPendingClosureHandler769=== PAUSE TestService_createPendingClosureHandler770=== RUN TestService_verifyS3Integrity771=== PAUSE TestService_verifyS3Integrity772=== RUN TestService_verifyS3ReuploadsStaleNarinfo773=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo774=== RUN TestCompleteMultipartUnregistered775=== PAUSE TestCompleteMultipartUnregistered776=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT777=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT778=== CONT TestPush_RejectsBadRequests779=== CONT TestGCTaskStore_StartNew780=== CONT TestOrphanedObjectsGC781=== CONT TestAbortSucceeded782=== CONT TestPushWaitsForSweepHoldingRow783=== CONT TestGCTaskStore_ConflictDifferentParams784=== CONT TestCacheConfigHandlerMaxNarSize785=== CONT TestObjectStatsTrigger786=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit787=== CONT TestSweepRechecksPendingAfterLocking788=== CONT TestPush_RootsAreLockedInByteOrder789=== CONT TestPush_CompleteWaitsForCleanupOfThePush790=== CONT TestPush_RepeatedRootCommitsOnce791=== CONT TestPush_CompleteCommitsEveryRoot792=== CONT TestGCTaskStore_DeduplicateSameParams793=== CONT TestPush_OverlappingRootsStoreOneRowPerKey794=== CONT TestEnsureAbortUploadsRule795=== CONT TestObjectStatsUpdatedOncePerCommit796=== CONT TestPendingCleanupUsesOneCutoff797=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded798--- PASS: TestGCTaskStore_StartNew (0.00s)799=== CONT TestMultipartCleanup800=== CONT TestGCEndsOnShutdown801=== RUN TestGCEndsOnShutdown/before_the_run802=== CONT TestServerTLSConfig803=== CONT TestService_NativeMTLS804=== CONT TestMetricsCommit805=== CONT TestMetricsInventory806=== CONT TestNARDeduplicationMetadataUploadBug807=== CONT TestCommitRacingPendingCleanupKeepsObjects808=== CONT TestCreatePendingClosureRejectsOversizedNAR809--- PASS: TestAbortSucceeded (0.00s)810=== PAUSE TestGCEndsOnShutdown/before_the_run811=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete812=== RUN TestServerTLSConfig/no_client_CA813=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete814--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)815=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark816=== PAUSE TestServerTLSConfig/no_client_CA817=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark818--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)8192026/11/11 23:37:19 INFO Received uploads request method=POST path=/api/pending_closures820--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)821=== RUN TestServerTLSConfig/missing_CA_file822=== PAUSE TestServerTLSConfig/missing_CA_file823=== CONT TestReadProxy404824=== RUN TestServerTLSConfig/not_a_PEM_file825=== PAUSE TestServerTLSConfig/not_a_PEM_file826=== CONT TestGetExistingObjectsLocksRows827--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.00s)828=== CONT TestGenerateLandingPage829--- PASS: TestGenerateLandingPage (0.01s)830=== CONT TestReadRedirectUsesPublicS3URL831=== RUN TestPush_RejectsBadRequests/no_objects832=== PAUSE TestPush_RejectsBadRequests/no_objects833=== RUN TestPush_RejectsBadRequests/bad_root834=== PAUSE TestPush_RejectsBadRequests/bad_root835=== RUN TestPush_RejectsBadRequests/root_not_in_objects836=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects837=== RUN TestPush_RejectsBadRequests/no_roots838=== PAUSE TestPush_RejectsBadRequests/no_roots839=== CONT TestReadProxyRangeRequest840--- PASS: TestReadRedirectUsesPublicS3URL (1.38s)841=== CONT TestReadRedirectKeepsNarinfoProxied8422026/11/11 23:37:20 INFO Received push request method=POST path=/api/pushes8432026/11/11 23:37:20 INFO Received complete push request method=POST path=/api/pushes/1/complete8442026/11/11 23:37:21 INFO Completed push id=1 objects=4 duration=30.214041ms8452026/11/11 23:37:21 INFO Received push request method=POST path=/api/pushes8462026/11/11 23:37:21 INFO Aborted multipart uploads count=0 kept=08472026/11/11 23:37:21 WARN Force mode enabled - objects will be deleted immediately without grace period8482026/11/11 23:37:21 INFO Received uploads request method=POST path=/api/pending_closures8492026/11/11 23:37:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=08502026/11/11 23:37:21 INFO Vacuumed table table=pending_closures8512026/11/11 23:37:21 INFO Vacuumed table table=pending_objects8522026/11/11 23:37:21 INFO Received uploads request method=POST path=/api/pending_closures8532026/11/11 23:37:21 INFO Vacuumed table table=multipart_uploads8542026/11/11 23:37:21 INFO Vacuumed table table=closures8552026/11/11 23:37:21 INFO Received cleanup request method=DELETE path=/api/pending_closures8562026/11/11 23:37:21 INFO Vacuumed table table=objects8572026/11/11 23:37:21 INFO Received complete push request method=POST path=/api/pushes/2/complete8582026/11/11 23:37:21 INFO Completed push id=2 objects=4 duration=6.6905ms859--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (2.24s)860=== CONT TestReadRedirectNar8612026/11/11 23:37:21 INFO Received push request method=POST path=/api/pushes8622026/11/11 23:37:21 INFO Received complete push request method=POST path=/api/pushes/1/complete8632026/11/11 23:37:21 INFO Completed push id=1 objects=4 duration=14.642792ms864--- PASS: TestPush_RepeatedRootCommitsOnce (2.44s)865=== CONT TestRegisterCompletedObjectNeedsPendingRow8662026/11/11 23:37:21 INFO Received uploads request method=POST path=/api/pending_closures8672026/11/11 23:37:21 INFO Received cleanup request method=DELETE path=/api/pending_closures8682026/11/11 23:37:21 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket10/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""8692026/11/11 23:37:21 INFO Aborted multipart uploads count=0 kept=18702026/11/11 23:37:21 INFO Received cleanup request method=DELETE path=/api/pending_closures8712026/11/11 23:37:21 INFO Aborted multipart uploads count=1 kept=0872--- PASS: TestMultipartCleanup (2.81s)873=== CONT TestReadProxyDisabled8742026/11/11 23:37:21 INFO Received uploads request method=POST path=/api/pending_closures875=== NAME TestOrphanedObjectsGC876 orphaned_objects_gc_test.go:287: GC Test Summary:877 orphaned_objects_gc_test.go:288: - Kept: 2 objects from closure A878 orphaned_objects_gc_test.go:289: - Deleted: 2 objects from closure B879 orphaned_objects_gc_test.go:290: - Deleted: 6 orphaned chain objects (X1->X2->X3)880 orphaned_objects_gc_test.go:291: - Deleted: 2 orphaned single objects (Y)881 orphaned_objects_gc_test.go:292: - Total deleted: 10 objects882--- PASS: TestOrphanedObjectsGC (3.19s)883=== CONT TestService_readinessHandler884--- PASS: TestObjectStatsTrigger (3.30s)885=== CONT TestSweepSkipsRowsLockedByPush886--- PASS: TestPushWaitsForSweepHoldingRow (3.37s)887=== CONT TestReadProxyRootRedirectsToIndexHTML888--- PASS: TestObjectStatsUpdatedOncePerCommit (3.47s)889=== CONT TestService_healthCheckHandler8902026/11/11 23:37:22 INFO Installed lifecycle rule aborting incomplete NAR uploads bucket=bucket14 days=2891--- PASS: TestEnsureAbortUploadsRule (3.63s)892=== CONT TestFailedS3DeleteKeepsTombstone8932026/11/11 23:37:22 INFO Received uploads request method=POST path=/api/pending_closures894--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (3.78s)895=== CONT TestReadProxyConditionalGet8962026/11/11 23:37:23 INFO Received push request method=POST path=/api/pushes8972026/11/11 23:37:23 INFO Received complete push request method=POST path=/api/pushes/1/complete8982026/11/11 23:37:23 INFO Completed push id=1 objects=6 duration=31.40125ms899--- PASS: TestPush_CompleteCommitsEveryRoot (4.30s)900=== CONT TestGracefulShutdownDrainsInflight9012026/11/11 23:37:23 INFO Starting HTTP server address=127.0.0.1:595399022026/11/11 23:37:23 INFO Shutdown signal received, draining in-flight requests timeout=10s903--- PASS: TestGracefulShutdownDrainsInflight (0.08s)904=== CONT TestForceGCDuringPushOffersSweptObject905=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows906=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows907=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check908=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check909=== CONT TestReadProxyOutlastsServerWriteTimeout910--- PASS: TestReadProxy404 (4.39s)911=== CONT TestGCTaskStore_Fail912--- PASS: TestGCTaskStore_Fail (0.00s)913=== CONT TestReadProxyHead9142026/11/11 23:37:23 INFO Received push request method=POST path=/api/pushes915--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (4.53s)916=== CONT TestGCTaskStore_PhaseUpdates917--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)918=== CONT TestReadProxyInvalidPath919--- PASS: TestGetExistingObjectsLocksRows (4.68s)920=== CONT TestGCTaskStore_CompletedAllowsNewTask921--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)922=== CONT TestParseSingleRange923=== RUN TestParseSingleRange/none924=== PAUSE TestParseSingleRange/none925=== RUN TestParseSingleRange/unknown_unit926=== PAUSE TestParseSingleRange/unknown_unit927=== RUN TestParseSingleRange/multi-range_ignored928=== PAUSE TestParseSingleRange/multi-range_ignored929=== RUN TestParseSingleRange/malformed_no_dash930=== PAUSE TestParseSingleRange/malformed_no_dash931=== RUN TestParseSingleRange/malformed_both_empty932=== PAUSE TestParseSingleRange/malformed_both_empty933=== RUN TestParseSingleRange/malformed_end_before_start934=== PAUSE TestParseSingleRange/malformed_end_before_start935=== RUN TestParseSingleRange/closed936=== PAUSE TestParseSingleRange/closed937=== RUN TestParseSingleRange/open-ended938=== PAUSE TestParseSingleRange/open-ended939=== RUN TestParseSingleRange/end_clamped_to_size940=== PAUSE TestParseSingleRange/end_clamped_to_size941=== RUN TestParseSingleRange/suffix942=== PAUSE TestParseSingleRange/suffix943=== RUN TestParseSingleRange/suffix_exceeds_size944=== PAUSE TestParseSingleRange/suffix_exceeds_size945=== RUN TestParseSingleRange/single_byte946=== PAUSE TestParseSingleRange/single_byte947=== RUN TestParseSingleRange/start_past_EOF948=== PAUSE TestParseSingleRange/start_past_EOF949=== RUN TestParseSingleRange/start_far_past_EOF950=== PAUSE TestParseSingleRange/start_far_past_EOF951=== CONT TestGCTaskStore_GetReturnsLatest952--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)953=== CONT TestReadProxyNarStreaming9542026/11/11 23:37:23 INFO Received push request method=POST path=/api/pushes9552026/11/11 23:37:24 INFO Aborted multipart uploads count=0 kept=09562026/11/11 23:37:24 WARN Force mode enabled - objects will be deleted immediately without grace period9572026/11/11 23:37:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=09582026-11-11 23:37:24.191 UTC [89844] ERROR: Push does not exist: id=19592026-11-11 23:37:24.191 UTC [89844] CONTEXT: PL/pgSQL function commit_push(bigint) line 11 at RAISE9602026-11-11 23:37:24.191 UTC [89844] STATEMENT: SELECT commit_push($1)961--- PASS: TestPush_CompleteWaitsForCleanupOfThePush (5.17s)962=== CONT TestGCTaskStore_GetEmpty963--- PASS: TestGCTaskStore_GetEmpty (0.00s)964=== CONT TestReadProxyContentNegotiation9652026/11/11 23:37:24 INFO Vacuumed table table=pending_closures9662026/11/11 23:37:24 INFO Vacuumed table table=pending_objects9672026/11/11 23:37:24 INFO Vacuumed table table=multipart_uploads9682026/11/11 23:37:24 INFO Vacuumed table table=closures9692026/11/11 23:37:24 INFO Vacuumed table table=objects970--- PASS: TestCommitRacingPendingCleanupKeepsObjects (5.22s)971=== CONT TestReadProxyNarinfoAlreadyDecompressed972--- PASS: TestMetricsInventory (5.22s)973=== CONT TestPresentReportsOnlyClosureRoots9742026/11/11 23:37:24 INFO Aborted multipart uploads count=0 kept=09752026/11/11 23:37:24 WARN Force mode enabled - objects will be deleted immediately without grace period9762026/11/11 23:37:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=09772026/11/11 23:37:24 INFO Vacuumed table table=pending_closures9782026/11/11 23:37:24 INFO Vacuumed table table=pending_objects9792026/11/11 23:37:24 INFO Vacuumed table table=multipart_uploads9802026/11/11 23:37:24 INFO Vacuumed table table=closures9812026/11/11 23:37:24 INFO Vacuumed table table=objects9822026/11/11 23:37:24 INFO Received push request method=POST path=/api/pushes983--- PASS: TestSweepRechecksPendingAfterLocking (5.57s)984=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection9852026/11/11 23:37:24 INFO Received complete push request method=POST path=/api/pushes/1/complete9862026/11/11 23:37:24 INFO Completed push id=1 objects=4 duration=13.843917ms9872026/11/11 23:37:24 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete9882026/11/11 23:37:24 INFO Completed upload id=2 objects=2 duration=686.583µs989--- PASS: TestMetricsCommit (5.67s)990=== CONT TestReadProxyNarinfo9912026/11/11 23:37:24 INFO Received push request method=POST path=/api/pushes992=== NAME TestNARDeduplicationMetadataUploadBug993 metadata_upload_test.go:48: First store path: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestNARDeduplicationMetadataUploadBug2851936887/001/store/x9rzx5394q54m990xgcnax3j4bkbls60-file1.txt9942026/11/11 23:37:25 INFO Received push request method=POST path=/api/pushes9952026-11-11 23:37:25.067 UTC [89871] ERROR: could not obtain lock on row in relation "closures"9962026-11-11 23:37:25.067 UTC [89871] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9972026-11-11 23:37:25.067 UTC [89871] ERROR: could not obtain lock on row in relation "closures"9982026-11-11 23:37:25.067 UTC [89871] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9992026-11-11 23:37:25.068 UTC [89871] ERROR: could not obtain lock on row in relation "closures"10002026-11-11 23:37:25.068 UTC [89871] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT10012026-11-11 23:37:25.069 UTC [89871] ERROR: could not obtain lock on row in relation "closures"10022026-11-11 23:37:25.069 UTC [89871] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT10032026-11-11 23:37:25.069 UTC [89871] ERROR: could not obtain lock on row in relation "closures"10042026-11-11 23:37:25.069 UTC [89871] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT10052026/11/11 23:37:25 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)10062026/11/11 23:37:25 INFO Uploading x9rzx5394q54m990xgcnax3j4bkbls60-file1.txt (160B)1007--- PASS: TestPush_RootsAreLockedInByteOrder (6.05s)1008=== CONT TestPresentNotReportedWhileGCDeletesClosure1009=== RUN TestPresentNotReportedWhileGCDeletesClosure/stale1010=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/stale1011=== RUN TestPresentNotReportedWhileGCDeletesClosure/fresh1012=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/fresh1013=== CONT TestIsValidCachePath1014=== RUN TestIsValidCachePath/narinfo1015=== PAUSE TestIsValidCachePath/narinfo1016=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars1017=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars1018=== RUN TestIsValidCachePath/nar_zst1019=== PAUSE TestIsValidCachePath/nar_zst1020=== RUN TestIsValidCachePath/nar_xz1021=== PAUSE TestIsValidCachePath/nar_xz1022=== RUN TestIsValidCachePath/nar_bz21023=== PAUSE TestIsValidCachePath/nar_bz21024=== RUN TestIsValidCachePath/nar_uncompressed1025=== PAUSE TestIsValidCachePath/nar_uncompressed1026=== RUN TestIsValidCachePath/ls1027=== PAUSE TestIsValidCachePath/ls1028=== RUN TestIsValidCachePath/log1029=== PAUSE TestIsValidCachePath/log1030=== RUN TestIsValidCachePath/realisation1031=== PAUSE TestIsValidCachePath/realisation1032=== RUN TestIsValidCachePath/nix-cache-info1033=== PAUSE TestIsValidCachePath/nix-cache-info1034=== RUN TestIsValidCachePath/index.html1035=== PAUSE TestIsValidCachePath/index.html1036=== RUN TestIsValidCachePath/traversal_parent1037=== PAUSE TestIsValidCachePath/traversal_parent1038=== RUN TestIsValidCachePath/traversal_in_middle1039=== PAUSE TestIsValidCachePath/traversal_in_middle1040=== RUN TestIsValidCachePath/invalid_char_e1041=== PAUSE TestIsValidCachePath/invalid_char_e1042=== RUN TestIsValidCachePath/invalid_char_u1043=== PAUSE TestIsValidCachePath/invalid_char_u1044=== RUN TestIsValidCachePath/random_path1045=== PAUSE TestIsValidCachePath/random_path1046=== RUN TestIsValidCachePath/empty1047=== PAUSE TestIsValidCachePath/empty1048=== RUN TestIsValidCachePath/leading_slash1049=== PAUSE TestIsValidCachePath/leading_slash1050=== RUN TestIsValidCachePath/wrong_extension1051=== PAUSE TestIsValidCachePath/wrong_extension1052=== RUN TestIsValidCachePath/short_hash1053=== PAUSE TestIsValidCachePath/short_hash1054=== CONT TestPresentRefreshesOnlyStaleClosures10552026/11/11 23:37:25 WARN Failed to register uploaded object key=x9rzx5394q54m990xgcnax3j4bkbls60.ls error="server returned 404: 404 page not found\n"10562026/11/11 23:37:25 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"10572026/11/11 23:37:25 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign10582026/11/11 23:37:25 INFO Signed narinfos id=1 count=110592026/11/11 23:37:25 INFO Uploading 1 narinfos10602026/11/11 23:37:25 INFO Received complete push request method=POST path=/api/pushes/1/complete10612026/11/11 23:37:25 WARN Failed to register uploaded object key=x9rzx5394q54m990xgcnax3j4bkbls60.narinfo error="server returned 404: 404 page not found\n"10622026/11/11 23:37:25 INFO Completed push id=1 objects=3 duration=22.726792ms10632026/11/11 23:37:25 INFO Upload complete. (211ms)1064=== NAME TestNARDeduplicationMetadataUploadBug1065 metadata_upload_test.go:54: Retrieved narinfo from S3:1066 StorePath: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestNARDeduplicationMetadataUploadBug2851936887/001/store/x9rzx5394q54m990xgcnax3j4bkbls60-file1.txt1067 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1068 Compression: zstd1069 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1070 NarSize: 1601071 References: 1072 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf10732026/11/11 23:37:25 WARN mTLS auth: subject not in bound subjects subject="CN=reader"10742026/11/11 23:37:25 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1075--- PASS: TestService_NativeMTLS (6.18s)1076=== CONT TestCreatePin_ReservedPins1077=== NAME TestNARDeduplicationMetadataUploadBug1078 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)1079 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):1080 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}1081 metadata_upload_test.go:64: Second store path (same content): /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestNARDeduplicationMetadataUploadBug2851936887/001/store/9sdp0cxd679f6fijib73xzqqjzc7b7q5-file2.txt10822026/11/11 23:37:25 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:59564/oidc10832026/11/11 23:37:25 INFO Received push request method=POST path=/api/pushes10842026/11/11 23:37:25 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)10852026/11/11 23:37:25 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign10862026/11/11 23:37:25 WARN Failed to register uploaded object key=9sdp0cxd679f6fijib73xzqqjzc7b7q5.ls error="server returned 404: 404 page not found\n"10872026/11/11 23:37:25 INFO Signed narinfos id=2 count=110882026/11/11 23:37:25 INFO Uploading 1 narinfos1089--- PASS: TestReadProxyRangeRequest (5.42s)1090=== CONT TestClientFallsBackToClosures10912026/11/11 23:37:25 INFO Received complete push request method=POST path=/api/pushes/2/complete10922026/11/11 23:37:25 WARN Failed to register uploaded object key=9sdp0cxd679f6fijib73xzqqjzc7b7q5.narinfo error="server returned 404: 404 page not found\n"10932026/11/11 23:37:25 INFO Completed push id=2 objects=3 duration=901.959µs10942026/11/11 23:37:25 INFO Upload complete. (137ms)1095=== NAME TestNARDeduplicationMetadataUploadBug1096 metadata_upload_test.go:76: Retrieved narinfo from S3:1097 StorePath: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestNARDeduplicationMetadataUploadBug2851936887/001/store/9sdp0cxd679f6fijib73xzqqjzc7b7q5-file2.txt1098 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1099 Compression: zstd1100 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1101 NarSize: 1601102 References: 1103 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1104 metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes)1105 metadata_upload_test.go:77: Decompressed .ls content (49 bytes):1106 {"version":1,"root":{"type":"regular","size":44}}1107--- PASS: TestNARDeduplicationMetadataUploadBug (6.44s)1108=== CONT TestGCSweepDeliversEachKeyOnce1109--- PASS: TestReadRedirectKeepsNarinfoProxied (5.18s)1110=== CONT TestTombstonedObjectOfferedWithoutWaiting1111--- PASS: TestReadRedirectNar (4.50s)1112=== CONT TestGCSweepSkipsPendingObjects1113--- PASS: TestRegisterCompletedObjectNeedsPendingRow (4.45s)1114=== CONT TestClientPushesUseOnePush1115--- PASS: TestReadProxyDisabled (4.21s)1116=== CONT TestDeduplicatedObjectsRecordedAsPending11172026/11/11 23:37:26 WARN readiness check failed error="closed pool"1118--- PASS: TestService_readinessHandler (4.01s)1119=== CONT TestCacheStatsHandler1120--- PASS: TestReadProxyRootRedirectsToIndexHTML (4.05s)1121=== CONT TestPushDedupSurvivesConcurrentGC11222026/11/11 23:37:26 INFO Aborted multipart uploads count=0 kept=011232026/11/11 23:37:26 WARN Force mode enabled - objects will be deleted immediately without grace period11242026/11/11 23:37:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011252026/11/11 23:37:26 INFO Vacuumed table table=pending_closures11262026/11/11 23:37:26 INFO Vacuumed table table=pending_objects11272026/11/11 23:37:26 INFO Vacuumed table table=multipart_uploads11282026/11/11 23:37:26 INFO Vacuumed table table=closures11292026/11/11 23:37:26 INFO Vacuumed table table=objects11302026/11/11 23:37:26 INFO Aborted multipart uploads count=0 kept=011312026/11/11 23:37:26 WARN Force mode enabled - objects will be deleted immediately without grace period1132--- PASS: TestService_healthCheckHandler (4.37s)1133=== CONT TestGCMetrics11342026/11/11 23:37:27 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=011352026/11/11 23:37:27 INFO Vacuumed table table=pending_closures11362026/11/11 23:37:27 INFO Vacuumed table table=pending_objects11372026/11/11 23:37:27 INFO Vacuumed table table=multipart_uploads11382026/11/11 23:37:27 INFO Vacuumed table table=closures11392026/11/11 23:37:27 INFO Vacuumed table table=objects1140--- PASS: TestSweepSkipsRowsLockedByPush (4.72s)1141=== CONT TestGCBugBareHashReferences11422026/11/11 23:37:27 INFO Received uploads request method=POST path=/api/pending_closures11432026/11/11 23:37:27 INFO Aborted multipart uploads count=1 kept=011442026/11/11 23:37:27 INFO Received cleanup request method=DELETE path=/api/pending_closures11452026/11/11 23:37:27 INFO Aborted multipart uploads count=1 kept=01146--- PASS: TestPendingCleanupUsesOneCutoff (8.31s)1147=== CONT TestGCAdvisoryLockBlocksConcurrentRun11482026/11/11 23:37:27 INFO Aborted multipart uploads count=0 kept=011492026/11/11 23:37:27 WARN Force mode enabled - objects will be deleted immediately without grace period11502026/11/11 23:37:27 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."11512026/11/11 23:37:27 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."1152--- PASS: TestFailedS3DeleteKeepsTombstone (4.76s)1153=== CONT TestClientIntegration1154--- PASS: TestReadProxyConditionalGet (4.64s)1155=== CONT TestClientMultipleUploads1156--- PASS: TestReadProxyHead (4.49s)1157=== CONT TestLeadEndsOnShutdown1158--- PASS: TestReadProxyInvalidPath (4.50s)1159=== CONT TestClientErrorHandling1160=== RUN TestClientErrorHandling/InvalidStorePath1161=== PAUSE TestClientErrorHandling/InvalidStorePath1162=== RUN TestClientErrorHandling/InvalidAuthToken1163=== PAUSE TestClientErrorHandling/InvalidAuthToken1164=== RUN TestClientErrorHandling/ServerNotAvailable1165=== PAUSE TestClientErrorHandling/ServerNotAvailable1166=== CONT TestLeadGenerationGrowsWithEveryGrant1167--- PASS: TestReadProxyNarStreaming (4.70s)1168=== CONT TestPinProtectsFromGC1169--- PASS: TestReadProxyContentNegotiation (4.70s)1170=== CONT TestClientCADerivations1171--- PASS: TestReadProxyNarinfoAlreadyDecompressed (4.83s)1172=== CONT TestLeadIncumbentAnnouncesAtOnce1173--- PASS: TestPresentReportsOnlyClosureRoots (5.07s)1174=== CONT TestClientSharedPathCommittedMidPush11752026/11/11 23:37:29 INFO Received uploads request method=POST path=/api/pending_closures1176--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (5.25s)1177=== CONT TestLeadUnannouncedStandbyYieldsToIncumbent11782026/11/11 23:37:30 ERROR Failed to decode narinfo key=3hcdxyjf9yiq7qf3i4548drb6sjmwa1v.narinfo limit=16777216 error=<nil>1179--- PASS: TestReadProxyNarinfo (5.30s)1180=== CONT TestClientWithDependencies1181--- PASS: TestPresentRefreshesOnlyStaleClosures (5.10s)1182=== CONT TestOrphanedObjectsGCStressTest11832026/11/11 23:37:30 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux11842026/11/11 23:37:30 WARN Refused reserved pin name=worker-x86_64-linux11852026/11/11 23:37:30 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux11862026/11/11 23:37:30 INFO Received create pin request method=POST path=/api/pins/my-app11872026/11/11 23:37:30 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux1188--- PASS: TestCreatePin_ReservedPins (5.21s)1189=== CONT TestLeadElectsOneAndHandsOver11902026/11/11 23:37:31 INFO Aborted multipart uploads count=0 kept=011912026/11/11 23:37:31 WARN Force mode enabled - objects will be deleted immediately without grace period11922026/11/11 23:37:31 INFO Received uploads request method=POST path=/api/pending_closures1193--- PASS: TestTombstonedObjectOfferedWithoutWaiting (5.83s)1194=== CONT TestConnectWaitsForAPeerMigration11952026/11/11 23:37:31 INFO Received uploads request method=POST path=/api/pending_closures11962026/11/11 23:37:31 INFO Received uploads request method=POST path=/api/pending_closures11972026/11/11 23:37:31 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)11982026/11/11 23:37:31 INFO Uploading xl1abmid4ny5afa6g663p7bg5m6mvlnw-b (256B)11992026/11/11 23:37:31 INFO Uploading km257i7cxr0l2krydfirmjcs44ci8h5d-shared-dep (136B)12002026/11/11 23:37:31 WARN Failed to register uploaded object key=x4rxsr0anrszlz74bayya1v79r43x1a4.ls error="server returned 404: 404 page not found\n"12012026/11/11 23:37:31 WARN Failed to register uploaded object key=nar/07fq692cxk9i9iss0h2kw74qjk0zwhdz13xzzsqgby6zjiry9crz.nar.zst error="server returned 404: 404 page not found\n"12022026/11/11 23:37:31 WARN Failed to register uploaded object key=km257i7cxr0l2krydfirmjcs44ci8h5d.ls error="server returned 404: 404 page not found\n"12032026/11/11 23:37:31 WARN Failed to register uploaded object key=xl1abmid4ny5afa6g663p7bg5m6mvlnw.ls error="server returned 404: 404 page not found\n"12042026/11/11 23:37:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign12052026/11/11 23:37:31 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"12062026/11/11 23:37:31 INFO Signed narinfos id=1 count=212072026/11/11 23:37:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign12082026/11/11 23:37:31 INFO Signed narinfos id=2 count=212092026/11/11 23:37:31 INFO Uploading 4 narinfos12102026/11/11 23:37:31 WARN Failed to register uploaded object key=km257i7cxr0l2krydfirmjcs44ci8h5d.narinfo error="server returned 404: 404 page not found\n"12112026/11/11 23:37:31 WARN Failed to register uploaded object key=x4rxsr0anrszlz74bayya1v79r43x1a4.narinfo error="server returned 404: 404 page not found\n"12122026/11/11 23:37:31 WARN Failed to register uploaded object key=xl1abmid4ny5afa6g663p7bg5m6mvlnw.narinfo error="server returned 404: 404 page not found\n"12132026/11/11 23:37:31 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=012142026/11/11 23:37:31 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete12152026/11/11 23:37:31 INFO Vacuumed table table=pending_closures12162026/11/11 23:37:31 WARN Failed to register uploaded object key=km257i7cxr0l2krydfirmjcs44ci8h5d.narinfo error="server returned 404: 404 page not found\n"12172026/11/11 23:37:31 INFO Vacuumed table table=pending_objects12182026/11/11 23:37:31 INFO Vacuumed table table=multipart_uploads12192026/11/11 23:37:31 INFO Vacuumed table table=closures12202026/11/11 23:37:31 INFO Completed upload id=1 objects=6 duration=50.316583ms12212026/11/11 23:37:31 INFO Received uploads request method=POST path=/api/pending_closures12222026/11/11 23:37:31 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete12232026/11/11 23:37:31 INFO Vacuumed table table=objects12242026/11/11 23:37:31 INFO Completed upload id=2 objects=6 duration=6.66575ms12252026/11/11 23:37:31 INFO Upload complete. (508ms)1226--- PASS: TestGCSweepDeliversEachKeyOnce (6.39s)1227=== CONT TestService_ReadAuthMiddleware1228=== NAME TestClientFallsBackToClosures1229 client_pushes_test.go:112: Retrieved narinfo from S3:1230 StorePath: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientFallsBackToClosures1183454986/001/store/km257i7cxr0l2krydfirmjcs44ci8h5d-shared-dep1231 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1232 Compression: zstd1233 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821234 NarSize: 1361235 References: 1236 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1237 client_pushes_test.go:112: Retrieved narinfo from S3:1238 StorePath: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientFallsBackToClosures1183454986/001/store/x4rxsr0anrszlz74bayya1v79r43x1a4-a1239 URL: nar/07fq692cxk9i9iss0h2kw74qjk0zwhdz13xzzsqgby6zjiry9crz.nar.zst1240 Compression: zstd1241 NarHash: sha256:07fq692cxk9i9iss0h2kw74qjk0zwhdz13xzzsqgby6zjiry9crz1242 NarSize: 2561243 References: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientFallsBackToClosures1183454986/001/store/km257i7cxr0l2krydfirmjcs44ci8h5d-shared-dep1244 CA: text:sha256:0xd8z9nqf4ka20kq1g9750r6rjrmql7ank4mc34kc5zny4fak5781245 client_pushes_test.go:112: Retrieved narinfo from S3:1246 StorePath: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientFallsBackToClosures1183454986/001/store/xl1abmid4ny5afa6g663p7bg5m6mvlnw-b1247 URL: nar/07fq692cxk9i9iss0h2kw74qjk0zwhdz13xzzsqgby6zjiry9crz.nar.zst1248 Compression: zstd1249 NarHash: sha256:07fq692cxk9i9iss0h2kw74qjk0zwhdz13xzzsqgby6zjiry9crz1250 NarSize: 2561251 References: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientFallsBackToClosures1183454986/001/store/km257i7cxr0l2krydfirmjcs44ci8h5d-shared-dep1252 CA: text:sha256:0xd8z9nqf4ka20kq1g9750r6rjrmql7ank4mc34kc5zny4fak5781253--- PASS: TestReadProxyOutlastsServerWriteTimeout (8.50s)1254=== CONT TestResolveDBConnectionString1255=== RUN TestResolveDBConnectionString/flag_wins1256=== PAUSE TestResolveDBConnectionString/flag_wins1257=== RUN TestResolveDBConnectionString/file_when_flag_empty1258=== PAUSE TestResolveDBConnectionString/file_when_flag_empty1259=== RUN TestResolveDBConnectionString/missing_file_is_an_error1260=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error1261=== RUN TestResolveDBConnectionString/PGHOST_allows_empty1262=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty1263=== RUN TestResolveDBConnectionString/nothing_configured1264=== PAUSE TestResolveDBConnectionString/nothing_configured1265=== CONT TestService_ReadScope_PublicByDefault12662026/11/11 23:37:31 INFO Aborted multipart uploads count=0 kept=01267--- PASS: TestClientFallsBackToClosures (6.56s)1268=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock12692026/11/11 23:37:31 WARN Force mode enabled - objects will be deleted immediately without grace period12702026/11/11 23:37:31 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012712026/11/11 23:37:32 INFO Vacuumed table table=pending_closures12722026/11/11 23:37:32 INFO Vacuumed table table=pending_objects12732026/11/11 23:37:32 INFO Vacuumed table table=multipart_uploads12742026/11/11 23:37:32 INFO Vacuumed table table=closures12752026/11/11 23:37:32 INFO Vacuumed table table=objects1276--- PASS: TestGCSweepSkipsPendingObjects (6.45s)1277=== CONT TestService_RequireScope_OIDC12782026/11/11 23:37:32 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:59641/oidc12792026/11/11 23:37:32 INFO Received uploads request method=POST path=/api/pending_closures1280--- PASS: TestDeduplicatedObjectsRecordedAsPending (6.50s)1281=== CONT TestDeleteClosuresRacingPins1282--- PASS: TestCacheStatsHandler (6.52s)1283=== CONT TestService_AuthMiddleware_OIDC12842026/11/11 23:37:32 INFO Received push request method=POST path=/api/pushes12852026/11/11 23:37:32 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)12862026/11/11 23:37:32 INFO Uploading cav785qk88qjgwhr4b8plrz2q8lyycyf-shared-dep (136B)12872026/11/11 23:37:32 INFO Uploading z90zd5hmah295i7lfkv5x8hik86967vg-b (256B)12882026/11/11 23:37:32 WARN Failed to register uploaded object key=nar/12crb9x8vvv24a0dlx0h4l932xx7ysbr65x0nxsi7a3yb7qjkwvz.nar.zst error="server returned 404: 404 page not found\n"12892026/11/11 23:37:32 WARN Failed to register uploaded object key=ipswgzm97arkf9bsvqk6xjadks0c9hhl.ls error="server returned 404: 404 page not found\n"12902026/11/11 23:37:32 WARN Failed to register uploaded object key=cav785qk88qjgwhr4b8plrz2q8lyycyf.ls error="server returned 404: 404 page not found\n"12912026/11/11 23:37:32 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"12922026/11/11 23:37:32 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:59651/oidc12932026/11/11 23:37:32 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12942026/11/11 23:37:32 INFO Signed narinfos id=1 count=312952026/11/11 23:37:32 WARN Failed to register uploaded object key=z90zd5hmah295i7lfkv5x8hik86967vg.ls error="server returned 404: 404 page not found\n"12962026/11/11 23:37:32 INFO Uploading 3 narinfos12972026/11/11 23:37:32 WARN Failed to register uploaded object key=ipswgzm97arkf9bsvqk6xjadks0c9hhl.narinfo error="server returned 404: 404 page not found\n"12982026/11/11 23:37:32 WARN Failed to register uploaded object key=z90zd5hmah295i7lfkv5x8hik86967vg.narinfo error="server returned 404: 404 page not found\n"12992026/11/11 23:37:32 INFO Received complete push request method=POST path=/api/pushes/1/complete13002026/11/11 23:37:32 WARN Failed to register uploaded object key=cav785qk88qjgwhr4b8plrz2q8lyycyf.narinfo error="server returned 404: 404 page not found\n"13012026/11/11 23:37:32 INFO Completed push id=1 objects=8 duration=30.771209ms13022026/11/11 23:37:32 INFO Upload complete. (242ms)1303=== NAME TestClientPushesUseOnePush1304 client_pushes_test.go:97: Retrieved narinfo from S3:1305 StorePath: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientPushesUseOnePush3279572037/001/store/cav785qk88qjgwhr4b8plrz2q8lyycyf-shared-dep1306 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1307 Compression: zstd1308 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821309 NarSize: 1361310 References: 1311 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1312 client_pushes_test.go:97: Retrieved narinfo from S3:1313 StorePath: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientPushesUseOnePush3279572037/001/store/ipswgzm97arkf9bsvqk6xjadks0c9hhl-a1314 URL: nar/12crb9x8vvv24a0dlx0h4l932xx7ysbr65x0nxsi7a3yb7qjkwvz.nar.zst1315 Compression: zstd1316 NarHash: sha256:12crb9x8vvv24a0dlx0h4l932xx7ysbr65x0nxsi7a3yb7qjkwvz1317 NarSize: 2561318 References: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientPushesUseOnePush3279572037/001/store/cav785qk88qjgwhr4b8plrz2q8lyycyf-shared-dep1319 CA: text:sha256:063v57x4v7bqab0596inlsmnyif78pbm2xgfjcb9ic0yasbclg3l1320 client_pushes_test.go:97: Retrieved narinfo from S3:1321 StorePath: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientPushesUseOnePush3279572037/001/store/z90zd5hmah295i7lfkv5x8hik86967vg-b1322 URL: nar/12crb9x8vvv24a0dlx0h4l932xx7ysbr65x0nxsi7a3yb7qjkwvz.nar.zst1323 Compression: zstd1324 NarHash: sha256:12crb9x8vvv24a0dlx0h4l932xx7ysbr65x0nxsi7a3yb7qjkwvz1325 NarSize: 2561326 References: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientPushesUseOnePush3279572037/001/store/cav785qk88qjgwhr4b8plrz2q8lyycyf-shared-dep1327 CA: text:sha256:063v57x4v7bqab0596inlsmnyif78pbm2xgfjcb9ic0yasbclg3l1328--- PASS: TestClientPushesUseOnePush (7.11s)1329=== CONT TestService_AuthMiddleware_MTLSBoundSubjects13302026/11/11 23:37:33 INFO Received uploads request method=POST path=/api/pending_closures13312026/11/11 23:37:33 INFO Aborted multipart uploads count=0 kept=013322026/11/11 23:37:33 WARN Force mode enabled - objects will be deleted immediately without grace period13332026/11/11 23:37:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013342026/11/11 23:37:33 INFO Vacuumed table table=pending_closures13352026/11/11 23:37:33 INFO Aborted multipart uploads count=0 kept=013362026/11/11 23:37:33 INFO Vacuumed table table=pending_objects13372026/11/11 23:37:33 INFO Vacuumed table table=multipart_uploads13382026/11/11 23:37:33 INFO Vacuumed table table=closures13392026/11/11 23:37:33 INFO Vacuumed table table=objects1340--- PASS: TestGCMetrics (6.37s)1341=== CONT TestService_AuthMiddleware13422026/11/11 23:37:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013432026/11/11 23:37:33 INFO Vacuumed table table=pending_closures13442026/11/11 23:37:33 INFO Vacuumed table table=pending_objects13452026/11/11 23:37:33 INFO Vacuumed table table=multipart_uploads13462026/11/11 23:37:33 INFO Vacuumed table table=closures13472026/11/11 23:37:33 INFO Vacuumed table table=objects13482026/11/11 23:37:33 INFO Aborted multipart uploads count=0 kept=013492026/11/11 23:37:33 WARN Force mode enabled - objects will be deleted immediately without grace period13502026/11/11 23:37:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013512026/11/11 23:37:33 INFO Vacuumed table table=pending_closures13522026/11/11 23:37:33 INFO Vacuumed table table=pending_objects13532026/11/11 23:37:33 INFO Vacuumed table table=multipart_uploads13542026/11/11 23:37:33 INFO Vacuumed table table=closures13552026/11/11 23:37:33 INFO Vacuumed table table=objects1356--- PASS: TestPushDedupSurvivesConcurrentGC (6.92s)1357=== CONT TestPendingClosureWriteTimeout1358=== RUN TestPendingClosureWriteTimeout/empty1359=== PAUSE TestPendingClosureWriteTimeout/empty1360=== RUN TestPendingClosureWriteTimeout/400_objects1361=== PAUSE TestPendingClosureWriteTimeout/400_objects1362=== RUN TestPendingClosureWriteTimeout/670k_objects1363=== PAUSE TestPendingClosureWriteTimeout/670k_objects1364=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1365=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1366=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT1367--- PASS: TestGCBugBareHashReferences (6.63s)1368=== CONT TestCompleteMultipartUnregistered13692026/11/11 23:37:33 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=_tribuchetbld7 database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"1370--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (6.36s)1371=== CONT TestService_Rustfstest1372=== NAME TestClientIntegration1373 client_integration_test.go:292: Created store path: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientIntegration164596237/002/store/a5smk9nb8a60hlkjbdrvn0iwjgv65zk4-test-file.txt13742026/11/11 23:37:34 INFO Received push request method=POST path=/api/pushes13752026/11/11 23:37:34 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)13762026/11/11 23:37:34 INFO Uploading a5smk9nb8a60hlkjbdrvn0iwjgv65zk4-test-file.txt (152B)13772026/11/11 23:37:34 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13782026/11/11 23:37:34 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"1379=== NAME TestClientMultipleUploads13802026/11/11 23:37:34 WARN Failed to register uploaded object key=a5smk9nb8a60hlkjbdrvn0iwjgv65zk4.ls error="server returned 404: 404 page not found\n"1381 client_integration_test.go:406: Created store path 0: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientMultipleUploads3418972633/001/store/0r7dr8qqfdff25ay050rq2n35giaa9li-test-file-0.txt13822026/11/11 23:37:34 INFO Signed narinfos id=1 count=113832026/11/11 23:37:34 INFO Uploading 1 narinfos13842026/11/11 23:37:34 INFO lead: acquired remote=192.0.2.1:1234 generation=113852026/11/11 23:37:34 INFO Received complete push request method=POST path=/api/pushes/1/complete13862026/11/11 23:37:34 WARN Failed to register uploaded object key=a5smk9nb8a60hlkjbdrvn0iwjgv65zk4.narinfo error="server returned 404: 404 page not found\n"1387 client_integration_test.go:406: Created store path 1: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientMultipleUploads3418972633/001/store/ayflhck15s2zkzw37iszaznrf6qdkqks-test-file-1.txt13882026/11/11 23:37:34 INFO Completed push id=1 objects=3 duration=36.79575ms13892026/11/11 23:37:34 INFO Upload complete. (257ms)1390 client_integration_test.go:406: Created store path 2: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientMultipleUploads3418972633/001/store/fgaiyz7nr1nxym4wh1brib0pr99wksln-test-file-2.txt13912026/11/11 23:37:34 INFO All 1 paths already cached1392=== NAME TestClientIntegration1393 client_integration_test.go:318: Retrieved narinfo from S3:1394 StorePath: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientIntegration164596237/002/store/a5smk9nb8a60hlkjbdrvn0iwjgv65zk4-test-file.txt1395 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1396 Compression: zstd1397 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11398 NarSize: 1521399 References: 1400 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11401 client_integration_test.go:319: Retrieved .ls file from S3 (compressed size: 77 bytes)1402 client_integration_test.go:319: Decompressed .ls content (64 bytes):1403 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}14042026/11/11 23:37:34 INFO Received push request method=POST path=/api/pushes14052026/11/11 23:37:34 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)14062026/11/11 23:37:34 INFO Uploading ayflhck15s2zkzw37iszaznrf6qdkqks-test-file-1.txt (160B)14072026/11/11 23:37:34 INFO Uploading fgaiyz7nr1nxym4wh1brib0pr99wksln-test-file-2.txt (160B)14082026/11/11 23:37:34 INFO Uploading 0r7dr8qqfdff25ay050rq2n35giaa9li-test-file-0.txt (160B)14092026/11/11 23:37:34 INFO lead: acquired remote=192.0.2.1:1234 generation=114102026/11/11 23:37:34 WARN Failed to register uploaded object key=fgaiyz7nr1nxym4wh1brib0pr99wksln.ls error="server returned 404: 404 page not found\n"14112026/11/11 23:37:34 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"14122026/11/11 23:37:34 WARN Failed to register uploaded object key=ayflhck15s2zkzw37iszaznrf6qdkqks.ls error="server returned 404: 404 page not found\n"14132026/11/11 23:37:34 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"14142026/11/11 23:37:34 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"14152026/11/11 23:37:34 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign14162026/11/11 23:37:34 WARN Failed to register uploaded object key=0r7dr8qqfdff25ay050rq2n35giaa9li.ls error="server returned 404: 404 page not found\n"14172026/11/11 23:37:34 INFO Signed narinfos id=1 count=314182026/11/11 23:37:34 INFO Uploading 3 narinfos14192026/11/11 23:37:34 WARN Failed to register uploaded object key=ayflhck15s2zkzw37iszaznrf6qdkqks.narinfo error="server returned 404: 404 page not found\n"14202026/11/11 23:37:34 WARN Failed to register uploaded object key=fgaiyz7nr1nxym4wh1brib0pr99wksln.narinfo error="server returned 404: 404 page not found\n"14212026/11/11 23:37:34 INFO Received complete push request method=POST path=/api/pushes/1/complete14222026/11/11 23:37:34 WARN Failed to register uploaded object key=0r7dr8qqfdff25ay050rq2n35giaa9li.narinfo error="server returned 404: 404 page not found\n"14232026/11/11 23:37:34 INFO Completed push id=1 objects=9 duration=23.349542ms14242026/11/11 23:37:34 INFO Upload complete. (317ms)1425=== NAME TestClientMultipleUploads1426 client_integration_test.go:417: Uploaded 3 paths in 349.940708ms14272026/11/11 23:37:34 INFO Received push request method=POST path=/api/pushes14282026/11/11 23:37:34 INFO Object in database but missing from S3 key=a5smk9nb8a60hlkjbdrvn0iwjgv65zk4.ls14292026/11/11 23:37:34 WARN Found objects in DB but missing from S3, will re-upload count=114302026/11/11 23:37:34 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)1431--- PASS: TestClientMultipleUploads (7.49s)1432=== CONT TestService_verifyS3ReuploadsStaleNarinfo1433=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1434=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1435=== RUN TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1436=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1437=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1438=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1439=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1440=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1441=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1442=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1443=== CONT TestProxyWriteTimeout1444=== RUN TestProxyWriteTimeout/narinfo1445=== PAUSE TestProxyWriteTimeout/narinfo1446=== RUN TestProxyWriteTimeout/1_GiB_nar1447=== PAUSE TestProxyWriteTimeout/1_GiB_nar1448=== RUN TestProxyWriteTimeout/10_GiB_nar1449=== PAUSE TestProxyWriteTimeout/10_GiB_nar1450=== RUN TestProxyWriteTimeout/unknown_size1451=== PAUSE TestProxyWriteTimeout/unknown_size1452=== CONT TestCacheConfigHandler1453=== RUN TestCacheConfigHandler/full_config,_no_issuer1454=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1455=== RUN TestCacheConfigHandler/no_cache_url_configured1456=== PAUSE TestCacheConfigHandler/no_cache_url_configured1457=== RUN TestCacheConfigHandler/no_signing_keys1458=== PAUSE TestCacheConfigHandler/no_signing_keys1459=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1460=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1461=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle14622026/11/11 23:37:34 INFO Received complete push request method=POST path=/api/pushes/2/complete14632026/11/11 23:37:34 WARN Failed to register uploaded object key=a5smk9nb8a60hlkjbdrvn0iwjgv65zk4.ls error="server returned 404: 404 page not found\n"14642026/11/11 23:37:34 INFO Completed push id=2 objects=3 duration=886.333µs14652026/11/11 23:37:34 INFO Upload complete. (95ms)1466=== NAME TestClientIntegration1467 client_integration_test.go:336: Retrieved .ls file from S3 (compressed size: 62 bytes)1468 client_integration_test.go:336: Decompressed .ls content (49 bytes):1469 {"version":1,"root":{"type":"regular","size":39}}14702026/11/11 23:37:35 INFO Received push request method=POST path=/api/pushes14712026/11/11 23:37:35 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)1472=== NAME TestPinProtectsFromGC1473 client_integration_test.go:779: Pinned store path: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestPinProtectsFromGC96148377/001/store/myvvhlwxjg842wf9nl5n3cwpr526dzwj-pinned-file.txt1474 client_integration_test.go:780: Unpinned store path: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestPinProtectsFromGC96148377/001/store/clnvym034m779kkvf8nlyy7r0fnalvjl-unpinned-file.txt14752026/11/11 23:37:35 INFO Received complete push request method=POST path=/api/pushes/3/complete14762026/11/11 23:37:35 WARN Failed to register uploaded object key=a5smk9nb8a60hlkjbdrvn0iwjgv65zk4.ls error="server returned 404: 404 page not found\n"14772026/11/11 23:37:35 INFO Completed push id=3 objects=3 duration=635.875µs14782026/11/11 23:37:35 INFO Upload complete. (93ms)1479=== NAME TestClientIntegration1480 client_integration_test.go:353: Retrieved .ls file from S3 (compressed size: 62 bytes)1481 client_integration_test.go:353: Decompressed .ls content (49 bytes):1482 {"version":1,"root":{"type":"regular","size":39}}1483 client_integration_test.go:364: Testing garbage collection...14842026/11/11 23:37:35 INFO Starting cleanup of old closures method=DELETE path=/api/closures14852026/11/11 23:37:35 INFO Garbage collection started14862026/11/11 23:37:35 INFO Aborted multipart uploads count=0 kept=014872026/11/11 23:37:35 WARN Force mode enabled - objects will be deleted immediately without grace period14882026/11/11 23:37:35 INFO Received push request method=POST path=/api/pushes14892026/11/11 23:37:35 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)14902026/11/11 23:37:35 INFO Uploading myvvhlwxjg842wf9nl5n3cwpr526dzwj-pinned-file.txt (128B)14912026/11/11 23:37:35 WARN Failed to register uploaded object key=myvvhlwxjg842wf9nl5n3cwpr526dzwj.ls error="server returned 404: 404 page not found\n"14922026/11/11 23:37:35 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign14932026/11/11 23:37:35 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"14942026/11/11 23:37:35 INFO Signed narinfos id=1 count=114952026/11/11 23:37:35 INFO Uploading 1 narinfos14962026/11/11 23:37:35 INFO lead: acquired remote=192.0.2.1:1234 generation=114972026/11/11 23:37:35 INFO lead: released remote=192.0.2.1:1234 generation=11498--- PASS: TestLeadIncumbentAnnouncesAtOnce (6.45s)1499=== CONT TestService_verifyS3Integrity15002026/11/11 23:37:35 INFO Received complete push request method=POST path=/api/pushes/1/complete15012026/11/11 23:37:35 WARN Failed to register uploaded object key=myvvhlwxjg842wf9nl5n3cwpr526dzwj.narinfo error="server returned 404: 404 page not found\n"15022026/11/11 23:37:35 INFO Completed push id=1 objects=3 duration=19.4745ms15032026/11/11 23:37:35 INFO Upload complete. (208ms)15042026/11/11 23:37:35 INFO lead: released remote=192.0.2.1:1234 generation=11505--- PASS: TestLeadEndsOnShutdown (7.65s)1506=== CONT TestService_createPendingClosureHandler15072026/11/11 23:37:35 INFO Received push request method=POST path=/api/pushes1508=== NAME TestClientCADerivations1509 client_ca_test.go:136: Built CA derivation: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientCADerivations4076054898/001/store/c4axp18q26zzdrgfcyf2ry4kcly0wqsq-ca-test15102026/11/11 23:37:35 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)15112026/11/11 23:37:35 INFO Uploading clnvym034m779kkvf8nlyy7r0fnalvjl-unpinned-file.txt (128B)1512 client_ca_test.go:139: Found 1 dependencies (including self)15132026/11/11 23:37:35 WARN Failed to register uploaded object key=clnvym034m779kkvf8nlyy7r0fnalvjl.ls error="server returned 404: 404 page not found\n"15142026/11/11 23:37:35 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign15152026/11/11 23:37:35 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"15162026/11/11 23:37:35 INFO Signed narinfos id=2 count=115172026/11/11 23:37:35 INFO Uploading 1 narinfos15182026/11/11 23:37:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=015192026/11/11 23:37:35 INFO Received complete push request method=POST path=/api/pushes/2/complete15202026/11/11 23:37:35 WARN Failed to register uploaded object key=clnvym034m779kkvf8nlyy7r0fnalvjl.narinfo error="server returned 404: 404 page not found\n"15212026/11/11 23:37:35 INFO Completed push id=2 objects=3 duration=585.917µs15222026/11/11 23:37:35 INFO Upload complete. (189ms)15232026/11/11 23:37:35 INFO Vacuumed table table=pending_closures15242026/11/11 23:37:35 INFO Vacuumed table table=pending_objects15252026/11/11 23:37:35 INFO Vacuumed table table=multipart_uploads15262026/11/11 23:37:35 INFO Vacuumed table table=closures15272026/11/11 23:37:35 INFO Starting cleanup of old closures method=DELETE path=/api/closures15282026/11/11 23:37:35 INFO Vacuumed table table=objects15292026/11/11 23:37:35 INFO Garbage collection started15302026/11/11 23:37:35 INFO Aborted multipart uploads count=0 kept=015312026/11/11 23:37:35 INFO Received push request method=POST path=/api/pushes15322026/11/11 23:37:35 WARN Force mode enabled - objects will be deleted immediately without grace period15332026/11/11 23:37:35 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)15342026/11/11 23:37:35 INFO Uploading c4axp18q26zzdrgfcyf2ry4kcly0wqsq-ca-test (144B)15352026/11/11 23:37:35 INFO lead: released remote=192.0.2.1:1234 generation=115362026/11/11 23:37:35 INFO lead: acquired remote=192.0.2.1:1234 generation=215372026/11/11 23:37:35 WARN Failed to register uploaded object key=c4axp18q26zzdrgfcyf2ry4kcly0wqsq.ls error="server returned 404: 404 page not found\n"15382026/11/11 23:37:35 WARN Failed to register uploaded object key=log/2myr9nygnd1spjj3wbc6x6hjdr244ljv-ca-test.drv error="server returned 404: 404 page not found\n"15392026/11/11 23:37:35 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15402026/11/11 23:37:35 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"15412026/11/11 23:37:35 INFO Signed narinfos id=1 count=115422026/11/11 23:37:35 INFO Uploading 1 narinfos15432026/11/11 23:37:35 INFO Received complete push request method=POST path=/api/pushes/1/complete15442026/11/11 23:37:35 WARN Failed to register uploaded object key=c4axp18q26zzdrgfcyf2ry4kcly0wqsq.narinfo error="server returned 404: 404 page not found\n"15452026/11/11 23:37:36 INFO Completed push id=1 objects=4 duration=15.564791ms15462026/11/11 23:37:36 INFO Upload complete. (276ms)1547 client_ca_test.go:180: Narinfo contains CA field: StorePath: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientCADerivations4076054898/001/store/c4axp18q26zzdrgfcyf2ry4kcly0wqsq-ca-test1548 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1549 Compression: zstd1550 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1551 NarSize: 1441552 References: 1553 Deriver: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientCADerivations4076054898/001/store/2myr9nygnd1spjj3wbc6x6hjdr244ljv-ca-test.drv1554 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1555 client_ca_test.go:185: Checking for realisation files in S3...1556 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1557 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache15582026/11/11 23:37:36 INFO lead: acquired remote=192.0.2.1:1234 generation=11559 client_ca_test.go:258: nix copy output: error: binary cache 's3://bucket66?endpoint=http://localhost:59495&region=eu-west-1' is for Nix stores with prefix '/nix/store', not '/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientCADerivations4076054898/001/store'1560 client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 11561--- PASS: TestClientCADerivations (7.30s)1562=== CONT TestSkippedUploadsHandler15632026/11/11 23:37:36 INFO Client skipped oversized paths paths=3 nar_bytes=50000000001564--- PASS: TestSkippedUploadsHandler (0.01s)1565=== CONT TestParseSize1566--- PASS: TestParseSize (0.00s)1567=== CONT TestService_cleanupPendingClosuresHandler15682026/11/11 23:37:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=015692026/11/11 23:37:36 INFO Vacuumed table table=pending_closures15702026/11/11 23:37:36 INFO Vacuumed table table=pending_objects15712026/11/11 23:37:36 INFO Vacuumed table table=multipart_uploads15722026/11/11 23:37:36 INFO Received push request method=POST path=/api/pushes15732026/11/11 23:37:36 INFO Vacuumed table table=closures15742026/11/11 23:37:36 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)15752026/11/11 23:37:36 INFO Uploading zz28bx9bn03j74ah331iawrafzc259q1-top (264B)15762026/11/11 23:37:36 INFO Uploading 436afkr7ghy8ni79vaxxclsp0rx26xn9-shared-dep (136B)15772026/11/11 23:37:36 INFO Vacuumed table table=objects15782026/11/11 23:37:36 WARN Failed to register uploaded object key=zz28bx9bn03j74ah331iawrafzc259q1.ls error="server returned 404: 404 page not found\n"15792026/11/11 23:37:36 WARN Failed to register uploaded object key=436afkr7ghy8ni79vaxxclsp0rx26xn9.ls error="server returned 404: 404 page not found\n"15802026/11/11 23:37:36 WARN Failed to register uploaded object key=nar/0ngbqwh8kmrng0mip6a4rlgv3gsrjk1sqiaswpvkdi97gg78qc9l.nar.zst error="server returned 404: 404 page not found\n"15812026/11/11 23:37:36 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15822026/11/11 23:37:36 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"15832026/11/11 23:37:36 INFO Signed narinfos id=1 count=215842026/11/11 23:37:36 INFO Uploading 2 narinfos15852026/11/11 23:37:36 WARN Failed to register uploaded object key=zz28bx9bn03j74ah331iawrafzc259q1.narinfo error="server returned 404: 404 page not found\n"15862026/11/11 23:37:36 INFO Received complete push request method=POST path=/api/pushes/1/complete15872026/11/11 23:37:36 WARN Failed to register uploaded object key=436afkr7ghy8ni79vaxxclsp0rx26xn9.narinfo error="server returned 404: 404 page not found\n"15882026/11/11 23:37:36 INFO Completed push id=1 objects=6 duration=15.405833ms15892026/11/11 23:37:36 INFO Upload complete. (222ms)1590=== NAME TestClientSharedPathCommittedMidPush1591 client_integration_test.go:728: Retrieved narinfo from S3:1592 StorePath: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientSharedPathCommittedMidPush3555660716/001/store/436afkr7ghy8ni79vaxxclsp0rx26xn9-shared-dep1593 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1594 Compression: zstd1595 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821596 NarSize: 1361597 References: 1598 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1599 client_integration_test.go:728: Retrieved narinfo from S3:1600 StorePath: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientSharedPathCommittedMidPush3555660716/001/store/zz28bx9bn03j74ah331iawrafzc259q1-top1601 URL: nar/0ngbqwh8kmrng0mip6a4rlgv3gsrjk1sqiaswpvkdi97gg78qc9l.nar.zst1602 Compression: zstd1603 NarHash: sha256:0ngbqwh8kmrng0mip6a4rlgv3gsrjk1sqiaswpvkdi97gg78qc9l1604 NarSize: 2641605 References: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientSharedPathCommittedMidPush3555660716/001/store/436afkr7ghy8ni79vaxxclsp0rx26xn9-shared-dep1606 CA: text:sha256:1yzkf087axhyxhf24lks1m538f3pglbmywp1h9z31sgjxxkxmh2h1607--- PASS: TestClientSharedPathCommittedMidPush (7.12s)1608=== CONT TestUploadHandlersRejectInvalidKeys1609=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1610=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1611=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal1612=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal1613=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key1614=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key1615=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key1616=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key1617=== CONT TestCompleteMultipartUpload_ErrorButObjectExists1618=== NAME TestClientWithDependencies1619 client_integration_test.go:661: Built derivation: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientWithDependencies2390910028/001/store/h1k4czyf4vs9qk2dm78la97an363ayzb-test-script16202026/11/11 23:37:36 INFO lead: acquired remote=192.0.2.1:1234 generation=11621 client_integration_test.go:663: Found 1 dependencies (including self)16222026/11/11 23:37:36 INFO Received push request method=POST path=/api/pushes16232026/11/11 23:37:36 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)16242026/11/11 23:37:36 INFO Uploading h1k4czyf4vs9qk2dm78la97an363ayzb-test-script (136B)16252026/11/11 23:37:36 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"16262026/11/11 23:37:36 WARN Failed to register uploaded object key=h1k4czyf4vs9qk2dm78la97an363ayzb.ls error="server returned 404: 404 page not found\n"16272026/11/11 23:37:36 WARN Failed to register uploaded object key=log/9qwm0s1c0q1f7w1fm0f98v2splzsgr6h-test-script.drv error="server returned 404: 404 page not found\n"16282026/11/11 23:37:36 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16292026/11/11 23:37:36 INFO Signed narinfos id=1 count=116302026/11/11 23:37:36 INFO Uploading 1 narinfos16312026/11/11 23:37:36 INFO lead: released remote=192.0.2.1:1234 generation=216322026/11/11 23:37:36 INFO lead: acquired remote=192.0.2.1:1234 generation=316332026/11/11 23:37:36 INFO lead: released remote=192.0.2.1:1234 generation=31634--- PASS: TestLeadGenerationGrowsWithEveryGrant (8.86s)1635=== CONT TestUploadHandlersRejectOversizedBody1636--- PASS: TestService_ReadScope_PublicByDefault (5.03s)1637=== CONT TestPresignedUploadRegisteredBeforeCommit16382026/11/11 23:37:36 INFO Received complete push request method=POST path=/api/pushes/1/complete16392026/11/11 23:37:36 WARN Failed to register uploaded object key=h1k4czyf4vs9qk2dm78la97an363ayzb.narinfo error="server returned 404: 404 page not found\n"1640=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure1641=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure1642=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart1643=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart1644=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts1645=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts1646=== CONT TestSharedNarBothUploadsComplete16472026/11/11 23:37:36 INFO Completed push id=1 objects=4 duration=21.667458ms16482026/11/11 23:37:36 INFO Upload complete. (185ms)1649=== NAME TestClientWithDependencies1650 client_integration_test.go:665: Skipping nix copy test - isolated store (/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestClientWithDependencies2390910028/001/store) requires matching store prefix1651--- PASS: TestClientWithDependencies (6.99s)1652=== CONT TestCompletedNarNotReofferedAcrossClosures16532026/11/11 23:37:37 INFO lead: released remote=192.0.2.1:1234 generation=116542026/11/11 23:37:37 INFO lead: acquired remote=192.0.2.1:1234 generation=216552026/11/11 23:37:37 INFO lead: given back to the incumbent remote=192.0.2.1:1234 generation=216562026/11/11 23:37:37 INFO lead: acquired remote=192.0.2.1:1234 generation=31657--- PASS: TestService_ReadAuthMiddleware (5.51s)1658=== CONT TestPush_SignsNarinfosOfItsPendingObjects16592026/11/11 23:37:37 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=01660=== NAME TestClientIntegration1661 client_integration_test.go:371: Objects in database after GC:1662 client_integration_test.go:371: Successfully deleted all objects with GC --force1663--- PASS: TestClientIntegration (10.04s)1664=== CONT TestService_AuthMiddleware_MTLSProxyHeader1665=== RUN TestService_RequireScope_OIDC/builder_may_write1666=== PAUSE TestService_RequireScope_OIDC/builder_may_write1667=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1668=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1669=== RUN TestService_RequireScope_OIDC/ops_may_admin1670=== PAUSE TestService_RequireScope_OIDC/ops_may_admin1671=== RUN TestService_RequireScope_OIDC/ops_may_not_write1672=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1673=== RUN TestService_RequireScope_OIDC/reader_may_not_write1674=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1675=== RUN TestService_RequireScope_OIDC/static_token_may_admin1676=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1677=== RUN TestService_RequireScope_OIDC/static_token_may_write1678=== PAUSE TestService_RequireScope_OIDC/static_token_may_write1679=== RUN TestService_RequireScope_OIDC/reader_may_read1680=== PAUSE TestService_RequireScope_OIDC/reader_may_read1681=== RUN TestService_RequireScope_OIDC/writer_implies_read1682=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1683=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1684=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read1685=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc1686=== PAUSE TestService_RequireScope_OIDC/reader_may_read_with_netrc1687=== RUN TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1688=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1689=== RUN TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1690=== PAUSE TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1691=== RUN TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1692=== PAUSE TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1693=== CONT TestIsValidUploadKey1694=== RUN TestIsValidUploadKey/narinfo1695=== PAUSE TestIsValidUploadKey/narinfo1696=== RUN TestIsValidUploadKey/nar_zst1697=== PAUSE TestIsValidUploadKey/nar_zst1698=== RUN TestIsValidUploadKey/nar_xz1699=== PAUSE TestIsValidUploadKey/nar_xz1700=== RUN TestIsValidUploadKey/nar_plain1701=== PAUSE TestIsValidUploadKey/nar_plain1702=== RUN TestIsValidUploadKey/listing1703=== PAUSE TestIsValidUploadKey/listing1704=== RUN TestIsValidUploadKey/build_log1705=== PAUSE TestIsValidUploadKey/build_log1706=== RUN TestIsValidUploadKey/build_log_home-manager_file1707=== PAUSE TestIsValidUploadKey/build_log_home-manager_file1708=== RUN TestIsValidUploadKey/build_log_plus_in_name1709=== PAUSE TestIsValidUploadKey/build_log_plus_in_name1710=== RUN TestIsValidUploadKey/build_log_question_mark1711=== PAUSE TestIsValidUploadKey/build_log_question_mark1712=== RUN TestIsValidUploadKey/build_log_equals1713=== PAUSE TestIsValidUploadKey/build_log_equals1714=== RUN TestIsValidUploadKey/realisation1715=== PAUSE TestIsValidUploadKey/realisation1716=== RUN TestIsValidUploadKey/realisation_plus_in_output1717=== PAUSE TestIsValidUploadKey/realisation_plus_in_output1718=== RUN TestIsValidUploadKey/nix-cache-info1719=== PAUSE TestIsValidUploadKey/nix-cache-info1720=== RUN TestIsValidUploadKey/index.html1721=== PAUSE TestIsValidUploadKey/index.html1722=== RUN TestIsValidUploadKey/narinfo_key,_nar_type1723=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type1724=== RUN TestIsValidUploadKey/nar_key,_narinfo_type1725=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type1726=== RUN TestIsValidUploadKey/listing_key,_narinfo_type1727=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type1728=== RUN TestIsValidUploadKey/traversal1729=== PAUSE TestIsValidUploadKey/traversal1730=== RUN TestIsValidUploadKey/traversal_nar1731=== PAUSE TestIsValidUploadKey/traversal_nar1732=== RUN TestIsValidUploadKey/absolute1733=== PAUSE TestIsValidUploadKey/absolute1734=== RUN TestIsValidUploadKey/empty_key1735=== PAUSE TestIsValidUploadKey/empty_key1736=== RUN TestIsValidUploadKey/unknown_type1737=== PAUSE TestIsValidUploadKey/unknown_type1738=== CONT TestGCEndsOnShutdown/before_the_run17392026/11/11 23:37:37 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=017402026/11/11 23:37:37 INFO Received create pin request method=POST path=/api/pins/myapp17412026/11/11 23:37:37 INFO Created/updated pin name=myapp store_path=/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestPinProtectsFromGC96148377/001/store/myvvhlwxjg842wf9nl5n3cwpr526dzwj-pinned-file.txt narinfo_key=myvvhlwxjg842wf9nl5n3cwpr526dzwj.narinfo17422026/11/11 23:37:37 INFO Starting cleanup of old closures method=DELETE path=/api/closures17432026/11/11 23:37:37 INFO Garbage collection started17442026/11/11 23:37:37 INFO Aborted multipart uploads count=0 kept=017452026/11/11 23:37:37 WARN Force mode enabled - objects will be deleted immediately without grace period17462026/11/11 23:37:37 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=017472026/11/11 23:37:37 INFO Vacuumed table table=pending_closures17482026/11/11 23:37:37 INFO Vacuumed table table=pending_objects17492026/11/11 23:37:37 INFO Vacuumed table table=multipart_uploads17502026/11/11 23:37:37 INFO Vacuumed table table=closures17512026/11/11 23:37:37 INFO lead: released remote=192.0.2.1:1234 generation=117522026/11/11 23:37:37 INFO Vacuumed table table=objects17532026/11/11 23:37:38 INFO lead: acquired remote=192.0.2.1:1234 generation=21754--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (6.14s)1755=== CONT TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1756=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token1757=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token1758=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1759=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1760=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1761=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1762=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1763=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1764=== CONT TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete17652026/11/11 23:37:38 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"17662026/11/11 23:37:38 WARN mTLS auth: bound subjects configured but subject DN unavailable17672026/11/11 23:37:38 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1768--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (5.42s)1769=== CONT TestServerTLSConfig/no_client_CA1770=== CONT TestServerTLSConfig/not_a_PEM_file1771=== CONT TestServerTLSConfig/missing_CA_file1772--- PASS: TestServerTLSConfig (0.00s)1773 --- PASS: TestServerTLSConfig/no_client_CA (0.00s)1774 --- PASS: TestServerTLSConfig/not_a_PEM_file (0.03s)1775 --- PASS: TestServerTLSConfig/missing_CA_file (0.00s)1776=== CONT TestPush_RejectsBadRequests/no_objects17772026/11/11 23:37:38 INFO Received push request method=POST path=/api/pushes1778=== CONT TestPush_RejectsBadRequests/root_not_in_objects17792026/11/11 23:37:38 INFO Received push request method=POST path=/api/pushes1780=== CONT TestPush_RejectsBadRequests/no_roots17812026/11/11 23:37:38 INFO Received push request method=POST path=/api/pushes1782=== CONT TestPush_RejectsBadRequests/bad_root17832026/11/11 23:37:38 INFO Received push request method=POST path=/api/pushes1784=== CONT TestForceGCDuringPushOffersSweptObject/before_pending_rows1785--- PASS: TestPush_RejectsBadRequests (0.96s)1786 --- PASS: TestPush_RejectsBadRequests/no_objects (0.00s)1787 --- PASS: TestPush_RejectsBadRequests/root_not_in_objects (0.00s)1788 --- PASS: TestPush_RejectsBadRequests/no_roots (0.00s)1789 --- PASS: TestPush_RejectsBadRequests/bad_root (0.00s)17902026/11/11 23:37:38 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"1791--- PASS: TestService_AuthMiddleware (5.54s)1792=== CONT TestForceGCDuringPushOffersSweptObject/after_presence_check1793--- PASS: TestConnectWaitsForAPeerMigration (7.49s)1794=== CONT TestParseSingleRange/none1795=== CONT TestParseSingleRange/open-ended1796=== CONT TestParseSingleRange/start_far_past_EOF1797=== CONT TestParseSingleRange/start_past_EOF1798=== CONT TestParseSingleRange/single_byte1799=== CONT TestParseSingleRange/suffix_exceeds_size1800=== CONT TestParseSingleRange/suffix1801=== CONT TestParseSingleRange/end_clamped_to_size1802=== CONT TestParseSingleRange/malformed_both_empty1803=== CONT TestParseSingleRange/closed1804=== CONT TestParseSingleRange/malformed_end_before_start1805=== CONT TestParseSingleRange/multi-range_ignored1806=== CONT TestParseSingleRange/malformed_no_dash1807=== CONT TestParseSingleRange/unknown_unit1808--- PASS: TestParseSingleRange (0.01s)1809 --- PASS: TestParseSingleRange/none (0.00s)1810 --- PASS: TestParseSingleRange/open-ended (0.00s)1811 --- PASS: TestParseSingleRange/start_far_past_EOF (0.00s)1812 --- PASS: TestParseSingleRange/start_past_EOF (0.00s)1813 --- PASS: TestParseSingleRange/single_byte (0.00s)1814 --- PASS: TestParseSingleRange/suffix_exceeds_size (0.00s)1815 --- PASS: TestParseSingleRange/suffix (0.00s)1816 --- PASS: TestParseSingleRange/end_clamped_to_size (0.00s)1817 --- PASS: TestParseSingleRange/malformed_both_empty (0.00s)1818 --- PASS: TestParseSingleRange/closed (0.00s)1819 --- PASS: TestParseSingleRange/malformed_end_before_start (0.00s)1820 --- PASS: TestParseSingleRange/multi-range_ignored (0.00s)1821 --- PASS: TestParseSingleRange/malformed_no_dash (0.00s)1822 --- PASS: TestParseSingleRange/unknown_unit (0.00s)1823=== CONT TestPresentNotReportedWhileGCDeletesClosure/stale18242026/11/11 23:37:39 INFO Received uploads request method=POST path=/api/pending_closures18252026/11/11 23:37:39 INFO lead: released remote=192.0.2.1:1234 generation=21826--- PASS: TestLeadElectsOneAndHandsOver (8.70s)1827=== CONT TestPresentNotReportedWhileGCDeletesClosure/fresh1828--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (5.78s)1829=== CONT TestIsValidCachePath/narinfo1830=== CONT TestIsValidCachePath/index.html1831=== CONT TestIsValidCachePath/short_hash1832=== CONT TestIsValidCachePath/wrong_extension1833=== CONT TestIsValidCachePath/leading_slash1834=== CONT TestIsValidCachePath/empty1835=== CONT TestIsValidCachePath/random_path1836=== CONT TestIsValidCachePath/invalid_char_u1837=== CONT TestIsValidCachePath/invalid_char_e1838=== CONT TestIsValidCachePath/traversal_in_middle1839=== CONT TestIsValidCachePath/traversal_parent1840=== CONT TestIsValidCachePath/nar_uncompressed1841=== CONT TestIsValidCachePath/nix-cache-info1842=== CONT TestIsValidCachePath/realisation1843=== CONT TestIsValidCachePath/log1844=== CONT TestIsValidCachePath/ls1845=== CONT TestIsValidCachePath/nar_xz1846=== CONT TestIsValidCachePath/nar_bz21847=== CONT TestIsValidCachePath/nar_zst1848=== CONT TestIsValidCachePath/narinfo_all_nix_base32_chars1849--- PASS: TestIsValidCachePath (0.00s)1850 --- PASS: TestIsValidCachePath/narinfo (0.00s)1851 --- PASS: TestIsValidCachePath/index.html (0.00s)1852 --- PASS: TestIsValidCachePath/short_hash (0.00s)1853 --- PASS: TestIsValidCachePath/wrong_extension (0.00s)1854 --- PASS: TestIsValidCachePath/leading_slash (0.00s)1855 --- PASS: TestIsValidCachePath/empty (0.00s)1856 --- PASS: TestIsValidCachePath/random_path (0.00s)1857 --- PASS: TestIsValidCachePath/invalid_char_u (0.00s)1858 --- PASS: TestIsValidCachePath/invalid_char_e (0.00s)1859 --- PASS: TestIsValidCachePath/traversal_in_middle (0.00s)1860 --- PASS: TestIsValidCachePath/traversal_parent (0.00s)1861 --- PASS: TestIsValidCachePath/nar_uncompressed (0.00s)1862 --- PASS: TestIsValidCachePath/nix-cache-info (0.00s)1863 --- PASS: TestIsValidCachePath/realisation (0.00s)1864 --- PASS: TestIsValidCachePath/log (0.00s)1865 --- PASS: TestIsValidCachePath/ls (0.00s)1866 --- PASS: TestIsValidCachePath/nar_xz (0.00s)1867 --- PASS: TestIsValidCachePath/nar_bz2 (0.00s)1868 --- PASS: TestIsValidCachePath/nar_zst (0.00s)1869 --- PASS: TestIsValidCachePath/narinfo_all_nix_base32_chars (0.00s)1870=== CONT TestClientErrorHandling/InvalidStorePath18712026/11/11 23:37:39 INFO lead: released remote=192.0.2.1:1234 generation=31872--- PASS: TestLeadUnannouncedStandbyYieldsToIncumbent (9.32s)1873=== CONT TestClientErrorHandling/ServerNotAvailable1874--- PASS: TestService_Rustfstest (5.67s)1875=== CONT TestClientErrorHandling/InvalidAuthToken18762026/11/11 23:37:39 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present18772026/11/11 23:37:39 WARN Request failed, retrying attempt=2 backoff=213.420343ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present18782026/11/11 23:37:39 INFO Received complete multipart upload request method=POST path=/api/multipart/complete18792026/11/11 23:37:39 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst1880--- PASS: TestCompleteMultipartUnregistered (5.94s)1881=== CONT TestResolveDBConnectionString/flag_wins1882=== CONT TestResolveDBConnectionString/PGHOST_allows_empty1883=== CONT TestResolveDBConnectionString/nothing_configured1884=== CONT TestResolveDBConnectionString/missing_file_is_an_error1885=== CONT TestResolveDBConnectionString/file_when_flag_empty1886=== CONT TestPendingClosureWriteTimeout/empty1887=== CONT TestPendingClosureWriteTimeout/670k_objects1888--- PASS: TestResolveDBConnectionString (0.01s)1889 --- PASS: TestResolveDBConnectionString/flag_wins (0.00s)1890 --- PASS: TestResolveDBConnectionString/PGHOST_allows_empty (0.00s)1891 --- PASS: TestResolveDBConnectionString/nothing_configured (0.00s)1892 --- PASS: TestResolveDBConnectionString/missing_file_is_an_error (0.00s)1893 --- PASS: TestResolveDBConnectionString/file_when_flag_empty (0.00s)1894=== CONT TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline18952026/11/11 23:37:39 WARN Request failed, retrying attempt=3 backoff=414.958408ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present18962026/11/11 23:37:39 INFO Received uploads request method=POST path=/api/pending_closures1897=== NAME TestPinProtectsFromGC1898 client_integration_test.go:888: Pin successfully protected closure from garbage collection1899--- PASS: TestPinProtectsFromGC (11.60s)1900=== CONT TestPendingClosureWriteTimeout/400_objects1901=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed19022026/11/11 23:37:40 WARN Request failed, retrying attempt=4 backoff=727.925566ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present19032026/11/11 23:37:40 INFO Received uploads request method=POST path=/api/pending_closures19042026/11/11 23:37:40 INFO Received complete multipart upload request method=POST path=/api/multipart/complete19052026/11/11 23:37:40 INFO Received uploads request method=POST path=/api/pending_closures19062026/11/11 23:37:40 INFO Received uploads request method=POST path=/api/pending_closures19072026/11/11 23:37:40 INFO Received uploads request method=POST path=/api/pending_closures19082026/11/11 23:37:40 WARN Request failed, retrying attempt=5 backoff=1.671894804s error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present19092026/11/11 23:37:41 INFO Received cleanup request method=DELETE path=/api/pending_closures19102026/11/11 23:37:41 INFO Aborted multipart uploads count=0 kept=019112026/11/11 23:37:41 INFO Received uploads request method=POST path=/api/pending_closures19122026/11/11 23:37:41 INFO Received cleanup request method=DELETE path=/api/pending_closures19132026/11/11 23:37:41 INFO Aborted multipart uploads count=1 kept=019142026/11/11 23:37:41 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete19152026-11-11 23:37:41.258 UTC [90081] ERROR: Closure does not exist: id=119162026-11-11 23:37:41.258 UTC [90081] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 13 at RAISE19172026-11-11 23:37:41.258 UTC [90081] STATEMENT: -- name: CommitPendingClosure :one1918 SELECT commit_pending_closure($1::bigint)::bigint1919 1920--- PASS: TestService_cleanupPendingClosuresHandler (5.04s)1921=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed19222026/11/11 23:37:41 INFO Received uploads request method=POST path=/api/pending_closures19232026/11/11 23:37:41 INFO Received uploads request method=POST path=/api/pending_closures19242026/11/11 23:37:42 INFO Received complete multipart upload request method=POST path=/api/multipart/complete19252026/11/11 23:37:42 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MmVjZmY2NzQtZTkyNS00YmMzLTg1NjItYzkyOTFmZjEyN2NhLmY0NTAwY2RlLTYzZTYtNDJjYS05MDQzLTlkMzBhMzU4NWZhYXgxNzk0NDQwMjYxNTg4MDE2MDAw19262026/11/11 23:37:42 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MmVjZmY2NzQtZTkyNS00YmMzLTg1NjItYzkyOTFmZjEyN2NhLmY0NTAwY2RlLTYzZTYtNDJjYS05MDQzLTlkMzBhMzU4NWZhYXgxNzk0NDQwMjYxNTg4MDE2MDAw parts=11927--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (5.65s)1928=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed19292026/11/11 23:37:42 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst19302026/11/11 23:37:42 INFO Received uploads request method=POST path=/api/pending_closures1931--- PASS: TestPresignedUploadRegisteredBeforeCommit (5.28s)1932=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost19332026/11/11 23:37:42 INFO Received complete multipart upload request method=POST path=/api/multipart/complete19342026/11/11 23:37:42 INFO Received uploads request method=POST path=/api/pending_closures19352026/11/11 23:37:42 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=MmVjZmY2NzQtZTkyNS00YmMzLTg1NjItYzkyOTFmZjEyN2NhLjhlYjhmNTA2LWNlZjgtNGI5OC04ODcwLWM2NzNjYTI5MmU2OXgxNzk0NDQwMjYwMzkyMjMyMDAw parts=1019362026/11/11 23:37:42 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete19372026/11/11 23:37:42 INFO Completed upload id=1 objects=2 duration=18.144625ms19382026/11/11 23:37:42 INFO Received uploads request method=POST path=/api/pending_closures19392026/11/11 23:37:42 INFO Received uploads request method=POST path=/api/pending_closures19402026/11/11 23:37:42 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo19412026/11/11 23:37:42 WARN Found objects in DB but missing from S3, will re-upload count=11942--- PASS: TestService_verifyS3Integrity (7.05s)1943=== CONT TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed19442026/11/11 23:37:42 INFO Received uploads request method=POST path=/api/pending_closures19452026/11/11 23:37:42 WARN Request failed, retrying attempt=1 backoff=100ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config19462026/11/11 23:37:42 WARN Request failed, retrying attempt=2 backoff=192.102205ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config19472026/11/11 23:37:42 INFO Received uploads request method=POST path=/api/pending_closures19482026/11/11 23:37:42 INFO Received complete multipart upload request method=POST path=/api/multipart/complete19492026/11/11 23:37:42 WARN Request failed, retrying attempt=3 backoff=431.187148ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config19502026/11/11 23:37:43 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=MmVjZmY2NzQtZTkyNS00YmMzLTg1NjItYzkyOTFmZjEyN2NhLmQ0NzhiZmQ2LTJkMzYtNGY1Yy04NjQwLWJlNzRhZjNkODBiOHgxNzk0NDQwMjYwNzkwODEwMDAw parts=1019512026/11/11 23:37:43 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete19522026/11/11 23:37:43 INFO Completed upload id=1 objects=2 duration=30.065209ms19532026/11/11 23:37:43 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000019542026/11/11 23:37:43 INFO Received uploads request method=POST path=/api/pending_closures19552026/11/11 23:37:43 INFO Starting cleanup of old closures method=DELETE path=/api/closures19562026/11/11 23:37:43 INFO Aborted multipart uploads count=0 kept=019572026/11/11 23:37:43 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=019582026/11/11 23:37:43 INFO Vacuumed table table=pending_closures1959--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (5.81s)1960=== CONT TestProxyWriteTimeout/narinfo1961=== CONT TestProxyWriteTimeout/10_GiB_nar1962=== CONT TestProxyWriteTimeout/unknown_size1963=== CONT TestProxyWriteTimeout/1_GiB_nar1964--- PASS: TestProxyWriteTimeout (0.00s)1965 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1966 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1967 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1968 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)1969=== CONT TestCacheConfigHandler/full_config,_no_issuer1970=== CONT TestCacheConfigHandler/no_signing_keys1971=== CONT TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1972=== CONT TestCacheConfigHandler/no_cache_url_configured1973--- PASS: TestCacheConfigHandler (0.00s)1974 --- PASS: TestCacheConfigHandler/full_config,_no_issuer (0.00s)1975 --- PASS: TestCacheConfigHandler/no_signing_keys (0.00s)1976 --- PASS: TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator (0.00s)1977 --- PASS: TestCacheConfigHandler/no_cache_url_configured (0.00s)1978=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info19792026/11/11 23:37:43 INFO Received uploads request method=POST path=/1980=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key19812026/11/11 23:37:43 INFO Received complete multipart upload request method=POST path=/1982=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key19832026/11/11 23:37:43 INFO Received request for more parts method=POST path=/19842026/11/11 23:37:43 INFO Vacuumed table table=pending_objects1985=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal19862026/11/11 23:37:43 INFO Received uploads request method=POST path=/1987--- PASS: TestUploadHandlersRejectInvalidKeys (0.00s)1988 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)1989 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.01s)1990 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1991 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s)1992=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure19932026/11/11 23:37:43 INFO Received uploads request method=POST path=/19942026/11/11 23:37:43 INFO Vacuumed table table=multipart_uploads19952026/11/11 23:37:43 INFO Vacuumed table table=closures19962026/11/11 23:37:43 INFO Vacuumed table table=objects19972026/11/11 23:37:43 WARN Request failed, retrying attempt=4 backoff=724.959545ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config19982026/11/11 23:37:43 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001999--- PASS: TestService_createPendingClosureHandler (7.85s)2000=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts

loading 650 hidden lines…

26512026/11/11 23:38:04 INFO Completed push id=1 objects=10 duration=32.0465ms26522026/11/11 23:38:04 INFO Received uploads request method=POST path=/26532026/11/11 23:38:04 INFO Received push request method=POST path=/26542026/11/11 23:38:04 INFO Received push request method=POST path=/26552026/11/11 23:38:04 INFO Received complete upload request method=POST path=/26562026/11/11 23:38:04 INFO Completed upload id=3 objects=5 duration=474.208µs26572026/11/11 23:38:04 INFO Received push request method=POST path=/26582026/11/11 23:38:04 INFO Received complete upload request method=POST path=/26592026/11/11 23:38:04 INFO Received complete push request method=POST path=/26602026/11/11 23:38:04 INFO Completed upload id=9 objects=16 duration=1.377875ms26612026/11/11 23:38:04 INFO Completed push id=4 objects=7 duration=1.396459ms26622026/11/11 23:38:04 INFO Received push request method=POST path=/26632026/11/11 23:38:04 INFO Received push request method=POST path=/26642026/11/11 23:38:04 INFO Received uploads request method=POST path=/26652026/11/11 23:38:04 INFO Received push request method=POST path=/26662026/11/11 23:38:04 INFO Received complete push request method=POST path=/26672026/11/11 23:38:04 INFO Received uploads request method=POST path=/26682026/11/11 23:38:04 INFO Received complete push request method=POST path=/26692026/11/11 23:38:04 INFO Completed push id=10 objects=3 duration=748.5µs26702026/11/11 23:38:04 INFO Received uploads request method=POST path=/26712026/11/11 23:38:04 INFO Completed push id=3 objects=5 duration=16.15625ms26722026/11/11 23:38:04 INFO Received uploads request method=POST path=/26732026/11/11 23:38:04 INFO Received push request method=POST path=/26742026/11/11 23:38:04 INFO Received complete push request method=POST path=/26752026/11/11 23:38:04 INFO Received push request method=POST path=/26762026/11/11 23:38:04 INFO Aborted multipart uploads count=0 kept=026772026/11/11 23:38:04 INFO Completed push id=2 objects=7 duration=6.261584ms26782026/11/11 23:38:04 INFO Aborted multipart uploads count=0 kept=026792026/11/11 23:38:04 INFO Aborted multipart uploads count=0 kept=026802026/11/11 23:38:04 WARN Force mode enabled - objects will be deleted immediately without grace period26812026/11/11 23:38:04 INFO Received complete push request method=POST path=/26822026/11/11 23:38:04 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=026832026/11/11 23:38:04 INFO Received complete upload request method=POST path=/26842026/11/11 23:38:04 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=026852026/11/11 23:38:04 INFO Completed push id=4 objects=3 duration=549.542µs26862026/11/11 23:38:04 WARN Force mode enabled - objects will be deleted immediately without grace period26872026/11/11 23:38:04 INFO Completed upload id=10 objects=23 duration=577.292µs26882026/11/11 23:38:04 INFO Vacuumed table table=pending_closures26892026/11/11 23:38:04 INFO Vacuumed table table=pending_closures26902026/11/11 23:38:04 INFO Vacuumed table table=pending_objects26912026/11/11 23:38:04 INFO Vacuumed table table=multipart_uploads26922026/11/11 23:38:04 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=5 objects-failed-to-delete=026932026/11/11 23:38:04 INFO Received uploads request method=POST path=/26942026/11/11 23:38:04 INFO Vacuumed table table=closures26952026/11/11 23:38:04 INFO Received push request method=POST path=/26962026/11/11 23:38:04 INFO Vacuumed table table=objects26972026/11/11 23:38:04 INFO Vacuumed table table=pending_objects26982026/11/11 23:38:04 INFO Received push request method=POST path=/26992026/11/11 23:38:04 INFO Vacuumed table table=multipart_uploads27002026/11/11 23:38:04 INFO Received uploads request method=POST path=/27012026/11/11 23:38:04 INFO Received push request method=POST path=/27022026/11/11 23:38:04 INFO Vacuumed table table=closures27032026/11/11 23:38:04 INFO Vacuumed table table=pending_closures27042026/11/11 23:38:04 INFO Vacuumed table table=objects27052026/11/11 23:38:04 INFO Received uploads request method=POST path=/27062026/11/11 23:38:04 INFO Received uploads request method=POST path=/27072026/11/11 23:38:04 INFO Received uploads request method=POST path=/27082026/11/11 23:38:04 INFO Received push request method=POST path=/27092026/11/11 23:38:04 INFO Aborted multipart uploads count=0 kept=027102026/11/11 23:38:04 INFO Received uploads request method=POST path=/27112026/11/11 23:38:04 INFO Vacuumed table table=pending_objects27122026/11/11 23:38:04 INFO Received complete upload request method=POST path=/27132026/11/11 23:38:04 WARN Force mode enabled - objects will be deleted immediately without grace period27142026/11/11 23:38:05 INFO Vacuumed table table=multipart_uploads27152026/11/11 23:38:05 INFO Completed upload id=11 objects=6 duration=402.917µs27162026/11/11 23:38:05 INFO Vacuumed table table=closures27172026/11/11 23:38:05 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027182026/11/11 23:38:05 INFO Received push request method=POST path=/27192026/11/11 23:38:05 INFO Vacuumed table table=pending_closures27202026/11/11 23:38:05 INFO Vacuumed table table=pending_objects27212026/11/11 23:38:05 INFO Vacuumed table table=objects27222026/11/11 23:38:05 INFO Vacuumed table table=multipart_uploads27232026/11/11 23:38:05 INFO Vacuumed table table=closures27242026/11/11 23:38:05 INFO Received complete upload request method=POST path=/27252026/11/11 23:38:05 INFO Vacuumed table table=objects27262026/11/11 23:38:05 INFO Completed upload id=17 objects=6 duration=767.583µs27272026/11/11 23:38:05 INFO Received push request method=POST path=/27282026/11/11 23:38:05 INFO Received push request method=POST path=/27292026/11/11 23:38:05 INFO Received uploads request method=POST path=/27302026/11/11 23:38:05 INFO Received push request method=POST path=/27312026/11/11 23:38:05 INFO Received complete upload request method=POST path=/27322026/11/11 23:38:05 INFO Received uploads request method=POST path=/27332026/11/11 23:38:05 INFO Completed upload id=3 objects=25 duration=702.625µs27342026/11/11 23:38:05 INFO Received complete upload request method=POST path=/27352026/11/11 23:38:05 INFO Completed upload id=15 objects=9 duration=302.666µs27362026/11/11 23:38:05 INFO Aborted multipart uploads count=0 kept=027372026/11/11 23:38:05 WARN Force mode enabled - objects will be deleted immediately without grace period27382026/11/11 23:38:05 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=027392026/11/11 23:38:05 INFO Received complete push request method=POST path=/27402026/11/11 23:38:05 INFO Completed push id=12 objects=15 duration=624.959µs27412026/11/11 23:38:05 INFO Received push request method=POST path=/27422026/11/11 23:38:05 INFO Vacuumed table table=pending_closures27432026/11/11 23:38:05 INFO Received push request method=POST path=/27442026/11/11 23:38:05 INFO Received uploads request method=POST path=/27452026/11/11 23:38:05 INFO Received complete push request method=POST path=/27462026/11/11 23:38:05 INFO Vacuumed table table=pending_objects27472026/11/11 23:38:05 INFO Vacuumed table table=multipart_uploads27482026/11/11 23:38:05 INFO Completed push id=21 objects=17 duration=667.208µs27492026/11/11 23:38:05 INFO Received uploads request method=POST path=/27502026/11/11 23:38:05 INFO Received complete push request method=POST path=/27512026/11/11 23:38:05 INFO Completed push id=19 objects=7 duration=456.959µs27522026/11/11 23:38:05 INFO Received push request method=POST path=/27532026/11/11 23:38:05 INFO Received push request method=POST path=/27542026/11/11 23:38:05 INFO Vacuumed table table=closures27552026/11/11 23:38:05 INFO Received complete upload request method=POST path=/27562026/11/11 23:38:05 INFO Completed upload id=15 objects=6 duration=255.333µs27572026/11/11 23:38:05 INFO Received push request method=POST path=/27582026/11/11 23:38:05 INFO Aborted multipart uploads count=0 kept=027592026/11/11 23:38:05 INFO Vacuumed table table=objects27602026/11/11 23:38:05 WARN Force mode enabled - objects will be deleted immediately without grace period27612026/11/11 23:38:05 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027622026/11/11 23:38:05 INFO Vacuumed table table=pending_closures27632026/11/11 23:38:05 INFO Vacuumed table table=pending_objects27642026/11/11 23:38:05 INFO Vacuumed table table=multipart_uploads27652026/11/11 23:38:05 INFO Received uploads request method=POST path=/27662026/11/11 23:38:05 INFO Vacuumed table table=closures27672026/11/11 23:38:05 INFO Received push request method=POST path=/27682026/11/11 23:38:05 INFO Vacuumed table table=objects27692026/11/11 23:38:05 INFO Received complete push request method=POST path=/27702026/11/11 23:38:05 INFO Completed push id=8 objects=13 duration=529.042µs27712026/11/11 23:38:05 INFO Received uploads request method=POST path=/27722026/11/11 23:38:05 INFO Aborted multipart uploads count=0 kept=027732026/11/11 23:38:05 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027742026/11/11 23:38:05 INFO Received complete push request method=POST path=/27752026/11/11 23:38:05 INFO Completed push id=6 objects=8 duration=1.036917ms27762026/11/11 23:38:05 INFO Received complete push request method=POST path=/27772026/11/11 23:38:05 INFO Received uploads request method=POST path=/27782026/11/11 23:38:05 INFO Vacuumed table table=pending_closures27792026/11/11 23:38:05 INFO Received uploads request method=POST path=/27802026/11/11 23:38:05 INFO Completed push id=1 objects=5 duration=14.758583ms27812026/11/11 23:38:05 INFO Received uploads request method=POST path=/27822026/11/11 23:38:05 INFO Received push request method=POST path=/27832026/11/11 23:38:05 INFO Received complete upload request method=POST path=/27842026/11/11 23:38:05 INFO Vacuumed table table=pending_objects27852026/11/11 23:38:05 INFO Received complete upload request method=POST path=/27862026/11/11 23:38:05 INFO Received uploads request method=POST path=/27872026/11/11 23:38:05 INFO Completed upload id=4 objects=7 duration=519.167µs27882026/11/11 23:38:05 INFO Vacuumed table table=multipart_uploads27892026/11/11 23:38:05 INFO Completed upload id=7 objects=4 duration=651.875µs27902026/11/11 23:38:05 INFO Received push request method=POST path=/27912026/11/11 23:38:05 INFO Received push request method=POST path=/27922026/11/11 23:38:05 INFO Received uploads request method=POST path=/27932026/11/11 23:38:05 INFO Aborted multipart uploads count=0 kept=027942026/11/11 23:38:05 INFO Received complete push request method=POST path=/27952026/11/11 23:38:05 WARN Force mode enabled - objects will be deleted immediately without grace period27962026/11/11 23:38:05 INFO Received push request method=POST path=/27972026/11/11 23:38:05 INFO Completed push id=7 objects=7 duration=540.417µs27982026/11/11 23:38:05 INFO Vacuumed table table=closures27992026/11/11 23:38:05 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028002026/11/11 23:38:05 INFO Received uploads request method=POST path=/28012026/11/11 23:38:05 INFO Received uploads request method=POST path=/28022026/11/11 23:38:05 INFO Vacuumed table table=pending_closures28032026/11/11 23:38:05 INFO Received push request method=POST path=/28042026/11/11 23:38:05 INFO Vacuumed table table=pending_objects28052026/11/11 23:38:05 INFO Aborted multipart uploads count=0 kept=028062026/11/11 23:38:05 INFO Received complete upload request method=POST path=/28072026/11/11 23:38:05 INFO Vacuumed table table=multipart_uploads28082026/11/11 23:38:05 WARN Force mode enabled - objects will be deleted immediately without grace period28092026/11/11 23:38:05 INFO Vacuumed table table=objects28102026/11/11 23:38:05 INFO Vacuumed table table=closures28112026/11/11 23:38:05 INFO Completed upload id=12 objects=26 duration=992.167µs28122026/11/11 23:38:05 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028132026/11/11 23:38:05 INFO Vacuumed table table=objects28142026/11/11 23:38:05 INFO Received uploads request method=POST path=/28152026/11/11 23:38:05 INFO Received uploads request method=POST path=/28162026/11/11 23:38:05 INFO Received complete push request method=POST path=/28172026/11/11 23:38:05 INFO Received uploads request method=POST path=/28182026/11/11 23:38:05 INFO Received complete push request method=POST path=/28192026/11/11 23:38:05 INFO Completed push id=16 objects=27 duration=741.208µs28202026/11/11 23:38:05 INFO Received complete upload request method=POST path=/28212026/11/11 23:38:05 INFO Completed push id=16 objects=8 duration=460.041µs28222026/11/11 23:38:05 INFO Completed upload id=9 objects=7 duration=780.875µs28232026/11/11 23:38:05 INFO Received uploads request method=POST path=/28242026/11/11 23:38:05 INFO Aborted multipart uploads count=0 kept=028252026/11/11 23:38:05 INFO Vacuumed table table=pending_closures28262026/11/11 23:38:05 WARN Force mode enabled - objects will be deleted immediately without grace period28272026/11/11 23:38:05 INFO Received push request method=POST path=/28282026/11/11 23:38:05 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028292026/11/11 23:38:05 INFO Aborted multipart uploads count=0 kept=028302026/11/11 23:38:05 INFO Vacuumed table table=pending_closures28312026/11/11 23:38:05 INFO Vacuumed table table=pending_objects28322026/11/11 23:38:05 INFO Vacuumed table table=pending_objects28332026/11/11 23:38:05 INFO Vacuumed table table=multipart_uploads28342026/11/11 23:38:05 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."28352026/11/11 23:38:05 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."28362026/11/11 23:38:05 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."28372026/11/11 23:38:05 INFO Vacuumed table table=multipart_uploads28382026/11/11 23:38:05 INFO Vacuumed table table=closures28392026/11/11 23:38:05 INFO Vacuumed table table=objects28402026/11/11 23:38:05 INFO Aborted multipart uploads count=0 kept=028412026/11/11 23:38:05 WARN Force mode enabled - objects will be deleted immediately without grace period28422026/11/11 23:38:05 INFO Received uploads request method=POST path=/28432026/11/11 23:38:05 INFO Received complete push request method=POST path=/28442026/11/11 23:38:05 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028452026/11/11 23:38:05 INFO Vacuumed table table=closures28462026/11/11 23:38:05 INFO Vacuumed table table=pending_closures28472026/11/11 23:38:05 INFO Received complete upload request method=POST path=/28482026/11/11 23:38:05 INFO Vacuumed table table=pending_objects28492026/11/11 23:38:05 INFO Completed upload id=10 objects=4 duration=5.732959ms28502026/11/11 23:38:05 INFO Vacuumed table table=multipart_uploads28512026/11/11 23:38:05 INFO Received complete push request method=POST path=/28522026/11/11 23:38:05 INFO Received push request method=POST path=/28532026/11/11 23:38:05 INFO Vacuumed table table=closures28542026/11/11 23:38:05 INFO Completed push id=1 objects=6 duration=41.455042ms28552026/11/11 23:38:05 INFO Completed push id=10 objects=2 duration=5.658708ms28562026/11/11 23:38:05 INFO Vacuumed table table=objects28572026/11/11 23:38:05 INFO Vacuumed table table=objects28582026/11/11 23:38:05 INFO Received push request method=POST path=/28592026/11/11 23:38:05 INFO Received push request method=POST path=/28602026/11/11 23:38:05 INFO Received uploads request method=POST path=/28612026/11/11 23:38:05 INFO Received uploads request method=POST path=/28622026/11/11 23:38:05 INFO Received uploads request method=POST path=/28632026/11/11 23:38:05 INFO Received complete push request method=POST path=/28642026/11/11 23:38:05 INFO Received complete push request method=POST path=/28652026/11/11 23:38:05 INFO Received uploads request method=POST path=/28662026/11/11 23:38:05 INFO Completed push id=18 objects=10 duration=818.125µs28672026/11/11 23:38:05 INFO Completed push id=11 objects=6 duration=6.751709ms28682026/11/11 23:38:05 INFO Received uploads request method=POST path=/28692026/11/11 23:38:05 INFO Aborted multipart uploads count=0 kept=028702026/11/11 23:38:05 INFO Received complete upload request method=POST path=/28712026/11/11 23:38:05 INFO Received uploads request method=POST path=/28722026/11/11 23:38:05 WARN Force mode enabled - objects will be deleted immediately without grace period28732026/11/11 23:38:05 INFO Received complete upload request method=POST path=/28742026/11/11 23:38:05 INFO Completed upload id=12 objects=21 duration=312.958µs28752026/11/11 23:38:05 INFO Garbage collection completed failed-uploads-deleted=5 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028762026/11/11 23:38:05 INFO Aborted multipart uploads count=0 kept=028772026/11/11 23:38:05 INFO Completed upload id=5 objects=5 duration=483.708µs28782026/11/11 23:38:05 INFO Received complete upload request method=POST path=/28792026/11/11 23:38:05 INFO Received push request method=POST path=/28802026/11/11 23:38:05 INFO Received uploads request method=POST path=/28812026/11/11 23:38:05 INFO Completed upload id=18 objects=11 duration=362.833µs28822026/11/11 23:38:05 WARN Force mode enabled - objects will be deleted immediately without grace period28832026/11/11 23:38:05 INFO Received uploads request method=POST path=/28842026/11/11 23:38:05 INFO Received uploads request method=POST path=/28852026/11/11 23:38:06 INFO Received push request method=POST path=/28862026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=4 objects-failed-to-delete=028872026/11/11 23:38:06 INFO Received push request method=POST path=/28882026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=028892026/11/11 23:38:06 INFO Received uploads request method=POST path=/28902026/11/11 23:38:06 WARN Force mode enabled - objects will be deleted immediately without grace period28912026/11/11 23:38:06 INFO Vacuumed table table=pending_closures28922026/11/11 23:38:06 INFO Vacuumed table table=pending_closures28932026/11/11 23:38:06 INFO Received uploads request method=POST path=/28942026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=028952026/11/11 23:38:06 INFO Received complete push request method=POST path=/28962026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=028972026/11/11 23:38:06 INFO Vacuumed table table=pending_objects28982026/11/11 23:38:06 INFO Completed push id=13 objects=23 duration=437.75µs28992026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads29002026/11/11 23:38:06 INFO Received complete push request method=POST path=/29012026/11/11 23:38:06 INFO Received complete upload request method=POST path=/29022026/11/11 23:38:06 INFO Vacuumed table table=pending_closures29032026/11/11 23:38:06 INFO Completed push id=18 objects=6 duration=666.292µs29042026/11/11 23:38:06 INFO Vacuumed table table=closures29052026/11/11 23:38:06 INFO Completed upload id=11 objects=14 duration=821.416µs29062026/11/11 23:38:06 INFO Vacuumed table table=pending_objects29072026/11/11 23:38:06 INFO Vacuumed table table=pending_objects29082026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=029092026/11/11 23:38:06 INFO Vacuumed table table=objects29102026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=029112026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads29122026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads29132026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029142026/11/11 23:38:06 INFO Received push request method=POST path=/29152026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=029162026/11/11 23:38:06 INFO Received complete upload request method=POST path=/29172026/11/11 23:38:06 INFO Vacuumed table table=pending_closures29182026/11/11 23:38:06 INFO Completed upload id=10 objects=6 duration=5.572667ms29192026/11/11 23:38:06 INFO Vacuumed table table=pending_objects29202026/11/11 23:38:06 INFO Received push request method=POST path=/29212026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads29222026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=029232026/11/11 23:38:06 INFO Vacuumed table table=closures29242026/11/11 23:38:06 WARN Force mode enabled - objects will be deleted immediately without grace period29252026/11/11 23:38:06 INFO Vacuumed table table=closures29262026/11/11 23:38:06 INFO Vacuumed table table=pending_closures29272026/11/11 23:38:06 INFO Received push request method=POST path=/29282026/11/11 23:38:06 INFO Vacuumed table table=objects29292026/11/11 23:38:06 INFO Vacuumed table table=objects29302026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029312026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=029322026/11/11 23:38:06 WARN Force mode enabled - objects will be deleted immediately without grace period29332026/11/11 23:38:06 INFO Vacuumed table table=pending_objects29342026/11/11 23:38:06 INFO Vacuumed table table=pending_closures29352026/11/11 23:38:06 INFO Vacuumed table table=closures29362026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029372026/11/11 23:38:06 INFO Received complete upload request method=POST path=/29382026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads29392026/11/11 23:38:06 INFO Vacuumed table table=pending_objects29402026/11/11 23:38:06 INFO Vacuumed table table=pending_closures29412026/11/11 23:38:06 INFO Completed upload id=20 objects=3 duration=912.042µs29422026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads29432026/11/11 23:38:06 INFO Vacuumed table table=pending_objects29442026/11/11 23:38:06 INFO Vacuumed table table=closures29452026/11/11 23:38:06 INFO Received push request method=POST path=/29462026/11/11 23:38:06 INFO Vacuumed table table=closures29472026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads29482026/11/11 23:38:06 INFO Vacuumed table table=objects29492026/11/11 23:38:06 INFO Vacuumed table table=objects29502026/11/11 23:38:06 INFO Received complete upload request method=POST path=/29512026/11/11 23:38:06 INFO Completed upload id=24 objects=7 duration=288.75µs29522026/11/11 23:38:06 INFO Vacuumed table table=closures29532026/11/11 23:38:06 INFO Received push request method=POST path=/29542026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=029552026/11/11 23:38:06 INFO Received complete upload request method=POST path=/29562026/11/11 23:38:06 INFO Vacuumed table table=objects29572026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=029582026/11/11 23:38:06 INFO Vacuumed table table=objects29592026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029602026/11/11 23:38:06 INFO Completed upload id=4 objects=14 duration=6.320875ms29612026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029622026/11/11 23:38:06 INFO Vacuumed table table=pending_closures29632026/11/11 23:38:06 INFO Vacuumed table table=pending_objects29642026/11/11 23:38:06 INFO Vacuumed table table=pending_closures29652026/11/11 23:38:06 INFO Received uploads request method=POST path=/29662026/11/11 23:38:06 INFO Received uploads request method=POST path=/29672026/11/11 23:38:06 INFO Received complete push request method=POST path=/29682026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads29692026/11/11 23:38:06 INFO Received complete push request method=POST path=/29702026/11/11 23:38:06 INFO Vacuumed table table=pending_objects29712026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=029722026/11/11 23:38:06 INFO Received push request method=POST path=/29732026/11/11 23:38:06 INFO Received complete push request method=POST path=/29742026/11/11 23:38:06 INFO Vacuumed table table=closures29752026/11/11 23:38:06 INFO Completed push id=17 objects=22 duration=602.042µs29762026/11/11 23:38:06 INFO Received uploads request method=POST path=/29772026/11/11 23:38:06 INFO Completed push id=2 objects=4 duration=431.459µs29782026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads29792026/11/11 23:38:06 INFO Vacuumed table table=objects29802026/11/11 23:38:06 INFO Vacuumed table table=closures29812026/11/11 23:38:06 WARN Force mode enabled - objects will be deleted immediately without grace period29822026/11/11 23:38:06 INFO Completed push id=14 objects=3 duration=1.115917ms29832026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=029842026/11/11 23:38:06 INFO Received complete upload request method=POST path=/29852026/11/11 23:38:06 INFO Received uploads request method=POST path=/29862026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029872026/11/11 23:38:06 INFO Received push request method=POST path=/29882026/11/11 23:38:06 INFO Vacuumed table table=objects29892026/11/11 23:38:06 INFO Completed upload id=15 objects=4 duration=621.125µs29902026/11/11 23:38:06 INFO Received complete push request method=POST path=/29912026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029922026/11/11 23:38:06 INFO Received push request method=POST path=/29932026/11/11 23:38:06 INFO Completed push id=5 objects=6 duration=436.291µs29942026/11/11 23:38:06 INFO Received push request method=POST path=/29952026/11/11 23:38:06 INFO Received uploads request method=POST path=/29962026/11/11 23:38:06 INFO Received push request method=POST path=/29972026/11/11 23:38:06 INFO Vacuumed table table=pending_closures29982026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=029992026/11/11 23:38:06 INFO Received uploads request method=POST path=/30002026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=030012026/11/11 23:38:06 WARN Force mode enabled - objects will be deleted immediately without grace period30022026/11/11 23:38:06 INFO Received uploads request method=POST path=/30032026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030042026/11/11 23:38:06 WARN Force mode enabled - objects will be deleted immediately without grace period30052026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=030062026/11/11 23:38:06 INFO Received complete upload request method=POST path=/30072026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=030082026/11/11 23:38:06 INFO Vacuumed table table=pending_closures30092026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030102026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030112026/11/11 23:38:06 INFO Vacuumed table table=pending_objects30122026/11/11 23:38:06 INFO Completed upload id=16 objects=3 duration=209.25µs30132026/11/11 23:38:06 INFO Vacuumed table table=pending_closures30142026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads30152026/11/11 23:38:06 INFO Vacuumed table table=pending_closures30162026/11/11 23:38:06 INFO Vacuumed table table=pending_closures30172026/11/11 23:38:06 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."30182026/11/11 23:38:06 INFO Vacuumed table table=closures30192026/11/11 23:38:06 INFO Vacuumed table table=pending_objects30202026/11/11 23:38:06 INFO Vacuumed table table=pending_objects30212026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030222026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=030232026/11/11 23:38:06 INFO Received uploads request method=POST path=/30242026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads30252026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads30262026/11/11 23:38:06 INFO Vacuumed table table=objects30272026/11/11 23:38:06 INFO Vacuumed table table=pending_objects30282026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030292026/11/11 23:38:06 INFO Vacuumed table table=closures30302026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads30312026/11/11 23:38:06 INFO Vacuumed table table=closures30322026/11/11 23:38:06 INFO Vacuumed table table=objects30332026/11/11 23:38:06 INFO Vacuumed table table=pending_closures30342026/11/11 23:38:06 INFO Received push request method=POST path=/30352026/11/11 23:38:06 INFO Vacuumed table table=closures30362026/11/11 23:38:06 INFO Vacuumed table table=pending_closures30372026/11/11 23:38:06 INFO Received uploads request method=POST path=/30382026/11/11 23:38:06 INFO Vacuumed table table=objects30392026/11/11 23:38:06 INFO Vacuumed table table=objects30402026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=030412026/11/11 23:38:06 INFO Vacuumed table table=pending_objects30422026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030432026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads30442026/11/11 23:38:06 INFO Vacuumed table table=pending_objects30452026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads30462026/11/11 23:38:06 INFO Vacuumed table table=closures30472026/11/11 23:38:06 INFO Vacuumed table table=pending_closures30482026/11/11 23:38:06 INFO Received push request method=POST path=/30492026/11/11 23:38:06 INFO Vacuumed table table=objects30502026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=030512026/11/11 23:38:06 INFO Vacuumed table table=pending_objects30522026/11/11 23:38:06 INFO Received complete push request method=POST path=/30532026/11/11 23:38:06 INFO Vacuumed table table=closures30542026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030552026/11/11 23:38:06 INFO Completed push id=15 objects=4 duration=380.584µs30562026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads30572026/11/11 23:38:06 INFO Vacuumed table table=pending_objects30582026/11/11 23:38:06 INFO Vacuumed table table=closures30592026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads30602026/11/11 23:38:06 INFO Received uploads request method=POST path=/30612026/11/11 23:38:06 INFO Vacuumed table table=closures30622026/11/11 23:38:06 INFO Received push request method=POST path=/30632026/11/11 23:38:06 INFO Vacuumed table table=objects30642026/11/11 23:38:06 INFO Vacuumed table table=objects30652026/11/11 23:38:06 INFO Received complete upload request method=POST path=/30662026/11/11 23:38:06 INFO Received uploads request method=POST path=/30672026/11/11 23:38:06 INFO Vacuumed table table=pending_closures30682026/11/11 23:38:06 INFO Completed upload id=18 objects=4 duration=717.25µs30692026/11/11 23:38:06 INFO Vacuumed table table=objects30702026/11/11 23:38:06 INFO Received uploads request method=POST path=/30712026/11/11 23:38:06 INFO Received complete push request method=POST path=/30722026/11/11 23:38:06 INFO Completed push id=17 objects=6 duration=636.167µs30732026/11/11 23:38:06 INFO Received complete push request method=POST path=/30742026/11/11 23:38:06 INFO Vacuumed table table=pending_objects30752026/11/11 23:38:06 INFO Received uploads request method=POST path=/30762026/11/11 23:38:06 INFO Completed push id=23 objects=28 duration=1.052666ms30772026/11/11 23:38:06 INFO Received push request method=POST path=/30782026/11/11 23:38:06 INFO Received push request method=POST path=/30792026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads30802026/11/11 23:38:06 INFO Received complete upload request method=POST path=/30812026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=030822026/11/11 23:38:06 INFO Received uploads request method=POST path=/30832026/11/11 23:38:06 INFO Vacuumed table table=closures30842026/11/11 23:38:06 INFO Completed upload id=27 objects=8 duration=278.75µs30852026/11/11 23:38:06 INFO Received complete push request method=POST path=/30862026/11/11 23:38:06 INFO Received push request method=POST path=/30872026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030882026/11/11 23:38:06 INFO Received complete upload request method=POST path=/30892026/11/11 23:38:06 INFO Vacuumed table table=objects30902026/11/11 23:38:06 INFO Completed push id=5 objects=8 duration=1.209541ms30912026/11/11 23:38:06 INFO Vacuumed table table=pending_closures30922026/11/11 23:38:06 INFO Received push request method=POST path=/30932026/11/11 23:38:06 INFO Received uploads request method=POST path=/30942026/11/11 23:38:06 INFO Received complete upload request method=POST path=/30952026/11/11 23:38:06 INFO Received push request method=POST path=/30962026/11/11 23:38:06 INFO Vacuumed table table=pending_objects30972026/11/11 23:38:06 INFO Completed upload id=23 objects=18 duration=742.958µs30982026/11/11 23:38:06 INFO Received push request method=POST path=/30992026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=031002026/11/11 23:38:06 INFO Received uploads request method=POST path=/31012026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads31022026/11/11 23:38:06 INFO Received complete upload request method=POST path=/31032026/11/11 23:38:06 INFO Completed upload id=1 objects=9 duration=37.1055ms31042026/11/11 23:38:06 INFO Received uploads request method=POST path=/31052026/11/11 23:38:06 INFO Received complete push request method=POST path=/31062026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031072026/11/11 23:38:06 INFO Vacuumed table table=closures31082026/11/11 23:38:06 INFO Completed upload id=11 objects=6 duration=773.625µs31092026/11/11 23:38:06 INFO Vacuumed table table=pending_closures31102026/11/11 23:38:06 INFO Completed push id=6 objects=7 duration=1.225125ms31112026/11/11 23:38:06 INFO Received push request method=POST path=/31122026/11/11 23:38:06 INFO Vacuumed table table=objects31132026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=031142026/11/11 23:38:06 INFO Received push request method=POST path=/31152026/11/11 23:38:06 INFO Received uploads request method=POST path=/31162026/11/11 23:38:06 INFO Received uploads request method=POST path=/31172026/11/11 23:38:06 INFO Received uploads request method=POST path=/31182026/11/11 23:38:06 INFO Vacuumed table table=pending_objects31192026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads31202026/11/11 23:38:06 INFO Vacuumed table table=closures31212026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031222026/11/11 23:38:06 INFO Vacuumed table table=pending_closures31232026/11/11 23:38:06 INFO Vacuumed table table=objects31242026/11/11 23:38:06 INFO Received uploads request method=POST path=/31252026/11/11 23:38:06 INFO Vacuumed table table=pending_objects31262026/11/11 23:38:06 INFO Received complete upload request method=POST path=/31272026/11/11 23:38:06 INFO Received push request method=POST path=/31282026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=031292026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads31302026/11/11 23:38:06 INFO Completed upload id=22 objects=22 duration=644.666µs31312026/11/11 23:38:06 INFO Received push request method=POST path=/31322026/11/11 23:38:06 INFO Received uploads request method=POST path=/31332026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031342026/11/11 23:38:06 INFO Received push request method=POST path=/31352026/11/11 23:38:06 INFO Vacuumed table table=closures31362026/11/11 23:38:06 INFO Received push request method=POST path=/31372026/11/11 23:38:06 INFO Vacuumed table table=pending_closures31382026/11/11 23:38:06 INFO Vacuumed table table=objects31392026/11/11 23:38:06 INFO Received push request method=POST path=/31402026/11/11 23:38:06 INFO Vacuumed table table=pending_objects31412026/11/11 23:38:06 INFO Received complete push request method=POST path=/31422026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads31432026/11/11 23:38:06 INFO Received complete push request method=POST path=/31442026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=031452026/11/11 23:38:06 INFO Received uploads request method=POST path=/31462026/11/11 23:38:06 INFO Received push request method=POST path=/31472026/11/11 23:38:06 INFO Completed push id=9 objects=5 duration=661.916µs31482026/11/11 23:38:06 INFO Vacuumed table table=closures31492026/11/11 23:38:06 INFO Completed push id=26 objects=5 duration=505.875µs31502026/11/11 23:38:06 INFO Received uploads request method=POST path=/31512026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=031522026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=031532026/11/11 23:38:06 INFO Vacuumed table table=objects31542026/11/11 23:38:06 WARN Force mode enabled - objects will be deleted immediately without grace period31552026/11/11 23:38:06 INFO Received push request method=POST path=/31562026/11/11 23:38:06 INFO Received complete upload request method=POST path=/31572026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031582026/11/11 23:38:06 WARN Force mode enabled - objects will be deleted immediately without grace period31592026/11/11 23:38:06 INFO Received complete upload request method=POST path=/31602026/11/11 23:38:06 INFO Received complete upload request method=POST path=/31612026/11/11 23:38:06 INFO Completed upload id=24 objects=14 duration=235.167µs31622026/11/11 23:38:06 INFO Received complete push request method=POST path=/31632026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031642026/11/11 23:38:06 INFO Vacuumed table table=pending_closures31652026/11/11 23:38:06 INFO Completed upload id=8 objects=3 duration=1.165958ms31662026/11/11 23:38:06 INFO Completed upload id=25 objects=16 duration=1.200708ms31672026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=031682026/11/11 23:38:06 INFO Received complete push request method=POST path=/31692026/11/11 23:38:06 INFO Completed push id=14 objects=3 duration=248.75µs31702026/11/11 23:38:06 INFO Vacuumed table table=pending_closures31712026/11/11 23:38:06 INFO Completed push id=19 objects=3 duration=740.041µs31722026/11/11 23:38:06 INFO Received uploads request method=POST path=/31732026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031742026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=031752026/11/11 23:38:06 INFO Vacuumed table table=pending_objects31762026/11/11 23:38:06 INFO Vacuumed table table=pending_objects31772026/11/11 23:38:06 INFO Received complete upload request method=POST path=/31782026/11/11 23:38:06 INFO Received uploads request method=POST path=/31792026/11/11 23:38:06 WARN Force mode enabled - objects will be deleted immediately without grace period31802026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads31812026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads31822026/11/11 23:38:06 INFO Completed upload id=20 objects=4 duration=352.334µs31832026/11/11 23:38:06 INFO Vacuumed table table=pending_closures31842026/11/11 23:38:06 INFO Received complete push request method=POST path=/31852026/11/11 23:38:06 INFO Vacuumed table table=closures31862026/11/11 23:38:06 INFO Vacuumed table table=closures31872026/11/11 23:38:06 INFO Vacuumed table table=pending_objects31882026/11/11 23:38:06 INFO Completed push id=6 objects=6 duration=6.5615ms31892026/11/11 23:38:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031902026/11/11 23:38:06 INFO Vacuumed table table=objects31912026/11/11 23:38:06 INFO Received uploads request method=POST path=/31922026/11/11 23:38:06 INFO Vacuumed table table=objects31932026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads31942026/11/11 23:38:06 INFO Vacuumed table table=pending_closures31952026/11/11 23:38:06 INFO Received push request method=POST path=/31962026/11/11 23:38:06 INFO Vacuumed table table=closures31972026/11/11 23:38:06 INFO Received push request method=POST path=/31982026/11/11 23:38:06 INFO Received complete upload request method=POST path=/31992026/11/11 23:38:06 INFO Received push request method=POST path=/32002026/11/11 23:38:06 INFO Vacuumed table table=pending_objects32012026/11/11 23:38:06 INFO Completed upload id=25 objects=4 duration=575.208µs32022026/11/11 23:38:06 INFO Vacuumed table table=objects32032026/11/11 23:38:06 INFO Vacuumed table table=multipart_uploads32042026/11/11 23:38:06 INFO Received uploads request method=POST path=/32052026/11/11 23:38:06 INFO Received push request method=POST path=/32062026/11/11 23:38:06 INFO Aborted multipart uploads count=0 kept=032072026/11/11 23:38:06 INFO Received push request method=POST path=/32082026/11/11 23:38:06 INFO Vacuumed table table=closures32092026/11/11 23:38:07 WARN Force mode enabled - objects will be deleted immediately without grace period32102026/11/11 23:38:07 INFO Received complete push request method=POST path=/32112026/11/11 23:38:07 INFO Received complete push request method=POST path=/32122026/11/11 23:38:07 INFO Vacuumed table table=objects32132026/11/11 23:38:07 INFO Completed push id=20 objects=8 duration=465.916µs32142026/11/11 23:38:07 INFO Completed push id=26 objects=14 duration=11.112667ms32152026/11/11 23:38:07 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032162026/11/11 23:38:07 INFO Received push request method=POST path=/32172026/11/11 23:38:07 INFO Received uploads request method=POST path=/32182026/11/11 23:38:07 INFO Received push request method=POST path=/32192026/11/11 23:38:07 INFO Vacuumed table table=pending_closures32202026/11/11 23:38:07 INFO Received push request method=POST path=/32212026/11/11 23:38:07 INFO Received push request method=POST path=/32222026/11/11 23:38:07 INFO Vacuumed table table=pending_objects32232026/11/11 23:38:07 INFO Received uploads request method=POST path=/32242026/11/11 23:38:07 INFO Received uploads request method=POST path=/32252026/11/11 23:38:07 INFO Received complete push request method=POST path=/32262026/11/11 23:38:07 INFO Vacuumed table table=multipart_uploads32272026/11/11 23:38:07 INFO Received complete upload request method=POST path=/32282026/11/11 23:38:07 INFO Vacuumed table table=closures32292026/11/11 23:38:07 INFO Completed push id=27 objects=11 duration=379.208µs32302026/11/11 23:38:07 INFO Completed upload id=22 objects=5 duration=715.708µs32312026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=032322026/11/11 23:38:07 INFO Vacuumed table table=objects32332026/11/11 23:38:07 INFO Received complete upload request method=POST path=/32342026/11/11 23:38:07 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032352026/11/11 23:38:07 INFO Received complete push request method=POST path=/32362026/11/11 23:38:07 INFO Received push request method=POST path=/32372026/11/11 23:38:07 INFO Completed upload id=29 objects=26 duration=315.75µs32382026/11/11 23:38:07 INFO Vacuumed table table=pending_closures32392026/11/11 23:38:07 INFO Completed push id=28 objects=3 duration=468.416µs32402026/11/11 23:38:07 INFO Vacuumed table table=pending_objects32412026/11/11 23:38:07 INFO Received complete push request method=POST path=/32422026/11/11 23:38:07 INFO Received complete push request method=POST path=/32432026/11/11 23:38:07 INFO Vacuumed table table=multipart_uploads32442026/11/11 23:38:07 INFO Completed push id=30 objects=8 duration=11.03075ms32452026/11/11 23:38:07 INFO Completed push id=29 objects=15 duration=351.042µs32462026/11/11 23:38:07 INFO Received complete push request method=POST path=/32472026/11/11 23:38:07 INFO Vacuumed table table=closures32482026/11/11 23:38:07 INFO Vacuumed table table=objects32492026/11/11 23:38:07 INFO Completed push id=10 objects=15 duration=6.296542ms32502026/11/11 23:38:07 INFO Received complete push request method=POST path=/32512026/11/11 23:38:07 INFO Received complete upload request method=POST path=/32522026/11/11 23:38:07 INFO Received push request method=POST path=/32532026/11/11 23:38:07 INFO Completed push id=30 objects=6 duration=360.542µs32542026/11/11 23:38:07 INFO Completed upload id=31 objects=5 duration=434.583µs32552026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=032562026/11/11 23:38:07 INFO Received push request method=POST path=/32572026/11/11 23:38:07 INFO Received complete upload request method=POST path=/32582026/11/11 23:38:07 INFO Received complete push request method=POST path=/32592026/11/11 23:38:07 WARN Force mode enabled - objects will be deleted immediately without grace period32602026/11/11 23:38:07 INFO Received push request method=POST path=/32612026/11/11 23:38:07 INFO Completed upload id=31 objects=3 duration=291.875µs32622026/11/11 23:38:07 INFO Completed push id=32 objects=3 duration=271.959µs32632026/11/11 23:38:07 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=032642026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=032652026/11/11 23:38:07 WARN Force mode enabled - objects will be deleted immediately without grace period32662026/11/11 23:38:07 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032672026/11/11 23:38:07 INFO Received push request method=POST path=/32682026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=032692026/11/11 23:38:07 INFO Vacuumed table table=pending_closures32702026/11/11 23:38:07 WARN Force mode enabled - objects will be deleted immediately without grace period32712026/11/11 23:38:07 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032722026/11/11 23:38:07 INFO Vacuumed table table=pending_closures32732026/11/11 23:38:07 INFO Vacuumed table table=pending_closures32742026/11/11 23:38:07 INFO Vacuumed table table=pending_objects32752026/11/11 23:38:07 INFO Vacuumed table table=multipart_uploads32762026/11/11 23:38:07 INFO Vacuumed table table=pending_objects32772026/11/11 23:38:07 INFO Vacuumed table table=closures32782026/11/11 23:38:07 INFO Vacuumed table table=multipart_uploads32792026/11/11 23:38:07 INFO Vacuumed table table=objects32802026/11/11 23:38:07 INFO Vacuumed table table=closures32812026/11/11 23:38:07 INFO Vacuumed table table=pending_objects32822026/11/11 23:38:07 INFO Vacuumed table table=multipart_uploads32832026/11/11 23:38:07 INFO Vacuumed table table=objects32842026/11/11 23:38:07 INFO Vacuumed table table=closures32852026/11/11 23:38:07 INFO Vacuumed table table=objects32862026/11/11 23:38:07 INFO Received complete push request method=POST path=/32872026/11/11 23:38:07 INFO Received push request method=POST path=/32882026/11/11 23:38:07 INFO Completed push id=16 objects=5 duration=487.708µs32892026/11/11 23:38:07 INFO Received uploads request method=POST path=/32902026/11/11 23:38:07 INFO Received complete push request method=POST path=/32912026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=032922026/11/11 23:38:07 INFO Completed push id=27 objects=25 duration=10.499459ms32932026/11/11 23:38:07 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=032942026/11/11 23:38:07 INFO Vacuumed table table=pending_closures32952026/11/11 23:38:07 INFO Received uploads request method=POST path=/32962026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=032972026/11/11 23:38:07 INFO Vacuumed table table=pending_objects32982026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=032992026/11/11 23:38:07 INFO Vacuumed table table=multipart_uploads33002026/11/11 23:38:07 WARN Force mode enabled - objects will be deleted immediately without grace period33012026/11/11 23:38:07 INFO Vacuumed table table=closures33022026/11/11 23:38:07 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033032026/11/11 23:38:07 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."33042026/11/11 23:38:07 INFO Vacuumed table table=objects33052026/11/11 23:38:07 INFO Vacuumed table table=pending_closures33062026/11/11 23:38:07 INFO Vacuumed table table=pending_objects33072026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=033082026/11/11 23:38:07 INFO Vacuumed table table=multipart_uploads33092026/11/11 23:38:07 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033102026/11/11 23:38:07 INFO Vacuumed table table=closures33112026/11/11 23:38:07 INFO Vacuumed table table=pending_closures33122026/11/11 23:38:07 INFO Received push request method=POST path=/33132026/11/11 23:38:07 INFO Vacuumed table table=pending_objects33142026/11/11 23:38:07 INFO Vacuumed table table=objects33152026/11/11 23:38:07 INFO Vacuumed table table=multipart_uploads33162026/11/11 23:38:07 INFO Vacuumed table table=closures33172026/11/11 23:38:07 INFO Vacuumed table table=objects33182026/11/11 23:38:07 INFO Received complete push request method=POST path=/33192026/11/11 23:38:07 INFO Completed push id=24 objects=5 duration=1.557291ms33202026/11/11 23:38:07 INFO Received uploads request method=POST path=/33212026/11/11 23:38:07 INFO Received complete upload request method=POST path=/33222026/11/11 23:38:07 INFO Received uploads request method=POST path=/33232026/11/11 23:38:07 INFO Completed upload id=14 objects=8 duration=1.0245ms33242026/11/11 23:38:07 INFO Received uploads request method=POST path=/33252026/11/11 23:38:07 INFO Received push request method=POST path=/33262026/11/11 23:38:07 INFO Received complete upload request method=POST path=/33272026/11/11 23:38:07 INFO Received uploads request method=POST path=/33282026/11/11 23:38:07 INFO Received complete upload request method=POST path=/33292026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=033302026/11/11 23:38:07 INFO Completed upload id=7 objects=10 duration=524.5µs33312026/11/11 23:38:07 INFO Completed upload id=6 objects=24 duration=1.059375ms33322026/11/11 23:38:07 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033332026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=033342026/11/11 23:38:07 INFO Received push request method=POST path=/33352026/11/11 23:38:07 INFO Vacuumed table table=pending_closures33362026/11/11 23:38:07 WARN Force mode enabled - objects will be deleted immediately without grace period33372026/11/11 23:38:07 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=033382026/11/11 23:38:07 INFO Vacuumed table table=pending_objects33392026/11/11 23:38:07 INFO Vacuumed table table=pending_closures33402026/11/11 23:38:07 INFO Vacuumed table table=multipart_uploads33412026/11/11 23:38:07 INFO Vacuumed table table=closures33422026/11/11 23:38:07 INFO Vacuumed table table=objects33432026/11/11 23:38:07 INFO Vacuumed table table=pending_objects33442026/11/11 23:38:07 INFO Vacuumed table table=multipart_uploads33452026/11/11 23:38:07 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."33462026/11/11 23:38:07 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."33472026/11/11 23:38:07 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."33482026/11/11 23:38:07 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."33492026/11/11 23:38:07 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."33502026/11/11 23:38:07 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."33512026/11/11 23:38:07 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."33522026/11/11 23:38:07 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."33532026/11/11 23:38:07 INFO Vacuumed table table=closures33542026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=033552026/11/11 23:38:07 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."33562026/11/11 23:38:07 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."33572026/11/11 23:38:07 INFO Vacuumed table table=objects33582026/11/11 23:38:07 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033592026/11/11 23:38:07 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."33602026/11/11 23:38:07 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."33612026/11/11 23:38:07 INFO Vacuumed table table=pending_closures33622026/11/11 23:38:07 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."33632026/11/11 23:38:07 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."33642026/11/11 23:38:07 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."33652026/11/11 23:38:07 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."33662026/11/11 23:38:07 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."33672026/11/11 23:38:07 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."33682026/11/11 23:38:07 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."33692026/11/11 23:38:07 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."33702026/11/11 23:38:07 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."33712026/11/11 23:38:07 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."33722026/11/11 23:38:07 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."33732026/11/11 23:38:07 INFO Vacuumed table table=pending_objects33742026/11/11 23:38:07 INFO Received uploads request method=POST path=/33752026/11/11 23:38:07 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."33762026/11/11 23:38:07 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."33772026/11/11 23:38:07 INFO Vacuumed table table=multipart_uploads33782026/11/11 23:38:07 INFO Vacuumed table table=closures33792026/11/11 23:38:07 INFO Received uploads request method=POST path=/33802026/11/11 23:38:07 INFO Vacuumed table table=objects33812026/11/11 23:38:07 INFO Received push request method=POST path=/33822026/11/11 23:38:07 INFO Received complete push request method=POST path=/33832026/11/11 23:38:07 INFO Received push request method=POST path=/33842026/11/11 23:38:07 INFO Received uploads request method=POST path=/33852026/11/11 23:38:07 INFO Completed push id=8 objects=6 duration=632.625µs33862026/11/11 23:38:07 INFO Received push request method=POST path=/33872026/11/11 23:38:07 INFO Received push request method=POST path=/33882026/11/11 23:38:07 INFO Received push request method=POST path=/33892026/11/11 23:38:07 INFO Received uploads request method=POST path=/33902026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=033912026/11/11 23:38:07 INFO Received push request method=POST path=/33922026/11/11 23:38:07 WARN Force mode enabled - objects will be deleted immediately without grace period33932026/11/11 23:38:07 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."33942026/11/11 23:38:07 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."33952026/11/11 23:38:07 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."33962026/11/11 23:38:07 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."33972026/11/11 23:38:07 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."33982026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=033992026/11/11 23:38:07 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."34002026/11/11 23:38:07 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."34012026/11/11 23:38:07 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."34022026/11/11 23:38:07 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."34032026/11/11 23:38:07 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034042026/11/11 23:38:07 INFO Received complete push request method=POST path=/34052026/11/11 23:38:07 INFO Vacuumed table table=pending_closures34062026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=034072026/11/11 23:38:07 INFO Completed push id=21 objects=26 duration=464.333µs34082026/11/11 23:38:07 INFO Vacuumed table table=pending_objects34092026/11/11 23:38:07 WARN Force mode enabled - objects will be deleted immediately without grace period34102026/11/11 23:38:07 INFO Received complete upload request method=POST path=/34112026/11/11 23:38:07 INFO Vacuumed table table=multipart_uploads34122026/11/11 23:38:07 INFO Received complete push request method=POST path=/34132026/11/11 23:38:07 INFO Completed upload id=16 objects=17 duration=936.167µs34142026/11/11 23:38:07 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=9 objects-failed-to-delete=034152026/11/11 23:38:07 INFO Vacuumed table table=closures34162026/11/11 23:38:07 INFO Vacuumed table table=objects34172026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=034182026/11/11 23:38:07 INFO Vacuumed table table=pending_closures34192026/11/11 23:38:07 WARN Force mode enabled - objects will be deleted immediately without grace period34202026/11/11 23:38:07 INFO Aborted multipart uploads count=0 kept=034212026/11/11 23:38:07 INFO Received complete upload request method=POST path=/34222026/11/11 23:38:07 WARN Force mode enabled - objects will be deleted immediately without grace period34232026/11/11 23:38:07 INFO Completed push id=1 objects=7 duration=33.257167ms34242026/11/11 23:38:07 INFO Completed upload id=9 objects=3 duration=423.375µs34252026/11/11 23:38:07 INFO Vacuumed table table=pending_objects34262026/11/11 23:38:07 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034272026/11/11 23:38:08 INFO Vacuumed table table=multipart_uploads34282026/11/11 23:38:08 INFO Vacuumed table table=pending_closures34292026/11/11 23:38:08 INFO Received uploads request method=POST path=/34302026/11/11 23:38:08 INFO Vacuumed table table=closures34312026/11/11 23:38:08 INFO Received push request method=POST path=/34322026/11/11 23:38:08 INFO Vacuumed table table=pending_objects34332026/11/11 23:38:08 INFO Vacuumed table table=objects34342026/11/11 23:38:08 INFO Vacuumed table table=multipart_uploads34352026/11/11 23:38:08 INFO Aborted multipart uploads count=0 kept=034362026/11/11 23:38:08 INFO Vacuumed table table=closures34372026/11/11 23:38:08 INFO Received push request method=POST path=/34382026/11/11 23:38:08 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."34392026/11/11 23:38:08 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."34402026/11/11 23:38:08 INFO Received uploads request method=POST path=/34412026/11/11 23:38:08 INFO Vacuumed table table=objects34422026/11/11 23:38:08 INFO Received complete upload request method=POST path=/34432026/11/11 23:38:08 INFO Received push request method=POST path=/34442026/11/11 23:38:08 INFO Received complete push request method=POST path=/34452026/11/11 23:38:08 INFO Received complete upload request method=POST path=/34462026/11/11 23:38:08 INFO Received uploads request method=POST path=/34472026/11/11 23:38:08 INFO Completed upload id=28 objects=17 duration=435.166µs34482026/11/11 23:38:08 INFO Received uploads request method=POST path=/34492026/11/11 23:38:08 INFO Completed upload id=19 objects=2 duration=1.047291ms34502026/11/11 23:38:08 INFO Received complete upload request method=POST path=/34512026/11/11 23:38:08 INFO Received push request method=POST path=/34522026/11/11 23:38:08 INFO Received uploads request method=POST path=/34532026/11/11 23:38:08 INFO Received complete push request method=POST path=/34542026/11/11 23:38:08 INFO Received uploads request method=POST path=/34552026/11/11 23:38:08 INFO Completed push id=1 objects=5 duration=29.012459ms34562026/11/11 23:38:08 INFO Received push request method=POST path=/34572026/11/11 23:38:08 INFO Received complete push request method=POST path=/34582026/11/11 23:38:08 INFO Completed push id=29 objects=2 duration=617.625µs34592026/11/11 23:38:08 INFO Received push request method=POST path=/34602026/11/11 23:38:08 INFO Received complete push request method=POST path=/34612026/11/11 23:38:08 INFO Completed push id=33 objects=7 duration=771.708µs34622026/11/11 23:38:08 INFO Completed upload id=1 objects=15 duration=32.543583ms34632026/11/11 23:38:08 INFO Completed push id=12 objects=3 duration=663.917µs34642026/11/11 23:38:08 INFO Received complete push request method=POST path=/34652026/11/11 23:38:08 INFO Received complete upload request method=POST path=/34662026/11/11 23:38:08 INFO Received uploads request method=POST path=/34672026/11/11 23:38:08 INFO Completed push id=31 objects=6 duration=372µs34682026/11/11 23:38:08 INFO Received uploads request method=POST path=/34692026/11/11 23:38:08 INFO Completed upload id=26 objects=3 duration=715.083µs34702026/11/11 23:38:08 INFO Received uploads request method=POST path=/34712026/11/11 23:38:08 INFO Aborted multipart uploads count=0 kept=034722026/11/11 23:38:08 INFO Received complete push request method=POST path=/34732026/11/11 23:38:08 INFO Received uploads request method=POST path=/34742026/11/11 23:38:08 INFO Aborted multipart uploads count=0 kept=034752026/11/11 23:38:08 INFO Aborted multipart uploads count=0 kept=034762026/11/11 23:38:08 INFO Completed push id=32 objects=9 duration=296.375µs34772026/11/11 23:38:08 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."34782026/11/11 23:38:08 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."34792026/11/11 23:38:08 WARN Force mode enabled - objects will be deleted immediately without grace period34802026/11/11 23:38:08 WARN Force mode enabled - objects will be deleted immediately without grace period34812026/11/11 23:38:08 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."34822026/11/11 23:38:08 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034832026/11/11 23:38:08 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."34842026/11/11 23:38:08 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."34852026/11/11 23:38:08 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."34862026/11/11 23:38:08 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."34872026/11/11 23:38:08 INFO Vacuumed table table=pending_closures34882026/11/11 23:38:08 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034892026/11/11 23:38:08 INFO Vacuumed table table=pending_objects34902026/11/11 23:38:08 INFO Aborted multipart uploads count=0 kept=034912026/11/11 23:38:08 INFO Vacuumed table table=multipart_uploads34922026/11/11 23:38:08 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."34932026/11/11 23:38:08 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."34942026/11/11 23:38:08 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."34952026/11/11 23:38:08 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."34962026/11/11 23:38:08 INFO Vacuumed table table=closures34972026/11/11 23:38:08 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."34982026/11/11 23:38:08 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."34992026/11/11 23:38:08 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."35002026/11/11 23:38:08 INFO Vacuumed table table=objects35012026/11/11 23:38:08 INFO Received push request method=POST path=/35022026/11/11 23:38:08 INFO Received complete upload request method=POST path=/35032026/11/11 23:38:08 INFO Received push request method=POST path=/35042026/11/11 23:38:08 INFO Completed upload id=16 objects=23 duration=536.042µs35052026/11/11 23:38:08 INFO Vacuumed table table=pending_closures35062026/11/11 23:38:08 INFO Aborted multipart uploads count=0 kept=035072026/11/11 23:38:08 INFO Received complete upload request method=POST path=/35082026/11/11 23:38:08 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."35092026/11/11 23:38:08 INFO Completed upload id=15 objects=6 duration=294.583µs35102026/11/11 23:38:08 INFO Received complete push request method=POST path=/35112026/11/11 23:38:08 INFO Received uploads request method=POST path=/35122026/11/11 23:38:08 INFO Completed push id=2 objects=25 duration=919.375µs35132026/11/11 23:38:08 INFO Received uploads request method=POST path=/35142026/11/11 23:38:08 INFO Vacuumed table table=pending_objects35152026/11/11 23:38:08 INFO Vacuumed table table=multipart_uploads35162026/11/11 23:38:08 INFO Received push request method=POST path=/35172026/11/11 23:38:08 INFO Vacuumed table table=closures35182026/11/11 23:38:08 INFO Received complete upload request method=POST path=/35192026/11/11 23:38:08 INFO Vacuumed table table=objects35202026/11/11 23:38:08 INFO Aborted multipart uploads count=0 kept=035212026/11/11 23:38:08 INFO Completed upload id=33 objects=9 duration=271.375µs35222026/11/11 23:38:08 INFO Aborted multipart uploads count=0 kept=035232026/11/11 23:38:08 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035242026/11/11 23:38:08 INFO Aborted multipart uploads count=0 kept=035252026/11/11 23:38:08 WARN Force mode enabled - objects will be deleted immediately without grace period35262026/11/11 23:38:08 INFO Received push request method=POST path=/35272026/11/11 23:38:08 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035282026/11/11 23:38:08 INFO Received push request method=POST path=/35292026/11/11 23:38:08 INFO Vacuumed table table=pending_closures35302026/11/11 23:38:08 INFO Vacuumed table table=pending_closures35312026/11/11 23:38:08 INFO Received complete upload request method=POST path=/35322026/11/11 23:38:08 INFO Completed upload id=17 objects=7 duration=372.5µs35332026/11/11 23:38:08 INFO Received complete upload request method=POST path=/35342026/11/11 23:38:08 INFO Aborted multipart uploads count=0 kept=035352026/11/11 23:38:08 INFO Vacuumed table table=pending_objects35362026/11/11 23:38:08 INFO Completed upload id=2 objects=10 duration=484.583µs35372026/11/11 23:38:08 INFO Vacuumed table table=multipart_uploads35382026/11/11 23:38:08 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."35392026/11/11 23:38:08 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."35402026/11/11 23:38:08 INFO Received push request method=POST path=/35412026/11/11 23:38:08 INFO Vacuumed table table=closures35422026/11/11 23:38:08 INFO Vacuumed table table=pending_objects35432026/11/11 23:38:08 INFO Vacuumed table table=objects35442026/11/11 23:38:08 INFO Vacuumed table table=multipart_uploads35452026/11/11 23:38:08 INFO Received complete push request method=POST path=/35462026/11/11 23:38:08 INFO Received complete push request method=POST path=/35472026/11/11 23:38:08 INFO Completed push id=6 objects=26 duration=1.955959ms35482026/11/11 23:38:08 INFO Completed push id=2 objects=4 duration=712.584µs35492026/11/11 23:38:08 INFO Received complete upload request method=POST path=/35502026/11/11 23:38:08 INFO Vacuumed table table=closures35512026/11/11 23:38:08 INFO Completed upload id=11 objects=7 duration=483.917µs35522026/11/11 23:38:08 INFO Aborted multipart uploads count=0 kept=035532026/11/11 23:38:08 WARN Force mode enabled - objects will be deleted immediately without grace period35542026/11/11 23:38:08 INFO Received uploads request method=POST path=/35552026/11/11 23:38:08 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035562026/11/11 23:38:08 INFO Received uploads request method=POST path=/35572026/11/11 23:38:08 INFO Received uploads request method=POST path=/35582026/11/11 23:38:08 INFO Vacuumed table table=pending_closures35592026/11/11 23:38:08 INFO Received complete upload request method=POST path=/35602026/11/11 23:38:08 INFO Vacuumed table table=pending_objects35612026/11/11 23:38:08 INFO Received uploads request method=POST path=/35622026/11/11 23:38:08 INFO Vacuumed table table=objects35632026/11/11 23:38:08 INFO Completed upload id=14 objects=6 duration=336.375µs35642026/11/11 23:38:08 INFO Vacuumed table table=multipart_uploads35652026/11/11 23:38:08 INFO Vacuumed table table=closures35662026/11/11 23:38:08 INFO Vacuumed table table=objects35672026/11/11 23:38:08 INFO Received push request method=POST path=/35682026/11/11 23:38:08 INFO Aborted multipart uploads count=0 kept=035692026/11/11 23:38:08 INFO Received uploads request method=POST path=/35702026/11/11 23:38:08 WARN Force mode enabled - objects will be deleted immediately without grace period35712026/11/11 23:38:08 INFO Received push request method=POST path=/35722026/11/11 23:38:08 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035732026/11/11 23:38:08 INFO Aborted multipart uploads count=0 kept=035742026/11/11 23:38:08 WARN Force mode enabled - objects will be deleted immediately without grace period35752026/11/11 23:38:08 INFO Received push request method=POST path=/35762026/11/11 23:38:08 INFO Aborted multipart uploads count=0 kept=035772026/11/11 23:38:08 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035782026/11/11 23:38:08 INFO Vacuumed table table=pending_closures35792026/11/11 23:38:08 INFO Vacuumed table table=pending_closures35802026/11/11 23:38:08 INFO Received uploads request method=POST path=/35812026/11/11 23:38:08 INFO Vacuumed table table=pending_objects35822026/11/11 23:38:08 INFO Vacuumed table table=multipart_uploads35832026/11/11 23:38:08 INFO Vacuumed table table=closures35842026/11/11 23:38:08 INFO Vacuumed table table=pending_objects35852026/11/11 23:38:08 INFO Vacuumed table table=multipart_uploads35862026/11/11 23:38:08 INFO Vacuumed table table=objects35872026/11/11 23:38:08 INFO Received complete upload request method=POST path=/35882026/11/11 23:38:08 INFO Completed upload id=7 objects=3 duration=612.917µs35892026/11/11 23:38:08 INFO Received complete push request method=POST path=/35902026/11/11 23:38:08 INFO Completed push id=4 objects=4 duration=1.547917ms35912026/11/11 23:38:08 INFO Vacuumed table table=closures35922026/11/11 23:38:08 INFO Received uploads request method=POST path=/35932026/11/11 23:38:08 INFO Aborted multipart uploads count=0 kept=035942026/11/11 23:38:08 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035952026/11/11 23:38:08 INFO Received complete push request method=POST path=/35962026/11/11 23:38:08 INFO Vacuumed table table=objects35972026/11/11 23:38:08 INFO Completed push id=15 objects=24 duration=1.386ms35982026/11/11 23:38:08 INFO Received uploads request method=POST path=/35992026/11/11 23:38:08 INFO Received uploads request method=POST path=/36002026/11/11 23:38:09 INFO Vacuumed table table=pending_closures36012026/11/11 23:38:09 INFO Received push request method=POST path=/36022026/11/11 23:38:09 INFO Received uploads request method=POST path=/36032026/11/11 23:38:09 INFO Received uploads request method=POST path=/36042026/11/11 23:38:09 INFO Vacuumed table table=pending_objects36052026/11/11 23:38:09 INFO Vacuumed table table=multipart_uploads36062026/11/11 23:38:09 INFO Vacuumed table table=closures36072026/11/11 23:38:09 INFO Received complete push request method=POST path=/36082026/11/11 23:38:09 INFO Completed push id=3 objects=5 duration=806.292µs36092026/11/11 23:38:09 INFO Received push request method=POST path=/36102026/11/11 23:38:09 INFO Received complete push request method=POST path=/36112026/11/11 23:38:09 INFO Vacuumed table table=objects36122026/11/11 23:38:09 INFO Completed push id=6 objects=6 duration=819.708µs36132026/11/11 23:38:09 INFO Received push request method=POST path=/36142026/11/11 23:38:09 INFO Received complete upload request method=POST path=/36152026/11/11 23:38:09 INFO Completed upload id=6 objects=6 duration=458.958µs36162026/11/11 23:38:09 INFO Aborted multipart uploads count=0 kept=036172026/11/11 23:38:09 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=036182026/11/11 23:38:09 WARN Force mode enabled - objects will be deleted immediately without grace period36192026/11/11 23:38:09 INFO Vacuumed table table=pending_closures36202026/11/11 23:38:09 INFO Received complete push request method=POST path=/36212026/11/11 23:38:09 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036222026/11/11 23:38:09 INFO Received complete push request method=POST path=/36232026/11/11 23:38:09 INFO Vacuumed table table=pending_objects36242026/11/11 23:38:09 INFO Completed push id=18 objects=23 duration=910.834µs36252026/11/11 23:38:09 INFO Vacuumed table table=multipart_uploads36262026/11/11 23:38:09 INFO Completed push id=13 objects=12 duration=6.432917ms36272026/11/11 23:38:09 INFO Vacuumed table table=pending_closures36282026/11/11 23:38:09 INFO Received uploads request method=POST path=/36292026/11/11 23:38:09 INFO Aborted multipart uploads count=0 kept=036302026/11/11 23:38:09 INFO Vacuumed table table=pending_objects36312026/11/11 23:38:09 INFO Vacuumed table table=multipart_uploads36322026/11/11 23:38:09 INFO Vacuumed table table=closures36332026/11/11 23:38:09 INFO Vacuumed table table=objects36342026/11/11 23:38:09 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=036352026/11/11 23:38:09 INFO Vacuumed table table=pending_closures36362026/11/11 23:38:09 INFO Vacuumed table table=pending_objects36372026/11/11 23:38:09 INFO Vacuumed table table=multipart_uploads36382026/11/11 23:38:09 INFO Vacuumed table table=closures36392026/11/11 23:38:09 INFO Received push request method=POST path=/36402026/11/11 23:38:09 INFO Vacuumed table table=closures36412026/11/11 23:38:09 INFO Received complete push request method=POST path=/36422026/11/11 23:38:09 INFO Received complete push request method=POST path=/36432026/11/11 23:38:09 INFO Received uploads request method=POST path=/36442026/11/11 23:38:09 INFO Vacuumed table table=objects36452026/11/11 23:38:09 INFO Completed push id=4 objects=4 duration=626.542µs36462026/11/11 23:38:09 INFO Completed push id=9 objects=9 duration=1.325667ms36472026/11/11 23:38:09 INFO Received push request method=POST path=/36482026/11/11 23:38:09 INFO Vacuumed table table=objects36492026/11/11 23:38:09 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."36502026/11/11 23:38:09 INFO Aborted multipart uploads count=0 kept=036512026/11/11 23:38:09 INFO Received push request method=POST path=/36522026/11/11 23:38:09 WARN Force mode enabled - objects will be deleted immediately without grace period36532026/11/11 23:38:09 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036542026/11/11 23:38:09 INFO Vacuumed table table=pending_closures36552026/11/11 23:38:09 INFO Received complete push request method=POST path=/36562026/11/11 23:38:09 INFO Received complete push request method=POST path=/36572026/11/11 23:38:09 INFO Completed push id=5 objects=3 duration=484.917µs36582026/11/11 23:38:09 INFO Completed push id=11 objects=9 duration=486.333µs36592026/11/11 23:38:09 INFO Vacuumed table table=pending_objects36602026/11/11 23:38:09 INFO Aborted multipart uploads count=0 kept=036612026/11/11 23:38:09 INFO Received push request method=POST path=/36622026/11/11 23:38:09 INFO Vacuumed table table=multipart_uploads36632026/11/11 23:38:09 INFO Received complete upload request method=POST path=/36642026/11/11 23:38:09 INFO Vacuumed table table=closures36652026/11/11 23:38:09 INFO Received push request method=POST path=/36662026/11/11 23:38:09 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=4 objects-deleted-after-grace-period=0 objects-failed-to-delete=036672026/11/11 23:38:09 INFO Completed upload id=6 objects=7 duration=909.25µs36682026/11/11 23:38:09 INFO Vacuumed table table=objects36692026/11/11 23:38:09 INFO Received complete upload request method=POST path=/36702026/11/11 23:38:09 INFO Completed upload id=21 objects=8 duration=665.75µs36712026/11/11 23:38:09 INFO Received push request method=POST path=/36722026/11/11 23:38:09 INFO Received uploads request method=POST path=/36732026/11/11 23:38:09 INFO Received complete push request method=POST path=/36742026/11/11 23:38:09 INFO Completed push id=22 objects=6 duration=626.959µs36752026/11/11 23:38:09 INFO Received push request method=POST path=/36762026/11/11 23:38:09 INFO Aborted multipart uploads count=0 kept=036772026/11/11 23:38:09 INFO Received complete push request method=POST path=/36782026/11/11 23:38:09 INFO Received uploads request method=POST path=/36792026/11/11 23:38:09 WARN Force mode enabled - objects will be deleted immediately without grace period36802026/11/11 23:38:09 INFO Completed push id=24 objects=8 duration=272.375µs36812026/11/11 23:38:09 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036822026/11/11 23:38:09 INFO Vacuumed table table=pending_closures36832026/11/11 23:38:09 INFO Vacuumed table table=pending_closures36842026/11/11 23:38:09 INFO Vacuumed table table=pending_objects36852026/11/11 23:38:09 INFO Vacuumed table table=multipart_uploads36862026/11/11 23:38:09 INFO Vacuumed table table=closures36872026/11/11 23:38:09 INFO Received complete upload request method=POST path=/36882026/11/11 23:38:09 INFO Vacuumed table table=objects36892026/11/11 23:38:09 INFO Aborted multipart uploads count=0 kept=036902026/11/11 23:38:09 INFO Completed upload id=25 objects=9 duration=323.042µs36912026/11/11 23:38:09 INFO Vacuumed table table=pending_objects36922026/11/11 23:38:09 INFO Received uploads request method=POST path=/36932026/11/11 23:38:09 WARN Force mode enabled - objects will be deleted immediately without grace period36942026/11/11 23:38:09 INFO Received complete upload request method=POST path=/36952026/11/11 23:38:09 INFO Completed upload id=27 objects=3 duration=291.584µs36962026/11/11 23:38:09 INFO Vacuumed table table=multipart_uploads36972026/11/11 23:38:09 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036982026/11/11 23:38:09 INFO Received complete upload request method=POST path=/36992026/11/11 23:38:09 INFO Received push request method=POST path=/37002026/11/11 23:38:09 INFO Completed upload id=28 objects=6 duration=209.375µs37012026/11/11 23:38:09 INFO Vacuumed table table=pending_closures37022026/11/11 23:38:09 INFO Received complete upload request method=POST path=/37032026/11/11 23:38:09 INFO Received complete push request method=POST path=/37042026/11/11 23:38:09 INFO Vacuumed table table=pending_objects37052026/11/11 23:38:09 INFO Completed upload id=23 objects=15 duration=5.520959ms37062026/11/11 23:38:09 INFO Completed push id=30 objects=4 duration=5.529417ms37072026/11/11 23:38:09 INFO Vacuumed table table=multipart_uploads37082026/11/11 23:38:09 INFO Vacuumed table table=closures37092026/11/11 23:38:09 INFO Vacuumed table table=objects37102026/11/11 23:38:09 INFO Vacuumed table table=closures37112026/11/11 23:38:09 INFO Received complete push request method=POST path=/37122026/11/11 23:38:09 INFO Received complete push request method=POST path=/37132026/11/11 23:38:09 INFO Aborted multipart uploads count=0 kept=037142026/11/11 23:38:09 INFO Completed push id=31 objects=7 duration=523.125µs37152026/11/11 23:38:09 WARN Force mode enabled - objects will be deleted immediately without grace period37162026/11/11 23:38:09 INFO Completed push id=7 objects=27 duration=658.958µs37172026/11/11 23:38:09 INFO Received complete upload request method=POST path=/37182026/11/11 23:38:09 INFO Completed upload id=32 objects=5 duration=292.583µs37192026/11/11 23:38:09 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037202026/11/11 23:38:09 INFO Vacuumed table table=objects37212026/11/11 23:38:09 INFO Vacuumed table table=pending_closures37222026/11/11 23:38:09 INFO Vacuumed table table=pending_objects37232026/11/11 23:38:09 INFO Vacuumed table table=multipart_uploads37242026/11/11 23:38:09 INFO Vacuumed table table=closures37252026/11/11 23:38:09 INFO Received push request method=POST path=/37262026/11/11 23:38:09 INFO Vacuumed table table=objects37272026/11/11 23:38:09 INFO Received push request method=POST path=/37282026/11/11 23:38:09 INFO Received uploads request method=POST path=/37292026/11/11 23:38:09 INFO Received complete upload request method=POST path=/37302026/11/11 23:38:09 INFO Completed upload id=19 objects=6 duration=365.5µs37312026/11/11 23:38:09 INFO Received push request method=POST path=/37322026/11/11 23:38:09 INFO Received uploads request method=POST path=/37332026/11/11 23:38:09 INFO Received uploads request method=POST path=/37342026/11/11 23:38:09 INFO Received uploads request method=POST path=/37352026/11/11 23:38:09 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=037362026/11/11 23:38:09 INFO Vacuumed table table=pending_closures37372026/11/11 23:38:09 INFO Vacuumed table table=pending_objects37382026/11/11 23:38:09 INFO Vacuumed table table=multipart_uploads37392026/11/11 23:38:09 INFO Vacuumed table table=closures37402026/11/11 23:38:09 INFO Vacuumed table table=objects37412026/11/11 23:38:09 INFO Aborted multipart uploads count=0 kept=037422026/11/11 23:38:09 WARN Force mode enabled - objects will be deleted immediately without grace period37432026/11/11 23:38:09 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037442026/11/11 23:38:09 INFO Vacuumed table table=pending_closures37452026/11/11 23:38:09 INFO Vacuumed table table=pending_objects37462026/11/11 23:38:09 INFO Vacuumed table table=multipart_uploads37472026/11/11 23:38:09 INFO Vacuumed table table=closures37482026/11/11 23:38:09 INFO Received complete upload request method=POST path=/37492026/11/11 23:38:09 INFO Vacuumed table table=objects37502026/11/11 23:38:09 INFO Received complete upload request method=POST path=/37512026/11/11 23:38:09 INFO Completed upload id=22 objects=7 duration=302.167µs37522026/11/11 23:38:09 INFO Completed upload id=3 objects=9 duration=477.375µs37532026/11/11 23:38:09 INFO Received complete upload request method=POST path=/37542026/11/11 23:38:09 INFO Completed upload id=15 objects=3 duration=599.292µs37552026/11/11 23:38:09 INFO Received push request method=POST path=/37562026/11/11 23:38:09 INFO Received complete push request method=POST path=/37572026/11/11 23:38:09 INFO Completed push id=24 objects=9 duration=495.292µs37582026/11/11 23:38:10 INFO Received push request method=POST path=/37592026/11/11 23:38:10 INFO Received push request method=POST path=/37602026/11/11 23:38:10 INFO Received uploads request method=POST path=/37612026/11/11 23:38:10 INFO Received complete push request method=POST path=/37622026/11/11 23:38:10 INFO Completed push id=21 objects=3 duration=352.5µs37632026/11/11 23:38:10 INFO Received uploads request method=POST path=/37642026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=037652026/11/11 23:38:10 INFO Received complete push request method=POST path=/37662026/11/11 23:38:10 INFO Received complete push request method=POST path=/37672026/11/11 23:38:10 INFO Received complete upload request method=POST path=/37682026/11/11 23:38:10 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=3 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037692026/11/11 23:38:10 INFO Completed push id=5 objects=6 duration=378.208µs37702026/11/11 23:38:10 INFO Completed upload id=23 objects=16 duration=617.834µs37712026/11/11 23:38:10 INFO Completed push id=19 objects=6 duration=802.458µs37722026/11/11 23:38:10 INFO Vacuumed table table=pending_closures37732026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=037742026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=037752026/11/11 23:38:10 WARN Force mode enabled - objects will be deleted immediately without grace period37762026/11/11 23:38:10 WARN Force mode enabled - objects will be deleted immediately without grace period37772026/11/11 23:38:10 INFO Vacuumed table table=pending_objects37782026/11/11 23:38:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037792026/11/11 23:38:10 INFO Vacuumed table table=multipart_uploads37802026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=037812026/11/11 23:38:10 INFO Vacuumed table table=pending_closures37822026/11/11 23:38:10 INFO Vacuumed table table=closures37832026/11/11 23:38:10 INFO Vacuumed table table=pending_objects37842026/11/11 23:38:10 WARN Force mode enabled - objects will be deleted immediately without grace period37852026/11/11 23:38:10 INFO Vacuumed table table=multipart_uploads37862026/11/11 23:38:10 INFO Vacuumed table table=objects37872026/11/11 23:38:10 INFO Vacuumed table table=closures37882026/11/11 23:38:10 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."37892026/11/11 23:38:10 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."37902026/11/11 23:38:10 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."37912026/11/11 23:38:10 INFO Vacuumed table table=objects37922026/11/11 23:38:10 INFO Received complete upload request method=POST path=/37932026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=037942026/11/11 23:38:10 INFO Received push request method=POST path=/37952026/11/11 23:38:10 WARN Force mode enabled - objects will be deleted immediately without grace period37962026/11/11 23:38:10 INFO Completed upload id=10 objects=6 duration=10.203167ms37972026/11/11 23:38:10 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."37982026/11/11 23:38:10 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."37992026/11/11 23:38:10 INFO Received uploads request method=POST path=/38002026/11/11 23:38:10 INFO Received push request method=POST path=/38012026/11/11 23:38:10 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."38022026/11/11 23:38:10 INFO Received push request method=POST path=/38032026/11/11 23:38:10 INFO Received complete push request method=POST path=/38042026/11/11 23:38:10 INFO Received complete upload request method=POST path=/38052026/11/11 23:38:10 INFO Completed push id=30 objects=4 duration=1.126458ms38062026/11/11 23:38:10 INFO Completed upload id=19 objects=25 duration=923.959µs38072026/11/11 23:38:10 INFO Received complete push request method=POST path=/38082026/11/11 23:38:10 INFO Received uploads request method=POST path=/38092026/11/11 23:38:10 INFO Completed push id=8 objects=6 duration=524.833µs38102026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=038112026/11/11 23:38:10 INFO Received push request method=POST path=/38122026/11/11 23:38:10 WARN Force mode enabled - objects will be deleted immediately without grace period38132026/11/11 23:38:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=038142026/11/11 23:38:10 INFO Vacuumed table table=pending_closures38152026/11/11 23:38:10 INFO Received push request method=POST path=/38162026/11/11 23:38:10 INFO Vacuumed table table=pending_objects38172026/11/11 23:38:10 INFO Received complete push request method=POST path=/38182026/11/11 23:38:10 INFO Vacuumed table table=multipart_uploads38192026/11/11 23:38:10 INFO Completed push id=32 objects=7 duration=6.6075ms38202026/11/11 23:38:10 INFO Vacuumed table table=closures38212026/11/11 23:38:10 INFO Received complete push request method=POST path=/38222026/11/11 23:38:10 INFO Completed push id=26 objects=4 duration=455.791µs38232026/11/11 23:38:10 INFO Vacuumed table table=objects38242026/11/11 23:38:10 INFO Received complete upload request method=POST path=/38252026/11/11 23:38:10 INFO Completed upload id=28 objects=13 duration=280.583µs38262026/11/11 23:38:10 INFO Received complete push request method=POST path=/38272026/11/11 23:38:10 INFO Completed push id=12 objects=7 duration=673.167µs38282026/11/11 23:38:10 INFO Received push request method=POST path=/38292026/11/11 23:38:10 INFO Received complete push request method=POST path=/38302026/11/11 23:38:10 INFO Completed push id=29 objects=3 duration=356.666µs38312026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=038322026/11/11 23:38:10 INFO Received uploads request method=POST path=/38332026/11/11 23:38:10 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=038342026/11/11 23:38:10 INFO Received complete upload request method=POST path=/38352026/11/11 23:38:10 INFO Completed upload id=31 objects=2 duration=276.625µs38362026/11/11 23:38:10 INFO Received uploads request method=POST path=/38372026/11/11 23:38:10 INFO Received complete push request method=POST path=/38382026/11/11 23:38:10 INFO Completed push id=33 objects=6 duration=247.709µs38392026/11/11 23:38:10 INFO Received push request method=POST path=/38402026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=038412026/11/11 23:38:10 INFO Received push request method=POST path=/38422026/11/11 23:38:10 INFO Vacuumed table table=pending_closures38432026/11/11 23:38:10 WARN Force mode enabled - objects will be deleted immediately without grace period38442026/11/11 23:38:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038452026/11/11 23:38:10 INFO Received complete upload request method=POST path=/38462026/11/11 23:38:10 INFO Received uploads request method=POST path=/38472026/11/11 23:38:10 INFO Completed upload id=34 objects=24 duration=396.834µs38482026/11/11 23:38:10 INFO Received complete push request method=POST path=/38492026/11/11 23:38:10 INFO Vacuumed table table=pending_closures38502026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=038512026/11/11 23:38:10 INFO Completed push id=16 objects=4 duration=763.709µs38522026/11/11 23:38:10 INFO Vacuumed table table=pending_objects38532026/11/11 23:38:10 WARN Force mode enabled - objects will be deleted immediately without grace period38542026/11/11 23:38:10 INFO Vacuumed table table=multipart_uploads38552026/11/11 23:38:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038562026/11/11 23:38:10 INFO Received complete push request method=POST path=/38572026/11/11 23:38:10 INFO Vacuumed table table=closures38582026/11/11 23:38:10 INFO Received uploads request method=POST path=/38592026/11/11 23:38:10 INFO Vacuumed table table=pending_objects38602026/11/11 23:38:10 INFO Vacuumed table table=multipart_uploads38612026/11/11 23:38:10 INFO Vacuumed table table=objects38622026/11/11 23:38:10 INFO Completed push id=5 objects=9 duration=10.970708ms38632026/11/11 23:38:10 INFO Vacuumed table table=closures38642026/11/11 23:38:10 INFO Received complete upload request method=POST path=/38652026/11/11 23:38:10 INFO Completed upload id=20 objects=5 duration=372.292µs38662026/11/11 23:38:10 INFO Vacuumed table table=objects38672026/11/11 23:38:10 INFO Vacuumed table table=pending_closures38682026/11/11 23:38:10 INFO Received push request method=POST path=/38692026/11/11 23:38:10 INFO Received uploads request method=POST path=/38702026/11/11 23:38:10 INFO Received uploads request method=POST path=/38712026/11/11 23:38:10 INFO Received push request method=POST path=/38722026/11/11 23:38:10 INFO Received push request method=POST path=/38732026/11/11 23:38:10 INFO Received uploads request method=POST path=/38742026/11/11 23:38:10 INFO Received push request method=POST path=/38752026/11/11 23:38:10 INFO Received uploads request method=POST path=/38762026/11/11 23:38:10 INFO Vacuumed table table=pending_objects38772026/11/11 23:38:10 INFO Received push request method=POST path=/38782026/11/11 23:38:10 INFO Received uploads request method=POST path=/38792026/11/11 23:38:10 INFO Received push request method=POST path=/38802026/11/11 23:38:10 INFO Vacuumed table table=multipart_uploads38812026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=038822026/11/11 23:38:10 INFO Received uploads request method=POST path=/38832026/11/11 23:38:10 INFO Vacuumed table table=closures38842026/11/11 23:38:10 WARN Force mode enabled - objects will be deleted immediately without grace period38852026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=038862026/11/11 23:38:10 INFO Vacuumed table table=objects38872026/11/11 23:38:10 INFO Received uploads request method=POST path=/38882026/11/11 23:38:10 WARN Force mode enabled - objects will be deleted immediately without grace period38892026/11/11 23:38:10 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038902026/11/11 23:38:10 INFO Received complete push request method=POST path=/38912026/11/11 23:38:10 INFO Vacuumed table table=pending_closures38922026/11/11 23:38:10 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038932026/11/11 23:38:10 INFO Received push request method=POST path=/38942026/11/11 23:38:10 INFO Completed push id=17 objects=22 duration=430.292µs38952026/11/11 23:38:10 INFO Vacuumed table table=pending_objects38962026/11/11 23:38:10 INFO Vacuumed table table=multipart_uploads38972026/11/11 23:38:10 INFO Vacuumed table table=pending_closures38982026/11/11 23:38:10 INFO Received uploads request method=POST path=/38992026/11/11 23:38:10 INFO Received uploads request method=POST path=/39002026/11/11 23:38:10 INFO Vacuumed table table=pending_objects39012026/11/11 23:38:10 INFO Vacuumed table table=multipart_uploads39022026/11/11 23:38:10 INFO Vacuumed table table=closures39032026/11/11 23:38:10 INFO Received push request method=POST path=/39042026/11/11 23:38:10 INFO Vacuumed table table=objects39052026/11/11 23:38:10 INFO Vacuumed table table=closures39062026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=039072026/11/11 23:38:10 INFO Vacuumed table table=objects39082026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=039092026/11/11 23:38:10 INFO Received push request method=POST path=/39102026/11/11 23:38:10 WARN Force mode enabled - objects will be deleted immediately without grace period39112026/11/11 23:38:10 WARN Force mode enabled - objects will be deleted immediately without grace period39122026/11/11 23:38:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039132026/11/11 23:38:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039142026/11/11 23:38:10 INFO Received complete push request method=POST path=/39152026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=039162026/11/11 23:38:10 INFO Vacuumed table table=pending_closures39172026/11/11 23:38:10 INFO Completed push id=19 objects=14 duration=5.127084ms39182026/11/11 23:38:10 INFO Received complete push request method=POST path=/39192026/11/11 23:38:10 INFO Vacuumed table table=pending_objects39202026/11/11 23:38:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039212026/11/11 23:38:10 INFO Vacuumed table table=pending_closures39222026/11/11 23:38:10 INFO Vacuumed table table=multipart_uploads39232026/11/11 23:38:10 INFO Vacuumed table table=pending_closures39242026/11/11 23:38:10 INFO Vacuumed table table=pending_objects39252026/11/11 23:38:10 INFO Received uploads request method=POST path=/39262026/11/11 23:38:10 INFO Vacuumed table table=closures39272026/11/11 23:38:10 INFO Vacuumed table table=multipart_uploads39282026/11/11 23:38:10 INFO Vacuumed table table=objects39292026/11/11 23:38:10 INFO Vacuumed table table=closures39302026/11/11 23:38:10 INFO Vacuumed table table=pending_objects39312026/11/11 23:38:10 INFO Completed push id=3 objects=22 duration=30.335417ms39322026/11/11 23:38:10 INFO Vacuumed table table=multipart_uploads39332026/11/11 23:38:10 INFO Received push request method=POST path=/39342026/11/11 23:38:10 INFO Vacuumed table table=closures39352026/11/11 23:38:10 INFO Vacuumed table table=objects39362026/11/11 23:38:10 INFO Vacuumed table table=objects39372026/11/11 23:38:10 INFO Received push request method=POST path=/39382026/11/11 23:38:10 INFO Received complete push request method=POST path=/39392026/11/11 23:38:10 INFO Completed push id=13 objects=15 duration=1.193125ms39402026/11/11 23:38:10 INFO Received push request method=POST path=/39412026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=039422026/11/11 23:38:10 INFO Received push request method=POST path=/39432026/11/11 23:38:10 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."39442026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=039452026/11/11 23:38:10 WARN Force mode enabled - objects will be deleted immediately without grace period39462026/11/11 23:38:10 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039472026/11/11 23:38:10 INFO Vacuumed table table=pending_closures39482026/11/11 23:38:10 INFO Vacuumed table table=pending_objects39492026/11/11 23:38:10 INFO Vacuumed table table=multipart_uploads39502026/11/11 23:38:10 INFO Vacuumed table table=closures39512026/11/11 23:38:10 INFO Vacuumed table table=objects39522026/11/11 23:38:10 INFO Received complete upload request method=POST path=/39532026/11/11 23:38:10 INFO Received push request method=POST path=/39542026/11/11 23:38:10 INFO Completed upload id=20 objects=6 duration=314.625µs39552026/11/11 23:38:10 INFO Received push request method=POST path=/39562026/11/11 23:38:10 INFO Received complete push request method=POST path=/39572026/11/11 23:38:10 INFO Completed push id=31 objects=14 duration=914µs39582026/11/11 23:38:10 INFO Received uploads request method=POST path=/39592026/11/11 23:38:10 INFO Received complete push request method=POST path=/39602026/11/11 23:38:10 INFO Received complete upload request method=POST path=/39612026/11/11 23:38:10 INFO Completed push id=11 objects=11 duration=1.155375ms39622026/11/11 23:38:10 INFO Received complete upload request method=POST path=/39632026/11/11 23:38:10 INFO Received uploads request method=POST path=/39642026/11/11 23:38:10 INFO Completed upload id=21 objects=2 duration=340.834µs39652026/11/11 23:38:10 INFO Completed upload id=3 objects=24 duration=592.291µs39662026/11/11 23:38:10 INFO Received uploads request method=POST path=/39672026/11/11 23:38:10 INFO Aborted multipart uploads count=0 kept=039682026/11/11 23:38:10 WARN Force mode enabled - objects will be deleted immediately without grace period39692026/11/11 23:38:10 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039702026/11/11 23:38:11 INFO Vacuumed table table=pending_closures39712026/11/11 23:38:11 INFO Received complete push request method=POST path=/39722026/11/11 23:38:11 INFO Vacuumed table table=pending_objects39732026/11/11 23:38:11 INFO Completed push id=15 objects=7 duration=418.583µs39742026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads39752026/11/11 23:38:11 INFO Vacuumed table table=closures39762026/11/11 23:38:11 INFO Received uploads request method=POST path=/39772026/11/11 23:38:11 INFO Vacuumed table table=objects39782026/11/11 23:38:11 INFO Received complete push request method=POST path=/39792026/11/11 23:38:11 INFO Received complete upload request method=POST path=/39802026/11/11 23:38:11 INFO Received uploads request method=POST path=/39812026/11/11 23:38:11 INFO Received uploads request method=POST path=/39822026/11/11 23:38:11 INFO Completed upload id=2 objects=2 duration=510.166µs39832026/11/11 23:38:11 INFO Completed push id=19 objects=8 duration=5.936916ms39842026/11/11 23:38:11 INFO Received push request method=POST path=/39852026/11/11 23:38:11 INFO Received uploads request method=POST path=/39862026/11/11 23:38:11 INFO Received complete push request method=POST path=/39872026/11/11 23:38:11 INFO Received push request method=POST path=/39882026/11/11 23:38:11 INFO Completed push id=24 objects=24 duration=12.062625ms39892026/11/11 23:38:11 INFO Received complete push request method=POST path=/39902026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=039912026/11/11 23:38:11 INFO Received complete upload request method=POST path=/39922026/11/11 23:38:11 INFO Completed push id=7 objects=15 duration=914.125µs39932026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=039942026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039952026/11/11 23:38:11 INFO Completed upload id=22 objects=2 duration=364.583µs39962026/11/11 23:38:11 INFO Received push request method=POST path=/39972026/11/11 23:38:11 WARN Force mode enabled - objects will be deleted immediately without grace period39982026/11/11 23:38:11 INFO Vacuumed table table=pending_closures39992026/11/11 23:38:11 INFO Received complete push request method=POST path=/40002026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=040012026/11/11 23:38:11 INFO Received push request method=POST path=/40022026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040032026/11/11 23:38:11 INFO Completed push id=6 objects=5 duration=486.459µs40042026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=040052026/11/11 23:38:11 INFO Vacuumed table table=pending_objects40062026/11/11 23:38:11 INFO Vacuumed table table=pending_closures40072026/11/11 23:38:11 INFO Received push request method=POST path=/40082026/11/11 23:38:11 INFO Vacuumed table table=pending_closures40092026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads40102026/11/11 23:38:11 INFO Received push request method=POST path=/40112026/11/11 23:38:11 INFO Vacuumed table table=pending_objects40122026/11/11 23:38:11 INFO Received push request method=POST path=/40132026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads40142026/11/11 23:38:11 INFO Vacuumed table table=pending_objects40152026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads40162026/11/11 23:38:11 INFO Vacuumed table table=closures40172026/11/11 23:38:11 INFO Vacuumed table table=closures40182026/11/11 23:38:11 INFO Received complete push request method=POST path=/40192026/11/11 23:38:11 INFO Vacuumed table table=closures40202026/11/11 23:38:11 INFO Vacuumed table table=objects40212026/11/11 23:38:11 INFO Vacuumed table table=objects40222026/11/11 23:38:11 INFO Received uploads request method=POST path=/40232026/11/11 23:38:11 INFO Completed push id=20 objects=3 duration=408.5µs40242026/11/11 23:38:11 INFO Received uploads request method=POST path=/40252026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=040262026/11/11 23:38:11 INFO Vacuumed table table=objects40272026/11/11 23:38:11 WARN Force mode enabled - objects will be deleted immediately without grace period40282026/11/11 23:38:11 INFO Received push request method=POST path=/40292026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040302026/11/11 23:38:11 INFO Received push request method=POST path=/40312026/11/11 23:38:11 INFO Received push request method=POST path=/40322026/11/11 23:38:11 INFO Vacuumed table table=pending_closures40332026/11/11 23:38:11 INFO Received push request method=POST path=/40342026/11/11 23:38:11 INFO Received uploads request method=POST path=/40352026/11/11 23:38:11 INFO Received push request method=POST path=/40362026/11/11 23:38:11 INFO Received push request method=POST path=/40372026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=040382026/11/11 23:38:11 INFO Vacuumed table table=pending_objects40392026/11/11 23:38:11 INFO Received uploads request method=POST path=/40402026/11/11 23:38:11 INFO Received push request method=POST path=/40412026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=040422026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads40432026/11/11 23:38:11 INFO Received push request method=POST path=/40442026/11/11 23:38:11 INFO Received complete upload request method=POST path=/40452026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040462026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040472026/11/11 23:38:11 INFO Vacuumed table table=closures40482026/11/11 23:38:11 INFO Completed upload id=22 objects=6 duration=278.542µs40492026/11/11 23:38:11 INFO Received uploads request method=POST path=/40502026/11/11 23:38:11 INFO Vacuumed table table=pending_closures40512026/11/11 23:38:11 INFO Vacuumed table table=pending_closures40522026/11/11 23:38:11 INFO Vacuumed table table=objects40532026/11/11 23:38:11 INFO Received uploads request method=POST path=/40542026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=040552026/11/11 23:38:11 INFO Vacuumed table table=pending_objects40562026/11/11 23:38:11 INFO Received push request method=POST path=/40572026/11/11 23:38:11 WARN Force mode enabled - objects will be deleted immediately without grace period40582026/11/11 23:38:11 INFO Received uploads request method=POST path=/40592026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads40602026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040612026/11/11 23:38:11 INFO Received uploads request method=POST path=/40622026/11/11 23:38:11 INFO Vacuumed table table=pending_objects40632026/11/11 23:38:11 INFO Vacuumed table table=pending_closures40642026/11/11 23:38:11 INFO Received push request method=POST path=/40652026/11/11 23:38:11 INFO Received uploads request method=POST path=/40662026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads40672026/11/11 23:38:11 INFO Vacuumed table table=closures40682026/11/11 23:38:11 INFO Vacuumed table table=pending_objects40692026/11/11 23:38:11 INFO Received push request method=POST path=/40702026/11/11 23:38:11 INFO Received uploads request method=POST path=/40712026/11/11 23:38:11 INFO Vacuumed table table=closures40722026/11/11 23:38:11 INFO Vacuumed table table=objects40732026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads40742026/11/11 23:38:11 INFO Received uploads request method=POST path=/40752026/11/11 23:38:11 INFO Received complete upload request method=POST path=/40762026/11/11 23:38:11 INFO Vacuumed table table=closures40772026/11/11 23:38:11 INFO Received push request method=POST path=/40782026/11/11 23:38:11 INFO Received uploads request method=POST path=/40792026/11/11 23:38:11 INFO Vacuumed table table=objects40802026/11/11 23:38:11 INFO Completed upload id=2 objects=26 duration=13.7345ms40812026/11/11 23:38:11 INFO Received push request method=POST path=/40822026/11/11 23:38:11 INFO Received complete push request method=POST path=/40832026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=040842026/11/11 23:38:11 INFO Received uploads request method=POST path=/40852026/11/11 23:38:11 INFO Received uploads request method=POST path=/40862026/11/11 23:38:11 INFO Vacuumed table table=objects40872026/11/11 23:38:11 INFO Received push request method=POST path=/40882026/11/11 23:38:11 INFO Completed push id=4 objects=6 duration=469.458µs40892026/11/11 23:38:11 INFO Received push request method=POST path=/40902026/11/11 23:38:11 INFO Received push request method=POST path=/40912026/11/11 23:38:11 INFO Vacuumed table table=pending_closures40922026/11/11 23:38:11 INFO Vacuumed table table=pending_objects40932026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads40942026/11/11 23:38:11 INFO Vacuumed table table=closures40952026/11/11 23:38:11 INFO Received uploads request method=POST path=/40962026/11/11 23:38:11 INFO Received uploads request method=POST path=/40972026/11/11 23:38:11 INFO Received push request method=POST path=/40982026/11/11 23:38:11 INFO Received uploads request method=POST path=/40992026/11/11 23:38:11 INFO Vacuumed table table=objects41002026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=041012026/11/11 23:38:11 INFO Received uploads request method=POST path=/41022026/11/11 23:38:11 WARN Force mode enabled - objects will be deleted immediately without grace period41032026/11/11 23:38:11 INFO Received uploads request method=POST path=/41042026/11/11 23:38:11 INFO Received push request method=POST path=/41052026/11/11 23:38:11 INFO Received complete push request method=POST path=/41062026/11/11 23:38:11 INFO Received push request method=POST path=/41072026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041082026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=041092026/11/11 23:38:11 INFO Received push request method=POST path=/41102026/11/11 23:38:11 INFO Completed push id=7 objects=7 duration=366.208µs41112026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=041122026/11/11 23:38:11 INFO Vacuumed table table=pending_closures41132026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041142026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=041152026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=041162026/11/11 23:38:11 INFO Received complete upload request method=POST path=/41172026/11/11 23:38:11 INFO Vacuumed table table=pending_closures41182026/11/11 23:38:11 INFO Received complete push request method=POST path=/41192026/11/11 23:38:11 INFO Vacuumed table table=pending_closures41202026/11/11 23:38:11 INFO Vacuumed table table=pending_objects41212026/11/11 23:38:11 INFO Completed upload id=23 objects=7 duration=6.256959ms41222026/11/11 23:38:11 INFO Vacuumed table table=pending_objects41232026/11/11 23:38:11 INFO Completed push id=12 objects=6 duration=6.228458ms41242026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads41252026/11/11 23:38:11 WARN Force mode enabled - objects will be deleted immediately without grace period41262026/11/11 23:38:11 INFO Vacuumed table table=pending_objects41272026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads41282026/11/11 23:38:11 INFO Vacuumed table table=closures41292026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads41302026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041312026/11/11 23:38:11 INFO Received complete upload request method=POST path=/41322026/11/11 23:38:11 INFO Completed upload id=26 objects=5 duration=310.875µs41332026/11/11 23:38:11 INFO Vacuumed table table=pending_closures41342026/11/11 23:38:11 INFO Vacuumed table table=objects41352026/11/11 23:38:11 INFO Vacuumed table table=closures41362026/11/11 23:38:11 INFO Vacuumed table table=pending_objects41372026/11/11 23:38:11 INFO Received complete push request method=POST path=/41382026/11/11 23:38:11 INFO Received push request method=POST path=/41392026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads41402026/11/11 23:38:11 INFO Completed push id=27 objects=3 duration=612.292µs41412026/11/11 23:38:11 INFO Vacuumed table table=objects41422026/11/11 23:38:11 INFO Vacuumed table table=closures41432026/11/11 23:38:11 INFO Received complete upload request method=POST path=/41442026/11/11 23:38:11 INFO Received push request method=POST path=/41452026/11/11 23:38:11 INFO Vacuumed table table=objects41462026/11/11 23:38:11 INFO Received complete push request method=POST path=/41472026/11/11 23:38:11 INFO Completed upload id=28 objects=5 duration=389.834µs41482026/11/11 23:38:11 INFO Vacuumed table table=closures41492026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=041502026/11/11 23:38:11 INFO Completed push id=8 objects=6 duration=752.417µs41512026/11/11 23:38:11 INFO Vacuumed table table=objects41522026/11/11 23:38:11 INFO Received complete push request method=POST path=/41532026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041542026/11/11 23:38:11 INFO Received uploads request method=POST path=/41552026/11/11 23:38:11 INFO Completed push id=29 objects=17 duration=391.458µs41562026/11/11 23:38:11 INFO Vacuumed table table=pending_closures41572026/11/11 23:38:11 INFO Received push request method=POST path=/41582026/11/11 23:38:11 INFO Received push request method=POST path=/41592026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=041602026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=041612026/11/11 23:38:11 INFO Vacuumed table table=pending_objects41622026/11/11 23:38:11 INFO Received push request method=POST path=/41632026/11/11 23:38:11 WARN Force mode enabled - objects will be deleted immediately without grace period41642026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041652026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads41662026/11/11 23:38:11 INFO Received push request method=POST path=/41672026/11/11 23:38:11 INFO Vacuumed table table=closures41682026/11/11 23:38:11 INFO Received complete upload request method=POST path=/41692026/11/11 23:38:11 INFO Vacuumed table table=pending_closures41702026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041712026/11/11 23:38:11 INFO Vacuumed table table=objects41722026/11/11 23:38:11 INFO Completed upload id=9 objects=2 duration=10.510583ms41732026/11/11 23:38:11 INFO Vacuumed table table=pending_objects41742026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads41752026/11/11 23:38:11 INFO Vacuumed table table=closures41762026/11/11 23:38:11 INFO Vacuumed table table=pending_closures41772026/11/11 23:38:11 INFO Received complete push request method=POST path=/41782026/11/11 23:38:11 INFO Received push request method=POST path=/41792026/11/11 23:38:11 INFO Received push request method=POST path=/41802026/11/11 23:38:11 INFO Vacuumed table table=objects41812026/11/11 23:38:11 INFO Completed push id=8 objects=3 duration=518.125µs41822026/11/11 23:38:11 INFO Received push request method=POST path=/41832026/11/11 23:38:11 INFO Received push request method=POST path=/41842026/11/11 23:38:11 INFO Vacuumed table table=pending_objects41852026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=041862026/11/11 23:38:11 INFO Received uploads request method=POST path=/41872026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads41882026/11/11 23:38:11 INFO Received push request method=POST path=/41892026/11/11 23:38:11 INFO Vacuumed table table=closures41902026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041912026/11/11 23:38:11 INFO Received complete upload request method=POST path=/41922026/11/11 23:38:11 INFO Completed upload id=13 objects=10 duration=270.625µs41932026/11/11 23:38:11 INFO Vacuumed table table=pending_closures41942026/11/11 23:38:11 INFO Vacuumed table table=objects41952026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=041962026/11/11 23:38:11 INFO Received uploads request method=POST path=/41972026/11/11 23:38:11 INFO Vacuumed table table=pending_objects41982026/11/11 23:38:11 WARN Force mode enabled - objects will be deleted immediately without grace period41992026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads42002026/11/11 23:38:11 INFO Received uploads request method=POST path=/42012026/11/11 23:38:11 INFO Vacuumed table table=closures42022026/11/11 23:38:11 INFO Vacuumed table table=objects42032026/11/11 23:38:11 INFO Received push request method=POST path=/42042026/11/11 23:38:11 INFO Received complete upload request method=POST path=/42052026/11/11 23:38:11 INFO Received push request method=POST path=/42062026/11/11 23:38:11 INFO Received uploads request method=POST path=/42072026/11/11 23:38:11 INFO Completed upload id=33 objects=5 duration=801.042µs42082026/11/11 23:38:11 INFO Received push request method=POST path=/42092026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=042102026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042112026/11/11 23:38:11 INFO Vacuumed table table=pending_closures42122026/11/11 23:38:11 INFO Vacuumed table table=pending_objects42132026/11/11 23:38:11 INFO Vacuumed table table=multipart_uploads42142026/11/11 23:38:11 INFO Received push request method=POST path=/42152026/11/11 23:38:11 INFO Received complete push request method=POST path=/42162026/11/11 23:38:11 INFO Vacuumed table table=closures42172026/11/11 23:38:11 INFO Received uploads request method=POST path=/42182026/11/11 23:38:11 INFO Completed push id=19 objects=14 duration=485.083µs42192026/11/11 23:38:11 INFO Vacuumed table table=objects42202026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=042212026/11/11 23:38:11 WARN Force mode enabled - objects will be deleted immediately without grace period42222026/11/11 23:38:11 INFO Received uploads request method=POST path=/42232026/11/11 23:38:11 INFO Received push request method=POST path=/42242026/11/11 23:38:11 INFO Received push request method=POST path=/42252026/11/11 23:38:11 INFO Aborted multipart uploads count=0 kept=042262026/11/11 23:38:11 WARN Force mode enabled - objects will be deleted immediately without grace period42272026/11/11 23:38:11 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042282026/11/11 23:38:11 INFO Vacuumed table table=pending_closures42292026/11/11 23:38:11 INFO Received complete push request method=POST path=/42302026/11/11 23:38:11 INFO Completed push id=7 objects=20 duration=614.334µs42312026/11/11 23:38:11 INFO Vacuumed table table=pending_objects42322026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads42332026/11/11 23:38:12 INFO Vacuumed table table=closures42342026/11/11 23:38:12 INFO Vacuumed table table=objects42352026/11/11 23:38:12 INFO Aborted multipart uploads count=0 kept=042362026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042372026/11/11 23:38:12 INFO Vacuumed table table=pending_closures42382026/11/11 23:38:12 INFO Aborted multipart uploads count=0 kept=042392026/11/11 23:38:12 WARN Force mode enabled - objects will be deleted immediately without grace period42402026/11/11 23:38:12 INFO Vacuumed table table=pending_objects42412026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042422026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads42432026/11/11 23:38:12 INFO Vacuumed table table=pending_closures42442026/11/11 23:38:12 INFO Vacuumed table table=closures42452026/11/11 23:38:12 INFO Vacuumed table table=objects42462026/11/11 23:38:12 INFO Vacuumed table table=pending_objects42472026/11/11 23:38:12 INFO Received complete push request method=POST path=/42482026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads42492026/11/11 23:38:12 INFO Received complete push request method=POST path=/42502026/11/11 23:38:12 INFO Vacuumed table table=closures42512026/11/11 23:38:12 INFO Completed push id=9 objects=7 duration=6.216625ms42522026/11/11 23:38:12 INFO Completed push id=13 objects=22 duration=6.210834ms42532026/11/11 23:38:12 INFO Received complete upload request method=POST path=/42542026/11/11 23:38:12 INFO Vacuumed table table=objects42552026/11/11 23:38:12 INFO Completed upload id=14 objects=16 duration=955.458µs42562026/11/11 23:38:12 INFO Received complete push request method=POST path=/42572026/11/11 23:38:12 INFO Received push request method=POST path=/42582026/11/11 23:38:12 INFO Completed push id=21 objects=6 duration=593.5µs42592026/11/11 23:38:12 INFO Received complete upload request method=POST path=/42602026/11/11 23:38:12 INFO Completed upload id=22 objects=7 duration=380.542µs42612026/11/11 23:38:12 INFO Received complete push request method=POST path=/42622026/11/11 23:38:12 INFO Completed push id=24 objects=4 duration=336.875µs42632026/11/11 23:38:12 INFO Received complete push request method=POST path=/42642026/11/11 23:38:12 INFO Completed push id=25 objects=9 duration=254.625µs42652026/11/11 23:38:12 INFO Received complete push request method=POST path=/42662026/11/11 23:38:12 INFO Received complete upload request method=POST path=/42672026/11/11 23:38:12 INFO Completed upload id=26 objects=3 duration=218.125µs42682026/11/11 23:38:12 INFO Completed push id=5 objects=3 duration=6.582833ms42692026/11/11 23:38:12 INFO Received push request method=POST path=/42702026/11/11 23:38:12 INFO Received complete push request method=POST path=/42712026/11/11 23:38:12 INFO Received complete push request method=POST path=/42722026/11/11 23:38:12 INFO Completed push id=8 objects=6 duration=499.5µs42732026/11/11 23:38:12 INFO Received complete push request method=POST path=/42742026/11/11 23:38:12 INFO Completed push id=16 objects=7 duration=534.417µs42752026/11/11 23:38:12 INFO Completed push id=9 objects=24 duration=417.416µs42762026/11/11 23:38:12 INFO Received push request method=POST path=/42772026/11/11 23:38:12 INFO Received complete upload request method=POST path=/42782026/11/11 23:38:12 INFO Received uploads request method=POST path=/42792026/11/11 23:38:12 INFO Received uploads request method=POST path=/42802026/11/11 23:38:12 INFO Completed upload id=10 objects=7 duration=330.416µs42812026/11/11 23:38:12 INFO Received uploads request method=POST path=/42822026/11/11 23:38:12 INFO Received push request method=POST path=/42832026/11/11 23:38:12 INFO Received uploads request method=POST path=/42842026/11/11 23:38:12 INFO Received push request method=POST path=/42852026/11/11 23:38:12 INFO Received uploads request method=POST path=/42862026/11/11 23:38:12 INFO Received complete upload request method=POST path=/42872026/11/11 23:38:12 INFO Received complete upload request method=POST path=/42882026/11/11 23:38:12 INFO Completed upload id=10 objects=10 duration=236.167µs42892026/11/11 23:38:12 INFO Completed upload id=18 objects=4 duration=334.708µs42902026/11/11 23:38:12 INFO Received uploads request method=POST path=/42912026/11/11 23:38:12 INFO Received complete upload request method=POST path=/42922026/11/11 23:38:12 INFO Received push request method=POST path=/42932026/11/11 23:38:12 INFO Completed upload id=27 objects=7 duration=271.583µs42942026/11/11 23:38:12 INFO Received push request method=POST path=/42952026/11/11 23:38:12 INFO Received complete upload request method=POST path=/42962026/11/11 23:38:12 INFO Received uploads request method=POST path=/42972026/11/11 23:38:12 INFO Completed upload id=28 objects=3 duration=234.666µs42982026/11/11 23:38:12 INFO Received complete push request method=POST path=/42992026/11/11 23:38:12 INFO Completed push id=14 objects=3 duration=253.667µs43002026/11/11 23:38:12 INFO Received push request method=POST path=/43012026/11/11 23:38:12 INFO Received push request method=POST path=/43022026/11/11 23:38:12 INFO Aborted multipart uploads count=0 kept=043032026/11/11 23:38:12 WARN Force mode enabled - objects will be deleted immediately without grace period43042026/11/11 23:38:12 INFO Received complete push request method=POST path=/43052026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043062026/11/11 23:38:12 INFO Completed push id=3 objects=21 duration=10.236667ms43072026/11/11 23:38:12 INFO Vacuumed table table=pending_closures43082026/11/11 23:38:12 INFO Received uploads request method=POST path=/43092026/11/11 23:38:12 INFO Received complete push request method=POST path=/43102026/11/11 23:38:12 INFO Received push request method=POST path=/43112026/11/11 23:38:12 INFO Vacuumed table table=pending_objects43122026/11/11 23:38:12 INFO Completed push id=29 objects=23 duration=6.586916ms43132026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads43142026/11/11 23:38:12 INFO Vacuumed table table=closures43152026/11/11 23:38:12 INFO Received complete push request method=POST path=/43162026/11/11 23:38:12 INFO Received uploads request method=POST path=/43172026/11/11 23:38:12 INFO Completed push id=30 objects=6 duration=495.25µs43182026/11/11 23:38:12 INFO Received complete upload request method=POST path=/43192026/11/11 23:38:12 INFO Received uploads request method=POST path=/43202026/11/11 23:38:12 INFO Vacuumed table table=objects43212026/11/11 23:38:12 INFO Completed upload id=21 objects=25 duration=377.791µs43222026/11/11 23:38:12 INFO Received uploads request method=POST path=/43232026/11/11 23:38:12 INFO Received complete push request method=POST path=/43242026/11/11 23:38:12 INFO Completed push id=15 objects=5 duration=714.625µs43252026/11/11 23:38:12 INFO Received push request method=POST path=/43262026/11/11 23:38:12 INFO Received uploads request method=POST path=/43272026/11/11 23:38:12 INFO Received uploads request method=POST path=/43282026/11/11 23:38:12 INFO Received uploads request method=POST path=/43292026/11/11 23:38:12 INFO Received push request method=POST path=/43302026/11/11 23:38:12 INFO Received complete push request method=POST path=/43312026/11/11 23:38:12 INFO Completed push id=23 objects=6 duration=662.584µs43322026/11/11 23:38:12 INFO Aborted multipart uploads count=0 kept=043332026/11/11 23:38:12 WARN Force mode enabled - objects will be deleted immediately without grace period43342026/11/11 23:38:12 INFO Received complete upload request method=POST path=/43352026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043362026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=11 objects-failed-to-delete=043372026/11/11 23:38:12 INFO Received complete push request method=POST path=/43382026/11/11 23:38:12 INFO Completed upload id=10 objects=17 duration=491.833µs43392026/11/11 23:38:12 INFO Vacuumed table table=pending_closures43402026/11/11 23:38:12 INFO Received complete push request method=POST path=/43412026/11/11 23:38:12 INFO Vacuumed table table=pending_closures43422026/11/11 23:38:12 INFO Completed push id=24 objects=6 duration=6.246583ms43432026/11/11 23:38:12 INFO Received complete push request method=POST path=/43442026/11/11 23:38:12 INFO Completed push id=33 objects=3 duration=419.583µs43452026/11/11 23:38:12 INFO Vacuumed table table=pending_objects43462026/11/11 23:38:12 INFO Completed push id=17 objects=11 duration=6.016416ms43472026/11/11 23:38:12 INFO Received uploads request method=POST path=/43482026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads43492026/11/11 23:38:12 INFO Vacuumed table table=pending_objects43502026/11/11 23:38:12 INFO Received push request method=POST path=/43512026/11/11 23:38:12 INFO Received push request method=POST path=/43522026/11/11 23:38:12 INFO Received complete push request method=POST path=/43532026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads43542026/11/11 23:38:12 INFO Vacuumed table table=closures43552026/11/11 23:38:12 INFO Completed push id=34 objects=6 duration=272.25µs43562026/11/11 23:38:12 INFO Vacuumed table table=closures43572026/11/11 23:38:12 INFO Vacuumed table table=objects43582026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=043592026/11/11 23:38:12 INFO Vacuumed table table=objects43602026/11/11 23:38:12 INFO Vacuumed table table=pending_closures43612026/11/11 23:38:12 INFO Received push request method=POST path=/43622026/11/11 23:38:12 INFO Aborted multipart uploads count=0 kept=043632026/11/11 23:38:12 WARN Force mode enabled - objects will be deleted immediately without grace period43642026/11/11 23:38:12 INFO Received complete push request method=POST path=/43652026/11/11 23:38:12 INFO Received push request method=POST path=/43662026/11/11 23:38:12 INFO Completed push id=35 objects=8 duration=264.917µs43672026/11/11 23:38:12 INFO Vacuumed table table=pending_objects43682026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043692026/11/11 23:38:12 INFO Received push request method=POST path=/43702026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads43712026/11/11 23:38:12 INFO Vacuumed table table=pending_closures43722026/11/11 23:38:12 INFO Vacuumed table table=closures43732026/11/11 23:38:12 INFO Received uploads request method=POST path=/43742026/11/11 23:38:12 INFO Received complete push request method=POST path=/43752026/11/11 23:38:12 INFO Received push request method=POST path=/43762026/11/11 23:38:12 INFO Received complete upload request method=POST path=/43772026/11/11 23:38:12 INFO Completed upload id=26 objects=6 duration=755.958µs43782026/11/11 23:38:12 INFO Vacuumed table table=pending_objects43792026/11/11 23:38:12 INFO Completed push id=14 objects=6 duration=928.208µs43802026/11/11 23:38:12 INFO Vacuumed table table=objects43812026/11/11 23:38:12 INFO Received complete upload request method=POST path=/43822026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads43832026/11/11 23:38:12 INFO Completed upload id=36 objects=9 duration=412.542µs43842026/11/11 23:38:12 INFO Received uploads request method=POST path=/43852026/11/11 23:38:12 INFO Received uploads request method=POST path=/43862026/11/11 23:38:12 INFO Vacuumed table table=closures43872026/11/11 23:38:12 INFO Received push request method=POST path=/43882026/11/11 23:38:12 INFO Received push request method=POST path=/43892026/11/11 23:38:12 INFO Aborted multipart uploads count=0 kept=043902026/11/11 23:38:12 INFO Received uploads request method=POST path=/43912026/11/11 23:38:12 INFO Received complete upload request method=POST path=/43922026/11/11 23:38:12 INFO Vacuumed table table=objects43932026/11/11 23:38:12 INFO Received push request method=POST path=/43942026/11/11 23:38:12 INFO Received uploads request method=POST path=/43952026/11/11 23:38:12 INFO Completed upload id=29 objects=5 duration=415.792µs43962026/11/11 23:38:12 WARN Force mode enabled - objects will be deleted immediately without grace period43972026/11/11 23:38:12 INFO Aborted multipart uploads count=0 kept=043982026/11/11 23:38:12 INFO Aborted multipart uploads count=0 kept=043992026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044002026/11/11 23:38:12 INFO Received uploads request method=POST path=/44012026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044022026/11/11 23:38:12 INFO Aborted multipart uploads count=0 kept=044032026/11/11 23:38:12 INFO Vacuumed table table=pending_closures44042026/11/11 23:38:12 INFO Received uploads request method=POST path=/44052026/11/11 23:38:12 INFO Received complete push request method=POST path=/44062026/11/11 23:38:12 INFO Vacuumed table table=pending_objects44072026/11/11 23:38:12 WARN Force mode enabled - objects will be deleted immediately without grace period44082026/11/11 23:38:12 INFO Received complete push request method=POST path=/44092026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=044102026/11/11 23:38:12 INFO Received complete push request method=POST path=/44112026/11/11 23:38:12 INFO Vacuumed table table=pending_closures44122026/11/11 23:38:12 INFO Completed push id=25 objects=7 duration=584.208µs44132026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads44142026/11/11 23:38:12 INFO Vacuumed table table=pending_closures44152026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044162026/11/11 23:38:12 INFO Completed push id=14 objects=4 duration=852.416µs44172026/11/11 23:38:12 INFO Completed push id=18 objects=6 duration=1.153792ms44182026/11/11 23:38:12 INFO Vacuumed table table=closures44192026/11/11 23:38:12 INFO Vacuumed table table=pending_closures44202026/11/11 23:38:12 INFO Vacuumed table table=pending_objects44212026/11/11 23:38:12 INFO Vacuumed table table=pending_objects44222026/11/11 23:38:12 INFO Vacuumed table table=objects44232026/11/11 23:38:12 INFO Received complete upload request method=POST path=/44242026/11/11 23:38:12 INFO Received push request method=POST path=/44252026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads44262026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads44272026/11/11 23:38:12 INFO Vacuumed table table=pending_objects44282026/11/11 23:38:12 INFO Completed upload id=23 objects=9 duration=419.917µs44292026/11/11 23:38:12 INFO Received uploads request method=POST path=/44302026/11/11 23:38:12 INFO Vacuumed table table=closures44312026/11/11 23:38:12 INFO Vacuumed table table=closures44322026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads44332026/11/11 23:38:12 INFO Vacuumed table table=objects44342026/11/11 23:38:12 INFO Vacuumed table table=objects44352026/11/11 23:38:12 INFO Received push request method=POST path=/44362026/11/11 23:38:12 INFO Vacuumed table table=closures44372026/11/11 23:38:12 INFO Received uploads request method=POST path=/44382026/11/11 23:38:12 INFO Vacuumed table table=objects44392026/11/11 23:38:12 INFO Received push request method=POST path=/44402026/11/11 23:38:12 INFO Aborted multipart uploads count=0 kept=044412026/11/11 23:38:12 WARN Force mode enabled - objects will be deleted immediately without grace period44422026/11/11 23:38:12 INFO Received complete push request method=POST path=/44432026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044442026/11/11 23:38:12 INFO Received uploads request method=POST path=/44452026/11/11 23:38:12 INFO Received push request method=POST path=/44462026/11/11 23:38:12 INFO Completed push id=14 objects=3 duration=877.541µs44472026/11/11 23:38:12 INFO Received push request method=POST path=/44482026/11/11 23:38:12 INFO Vacuumed table table=pending_closures44492026/11/11 23:38:12 INFO Received uploads request method=POST path=/44502026/11/11 23:38:12 INFO Received complete push request method=POST path=/44512026/11/11 23:38:12 INFO Received uploads request method=POST path=/44522026/11/11 23:38:12 INFO Completed push id=12 objects=2 duration=1.583834ms44532026/11/11 23:38:12 INFO Vacuumed table table=pending_objects44542026/11/11 23:38:12 INFO Received complete push request method=POST path=/44552026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads44562026/11/11 23:38:12 INFO Received uploads request method=POST path=/44572026/11/11 23:38:12 INFO Completed push id=16 objects=26 duration=713.584µs44582026/11/11 23:38:12 INFO Vacuumed table table=closures44592026/11/11 23:38:12 INFO Vacuumed table table=objects44602026/11/11 23:38:12 INFO Received complete upload request method=POST path=/44612026/11/11 23:38:12 INFO Aborted multipart uploads count=0 kept=044622026/11/11 23:38:12 INFO Received complete upload request method=POST path=/44632026/11/11 23:38:12 INFO Completed upload id=31 objects=3 duration=1.067958ms44642026/11/11 23:38:12 INFO Completed upload id=12 objects=6 duration=336.625µs44652026/11/11 23:38:12 INFO Received uploads request method=POST path=/44662026/11/11 23:38:12 WARN Force mode enabled - objects will be deleted immediately without grace period44672026/11/11 23:38:12 INFO Received complete upload request method=POST path=/44682026/11/11 23:38:12 INFO Received complete push request method=POST path=/44692026/11/11 23:38:12 INFO Aborted multipart uploads count=0 kept=044702026/11/11 23:38:12 INFO Completed upload id=22 objects=2 duration=240.667µs44712026/11/11 23:38:12 INFO Received complete push request method=POST path=/44722026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044732026/11/11 23:38:12 INFO Completed push id=17 objects=3 duration=377.959µs44742026/11/11 23:38:12 INFO Completed push id=21 objects=9 duration=635.375µs44752026/11/11 23:38:12 INFO Vacuumed table table=pending_closures44762026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044772026/11/11 23:38:12 INFO Aborted multipart uploads count=0 kept=044782026/11/11 23:38:12 INFO Vacuumed table table=pending_closures44792026/11/11 23:38:12 WARN Force mode enabled - objects will be deleted immediately without grace period44802026/11/11 23:38:12 INFO Vacuumed table table=pending_objects44812026/11/11 23:38:12 INFO Received uploads request method=POST path=/44822026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads44832026/11/11 23:38:12 INFO Vacuumed table table=pending_objects44842026/11/11 23:38:12 INFO Vacuumed table table=closures44852026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044862026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads44872026/11/11 23:38:12 INFO Vacuumed table table=pending_closures44882026/11/11 23:38:12 INFO Vacuumed table table=objects44892026/11/11 23:38:12 INFO Vacuumed table table=closures44902026/11/11 23:38:12 INFO Vacuumed table table=pending_objects44912026/11/11 23:38:12 INFO Received push request method=POST path=/44922026/11/11 23:38:12 INFO Received complete push request method=POST path=/44932026/11/11 23:38:12 INFO Vacuumed table table=objects44942026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads44952026/11/11 23:38:12 INFO Received push request method=POST path=/44962026/11/11 23:38:12 INFO Received uploads request method=POST path=/44972026/11/11 23:38:12 INFO Completed push id=23 objects=5 duration=331.167µs44982026/11/11 23:38:12 INFO Vacuumed table table=closures44992026/11/11 23:38:12 INFO Received complete push request method=POST path=/45002026/11/11 23:38:12 INFO Received push request method=POST path=/45012026/11/11 23:38:12 INFO Aborted multipart uploads count=0 kept=045022026/11/11 23:38:12 INFO Received complete upload request method=POST path=/45032026/11/11 23:38:12 INFO Vacuumed table table=objects45042026/11/11 23:38:12 INFO Completed upload id=24 objects=3 duration=264.5µs45052026/11/11 23:38:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045062026/11/11 23:38:12 INFO Completed push id=27 objects=6 duration=877.583µs45072026/11/11 23:38:12 INFO Received complete upload request method=POST path=/45082026/11/11 23:38:12 INFO Vacuumed table table=pending_closures45092026/11/11 23:38:12 INFO Completed upload id=18 objects=2 duration=800.625µs45102026/11/11 23:38:12 INFO Received complete push request method=POST path=/45112026/11/11 23:38:12 INFO Received complete upload request method=POST path=/45122026/11/11 23:38:12 INFO Completed upload id=15 objects=2 duration=307.625µs45132026/11/11 23:38:12 INFO Completed push id=25 objects=4 duration=6.406125ms45142026/11/11 23:38:12 INFO Vacuumed table table=pending_objects45152026/11/11 23:38:12 INFO Vacuumed table table=multipart_uploads45162026/11/11 23:38:12 INFO Received push request method=POST path=/45172026/11/11 23:38:12 INFO Vacuumed table table=closures45182026/11/11 23:38:12 INFO Vacuumed table table=objects45192026/11/11 23:38:12 INFO Received complete upload request method=POST path=/45202026/11/11 23:38:12 INFO Completed upload id=13 objects=8 duration=500.458µs45212026/11/11 23:38:13 INFO Received uploads request method=POST path=/45222026/11/11 23:38:13 INFO Received complete push request method=POST path=/45232026/11/11 23:38:13 INFO Received push request method=POST path=/45242026/11/11 23:38:13 INFO Received uploads request method=POST path=/45252026/11/11 23:38:13 INFO Received complete upload request method=POST path=/45262026/11/11 23:38:13 INFO Completed push id=22 objects=3 duration=952.292µs45272026/11/11 23:38:13 INFO Received uploads request method=POST path=/45282026/11/11 23:38:13 INFO Completed upload id=26 objects=24 duration=289.542µs45292026/11/11 23:38:13 INFO Received push request method=POST path=/45302026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=045312026/11/11 23:38:13 WARN Force mode enabled - objects will be deleted immediately without grace period45322026/11/11 23:38:13 INFO Received uploads request method=POST path=/45332026/11/11 23:38:13 INFO Received complete upload request method=POST path=/45342026/11/11 23:38:13 INFO Completed upload id=26 objects=8 duration=309.417µs45352026/11/11 23:38:13 INFO Received complete push request method=POST path=/45362026/11/11 23:38:13 INFO Completed push id=28 objects=14 duration=236.5µs45372026/11/11 23:38:13 INFO Received push request method=POST path=/45382026/11/11 23:38:13 INFO Received uploads request method=POST path=/45392026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=045402026/11/11 23:38:13 WARN Force mode enabled - objects will be deleted immediately without grace period45412026/11/11 23:38:13 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045422026/11/11 23:38:13 INFO Vacuumed table table=pending_closures45432026/11/11 23:38:13 INFO Vacuumed table table=pending_objects45442026/11/11 23:38:13 INFO Vacuumed table table=multipart_uploads45452026/11/11 23:38:13 INFO Vacuumed table table=closures45462026/11/11 23:38:13 INFO Vacuumed table table=objects45472026/11/11 23:38:13 INFO Received uploads request method=POST path=/45482026/11/11 23:38:13 INFO Received complete upload request method=POST path=/45492026/11/11 23:38:13 INFO Completed upload id=21 objects=9 duration=600.875µs45502026/11/11 23:38:13 INFO Received complete push request method=POST path=/45512026/11/11 23:38:13 INFO Received complete upload request method=POST path=/45522026/11/11 23:38:13 INFO Completed push id=25 objects=10 duration=736.042µs45532026/11/11 23:38:13 INFO Completed upload id=13 objects=7 duration=402.959µs45542026/11/11 23:38:13 INFO Received complete upload request method=POST path=/45552026/11/11 23:38:13 INFO Completed upload id=29 objects=5 duration=525.25µs45562026/11/11 23:38:13 INFO Received complete upload request method=POST path=/45572026/11/11 23:38:13 INFO Completed upload id=20 objects=4 duration=223.75µs45582026/11/11 23:38:13 INFO Received complete upload request method=POST path=/45592026/11/11 23:38:13 INFO Completed upload id=30 objects=2 duration=206.25µs45602026/11/11 23:38:13 INFO Received complete upload request method=POST path=/45612026/11/11 23:38:13 INFO Completed upload id=23 objects=7 duration=426.708µs45622026/11/11 23:38:13 INFO Received complete push request method=POST path=/45632026/11/11 23:38:13 INFO Received complete push request method=POST path=/45642026/11/11 23:38:13 INFO Completed push id=31 objects=20 duration=321.75µs45652026/11/11 23:38:13 INFO Completed push id=16 objects=16 duration=5.947375ms45662026/11/11 23:38:13 INFO Received complete upload request method=POST path=/45672026/11/11 23:38:13 INFO Completed upload id=32 objects=5 duration=256.875µs45682026/11/11 23:38:13 INFO Received complete push request method=POST path=/45692026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=045702026/11/11 23:38:13 INFO Completed push id=1 objects=11 duration=680.083µs45712026/11/11 23:38:13 INFO Received complete upload request method=POST path=/45722026/11/11 23:38:13 INFO Completed upload id=33 objects=3 duration=269.583µs45732026/11/11 23:38:13 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."45742026/11/11 23:38:13 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."45752026/11/11 23:38:13 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."45762026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=045772026/11/11 23:38:13 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."45782026/11/11 23:38:13 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."45792026/11/11 23:38:13 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."45802026/11/11 23:38:13 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."45812026/11/11 23:38:13 WARN Force mode enabled - objects will be deleted immediately without grace period45822026/11/11 23:38:13 INFO Received uploads request method=POST path=/45832026/11/11 23:38:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045842026/11/11 23:38:13 INFO Received uploads request method=POST path=/45852026/11/11 23:38:13 INFO Received push request method=POST path=/45862026/11/11 23:38:13 INFO Vacuumed table table=pending_closures45872026/11/11 23:38:13 INFO Received uploads request method=POST path=/45882026/11/11 23:38:13 INFO Vacuumed table table=pending_objects45892026/11/11 23:38:13 INFO Vacuumed table table=multipart_uploads45902026/11/11 23:38:13 INFO Vacuumed table table=closures45912026/11/11 23:38:13 INFO Vacuumed table table=objects45922026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=045932026/11/11 23:38:13 WARN Force mode enabled - objects will be deleted immediately without grace period45942026/11/11 23:38:13 INFO Received complete upload request method=POST path=/45952026/11/11 23:38:13 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."45962026/11/11 23:38:13 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."45972026/11/11 23:38:13 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."45982026/11/11 23:38:13 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."45992026/11/11 23:38:13 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."46002026/11/11 23:38:13 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."46012026/11/11 23:38:13 INFO Completed upload id=16 objects=3 duration=351.75µs46022026/11/11 23:38:13 INFO Received uploads request method=POST path=/46032026/11/11 23:38:13 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."46042026/11/11 23:38:13 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."46052026/11/11 23:38:13 INFO Received push request method=POST path=/46062026/11/11 23:38:13 INFO Received complete upload request method=POST path=/46072026/11/11 23:38:13 INFO Received uploads request method=POST path=/46082026/11/11 23:38:13 INFO Received push request method=POST path=/46092026/11/11 23:38:13 INFO Completed upload id=21 objects=3 duration=314.416µs46102026/11/11 23:38:13 INFO Received complete upload request method=POST path=/46112026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=046122026/11/11 23:38:13 INFO Completed upload id=27 objects=26 duration=294.583µs46132026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=046142026/11/11 23:38:13 WARN Force mode enabled - objects will be deleted immediately without grace period46152026/11/11 23:38:13 INFO Received complete push request method=POST path=/46162026/11/11 23:38:13 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."46172026/11/11 23:38:13 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."46182026/11/11 23:38:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=7 objects-failed-to-delete=046192026/11/11 23:38:13 INFO Completed push id=28 objects=6 duration=231.458µs46202026/11/11 23:38:13 INFO Received complete push request method=POST path=/46212026/11/11 23:38:13 INFO Received complete push request method=POST path=/46222026/11/11 23:38:13 INFO Completed push id=5 objects=28 duration=535.375µs46232026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=046242026/11/11 23:38:13 INFO Completed push id=29 objects=6 duration=417.458µs46252026/11/11 23:38:13 WARN Force mode enabled - objects will be deleted immediately without grace period46262026/11/11 23:38:13 INFO Vacuumed table table=pending_closures46272026/11/11 23:38:13 INFO Received complete push request method=POST path=/46282026/11/11 23:38:13 INFO Completed push id=10 objects=2 duration=765.375µs46292026/11/11 23:38:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=046302026/11/11 23:38:13 INFO Vacuumed table table=pending_objects46312026/11/11 23:38:13 INFO Vacuumed table table=pending_closures46322026/11/11 23:38:13 INFO Vacuumed table table=multipart_uploads46332026/11/11 23:38:13 INFO Vacuumed table table=pending_objects46342026/11/11 23:38:13 INFO Received complete push request method=POST path=/46352026/11/11 23:38:13 INFO Vacuumed table table=closures46362026/11/11 23:38:13 INFO Vacuumed table table=multipart_uploads46372026/11/11 23:38:13 INFO Completed push id=26 objects=23 duration=5.853083ms46382026/11/11 23:38:13 INFO Vacuumed table table=closures46392026/11/11 23:38:13 INFO Vacuumed table table=objects46402026/11/11 23:38:13 INFO Vacuumed table table=objects46412026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=046422026/11/11 23:38:13 INFO Received complete upload request method=POST path=/46432026/11/11 23:38:13 WARN Force mode enabled - objects will be deleted immediately without grace period46442026/11/11 23:38:13 INFO Received complete upload request method=POST path=/46452026/11/11 23:38:13 INFO Completed upload id=13 objects=14 duration=1.347875ms46462026/11/11 23:38:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046472026/11/11 23:38:13 INFO Completed upload id=21 objects=7 duration=5.801125ms46482026/11/11 23:38:13 INFO Vacuumed table table=pending_closures46492026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=046502026/11/11 23:38:13 INFO Vacuumed table table=pending_objects46512026/11/11 23:38:13 INFO Vacuumed table table=multipart_uploads46522026/11/11 23:38:13 INFO Vacuumed table table=closures46532026/11/11 23:38:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046542026/11/11 23:38:13 INFO Received push request method=POST path=/46552026/11/11 23:38:13 INFO Vacuumed table table=objects46562026/11/11 23:38:13 INFO Vacuumed table table=pending_closures46572026/11/11 23:38:13 INFO Vacuumed table table=pending_objects46582026/11/11 23:38:13 INFO Received complete upload request method=POST path=/46592026/11/11 23:38:13 INFO Vacuumed table table=multipart_uploads46602026/11/11 23:38:13 INFO Received push request method=POST path=/46612026/11/11 23:38:13 INFO Received uploads request method=POST path=/46622026/11/11 23:38:13 INFO Vacuumed table table=closures46632026/11/11 23:38:13 INFO Completed upload id=17 objects=5 duration=479.375µs46642026/11/11 23:38:13 INFO Received uploads request method=POST path=/46652026/11/11 23:38:13 INFO Vacuumed table table=objects46662026/11/11 23:38:13 INFO Received complete upload request method=POST path=/46672026/11/11 23:38:13 INFO Completed upload id=27 objects=4 duration=825.333µs46682026/11/11 23:38:13 INFO Received complete push request method=POST path=/46692026/11/11 23:38:13 INFO Completed push id=22 objects=3 duration=732.042µs46702026/11/11 23:38:13 INFO Received complete push request method=POST path=/46712026/11/11 23:38:13 INFO Completed push id=13 objects=11 duration=434.959µs46722026/11/11 23:38:13 INFO Received uploads request method=POST path=/46732026/11/11 23:38:13 INFO Received uploads request method=POST path=/46742026/11/11 23:38:13 INFO Received push request method=POST path=/46752026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=046762026/11/11 23:38:13 WARN Force mode enabled - objects will be deleted immediately without grace period46772026/11/11 23:38:13 INFO Received complete push request method=POST path=/46782026/11/11 23:38:13 INFO Received complete upload request method=POST path=/46792026/11/11 23:38:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046802026/11/11 23:38:13 INFO Completed push id=24 objects=3 duration=296.084µs46812026/11/11 23:38:13 INFO Completed upload id=24 objects=18 duration=221.625µs46822026/11/11 23:38:13 INFO Vacuumed table table=pending_closures46832026/11/11 23:38:13 INFO Received complete push request method=POST path=/46842026/11/11 23:38:13 INFO Received complete push request method=POST path=/46852026/11/11 23:38:13 INFO Completed push id=19 objects=3 duration=226.709µs46862026/11/11 23:38:13 INFO Vacuumed table table=pending_objects46872026/11/11 23:38:13 INFO Completed push id=31 objects=6 duration=6.205459ms46882026/11/11 23:38:13 INFO Vacuumed table table=multipart_uploads46892026/11/11 23:38:13 INFO Received push request method=POST path=/46902026/11/11 23:38:13 INFO Received complete push request method=POST path=/46912026/11/11 23:38:13 INFO Vacuumed table table=closures46922026/11/11 23:38:13 INFO Completed push id=25 objects=2 duration=6.736709ms46932026/11/11 23:38:13 INFO Received complete upload request method=POST path=/46942026/11/11 23:38:13 INFO Received uploads request method=POST path=/46952026/11/11 23:38:13 INFO Vacuumed table table=objects46962026/11/11 23:38:13 INFO Completed upload id=30 objects=8 duration=528.708µs46972026/11/11 23:38:13 INFO Received complete push request method=POST path=/46982026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=046992026/11/11 23:38:13 INFO Completed push id=27 objects=4 duration=374.25µs47002026/11/11 23:38:13 WARN Force mode enabled - objects will be deleted immediately without grace period47012026/11/11 23:38:13 INFO Received uploads request method=POST path=/47022026/11/11 23:38:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047032026/11/11 23:38:13 INFO Vacuumed table table=pending_closures47042026/11/11 23:38:13 INFO Received complete push request method=POST path=/47052026/11/11 23:38:13 INFO Completed push id=27 objects=15 duration=322.708µs47062026/11/11 23:38:13 INFO Vacuumed table table=pending_objects47072026/11/11 23:38:13 INFO Vacuumed table table=multipart_uploads47082026/11/11 23:38:13 INFO Vacuumed table table=closures47092026/11/11 23:38:13 INFO Vacuumed table table=objects47102026/11/11 23:38:13 INFO Received complete push request method=POST path=/47112026/11/11 23:38:13 INFO Completed push id=16 objects=14 duration=700.791µs47122026/11/11 23:38:13 INFO Received complete upload request method=POST path=/47132026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=047142026/11/11 23:38:13 INFO Completed upload id=31 objects=6 duration=301.666µs47152026/11/11 23:38:13 WARN Force mode enabled - objects will be deleted immediately without grace period47162026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=047172026/11/11 23:38:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047182026/11/11 23:38:13 INFO Received complete upload request method=POST path=/47192026/11/11 23:38:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=047202026/11/11 23:38:13 INFO Vacuumed table table=pending_closures47212026/11/11 23:38:13 INFO Completed upload id=32 objects=4 duration=304.709µs47222026/11/11 23:38:13 INFO Vacuumed table table=pending_objects47232026/11/11 23:38:13 INFO Vacuumed table table=multipart_uploads47242026/11/11 23:38:13 INFO Vacuumed table table=pending_closures47252026/11/11 23:38:13 INFO Vacuumed table table=closures47262026/11/11 23:38:13 INFO Vacuumed table table=objects47272026/11/11 23:38:13 INFO Vacuumed table table=pending_objects47282026/11/11 23:38:13 INFO Vacuumed table table=multipart_uploads47292026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=047302026/11/11 23:38:13 INFO Vacuumed table table=closures47312026/11/11 23:38:13 INFO Received complete push request method=POST path=/47322026/11/11 23:38:13 INFO Vacuumed table table=objects47332026/11/11 23:38:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047342026/11/11 23:38:13 INFO Completed push id=20 objects=4 duration=500.042µs47352026/11/11 23:38:13 INFO Vacuumed table table=pending_closures47362026/11/11 23:38:13 INFO Vacuumed table table=pending_objects47372026/11/11 23:38:13 INFO Received complete push request method=POST path=/47382026/11/11 23:38:13 INFO Received complete push request method=POST path=/47392026/11/11 23:38:13 INFO Vacuumed table table=multipart_uploads47402026/11/11 23:38:13 INFO Received uploads request method=POST path=/47412026/11/11 23:38:13 INFO Vacuumed table table=closures47422026/11/11 23:38:13 INFO Completed push id=4 objects=6 duration=4.169208ms47432026/11/11 23:38:13 INFO Completed push id=28 objects=26 duration=4.196917ms47442026/11/11 23:38:13 INFO Vacuumed table table=objects47452026/11/11 23:38:13 INFO Received complete push request method=POST path=/47462026/11/11 23:38:13 INFO Received uploads request method=POST path=/47472026/11/11 23:38:13 INFO Received complete upload request method=POST path=/47482026/11/11 23:38:13 INFO Completed push id=30 objects=3 duration=659.042µs47492026/11/11 23:38:13 INFO Completed upload id=33 objects=10 duration=795.833µs47502026/11/11 23:38:13 INFO Received complete upload request method=POST path=/47512026/11/11 23:38:13 INFO Received push request method=POST path=/47522026/11/11 23:38:13 INFO Received push request method=POST path=/47532026/11/11 23:38:13 INFO Received complete push request method=POST path=/47542026/11/11 23:38:13 INFO Completed upload id=26 objects=2 duration=737.542µs47552026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=047562026/11/11 23:38:13 INFO Received uploads request method=POST path=/47572026/11/11 23:38:13 INFO Completed push id=32 objects=19 duration=358.042µs47582026/11/11 23:38:13 WARN Force mode enabled - objects will be deleted immediately without grace period47592026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=047602026/11/11 23:38:13 INFO Received complete upload request method=POST path=/47612026/11/11 23:38:13 INFO Completed upload id=25 objects=6 duration=592µs47622026/11/11 23:38:13 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047632026/11/11 23:38:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047642026/11/11 23:38:13 INFO Aborted multipart uploads count=0 kept=047652026/11/11 23:38:13 WARN Force mode enabled - objects will be deleted immediately without grace period47662026/11/11 23:38:13 INFO Vacuumed table table=pending_closures47672026/11/11 23:38:13 INFO Vacuumed table table=pending_objects47682026/11/11 23:38:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047692026/11/11 23:38:13 INFO Vacuumed table table=multipart_uploads47702026/11/11 23:38:13 INFO Received push request method=POST path=/47712026/11/11 23:38:13 INFO Vacuumed table table=closures47722026/11/11 23:38:13 INFO Received uploads request method=POST path=/47732026/11/11 23:38:14 INFO Vacuumed table table=objects47742026/11/11 23:38:14 INFO Vacuumed table table=pending_closures47752026/11/11 23:38:14 INFO Vacuumed table table=pending_closures47762026/11/11 23:38:14 INFO Received complete push request method=POST path=/47772026/11/11 23:38:14 INFO Received uploads request method=POST path=/47782026/11/11 23:38:14 INFO Completed push id=11 objects=9 duration=370.125µs47792026/11/11 23:38:14 INFO Aborted multipart uploads count=0 kept=047802026/11/11 23:38:14 WARN Force mode enabled - objects will be deleted immediately without grace period47812026/11/11 23:38:14 INFO Vacuumed table table=pending_objects47822026/11/11 23:38:14 INFO Vacuumed table table=pending_objects47832026/11/11 23:38:14 INFO Vacuumed table table=multipart_uploads47842026/11/11 23:38:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047852026/11/11 23:38:14 INFO Vacuumed table table=multipart_uploads47862026/11/11 23:38:14 INFO Vacuumed table table=closures47872026/11/11 23:38:14 INFO Vacuumed table table=pending_closures47882026/11/11 23:38:14 INFO Vacuumed table table=closures47892026/11/11 23:38:14 INFO Vacuumed table table=objects47902026/11/11 23:38:14 INFO Vacuumed table table=objects47912026/11/11 23:38:14 INFO Vacuumed table table=pending_objects47922026/11/11 23:38:14 INFO Vacuumed table table=multipart_uploads47932026/11/11 23:38:14 INFO Received push request method=POST path=/47942026/11/11 23:38:14 INFO Vacuumed table table=closures47952026/11/11 23:38:14 INFO Vacuumed table table=objects47962026/11/11 23:38:14 INFO Received complete push request method=POST path=/47972026/11/11 23:38:14 INFO Received complete upload request method=POST path=/47982026/11/11 23:38:14 INFO Completed push id=15 objects=24 duration=837.125µs47992026/11/11 23:38:14 INFO Completed upload id=22 objects=8 duration=962.625µs48002026/11/11 23:38:14 INFO Aborted multipart uploads count=0 kept=048012026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48022026/11/11 23:38:14 WARN Force mode enabled - objects will be deleted immediately without grace period48032026/11/11 23:38:14 INFO Aborted multipart uploads count=0 kept=048042026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48052026/11/11 23:38:14 INFO Received complete push request method=POST path=/48062026/11/11 23:38:14 INFO Completed upload id=34 objects=23 duration=302.875µs48072026/11/11 23:38:14 INFO Completed push id=23 objects=5 duration=649.25µs48082026/11/11 23:38:14 INFO Completed upload id=19 objects=2 duration=670.833µs48092026/11/11 23:38:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048102026/11/11 23:38:14 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."48112026/11/11 23:38:14 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."48122026/11/11 23:38:14 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."48132026/11/11 23:38:14 INFO Vacuumed table table=pending_closures48142026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48152026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48162026/11/11 23:38:14 INFO Vacuumed table table=pending_objects48172026/11/11 23:38:14 INFO Completed upload id=24 objects=7 duration=231.291µs48182026/11/11 23:38:14 INFO Completed upload id=24 objects=8 duration=355.125µs48192026/11/11 23:38:14 INFO Vacuumed table table=multipart_uploads48202026/11/11 23:38:14 INFO Vacuumed table table=closures48212026/11/11 23:38:14 INFO Vacuumed table table=objects48222026/11/11 23:38:14 INFO Received push request method=POST path=/48232026/11/11 23:38:14 INFO Received complete push request method=POST path=/48242026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48252026/11/11 23:38:14 INFO Completed push id=26 objects=3 duration=374.625µs48262026/11/11 23:38:14 INFO Completed upload id=28 objects=5 duration=286.417µs48272026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48282026/11/11 23:38:14 INFO Received push request method=POST path=/48292026/11/11 23:38:14 INFO Completed upload id=28 objects=8 duration=215.625µs48302026/11/11 23:38:14 INFO Received complete push request method=POST path=/48312026/11/11 23:38:14 INFO Completed push id=28 objects=11 duration=615.5µs48322026/11/11 23:38:14 INFO Received complete push request method=POST path=/48332026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48342026/11/11 23:38:14 INFO Completed push id=28 objects=19 duration=273.583µs48352026/11/11 23:38:14 INFO Completed upload id=29 objects=10 duration=171.291µs48362026/11/11 23:38:14 INFO Received complete push request method=POST path=/48372026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48382026/11/11 23:38:14 INFO Completed upload id=30 objects=20 duration=212.166µs48392026/11/11 23:38:14 INFO Completed push id=35 objects=6 duration=3.845ms48402026/11/11 23:38:14 INFO Received complete push request method=POST path=/48412026/11/11 23:38:14 INFO Aborted multipart uploads count=0 kept=048422026/11/11 23:38:14 INFO Completed push id=31 objects=11 duration=348.958µs48432026/11/11 23:38:14 WARN Force mode enabled - objects will be deleted immediately without grace period48442026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48452026/11/11 23:38:14 INFO Completed upload id=32 objects=2 duration=181.959µs48462026/11/11 23:38:14 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=048472026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48482026/11/11 23:38:14 INFO Received complete push request method=POST path=/48492026/11/11 23:38:14 INFO Completed upload id=30 objects=5 duration=310.417µs48502026/11/11 23:38:14 INFO Completed push id=33 objects=3 duration=259.75µs48512026/11/11 23:38:14 INFO Vacuumed table table=pending_closures48522026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48532026/11/11 23:38:14 INFO Completed upload id=34 objects=3 duration=157.333µs48542026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48552026/11/11 23:38:14 INFO Vacuumed table table=pending_objects48562026/11/11 23:38:14 INFO Aborted multipart uploads count=0 kept=048572026/11/11 23:38:14 INFO Completed upload id=17 objects=25 duration=1.130041ms48582026/11/11 23:38:14 INFO Vacuumed table table=multipart_uploads48592026/11/11 23:38:14 WARN Force mode enabled - objects will be deleted immediately without grace period48602026/11/11 23:38:14 INFO Vacuumed table table=closures48612026/11/11 23:38:14 INFO Received push request method=POST path=/48622026/11/11 23:38:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048632026/11/11 23:38:14 INFO Vacuumed table table=objects48642026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48652026/11/11 23:38:14 INFO Completed upload id=21 objects=2 duration=192.75µs48662026/11/11 23:38:14 INFO Vacuumed table table=pending_closures48672026/11/11 23:38:14 INFO Received complete push request method=POST path=/48682026/11/11 23:38:14 INFO Aborted multipart uploads count=0 kept=048692026/11/11 23:38:14 INFO Completed push id=31 objects=3 duration=384.833µs48702026/11/11 23:38:14 INFO Vacuumed table table=pending_objects48712026/11/11 23:38:14 WARN Force mode enabled - objects will be deleted immediately without grace period48722026/11/11 23:38:14 INFO Vacuumed table table=multipart_uploads48732026/11/11 23:38:14 INFO Received complete push request method=POST path=/48742026/11/11 23:38:14 INFO Vacuumed table table=closures48752026/11/11 23:38:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048762026/11/11 23:38:14 INFO Completed push id=29 objects=10 duration=434.917µs48772026/11/11 23:38:14 INFO Vacuumed table table=pending_closures48782026/11/11 23:38:14 INFO Vacuumed table table=objects48792026/11/11 23:38:14 INFO Vacuumed table table=pending_objects48802026/11/11 23:38:14 INFO Vacuumed table table=multipart_uploads48812026/11/11 23:38:14 INFO Vacuumed table table=closures48822026/11/11 23:38:14 INFO Vacuumed table table=objects48832026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48842026/11/11 23:38:14 INFO Received complete push request method=POST path=/48852026/11/11 23:38:14 INFO Received complete push request method=POST path=/48862026/11/11 23:38:14 INFO Completed upload id=30 objects=6 duration=412.708µs48872026/11/11 23:38:14 INFO Completed push id=1 objects=9 duration=430.292µs48882026/11/11 23:38:14 INFO Completed push id=29 objects=24 duration=410.791µs48892026/11/11 23:38:14 INFO Aborted multipart uploads count=0 kept=048902026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48912026/11/11 23:38:14 INFO Completed upload id=31 objects=5 duration=167.75µs48922026/11/11 23:38:14 WARN Force mode enabled - objects will be deleted immediately without grace period48932026/11/11 23:38:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=048942026/11/11 23:38:14 INFO Received complete upload request method=POST path=/48952026/11/11 23:38:14 INFO Completed upload id=34 objects=3 duration=185.958µs48962026/11/11 23:38:14 INFO Aborted multipart uploads count=0 kept=048972026/11/11 23:38:14 INFO Vacuumed table table=pending_closures48982026/11/11 23:38:14 INFO Received complete push request method=POST path=/48992026/11/11 23:38:14 INFO Completed push id=32 objects=6 duration=255.542µs49002026/11/11 23:38:14 WARN Force mode enabled - objects will be deleted immediately without grace period49012026/11/11 23:38:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049022026/11/11 23:38:14 INFO Vacuumed table table=pending_objects49032026/11/11 23:38:14 INFO Vacuumed table table=multipart_uploads49042026/11/11 23:38:14 INFO Vacuumed table table=pending_closures49052026/11/11 23:38:14 INFO Received complete upload request method=POST path=/49062026/11/11 23:38:14 INFO Completed upload id=33 objects=7 duration=237.292µs49072026/11/11 23:38:14 INFO Vacuumed table table=closures49082026/11/11 23:38:14 INFO Vacuumed table table=objects49092026/11/11 23:38:14 INFO Vacuumed table table=pending_objects49102026/11/11 23:38:14 INFO Vacuumed table table=multipart_uploads49112026/11/11 23:38:14 INFO Vacuumed table table=closures49122026/11/11 23:38:14 INFO Received complete upload request method=POST path=/49132026/11/11 23:38:14 INFO Completed upload id=34 objects=3 duration=287.791µs49142026/11/11 23:38:14 INFO Vacuumed table table=objects49152026/11/11 23:38:14 INFO Received complete push request method=POST path=/49162026/11/11 23:38:14 INFO Completed push id=35 objects=11 duration=256.542µs49172026/11/11 23:38:14 INFO Aborted multipart uploads count=0 kept=049182026/11/11 23:38:14 WARN Force mode enabled - objects will be deleted immediately without grace period49192026/11/11 23:38:14 INFO Garbage collection completed failed-uploads-deleted=6 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049202026/11/11 23:38:14 INFO Vacuumed table table=pending_closures49212026/11/11 23:38:14 INFO Vacuumed table table=pending_objects49222026/11/11 23:38:14 INFO Vacuumed table table=multipart_uploads49232026/11/11 23:38:14 INFO Vacuumed table table=closures49242026/11/11 23:38:14 INFO Vacuumed table table=objects49252026/11/11 23:38:14 INFO Received complete push request method=POST path=/49262026/11/11 23:38:14 INFO Completed push id=7 objects=24 duration=1.147292ms49272026/11/11 23:38:14 INFO Received uploads request method=POST path=/49282026/11/11 23:38:14 INFO Received uploads request method=POST path=/49292026/11/11 23:38:14 INFO Received uploads request method=POST path=/49302026/11/11 23:38:14 INFO Received push request method=POST path=/49312026/11/11 23:38:14 INFO Received uploads request method=POST path=/49322026/11/11 23:38:14 INFO Received complete push request method=POST path=/49332026/11/11 23:38:14 INFO Completed push id=13 objects=2 duration=250.916µs49342026/11/11 23:38:14 INFO Received complete upload request method=POST path=/49352026/11/11 23:38:14 INFO Completed upload id=27 objects=13 duration=211.291µs49362026/11/11 23:38:14 INFO Aborted multipart uploads count=0 kept=049372026/11/11 23:38:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049382026/11/11 23:38:14 INFO Vacuumed table table=pending_closures49392026/11/11 23:38:14 INFO Vacuumed table table=pending_objects49402026/11/11 23:38:14 INFO Vacuumed table table=multipart_uploads49412026/11/11 23:38:14 INFO Vacuumed table table=closures49422026/11/11 23:38:14 INFO Vacuumed table table=objects49432026/11/11 23:38:14 INFO Received uploads request method=POST path=/49442026/11/11 23:38:14 INFO Received push request method=POST path=/49452026/11/11 23:38:14 INFO Received push request method=POST path=/49462026/11/11 23:38:14 INFO Received uploads request method=POST path=/49472026/11/11 23:38:14 INFO Received uploads request method=POST path=/49482026/11/11 23:38:14 INFO Received complete upload request method=POST path=/49492026/11/11 23:38:14 INFO Received complete upload request method=POST path=/49502026/11/11 23:38:14 INFO Completed upload id=29 objects=4 duration=434.5µs49512026/11/11 23:38:14 INFO Completed upload id=2 objects=7 duration=224.625µs49522026/11/11 23:38:14 INFO Received uploads request method=POST path=/49532026/11/11 23:38:14 INFO Received uploads request method=POST path=/49542026/11/11 23:38:14 INFO Received complete push request method=POST path=/49552026/11/11 23:38:14 INFO Completed push id=11 objects=6 duration=441.125µs49562026/11/11 23:38:14 INFO Received complete push request method=POST path=/49572026/11/11 23:38:14 INFO Completed push id=20 objects=13 duration=288.792µs49582026/11/11 23:38:14 INFO Received complete push request method=POST path=/49592026/11/11 23:38:14 INFO Completed push id=22 objects=4 duration=194.5µs49602026/11/11 23:38:14 INFO Received complete upload request method=POST path=/49612026/11/11 23:38:14 INFO Completed upload id=8 objects=6 duration=214.209µs49622026/11/11 23:38:14 INFO Received complete upload request method=POST path=/49632026/11/11 23:38:14 INFO Completed upload id=23 objects=7 duration=230.875µs49642026/11/11 23:38:14 INFO Received complete push request method=POST path=/49652026/11/11 23:38:14 INFO Completed push id=24 objects=5 duration=443.334µs49662026/11/11 23:38:14 INFO Received complete upload request method=POST path=/49672026/11/11 23:38:14 INFO Completed upload id=25 objects=2 duration=149.75µs49682026/11/11 23:38:14 INFO Received complete upload request method=POST path=/49692026/11/11 23:38:14 INFO Completed upload id=26 objects=20 duration=143.125µs49702026/11/11 23:38:14 INFO Received complete push request method=POST path=/49712026/11/11 23:38:14 INFO Completed push id=28 objects=7 duration=233.167µs49722026/11/11 23:38:14 INFO Received complete upload request method=POST path=/49732026/11/11 23:38:14 INFO Completed upload id=30 objects=6 duration=194.333µs49742026/11/11 23:38:14 INFO Received complete push request method=POST path=/49752026/11/11 23:38:14 INFO Completed push id=31 objects=5 duration=182.167µs49762026/11/11 23:38:14 INFO Received complete push request method=POST path=/49772026/11/11 23:38:14 INFO Completed push id=32 objects=26 duration=406.167µs49782026/11/11 23:38:14 INFO Received complete upload request method=POST path=/49792026/11/11 23:38:14 INFO Completed upload id=12 objects=9 duration=316.125µs49802026/11/11 23:38:14 INFO Received complete upload request method=POST path=/49812026/11/11 23:38:14 INFO Completed upload id=33 objects=2 duration=163.541µs49822026/11/11 23:38:14 INFO Received complete upload request method=POST path=/49832026/11/11 23:38:14 INFO Completed upload id=34 objects=5 duration=154.125µs49842026/11/11 23:38:14 INFO Received complete upload request method=POST path=/49852026/11/11 23:38:14 INFO Completed upload id=35 objects=11 duration=154.834µs49862026/11/11 23:38:14 INFO Received complete upload request method=POST path=/49872026/11/11 23:38:14 INFO Completed upload id=36 objects=3 duration=461.417µs49882026/11/11 23:38:14 INFO Aborted multipart uploads count=0 kept=049892026/11/11 23:38:14 WARN Force mode enabled - objects will be deleted immediately without grace period49902026/11/11 23:38:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049912026/11/11 23:38:14 INFO Vacuumed table table=pending_closures49922026/11/11 23:38:14 INFO Vacuumed table table=pending_objects49932026/11/11 23:38:14 INFO Vacuumed table table=multipart_uploads49942026/11/11 23:38:14 INFO Vacuumed table table=closures49952026/11/11 23:38:14 INFO Vacuumed table table=objects49962026/11/11 23:38:15 INFO Received complete upload request method=POST path=/49972026/11/11 23:38:15 INFO Completed upload id=18 objects=22 duration=1.414ms49982026/11/11 23:38:15 INFO Received complete upload request method=POST path=/49992026/11/11 23:38:15 INFO Completed upload id=21 objects=8 duration=692.541µs50002026/11/11 23:38:15 INFO Received complete upload request method=POST path=/50012026/11/11 23:38:15 INFO Completed upload id=22 objects=7 duration=733.875µs50022026/11/11 23:38:15 INFO Received complete push request method=POST path=/50032026/11/11 23:38:15 INFO Completed push id=24 objects=5 duration=416.625µs50042026/11/11 23:38:15 INFO Received complete upload request method=POST path=/50052026/11/11 23:38:15 INFO Completed upload id=25 objects=8 duration=280.292µs50062026/11/11 23:38:15 INFO Received complete push request method=POST path=/50072026/11/11 23:38:15 INFO Completed push id=26 objects=4 duration=323.917µs50082026/11/11 23:38:15 INFO Received complete push request method=POST path=/50092026/11/11 23:38:15 INFO Completed push id=30 objects=2 duration=242.709µs50102026/11/11 23:38:15 INFO Received complete push request method=POST path=/50112026/11/11 23:38:15 INFO Completed push id=32 objects=7 duration=417.416µs50122026/11/11 23:38:15 INFO Received complete upload request method=POST path=/50132026/11/11 23:38:15 INFO Completed upload id=33 objects=6 duration=243.5µs50142026/11/11 23:38:15 INFO Received complete upload request method=POST path=/50152026/11/11 23:38:15 INFO Completed upload id=34 objects=3 duration=152.75µs50162026/11/11 23:38:15 INFO Aborted multipart uploads count=0 kept=050172026/11/11 23:38:15 WARN Force mode enabled - objects will be deleted immediately without grace period50182026/11/11 23:38:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050192026/11/11 23:38:15 INFO Vacuumed table table=pending_closures50202026/11/11 23:38:15 INFO Vacuumed table table=pending_objects50212026/11/11 23:38:15 INFO Vacuumed table table=multipart_uploads50222026/11/11 23:38:15 INFO Vacuumed table table=closures50232026/11/11 23:38:15 INFO Vacuumed table table=objects5024--- PASS: FuzzPushGC (0.01s)5025 --- PASS: FuzzPushGC/seed#3 (7.26s)5026 --- PASS: FuzzPushGC/seed#15 (7.27s)5027 --- PASS: FuzzPushGC/seed#0 (8.46s)5028 --- PASS: FuzzPushGC/seed#14 (10.40s)5029 --- PASS: FuzzPushGC/seed#9 (10.50s)5030 --- PASS: FuzzPushGC/seed#1 (11.67s)5031 --- PASS: FuzzPushGC/seed#4 (12.65s)5032 --- PASS: FuzzPushGC/seed#6 (13.28s)5033 --- PASS: FuzzPushGC/seed#7 (13.93s)5034 --- PASS: FuzzPushGC/seed#12 (14.14s)5035 --- PASS: FuzzPushGC/seed#5 (14.19s)5036 --- PASS: FuzzPushGC/seed#2 (14.31s)5037 --- PASS: FuzzPushGC/seed#13 (14.32s)5038 --- PASS: FuzzPushGC/seed#8 (14.37s)5039 --- PASS: FuzzPushGC/seed#11 (14.80s)5040 --- PASS: FuzzPushGC/seed#10 (15.24s)5041PASS5042{"timestamp":"2026-11-11T23:38:15.350983Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:59828","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(11)"}5043{"timestamp":"2026-11-11T23:38:15.351036Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:59833","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(11)"}50442026-11-11 23:38:15.501 UTC [89673] LOG: received smart shutdown request50452026-11-11 23:38:15.502 UTC [89673] LOG: background worker "logical replication launcher" (PID 89683) exited with exit code 150462026-11-11 23:38:16.463 UTC [89678] LOG: checkpoint complete: wrote 3663 buffers (22.4%), wrote 5 SLRU buffers; 0 WAL file(s) added, 0 removed, 33 recycled; write=31.663 s, sync=0.813 s, total=32.565 s; sync files=42365, longest=0.001 s, average=0.001 s; distance=533429 kB, estimate=533429 kB; lsn=0/2785AAD0, redo lsn=0/2207A4D050472026-11-11 23:38:16.465 UTC [89678] LOG: shutting down50482026-11-11 23:38:16.465 UTC [89678] LOG: checkpoint starting: shutdown immediate50492026-11-11 23:38:16.952 UTC [89678] LOG: checkpoint complete: wrote 8577 buffers (52.3%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 5 recycled; write=0.439 s, sync=0.046 s, total=0.488 s; sync files=2832, longest=0.001 s, average=0.001 s; distance=89985 kB, estimate=489084 kB; lsn=0/2785AB48, redo lsn=0/2785AB4850502026-11-11 23:38:16.959 UTC [89673] LOG: database system is shut down5051Running OIDC tests...5052=== RUN TestAudienceForIssuer5053=== PAUSE TestAudienceForIssuer5054=== RUN TestGlobMatch5055=== PAUSE TestGlobMatch5056=== RUN TestValidateToken_ValidToken5057=== PAUSE TestValidateToken_ValidToken5058=== RUN TestValidateToken_WrongAudience5059=== PAUSE TestValidateToken_WrongAudience5060=== RUN TestValidateToken_Expired5061=== PAUSE TestValidateToken_Expired5062=== RUN TestValidateToken_BoundClaimsMismatch5063=== PAUSE TestValidateToken_BoundClaimsMismatch5064=== RUN TestValidateToken_BoundSubjectMismatch5065=== PAUSE TestValidateToken_BoundSubjectMismatch5066=== RUN TestValidateToken_MultipleProviders5067=== PAUSE TestValidateToken_MultipleProviders5068=== RUN TestValidateToken_NoMatchingProvider5069=== PAUSE TestValidateToken_NoMatchingProvider5070=== RUN TestValidateToken_KubernetesServiceAccount5071=== PAUSE TestValidateToken_KubernetesServiceAccount5072=== RUN TestNewValidator_KubernetesRequiresCA5073=== PAUSE TestNewValidator_KubernetesRequiresCA5074=== RUN TestValidateToken_KubernetesIssuerFromOwnToken5075=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken5076=== RUN TestPins_ReservedForMatchingRule5077=== PAUSE TestPins_ReservedForMatchingRule5078=== RUN TestPins_TopLevelShorthand5079=== PAUSE TestPins_TopLevelShorthand5080=== RUN TestPins_ConfigValidation5081=== PAUSE TestPins_ConfigValidation5082=== RUN TestScopes_LegacyProviderDefaultsToWrite5083=== PAUSE TestScopes_LegacyProviderDefaultsToWrite5084=== RUN TestScopes_Rules5085=== PAUSE TestScopes_Rules5086=== RUN TestScopes_ConfigValidation5087=== PAUSE TestScopes_ConfigValidation5088=== CONT TestAudienceForIssuer5089--- PASS: TestAudienceForIssuer (0.00s)5090=== CONT TestValidateToken_WrongAudience5091=== CONT TestNewValidator_KubernetesRequiresCA5092=== CONT TestPins_ConfigValidation5093=== CONT TestValidateToken_Expired5094=== CONT TestValidateToken_ValidToken5095=== CONT TestGlobMatch5096=== RUN TestGlobMatch/foo_foo5097=== CONT TestValidateToken_KubernetesServiceAccount5098=== CONT TestValidateToken_MultipleProviders5099=== PAUSE TestGlobMatch/foo_foo5100=== RUN TestGlobMatch/foo_bar5101=== PAUSE TestGlobMatch/foo_bar5102=== RUN TestGlobMatch/*_5103=== PAUSE TestGlobMatch/*_5104=== RUN TestGlobMatch/*_anything5105=== PAUSE TestGlobMatch/*_anything5106=== RUN TestGlobMatch/foo*_foo5107=== PAUSE TestGlobMatch/foo*_foo5108=== RUN TestGlobMatch/foo*_foobar5109=== PAUSE TestGlobMatch/foo*_foobar5110=== RUN TestGlobMatch/foo*_bar5111=== PAUSE TestGlobMatch/foo*_bar5112=== RUN TestGlobMatch/*bar_bar5113=== PAUSE TestGlobMatch/*bar_bar5114=== RUN TestGlobMatch/*bar_foobar5115=== PAUSE TestGlobMatch/*bar_foobar5116=== RUN TestGlobMatch/*bar_foo5117=== PAUSE TestGlobMatch/*bar_foo5118=== RUN TestGlobMatch/foo*bar_foobar5119=== PAUSE TestGlobMatch/foo*bar_foobar5120=== RUN TestGlobMatch/foo*bar_foo123bar5121=== PAUSE TestGlobMatch/foo*bar_foo123bar5122=== RUN TestGlobMatch/foo*bar_foobarbaz5123=== CONT TestPins_ReservedForMatchingRule5124=== CONT TestValidateToken_KubernetesIssuerFromOwnToken5125=== CONT TestValidateToken_NoMatchingProvider5126=== CONT TestValidateToken_BoundSubjectMismatch5127=== CONT TestScopes_Rules5128=== CONT TestScopes_ConfigValidation5129=== CONT TestScopes_LegacyProviderDefaultsToWrite5130=== CONT TestValidateToken_BoundClaimsMismatch5131--- PASS: TestPins_ConfigValidation (0.00s)5132--- PASS: TestScopes_ConfigValidation (0.00s)5133=== PAUSE TestGlobMatch/foo*bar_foobarbaz5134=== RUN TestGlobMatch/*/*_foo/bar5135=== PAUSE TestGlobMatch/*/*_foo/bar5136=== RUN TestGlobMatch/*/*_foo5137=== PAUSE TestGlobMatch/*/*_foo5138=== RUN TestGlobMatch/refs/heads/*_refs/heads/main5139=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main5140=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.05141=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.05142=== RUN TestGlobMatch/refs/*/main_refs/heads/main5143=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main5144=== RUN TestGlobMatch/fo?_foo5145=== PAUSE TestGlobMatch/fo?_foo5146=== RUN TestGlobMatch/fo?_fo5147=== PAUSE TestGlobMatch/fo?_fo5148=== RUN TestGlobMatch/fo?_fooo5149=== PAUSE TestGlobMatch/fo?_fooo5150=== RUN TestGlobMatch/?oo_foo5151=== PAUSE TestGlobMatch/?oo_foo5152=== RUN TestGlobMatch/?oo_boo5153=== PAUSE TestGlobMatch/?oo_boo5154=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5155=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5156=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5157=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5158=== CONT TestGlobMatch/foo_foo5159=== CONT TestPins_TopLevelShorthand5160=== CONT TestGlobMatch/*/*_foo/bar5161=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5162=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5163=== CONT TestGlobMatch/?oo_boo5164=== CONT TestGlobMatch/?oo_foo5165=== CONT TestGlobMatch/fo?_fooo5166=== CONT TestGlobMatch/fo?_fo5167=== CONT TestGlobMatch/fo?_foo5168=== CONT TestGlobMatch/refs/*/main_refs/heads/main5169=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.05170=== CONT TestGlobMatch/refs/heads/*_refs/heads/main5171=== CONT TestGlobMatch/*/*_foo5172=== CONT TestGlobMatch/*bar_bar5173=== CONT TestGlobMatch/foo*bar_foobarbaz5174=== CONT TestGlobMatch/foo*bar_foo123bar5175=== CONT TestGlobMatch/foo*bar_foobar5176=== CONT TestGlobMatch/*bar_foo5177=== CONT TestGlobMatch/*bar_foobar5178=== CONT TestGlobMatch/foo*_foo5179=== CONT TestGlobMatch/foo*_bar5180=== CONT TestGlobMatch/foo*_foobar5181=== CONT TestGlobMatch/*_5182=== CONT TestGlobMatch/*_anything5183=== CONT TestGlobMatch/foo_bar5184--- PASS: TestGlobMatch (0.01s)5185 --- PASS: TestGlobMatch/foo_foo (0.00s)5186 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)5187 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)5188 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)5189 --- PASS: TestGlobMatch/?oo_boo (0.00s)5190 --- PASS: TestGlobMatch/?oo_foo (0.00s)5191 --- PASS: TestGlobMatch/fo?_fooo (0.00s)5192 --- PASS: TestGlobMatch/fo?_fo (0.00s)5193 --- PASS: TestGlobMatch/fo?_foo (0.00s)5194 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)5195 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)5196 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)5197 --- PASS: TestGlobMatch/*/*_foo (0.00s)5198 --- PASS: TestGlobMatch/*bar_bar (0.00s)5199 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)5200 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)5201 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)5202 --- PASS: TestGlobMatch/*bar_foo (0.00s)5203 --- PASS: TestGlobMatch/*bar_foobar (0.00s)5204 --- PASS: TestGlobMatch/foo*_foo (0.00s)5205 --- PASS: TestGlobMatch/foo*_bar (0.00s)5206 --- PASS: TestGlobMatch/foo*_foobar (0.00s)5207 --- PASS: TestGlobMatch/*_ (0.00s)5208 --- PASS: TestGlobMatch/*_anything (0.00s)5209 --- PASS: TestGlobMatch/foo_bar (0.00s)52102026/11/11 23:38:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:60195/oidc5211--- PASS: TestScopes_Rules (0.04s)52122026/11/11 23:38:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:60197/oidc5213--- PASS: TestValidateToken_Expired (0.06s)52142026/11/11 23:38:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:60199/oidc52152026/11/11 23:38:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:60201/oidc5216--- PASS: TestPins_TopLevelShorthand (0.06s)5217--- PASS: TestValidateToken_BoundSubjectMismatch (0.07s)52182026/11/11 23:38:18 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:6020352192026/11/11 23:38:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:60205/oidc52202026/11/11 23:38:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:60207/oidc5221--- PASS: TestValidateToken_KubernetesServiceAccount (0.08s)52222026/11/11 23:38:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:60209/oidc5223--- PASS: TestValidateToken_BoundClaimsMismatch (0.07s)5224--- PASS: TestScopes_LegacyProviderDefaultsToWrite (0.07s)5225--- PASS: TestPins_ReservedForMatchingRule (0.08s)52262026/11/11 23:38:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:60213/oidc52272026/11/11 23:38:18 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:60215/oidc5228--- PASS: TestValidateToken_ValidToken (0.10s)5229--- PASS: TestValidateToken_WrongAudience (0.10s)52302026/11/11 23:38:18 http: TLS handshake error from 127.0.0.1:60219: remote error: tls: bad certificate5231--- PASS: TestNewValidator_KubernetesRequiresCA (0.11s)52322026/11/11 23:38:18 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC1235233--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (0.12s)52342026/11/11 23:38:18 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:60222/oidc52352026/11/11 23:38:18 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:60223/oidc5236--- PASS: TestValidateToken_MultipleProviders (0.15s)52372026/11/11 23:38:18 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:60217/oidc5238--- PASS: TestValidateToken_NoMatchingProvider (0.16s)5239PASS5240Running hook tests...5241=== RUN TestSendPathsEmpty5242=== PAUSE TestSendPathsEmpty5243=== RUN TestQueueEnqueueAndFetch5244=== PAUSE TestQueueEnqueueAndFetch5245=== RUN TestQueueDeduplication5246=== PAUSE TestQueueDeduplication5247=== RUN TestQueueRemove5248=== PAUSE TestQueueRemove5249=== RUN TestQueueFetchBatchLimit5250=== PAUSE TestQueueFetchBatchLimit5251=== RUN TestQueueRetryMovesToBack5252=== PAUSE TestQueueRetryMovesToBack5253=== RUN TestQueueFetchRemoveLifecycle5254=== PAUSE TestQueueFetchRemoveLifecycle5255=== RUN TestQueueConcurrentWriters5256=== PAUSE TestQueueConcurrentWriters5257=== RUN TestQueueRemoveLargeClosure5258=== PAUSE TestQueueRemoveLargeClosure5259=== RUN TestServerClientIntegration5260--- PASS: TestServerClientIntegration (0.00s)5261=== RUN TestServerQueueError52622026/11/11 23:38:19 ERROR Failed to queue paths error="permission denied" count=15263--- PASS: TestServerQueueError (0.00s)5264=== RUN TestGetListenerSocketActivation5265 server_test.go:147: systemd socket activation does not exist on darwin5266--- SKIP: TestGetListenerSocketActivation (0.00s)5267=== RUN TestServerStalledClientDoesNotBlockShutdown5268=== PAUSE TestServerStalledClientDoesNotBlockShutdown5269=== RUN TestServerBacksOffOnAcceptErrors5270=== PAUSE TestServerBacksOffOnAcceptErrors5271=== RUN TestDrainIsolatesPoisonPath5272=== PAUSE TestDrainIsolatesPoisonPath5273=== RUN TestRunNotBlockedByPoisonHead5274=== PAUSE TestRunNotBlockedByPoisonHead5275=== RUN TestDrainGivesUpWhenServerDown5276=== PAUSE TestDrainGivesUpWhenServerDown5277=== RUN TestFailedPathPrunedByLaterClosure5278=== PAUSE TestFailedPathPrunedByLaterClosure5279=== RUN TestWorkerUploadsAndRemoves5280=== PAUSE TestWorkerUploadsAndRemoves5281=== RUN TestWorkerSkipsGCdPaths5282=== PAUSE TestWorkerSkipsGCdPaths5283=== RUN TestWorkerPrunesClosureDeps5284=== PAUSE TestWorkerPrunesClosureDeps5285=== RUN TestDrainTimeout5286=== PAUSE TestDrainTimeout5287=== RUN TestWorkerRemoveFailureIsNotProgress5288=== PAUSE TestWorkerRemoveFailureIsNotProgress5289=== RUN TestWorkerKeepsPathItCannotStat5290=== PAUSE TestWorkerKeepsPathItCannotStat5291=== CONT TestSendPathsEmpty5292=== CONT TestWorkerUploadsAndRemoves5293--- PASS: TestSendPathsEmpty (0.00s)5294=== CONT TestWorkerKeepsPathItCannotStat5295=== CONT TestWorkerRemoveFailureIsNotProgress5296=== CONT TestDrainTimeout5297=== CONT TestWorkerPrunesClosureDeps5298=== CONT TestQueueRemoveLargeClosure5299=== CONT TestWorkerSkipsGCdPaths5300=== CONT TestQueueConcurrentWriters5301=== CONT TestQueueRemove5302=== CONT TestQueueDeduplication5303=== CONT TestQueueFetchBatchLimit5304=== CONT TestQueueEnqueueAndFetch5305=== CONT TestQueueFetchRemoveLifecycle5306=== CONT TestRunNotBlockedByPoisonHead5307=== CONT TestFailedPathPrunedByLaterClosure5308=== CONT TestQueueRetryMovesToBack5309=== CONT TestDrainGivesUpWhenServerDown5310=== CONT TestServerBacksOffOnAcceptErrors5311=== CONT TestDrainIsolatesPoisonPath53122000/01/01 00:00:00 ERROR Accept failed error="too many open files"53132000/01/01 00:00:00 ERROR Accept failed error="too many open files"53142000/01/01 00:00:00 ERROR Accept failed error="too many open files"53152000/01/01 00:00:00 ERROR Accept failed error="too many open files"53162000/01/01 00:00:00 ERROR Accept failed error="too many open files"53172000/01/01 00:00:00 ERROR Accept failed error="too many open files"53182000/01/01 00:00:00 ERROR Accept failed error="too many open files"53192000/01/01 00:00:00 ERROR Accept failed error="too many open files"5320--- PASS: TestServerBacksOffOnAcceptErrors (0.00s)5321=== CONT TestServerStalledClientDoesNotBlockShutdown53222026/11/11 23:38:19 INFO Uploading batch count=453232026/11/11 23:38:19 ERROR Upload failed error="upload failed" count=453242026/11/11 23:38:19 WARN Cannot stat store path, will retry later path=/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestWorkerKeepsPathItCannotStat4003833715/002/locked/aaa error="stat /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestWorkerKeepsPathItCannotStat4003833715/002/locked/aaa: permission denied"53252026/11/11 23:38:19 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestDrainIsolatesPoisonPath104375357/002/bbb53262026/11/11 23:38:19 INFO Uploading batch count=153272026/11/11 23:38:19 ERROR Upload failed error="upload failed" count=153282026/11/11 23:38:19 INFO Upload queue status pending=25329--- PASS: TestQueueFetchBatchLimit (0.01s)53302026/11/11 23:38:19 INFO Upload queue status pending=253312026/11/11 23:38:19 INFO Uploading batch count=15332--- PASS: TestQueueEnqueueAndFetch (0.01s)5333--- PASS: TestQueueRemove (0.01s)53342026/11/11 23:38:19 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestWorkerSkipsGCdPaths3600592227/002/nonexistent53352026/11/11 23:38:19 INFO Uploading batch count=153362026/11/11 23:38:19 INFO Uploading batch count=253372026/11/11 23:38:19 ERROR Upload failed error="upload failed" count=253382026/11/11 23:38:19 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestDrainGivesUpWhenServerDown4159874820/002/a53392026/11/11 23:38:19 WARN Cannot stat store path, will retry later path=/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestWorkerKeepsPathItCannotStat4003833715/002/locked/aaa error="stat /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestWorkerKeepsPathItCannotStat4003833715/002/locked/aaa: permission denied"53402026/11/11 23:38:19 INFO Upload queue status pending=253412026/11/11 23:38:19 INFO Uploading batch count=153422026/11/11 23:38:19 INFO Uploading batch count=253432026/11/11 23:38:19 INFO Uploading batch count=153442026/11/11 23:38:19 INFO Upload queue status pending=353452000/01/01 00:00:00 INFO Uploading batch count=253462000/01/01 00:00:00 ERROR Upload failed error="context deadline exceeded" count=253472026/11/11 23:38:19 INFO Uploading batch count=153482026/11/11 23:38:19 INFO Uploading batch count=153492026/11/11 23:38:19 ERROR Upload failed error="upload failed" count=153502026/11/11 23:38:19 ERROR Upload failed error="upload failed" count=153512000/01/01 00:00:00 ERROR Drain finished with paths left in queue remaining=45352--- PASS: TestQueueRetryMovesToBack (0.01s)5353--- PASS: TestQueueDeduplication (0.01s)53542026/11/11 23:38:19 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestDrainGivesUpWhenServerDown4159874820/002/b53552026/11/11 23:38:19 WARN Cannot stat store path, will retry later path=/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestWorkerKeepsPathItCannotStat4003833715/002/locked/aaa error="stat /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestWorkerKeepsPathItCannotStat4003833715/002/locked/aaa: permission denied"53562026/11/11 23:38:19 INFO Uploading batch count=253572026/11/11 23:38:19 ERROR Upload failed error="upload failed" count=253582026/11/11 23:38:19 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestDrainGivesUpWhenServerDown4159874820/002/c53592026/11/11 23:38:19 INFO Uploading batch count=153602026/11/11 23:38:19 ERROR Upload failed error="upload failed" count=15361--- PASS: TestQueueFetchRemoveLifecycle (0.01s)53622026/11/11 23:38:19 ERROR Drain finished with paths left in queue remaining=153632026/11/11 23:38:19 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestDrainGivesUpWhenServerDown4159874820/002/d53642026/11/11 23:38:19 INFO Uploading batch count=153652026/11/11 23:38:19 ERROR Upload failed error="upload failed" count=153662026/11/11 23:38:19 ERROR Drain finished with paths left in queue remaining=153672026/11/11 23:38:19 INFO Uploading batch count=253682026/11/11 23:38:19 ERROR Upload failed error="upload failed" count=253692026/11/11 23:38:19 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestWorkerRemoveFailureIsNotProgress4221779632/002/nonexistent53702026/11/11 23:38:19 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestDrainGivesUpWhenServerDown4159874820/002/e53712026/11/11 23:38:19 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=15372--- PASS: TestFailedPathPrunedByLaterClosure (0.01s)53732026/11/11 23:38:19 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestWorkerRemoveFailureIsNotProgress4221779632/002/nonexistent53742026/11/11 23:38:19 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=153752026/11/11 23:38:19 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestWorkerRemoveFailureIsNotProgress4221779632/002/nonexistent53762026/11/11 23:38:19 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=153772026/11/11 23:38:19 ERROR Drain finished with paths left in queue remaining=153782026/11/11 23:38:19 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestDrainGivesUpWhenServerDown4159874820/002/f53792026/11/11 23:38:19 ERROR Drain finished with paths left in queue remaining=105380--- PASS: TestWorkerKeepsPathItCannotStat (0.02s)5381--- PASS: TestDrainTimeout (0.02s)5382--- PASS: TestWorkerRemoveFailureIsNotProgress (0.02s)5383--- PASS: TestDrainIsolatesPoisonPath (0.02s)5384--- PASS: TestDrainGivesUpWhenServerDown (0.02s)5385--- PASS: TestWorkerPrunesClosureDeps (0.03s)5386--- PASS: TestWorkerSkipsGCdPaths (0.03s)5387--- PASS: TestWorkerUploadsAndRemoves (0.03s)5388--- PASS: TestQueueRemoveLargeClosure (0.05s)5389--- PASS: TestQueueConcurrentWriters (0.05s)53902026/11/11 23:38:19 ERROR Failed to decode request error="read unix /private/var/tb/7/scratch/064274e7a4e82b9c/build/hook989001410/test.sock->: i/o timeout"53912026/11/11 23:38:19 ERROR Failed to write response error="write unix /private/var/tb/7/scratch/064274e7a4e82b9c/build/hook989001410/test.sock->: i/o timeout"5392--- PASS: TestServerStalledClientDoesNotBlockShutdown (0.20s)53932026/11/11 23:38:20 INFO Uploading batch count=153942026/11/11 23:38:20 INFO Uploading batch count=153952026/11/11 23:38:20 INFO Uploading batch count=153962026/11/11 23:38:20 ERROR Upload failed error="upload failed" count=153972026/11/11 23:38:20 INFO Uploading batch count=153982026/11/11 23:38:20 ERROR Upload failed error="upload failed" count=153992026/11/11 23:38:20 INFO Uploading batch count=154002026/11/11 23:38:20 ERROR Upload failed error="upload failed" count=154012026/11/11 23:38:20 INFO Uploading batch count=154022026/11/11 23:38:20 ERROR Upload failed error="upload failed" count=154032026/11/11 23:38:20 ERROR Drain finished with paths left in queue remaining=15404--- PASS: TestRunNotBlockedByPoisonHead (1.04s)5405PASS5406Running signing tests...5407=== RUN TestExternalSigner5408=== PAUSE TestExternalSigner5409=== RUN TestExternalSignerWrongPublicKey5410=== PAUSE TestExternalSignerWrongPublicKey5411=== RUN TestExternalSignerErrors5412=== PAUSE TestExternalSignerErrors5413=== RUN TestExternalSignerRestartsAfterCrash5414=== PAUSE TestExternalSignerRestartsAfterCrash5415=== RUN TestExternalSignerProgramErrorKeepsProcess5416=== PAUSE TestExternalSignerProgramErrorKeepsProcess5417=== RUN TestExternalSignerClose5418=== PAUSE TestExternalSignerClose5419=== RUN TestExternalSignerRejectsInvalidPublicKey5420=== PAUSE TestExternalSignerRejectsInvalidPublicKey5421=== RUN TestExternalSignerCancellation5422=== PAUSE TestExternalSignerCancellation5423=== RUN TestGenerateFingerprint5424=== PAUSE TestGenerateFingerprint5425=== RUN TestParseSigningKey5426=== PAUSE TestParseSigningKey5427=== RUN TestSign5428=== PAUSE TestSign5429=== RUN TestSignBatch5430=== PAUSE TestSignBatch5431=== RUN TestNilKeyPublicKey5432=== PAUSE TestNilKeyPublicKey5433=== RUN TestSignErrors5434=== PAUSE TestSignErrors5435=== CONT TestExternalSigner5436=== CONT TestSignBatch5437=== CONT TestSignErrors5438=== CONT TestExternalSignerRejectsInvalidPublicKey5439=== RUN TestExternalSignerRejectsInvalidPublicKey/#005440=== CONT TestExternalSignerProgramErrorKeepsProcess5441=== PAUSE TestExternalSignerRejectsInvalidPublicKey/#005442=== RUN TestExternalSignerRejectsInvalidPublicKey/no-colon5443=== PAUSE TestExternalSignerRejectsInvalidPublicKey/no-colon5444=== CONT TestParseSigningKey5445=== RUN TestParseSigningKey/valid_32-byte_key5446=== RUN TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5447=== PAUSE TestParseSigningKey/valid_32-byte_key5448=== CONT TestExternalSignerClose5449=== CONT TestGenerateFingerprint5450=== RUN TestGenerateFingerprint/basic_with_references5451=== PAUSE TestGenerateFingerprint/basic_with_references5452=== RUN TestGenerateFingerprint/no_references5453=== PAUSE TestGenerateFingerprint/no_references5454=== RUN TestGenerateFingerprint/unsorted_references_get_sorted5455=== PAUSE TestGenerateFingerprint/unsorted_references_get_sorted5456=== RUN TestGenerateFingerprint/invalid_nar_hash_prefix5457=== PAUSE TestGenerateFingerprint/invalid_nar_hash_prefix5458=== RUN TestGenerateFingerprint/invalid_nar_hash_length5459=== PAUSE TestGenerateFingerprint/invalid_nar_hash_length5460=== RUN TestGenerateFingerprint/invalid_store_path_prefix5461=== PAUSE TestGenerateFingerprint/invalid_store_path_prefix5462=== RUN TestGenerateFingerprint/invalid_reference_prefix5463=== PAUSE TestGenerateFingerprint/invalid_reference_prefix5464=== CONT TestGenerateFingerprint/basic_with_references5465=== CONT TestExternalSignerCancellation5466=== CONT TestGenerateFingerprint/invalid_nar_hash_length5467=== CONT TestGenerateFingerprint/invalid_reference_prefix5468=== CONT TestGenerateFingerprint/invalid_store_path_prefix5469=== CONT TestGenerateFingerprint/unsorted_references_get_sorted5470=== CONT TestGenerateFingerprint/invalid_nar_hash_prefix5471=== CONT TestGenerateFingerprint/no_references5472--- PASS: TestGenerateFingerprint (0.00s)5473 --- PASS: TestGenerateFingerprint/basic_with_references (0.00s)5474 --- PASS: TestGenerateFingerprint/invalid_nar_hash_length (0.00s)5475 --- PASS: TestGenerateFingerprint/invalid_reference_prefix (0.00s)5476 --- PASS: TestGenerateFingerprint/invalid_store_path_prefix (0.00s)5477 --- PASS: TestGenerateFingerprint/unsorted_references_get_sorted (0.00s)5478 --- PASS: TestGenerateFingerprint/invalid_nar_hash_prefix (0.00s)5479 --- PASS: TestGenerateFingerprint/no_references (0.00s)5480=== CONT TestNilKeyPublicKey5481--- PASS: TestNilKeyPublicKey (0.00s)5482=== CONT TestSign5483=== CONT TestExternalSignerErrors5484=== RUN TestExternalSignerErrors/exit_on_request5485=== CONT TestExternalSignerWrongPublicKey5486=== PAUSE TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5487=== PAUSE TestExternalSignerErrors/exit_on_request5488=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5489=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5490=== RUN TestParseSigningKey/valid_32-byte_key_with_different_name5491=== RUN TestExternalSignerErrors/invalid_JSON5492=== CONT TestExternalSignerRestartsAfterCrash5493=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5494=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5495=== CONT TestExternalSignerRejectsInvalidPublicKey/#005496=== CONT TestExternalSignerRejectsInvalidPublicKey/no-colon5497=== PAUSE TestParseSigningKey/valid_32-byte_key_with_different_name5498=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5499=== RUN TestParseSigningKey/no_colon5500=== PAUSE TestParseSigningKey/no_colon5501=== RUN TestParseSigningKey/empty_name5502=== PAUSE TestExternalSignerErrors/invalid_JSON5503=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5504=== CONT TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5505=== RUN TestExternalSignerErrors/null_signatures5506--- PASS: TestExternalSignerRejectsInvalidPublicKey (0.00s)5507 --- PASS: TestExternalSignerRejectsInvalidPublicKey/no-colon (0.00s)5508 --- PASS: TestExternalSignerRejectsInvalidPublicKey/#00 (0.00s)5509 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:YQ== (0.00s)5510 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:invalid! (0.00s)5511 --- PASS: TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= (0.00s)5512--- PASS: TestSign (0.00s)5513=== PAUSE TestExternalSignerErrors/null_signatures5514=== PAUSE TestParseSigningKey/empty_name5515=== RUN TestSignErrors/nil_key5516=== PAUSE TestSignErrors/nil_key5517--- PASS: TestSignBatch (0.00s)5518=== RUN TestExternalSignerErrors/missing_signature_name5519=== PAUSE TestExternalSignerErrors/missing_signature_name5520=== RUN TestExternalSignerErrors/wrong_id5521=== RUN TestSignErrors/nil_narinfo5522=== PAUSE TestSignErrors/nil_narinfo5523=== RUN TestSignErrors/invalid_narinfo5524=== RUN TestParseSigningKey/invalid_base645525=== PAUSE TestExternalSignerErrors/wrong_id5526=== RUN TestExternalSignerErrors/missing_signatures5527=== PAUSE TestSignErrors/invalid_narinfo5528=== PAUSE TestParseSigningKey/invalid_base645529=== RUN TestParseSigningKey/wrong_length5530=== PAUSE TestExternalSignerErrors/missing_signatures5531=== RUN TestExternalSignerErrors/exit_at_startup5532=== RUN TestSignErrors/canceled5533=== PAUSE TestExternalSignerErrors/exit_at_startup5534=== PAUSE TestSignErrors/canceled5535=== CONT TestSignErrors/nil_key5536=== CONT TestSignErrors/canceled5537=== PAUSE TestParseSigningKey/wrong_length5538=== CONT TestParseSigningKey/valid_32-byte_key5539=== RUN TestExternalSignerErrors/wrong_shape5540=== CONT TestParseSigningKey/invalid_base645541=== CONT TestParseSigningKey/no_colon5542=== CONT TestParseSigningKey/empty_name5543=== CONT TestParseSigningKey/wrong_length5544=== CONT TestSignErrors/nil_narinfo5545=== PAUSE TestExternalSignerErrors/wrong_shape5546=== CONT TestParseSigningKey/valid_32-byte_key_with_different_name5547=== RUN TestExternalSignerErrors/null5548--- PASS: TestParseSigningKey (0.00s)5549 --- PASS: TestParseSigningKey/no_colon (0.00s)5550 --- PASS: TestParseSigningKey/invalid_base64 (0.00s)5551 --- PASS: TestParseSigningKey/valid_32-byte_key (0.00s)5552 --- PASS: TestParseSigningKey/empty_name (0.00s)5553 --- PASS: TestParseSigningKey/wrong_length (0.00s)5554 --- PASS: TestParseSigningKey/valid_32-byte_key_with_different_name (0.00s)5555=== CONT TestSignErrors/invalid_narinfo5556--- PASS: TestSignErrors (0.00s)5557 --- PASS: TestSignErrors/nil_key (0.00s)5558 --- PASS: TestSignErrors/canceled (0.00s)5559 --- PASS: TestSignErrors/nil_narinfo (0.00s)5560 --- PASS: TestSignErrors/invalid_narinfo (0.00s)5561=== PAUSE TestExternalSignerErrors/null5562=== RUN TestExternalSignerErrors/unrequested_key5563=== PAUSE TestExternalSignerErrors/unrequested_key5564=== RUN TestExternalSignerErrors/wrong_signature_length5565=== PAUSE TestExternalSignerErrors/wrong_signature_length5566=== RUN TestExternalSignerErrors/program_error5567=== PAUSE TestExternalSignerErrors/program_error5568=== RUN TestExternalSignerErrors/wrong_signatures_shape5569=== PAUSE TestExternalSignerErrors/wrong_signatures_shape5570=== RUN TestExternalSignerErrors/too_few_signatures5571=== PAUSE TestExternalSignerErrors/too_few_signatures5572=== RUN TestExternalSignerErrors/wrong_signature_name5573=== PAUSE TestExternalSignerErrors/wrong_signature_name5574=== RUN TestExternalSignerErrors/invalid_signature_base645575=== PAUSE TestExternalSignerErrors/invalid_signature_base645576=== CONT TestExternalSignerErrors/null5577=== CONT TestExternalSignerErrors/wrong_signatures_shape5578=== CONT TestExternalSignerErrors/wrong_id5579=== CONT TestExternalSignerErrors/too_few_signatures5580=== CONT TestExternalSignerErrors/wrong_signature_length5581=== CONT TestExternalSignerErrors/unrequested_key5582=== CONT TestExternalSignerErrors/missing_signatures5583=== CONT TestExternalSignerErrors/exit_at_startup5584=== CONT TestExternalSignerErrors/program_error5585=== CONT TestExternalSignerErrors/null_signatures5586=== CONT TestExternalSignerErrors/invalid_JSON5587=== CONT TestExternalSignerErrors/wrong_shape5588=== CONT TestExternalSignerErrors/wrong_signature_name5589=== CONT TestExternalSignerErrors/invalid_signature_base645590=== CONT TestExternalSignerErrors/exit_on_request5591=== CONT TestExternalSignerErrors/missing_signature_name5592--- PASS: TestExternalSignerClose (0.10s)5593--- PASS: TestExternalSignerProgramErrorKeepsProcess (0.18s)5594--- PASS: TestExternalSignerRestartsAfterCrash (0.27s)5595/private/var/tb/7/scratch/064274e7a4e82b9c/build/TestExternalSignerRestartsAfterCrash4081065996/001/sign program: line 7: /private/var/tb/7/scratch/064274e7a4e82b9c/build/TestExternalSignerRestartsAfterCrash4081065996/001/sign program.starts: No such file or directory5596--- PASS: TestExternalSigner (0.34s)5597--- PASS: TestExternalSignerCancellation (0.43s)5598--- PASS: TestExternalSignerWrongPublicKey (0.48s)5599--- PASS: TestExternalSignerErrors (0.00s)5600 --- PASS: TestExternalSignerErrors/wrong_signature_length (0.55s)5601 --- PASS: TestExternalSignerErrors/wrong_signature_name (0.62s)5602 --- PASS: TestExternalSignerErrors/missing_signatures (0.70s)5603 --- PASS: TestExternalSignerErrors/too_few_signatures (0.78s)5604 --- PASS: TestExternalSignerErrors/null (0.85s)5605 --- PASS: TestExternalSignerErrors/exit_at_startup (0.92s)5606 --- PASS: TestExternalSignerErrors/null_signatures (1.00s)5607 --- PASS: TestExternalSignerErrors/wrong_id (1.07s)5608 --- PASS: TestExternalSignerErrors/unrequested_key (1.15s)5609 --- PASS: TestExternalSignerErrors/wrong_signatures_shape (1.22s)5610 --- PASS: TestExternalSignerErrors/invalid_JSON (1.30s)5611 --- PASS: TestExternalSignerErrors/program_error (1.37s)5612 --- PASS: TestExternalSignerErrors/exit_on_request (1.44s)5613 --- PASS: TestExternalSignerErrors/wrong_shape (1.52s)5614 --- PASS: TestExternalSignerErrors/invalid_signature_base64 (1.59s)5615 --- PASS: TestExternalSignerErrors/missing_signature_name (1.67s)5616PASS5617Running OpenBao signer tests...5618=== RUN TestSign5619=== PAUSE TestSign5620=== RUN TestSignErrors5621=== PAUSE TestSignErrors5622=== RUN TestServe5623=== PAUSE TestServe5624=== RUN TestServeRejectsGarbage5625=== PAUSE TestServeRejectsGarbage5626=== CONT TestSign5627=== CONT TestServeRejectsGarbage5628=== CONT TestSignErrors5629=== CONT TestServe5630=== RUN TestSignErrors/invalid_json5631=== PAUSE TestSignErrors/invalid_json5632=== RUN TestSignErrors/no_prefix5633=== PAUSE TestSignErrors/no_prefix5634=== RUN TestSignErrors/invalid_base645635=== PAUSE TestSignErrors/invalid_base645636=== RUN TestSignErrors/http_error5637--- PASS: TestServeRejectsGarbage (0.00s)5638=== PAUSE TestSignErrors/http_error5639=== CONT TestSignErrors/invalid_json5640=== CONT TestSignErrors/invalid_base645641=== CONT TestSignErrors/http_error5642=== CONT TestSignErrors/no_prefix5643--- PASS: TestServe (0.00s)5644--- PASS: TestSignErrors (0.00s)5645 --- PASS: TestSignErrors/invalid_base64 (0.00s)5646 --- PASS: TestSignErrors/http_error (0.00s)5647 --- PASS: TestSignErrors/invalid_json (0.00s)5648 --- PASS: TestSignErrors/no_prefix (0.00s)5649--- PASS: TestSign (0.00s)5650PASS

succeeded niks3-go-unit-tests checks.aarch64-darwin.go-unit-tests · build #491 · raw

succeeded `niks3-go-unit-tests` `checks.aarch64-darwin.go-unit-tests` · build #491 · raw