niks3-go-unit-tests log

succeeded `niks3-go-unit-tests` `checks.aarch64-linux.go-unit-tests` · build #492 · raw

1tribuchet: building on eliza2Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestUploadBuildLog_FileBodyReplayedOnRetry6=== PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry7=== RUN TestRegisterUploadedObjectReusesConnections8=== PAUSE TestRegisterUploadedObjectReusesConnections9=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica10=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica11=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun12=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun13=== RUN TestCaseHackSuffix14=== PAUSE TestCaseHackSuffix15=== RUN TestFilterOversizedClosures16=== PAUSE TestFilterOversizedClosures17=== RUN TestUploadMultipart_PartsInParallel18=== PAUSE TestUploadMultipart_PartsInParallel19=== RUN TestUploadMultipart_ReturnsAfterDroppedBody20=== PAUSE TestUploadMultipart_ReturnsAfterDroppedBody21=== RUN TestPartSizeForNAR22=== PAUSE TestPartSizeForNAR23=== RUN TestDumpPathCaseHackMatchesNix24--- PASS: TestDumpPathCaseHackMatchesNix (3.41s)25=== RUN TestDumpPathCaseHackCollision26--- PASS: TestDumpPathCaseHackCollision (0.00s)27=== RUN TestDumpPathMatchesNix28=== PAUSE TestDumpPathMatchesNix29=== RUN TestDumpPathSingleFile30=== PAUSE TestDumpPathSingleFile31=== RUN TestDumpPathWriterError32=== PAUSE TestDumpPathWriterError33=== RUN TestEncodeNixBase3234=== PAUSE TestEncodeNixBase3235=== RUN TestEncodeNixBase32WithRealHash36=== PAUSE TestEncodeNixBase32WithRealHash37=== RUN TestConvertHashToNix3238=== PAUSE TestConvertHashToNix3239=== RUN TestGetStorePathHash40=== PAUSE TestGetStorePathHash41=== RUN TestPathInfoHashCompatibility42=== PAUSE TestPathInfoHashCompatibility43=== RUN TestParsePathInfoJSON44=== PAUSE TestParsePathInfoJSON45=== RUN TestParsePathInfoJSONMultiplePaths46=== PAUSE TestParsePathInfoJSONMultiplePaths47=== RUN TestPathInfoCACompatibility48=== PAUSE TestPathInfoCACompatibility49=== RUN TestRateLimiterFeedback50=== PAUSE TestRateLimiterFeedback51=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess52=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess53=== RUN TestRegisterUploadedObject_BoundedAgainstSilentServer54=== PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer55=== RUN TestRegisterUploadedObject_OutlivesPush56=== PAUSE TestRegisterUploadedObject_OutlivesPush57=== RUN TestResolveStorePath58=== PAUSE TestResolveStorePath59=== RUN TestDoWithRetry_BodyReplayedViaGetBody60=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody61=== RUN TestRetryTimeoutOutlastsMaxRetries62=== PAUSE TestRetryTimeoutOutlastsMaxRetries63=== RUN TestDoWithRetry_FinalResponseBodyReadable64=== PAUSE TestDoWithRetry_FinalResponseBodyReadable65=== RUN TestShellSplit66=== PAUSE TestShellSplit67=== RUN TestShellSplitErrors68=== PAUSE TestShellSplitErrors69=== RUN TestStreamPushReportsEveryPath70=== PAUSE TestStreamPushReportsEveryPath71=== RUN TestStreamPushBatchesUnderLoad72=== PAUSE TestStreamPushBatchesUnderLoad73=== RUN TestStreamPushIsolatesFailures74=== PAUSE TestStreamPushIsolatesFailures75=== RUN TestStreamPushGivesUpOnDeadServer76=== PAUSE TestStreamPushGivesUpOnDeadServer77=== RUN TestStreamPushRequestLine78=== PAUSE TestStreamPushRequestLine79=== RUN TestStreamPushReportsSignatures80=== PAUSE TestStreamPushReportsSignatures81=== RUN TestStreamPushReportsSkippedPaths82=== PAUSE TestStreamPushReportsSkippedPaths83=== RUN TestClientSignaturesByStorePath84=== PAUSE TestClientSignaturesByStorePath85=== RUN TestStreamPushAnswersEveryLineWhenCanceled86=== PAUSE TestStreamPushAnswersEveryLineWhenCanceled87=== RUN TestStreamPushStopsWhenCanceled88=== PAUSE TestStreamPushStopsWhenCanceled89=== RUN TestSetClientTLS90=== PAUSE TestSetClientTLS91=== RUN TestSetClientTLSDoesNotMutateDefaultTransport92=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport93=== RUN TestSetClientTLSErrors94=== PAUSE TestSetClientTLSErrors95=== RUN TestStaticToken96=== PAUSE TestStaticToken97=== RUN TestFileTokenReadsAndCaches98=== PAUSE TestFileTokenReadsAndCaches99=== RUN TestFileTokenMissing100=== PAUSE TestFileTokenMissing101=== RUN TestFileTokenEmpty102=== PAUSE TestFileTokenEmpty103=== RUN TestScriptTokenNoExpiryRerunsEveryCall104=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall105=== RUN TestScriptTokenCachesUntilRefresh106=== PAUSE TestScriptTokenCachesUntilRefresh107=== RUN TestScriptTokenEmptyToken108=== PAUSE TestScriptTokenEmptyToken109=== RUN TestScriptTokenBadJSON110=== PAUSE TestScriptTokenBadJSON111=== RUN TestScriptTokenScriptFails112=== PAUSE TestScriptTokenScriptFails113=== RUN TestScriptTokenEmptyCommand114=== PAUSE TestScriptTokenEmptyCommand115=== CONT TestDoServerRequestAttachesToken116=== CONT TestScriptTokenEmptyCommand117=== CONT TestDoWithRetry_BodyReplayedViaGetBody118=== CONT TestScriptTokenScriptFails119=== CONT TestStreamPushReportsSkippedPaths120=== CONT TestClientSignaturesByStorePath121=== CONT TestRegisterUploadedObject_BoundedAgainstSilentServer122=== CONT TestStreamPushRequestLine123=== CONT TestStreamPushReportsSignatures124=== CONT TestStreamPushIsolatesFailures125=== CONT TestStreamPushGivesUpOnDeadServer126=== CONT TestStreamPushReportsEveryPath127=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess128=== CONT TestShellSplitErrors129=== CONT TestRateLimiterFeedback130=== RUN TestRateLimiterFeedback/429_enables_limiter131=== PAUSE TestRateLimiterFeedback/429_enables_limiter132=== RUN TestRateLimiterFeedback/503_enables_limiter133=== PAUSE TestRateLimiterFeedback/503_enables_limiter134=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter135=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter136=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter137=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter1382026/11/11 23:56:48 ERROR Upload failed error="bad path" count=3139=== CONT TestShellSplit140=== CONT TestParsePathInfoJSONMultiplePaths141=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths142=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths143=== CONT TestParsePathInfoJSON144=== RUN TestParsePathInfoJSON/Nix_format145=== PAUSE TestParsePathInfoJSON/Nix_format146=== RUN TestParsePathInfoJSON/Lix_format147=== PAUSE TestParsePathInfoJSON/Lix_format1482026/11/11 23:56:48 WARN Rate limiter enabled after throttle name=server-test rate=5149=== RUN TestParsePathInfoJSON/empty_input150=== PAUSE TestParsePathInfoJSON/empty_input151=== RUN TestParsePathInfoJSON/whitespace_only152=== PAUSE TestParsePathInfoJSON/whitespace_only153=== RUN TestParsePathInfoJSON/invalid_JSON154=== PAUSE TestParsePathInfoJSON/invalid_JSON1552026/11/11 23:56:48 ERROR Upload failed error=boom count=1156=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths157=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths158=== CONT TestPathInfoHashCompatibility159=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)160=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)161=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon162=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon163=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI164=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI165=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512166=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512167=== CONT TestGetStorePathHash168=== RUN TestGetStorePathHash/valid_store_path169=== PAUSE TestGetStorePathHash/valid_store_path170=== RUN TestGetStorePathHash/basename_without_hyphen_should_error171=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error172=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error173=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error174=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error175=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error1762026/11/11 23:56:48 ERROR Upload failed error="connection refused" count=201772026/11/11 23:56:48 ERROR Server seems unavailable, giving up on batch untried=17178=== CONT TestDumpPathWriterError179=== CONT TestRetryTimeoutOutlastsMaxRetries180=== CONT TestUploadMultipart_PartsInParallel1812026/11/11 23:56:52 ERROR Upload failed error=boom count=1182=== CONT TestEncodeNixBase32WithRealHash183=== CONT TestConvertHashToNix32184=== CONT TestScriptTokenBadJSON185=== CONT TestDumpPathMatchesNix186=== RUN TestConvertHashToNix32/SRI_format_to_Nix32187=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32188=== RUN TestConvertHashToNix32/already_Nix32_format189=== PAUSE TestConvertHashToNix32/already_Nix32_format190=== RUN TestConvertHashToNix32/invalid_format191=== PAUSE TestConvertHashToNix32/invalid_format192=== CONT TestPartSizeForNAR193=== RUN TestPartSizeForNAR/zero_stays_at_minimum194=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum195=== RUN TestPartSizeForNAR/small_stays_at_minimum196=== PAUSE TestPartSizeForNAR/small_stays_at_minimum197=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum198=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum199=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts200=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts201=== RUN TestPartSizeForNAR/1_TiB202=== PAUSE TestPartSizeForNAR/1_TiB203=== RUN TestPartSizeForNAR/5_TiB_S3_max_object204=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object205=== RUN TestPartSizeForNAR/capped_at_5_GiB206=== PAUSE TestPartSizeForNAR/capped_at_5_GiB207=== CONT TestUploadMultipart_ReturnsAfterDroppedBody208=== CONT TestScriptTokenEmptyToken209=== CONT TestScriptTokenCachesUntilRefresh210=== CONT TestFileTokenMissing211=== CONT TestScriptTokenNoExpiryRerunsEveryCall212=== CONT TestFileTokenEmpty213=== CONT TestEncodeNixBase32214=== RUN TestEncodeNixBase32/test_string_hash215=== PAUSE TestEncodeNixBase32/test_string_hash216=== RUN TestEncodeNixBase32/empty_input217=== PAUSE TestEncodeNixBase32/empty_input218--- PASS: TestScriptTokenEmptyCommand (0.00s)219--- PASS: TestClientSignaturesByStorePath (0.00s)220--- PASS: TestStreamPushReportsSkippedPaths (0.00s)221--- PASS: TestShellSplitErrors (0.00s)222--- PASS: TestShellSplit (0.00s)223--- PASS: TestStreamPushIsolatesFailures (0.00s)224--- PASS: TestStreamPushReportsEveryPath (0.00s)225--- PASS: TestStreamPushGivesUpOnDeadServer (0.00s)226--- PASS: TestStreamPushReportsSignatures (4.39s)227--- PASS: TestEncodeNixBase32WithRealHash (0.00s)228--- PASS: TestScriptTokenScriptFails (4.39s)229--- PASS: TestFileTokenMissing (0.00s)230=== CONT TestFilterOversizedClosures231=== RUN TestFilterOversizedClosures/no_limit_keeps_everything232=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything233=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped234=== CONT TestResolveStorePath235--- PASS: TestResolveStorePath (0.00s)236=== CONT TestSetClientTLSErrors237--- PASS: TestScriptTokenEmptyToken (0.00s)238=== CONT TestFileTokenReadsAndCaches239=== CONT TestDumpPathSingleFile240--- PASS: TestFileTokenEmpty (0.00s)241=== CONT TestStaticToken242--- PASS: TestStaticToken (0.00s)243=== CONT TestSetClientTLS244=== CONT TestStreamPushBatchesUnderLoad245=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=false246=== CONT TestPathInfoCACompatibility247=== RUN TestPathInfoCACompatibility/null_ca_field248=== PAUSE TestPathInfoCACompatibility/null_ca_field249=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=false250--- PASS: TestFileTokenReadsAndCaches (0.00s)251=== RUN TestPathInfoCACompatibility/old_string_format_-_text252=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text253=== CONT TestDoWithRetry_FinalResponseBodyReadable254=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive255=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive256=== RUN TestPathInfoCACompatibility/new_structured_format_-_text257=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text258=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method259=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method260=== CONT TestRegisterUploadedObjectReusesConnections261=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun262=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=true263=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=true264=== CONT TestCaseHackSuffix265--- PASS: TestScriptTokenBadJSON (0.01s)266=== CONT TestUploadBuildLog_FileBodyReplayedOnRetry267=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped268=== RUN TestFilterOversizedClosures/all_closures_skipped269=== PAUSE TestFilterOversizedClosures/all_closures_skipped270=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica271=== CONT TestRegisterUploadedObject_OutlivesPush272=== CONT TestSetClientTLSDoesNotMutateDefaultTransport2732026/11/11 23:56:56 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:40983/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:40983/api/uploads/complete2742026/11/11 23:56:56 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:40983/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:40983/api/uploads/complete2752026/11/11 23:56:56 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"2762026/11/11 23:56:56 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:40983/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:40983/api/uploads/complete2772026/11/11 23:56:56 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"2782026/11/11 23:56:56 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"2792026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:442392802026/11/11 23:56:56 WARN Rate limiter enabled after throttle name=server-test rate=52812026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:401432822026/11/11 23:56:56 WARN Rate limiter enabled after throttle name=server-test rate=52832026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:40735284--- PASS: TestDumpPathSingleFile (4.04s)285=== CONT TestStreamPushAnswersEveryLineWhenCanceled286--- PASS: TestRegisterUploadedObject_BoundedAgainstSilentServer (8.43s)2872026/11/11 23:56:56 WARN Rate limiter backed off name=server-test rate=5288=== CONT TestRateLimiterFeedback/429_enables_limiter2892026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:407352902026/11/11 23:56:56 WARN Rate limiter backed off name=server-test rate=5291--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (8.43s)292--- PASS: TestDoWithRetry_FinalResponseBodyReadable (4.04s)293=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter294=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter295=== CONT TestStreamPushStopsWhenCanceled2962026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:44239297--- PASS: TestStreamPushStopsWhenCanceled (0.17s)298=== CONT TestRateLimiterFeedback/503_enables_limiter2992000/01/01 00:00:00 ERROR Upload failed error="context canceled" count=2300--- PASS: TestRegisterUploadedObject_OutlivesPush (4.21s)3012026/11/11 23:56:56 WARN Rate limiter enabled after throttle name=server-test rate=5302--- PASS: TestUploadMultipart_ReturnsAfterDroppedBody (4.22s)3032026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:36511304=== CONT TestParsePathInfoJSON/Nix_format305=== CONT TestGetStorePathHash/valid_store_path306=== CONT TestParsePathInfoJSON/invalid_JSON3072026/11/11 23:56:56 WARN Rate limiter backed off name=server-test rate=5308=== CONT TestParsePathInfoJSON/whitespace_only3092026/11/11 23:56:56 WARN Rate limiter enabled after throttle name=server-test rate=5310--- PASS: TestStreamPushAnswersEveryLineWhenCanceled (0.18s)3112026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:37301312=== CONT TestParsePathInfoJSON/empty_input313=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths314=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error3152026/11/11 23:56:56 INFO Garbage collection started316=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths3172026/11/11 23:56:56 INFO Garbage collection started3182026/11/11 23:56:56 WARN Rate limiter backed off name=server-test rate=5319=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error320=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512321--- PASS: TestParsePathInfoJSONMultiplePaths (0.00s)322 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)323 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)324=== CONT TestParsePathInfoJSON/Lix_format325=== CONT TestConvertHashToNix32/SRI_format_to_Nix32326=== CONT TestConvertHashToNix32/invalid_format327=== CONT TestConvertHashToNix32/already_Nix32_format328=== CONT TestPartSizeForNAR/capped_at_5_GiB329=== CONT TestPartSizeForNAR/5_TiB_S3_max_object330=== CONT TestPartSizeForNAR/1_TiB331=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts332=== CONT TestEncodeNixBase32/test_string_hash333=== CONT TestEncodeNixBase32/empty_input334=== CONT TestPathInfoCACompatibility/null_ca_field335=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=false336--- PASS: TestParsePathInfoJSON (0.00s)337 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)338 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)339 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)340 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)341 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)342--- PASS: TestRateLimiterFeedback (0.00s)343 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.17s)344 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.18s)345 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.18s)346 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s)347=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum348=== CONT TestFilterOversizedClosures/no_limit_keeps_everything349--- PASS: TestCaseHackSuffix (4.21s)350--- PASS: TestConvertHashToNix32 (0.00s)351 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)352 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)353 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)354=== CONT TestGetStorePathHash/basename_without_hyphen_should_error355--- PASS: TestGetStorePathHash (0.00s)356 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)357 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)358 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)359 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)3602026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:44239361=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=true362=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI363=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive364=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method365=== CONT TestFilterOversizedClosures/all_closures_skipped3662026/11/11 23:56:56 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=50367=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped3682026/11/11 23:56:56 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000369=== CONT TestPathInfoCACompatibility/old_string_format_-_text370=== CONT TestPartSizeForNAR/zero_stays_at_minimum371=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon372=== CONT TestPartSizeForNAR/small_stays_at_minimum373=== RUN TestSetClientTLSErrors/missing_cert_file374=== PAUSE TestSetClientTLSErrors/missing_cert_file375=== RUN TestSetClientTLSErrors/missing_key_file376=== PAUSE TestSetClientTLSErrors/missing_key_file377=== RUN TestSetClientTLSErrors/missing_ca_file378=== PAUSE TestSetClientTLSErrors/missing_ca_file379=== RUN TestSetClientTLSErrors/invalid_ca_file3802026/11/11 23:56:56 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=0381=== PAUSE TestSetClientTLSErrors/invalid_ca_file382=== CONT TestSetClientTLSErrors/missing_cert_file383=== CONT TestSetClientTLSErrors/invalid_ca_file384=== CONT TestSetClientTLSErrors/missing_key_file3852026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:442393862026/11/11 23:56:56 WARN Rate limiter enabled after throttle name=s3-test rate=53872026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:35299388--- PASS: TestEncodeNixBase32 (0.00s)389 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)390 --- PASS: TestEncodeNixBase32/empty_input (0.00s)391=== CONT TestPathInfoCACompatibility/new_structured_format_-_text392=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)393--- PASS: TestDoServerRequestAttachesToken (8.61s)394=== CONT TestSetClientTLSErrors/missing_ca_file395--- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (4.21s)396--- PASS: TestStreamPushBatchesUnderLoad (0.00s)397 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=false (0.00s)398 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=true (0.00s)399--- PASS: TestFilterOversizedClosures (0.00s)400 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)401 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)402 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)403--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (4.21s)404--- PASS: TestPathInfoCACompatibility (0.00s)405 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)406 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)407 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)408 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)409 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)410--- PASS: TestPathInfoHashCompatibility (0.00s)411 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)412 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)413 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)414 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)415--- PASS: TestPartSizeForNAR (0.00s)416 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)417 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)418 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)419 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)420 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)421 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)422 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)423--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (4.21s)4242026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:44239425=== RUN TestSetClientTLS/rejects_connection_without_client_cert426=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert427=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA428=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA429=== RUN TestSetClientTLS/preserves_debug_logging_transport430=== PAUSE TestSetClientTLS/preserves_debug_logging_transport431=== CONT TestSetClientTLS/rejects_connection_without_client_cert432=== CONT TestSetClientTLS/preserves_debug_logging_transport433=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA4342026/11/11 23:56:56 WARN Garbage collection finished on another replica. Its statistics are not available here435--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (4.22s)436--- PASS: TestSetClientTLSErrors (4.22s)437 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)438 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)439 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)440 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)4412026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:44239442--- PASS: TestScriptTokenCachesUntilRefresh (4.22s)4432026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:442394442026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:44239445--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (4.23s)4462026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:442394472026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:442394482026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:442394492026/11/11 23:56:56 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:44239450--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (9.61s)4512026/11/11 23:56:57 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:442394522026/11/11 23:56:57 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:44239453--- PASS: TestRegisterUploadedObjectReusesConnections (5.22s)4542026/11/11 23:56:57 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:442394552026/11/11 23:56:57 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:442394562026/11/11 23:56:57 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:44239457--- PASS: TestStreamPushRequestLine (9.62s)4582026/11/11 23:56:57 http: TLS handshake error from 127.0.0.1:54914: remote error: tls: bad certificate4592026/11/11 23:56:57 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:44239460--- PASS: TestSetClientTLS (4.22s)461 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.00s)462 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.00s)463 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (1.01s)4642026/11/11 23:56:57 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:44239465--- PASS: TestRetryTimeoutOutlastsMaxRetries (5.24s)466--- PASS: TestDumpPathWriterError (9.63s)467--- PASS: TestUploadMultipart_PartsInParallel (5.84s)468--- PASS: TestDumpPathMatchesNix (6.05s)469PASS470Running server tests...471The files belonging to this database system will be owned by user "nixbld".472This user must also own the server process.473474The database cluster will be initialized with locale "C".475The default database encoding has accordingly been set to "SQL_ASCII".476The default text search configuration will be set to "english".477478Data page checksums are enabled.479480creating directory /build/postgres3621723541/data ... ok481creating subdirectories ... ok482selecting dynamic shared memory implementation ... posix483selecting default "max_connections" ... 100484selecting default "shared_buffers" ... 128MB485selecting default time zone ... UTC486creating configuration files ... ok487running bootstrap script ... ok488performing post-bootstrap initialization ... ok489syncing data to disk ... ok490491initdb: warning: enabling "trust" authentication for local connections492initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.493494Success. You can now start the database server using:495496 pg_ctl -D /build/postgres3621723541/data -l logfile start497498/build/postgres3621723541:5432 - no response499/build/postgres3621723541:5432 - no response500/build/postgres3621723541:5432 - no response501/build/postgres3621723541:5432 - no response502/build/postgres3621723541:5432 - no response503/build/postgres3621723541:5432 - no response504/build/postgres3621723541:5432 - no response505/build/postgres3621723541:5432 - no response506/build/postgres3621723541:5432 - no response507/build/postgres3621723541:5432 - no response508/build/postgres3621723541:5432 - no response509/build/postgres3621723541:5432 - no response510/build/postgres3621723541:5432 - no response511/build/postgres3621723541:5432 - no response512/build/postgres3621723541:5432 - no response513/build/postgres3621723541:5432 - no response514/build/postgres3621723541:5432 - no response515/build/postgres3621723541:5432 - no response516/build/postgres3621723541:5432 - no response517/build/postgres3621723541:5432 - no response518/build/postgres3621723541:5432 - no response519/build/postgres3621723541:5432 - no response520/build/postgres3621723541:5432 - no response521/build/postgres3621723541:5432 - no response522/build/postgres3621723541:5432 - no response523/build/postgres3621723541:5432 - no response524/build/postgres3621723541:5432 - no response525/build/postgres3621723541:5432 - no response526/build/postgres3621723541:5432 - no response527/build/postgres3621723541:5432 - no response528/build/postgres3621723541:5432 - no response529/build/postgres3621723541:5432 - no response530/build/postgres3621723541:5432 - no response5312026-11-11 23:57:15.364 UTC [130] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit5322026-11-11 23:57:15.365 UTC [130] LOG: listening on Unix socket "/build/postgres3621723541/.s.PGSQL.5432"5332026-11-11 23:57:15.374 UTC [169] LOG: database system was shut down at 2026-11-11 23:57:11 UTC5342026-11-11 23:57:15.378 UTC [130] LOG: database system is ready to accept connections535/build/postgres3621723541:5432 - accepting connections536{"timestamp":"2026-11-11T23:57:47.521892785Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"127bd1c4-ae2f-4b21-a3cc-ba8ee720ded7","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":23,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(199)"}537{"timestamp":"2026-11-11T23:57:47.639157244Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"9ac74d6d-6a13-47c9-a68e-18e5d3fcd055","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(199)"}538{"timestamp":"2026-11-11T23:57:47.743399038Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"3fa2c801-6f53-4978-b7c1-6f938331c752","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(209)"}539=== RUN TestAbortSucceeded540=== PAUSE TestAbortSucceeded541=== RUN TestService_AuthMiddleware542=== PAUSE TestService_AuthMiddleware543=== RUN TestService_AuthMiddleware_MTLSProxyHeader544=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader545=== RUN TestService_AuthMiddleware_MTLSBoundSubjects546=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects547=== RUN TestService_ReadAuthMiddleware548=== PAUSE TestService_ReadAuthMiddleware549=== RUN TestService_AuthMiddleware_OIDC550=== PAUSE TestService_AuthMiddleware_OIDC551=== RUN TestService_RequireScope_OIDC552=== PAUSE TestService_RequireScope_OIDC553=== RUN TestService_ReadScope_PublicByDefault554=== PAUSE TestService_ReadScope_PublicByDefault555=== RUN TestCacheConfigHandler556=== PAUSE TestCacheConfigHandler557=== RUN TestCacheStatsHandler558=== PAUSE TestCacheStatsHandler559=== RUN TestClientCADerivations560=== PAUSE TestClientCADerivations561=== RUN TestClientErrorHandling562=== PAUSE TestClientErrorHandling563=== RUN TestClientIntegration564=== PAUSE TestClientIntegration565=== RUN TestClientMultipleUploads566=== PAUSE TestClientMultipleUploads567=== RUN TestClientWithDependencies568=== PAUSE TestClientWithDependencies569=== RUN TestClientSharedPathCommittedMidPush570=== PAUSE TestClientSharedPathCommittedMidPush571=== RUN TestPinProtectsFromGC572=== PAUSE TestPinProtectsFromGC573=== RUN TestClientPushesUseOnePush574=== PAUSE TestClientPushesUseOnePush575=== RUN TestClientFallsBackToClosures576=== PAUSE TestClientFallsBackToClosures577=== RUN TestDeleteClosuresRacingPins578=== PAUSE TestDeleteClosuresRacingPins579=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock580=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock581=== RUN TestResolveDBConnectionString582=== PAUSE TestResolveDBConnectionString583=== RUN TestConnectWaitsForAPeerMigration584=== PAUSE TestConnectWaitsForAPeerMigration585=== RUN TestLeadElectsOneAndHandsOver586=== PAUSE TestLeadElectsOneAndHandsOver587=== RUN TestLeadIncumbentWinsAfterRestart5882026/11/11 23:57:51 INFO lead: acquired remote=192.0.2.1:1234 generation=15892026/11/11 23:57:51 INFO lead: released remote=192.0.2.1:1234 generation=15902026/11/11 23:57:51 INFO lead: acquired remote=192.0.2.1:1234 generation=25912026/11/11 23:57:52 INFO lead: released remote=192.0.2.1:1234 generation=2592--- PASS: TestLeadIncumbentWinsAfterRestart (5.10s)593=== RUN TestLeadUnannouncedStandbyYieldsToIncumbent594=== PAUSE TestLeadUnannouncedStandbyYieldsToIncumbent595=== RUN TestLeadIncumbentAnnouncesAtOnce596=== PAUSE TestLeadIncumbentAnnouncesAtOnce597=== RUN TestLeadGenerationGrowsWithEveryGrant598=== PAUSE TestLeadGenerationGrowsWithEveryGrant599=== RUN TestLeadEndsOnShutdown600=== PAUSE TestLeadEndsOnShutdown601=== RUN TestLeadEndsWhenItsConnectionHangs6022026/11/11 23:57:54 INFO lead: acquired remote=192.0.2.1:1234 generation=16032026-11-11 23:57:55.426 UTC [415] FATAL: terminating connection due to administrator command6042026/11/11 23:57:55 INFO lead: acquired remote=192.0.2.1:1234 generation=26052026/11/11 23:57:56 WARN lead: lock connection lost error="timeout: context deadline exceeded"6062026/11/11 23:57:56 INFO lead: released remote=192.0.2.1:1234 generation=16072026/11/11 23:57:57 INFO lead: released remote=192.0.2.1:1234 generation=2608--- PASS: TestLeadEndsWhenItsConnectionHangs (4.05s)609=== RUN TestGCAdvisoryLockBlocksConcurrentRun610=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun611=== RUN TestGCBugBareHashReferences612=== PAUSE TestGCBugBareHashReferences613=== RUN TestGCMetrics614=== PAUSE TestGCMetrics615=== RUN TestPushDedupSurvivesConcurrentGC616=== PAUSE TestPushDedupSurvivesConcurrentGC617=== RUN TestDeduplicatedObjectsRecordedAsPending618=== PAUSE TestDeduplicatedObjectsRecordedAsPending619=== RUN TestGCSweepSkipsPendingObjects620=== PAUSE TestGCSweepSkipsPendingObjects621=== RUN TestTombstonedObjectOfferedWithoutWaiting622=== PAUSE TestTombstonedObjectOfferedWithoutWaiting623=== RUN TestGCSweepDeliversEachKeyOnce624=== PAUSE TestGCSweepDeliversEachKeyOnce625=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection626=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection627=== RUN TestForceGCDuringPushOffersSweptObject628=== PAUSE TestForceGCDuringPushOffersSweptObject629=== RUN TestFailedS3DeleteKeepsTombstone630=== PAUSE TestFailedS3DeleteKeepsTombstone631=== RUN TestSweepSkipsRowsLockedByPush632=== PAUSE TestSweepSkipsRowsLockedByPush633=== RUN TestRegisterCompletedObjectNeedsPendingRow634=== PAUSE TestRegisterCompletedObjectNeedsPendingRow635=== RUN TestGetExistingObjectsLocksRows636=== PAUSE TestGetExistingObjectsLocksRows637=== RUN TestSweepRechecksPendingAfterLocking638=== PAUSE TestSweepRechecksPendingAfterLocking639=== RUN TestPushWaitsForSweepHoldingRow640=== PAUSE TestPushWaitsForSweepHoldingRow641=== RUN TestCommitRacingPendingCleanupKeepsObjects642=== PAUSE TestCommitRacingPendingCleanupKeepsObjects643=== RUN TestGCEndsOnShutdown644=== PAUSE TestGCEndsOnShutdown645=== RUN TestGCTaskStore_StartNew646=== PAUSE TestGCTaskStore_StartNew647=== RUN TestGCTaskStore_DeduplicateSameParams648=== PAUSE TestGCTaskStore_DeduplicateSameParams649=== RUN TestGCTaskStore_ConflictDifferentParams650=== PAUSE TestGCTaskStore_ConflictDifferentParams651=== RUN TestGCTaskStore_GetEmpty652=== PAUSE TestGCTaskStore_GetEmpty653=== RUN TestGCTaskStore_GetReturnsLatest654=== PAUSE TestGCTaskStore_GetReturnsLatest655=== RUN TestGCTaskStore_CompletedAllowsNewTask656=== PAUSE TestGCTaskStore_CompletedAllowsNewTask657=== RUN TestGCTaskStore_PhaseUpdates658=== PAUSE TestGCTaskStore_PhaseUpdates659=== RUN TestGCTaskStore_Fail660=== PAUSE TestGCTaskStore_Fail661=== RUN TestGracefulShutdownDrainsInflight662=== PAUSE TestGracefulShutdownDrainsInflight663=== RUN TestService_healthCheckHandler664=== PAUSE TestService_healthCheckHandler665=== RUN TestService_readinessHandler666=== PAUSE TestService_readinessHandler667=== RUN TestGenerateLandingPage668=== PAUSE TestGenerateLandingPage669=== RUN TestEnsureAbortUploadsRule670=== PAUSE TestEnsureAbortUploadsRule671=== RUN TestCacheConfigHandlerMaxNarSize672=== PAUSE TestCacheConfigHandlerMaxNarSize673=== RUN TestCreatePendingClosureRejectsOversizedNAR674=== PAUSE TestCreatePendingClosureRejectsOversizedNAR675=== RUN TestNARDeduplicationMetadataUploadBug676=== PAUSE TestNARDeduplicationMetadataUploadBug677=== RUN TestMetricsInventory678=== PAUSE TestMetricsInventory679=== RUN TestMetricsCommit680=== PAUSE TestMetricsCommit681=== RUN TestService_NativeMTLS682=== PAUSE TestService_NativeMTLS683=== RUN TestServerTLSConfig684=== PAUSE TestServerTLSConfig685=== RUN TestMultipartCleanup686=== PAUSE TestMultipartCleanup687=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded688=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded689=== RUN TestPendingCleanupUsesOneCutoff690=== PAUSE TestPendingCleanupUsesOneCutoff691=== RUN TestObjectStatsTrigger692=== PAUSE TestObjectStatsTrigger693=== RUN TestObjectStatsUpdatedOncePerCommit694=== PAUSE TestObjectStatsUpdatedOncePerCommit695=== RUN TestOrphanedObjectsGC696=== PAUSE TestOrphanedObjectsGC697=== RUN TestOrphanedObjectsGCStressTest698=== PAUSE TestOrphanedObjectsGCStressTest699=== RUN TestCreatePin_ReservedPins700=== PAUSE TestCreatePin_ReservedPins701=== RUN TestPresentReportsOnlyClosureRoots702=== PAUSE TestPresentReportsOnlyClosureRoots703=== RUN TestPresentRefreshesOnlyStaleClosures704=== PAUSE TestPresentRefreshesOnlyStaleClosures705=== RUN TestPresentNotReportedWhileGCDeletesClosure706=== PAUSE TestPresentNotReportedWhileGCDeletesClosure707=== RUN TestParseSingleRange708=== PAUSE TestParseSingleRange709=== RUN TestProxyHeadersOnlyTrustedOnSocket7102026/11/11 23:58:00 INFO Starting HTTP server address=127.0.0.1:410977112026/11/11 23:58:00 INFO Starting HTTP server address=proxy.sock7122026/11/11 23:58:00 WARN mTLS auth: subject not in bound subjects subject="CN=someone"7132026/11/11 23:58:00 INFO Shutdown signal received, draining in-flight requests timeout=10s714--- PASS: TestProxyHeadersOnlyTrustedOnSocket (3.62s)715=== RUN TestIsValidCachePath716=== PAUSE TestIsValidCachePath717=== RUN TestReadProxyNarinfo718=== PAUSE TestReadProxyNarinfo719=== RUN TestReadProxyNarinfoAlreadyDecompressed720=== PAUSE TestReadProxyNarinfoAlreadyDecompressed721=== RUN TestReadProxyContentNegotiation722=== PAUSE TestReadProxyContentNegotiation723=== RUN TestReadProxyNarStreaming724=== PAUSE TestReadProxyNarStreaming725=== RUN TestReadProxy404726=== PAUSE TestReadProxy404727=== RUN TestReadProxyInvalidPath728=== PAUSE TestReadProxyInvalidPath729=== RUN TestReadProxyHead730=== PAUSE TestReadProxyHead731=== RUN TestReadProxyOutlastsServerWriteTimeout732=== PAUSE TestReadProxyOutlastsServerWriteTimeout733=== RUN TestReadProxyConditionalGet734=== PAUSE TestReadProxyConditionalGet735=== RUN TestReadProxyRootRedirectsToIndexHTML736=== PAUSE TestReadProxyRootRedirectsToIndexHTML737=== RUN TestReadProxyDisabled738=== PAUSE TestReadProxyDisabled739=== RUN TestReadRedirectNar740=== PAUSE TestReadRedirectNar741=== RUN TestReadRedirectKeepsNarinfoProxied742=== PAUSE TestReadRedirectKeepsNarinfoProxied743=== RUN TestReadProxyRangeRequest744=== PAUSE TestReadProxyRangeRequest745=== RUN TestReadRedirectUsesPublicS3URL746=== PAUSE TestReadRedirectUsesPublicS3URL747=== RUN TestPush_OverlappingRootsStoreOneRowPerKey748=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey749=== RUN TestPush_CompleteCommitsEveryRoot750=== PAUSE TestPush_CompleteCommitsEveryRoot751=== RUN TestPush_RepeatedRootCommitsOnce752=== PAUSE TestPush_RepeatedRootCommitsOnce753=== RUN TestPush_CompleteWaitsForCleanupOfThePush754=== PAUSE TestPush_CompleteWaitsForCleanupOfThePush755=== RUN TestPush_RootsAreLockedInByteOrder756=== PAUSE TestPush_RootsAreLockedInByteOrder757=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit758=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit759=== RUN TestPush_RejectsBadRequests760=== PAUSE TestPush_RejectsBadRequests761=== RUN TestPush_SignsNarinfosOfItsPendingObjects762=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects763=== RUN TestSharedNarBothUploadsComplete764=== PAUSE TestSharedNarBothUploadsComplete765=== RUN TestCompleteMultipartUpload_ErrorButObjectExists766=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists767=== RUN TestCompletedNarNotReofferedAcrossClosures768=== PAUSE TestCompletedNarNotReofferedAcrossClosures769=== RUN TestPresignedUploadRegisteredBeforeCommit770=== PAUSE TestPresignedUploadRegisteredBeforeCommit771=== RUN TestService_Rustfstest772=== PAUSE TestService_Rustfstest773=== RUN TestParseSize774=== PAUSE TestParseSize775=== RUN TestSkippedUploadsHandler776=== PAUSE TestSkippedUploadsHandler777=== RUN TestSystemdListenerNotActivated778--- PASS: TestSystemdListenerNotActivated (0.00s)779=== RUN TestWatchdogBeatsWhenHealthy780--- PASS: TestWatchdogBeatsWhenHealthy (0.02s)781=== RUN TestWatchdogSkipsWhenUnhealthy7822026/11/11 23:58:00 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7832026/11/11 23:58:00 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7842026/11/11 23:58:00 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7852026/11/11 23:58:00 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7862026/11/11 23:58:00 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7872026/11/11 23:58:00 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7882026/11/11 23:58:00 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7892026/11/11 23:58:00 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7902026/11/11 23:58:00 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7912026/11/11 23:58:00 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"792--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)793=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle794=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle795=== RUN TestProxyWriteTimeout796=== PAUSE TestProxyWriteTimeout797=== RUN TestPendingClosureWriteTimeout798=== PAUSE TestPendingClosureWriteTimeout799=== RUN TestIsValidUploadKey800=== PAUSE TestIsValidUploadKey801=== RUN TestUploadHandlersRejectInvalidKeys802=== PAUSE TestUploadHandlersRejectInvalidKeys803=== RUN TestUploadHandlersRejectOversizedBody804=== PAUSE TestUploadHandlersRejectOversizedBody805=== RUN TestService_cleanupPendingClosuresHandler806=== PAUSE TestService_cleanupPendingClosuresHandler807=== RUN TestService_createPendingClosureHandler808=== PAUSE TestService_createPendingClosureHandler809=== RUN TestService_verifyS3Integrity810=== PAUSE TestService_verifyS3Integrity811=== RUN TestService_verifyS3ReuploadsStaleNarinfo812=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo813=== RUN TestCompleteMultipartUnregistered814=== PAUSE TestCompleteMultipartUnregistered815=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT816=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT817=== CONT TestPendingClosureWriteTimeout818=== CONT TestAbortSucceeded819=== CONT TestProxyWriteTimeout820=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle821=== RUN TestPendingClosureWriteTimeout/empty822=== PAUSE TestPendingClosureWriteTimeout/empty823=== CONT TestSkippedUploadsHandler824=== CONT TestService_Rustfstest825=== CONT TestPresignedUploadRegisteredBeforeCommit826=== CONT TestGCTaskStore_PhaseUpdates827=== CONT TestGCTaskStore_CompletedAllowsNewTask828=== CONT TestGCTaskStore_GetReturnsLatest829=== CONT TestGCTaskStore_GetEmpty830=== CONT TestGCTaskStore_ConflictDifferentParams831=== CONT TestGCTaskStore_DeduplicateSameParams832=== CONT TestGCTaskStore_StartNew833=== CONT TestGCEndsOnShutdown834=== CONT TestCommitRacingPendingCleanupKeepsObjects835=== CONT TestPushWaitsForSweepHoldingRow836=== CONT TestGCTaskStore_Fail837=== CONT TestCompletedNarNotReofferedAcrossClosures838=== CONT TestCompleteMultipartUpload_ErrorButObjectExists839=== CONT TestSweepRechecksPendingAfterLocking840=== CONT TestSharedNarBothUploadsComplete841=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock842--- PASS: TestAbortSucceeded (0.00s)843=== CONT TestPush_SignsNarinfosOfItsPendingObjects844=== RUN TestPendingClosureWriteTimeout/400_objects845=== PAUSE TestPendingClosureWriteTimeout/400_objects846=== RUN TestPendingClosureWriteTimeout/670k_objects847=== PAUSE TestPendingClosureWriteTimeout/670k_objects848=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline849=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline850=== CONT TestResolveDBConnectionString851=== RUN TestResolveDBConnectionString/flag_wins852=== PAUSE TestResolveDBConnectionString/flag_wins853=== RUN TestResolveDBConnectionString/file_when_flag_empty854=== PAUSE TestResolveDBConnectionString/file_when_flag_empty855=== RUN TestResolveDBConnectionString/missing_file_is_an_error856=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error857=== RUN TestResolveDBConnectionString/PGHOST_allows_empty858=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty859=== RUN TestResolveDBConnectionString/nothing_configured860=== PAUSE TestResolveDBConnectionString/nothing_configured861=== CONT TestPush_RejectsBadRequests862--- PASS: TestGCTaskStore_StartNew (0.00s)863=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit864--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)865=== CONT TestClientPushesUseOnePush866--- PASS: TestGCTaskStore_GetEmpty (0.00s)867=== CONT TestGetExistingObjectsLocksRows8682026/11/11 23:58:00 INFO Client skipped oversized paths paths=3 nar_bytes=5000000000869--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)870=== CONT TestPush_RootsAreLockedInByteOrder871=== CONT TestParseSize872--- PASS: TestParseSize (0.00s)873=== CONT TestPinProtectsFromGC874=== CONT TestDeleteClosuresRacingPins875--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)876--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)877=== RUN TestGCEndsOnShutdown/before_the_run878=== RUN TestProxyWriteTimeout/narinfo879--- PASS: TestGCTaskStore_Fail (0.00s)880=== CONT TestRegisterCompletedObjectNeedsPendingRow881=== PAUSE TestGCEndsOnShutdown/before_the_run882=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete883=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete884=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark885=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark886=== CONT TestPush_CompleteWaitsForCleanupOfThePush887--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)888=== PAUSE TestProxyWriteTimeout/narinfo889=== CONT TestClientFallsBackToClosures890=== RUN TestProxyWriteTimeout/1_GiB_nar891=== PAUSE TestProxyWriteTimeout/1_GiB_nar892=== RUN TestProxyWriteTimeout/10_GiB_nar893=== PAUSE TestProxyWriteTimeout/10_GiB_nar894=== RUN TestProxyWriteTimeout/unknown_size895=== PAUSE TestProxyWriteTimeout/unknown_size896=== CONT TestSweepSkipsRowsLockedByPush897=== CONT TestClientSharedPathCommittedMidPush898--- PASS: TestSkippedUploadsHandler (4.34s)899=== CONT TestPush_RepeatedRootCommitsOnce9002026/11/11 23:58:05 INFO Received push request method=POST path=/api/pushes901--- PASS: TestService_Rustfstest (4.61s)902=== CONT TestClientWithDependencies9032026/11/11 23:58:05 INFO Received push request method=POST path=/api/pushes9042026/11/11 23:58:05 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9052026/11/11 23:58:05 INFO Signed narinfos id=1 count=19062026/11/11 23:58:05 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9072026/11/11 23:58:05 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9082026/11/11 23:58:05 ERROR Failed to sign narinfos signer="index 1" error="signing key cannot be nil"9092026/11/11 23:58:05 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9102026/11/11 23:58:05 ERROR Failed to sign narinfos signer=niks3-test-1 error="signer failed"9112026/11/11 23:58:05 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9122026/11/11 23:58:05 ERROR Signer returned unexpected signature count signer=niks3-test-1 got=0 want=19132026/11/11 23:58:05 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9142026/11/11 23:58:05 ERROR Signer returned unexpected object key signer=niks3-test-1 object_key=unexpected.narinfo915--- PASS: TestPush_SignsNarinfosOfItsPendingObjects (4.64s)916=== CONT TestFailedS3DeleteKeepsTombstone9172026/11/11 23:58:05 INFO Received complete push request method=POST path=/api/pushes/1/complete918=== RUN TestPush_RejectsBadRequests/no_roots919=== PAUSE TestPush_RejectsBadRequests/no_roots920=== RUN TestPush_RejectsBadRequests/no_objects921=== PAUSE TestPush_RejectsBadRequests/no_objects922=== RUN TestPush_RejectsBadRequests/bad_root923=== PAUSE TestPush_RejectsBadRequests/bad_root924=== RUN TestPush_RejectsBadRequests/root_not_in_objects925=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects926=== CONT TestPush_CompleteCommitsEveryRoot9272026/11/11 23:58:05 INFO Received push request method=POST path=/api/pushes928=== CONT TestClientMultipleUploads929--- PASS: TestGetExistingObjectsLocksRows (4.73s)9302026/11/11 23:58:05 INFO Completed push id=1 objects=4 duration=86.801108ms9312026/11/11 23:58:05 INFO Received push request method=POST path=/api/pushes9322026/11/11 23:58:05 INFO Received uploads request method=POST path=/api/pending_closures9332026/11/11 23:58:05 INFO Received uploads request method=POST path=/api/pending_closures9342026/11/11 23:58:05 INFO Aborted multipart uploads count=0 kept=09352026/11/11 23:58:05 WARN Force mode enabled - objects will be deleted immediately without grace period9362026/11/11 23:58:05 INFO Received uploads request method=POST path=/api/pending_closures9372026/11/11 23:58:05 INFO Received uploads request method=POST path=/api/pending_closures9382026/11/11 23:58:05 INFO Aborted multipart uploads count=0 kept=09392026/11/11 23:58:05 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9402026/11/11 23:58:05 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=NTQxMjdlZDgtNzdlNS00OThlLTk0MjItOTBjZGEzMzNjNThkLjYyY2UyMTcxLTBkMjMtNDM2Yy05MzU2LTFjNjkxNGYwNDkzM3gxNzk0NDQxNDg1Njk0OTcwNzA39412026/11/11 23:58:05 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=NTQxMjdlZDgtNzdlNS00OThlLTk0MjItOTBjZGEzMzNjNThkLjYyY2UyMTcxLTBkMjMtNDM2Yy05MzU2LTFjNjkxNGYwNDkzM3gxNzk0NDQxNDg1Njk0OTcwNzA3 parts=1942=== CONT TestPush_OverlappingRootsStoreOneRowPerKey943--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (3.63s)9442026/11/11 23:58:05 INFO Aborted multipart uploads count=0 kept=09452026/11/11 23:58:05 WARN Force mode enabled - objects will be deleted immediately without grace period9462026/11/11 23:58:05 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=09472026/11/11 23:58:05 INFO Vacuumed table table=pending_closures9482026/11/11 23:58:05 INFO Received push request method=POST path=/api/pushes9492026/11/11 23:58:05 INFO Vacuumed table table=pending_objects9502026/11/11 23:58:05 INFO Vacuumed table table=multipart_uploads9512026/11/11 23:58:05 INFO Vacuumed table table=closures9522026/11/11 23:58:05 INFO Vacuumed table table=objects9532026/11/11 23:58:05 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=09542026/11/11 23:58:05 INFO Aborted multipart uploads count=0 kept=09552026/11/11 23:58:05 INFO Vacuumed table table=pending_closures956--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (5.05s)957=== CONT TestClientIntegration958--- PASS: TestSweepRechecksPendingAfterLocking (3.80s)959=== CONT TestReadRedirectUsesPublicS3URL9602026/11/11 23:58:05 INFO Vacuumed table table=pending_objects9612026/11/11 23:58:05 INFO Vacuumed table table=multipart_uploads9622026/11/11 23:58:05 INFO Vacuumed table table=closures9632026/11/11 23:58:05 INFO Vacuumed table table=objects9642026/11/11 23:58:05 INFO Received complete push request method=POST path=/api/pushes/2/complete9652026-11-11 23:58:05.946 UTC [520] ERROR: Push does not exist: id=19662026-11-11 23:58:05.946 UTC [520] CONTEXT: PL/pgSQL function commit_push(bigint) line 11 at RAISE9672026-11-11 23:58:05.946 UTC [520] STATEMENT: SELECT commit_push($1)968--- PASS: TestPush_CompleteWaitsForCleanupOfThePush (3.81s)969=== CONT TestClientErrorHandling970=== RUN TestClientErrorHandling/InvalidStorePath971=== PAUSE TestClientErrorHandling/InvalidStorePath972=== RUN TestClientErrorHandling/InvalidAuthToken973=== PAUSE TestClientErrorHandling/InvalidAuthToken974=== RUN TestClientErrorHandling/ServerNotAvailable975=== PAUSE TestClientErrorHandling/ServerNotAvailable976=== CONT TestReadProxyRangeRequest9772026/11/11 23:58:05 INFO Completed push id=2 objects=4 duration=3.581314ms978--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (5.07s)979=== CONT TestClientCADerivations9802026/11/11 23:58:05 INFO Received uploads request method=POST path=/api/pending_closures9812026/11/11 23:58:06 INFO Received push request method=POST path=/api/pushes9822026/11/11 23:58:06 INFO Received complete push request method=POST path=/api/pushes/1/complete983--- PASS: TestRegisterCompletedObjectNeedsPendingRow (3.98s)984=== CONT TestForceGCDuringPushOffersSweptObject985=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows986=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows987=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check988=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check989=== CONT TestCacheStatsHandler9902026/11/11 23:58:06 INFO Received uploads request method=POST path=/api/pending_closures9912026/11/11 23:58:06 INFO Received uploads request method=POST path=/api/pending_closures9922026/11/11 23:58:06 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9932026/11/11 23:58:06 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst9942026/11/11 23:58:06 INFO Received uploads request method=POST path=/api/pending_closures995--- PASS: TestPresignedUploadRegisteredBeforeCommit (5.33s)996=== CONT TestReadRedirectKeepsNarinfoProxied9972026/11/11 23:58:06 INFO Received push request method=POST path=/api/pushes9982026/11/11 23:58:06 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=NTQxMjdlZDgtNzdlNS00OThlLTk0MjItOTBjZGEzMzNjNThkLjM1NzFkYWUyLTgxMWItNGJiYS1hNWFmLWY3Y2Y3ODM2ZGZiNXgxNzk0NDQxNDg1NjM5NTU5MDE5 parts=129992026/11/11 23:58:06 INFO Received uploads request method=POST path=/api/pending_closures10002026/11/11 23:58:06 INFO Received complete push request method=POST path=/api/pushes/1/complete1001--- PASS: TestCompletedNarNotReofferedAcrossClosures (4.12s)1002=== CONT TestReadRedirectNar10032026/11/11 23:58:06 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10042026/11/11 23:58:06 INFO Received uploads request method=POST path=/api/pending_closures10052026/11/11 23:58:06 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=NTQxMjdlZDgtNzdlNS00OThlLTk0MjItOTBjZGEzMzNjNThkLjAxNjIzYjIzLTA2MTUtNDdmYy05MzdjLWVhNWNkOTE2OWUwNHgxNzk0NDQxNDg1NzI3NTQ4NjM4 parts=1210062026/11/11 23:58:06 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10072026/11/11 23:58:06 INFO Aborted multipart uploads count=0 kept=01008--- PASS: TestPushWaitsForSweepHoldingRow (4.34s)1009=== CONT TestService_ReadScope_PublicByDefault10102026-11-11 23:58:06.554 UTC [563] ERROR: could not obtain lock on row in relation "closures"10112026-11-11 23:58:06.554 UTC [563] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT10122026/11/11 23:58:06 INFO Completed push id=1 objects=4 duration=461.092138ms10132026-11-11 23:58:06.555 UTC [563] ERROR: could not obtain lock on row in relation "closures"10142026-11-11 23:58:06.555 UTC [563] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT10152026-11-11 23:58:06.555 UTC [563] ERROR: could not obtain lock on row in relation "closures"10162026-11-11 23:58:06.555 UTC [563] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT10172026-11-11 23:58:06.556 UTC [563] ERROR: could not obtain lock on row in relation "closures"10182026-11-11 23:58:06.556 UTC [563] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT10192026-11-11 23:58:06.556 UTC [563] ERROR: could not obtain lock on row in relation "closures"10202026-11-11 23:58:06.556 UTC [563] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT10212026/11/11 23:58:06 WARN Force mode enabled - objects will be deleted immediately without grace period10222026/11/11 23:58:06 WARN Force mode enabled - objects will be deleted immediately without grace period1023--- PASS: TestPush_RepeatedRootCommitsOnce (1.35s)1024=== CONT TestReadProxyDisabled1025--- PASS: TestPush_RootsAreLockedInByteOrder (5.69s)1026=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection10272026/11/11 23:58:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010282026/11/11 23:58:06 INFO Completed push id=1 objects=6 duration=324.997121ms1029--- PASS: TestPush_CompleteCommitsEveryRoot (1.03s)1030=== CONT TestService_RequireScope_OIDC10312026/11/11 23:58:06 INFO Vacuumed table table=pending_closures10322026/11/11 23:58:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010332026/11/11 23:58:06 INFO Vacuumed table table=pending_closures10342026/11/11 23:58:06 INFO Vacuumed table table=pending_objects10352026/11/11 23:58:06 INFO Vacuumed table table=pending_objects10362026/11/11 23:58:06 INFO Vacuumed table table=multipart_uploads10372026/11/11 23:58:06 INFO Vacuumed table table=closures10382026/11/11 23:58:06 INFO Vacuumed table table=multipart_uploads10392026/11/11 23:58:06 INFO Vacuumed table table=objects10402026/11/11 23:58:06 INFO Vacuumed table table=closures10412026/11/11 23:58:06 INFO Vacuumed table table=objects10422026/11/11 23:58:06 INFO Aborted multipart uploads count=0 kept=010432026/11/11 23:58:06 WARN Force mode enabled - objects will be deleted immediately without grace period1044=== NAME TestPinProtectsFromGC1045 client_integration_test.go:779: Pinned store path: /build/TestPinProtectsFromGC2187224002/001/store/wl42c431pg960d739hhvfsza6lx90a1a-pinned-file.txt1046 client_integration_test.go:780: Unpinned store path: /build/TestPinProtectsFromGC2187224002/001/store/aiyz46fc59jlgi3xzcb1578j3fbvvn6c-unpinned-file.txt1047=== NAME TestClientMultipleUploads1048 client_integration_test.go:406: Created store path 0: /build/TestClientMultipleUploads1689975283/001/store/xajswyi49nbalwmnf87fzbs1h0cc2rh5-test-file-0.txt1049--- PASS: TestCacheStatsHandler (0.58s)1050=== CONT TestReadProxyRootRedirectsToIndexHTML10512026/11/11 23:58:06 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:44575/oidc10522026/11/11 23:58:06 WARN Force mode enabled - objects will be deleted immediately without grace period1053=== NAME TestClientWithDependencies1054 client_integration_test.go:661: Built derivation: /build/TestClientWithDependencies1992346860/001/store/vf4rkl4rcbins7y6mcxx12nwxv9dwrax-test-script1055--- PASS: TestCommitRacingPendingCleanupKeepsObjects (4.58s)1056=== CONT TestCacheConfigHandler1057=== RUN TestCacheConfigHandler/full_config,_no_issuer1058=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1059=== RUN TestCacheConfigHandler/no_cache_url_configured1060=== PAUSE TestCacheConfigHandler/no_cache_url_configured1061=== RUN TestCacheConfigHandler/no_signing_keys1062=== PAUSE TestCacheConfigHandler/no_signing_keys1063=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1064=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1065=== CONT TestService_AuthMiddleware_OIDC10662026/11/11 23:58:06 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."10672026/11/11 23:58:06 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."1068--- PASS: TestFailedS3DeleteKeepsTombstone (1.24s)1069=== CONT TestReadProxyConditionalGet1070--- PASS: TestReadProxyRangeRequest (0.81s)1071=== CONT TestGCSweepDeliversEachKeyOnce10722026/11/11 23:58:06 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:34449/oidc1073=== NAME TestClientMultipleUploads1074 client_integration_test.go:406: Created store path 1: /build/TestClientMultipleUploads1689975283/001/store/158qqvbkxlrfqj4kbwq4rxjcr3hx2wqm-test-file-1.txt10752026/11/11 23:58:06 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10762026/11/11 23:58:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=010772026/11/11 23:58:06 INFO Vacuumed table table=pending_closures10782026/11/11 23:58:06 INFO Vacuumed table table=pending_objects10792026/11/11 23:58:06 INFO Vacuumed table table=multipart_uploads10802026/11/11 23:58:06 INFO Vacuumed table table=closures10812026/11/11 23:58:06 INFO Vacuumed table table=objects1082--- PASS: TestSweepSkipsRowsLockedByPush (4.69s)1083=== CONT TestService_ReadAuthMiddleware10842026/11/11 23:58:06 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=NTQxMjdlZDgtNzdlNS00OThlLTk0MjItOTBjZGEzMzNjNThkLjcyN2FjMDQ1LTU5Y2EtNGUwZi1iZDMwLTFmM2IyYTlhNzQ5MngxNzk0NDQxNDg1NzM2MTI0MzIw parts=121085--- PASS: TestSharedNarBothUploadsComplete (4.71s)1086=== CONT TestReadProxyOutlastsServerWriteTimeout1087--- PASS: TestReadRedirectKeepsNarinfoProxied (0.65s)1088=== CONT TestTombstonedObjectOfferedWithoutWaiting10892026/11/11 23:58:06 INFO Received push request method=POST path=/api/pushes1090=== NAME TestClientIntegration1091 client_integration_test.go:292: Created store path: /build/TestClientIntegration1903960095/002/store/29ixakasnvkxmj4p9k04zxybqzlhcszi-test-file.txt1092=== NAME TestClientWithDependencies1093 client_integration_test.go:663: Found 1 dependencies (including self)1094--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (1.14s)1095=== CONT TestService_AuthMiddleware_MTLSBoundSubjects1096=== NAME TestClientMultipleUploads1097 client_integration_test.go:406: Created store path 2: /build/TestClientMultipleUploads1689975283/001/store/44nh3zi7cg4hhrb42q41653kb29m24cz-test-file-2.txt1098--- PASS: TestReadRedirectNar (0.68s)1099=== CONT TestReadProxyHead1100--- PASS: TestReadRedirectUsesPublicS3URL (1.05s)1101=== CONT TestGCSweepSkipsPendingObjects1102--- PASS: TestReadProxyDisabled (0.46s)1103=== CONT TestService_AuthMiddleware_MTLSProxyHeader1104--- PASS: TestService_ReadScope_PublicByDefault (0.57s)1105=== CONT TestReadProxyInvalidPath11062026/11/11 23:58:07 INFO Received uploads request method=POST path=/api/pending_closures1107=== RUN TestService_RequireScope_OIDC/builder_may_write1108=== PAUSE TestService_RequireScope_OIDC/builder_may_write1109=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1110=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1111=== RUN TestService_RequireScope_OIDC/ops_may_admin1112=== PAUSE TestService_RequireScope_OIDC/ops_may_admin1113=== RUN TestService_RequireScope_OIDC/ops_may_not_write1114=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1115=== RUN TestService_RequireScope_OIDC/reader_may_not_write1116=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1117=== RUN TestService_RequireScope_OIDC/static_token_may_admin1118=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1119=== RUN TestService_RequireScope_OIDC/static_token_may_write1120=== PAUSE TestService_RequireScope_OIDC/static_token_may_write1121=== RUN TestService_RequireScope_OIDC/reader_may_read1122=== PAUSE TestService_RequireScope_OIDC/reader_may_read1123=== RUN TestService_RequireScope_OIDC/writer_implies_read1124=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1125=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1126=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read1127=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc1128=== PAUSE TestService_RequireScope_OIDC/reader_may_read_with_netrc1129=== RUN TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1130=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1131=== RUN TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1132=== PAUSE TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1133=== RUN TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1134=== PAUSE TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1135=== CONT TestDeduplicatedObjectsRecordedAsPending1136--- PASS: TestReadProxyRootRedirectsToIndexHTML (0.44s)1137=== CONT TestService_AuthMiddleware1138--- PASS: TestReadProxyConditionalGet (0.46s)1139=== CONT TestReadProxy4041140=== NAME TestClientCADerivations1141 client_ca_test.go:136: Built CA derivation: /build/TestClientCADerivations3815275722/001/store/mj0xw5xq026yqqm7k870r4mfxg5p784l-ca-test1142--- PASS: TestReadProxyHead (0.31s)1143=== CONT TestPushDedupSurvivesConcurrentGC1144--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (0.72s)1145=== CONT TestUploadHandlersRejectOversizedBody11462026/11/11 23:58:07 INFO Received uploads request method=POST path=/api/pending_closures1147--- PASS: TestTombstonedObjectOfferedWithoutWaiting (0.48s)1148=== CONT TestGCMetrics11492026/11/11 23:58:07 INFO Received uploads request method=POST path=/api/pending_closures1150--- PASS: TestDeduplicatedObjectsRecordedAsPending (0.24s)1151=== CONT TestService_createPendingClosureHandler1152--- PASS: TestService_ReadAuthMiddleware (0.55s)1153=== CONT TestGCBugBareHashReferences1154--- PASS: TestReadProxyInvalidPath (0.37s)1155=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT1156=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token1157=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token1158=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1159=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1160=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1161=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1162=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1163=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1164=== CONT TestGCAdvisoryLockBlocksConcurrentRun1165=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure1166=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure1167=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart1168=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart1169=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts1170=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts1171=== CONT TestCompleteMultipartUnregistered11722026/11/11 23:58:07 INFO Received uploads request method=POST path=/api/pending_closures11732026/11/11 23:58:07 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"11742026/11/11 23:58:07 WARN mTLS auth: bound subjects configured but subject DN unavailable11752026/11/11 23:58:07 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1176--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (0.89s)1177=== CONT TestReadProxyNarStreaming11782026/11/11 23:58:07 INFO Received uploads request method=POST path=/api/pending_closures1179--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (0.39s)1180=== CONT TestService_verifyS3ReuploadsStaleNarinfo1181=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1182=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1183=== RUN TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1184=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1185=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1186=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1187=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1188=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1189=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1190=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1191=== CONT TestLeadEndsOnShutdown11922026/11/11 23:58:07 INFO Aborted multipart uploads count=0 kept=011932026/11/11 23:58:07 INFO Aborted multipart uploads count=0 kept=011942026/11/11 23:58:08 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=nixbld database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"1195--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (0.58s)1196=== CONT TestReadProxyContentNegotiation11972026/11/11 23:58:08 INFO Received complete multipart upload request method=POST path=/api/multipart/complete11982026/11/11 23:58:08 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst1199--- PASS: TestCompleteMultipartUnregistered (0.62s)1200=== CONT TestLeadGenerationGrowsWithEveryGrant12012026/11/11 23:58:08 INFO Aborted multipart uploads count=0 kept=01202--- PASS: TestGCBugBareHashReferences (1.06s)1203=== CONT TestReadProxyNarinfoAlreadyDecompressed12042026/11/11 23:58:08 WARN Force mode enabled - objects will be deleted immediately without grace period12052026/11/11 23:58:08 WARN Force mode enabled - objects will be deleted immediately without grace period12062026/11/11 23:58:08 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=01207=== NAME TestClientCADerivations1208 client_ca_test.go:139: Found 1 dependencies (including self)12092026/11/11 23:58:08 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012102026/11/11 23:58:08 INFO Vacuumed table table=pending_closures12112026/11/11 23:58:08 INFO Vacuumed table table=pending_objects12122026/11/11 23:58:08 INFO Vacuumed table table=multipart_uploads12132026/11/11 23:58:08 INFO Vacuumed table table=closures12142026/11/11 23:58:08 INFO Vacuumed table table=objects12152026/11/11 23:58:08 INFO Received push request method=POST path=/api/pushes12162026/11/11 23:58:08 INFO Received push request method=POST path=/api/pushes1217--- PASS: TestGCMetrics (1.14s)1218=== CONT TestLeadIncumbentAnnouncesAtOnce12192026/11/11 23:58:08 INFO Vacuumed table table=pending_closures12202026/11/11 23:58:08 INFO Vacuumed table table=pending_objects12212026/11/11 23:58:08 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)12222026/11/11 23:58:08 INFO Uploading wl42c431pg960d739hhvfsza6lx90a1a-pinned-file.txt (128B)12232026/11/11 23:58:08 INFO Vacuumed table table=multipart_uploads12242026/11/11 23:58:08 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)12252026/11/11 23:58:08 INFO Uploading kl23wj1jpy0f2zrjiqknkkj62m46cb7a-b (208B)12262026/11/11 23:58:08 INFO Uploading kmvcbp322ki89ivvq0zxa9m6bgfzzxi7-shared-dep (136B)12272026/11/11 23:58:08 INFO Vacuumed table table=closures12282026/11/11 23:58:08 INFO Vacuumed table table=objects12292026/11/11 23:58:08 INFO Aborted multipart uploads count=0 kept=012302026/11/11 23:58:08 WARN Force mode enabled - objects will be deleted immediately without grace period12312026/11/11 23:58:08 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012322026/11/11 23:58:08 INFO Vacuumed table table=pending_closures12332026/11/11 23:58:08 INFO Vacuumed table table=pending_objects12342026/11/11 23:58:08 INFO Vacuumed table table=multipart_uploads12352026/11/11 23:58:08 INFO Vacuumed table table=closures12362026/11/11 23:58:08 INFO Vacuumed table table=objects1237--- PASS: TestPushDedupSurvivesConcurrentGC (1.28s)1238=== CONT TestReadProxyNarinfo12392026/11/11 23:58:08 WARN Failed to register uploaded object key=wl42c431pg960d739hhvfsza6lx90a1a.ls error="server returned 404: 404 page not found\n"12402026/11/11 23:58:08 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"12412026/11/11 23:58:08 WARN Failed to register uploaded object key=kl23wj1jpy0f2zrjiqknkkj62m46cb7a.ls error="server returned 404: 404 page not found\n"12422026/11/11 23:58:08 WARN Failed to register uploaded object key=2vrchy3wcj02iw5q6g3rqyigfvnk3qn3.ls error="server returned 404: 404 page not found\n"12432026/11/11 23:58:08 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"12442026/11/11 23:58:08 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12452026/11/11 23:58:08 WARN Failed to register uploaded object key=kmvcbp322ki89ivvq0zxa9m6bgfzzxi7.ls error="server returned 404: 404 page not found\n"12462026/11/11 23:58:08 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12472026/11/11 23:58:08 WARN Failed to register uploaded object key=nar/1q4dn4q0ygk5dshlki4k2npa8rp3j9a5n0s8iskbwp8zls9la703.nar.zst error="server returned 404: 404 page not found\n"12482026/11/11 23:58:08 INFO Signed narinfos id=1 count=312492026/11/11 23:58:08 INFO Signed narinfos id=1 count=112502026/11/11 23:58:08 INFO Uploading 3 narinfos12512026/11/11 23:58:08 INFO Uploading 1 narinfos12522026/11/11 23:58:08 WARN Failed to register uploaded object key=2vrchy3wcj02iw5q6g3rqyigfvnk3qn3.narinfo error="server returned 404: 404 page not found\n"12532026/11/11 23:58:08 WARN Failed to register uploaded object key=kmvcbp322ki89ivvq0zxa9m6bgfzzxi7.narinfo error="server returned 404: 404 page not found\n"12542026/11/11 23:58:08 INFO Received complete push request method=POST path=/api/pushes/1/complete12552026/11/11 23:58:08 INFO Received complete push request method=POST path=/api/pushes/1/complete12562026/11/11 23:58:08 WARN Failed to register uploaded object key=wl42c431pg960d739hhvfsza6lx90a1a.narinfo error="server returned 404: 404 page not found\n"12572026/11/11 23:58:08 WARN Failed to register uploaded object key=kl23wj1jpy0f2zrjiqknkkj62m46cb7a.narinfo error="server returned 404: 404 page not found\n"12582026/11/11 23:58:08 INFO Completed push id=1 objects=8 duration=14.961903ms12592026/11/11 23:58:08 INFO Completed push id=1 objects=3 duration=14.911102ms12602026/11/11 23:58:08 INFO Upload complete. (1.331s)12612026/11/11 23:58:08 INFO Upload complete. (1.704s)12622026/11/11 23:58:08 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=012632026/11/11 23:58:08 INFO Vacuumed table table=pending_closures12642026/11/11 23:58:08 INFO Vacuumed table table=pending_objects1265=== NAME TestClientPushesUseOnePush1266 client_pushes_test.go:97: Retrieved narinfo from S3:1267 StorePath: /build/TestClientPushesUseOnePush685138421/001/store/kmvcbp322ki89ivvq0zxa9m6bgfzzxi7-shared-dep1268 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1269 Compression: zstd1270 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821271 NarSize: 1361272 References: 1273 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n12742026/11/11 23:58:08 INFO Vacuumed table table=multipart_uploads12752026/11/11 23:58:08 INFO lead: acquired remote=192.0.2.1:1234 generation=112762026/11/11 23:58:08 INFO Vacuumed table table=closures12772026/11/11 23:58:08 INFO Vacuumed table table=objects1278 client_pushes_test.go:97: Retrieved narinfo from S3:1279 StorePath: /build/TestClientPushesUseOnePush685138421/001/store/2vrchy3wcj02iw5q6g3rqyigfvnk3qn3-a1280 URL: nar/1q4dn4q0ygk5dshlki4k2npa8rp3j9a5n0s8iskbwp8zls9la703.nar.zst1281 Compression: zstd1282 NarHash: sha256:1q4dn4q0ygk5dshlki4k2npa8rp3j9a5n0s8iskbwp8zls9la7031283 NarSize: 2081284 References: /build/TestClientPushesUseOnePush685138421/001/store/kmvcbp322ki89ivvq0zxa9m6bgfzzxi7-shared-dep1285 CA: text:sha256:1xn71kc5zifs6wsm410i1fzhajzf41bf71x6zp9iqaczvfmfys0r1286--- PASS: TestGCSweepDeliversEachKeyOnce (1.82s)1287=== CONT TestLeadUnannouncedStandbyYieldsToIncumbent1288=== NAME TestClientPushesUseOnePush1289 client_pushes_test.go:97: Retrieved narinfo from S3:1290 StorePath: /build/TestClientPushesUseOnePush685138421/001/store/kl23wj1jpy0f2zrjiqknkkj62m46cb7a-b1291 URL: nar/1q4dn4q0ygk5dshlki4k2npa8rp3j9a5n0s8iskbwp8zls9la703.nar.zst1292 Compression: zstd1293 NarHash: sha256:1q4dn4q0ygk5dshlki4k2npa8rp3j9a5n0s8iskbwp8zls9la7031294 NarSize: 2081295 References: /build/TestClientPushesUseOnePush685138421/001/store/kmvcbp322ki89ivvq0zxa9m6bgfzzxi7-shared-dep1296 CA: text:sha256:1xn71kc5zifs6wsm410i1fzhajzf41bf71x6zp9iqaczvfmfys0r1297--- PASS: TestClientPushesUseOnePush (7.71s)1298=== CONT TestIsValidCachePath1299=== RUN TestIsValidCachePath/narinfo1300=== PAUSE TestIsValidCachePath/narinfo1301=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars1302=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars1303=== RUN TestIsValidCachePath/nar_zst1304=== PAUSE TestIsValidCachePath/nar_zst1305=== RUN TestIsValidCachePath/nar_xz1306=== PAUSE TestIsValidCachePath/nar_xz1307=== RUN TestIsValidCachePath/nar_bz21308=== PAUSE TestIsValidCachePath/nar_bz21309=== RUN TestIsValidCachePath/nar_uncompressed1310=== PAUSE TestIsValidCachePath/nar_uncompressed1311=== RUN TestIsValidCachePath/ls1312=== PAUSE TestIsValidCachePath/ls1313=== RUN TestIsValidCachePath/log1314=== PAUSE TestIsValidCachePath/log1315=== RUN TestIsValidCachePath/realisation1316=== PAUSE TestIsValidCachePath/realisation1317=== RUN TestIsValidCachePath/nix-cache-info1318=== PAUSE TestIsValidCachePath/nix-cache-info1319=== RUN TestIsValidCachePath/index.html1320=== PAUSE TestIsValidCachePath/index.html1321=== RUN TestIsValidCachePath/traversal_parent1322=== PAUSE TestIsValidCachePath/traversal_parent1323=== RUN TestIsValidCachePath/traversal_in_middle1324=== PAUSE TestIsValidCachePath/traversal_in_middle1325=== RUN TestIsValidCachePath/invalid_char_e1326=== PAUSE TestIsValidCachePath/invalid_char_e1327=== RUN TestIsValidCachePath/invalid_char_u1328=== PAUSE TestIsValidCachePath/invalid_char_u1329=== RUN TestIsValidCachePath/random_path1330=== PAUSE TestIsValidCachePath/random_path1331=== RUN TestIsValidCachePath/empty1332=== PAUSE TestIsValidCachePath/empty1333=== RUN TestIsValidCachePath/leading_slash1334=== PAUSE TestIsValidCachePath/leading_slash1335=== RUN TestIsValidCachePath/wrong_extension1336=== PAUSE TestIsValidCachePath/wrong_extension1337=== RUN TestIsValidCachePath/short_hash1338=== PAUSE TestIsValidCachePath/short_hash1339=== CONT TestLeadElectsOneAndHandsOver1340=== CONT TestParseSingleRange1341--- PASS: TestReadProxyNarStreaming (1.79s)1342=== RUN TestParseSingleRange/none1343=== PAUSE TestParseSingleRange/none1344=== RUN TestParseSingleRange/unknown_unit1345=== PAUSE TestParseSingleRange/unknown_unit1346=== RUN TestParseSingleRange/multi-range_ignored1347=== PAUSE TestParseSingleRange/multi-range_ignored1348=== RUN TestParseSingleRange/malformed_no_dash1349=== PAUSE TestParseSingleRange/malformed_no_dash1350=== RUN TestParseSingleRange/malformed_both_empty1351=== PAUSE TestParseSingleRange/malformed_both_empty1352=== RUN TestParseSingleRange/malformed_end_before_start1353=== PAUSE TestParseSingleRange/malformed_end_before_start1354=== RUN TestParseSingleRange/closed1355=== PAUSE TestParseSingleRange/closed1356=== RUN TestParseSingleRange/open-ended1357=== PAUSE TestParseSingleRange/open-ended1358=== RUN TestParseSingleRange/end_clamped_to_size1359=== PAUSE TestParseSingleRange/end_clamped_to_size1360=== RUN TestParseSingleRange/suffix1361=== PAUSE TestParseSingleRange/suffix1362=== RUN TestParseSingleRange/suffix_exceeds_size1363=== PAUSE TestParseSingleRange/suffix_exceeds_size1364=== RUN TestParseSingleRange/single_byte1365=== PAUSE TestParseSingleRange/single_byte1366=== RUN TestParseSingleRange/start_past_EOF1367=== PAUSE TestParseSingleRange/start_past_EOF1368=== RUN TestParseSingleRange/start_far_past_EOF1369=== PAUSE TestParseSingleRange/start_far_past_EOF1370=== CONT TestConnectWaitsForAPeerMigration13712026/11/11 23:58:09 INFO lead: acquired remote=192.0.2.1:1234 generation=113722026/11/11 23:58:09 INFO Received uploads request method=POST path=/api/pending_closures13732026/11/11 23:58:09 INFO Received uploads request method=POST path=/api/pending_closures13742026/11/11 23:58:09 INFO Received uploads request method=POST path=/api/pending_closures1375--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (2.63s)1376=== CONT TestPresentNotReportedWhileGCDeletesClosure1377--- PASS: TestReadProxyContentNegotiation (1.64s)1378=== CONT TestPresentRefreshesOnlyStaleClosures1379=== RUN TestPresentNotReportedWhileGCDeletesClosure/fresh1380=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/fresh1381=== RUN TestPresentNotReportedWhileGCDeletesClosure/stale1382=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/stale1383=== CONT TestUploadHandlersRejectInvalidKeys1384=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1385=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1386=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal1387=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal1388=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key1389=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key1390=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key1391=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key1392=== CONT TestPresentReportsOnlyClosureRoots13932026/11/11 23:58:09 INFO lead: released remote=192.0.2.1:1234 generation=113942026/11/11 23:58:09 INFO lead: acquired remote=192.0.2.1:1234 generation=213952026/11/11 23:58:09 INFO Received uploads request method=POST path=/api/pending_closures13962026/11/11 23:58:09 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"1397--- PASS: TestService_AuthMiddleware (2.59s)1398=== CONT TestIsValidUploadKey1399=== RUN TestIsValidUploadKey/narinfo1400=== PAUSE TestIsValidUploadKey/narinfo1401=== RUN TestIsValidUploadKey/nar_zst1402=== PAUSE TestIsValidUploadKey/nar_zst1403=== RUN TestIsValidUploadKey/nar_xz1404=== PAUSE TestIsValidUploadKey/nar_xz1405=== RUN TestIsValidUploadKey/nar_plain1406=== PAUSE TestIsValidUploadKey/nar_plain1407=== RUN TestIsValidUploadKey/listing1408=== PAUSE TestIsValidUploadKey/listing1409=== RUN TestIsValidUploadKey/build_log1410=== PAUSE TestIsValidUploadKey/build_log1411=== RUN TestIsValidUploadKey/build_log_home-manager_file1412=== PAUSE TestIsValidUploadKey/build_log_home-manager_file1413=== RUN TestIsValidUploadKey/build_log_plus_in_name1414=== PAUSE TestIsValidUploadKey/build_log_plus_in_name1415=== RUN TestIsValidUploadKey/build_log_question_mark1416=== PAUSE TestIsValidUploadKey/build_log_question_mark1417=== RUN TestIsValidUploadKey/build_log_equals1418=== PAUSE TestIsValidUploadKey/build_log_equals1419=== RUN TestIsValidUploadKey/realisation1420=== PAUSE TestIsValidUploadKey/realisation1421=== RUN TestIsValidUploadKey/realisation_plus_in_output1422=== PAUSE TestIsValidUploadKey/realisation_plus_in_output1423=== RUN TestIsValidUploadKey/nix-cache-info1424=== PAUSE TestIsValidUploadKey/nix-cache-info1425=== RUN TestIsValidUploadKey/index.html1426=== PAUSE TestIsValidUploadKey/index.html1427=== RUN TestIsValidUploadKey/narinfo_key,_nar_type1428=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type1429=== RUN TestIsValidUploadKey/nar_key,_narinfo_type1430=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type1431=== RUN TestIsValidUploadKey/listing_key,_narinfo_type1432=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type1433=== RUN TestIsValidUploadKey/traversal1434=== PAUSE TestIsValidUploadKey/traversal1435=== RUN TestIsValidUploadKey/traversal_nar1436=== PAUSE TestIsValidUploadKey/traversal_nar1437=== RUN TestIsValidUploadKey/absolute1438=== PAUSE TestIsValidUploadKey/absolute1439=== RUN TestIsValidUploadKey/empty_key1440=== PAUSE TestIsValidUploadKey/empty_key1441=== RUN TestIsValidUploadKey/unknown_type1442=== PAUSE TestIsValidUploadKey/unknown_type1443=== CONT TestCreatePin_ReservedPins14442026/11/11 23:58:09 INFO Aborted multipart uploads count=0 kept=01445--- PASS: TestReadProxyNarinfoAlreadyDecompressed (1.34s)1446=== CONT TestMetricsCommit1447--- PASS: TestReadProxy404 (2.60s)1448=== CONT TestService_verifyS3Integrity14492026/11/11 23:58:09 INFO lead: acquired remote=192.0.2.1:1234 generation=114502026/11/11 23:58:09 INFO lead: released remote=192.0.2.1:1234 generation=11451--- PASS: TestLeadIncumbentAnnouncesAtOnce (1.34s)1452=== CONT TestMetricsInventory14532026/11/11 23:58:09 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:36059/oidc14542026/11/11 23:58:10 ERROR Failed to decode narinfo key=3hcdxyjf9yiq7qf3i4548drb6sjmwa1v.narinfo limit=16777216 error=<nil>1455--- PASS: TestReadProxyNarinfo (1.50s)1456=== CONT TestOrphanedObjectsGC14572026/11/11 23:58:10 INFO Received complete multipart upload request method=POST path=/api/multipart/complete14582026/11/11 23:58:10 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=NTQxMjdlZDgtNzdlNS00OThlLTk0MjItOTBjZGEzMzNjNThkLjFlOGJmMTI1LTA3NDctNDRmYi04ZmVjLTViYzY4OWVkYTBiOHgxNzk0NDQxNDg5NjM2NTYzMjEy parts=1014592026/11/11 23:58:10 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete14602026/11/11 23:58:10 INFO Completed upload id=1 objects=2 duration=4.769185ms14612026/11/11 23:58:10 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000014622026/11/11 23:58:10 INFO Received uploads request method=POST path=/api/pending_closures14632026/11/11 23:58:10 INFO Starting cleanup of old closures method=DELETE path=/api/closures14642026/11/11 23:58:10 INFO Aborted multipart uploads count=0 kept=014652026/11/11 23:58:10 WARN Rate limiter enabled after throttle name=s3-test rate=514662026/11/11 23:58:10 WARN S3 rate limit hit operation="complete multipart upload" error="Please reduce your request rate."1467=== NAME TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle1468 throttle_test.go:213: Proxy stats: total=13, throttled=10, completeMultipart=101469 throttle_test.go:215: Rate limiter: enabled=true, rate=5.001470--- PASS: TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle (9.76s)1471=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded14722026/11/11 23:58:10 INFO lead: released remote=192.0.2.1:1234 generation=11473--- PASS: TestLeadEndsOnShutdown (2.89s)1474=== CONT TestObjectStatsUpdatedOncePerCommit14752026/11/11 23:58:10 INFO lead: released remote=192.0.2.1:1234 generation=214762026/11/11 23:58:10 INFO lead: acquired remote=192.0.2.1:1234 generation=314772026/11/11 23:58:10 INFO lead: released remote=192.0.2.1:1234 generation=31478--- PASS: TestLeadGenerationGrowsWithEveryGrant (2.75s)1479=== CONT TestOrphanedObjectsGCStressTest14802026/11/11 23:58:11 INFO Received push request method=POST path=/api/pushes14812026/11/11 23:58:11 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)14822026/11/11 23:58:11 INFO Uploading vf4rkl4rcbins7y6mcxx12nwxv9dwrax-test-script (136B)14832026/11/11 23:58:11 WARN Failed to register uploaded object key=log/qzyq9v1j6wfpiwfcnbygy1lmig7151qw-test-script.drv error="server returned 404: 404 page not found\n"14842026/11/11 23:58:11 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"14852026/11/11 23:58:11 WARN Failed to register uploaded object key=vf4rkl4rcbins7y6mcxx12nwxv9dwrax.ls error="server returned 404: 404 page not found\n"14862026/11/11 23:58:11 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign14872026/11/11 23:58:11 INFO Signed narinfos id=1 count=114882026/11/11 23:58:11 INFO Uploading 1 narinfos14892026/11/11 23:58:11 INFO Received complete push request method=POST path=/api/pushes/1/complete14902026/11/11 23:58:11 WARN Failed to register uploaded object key=vf4rkl4rcbins7y6mcxx12nwxv9dwrax.narinfo error="server returned 404: 404 page not found\n"14912026/11/11 23:58:11 INFO Completed push id=1 objects=4 duration=11.374328ms14922026/11/11 23:58:11 INFO Upload complete. (2.823s)1493--- PASS: TestPresentRefreshesOnlyStaleClosures (1.66s)1494=== CONT TestNARDeduplicationMetadataUploadBug1495=== NAME TestClientWithDependencies1496 client_integration_test.go:665: Skipping nix copy test - isolated store (/build/TestClientWithDependencies1992346860/001/store) requires matching store prefix1497--- PASS: TestClientWithDependencies (5.84s)1498=== CONT TestObjectStatsTrigger1499--- PASS: TestReadProxyOutlastsServerWriteTimeout (4.58s)1500=== CONT TestGenerateLandingPage1501--- PASS: TestGenerateLandingPage (0.00s)1502=== CONT TestService_readinessHandler15032026/11/11 23:58:11 INFO lead: acquired remote=192.0.2.1:1234 generation=115042026/11/11 23:58:11 WARN Force mode enabled - objects will be deleted immediately without grace period15052026/11/11 23:58:11 INFO Received push request method=POST path=/api/pushes15062026/11/11 23:58:11 INFO Received push request method=POST path=/api/pushes15072026/11/11 23:58:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=015082026/11/11 23:58:11 INFO Received push request method=POST path=/api/pushes15092026/11/11 23:58:11 INFO Received push request method=POST path=/api/pushes15102026/11/11 23:58:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=015112026/11/11 23:58:11 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)15122026/11/11 23:58:11 INFO Uploading 29ixakasnvkxmj4p9k04zxybqzlhcszi-test-file.txt (152B)15132026/11/11 23:58:11 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)15142026/11/11 23:58:11 INFO Uploading aiyz46fc59jlgi3xzcb1578j3fbvvn6c-unpinned-file.txt (128B)15152026/11/11 23:58:11 INFO Received uploads request method=POST path=/api/pending_closures15162026/11/11 23:58:11 INFO Vacuumed table table=pending_closures15172026/11/11 23:58:11 INFO Vacuumed table table=pending_closures15182026/11/11 23:58:11 INFO Vacuumed table table=pending_objects15192026/11/11 23:58:11 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)15202026/11/11 23:58:11 INFO Uploading smdw3pzq2ja6xp5p0s93dvjrdk1774pz-shared-dep (136B)15212026/11/11 23:58:11 INFO Uploading zzjyzzgqdfdyz1kr6vzhfnb4cf438qvs-top (224B)15222026/11/11 23:58:11 INFO Vacuumed table table=pending_objects15232026/11/11 23:58:11 INFO Vacuumed table table=multipart_uploads15242026/11/11 23:58:11 INFO Vacuumed table table=closures15252026/11/11 23:58:11 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)15262026/11/11 23:58:11 INFO Uploading 158qqvbkxlrfqj4kbwq4rxjcr3hx2wqm-test-file-1.txt (160B)15272026/11/11 23:58:11 INFO Uploading xajswyi49nbalwmnf87fzbs1h0cc2rh5-test-file-0.txt (160B)15282026/11/11 23:58:11 INFO Uploading 44nh3zi7cg4hhrb42q41653kb29m24cz-test-file-2.txt (160B)15292026/11/11 23:58:11 WARN Failed to register uploaded object key=29ixakasnvkxmj4p9k04zxybqzlhcszi.ls error="server returned 404: 404 page not found\n"15302026/11/11 23:58:11 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15312026/11/11 23:58:11 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"15322026/11/11 23:58:11 INFO Signed narinfos id=1 count=115332026/11/11 23:58:11 INFO Uploading 1 narinfos15342026/11/11 23:58:11 INFO Received uploads request method=POST path=/api/pending_closures15352026/11/11 23:58:11 INFO Vacuumed table table=objects15362026/11/11 23:58:11 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)15372026/11/11 23:58:11 INFO Uploading 29pw15bvk9f9prs4psmra8chzn9ixn0m-b (216B)15382026/11/11 23:58:11 INFO Uploading l03n9mc77vk5y1fkpw9wjjk41a33y53g-shared-dep (136B)15392026/11/11 23:58:11 INFO Vacuumed table table=multipart_uploads15402026/11/11 23:58:11 INFO Vacuumed table table=closures15412026/11/11 23:58:11 INFO Vacuumed table table=objects15422026/11/11 23:58:11 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"15432026/11/11 23:58:11 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign15442026/11/11 23:58:11 INFO Received complete push request method=POST path=/api/pushes/1/complete15452026/11/11 23:58:11 INFO Signed narinfos id=2 count=115462026/11/11 23:58:11 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"15472026/11/11 23:58:11 WARN Failed to register uploaded object key=aiyz46fc59jlgi3xzcb1578j3fbvvn6c.ls error="server returned 404: 404 page not found\n"15482026/11/11 23:58:11 WARN Failed to register uploaded object key=29ixakasnvkxmj4p9k04zxybqzlhcszi.narinfo error="server returned 404: 404 page not found\n"15492026/11/11 23:58:11 WARN Failed to register uploaded object key=zzjyzzgqdfdyz1kr6vzhfnb4cf438qvs.ls error="server returned 404: 404 page not found\n"15502026/11/11 23:58:11 INFO Uploading 1 narinfos15512026/11/11 23:58:11 WARN Failed to register uploaded object key=158qqvbkxlrfqj4kbwq4rxjcr3hx2wqm.ls error="server returned 404: 404 page not found\n"15522026/11/11 23:58:11 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15532026/11/11 23:58:11 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"15542026/11/11 23:58:11 WARN Failed to register uploaded object key=qra852iiz4va1h8nkjm0a5fbnbv1c1lw.ls error="server returned 404: 404 page not found\n"15552026/11/11 23:58:11 WARN Failed to register uploaded object key=xajswyi49nbalwmnf87fzbs1h0cc2rh5.ls error="server returned 404: 404 page not found\n"15562026/11/11 23:58:11 WARN Failed to register uploaded object key=l03n9mc77vk5y1fkpw9wjjk41a33y53g.ls error="server returned 404: 404 page not found\n"15572026/11/11 23:58:11 WARN Failed to register uploaded object key=nar/11xircjbc1q5gvlxm163l1wv4xvii0iylh13lhm1dfh5jhxwjy03.nar.zst error="server returned 404: 404 page not found\n"15582026/11/11 23:58:11 WARN Failed to register uploaded object key=smdw3pzq2ja6xp5p0s93dvjrdk1774pz.ls error="server returned 404: 404 page not found\n"15592026/11/11 23:58:11 INFO Signed narinfos id=1 count=215602026/11/11 23:58:11 INFO Uploading 2 narinfos15612026/11/11 23:58:11 WARN Failed to register uploaded object key=29pw15bvk9f9prs4psmra8chzn9ixn0m.ls error="server returned 404: 404 page not found\n"15622026/11/11 23:58:11 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"15632026/11/11 23:58:11 WARN Failed to register uploaded object key=44nh3zi7cg4hhrb42q41653kb29m24cz.ls error="server returned 404: 404 page not found\n"15642026/11/11 23:58:11 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15652026/11/11 23:58:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign15662026/11/11 23:58:11 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"15672026/11/11 23:58:11 WARN Failed to register uploaded object key=nar/0bja1wp3vq9kfiywyyfc0lzsjdy8q4hcylzg71gyv962wqwdwzsn.nar.zst error="server returned 404: 404 page not found\n"15682026/11/11 23:58:11 INFO Signed narinfos id=1 count=315692026/11/11 23:58:11 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"15702026/11/11 23:58:11 INFO Uploading 3 narinfos15712026/11/11 23:58:11 INFO Signed narinfos id=1 count=215722026/11/11 23:58:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign15732026/11/11 23:58:11 INFO Signed narinfos id=2 count=215742026/11/11 23:58:11 INFO Uploading 4 narinfos1575--- PASS: TestGCSweepSkipsPendingObjects (4.56s)1576=== CONT TestCreatePendingClosureRejectsOversizedNAR15772026/11/11 23:58:11 INFO Received uploads request method=POST path=/api/pending_closures1578--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.00s)1579=== CONT TestService_healthCheckHandler15802026/11/11 23:58:11 INFO Received complete push request method=POST path=/api/pushes/2/complete15812026/11/11 23:58:11 WARN Failed to register uploaded object key=aiyz46fc59jlgi3xzcb1578j3fbvvn6c.narinfo error="server returned 404: 404 page not found\n"15822026/11/11 23:58:11 INFO Received complete push request method=POST path=/api/pushes/1/complete15832026/11/11 23:58:11 WARN Failed to register uploaded object key=zzjyzzgqdfdyz1kr6vzhfnb4cf438qvs.narinfo error="server returned 404: 404 page not found\n"15842026/11/11 23:58:11 WARN Failed to register uploaded object key=smdw3pzq2ja6xp5p0s93dvjrdk1774pz.narinfo error="server returned 404: 404 page not found\n"15852026/11/11 23:58:11 WARN Failed to register uploaded object key=158qqvbkxlrfqj4kbwq4rxjcr3hx2wqm.narinfo error="server returned 404: 404 page not found\n"15862026/11/11 23:58:11 WARN Failed to register uploaded object key=xajswyi49nbalwmnf87fzbs1h0cc2rh5.narinfo error="server returned 404: 404 page not found\n"15872026/11/11 23:58:11 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001588--- PASS: TestService_createPendingClosureHandler (4.18s)15892026/11/11 23:58:11 INFO Completed push id=2 objects=3 duration=2.889108ms1590=== CONT TestCacheConfigHandlerMaxNarSize15912026/11/11 23:58:11 WARN Failed to register uploaded object key=29pw15bvk9f9prs4psmra8chzn9ixn0m.narinfo error="server returned 404: 404 page not found\n"15922026/11/11 23:58:11 INFO Received complete push request method=POST path=/api/pushes/1/complete1593--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)1594=== CONT TestGracefulShutdownDrainsInflight15952026/11/11 23:58:11 WARN Failed to register uploaded object key=44nh3zi7cg4hhrb42q41653kb29m24cz.narinfo error="server returned 404: 404 page not found\n"15962026/11/11 23:58:11 WARN Failed to register uploaded object key=qra852iiz4va1h8nkjm0a5fbnbv1c1lw.narinfo error="server returned 404: 404 page not found\n"15972026/11/11 23:58:11 INFO Upload complete. (1.17s)15982026/11/11 23:58:11 WARN Failed to register uploaded object key=l03n9mc77vk5y1fkpw9wjjk41a33y53g.narinfo error="server returned 404: 404 page not found\n"15992026/11/11 23:58:11 INFO Starting HTTP server address=127.0.0.1:4579716002026/11/11 23:58:11 INFO Shutdown signal received, draining in-flight requests timeout=10s16012026/11/11 23:58:11 INFO Completed push id=1 objects=6 duration=17.347826ms16022026/11/11 23:58:11 INFO Completed push id=1 objects=3 duration=25.358642ms16032026/11/11 23:58:11 INFO Upload complete. (3.09s)16042026/11/11 23:58:11 INFO Upload complete. (3.094s)16052026/11/11 23:58:11 INFO Completed push id=1 objects=9 duration=16.142134ms16062026/11/11 23:58:11 INFO Upload complete. (3.096s)1607=== NAME TestClientMultipleUploads1608 client_integration_test.go:417: Uploaded 3 paths in 4.657429935s1609=== NAME TestClientSharedPathCommittedMidPush1610 client_integration_test.go:728: Retrieved narinfo from S3:1611 StorePath: /build/TestClientSharedPathCommittedMidPush3809885284/001/store/smdw3pzq2ja6xp5p0s93dvjrdk1774pz-shared-dep1612 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1613 Compression: zstd1614 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821615 NarSize: 1361616 References: 1617 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n16182026/11/11 23:58:11 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete16192026/11/11 23:58:11 WARN Failed to register uploaded object key=l03n9mc77vk5y1fkpw9wjjk41a33y53g.narinfo error="server returned 404: 404 page not found\n"1620 client_integration_test.go:728: Retrieved narinfo from S3:1621 StorePath: /build/TestClientSharedPathCommittedMidPush3809885284/001/store/zzjyzzgqdfdyz1kr6vzhfnb4cf438qvs-top1622 URL: nar/11xircjbc1q5gvlxm163l1wv4xvii0iylh13lhm1dfh5jhxwjy03.nar.zst1623 Compression: zstd1624 NarHash: sha256:11xircjbc1q5gvlxm163l1wv4xvii0iylh13lhm1dfh5jhxwjy031625 NarSize: 2241626 References: /build/TestClientSharedPathCommittedMidPush3809885284/001/store/smdw3pzq2ja6xp5p0s93dvjrdk1774pz-shared-dep1627 CA: text:sha256:1ck1nxy524vzp1w5zxfyac2zssmhli0nfkblk3675d1kvgzrxiik1628--- PASS: TestClientSharedPathCommittedMidPush (9.43s)1629=== CONT TestEnsureAbortUploadsRule1630--- PASS: TestClientMultipleUploads (5.97s)1631=== CONT TestService_cleanupPendingClosuresHandler16322026/11/11 23:58:11 INFO Completed upload id=1 objects=6 duration=12.371438ms16332026/11/11 23:58:11 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete16342026/11/11 23:58:11 INFO Completed upload id=2 objects=6 duration=7.232329ms16352026/11/11 23:58:11 INFO Upload complete. (3.121s)1636=== NAME TestClientFallsBackToClosures1637 client_pushes_test.go:112: Retrieved narinfo from S3:1638 StorePath: /build/TestClientFallsBackToClosures2347019482/001/store/l03n9mc77vk5y1fkpw9wjjk41a33y53g-shared-dep1639 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1640 Compression: zstd1641 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821642 NarSize: 1361643 References: 1644 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1645 client_pushes_test.go:112: Retrieved narinfo from S3:1646 StorePath: /build/TestClientFallsBackToClosures2347019482/001/store/qra852iiz4va1h8nkjm0a5fbnbv1c1lw-a1647 URL: nar/0bja1wp3vq9kfiywyyfc0lzsjdy8q4hcylzg71gyv962wqwdwzsn.nar.zst1648 Compression: zstd1649 NarHash: sha256:0bja1wp3vq9kfiywyyfc0lzsjdy8q4hcylzg71gyv962wqwdwzsn1650 NarSize: 2161651 References: /build/TestClientFallsBackToClosures2347019482/001/store/l03n9mc77vk5y1fkpw9wjjk41a33y53g-shared-dep1652 CA: text:sha256:160cindvmsmxafkbq06x73h5r6wpdsa2a91ygadj9spihi2vyz4q1653 client_pushes_test.go:112: Retrieved narinfo from S3:1654 StorePath: /build/TestClientFallsBackToClosures2347019482/001/store/29pw15bvk9f9prs4psmra8chzn9ixn0m-b1655 URL: nar/0bja1wp3vq9kfiywyyfc0lzsjdy8q4hcylzg71gyv962wqwdwzsn.nar.zst1656 Compression: zstd1657 NarHash: sha256:0bja1wp3vq9kfiywyyfc0lzsjdy8q4hcylzg71gyv962wqwdwzsn1658 NarSize: 2161659 References: /build/TestClientFallsBackToClosures2347019482/001/store/l03n9mc77vk5y1fkpw9wjjk41a33y53g-shared-dep1660 CA: text:sha256:160cindvmsmxafkbq06x73h5r6wpdsa2a91ygadj9spihi2vyz4q1661--- PASS: TestClientFallsBackToClosures (9.47s)1662=== CONT TestService_NativeMTLS1663--- PASS: TestMetricsInventory (1.79s)1664=== CONT TestMultipartCleanup1665--- PASS: TestGracefulShutdownDrainsInflight (0.07s)1666=== CONT TestPendingCleanupUsesOneCutoff16672026/11/11 23:58:11 INFO Received uploads request method=POST path=/api/pending_closures16682026/11/11 23:58:11 INFO Received push request method=POST path=/api/pushes1669--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (1.07s)1670=== CONT TestServerTLSConfig1671=== RUN TestServerTLSConfig/no_client_CA1672=== PAUSE TestServerTLSConfig/no_client_CA1673=== RUN TestServerTLSConfig/missing_CA_file1674=== PAUSE TestServerTLSConfig/missing_CA_file1675=== RUN TestServerTLSConfig/not_a_PEM_file1676=== PAUSE TestServerTLSConfig/not_a_PEM_file1677=== CONT TestPendingClosureWriteTimeout/empty1678=== CONT TestResolveDBConnectionString/flag_wins1679=== CONT TestResolveDBConnectionString/nothing_configured1680=== CONT TestResolveDBConnectionString/PGHOST_allows_empty1681=== CONT TestResolveDBConnectionString/missing_file_is_an_error1682=== CONT TestResolveDBConnectionString/file_when_flag_empty1683=== CONT TestPendingClosureWriteTimeout/400_objects1684=== CONT TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1685--- PASS: TestResolveDBConnectionString (0.00s)1686 --- PASS: TestResolveDBConnectionString/flag_wins (0.00s)1687 --- PASS: TestResolveDBConnectionString/nothing_configured (0.00s)1688 --- PASS: TestResolveDBConnectionString/PGHOST_allows_empty (0.00s)1689 --- PASS: TestResolveDBConnectionString/missing_file_is_an_error (0.00s)1690 --- PASS: TestResolveDBConnectionString/file_when_flag_empty (0.00s)16912026/11/11 23:58:11 INFO Received complete push request method=POST path=/api/pushes/1/complete1692--- PASS: TestObjectStatsTrigger (0.45s)1693=== CONT TestPendingClosureWriteTimeout/670k_objects1694=== CONT TestGCEndsOnShutdown/before_the_run1695--- PASS: TestPresentReportsOnlyClosureRoots (2.18s)1696=== CONT TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark16972026/11/11 23:58:11 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux16982026/11/11 23:58:11 WARN Refused reserved pin name=worker-x86_64-linux16992026/11/11 23:58:11 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux17002026/11/11 23:58:11 INFO Received create pin request method=POST path=/api/pins/my-app17012026/11/11 23:58:11 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux1702--- PASS: TestCreatePin_ReservedPins (2.14s)1703=== CONT TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1704--- PASS: TestObjectStatsUpdatedOncePerCommit (1.33s)1705=== CONT TestProxyWriteTimeout/narinfo1706=== CONT TestProxyWriteTimeout/unknown_size1707=== CONT TestProxyWriteTimeout/10_GiB_nar1708=== CONT TestProxyWriteTimeout/1_GiB_nar1709--- PASS: TestProxyWriteTimeout (1.26s)1710 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1711 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1712 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1713 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)1714=== CONT TestPush_RejectsBadRequests/no_roots17152026/11/11 23:58:12 INFO Received push request method=POST path=/api/pushes1716=== CONT TestPush_RejectsBadRequests/bad_root17172026/11/11 23:58:12 INFO Received push request method=POST path=/api/pushes1718=== CONT TestPush_RejectsBadRequests/root_not_in_objects17192026/11/11 23:58:12 INFO Received push request method=POST path=/api/pushes1720=== CONT TestPush_RejectsBadRequests/no_objects17212026/11/11 23:58:12 INFO Received push request method=POST path=/api/pushes1722=== CONT TestClientErrorHandling/InvalidStorePath1723--- PASS: TestPush_RejectsBadRequests (4.67s)1724 --- PASS: TestPush_RejectsBadRequests/no_roots (0.00s)1725 --- PASS: TestPush_RejectsBadRequests/bad_root (0.00s)1726 --- PASS: TestPush_RejectsBadRequests/root_not_in_objects (0.00s)1727 --- PASS: TestPush_RejectsBadRequests/no_objects (0.00s)17282026/11/11 23:58:12 INFO Received uploads request method=POST path=/api/pending_closures17292026/11/11 23:58:12 INFO lead: acquired remote=192.0.2.1:1234 generation=117302026/11/11 23:58:12 INFO Received complete multipart upload request method=POST path=/api/multipart/complete17312026/11/11 23:58:12 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=NTQxMjdlZDgtNzdlNS00OThlLTk0MjItOTBjZGEzMzNjNThkLmM3MzcwY2Y4LWE3ZDktNDFlZS1hNThmLWVlNTk2NGJjN2U0OXgxNzk0NDQxNDkyMDQ4NjM5NzI1 parts=1017322026/11/11 23:58:12 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete17332026/11/11 23:58:12 INFO Completed upload id=1 objects=2 duration=2.225381ms17342026/11/11 23:58:12 INFO Received uploads request method=POST path=/api/pending_closures17352026/11/11 23:58:12 INFO Received uploads request method=POST path=/api/pending_closures17362026/11/11 23:58:12 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo17372026/11/11 23:58:12 WARN Found objects in DB but missing from S3, will re-upload count=11738--- PASS: TestService_verifyS3Integrity (2.72s)1739=== CONT TestClientErrorHandling/ServerNotAvailable17402026/11/11 23:58:12 INFO lead: released remote=192.0.2.1:1234 generation=117412026/11/11 23:58:12 INFO lead: acquired remote=192.0.2.1:1234 generation=217422026/11/11 23:58:13 INFO lead: released remote=192.0.2.1:1234 generation=117432026/11/11 23:58:13 INFO lead: acquired remote=192.0.2.1:1234 generation=217442026/11/11 23:58:13 INFO Received push request method=POST path=/api/pushes17452026/11/11 23:58:13 INFO Completed push id=1 objects=4 duration=1.524182822s17462026/11/11 23:58:13 INFO All 1 paths already cached17472026/11/11 23:58:13 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete17482026/11/11 23:58:13 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)17492026/11/11 23:58:13 INFO Uploading mj0xw5xq026yqqm7k870r4mfxg5p784l-ca-test (144B)17502026/11/11 23:58:13 INFO Starting cleanup of old closures method=DELETE path=/api/closures1751=== NAME TestNARDeduplicationMetadataUploadBug1752 metadata_upload_test.go:48: First store path: /build/TestNARDeduplicationMetadataUploadBug2725575155/001/store/jv6v7c394aqsj4gfd4m740dm828sdclk-file1.txt17532026/11/11 23:58:13 INFO Garbage collection started17542026/11/11 23:58:13 INFO Completed upload id=2 objects=2 duration=5.294611ms1755--- PASS: TestMetricsCommit (3.51s)1756=== CONT TestClientErrorHandling/InvalidAuthToken1757=== NAME TestClientIntegration1758 client_integration_test.go:318: Retrieved narinfo from S3:1759 StorePath: /build/TestClientIntegration1903960095/002/store/29ixakasnvkxmj4p9k04zxybqzlhcszi-test-file.txt1760 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1761 Compression: zstd1762 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11763 NarSize: 1521764 References: 1765 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11766 client_integration_test.go:319: Retrieved .ls file from S3 (compressed size: 77 bytes)1767 client_integration_test.go:319: Decompressed .ls content (64 bytes):1768 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}17692026/11/11 23:58:13 INFO lead: given back to the incumbent remote=192.0.2.1:1234 generation=217702026/11/11 23:58:13 INFO Aborted multipart uploads count=0 kept=017712026/11/11 23:58:13 WARN Failed to register uploaded object key=mj0xw5xq026yqqm7k870r4mfxg5p784l.ls error="server returned 404: 404 page not found\n"17722026/11/11 23:58:13 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"17732026/11/11 23:58:13 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign17742026/11/11 23:58:13 WARN Failed to register uploaded object key=log/71lav4a6jh64x86cjfzkv584ryfxh8jn-ca-test.drv error="server returned 404: 404 page not found\n"17752026/11/11 23:58:13 INFO Signed narinfos id=1 count=117762026/11/11 23:58:13 INFO Uploading 1 narinfos17772026/11/11 23:58:13 INFO lead: acquired remote=192.0.2.1:1234 generation=317782026/11/11 23:58:13 INFO Received complete push request method=POST path=/api/pushes/1/complete17792026/11/11 23:58:13 WARN Failed to register uploaded object key=mj0xw5xq026yqqm7k870r4mfxg5p784l.narinfo error="server returned 404: 404 page not found\n"17802026/11/11 23:58:13 INFO Completed push id=1 objects=4 duration=17.096003ms17812026/11/11 23:58:13 INFO Upload complete. (2.175s)1782=== NAME TestClientCADerivations1783 client_ca_test.go:180: Narinfo contains CA field: StorePath: /build/TestClientCADerivations3815275722/001/store/mj0xw5xq026yqqm7k870r4mfxg5p784l-ca-test1784 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1785 Compression: zstd1786 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1787 NarSize: 1441788 References: 1789 Deriver: /build/TestClientCADerivations3815275722/001/store/71lav4a6jh64x86cjfzkv584ryfxh8jn-ca-test.drv1790 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1791 client_ca_test.go:185: Checking for realisation files in S3...1792 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1793 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache1794=== NAME TestOrphanedObjectsGC1795 orphaned_objects_gc_test.go:287: GC Test Summary:1796 orphaned_objects_gc_test.go:288: - Kept: 2 objects from closure A1797 orphaned_objects_gc_test.go:289: - Deleted: 2 objects from closure B1798 orphaned_objects_gc_test.go:290: - Deleted: 6 orphaned chain objects (X1->X2->X3)1799 orphaned_objects_gc_test.go:291: - Deleted: 2 orphaned single objects (Y)1800 orphaned_objects_gc_test.go:292: - Total deleted: 10 objects1801--- PASS: TestOrphanedObjectsGC (3.41s)1802=== CONT TestForceGCDuringPushOffersSweptObject/before_pending_rows18032026/11/11 23:58:13 INFO Aborted multipart uploads count=0 kept=018042026/11/11 23:58:13 WARN Force mode enabled - objects will be deleted immediately without grace period1805--- PASS: TestDeleteClosuresRacingPins (11.40s)1806=== CONT TestForceGCDuringPushOffersSweptObject/after_presence_check1807--- PASS: TestService_healthCheckHandler (2.00s)1808=== CONT TestCacheConfigHandler/full_config,_no_issuer1809=== CONT TestCacheConfigHandler/no_signing_keys1810=== CONT TestCacheConfigHandler/no_cache_url_configured1811=== CONT TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1812--- PASS: TestCacheConfigHandler (0.00s)1813 --- PASS: TestCacheConfigHandler/full_config,_no_issuer (0.00s)1814 --- PASS: TestCacheConfigHandler/no_signing_keys (0.00s)1815 --- PASS: TestCacheConfigHandler/no_cache_url_configured (0.00s)1816 --- PASS: TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator (0.00s)1817=== CONT TestService_RequireScope_OIDC/builder_may_write1818=== CONT TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1819=== CONT TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1820=== CONT TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read18212026/11/11 23:58:13 WARN Authentication failed token_preview=wrong token_length=5 oidc_error="no provider could verify the token (signature or issuer mismatch)" oidc_provider="" tried_providers=[test]1822=== CONT TestService_RequireScope_OIDC/reader_may_read1823=== CONT TestService_RequireScope_OIDC/reader_may_not_write1824=== CONT TestService_RequireScope_OIDC/ops_may_admin1825=== CONT TestService_RequireScope_OIDC/static_token_may_write1826=== CONT TestService_RequireScope_OIDC/ops_may_not_write1827=== CONT TestService_RequireScope_OIDC/reader_may_read_with_netrc1828=== CONT TestService_RequireScope_OIDC/static_token_may_admin1829=== CONT TestService_RequireScope_OIDC/anonymous_may_not_read1830=== CONT TestService_RequireScope_OIDC/builder_may_not_admin1831=== CONT TestService_RequireScope_OIDC/writer_implies_read1832=== CONT TestService_AuthMiddleware_OIDC/valid_OIDC_token1833--- PASS: TestService_RequireScope_OIDC (0.54s)1834 --- PASS: TestService_RequireScope_OIDC/builder_may_write (0.00s)1835 --- PASS: TestService_RequireScope_OIDC/static_token_may_admin_with_netrc (0.00s)1836 --- PASS: TestService_RequireScope_OIDC/empty_netrc_password_may_not_read (0.00s)1837 --- PASS: TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read (0.00s)1838 --- PASS: TestService_RequireScope_OIDC/reader_may_read (0.00s)1839 --- PASS: TestService_RequireScope_OIDC/reader_may_not_write (0.00s)1840 --- PASS: TestService_RequireScope_OIDC/ops_may_admin (0.00s)1841 --- PASS: TestService_RequireScope_OIDC/static_token_may_write (0.00s)1842 --- PASS: TestService_RequireScope_OIDC/ops_may_not_write (0.00s)1843 --- PASS: TestService_RequireScope_OIDC/reader_may_read_with_netrc (0.00s)1844 --- PASS: TestService_RequireScope_OIDC/static_token_may_admin (0.00s)1845 --- PASS: TestService_RequireScope_OIDC/anonymous_may_not_read (0.00s)1846 --- PASS: TestService_RequireScope_OIDC/builder_may_not_admin (0.00s)1847 --- PASS: TestService_RequireScope_OIDC/writer_implies_read (0.00s)1848=== CONT TestService_AuthMiddleware_OIDC/malformed_token_rejected18492026/11/11 23:58:13 WARN Authentication failed token_preview=not-a-valid-jwt token_length=15 oidc_error="no provider could verify the token (signature or issuer mismatch)" oidc_provider="" tried_providers=[test]1850=== CONT TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1851=== CONT TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected18522026/11/11 23:58:13 WARN Authentication failed token_preview=eyJhbGciOi...JkAA6gbsdA token_length=701 oidc_error="no rule matched: claim \"repository_owner\" value [otherorg] not in allowed patterns [myorg]" oidc_provider=test tried_providers=[test]1853=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure18542026/11/11 23:58:13 INFO Received uploads request method=POST path=/1855--- PASS: TestService_AuthMiddleware_OIDC (0.73s)1856 --- PASS: TestService_AuthMiddleware_OIDC/valid_OIDC_token (0.00s)1857 --- PASS: TestService_AuthMiddleware_OIDC/malformed_token_rejected (0.00s)1858 --- PASS: TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured (0.00s)1859 --- PASS: TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected (0.00s)18602026/11/11 23:58:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=018612026/11/11 23:58:13 INFO Vacuumed table table=pending_closures18622026/11/11 23:58:13 INFO Vacuumed table table=pending_objects18632026/11/11 23:58:13 INFO Vacuumed table table=multipart_uploads18642026/11/11 23:58:13 INFO Received uploads request method=POST path=/api/pending_closures18652026/11/11 23:58:13 INFO Vacuumed table table=closures18662026/11/11 23:58:13 INFO Vacuumed table table=objects18672026/11/11 23:58:13 INFO Aborted multipart uploads count=0 kept=018682026/11/11 23:58:13 INFO Received uploads request method=POST path=/api/pending_closures18692026/11/11 23:58:13 INFO Received uploads request method=POST path=/api/pending_closures18702026/11/11 23:58:13 INFO Received cleanup request method=DELETE path=/api/pending_closures18712026/11/11 23:58:13 INFO lead: released remote=192.0.2.1:1234 generation=21872--- PASS: TestLeadElectsOneAndHandsOver (5.23s)1873=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts18742026/11/11 23:58:13 INFO Received request for more parts method=POST path=/1875=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart18762026/11/11 23:58:13 INFO Received complete multipart upload request method=POST path=/1877=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed18782026/11/11 23:58:13 INFO Received uploads request method=POST path=/api/pending_closures1879=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed18802026/11/11 23:58:13 INFO Received cleanup request method=DELETE path=/api/pending_closures18812026/11/11 23:58:13 INFO Aborted multipart uploads count=0 kept=018822026/11/11 23:58:13 INFO Received uploads request method=POST path=/api/pending_closures18832026/11/11 23:58:13 INFO Received cleanup request method=DELETE path=/api/pending_closures18842026/11/11 23:58:13 INFO Aborted multipart uploads count=1 kept=018852026/11/11 23:58:13 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete18862026/11/11 23:58:13 INFO Received cleanup request method=DELETE path=/api/pending_closures18872026-11-11 23:58:13.996 UTC [1488] ERROR: Closure does not exist: id=118882026-11-11 23:58:13.996 UTC [1488] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 13 at RAISE18892026-11-11 23:58:13.996 UTC [1488] STATEMENT: -- name: CommitPendingClosure :one1890 SELECT commit_pending_closure($1::bigint)::bigint1891 1892--- PASS: TestService_cleanupPendingClosuresHandler (2.42s)1893=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed18942026/11/11 23:58:14 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket94/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""18952026/11/11 23:58:14 INFO Aborted multipart uploads count=0 kept=118962026/11/11 23:58:14 INFO Received cleanup request method=DELETE path=/api/pending_closures18972026/11/11 23:58:14 INFO Aborted multipart uploads count=1 kept=01898--- PASS: TestMultipartCleanup (2.43s)1899=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost19002026/11/11 23:58:14 INFO Installed lifecycle rule aborting incomplete NAR uploads bucket=bucket96 days=21901--- PASS: TestEnsureAbortUploadsRule (2.50s)1902=== CONT TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed19032026/11/11 23:58:14 WARN readiness check failed error="closed pool"1904--- PASS: TestService_readinessHandler (2.67s)1905=== CONT TestIsValidCachePath/narinfo1906=== CONT TestIsValidCachePath/index.html1907=== CONT TestIsValidCachePath/short_hash1908=== CONT TestIsValidCachePath/wrong_extension1909=== CONT TestIsValidCachePath/leading_slash1910=== CONT TestIsValidCachePath/empty1911=== CONT TestIsValidCachePath/random_path1912=== CONT TestIsValidCachePath/invalid_char_u1913=== CONT TestIsValidCachePath/invalid_char_e1914=== CONT TestIsValidCachePath/traversal_in_middle1915=== CONT TestIsValidCachePath/traversal_parent1916=== CONT TestIsValidCachePath/nar_uncompressed1917=== CONT TestIsValidCachePath/realisation1918=== CONT TestIsValidCachePath/log1919=== CONT TestIsValidCachePath/nix-cache-info1920=== CONT TestIsValidCachePath/ls1921=== CONT TestIsValidCachePath/nar_xz1922=== CONT TestIsValidCachePath/nar_bz21923=== CONT TestIsValidCachePath/narinfo_all_nix_base32_chars1924=== CONT TestIsValidCachePath/nar_zst1925--- PASS: TestIsValidCachePath (0.00s)1926 --- PASS: TestIsValidCachePath/narinfo (0.00s)1927 --- PASS: TestIsValidCachePath/index.html (0.00s)1928 --- PASS: TestIsValidCachePath/short_hash (0.00s)1929 --- PASS: TestIsValidCachePath/wrong_extension (0.00s)1930 --- PASS: TestIsValidCachePath/leading_slash (0.00s)1931 --- PASS: TestIsValidCachePath/empty (0.00s)1932 --- PASS: TestIsValidCachePath/random_path (0.00s)1933 --- PASS: TestIsValidCachePath/invalid_char_u (0.00s)1934 --- PASS: TestIsValidCachePath/invalid_char_e (0.00s)1935 --- PASS: TestIsValidCachePath/traversal_in_middle (0.00s)1936 --- PASS: TestIsValidCachePath/traversal_parent (0.00s)1937 --- PASS: TestIsValidCachePath/nar_uncompressed (0.00s)1938 --- PASS: TestIsValidCachePath/realisation (0.00s)1939 --- PASS: TestIsValidCachePath/log (0.00s)1940 --- PASS: TestIsValidCachePath/nix-cache-info (0.00s)1941 --- PASS: TestIsValidCachePath/ls (0.00s)1942 --- PASS: TestIsValidCachePath/nar_xz (0.00s)1943 --- PASS: TestIsValidCachePath/nar_bz2 (0.00s)1944 --- PASS: TestIsValidCachePath/narinfo_all_nix_base32_chars (0.00s)1945 --- PASS: TestIsValidCachePath/nar_zst (0.00s)1946=== CONT TestParseSingleRange/none1947=== CONT TestParseSingleRange/open-ended1948=== CONT TestParseSingleRange/start_far_past_EOF1949=== CONT TestParseSingleRange/start_past_EOF1950=== CONT TestParseSingleRange/single_byte1951=== CONT TestParseSingleRange/end_clamped_to_size1952=== CONT TestParseSingleRange/suffix_exceeds_size1953=== CONT TestParseSingleRange/malformed_both_empty1954=== CONT TestParseSingleRange/suffix1955=== CONT TestParseSingleRange/closed1956=== CONT TestParseSingleRange/malformed_end_before_start1957=== CONT TestParseSingleRange/multi-range_ignored1958=== CONT TestParseSingleRange/unknown_unit1959=== CONT TestParseSingleRange/malformed_no_dash1960--- PASS: TestParseSingleRange (0.00s)1961 --- PASS: TestParseSingleRange/none (0.00s)1962 --- PASS: TestParseSingleRange/open-ended (0.00s)1963 --- PASS: TestParseSingleRange/start_far_past_EOF (0.00s)1964 --- PASS: TestParseSingleRange/start_past_EOF (0.00s)1965 --- PASS: TestParseSingleRange/single_byte (0.00s)1966 --- PASS: TestParseSingleRange/end_clamped_to_size (0.00s)1967 --- PASS: TestParseSingleRange/suffix_exceeds_size (0.00s)1968 --- PASS: TestParseSingleRange/malformed_both_empty (0.00s)1969 --- PASS: TestParseSingleRange/suffix (0.00s)1970 --- PASS: TestParseSingleRange/closed (0.00s)1971 --- PASS: TestParseSingleRange/malformed_end_before_start (0.00s)1972 --- PASS: TestParseSingleRange/multi-range_ignored (0.00s)1973 --- PASS: TestParseSingleRange/unknown_unit (0.00s)1974 --- PASS: TestParseSingleRange/malformed_no_dash (0.00s)1975=== CONT TestPresentNotReportedWhileGCDeletesClosure/fresh19762026/11/11 23:58:14 WARN mTLS auth: subject not in bound subjects subject="CN=reader"19772026/11/11 23:58:14 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1978--- PASS: TestService_NativeMTLS (2.54s)1979=== CONT TestPresentNotReportedWhileGCDeletesClosure/stale1980=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1981--- PASS: TestUploadHandlersRejectOversizedBody (0.18s)1982 --- PASS: TestUploadHandlersRejectOversizedBody/request_more_parts (0.04s)1983 --- PASS: TestUploadHandlersRejectOversizedBody/complete_multipart (0.04s)1984 --- PASS: TestUploadHandlersRejectOversizedBody/create_pending_closure (0.85s)19852026/11/11 23:58:14 INFO Received uploads request method=POST path=/1986=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key19872026/11/11 23:58:14 INFO Received complete multipart upload request method=POST path=/1988=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal19892026/11/11 23:58:14 INFO Received uploads request method=POST path=/1990=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key19912026/11/11 23:58:14 INFO Received request for more parts method=POST path=/1992--- PASS: TestUploadHandlersRejectInvalidKeys (0.00s)1993 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)1994 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1995 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s)1996 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1997=== CONT TestIsValidUploadKey/narinfo1998=== CONT TestIsValidUploadKey/realisation_plus_in_output1999=== CONT TestIsValidUploadKey/unknown_type2000=== CONT TestIsValidUploadKey/empty_key

loading 714 hidden lines…

27152026/11/11 23:58:45 INFO Vacuumed table table=closures27162026/11/11 23:58:45 INFO Received push request method=POST path=/27172026/11/11 23:58:45 INFO Vacuumed table table=objects27182026/11/11 23:58:45 INFO Received uploads request method=POST path=/27192026/11/11 23:58:45 INFO Received complete push request method=POST path=/27202026/11/11 23:58:45 INFO Received push request method=POST path=/27212026/11/11 23:58:45 INFO Completed push id=3 objects=5 duration=2.465683ms27222026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=027232026/11/11 23:58:45 INFO Received push request method=POST path=/27242026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period27252026/11/11 23:58:45 INFO Received push request method=POST path=/27262026/11/11 23:58:45 INFO Received push request method=POST path=/27272026/11/11 23:58:45 INFO Received uploads request method=POST path=/27282026/11/11 23:58:45 INFO Received uploads request method=POST path=/27292026/11/11 23:58:45 INFO Received complete push request method=POST path=/27302026/11/11 23:58:45 INFO Received complete push request method=POST path=/27312026/11/11 23:58:45 INFO Completed push id=5 objects=3 duration=3.13567ms27322026/11/11 23:58:45 INFO Completed push id=2 objects=25 duration=4.064519ms27332026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=027342026/11/11 23:58:45 INFO Received push request method=POST path=/27352026/11/11 23:58:45 INFO Received push request method=POST path=/27362026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period27372026/11/11 23:58:45 INFO Received push request method=POST path=/27382026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027392026/11/11 23:58:45 INFO Vacuumed table table=pending_closures27402026/11/11 23:58:45 INFO Vacuumed table table=pending_objects27412026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads27422026/11/11 23:58:45 INFO Received complete upload request method=POST path=/27432026/11/11 23:58:45 INFO Received complete upload request method=POST path=/27442026/11/11 23:58:45 INFO Vacuumed table table=closures27452026/11/11 23:58:45 INFO Completed upload id=2 objects=2 duration=1.928338ms27462026/11/11 23:58:45 INFO Completed upload id=3 objects=5 duration=1.400953ms27472026/11/11 23:58:45 INFO Vacuumed table table=objects27482026/11/11 23:58:45 INFO Received push request method=POST path=/27492026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=027502026/11/11 23:58:45 INFO Received uploads request method=POST path=/27512026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period27522026/11/11 23:58:45 INFO Received push request method=POST path=/27532026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=027542026/11/11 23:58:45 INFO Received complete push request method=POST path=/27552026/11/11 23:58:45 INFO Received uploads request method=POST path=/27562026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=027572026/11/11 23:58:45 INFO Completed push id=2 objects=7 duration=5.143949ms27582026/11/11 23:58:45 INFO Received push request method=POST path=/27592026/11/11 23:58:45 INFO Received complete push request method=POST path=/27602026/11/11 23:58:45 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."27612026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."27622026/11/11 23:58:45 INFO Completed push id=6 objects=5 duration=1.936018ms27632026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=027642026/11/11 23:58:45 INFO Received complete push request method=POST path=/27652026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=027662026/11/11 23:58:45 INFO Received push request method=POST path=/27672026/11/11 23:58:45 INFO Completed push id=4 objects=4 duration=1.979579ms27682026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=027692026/11/11 23:58:45 INFO Vacuumed table table=pending_closures27702026/11/11 23:58:45 INFO Vacuumed table table=pending_objects27712026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads27722026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=027732026/11/11 23:58:45 INFO Received push request method=POST path=/27742026/11/11 23:58:45 INFO Vacuumed table table=closures27752026/11/11 23:58:45 INFO Received uploads request method=POST path=/27762026/11/11 23:58:45 INFO Vacuumed table table=objects27772026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027782026/11/11 23:58:45 INFO Vacuumed table table=pending_closures27792026/11/11 23:58:45 INFO Received complete push request method=POST path=/27802026/11/11 23:58:45 INFO Vacuumed table table=pending_objects27812026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads27822026/11/11 23:58:45 INFO Vacuumed table table=closures27832026/11/11 23:58:45 INFO Received uploads request method=POST path=/27842026/11/11 23:58:45 INFO Completed push id=8 objects=6 duration=2.029419ms27852026/11/11 23:58:45 INFO Received complete upload request method=POST path=/27862026/11/11 23:58:45 INFO Received complete push request method=POST path=/27872026/11/11 23:58:45 INFO Vacuumed table table=objects27882026/11/11 23:58:45 INFO Completed push id=8 objects=13 duration=2.173321ms27892026/11/11 23:58:45 INFO Completed upload id=3 objects=8 duration=2.859167ms27902026/11/11 23:58:45 INFO Received push request method=POST path=/27912026/11/11 23:58:45 INFO Received uploads request method=POST path=/27922026/11/11 23:58:45 INFO Received push request method=POST path=/27932026/11/11 23:58:45 INFO Received push request method=POST path=/27942026/11/11 23:58:45 INFO Received push request method=POST path=/27952026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=027962026/11/11 23:58:45 INFO Received complete push request method=POST path=/27972026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=027982026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period27992026/11/11 23:58:45 INFO Completed push id=2 objects=10 duration=3.254591ms28002026/11/11 23:58:45 INFO Received push request method=POST path=/28012026/11/11 23:58:45 INFO Received uploads request method=POST path=/28022026/11/11 23:58:45 INFO Received push request method=POST path=/28032026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=4 objects-failed-to-delete=028042026/11/11 23:58:45 INFO Vacuumed table table=pending_closures28052026/11/11 23:58:45 INFO Received uploads request method=POST path=/28062026/11/11 23:58:45 INFO Vacuumed table table=pending_objects28072026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads28082026/11/11 23:58:45 INFO Vacuumed table table=closures28092026/11/11 23:58:45 INFO Vacuumed table table=objects28102026/11/11 23:58:45 INFO Received push request method=POST path=/28112026/11/11 23:58:45 INFO Received complete push request method=POST path=/28122026/11/11 23:58:45 INFO Received push request method=POST path=/28132026/11/11 23:58:45 INFO Completed push id=5 objects=3 duration=1.840197ms28142026/11/11 23:58:45 INFO Received complete upload request method=POST path=/28152026/11/11 23:58:45 INFO Received push request method=POST path=/28162026/11/11 23:58:45 INFO Completed upload id=10 objects=6 duration=2.560144ms28172026/11/11 23:58:45 INFO Received uploads request method=POST path=/28182026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=028192026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=028202026/11/11 23:58:45 INFO Received complete push request method=POST path=/28212026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period28222026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=028232026/11/11 23:58:45 INFO Completed push id=8 objects=7 duration=1.234712ms28242026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028252026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028262026/11/11 23:58:45 INFO Received push request method=POST path=/28272026/11/11 23:58:45 INFO Vacuumed table table=pending_closures28282026/11/11 23:58:45 INFO Vacuumed table table=pending_closures28292026/11/11 23:58:45 INFO Vacuumed table table=pending_objects28302026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads28312026/11/11 23:58:45 INFO Vacuumed table table=pending_objects28322026/11/11 23:58:45 INFO Vacuumed table table=closures28332026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads28342026/11/11 23:58:45 INFO Vacuumed table table=closures28352026/11/11 23:58:45 INFO Vacuumed table table=objects28362026/11/11 23:58:45 INFO Vacuumed table table=objects28372026/11/11 23:58:45 INFO Received uploads request method=POST path=/28382026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=028392026/11/11 23:58:45 INFO Received push request method=POST path=/28402026/11/11 23:58:45 INFO Received complete push request method=POST path=/28412026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=028422026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028432026/11/11 23:58:45 INFO Vacuumed table table=pending_closures28442026/11/11 23:58:45 INFO Completed push id=4 objects=4 duration=2.723266ms28452026/11/11 23:58:45 INFO Vacuumed table table=pending_objects28462026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads28472026/11/11 23:58:45 INFO Vacuumed table table=closures28482026/11/11 23:58:45 INFO Vacuumed table table=objects28492026/11/11 23:58:45 INFO Received uploads request method=POST path=/28502026/11/11 23:58:45 INFO Received complete upload request method=POST path=/28512026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=028522026/11/11 23:58:45 INFO Completed upload id=10 objects=17 duration=1.280712ms28532026/11/11 23:58:45 INFO Received complete upload request method=POST path=/28542026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028552026/11/11 23:58:45 INFO Completed upload id=9 objects=16 duration=1.263692ms28562026/11/11 23:58:45 INFO Vacuumed table table=pending_closures28572026/11/11 23:58:45 INFO Vacuumed table table=pending_objects28582026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads28592026/11/11 23:58:45 INFO Vacuumed table table=closures28602026/11/11 23:58:45 INFO Received complete push request method=POST path=/28612026/11/11 23:58:45 INFO Vacuumed table table=objects28622026/11/11 23:58:45 INFO Completed push id=6 objects=6 duration=1.826497ms28632026/11/11 23:58:45 INFO Received push request method=POST path=/28642026/11/11 23:58:45 INFO Received push request method=POST path=/28652026/11/11 23:58:45 INFO Received push request method=POST path=/28662026/11/11 23:58:45 INFO Received push request method=POST path=/28672026/11/11 23:58:45 INFO Received uploads request method=POST path=/28682026/11/11 23:58:45 INFO Received push request method=POST path=/28692026/11/11 23:58:45 INFO Received uploads request method=POST path=/28702026/11/11 23:58:45 INFO Received push request method=POST path=/28712026/11/11 23:58:45 INFO Received complete push request method=POST path=/28722026/11/11 23:58:45 INFO Completed push id=10 objects=3 duration=1.608816ms28732026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=028742026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period28752026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028762026/11/11 23:58:45 INFO Received complete push request method=POST path=/28772026/11/11 23:58:45 INFO Vacuumed table table=pending_closures28782026/11/11 23:58:45 INFO Received complete upload request method=POST path=/28792026/11/11 23:58:45 INFO Vacuumed table table=pending_objects28802026/11/11 23:58:45 INFO Completed push id=14 objects=6 duration=2.018ms28812026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads28822026/11/11 23:58:45 INFO Received complete push request method=POST path=/28832026/11/11 23:58:45 INFO Vacuumed table table=closures28842026/11/11 23:58:45 INFO Vacuumed table table=objects28852026/11/11 23:58:45 INFO Completed upload id=11 objects=6 duration=2.423203ms28862026/11/11 23:58:45 INFO Completed push id=7 objects=27 duration=1.702996ms28872026/11/11 23:58:45 INFO Received uploads request method=POST path=/28882026/11/11 23:58:45 INFO Received uploads request method=POST path=/28892026/11/11 23:58:45 INFO Received push request method=POST path=/28902026/11/11 23:58:45 INFO Received push request method=POST path=/28912026/11/11 23:58:45 INFO Received push request method=POST path=/28922026/11/11 23:58:45 INFO Received uploads request method=POST path=/28932026/11/11 23:58:45 INFO Received push request method=POST path=/28942026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=028952026/11/11 23:58:45 INFO Received complete upload request method=POST path=/28962026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028972026/11/11 23:58:45 INFO Completed upload id=3 objects=9 duration=1.468314ms28982026/11/11 23:58:45 INFO Vacuumed table table=pending_closures28992026/11/11 23:58:45 INFO Vacuumed table table=pending_objects29002026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads29012026/11/11 23:58:45 INFO Vacuumed table table=closures29022026/11/11 23:58:45 INFO Vacuumed table table=objects29032026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=029042026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period29052026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=029062026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029072026/11/11 23:58:45 INFO Received push request method=POST path=/29082026/11/11 23:58:45 INFO Vacuumed table table=pending_closures29092026/11/11 23:58:45 INFO Vacuumed table table=pending_objects29102026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads29112026/11/11 23:58:45 INFO Received push request method=POST path=/29122026/11/11 23:58:45 INFO Vacuumed table table=closures29132026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=029142026/11/11 23:58:45 INFO Vacuumed table table=objects29152026/11/11 23:58:45 INFO Vacuumed table table=pending_closures29162026/11/11 23:58:45 INFO Vacuumed table table=pending_objects29172026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads29182026/11/11 23:58:45 INFO Vacuumed table table=closures29192026/11/11 23:58:45 INFO Vacuumed table table=objects29202026/11/11 23:58:45 INFO Received complete push request method=POST path=/29212026/11/11 23:58:45 INFO Completed push id=9 objects=5 duration=3.633695ms29222026/11/11 23:58:45 INFO Received uploads request method=POST path=/29232026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=029242026/11/11 23:58:45 INFO Received complete push request method=POST path=/29252026/11/11 23:58:45 INFO Completed push id=5 objects=6 duration=1.588355ms29262026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029272026/11/11 23:58:45 INFO Vacuumed table table=pending_closures29282026/11/11 23:58:45 INFO Vacuumed table table=pending_objects29292026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads29302026/11/11 23:58:45 INFO Vacuumed table table=closures29312026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=029322026/11/11 23:58:45 INFO Vacuumed table table=objects29332026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period29342026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029352026/11/11 23:58:45 INFO Vacuumed table table=pending_closures29362026/11/11 23:58:45 INFO Vacuumed table table=pending_objects29372026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads29382026/11/11 23:58:45 INFO Vacuumed table table=closures29392026/11/11 23:58:45 INFO Vacuumed table table=objects29402026/11/11 23:58:45 INFO Received push request method=POST path=/29412026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=029422026/11/11 23:58:45 INFO Received push request method=POST path=/29432026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period29442026/11/11 23:58:45 INFO Received uploads request method=POST path=/29452026/11/11 23:58:45 INFO Received uploads request method=POST path=/29462026/11/11 23:58:45 INFO Received uploads request method=POST path=/29472026/11/11 23:58:45 INFO Received push request method=POST path=/29482026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029492026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=029502026/11/11 23:58:45 INFO Vacuumed table table=pending_closures29512026/11/11 23:58:45 INFO Vacuumed table table=pending_closures29522026/11/11 23:58:45 INFO Vacuumed table table=pending_objects29532026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads29542026/11/11 23:58:45 INFO Vacuumed table table=closures29552026/11/11 23:58:45 INFO Vacuumed table table=pending_objects29562026/11/11 23:58:45 INFO Vacuumed table table=objects29572026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads29582026/11/11 23:58:45 INFO Vacuumed table table=closures29592026/11/11 23:58:45 INFO Vacuumed table table=objects29602026/11/11 23:58:45 INFO Received uploads request method=POST path=/29612026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period29622026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period29632026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=029642026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029652026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029662026/11/11 23:58:45 INFO Vacuumed table table=pending_closures29672026/11/11 23:58:45 INFO Vacuumed table table=pending_closures29682026/11/11 23:58:45 INFO Vacuumed table table=pending_objects29692026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads29702026/11/11 23:58:45 INFO Vacuumed table table=closures29712026/11/11 23:58:45 INFO Vacuumed table table=objects29722026/11/11 23:58:45 INFO Vacuumed table table=pending_objects29732026/11/11 23:58:45 INFO Vacuumed table table=pending_closures29742026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads29752026/11/11 23:58:45 INFO Received push request method=POST path=/29762026/11/11 23:58:45 INFO Received uploads request method=POST path=/29772026/11/11 23:58:45 INFO Vacuumed table table=closures29782026/11/11 23:58:45 INFO Vacuumed table table=pending_objects29792026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads29802026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=029812026/11/11 23:58:45 INFO Vacuumed table table=objects29822026/11/11 23:58:45 INFO Vacuumed table table=closures29832026/11/11 23:58:45 INFO Received push request method=POST path=/29842026/11/11 23:58:45 INFO Vacuumed table table=objects29852026/11/11 23:58:45 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."29862026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."29872026/11/11 23:58:45 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."29882026/11/11 23:58:45 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."29892026/11/11 23:58:45 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."29902026/11/11 23:58:45 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."29912026/11/11 23:58:45 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."29922026/11/11 23:58:45 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."29932026/11/11 23:58:45 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."29942026/11/11 23:58:45 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."29952026/11/11 23:58:45 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."29962026/11/11 23:58:45 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."29972026/11/11 23:58:45 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."29982026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:aaaaaaaa!out.doi error="Access Denied."29992026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."30002026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."30012026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."30022026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."30032026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."30042026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."30052026/11/11 23:58:45 INFO Received complete push request method=POST path=/30062026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030072026/11/11 23:58:45 INFO Vacuumed table table=pending_closures30082026/11/11 23:58:45 INFO Completed push id=12 objects=2 duration=3.15557ms30092026/11/11 23:58:45 INFO Vacuumed table table=pending_objects30102026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads30112026/11/11 23:58:45 INFO Vacuumed table table=closures30122026/11/11 23:58:45 INFO Vacuumed table table=objects30132026/11/11 23:58:45 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."30142026/11/11 23:58:45 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."30152026/11/11 23:58:45 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."30162026/11/11 23:58:45 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."30172026/11/11 23:58:45 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."30182026/11/11 23:58:45 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."30192026/11/11 23:58:45 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."30202026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."30212026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."30222026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."30232026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."30242026/11/11 23:58:45 INFO Received uploads request method=POST path=/30252026/11/11 23:58:45 INFO Received push request method=POST path=/30262026/11/11 23:58:45 INFO Received complete upload request method=POST path=/30272026/11/11 23:58:45 INFO Received complete push request method=POST path=/30282026/11/11 23:58:45 INFO Received push request method=POST path=/30292026/11/11 23:58:45 INFO Completed upload id=9 objects=7 duration=2.10236ms30302026/11/11 23:58:45 INFO Completed push id=6 objects=5 duration=3.626095ms30312026/11/11 23:58:45 INFO Received uploads request method=POST path=/30322026/11/11 23:58:45 INFO Received push request method=POST path=/30332026/11/11 23:58:45 INFO Received push request method=POST path=/30342026/11/11 23:58:45 INFO Received uploads request method=POST path=/30352026/11/11 23:58:45 INFO Received uploads request method=POST path=/30362026/11/11 23:58:45 INFO Received push request method=POST path=/30372026/11/11 23:58:45 INFO Received complete push request method=POST path=/30382026/11/11 23:58:45 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."30392026/11/11 23:58:45 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."30402026/11/11 23:58:45 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."30412026/11/11 23:58:45 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."30422026/11/11 23:58:45 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."30432026/11/11 23:58:45 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."30442026/11/11 23:58:45 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."30452026/11/11 23:58:45 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."30462026/11/11 23:58:45 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."30472026/11/11 23:58:45 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."30482026/11/11 23:58:45 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."30492026/11/11 23:58:45 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."30502026/11/11 23:58:45 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."30512026/11/11 23:58:45 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."30522026/11/11 23:58:45 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."30532026/11/11 23:58:45 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."30542026/11/11 23:58:45 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."30552026/11/11 23:58:45 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."30562026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."30572026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."30582026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."30592026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."30602026/11/11 23:58:45 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."30612026/11/11 23:58:45 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."30622026/11/11 23:58:45 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."30632026/11/11 23:58:45 INFO Received push request method=POST path=/30642026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=030652026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=030662026/11/11 23:58:45 INFO Completed push id=9 objects=7 duration=3.623235ms30672026/11/11 23:58:45 INFO Vacuumed table table=pending_closures30682026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030692026/11/11 23:58:45 INFO Vacuumed table table=pending_objects30702026/11/11 23:58:45 INFO Vacuumed table table=pending_closures30712026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads30722026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=030732026/11/11 23:58:45 INFO Vacuumed table table=closures30742026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period30752026/11/11 23:58:45 INFO Vacuumed table table=pending_objects30762026/11/11 23:58:45 INFO Received complete push request method=POST path=/30772026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads30782026/11/11 23:58:45 INFO Vacuumed table table=closures30792026/11/11 23:58:45 INFO Vacuumed table table=objects30802026/11/11 23:58:45 INFO Vacuumed table table=objects30812026/11/11 23:58:45 INFO Completed push id=8 objects=2 duration=2.204401ms30822026/11/11 23:58:45 INFO Received uploads request method=POST path=/30832026/11/11 23:58:45 INFO Received uploads request method=POST path=/30842026/11/11 23:58:45 INFO Received uploads request method=POST path=/30852026/11/11 23:58:45 INFO Received complete upload request method=POST path=/30862026/11/11 23:58:45 INFO Completed upload id=10 objects=7 duration=1.648996ms30872026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=030882026/11/11 23:58:45 INFO Received complete upload request method=POST path=/30892026/11/11 23:58:45 INFO Vacuumed table table=pending_closures30902026/11/11 23:58:45 INFO Vacuumed table table=pending_objects30912026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads30922026/11/11 23:58:45 INFO Completed upload id=17 objects=6 duration=2.941368ms30932026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=030942026/11/11 23:58:45 INFO Vacuumed table table=closures30952026/11/11 23:58:45 INFO Vacuumed table table=objects30962026/11/11 23:58:45 INFO Received push request method=POST path=/30972026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030982026/11/11 23:58:45 INFO Vacuumed table table=pending_closures30992026/11/11 23:58:45 INFO Received uploads request method=POST path=/31002026/11/11 23:58:45 INFO Vacuumed table table=pending_objects31012026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads31022026/11/11 23:58:45 INFO Vacuumed table table=closures31032026/11/11 23:58:45 INFO Vacuumed table table=objects31042026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=031052026/11/11 23:58:45 INFO Received push request method=POST path=/31062026/11/11 23:58:45 INFO Received uploads request method=POST path=/31072026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=031082026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period31092026/11/11 23:58:45 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."31102026/11/11 23:58:45 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."31112026/11/11 23:58:45 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."31122026/11/11 23:58:45 INFO Received push request method=POST path=/31132026/11/11 23:58:45 INFO Received push request method=POST path=/31142026/11/11 23:58:45 INFO Received complete upload request method=POST path=/31152026/11/11 23:58:45 INFO Received push request method=POST path=/31162026/11/11 23:58:45 INFO Completed upload id=6 objects=24 duration=2.716585ms31172026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=031182026/11/11 23:58:45 INFO Received push request method=POST path=/31192026/11/11 23:58:45 INFO Received push request method=POST path=/31202026/11/11 23:58:45 INFO Received complete push request method=POST path=/31212026/11/11 23:58:45 INFO Received push request method=POST path=/31222026/11/11 23:58:45 INFO Received uploads request method=POST path=/31232026/11/11 23:58:45 INFO Received uploads request method=POST path=/31242026/11/11 23:58:45 INFO Completed push id=15 objects=24 duration=3.653135ms31252026/11/11 23:58:45 INFO Received uploads request method=POST path=/31262026/11/11 23:58:45 INFO Received push request method=POST path=/31272026/11/11 23:58:45 INFO Received push request method=POST path=/31282026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=031292026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period31302026/11/11 23:58:45 INFO Received push request method=POST path=/31312026/11/11 23:58:45 INFO Received complete upload request method=POST path=/31322026/11/11 23:58:45 INFO Received uploads request method=POST path=/31332026/11/11 23:58:45 INFO Received uploads request method=POST path=/31342026/11/11 23:58:45 INFO Completed upload id=18 objects=4 duration=7.831674ms31352026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=031362026/11/11 23:58:45 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."31372026/11/11 23:58:45 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."31382026/11/11 23:58:45 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."31392026/11/11 23:58:45 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."31402026/11/11 23:58:45 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."31412026/11/11 23:58:45 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."31422026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."31432026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."31442026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."31452026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031462026/11/11 23:58:45 INFO Vacuumed table table=pending_closures31472026/11/11 23:58:45 INFO Received complete push request method=POST path=/31482026/11/11 23:58:45 INFO Vacuumed table table=pending_objects31492026/11/11 23:58:45 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."31502026/11/11 23:58:45 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."31512026/11/11 23:58:45 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."31522026/11/11 23:58:45 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."31532026/11/11 23:58:45 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."31542026/11/11 23:58:45 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."31552026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."31562026/11/11 23:58:45 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."31572026/11/11 23:58:45 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."31582026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."31592026/11/11 23:58:45 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."31602026/11/11 23:58:45 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."31612026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads31622026/11/11 23:58:45 INFO Completed push id=8 objects=6 duration=1.970719ms31632026/11/11 23:58:45 INFO Received push request method=POST path=/31642026/11/11 23:58:45 INFO Vacuumed table table=closures31652026/11/11 23:58:45 INFO Vacuumed table table=objects31662026/11/11 23:58:45 INFO Received complete upload request method=POST path=/31672026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=031682026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period31692026/11/11 23:58:45 INFO Received uploads request method=POST path=/31702026/11/11 23:58:45 INFO Received complete push request method=POST path=/31712026/11/11 23:58:45 INFO Received uploads request method=POST path=/31722026/11/11 23:58:45 INFO Received uploads request method=POST path=/31732026/11/11 23:58:45 INFO Received uploads request method=POST path=/31742026/11/11 23:58:45 INFO Completed upload id=1 objects=15 duration=9.421429ms31752026/11/11 23:58:45 INFO Completed push id=10 objects=2 duration=5.378112ms31762026/11/11 23:58:45 INFO Received push request method=POST path=/31772026/11/11 23:58:45 INFO Received complete push request method=POST path=/31782026/11/11 23:58:45 INFO Completed push id=22 objects=3 duration=2.562605ms31792026/11/11 23:58:45 INFO Received complete upload request method=POST path=/31802026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=9 objects-failed-to-delete=031812026/11/11 23:58:45 INFO Completed upload id=9 objects=3 duration=1.267972ms31822026/11/11 23:58:45 INFO Received push request method=POST path=/31832026/11/11 23:58:45 INFO Received push request method=POST path=/31842026/11/11 23:58:45 INFO Vacuumed table table=pending_closures31852026/11/11 23:58:45 INFO Received uploads request method=POST path=/31862026/11/11 23:58:45 INFO Vacuumed table table=pending_objects31872026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads31882026/11/11 23:58:45 INFO Received uploads request method=POST path=/31892026/11/11 23:58:45 INFO Vacuumed table table=closures31902026/11/11 23:58:45 INFO Vacuumed table table=objects31912026/11/11 23:58:45 INFO Received uploads request method=POST path=/31922026/11/11 23:58:45 INFO Received complete upload request method=POST path=/31932026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=031942026/11/11 23:58:45 INFO Received uploads request method=POST path=/31952026/11/11 23:58:45 INFO Completed upload id=12 objects=21 duration=1.297293ms31962026/11/11 23:58:45 INFO Received push request method=POST path=/31972026/11/11 23:58:45 INFO Received push request method=POST path=/31982026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period31992026/11/11 23:58:45 INFO Received push request method=POST path=/32002026/11/11 23:58:45 INFO Received push request method=POST path=/32012026/11/11 23:58:45 INFO Received push request method=POST path=/32022026/11/11 23:58:45 INFO Received uploads request method=POST path=/32032026/11/11 23:58:45 INFO Received uploads request method=POST path=/32042026/11/11 23:58:45 INFO Received push request method=POST path=/32052026/11/11 23:58:45 INFO Received uploads request method=POST path=/32062026/11/11 23:58:45 INFO Received push request method=POST path=/32072026/11/11 23:58:45 INFO Received push request method=POST path=/32082026/11/11 23:58:45 INFO Received push request method=POST path=/32092026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=032102026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."32112026/11/11 23:58:45 INFO Received push request method=POST path=/32122026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=032132026/11/11 23:58:45 INFO Received uploads request method=POST path=/32142026/11/11 23:58:45 INFO Received complete upload request method=POST path=/32152026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=032162026/11/11 23:58:45 INFO Received complete upload request method=POST path=/32172026/11/11 23:58:45 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."32182026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."32192026/11/11 23:58:45 INFO Completed upload id=6 objects=6 duration=1.200012ms32202026/11/11 23:58:45 INFO Received push request method=POST path=/32212026/11/11 23:58:45 INFO Completed upload id=10 objects=23 duration=2.656565ms32222026/11/11 23:58:45 INFO Received complete push request method=POST path=/32232026/11/11 23:58:45 INFO Completed push id=13 objects=23 duration=1.968959ms32242026/11/11 23:58:45 INFO Received uploads request method=POST path=/32252026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=032262026/11/11 23:58:45 INFO Received uploads request method=POST path=/32272026/11/11 23:58:45 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."32282026/11/11 23:58:45 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."32292026/11/11 23:58:45 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."32302026/11/11 23:58:45 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."32312026/11/11 23:58:45 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."32322026/11/11 23:58:45 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."32332026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."32342026/11/11 23:58:45 INFO Vacuumed table table=pending_closures32352026/11/11 23:58:45 INFO Received complete upload request method=POST path=/32362026/11/11 23:58:45 INFO Vacuumed table table=pending_objects32372026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads32382026/11/11 23:58:45 INFO Completed upload id=15 objects=9 duration=1.522554ms32392026/11/11 23:58:45 INFO Received uploads request method=POST path=/32402026/11/11 23:58:45 INFO Vacuumed table table=closures32412026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=032422026/11/11 23:58:45 INFO Vacuumed table table=objects32432026/11/11 23:58:45 INFO Received push request method=POST path=/32442026/11/11 23:58:45 WARN Force mode enabled - objects will be deleted immediately without grace period32452026/11/11 23:58:45 INFO Received push request method=POST path=/32462026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=032472026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=032482026/11/11 23:58:45 INFO Received uploads request method=POST path=/32492026/11/11 23:58:45 INFO Received complete push request method=POST path=/32502026/11/11 23:58:45 INFO Received push request method=POST path=/32512026/11/11 23:58:45 INFO Completed push id=1 objects=11 duration=2.145641ms32522026/11/11 23:58:45 INFO Received uploads request method=POST path=/32532026/11/11 23:58:45 INFO Received uploads request method=POST path=/32542026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=032552026/11/11 23:58:45 INFO Vacuumed table table=pending_closures32562026/11/11 23:58:45 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."32572026/11/11 23:58:45 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."32582026/11/11 23:58:45 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."32592026/11/11 23:58:45 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."32602026/11/11 23:58:45 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."32612026/11/11 23:58:45 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."32622026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."32632026/11/11 23:58:45 INFO Vacuumed table table=pending_objects32642026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads32652026/11/11 23:58:45 INFO Vacuumed table table=closures32662026/11/11 23:58:45 INFO Vacuumed table table=objects32672026/11/11 23:58:45 INFO Received uploads request method=POST path=/32682026/11/11 23:58:45 INFO Received complete upload request method=POST path=/32692026/11/11 23:58:45 INFO Received complete upload request method=POST path=/32702026/11/11 23:58:45 INFO Received complete push request method=POST path=/32712026/11/11 23:58:45 INFO Completed upload id=11 objects=6 duration=1.895778ms32722026/11/11 23:58:45 INFO Completed upload id=10 objects=2 duration=3.401093ms32732026/11/11 23:58:45 INFO Received uploads request method=POST path=/32742026/11/11 23:58:45 INFO Completed push id=12 objects=3 duration=2.343082ms32752026/11/11 23:58:45 INFO Received push request method=POST path=/32762026/11/11 23:58:45 INFO Received complete upload request method=POST path=/32772026/11/11 23:58:45 INFO Received complete push request method=POST path=/32782026/11/11 23:58:45 INFO Completed upload id=16 objects=3 duration=1.278573ms32792026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=032802026/11/11 23:58:45 INFO Completed push id=21 objects=17 duration=2.269242ms32812026/11/11 23:58:45 INFO Received push request method=POST path=/32822026/11/11 23:58:45 INFO Received push request method=POST path=/32832026/11/11 23:58:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=032842026/11/11 23:58:45 INFO Received push request method=POST path=/32852026/11/11 23:58:45 INFO Received uploads request method=POST path=/32862026/11/11 23:58:45 INFO Received push request method=POST path=/32872026/11/11 23:58:45 INFO Vacuumed table table=pending_closures32882026/11/11 23:58:45 INFO Vacuumed table table=pending_objects32892026/11/11 23:58:45 INFO Received push request method=POST path=/32902026/11/11 23:58:45 INFO Vacuumed table table=multipart_uploads32912026/11/11 23:58:45 INFO Vacuumed table table=closures32922026/11/11 23:58:45 INFO Received push request method=POST path=/32932026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=032942026/11/11 23:58:45 INFO Vacuumed table table=objects32952026/11/11 23:58:45 INFO Received complete push request method=POST path=/32962026/11/11 23:58:45 INFO Completed push id=19 objects=7 duration=1.480514ms32972026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=032982026/11/11 23:58:45 INFO Received uploads request method=POST path=/32992026/11/11 23:58:45 INFO Received uploads request method=POST path=/33002026/11/11 23:58:45 INFO Received complete push request method=POST path=/33012026/11/11 23:58:45 INFO Received push request method=POST path=/33022026/11/11 23:58:45 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."33032026/11/11 23:58:45 INFO Received push request method=POST path=/33042026/11/11 23:58:45 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."33052026/11/11 23:58:45 INFO Received push request method=POST path=/33062026/11/11 23:58:45 INFO Completed push id=13 objects=12 duration=3.657895ms33072026/11/11 23:58:45 INFO Received push request method=POST path=/33082026/11/11 23:58:45 INFO Received push request method=POST path=/33092026/11/11 23:58:45 INFO Received complete push request method=POST path=/33102026/11/11 23:58:45 INFO Received uploads request method=POST path=/33112026/11/11 23:58:45 INFO Completed push id=12 objects=15 duration=2.343263ms33122026/11/11 23:58:45 INFO Aborted multipart uploads count=0 kept=033132026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=033142026/11/11 23:58:46 INFO Received push request method=POST path=/33152026/11/11 23:58:46 WARN Force mode enabled - objects will be deleted immediately without grace period33162026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=033172026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033182026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=033192026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=033202026/11/11 23:58:46 WARN Force mode enabled - objects will be deleted immediately without grace period33212026/11/11 23:58:46 INFO Vacuumed table table=pending_closures33222026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=033232026/11/11 23:58:46 WARN Force mode enabled - objects will be deleted immediately without grace period33242026/11/11 23:58:46 INFO Vacuumed table table=pending_objects33252026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads33262026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033272026/11/11 23:58:46 INFO Vacuumed table table=pending_closures33282026/11/11 23:58:46 INFO Vacuumed table table=closures33292026/11/11 23:58:46 INFO Vacuumed table table=objects33302026/11/11 23:58:46 INFO Vacuumed table table=pending_closures33312026/11/11 23:58:46 INFO Vacuumed table table=pending_objects33322026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads33332026/11/11 23:58:46 INFO Vacuumed table table=pending_objects33342026/11/11 23:58:46 INFO Vacuumed table table=closures33352026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads33362026/11/11 23:58:46 INFO Vacuumed table table=closures33372026/11/11 23:58:46 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."33382026/11/11 23:58:46 INFO Vacuumed table table=objects33392026/11/11 23:58:46 INFO Vacuumed table table=objects33402026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=033412026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=033422026/11/11 23:58:46 INFO Vacuumed table table=pending_closures33432026/11/11 23:58:46 WARN Force mode enabled - objects will be deleted immediately without grace period33442026/11/11 23:58:46 INFO Received uploads request method=POST path=/33452026/11/11 23:58:46 INFO Vacuumed table table=pending_objects33462026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads33472026/11/11 23:58:46 INFO Vacuumed table table=closures33482026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033492026/11/11 23:58:46 INFO Vacuumed table table=pending_closures33502026/11/11 23:58:46 INFO Vacuumed table table=objects33512026/11/11 23:58:46 INFO Received uploads request method=POST path=/33522026/11/11 23:58:46 INFO Vacuumed table table=pending_objects33532026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads33542026/11/11 23:58:46 INFO Vacuumed table table=closures33552026/11/11 23:58:46 INFO Received uploads request method=POST path=/33562026/11/11 23:58:46 INFO Vacuumed table table=objects33572026/11/11 23:58:46 INFO Received push request method=POST path=/33582026/11/11 23:58:46 INFO Received uploads request method=POST path=/33592026/11/11 23:58:46 INFO Received complete upload request method=POST path=/33602026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=033612026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=033622026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=033632026/11/11 23:58:46 INFO Completed upload id=21 objects=7 duration=3.300731ms33642026/11/11 23:58:46 WARN Force mode enabled - objects will be deleted immediately without grace period33652026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033662026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033672026/11/11 23:58:46 INFO Vacuumed table table=pending_closures33682026/11/11 23:58:46 INFO Vacuumed table table=pending_closures33692026/11/11 23:58:46 INFO Vacuumed table table=pending_objects33702026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads33712026/11/11 23:58:46 INFO Received complete upload request method=POST path=/33722026/11/11 23:58:46 INFO Vacuumed table table=pending_objects33732026/11/11 23:58:46 INFO Vacuumed table table=closures33742026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads33752026/11/11 23:58:46 INFO Vacuumed table table=closures33762026/11/11 23:58:46 INFO Vacuumed table table=objects33772026/11/11 23:58:46 INFO Vacuumed table table=objects33782026/11/11 23:58:46 INFO Completed upload id=16 objects=23 duration=4.757666ms33792026/11/11 23:58:46 INFO Received complete upload request method=POST path=/33802026/11/11 23:58:46 INFO Received complete upload request method=POST path=/33812026/11/11 23:58:46 INFO Completed upload id=15 objects=4 duration=2.324922ms33822026/11/11 23:58:46 INFO Completed upload id=12 objects=26 duration=3.057549ms33832026/11/11 23:58:46 INFO Received push request method=POST path=/33842026/11/11 23:58:46 INFO Received complete upload request method=POST path=/33852026/11/11 23:58:46 INFO Received push request method=POST path=/33862026/11/11 23:58:46 INFO Received push request method=POST path=/33872026/11/11 23:58:46 INFO Completed upload id=15 objects=6 duration=5.440912ms33882026/11/11 23:58:46 INFO Received push request method=POST path=/33892026/11/11 23:58:46 INFO Received complete push request method=POST path=/33902026/11/11 23:58:46 INFO Received complete push request method=POST path=/33912026/11/11 23:58:46 INFO Completed push id=4 objects=6 duration=3.344332ms33922026/11/11 23:58:46 INFO Completed push id=19 objects=6 duration=3.842037ms33932026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=033942026/11/11 23:58:46 INFO Received uploads request method=POST path=/33952026/11/11 23:58:46 INFO Received uploads request method=POST path=/33962026/11/11 23:58:46 INFO Vacuumed table table=pending_closures33972026/11/11 23:58:46 INFO Received uploads request method=POST path=/33982026/11/11 23:58:46 INFO Vacuumed table table=pending_objects33992026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads34002026/11/11 23:58:46 INFO Vacuumed table table=closures34012026/11/11 23:58:46 INFO Received uploads request method=POST path=/34022026/11/11 23:58:46 INFO Received uploads request method=POST path=/34032026/11/11 23:58:46 INFO Vacuumed table table=objects34042026/11/11 23:58:46 INFO Received uploads request method=POST path=/34052026/11/11 23:58:46 INFO Received complete push request method=POST path=/34062026/11/11 23:58:46 INFO Received uploads request method=POST path=/34072026/11/11 23:58:46 INFO Received complete upload request method=POST path=/34082026/11/11 23:58:46 INFO Completed push id=9 objects=4 duration=3.297672ms34092026/11/11 23:58:46 INFO Completed upload id=16 objects=3 duration=1.694156ms34102026/11/11 23:58:46 INFO Received complete upload request method=POST path=/34112026/11/11 23:58:46 INFO Completed upload id=17 objects=7 duration=1.471974ms34122026/11/11 23:58:46 INFO Received push request method=POST path=/34132026/11/11 23:58:46 INFO Received uploads request method=POST path=/34142026/11/11 23:58:46 INFO Received complete push request method=POST path=/34152026/11/11 23:58:46 INFO Completed push id=16 objects=8 duration=1.591195ms34162026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=034172026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=034182026/11/11 23:58:46 INFO Received complete push request method=POST path=/34192026/11/11 23:58:46 INFO Received complete upload request method=POST path=/34202026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=034212026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=034222026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034232026/11/11 23:58:46 INFO Completed upload id=27 objects=4 duration=1.169951ms34242026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=034252026/11/11 23:58:46 INFO Vacuumed table table=pending_closures34262026/11/11 23:58:46 WARN Force mode enabled - objects will be deleted immediately without grace period34272026/11/11 23:58:46 WARN Force mode enabled - objects will be deleted immediately without grace period34282026/11/11 23:58:46 WARN Force mode enabled - objects will be deleted immediately without grace period34292026/11/11 23:58:46 INFO Vacuumed table table=pending_objects34302026/11/11 23:58:46 INFO Completed push id=3 objects=21 duration=6.205279ms34312026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads34322026/11/11 23:58:46 INFO Vacuumed table table=closures34332026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034342026/11/11 23:58:46 INFO Vacuumed table table=objects34352026/11/11 23:58:46 INFO Vacuumed table table=pending_closures34362026/11/11 23:58:46 INFO Vacuumed table table=pending_objects34372026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads34382026/11/11 23:58:46 INFO Vacuumed table table=closures34392026/11/11 23:58:46 INFO Vacuumed table table=objects34402026/11/11 23:58:46 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."34412026/11/11 23:58:46 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."34422026/11/11 23:58:46 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."34432026/11/11 23:58:46 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."34442026/11/11 23:58:46 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."34452026/11/11 23:58:46 INFO Received complete push request method=POST path=/34462026/11/11 23:58:46 INFO Completed push id=13 objects=11 duration=1.291032ms34472026/11/11 23:58:46 INFO Received uploads request method=POST path=/34482026/11/11 23:58:46 INFO Received uploads request method=POST path=/34492026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=034502026/11/11 23:58:46 WARN Force mode enabled - objects will be deleted immediately without grace period34512026/11/11 23:58:46 INFO Received complete push request method=POST path=/34522026/11/11 23:58:46 INFO Received uploads request method=POST path=/34532026/11/11 23:58:46 INFO Received complete push request method=POST path=/34542026/11/11 23:58:46 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."34552026/11/11 23:58:46 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."34562026/11/11 23:58:46 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."34572026/11/11 23:58:46 INFO Completed push id=5 objects=9 duration=5.015368ms34582026/11/11 23:58:46 INFO Completed push id=11 objects=6 duration=3.489633ms34592026/11/11 23:58:46 INFO Received complete upload request method=POST path=/34602026/11/11 23:58:46 INFO Completed upload id=15 objects=6 duration=1.575795ms34612026/11/11 23:58:46 INFO Received push request method=POST path=/34622026/11/11 23:58:46 INFO Received push request method=POST path=/34632026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=034642026/11/11 23:58:46 INFO Received complete upload request method=POST path=/34652026/11/11 23:58:46 INFO Completed upload id=14 objects=6 duration=1.06375ms34662026/11/11 23:58:46 WARN Force mode enabled - objects will be deleted immediately without grace period34672026/11/11 23:58:46 INFO Received complete upload request method=POST path=/34682026/11/11 23:58:46 INFO Received uploads request method=POST path=/34692026/11/11 23:58:46 INFO Received push request method=POST path=/34702026/11/11 23:58:46 INFO Completed upload id=18 objects=4 duration=2.09472ms34712026/11/11 23:58:46 INFO Received complete push request method=POST path=/34722026/11/11 23:58:46 INFO Completed push id=18 objects=6 duration=1.273312ms34732026/11/11 23:58:46 INFO Received push request method=POST path=/34742026/11/11 23:58:46 INFO Received push request method=POST path=/34752026/11/11 23:58:46 INFO Received uploads request method=POST path=/34762026/11/11 23:58:46 INFO Received uploads request method=POST path=/34772026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=034782026/11/11 23:58:46 INFO Received complete push request method=POST path=/34792026/11/11 23:58:46 INFO Received complete push request method=POST path=/34802026/11/11 23:58:46 INFO Received complete upload request method=POST path=/34812026/11/11 23:58:46 INFO Completed push id=17 objects=6 duration=1.985259ms34822026/11/11 23:58:46 INFO Completed push id=15 objects=7 duration=2.747887ms34832026/11/11 23:58:46 INFO Completed upload id=7 objects=3 duration=2.219781ms34842026/11/11 23:58:46 INFO Received uploads request method=POST path=/34852026/11/11 23:58:46 INFO Received complete push request method=POST path=/34862026/11/11 23:58:46 INFO Completed push id=17 objects=22 duration=2.384423ms34872026/11/11 23:58:46 INFO Received uploads request method=POST path=/34882026/11/11 23:58:46 INFO Received push request method=POST path=/34892026/11/11 23:58:46 INFO Received complete push request method=POST path=/34902026/11/11 23:58:46 INFO Completed push id=16 objects=27 duration=1.921258ms34912026/11/11 23:58:46 INFO Received complete push request method=POST path=/34922026/11/11 23:58:46 INFO Completed push id=15 objects=5 duration=1.498155ms34932026/11/11 23:58:46 INFO Received complete push request method=POST path=/34942026/11/11 23:58:46 INFO Received uploads request method=POST path=/34952026/11/11 23:58:46 INFO Received push request method=POST path=/34962026/11/11 23:58:46 INFO Received uploads request method=POST path=/34972026/11/11 23:58:46 INFO Completed push id=18 objects=23 duration=2.339122ms34982026/11/11 23:58:46 INFO Received uploads request method=POST path=/34992026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=035002026/11/11 23:58:46 INFO Received uploads request method=POST path=/35012026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=035022026/11/11 23:58:46 WARN Force mode enabled - objects will be deleted immediately without grace period35032026/11/11 23:58:46 INFO Received push request method=POST path=/35042026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035052026/11/11 23:58:46 INFO Vacuumed table table=pending_closures35062026/11/11 23:58:46 INFO Vacuumed table table=pending_objects35072026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads35082026/11/11 23:58:46 INFO Vacuumed table table=closures35092026/11/11 23:58:46 INFO Vacuumed table table=objects35102026/11/11 23:58:46 INFO Received push request method=POST path=/35112026/11/11 23:58:46 INFO Received complete push request method=POST path=/35122026/11/11 23:58:46 INFO Completed push id=11 objects=9 duration=1.673256ms35132026/11/11 23:58:46 INFO Received complete push request method=POST path=/35142026/11/11 23:58:46 INFO Received complete upload request method=POST path=/35152026/11/11 23:58:46 INFO Completed push id=18 objects=10 duration=1.879178ms35162026/11/11 23:58:46 INFO Received push request method=POST path=/35172026/11/11 23:58:46 INFO Completed upload id=7 objects=10 duration=2.06456ms35182026/11/11 23:58:46 INFO Received push request method=POST path=/35192026/11/11 23:58:46 INFO Received complete upload request method=POST path=/35202026/11/11 23:58:46 INFO Completed upload id=22 objects=2 duration=2.342183ms35212026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=035222026/11/11 23:58:46 INFO Received push request method=POST path=/35232026/11/11 23:58:46 INFO Received uploads request method=POST path=/35242026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=035252026/11/11 23:58:46 INFO Received uploads request method=POST path=/35262026/11/11 23:58:46 WARN Force mode enabled - objects will be deleted immediately without grace period35272026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=035282026/11/11 23:58:46 INFO Vacuumed table table=pending_closures35292026/11/11 23:58:46 INFO Vacuumed table table=pending_objects35302026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads35312026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=035322026/11/11 23:58:46 INFO Vacuumed table table=closures35332026/11/11 23:58:46 INFO Vacuumed table table=objects35342026/11/11 23:58:46 WARN Force mode enabled - objects will be deleted immediately without grace period35352026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035362026/11/11 23:58:46 INFO Vacuumed table table=pending_closures35372026/11/11 23:58:46 INFO Vacuumed table table=pending_objects35382026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads35392026/11/11 23:58:46 INFO Vacuumed table table=closures35402026/11/11 23:58:46 INFO Vacuumed table table=objects35412026/11/11 23:58:46 INFO Received push request method=POST path=/35422026/11/11 23:58:46 INFO Received uploads request method=POST path=/35432026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=035442026/11/11 23:58:46 WARN Force mode enabled - objects will be deleted immediately without grace period35452026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035462026/11/11 23:58:46 INFO Vacuumed table table=pending_closures35472026/11/11 23:58:46 INFO Vacuumed table table=pending_objects35482026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads35492026/11/11 23:58:46 INFO Received uploads request method=POST path=/35502026/11/11 23:58:46 INFO Vacuumed table table=closures35512026/11/11 23:58:46 INFO Vacuumed table table=objects35522026/11/11 23:58:46 INFO Received complete upload request method=POST path=/35532026/11/11 23:58:46 INFO Completed upload id=11 objects=7 duration=2.151ms35542026/11/11 23:58:46 INFO Received push request method=POST path=/35552026/11/11 23:58:46 INFO Received push request method=POST path=/35562026/11/11 23:58:46 INFO Received uploads request method=POST path=/35572026/11/11 23:58:46 INFO Received push request method=POST path=/35582026/11/11 23:58:46 INFO Received uploads request method=POST path=/35592026/11/11 23:58:46 INFO Received push request method=POST path=/35602026/11/11 23:58:46 INFO Received push request method=POST path=/35612026/11/11 23:58:46 INFO Received uploads request method=POST path=/35622026/11/11 23:58:46 INFO Received push request method=POST path=/35632026/11/11 23:58:46 INFO Received complete upload request method=POST path=/35642026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=035652026/11/11 23:58:46 INFO Completed upload id=23 objects=15 duration=12.434679ms35662026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035672026/11/11 23:58:46 INFO Vacuumed table table=pending_closures35682026/11/11 23:58:46 INFO Vacuumed table table=pending_objects35692026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads35702026/11/11 23:58:46 INFO Vacuumed table table=closures35712026/11/11 23:58:46 INFO Vacuumed table table=objects35722026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=035732026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035742026/11/11 23:58:46 INFO Received complete push request method=POST path=/35752026/11/11 23:58:46 INFO Vacuumed table table=pending_closures35762026/11/11 23:58:46 INFO Received push request method=POST path=/35772026/11/11 23:58:46 INFO Vacuumed table table=pending_objects35782026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads35792026/11/11 23:58:46 INFO Vacuumed table table=closures35802026/11/11 23:58:46 INFO Vacuumed table table=objects35812026/11/11 23:58:46 INFO Completed push id=14 objects=3 duration=4.388742ms35822026/11/11 23:58:46 INFO Received complete upload request method=POST path=/35832026/11/11 23:58:46 INFO Completed upload id=19 objects=6 duration=1.923719ms35842026/11/11 23:58:46 INFO Received push request method=POST path=/35852026/11/11 23:58:46 INFO Received uploads request method=POST path=/35862026/11/11 23:58:46 INFO Received complete push request method=POST path=/35872026/11/11 23:58:46 INFO Completed push id=21 objects=9 duration=2.205761ms35882026/11/11 23:58:46 INFO Received push request method=POST path=/35892026/11/11 23:58:46 INFO Received push request method=POST path=/35902026/11/11 23:58:46 INFO Received complete upload request method=POST path=/35912026/11/11 23:58:46 INFO Completed upload id=22 objects=7 duration=1.208291ms35922026/11/11 23:58:46 INFO Received uploads request method=POST path=/35932026/11/11 23:58:46 INFO Received complete push request method=POST path=/35942026/11/11 23:58:46 INFO Completed push id=26 objects=14 duration=9.195547ms35952026/11/11 23:58:46 INFO Received uploads request method=POST path=/35962026/11/11 23:58:46 INFO Received complete push request method=POST path=/35972026/11/11 23:58:46 INFO Completed push id=13 objects=15 duration=2.014439ms35982026/11/11 23:58:46 INFO Received push request method=POST path=/35992026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=036002026/11/11 23:58:46 INFO Received uploads request method=POST path=/36012026/11/11 23:58:46 INFO Received complete push request method=POST path=/36022026/11/11 23:58:46 INFO Completed push id=24 objects=9 duration=1.978179ms36032026/11/11 23:58:46 INFO Received push request method=POST path=/36042026/11/11 23:58:46 INFO Received uploads request method=POST path=/36052026/11/11 23:58:46 INFO Received push request method=POST path=/36062026/11/11 23:58:46 INFO Received push request method=POST path=/36072026/11/11 23:58:46 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."36082026/11/11 23:58:46 INFO Received push request method=POST path=/36092026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=036102026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=036112026/11/11 23:58:46 INFO Received push request method=POST path=/36122026/11/11 23:58:46 WARN Force mode enabled - objects will be deleted immediately without grace period36132026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036142026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036152026/11/11 23:58:46 INFO Vacuumed table table=pending_closures36162026/11/11 23:58:46 INFO Vacuumed table table=pending_closures36172026/11/11 23:58:46 INFO Vacuumed table table=pending_objects36182026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads36192026/11/11 23:58:46 INFO Vacuumed table table=pending_objects36202026/11/11 23:58:46 INFO Vacuumed table table=closures36212026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads36222026/11/11 23:58:46 INFO Vacuumed table table=closures36232026/11/11 23:58:46 INFO Vacuumed table table=objects36242026/11/11 23:58:46 INFO Vacuumed table table=objects36252026/11/11 23:58:46 INFO Received uploads request method=POST path=/36262026/11/11 23:58:46 INFO Received push request method=POST path=/36272026/11/11 23:58:46 INFO Received complete push request method=POST path=/36282026/11/11 23:58:46 INFO Received uploads request method=POST path=/36292026/11/11 23:58:46 INFO Completed push id=20 objects=3 duration=2.859507ms36302026/11/11 23:58:46 INFO Received push request method=POST path=/36312026/11/11 23:58:46 INFO Received push request method=POST path=/36322026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=036332026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036342026/11/11 23:58:46 INFO Vacuumed table table=pending_closures36352026/11/11 23:58:46 INFO Vacuumed table table=pending_objects36362026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads36372026/11/11 23:58:46 INFO Vacuumed table table=closures36382026/11/11 23:58:46 INFO Received complete upload request method=POST path=/36392026/11/11 23:58:46 INFO Vacuumed table table=objects36402026/11/11 23:58:46 INFO Completed upload id=13 objects=8 duration=1.581115ms36412026/11/11 23:58:46 INFO Received push request method=POST path=/36422026/11/11 23:58:46 INFO Received complete push request method=POST path=/36432026/11/11 23:58:46 INFO Completed push id=21 objects=3 duration=1.875758ms36442026/11/11 23:58:46 INFO Received uploads request method=POST path=/36452026/11/11 23:58:46 INFO Received push request method=POST path=/36462026/11/11 23:58:46 INFO Received uploads request method=POST path=/36472026/11/11 23:58:46 INFO Received uploads request method=POST path=/36482026/11/11 23:58:46 INFO Received push request method=POST path=/36492026/11/11 23:58:46 INFO Received push request method=POST path=/36502026/11/11 23:58:46 INFO Received complete upload request method=POST path=/36512026/11/11 23:58:46 INFO Received push request method=POST path=/36522026/11/11 23:58:46 INFO Completed upload id=33 objects=5 duration=4.006798ms36532026/11/11 23:58:46 INFO Received complete upload request method=POST path=/36542026/11/11 23:58:46 INFO Completed upload id=22 objects=6 duration=1.160291ms36552026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=036562026/11/11 23:58:46 INFO Received complete push request method=POST path=/36572026/11/11 23:58:46 INFO Completed push id=27 objects=25 duration=4.972408ms36582026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=3 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036592026/11/11 23:58:46 INFO Vacuumed table table=pending_closures36602026/11/11 23:58:46 INFO Vacuumed table table=pending_objects36612026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads36622026/11/11 23:58:46 INFO Vacuumed table table=closures36632026/11/11 23:58:46 INFO Vacuumed table table=objects36642026/11/11 23:58:46 INFO Received uploads request method=POST path=/36652026/11/11 23:58:46 INFO Aborted multipart uploads count=0 kept=036662026/11/11 23:58:46 INFO Received push request method=POST path=/36672026/11/11 23:58:46 INFO Received complete push request method=POST path=/36682026/11/11 23:58:46 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036692026/11/11 23:58:46 INFO Completed push id=5 objects=28 duration=2.507524ms36702026/11/11 23:58:46 INFO Vacuumed table table=pending_closures36712026/11/11 23:58:46 INFO Vacuumed table table=pending_objects36722026/11/11 23:58:46 INFO Vacuumed table table=multipart_uploads36732026/11/11 23:58:46 INFO Vacuumed table table=closures36742026/11/11 23:58:46 INFO Vacuumed table table=objects36752026/11/11 23:58:46 INFO Received complete push request method=POST path=/36762026/11/11 23:58:46 INFO Received push request method=POST path=/36772026/11/11 23:58:46 INFO Received complete upload request method=POST path=/36782026/11/11 23:58:46 INFO Completed push id=10 objects=2 duration=2.10664ms36792026/11/11 23:58:46 INFO Completed upload id=21 objects=25 duration=2.632865ms36802026/11/11 23:58:46 INFO Received complete push request method=POST path=/36812026/11/11 23:58:46 INFO Completed push id=24 objects=5 duration=6.668184ms36822026/11/11 23:58:46 INFO Received uploads request method=POST path=/36832026/11/11 23:58:46 INFO Received uploads request method=POST path=/36842026/11/11 23:58:46 INFO Received complete push request method=POST path=/36852026/11/11 23:58:46 INFO Completed push id=30 objects=4 duration=3.972958ms36862026/11/11 23:58:46 INFO Received push request method=POST path=/36872026/11/11 23:58:46 INFO Received complete upload request method=POST path=/36882026/11/11 23:58:46 INFO Completed upload id=17 objects=5 duration=1.224372ms36892026/11/11 23:58:46 INFO Received uploads request method=POST path=/36902026/11/11 23:58:46 INFO Received uploads request method=POST path=/36912026/11/11 23:58:46 INFO Received push request method=POST path=/36922026/11/11 23:58:46 INFO Received push request method=POST path=/36932026/11/11 23:58:47 INFO Received complete upload request method=POST path=/36942026/11/11 23:58:47 INFO Received complete push request method=POST path=/36952026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036962026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036972026/11/11 23:58:47 INFO Completed push id=19 objects=3 duration=2.102ms36982026/11/11 23:58:47 INFO Completed upload id=12 objects=6 duration=2.859208ms36992026/11/11 23:58:47 INFO Vacuumed table table=pending_closures37002026/11/11 23:58:47 INFO Vacuumed table table=pending_closures37012026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=5 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037022026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037032026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037042026/11/11 23:58:47 INFO Vacuumed table table=pending_objects37052026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037062026/11/11 23:58:47 INFO Vacuumed table table=pending_objects37072026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads37082026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037092026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037102026/11/11 23:58:47 INFO Vacuumed table table=pending_closures37112026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads37122026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=4 objects-deleted-after-grace-period=0 objects-failed-to-delete=037132026/11/11 23:58:47 INFO Vacuumed table table=pending_objects37142026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads37152026/11/11 23:58:47 INFO Vacuumed table table=closures37162026/11/11 23:58:47 INFO Vacuumed table table=closures37172026/11/11 23:58:47 INFO Vacuumed table table=pending_closures37182026/11/11 23:58:47 INFO Vacuumed table table=pending_closures37192026/11/11 23:58:47 INFO Vacuumed table table=pending_closures37202026/11/11 23:58:47 INFO Vacuumed table table=closures37212026/11/11 23:58:47 INFO Vacuumed table table=objects37222026/11/11 23:58:47 INFO Vacuumed table table=objects37232026/11/11 23:58:47 INFO Vacuumed table table=pending_objects37242026/11/11 23:58:47 INFO Vacuumed table table=pending_closures37252026/11/11 23:58:47 INFO Vacuumed table table=pending_closures37262026/11/11 23:58:47 INFO Vacuumed table table=pending_objects37272026/11/11 23:58:47 INFO Vacuumed table table=objects37282026/11/11 23:58:47 INFO Vacuumed table table=pending_objects37292026/11/11 23:58:47 INFO Vacuumed table table=pending_closures37302026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads37312026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads37322026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads37332026/11/11 23:58:47 INFO Vacuumed table table=closures37342026/11/11 23:58:47 INFO Vacuumed table table=closures37352026/11/11 23:58:47 INFO Vacuumed table table=closures37362026/11/11 23:58:47 INFO Vacuumed table table=pending_objects37372026/11/11 23:58:47 INFO Vacuumed table table=pending_objects37382026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads37392026/11/11 23:58:47 INFO Vacuumed table table=pending_objects37402026/11/11 23:58:47 INFO Vacuumed table table=objects37412026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads37422026/11/11 23:58:47 INFO Vacuumed table table=objects37432026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads37442026/11/11 23:58:47 INFO Vacuumed table table=closures37452026/11/11 23:58:47 INFO Vacuumed table table=objects37462026/11/11 23:58:47 INFO Received push request method=POST path=/37472026/11/11 23:58:47 INFO Vacuumed table table=objects37482026/11/11 23:58:47 INFO Vacuumed table table=closures37492026/11/11 23:58:47 INFO Vacuumed table table=closures37502026/11/11 23:58:47 INFO Received complete push request method=POST path=/37512026/11/11 23:58:47 INFO Vacuumed table table=objects37522026/11/11 23:58:47 INFO Received uploads request method=POST path=/37532026/11/11 23:58:47 INFO Vacuumed table table=objects37542026/11/11 23:58:47 INFO Completed push id=17 objects=3 duration=1.991639ms37552026/11/11 23:58:47 INFO Received complete push request method=POST path=/37562026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=037572026/11/11 23:58:47 INFO Completed push id=21 objects=26 duration=1.630115ms37582026/11/11 23:58:47 INFO Received push request method=POST path=/37592026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period37602026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037612026/11/11 23:58:47 INFO Vacuumed table table=pending_closures37622026/11/11 23:58:47 INFO Received push request method=POST path=/37632026/11/11 23:58:47 INFO Vacuumed table table=pending_objects37642026/11/11 23:58:47 INFO Received uploads request method=POST path=/37652026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037662026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads37672026/11/11 23:58:47 INFO Vacuumed table table=closures37682026/11/11 23:58:47 INFO Vacuumed table table=objects37692026/11/11 23:58:47 INFO Vacuumed table table=pending_closures37702026/11/11 23:58:47 INFO Received complete push request method=POST path=/37712026/11/11 23:58:47 INFO Received complete upload request method=POST path=/37722026/11/11 23:58:47 INFO Vacuumed table table=pending_objects37732026/11/11 23:58:47 INFO Received uploads request method=POST path=/37742026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads37752026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=037762026/11/11 23:58:47 INFO Completed push id=20 objects=4 duration=1.286792ms37772026/11/11 23:58:47 INFO Vacuumed table table=closures37782026/11/11 23:58:47 INFO Received complete push request method=POST path=/37792026/11/11 23:58:47 INFO Vacuumed table table=pending_closures37802026/11/11 23:58:47 INFO Received uploads request method=POST path=/37812026/11/11 23:58:47 INFO Completed upload id=20 objects=3 duration=2.837687ms37822026/11/11 23:58:47 INFO Vacuumed table table=objects37832026/11/11 23:58:47 INFO Vacuumed table table=pending_objects37842026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads37852026/11/11 23:58:47 INFO Completed push id=32 objects=7 duration=1.645115ms37862026/11/11 23:58:47 INFO Vacuumed table table=closures37872026/11/11 23:58:47 INFO Vacuumed table table=objects37882026/11/11 23:58:47 INFO Received complete push request method=POST path=/37892026/11/11 23:58:47 INFO Received complete push request method=POST path=/37902026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=037912026/11/11 23:58:47 INFO Received complete upload request method=POST path=/37922026/11/11 23:58:47 INFO Completed push id=23 objects=5 duration=1.483194ms37932026/11/11 23:58:47 INFO Completed push id=8 objects=6 duration=3.07117ms37942026/11/11 23:58:47 INFO Completed upload id=28 objects=17 duration=1.189952ms37952026/11/11 23:58:47 INFO Received push request method=POST path=/37962026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period37972026/11/11 23:58:47 INFO Received uploads request method=POST path=/37982026/11/11 23:58:47 INFO Received uploads request method=POST path=/37992026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038002026/11/11 23:58:47 INFO Vacuumed table table=pending_closures38012026/11/11 23:58:47 INFO Vacuumed table table=pending_objects38022026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads38032026/11/11 23:58:47 INFO Vacuumed table table=closures38042026/11/11 23:58:47 INFO Received uploads request method=POST path=/38052026/11/11 23:58:47 INFO Vacuumed table table=objects38062026/11/11 23:58:47 INFO Received uploads request method=POST path=/38072026/11/11 23:58:47 INFO Received push request method=POST path=/38082026/11/11 23:58:47 INFO Received complete push request method=POST path=/38092026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=038102026/11/11 23:58:47 INFO Completed push id=26 objects=4 duration=1.05987ms38112026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period38122026/11/11 23:58:47 INFO Received push request method=POST path=/38132026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038142026/11/11 23:58:47 INFO Vacuumed table table=pending_closures38152026/11/11 23:58:47 INFO Vacuumed table table=pending_objects38162026/11/11 23:58:47 INFO Received complete upload request method=POST path=/38172026/11/11 23:58:47 INFO Received complete upload request method=POST path=/38182026/11/11 23:58:47 INFO Received complete push request method=POST path=/38192026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads38202026/11/11 23:58:47 INFO Completed upload id=24 objects=3 duration=1.121391ms38212026/11/11 23:58:47 INFO Completed upload id=22 objects=8 duration=1.305512ms38222026/11/11 23:58:47 INFO Vacuumed table table=closures38232026/11/11 23:58:47 INFO Completed push id=29 objects=2 duration=1.246892ms38242026/11/11 23:58:47 INFO Vacuumed table table=objects38252026/11/11 23:58:47 INFO Received uploads request method=POST path=/38262026/11/11 23:58:47 INFO Received push request method=POST path=/38272026/11/11 23:58:47 INFO Received complete push request method=POST path=/38282026/11/11 23:58:47 INFO Received complete upload request method=POST path=/38292026/11/11 23:58:47 INFO Completed upload id=28 objects=13 duration=1.246352ms38302026/11/11 23:58:47 INFO Completed push id=16 objects=14 duration=3.275951ms38312026/11/11 23:58:47 INFO Received complete push request method=POST path=/38322026/11/11 23:58:47 INFO Received push request method=POST path=/38332026/11/11 23:58:47 INFO Received complete push request method=POST path=/38342026/11/11 23:58:47 INFO Completed push id=23 objects=5 duration=1.191372ms38352026/11/11 23:58:47 INFO Received complete push request method=POST path=/38362026/11/11 23:58:47 INFO Received complete upload request method=POST path=/38372026/11/11 23:58:47 INFO Completed push id=11 objects=11 duration=3.507913ms38382026/11/11 23:58:47 INFO Received complete push request method=POST path=/38392026/11/11 23:58:47 INFO Completed push id=31 objects=6 duration=1.696177ms38402026/11/11 23:58:47 INFO Completed push id=25 objects=4 duration=1.181672ms38412026/11/11 23:58:47 INFO Completed upload id=10 objects=4 duration=2.659185ms38422026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=038432026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=038442026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period38452026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period38462026/11/11 23:58:47 INFO Received uploads request method=POST path=/38472026/11/11 23:58:47 INFO Received uploads request method=POST path=/38482026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038492026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038502026/11/11 23:58:47 INFO Vacuumed table table=pending_closures38512026/11/11 23:58:47 INFO Vacuumed table table=pending_closures38522026/11/11 23:58:47 INFO Vacuumed table table=pending_objects38532026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=038542026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads38552026/11/11 23:58:47 INFO Vacuumed table table=pending_objects38562026/11/11 23:58:47 INFO Vacuumed table table=closures38572026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads38582026/11/11 23:58:47 INFO Vacuumed table table=objects38592026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period38602026/11/11 23:58:47 INFO Received push request method=POST path=/38612026/11/11 23:58:47 INFO Received complete push request method=POST path=/38622026/11/11 23:58:47 INFO Vacuumed table table=closures38632026/11/11 23:58:47 INFO Received complete upload request method=POST path=/38642026/11/11 23:58:47 INFO Completed push id=29 objects=3 duration=976.909µs38652026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038662026/11/11 23:58:47 INFO Vacuumed table table=objects38672026/11/11 23:58:47 INFO Completed upload id=24 objects=7 duration=1.221452ms38682026/11/11 23:58:47 INFO Vacuumed table table=pending_closures38692026/11/11 23:58:47 INFO Received complete push request method=POST path=/38702026/11/11 23:58:47 INFO Received complete upload request method=POST path=/38712026/11/11 23:58:47 INFO Vacuumed table table=pending_objects38722026/11/11 23:58:47 INFO Received push request method=POST path=/38732026/11/11 23:58:47 INFO Completed upload id=24 objects=8 duration=766.607µs38742026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads38752026/11/11 23:58:47 INFO Vacuumed table table=closures38762026/11/11 23:58:47 INFO Completed push id=11 objects=8 duration=3.253231ms38772026/11/11 23:58:47 INFO Vacuumed table table=objects38782026/11/11 23:58:47 INFO Received complete push request method=POST path=/38792026/11/11 23:58:47 INFO Completed push id=32 objects=9 duration=1.187391ms38802026/11/11 23:58:47 INFO Received push request method=POST path=/38812026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=038822026/11/11 23:58:47 INFO Received push request method=POST path=/38832026/11/11 23:58:47 INFO Received complete upload request method=POST path=/38842026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=038852026/11/11 23:58:47 INFO Completed upload id=31 objects=2 duration=1.000369ms38862026/11/11 23:58:47 INFO Received push request method=POST path=/38872026/11/11 23:58:47 INFO Received push request method=POST path=/38882026/11/11 23:58:47 INFO Received complete upload request method=POST path=/38892026/11/11 23:58:47 INFO Completed upload id=28 objects=5 duration=1.375013ms38902026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038912026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038922026/11/11 23:58:47 INFO Vacuumed table table=pending_closures38932026/11/11 23:58:47 INFO Vacuumed table table=pending_closures38942026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=038952026/11/11 23:58:47 INFO Vacuumed table table=pending_objects38962026/11/11 23:58:47 INFO Received complete upload request method=POST path=/38972026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads38982026/11/11 23:58:47 INFO Vacuumed table table=pending_objects38992026/11/11 23:58:47 INFO Vacuumed table table=closures39002026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads39012026/11/11 23:58:47 INFO Vacuumed table table=closures39022026/11/11 23:58:47 INFO Vacuumed table table=objects39032026/11/11 23:58:47 INFO Vacuumed table table=objects39042026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039052026/11/11 23:58:47 INFO Received complete push request method=POST path=/39062026/11/11 23:58:47 INFO Vacuumed table table=pending_closures39072026/11/11 23:58:47 INFO Completed upload id=17 objects=25 duration=3.399492ms39082026/11/11 23:58:47 INFO Vacuumed table table=pending_objects39092026/11/11 23:58:47 INFO Received complete upload request method=POST path=/39102026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads39112026/11/11 23:58:47 INFO Vacuumed table table=closures39122026/11/11 23:58:47 INFO Completed upload id=33 objects=9 duration=1.06369ms39132026/11/11 23:58:47 INFO Completed push id=4 objects=6 duration=2.391343ms39142026/11/11 23:58:47 INFO Vacuumed table table=objects39152026/11/11 23:58:47 INFO Received complete push request method=POST path=/39162026/11/11 23:58:47 INFO Received push request method=POST path=/39172026/11/11 23:58:47 INFO Completed push id=33 objects=6 duration=853.828µs39182026/11/11 23:58:47 INFO Received uploads request method=POST path=/39192026/11/11 23:58:47 INFO Received complete push request method=POST path=/39202026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=039212026/11/11 23:58:47 INFO Completed push id=26 objects=23 duration=1.415114ms39222026/11/11 23:58:47 INFO Received complete push request method=POST path=/39232026/11/11 23:58:47 INFO Received complete upload request method=POST path=/39242026/11/11 23:58:47 INFO Received uploads request method=POST path=/39252026/11/11 23:58:47 INFO Received push request method=POST path=/39262026/11/11 23:58:47 INFO Completed push id=9 objects=24 duration=1.850838ms39272026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039282026/11/11 23:58:47 INFO Received complete upload request method=POST path=/39292026/11/11 23:58:47 INFO Completed upload id=4 objects=14 duration=3.363312ms39302026/11/11 23:58:47 INFO Completed upload id=30 objects=5 duration=851.288µs39312026/11/11 23:58:47 INFO Vacuumed table table=pending_closures39322026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=039332026/11/11 23:58:47 INFO Received push request method=POST path=/39342026/11/11 23:58:47 INFO Received uploads request method=POST path=/39352026/11/11 23:58:47 INFO Vacuumed table table=pending_objects39362026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads39372026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period39382026/11/11 23:58:47 INFO Vacuumed table table=closures39392026/11/11 23:58:47 INFO Vacuumed table table=objects39402026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039412026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=039422026/11/11 23:58:47 INFO Vacuumed table table=pending_closures39432026/11/11 23:58:47 INFO Received complete push request method=POST path=/39442026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period39452026/11/11 23:58:47 INFO Vacuumed table table=pending_objects39462026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads39472026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=039482026/11/11 23:58:47 INFO Vacuumed table table=closures39492026/11/11 23:58:47 INFO Completed push id=13 objects=6 duration=6.464262ms39502026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039512026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=039522026/11/11 23:58:47 INFO Vacuumed table table=objects39532026/11/11 23:58:47 INFO Received complete upload request method=POST path=/39542026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period39552026/11/11 23:58:47 INFO Vacuumed table table=pending_closures39562026/11/11 23:58:47 INFO Received uploads request method=POST path=/39572026/11/11 23:58:47 INFO Completed upload id=21 objects=2 duration=1.07359ms39582026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period39592026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039602026/11/11 23:58:47 INFO Vacuumed table table=pending_objects39612026/11/11 23:58:47 INFO Received push request method=POST path=/39622026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads39632026/11/11 23:58:47 INFO Vacuumed table table=pending_closures39642026/11/11 23:58:47 INFO Vacuumed table table=closures39652026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039662026/11/11 23:58:47 INFO Vacuumed table table=pending_objects39672026/11/11 23:58:47 INFO Received complete push request method=POST path=/39682026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads39692026/11/11 23:58:47 INFO Vacuumed table table=objects39702026/11/11 23:58:47 INFO Received uploads request method=POST path=/39712026/11/11 23:58:47 INFO Received uploads request method=POST path=/39722026/11/11 23:58:47 INFO Vacuumed table table=pending_closures39732026/11/11 23:58:47 INFO Vacuumed table table=closures39742026/11/11 23:58:47 INFO Completed push id=31 objects=3 duration=1.161471ms39752026/11/11 23:58:47 INFO Vacuumed table table=objects39762026/11/11 23:58:47 INFO Vacuumed table table=pending_objects39772026/11/11 23:58:47 INFO Received complete upload request method=POST path=/39782026/11/11 23:58:47 INFO Received complete push request method=POST path=/39792026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads39802026/11/11 23:58:47 INFO Received push request method=POST path=/39812026/11/11 23:58:47 INFO Vacuumed table table=closures39822026/11/11 23:58:47 INFO Completed upload id=9 objects=2 duration=1.246912ms39832026/11/11 23:58:47 INFO Vacuumed table table=objects39842026/11/11 23:58:47 INFO Completed push id=16 objects=7 duration=1.852897ms39852026/11/11 23:58:47 INFO Received complete push request method=POST path=/39862026/11/11 23:58:47 INFO Received complete upload request method=POST path=/39872026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=039882026/11/11 23:58:47 INFO Completed push id=27 objects=15 duration=1.175971ms39892026/11/11 23:58:47 INFO Completed upload id=11 objects=14 duration=1.792937ms39902026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period39912026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039922026/11/11 23:58:47 INFO Received uploads request method=POST path=/39932026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=039942026/11/11 23:58:47 INFO Vacuumed table table=pending_closures39952026/11/11 23:58:47 INFO Received uploads request method=POST path=/39962026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period39972026/11/11 23:58:47 INFO Vacuumed table table=pending_objects39982026/11/11 23:58:47 INFO Received push request method=POST path=/39992026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads40002026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040012026/11/11 23:58:47 INFO Vacuumed table table=closures40022026/11/11 23:58:47 INFO Vacuumed table table=pending_closures40032026/11/11 23:58:47 INFO Received complete push request method=POST path=/40042026/11/11 23:58:47 INFO Vacuumed table table=objects40052026/11/11 23:58:47 INFO Vacuumed table table=pending_objects40062026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads40072026/11/11 23:58:47 INFO Received complete upload request method=POST path=/40082026/11/11 23:58:47 INFO Vacuumed table table=closures40092026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=040102026/11/11 23:58:47 INFO Completed push id=8 objects=3 duration=1.604415ms40112026/11/11 23:58:47 INFO Completed upload id=25 objects=6 duration=1.373513ms40122026/11/11 23:58:47 INFO Vacuumed table table=objects40132026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period40142026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040152026/11/11 23:58:47 INFO Received push request method=POST path=/40162026/11/11 23:58:47 INFO Vacuumed table table=pending_closures40172026/11/11 23:58:47 INFO Vacuumed table table=pending_objects40182026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads40192026/11/11 23:58:47 INFO Vacuumed table table=closures40202026/11/11 23:58:47 INFO Received complete push request method=POST path=/40212026/11/11 23:58:47 INFO Vacuumed table table=objects40222026/11/11 23:58:47 INFO Received uploads request method=POST path=/40232026/11/11 23:58:47 INFO Received complete push request method=POST path=/40242026/11/11 23:58:47 INFO Completed push id=23 objects=28 duration=2.920768ms40252026/11/11 23:58:47 INFO Received complete push request method=POST path=/40262026/11/11 23:58:47 INFO Received push request method=POST path=/40272026/11/11 23:58:47 INFO Completed push id=23 objects=6 duration=1.171491ms40282026/11/11 23:58:47 INFO Completed push id=32 objects=6 duration=1.152531ms40292026/11/11 23:58:47 INFO Received complete upload request method=POST path=/40302026/11/11 23:58:47 INFO Completed upload id=10 objects=10 duration=1.01987ms40312026/11/11 23:58:47 INFO Received uploads request method=POST path=/40322026/11/11 23:58:47 INFO Received push request method=POST path=/40332026/11/11 23:58:47 INFO Received uploads request method=POST path=/40342026/11/11 23:58:47 INFO Received push request method=POST path=/40352026/11/11 23:58:47 INFO Received push request method=POST path=/40362026/11/11 23:58:47 INFO Received uploads request method=POST path=/40372026/11/11 23:58:47 INFO Received complete push request method=POST path=/40382026/11/11 23:58:47 INFO Received complete upload request method=POST path=/40392026/11/11 23:58:47 INFO Completed push id=24 objects=6 duration=1.146671ms40402026/11/11 23:58:47 INFO Completed upload id=33 objects=7 duration=940.229µs40412026/11/11 23:58:47 INFO Received push request method=POST path=/40422026/11/11 23:58:47 INFO Received push request method=POST path=/40432026/11/11 23:58:47 INFO Received complete push request method=POST path=/40442026/11/11 23:58:47 INFO Completed push id=28 objects=19 duration=1.280832ms40452026/11/11 23:58:47 INFO Received complete push request method=POST path=/40462026/11/11 23:58:47 INFO Completed push id=17 objects=6 duration=3.024449ms40472026/11/11 23:58:47 INFO Received complete upload request method=POST path=/40482026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=040492026/11/11 23:58:47 INFO Received complete upload request method=POST path=/40502026/11/11 23:58:47 INFO Completed upload id=13 objects=10 duration=993.85µs40512026/11/11 23:58:47 INFO Completed upload id=27 objects=8 duration=1.03771ms40522026/11/11 23:58:47 INFO Received uploads request method=POST path=/40532026/11/11 23:58:47 INFO Received complete upload request method=POST path=/40542026/11/11 23:58:47 INFO Completed upload id=34 objects=3 duration=759.527µs40552026/11/11 23:58:47 INFO Received uploads request method=POST path=/40562026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040572026/11/11 23:58:47 INFO Received uploads request method=POST path=/40582026/11/11 23:58:47 INFO Vacuumed table table=pending_closures40592026/11/11 23:58:47 INFO Vacuumed table table=pending_objects40602026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads40612026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=040622026/11/11 23:58:47 INFO Vacuumed table table=closures40632026/11/11 23:58:47 INFO Received complete push request method=POST path=/40642026/11/11 23:58:47 INFO Vacuumed table table=objects40652026/11/11 23:58:47 INFO Received complete push request method=POST path=/40662026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period40672026/11/11 23:58:47 INFO Completed push id=25 objects=7 duration=1.169571ms40682026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040692026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=040702026/11/11 23:58:47 INFO Completed push id=7 objects=24 duration=3.651755ms40712026/11/11 23:58:47 INFO Vacuumed table table=pending_closures40722026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period40732026/11/11 23:58:47 INFO Vacuumed table table=pending_objects40742026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads40752026/11/11 23:58:47 INFO Received push request method=POST path=/40762026/11/11 23:58:47 INFO Vacuumed table table=closures40772026/11/11 23:58:47 INFO Received push request method=POST path=/40782026/11/11 23:58:47 INFO Vacuumed table table=objects40792026/11/11 23:58:47 INFO Received complete push request method=POST path=/40802026/11/11 23:58:47 INFO Received complete push request method=POST path=/40812026/11/11 23:58:47 INFO Completed push id=29 objects=10 duration=1.201971ms40822026/11/11 23:58:47 INFO Completed push id=35 objects=11 duration=1.193152ms40832026/11/11 23:58:47 INFO Received complete push request method=POST path=/40842026/11/11 23:58:47 INFO Completed push id=11 objects=9 duration=1.253472ms40852026/11/11 23:58:47 INFO Received complete upload request method=POST path=/40862026/11/11 23:58:47 INFO Received complete push request method=POST path=/40872026/11/11 23:58:47 INFO Received complete upload request method=POST path=/40882026/11/11 23:58:47 INFO Completed upload id=23 objects=18 duration=2.404303ms40892026/11/11 23:58:47 INFO Received complete push request method=POST path=/40902026/11/11 23:58:47 INFO Completed push id=4 objects=7 duration=1.790617ms40912026/11/11 23:58:47 INFO Completed upload id=23 objects=16 duration=2.493963ms40922026/11/11 23:58:47 INFO Received uploads request method=POST path=/40932026/11/11 23:58:47 INFO Received complete upload request method=POST path=/40942026/11/11 23:58:47 INFO Completed push id=5 objects=8 duration=2.877887ms40952026/11/11 23:58:47 INFO Completed upload id=10 objects=6 duration=2.807847ms40962026/11/11 23:58:47 INFO Received push request method=POST path=/40972026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=040982026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=040992026/11/11 23:58:47 INFO Received uploads request method=POST path=/41002026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period41012026/11/11 23:58:47 INFO Received uploads request method=POST path=/41022026/11/11 23:58:47 INFO Received complete upload request method=POST path=/41032026/11/11 23:58:47 INFO Received complete push request method=POST path=/41042026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041052026/11/11 23:58:47 INFO Completed upload id=30 objects=6 duration=1.003349ms41062026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=6 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041072026/11/11 23:58:47 INFO Completed push id=15 objects=24 duration=1.280092ms41082026/11/11 23:58:47 INFO Received push request method=POST path=/41092026/11/11 23:58:47 INFO Vacuumed table table=pending_closures41102026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=041112026/11/11 23:58:47 INFO Vacuumed table table=pending_closures41122026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period41132026/11/11 23:58:47 INFO Vacuumed table table=pending_objects41142026/11/11 23:58:47 INFO Received complete upload request method=POST path=/41152026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads41162026/11/11 23:58:47 INFO Received uploads request method=POST path=/41172026/11/11 23:58:47 INFO Vacuumed table table=closures41182026/11/11 23:58:47 INFO Vacuumed table table=pending_objects41192026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads41202026/11/11 23:58:47 INFO Completed upload id=26 objects=24 duration=1.279973ms41212026/11/11 23:58:47 INFO Received uploads request method=POST path=/41222026/11/11 23:58:47 INFO Vacuumed table table=objects41232026/11/11 23:58:47 INFO Vacuumed table table=closures41242026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041252026/11/11 23:58:47 INFO Vacuumed table table=objects41262026/11/11 23:58:47 INFO Vacuumed table table=pending_closures41272026/11/11 23:58:47 INFO Vacuumed table table=pending_objects41282026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads41292026/11/11 23:58:47 INFO Vacuumed table table=closures41302026/11/11 23:58:47 INFO Vacuumed table table=objects41312026/11/11 23:58:47 INFO Received uploads request method=POST path=/41322026/11/11 23:58:47 INFO Received complete upload request method=POST path=/41332026/11/11 23:58:47 INFO Received uploads request method=POST path=/41342026/11/11 23:58:47 INFO Received complete upload request method=POST path=/41352026/11/11 23:58:47 INFO Received push request method=POST path=/41362026/11/11 23:58:47 INFO Completed upload id=19 objects=2 duration=927.649µs41372026/11/11 23:58:47 INFO Completed upload id=31 objects=5 duration=958.329µs41382026/11/11 23:58:47 INFO Received complete push request method=POST path=/41392026/11/11 23:58:47 INFO Completed push id=14 objects=3 duration=2.889008ms41402026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=041412026/11/11 23:58:47 INFO Received uploads request method=POST path=/41422026/11/11 23:58:47 INFO Received uploads request method=POST path=/41432026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041442026/11/11 23:58:47 INFO Vacuumed table table=pending_closures41452026/11/11 23:58:47 INFO Vacuumed table table=pending_objects41462026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads41472026/11/11 23:58:47 INFO Vacuumed table table=closures41482026/11/11 23:58:47 INFO Vacuumed table table=objects41492026/11/11 23:58:47 INFO Received uploads request method=POST path=/41502026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=041512026/11/11 23:58:47 INFO Received complete upload request method=POST path=/41522026/11/11 23:58:47 INFO Received complete upload request method=POST path=/41532026/11/11 23:58:47 INFO Received push request method=POST path=/41542026/11/11 23:58:47 INFO Completed upload id=34 objects=3 duration=925.009µs41552026/11/11 23:58:47 INFO Completed upload id=24 objects=7 duration=1.00611ms41562026/11/11 23:58:47 INFO Received complete upload request method=POST path=/41572026/11/11 23:58:47 INFO Completed upload id=22 objects=22 duration=1.060991ms41582026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041592026/11/11 23:58:47 INFO Vacuumed table table=pending_closures41602026/11/11 23:58:47 INFO Vacuumed table table=pending_objects41612026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads41622026/11/11 23:58:47 INFO Vacuumed table table=closures41632026/11/11 23:58:47 INFO Vacuumed table table=objects41642026/11/11 23:58:47 INFO Received push request method=POST path=/41652026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=11 objects-failed-to-delete=041662026/11/11 23:58:47 INFO Received complete upload request method=POST path=/41672026/11/11 23:58:47 INFO Vacuumed table table=pending_closures41682026/11/11 23:58:47 INFO Completed upload id=27 objects=26 duration=1.104211ms41692026/11/11 23:58:47 INFO Vacuumed table table=pending_objects41702026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads41712026/11/11 23:58:47 INFO Vacuumed table table=closures41722026/11/11 23:58:47 INFO Vacuumed table table=objects41732026/11/11 23:58:47 INFO Received complete push request method=POST path=/41742026/11/11 23:58:47 INFO Received complete upload request method=POST path=/41752026/11/11 23:58:47 INFO Received complete push request method=POST path=/41762026/11/11 23:58:47 INFO Received complete push request method=POST path=/41772026/11/11 23:58:47 INFO Completed push id=14 objects=3 duration=1.110831ms41782026/11/11 23:58:47 INFO Completed push id=26 objects=3 duration=965.31µs41792026/11/11 23:58:47 INFO Received push request method=POST path=/41802026/11/11 23:58:47 INFO Completed push id=8 objects=6 duration=1.776217ms41812026/11/11 23:58:47 INFO Received complete push request method=POST path=/41822026/11/11 23:58:47 INFO Completed upload id=25 objects=16 duration=2.035299ms41832026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=041842026/11/11 23:58:47 INFO Completed push id=5 objects=6 duration=1.873858ms41852026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period41862026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041872026/11/11 23:58:47 INFO Vacuumed table table=pending_closures41882026/11/11 23:58:47 INFO Received push request method=POST path=/41892026/11/11 23:58:47 INFO Received complete push request method=POST path=/41902026/11/11 23:58:47 INFO Completed push id=28 objects=6 duration=890.908µs41912026/11/11 23:58:47 INFO Vacuumed table table=pending_objects41922026/11/11 23:58:47 INFO Received uploads request method=POST path=/41932026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads41942026/11/11 23:58:47 INFO Vacuumed table table=closures41952026/11/11 23:58:47 INFO Vacuumed table table=objects41962026/11/11 23:58:47 INFO Received complete push request method=POST path=/41972026/11/11 23:58:47 INFO Received push request method=POST path=/41982026/11/11 23:58:47 INFO Received complete upload request method=POST path=/41992026/11/11 23:58:47 INFO Completed push id=7 objects=7 duration=1.330053ms42002026/11/11 23:58:47 INFO Completed upload id=28 objects=8 duration=732.607µs42012026/11/11 23:58:47 INFO Received uploads request method=POST path=/42022026/11/11 23:58:47 INFO Received complete push request method=POST path=/42032026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=042042026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=042052026/11/11 23:58:47 INFO Received complete push request method=POST path=/42062026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period42072026/11/11 23:58:47 INFO Completed push id=12 objects=7 duration=1.471794ms42082026/11/11 23:58:47 INFO Completed push id=26 objects=5 duration=2.168321ms42092026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042102026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period42112026/11/11 23:58:47 INFO Vacuumed table table=pending_closures42122026/11/11 23:58:47 INFO Received push request method=POST path=/42132026/11/11 23:58:47 INFO Vacuumed table table=pending_objects42142026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads42152026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042162026/11/11 23:58:47 INFO Received complete push request method=POST path=/42172026/11/11 23:58:47 INFO Vacuumed table table=closures42182026/11/11 23:58:47 INFO Vacuumed table table=pending_closures42192026/11/11 23:58:47 INFO Completed push id=29 objects=6 duration=1.100571ms42202026/11/11 23:58:47 INFO Vacuumed table table=objects42212026/11/11 23:58:47 INFO Received uploads request method=POST path=/42222026/11/11 23:58:47 INFO Received complete push request method=POST path=/42232026/11/11 23:58:47 INFO Vacuumed table table=pending_objects42242026/11/11 23:58:47 INFO Completed push id=13 objects=2 duration=856.148µs42252026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads42262026/11/11 23:58:47 INFO Vacuumed table table=closures42272026/11/11 23:58:47 INFO Received complete upload request method=POST path=/42282026/11/11 23:58:47 INFO Vacuumed table table=objects42292026/11/11 23:58:47 INFO Completed upload id=29 objects=10 duration=797.328µs42302026/11/11 23:58:47 INFO Received uploads request method=POST path=/42312026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=042322026/11/11 23:58:47 INFO Received complete upload request method=POST path=/42332026/11/11 23:58:47 INFO Completed upload id=24 objects=14 duration=910.809µs42342026/11/11 23:58:47 INFO Received complete push request method=POST path=/42352026/11/11 23:58:47 INFO Received push request method=POST path=/42362026/11/11 23:58:47 INFO Received complete upload request method=POST path=/42372026/11/11 23:58:47 INFO Received complete upload request method=POST path=/42382026/11/11 23:58:47 INFO Completed push id=14 objects=4 duration=2.998808ms42392026/11/11 23:58:47 INFO Completed upload id=27 objects=13 duration=1.463354ms42402026/11/11 23:58:47 INFO Completed upload id=30 objects=20 duration=789.447µs42412026/11/11 23:58:47 INFO Received push request method=POST path=/42422026/11/11 23:58:47 INFO Received push request method=POST path=/42432026/11/11 23:58:47 INFO Received complete push request method=POST path=/42442026/11/11 23:58:47 INFO Completed push id=19 objects=3 duration=1.381534ms42452026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=042462026/11/11 23:58:47 INFO Received complete upload request method=POST path=/42472026/11/11 23:58:47 INFO Vacuumed table table=pending_closures42482026/11/11 23:58:47 INFO Received complete upload request method=POST path=/42492026/11/11 23:58:47 INFO Completed upload id=30 objects=8 duration=1.739537ms42502026/11/11 23:58:47 INFO Completed upload id=18 objects=11 duration=1.190771ms42512026/11/11 23:58:47 INFO Vacuumed table table=pending_objects42522026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads42532026/11/11 23:58:47 INFO Received push request method=POST path=/42542026/11/11 23:58:47 INFO Vacuumed table table=closures42552026/11/11 23:58:47 INFO Received complete push request method=POST path=/42562026/11/11 23:58:47 INFO Vacuumed table table=objects42572026/11/11 23:58:47 INFO Completed push id=31 objects=11 duration=907.848µs42582026/11/11 23:58:47 INFO Received complete upload request method=POST path=/42592026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=042602026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=042612026/11/11 23:58:47 INFO Completed upload id=8 objects=3 duration=2.07402ms42622026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period42632026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042642026/11/11 23:58:47 INFO Received push request method=POST path=/42652026/11/11 23:58:47 INFO Vacuumed table table=pending_closures42662026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042672026/11/11 23:58:47 INFO Received complete push request method=POST path=/42682026/11/11 23:58:47 INFO Vacuumed table table=pending_objects42692026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads42702026/11/11 23:58:47 INFO Vacuumed table table=pending_closures42712026/11/11 23:58:47 INFO Vacuumed table table=closures42722026/11/11 23:58:47 INFO Received uploads request method=POST path=/42732026/11/11 23:58:47 INFO Vacuumed table table=pending_objects42742026/11/11 23:58:47 INFO Completed push id=7 objects=15 duration=2.653765ms42752026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads42762026/11/11 23:58:47 INFO Vacuumed table table=objects42772026/11/11 23:58:47 INFO Vacuumed table table=closures42782026/11/11 23:58:47 INFO Received complete upload request method=POST path=/42792026/11/11 23:58:47 INFO Vacuumed table table=objects42802026/11/11 23:58:47 INFO Completed upload id=32 objects=2 duration=766.388µs42812026/11/11 23:58:47 INFO Received uploads request method=POST path=/42822026/11/11 23:58:47 INFO Received complete upload request method=POST path=/42832026/11/11 23:58:47 INFO Completed upload id=31 objects=6 duration=1.08447ms42842026/11/11 23:58:47 INFO Received uploads request method=POST path=/42852026/11/11 23:58:47 INFO Received uploads request method=POST path=/42862026/11/11 23:58:47 INFO Received uploads request method=POST path=/42872026/11/11 23:58:47 INFO Received push request method=POST path=/42882026/11/11 23:58:47 INFO Received complete push request method=POST path=/42892026/11/11 23:58:47 INFO Completed push id=33 objects=3 duration=1.088611ms42902026/11/11 23:58:47 INFO Received push request method=POST path=/42912026/11/11 23:58:47 INFO Received complete upload request method=POST path=/42922026/11/11 23:58:47 INFO Completed upload id=32 objects=4 duration=826.548µs42932026/11/11 23:58:47 INFO Received uploads request method=POST path=/42942026/11/11 23:58:47 INFO Received push request method=POST path=/42952026/11/11 23:58:47 INFO Received complete upload request method=POST path=/42962026/11/11 23:58:47 INFO Received push request method=POST path=/42972026/11/11 23:58:47 INFO Completed upload id=34 objects=3 duration=586.505µs42982026/11/11 23:58:47 INFO Received uploads request method=POST path=/42992026/11/11 23:58:47 INFO Received uploads request method=POST path=/43002026/11/11 23:58:47 INFO Received complete upload request method=POST path=/43012026/11/11 23:58:47 INFO Completed upload id=33 objects=10 duration=917.208µs43022026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=043032026/11/11 23:58:47 INFO Received uploads request method=POST path=/43042026/11/11 23:58:47 INFO Received uploads request method=POST path=/43052026/11/11 23:58:47 INFO Received complete push request method=POST path=/43062026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period43072026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=043082026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=043092026/11/11 23:58:47 INFO Received push request method=POST path=/43102026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period43112026/11/11 23:58:47 INFO Completed push id=6 objects=6 duration=2.16924ms43122026/11/11 23:58:47 INFO Received push request method=POST path=/43132026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043142026/11/11 23:58:47 INFO Received complete upload request method=POST path=/43152026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period43162026/11/11 23:58:47 INFO Completed upload id=22 objects=5 duration=1.884938ms43172026/11/11 23:58:47 INFO Vacuumed table table=pending_closures43182026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043192026/11/11 23:58:47 INFO Vacuumed table table=pending_closures43202026/11/11 23:58:47 INFO Received uploads request method=POST path=/43212026/11/11 23:58:47 INFO Vacuumed table table=pending_objects43222026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads43232026/11/11 23:58:47 INFO Vacuumed table table=pending_objects43242026/11/11 23:58:47 INFO Vacuumed table table=closures43252026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads43262026/11/11 23:58:47 INFO Vacuumed table table=closures43272026/11/11 23:58:47 INFO Vacuumed table table=objects43282026/11/11 23:58:47 INFO Vacuumed table table=objects43292026/11/11 23:58:47 INFO Received push request method=POST path=/43302026/11/11 23:58:47 INFO Received uploads request method=POST path=/43312026/11/11 23:58:47 INFO Received complete upload request method=POST path=/43322026/11/11 23:58:47 INFO Received complete push request method=POST path=/43332026/11/11 23:58:47 INFO Completed upload id=25 objects=4 duration=1.06867ms43342026/11/11 23:58:47 INFO Received uploads request method=POST path=/43352026/11/11 23:58:47 INFO Completed push id=28 objects=3 duration=1.984799ms43362026/11/11 23:58:47 INFO Received complete push request method=POST path=/43372026/11/11 23:58:47 INFO Received complete upload request method=POST path=/43382026/11/11 23:58:47 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."43392026/11/11 23:58:47 INFO Completed push id=6 objects=26 duration=3.900438ms43402026/11/11 23:58:47 INFO Completed upload id=34 objects=23 duration=1.298512ms43412026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=043422026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period43432026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043442026/11/11 23:58:47 INFO Vacuumed table table=pending_closures43452026/11/11 23:58:47 INFO Received uploads request method=POST path=/43462026/11/11 23:58:47 INFO Vacuumed table table=pending_objects43472026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads43482026/11/11 23:58:47 INFO Received push request method=POST path=/43492026/11/11 23:58:47 INFO Vacuumed table table=closures43502026/11/11 23:58:47 INFO Vacuumed table table=objects43512026/11/11 23:58:47 INFO Received complete push request method=POST path=/43522026/11/11 23:58:47 INFO Completed push id=29 objects=15 duration=1.02839ms43532026/11/11 23:58:47 INFO Received uploads request method=POST path=/43542026/11/11 23:58:47 INFO Received uploads request method=POST path=/43552026/11/11 23:58:47 INFO Received complete push request method=POST path=/43562026/11/11 23:58:47 INFO Received push request method=POST path=/43572026/11/11 23:58:47 INFO Completed push id=35 objects=6 duration=995.389µs43582026/11/11 23:58:47 INFO Received complete push request method=POST path=/43592026/11/11 23:58:47 INFO Completed push id=30 objects=6 duration=975.509µs43602026/11/11 23:58:47 INFO Received complete upload request method=POST path=/43612026/11/11 23:58:47 INFO Completed upload id=29 objects=4 duration=1.412513ms43622026/11/11 23:58:47 INFO Received complete push request method=POST path=/43632026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=043642026/11/11 23:58:47 INFO Received complete push request method=POST path=/43652026/11/11 23:58:47 INFO Completed push id=10 objects=15 duration=1.719036ms43662026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period43672026/11/11 23:58:47 INFO Received complete push request method=POST path=/43682026/11/11 23:58:47 INFO Completed push id=16 objects=16 duration=2.910048ms43692026/11/11 23:58:47 INFO Completed push id=15 objects=4 duration=1.181411ms43702026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043712026/11/11 23:58:47 INFO Vacuumed table table=pending_closures43722026/11/11 23:58:47 INFO Vacuumed table table=pending_objects43732026/11/11 23:58:47 INFO Received uploads request method=POST path=/43742026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads43752026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=043762026/11/11 23:58:47 INFO Vacuumed table table=closures43772026/11/11 23:58:47 INFO Vacuumed table table=objects43782026/11/11 23:58:47 INFO Received uploads request method=POST path=/43792026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period43802026/11/11 23:58:47 INFO Received complete upload request method=POST path=/43812026/11/11 23:58:47 INFO Received complete push request method=POST path=/43822026/11/11 23:58:47 INFO Received uploads request method=POST path=/43832026/11/11 23:58:47 INFO Completed upload id=31 objects=3 duration=828.388µs43842026/11/11 23:58:47 INFO Completed push id=20 objects=8 duration=989.689µs43852026/11/11 23:58:47 INFO Received push request method=POST path=/43862026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=043872026/11/11 23:58:47 INFO Vacuumed table table=pending_closures43882026/11/11 23:58:47 INFO Vacuumed table table=pending_objects43892026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads43902026/11/11 23:58:47 INFO Received push request method=POST path=/43912026/11/11 23:58:47 INFO Vacuumed table table=closures43922026/11/11 23:58:47 INFO Vacuumed table table=objects43932026/11/11 23:58:47 INFO Received uploads request method=POST path=/43942026/11/11 23:58:47 INFO Received uploads request method=POST path=/43952026/11/11 23:58:47 INFO Received uploads request method=POST path=/43962026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=043972026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=043982026/11/11 23:58:47 INFO Received uploads request method=POST path=/43992026/11/11 23:58:47 INFO Received uploads request method=POST path=/44002026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period44012026/11/11 23:58:47 INFO Received complete push request method=POST path=/44022026/11/11 23:58:47 INFO Completed push id=19 objects=14 duration=1.996459ms44032026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044042026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=044052026/11/11 23:58:47 INFO Vacuumed table table=pending_closures44062026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period44072026/11/11 23:58:47 INFO Vacuumed table table=pending_objects44082026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads44092026/11/11 23:58:47 INFO Vacuumed table table=closures44102026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044112026/11/11 23:58:47 INFO Vacuumed table table=objects44122026/11/11 23:58:47 INFO Received uploads request method=POST path=/44132026/11/11 23:58:47 INFO Received push request method=POST path=/44142026/11/11 23:58:47 INFO Vacuumed table table=pending_closures44152026/11/11 23:58:47 INFO Vacuumed table table=pending_objects44162026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads44172026/11/11 23:58:47 INFO Vacuumed table table=closures44182026/11/11 23:58:47 INFO Vacuumed table table=objects44192026/11/11 23:58:47 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."44202026/11/11 23:58:47 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."44212026/11/11 23:58:47 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."44222026/11/11 23:58:47 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."44232026/11/11 23:58:47 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."44242026/11/11 23:58:47 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."44252026/11/11 23:58:47 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."44262026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=044272026/11/11 23:58:47 INFO Received complete push request method=POST path=/44282026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period44292026/11/11 23:58:47 INFO Completed push id=11 objects=6 duration=1.569635ms44302026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=044312026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period44322026/11/11 23:58:47 INFO Received uploads request method=POST path=/44332026/11/11 23:58:47 INFO Received complete push request method=POST path=/44342026/11/11 23:58:47 INFO Completed push id=14 objects=3 duration=1.432694ms44352026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=044362026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period44372026/11/11 23:58:47 INFO Received complete push request method=POST path=/44382026/11/11 23:58:47 INFO Received complete push request method=POST path=/44392026/11/11 23:58:47 INFO Received complete push request method=POST path=/44402026/11/11 23:58:47 INFO Completed push id=27 objects=11 duration=998.85µs44412026/11/11 23:58:47 INFO Completed push id=20 objects=13 duration=1.349113ms44422026/11/11 23:58:47 INFO Received uploads request method=POST path=/44432026/11/11 23:58:47 INFO Completed push id=19 objects=8 duration=4.18286ms44442026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=044452026/11/11 23:58:47 INFO Received uploads request method=POST path=/44462026/11/11 23:58:47 INFO Vacuumed table table=pending_closures44472026/11/11 23:58:47 INFO Vacuumed table table=pending_objects44482026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads44492026/11/11 23:58:47 INFO Vacuumed table table=closures44502026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=044512026/11/11 23:58:47 INFO Vacuumed table table=objects44522026/11/11 23:58:47 INFO Vacuumed table table=pending_closures44532026/11/11 23:58:47 INFO Vacuumed table table=pending_objects44542026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads44552026/11/11 23:58:47 INFO Vacuumed table table=closures44562026/11/11 23:58:47 INFO Vacuumed table table=objects44572026/11/11 23:58:47 INFO Received uploads request method=POST path=/44582026/11/11 23:58:47 INFO Received complete push request method=POST path=/44592026/11/11 23:58:47 INFO Received complete upload request method=POST path=/44602026/11/11 23:58:47 INFO Completed push id=22 objects=4 duration=1.556214ms44612026/11/11 23:58:47 INFO Completed upload id=29 objects=26 duration=1.03629ms44622026/11/11 23:58:47 INFO Received push request method=POST path=/44632026/11/11 23:58:47 INFO Received push request method=POST path=/44642026/11/11 23:58:47 INFO Received complete upload request method=POST path=/44652026/11/11 23:58:47 INFO Completed upload id=20 objects=4 duration=902.368µs44662026/11/11 23:58:47 INFO Received push request method=POST path=/44672026/11/11 23:58:47 INFO Received complete push request method=POST path=/44682026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=044692026/11/11 23:58:47 INFO Received complete push request method=POST path=/44702026/11/11 23:58:47 INFO Completed push id=30 objects=8 duration=1.204532ms44712026/11/11 23:58:47 INFO Completed push id=18 objects=6 duration=3.533433ms44722026/11/11 23:58:47 INFO Received complete upload request method=POST path=/44732026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044742026/11/11 23:58:47 INFO Vacuumed table table=pending_closures44752026/11/11 23:58:47 INFO Completed upload id=23 objects=7 duration=1.170471ms44762026/11/11 23:58:47 INFO Vacuumed table table=pending_objects44772026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads44782026/11/11 23:58:47 INFO Vacuumed table table=closures44792026/11/11 23:58:47 INFO Vacuumed table table=objects44802026/11/11 23:58:47 INFO Received uploads request method=POST path=/44812026/11/11 23:58:47 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."44822026/11/11 23:58:47 INFO Received uploads request method=POST path=/44832026/11/11 23:58:47 INFO Received complete upload request method=POST path=/44842026/11/11 23:58:47 INFO Received uploads request method=POST path=/44852026/11/11 23:58:47 INFO Completed upload id=31 objects=5 duration=1.06661ms44862026/11/11 23:58:47 INFO Received complete upload request method=POST path=/44872026/11/11 23:58:47 INFO Completed upload id=23 objects=9 duration=1.544034ms44882026/11/11 23:58:47 INFO Received complete push request method=POST path=/44892026/11/11 23:58:47 INFO Completed push id=24 objects=5 duration=1.116711ms44902026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=044912026/11/11 23:58:47 INFO Received complete push request method=POST path=/44922026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period44932026/11/11 23:58:47 INFO Completed push id=16 objects=5 duration=1.07103ms44942026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044952026/11/11 23:58:47 INFO Vacuumed table table=pending_closures44962026/11/11 23:58:47 INFO Vacuumed table table=pending_objects44972026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads44982026/11/11 23:58:47 INFO Vacuumed table table=closures44992026/11/11 23:58:47 INFO Vacuumed table table=objects45002026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=045012026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=045022026/11/11 23:58:47 INFO Received complete push request method=POST path=/45032026/11/11 23:58:47 INFO Completed push id=32 objects=3 duration=1.331652ms45042026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period45052026/11/11 23:58:47 INFO Received complete upload request method=POST path=/45062026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045072026/11/11 23:58:47 INFO Completed upload id=25 objects=2 duration=913.009µs45082026/11/11 23:58:47 INFO Vacuumed table table=pending_closures45092026/11/11 23:58:47 INFO Received uploads request method=POST path=/45102026/11/11 23:58:47 INFO Vacuumed table table=pending_objects45112026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads45122026/11/11 23:58:47 INFO Vacuumed table table=closures45132026/11/11 23:58:47 INFO Vacuumed table table=objects45142026/11/11 23:58:47 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."45152026/11/11 23:58:47 INFO Received push request method=POST path=/45162026/11/11 23:58:47 INFO Received push request method=POST path=/45172026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=045182026/11/11 23:58:47 INFO Received complete upload request method=POST path=/45192026/11/11 23:58:47 INFO Completed upload id=26 objects=20 duration=1.487454ms45202026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=045212026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period45222026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=045232026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period45242026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045252026/11/11 23:58:47 INFO Vacuumed table table=pending_closures45262026/11/11 23:58:47 INFO Received uploads request method=POST path=/45272026/11/11 23:58:47 INFO Vacuumed table table=pending_objects45282026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads45292026/11/11 23:58:47 INFO Vacuumed table table=closures45302026/11/11 23:58:47 INFO Vacuumed table table=objects45312026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=045322026/11/11 23:58:47 INFO Vacuumed table table=pending_closures45332026/11/11 23:58:47 INFO Received complete upload request method=POST path=/45342026/11/11 23:58:47 INFO Vacuumed table table=pending_objects45352026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads45362026/11/11 23:58:47 INFO Vacuumed table table=closures45372026/11/11 23:58:47 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."45382026/11/11 23:58:47 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."45392026/11/11 23:58:47 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."45402026/11/11 23:58:47 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."45412026/11/11 23:58:47 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."45422026/11/11 23:58:47 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."45432026/11/11 23:58:47 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."45442026/11/11 23:58:47 INFO Vacuumed table table=objects45452026/11/11 23:58:47 INFO Completed upload id=19 objects=25 duration=2.996168ms45462026/11/11 23:58:47 INFO Received uploads request method=POST path=/45472026/11/11 23:58:47 INFO Received complete upload request method=POST path=/45482026/11/11 23:58:47 INFO Received complete push request method=POST path=/45492026/11/11 23:58:47 INFO Completed upload id=14 objects=8 duration=1.947718ms45502026/11/11 23:58:47 INFO Completed push id=28 objects=7 duration=1.670556ms45512026/11/11 23:58:47 INFO Received uploads request method=POST path=/45522026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=045532026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045542026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=045552026/11/11 23:58:47 INFO Vacuumed table table=pending_closures45562026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period45572026/11/11 23:58:47 INFO Vacuumed table table=pending_objects45582026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads45592026/11/11 23:58:47 INFO Vacuumed table table=closures45602026/11/11 23:58:47 INFO Vacuumed table table=objects45612026/11/11 23:58:47 INFO Received uploads request method=POST path=/45622026/11/11 23:58:47 INFO Received complete upload request method=POST path=/45632026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=045642026/11/11 23:58:47 INFO Received complete upload request method=POST path=/45652026/11/11 23:58:47 INFO Vacuumed table table=pending_closures45662026/11/11 23:58:47 INFO Completed upload id=21 objects=3 duration=2.192701ms45672026/11/11 23:58:47 INFO Completed upload id=30 objects=6 duration=1.103191ms45682026/11/11 23:58:47 INFO Vacuumed table table=pending_objects45692026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads45702026/11/11 23:58:47 INFO Vacuumed table table=closures45712026/11/11 23:58:47 INFO Vacuumed table table=objects45722026/11/11 23:58:47 INFO Received uploads request method=POST path=/45732026/11/11 23:58:47 INFO Received push request method=POST path=/45742026/11/11 23:58:47 INFO Received complete upload request method=POST path=/45752026/11/11 23:58:47 INFO Completed upload id=13 objects=14 duration=3.231971ms45762026/11/11 23:58:47 INFO Received push request method=POST path=/45772026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=045782026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period45792026/11/11 23:58:47 INFO Received complete push request method=POST path=/45802026/11/11 23:58:47 INFO Completed push id=31 objects=5 duration=989.03µs45812026/11/11 23:58:47 INFO Received uploads request method=POST path=/45822026/11/11 23:58:47 INFO Received uploads request method=POST path=/45832026/11/11 23:58:47 INFO Received uploads request method=POST path=/45842026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=045852026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=7 objects-failed-to-delete=045862026/11/11 23:58:47 INFO Vacuumed table table=pending_closures45872026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045882026/11/11 23:58:47 INFO Vacuumed table table=pending_objects45892026/11/11 23:58:47 INFO Vacuumed table table=pending_closures45902026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads45912026/11/11 23:58:47 INFO Vacuumed table table=closures45922026/11/11 23:58:47 INFO Vacuumed table table=pending_objects45932026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads45942026/11/11 23:58:47 INFO Vacuumed table table=objects45952026/11/11 23:58:47 INFO Vacuumed table table=closures45962026/11/11 23:58:47 INFO Vacuumed table table=objects45972026/11/11 23:58:47 INFO Received push request method=POST path=/45982026/11/11 23:58:47 INFO Received push request method=POST path=/45992026/11/11 23:58:47 INFO Received complete push request method=POST path=/46002026/11/11 23:58:47 INFO Completed push id=16 objects=26 duration=1.194811ms46012026/11/11 23:58:47 INFO Received complete push request method=POST path=/46022026/11/11 23:58:47 INFO Completed push id=32 objects=26 duration=1.395894ms46032026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=046042026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=046052026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period46062026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046072026/11/11 23:58:47 INFO Vacuumed table table=pending_closures46082026/11/11 23:58:47 INFO Received uploads request method=POST path=/46092026/11/11 23:58:47 INFO Vacuumed table table=pending_objects46102026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads46112026/11/11 23:58:47 INFO Received complete push request method=POST path=/46122026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=046132026/11/11 23:58:47 INFO Vacuumed table table=closures46142026/11/11 23:58:47 INFO Vacuumed table table=objects46152026/11/11 23:58:47 INFO Completed push id=31 objects=6 duration=2.386043ms46162026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=046172026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period46182026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=046192026/11/11 23:58:47 INFO Received complete upload request method=POST path=/46202026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period46212026/11/11 23:58:47 INFO Vacuumed table table=pending_closures46222026/11/11 23:58:47 INFO Completed upload id=33 objects=2 duration=1.421593ms46232026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046242026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046252026/11/11 23:58:47 INFO Vacuumed table table=pending_closures46262026/11/11 23:58:47 INFO Vacuumed table table=pending_objects46272026/11/11 23:58:47 INFO Vacuumed table table=pending_objects46282026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads46292026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads46302026/11/11 23:58:47 INFO Vacuumed table table=pending_closures46312026/11/11 23:58:47 INFO Vacuumed table table=closures46322026/11/11 23:58:47 INFO Vacuumed table table=closures46332026/11/11 23:58:47 INFO Vacuumed table table=objects46342026/11/11 23:58:47 INFO Vacuumed table table=pending_objects46352026/11/11 23:58:47 INFO Vacuumed table table=objects46362026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads46372026/11/11 23:58:47 INFO Vacuumed table table=closures46382026/11/11 23:58:47 INFO Vacuumed table table=objects46392026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=046402026/11/11 23:58:47 INFO Received uploads request method=POST path=/46412026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046422026/11/11 23:58:47 INFO Vacuumed table table=pending_closures46432026/11/11 23:58:47 INFO Vacuumed table table=pending_objects46442026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads46452026/11/11 23:58:47 INFO Received complete upload request method=POST path=/46462026/11/11 23:58:47 INFO Vacuumed table table=closures46472026/11/11 23:58:47 INFO Received push request method=POST path=/46482026/11/11 23:58:47 INFO Completed upload id=34 objects=5 duration=888.288µs46492026/11/11 23:58:47 INFO Vacuumed table table=objects46502026/11/11 23:58:47 INFO Received complete push request method=POST path=/46512026/11/11 23:58:47 INFO Received uploads request method=POST path=/46522026/11/11 23:58:47 INFO Completed push id=27 objects=6 duration=2.768127ms46532026/11/11 23:58:47 INFO Received complete push request method=POST path=/46542026/11/11 23:58:47 INFO Received complete upload request method=POST path=/46552026/11/11 23:58:47 INFO Received uploads request method=POST path=/46562026/11/11 23:58:47 INFO Completed upload id=35 objects=11 duration=680.307µs46572026/11/11 23:58:47 INFO Received complete upload request method=POST path=/46582026/11/11 23:58:47 INFO Completed push id=22 objects=3 duration=3.077369ms46592026/11/11 23:58:47 INFO Completed upload id=14 objects=16 duration=2.383943ms46602026/11/11 23:58:47 INFO Received push request method=POST path=/46612026/11/11 23:58:47 INFO Received push request method=POST path=/46622026/11/11 23:58:47 INFO Received uploads request method=POST path=/46632026/11/11 23:58:47 INFO Received complete upload request method=POST path=/46642026/11/11 23:58:47 INFO Completed upload id=36 objects=3 duration=704.167µs46652026/11/11 23:58:47 INFO Received push request method=POST path=/46662026/11/11 23:58:47 INFO Received push request method=POST path=/46672026/11/11 23:58:47 INFO Received complete upload request method=POST path=/46682026/11/11 23:58:47 INFO Completed upload id=15 objects=2 duration=988.669µs46692026/11/11 23:58:47 INFO Received complete upload request method=POST path=/46702026/11/11 23:58:47 INFO Completed upload id=16 objects=17 duration=2.306582ms46712026/11/11 23:58:47 INFO Received uploads request method=POST path=/46722026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=046732026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period46742026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046752026/11/11 23:58:47 INFO Vacuumed table table=pending_closures46762026/11/11 23:58:47 INFO Vacuumed table table=pending_objects46772026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=046782026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads46792026/11/11 23:58:47 INFO Received complete push request method=POST path=/46802026/11/11 23:58:47 INFO Vacuumed table table=closures46812026/11/11 23:58:47 INFO Vacuumed table table=objects46822026/11/11 23:58:47 INFO Completed push id=1 objects=9 duration=1.457134ms46832026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period46842026/11/11 23:58:47 INFO Received push request method=POST path=/46852026/11/11 23:58:47 INFO Received push request method=POST path=/46862026/11/11 23:58:47 INFO Received complete push request method=POST path=/46872026/11/11 23:58:47 INFO Completed push id=17 objects=11 duration=1.455654ms46882026/11/11 23:58:47 INFO Received uploads request method=POST path=/46892026/11/11 23:58:47 INFO Received complete push request method=POST path=/46902026/11/11 23:58:47 INFO Completed push id=24 objects=3 duration=1.160751ms46912026/11/11 23:58:47 INFO Received push request method=POST path=/46922026/11/11 23:58:47 INFO Received complete push request method=POST path=/46932026/11/11 23:58:47 INFO Completed push id=16 objects=4 duration=2.435023ms46942026/11/11 23:58:47 INFO Received push request method=POST path=/46952026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=046962026/11/11 23:58:47 INFO Received push request method=POST path=/46972026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046982026/11/11 23:58:47 INFO Vacuumed table table=pending_closures46992026/11/11 23:58:47 INFO Vacuumed table table=pending_objects47002026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads47012026/11/11 23:58:47 INFO Vacuumed table table=closures47022026/11/11 23:58:47 INFO Vacuumed table table=objects47032026/11/11 23:58:47 INFO Received push request method=POST path=/47042026/11/11 23:58:47 INFO Received uploads request method=POST path=/47052026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=047062026/11/11 23:58:47 INFO Vacuumed table table=pending_closures47072026/11/11 23:58:47 INFO Vacuumed table table=pending_objects47082026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads47092026/11/11 23:58:47 INFO Vacuumed table table=closures47102026/11/11 23:58:47 INFO Vacuumed table table=objects47112026/11/11 23:58:47 INFO Received complete upload request method=POST path=/47122026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=047132026/11/11 23:58:47 INFO Completed upload id=2 objects=7 duration=1.062491ms47142026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period47152026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047162026/11/11 23:58:47 INFO Vacuumed table table=pending_closures47172026/11/11 23:58:47 INFO Vacuumed table table=pending_objects47182026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads47192026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=047202026/11/11 23:58:47 INFO Vacuumed table table=closures47212026/11/11 23:58:47 INFO Received uploads request method=POST path=/47222026/11/11 23:58:47 INFO Vacuumed table table=objects47232026/11/11 23:58:47 INFO Received complete upload request method=POST path=/47242026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period47252026/11/11 23:58:47 INFO Completed upload id=20 objects=5 duration=909.528µs47262026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047272026/11/11 23:58:47 INFO Vacuumed table table=pending_closures47282026/11/11 23:58:47 INFO Received complete upload request method=POST path=/47292026/11/11 23:58:47 INFO Vacuumed table table=pending_objects47302026/11/11 23:58:47 INFO Completed upload id=26 objects=6 duration=1.09371ms47312026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads47322026/11/11 23:58:47 INFO Vacuumed table table=closures47332026/11/11 23:58:47 INFO Vacuumed table table=objects47342026/11/11 23:58:47 INFO Received complete upload request method=POST path=/47352026/11/11 23:58:47 INFO Completed upload id=19 objects=2 duration=1.892678ms47362026/11/11 23:58:47 INFO Received uploads request method=POST path=/47372026/11/11 23:58:47 INFO Received uploads request method=POST path=/47382026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=047392026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period47402026/11/11 23:58:47 INFO Received uploads request method=POST path=/47412026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047422026/11/11 23:58:47 INFO Vacuumed table table=pending_closures47432026/11/11 23:58:47 INFO Vacuumed table table=pending_objects47442026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads47452026/11/11 23:58:47 INFO Vacuumed table table=closures47462026/11/11 23:58:47 INFO Vacuumed table table=objects47472026/11/11 23:58:47 INFO Received uploads request method=POST path=/47482026/11/11 23:58:47 INFO Received complete upload request method=POST path=/47492026/11/11 23:58:47 INFO Completed upload id=8 objects=6 duration=1.059311ms47502026/11/11 23:58:47 INFO Received complete upload request method=POST path=/47512026/11/11 23:58:47 INFO Received push request method=POST path=/47522026/11/11 23:58:47 INFO Completed upload id=26 objects=8 duration=1.091751ms47532026/11/11 23:58:47 INFO Received complete upload request method=POST path=/47542026/11/11 23:58:47 INFO Completed upload id=29 objects=5 duration=957.869µs47552026/11/11 23:58:47 INFO Received complete upload request method=POST path=/47562026/11/11 23:58:47 INFO Completed upload id=15 objects=3 duration=1.606236ms47572026/11/11 23:58:47 INFO Received complete push request method=POST path=/47582026/11/11 23:58:47 INFO Completed push id=33 objects=7 duration=1.721356ms47592026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=047602026/11/11 23:58:47 INFO Received complete push request method=POST path=/47612026/11/11 23:58:47 INFO Received push request method=POST path=/47622026/11/11 23:58:47 INFO Completed push id=28 objects=14 duration=1.248092ms47632026/11/11 23:58:47 INFO Received complete upload request method=POST path=/47642026/11/11 23:58:47 INFO Completed upload id=26 objects=3 duration=898.469µs47652026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=047662026/11/11 23:58:47 INFO Received push request method=POST path=/47672026/11/11 23:58:47 INFO Vacuumed table table=pending_closures47682026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=047692026/11/11 23:58:47 INFO Received push request method=POST path=/47702026/11/11 23:58:47 INFO Vacuumed table table=pending_objects47712026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads47722026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period47732026/11/11 23:58:47 INFO Vacuumed table table=closures47742026/11/11 23:58:47 INFO Vacuumed table table=objects47752026/11/11 23:58:47 INFO Received push request method=POST path=/47762026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047772026/11/11 23:58:47 INFO Vacuumed table table=pending_closures47782026/11/11 23:58:47 INFO Vacuumed table table=pending_objects47792026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads47802026/11/11 23:58:47 INFO Vacuumed table table=closures47812026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=047822026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period47832026/11/11 23:58:47 INFO Vacuumed table table=objects47842026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047852026/11/11 23:58:47 INFO Vacuumed table table=pending_closures47862026/11/11 23:58:47 INFO Received complete upload request method=POST path=/47872026/11/11 23:58:47 INFO Vacuumed table table=pending_objects47882026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads47892026/11/11 23:58:47 INFO Vacuumed table table=closures47902026/11/11 23:58:47 INFO Completed upload id=12 objects=9 duration=938.529µs47912026/11/11 23:58:47 INFO Vacuumed table table=objects47922026/11/11 23:58:47 INFO Received uploads request method=POST path=/47932026/11/11 23:58:47 INFO Received push request method=POST path=/47942026/11/11 23:58:47 INFO Received uploads request method=POST path=/47952026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=047962026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period47972026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047982026/11/11 23:58:47 INFO Vacuumed table table=pending_closures47992026/11/11 23:58:47 INFO Vacuumed table table=pending_objects48002026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads48012026/11/11 23:58:47 INFO Received complete push request method=POST path=/48022026/11/11 23:58:47 INFO Vacuumed table table=closures48032026/11/11 23:58:47 INFO Vacuumed table table=objects48042026/11/11 23:58:47 INFO Received complete upload request method=POST path=/48052026/11/11 23:58:47 INFO Completed push id=17 objects=22 duration=2.09376ms48062026/11/11 23:58:47 INFO Completed upload id=26 objects=2 duration=1.652776ms48072026/11/11 23:58:47 INFO Received uploads request method=POST path=/48082026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=048092026/11/11 23:58:47 WARN Force mode enabled - objects will be deleted immediately without grace period48102026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048112026/11/11 23:58:47 INFO Vacuumed table table=pending_closures48122026/11/11 23:58:47 INFO Vacuumed table table=pending_objects48132026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads48142026/11/11 23:58:47 INFO Vacuumed table table=closures48152026/11/11 23:58:47 INFO Received uploads request method=POST path=/48162026/11/11 23:58:47 INFO Vacuumed table table=objects48172026/11/11 23:58:47 INFO Received complete push request method=POST path=/48182026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=048192026/11/11 23:58:47 INFO Received uploads request method=POST path=/48202026/11/11 23:58:47 INFO Completed push id=9 objects=9 duration=2.380003ms48212026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048222026/11/11 23:58:47 INFO Vacuumed table table=pending_closures48232026/11/11 23:58:47 INFO Vacuumed table table=pending_objects48242026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads48252026/11/11 23:58:47 INFO Vacuumed table table=closures48262026/11/11 23:58:47 INFO Vacuumed table table=objects48272026/11/11 23:58:47 INFO Aborted multipart uploads count=0 kept=048282026/11/11 23:58:47 INFO Received complete upload request method=POST path=/48292026/11/11 23:58:47 INFO Received complete upload request method=POST path=/48302026/11/11 23:58:47 INFO Completed upload id=31 objects=3 duration=2.08538ms48312026/11/11 23:58:47 INFO Received complete upload request method=POST path=/48322026/11/11 23:58:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048332026/11/11 23:58:47 INFO Completed upload id=21 objects=8 duration=1.082731ms48342026/11/11 23:58:47 INFO Vacuumed table table=pending_closures48352026/11/11 23:58:47 INFO Completed upload id=21 objects=9 duration=2.047ms48362026/11/11 23:58:47 INFO Vacuumed table table=pending_objects48372026/11/11 23:58:47 INFO Vacuumed table table=multipart_uploads48382026/11/11 23:58:47 INFO Vacuumed table table=closures48392026/11/11 23:58:48 INFO Vacuumed table table=objects48402026/11/11 23:58:48 INFO Received complete push request method=POST path=/48412026/11/11 23:58:48 INFO Received complete upload request method=POST path=/48422026/11/11 23:58:48 INFO Completed push id=22 objects=6 duration=1.222672ms48432026/11/11 23:58:48 INFO Received push request method=POST path=/48442026/11/11 23:58:48 INFO Completed upload id=22 objects=2 duration=886.348µs48452026/11/11 23:58:48 INFO Received complete push request method=POST path=/48462026/11/11 23:58:48 INFO Received complete upload request method=POST path=/48472026/11/11 23:58:48 INFO Completed push id=25 objects=10 duration=1.566075ms48482026/11/11 23:58:48 INFO Completed upload id=18 objects=22 duration=1.468914ms48492026/11/11 23:58:48 INFO Received push request method=POST path=/48502026/11/11 23:58:48 INFO Received complete push request method=POST path=/48512026/11/11 23:58:48 INFO Completed push id=24 objects=8 duration=1.088971ms48522026/11/11 23:58:48 INFO Aborted multipart uploads count=0 kept=048532026/11/11 23:58:48 WARN Force mode enabled - objects will be deleted immediately without grace period48542026/11/11 23:58:48 INFO Received complete upload request method=POST path=/48552026/11/11 23:58:48 INFO Received complete upload request method=POST path=/48562026/11/11 23:58:48 INFO Completed upload id=21 objects=8 duration=830.548µs48572026/11/11 23:58:48 INFO Completed upload id=29 objects=5 duration=959.369µs48582026/11/11 23:58:48 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048592026/11/11 23:58:48 INFO Received push request method=POST path=/48602026/11/11 23:58:48 INFO Vacuumed table table=pending_closures48612026/11/11 23:58:48 INFO Vacuumed table table=pending_objects48622026/11/11 23:58:48 INFO Vacuumed table table=multipart_uploads48632026/11/11 23:58:48 INFO Vacuumed table table=closures48642026/11/11 23:58:48 INFO Vacuumed table table=objects48652026/11/11 23:58:48 INFO Received complete upload request method=POST path=/48662026/11/11 23:58:48 INFO Completed upload id=25 objects=9 duration=1.06395ms48672026/11/11 23:58:48 INFO Aborted multipart uploads count=0 kept=048682026/11/11 23:58:48 INFO Received complete push request method=POST path=/48692026/11/11 23:58:48 INFO Received complete upload request method=POST path=/48702026/11/11 23:58:48 INFO Completed upload id=30 objects=2 duration=1.02557ms48712026/11/11 23:58:48 INFO Completed push id=19 objects=14 duration=1.643636ms48722026/11/11 23:58:48 INFO Received complete upload request method=POST path=/48732026/11/11 23:58:48 INFO Completed upload id=18 objects=2 duration=2.0699ms48742026/11/11 23:58:48 INFO Received complete upload request method=POST path=/48752026/11/11 23:58:48 INFO Received complete upload request method=POST path=/48762026/11/11 23:58:48 INFO Completed upload id=22 objects=7 duration=1.123671ms48772026/11/11 23:58:48 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."48782026/11/11 23:58:48 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."48792026/11/11 23:58:48 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."48802026/11/11 23:58:48 INFO Completed upload id=27 objects=3 duration=2.502144ms48812026/11/11 23:58:48 INFO Received push request method=POST path=/48822026/11/11 23:58:48 INFO Received uploads request method=POST path=/48832026/11/11 23:58:48 INFO Received complete push request method=POST path=/48842026/11/11 23:58:48 INFO Completed push id=31 objects=20 duration=1.140051ms48852026/11/11 23:58:48 INFO Received push request method=POST path=/48862026/11/11 23:58:48 INFO Received push request method=POST path=/48872026/11/11 23:58:48 INFO Received complete push request method=POST path=/48882026/11/11 23:58:48 INFO Completed push id=24 objects=5 duration=987.19µs48892026/11/11 23:58:48 INFO Received complete upload request method=POST path=/48902026/11/11 23:58:48 INFO Completed upload id=28 objects=6 duration=737.347µs48912026/11/11 23:58:48 INFO Received complete upload request method=POST path=/48922026/11/11 23:58:48 INFO Received complete push request method=POST path=/48932026/11/11 23:58:48 INFO Completed upload id=32 objects=5 duration=1.001069ms48942026/11/11 23:58:48 INFO Completed push id=30 objects=4 duration=1.132031ms48952026/11/11 23:58:48 INFO Received complete upload request method=POST path=/48962026/11/11 23:58:48 INFO Completed upload id=20 objects=6 duration=1.216631ms48972026/11/11 23:58:48 INFO Received complete upload request method=POST path=/48982026/11/11 23:58:48 INFO Completed upload id=25 objects=8 duration=873.728µs48992026/11/11 23:58:48 INFO Received complete upload request method=POST path=/49002026/11/11 23:58:48 INFO Received push request method=POST path=/49012026/11/11 23:58:48 INFO Aborted multipart uploads count=0 kept=049022026/11/11 23:58:48 INFO Completed upload id=13 objects=7 duration=1.02825ms49032026/11/11 23:58:48 INFO Received complete push request method=POST path=/49042026/11/11 23:58:48 WARN Force mode enabled - objects will be deleted immediately without grace period49052026/11/11 23:58:48 INFO Completed push id=31 objects=7 duration=1.070211ms49062026/11/11 23:58:48 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049072026/11/11 23:58:48 INFO Vacuumed table table=pending_closures49082026/11/11 23:58:48 INFO Received complete upload request method=POST path=/49092026/11/11 23:58:48 INFO Vacuumed table table=pending_objects49102026/11/11 23:58:48 INFO Vacuumed table table=multipart_uploads49112026/11/11 23:58:48 INFO Completed upload id=33 objects=3 duration=774.528µs49122026/11/11 23:58:48 INFO Vacuumed table table=closures49132026/11/11 23:58:48 INFO Vacuumed table table=objects49142026/11/11 23:58:48 INFO Received complete push request method=POST path=/49152026/11/11 23:58:48 INFO Completed push id=26 objects=4 duration=926.189µs49162026/11/11 23:58:48 INFO Received complete upload request method=POST path=/49172026/11/11 23:58:48 INFO Received complete push request method=POST path=/49182026/11/11 23:58:48 INFO Completed upload id=20 objects=4 duration=1.089911ms49192026/11/11 23:58:48 INFO Received complete upload request method=POST path=/49202026/11/11 23:58:48 INFO Completed push id=28 objects=11 duration=2.04474ms49212026/11/11 23:58:48 INFO Completed upload id=32 objects=5 duration=1.320493ms49222026/11/11 23:58:48 INFO Aborted multipart uploads count=0 kept=049232026/11/11 23:58:48 WARN Force mode enabled - objects will be deleted immediately without grace period49242026/11/11 23:58:48 INFO Received uploads request method=POST path=/49252026/11/11 23:58:48 INFO Received complete push request method=POST path=/49262026/11/11 23:58:48 INFO Completed push id=30 objects=2 duration=829.628µs49272026/11/11 23:58:48 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049282026/11/11 23:58:48 INFO Received complete upload request method=POST path=/49292026/11/11 23:58:48 INFO Vacuumed table table=pending_closures49302026/11/11 23:58:48 INFO Completed upload id=23 objects=7 duration=749.027µs49312026/11/11 23:58:48 INFO Vacuumed table table=pending_objects49322026/11/11 23:58:48 INFO Vacuumed table table=multipart_uploads49332026/11/11 23:58:48 INFO Vacuumed table table=closures49342026/11/11 23:58:48 INFO Vacuumed table table=objects49352026/11/11 23:58:48 INFO Received push request method=POST path=/49362026/11/11 23:58:48 INFO Received complete push request method=POST path=/49372026/11/11 23:58:48 INFO Received complete upload request method=POST path=/49382026/11/11 23:58:48 INFO Completed push id=32 objects=7 duration=1.09617ms49392026/11/11 23:58:48 INFO Completed upload id=21 objects=2 duration=906.909µs49402026/11/11 23:58:48 INFO Received push request method=POST path=/49412026/11/11 23:58:48 INFO Received complete push request method=POST path=/49422026/11/11 23:58:48 INFO Completed push id=29 objects=24 duration=2.845887ms49432026/11/11 23:58:48 INFO Received complete upload request method=POST path=/49442026/11/11 23:58:48 INFO Received uploads request method=POST path=/49452026/11/11 23:58:48 INFO Completed upload id=34 objects=24 duration=1.168931ms49462026/11/11 23:58:48 INFO Received complete push request method=POST path=/49472026/11/11 23:58:48 INFO Received complete upload request method=POST path=/49482026/11/11 23:58:48 INFO Received complete upload request method=POST path=/49492026/11/11 23:58:48 INFO Completed upload id=33 objects=6 duration=926.209µs49502026/11/11 23:58:48 INFO Completed upload id=24 objects=18 duration=1.116271ms49512026/11/11 23:58:48 INFO Aborted multipart uploads count=0 kept=049522026/11/11 23:58:48 INFO Completed push id=31 objects=14 duration=2.682025ms49532026/11/11 23:58:48 WARN Force mode enabled - objects will be deleted immediately without grace period49542026/11/11 23:58:48 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=049552026/11/11 23:58:48 INFO Vacuumed table table=pending_closures49562026/11/11 23:58:48 INFO Aborted multipart uploads count=0 kept=049572026/11/11 23:58:48 INFO Aborted multipart uploads count=0 kept=049582026/11/11 23:58:48 INFO Vacuumed table table=pending_objects49592026/11/11 23:58:48 INFO Vacuumed table table=multipart_uploads49602026/11/11 23:58:48 WARN Force mode enabled - objects will be deleted immediately without grace period49612026/11/11 23:58:48 WARN Force mode enabled - objects will be deleted immediately without grace period49622026/11/11 23:58:48 INFO Received complete upload request method=POST path=/49632026/11/11 23:58:48 INFO Vacuumed table table=closures49642026/11/11 23:58:48 INFO Completed upload id=34 objects=3 duration=674.267µs49652026/11/11 23:58:48 INFO Received complete push request method=POST path=/49662026/11/11 23:58:48 INFO Vacuumed table table=objects49672026/11/11 23:58:48 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049682026/11/11 23:58:48 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049692026/11/11 23:58:48 INFO Vacuumed table table=pending_closures49702026/11/11 23:58:48 INFO Completed push id=25 objects=2 duration=2.024099ms49712026/11/11 23:58:48 INFO Vacuumed table table=pending_closures49722026/11/11 23:58:48 INFO Vacuumed table table=pending_objects49732026/11/11 23:58:48 INFO Vacuumed table table=multipart_uploads49742026/11/11 23:58:48 INFO Vacuumed table table=closures49752026/11/11 23:58:48 INFO Vacuumed table table=pending_objects49762026/11/11 23:58:48 INFO Vacuumed table table=objects49772026/11/11 23:58:48 INFO Vacuumed table table=multipart_uploads49782026/11/11 23:58:48 INFO Vacuumed table table=closures49792026/11/11 23:58:48 INFO Vacuumed table table=objects49802026/11/11 23:58:48 INFO Aborted multipart uploads count=0 kept=049812026/11/11 23:58:48 INFO Received complete push request method=POST path=/49822026/11/11 23:58:48 INFO Received uploads request method=POST path=/49832026/11/11 23:58:48 WARN Force mode enabled - objects will be deleted immediately without grace period49842026/11/11 23:58:48 INFO Completed push id=27 objects=4 duration=963.629µs49852026/11/11 23:58:48 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049862026/11/11 23:58:48 INFO Vacuumed table table=pending_closures49872026/11/11 23:58:48 INFO Vacuumed table table=pending_objects49882026/11/11 23:58:48 INFO Vacuumed table table=multipart_uploads49892026/11/11 23:58:48 INFO Vacuumed table table=closures49902026/11/11 23:58:48 INFO Vacuumed table table=objects49912026/11/11 23:58:48 INFO Received uploads request method=POST path=/49922026/11/11 23:58:48 INFO Received complete push request method=POST path=/49932026/11/11 23:58:48 INFO Completed push id=28 objects=26 duration=1.392513ms49942026/11/11 23:58:48 INFO Received push request method=POST path=/49952026/11/11 23:58:48 INFO Received complete push request method=POST path=/49962026/11/11 23:58:48 INFO Completed push id=7 objects=7 duration=1.390513ms49972026/11/11 23:58:48 INFO Received complete push request method=POST path=/49982026/11/11 23:58:48 INFO Completed push id=30 objects=3 duration=922.889µs49992026/11/11 23:58:48 INFO Received complete push request method=POST path=/50002026/11/11 23:58:48 INFO Completed push id=24 objects=24 duration=6.196579ms50012026/11/11 23:58:48 INFO Received complete push request method=POST path=/50022026/11/11 23:58:48 INFO Received complete push request method=POST path=/50032026/11/11 23:58:48 INFO Completed push id=32 objects=19 duration=10.181197ms50042026/11/11 23:58:48 INFO Completed push id=12 objects=6 duration=15.486068ms50052026/11/11 23:58:48 INFO Aborted multipart uploads count=0 kept=050062026/11/11 23:58:48 INFO Aborted multipart uploads count=0 kept=050072026/11/11 23:58:48 WARN Force mode enabled - objects will be deleted immediately without grace period50082026/11/11 23:58:48 WARN Force mode enabled - objects will be deleted immediately without grace period50092026/11/11 23:58:48 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050102026/11/11 23:58:48 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050112026/11/11 23:58:48 INFO Vacuumed table table=pending_closures50122026/11/11 23:58:48 INFO Vacuumed table table=pending_closures50132026/11/11 23:58:48 INFO Vacuumed table table=pending_objects50142026/11/11 23:58:48 INFO Vacuumed table table=pending_objects50152026/11/11 23:58:48 INFO Vacuumed table table=multipart_uploads50162026/11/11 23:58:48 INFO Vacuumed table table=multipart_uploads50172026/11/11 23:58:48 INFO Vacuumed table table=closures50182026/11/11 23:58:48 INFO Vacuumed table table=closures50192026/11/11 23:58:48 INFO Vacuumed table table=objects50202026/11/11 23:58:48 INFO Vacuumed table table=objects50212026/11/11 23:58:48 INFO Received uploads request method=POST path=/50222026/11/11 23:58:48 INFO Received complete push request method=POST path=/50232026/11/11 23:58:48 INFO Completed push id=13 objects=22 duration=1.201051ms50242026/11/11 23:58:48 INFO Received uploads request method=POST path=/50252026/11/11 23:58:48 INFO Received complete push request method=POST path=/50262026/11/11 23:58:48 INFO Completed push id=21 objects=6 duration=968.689µs50272026/11/11 23:58:48 INFO Received complete upload request method=POST path=/50282026/11/11 23:58:48 INFO Completed upload id=22 objects=7 duration=838.708µs50292026/11/11 23:58:48 INFO Received push request method=POST path=/50302026/11/11 23:58:48 INFO Received complete push request method=POST path=/50312026/11/11 23:58:48 INFO Completed push id=24 objects=4 duration=965.029µs50322026/11/11 23:58:48 INFO Received uploads request method=POST path=/50332026/11/11 23:58:48 INFO Received complete push request method=POST path=/50342026/11/11 23:58:48 INFO Completed push id=25 objects=9 duration=1.058071ms50352026/11/11 23:58:48 INFO Aborted multipart uploads count=0 kept=050362026/11/11 23:58:48 WARN Force mode enabled - objects will be deleted immediately without grace period50372026/11/11 23:58:48 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050382026/11/11 23:58:48 INFO Vacuumed table table=pending_closures50392026/11/11 23:58:48 INFO Vacuumed table table=pending_objects50402026/11/11 23:58:48 INFO Vacuumed table table=multipart_uploads50412026/11/11 23:58:48 INFO Vacuumed table table=closures50422026/11/11 23:58:48 INFO Vacuumed table table=objects50432026/11/11 23:58:48 INFO Received complete upload request method=POST path=/50442026/11/11 23:58:48 INFO Completed upload id=26 objects=3 duration=858.808µs50452026/11/11 23:58:48 INFO Received complete upload request method=POST path=/50462026/11/11 23:58:48 INFO Completed upload id=27 objects=7 duration=1.12751ms50472026/11/11 23:58:48 INFO Received push request method=POST path=/50482026/11/11 23:58:48 INFO Received complete upload request method=POST path=/50492026/11/11 23:58:48 INFO Completed upload id=28 objects=3 duration=848.029µs50502026/11/11 23:58:48 INFO Received complete upload request method=POST path=/50512026/11/11 23:58:48 INFO Completed upload id=23 objects=7 duration=2.525884ms50522026/11/11 23:58:48 INFO Received complete push request method=POST path=/50532026/11/11 23:58:48 INFO Completed push id=29 objects=23 duration=1.313412ms50542026/11/11 23:58:48 INFO Received complete upload request method=POST path=/50552026/11/11 23:58:48 INFO Completed upload id=26 objects=5 duration=1.626875ms50562026/11/11 23:58:48 INFO Received complete push request method=POST path=/50572026/11/11 23:58:48 INFO Completed push id=30 objects=6 duration=1.182191ms50582026/11/11 23:58:48 INFO Received complete push request method=POST path=/50592026/11/11 23:58:48 INFO Completed push id=27 objects=3 duration=1.701596ms50602026/11/11 23:58:48 INFO Received complete push request method=POST path=/50612026/11/11 23:58:48 INFO Completed push id=33 objects=3 duration=1.07595ms50622026/11/11 23:58:48 INFO Received complete upload request method=POST path=/50632026/11/11 23:58:48 INFO Completed upload id=28 objects=5 duration=841.468µs50642026/11/11 23:58:48 INFO Received complete push request method=POST path=/50652026/11/11 23:58:48 INFO Received complete push request method=POST path=/50662026/11/11 23:58:48 INFO Completed push id=29 objects=17 duration=1.097191ms50672026/11/11 23:58:48 INFO Completed push id=34 objects=6 duration=10.998384ms50682026/11/11 23:58:48 INFO Received complete push request method=POST path=/50692026/11/11 23:58:48 INFO Aborted multipart uploads count=0 kept=050702026/11/11 23:58:48 INFO Completed push id=35 objects=8 duration=1.07795ms50712026/11/11 23:58:48 WARN Force mode enabled - objects will be deleted immediately without grace period50722026/11/11 23:58:48 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050732026/11/11 23:58:48 INFO Vacuumed table table=pending_closures50742026/11/11 23:58:48 INFO Vacuumed table table=pending_objects50752026/11/11 23:58:48 INFO Vacuumed table table=multipart_uploads50762026/11/11 23:58:48 INFO Vacuumed table table=closures50772026/11/11 23:58:48 INFO Vacuumed table table=objects50782026/11/11 23:58:48 INFO Received complete upload request method=POST path=/50792026/11/11 23:58:48 INFO Completed upload id=36 objects=9 duration=858.788µs50802026/11/11 23:58:48 INFO Aborted multipart uploads count=0 kept=050812026/11/11 23:58:48 WARN Force mode enabled - objects will be deleted immediately without grace period50822026/11/11 23:58:48 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050832026/11/11 23:58:48 INFO Vacuumed table table=pending_closures50842026/11/11 23:58:48 INFO Vacuumed table table=pending_objects50852026/11/11 23:58:48 INFO Vacuumed table table=multipart_uploads50862026/11/11 23:58:48 INFO Vacuumed table table=closures50872026/11/11 23:58:48 INFO Vacuumed table table=objects5088--- PASS: FuzzPushGC (0.01s)5089 --- PASS: FuzzPushGC/seed#0 (4.11s)5090 --- PASS: FuzzPushGC/seed#14 (4.11s)5091 --- PASS: FuzzPushGC/seed#8 (4.21s)5092 --- PASS: FuzzPushGC/seed#13 (4.26s)5093 --- PASS: FuzzPushGC/seed#5 (4.38s)5094 --- PASS: FuzzPushGC/seed#12 (4.45s)5095 --- PASS: FuzzPushGC/seed#15 (4.46s)5096 --- PASS: FuzzPushGC/seed#3 (4.59s)5097 --- PASS: FuzzPushGC/seed#11 (4.75s)5098 --- PASS: FuzzPushGC/seed#6 (5.06s)5099 --- PASS: FuzzPushGC/seed#2 (5.11s)5100 --- PASS: FuzzPushGC/seed#9 (5.12s)5101 --- PASS: FuzzPushGC/seed#10 (5.13s)5102 --- PASS: FuzzPushGC/seed#7 (5.26s)5103 --- PASS: FuzzPushGC/seed#1 (5.55s)5104 --- PASS: FuzzPushGC/seed#4 (5.60s)5105PASS51062026-11-11 23:58:49.082 UTC [130] LOG: received smart shutdown request51072026-11-11 23:58:49.091 UTC [130] LOG: background worker "logical replication launcher" (PID 172) exited with exit code 151082026-11-11 23:58:49.768 UTC [167] LOG: checkpoint complete: wrote 3799 buffers (23.2%), wrote 5 SLRU buffers; 0 WAL file(s) added, 0 removed, 33 recycled; write=6.084 s, sync=0.433 s, total=6.647 s; sync files=42371, longest=0.003 s, average=0.001 s; distance=535473 kB, estimate=535473 kB; lsn=0/26DE8F40, redo lsn=0/2225232051092026-11-11 23:58:49.774 UTC [167] LOG: shutting down51102026-11-11 23:58:49.775 UTC [167] LOG: checkpoint starting: shutdown immediate51112026-11-11 23:58:50.316 UTC [167] LOG: checkpoint complete: wrote 8646 buffers (52.8%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 4 recycled; write=0.179 s, sync=0.352 s, total=0.542 s; sync files=2600, longest=0.004 s, average=0.001 s; distance=77403 kB, estimate=489666 kB; lsn=0/26DE8FB8, redo lsn=0/26DE8FB851122026-11-11 23:58:50.524 UTC [130] LOG: database system is shut down5113Running OIDC tests...5114=== RUN TestAudienceForIssuer5115=== PAUSE TestAudienceForIssuer5116=== RUN TestGlobMatch5117=== PAUSE TestGlobMatch5118=== RUN TestValidateToken_ValidToken5119=== PAUSE TestValidateToken_ValidToken5120=== RUN TestValidateToken_WrongAudience5121=== PAUSE TestValidateToken_WrongAudience5122=== RUN TestValidateToken_Expired5123=== PAUSE TestValidateToken_Expired5124=== RUN TestValidateToken_BoundClaimsMismatch5125=== PAUSE TestValidateToken_BoundClaimsMismatch5126=== RUN TestValidateToken_BoundSubjectMismatch5127=== PAUSE TestValidateToken_BoundSubjectMismatch5128=== RUN TestValidateToken_MultipleProviders5129=== PAUSE TestValidateToken_MultipleProviders5130=== RUN TestValidateToken_NoMatchingProvider5131=== PAUSE TestValidateToken_NoMatchingProvider5132=== RUN TestValidateToken_KubernetesServiceAccount5133=== PAUSE TestValidateToken_KubernetesServiceAccount5134=== RUN TestNewValidator_KubernetesRequiresCA5135=== PAUSE TestNewValidator_KubernetesRequiresCA5136=== RUN TestValidateToken_KubernetesIssuerFromOwnToken5137=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken5138=== RUN TestPins_ReservedForMatchingRule5139=== PAUSE TestPins_ReservedForMatchingRule5140=== RUN TestPins_TopLevelShorthand5141=== PAUSE TestPins_TopLevelShorthand5142=== RUN TestPins_ConfigValidation5143=== PAUSE TestPins_ConfigValidation5144=== RUN TestScopes_LegacyProviderDefaultsToWrite5145=== PAUSE TestScopes_LegacyProviderDefaultsToWrite5146=== RUN TestScopes_Rules5147=== PAUSE TestScopes_Rules5148=== RUN TestScopes_ConfigValidation5149=== PAUSE TestScopes_ConfigValidation5150=== CONT TestAudienceForIssuer5151=== CONT TestValidateToken_KubernetesServiceAccount5152=== CONT TestValidateToken_MultipleProviders5153--- PASS: TestAudienceForIssuer (0.00s)5154=== CONT TestValidateToken_Expired5155=== CONT TestValidateToken_WrongAudience5156=== CONT TestValidateToken_ValidToken5157=== CONT TestGlobMatch5158=== RUN TestGlobMatch/foo_foo5159=== PAUSE TestGlobMatch/foo_foo5160=== RUN TestGlobMatch/foo_bar5161=== PAUSE TestGlobMatch/foo_bar5162=== RUN TestGlobMatch/*_5163=== PAUSE TestGlobMatch/*_5164=== RUN TestGlobMatch/*_anything5165=== PAUSE TestGlobMatch/*_anything5166=== RUN TestGlobMatch/foo*_foo5167=== PAUSE TestGlobMatch/foo*_foo5168=== RUN TestGlobMatch/foo*_foobar5169=== PAUSE TestGlobMatch/foo*_foobar5170=== RUN TestGlobMatch/foo*_bar5171=== PAUSE TestGlobMatch/foo*_bar5172=== RUN TestGlobMatch/*bar_bar5173=== PAUSE TestGlobMatch/*bar_bar5174=== RUN TestGlobMatch/*bar_foobar5175=== PAUSE TestGlobMatch/*bar_foobar5176=== RUN TestGlobMatch/*bar_foo5177=== PAUSE TestGlobMatch/*bar_foo5178=== RUN TestGlobMatch/foo*bar_foobar5179=== PAUSE TestGlobMatch/foo*bar_foobar5180=== RUN TestGlobMatch/foo*bar_foo123bar5181=== PAUSE TestGlobMatch/foo*bar_foo123bar5182=== RUN TestGlobMatch/foo*bar_foobarbaz5183=== PAUSE TestGlobMatch/foo*bar_foobarbaz5184=== RUN TestGlobMatch/*/*_foo/bar5185=== PAUSE TestGlobMatch/*/*_foo/bar5186=== RUN TestGlobMatch/*/*_foo5187=== PAUSE TestGlobMatch/*/*_foo5188=== RUN TestGlobMatch/refs/heads/*_refs/heads/main5189=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main5190=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.05191=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.05192=== RUN TestGlobMatch/refs/*/main_refs/heads/main5193=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main5194=== RUN TestGlobMatch/fo?_foo5195=== PAUSE TestGlobMatch/fo?_foo5196=== RUN TestGlobMatch/fo?_fo5197=== PAUSE TestGlobMatch/fo?_fo5198=== RUN TestGlobMatch/fo?_fooo5199=== PAUSE TestGlobMatch/fo?_fooo5200=== RUN TestGlobMatch/?oo_foo5201=== PAUSE TestGlobMatch/?oo_foo5202=== RUN TestGlobMatch/?oo_boo5203=== PAUSE TestGlobMatch/?oo_boo5204=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5205=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5206=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5207=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5208=== CONT TestGlobMatch/foo_foo5209=== CONT TestValidateToken_BoundSubjectMismatch5210=== CONT TestValidateToken_NoMatchingProvider5211=== CONT TestPins_ConfigValidation5212=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5213=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5214=== CONT TestGlobMatch/?oo_boo5215=== CONT TestGlobMatch/?oo_foo5216=== CONT TestGlobMatch/fo?_fooo5217=== CONT TestGlobMatch/fo?_fo5218=== CONT TestGlobMatch/fo?_foo5219=== CONT TestGlobMatch/refs/*/main_refs/heads/main5220=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.05221=== CONT TestGlobMatch/refs/heads/*_refs/heads/main5222=== CONT TestGlobMatch/*/*_foo5223=== CONT TestGlobMatch/*/*_foo/bar5224=== CONT TestGlobMatch/foo*bar_foobarbaz5225=== CONT TestGlobMatch/foo*bar_foo123bar5226=== CONT TestGlobMatch/foo*bar_foobar5227=== CONT TestGlobMatch/*bar_foo5228=== CONT TestGlobMatch/*bar_foobar5229=== CONT TestGlobMatch/*bar_bar5230=== CONT TestGlobMatch/foo*_bar5231=== CONT TestGlobMatch/foo*_foobar5232=== CONT TestGlobMatch/foo*_foo5233=== CONT TestGlobMatch/*_anything5234=== CONT TestGlobMatch/*_5235=== CONT TestGlobMatch/foo_bar5236--- PASS: TestGlobMatch (0.00s)5237 --- PASS: TestGlobMatch/foo_foo (0.00s)5238 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)5239 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)5240 --- PASS: TestGlobMatch/?oo_boo (0.00s)5241 --- PASS: TestGlobMatch/?oo_foo (0.00s)5242 --- PASS: TestGlobMatch/fo?_fooo (0.00s)5243 --- PASS: TestGlobMatch/fo?_fo (0.00s)5244 --- PASS: TestGlobMatch/fo?_foo (0.00s)5245 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)5246 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)5247 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)5248 --- PASS: TestGlobMatch/*/*_foo (0.00s)5249 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)5250 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)5251 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)5252 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)5253 --- PASS: TestGlobMatch/*bar_foo (0.00s)5254 --- PASS: TestGlobMatch/*bar_foobar (0.00s)5255 --- PASS: TestGlobMatch/*bar_bar (0.00s)5256 --- PASS: TestGlobMatch/foo*_bar (0.00s)5257 --- PASS: TestGlobMatch/foo*_foobar (0.00s)5258 --- PASS: TestGlobMatch/foo*_foo (0.00s)5259 --- PASS: TestGlobMatch/*_anything (0.00s)5260 --- PASS: TestGlobMatch/*_ (0.00s)5261 --- PASS: TestGlobMatch/foo_bar (0.00s)5262=== CONT TestScopes_ConfigValidation5263=== CONT TestScopes_Rules5264=== CONT TestScopes_LegacyProviderDefaultsToWrite5265=== CONT TestPins_ReservedForMatchingRule5266=== CONT TestPins_TopLevelShorthand5267=== CONT TestValidateToken_KubernetesIssuerFromOwnToken5268=== CONT TestNewValidator_KubernetesRequiresCA5269=== CONT TestValidateToken_BoundClaimsMismatch5270--- PASS: TestScopes_ConfigValidation (0.00s)5271--- PASS: TestPins_ConfigValidation (1.77s)52722026/11/11 23:58:56 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:3359952732026/11/11 23:58:57 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:42667/oidc52742026/11/11 23:58:57 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:46787/oidc52752026/11/11 23:58:57 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:42263/oidc5276--- PASS: TestValidateToken_KubernetesServiceAccount (3.07s)5277--- PASS: TestScopes_LegacyProviderDefaultsToWrite (2.33s)5278--- PASS: TestValidateToken_Expired (3.06s)5279--- PASS: TestValidateToken_ValidToken (3.03s)52802026/11/11 23:58:57 http: TLS handshake error from 127.0.0.1:35308: remote error: tls: bad certificate5281--- PASS: TestNewValidator_KubernetesRequiresCA (2.33s)52822026/11/11 23:58:57 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC12352832026/11/11 23:58:57 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:35229/oidc5284--- PASS: TestPins_TopLevelShorthand (2.38s)52852026/11/11 23:58:57 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:39045/oidc5286--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (2.38s)5287--- PASS: TestPins_ReservedForMatchingRule (2.38s)52882026/11/11 23:58:57 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:41039/oidc5289--- PASS: TestValidateToken_WrongAudience (3.14s)52902026/11/11 23:58:57 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:35231/oidc5291--- PASS: TestValidateToken_BoundSubjectMismatch (3.12s)52922026/11/11 23:58:57 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:37869/oidc5293--- PASS: TestScopes_Rules (2.48s)52942026/11/11 23:58:57 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:34691/oidc5295--- PASS: TestValidateToken_BoundClaimsMismatch (2.52s)52962026/11/11 23:58:57 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:43499/oidc5297--- PASS: TestValidateToken_NoMatchingProvider (3.21s)52982026/11/11 23:58:57 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:39225/oidc52992026/11/11 23:58:57 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:34661/oidc5300--- PASS: TestValidateToken_MultipleProviders (3.45s)5301PASS5302Running hook tests...5303=== RUN TestSendPathsEmpty5304=== PAUSE TestSendPathsEmpty5305=== RUN TestQueueEnqueueAndFetch5306=== PAUSE TestQueueEnqueueAndFetch5307=== RUN TestQueueDeduplication5308=== PAUSE TestQueueDeduplication5309=== RUN TestQueueRemove5310=== PAUSE TestQueueRemove5311=== RUN TestQueueFetchBatchLimit5312=== PAUSE TestQueueFetchBatchLimit5313=== RUN TestQueueRetryMovesToBack5314=== PAUSE TestQueueRetryMovesToBack5315=== RUN TestQueueFetchRemoveLifecycle5316=== PAUSE TestQueueFetchRemoveLifecycle5317=== RUN TestQueueConcurrentWriters5318=== PAUSE TestQueueConcurrentWriters5319=== RUN TestQueueRemoveLargeClosure5320=== PAUSE TestQueueRemoveLargeClosure5321=== RUN TestServerClientIntegration5322--- PASS: TestServerClientIntegration (0.00s)5323=== RUN TestServerQueueError53242026/11/11 23:58:57 ERROR Failed to queue paths error="permission denied" count=15325--- PASS: TestServerQueueError (0.00s)5326=== RUN TestGetListenerSocketActivation5327 server_test.go:220: === RUN TestGetListenerSocketActivation5328 --- PASS: TestGetListenerSocketActivation (0.00s)5329 PASS5330 5331--- PASS: TestGetListenerSocketActivation (1.40s)5332=== RUN TestServerStalledClientDoesNotBlockShutdown5333=== PAUSE TestServerStalledClientDoesNotBlockShutdown5334=== RUN TestServerBacksOffOnAcceptErrors5335=== PAUSE TestServerBacksOffOnAcceptErrors5336=== RUN TestDrainIsolatesPoisonPath5337=== PAUSE TestDrainIsolatesPoisonPath5338=== RUN TestRunNotBlockedByPoisonHead5339=== PAUSE TestRunNotBlockedByPoisonHead5340=== RUN TestDrainGivesUpWhenServerDown5341=== PAUSE TestDrainGivesUpWhenServerDown5342=== RUN TestFailedPathPrunedByLaterClosure5343=== PAUSE TestFailedPathPrunedByLaterClosure5344=== RUN TestWorkerUploadsAndRemoves5345=== PAUSE TestWorkerUploadsAndRemoves5346=== RUN TestWorkerSkipsGCdPaths5347=== PAUSE TestWorkerSkipsGCdPaths5348=== RUN TestWorkerPrunesClosureDeps5349=== PAUSE TestWorkerPrunesClosureDeps5350=== RUN TestDrainTimeout5351=== PAUSE TestDrainTimeout5352=== RUN TestWorkerRemoveFailureIsNotProgress5353=== PAUSE TestWorkerRemoveFailureIsNotProgress5354=== RUN TestWorkerKeepsPathItCannotStat5355=== PAUSE TestWorkerKeepsPathItCannotStat5356=== CONT TestSendPathsEmpty5357--- PASS: TestSendPathsEmpty (0.00s)5358=== CONT TestWorkerKeepsPathItCannotStat5359=== CONT TestWorkerRemoveFailureIsNotProgress5360=== CONT TestServerStalledClientDoesNotBlockShutdown5361=== CONT TestQueueRemove5362=== CONT TestDrainTimeout5363=== CONT TestQueueFetchBatchLimit5364=== CONT TestQueueFetchRemoveLifecycle5365=== CONT TestWorkerPrunesClosureDeps5366=== CONT TestWorkerSkipsGCdPaths5367=== CONT TestWorkerUploadsAndRemoves5368=== CONT TestQueueRetryMovesToBack5369=== CONT TestQueueDeduplication5370=== CONT TestDrainIsolatesPoisonPath5371=== CONT TestQueueEnqueueAndFetch5372=== CONT TestFailedPathPrunedByLaterClosure5373=== CONT TestDrainGivesUpWhenServerDown5374=== CONT TestServerBacksOffOnAcceptErrors53752000/01/01 00:00:00 ERROR Accept failed error="too many open files"53762000/01/01 00:00:00 ERROR Accept failed error="too many open files"53772000/01/01 00:00:00 ERROR Accept failed error="too many open files"53782000/01/01 00:00:00 ERROR Accept failed error="too many open files"53792000/01/01 00:00:00 ERROR Accept failed error="too many open files"53802000/01/01 00:00:00 ERROR Accept failed error="too many open files"53812000/01/01 00:00:00 ERROR Accept failed error="too many open files"53822000/01/01 00:00:00 ERROR Accept failed error="too many open files"5383--- PASS: TestQueueFetchRemoveLifecycle (0.00s)5384--- PASS: TestServerBacksOffOnAcceptErrors (0.00s)5385=== CONT TestQueueRemoveLargeClosure5386=== CONT TestQueueConcurrentWriters5387=== CONT TestRunNotBlockedByPoisonHead53882026/11/11 23:58:59 INFO Uploading batch count=253892026/11/11 23:58:59 ERROR Upload failed error="upload failed" count=253902026/11/11 23:58:59 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1130590893/002/a5391--- PASS: TestQueueRemove (0.01s)53922026/11/11 23:58:59 INFO Upload queue status pending=253932026/11/11 23:58:59 INFO Uploading batch count=25394--- PASS: TestQueueDeduplication (0.01s)5395--- PASS: TestQueueEnqueueAndFetch (0.02s)53962026/11/11 23:58:59 INFO Upload queue status pending=253972026/11/11 23:58:59 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerSkipsGCdPaths4087246852/002/nonexistent53982026/11/11 23:58:59 INFO Uploading batch count=15399--- PASS: TestQueueRetryMovesToBack (0.02s)54002026/11/11 23:58:59 INFO Upload queue status pending=354012026/11/11 23:58:59 INFO Uploading batch count=154022026/11/11 23:58:59 ERROR Upload failed error="upload failed" count=154032026/11/11 23:58:59 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1130590893/002/b54042026/11/11 23:58:59 INFO Uploading batch count=254052026/11/11 23:58:59 ERROR Upload failed error="upload failed" count=254062026/11/11 23:58:59 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1130590893/002/c54072026/11/11 23:58:59 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1130590893/002/d5408--- PASS: TestQueueFetchBatchLimit (0.03s)54092026/11/11 23:58:59 INFO Uploading batch count=254102026/11/11 23:58:59 ERROR Upload failed error="upload failed" count=254112026/11/11 23:58:59 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1130590893/002/e54122026/11/11 23:58:59 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1130590893/002/f54132026/11/11 23:58:59 ERROR Drain finished with paths left in queue remaining=1054142000/01/01 00:00:00 INFO Uploading batch count=25415--- PASS: TestDrainGivesUpWhenServerDown (0.03s)54162000/01/01 00:00:00 ERROR Upload failed error="context deadline exceeded" count=254172000/01/01 00:00:00 ERROR Drain finished with paths left in queue remaining=454182026/11/11 23:58:59 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress3807645951/002/nonexistent54192026/11/11 23:58:59 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=154202026/11/11 23:58:59 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress3807645951/002/nonexistent54212026/11/11 23:58:59 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=154222026/11/11 23:58:59 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerRemoveFailureIsNotProgress3807645951/002/nonexistent54232026/11/11 23:58:59 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=154242026/11/11 23:58:59 ERROR Drain finished with paths left in queue remaining=15425--- PASS: TestDrainTimeout (0.03s)5426--- PASS: TestWorkerRemoveFailureIsNotProgress (0.04s)54272026/11/11 23:58:59 INFO Upload queue status pending=254282026/11/11 23:58:59 INFO Uploading batch count=154292026/11/11 23:58:59 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat4223743043/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat4223743043/002/locked/aaa: permission denied"54302026/11/11 23:58:59 INFO Uploading batch count=154312026/11/11 23:58:59 ERROR Upload failed error="upload failed" count=154322026/11/11 23:58:59 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat4223743043/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat4223743043/002/locked/aaa: permission denied"54332026/11/11 23:58:59 INFO Uploading batch count=154342026/11/11 23:58:59 WARN Cannot stat store path, will retry later path=/build/TestWorkerKeepsPathItCannotStat4223743043/002/locked/aaa error="stat /build/TestWorkerKeepsPathItCannotStat4223743043/002/locked/aaa: permission denied"54352026/11/11 23:58:59 ERROR Drain finished with paths left in queue remaining=154362026/11/11 23:58:59 INFO Uploading batch count=15437--- PASS: TestWorkerUploadsAndRemoves (0.03s)5438--- PASS: TestWorkerKeepsPathItCannotStat (0.06s)5439--- PASS: TestFailedPathPrunedByLaterClosure (0.04s)5440--- PASS: TestWorkerSkipsGCdPaths (0.04s)54412026/11/11 23:59:00 ERROR Failed to decode request error="read unix /build/hook4017464299/test.sock->@: i/o timeout"54422026/11/11 23:59:00 ERROR Failed to write response error="write unix /build/hook4017464299/test.sock->@: i/o timeout"54432026/11/11 23:59:00 INFO Uploading batch count=454442026/11/11 23:59:00 ERROR Upload failed error="upload failed" count=45445--- PASS: TestServerStalledClientDoesNotBlockShutdown (0.68s)54462026/11/11 23:59:00 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainIsolatesPoisonPath3917253390/002/bbb5447--- PASS: TestWorkerPrunesClosureDeps (0.68s)54482026/11/11 23:59:00 INFO Uploading batch count=154492026/11/11 23:59:00 ERROR Upload failed error="upload failed" count=154502026/11/11 23:59:00 INFO Uploading batch count=154512026/11/11 23:59:00 ERROR Upload failed error="upload failed" count=154522026/11/11 23:59:00 INFO Uploading batch count=154532026/11/11 23:59:00 ERROR Upload failed error="upload failed" count=154542026/11/11 23:59:00 ERROR Drain finished with paths left in queue remaining=15455--- PASS: TestDrainIsolatesPoisonPath (0.68s)5456--- PASS: TestQueueRemoveLargeClosure (0.77s)5457--- PASS: TestQueueConcurrentWriters (0.80s)54582026/11/11 23:59:00 INFO Uploading batch count=154592026/11/11 23:59:00 INFO Uploading batch count=154602026/11/11 23:59:00 INFO Uploading batch count=154612026/11/11 23:59:00 ERROR Upload failed error="upload failed" count=154622026/11/11 23:59:00 INFO Uploading batch count=154632026/11/11 23:59:00 ERROR Upload failed error="upload failed" count=154642026/11/11 23:59:00 INFO Uploading batch count=154652026/11/11 23:59:00 ERROR Upload failed error="upload failed" count=154662026/11/11 23:59:00 INFO Uploading batch count=154672026/11/11 23:59:00 ERROR Upload failed error="upload failed" count=154682026/11/11 23:59:00 ERROR Drain finished with paths left in queue remaining=15469--- PASS: TestRunNotBlockedByPoisonHead (1.04s)5470PASS5471Running signing tests...5472=== RUN TestExternalSigner5473=== PAUSE TestExternalSigner5474=== RUN TestExternalSignerWrongPublicKey5475=== PAUSE TestExternalSignerWrongPublicKey5476=== RUN TestExternalSignerErrors5477=== PAUSE TestExternalSignerErrors5478=== RUN TestExternalSignerRestartsAfterCrash5479=== PAUSE TestExternalSignerRestartsAfterCrash5480=== RUN TestExternalSignerProgramErrorKeepsProcess5481=== PAUSE TestExternalSignerProgramErrorKeepsProcess5482=== RUN TestExternalSignerClose5483=== PAUSE TestExternalSignerClose5484=== RUN TestExternalSignerRejectsInvalidPublicKey5485=== PAUSE TestExternalSignerRejectsInvalidPublicKey5486=== RUN TestExternalSignerCancellation5487=== PAUSE TestExternalSignerCancellation5488=== RUN TestGenerateFingerprint5489=== PAUSE TestGenerateFingerprint5490=== RUN TestParseSigningKey5491=== PAUSE TestParseSigningKey5492=== RUN TestSign5493=== PAUSE TestSign5494=== RUN TestSignBatch5495=== PAUSE TestSignBatch5496=== RUN TestNilKeyPublicKey5497=== PAUSE TestNilKeyPublicKey5498=== RUN TestSignErrors5499=== PAUSE TestSignErrors5500=== CONT TestExternalSigner5501=== CONT TestExternalSignerCancellation5502=== CONT TestExternalSignerProgramErrorKeepsProcess5503=== CONT TestSignErrors5504=== CONT TestExternalSignerErrors5505=== RUN TestExternalSignerErrors/exit_on_request5506=== PAUSE TestExternalSignerErrors/exit_on_request5507=== RUN TestExternalSignerErrors/invalid_JSON5508=== PAUSE TestExternalSignerErrors/invalid_JSON5509=== RUN TestExternalSignerErrors/null_signatures5510=== PAUSE TestExternalSignerErrors/null_signatures5511=== RUN TestExternalSignerErrors/wrong_signatures_shape5512=== PAUSE TestExternalSignerErrors/wrong_signatures_shape5513=== RUN TestExternalSignerErrors/too_few_signatures5514=== PAUSE TestExternalSignerErrors/too_few_signatures5515=== RUN TestExternalSignerErrors/unrequested_key5516=== PAUSE TestExternalSignerErrors/unrequested_key5517=== RUN TestExternalSignerErrors/wrong_signature_name5518=== PAUSE TestExternalSignerErrors/wrong_signature_name5519=== RUN TestExternalSignerErrors/wrong_signature_length5520=== PAUSE TestExternalSignerErrors/wrong_signature_length5521=== RUN TestExternalSignerErrors/wrong_shape5522=== PAUSE TestExternalSignerErrors/wrong_shape5523=== RUN TestExternalSignerErrors/invalid_signature_base645524=== PAUSE TestExternalSignerErrors/invalid_signature_base645525=== RUN TestExternalSignerErrors/program_error5526=== PAUSE TestExternalSignerErrors/program_error5527=== RUN TestExternalSignerErrors/missing_signatures5528=== PAUSE TestExternalSignerErrors/missing_signatures5529=== RUN TestExternalSignerErrors/missing_signature_name5530=== PAUSE TestExternalSignerErrors/missing_signature_name5531=== RUN TestExternalSignerErrors/exit_at_startup5532=== PAUSE TestExternalSignerErrors/exit_at_startup5533=== RUN TestExternalSignerErrors/null5534=== PAUSE TestExternalSignerErrors/null5535=== RUN TestExternalSignerErrors/wrong_id5536=== PAUSE TestExternalSignerErrors/wrong_id5537=== CONT TestExternalSignerErrors/exit_on_request5538=== CONT TestNilKeyPublicKey5539--- PASS: TestNilKeyPublicKey (0.00s)5540=== CONT TestSignBatch5541=== CONT TestSign5542=== CONT TestExternalSignerErrors/invalid_signature_base645543=== CONT TestParseSigningKey5544=== CONT TestGenerateFingerprint5545=== CONT TestExternalSignerRejectsInvalidPublicKey5546=== CONT TestExternalSignerRestartsAfterCrash5547=== CONT TestExternalSignerWrongPublicKey5548=== CONT TestExternalSignerClose5549=== CONT TestExternalSignerErrors/wrong_id5550=== RUN TestSignErrors/nil_key5551=== PAUSE TestSignErrors/nil_key5552=== RUN TestSignErrors/nil_narinfo5553=== PAUSE TestSignErrors/nil_narinfo5554=== RUN TestSignErrors/invalid_narinfo5555=== RUN TestExternalSignerRejectsInvalidPublicKey/#005556=== PAUSE TestExternalSignerRejectsInvalidPublicKey/#005557=== CONT TestExternalSignerErrors/null5558=== RUN TestGenerateFingerprint/basic_with_references5559=== PAUSE TestGenerateFingerprint/basic_with_references5560=== RUN TestGenerateFingerprint/no_references5561=== PAUSE TestGenerateFingerprint/no_references5562=== RUN TestGenerateFingerprint/unsorted_references_get_sorted5563=== PAUSE TestGenerateFingerprint/unsorted_references_get_sorted5564=== RUN TestGenerateFingerprint/invalid_nar_hash_prefix5565=== PAUSE TestGenerateFingerprint/invalid_nar_hash_prefix5566=== RUN TestGenerateFingerprint/invalid_nar_hash_length5567=== PAUSE TestGenerateFingerprint/invalid_nar_hash_length5568=== RUN TestGenerateFingerprint/invalid_store_path_prefix5569=== PAUSE TestGenerateFingerprint/invalid_store_path_prefix5570=== RUN TestGenerateFingerprint/invalid_reference_prefix5571=== PAUSE TestGenerateFingerprint/invalid_reference_prefix5572=== CONT TestExternalSignerErrors/wrong_signatures_shape5573=== PAUSE TestSignErrors/invalid_narinfo5574=== RUN TestSignErrors/canceled5575=== PAUSE TestSignErrors/canceled5576=== CONT TestExternalSignerErrors/missing_signature_name5577--- PASS: TestSignBatch (0.00s)5578=== CONT TestExternalSignerErrors/null_signatures5579=== CONT TestExternalSignerErrors/too_few_signatures5580--- PASS: TestSign (0.00s)5581=== CONT TestExternalSignerErrors/missing_signatures5582=== CONT TestExternalSignerErrors/program_error5583=== CONT TestExternalSignerErrors/unrequested_key5584=== CONT TestExternalSignerErrors/wrong_shape5585=== CONT TestExternalSignerErrors/wrong_signature_length5586=== CONT TestExternalSignerErrors/wrong_signature_name5587=== RUN TestParseSigningKey/valid_32-byte_key5588=== RUN TestExternalSignerRejectsInvalidPublicKey/no-colon5589=== CONT TestExternalSignerErrors/invalid_JSON5590=== CONT TestExternalSignerErrors/exit_at_startup5591=== PAUSE TestParseSigningKey/valid_32-byte_key5592=== PAUSE TestExternalSignerRejectsInvalidPublicKey/no-colon5593=== RUN TestParseSigningKey/valid_32-byte_key_with_different_name5594=== PAUSE TestParseSigningKey/valid_32-byte_key_with_different_name5595=== RUN TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5596=== PAUSE TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5597=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5598=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5599=== RUN TestParseSigningKey/no_colon5600=== PAUSE TestParseSigningKey/no_colon5601=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5602=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5603=== RUN TestParseSigningKey/empty_name5604=== CONT TestGenerateFingerprint/basic_with_references5605=== CONT TestSignErrors/nil_key5606=== CONT TestGenerateFingerprint/invalid_reference_prefix5607=== PAUSE TestParseSigningKey/empty_name5608=== CONT TestGenerateFingerprint/invalid_store_path_prefix5609=== RUN TestParseSigningKey/invalid_base645610=== CONT TestGenerateFingerprint/invalid_nar_hash_length5611=== CONT TestGenerateFingerprint/invalid_nar_hash_prefix5612=== CONT TestGenerateFingerprint/unsorted_references_get_sorted5613=== CONT TestGenerateFingerprint/no_references5614--- PASS: TestGenerateFingerprint (0.00s)5615 --- PASS: TestGenerateFingerprint/basic_with_references (0.00s)5616 --- PASS: TestGenerateFingerprint/invalid_reference_prefix (0.00s)5617 --- PASS: TestGenerateFingerprint/invalid_store_path_prefix (0.00s)5618 --- PASS: TestGenerateFingerprint/invalid_nar_hash_length (0.00s)5619 --- PASS: TestGenerateFingerprint/invalid_nar_hash_prefix (0.00s)5620 --- PASS: TestGenerateFingerprint/unsorted_references_get_sorted (0.00s)5621 --- PASS: TestGenerateFingerprint/no_references (0.00s)5622=== PAUSE TestParseSigningKey/invalid_base645623=== RUN TestParseSigningKey/wrong_length5624=== CONT TestSignErrors/canceled5625=== PAUSE TestParseSigningKey/wrong_length5626=== CONT TestSignErrors/invalid_narinfo5627=== CONT TestSignErrors/nil_narinfo5628=== CONT TestExternalSignerRejectsInvalidPublicKey/#005629=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5630=== CONT TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5631=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5632=== CONT TestExternalSignerRejectsInvalidPublicKey/no-colon5633=== CONT TestParseSigningKey/valid_32-byte_key5634=== CONT TestParseSigningKey/invalid_base645635=== CONT TestParseSigningKey/no_colon5636=== CONT TestParseSigningKey/empty_name5637=== CONT TestParseSigningKey/valid_32-byte_key_with_different_name5638--- PASS: TestExternalSignerRejectsInvalidPublicKey (0.00s)5639 --- PASS: TestExternalSignerRejectsInvalidPublicKey/#00 (0.00s)5640 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:invalid! (0.00s)5641 --- PASS: TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= (0.00s)5642 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:YQ== (0.00s)5643 --- PASS: TestExternalSignerRejectsInvalidPublicKey/no-colon (0.00s)5644--- PASS: TestSignErrors (0.00s)5645 --- PASS: TestSignErrors/nil_key (0.00s)5646 --- PASS: TestSignErrors/canceled (0.00s)5647 --- PASS: TestSignErrors/nil_narinfo (0.00s)5648 --- PASS: TestSignErrors/invalid_narinfo (0.00s)5649=== CONT TestParseSigningKey/wrong_length5650--- PASS: TestParseSigningKey (0.00s)5651 --- PASS: TestParseSigningKey/valid_32-byte_key (0.00s)5652 --- PASS: TestParseSigningKey/invalid_base64 (0.00s)5653 --- PASS: TestParseSigningKey/no_colon (0.00s)5654 --- PASS: TestParseSigningKey/empty_name (0.00s)5655 --- PASS: TestParseSigningKey/valid_32-byte_key_with_different_name (0.00s)5656 --- PASS: TestParseSigningKey/wrong_length (0.00s)5657--- PASS: TestExternalSignerWrongPublicKey (0.01s)5658--- PASS: TestExternalSignerProgramErrorKeepsProcess (0.01s)5659--- PASS: TestExternalSignerClose (0.02s)5660--- PASS: TestExternalSigner (0.02s)5661--- PASS: TestExternalSignerErrors (0.00s)5662 --- PASS: TestExternalSignerErrors/null (0.01s)5663 --- PASS: TestExternalSignerErrors/exit_on_request (0.01s)5664 --- PASS: TestExternalSignerErrors/wrong_signature_length (0.01s)5665 --- PASS: TestExternalSignerErrors/too_few_signatures (0.01s)5666 --- PASS: TestExternalSignerErrors/missing_signatures (0.01s)5667 --- PASS: TestExternalSignerErrors/invalid_JSON (0.01s)5668 --- PASS: TestExternalSignerErrors/null_signatures (0.01s)5669 --- PASS: TestExternalSignerErrors/wrong_id (0.02s)5670 --- PASS: TestExternalSignerErrors/exit_at_startup (0.01s)5671 --- PASS: TestExternalSignerErrors/wrong_signature_name (0.01s)5672 --- PASS: TestExternalSignerErrors/wrong_shape (0.02s)5673 --- PASS: TestExternalSignerErrors/wrong_signatures_shape (0.02s)5674 --- PASS: TestExternalSignerErrors/invalid_signature_base64 (0.02s)5675 --- PASS: TestExternalSignerErrors/missing_signature_name (0.02s)5676 --- PASS: TestExternalSignerErrors/program_error (0.02s)5677 --- PASS: TestExternalSignerErrors/unrequested_key (0.02s)5678--- PASS: TestExternalSignerRestartsAfterCrash (0.02s)5679--- PASS: TestExternalSignerCancellation (0.03s)5680PASS5681Running OpenBao signer tests...5682=== RUN TestSign5683=== PAUSE TestSign5684=== RUN TestSignErrors5685=== PAUSE TestSignErrors5686=== RUN TestServe5687=== PAUSE TestServe5688=== RUN TestServeRejectsGarbage5689=== PAUSE TestServeRejectsGarbage5690=== CONT TestSign5691=== CONT TestServeRejectsGarbage5692--- PASS: TestServeRejectsGarbage (0.00s)5693=== CONT TestServe5694--- PASS: TestServe (0.00s)5695=== CONT TestSignErrors5696=== RUN TestSignErrors/http_error5697=== PAUSE TestSignErrors/http_error5698=== RUN TestSignErrors/invalid_json5699=== PAUSE TestSignErrors/invalid_json5700=== RUN TestSignErrors/no_prefix5701=== PAUSE TestSignErrors/no_prefix5702=== RUN TestSignErrors/invalid_base645703=== PAUSE TestSignErrors/invalid_base645704=== CONT TestSignErrors/http_error5705=== CONT TestSignErrors/invalid_base645706=== CONT TestSignErrors/no_prefix5707=== CONT TestSignErrors/invalid_json5708--- PASS: TestSign (0.01s)5709--- PASS: TestSignErrors (0.00s)5710 --- PASS: TestSignErrors/http_error (0.00s)5711 --- PASS: TestSignErrors/no_prefix (0.00s)5712 --- PASS: TestSignErrors/invalid_json (6.16s)5713 --- PASS: TestSignErrors/invalid_base64 (6.17s)5714PASS

succeeded niks3-go-unit-tests checks.aarch64-linux.go-unit-tests · build #492 · raw

succeeded `niks3-go-unit-tests` `checks.aarch64-linux.go-unit-tests` · build #492 · raw