niks3-go-unit-tests log

succeeded `niks3-go-unit-tests` `checks.aarch64-darwin.go-unit-tests` · build #489 · raw

1tribuchet: building on mac022Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestUploadBuildLog_FileBodyReplayedOnRetry6=== PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry7=== RUN TestRegisterUploadedObjectReusesConnections8=== PAUSE TestRegisterUploadedObjectReusesConnections9=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica10=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica11=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun12=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun13=== RUN TestCaseHackSuffix14=== PAUSE TestCaseHackSuffix15=== RUN TestFilterOversizedClosures16=== PAUSE TestFilterOversizedClosures17=== RUN TestUploadMultipart_PartsInParallel18=== PAUSE TestUploadMultipart_PartsInParallel19=== RUN TestUploadMultipart_ReturnsAfterDroppedBody20=== PAUSE TestUploadMultipart_ReturnsAfterDroppedBody21=== RUN TestPartSizeForNAR22=== PAUSE TestPartSizeForNAR23=== RUN TestDumpPathCaseHackMatchesNix24--- PASS: TestDumpPathCaseHackMatchesNix (0.05s)25=== RUN TestDumpPathCaseHackCollision26--- PASS: TestDumpPathCaseHackCollision (0.00s)27=== RUN TestDumpPathMatchesNix28=== PAUSE TestDumpPathMatchesNix29=== RUN TestDumpPathSingleFile30=== PAUSE TestDumpPathSingleFile31=== RUN TestDumpPathWriterError32=== PAUSE TestDumpPathWriterError33=== RUN TestEncodeNixBase3234=== PAUSE TestEncodeNixBase3235=== RUN TestEncodeNixBase32WithRealHash36=== PAUSE TestEncodeNixBase32WithRealHash37=== RUN TestConvertHashToNix3238=== PAUSE TestConvertHashToNix3239=== RUN TestGetStorePathHash40=== PAUSE TestGetStorePathHash41=== RUN TestPathInfoHashCompatibility42=== PAUSE TestPathInfoHashCompatibility43=== RUN TestParsePathInfoJSON44=== PAUSE TestParsePathInfoJSON45=== RUN TestParsePathInfoJSONMultiplePaths46=== PAUSE TestParsePathInfoJSONMultiplePaths47=== RUN TestPathInfoCACompatibility48=== PAUSE TestPathInfoCACompatibility49=== RUN TestRateLimiterFeedback50=== PAUSE TestRateLimiterFeedback51=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess52=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess53=== RUN TestRegisterUploadedObject_BoundedAgainstSilentServer54=== PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer55=== RUN TestRegisterUploadedObject_OutlivesPush56=== PAUSE TestRegisterUploadedObject_OutlivesPush57=== RUN TestResolveStorePath58=== PAUSE TestResolveStorePath59=== RUN TestDoWithRetry_BodyReplayedViaGetBody60=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody61=== RUN TestRetryTimeoutOutlastsMaxRetries62=== PAUSE TestRetryTimeoutOutlastsMaxRetries63=== RUN TestDoWithRetry_FinalResponseBodyReadable64=== PAUSE TestDoWithRetry_FinalResponseBodyReadable65=== RUN TestShellSplit66=== PAUSE TestShellSplit67=== RUN TestShellSplitErrors68=== PAUSE TestShellSplitErrors69=== RUN TestStreamPushReportsEveryPath70=== PAUSE TestStreamPushReportsEveryPath71=== RUN TestStreamPushBatchesUnderLoad72=== PAUSE TestStreamPushBatchesUnderLoad73=== RUN TestStreamPushIsolatesFailures74=== PAUSE TestStreamPushIsolatesFailures75=== RUN TestStreamPushGivesUpOnDeadServer76=== PAUSE TestStreamPushGivesUpOnDeadServer77=== RUN TestStreamPushRequestLine78=== PAUSE TestStreamPushRequestLine79=== RUN TestStreamPushReportsSignatures80=== PAUSE TestStreamPushReportsSignatures81=== RUN TestStreamPushReportsSkippedPaths82=== PAUSE TestStreamPushReportsSkippedPaths83=== RUN TestClientSignaturesByStorePath84=== PAUSE TestClientSignaturesByStorePath85=== RUN TestStreamPushStopsWhenCanceled86=== PAUSE TestStreamPushStopsWhenCanceled87=== RUN TestSetClientTLS88=== PAUSE TestSetClientTLS89=== RUN TestSetClientTLSDoesNotMutateDefaultTransport90=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport91=== RUN TestSetClientTLSErrors92=== PAUSE TestSetClientTLSErrors93=== RUN TestStaticToken94=== PAUSE TestStaticToken95=== RUN TestFileTokenReadsAndCaches96=== PAUSE TestFileTokenReadsAndCaches97=== RUN TestFileTokenMissing98=== PAUSE TestFileTokenMissing99=== RUN TestFileTokenEmpty100=== PAUSE TestFileTokenEmpty101=== RUN TestScriptTokenNoExpiryRerunsEveryCall102=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall103=== RUN TestScriptTokenCachesUntilRefresh104=== PAUSE TestScriptTokenCachesUntilRefresh105=== RUN TestScriptTokenEmptyToken106=== PAUSE TestScriptTokenEmptyToken107=== RUN TestScriptTokenBadJSON108=== PAUSE TestScriptTokenBadJSON109=== RUN TestScriptTokenScriptFails110=== PAUSE TestScriptTokenScriptFails111=== RUN TestScriptTokenEmptyCommand112=== PAUSE TestScriptTokenEmptyCommand113=== CONT TestDoServerRequestAttachesToken114=== CONT TestRetryTimeoutOutlastsMaxRetries115=== CONT TestStreamPushGivesUpOnDeadServer116=== CONT TestSetClientTLS117=== CONT TestScriptTokenEmptyCommand118--- PASS: TestScriptTokenEmptyCommand (0.00s)119=== CONT TestShellSplit120=== CONT TestScriptTokenScriptFails121=== CONT TestScriptTokenBadJSON122--- PASS: TestShellSplit (0.00s)123=== CONT TestShellSplitErrors124=== CONT TestScriptTokenEmptyToken125=== CONT TestScriptTokenCachesUntilRefresh126=== CONT TestScriptTokenNoExpiryRerunsEveryCall127=== CONT TestFileTokenEmpty128=== CONT TestFileTokenMissing129=== CONT TestFileTokenReadsAndCaches130=== CONT TestStaticToken131=== CONT TestSetClientTLSErrors132=== CONT TestEncodeNixBase32WithRealHash133=== CONT TestSetClientTLSDoesNotMutateDefaultTransport134=== CONT TestStreamPushReportsEveryPath1352026/11/11 23:26:39 ERROR Upload failed error="connection refused" count=201362026/11/11 23:26:39 ERROR Server seems unavailable, giving up on batch untried=17137=== CONT TestStreamPushIsolatesFailures138=== CONT TestRegisterUploadedObject_OutlivesPush139=== CONT TestStreamPushBatchesUnderLoad140=== CONT TestStreamPushReportsSkippedPaths141=== CONT TestStreamPushStopsWhenCanceled142=== CONT TestClientSignaturesByStorePath143=== CONT TestStreamPushReportsSignatures144=== CONT TestStreamPushRequestLine145--- PASS: TestShellSplitErrors (0.00s)146=== CONT TestDoWithRetry_FinalResponseBodyReadable147=== CONT TestDoWithRetry_BodyReplayedViaGetBody148=== CONT TestResolveStorePath149--- PASS: TestFileTokenEmpty (0.00s)150--- PASS: TestEncodeNixBase32WithRealHash (0.00s)151--- PASS: TestFileTokenMissing (0.00s)152--- PASS: TestFileTokenReadsAndCaches (0.00s)153--- PASS: TestStaticToken (0.00s)154--- PASS: TestScriptTokenScriptFails (0.00s)155=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess1562026/11/11 23:26:39 ERROR Upload failed error="bad path" count=31572026/11/11 23:26:39 WARN Rate limiter enabled after throttle name=server-test rate=5158--- PASS: TestStreamPushIsolatesFailures (0.00s)159=== CONT TestRateLimiterFeedback160=== RUN TestRateLimiterFeedback/429_enables_limiter161=== PAUSE TestRateLimiterFeedback/429_enables_limiter162=== RUN TestRateLimiterFeedback/503_enables_limiter163=== PAUSE TestRateLimiterFeedback/503_enables_limiter164=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter165=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter166=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter167=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter168=== CONT TestPathInfoCACompatibility169=== RUN TestPathInfoCACompatibility/null_ca_field170=== PAUSE TestPathInfoCACompatibility/null_ca_field171=== RUN TestPathInfoCACompatibility/old_string_format_-_text172=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text173=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive174=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive175=== RUN TestPathInfoCACompatibility/new_structured_format_-_text176=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text177=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method178--- PASS: TestStreamPushReportsEveryPath (0.00s)179=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method180=== CONT TestParsePathInfoJSONMultiplePaths181=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths182=== CONT TestParsePathInfoJSON183=== RUN TestParsePathInfoJSON/Nix_format184=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=false185=== PAUSE TestParsePathInfoJSON/Nix_format186=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=false187=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=true188--- PASS: TestStreamPushGivesUpOnDeadServer (0.01s)189=== RUN TestParsePathInfoJSON/Lix_format190=== PAUSE TestParsePathInfoJSON/Lix_format191=== RUN TestParsePathInfoJSON/empty_input192=== PAUSE TestParsePathInfoJSON/empty_input193=== RUN TestParsePathInfoJSON/whitespace_only194=== PAUSE TestParsePathInfoJSON/whitespace_only195=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=true196=== CONT TestPathInfoHashCompatibility197=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)198=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)199=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon200=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon201=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI202=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI203=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512204=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512205=== CONT TestGetStorePathHash206--- PASS: TestRegisterUploadedObject_OutlivesPush (0.00s)207=== RUN TestGetStorePathHash/valid_store_path208=== CONT TestEncodeNixBase32209=== RUN TestEncodeNixBase32/test_string_hash210=== PAUSE TestEncodeNixBase32/test_string_hash211=== CONT TestDumpPathWriterError212=== CONT TestDumpPathSingleFile213=== CONT TestDumpPathMatchesNix2142026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:578862152026/11/11 23:26:39 ERROR Upload failed error=boom count=1216=== RUN TestParsePathInfoJSON/invalid_JSON217=== PAUSE TestParsePathInfoJSON/invalid_JSON218=== CONT TestConvertHashToNix32219=== RUN TestConvertHashToNix32/SRI_format_to_Nix32220=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32221=== RUN TestConvertHashToNix32/already_Nix32_format222=== PAUSE TestConvertHashToNix32/already_Nix32_format223=== RUN TestConvertHashToNix32/invalid_format224=== PAUSE TestConvertHashToNix32/invalid_format225=== CONT TestPartSizeForNAR226=== RUN TestPartSizeForNAR/zero_stays_at_minimum227=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum228=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun229=== RUN TestPartSizeForNAR/small_stays_at_minimum230=== PAUSE TestPartSizeForNAR/small_stays_at_minimum231=== CONT TestUploadMultipart_PartsInParallel2322026/11/11 23:26:39 ERROR Upload failed error=boom count=1233=== CONT TestFilterOversizedClosures234=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum235=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum236=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts237=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts238=== RUN TestPartSizeForNAR/1_TiB239=== PAUSE TestPartSizeForNAR/1_TiB240=== RUN TestPartSizeForNAR/5_TiB_S3_max_object241=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object242=== RUN TestPartSizeForNAR/capped_at_5_GiB243=== PAUSE TestPartSizeForNAR/capped_at_5_GiB244=== RUN TestFilterOversizedClosures/no_limit_keeps_everything245=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything246=== PAUSE TestGetStorePathHash/valid_store_path247=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths248=== RUN TestEncodeNixBase32/empty_input249=== CONT TestRegisterUploadedObject_BoundedAgainstSilentServer250=== RUN TestSetClientTLSErrors/missing_cert_file251--- PASS: TestStreamPushReportsSkippedPaths (0.00s)252=== CONT TestUploadMultipart_ReturnsAfterDroppedBody253=== CONT TestCaseHackSuffix254=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped255--- PASS: TestResolveStorePath (0.00s)256--- PASS: TestStreamPushStopsWhenCanceled (0.00s)257--- PASS: TestClientSignaturesByStorePath (0.00s)258--- PASS: TestDoServerRequestAttachesToken (0.01s)259--- PASS: TestStreamPushReportsSignatures (0.00s)2602026/11/11 23:26:39 WARN Rate limiter enabled after throttle name=server-test rate=52612026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:57893262=== PAUSE TestSetClientTLSErrors/missing_cert_file263=== RUN TestSetClientTLSErrors/missing_key_file264=== PAUSE TestSetClientTLSErrors/missing_key_file265=== RUN TestSetClientTLSErrors/missing_ca_file266=== PAUSE TestSetClientTLSErrors/missing_ca_file267=== RUN TestSetClientTLSErrors/invalid_ca_file268=== PAUSE TestSetClientTLSErrors/invalid_ca_file269=== PAUSE TestEncodeNixBase32/empty_input2702026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:57886271=== CONT TestRegisterUploadedObjectReusesConnections272=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica2732026/11/11 23:26:39 WARN Rate limiter enabled after throttle name=server-test rate=5274--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.01s)2752026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:578952762026/11/11 23:26:39 WARN Rate limiter backed off name=server-test rate=5277=== CONT TestUploadBuildLog_FileBodyReplayedOnRetry278=== RUN TestSetClientTLS/rejects_connection_without_client_cert279=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths280=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths2812026/11/11 23:26:39 INFO Garbage collection started282=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter283=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert284=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA285=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA286=== RUN TestSetClientTLS/preserves_debug_logging_transport287=== PAUSE TestSetClientTLS/preserves_debug_logging_transport288=== RUN TestGetStorePathHash/basename_without_hyphen_should_error289=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error290=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error291=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error292=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error293=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error294=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter295=== CONT TestPathInfoCACompatibility/null_ca_field296=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method297=== CONT TestPathInfoCACompatibility/new_structured_format_-_text298=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped299=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive300=== CONT TestPathInfoCACompatibility/old_string_format_-_text3012026/11/11 23:26:39 WARN Rate limiter enabled after throttle name=s3-test rate=53022026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:579553032026/11/11 23:26:39 WARN Rate limiter backed off name=server-test rate=53042026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:57895305=== CONT TestRateLimiterFeedback/503_enables_limiter306=== CONT TestRateLimiterFeedback/429_enables_limiter307--- PASS: TestDoWithRetry_FinalResponseBodyReadable (0.01s)308--- PASS: TestUploadMultipart_ReturnsAfterDroppedBody (0.01s)309=== RUN TestFilterOversizedClosures/all_closures_skipped310=== PAUSE TestFilterOversizedClosures/all_closures_skipped311=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=false312=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=true313--- PASS: TestStreamPushBatchesUnderLoad (0.00s)314 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=false (0.00s)315 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=true (0.00s)316=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)317=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512318=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI319=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon320--- PASS: TestPathInfoHashCompatibility (0.00s)321 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)322 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)323 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)324 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)325=== CONT TestParsePathInfoJSON/Nix_format326=== CONT TestConvertHashToNix32/SRI_format_to_Nix32327=== CONT TestParsePathInfoJSON/invalid_JSON328=== CONT TestParsePathInfoJSON/empty_input329=== CONT TestParsePathInfoJSON/Lix_format330=== CONT TestConvertHashToNix32/invalid_format331=== CONT TestConvertHashToNix32/already_Nix32_format332=== CONT TestPartSizeForNAR/zero_stays_at_minimum333=== CONT TestPartSizeForNAR/1_TiB334=== CONT TestPartSizeForNAR/capped_at_5_GiB335=== CONT TestPartSizeForNAR/5_TiB_S3_max_object336=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum337=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts338=== CONT TestPartSizeForNAR/small_stays_at_minimum339=== CONT TestSetClientTLSErrors/missing_cert_file340=== CONT TestEncodeNixBase32/test_string_hash341=== CONT TestSetClientTLSErrors/missing_key_file342=== CONT TestSetClientTLSErrors/invalid_ca_file343--- PASS: TestPathInfoCACompatibility (0.00s)344 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)345 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)346 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)347 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)348 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)349=== CONT TestParsePathInfoJSON/whitespace_only350--- PASS: TestParsePathInfoJSON (0.00s)351 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)352 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)353 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)354 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)355 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)356--- PASS: TestConvertHashToNix32 (0.00s)357 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)358 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)359 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)360=== CONT TestSetClientTLSErrors/missing_ca_file361=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths362=== CONT TestEncodeNixBase32/empty_input363=== CONT TestGetStorePathHash/valid_store_path364=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths365=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error366=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error367=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA368=== CONT TestGetStorePathHash/basename_without_hyphen_should_error369=== CONT TestFilterOversizedClosures/no_limit_keeps_everything370=== CONT TestFilterOversizedClosures/all_closures_skipped3712026/11/11 23:26:39 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50372=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped3732026/11/11 23:26:39 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000374--- PASS: TestPartSizeForNAR (0.00s)375 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)376 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)377 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)378 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)379 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)380 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)381 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)382=== CONT TestSetClientTLS/preserves_debug_logging_transport383--- PASS: TestEncodeNixBase32 (0.00s)384 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)385 --- PASS: TestEncodeNixBase32/empty_input (0.00s)386=== CONT TestSetClientTLS/rejects_connection_without_client_cert387--- PASS: TestScriptTokenBadJSON (0.02s)388--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.01s)3892026/11/11 23:26:39 INFO Garbage collection started3902026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:57886391--- PASS: TestParsePathInfoJSONMultiplePaths (0.01s)392 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)393 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)394--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (0.01s)395--- PASS: TestGetStorePathHash (0.01s)396 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)397 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)398 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)399 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)400--- PASS: TestFilterOversizedClosures (0.01s)401 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)402 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)403 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)4042026/11/11 23:26:39 WARN Rate limiter enabled after throttle name=server-test rate=54052026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:579794062026/11/11 23:26:39 WARN Rate limiter enabled after throttle name=server-test rate=54072026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:57981408--- PASS: TestScriptTokenEmptyToken (0.02s)409--- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (0.01s)410--- PASS: TestSetClientTLSErrors (0.01s)411 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)412 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)413 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)414 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)4152026/11/11 23:26:39 WARN Rate limiter backed off name=server-test rate=54162026/11/11 23:26:39 WARN Rate limiter backed off name=server-test rate=5417--- PASS: TestRateLimiterFeedback (0.00s)418 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s)419 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.00s)420 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.01s)421 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.01s)4222026/11/11 23:26:39 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=04232026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:57886424--- PASS: TestStreamPushRequestLine (0.02s)4252026/11/11 23:26:39 WARN Garbage collection finished on another replica. Its statistics are not available here426--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (0.01s)4272026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:578864282026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:578864292026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:578864302026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:578864312026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:578864322026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:578864332026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:578864342026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:578864352026/11/11 23:26:39 http: TLS handshake error from 127.0.0.1:57985: remote error: tls: bad certificate436--- PASS: TestSetClientTLS (0.01s)437 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.01s)438 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.01s)439 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.02s)4402026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:578864412026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:57886442--- PASS: TestRegisterUploadedObjectReusesConnections (0.04s)4432026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:578864442026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:578864452026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:578864462026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:57886447--- PASS: TestScriptTokenCachesUntilRefresh (0.05s)448--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.05s)4492026/11/11 23:26:39 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:57886450--- PASS: TestRetryTimeoutOutlastsMaxRetries (0.05s)451--- PASS: TestDumpPathWriterError (0.05s)452--- PASS: TestDumpPathSingleFile (0.05s)453--- PASS: TestCaseHackSuffix (0.05s)454--- PASS: TestDumpPathMatchesNix (0.08s)4552026/11/11 23:26:39 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:57901/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:57901/api/uploads/complete4562026/11/11 23:26:39 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4572026/11/11 23:26:39 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:57901/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:57901/api/uploads/complete4582026/11/11 23:26:39 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4592026/11/11 23:26:39 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:57901/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:57901/api/uploads/complete4602026/11/11 23:26:39 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"461--- PASS: TestRegisterUploadedObject_BoundedAgainstSilentServer (0.10s)462--- PASS: TestUploadMultipart_PartsInParallel (0.62s)463--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)464PASS465Running server tests...466The files belonging to this database system will be owned by user "_tribuchetbld5".467This user must also own the server process.468469The database cluster will be initialized with locale "C".470The default database encoding has accordingly been set to "SQL_ASCII".471The default text search configuration will be set to "english".472473Data page checksums are enabled.474475creating directory /private/var/tb/5/scratch/f4d39195ead3f86e/build/postgres4233933343/data ... ok476creating subdirectories ... ok477selecting dynamic shared memory implementation ... posix478selecting default "max_connections" ... 100479selecting default "shared_buffers" ... 128MB480selecting default time zone ... UTC481creating configuration files ... ok482running bootstrap script ... ok483performing post-bootstrap initialization ... ok484syncing data to disk ... ok485486initdb: warning: enabling "trust" authentication for local connections487initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.488489Success. You can now start the database server using:490491 pg_ctl -D /private/var/tb/5/scratch/f4d39195ead3f86e/build/postgres4233933343/data -l logfile start492493/private/var/tb/5/scratch/f4d39195ead3f86e/build/postgres4233933343:5432 - no response4942026-11-11 23:26:41.091 UTC [78319] LOG: starting PostgreSQL 18.6 on aarch64-apple-darwin25.6.0, compiled by clang version 21.1.8, 64-bit4952026-11-11 23:26:41.091 UTC [78319] LOG: listening on Unix socket "/private/var/tb/5/scratch/f4d39195ead3f86e/build/postgres4233933343/.s.PGSQL.5432"4962026-11-11 23:26:41.094 UTC [78326] LOG: database system was shut down at 2026-11-11 23:26:41 UTC4972026-11-11 23:26:41.095 UTC [78319] LOG: database system is ready to accept connections498/private/var/tb/5/scratch/f4d39195ead3f86e/build/postgres4233933343:5432 - accepting connections499{"timestamp":"2026-11-11T23:26:41.309481Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"b566fedd-a70c-4b22-b73e-708b0e22ea73","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(6)"}500=== RUN TestAbortSucceeded501=== PAUSE TestAbortSucceeded502=== RUN TestService_AuthMiddleware503=== PAUSE TestService_AuthMiddleware504=== RUN TestService_AuthMiddleware_MTLSProxyHeader505=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader506=== RUN TestService_AuthMiddleware_MTLSBoundSubjects507=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects508=== RUN TestService_ReadAuthMiddleware509=== PAUSE TestService_ReadAuthMiddleware510=== RUN TestService_AuthMiddleware_OIDC511=== PAUSE TestService_AuthMiddleware_OIDC512=== RUN TestService_RequireScope_OIDC513=== PAUSE TestService_RequireScope_OIDC514=== RUN TestService_ReadScope_PublicByDefault515=== PAUSE TestService_ReadScope_PublicByDefault516=== RUN TestCacheConfigHandler517=== PAUSE TestCacheConfigHandler518=== RUN TestCacheStatsHandler519=== PAUSE TestCacheStatsHandler520=== RUN TestClientCADerivations521=== PAUSE TestClientCADerivations522=== RUN TestClientErrorHandling523=== PAUSE TestClientErrorHandling524=== RUN TestClientIntegration525=== PAUSE TestClientIntegration526=== RUN TestClientMultipleUploads527=== PAUSE TestClientMultipleUploads528=== RUN TestClientWithDependencies529=== PAUSE TestClientWithDependencies530=== RUN TestClientSharedPathCommittedMidPush531=== PAUSE TestClientSharedPathCommittedMidPush532=== RUN TestPinProtectsFromGC533=== PAUSE TestPinProtectsFromGC534=== RUN TestClientPushesUseOnePush535=== PAUSE TestClientPushesUseOnePush536=== RUN TestClientFallsBackToClosures537=== PAUSE TestClientFallsBackToClosures538=== RUN TestDeleteClosuresRacingPins539=== PAUSE TestDeleteClosuresRacingPins540=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock541=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock542=== RUN TestResolveDBConnectionString543=== PAUSE TestResolveDBConnectionString544=== RUN TestConnectWaitsForAPeerMigration545=== PAUSE TestConnectWaitsForAPeerMigration546=== RUN TestLeadElectsOneAndHandsOver547=== PAUSE TestLeadElectsOneAndHandsOver548=== RUN TestLeadIncumbentWinsAfterRestart5492026/11/11 23:26:41 INFO lead: acquired remote=192.0.2.1:1234 generation=15502026/11/11 23:26:42 INFO lead: released remote=192.0.2.1:1234 generation=15512026/11/11 23:26:42 INFO lead: acquired remote=192.0.2.1:1234 generation=25522026/11/11 23:26:43 INFO lead: released remote=192.0.2.1:1234 generation=2553--- PASS: TestLeadIncumbentWinsAfterRestart (1.92s)554=== RUN TestLeadUnannouncedStandbyYieldsToIncumbent555=== PAUSE TestLeadUnannouncedStandbyYieldsToIncumbent556=== RUN TestLeadIncumbentAnnouncesAtOnce557=== PAUSE TestLeadIncumbentAnnouncesAtOnce558=== RUN TestLeadGenerationGrowsWithEveryGrant559=== PAUSE TestLeadGenerationGrowsWithEveryGrant560=== RUN TestLeadEndsOnShutdown561=== PAUSE TestLeadEndsOnShutdown562=== RUN TestLeadEndsWhenItsConnectionHangs5632026/11/11 23:26:43 INFO lead: acquired remote=192.0.2.1:1234 generation=15642026-11-11 23:26:44.905 UTC [78362] FATAL: terminating connection due to administrator command5652026/11/11 23:26:45 INFO lead: acquired remote=192.0.2.1:1234 generation=25662026/11/11 23:26:46 WARN lead: lock connection lost error="timeout: context deadline exceeded"5672026/11/11 23:26:46 INFO lead: released remote=192.0.2.1:1234 generation=15682026/11/11 23:26:46 INFO lead: released remote=192.0.2.1:1234 generation=2569--- PASS: TestLeadEndsWhenItsConnectionHangs (3.16s)570=== RUN TestGCAdvisoryLockBlocksConcurrentRun571=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun572=== RUN TestGCBugBareHashReferences573=== PAUSE TestGCBugBareHashReferences574=== RUN TestGCMetrics575=== PAUSE TestGCMetrics576=== RUN TestPushDedupSurvivesConcurrentGC577=== PAUSE TestPushDedupSurvivesConcurrentGC578=== RUN TestDeduplicatedObjectsRecordedAsPending579=== PAUSE TestDeduplicatedObjectsRecordedAsPending580=== RUN TestGCSweepSkipsPendingObjects581=== PAUSE TestGCSweepSkipsPendingObjects582=== RUN TestTombstonedObjectOfferedWithoutWaiting583=== PAUSE TestTombstonedObjectOfferedWithoutWaiting584=== RUN TestGCSweepDeliversEachKeyOnce585=== PAUSE TestGCSweepDeliversEachKeyOnce586=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection587=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection588=== RUN TestForceGCDuringPushOffersSweptObject589=== PAUSE TestForceGCDuringPushOffersSweptObject590=== RUN TestFailedS3DeleteKeepsTombstone591=== PAUSE TestFailedS3DeleteKeepsTombstone592=== RUN TestSweepSkipsRowsLockedByPush593=== PAUSE TestSweepSkipsRowsLockedByPush594=== RUN TestRegisterCompletedObjectNeedsPendingRow595=== PAUSE TestRegisterCompletedObjectNeedsPendingRow596=== RUN TestGetExistingObjectsLocksRows597=== PAUSE TestGetExistingObjectsLocksRows598=== RUN TestSweepRechecksPendingAfterLocking599=== PAUSE TestSweepRechecksPendingAfterLocking600=== RUN TestPushWaitsForSweepHoldingRow601=== PAUSE TestPushWaitsForSweepHoldingRow602=== RUN TestCommitRacingPendingCleanupKeepsObjects603=== PAUSE TestCommitRacingPendingCleanupKeepsObjects604=== RUN TestGCEndsOnShutdown605=== PAUSE TestGCEndsOnShutdown606=== RUN TestGCTaskStore_StartNew607=== PAUSE TestGCTaskStore_StartNew608=== RUN TestGCTaskStore_DeduplicateSameParams609=== PAUSE TestGCTaskStore_DeduplicateSameParams610=== RUN TestGCTaskStore_ConflictDifferentParams611=== PAUSE TestGCTaskStore_ConflictDifferentParams612=== RUN TestGCTaskStore_GetEmpty613=== PAUSE TestGCTaskStore_GetEmpty614=== RUN TestGCTaskStore_GetReturnsLatest615=== PAUSE TestGCTaskStore_GetReturnsLatest616=== RUN TestGCTaskStore_CompletedAllowsNewTask617=== PAUSE TestGCTaskStore_CompletedAllowsNewTask618=== RUN TestGCTaskStore_PhaseUpdates619=== PAUSE TestGCTaskStore_PhaseUpdates620=== RUN TestGCTaskStore_Fail621=== PAUSE TestGCTaskStore_Fail622=== RUN TestGracefulShutdownDrainsInflight623=== PAUSE TestGracefulShutdownDrainsInflight624=== RUN TestService_healthCheckHandler625=== PAUSE TestService_healthCheckHandler626=== RUN TestService_readinessHandler627=== PAUSE TestService_readinessHandler628=== RUN TestGenerateLandingPage629=== PAUSE TestGenerateLandingPage630=== RUN TestEnsureAbortUploadsRule631=== PAUSE TestEnsureAbortUploadsRule632=== RUN TestCacheConfigHandlerMaxNarSize633=== PAUSE TestCacheConfigHandlerMaxNarSize634=== RUN TestCreatePendingClosureRejectsOversizedNAR635=== PAUSE TestCreatePendingClosureRejectsOversizedNAR636=== RUN TestNARDeduplicationMetadataUploadBug637=== PAUSE TestNARDeduplicationMetadataUploadBug638=== RUN TestMetricsInventory639=== PAUSE TestMetricsInventory640=== RUN TestMetricsCommit641=== PAUSE TestMetricsCommit642=== RUN TestService_NativeMTLS643=== PAUSE TestService_NativeMTLS644=== RUN TestServerTLSConfig645=== PAUSE TestServerTLSConfig646=== RUN TestMultipartCleanup647=== PAUSE TestMultipartCleanup648=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded649=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded650=== RUN TestPendingCleanupUsesOneCutoff651=== PAUSE TestPendingCleanupUsesOneCutoff652=== RUN TestObjectStatsTrigger653=== PAUSE TestObjectStatsTrigger654=== RUN TestObjectStatsUpdatedOncePerCommit655=== PAUSE TestObjectStatsUpdatedOncePerCommit656=== RUN TestOrphanedObjectsGC657=== PAUSE TestOrphanedObjectsGC658=== RUN TestOrphanedObjectsGCStressTest659=== PAUSE TestOrphanedObjectsGCStressTest660=== RUN TestCreatePin_ReservedPins661=== PAUSE TestCreatePin_ReservedPins662=== RUN TestPresentReportsOnlyClosureRoots663=== PAUSE TestPresentReportsOnlyClosureRoots664=== RUN TestPresentRefreshesOnlyStaleClosures665=== PAUSE TestPresentRefreshesOnlyStaleClosures666=== RUN TestPresentNotReportedWhileGCDeletesClosure667=== PAUSE TestPresentNotReportedWhileGCDeletesClosure668=== RUN TestParseSingleRange669=== PAUSE TestParseSingleRange670=== RUN TestProxyHeadersOnlyTrustedOnSocket6712026/11/11 23:26:46 INFO Starting HTTP server address=proxy.sock6722026/11/11 23:26:46 INFO Starting HTTP server address=127.0.0.1:580086732026/11/11 23:26:46 WARN mTLS auth: subject not in bound subjects subject="CN=someone"6742026/11/11 23:26:46 INFO Shutdown signal received, draining in-flight requests timeout=10s675--- PASS: TestProxyHeadersOnlyTrustedOnSocket (0.16s)676=== RUN TestIsValidCachePath677=== PAUSE TestIsValidCachePath678=== RUN TestReadProxyNarinfo679=== PAUSE TestReadProxyNarinfo680=== RUN TestReadProxyNarinfoAlreadyDecompressed681=== PAUSE TestReadProxyNarinfoAlreadyDecompressed682=== RUN TestReadProxyContentNegotiation683=== PAUSE TestReadProxyContentNegotiation684=== RUN TestReadProxyNarStreaming685=== PAUSE TestReadProxyNarStreaming686=== RUN TestReadProxy404687=== PAUSE TestReadProxy404688=== RUN TestReadProxyInvalidPath689=== PAUSE TestReadProxyInvalidPath690=== RUN TestReadProxyHead691=== PAUSE TestReadProxyHead692=== RUN TestReadProxyOutlastsServerWriteTimeout693=== PAUSE TestReadProxyOutlastsServerWriteTimeout694=== RUN TestReadProxyConditionalGet695=== PAUSE TestReadProxyConditionalGet696=== RUN TestReadProxyRootRedirectsToIndexHTML697=== PAUSE TestReadProxyRootRedirectsToIndexHTML698=== RUN TestReadProxyDisabled699=== PAUSE TestReadProxyDisabled700=== RUN TestReadRedirectNar701=== PAUSE TestReadRedirectNar702=== RUN TestReadRedirectKeepsNarinfoProxied703=== PAUSE TestReadRedirectKeepsNarinfoProxied704=== RUN TestReadProxyRangeRequest705=== PAUSE TestReadProxyRangeRequest706=== RUN TestReadRedirectUsesPublicS3URL707=== PAUSE TestReadRedirectUsesPublicS3URL708=== RUN TestPush_OverlappingRootsStoreOneRowPerKey709=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey710=== RUN TestPush_CompleteCommitsEveryRoot711=== PAUSE TestPush_CompleteCommitsEveryRoot712=== RUN TestPush_RepeatedRootCommitsOnce713=== PAUSE TestPush_RepeatedRootCommitsOnce714=== RUN TestPush_CompleteWaitsForCleanupOfThePush715=== PAUSE TestPush_CompleteWaitsForCleanupOfThePush716=== RUN TestPush_RootsAreLockedInByteOrder717=== PAUSE TestPush_RootsAreLockedInByteOrder718=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit719=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit720=== RUN TestPush_RejectsBadRequests721=== PAUSE TestPush_RejectsBadRequests722=== RUN TestPush_SignsNarinfosOfItsPendingObjects723=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects724=== RUN TestSharedNarBothUploadsComplete725=== PAUSE TestSharedNarBothUploadsComplete726=== RUN TestCompleteMultipartUpload_ErrorButObjectExists727=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists728=== RUN TestCompletedNarNotReofferedAcrossClosures729=== PAUSE TestCompletedNarNotReofferedAcrossClosures730=== RUN TestPresignedUploadRegisteredBeforeCommit731=== PAUSE TestPresignedUploadRegisteredBeforeCommit732=== RUN TestService_Rustfstest733=== PAUSE TestService_Rustfstest734=== RUN TestParseSize735=== PAUSE TestParseSize736=== RUN TestSkippedUploadsHandler737=== PAUSE TestSkippedUploadsHandler738=== RUN TestSystemdListenerNotActivated739--- PASS: TestSystemdListenerNotActivated (0.00s)740=== RUN TestWatchdogBeatsWhenHealthy741--- PASS: TestWatchdogBeatsWhenHealthy (0.03s)742=== RUN TestWatchdogSkipsWhenUnhealthy7432026/11/11 23:26:46 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7442026/11/11 23:26:46 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7452026/11/11 23:26:46 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7462026/11/11 23:26:46 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7472026/11/11 23:26:46 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7482026/11/11 23:26:46 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7492026/11/11 23:26:46 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7502026/11/11 23:26:46 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7512026/11/11 23:26:46 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7522026/11/11 23:26:46 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"753--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)754=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle755=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle756=== RUN TestProxyWriteTimeout757=== PAUSE TestProxyWriteTimeout758=== RUN TestPendingClosureWriteTimeout759=== PAUSE TestPendingClosureWriteTimeout760=== RUN TestIsValidUploadKey761=== PAUSE TestIsValidUploadKey762=== RUN TestUploadHandlersRejectInvalidKeys763=== PAUSE TestUploadHandlersRejectInvalidKeys764=== RUN TestUploadHandlersRejectOversizedBody765=== PAUSE TestUploadHandlersRejectOversizedBody766=== RUN TestService_cleanupPendingClosuresHandler767=== PAUSE TestService_cleanupPendingClosuresHandler768=== RUN TestService_createPendingClosureHandler769=== PAUSE TestService_createPendingClosureHandler770=== RUN TestService_verifyS3Integrity771=== PAUSE TestService_verifyS3Integrity772=== RUN TestService_verifyS3ReuploadsStaleNarinfo773=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo774=== RUN TestCompleteMultipartUnregistered775=== PAUSE TestCompleteMultipartUnregistered776=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT777=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT778=== CONT TestAbortSucceeded779=== CONT TestPush_CompleteCommitsEveryRoot780=== CONT TestService_verifyS3Integrity781=== CONT TestPresentRefreshesOnlyStaleClosures782=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT783=== CONT TestPush_CompleteWaitsForCleanupOfThePush784=== CONT TestCompleteMultipartUnregistered785=== CONT TestPush_RepeatedRootCommitsOnce786=== CONT TestService_verifyS3ReuploadsStaleNarinfo787=== CONT TestService_createPendingClosureHandler788=== CONT TestService_cleanupPendingClosuresHandler789=== CONT TestUploadHandlersRejectOversizedBody790=== CONT TestUploadHandlersRejectInvalidKeys791=== CONT TestIsValidUploadKey792=== CONT TestPendingClosureWriteTimeout793=== CONT TestProxyWriteTimeout794=== RUN TestIsValidUploadKey/narinfo795=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle796=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info797=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info798=== RUN TestProxyWriteTimeout/narinfo799=== CONT TestSkippedUploadsHandler800--- PASS: TestAbortSucceeded (0.00s)801=== CONT TestParseSize802=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal803--- PASS: TestParseSize (0.00s)804=== CONT TestPush_RootsAreLockedInByteOrder805=== CONT TestService_Rustfstest8062026/11/11 23:26:46 INFO Client skipped oversized paths paths=3 nar_bytes=5000000000807=== CONT TestPresignedUploadRegisteredBeforeCommit808=== CONT TestCompletedNarNotReofferedAcrossClosures809=== CONT TestCompleteMultipartUpload_ErrorButObjectExists810=== CONT TestSharedNarBothUploadsComplete811=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed812=== PAUSE TestIsValidUploadKey/narinfo813=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed814=== RUN TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed815=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed816=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost817=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost818=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed819=== RUN TestPendingClosureWriteTimeout/empty820=== PAUSE TestProxyWriteTimeout/narinfo821=== RUN TestProxyWriteTimeout/1_GiB_nar822=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal823=== CONT TestFailedS3DeleteKeepsTombstone824--- PASS: TestSkippedUploadsHandler (0.01s)825=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed826=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed827=== PAUSE TestPendingClosureWriteTimeout/empty828=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed829=== CONT TestForceGCDuringPushOffersSweptObject830=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows831=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows832=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check833=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check834=== RUN TestPendingClosureWriteTimeout/400_objects835=== PAUSE TestPendingClosureWriteTimeout/400_objects836=== RUN TestPendingClosureWriteTimeout/670k_objects837=== PAUSE TestPendingClosureWriteTimeout/670k_objects838=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline839=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline840=== CONT TestPresentReportsOnlyClosureRoots841=== CONT TestPush_RejectsBadRequests842=== PAUSE TestProxyWriteTimeout/1_GiB_nar843=== RUN TestProxyWriteTimeout/10_GiB_nar844=== PAUSE TestProxyWriteTimeout/10_GiB_nar845=== RUN TestProxyWriteTimeout/unknown_size846=== PAUSE TestProxyWriteTimeout/unknown_size847=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key848=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key849=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key850=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key851=== CONT TestCreatePin_ReservedPins852=== RUN TestIsValidUploadKey/nar_zst853=== PAUSE TestIsValidUploadKey/nar_zst854=== RUN TestIsValidUploadKey/nar_xz855=== PAUSE TestIsValidUploadKey/nar_xz856=== RUN TestIsValidUploadKey/nar_plain857=== PAUSE TestIsValidUploadKey/nar_plain858=== RUN TestIsValidUploadKey/listing859=== PAUSE TestIsValidUploadKey/listing860=== RUN TestIsValidUploadKey/build_log861=== PAUSE TestIsValidUploadKey/build_log862=== RUN TestIsValidUploadKey/build_log_home-manager_file863=== PAUSE TestIsValidUploadKey/build_log_home-manager_file864=== RUN TestIsValidUploadKey/build_log_plus_in_name865=== PAUSE TestIsValidUploadKey/build_log_plus_in_name866=== RUN TestIsValidUploadKey/build_log_question_mark867=== PAUSE TestIsValidUploadKey/build_log_question_mark868=== RUN TestIsValidUploadKey/build_log_equals869=== PAUSE TestIsValidUploadKey/build_log_equals870=== RUN TestIsValidUploadKey/realisation871=== PAUSE TestIsValidUploadKey/realisation872=== RUN TestIsValidUploadKey/realisation_plus_in_output873=== PAUSE TestIsValidUploadKey/realisation_plus_in_output874=== RUN TestIsValidUploadKey/nix-cache-info875=== PAUSE TestIsValidUploadKey/nix-cache-info876=== RUN TestIsValidUploadKey/index.html877=== PAUSE TestIsValidUploadKey/index.html878=== RUN TestIsValidUploadKey/narinfo_key,_nar_type879=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type880=== RUN TestIsValidUploadKey/nar_key,_narinfo_type881=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type882=== RUN TestIsValidUploadKey/listing_key,_narinfo_type883=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type884=== RUN TestIsValidUploadKey/traversal885=== PAUSE TestIsValidUploadKey/traversal886=== RUN TestIsValidUploadKey/traversal_nar887=== PAUSE TestIsValidUploadKey/traversal_nar888=== RUN TestIsValidUploadKey/absolute889=== PAUSE TestIsValidUploadKey/absolute890=== RUN TestIsValidUploadKey/empty_key891=== PAUSE TestIsValidUploadKey/empty_key892=== RUN TestIsValidUploadKey/unknown_type893=== PAUSE TestIsValidUploadKey/unknown_type894=== CONT TestPush_SignsNarinfosOfItsPendingObjects895=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit896=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection897=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure898=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure899=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart900=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart901=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts902=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts903=== CONT TestGCSweepDeliversEachKeyOnce9042026/11/11 23:26:47 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58011/oidc9052026/11/11 23:26:47 INFO Received push request method=POST path=/api/pushes9062026-11-11 23:26:47.945 UTC [78431] ERROR: could not obtain lock on row in relation "closures"9072026-11-11 23:26:47.945 UTC [78431] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9082026-11-11 23:26:47.948 UTC [78431] ERROR: could not obtain lock on row in relation "closures"9092026-11-11 23:26:47.948 UTC [78431] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9102026-11-11 23:26:47.951 UTC [78431] ERROR: could not obtain lock on row in relation "closures"9112026-11-11 23:26:47.951 UTC [78431] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9122026-11-11 23:26:47.953 UTC [78431] ERROR: could not obtain lock on row in relation "closures"9132026-11-11 23:26:47.953 UTC [78431] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9142026-11-11 23:26:47.959 UTC [78431] ERROR: could not obtain lock on row in relation "closures"9152026-11-11 23:26:47.959 UTC [78431] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT916--- PASS: TestPush_RootsAreLockedInByteOrder (1.06s)917=== CONT TestTombstonedObjectOfferedWithoutWaiting9182026/11/11 23:26:48 INFO Received push request method=POST path=/api/pushes9192026/11/11 23:26:48 INFO Received complete push request method=POST path=/api/pushes/1/complete9202026/11/11 23:26:48 INFO Completed push id=1 objects=6 duration=33.199209ms921--- PASS: TestPush_CompleteCommitsEveryRoot (1.35s)922=== CONT TestGCSweepSkipsPendingObjects923--- PASS: TestPresentRefreshesOnlyStaleClosures (1.51s)924=== CONT TestDeduplicatedObjectsRecordedAsPending9252026/11/11 23:26:48 INFO Received uploads request method=POST path=/api/pending_closures926--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (1.87s)927=== CONT TestPushDedupSurvivesConcurrentGC9282026/11/11 23:26:48 INFO Received uploads request method=POST path=/api/pending_closures9292026/11/11 23:26:49 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9302026/11/11 23:26:49 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=Mjg5YWE3MzYtZDM4NC00ZDJhLWIwODQtZTAxNjc2OWMzNGNlLmZiYmEzZmJiLTZhMzctNGViYi04Mzg5LTE3Y2YwNzVjNDU5NXgxNzk0NDM5NjA4OTUzNzY0MDAw931--- PASS: TestService_Rustfstest (2.22s)932=== CONT TestGCMetrics9332026/11/11 23:26:49 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=Mjg5YWE3MzYtZDM4NC00ZDJhLWIwODQtZTAxNjc2OWMzNGNlLmZiYmEzZmJiLTZhMzctNGViYi04Mzg5LTE3Y2YwNzVjNDU5NXgxNzk0NDM5NjA4OTUzNzY0MDAw parts=1934--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (2.23s)935=== CONT TestGCBugBareHashReferences9362026/11/11 23:26:49 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9372026/11/11 23:26:49 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst938--- PASS: TestCompleteMultipartUnregistered (2.38s)939=== CONT TestGCAdvisoryLockBlocksConcurrentRun9402026/11/11 23:26:49 INFO Received uploads request method=POST path=/api/pending_closures9412026/11/11 23:26:49 INFO Received push request method=POST path=/api/pushes9422026/11/11 23:26:49 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9432026/11/11 23:26:49 INFO Received uploads request method=POST path=/api/pending_closures9442026-11-11 23:26:50.060 UTC [78463] ERROR: Push does not exist: id=19452026-11-11 23:26:50.060 UTC [78463] CONTEXT: PL/pgSQL function commit_push(bigint) line 11 at RAISE9462026-11-11 23:26:50.060 UTC [78463] STATEMENT: SELECT commit_push($1)947--- PASS: TestPush_CompleteWaitsForCleanupOfThePush (3.16s)948=== CONT TestLeadEndsOnShutdown9492026/11/11 23:26:50 INFO Received uploads request method=POST path=/api/pending_closures9502026/11/11 23:26:50 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst9512026/11/11 23:26:50 INFO Received uploads request method=POST path=/api/pending_closures952--- PASS: TestPresignedUploadRegisteredBeforeCommit (3.30s)953=== CONT TestOrphanedObjectsGCStressTest9542026/11/11 23:26:50 INFO Received push request method=POST path=/api/pushes9552026/11/11 23:26:50 INFO Received complete push request method=POST path=/api/pushes/1/complete9562026/11/11 23:26:50 INFO Completed push id=1 objects=4 duration=31.412667ms957--- PASS: TestPush_RepeatedRootCommitsOnce (3.59s)958=== CONT TestLeadGenerationGrowsWithEveryGrant9592026/11/11 23:26:50 INFO Received uploads request method=POST path=/api/pending_closures9602026/11/11 23:26:50 INFO Aborted multipart uploads count=0 kept=09612026/11/11 23:26:50 WARN Force mode enabled - objects will be deleted immediately without grace period9622026/11/11 23:26:50 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."9632026/11/11 23:26:50 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."964--- PASS: TestFailedS3DeleteKeepsTombstone (3.98s)965=== CONT TestService_healthCheckHandler9662026/11/11 23:26:50 INFO Received push request method=POST path=/api/pushes9672026/11/11 23:26:50 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9682026/11/11 23:26:51 INFO Signed narinfos id=1 count=19692026/11/11 23:26:51 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9702026/11/11 23:26:51 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9712026/11/11 23:26:51 ERROR Failed to sign narinfos signer="index 1" error="signing key cannot be nil"9722026/11/11 23:26:51 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9732026/11/11 23:26:51 ERROR Failed to sign narinfos signer=niks3-test-1 error="signer failed"9742026/11/11 23:26:51 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9752026/11/11 23:26:51 ERROR Signer returned unexpected signature count signer=niks3-test-1 got=0 want=19762026/11/11 23:26:51 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9772026/11/11 23:26:51 ERROR Signer returned unexpected object key signer=niks3-test-1 object_key=unexpected.narinfo978--- PASS: TestPush_SignsNarinfosOfItsPendingObjects (4.06s)979=== CONT TestLeadIncumbentAnnouncesAtOnce9802026/11/11 23:26:51 INFO Received uploads request method=POST path=/api/pending_closures9812026/11/11 23:26:51 INFO Received uploads request method=POST path=/api/pending_closures9822026/11/11 23:26:51 INFO Received uploads request method=POST path=/api/pending_closures9832026/11/11 23:26:51 INFO Received uploads request method=POST path=/api/pending_closures9842026/11/11 23:26:51 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9852026/11/11 23:26:51 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=Mjg5YWE3MzYtZDM4NC00ZDJhLWIwODQtZTAxNjc2OWMzNGNlLjRiY2QzYjM5LWM5YmUtNDE1Ny1iZDI0LTBiMGUyN2EzN2JhZXgxNzk0NDM5NjA5OTE3NDMwMDAw parts=129862026/11/11 23:26:51 INFO Received uploads request method=POST path=/api/pending_closures987--- PASS: TestCompletedNarNotReofferedAcrossClosures (4.58s)988=== CONT TestOrphanedObjectsGC9892026/11/11 23:26:51 INFO Received uploads request method=POST path=/api/pending_closures990--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (4.87s)991=== CONT TestObjectStatsUpdatedOncePerCommit9922026/11/11 23:26:51 INFO Received cleanup request method=DELETE path=/api/pending_closures9932026/11/11 23:26:51 INFO Aborted multipart uploads count=0 kept=09942026/11/11 23:26:51 INFO Received uploads request method=POST path=/api/pending_closures9952026/11/11 23:26:51 INFO Received cleanup request method=DELETE path=/api/pending_closures9962026/11/11 23:26:51 INFO Aborted multipart uploads count=1 kept=09972026/11/11 23:26:51 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete9982026-11-11 23:26:51.996 UTC [78440] ERROR: Closure does not exist: id=19992026-11-11 23:26:51.996 UTC [78440] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 13 at RAISE10002026-11-11 23:26:51.996 UTC [78440] STATEMENT: -- name: CommitPendingClosure :one1001 SELECT commit_pending_closure($1::bigint)::bigint1002 1003--- PASS: TestService_cleanupPendingClosuresHandler (5.09s)1004=== CONT TestObjectStatsTrigger10052026/11/11 23:26:52 INFO Aborted multipart uploads count=0 kept=010062026/11/11 23:26:52 WARN Force mode enabled - objects will be deleted immediately without grace period10072026/11/11 23:26:52 INFO Received push request method=POST path=/api/pushes10082026/11/11 23:26:52 INFO Received complete push request method=POST path=/api/pushes/1/complete10092026/11/11 23:26:52 INFO Completed push id=1 objects=4 duration=32.654625ms10102026/11/11 23:26:52 INFO Received push request method=POST path=/api/pushes10112026/11/11 23:26:52 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10122026/11/11 23:26:52 INFO Aborted multipart uploads count=0 kept=010132026/11/11 23:26:52 WARN Force mode enabled - objects will be deleted immediately without grace period10142026/11/11 23:26:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=010152026/11/11 23:26:52 INFO Vacuumed table table=pending_closures10162026/11/11 23:26:52 INFO Vacuumed table table=pending_objects10172026/11/11 23:26:52 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=Mjg5YWE3MzYtZDM4NC00ZDJhLWIwODQtZTAxNjc2OWMzNGNlLjhiMmMzYmE2LWY3NTMtNDkwOS1iMzdlLWRlOGM4YTZlMDE5ZXgxNzk0NDM5NjExMTcxNDAyMDAw parts=1010182026/11/11 23:26:52 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10192026/11/11 23:26:52 INFO Received uploads request method=POST path=/api/pending_closures10202026/11/11 23:26:52 INFO Completed upload id=1 objects=2 duration=19.516042ms10212026/11/11 23:26:52 INFO Vacuumed table table=multipart_uploads10222026/11/11 23:26:52 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000010232026/11/11 23:26:52 INFO Received uploads request method=POST path=/api/pending_closures10242026/11/11 23:26:52 INFO Starting cleanup of old closures method=DELETE path=/api/closures10252026/11/11 23:26:52 INFO Vacuumed table table=closures10262026/11/11 23:26:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=010272026/11/11 23:26:52 INFO Vacuumed table table=pending_closures10282026/11/11 23:26:52 INFO Aborted multipart uploads count=0 kept=010292026/11/11 23:26:52 INFO Vacuumed table table=pending_objects10302026/11/11 23:26:52 INFO Vacuumed table table=multipart_uploads10312026/11/11 23:26:52 INFO Vacuumed table table=closures10322026/11/11 23:26:52 INFO Vacuumed table table=objects10332026/11/11 23:26:52 INFO Received uploads request method=POST path=/api/pending_closures10342026/11/11 23:26:52 INFO Received complete push request method=POST path=/api/pushes/2/complete10352026/11/11 23:26:52 INFO Completed push id=2 objects=4 duration=12.979ms10362026/11/11 23:26:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=01037--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (5.95s)1038=== CONT TestPendingCleanupUsesOneCutoff10392026/11/11 23:26:52 INFO Vacuumed table table=pending_closures10402026/11/11 23:26:52 INFO Vacuumed table table=objects1041--- PASS: TestGCSweepDeliversEachKeyOnce (5.99s)1042=== CONT TestGracefulShutdownDrainsInflight10432026/11/11 23:26:52 INFO Starting HTTP server address=127.0.0.1:5805210442026/11/11 23:26:52 INFO Shutdown signal received, draining in-flight requests timeout=10s10452026/11/11 23:26:53 INFO Vacuumed table table=pending_objects10462026/11/11 23:26:53 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1047--- PASS: TestGracefulShutdownDrainsInflight (0.07s)1048=== CONT TestLeadUnannouncedStandbyYieldsToIncumbent10492026/11/11 23:26:53 INFO Vacuumed table table=multipart_uploads10502026/11/11 23:26:53 INFO Vacuumed table table=closures10512026/11/11 23:26:53 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=Mjg5YWE3MzYtZDM4NC00ZDJhLWIwODQtZTAxNjc2OWMzNGNlLjAxY2E4ZTVmLTc4MzAtNGYyZC1iZTM4LWFjYzZhMTEzMDEzN3gxNzk0NDM5NjExNDg3NDc3MDAw parts=1010522026/11/11 23:26:53 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10532026/11/11 23:26:53 INFO Vacuumed table table=objects10542026/11/11 23:26:53 INFO Completed upload id=1 objects=2 duration=10.481708ms10552026/11/11 23:26:53 INFO Received uploads request method=POST path=/api/pending_closures10562026/11/11 23:26:53 INFO Received uploads request method=POST path=/api/pending_closures10572026/11/11 23:26:53 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001058=== CONT TestGCTaskStore_Fail1059--- PASS: TestService_createPendingClosureHandler (6.24s)1060=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded1061--- PASS: TestGCTaskStore_Fail (0.00s)10622026/11/11 23:26:53 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo10632026/11/11 23:26:53 WARN Found objects in DB but missing from S3, will re-upload count=11064--- PASS: TestService_verifyS3Integrity (6.24s)1065=== CONT TestMultipartCleanup1066--- PASS: TestPresentReportsOnlyClosureRoots (6.23s)1067=== CONT TestServerTLSConfig1068=== RUN TestServerTLSConfig/no_client_CA1069=== PAUSE TestServerTLSConfig/no_client_CA1070=== RUN TestServerTLSConfig/missing_CA_file1071=== PAUSE TestServerTLSConfig/missing_CA_file1072=== RUN TestServerTLSConfig/not_a_PEM_file1073=== PAUSE TestServerTLSConfig/not_a_PEM_file1074=== CONT TestGCTaskStore_PhaseUpdates1075--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)1076=== CONT TestService_NativeMTLS1077=== RUN TestPush_RejectsBadRequests/no_roots1078=== PAUSE TestPush_RejectsBadRequests/no_roots1079=== RUN TestPush_RejectsBadRequests/no_objects1080=== PAUSE TestPush_RejectsBadRequests/no_objects1081=== RUN TestPush_RejectsBadRequests/bad_root1082=== PAUSE TestPush_RejectsBadRequests/bad_root1083=== RUN TestPush_RejectsBadRequests/root_not_in_objects1084=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects1085=== CONT TestClientErrorHandling1086=== RUN TestClientErrorHandling/InvalidStorePath1087=== PAUSE TestClientErrorHandling/InvalidStorePath1088=== RUN TestClientErrorHandling/InvalidAuthToken1089=== PAUSE TestClientErrorHandling/InvalidAuthToken1090=== RUN TestClientErrorHandling/ServerNotAvailable1091=== PAUSE TestClientErrorHandling/ServerNotAvailable1092=== CONT TestGCTaskStore_CompletedAllowsNewTask1093--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)1094=== CONT TestClientCADerivations10952026/11/11 23:26:53 WARN Rate limiter enabled after throttle name=s3-test rate=510962026/11/11 23:26:53 WARN S3 rate limit hit operation="complete multipart upload" error="Please reduce your request rate."1097=== NAME TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle1098 throttle_test.go:213: Proxy stats: total=13, throttled=10, completeMultipart=101099 throttle_test.go:215: Rate limiter: enabled=true, rate=5.001100--- PASS: TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle (6.61s)1101=== CONT TestClientIntegration11022026/11/11 23:26:53 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux11032026/11/11 23:26:53 WARN Refused reserved pin name=worker-x86_64-linux11042026/11/11 23:26:53 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux11052026/11/11 23:26:53 INFO Received create pin request method=POST path=/api/pins/my-app11062026/11/11 23:26:53 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux1107=== CONT TestMetricsCommit1108--- PASS: TestCreatePin_ReservedPins (6.66s)11092026/11/11 23:26:53 INFO Received uploads request method=POST path=/api/pending_closures1110--- PASS: TestDeduplicatedObjectsRecordedAsPending (5.55s)1111=== CONT TestMetricsInventory11122026/11/11 23:26:54 INFO Received uploads request method=POST path=/api/pending_closures11132026/11/11 23:26:54 INFO Aborted multipart uploads count=0 kept=011142026/11/11 23:26:54 WARN Force mode enabled - objects will be deleted immediately without grace period11152026/11/11 23:26:54 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011162026/11/11 23:26:54 INFO Vacuumed table table=pending_closures11172026/11/11 23:26:54 INFO Vacuumed table table=pending_objects11182026/11/11 23:26:54 INFO Vacuumed table table=multipart_uploads11192026/11/11 23:26:54 INFO Vacuumed table table=closures11202026/11/11 23:26:54 INFO Vacuumed table table=objects1121--- PASS: TestGCSweepSkipsPendingObjects (6.29s)1122=== CONT TestLeadElectsOneAndHandsOver11232026/11/11 23:26:54 INFO Received uploads request method=POST path=/api/pending_closures11242026/11/11 23:26:54 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1125--- PASS: TestTombstonedObjectOfferedWithoutWaiting (6.66s)1126=== CONT TestCacheStatsHandler11272026/11/11 23:26:54 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=Mjg5YWE3MzYtZDM4NC00ZDJhLWIwODQtZTAxNjc2OWMzNGNlLjc2ZTQwY2I4LTZjYmMtNDI2Mi1iNmI5LTE2YzY0Mzk1NTgwZXgxNzk0NDM5NjEyODU0MDUwMDAw parts=1211282026/11/11 23:26:55 INFO Received uploads request method=POST path=/api/pending_closures11292026/11/11 23:26:55 INFO Aborted multipart uploads count=0 kept=011302026/11/11 23:26:55 INFO Aborted multipart uploads count=0 kept=011312026/11/11 23:26:55 WARN Force mode enabled - objects will be deleted immediately without grace period11322026/11/11 23:26:55 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011332026/11/11 23:26:55 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011342026/11/11 23:26:55 INFO Vacuumed table table=pending_closures11352026/11/11 23:26:55 INFO Vacuumed table table=pending_objects11362026/11/11 23:26:55 INFO Vacuumed table table=multipart_uploads11372026/11/11 23:26:55 INFO Vacuumed table table=closures11382026/11/11 23:26:55 INFO Vacuumed table table=objects1139--- PASS: TestGCMetrics (6.31s)1140=== CONT TestGCTaskStore_GetReturnsLatest1141--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)1142=== CONT TestCacheConfigHandler1143=== RUN TestCacheConfigHandler/full_config,_no_issuer1144=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1145=== RUN TestCacheConfigHandler/no_cache_url_configured1146=== PAUSE TestCacheConfigHandler/no_cache_url_configured1147=== RUN TestCacheConfigHandler/no_signing_keys1148=== PAUSE TestCacheConfigHandler/no_signing_keys1149=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1150=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1151=== CONT TestGCTaskStore_GetEmpty1152--- PASS: TestGCTaskStore_GetEmpty (0.00s)1153=== CONT TestService_ReadScope_PublicByDefault11542026/11/11 23:26:55 INFO Vacuumed table table=pending_closures11552026/11/11 23:26:55 INFO Vacuumed table table=pending_objects11562026/11/11 23:26:55 INFO Vacuumed table table=multipart_uploads11572026/11/11 23:26:55 INFO Vacuumed table table=closures11582026/11/11 23:26:55 INFO Vacuumed table table=objects11592026/11/11 23:26:55 INFO Aborted multipart uploads count=0 kept=011602026/11/11 23:26:55 WARN Force mode enabled - objects will be deleted immediately without grace period11612026/11/11 23:26:55 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011622026/11/11 23:26:55 INFO Vacuumed table table=pending_closures11632026/11/11 23:26:55 INFO Vacuumed table table=pending_objects11642026/11/11 23:26:55 INFO Vacuumed table table=multipart_uploads11652026/11/11 23:26:55 INFO Vacuumed table table=closures11662026/11/11 23:26:55 INFO Vacuumed table table=objects1167--- PASS: TestPushDedupSurvivesConcurrentGC (6.88s)1168=== CONT TestGCTaskStore_ConflictDifferentParams1169--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)1170=== CONT TestService_RequireScope_OIDC11712026/11/11 23:26:55 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58066/oidc1172--- PASS: TestGCBugBareHashReferences (6.73s)1173=== CONT TestNARDeduplicationMetadataUploadBug11742026/11/11 23:26:55 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=_tribuchetbld5 database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"1175--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (6.64s)1176=== CONT TestGCTaskStore_DeduplicateSameParams1177--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)1178=== CONT TestConnectWaitsForAPeerMigration11792026/11/11 23:26:56 INFO lead: acquired remote=192.0.2.1:1234 generation=111802026/11/11 23:26:56 INFO Received complete multipart upload request method=POST path=/api/multipart/complete11812026/11/11 23:26:56 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=Mjg5YWE3MzYtZDM4NC00ZDJhLWIwODQtZTAxNjc2OWMzNGNlLjUwMWZmMTZlLTVjODktNDRiNS1iNjFiLWFjNzc3ODhlNjYxN3gxNzk0NDM5NjEyODk5OTQ1MDAw parts=121182--- PASS: TestSharedNarBothUploadsComplete (9.63s)1183=== CONT TestService_AuthMiddleware_OIDC11842026/11/11 23:26:56 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58071/oidc11852026/11/11 23:26:56 INFO lead: acquired remote=192.0.2.1:1234 generation=11186--- PASS: TestService_healthCheckHandler (6.23s)1187=== CONT TestCreatePendingClosureRejectsOversizedNAR11882026/11/11 23:26:57 INFO Received uploads request method=POST path=/api/pending_closures1189--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.01s)1190=== CONT TestResolveDBConnectionString1191=== RUN TestResolveDBConnectionString/flag_wins1192=== PAUSE TestResolveDBConnectionString/flag_wins1193=== RUN TestResolveDBConnectionString/file_when_flag_empty1194=== PAUSE TestResolveDBConnectionString/file_when_flag_empty1195=== RUN TestResolveDBConnectionString/missing_file_is_an_error1196=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error1197=== RUN TestResolveDBConnectionString/PGHOST_allows_empty1198=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty1199=== RUN TestResolveDBConnectionString/nothing_configured1200=== PAUSE TestResolveDBConnectionString/nothing_configured1201=== CONT TestService_ReadAuthMiddleware12022026/11/11 23:26:57 INFO lead: released remote=192.0.2.1:1234 generation=11203--- PASS: TestLeadEndsOnShutdown (7.20s)1204=== CONT TestCacheConfigHandlerMaxNarSize1205--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)1206=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock12072026/11/11 23:26:57 INFO lead: acquired remote=192.0.2.1:1234 generation=112082026/11/11 23:26:57 INFO lead: released remote=192.0.2.1:1234 generation=11209--- PASS: TestLeadIncumbentAnnouncesAtOnce (6.45s)1210=== CONT TestEnsureAbortUploadsRule12112026/11/11 23:26:57 INFO lead: released remote=192.0.2.1:1234 generation=112122026/11/11 23:26:57 INFO lead: acquired remote=192.0.2.1:1234 generation=21213--- PASS: TestObjectStatsUpdatedOncePerCommit (6.93s)1214=== CONT TestDeleteClosuresRacingPins1215--- PASS: TestObjectStatsTrigger (6.92s)1216=== CONT TestGenerateLandingPage1217--- PASS: TestGenerateLandingPage (0.00s)1218=== CONT TestGCTaskStore_StartNew1219--- PASS: TestGCTaskStore_StartNew (0.00s)1220=== CONT TestService_readinessHandler12212026/11/11 23:26:59 INFO lead: released remote=192.0.2.1:1234 generation=212222026/11/11 23:26:59 INFO lead: acquired remote=192.0.2.1:1234 generation=312232026/11/11 23:26:59 INFO lead: released remote=192.0.2.1:1234 generation=31224--- PASS: TestLeadGenerationGrowsWithEveryGrant (8.53s)1225=== CONT TestService_AuthMiddleware_MTLSBoundSubjects12262026/11/11 23:26:59 INFO Received uploads request method=POST path=/api/pending_closures12272026/11/11 23:26:59 INFO Received uploads request method=POST path=/api/pending_closures12282026/11/11 23:26:59 INFO Received cleanup request method=DELETE path=/api/pending_closures12292026/11/11 23:26:59 INFO lead: acquired remote=192.0.2.1:1234 generation=112302026/11/11 23:27:00 INFO Received uploads request method=POST path=/api/pending_closures1231=== NAME TestOrphanedObjectsGC1232 orphaned_objects_gc_test.go:287: GC Test Summary:1233 orphaned_objects_gc_test.go:288: - Kept: 2 objects from closure A1234 orphaned_objects_gc_test.go:289: - Deleted: 2 objects from closure B1235 orphaned_objects_gc_test.go:290: - Deleted: 6 orphaned chain objects (X1->X2->X3)1236 orphaned_objects_gc_test.go:291: - Deleted: 2 orphaned single objects (Y)1237 orphaned_objects_gc_test.go:292: - Total deleted: 10 objects1238--- PASS: TestOrphanedObjectsGC (8.90s)1239=== CONT TestGCEndsOnShutdown1240=== RUN TestGCEndsOnShutdown/before_the_run1241=== PAUSE TestGCEndsOnShutdown/before_the_run1242=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1243=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1244=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1245=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1246=== CONT TestService_AuthMiddleware_MTLSProxyHeader1247--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (7.30s)1248=== CONT TestCommitRacingPendingCleanupKeepsObjects12492026/11/11 23:27:00 WARN mTLS auth: subject not in bound subjects subject="CN=reader"12502026/11/11 23:27:00 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1251--- PASS: TestService_NativeMTLS (7.46s)1252=== CONT TestService_AuthMiddleware12532026/11/11 23:27:00 INFO lead: released remote=192.0.2.1:1234 generation=112542026/11/11 23:27:00 INFO lead: acquired remote=192.0.2.1:1234 generation=212552026/11/11 23:27:00 INFO Received uploads request method=POST path=/api/pending_closures12562026/11/11 23:27:00 INFO lead: given back to the incumbent remote=192.0.2.1:1234 generation=212572026/11/11 23:27:00 INFO lead: acquired remote=192.0.2.1:1234 generation=312582026/11/11 23:27:01 INFO Received cleanup request method=DELETE path=/api/pending_closures12592026/11/11 23:27:01 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket47/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""12602026/11/11 23:27:01 INFO Aborted multipart uploads count=0 kept=112612026/11/11 23:27:01 INFO Received cleanup request method=DELETE path=/api/pending_closures12622026/11/11 23:27:01 INFO Aborted multipart uploads count=1 kept=01263--- PASS: TestMultipartCleanup (8.04s)1264=== CONT TestClientFallsBackToClosures1265=== NAME TestClientIntegration1266 client_integration_test.go:292: Created store path: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientIntegration2375891209/002/store/6r6l4y5sagi5yvk71y16hxwfh7pbyk48-test-file.txt1267=== NAME TestClientCADerivations1268 client_ca_test.go:136: Built CA derivation: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientCADerivations3586130383/001/store/sg9ppgc8j89k7j80bb5h7gdc4bal2cn5-ca-test12692026/11/11 23:27:01 INFO Received push request method=POST path=/api/pushes1270 client_ca_test.go:139: Found 1 dependencies (including self)12712026/11/11 23:27:01 INFO Received push request method=POST path=/api/pushes12722026/11/11 23:27:01 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)12732026/11/11 23:27:01 INFO Uploading 6r6l4y5sagi5yvk71y16hxwfh7pbyk48-test-file.txt (152B)12742026/11/11 23:27:01 INFO Received complete push request method=POST path=/api/pushes/1/complete12752026/11/11 23:27:01 INFO Completed push id=1 objects=4 duration=42.046916ms12762026/11/11 23:27:01 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete12772026/11/11 23:27:01 INFO Completed upload id=2 objects=2 duration=440.75µs1278--- PASS: TestMetricsCommit (8.33s)12792026/11/11 23:27:01 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12802026/11/11 23:27:01 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"12812026/11/11 23:27:01 WARN Failed to register uploaded object key=6r6l4y5sagi5yvk71y16hxwfh7pbyk48.ls error="server returned 404: 404 page not found\n"1282=== CONT TestPushWaitsForSweepHoldingRow12832026/11/11 23:27:01 INFO Signed narinfos id=1 count=112842026/11/11 23:27:01 INFO Uploading 1 narinfos12852026/11/11 23:27:01 INFO Received push request method=POST path=/api/pushes12862026/11/11 23:27:02 INFO Received complete push request method=POST path=/api/pushes/1/complete12872026/11/11 23:27:02 WARN Failed to register uploaded object key=6r6l4y5sagi5yvk71y16hxwfh7pbyk48.narinfo error="server returned 404: 404 page not found\n"12882026/11/11 23:27:02 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)12892026/11/11 23:27:02 INFO Uploading sg9ppgc8j89k7j80bb5h7gdc4bal2cn5-ca-test (144B)12902026/11/11 23:27:02 INFO Completed push id=1 objects=3 duration=31.890875ms12912026/11/11 23:27:02 INFO Upload complete. (270ms)12922026/11/11 23:27:02 INFO All 1 paths already cached1293=== NAME TestClientIntegration1294 client_integration_test.go:318: Retrieved narinfo from S3:12952026/11/11 23:27:02 WARN Failed to register uploaded object key=sg9ppgc8j89k7j80bb5h7gdc4bal2cn5.ls error="server returned 404: 404 page not found\n"1296 StorePath: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientIntegration2375891209/002/store/6r6l4y5sagi5yvk71y16hxwfh7pbyk48-test-file.txt1297 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1298 Compression: zstd1299 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11300 NarSize: 1521301 References: 1302 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11303 client_integration_test.go:319: Retrieved .ls file from S3 (compressed size: 77 bytes)1304 client_integration_test.go:319: Decompressed .ls content (64 bytes):13052026/11/11 23:27:02 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"1306 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}13072026/11/11 23:27:02 WARN Failed to register uploaded object key=log/kibf0nzvszysdka7hypbwi2wps6gj650-ca-test.drv error="server returned 404: 404 page not found\n"13082026/11/11 23:27:02 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13092026/11/11 23:27:02 INFO Signed narinfos id=1 count=113102026/11/11 23:27:02 INFO Uploading 1 narinfos13112026/11/11 23:27:02 INFO Received complete push request method=POST path=/api/pushes/1/complete13122026/11/11 23:27:02 WARN Failed to register uploaded object key=sg9ppgc8j89k7j80bb5h7gdc4bal2cn5.narinfo error="server returned 404: 404 page not found\n"13132026/11/11 23:27:02 INFO Completed push id=1 objects=4 duration=34.811ms13142026/11/11 23:27:02 INFO Upload complete. (355ms)1315--- PASS: TestMetricsInventory (8.23s)1316=== CONT TestClientPushesUseOnePush1317=== NAME TestClientCADerivations1318 client_ca_test.go:180: Narinfo contains CA field: StorePath: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientCADerivations3586130383/001/store/sg9ppgc8j89k7j80bb5h7gdc4bal2cn5-ca-test1319 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1320 Compression: zstd1321 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1322 NarSize: 1441323 References: 1324 Deriver: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientCADerivations3586130383/001/store/kibf0nzvszysdka7hypbwi2wps6gj650-ca-test.drv1325 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1326 client_ca_test.go:185: Checking for realisation files in S3...1327 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1328 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache1329 client_ca_test.go:258: nix copy output: error: binary cache 's3://bucket48?endpoint=http://localhost:57993&region=eu-west-1' is for Nix stores with prefix '/nix/store', not '/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientCADerivations3586130383/001/store'1330 client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 113312026/11/11 23:27:02 INFO Received push request method=POST path=/api/pushes13322026/11/11 23:27:02 INFO Object in database but missing from S3 key=6r6l4y5sagi5yvk71y16hxwfh7pbyk48.ls13332026/11/11 23:27:02 WARN Found objects in DB but missing from S3, will re-upload count=11334--- PASS: TestClientCADerivations (9.03s)1335=== CONT TestReadRedirectNar13362026/11/11 23:27:02 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)13372026/11/11 23:27:02 INFO lead: acquired remote=192.0.2.1:1234 generation=113382026/11/11 23:27:02 INFO Received complete push request method=POST path=/api/pushes/2/complete13392026/11/11 23:27:02 WARN Failed to register uploaded object key=6r6l4y5sagi5yvk71y16hxwfh7pbyk48.ls error="server returned 404: 404 page not found\n"13402026/11/11 23:27:02 INFO Completed push id=2 objects=3 duration=744.833µs13412026/11/11 23:27:02 INFO Upload complete. (112ms)1342=== NAME TestClientIntegration1343 client_integration_test.go:336: Retrieved .ls file from S3 (compressed size: 62 bytes)1344 client_integration_test.go:336: Decompressed .ls content (49 bytes):1345 {"version":1,"root":{"type":"regular","size":39}}1346--- PASS: TestCacheStatsHandler (8.15s)1347=== CONT TestReadProxyDisabled13482026/11/11 23:27:02 INFO Received push request method=POST path=/api/pushes13492026/11/11 23:27:02 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)13502026/11/11 23:27:02 INFO lead: released remote=192.0.2.1:1234 generation=31351--- PASS: TestLeadUnannouncedStandbyYieldsToIncumbent (9.78s)1352=== CONT TestSweepRechecksPendingAfterLocking13532026/11/11 23:27:02 INFO Received complete push request method=POST path=/api/pushes/3/complete13542026/11/11 23:27:02 WARN Failed to register uploaded object key=6r6l4y5sagi5yvk71y16hxwfh7pbyk48.ls error="server returned 404: 404 page not found\n"13552026/11/11 23:27:02 INFO Completed push id=3 objects=3 duration=809.208µs13562026/11/11 23:27:02 INFO Upload complete. (133ms)1357=== NAME TestClientIntegration1358 client_integration_test.go:353: Retrieved .ls file from S3 (compressed size: 62 bytes)1359 client_integration_test.go:353: Decompressed .ls content (49 bytes):1360 {"version":1,"root":{"type":"regular","size":39}}1361 client_integration_test.go:364: Testing garbage collection...13622026/11/11 23:27:02 INFO Starting cleanup of old closures method=DELETE path=/api/closures13632026/11/11 23:27:02 INFO Garbage collection started13642026/11/11 23:27:02 INFO Aborted multipart uploads count=0 kept=013652026/11/11 23:27:02 WARN Force mode enabled - objects will be deleted immediately without grace period1366--- PASS: TestService_ReadScope_PublicByDefault (7.53s)1367=== CONT TestReadProxyRootRedirectsToIndexHTML1368=== RUN TestService_RequireScope_OIDC/builder_may_write1369=== PAUSE TestService_RequireScope_OIDC/builder_may_write1370=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1371=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1372=== RUN TestService_RequireScope_OIDC/ops_may_admin1373=== PAUSE TestService_RequireScope_OIDC/ops_may_admin1374=== RUN TestService_RequireScope_OIDC/ops_may_not_write1375=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1376=== RUN TestService_RequireScope_OIDC/reader_may_not_write1377=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1378=== RUN TestService_RequireScope_OIDC/static_token_may_admin1379=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1380=== RUN TestService_RequireScope_OIDC/static_token_may_write1381=== PAUSE TestService_RequireScope_OIDC/static_token_may_write1382=== RUN TestService_RequireScope_OIDC/reader_may_read1383=== PAUSE TestService_RequireScope_OIDC/reader_may_read1384=== RUN TestService_RequireScope_OIDC/writer_implies_read1385=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1386=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1387=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read1388=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc1389=== PAUSE TestService_RequireScope_OIDC/reader_may_read_with_netrc1390=== RUN TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1391=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1392=== RUN TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1393=== PAUSE TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1394=== RUN TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1395=== PAUSE TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1396=== CONT TestGetExistingObjectsLocksRows13972026/11/11 23:27:03 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=013982026/11/11 23:27:03 INFO Vacuumed table table=pending_closures13992026/11/11 23:27:03 INFO Vacuumed table table=pending_objects14002026/11/11 23:27:03 INFO Vacuumed table table=multipart_uploads14012026/11/11 23:27:03 INFO Vacuumed table table=closures14022026/11/11 23:27:03 INFO Vacuumed table table=objects14032026/11/11 23:27:03 INFO lead: released remote=192.0.2.1:1234 generation=114042026/11/11 23:27:03 INFO lead: acquired remote=192.0.2.1:1234 generation=21405=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token1406=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token1407=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1408=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1409=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1410=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1411=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1412=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1413=== CONT TestPinProtectsFromGC1414=== NAME TestNARDeduplicationMetadataUploadBug1415 metadata_upload_test.go:48: First store path: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestNARDeduplicationMetadataUploadBug2951935436/001/store/rknhcv2kl6kwbn2ffv4gh0syrbpxbyir-file1.txt14162026/11/11 23:27:03 INFO Received push request method=POST path=/api/pushes14172026/11/11 23:27:04 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)14182026/11/11 23:27:04 INFO Uploading rknhcv2kl6kwbn2ffv4gh0syrbpxbyir-file1.txt (160B)14192026/11/11 23:27:04 WARN Failed to register uploaded object key=rknhcv2kl6kwbn2ffv4gh0syrbpxbyir.ls error="server returned 404: 404 page not found\n"14202026/11/11 23:27:04 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign14212026/11/11 23:27:04 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"14222026/11/11 23:27:04 INFO Signed narinfos id=1 count=114232026/11/11 23:27:04 INFO Uploading 1 narinfos1424--- PASS: TestService_ReadAuthMiddleware (6.97s)1425=== CONT TestPush_OverlappingRootsStoreOneRowPerKey14262026/11/11 23:27:04 INFO Received complete push request method=POST path=/api/pushes/1/complete14272026/11/11 23:27:04 WARN Failed to register uploaded object key=rknhcv2kl6kwbn2ffv4gh0syrbpxbyir.narinfo error="server returned 404: 404 page not found\n"14282026/11/11 23:27:04 INFO Completed push id=1 objects=3 duration=22.077041ms14292026/11/11 23:27:04 INFO Upload complete. (236ms)1430=== NAME TestNARDeduplicationMetadataUploadBug1431 metadata_upload_test.go:54: Retrieved narinfo from S3:1432 StorePath: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestNARDeduplicationMetadataUploadBug2951935436/001/store/rknhcv2kl6kwbn2ffv4gh0syrbpxbyir-file1.txt1433 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1434 Compression: zstd1435 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1436 NarSize: 1601437 References: 1438 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1439 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)1440 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):1441 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}1442 metadata_upload_test.go:64: Second store path (same content): /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestNARDeduplicationMetadataUploadBug2951935436/001/store/75yphbczs03g2gx6wxhxzwjwfvf3bk96-file2.txt14432026/11/11 23:27:04 INFO Received push request method=POST path=/api/pushes14442026/11/11 23:27:04 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)14452026/11/11 23:27:04 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign14462026/11/11 23:27:04 WARN Failed to register uploaded object key=75yphbczs03g2gx6wxhxzwjwfvf3bk96.ls error="server returned 404: 404 page not found\n"14472026/11/11 23:27:04 INFO Signed narinfos id=2 count=114482026/11/11 23:27:04 INFO Uploading 1 narinfos14492026/11/11 23:27:04 INFO Received complete push request method=POST path=/api/pushes/2/complete14502026/11/11 23:27:04 WARN Failed to register uploaded object key=75yphbczs03g2gx6wxhxzwjwfvf3bk96.narinfo error="server returned 404: 404 page not found\n"14512026/11/11 23:27:04 INFO Completed push id=2 objects=3 duration=1.387416ms14522026/11/11 23:27:04 INFO Upload complete. (163ms)1453 metadata_upload_test.go:76: Retrieved narinfo from S3:1454 StorePath: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestNARDeduplicationMetadataUploadBug2951935436/001/store/75yphbczs03g2gx6wxhxzwjwfvf3bk96-file2.txt1455 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1456 Compression: zstd1457 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1458 NarSize: 1601459 References: 1460 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1461 metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes)1462 metadata_upload_test.go:77: Decompressed .ls content (49 bytes):1463 {"version":1,"root":{"type":"regular","size":44}}1464--- PASS: TestNARDeduplicationMetadataUploadBug (8.68s)1465=== CONT TestReadProxyConditionalGet14662026/11/11 23:27:04 INFO lead: released remote=192.0.2.1:1234 generation=21467--- PASS: TestLeadElectsOneAndHandsOver (10.27s)1468=== CONT TestRegisterCompletedObjectNeedsPendingRow14692026/11/11 23:27:04 INFO Installed lifecycle rule aborting incomplete NAR uploads bucket=bucket60 days=21470--- PASS: TestEnsureAbortUploadsRule (7.38s)1471=== CONT TestReadProxyHead1472--- PASS: TestConnectWaitsForAPeerMigration (8.97s)1473=== CONT TestReadProxyOutlastsServerWriteTimeout14742026/11/11 23:27:04 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=01475=== NAME TestClientIntegration1476 client_integration_test.go:371: Objects in database after GC:1477 client_integration_test.go:371: Successfully deleted all objects with GC --force1478--- PASS: TestClientIntegration (11.51s)1479=== CONT TestReadRedirectUsesPublicS3URL14802026/11/11 23:27:05 WARN readiness check failed error="closed pool"1481--- PASS: TestService_readinessHandler (6.36s)1482=== CONT TestSweepSkipsRowsLockedByPush14832026/11/11 23:27:05 INFO Aborted multipart uploads count=1 kept=014842026/11/11 23:27:05 INFO Received cleanup request method=DELETE path=/api/pending_closures14852026/11/11 23:27:05 INFO Aborted multipart uploads count=1 kept=01486--- PASS: TestPendingCleanupUsesOneCutoff (12.61s)1487=== CONT TestClientSharedPathCommittedMidPush14882026/11/11 23:27:05 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"14892026/11/11 23:27:05 WARN mTLS auth: bound subjects configured but subject DN unavailable14902026/11/11 23:27:05 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1491--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (6.58s)1492=== CONT TestClientWithDependencies1493--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (8.35s)1494=== CONT TestReadProxyRangeRequest1495--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (5.47s)1496=== CONT TestReadProxyNarinfoAlreadyDecompressed14972026/11/11 23:27:06 INFO Aborted multipart uploads count=0 kept=014982026/11/11 23:27:06 WARN Force mode enabled - objects will be deleted immediately without grace period14992026/11/11 23:27:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=015002026/11/11 23:27:06 INFO Vacuumed table table=pending_closures15012026/11/11 23:27:06 INFO Vacuumed table table=pending_objects15022026/11/11 23:27:06 INFO Vacuumed table table=multipart_uploads15032026/11/11 23:27:06 INFO Vacuumed table table=closures15042026/11/11 23:27:06 INFO Vacuumed table table=objects1505--- PASS: TestCommitRacingPendingCleanupKeepsObjects (6.05s)1506=== CONT TestClientMultipleUploads15072026/11/11 23:27:06 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"1508--- PASS: TestService_AuthMiddleware (5.88s)1509=== CONT TestReadRedirectKeepsNarinfoProxied15102026/11/11 23:27:07 INFO Received uploads request method=POST path=/api/pending_closures15112026/11/11 23:27:07 INFO Received uploads request method=POST path=/api/pending_closures15122026/11/11 23:27:07 INFO Received uploads request method=POST path=/api/pending_closures15132026/11/11 23:27:07 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)15142026/11/11 23:27:07 INFO Uploading prbvm320k42fz27iwy2hxkcgzvdaf75c-a (256B)15152026/11/11 23:27:07 INFO Uploading sx6jd2ncd9h2b60azk4bv61dymzv96qw-shared-dep (136B)15162026/11/11 23:27:07 WARN Failed to register uploaded object key=1vvrmr1kp6mf70c7fv9kq7qcnr2la6iy.ls error="server returned 404: 404 page not found\n"15172026/11/11 23:27:07 WARN Failed to register uploaded object key=sx6jd2ncd9h2b60azk4bv61dymzv96qw.ls error="server returned 404: 404 page not found\n"15182026/11/11 23:27:07 WARN Failed to register uploaded object key=nar/16kwqs5knw3fd1z18imnly09jnbxfgmjqx91zkx2c2a4s938n8xg.nar.zst error="server returned 404: 404 page not found\n"15192026/11/11 23:27:07 WARN Failed to register uploaded object key=prbvm320k42fz27iwy2hxkcgzvdaf75c.ls error="server returned 404: 404 page not found\n"15202026/11/11 23:27:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign15212026/11/11 23:27:07 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"15222026/11/11 23:27:07 INFO Signed narinfos id=1 count=215232026/11/11 23:27:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign15242026/11/11 23:27:07 INFO Signed narinfos id=2 count=215252026/11/11 23:27:07 INFO Uploading 4 narinfos1526--- PASS: TestPushWaitsForSweepHoldingRow (5.76s)1527=== CONT TestReadProxyNarStreaming15282026/11/11 23:27:07 WARN Failed to register uploaded object key=sx6jd2ncd9h2b60azk4bv61dymzv96qw.narinfo error="server returned 404: 404 page not found\n"15292026/11/11 23:27:07 WARN Failed to register uploaded object key=1vvrmr1kp6mf70c7fv9kq7qcnr2la6iy.narinfo error="server returned 404: 404 page not found\n"15302026/11/11 23:27:07 WARN Failed to register uploaded object key=prbvm320k42fz27iwy2hxkcgzvdaf75c.narinfo error="server returned 404: 404 page not found\n"15312026/11/11 23:27:07 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete15322026/11/11 23:27:07 WARN Failed to register uploaded object key=sx6jd2ncd9h2b60azk4bv61dymzv96qw.narinfo error="server returned 404: 404 page not found\n"15332026/11/11 23:27:07 INFO Completed upload id=1 objects=6 duration=51.847042ms15342026/11/11 23:27:07 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete15352026/11/11 23:27:07 INFO Completed upload id=2 objects=6 duration=641µs15362026/11/11 23:27:07 INFO Upload complete. (470ms)1537=== NAME TestClientFallsBackToClosures1538 client_pushes_test.go:112: Retrieved narinfo from S3:1539 StorePath: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientFallsBackToClosures3469151693/001/store/sx6jd2ncd9h2b60azk4bv61dymzv96qw-shared-dep1540 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1541 Compression: zstd1542 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821543 NarSize: 1361544 References: 1545 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1546 client_pushes_test.go:112: Retrieved narinfo from S3:1547 StorePath: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientFallsBackToClosures3469151693/001/store/prbvm320k42fz27iwy2hxkcgzvdaf75c-a1548 URL: nar/16kwqs5knw3fd1z18imnly09jnbxfgmjqx91zkx2c2a4s938n8xg.nar.zst1549 Compression: zstd1550 NarHash: sha256:16kwqs5knw3fd1z18imnly09jnbxfgmjqx91zkx2c2a4s938n8xg1551 NarSize: 2561552 References: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientFallsBackToClosures3469151693/001/store/sx6jd2ncd9h2b60azk4bv61dymzv96qw-shared-dep1553 CA: text:sha256:1lzfh6lqh892sv1p7v894w164nn1am3c9cqapz8gy0n1v1qymfcg1554 client_pushes_test.go:112: Retrieved narinfo from S3:1555 StorePath: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientFallsBackToClosures3469151693/001/store/1vvrmr1kp6mf70c7fv9kq7qcnr2la6iy-b1556 URL: nar/16kwqs5knw3fd1z18imnly09jnbxfgmjqx91zkx2c2a4s938n8xg.nar.zst1557 Compression: zstd1558 NarHash: sha256:16kwqs5knw3fd1z18imnly09jnbxfgmjqx91zkx2c2a4s938n8xg1559 NarSize: 2561560 References: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientFallsBackToClosures3469151693/001/store/sx6jd2ncd9h2b60azk4bv61dymzv96qw-shared-dep1561 CA: text:sha256:1lzfh6lqh892sv1p7v894w164nn1am3c9cqapz8gy0n1v1qymfcg1562--- PASS: TestReadRedirectNar (5.58s)1563=== CONT TestReadProxyContentNegotiation1564--- PASS: TestClientFallsBackToClosures (6.80s)1565=== CONT TestReadProxy40415662026/11/11 23:27:08 INFO Received push request method=POST path=/api/pushes15672026/11/11 23:27:08 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)15682026/11/11 23:27:08 INFO Uploading qg9w5dyffamdzkdk6mg1r3hmgz0qqlcy-shared-dep (136B)15692026/11/11 23:27:08 INFO Uploading kazdwwrmd082dg0yh6v697j81w6nzvjx-a (256B)15702026/11/11 23:27:08 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"15712026/11/11 23:27:08 WARN Failed to register uploaded object key=k7ipmiwm2rlxfs476hw38zqnx180q08r.ls error="server returned 404: 404 page not found\n"15722026/11/11 23:27:08 WARN Failed to register uploaded object key=qg9w5dyffamdzkdk6mg1r3hmgz0qqlcy.ls error="server returned 404: 404 page not found\n"15732026/11/11 23:27:08 WARN Failed to register uploaded object key=nar/0f9bjz6rmxnw43ajpzkr9dkaa1xfnbqffg5kza91b3i2lfy89f73.nar.zst error="server returned 404: 404 page not found\n"15742026/11/11 23:27:08 WARN Failed to register uploaded object key=kazdwwrmd082dg0yh6v697j81w6nzvjx.ls error="server returned 404: 404 page not found\n"15752026/11/11 23:27:08 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15762026/11/11 23:27:08 INFO Signed narinfos id=1 count=315772026/11/11 23:27:08 INFO Uploading 3 narinfos15782026/11/11 23:27:08 INFO Aborted multipart uploads count=0 kept=015792026/11/11 23:27:08 WARN Failed to register uploaded object key=k7ipmiwm2rlxfs476hw38zqnx180q08r.narinfo error="server returned 404: 404 page not found\n"15802026/11/11 23:27:08 WARN Failed to register uploaded object key=qg9w5dyffamdzkdk6mg1r3hmgz0qqlcy.narinfo error="server returned 404: 404 page not found\n"15812026/11/11 23:27:08 WARN Force mode enabled - objects will be deleted immediately without grace period15822026/11/11 23:27:08 INFO Received complete push request method=POST path=/api/pushes/1/complete15832026/11/11 23:27:08 WARN Failed to register uploaded object key=kazdwwrmd082dg0yh6v697j81w6nzvjx.narinfo error="server returned 404: 404 page not found\n"15842026/11/11 23:27:08 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=015852026/11/11 23:27:08 INFO Completed push id=1 objects=8 duration=48.5915ms15862026/11/11 23:27:08 INFO Upload complete. (339ms)1587=== NAME TestClientPushesUseOnePush1588 client_pushes_test.go:97: Retrieved narinfo from S3:1589 StorePath: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientPushesUseOnePush397435451/001/store/qg9w5dyffamdzkdk6mg1r3hmgz0qqlcy-shared-dep1590 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1591 Compression: zstd1592 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821593 NarSize: 1361594 References: 1595 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1596 client_pushes_test.go:97: Retrieved narinfo from S3:1597 StorePath: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientPushesUseOnePush397435451/001/store/kazdwwrmd082dg0yh6v697j81w6nzvjx-a1598 URL: nar/0f9bjz6rmxnw43ajpzkr9dkaa1xfnbqffg5kza91b3i2lfy89f73.nar.zst1599 Compression: zstd1600 NarHash: sha256:0f9bjz6rmxnw43ajpzkr9dkaa1xfnbqffg5kza91b3i2lfy89f731601 NarSize: 2561602 References: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientPushesUseOnePush397435451/001/store/qg9w5dyffamdzkdk6mg1r3hmgz0qqlcy-shared-dep1603 CA: text:sha256:0iscxbqd3zg9c4jwzjyljpynyiji58a6rapgypw7gz2xvszr6ncg1604 client_pushes_test.go:97: Retrieved narinfo from S3:1605 StorePath: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientPushesUseOnePush397435451/001/store/k7ipmiwm2rlxfs476hw38zqnx180q08r-b1606 URL: nar/0f9bjz6rmxnw43ajpzkr9dkaa1xfnbqffg5kza91b3i2lfy89f73.nar.zst1607 Compression: zstd1608 NarHash: sha256:0f9bjz6rmxnw43ajpzkr9dkaa1xfnbqffg5kza91b3i2lfy89f731609 NarSize: 2561610 References: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientPushesUseOnePush397435451/001/store/qg9w5dyffamdzkdk6mg1r3hmgz0qqlcy-shared-dep1611 CA: text:sha256:0iscxbqd3zg9c4jwzjyljpynyiji58a6rapgypw7gz2xvszr6ncg16122026/11/11 23:27:08 INFO Vacuumed table table=pending_closures16132026/11/11 23:27:08 INFO Vacuumed table table=pending_objects16142026/11/11 23:27:08 INFO Vacuumed table table=multipart_uploads16152026/11/11 23:27:08 INFO Vacuumed table table=closures16162026/11/11 23:27:08 INFO Vacuumed table table=objects1617--- PASS: TestSweepRechecksPendingAfterLocking (5.69s)1618=== CONT TestIsValidCachePath1619=== RUN TestIsValidCachePath/narinfo1620=== PAUSE TestIsValidCachePath/narinfo1621=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars1622=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars1623=== RUN TestIsValidCachePath/nar_zst1624=== PAUSE TestIsValidCachePath/nar_zst1625=== RUN TestIsValidCachePath/nar_xz1626=== PAUSE TestIsValidCachePath/nar_xz1627=== RUN TestIsValidCachePath/nar_bz21628=== PAUSE TestIsValidCachePath/nar_bz21629=== RUN TestIsValidCachePath/nar_uncompressed1630=== PAUSE TestIsValidCachePath/nar_uncompressed1631=== RUN TestIsValidCachePath/ls1632=== PAUSE TestIsValidCachePath/ls1633=== RUN TestIsValidCachePath/log1634=== PAUSE TestIsValidCachePath/log1635=== RUN TestIsValidCachePath/realisation1636=== PAUSE TestIsValidCachePath/realisation1637=== RUN TestIsValidCachePath/nix-cache-info1638=== PAUSE TestIsValidCachePath/nix-cache-info1639=== RUN TestIsValidCachePath/index.html1640=== PAUSE TestIsValidCachePath/index.html1641=== RUN TestIsValidCachePath/traversal_parent1642=== PAUSE TestIsValidCachePath/traversal_parent1643=== RUN TestIsValidCachePath/traversal_in_middle1644=== PAUSE TestIsValidCachePath/traversal_in_middle1645=== RUN TestIsValidCachePath/invalid_char_e1646=== PAUSE TestIsValidCachePath/invalid_char_e1647=== RUN TestIsValidCachePath/invalid_char_u1648=== PAUSE TestIsValidCachePath/invalid_char_u1649=== RUN TestIsValidCachePath/random_path1650=== PAUSE TestIsValidCachePath/random_path1651=== RUN TestIsValidCachePath/empty1652=== PAUSE TestIsValidCachePath/empty1653=== RUN TestIsValidCachePath/leading_slash1654=== PAUSE TestIsValidCachePath/leading_slash1655=== RUN TestIsValidCachePath/wrong_extension1656=== PAUSE TestIsValidCachePath/wrong_extension1657=== RUN TestIsValidCachePath/short_hash1658=== PAUSE TestIsValidCachePath/short_hash1659=== CONT TestReadProxyNarinfo1660--- PASS: TestClientPushesUseOnePush (6.34s)1661=== CONT TestReadProxyInvalidPath1662--- PASS: TestReadProxyDisabled (5.82s)1663=== CONT TestPresentNotReportedWhileGCDeletesClosure1664=== RUN TestPresentNotReportedWhileGCDeletesClosure/stale1665=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/stale1666=== RUN TestPresentNotReportedWhileGCDeletesClosure/fresh1667=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/fresh1668=== CONT TestParseSingleRange1669=== RUN TestParseSingleRange/none1670=== PAUSE TestParseSingleRange/none1671=== RUN TestParseSingleRange/unknown_unit1672=== PAUSE TestParseSingleRange/unknown_unit1673=== RUN TestParseSingleRange/multi-range_ignored1674=== PAUSE TestParseSingleRange/multi-range_ignored1675=== RUN TestParseSingleRange/malformed_no_dash1676=== PAUSE TestParseSingleRange/malformed_no_dash1677=== RUN TestParseSingleRange/malformed_both_empty1678=== PAUSE TestParseSingleRange/malformed_both_empty1679=== RUN TestParseSingleRange/malformed_end_before_start1680=== PAUSE TestParseSingleRange/malformed_end_before_start1681=== RUN TestParseSingleRange/closed1682=== PAUSE TestParseSingleRange/closed1683=== RUN TestParseSingleRange/open-ended1684=== PAUSE TestParseSingleRange/open-ended1685=== RUN TestParseSingleRange/end_clamped_to_size1686=== PAUSE TestParseSingleRange/end_clamped_to_size1687=== RUN TestParseSingleRange/suffix1688=== PAUSE TestParseSingleRange/suffix1689=== RUN TestParseSingleRange/suffix_exceeds_size1690=== PAUSE TestParseSingleRange/suffix_exceeds_size1691=== RUN TestParseSingleRange/single_byte1692=== PAUSE TestParseSingleRange/single_byte1693=== RUN TestParseSingleRange/start_past_EOF1694=== PAUSE TestParseSingleRange/start_past_EOF1695=== RUN TestParseSingleRange/start_far_past_EOF1696=== PAUSE TestParseSingleRange/start_far_past_EOF1697=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1698--- PASS: TestReadProxyRootRedirectsToIndexHTML (5.93s)1699=== CONT TestForceGCDuringPushOffersSweptObject/before_pending_rows1700=== NAME TestOrphanedObjectsGCStressTest1701 orphaned_objects_gc_test.go:422: Created 10 active closures, 5 to-delete closures, 20 orphaned chains1702 orphaned_objects_gc_test.go:440: Marked 210 objects for deletion1703--- PASS: TestGetExistingObjectsLocksRows (5.83s)1704=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1705--- PASS: TestDeleteClosuresRacingPins (10.96s)1706=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed17072026/11/11 23:27:09 INFO Received push request method=POST path=/api/pushes1708--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (5.75s)1709=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1710=== NAME TestPinProtectsFromGC1711 client_integration_test.go:779: Pinned store path: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestPinProtectsFromGC1267649327/001/store/zdflb85ailijbn80pdsay58m02apl2l4-pinned-file.txt1712 client_integration_test.go:780: Unpinned store path: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestPinProtectsFromGC1267649327/001/store/nsps1v9h41q8358ckd18yz2di922lxv4-unpinned-file.txt17132026/11/11 23:27:10 INFO Received push request method=POST path=/api/pushes17142026/11/11 23:27:10 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)17152026/11/11 23:27:10 INFO Uploading zdflb85ailijbn80pdsay58m02apl2l4-pinned-file.txt (128B)17162026/11/11 23:27:10 WARN Failed to register uploaded object key=zdflb85ailijbn80pdsay58m02apl2l4.ls error="server returned 404: 404 page not found\n"17172026/11/11 23:27:10 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign17182026/11/11 23:27:10 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"17192026/11/11 23:27:10 INFO Signed narinfos id=1 count=117202026/11/11 23:27:10 INFO Uploading 1 narinfos17212026/11/11 23:27:10 INFO Received complete push request method=POST path=/api/pushes/1/complete17222026/11/11 23:27:10 WARN Failed to register uploaded object key=zdflb85ailijbn80pdsay58m02apl2l4.narinfo error="server returned 404: 404 page not found\n"1723--- PASS: TestReadProxyConditionalGet (5.60s)1724=== CONT TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed17252026/11/11 23:27:10 INFO Completed push id=1 objects=3 duration=38.648292ms17262026/11/11 23:27:10 INFO Upload complete. (271ms)17272026/11/11 23:27:10 INFO Received push request method=POST path=/api/pushes17282026/11/11 23:27:10 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)17292026/11/11 23:27:10 INFO Uploading nsps1v9h41q8358ckd18yz2di922lxv4-unpinned-file.txt (128B)17302026/11/11 23:27:10 WARN Failed to register uploaded object key=nsps1v9h41q8358ckd18yz2di922lxv4.ls error="server returned 404: 404 page not found\n"17312026/11/11 23:27:10 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign17322026/11/11 23:27:10 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"17332026/11/11 23:27:10 INFO Signed narinfos id=2 count=117342026/11/11 23:27:10 INFO Uploading 1 narinfos17352026/11/11 23:27:10 INFO Received complete push request method=POST path=/api/pushes/2/complete17362026/11/11 23:27:10 WARN Failed to register uploaded object key=nsps1v9h41q8358ckd18yz2di922lxv4.narinfo error="server returned 404: 404 page not found\n"17372026/11/11 23:27:10 INFO Completed push id=2 objects=3 duration=988.416µs17382026/11/11 23:27:10 INFO Upload complete. (206ms)17392026/11/11 23:27:10 INFO Starting cleanup of old closures method=DELETE path=/api/closures17402026/11/11 23:27:10 INFO Garbage collection started17412026/11/11 23:27:10 INFO Aborted multipart uploads count=0 kept=01742--- PASS: TestReadProxyHead (5.71s)1743=== CONT TestPendingClosureWriteTimeout/empty1744=== CONT TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline17452026/11/11 23:27:10 WARN Force mode enabled - objects will be deleted immediately without grace period1746--- PASS: TestRegisterCompletedObjectNeedsPendingRow (6.19s)1747=== CONT TestForceGCDuringPushOffersSweptObject/after_presence_check17482026/11/11 23:27:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=017492026/11/11 23:27:11 INFO Vacuumed table table=pending_closures17502026/11/11 23:27:11 INFO Vacuumed table table=pending_objects17512026/11/11 23:27:11 INFO Vacuumed table table=multipart_uploads17522026/11/11 23:27:11 INFO Vacuumed table table=closures17532026/11/11 23:27:11 INFO Vacuumed table table=objects1754--- PASS: TestReadRedirectUsesPublicS3URL (6.89s)1755=== CONT TestPendingClosureWriteTimeout/670k_objects1756=== CONT TestPendingClosureWriteTimeout/400_objects1757=== CONT TestProxyWriteTimeout/narinfo1758=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info17592026/11/11 23:27:11 INFO Received uploads request method=POST path=/1760=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key17612026/11/11 23:27:11 INFO Received request for more parts method=POST path=/1762=== CONT TestProxyWriteTimeout/10_GiB_nar1763=== CONT TestProxyWriteTimeout/unknown_size1764=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key17652026/11/11 23:27:11 INFO Received complete multipart upload request method=POST path=/1766=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal17672026/11/11 23:27:11 INFO Received uploads request method=POST path=/1768--- PASS: TestUploadHandlersRejectInvalidKeys (0.05s)1769 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)1770 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1771 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1772 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s)1773=== CONT TestIsValidUploadKey/narinfo1774=== CONT TestProxyWriteTimeout/1_GiB_nar1775--- PASS: TestProxyWriteTimeout (0.05s)1776 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1777 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1778 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1779 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)1780=== CONT TestIsValidUploadKey/nix-cache-info1781=== CONT TestIsValidUploadKey/unknown_type1782=== CONT TestIsValidUploadKey/empty_key1783=== CONT TestIsValidUploadKey/absolute1784=== CONT TestIsValidUploadKey/traversal_nar1785=== CONT TestIsValidUploadKey/traversal1786=== CONT TestIsValidUploadKey/listing_key,_narinfo_type1787=== CONT TestIsValidUploadKey/nar_key,_narinfo_type1788=== CONT TestIsValidUploadKey/narinfo_key,_nar_type1789=== CONT TestIsValidUploadKey/index.html1790=== CONT TestIsValidUploadKey/build_log_home-manager_file1791=== CONT TestIsValidUploadKey/realisation_plus_in_output1792=== CONT TestIsValidUploadKey/realisation1793=== CONT TestIsValidUploadKey/build_log_equals1794=== CONT TestIsValidUploadKey/build_log_question_mark1795=== CONT TestIsValidUploadKey/build_log_plus_in_name1796=== CONT TestIsValidUploadKey/nar_plain1797=== CONT TestIsValidUploadKey/build_log1798=== CONT TestIsValidUploadKey/listing1799=== CONT TestIsValidUploadKey/nar_xz1800=== CONT TestIsValidUploadKey/nar_zst1801--- PASS: TestIsValidUploadKey (0.05s)1802 --- PASS: TestIsValidUploadKey/narinfo (0.00s)1803 --- PASS: TestIsValidUploadKey/nix-cache-info (0.00s)1804 --- PASS: TestIsValidUploadKey/unknown_type (0.00s)1805 --- PASS: TestIsValidUploadKey/empty_key (0.00s)1806 --- PASS: TestIsValidUploadKey/absolute (0.00s)1807 --- PASS: TestIsValidUploadKey/traversal_nar (0.00s)1808 --- PASS: TestIsValidUploadKey/traversal (0.00s)1809 --- PASS: TestIsValidUploadKey/listing_key,_narinfo_type (0.00s)1810 --- PASS: TestIsValidUploadKey/nar_key,_narinfo_type (0.00s)1811 --- PASS: TestIsValidUploadKey/narinfo_key,_nar_type (0.00s)1812 --- PASS: TestIsValidUploadKey/index.html (0.00s)1813 --- PASS: TestIsValidUploadKey/build_log_home-manager_file (0.00s)1814 --- PASS: TestIsValidUploadKey/realisation_plus_in_output (0.00s)1815 --- PASS: TestIsValidUploadKey/realisation (0.00s)1816 --- PASS: TestIsValidUploadKey/build_log_equals (0.00s)1817 --- PASS: TestIsValidUploadKey/build_log_question_mark (0.00s)1818 --- PASS: TestIsValidUploadKey/build_log_plus_in_name (0.00s)1819 --- PASS: TestIsValidUploadKey/nar_plain (0.00s)1820 --- PASS: TestIsValidUploadKey/build_log (0.00s)1821 --- PASS: TestIsValidUploadKey/listing (0.00s)1822 --- PASS: TestIsValidUploadKey/nar_xz (0.00s)1823 --- PASS: TestIsValidUploadKey/nar_zst (0.00s)1824=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure18252026/11/11 23:27:11 INFO Received uploads request method=POST path=/1826=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts18272026/11/11 23:27:12 INFO Received request for more parts method=POST path=/1828=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart18292026/11/11 23:27:12 INFO Received complete multipart upload request method=POST path=/1830--- PASS: TestUploadHandlersRejectOversizedBody (0.09s)1831 --- PASS: TestUploadHandlersRejectOversizedBody/create_pending_closure (0.30s)1832 --- PASS: TestUploadHandlersRejectOversizedBody/request_more_parts (0.02s)1833 --- PASS: TestUploadHandlersRejectOversizedBody/complete_multipart (0.02s)1834=== CONT TestServerTLSConfig/no_client_CA1835=== CONT TestServerTLSConfig/not_a_PEM_file1836=== CONT TestServerTLSConfig/missing_CA_file1837--- PASS: TestServerTLSConfig (0.00s)1838 --- PASS: TestServerTLSConfig/no_client_CA (0.00s)1839 --- PASS: TestServerTLSConfig/not_a_PEM_file (0.03s)1840 --- PASS: TestServerTLSConfig/missing_CA_file (0.00s)1841=== CONT TestPush_RejectsBadRequests/no_roots18422026/11/11 23:27:12 INFO Received push request method=POST path=/api/pushes1843=== CONT TestClientErrorHandling/InvalidStorePath18442026/11/11 23:27:12 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=018452026/11/11 23:27:12 INFO Received create pin request method=POST path=/api/pins/myapp18462026/11/11 23:27:12 INFO Aborted multipart uploads count=0 kept=018472026/11/11 23:27:12 INFO Created/updated pin name=myapp store_path=/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestPinProtectsFromGC1267649327/001/store/zdflb85ailijbn80pdsay58m02apl2l4-pinned-file.txt narinfo_key=zdflb85ailijbn80pdsay58m02apl2l4.narinfo18482026/11/11 23:27:12 INFO Starting cleanup of old closures method=DELETE path=/api/closures18492026/11/11 23:27:12 INFO Garbage collection started18502026/11/11 23:27:12 INFO Aborted multipart uploads count=0 kept=018512026/11/11 23:27:12 WARN Force mode enabled - objects will be deleted immediately without grace period18522026/11/11 23:27:12 WARN Force mode enabled - objects will be deleted immediately without grace period18532026/11/11 23:27:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=018542026/11/11 23:27:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=018552026/11/11 23:27:12 INFO Vacuumed table table=pending_closures18562026/11/11 23:27:12 INFO Vacuumed table table=pending_closures18572026/11/11 23:27:12 INFO Vacuumed table table=pending_objects18582026/11/11 23:27:12 INFO Vacuumed table table=pending_objects18592026/11/11 23:27:12 INFO Vacuumed table table=multipart_uploads18602026/11/11 23:27:12 INFO Vacuumed table table=multipart_uploads18612026/11/11 23:27:12 INFO Vacuumed table table=closures18622026/11/11 23:27:12 INFO Vacuumed table table=closures18632026/11/11 23:27:12 INFO Vacuumed table table=objects18642026/11/11 23:27:12 INFO Received push request method=POST path=/api/pushes18652026/11/11 23:27:12 INFO Vacuumed table table=objects18662026/11/11 23:27:12 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)18672026/11/11 23:27:12 INFO Uploading zr88yfsxk3d960hbzjyjvz6l1kqmxwl2-shared-dep (136B)18682026/11/11 23:27:12 INFO Uploading bzxvvz7g41lf1fhy5052d1avdjn43s8k-top (264B)18692026/11/11 23:27:12 INFO Aborted multipart uploads count=0 kept=018702026/11/11 23:27:12 WARN Force mode enabled - objects will be deleted immediately without grace period18712026/11/11 23:27:12 WARN Failed to register uploaded object key=zr88yfsxk3d960hbzjyjvz6l1kqmxwl2.ls error="server returned 404: 404 page not found\n"18722026/11/11 23:27:12 WARN Failed to register uploaded object key=bzxvvz7g41lf1fhy5052d1avdjn43s8k.ls error="server returned 404: 404 page not found\n"18732026/11/11 23:27:12 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"18742026/11/11 23:27:12 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign18752026/11/11 23:27:12 WARN Failed to register uploaded object key=nar/0chb0axr5x3cvh4a388r7nn6a70r3z83p3j17vypj2dr9y35anr3.nar.zst error="server returned 404: 404 page not found\n"18762026/11/11 23:27:12 INFO Signed narinfos id=1 count=218772026/11/11 23:27:12 INFO Uploading 2 narinfos18782026/11/11 23:27:12 WARN Failed to register uploaded object key=zr88yfsxk3d960hbzjyjvz6l1kqmxwl2.narinfo error="server returned 404: 404 page not found\n"18792026/11/11 23:27:12 INFO Received complete push request method=POST path=/api/pushes/1/complete18802026/11/11 23:27:12 WARN Failed to register uploaded object key=bzxvvz7g41lf1fhy5052d1avdjn43s8k.narinfo error="server returned 404: 404 page not found\n"1881--- PASS: TestReadProxyRangeRequest (7.27s)1882=== CONT TestPush_RejectsBadRequests/root_not_in_objects18832026/11/11 23:27:12 INFO Received push request method=POST path=/api/pushes1884=== CONT TestPush_RejectsBadRequests/bad_root18852026/11/11 23:27:12 INFO Received push request method=POST path=/api/pushes1886=== CONT TestPush_RejectsBadRequests/no_objects18872026/11/11 23:27:12 INFO Completed push id=1 objects=6 duration=41.81525ms18882026/11/11 23:27:12 INFO Received push request method=POST path=/api/pushes1889=== CONT TestClientErrorHandling/ServerNotAvailable18902026/11/11 23:27:12 INFO Upload complete. (323ms)1891--- PASS: TestPush_RejectsBadRequests (6.41s)1892 --- PASS: TestPush_RejectsBadRequests/no_roots (0.01s)1893 --- PASS: TestPush_RejectsBadRequests/root_not_in_objects (0.01s)1894 --- PASS: TestPush_RejectsBadRequests/bad_root (0.01s)1895 --- PASS: TestPush_RejectsBadRequests/no_objects (0.00s)1896=== NAME TestClientSharedPathCommittedMidPush1897 client_integration_test.go:728: Retrieved narinfo from S3:1898 StorePath: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientSharedPathCommittedMidPush1172943376/001/store/zr88yfsxk3d960hbzjyjvz6l1kqmxwl2-shared-dep1899 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1900 Compression: zstd1901 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821902 NarSize: 1361903 References: 1904 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1905 client_integration_test.go:728: Retrieved narinfo from S3:1906 StorePath: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientSharedPathCommittedMidPush1172943376/001/store/bzxvvz7g41lf1fhy5052d1avdjn43s8k-top1907 URL: nar/0chb0axr5x3cvh4a388r7nn6a70r3z83p3j17vypj2dr9y35anr3.nar.zst1908 Compression: zstd1909 NarHash: sha256:0chb0axr5x3cvh4a388r7nn6a70r3z83p3j17vypj2dr9y35anr31910 NarSize: 2641911 References: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientSharedPathCommittedMidPush1172943376/001/store/zr88yfsxk3d960hbzjyjvz6l1kqmxwl2-shared-dep1912 CA: text:sha256:0gq1w66asshpgjqljilgmxwkqfi054ga39gxzhxdakzwmi03ii6d19132026/11/11 23:27:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=019142026/11/11 23:27:13 INFO Vacuumed table table=pending_closures19152026/11/11 23:27:13 INFO Vacuumed table table=pending_objects19162026/11/11 23:27:13 INFO Vacuumed table table=multipart_uploads19172026/11/11 23:27:13 INFO Vacuumed table table=closures19182026/11/11 23:27:13 INFO Vacuumed table table=objects1919--- PASS: TestClientSharedPathCommittedMidPush (7.53s)1920--- PASS: TestSweepSkipsRowsLockedByPush (7.74s)1921=== CONT TestCacheConfigHandler/full_config,_no_issuer1922=== CONT TestClientErrorHandling/InvalidAuthToken1923=== CONT TestCacheConfigHandler/no_signing_keys1924=== CONT TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1925=== CONT TestCacheConfigHandler/no_cache_url_configured1926--- PASS: TestCacheConfigHandler (0.00s)1927 --- PASS: TestCacheConfigHandler/full_config,_no_issuer (0.00s)1928 --- PASS: TestCacheConfigHandler/no_signing_keys (0.00s)1929 --- PASS: TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator (0.00s)1930 --- PASS: TestCacheConfigHandler/no_cache_url_configured (0.00s)1931=== CONT TestResolveDBConnectionString/flag_wins1932=== CONT TestResolveDBConnectionString/PGHOST_allows_empty1933=== CONT TestResolveDBConnectionString/nothing_configured1934=== CONT TestResolveDBConnectionString/missing_file_is_an_error1935=== CONT TestResolveDBConnectionString/file_when_flag_empty1936=== CONT TestGCEndsOnShutdown/before_the_run1937--- PASS: TestResolveDBConnectionString (0.03s)1938 --- PASS: TestResolveDBConnectionString/flag_wins (0.00s)1939 --- PASS: TestResolveDBConnectionString/PGHOST_allows_empty (0.00s)1940 --- PASS: TestResolveDBConnectionString/nothing_configured (0.00s)1941 --- PASS: TestResolveDBConnectionString/missing_file_is_an_error (0.00s)1942 --- PASS: TestResolveDBConnectionString/file_when_flag_empty (0.00s)19432026/11/11 23:27:13 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present19442026/11/11 23:27:13 WARN Request failed, retrying attempt=2 backoff=183.949934ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present19452026/11/11 23:27:13 WARN Request failed, retrying attempt=3 backoff=409.380732ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present1946--- PASS: TestReadProxyNarinfoAlreadyDecompressed (7.70s)1947=== CONT TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1948=== NAME TestClientWithDependencies1949 client_integration_test.go:661: Built derivation: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientWithDependencies3394166054/001/store/2icab7v9zrk57lgmr463jc1flwaxhjc3-test-script1950 client_integration_test.go:663: Found 1 dependencies (including self)19512026/11/11 23:27:13 INFO Received push request method=POST path=/api/pushes19522026/11/11 23:27:13 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)19532026/11/11 23:27:13 INFO Uploading 2icab7v9zrk57lgmr463jc1flwaxhjc3-test-script (136B)19542026/11/11 23:27:13 WARN Request failed, retrying attempt=4 backoff=765.141358ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present19552026/11/11 23:27:14 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"19562026/11/11 23:27:14 WARN Failed to register uploaded object key=2icab7v9zrk57lgmr463jc1flwaxhjc3.ls error="server returned 404: 404 page not found\n"19572026/11/11 23:27:14 WARN Failed to register uploaded object key=log/75v7xxna7fdmwzpnskwk6a2aa64kaskq-test-script.drv error="server returned 404: 404 page not found\n"19582026/11/11 23:27:14 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign19592026/11/11 23:27:14 INFO Signed narinfos id=1 count=119602026/11/11 23:27:14 INFO Uploading 1 narinfos19612026/11/11 23:27:14 INFO Received complete push request method=POST path=/api/pushes/1/complete19622026/11/11 23:27:14 WARN Failed to register uploaded object key=2icab7v9zrk57lgmr463jc1flwaxhjc3.narinfo error="server returned 404: 404 page not found\n"1963=== NAME TestClientMultipleUploads1964 client_integration_test.go:406: Created store path 0: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientMultipleUploads3848545978/001/store/c8hj3bl62i6jsbxxzwxv7vxlhbalhywb-test-file-0.txt1965--- PASS: TestReadRedirectKeepsNarinfoProxied (7.61s)1966=== CONT TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete19672026/11/11 23:27:14 INFO Completed push id=1 objects=4 duration=59.14275ms19682026/11/11 23:27:14 INFO Upload complete. (326ms)1969=== NAME TestClientWithDependencies1970 client_integration_test.go:665: Skipping nix copy test - isolated store (/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientWithDependencies3394166054/001/store) requires matching store prefix1971=== NAME TestClientMultipleUploads1972 client_integration_test.go:406: Created store path 1: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestClientMultipleUploads3848545978/001/store/cvggxm7aq4lcrp8ilmd966svd1p8pc93-test-file-1.txt1973--- PASS: TestClientWithDependencies (8.67s)1974=== CONT TestService_RequireScope_OIDC/builder_may_write1975=== CONT TestService_RequireScope_OIDC/reader_may_read1976=== CONT TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1977=== CONT TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read19782026/11/11 23:27:14 WARN Authentication failed token_preview=wrong token_length=5 oidc_error="no provider could verify the token (signature or issuer mismatch)" oidc_provider="" tried_providers=[test]1979=== CONT TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1980=== CONT TestService_RequireScope_OIDC/reader_may_read_with_netrc1981=== CONT TestService_RequireScope_OIDC/anonymous_may_not_read1982=== CONT TestService_RequireScope_OIDC/writer_implies_read1983=== CONT TestService_RequireScope_OIDC/reader_may_not_write1984=== CONT TestService_RequireScope_OIDC/static_token_may_write1985=== CONT TestService_RequireScope_OIDC/static_token_may_admin1986=== CONT TestService_RequireScope_OIDC/ops_may_admin1987=== CONT TestService_RequireScope_OIDC/ops_may_not_write1988=== CONT TestService_RequireScope_OIDC/builder_may_not_admin1989=== CONT TestService_AuthMiddleware_OIDC/valid_OIDC_token1990--- PASS: TestService_RequireScope_OIDC (7.67s)1991 --- PASS: TestService_RequireScope_OIDC/builder_may_write (0.00s)1992 --- PASS: TestService_RequireScope_OIDC/reader_may_read (0.00s)1993 --- PASS: TestService_RequireScope_OIDC/empty_netrc_password_may_not_read (0.00s)1994 --- PASS: TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read (0.01s)1995 --- PASS: TestService_RequireScope_OIDC/static_token_may_admin_with_netrc (0.00s)1996 --- PASS: TestService_RequireScope_OIDC/reader_may_read_with_netrc (0.00s)1997 --- PASS: TestService_RequireScope_OIDC/anonymous_may_not_read (0.00s)1998 --- PASS: TestService_RequireScope_OIDC/writer_implies_read (0.00s)1999 --- PASS: TestService_RequireScope_OIDC/reader_may_not_write (0.00s)2000 --- PASS: TestService_RequireScope_OIDC/static_token_may_write (0.00s)

loading 654 hidden lines…

26552026/11/11 23:27:32 INFO Aborted multipart uploads count=0 kept=026562026/11/11 23:27:32 INFO Vacuumed table table=pending_objects26572026/11/11 23:27:32 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=026582026/11/11 23:27:32 WARN Force mode enabled - objects will be deleted immediately without grace period26592026/11/11 23:27:32 INFO Received push request method=POST path=/26602026/11/11 23:27:32 INFO Vacuumed table table=pending_closures26612026/11/11 23:27:32 INFO Vacuumed table table=multipart_uploads26622026/11/11 23:27:32 INFO Vacuumed table table=pending_objects26632026/11/11 23:27:32 INFO Vacuumed table table=closures26642026/11/11 23:27:32 INFO Received push request method=POST path=/26652026/11/11 23:27:32 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."26662026/11/11 23:27:32 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."26672026/11/11 23:27:32 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."26682026/11/11 23:27:32 INFO Vacuumed table table=multipart_uploads26692026/11/11 23:27:32 INFO Aborted multipart uploads count=0 kept=026702026/11/11 23:27:32 INFO Vacuumed table table=objects26712026/11/11 23:27:32 INFO Vacuumed table table=closures26722026/11/11 23:27:32 WARN Force mode enabled - objects will be deleted immediately without grace period26732026/11/11 23:27:32 INFO Received uploads request method=POST path=/26742026/11/11 23:27:32 INFO Aborted multipart uploads count=0 kept=026752026/11/11 23:27:32 INFO Received complete upload request method=POST path=/26762026/11/11 23:27:32 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=026772026/11/11 23:27:32 INFO Vacuumed table table=objects26782026/11/11 23:27:32 WARN Force mode enabled - objects will be deleted immediately without grace period26792026/11/11 23:27:32 INFO Vacuumed table table=pending_closures26802026/11/11 23:27:32 INFO Received uploads request method=POST path=/26812026/11/11 23:27:32 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."26822026/11/11 23:27:32 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."26832026/11/11 23:27:32 INFO Aborted multipart uploads count=0 kept=026842026/11/11 23:27:32 INFO Received uploads request method=POST path=/26852026/11/11 23:27:32 INFO Vacuumed table table=pending_objects26862026/11/11 23:27:32 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."26872026/11/11 23:27:32 INFO Vacuumed table table=multipart_uploads26882026/11/11 23:27:32 WARN Force mode enabled - objects will be deleted immediately without grace period26892026/11/11 23:27:32 INFO Vacuumed table table=closures26902026/11/11 23:27:32 INFO Completed upload id=1 objects=9 duration=33.882458ms26912026/11/11 23:27:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=026922026/11/11 23:27:32 INFO Vacuumed table table=objects26932026/11/11 23:27:32 INFO Received push request method=POST path=/26942026/11/11 23:27:32 INFO Vacuumed table table=pending_closures26952026/11/11 23:27:32 INFO Vacuumed table table=pending_objects26962026/11/11 23:27:32 INFO Received uploads request method=POST path=/26972026/11/11 23:27:32 INFO Vacuumed table table=multipart_uploads26982026/11/11 23:27:32 INFO Received uploads request method=POST path=/26992026/11/11 23:27:32 INFO Vacuumed table table=closures27002026/11/11 23:27:32 INFO Vacuumed table table=objects27012026/11/11 23:27:32 INFO Received complete upload request method=POST path=/27022026/11/11 23:27:32 INFO Aborted multipart uploads count=0 kept=027032026/11/11 23:27:32 INFO Completed upload id=9 objects=7 duration=585.041µs27042026/11/11 23:27:32 WARN Force mode enabled - objects will be deleted immediately without grace period27052026/11/11 23:27:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027062026/11/11 23:27:32 INFO Vacuumed table table=pending_closures27072026/11/11 23:27:32 INFO Received complete push request method=POST path=/27082026/11/11 23:27:32 INFO Vacuumed table table=pending_objects27092026/11/11 23:27:32 INFO Aborted multipart uploads count=0 kept=027102026/11/11 23:27:32 INFO Vacuumed table table=multipart_uploads27112026/11/11 23:27:32 INFO Vacuumed table table=closures27122026/11/11 23:27:32 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."27132026/11/11 23:27:32 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."27142026/11/11 23:27:32 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."27152026/11/11 23:27:32 INFO Vacuumed table table=objects27162026/11/11 23:27:32 INFO Received uploads request method=POST path=/27172026/11/11 23:27:32 INFO Completed push id=1 objects=24 duration=27.995583ms27182026/11/11 23:27:32 INFO Received complete push request method=POST path=/27192026/11/11 23:27:32 INFO Completed push id=10 objects=2 duration=514.333µs27202026/11/11 23:27:32 INFO Aborted multipart uploads count=0 kept=027212026/11/11 23:27:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027222026/11/11 23:27:32 INFO Received uploads request method=POST path=/27232026/11/11 23:27:33 INFO Vacuumed table table=pending_closures27242026/11/11 23:27:33 INFO Received complete upload request method=POST path=/27252026/11/11 23:27:33 INFO Received uploads request method=POST path=/27262026/11/11 23:27:33 INFO Completed upload id=12 objects=21 duration=406µs27272026/11/11 23:27:33 INFO Received complete push request method=POST path=/27282026/11/11 23:27:33 INFO Aborted multipart uploads count=0 kept=027292026/11/11 23:27:33 INFO Received push request method=POST path=/27302026/11/11 23:27:33 INFO Completed push id=1 objects=11 duration=660.25µs27312026/11/11 23:27:33 INFO Vacuumed table table=pending_objects27322026/11/11 23:27:33 INFO Vacuumed table table=multipart_uploads27332026/11/11 23:27:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027342026/11/11 23:27:33 INFO Received complete upload request method=POST path=/27352026/11/11 23:27:33 INFO Completed upload id=10 objects=6 duration=934µs27362026/11/11 23:27:33 INFO Vacuumed table table=closures27372026/11/11 23:27:33 INFO Vacuumed table table=pending_closures27382026/11/11 23:27:33 INFO Received complete push request method=POST path=/27392026/11/11 23:27:33 INFO Completed push id=13 objects=23 duration=500µs27402026/11/11 23:27:33 INFO Received push request method=POST path=/27412026/11/11 23:27:33 INFO Received complete upload request method=POST path=/27422026/11/11 23:27:33 INFO Received complete push request method=POST path=/27432026/11/11 23:27:33 INFO Vacuumed table table=objects27442026/11/11 23:27:33 INFO Completed upload id=16 objects=3 duration=673.791µs27452026/11/11 23:27:33 INFO Aborted multipart uploads count=0 kept=027462026/11/11 23:27:33 INFO Completed push id=8 objects=6 duration=508.042µs27472026/11/11 23:27:33 INFO Vacuumed table table=pending_objects27482026/11/11 23:27:33 INFO Received push request method=POST path=/27492026/11/11 23:27:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=027502026/11/11 23:27:33 INFO Vacuumed table table=multipart_uploads27512026/11/11 23:27:33 INFO Received push request method=POST path=/27522026/11/11 23:27:33 INFO Vacuumed table table=closures27532026/11/11 23:27:33 INFO Vacuumed table table=pending_closures27542026/11/11 23:27:33 INFO Received push request method=POST path=/27552026/11/11 23:27:33 INFO Received uploads request method=POST path=/27562026/11/11 23:27:33 INFO Vacuumed table table=pending_objects27572026/11/11 23:27:33 INFO Received push request method=POST path=/27582026/11/11 23:27:33 INFO Vacuumed table table=multipart_uploads27592026/11/11 23:27:33 INFO Received uploads request method=POST path=/27602026/11/11 23:27:33 INFO Vacuumed table table=closures27612026/11/11 23:27:33 INFO Vacuumed table table=objects27622026/11/11 23:27:33 INFO Received uploads request method=POST path=/27632026/11/11 23:27:33 INFO Received complete push request method=POST path=/27642026/11/11 23:27:33 INFO Vacuumed table table=objects27652026/11/11 23:27:33 INFO Completed push id=12 objects=7 duration=540.792µs27662026/11/11 23:27:33 INFO Aborted multipart uploads count=0 kept=027672026/11/11 23:27:33 INFO Received push request method=POST path=/27682026/11/11 23:27:33 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=027692026/11/11 23:27:33 INFO Received uploads request method=POST path=/27702026/11/11 23:27:33 INFO Received uploads request method=POST path=/27712026/11/11 23:27:33 INFO Received complete upload request method=POST path=/27722026/11/11 23:27:33 INFO Completed upload id=15 objects=4 duration=615.708µs27732026/11/11 23:27:33 INFO Received push request method=POST path=/27742026/11/11 23:27:33 INFO Vacuumed table table=pending_closures27752026/11/11 23:27:33 INFO Received uploads request method=POST path=/27762026/11/11 23:27:33 INFO Received complete upload request method=POST path=/27772026/11/11 23:27:33 INFO Completed upload id=16 objects=3 duration=252.75µs27782026/11/11 23:27:33 INFO Aborted multipart uploads count=0 kept=027792026/11/11 23:27:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027802026/11/11 23:27:33 INFO Vacuumed table table=pending_closures27812026/11/11 23:27:33 INFO Vacuumed table table=pending_objects27822026/11/11 23:27:33 INFO Vacuumed table table=multipart_uploads27832026/11/11 23:27:33 INFO Vacuumed table table=pending_objects27842026/11/11 23:27:33 INFO Received complete upload request method=POST path=/27852026/11/11 23:27:33 INFO Vacuumed table table=closures27862026/11/11 23:27:33 INFO Vacuumed table table=multipart_uploads27872026/11/11 23:27:33 INFO Received complete upload request method=POST path=/27882026/11/11 23:27:33 INFO Vacuumed table table=closures27892026/11/11 23:27:33 INFO Vacuumed table table=objects27902026/11/11 23:27:33 INFO Completed upload id=5 objects=7 duration=776.792µs27912026/11/11 23:27:33 INFO Vacuumed table table=objects27922026/11/11 23:27:33 INFO Received complete push request method=POST path=/27932026/11/11 23:27:33 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."27942026/11/11 23:27:33 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."27952026/11/11 23:27:33 INFO Completed push id=15 objects=7 duration=751.75µs27962026/11/11 23:27:33 INFO Received uploads request method=POST path=/27972026/11/11 23:27:33 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."27982026/11/11 23:27:33 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."27992026/11/11 23:27:33 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."28002026/11/11 23:27:33 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."28012026/11/11 23:27:33 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."28022026/11/11 23:27:33 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."28032026/11/11 23:27:33 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."28042026/11/11 23:27:33 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."28052026/11/11 23:27:33 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."28062026/11/11 23:27:33 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."28072026/11/11 23:27:33 INFO Completed upload id=2 objects=7 duration=35.619875ms28082026/11/11 23:27:33 INFO Received uploads request method=POST path=/28092026/11/11 23:27:33 INFO Received uploads request method=POST path=/28102026/11/11 23:27:33 INFO Received uploads request method=POST path=/28112026/11/11 23:27:33 INFO Received uploads request method=POST path=/28122026/11/11 23:27:33 INFO Received uploads request method=POST path=/28132026/11/11 23:27:33 INFO Received push request method=POST path=/28142026/11/11 23:27:33 INFO Received uploads request method=POST path=/28152026/11/11 23:27:33 INFO Received uploads request method=POST path=/28162026/11/11 23:27:33 INFO Received complete upload request method=POST path=/28172026/11/11 23:27:33 INFO Received uploads request method=POST path=/28182026/11/11 23:27:33 INFO Completed upload id=18 objects=4 duration=608.125µs28192026/11/11 23:27:33 INFO Received push request method=POST path=/28202026/11/11 23:27:33 INFO Received push request method=POST path=/28212026/11/11 23:27:33 INFO Received complete upload request method=POST path=/28222026/11/11 23:27:33 INFO Received push request method=POST path=/28232026/11/11 23:27:33 INFO Received complete upload request method=POST path=/28242026/11/11 23:27:33 INFO Received push request method=POST path=/28252026/11/11 23:27:33 INFO Received complete push request method=POST path=/28262026/11/11 23:27:33 INFO Received push request method=POST path=/28272026/11/11 23:27:33 INFO Completed upload id=6 objects=9 duration=344.167µs28282026/11/11 23:27:33 INFO Completed upload id=22 objects=2 duration=363.875µs28292026/11/11 23:27:33 INFO Received uploads request method=POST path=/28302026/11/11 23:27:33 INFO Completed push id=17 objects=6 duration=563.583µs28312026/11/11 23:27:33 INFO Aborted multipart uploads count=0 kept=028322026/11/11 23:27:33 INFO Aborted multipart uploads count=0 kept=028332026/11/11 23:27:33 INFO Received push request method=POST path=/28342026/11/11 23:27:33 INFO Received uploads request method=POST path=/28352026/11/11 23:27:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=028362026/11/11 23:27:33 INFO Received push request method=POST path=/28372026/11/11 23:27:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=028382026/11/11 23:27:33 INFO Aborted multipart uploads count=0 kept=028392026/11/11 23:27:33 INFO Aborted multipart uploads count=0 kept=028402026/11/11 23:27:33 INFO Vacuumed table table=pending_closures28412026/11/11 23:27:33 INFO Aborted multipart uploads count=0 kept=028422026/11/11 23:27:33 INFO Vacuumed table table=pending_closures28432026/11/11 23:27:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028442026/11/11 23:27:33 INFO Received complete upload request method=POST path=/28452026/11/11 23:27:33 WARN Force mode enabled - objects will be deleted immediately without grace period28462026/11/11 23:27:33 INFO Completed upload id=3 objects=5 duration=540.334µs28472026/11/11 23:27:33 INFO Vacuumed table table=pending_objects28482026/11/11 23:27:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028492026/11/11 23:27:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028502026/11/11 23:27:33 INFO Vacuumed table table=pending_objects28512026/11/11 23:27:33 INFO Aborted multipart uploads count=0 kept=028522026/11/11 23:27:33 INFO Vacuumed table table=multipart_uploads28532026/11/11 23:27:33 INFO Vacuumed table table=pending_closures28542026/11/11 23:27:33 INFO Vacuumed table table=multipart_uploads28552026/11/11 23:27:33 INFO Received complete push request method=POST path=/28562026/11/11 23:27:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=028572026/11/11 23:27:33 INFO Vacuumed table table=closures28582026/11/11 23:27:33 INFO Vacuumed table table=pending_objects28592026/11/11 23:27:33 INFO Vacuumed table table=pending_closures28602026/11/11 23:27:33 INFO Vacuumed table table=pending_closures28612026/11/11 23:27:33 INFO Completed push id=1 objects=2 duration=681.375µs28622026/11/11 23:27:33 INFO Vacuumed table table=multipart_uploads28632026/11/11 23:27:33 INFO Vacuumed table table=closures28642026/11/11 23:27:33 INFO Vacuumed table table=objects28652026/11/11 23:27:33 INFO Vacuumed table table=closures28662026/11/11 23:27:33 INFO Aborted multipart uploads count=0 kept=028672026/11/11 23:27:33 INFO Vacuumed table table=pending_objects28682026/11/11 23:27:33 INFO Vacuumed table table=pending_closures28692026/11/11 23:27:33 INFO Vacuumed table table=objects28702026/11/11 23:27:33 INFO Vacuumed table table=objects28712026/11/11 23:27:33 INFO Received uploads request method=POST path=/28722026/11/11 23:27:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028732026/11/11 23:27:33 INFO Vacuumed table table=multipart_uploads28742026/11/11 23:27:33 INFO Received push request method=POST path=/28752026/11/11 23:27:33 INFO Vacuumed table table=pending_objects28762026/11/11 23:27:33 INFO Vacuumed table table=closures28772026/11/11 23:27:33 INFO Vacuumed table table=multipart_uploads28782026/11/11 23:27:33 INFO Vacuumed table table=pending_objects28792026/11/11 23:27:33 INFO Received push request method=POST path=/28802026/11/11 23:27:33 INFO Vacuumed table table=closures28812026/11/11 23:27:33 INFO Vacuumed table table=multipart_uploads28822026/11/11 23:27:33 INFO Vacuumed table table=objects28832026/11/11 23:27:33 INFO Received complete upload request method=POST path=/28842026/11/11 23:27:33 INFO Vacuumed table table=pending_closures28852026/11/11 23:27:33 INFO Vacuumed table table=objects28862026/11/11 23:27:33 INFO Vacuumed table table=closures28872026/11/11 23:27:33 INFO Received push request method=POST path=/28882026/11/11 23:27:33 INFO Received push request method=POST path=/28892026/11/11 23:27:33 INFO Received push request method=POST path=/28902026/11/11 23:27:33 INFO Completed upload id=10 objects=2 duration=12.471791ms28912026/11/11 23:27:33 INFO Vacuumed table table=objects28922026/11/11 23:27:33 INFO Aborted multipart uploads count=0 kept=028932026/11/11 23:27:33 INFO Vacuumed table table=pending_objects28942026/11/11 23:27:33 INFO Received push request method=POST path=/28952026/11/11 23:27:33 INFO Received push request method=POST path=/28962026/11/11 23:27:33 INFO Vacuumed table table=multipart_uploads28972026/11/11 23:27:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028982026/11/11 23:27:33 INFO Received push request method=POST path=/28992026/11/11 23:27:33 INFO Aborted multipart uploads count=0 kept=029002026/11/11 23:27:33 INFO Vacuumed table table=pending_closures29012026/11/11 23:27:33 INFO Received push request method=POST path=/29022026/11/11 23:27:33 INFO Received uploads request method=POST path=/29032026/11/11 23:27:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=029042026/11/11 23:27:33 INFO Aborted multipart uploads count=0 kept=029052026/11/11 23:27:33 INFO Vacuumed table table=pending_closures29062026/11/11 23:27:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029072026/11/11 23:27:33 INFO Vacuumed table table=pending_objects29082026/11/11 23:27:33 INFO Vacuumed table table=pending_closures29092026/11/11 23:27:33 INFO Vacuumed table table=pending_objects29102026/11/11 23:27:33 INFO Vacuumed table table=closures29112026/11/11 23:27:33 INFO Vacuumed table table=multipart_uploads29122026/11/11 23:27:33 INFO Vacuumed table table=multipart_uploads29132026/11/11 23:27:33 INFO Vacuumed table table=closures29142026/11/11 23:27:33 INFO Vacuumed table table=objects29152026/11/11 23:27:33 INFO Vacuumed table table=pending_objects29162026/11/11 23:27:33 INFO Vacuumed table table=multipart_uploads29172026/11/11 23:27:33 INFO Vacuumed table table=closures29182026/11/11 23:27:33 INFO Received push request method=POST path=/29192026/11/11 23:27:33 INFO Vacuumed table table=closures29202026/11/11 23:27:33 INFO Vacuumed table table=objects29212026/11/11 23:27:33 INFO Received complete push request method=POST path=/29222026/11/11 23:27:33 INFO Vacuumed table table=objects29232026/11/11 23:27:33 INFO Vacuumed table table=objects29242026/11/11 23:27:33 INFO Received push request method=POST path=/29252026/11/11 23:27:33 INFO Completed push id=4 objects=6 duration=928.458µs29262026/11/11 23:27:33 INFO Received push request method=POST path=/29272026/11/11 23:27:33 INFO Received complete push request method=POST path=/29282026/11/11 23:27:33 INFO Received uploads request method=POST path=/29292026/11/11 23:27:33 INFO Completed push id=2 objects=10 duration=2.384875ms29302026/11/11 23:27:33 INFO Received uploads request method=POST path=/29312026/11/11 23:27:33 INFO Received push request method=POST path=/29322026/11/11 23:27:33 INFO Aborted multipart uploads count=0 kept=029332026/11/11 23:27:33 INFO Received push request method=POST path=/29342026/11/11 23:27:34 INFO Received uploads request method=POST path=/29352026/11/11 23:27:34 INFO Received push request method=POST path=/29362026/11/11 23:27:34 INFO Received push request method=POST path=/29372026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029382026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=029392026/11/11 23:27:34 INFO Vacuumed table table=pending_closures29402026/11/11 23:27:34 INFO Received uploads request method=POST path=/29412026/11/11 23:27:34 INFO Vacuumed table table=pending_objects29422026/11/11 23:27:34 WARN Force mode enabled - objects will be deleted immediately without grace period29432026/11/11 23:27:34 INFO Received complete push request method=POST path=/29442026/11/11 23:27:34 INFO Received complete push request method=POST path=/29452026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads29462026/11/11 23:27:34 INFO Vacuumed table table=closures29472026/11/11 23:27:34 INFO Completed push id=26 objects=14 duration=5.928292ms29482026/11/11 23:27:34 INFO Completed push id=8 objects=7 duration=5.932333ms29492026/11/11 23:27:34 INFO Vacuumed table table=objects29502026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029512026/11/11 23:27:34 INFO Received push request method=POST path=/29522026/11/11 23:27:34 INFO Received push request method=POST path=/29532026/11/11 23:27:34 INFO Vacuumed table table=pending_closures29542026/11/11 23:27:34 INFO Received uploads request method=POST path=/29552026/11/11 23:27:34 INFO Received uploads request method=POST path=/29562026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=029572026/11/11 23:27:34 INFO Received complete push request method=POST path=/29582026/11/11 23:27:34 INFO Vacuumed table table=pending_objects29592026/11/11 23:27:34 WARN Force mode enabled - objects will be deleted immediately without grace period29602026/11/11 23:27:34 INFO Received uploads request method=POST path=/29612026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads29622026/11/11 23:27:34 INFO Completed push id=19 objects=8 duration=6.126ms29632026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029642026/11/11 23:27:34 INFO Vacuumed table table=closures29652026/11/11 23:27:34 INFO Received push request method=POST path=/29662026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=029672026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=029682026/11/11 23:27:34 INFO Vacuumed table table=pending_closures29692026/11/11 23:27:34 WARN Force mode enabled - objects will be deleted immediately without grace period29702026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029712026/11/11 23:27:34 INFO Received uploads request method=POST path=/29722026/11/11 23:27:34 INFO Vacuumed table table=objects29732026/11/11 23:27:34 INFO Vacuumed table table=pending_closures29742026/11/11 23:27:34 INFO Vacuumed table table=pending_objects29752026/11/11 23:27:34 INFO Vacuumed table table=pending_objects29762026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads29772026/11/11 23:27:34 INFO Received uploads request method=POST path=/29782026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=029792026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads29802026/11/11 23:27:34 INFO Vacuumed table table=closures29812026/11/11 23:27:34 INFO Vacuumed table table=closures29822026/11/11 23:27:34 INFO Vacuumed table table=objects29832026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029842026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=029852026/11/11 23:27:34 INFO Vacuumed table table=objects29862026/11/11 23:27:34 INFO Vacuumed table table=pending_closures29872026/11/11 23:27:34 WARN Force mode enabled - objects will be deleted immediately without grace period29882026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029892026/11/11 23:27:34 INFO Vacuumed table table=pending_objects29902026/11/11 23:27:34 INFO Vacuumed table table=pending_closures29912026/11/11 23:27:34 INFO Received push request method=POST path=/29922026/11/11 23:27:34 INFO Received uploads request method=POST path=/29932026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads29942026/11/11 23:27:34 INFO Vacuumed table table=pending_objects29952026/11/11 23:27:34 INFO Received push request method=POST path=/29962026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=029972026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads29982026/11/11 23:27:34 INFO Received push request method=POST path=/29992026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030002026/11/11 23:27:34 INFO Vacuumed table table=closures30012026/11/11 23:27:34 INFO Vacuumed table table=closures30022026/11/11 23:27:34 INFO Vacuumed table table=pending_closures30032026/11/11 23:27:34 INFO Vacuumed table table=objects30042026/11/11 23:27:34 INFO Vacuumed table table=objects30052026/11/11 23:27:34 INFO Received push request method=POST path=/30062026/11/11 23:27:34 INFO Vacuumed table table=pending_objects30072026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=030082026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=030092026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads30102026/11/11 23:27:34 INFO Received complete upload request method=POST path=/30112026/11/11 23:27:34 WARN Force mode enabled - objects will be deleted immediately without grace period30122026/11/11 23:27:34 WARN Force mode enabled - objects will be deleted immediately without grace period30132026/11/11 23:27:34 INFO Vacuumed table table=closures30142026/11/11 23:27:34 INFO Completed upload id=9 objects=16 duration=481.583µs30152026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030162026/11/11 23:27:34 INFO Received complete push request method=POST path=/30172026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030182026/11/11 23:27:34 INFO Vacuumed table table=objects30192026/11/11 23:27:34 INFO Vacuumed table table=pending_closures30202026/11/11 23:27:34 INFO Vacuumed table table=pending_closures30212026/11/11 23:27:34 INFO Received push request method=POST path=/30222026/11/11 23:27:34 INFO Completed push id=2 objects=25 duration=1.362459ms30232026/11/11 23:27:34 INFO Vacuumed table table=pending_objects30242026/11/11 23:27:34 INFO Vacuumed table table=pending_objects30252026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads30262026/11/11 23:27:34 INFO Received push request method=POST path=/30272026/11/11 23:27:34 INFO Received push request method=POST path=/30282026/11/11 23:27:34 INFO Vacuumed table table=closures30292026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads30302026/11/11 23:27:34 INFO Received complete push request method=POST path=/30312026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=030322026/11/11 23:27:34 INFO Vacuumed table table=objects30332026/11/11 23:27:34 INFO Vacuumed table table=closures30342026/11/11 23:27:34 INFO Completed push id=10 objects=3 duration=387.708µs30352026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030362026/11/11 23:27:34 INFO Received complete push request method=POST path=/30372026/11/11 23:27:34 INFO Vacuumed table table=objects30382026/11/11 23:27:34 INFO Completed push id=27 objects=25 duration=1.186208ms30392026/11/11 23:27:34 INFO Received uploads request method=POST path=/30402026/11/11 23:27:34 INFO Received push request method=POST path=/30412026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=030422026/11/11 23:27:34 INFO Received push request method=POST path=/30432026/11/11 23:27:34 INFO Received uploads request method=POST path=/30442026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030452026/11/11 23:27:34 INFO Vacuumed table table=pending_closures30462026/11/11 23:27:34 INFO Received complete upload request method=POST path=/30472026/11/11 23:27:34 INFO Vacuumed table table=pending_objects30482026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads30492026/11/11 23:27:34 INFO Completed upload id=33 objects=5 duration=7.153125ms30502026/11/11 23:27:34 INFO Received uploads request method=POST path=/30512026/11/11 23:27:34 INFO Vacuumed table table=closures30522026/11/11 23:27:34 INFO Vacuumed table table=pending_closures30532026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=030542026/11/11 23:27:34 INFO Received complete push request method=POST path=/30552026/11/11 23:27:34 INFO Received uploads request method=POST path=/30562026/11/11 23:27:34 WARN Force mode enabled - objects will be deleted immediately without grace period30572026/11/11 23:27:34 INFO Completed push id=11 objects=8 duration=852.875µs30582026/11/11 23:27:34 INFO Vacuumed table table=objects30592026/11/11 23:27:34 INFO Received push request method=POST path=/30602026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030612026/11/11 23:27:34 INFO Received complete upload request method=POST path=/30622026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=030632026/11/11 23:27:34 INFO Vacuumed table table=pending_objects30642026/11/11 23:27:34 INFO Received uploads request method=POST path=/30652026/11/11 23:27:34 INFO Vacuumed table table=pending_closures30662026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads30672026/11/11 23:27:34 INFO Vacuumed table table=pending_objects30682026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030692026/11/11 23:27:34 INFO Received complete push request method=POST path=/30702026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads30712026/11/11 23:27:34 INFO Vacuumed table table=pending_closures30722026/11/11 23:27:34 INFO Completed upload id=3 objects=6 duration=27.40225ms30732026/11/11 23:27:34 INFO Vacuumed table table=closures30742026/11/11 23:27:34 INFO Completed push id=24 objects=5 duration=1.051125ms30752026/11/11 23:27:34 INFO Vacuumed table table=objects30762026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=030772026/11/11 23:27:34 INFO Vacuumed table table=pending_objects30782026/11/11 23:27:34 INFO Received uploads request method=POST path=/30792026/11/11 23:27:34 INFO Vacuumed table table=closures30802026/11/11 23:27:34 INFO Received uploads request method=POST path=/30812026/11/11 23:27:34 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo error="Access Denied."30822026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads30832026/11/11 23:27:34 INFO Received complete upload request method=POST path=/30842026/11/11 23:27:34 INFO Received uploads request method=POST path=/30852026/11/11 23:27:34 INFO Received complete push request method=POST path=/30862026/11/11 23:27:34 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."30872026/11/11 23:27:34 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."30882026/11/11 23:27:34 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."30892026/11/11 23:27:34 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."30902026/11/11 23:27:34 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."30912026/11/11 23:27:34 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."30922026/11/11 23:27:34 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."30932026/11/11 23:27:34 INFO Vacuumed table table=closures30942026/11/11 23:27:34 INFO Completed upload id=17 objects=25 duration=1.033291ms30952026/11/11 23:27:34 INFO Received push request method=POST path=/30962026/11/11 23:27:34 ERROR failed to remove object object=nar/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.nar.zst error="Access Denied."30972026/11/11 23:27:34 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."30982026/11/11 23:27:34 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."30992026/11/11 23:27:34 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."31002026/11/11 23:27:34 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."31012026/11/11 23:27:34 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."31022026/11/11 23:27:34 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."31032026/11/11 23:27:34 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."31042026/11/11 23:27:34 INFO Vacuumed table table=objects31052026/11/11 23:27:34 INFO Vacuumed table table=objects31062026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=031072026/11/11 23:27:34 INFO Received push request method=POST path=/31082026/11/11 23:27:34 INFO Received uploads request method=POST path=/31092026/11/11 23:27:34 INFO Completed push id=1 objects=5 duration=37.624208ms31102026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=031112026/11/11 23:27:34 INFO Vacuumed table table=pending_closures31122026/11/11 23:27:34 INFO Vacuumed table table=pending_objects31132026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads31142026/11/11 23:27:34 INFO Received push request method=POST path=/31152026/11/11 23:27:34 INFO Received push request method=POST path=/31162026/11/11 23:27:34 INFO Received complete upload request method=POST path=/31172026/11/11 23:27:34 INFO Received push request method=POST path=/31182026/11/11 23:27:34 INFO Completed upload id=21 objects=2 duration=216.792µs31192026/11/11 23:27:34 INFO Received uploads request method=POST path=/31202026/11/11 23:27:34 INFO Vacuumed table table=closures31212026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=031222026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=031232026/11/11 23:27:34 WARN Force mode enabled - objects will be deleted immediately without grace period31242026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031252026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031262026/11/11 23:27:34 INFO Vacuumed table table=pending_closures31272026/11/11 23:27:34 INFO Vacuumed table table=pending_closures31282026/11/11 23:27:34 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."31292026/11/11 23:27:34 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."31302026/11/11 23:27:34 INFO Vacuumed table table=pending_objects31312026/11/11 23:27:34 INFO Received complete push request method=POST path=/31322026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads31332026/11/11 23:27:34 INFO Received push request method=POST path=/31342026/11/11 23:27:34 INFO Vacuumed table table=objects31352026/11/11 23:27:34 INFO Vacuumed table table=pending_objects31362026/11/11 23:27:34 INFO Completed push id=6 objects=26 duration=1.532833ms31372026/11/11 23:27:34 INFO Received push request method=POST path=/31382026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads31392026/11/11 23:27:34 INFO Vacuumed table table=closures31402026/11/11 23:27:34 INFO Received complete upload request method=POST path=/31412026/11/11 23:27:34 INFO Vacuumed table table=closures31422026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=031432026/11/11 23:27:34 INFO Vacuumed table table=objects31442026/11/11 23:27:34 INFO Vacuumed table table=objects31452026/11/11 23:27:34 INFO Completed upload id=17 objects=6 duration=7.319ms31462026/11/11 23:27:34 WARN Force mode enabled - objects will be deleted immediately without grace period31472026/11/11 23:27:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031482026/11/11 23:27:34 INFO Vacuumed table table=pending_closures31492026/11/11 23:27:34 INFO Vacuumed table table=pending_objects31502026/11/11 23:27:34 INFO Received push request method=POST path=/31512026/11/11 23:27:34 INFO Vacuumed table table=multipart_uploads31522026/11/11 23:27:34 INFO Received complete push request method=POST path=/31532026/11/11 23:27:34 INFO Vacuumed table table=closures31542026/11/11 23:27:34 INFO Completed push id=4 objects=4 duration=5.341209ms31552026/11/11 23:27:34 INFO Vacuumed table table=objects31562026/11/11 23:27:34 INFO Received push request method=POST path=/31572026/11/11 23:27:34 INFO Received uploads request method=POST path=/31582026/11/11 23:27:34 INFO Received uploads request method=POST path=/31592026/11/11 23:27:34 INFO Received push request method=POST path=/31602026/11/11 23:27:34 INFO Aborted multipart uploads count=0 kept=031612026/11/11 23:27:34 INFO Received complete push request method=POST path=/31622026/11/11 23:27:34 WARN Force mode enabled - objects will be deleted immediately without grace period31632026/11/11 23:27:34 INFO Received uploads request method=POST path=/31642026/11/11 23:27:34 INFO Completed push id=21 objects=26 duration=360.25µs31652026/11/11 23:27:34 INFO Received complete upload request method=POST path=/31662026/11/11 23:27:34 INFO Completed upload id=15 objects=9 duration=371.708µs31672026/11/11 23:27:34 INFO Received complete upload request method=POST path=/31682026/11/11 23:27:35 INFO Completed upload id=3 objects=8 duration=794.875µs31692026/11/11 23:27:35 INFO Received push request method=POST path=/31702026/11/11 23:27:35 INFO Received complete push request method=POST path=/31712026/11/11 23:27:35 INFO Received uploads request method=POST path=/31722026/11/11 23:27:35 INFO Completed push id=13 objects=6 duration=1.097334ms31732026/11/11 23:27:35 INFO Received push request method=POST path=/31742026/11/11 23:27:35 INFO Received push request method=POST path=/31752026/11/11 23:27:35 INFO Received complete push request method=POST path=/31762026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=031772026/11/11 23:27:35 INFO Completed push id=21 objects=17 duration=592.167µs31782026/11/11 23:27:35 INFO Received complete push request method=POST path=/31792026/11/11 23:27:35 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."31802026/11/11 23:27:35 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."31812026/11/11 23:27:35 INFO Received complete upload request method=POST path=/31822026/11/11 23:27:35 INFO Completed push id=19 objects=7 duration=510.166µs31832026/11/11 23:27:35 INFO Received complete push request method=POST path=/31842026/11/11 23:27:35 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."31852026/11/11 23:27:35 INFO Received complete push request method=POST path=/31862026/11/11 23:27:35 INFO Completed upload id=28 objects=17 duration=247.333µs31872026/11/11 23:27:35 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."31882026/11/11 23:27:35 INFO Completed push id=2 objects=4 duration=395.416µs31892026/11/11 23:27:35 INFO Received push request method=POST path=/31902026/11/11 23:27:35 INFO Completed push id=6 objects=6 duration=518.333µs31912026/11/11 23:27:35 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."31922026/11/11 23:27:35 INFO Received complete push request method=POST path=/31932026/11/11 23:27:35 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."31942026/11/11 23:27:35 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."31952026/11/11 23:27:35 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."31962026/11/11 23:27:35 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."31972026/11/11 23:27:35 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."31982026/11/11 23:27:35 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."31992026/11/11 23:27:35 INFO Completed push id=29 objects=2 duration=829.333µs32002026/11/11 23:27:35 INFO Received push request method=POST path=/32012026/11/11 23:27:35 INFO Received uploads request method=POST path=/32022026/11/11 23:27:35 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."32032026/11/11 23:27:35 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."32042026/11/11 23:27:35 ERROR failed to remove object object=realisations/sha256:aaaaaaaa!out.doi error="Access Denied."32052026/11/11 23:27:35 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."32062026/11/11 23:27:35 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."32072026/11/11 23:27:35 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."32082026/11/11 23:27:35 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."32092026/11/11 23:27:35 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."32102026/11/11 23:27:35 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."32112026/11/11 23:27:35 INFO Received uploads request method=POST path=/32122026/11/11 23:27:35 INFO Received complete push request method=POST path=/32132026/11/11 23:27:35 INFO Received push request method=POST path=/32142026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=032152026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=032162026/11/11 23:27:35 INFO Completed push id=31 objects=6 duration=343.209µs32172026/11/11 23:27:35 INFO Received complete upload request method=POST path=/32182026/11/11 23:27:35 INFO Received uploads request method=POST path=/32192026/11/11 23:27:35 WARN Force mode enabled - objects will be deleted immediately without grace period32202026/11/11 23:27:35 WARN Force mode enabled - objects will be deleted immediately without grace period32212026/11/11 23:27:35 INFO Completed upload id=21 objects=25 duration=665.083µs32222026/11/11 23:27:35 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032232026/11/11 23:27:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032242026/11/11 23:27:35 INFO Received complete push request method=POST path=/32252026/11/11 23:27:35 INFO Received uploads request method=POST path=/32262026/11/11 23:27:35 INFO Vacuumed table table=pending_closures32272026/11/11 23:27:35 INFO Completed push id=32 objects=9 duration=523.25µs32282026/11/11 23:27:35 INFO Vacuumed table table=pending_objects32292026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=032302026/11/11 23:27:35 INFO Vacuumed table table=multipart_uploads32312026/11/11 23:27:35 WARN Force mode enabled - objects will be deleted immediately without grace period32322026/11/11 23:27:35 INFO Vacuumed table table=closures32332026/11/11 23:27:35 INFO Vacuumed table table=objects32342026/11/11 23:27:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=032352026/11/11 23:27:35 INFO Vacuumed table table=pending_closures32362026/11/11 23:27:35 INFO Received uploads request method=POST path=/32372026/11/11 23:27:35 INFO Vacuumed table table=pending_closures32382026/11/11 23:27:35 INFO Vacuumed table table=pending_objects32392026/11/11 23:27:35 INFO Vacuumed table table=multipart_uploads32402026/11/11 23:27:35 INFO Vacuumed table table=pending_objects32412026/11/11 23:27:35 INFO Vacuumed table table=closures32422026/11/11 23:27:35 INFO Vacuumed table table=multipart_uploads32432026/11/11 23:27:35 INFO Vacuumed table table=objects32442026/11/11 23:27:35 INFO Received push request method=POST path=/32452026/11/11 23:27:35 INFO Received push request method=POST path=/32462026/11/11 23:27:35 INFO Received complete upload request method=POST path=/32472026/11/11 23:27:35 INFO Received push request method=POST path=/32482026/11/11 23:27:35 INFO Vacuumed table table=closures32492026/11/11 23:27:35 INFO Received complete push request method=POST path=/32502026/11/11 23:27:35 INFO Completed upload id=33 objects=9 duration=475.917µs32512026/11/11 23:27:35 INFO Received push request method=POST path=/32522026/11/11 23:27:35 INFO Completed push id=7 objects=24 duration=1.122625ms32532026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=032542026/11/11 23:27:35 INFO Received uploads request method=POST path=/32552026/11/11 23:27:35 INFO Received uploads request method=POST path=/32562026/11/11 23:27:35 INFO Received uploads request method=POST path=/32572026/11/11 23:27:35 WARN Force mode enabled - objects will be deleted immediately without grace period32582026/11/11 23:27:35 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032592026/11/11 23:27:35 INFO Received push request method=POST path=/32602026/11/11 23:27:35 INFO Vacuumed table table=objects32612026/11/11 23:27:35 INFO Received push request method=POST path=/32622026/11/11 23:27:35 INFO Received uploads request method=POST path=/32632026/11/11 23:27:35 INFO Received push request method=POST path=/32642026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=032652026/11/11 23:27:35 INFO Received uploads request method=POST path=/32662026/11/11 23:27:35 INFO Received uploads request method=POST path=/32672026/11/11 23:27:35 INFO Vacuumed table table=pending_closures32682026/11/11 23:27:35 INFO Received push request method=POST path=/32692026/11/11 23:27:35 INFO Received push request method=POST path=/32702026/11/11 23:27:35 WARN Force mode enabled - objects will be deleted immediately without grace period32712026/11/11 23:27:35 INFO Received uploads request method=POST path=/32722026/11/11 23:27:35 INFO Received uploads request method=POST path=/32732026/11/11 23:27:35 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."32742026/11/11 23:27:35 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=5 objects-failed-to-delete=032752026/11/11 23:27:35 INFO Received complete push request method=POST path=/32762026/11/11 23:27:35 INFO Received push request method=POST path=/32772026/11/11 23:27:35 INFO Vacuumed table table=pending_objects32782026/11/11 23:27:35 INFO Completed push id=13 objects=2 duration=377.917µs32792026/11/11 23:27:35 INFO Vacuumed table table=multipart_uploads32802026/11/11 23:27:35 INFO Received complete upload request method=POST path=/32812026/11/11 23:27:35 INFO Received complete upload request method=POST path=/32822026/11/11 23:27:35 INFO Vacuumed table table=closures32832026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=032842026/11/11 23:27:35 INFO Completed upload id=27 objects=13 duration=453.208µs32852026/11/11 23:27:35 INFO Vacuumed table table=pending_closures32862026/11/11 23:27:35 INFO Vacuumed table table=objects32872026/11/11 23:27:35 INFO Completed upload id=12 objects=26 duration=7.352542ms32882026/11/11 23:27:35 INFO Received complete upload request method=POST path=/32892026/11/11 23:27:35 INFO Received complete push request method=POST path=/32902026/11/11 23:27:35 INFO Completed upload id=14 objects=16 duration=831.167µs32912026/11/11 23:27:35 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=032922026/11/11 23:27:35 INFO Completed push id=7 objects=27 duration=1.258833ms32932026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=032942026/11/11 23:27:35 INFO Received uploads request method=POST path=/32952026/11/11 23:27:35 INFO Vacuumed table table=pending_closures32962026/11/11 23:27:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032972026/11/11 23:27:35 INFO Received complete push request method=POST path=/32982026/11/11 23:27:35 INFO Vacuumed table table=pending_objects32992026/11/11 23:27:35 INFO Vacuumed table table=pending_objects33002026/11/11 23:27:35 INFO Vacuumed table table=pending_closures33012026/11/11 23:27:35 INFO Vacuumed table table=multipart_uploads33022026/11/11 23:27:35 INFO Vacuumed table table=multipart_uploads33032026/11/11 23:27:35 INFO Vacuumed table table=pending_objects33042026/11/11 23:27:35 INFO Vacuumed table table=closures33052026/11/11 23:27:35 INFO Completed push id=16 objects=8 duration=4.651958ms33062026/11/11 23:27:35 INFO Vacuumed table table=closures33072026/11/11 23:27:35 INFO Vacuumed table table=multipart_uploads33082026/11/11 23:27:35 INFO Vacuumed table table=objects33092026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=033102026/11/11 23:27:35 INFO Vacuumed table table=closures33112026/11/11 23:27:35 INFO Received push request method=POST path=/33122026/11/11 23:27:35 WARN Force mode enabled - objects will be deleted immediately without grace period33132026/11/11 23:27:35 INFO Vacuumed table table=objects33142026/11/11 23:27:35 INFO Received complete push request method=POST path=/33152026/11/11 23:27:35 INFO Received push request method=POST path=/33162026/11/11 23:27:35 INFO Vacuumed table table=objects33172026/11/11 23:27:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033182026/11/11 23:27:35 INFO Received uploads request method=POST path=/33192026/11/11 23:27:35 INFO Vacuumed table table=pending_closures33202026/11/11 23:27:35 INFO Received uploads request method=POST path=/33212026/11/11 23:27:35 INFO Received push request method=POST path=/33222026/11/11 23:27:35 INFO Vacuumed table table=pending_objects33232026/11/11 23:27:35 INFO Received uploads request method=POST path=/33242026/11/11 23:27:35 INFO Received push request method=POST path=/33252026/11/11 23:27:35 INFO Vacuumed table table=multipart_uploads33262026/11/11 23:27:35 INFO Completed push id=3 objects=8 duration=36.10275ms33272026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=033282026/11/11 23:27:35 INFO Received uploads request method=POST path=/33292026/11/11 23:27:35 INFO Vacuumed table table=closures33302026/11/11 23:27:35 INFO Received complete upload request method=POST path=/33312026/11/11 23:27:35 INFO Received complete upload request method=POST path=/33322026/11/11 23:27:35 INFO Received push request method=POST path=/33332026/11/11 23:27:35 INFO Received complete push request method=POST path=/33342026/11/11 23:27:35 INFO Completed upload id=12 objects=6 duration=481.291µs33352026/11/11 23:27:35 INFO Vacuumed table table=objects33362026/11/11 23:27:35 WARN Force mode enabled - objects will be deleted immediately without grace period33372026/11/11 23:27:35 INFO Received complete push request method=POST path=/33382026/11/11 23:27:35 INFO Received uploads request method=POST path=/33392026/11/11 23:27:35 INFO Completed push id=17 objects=6 duration=597.459µs33402026/11/11 23:27:35 INFO Completed push id=4 objects=7 duration=657.584µs33412026/11/11 23:27:35 INFO Received complete push request method=POST path=/33422026/11/11 23:27:35 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033432026/11/11 23:27:35 INFO Completed upload id=2 objects=26 duration=18.508084ms33442026/11/11 23:27:35 INFO Vacuumed table table=pending_closures33452026/11/11 23:27:35 INFO Completed push id=2 objects=7 duration=2.207917ms33462026/11/11 23:27:35 INFO Received complete push request method=POST path=/33472026/11/11 23:27:35 INFO Completed push id=17 objects=3 duration=690.791µs33482026/11/11 23:27:35 INFO Received complete upload request method=POST path=/33492026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=033502026/11/11 23:27:35 INFO Received uploads request method=POST path=/33512026/11/11 23:27:35 INFO Vacuumed table table=pending_objects33522026/11/11 23:27:35 INFO Received push request method=POST path=/33532026/11/11 23:27:35 INFO Vacuumed table table=multipart_uploads33542026/11/11 23:27:35 INFO Completed upload id=29 objects=4 duration=728.583µs33552026/11/11 23:27:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033562026/11/11 23:27:35 INFO Vacuumed table table=closures33572026/11/11 23:27:35 INFO Received uploads request method=POST path=/33582026/11/11 23:27:35 INFO Received push request method=POST path=/33592026/11/11 23:27:35 INFO Received uploads request method=POST path=/33602026/11/11 23:27:35 INFO Vacuumed table table=objects33612026/11/11 23:27:35 INFO Received push request method=POST path=/33622026/11/11 23:27:35 INFO Received uploads request method=POST path=/33632026/11/11 23:27:35 INFO Vacuumed table table=pending_closures33642026/11/11 23:27:35 INFO Received push request method=POST path=/33652026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=033662026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=033672026/11/11 23:27:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033682026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=033692026/11/11 23:27:35 INFO Vacuumed table table=pending_closures33702026/11/11 23:27:35 WARN Force mode enabled - objects will be deleted immediately without grace period33712026/11/11 23:27:35 INFO Received complete push request method=POST path=/33722026/11/11 23:27:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=033732026/11/11 23:27:35 INFO Received complete upload request method=POST path=/33742026/11/11 23:27:35 INFO Vacuumed table table=pending_objects33752026/11/11 23:27:35 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."33762026/11/11 23:27:35 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."33772026/11/11 23:27:35 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."33782026/11/11 23:27:35 INFO Vacuumed table table=pending_objects33792026/11/11 23:27:35 INFO Received complete push request method=POST path=/33802026/11/11 23:27:35 INFO Completed push id=23 objects=5 duration=321.625µs33812026/11/11 23:27:35 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."33822026/11/11 23:27:35 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."33832026/11/11 23:27:35 INFO Vacuumed table table=multipart_uploads33842026/11/11 23:27:35 INFO Received complete upload request method=POST path=/33852026/11/11 23:27:35 INFO Vacuumed table table=multipart_uploads33862026/11/11 23:27:35 INFO Completed upload id=3 objects=9 duration=583.5µs33872026/11/11 23:27:35 INFO Vacuumed table table=pending_closures33882026/11/11 23:27:35 INFO Completed push id=11 objects=6 duration=944.459µs33892026/11/11 23:27:35 INFO Received complete upload request method=POST path=/33902026/11/11 23:27:35 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."33912026/11/11 23:27:35 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."33922026/11/11 23:27:35 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."33932026/11/11 23:27:35 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."33942026/11/11 23:27:35 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."33952026/11/11 23:27:35 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."33962026/11/11 23:27:35 INFO Vacuumed table table=closures33972026/11/11 23:27:35 INFO Completed upload id=24 objects=3 duration=333.709µs33982026/11/11 23:27:35 INFO Completed upload id=6 objects=7 duration=978.75µs33992026/11/11 23:27:35 INFO Received push request method=POST path=/34002026/11/11 23:27:35 INFO Vacuumed table table=pending_objects34012026/11/11 23:27:35 INFO Vacuumed table table=objects34022026/11/11 23:27:35 INFO Vacuumed table table=multipart_uploads34032026/11/11 23:27:35 INFO Vacuumed table table=closures34042026/11/11 23:27:35 INFO Received complete push request method=POST path=/34052026/11/11 23:27:35 INFO Received complete push request method=POST path=/34062026/11/11 23:27:35 INFO Vacuumed table table=objects34072026/11/11 23:27:35 INFO Received push request method=POST path=/34082026/11/11 23:27:35 INFO Received push request method=POST path=/34092026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=034102026/11/11 23:27:35 INFO Completed push id=8 objects=2 duration=589.417µs34112026/11/11 23:27:35 INFO Completed push id=25 objects=4 duration=202.666µs34122026/11/11 23:27:35 INFO Received complete push request method=POST path=/34132026/11/11 23:27:35 WARN Force mode enabled - objects will be deleted immediately without grace period34142026/11/11 23:27:35 INFO Received uploads request method=POST path=/34152026/11/11 23:27:35 INFO Received uploads request method=POST path=/34162026/11/11 23:27:35 INFO Completed push id=17 objects=11 duration=545.5µs34172026/11/11 23:27:35 INFO Received complete push request method=POST path=/34182026/11/11 23:27:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034192026/11/11 23:27:35 INFO Vacuumed table table=closures34202026/11/11 23:27:35 INFO Aborted multipart uploads count=0 kept=034212026/11/11 23:27:35 INFO Received uploads request method=POST path=/34222026/11/11 23:27:35 INFO Received push request method=POST path=/34232026/11/11 23:27:35 INFO Completed push id=11 objects=6 duration=423.375µs34242026/11/11 23:27:35 INFO Vacuumed table table=objects34252026/11/11 23:27:35 INFO Vacuumed table table=pending_closures34262026/11/11 23:27:35 WARN Force mode enabled - objects will be deleted immediately without grace period34272026/11/11 23:27:35 INFO Received complete push request method=POST path=/34282026/11/11 23:27:35 INFO Received complete push request method=POST path=/34292026/11/11 23:27:35 INFO Received push request method=POST path=/34302026/11/11 23:27:35 INFO Vacuumed table table=pending_objects34312026/11/11 23:27:35 INFO Received push request method=POST path=/34322026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034332026/11/11 23:27:36 INFO Completed push id=18 objects=6 duration=301.958µs34342026/11/11 23:27:36 INFO Completed push id=5 objects=6 duration=407.208µs34352026/11/11 23:27:36 INFO Received push request method=POST path=/34362026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads34372026/11/11 23:27:36 INFO Vacuumed table table=closures34382026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=034392026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=034402026/11/11 23:27:36 INFO Vacuumed table table=objects34412026/11/11 23:27:36 INFO Received push request method=POST path=/34422026/11/11 23:27:36 INFO Vacuumed table table=pending_closures34432026/11/11 23:27:36 INFO Received complete upload request method=POST path=/34442026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034452026/11/11 23:27:36 WARN Force mode enabled - objects will be deleted immediately without grace period34462026/11/11 23:27:36 INFO Completed upload id=26 objects=6 duration=395.333µs34472026/11/11 23:27:36 INFO Vacuumed table table=pending_closures34482026/11/11 23:27:36 INFO Received push request method=POST path=/34492026/11/11 23:27:36 INFO Received uploads request method=POST path=/34502026/11/11 23:27:36 INFO Vacuumed table table=pending_objects34512026/11/11 23:27:36 INFO Received complete upload request method=POST path=/34522026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads34532026/11/11 23:27:36 INFO Received push request method=POST path=/34542026/11/11 23:27:36 INFO Received complete push request method=POST path=/34552026/11/11 23:27:36 INFO Vacuumed table table=pending_objects34562026/11/11 23:27:36 INFO Received complete push request method=POST path=/34572026/11/11 23:27:36 INFO Completed upload id=29 objects=5 duration=222.125µs34582026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads34592026/11/11 23:27:36 INFO Completed push id=20 objects=13 duration=385.375µs34602026/11/11 23:27:36 INFO Vacuumed table table=closures34612026/11/11 23:27:36 INFO Received push request method=POST path=/34622026/11/11 23:27:36 INFO Completed push id=4 objects=7 duration=393.833µs34632026/11/11 23:27:36 INFO Vacuumed table table=closures34642026/11/11 23:27:36 INFO Vacuumed table table=objects34652026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=034662026/11/11 23:27:36 INFO Received complete push request method=POST path=/34672026/11/11 23:27:36 INFO Vacuumed table table=objects34682026/11/11 23:27:36 INFO Received uploads request method=POST path=/34692026/11/11 23:27:36 INFO Received uploads request method=POST path=/34702026/11/11 23:27:36 INFO Received push request method=POST path=/34712026/11/11 23:27:36 INFO Received complete upload request method=POST path=/34722026/11/11 23:27:36 WARN Force mode enabled - objects will be deleted immediately without grace period34732026/11/11 23:27:36 INFO Completed push id=22 objects=4 duration=389.5µs34742026/11/11 23:27:36 INFO Received push request method=POST path=/34752026/11/11 23:27:36 INFO Received uploads request method=POST path=/34762026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034772026/11/11 23:27:36 INFO Completed upload id=20 objects=3 duration=6.745375ms34782026/11/11 23:27:36 INFO Received complete push request method=POST path=/34792026/11/11 23:27:36 INFO Vacuumed table table=pending_closures34802026/11/11 23:27:36 INFO Received uploads request method=POST path=/34812026/11/11 23:27:36 INFO Received uploads request method=POST path=/34822026/11/11 23:27:36 INFO Vacuumed table table=pending_objects34832026/11/11 23:27:36 INFO Received push request method=POST path=/34842026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads34852026/11/11 23:27:36 INFO Received uploads request method=POST path=/34862026/11/11 23:27:36 INFO Completed push id=1 objects=5 duration=17.156417ms34872026/11/11 23:27:36 INFO Received complete upload request method=POST path=/34882026/11/11 23:27:36 INFO Vacuumed table table=closures34892026/11/11 23:27:36 INFO Completed upload id=24 objects=7 duration=266.291µs34902026/11/11 23:27:36 INFO Received push request method=POST path=/34912026/11/11 23:27:36 INFO Vacuumed table table=objects34922026/11/11 23:27:36 INFO Received uploads request method=POST path=/34932026/11/11 23:27:36 INFO Received push request method=POST path=/34942026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=034952026/11/11 23:27:36 INFO Received push request method=POST path=/34962026/11/11 23:27:36 INFO Received push request method=POST path=/34972026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034982026/11/11 23:27:36 INFO Received push request method=POST path=/34992026/11/11 23:27:36 INFO Received uploads request method=POST path=/35002026/11/11 23:27:36 INFO Vacuumed table table=pending_closures35012026/11/11 23:27:36 INFO Received uploads request method=POST path=/35022026/11/11 23:27:36 INFO Received complete upload request method=POST path=/35032026/11/11 23:27:36 INFO Vacuumed table table=pending_objects35042026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads35052026/11/11 23:27:36 INFO Completed upload id=31 objects=3 duration=11.400667ms35062026/11/11 23:27:36 INFO Vacuumed table table=closures35072026/11/11 23:27:36 INFO Received uploads request method=POST path=/35082026/11/11 23:27:36 INFO Vacuumed table table=objects35092026/11/11 23:27:36 INFO Received complete upload request method=POST path=/35102026/11/11 23:27:36 INFO Completed upload id=22 objects=2 duration=317.042µs35112026/11/11 23:27:36 INFO Received complete upload request method=POST path=/35122026/11/11 23:27:36 INFO Received uploads request method=POST path=/35132026/11/11 23:27:36 INFO Received complete upload request method=POST path=/35142026/11/11 23:27:36 INFO Received push request method=POST path=/35152026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=035162026/11/11 23:27:36 INFO Completed upload id=23 objects=7 duration=221.084µs35172026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=035182026/11/11 23:27:36 INFO Completed upload id=19 objects=25 duration=730.917µs35192026/11/11 23:27:36 WARN Force mode enabled - objects will be deleted immediately without grace period35202026/11/11 23:27:36 INFO Received push request method=POST path=/35212026/11/11 23:27:36 INFO Received complete push request method=POST path=/35222026/11/11 23:27:36 WARN Force mode enabled - objects will be deleted immediately without grace period35232026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=035242026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035252026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=035262026/11/11 23:27:36 INFO Completed push id=24 objects=5 duration=268.541µs35272026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035282026/11/11 23:27:36 WARN Force mode enabled - objects will be deleted immediately without grace period35292026/11/11 23:27:36 INFO Vacuumed table table=pending_closures35302026/11/11 23:27:36 WARN Force mode enabled - objects will be deleted immediately without grace period35312026/11/11 23:27:36 INFO Received complete upload request method=POST path=/35322026/11/11 23:27:36 INFO Vacuumed table table=pending_closures35332026/11/11 23:27:36 INFO Vacuumed table table=pending_objects35342026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035352026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=035362026/11/11 23:27:36 INFO Completed upload id=25 objects=2 duration=277.666µs35372026/11/11 23:27:36 INFO Vacuumed table table=pending_closures35382026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads35392026/11/11 23:27:36 INFO Vacuumed table table=pending_closures35402026/11/11 23:27:36 INFO Vacuumed table table=pending_objects35412026/11/11 23:27:36 INFO Received complete upload request method=POST path=/35422026/11/11 23:27:36 INFO Vacuumed table table=closures35432026/11/11 23:27:36 INFO Vacuumed table table=pending_objects35442026/11/11 23:27:36 INFO Completed upload id=26 objects=20 duration=200.292µs35452026/11/11 23:27:36 INFO Vacuumed table table=objects35462026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads35472026/11/11 23:27:36 INFO Received complete push request method=POST path=/35482026/11/11 23:27:36 INFO Vacuumed table table=closures35492026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads35502026/11/11 23:27:36 INFO Received complete upload request method=POST path=/35512026/11/11 23:27:36 INFO Vacuumed table table=pending_objects35522026/11/11 23:27:36 INFO Completed push id=9 objects=4 duration=902.125µs35532026/11/11 23:27:36 INFO Vacuumed table table=objects35542026/11/11 23:27:36 INFO Vacuumed table table=closures35552026/11/11 23:27:36 INFO Completed upload id=18 objects=2 duration=692.125µs35562026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads35572026/11/11 23:27:36 INFO Vacuumed table table=objects35582026/11/11 23:27:36 INFO Vacuumed table table=closures35592026/11/11 23:27:36 INFO Received push request method=POST path=/35602026/11/11 23:27:36 INFO Received push request method=POST path=/35612026/11/11 23:27:36 INFO Vacuumed table table=objects35622026/11/11 23:27:36 INFO Received uploads request method=POST path=/35632026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=035642026/11/11 23:27:36 WARN Force mode enabled - objects will be deleted immediately without grace period35652026/11/11 23:27:36 INFO Received uploads request method=POST path=/35662026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035672026/11/11 23:27:36 INFO Vacuumed table table=pending_closures35682026/11/11 23:27:36 INFO Received push request method=POST path=/35692026/11/11 23:27:36 INFO Received complete push request method=POST path=/35702026/11/11 23:27:36 INFO Received push request method=POST path=/35712026/11/11 23:27:36 INFO Completed push id=28 objects=7 duration=362.25µs35722026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=035732026/11/11 23:27:36 INFO Vacuumed table table=pending_objects35742026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads35752026/11/11 23:27:36 INFO Received uploads request method=POST path=/35762026/11/11 23:27:36 INFO Vacuumed table table=closures35772026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035782026/11/11 23:27:36 INFO Received complete push request method=POST path=/35792026/11/11 23:27:36 INFO Vacuumed table table=pending_closures35802026/11/11 23:27:36 INFO Vacuumed table table=objects35812026/11/11 23:27:36 INFO Completed push id=16 objects=4 duration=11.443167ms35822026/11/11 23:27:36 INFO Vacuumed table table=pending_objects35832026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads35842026/11/11 23:27:36 INFO Received push request method=POST path=/35852026/11/11 23:27:36 INFO Received uploads request method=POST path=/35862026/11/11 23:27:36 INFO Vacuumed table table=closures35872026/11/11 23:27:36 INFO Received complete upload request method=POST path=/35882026/11/11 23:27:36 INFO Vacuumed table table=objects35892026/11/11 23:27:36 INFO Completed upload id=20 objects=5 duration=312.791µs35902026/11/11 23:27:36 INFO Received uploads request method=POST path=/35912026/11/11 23:27:36 INFO Received complete push request method=POST path=/35922026/11/11 23:27:36 INFO Received uploads request method=POST path=/35932026/11/11 23:27:36 INFO Completed push id=23 objects=28 duration=875.084µs35942026/11/11 23:27:36 INFO Received complete push request method=POST path=/35952026/11/11 23:27:36 INFO Received push request method=POST path=/35962026/11/11 23:27:36 INFO Completed push id=26 objects=23 duration=599.583µs35972026/11/11 23:27:36 INFO Received complete upload request method=POST path=/35982026/11/11 23:27:36 INFO Received push request method=POST path=/35992026/11/11 23:27:36 INFO Completed upload id=27 objects=8 duration=298.625µs36002026/11/11 23:27:36 INFO Received push request method=POST path=/36012026/11/11 23:27:36 INFO Received complete upload request method=POST path=/36022026/11/11 23:27:36 INFO Received push request method=POST path=/36032026/11/11 23:27:36 INFO Received push request method=POST path=/36042026/11/11 23:27:36 INFO Completed upload id=30 objects=6 duration=374.834µs36052026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=036062026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=036072026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=036082026/11/11 23:27:36 INFO Received complete upload request method=POST path=/36092026/11/11 23:27:36 WARN Force mode enabled - objects will be deleted immediately without grace period36102026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036112026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036122026/11/11 23:27:36 INFO Received complete push request method=POST path=/36132026/11/11 23:27:36 INFO Completed upload id=5 objects=5 duration=497.959µs36142026/11/11 23:27:36 INFO Completed push id=31 objects=5 duration=322.333µs36152026/11/11 23:27:36 INFO Vacuumed table table=pending_closures36162026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036172026/11/11 23:27:36 INFO Received uploads request method=POST path=/36182026/11/11 23:27:36 INFO Received complete upload request method=POST path=/36192026/11/11 23:27:36 INFO Vacuumed table table=pending_closures36202026/11/11 23:27:36 INFO Vacuumed table table=pending_objects36212026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads36222026/11/11 23:27:36 INFO Completed upload id=13 objects=7 duration=610.542µs36232026/11/11 23:27:36 INFO Vacuumed table table=pending_objects36242026/11/11 23:27:36 INFO Received uploads request method=POST path=/36252026/11/11 23:27:36 INFO Vacuumed table table=pending_closures36262026/11/11 23:27:36 INFO Vacuumed table table=closures36272026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads36282026/11/11 23:27:36 INFO Received push request method=POST path=/36292026/11/11 23:27:36 INFO Received complete upload request method=POST path=/36302026/11/11 23:27:36 INFO Completed upload id=20 objects=4 duration=465.541µs36312026/11/11 23:27:36 INFO Vacuumed table table=objects36322026/11/11 23:27:36 INFO Vacuumed table table=closures36332026/11/11 23:27:36 INFO Received uploads request method=POST path=/36342026/11/11 23:27:36 INFO Received complete upload request method=POST path=/36352026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=036362026/11/11 23:27:36 INFO Vacuumed table table=objects36372026/11/11 23:27:36 INFO Completed upload id=23 objects=7 duration=6.18875ms36382026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036392026/11/11 23:27:36 INFO Received uploads request method=POST path=/36402026/11/11 23:27:36 INFO Received complete push request method=POST path=/36412026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=036422026/11/11 23:27:36 INFO Vacuumed table table=pending_closures36432026/11/11 23:27:36 INFO Vacuumed table table=pending_objects36442026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads36452026/11/11 23:27:36 INFO Vacuumed table table=pending_objects36462026/11/11 23:27:36 INFO Vacuumed table table=closures36472026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads36482026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=036492026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036502026/11/11 23:27:36 INFO Vacuumed table table=objects36512026/11/11 23:27:36 INFO Vacuumed table table=closures36522026/11/11 23:27:36 INFO Received complete push request method=POST path=/36532026/11/11 23:27:36 INFO Completed push id=3 objects=5 duration=29.599042ms36542026/11/11 23:27:36 INFO Vacuumed table table=pending_closures36552026/11/11 23:27:36 INFO Vacuumed table table=objects36562026/11/11 23:27:36 INFO Vacuumed table table=pending_objects36572026/11/11 23:27:36 INFO Completed push id=3 objects=21 duration=7.6175ms36582026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=036592026/11/11 23:27:36 INFO Received complete upload request method=POST path=/36602026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads36612026/11/11 23:27:36 INFO Received uploads request method=POST path=/36622026/11/11 23:27:36 INFO Vacuumed table table=pending_closures36632026/11/11 23:27:36 INFO Completed upload id=25 objects=16 duration=717.709µs36642026/11/11 23:27:36 INFO Vacuumed table table=closures36652026/11/11 23:27:36 INFO Received push request method=POST path=/36662026/11/11 23:27:36 INFO Vacuumed table table=pending_objects36672026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=036682026/11/11 23:27:36 INFO Received uploads request method=POST path=/36692026/11/11 23:27:36 INFO Vacuumed table table=objects36702026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads36712026/11/11 23:27:36 WARN Force mode enabled - objects will be deleted immediately without grace period36722026/11/11 23:27:36 INFO Received uploads request method=POST path=/36732026/11/11 23:27:36 INFO Received push request method=POST path=/36742026/11/11 23:27:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036752026/11/11 23:27:36 INFO Vacuumed table table=pending_closures36762026/11/11 23:27:36 INFO Vacuumed table table=closures36772026/11/11 23:27:36 INFO Received complete push request method=POST path=/36782026/11/11 23:27:36 INFO Vacuumed table table=pending_objects36792026/11/11 23:27:36 INFO Vacuumed table table=objects36802026/11/11 23:27:36 INFO Received push request method=POST path=/36812026/11/11 23:27:36 INFO Completed push id=15 objects=5 duration=525.833µs36822026/11/11 23:27:36 INFO Vacuumed table table=multipart_uploads36832026/11/11 23:27:36 INFO Received uploads request method=POST path=/36842026/11/11 23:27:36 INFO Aborted multipart uploads count=0 kept=036852026/11/11 23:27:37 INFO Received complete push request method=POST path=/36862026/11/11 23:27:37 INFO Vacuumed table table=closures36872026/11/11 23:27:37 WARN Force mode enabled - objects will be deleted immediately without grace period36882026/11/11 23:27:37 INFO Received push request method=POST path=/36892026/11/11 23:27:37 INFO Completed push id=32 objects=26 duration=461.375µs36902026/11/11 23:27:37 INFO Vacuumed table table=objects36912026/11/11 23:27:37 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036922026/11/11 23:27:37 INFO Vacuumed table table=pending_closures36932026/11/11 23:27:37 INFO Received complete upload request method=POST path=/36942026/11/11 23:27:37 INFO Received uploads request method=POST path=/36952026/11/11 23:27:37 INFO Received push request method=POST path=/36962026/11/11 23:27:37 INFO Completed upload id=33 objects=2 duration=243.709µs36972026/11/11 23:27:37 INFO Received complete push request method=POST path=/36982026/11/11 23:27:37 INFO Received push request method=POST path=/36992026/11/11 23:27:37 INFO Completed push id=7 objects=7 duration=741.083µs37002026/11/11 23:27:37 INFO Vacuumed table table=pending_objects37012026/11/11 23:27:37 INFO Aborted multipart uploads count=0 kept=037022026/11/11 23:27:37 INFO Received complete upload request method=POST path=/37032026/11/11 23:27:37 INFO Vacuumed table table=multipart_uploads37042026/11/11 23:27:37 INFO Completed upload id=34 objects=5 duration=379.25µs37052026/11/11 23:27:37 INFO Received uploads request method=POST path=/37062026/11/11 23:27:37 WARN Force mode enabled - objects will be deleted immediately without grace period37072026/11/11 23:27:37 INFO Vacuumed table table=closures37082026/11/11 23:27:37 INFO Received uploads request method=POST path=/37092026/11/11 23:27:37 INFO Received complete upload request method=POST path=/37102026/11/11 23:27:37 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037112026/11/11 23:27:37 INFO Vacuumed table table=objects37122026/11/11 23:27:37 INFO Received complete upload request method=POST path=/37132026/11/11 23:27:37 INFO Completed upload id=35 objects=11 duration=413.916µs37142026/11/11 23:27:37 INFO Vacuumed table table=pending_closures37152026/11/11 23:27:37 INFO Completed upload id=22 objects=5 duration=6.306833ms37162026/11/11 23:27:37 INFO Received complete upload request method=POST path=/37172026/11/11 23:27:37 INFO Received push request method=POST path=/37182026/11/11 23:27:37 INFO Vacuumed table table=pending_objects37192026/11/11 23:27:37 INFO Received complete push request method=POST path=/37202026/11/11 23:27:37 INFO Completed upload id=36 objects=3 duration=165.041µs37212026/11/11 23:27:37 INFO Vacuumed table table=multipart_uploads37222026/11/11 23:27:37 INFO Received complete push request method=POST path=/37232026/11/11 23:27:37 INFO Completed push id=28 objects=3 duration=518.875µs37242026/11/11 23:27:37 INFO Received push request method=POST path=/37252026/11/11 23:27:37 INFO Aborted multipart uploads count=0 kept=037262026/11/11 23:27:37 INFO Completed push id=27 objects=15 duration=527.541µs37272026/11/11 23:27:37 INFO Received complete push request method=POST path=/37282026/11/11 23:27:37 INFO Received complete push request method=POST path=/37292026/11/11 23:27:37 WARN Force mode enabled - objects will be deleted immediately without grace period37302026/11/11 23:27:37 INFO Completed push id=29 objects=15 duration=407.709µs37312026/11/11 23:27:37 INFO Vacuumed table table=closures37322026/11/11 23:27:37 INFO Completed push id=31 objects=14 duration=908.333µs37332026/11/11 23:27:37 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037342026/11/11 23:27:37 INFO Received complete push request method=POST path=/37352026/11/11 23:27:37 INFO Vacuumed table table=objects37362026/11/11 23:27:37 INFO Received uploads request method=POST path=/37372026/11/11 23:27:37 INFO Completed push id=30 objects=6 duration=363.917µs37382026/11/11 23:27:37 INFO Vacuumed table table=pending_closures37392026/11/11 23:27:37 INFO Received complete upload request method=POST path=/37402026/11/11 23:27:37 INFO Received uploads request method=POST path=/37412026/11/11 23:27:37 INFO Completed upload id=31 objects=3 duration=300.542µs37422026/11/11 23:27:37 INFO Aborted multipart uploads count=0 kept=037432026/11/11 23:27:37 INFO Vacuumed table table=pending_objects37442026/11/11 23:27:37 WARN Force mode enabled - objects will be deleted immediately without grace period37452026/11/11 23:27:37 INFO Vacuumed table table=multipart_uploads37462026/11/11 23:27:37 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037472026/11/11 23:27:37 INFO Vacuumed table table=closures37482026/11/11 23:27:37 INFO Vacuumed table table=objects37492026/11/11 23:27:37 INFO Vacuumed table table=pending_closures37502026/11/11 23:27:37 INFO Vacuumed table table=pending_objects37512026/11/11 23:27:37 INFO Vacuumed table table=multipart_uploads37522026/11/11 23:27:37 INFO Vacuumed table table=closures37532026/11/11 23:27:37 INFO Vacuumed table table=objects37542026/11/11 23:27:37 INFO Received complete push request method=POST path=/37552026/11/11 23:27:37 INFO Completed push id=14 objects=3 duration=1.266416ms37562026/11/11 23:27:37 INFO Received complete upload request method=POST path=/37572026/11/11 23:27:37 INFO Completed upload id=24 objects=18 duration=443.833µs37582026/11/11 23:27:37 INFO Received complete upload request method=POST path=/37592026/11/11 23:27:37 INFO Received complete push request method=POST path=/37602026/11/11 23:27:37 INFO Completed push id=25 objects=2 duration=726.375µs37612026/11/11 23:27:37 INFO Completed upload id=10 objects=23 duration=11.442375ms37622026/11/11 23:27:37 INFO Received complete upload request method=POST path=/37632026/11/11 23:27:37 INFO Received complete push request method=POST path=/37642026/11/11 23:27:37 INFO Received uploads request method=POST path=/37652026/11/11 23:27:37 INFO Completed upload id=18 objects=11 duration=543.875µs37662026/11/11 23:27:37 INFO Completed push id=27 objects=4 duration=381.125µs37672026/11/11 23:27:37 INFO Received complete push request method=POST path=/37682026/11/11 23:27:37 INFO Completed push id=21 objects=9 duration=676.375µs37692026/11/11 23:27:37 INFO Received uploads request method=POST path=/37702026/11/11 23:27:37 INFO Received push request method=POST path=/37712026/11/11 23:27:37 INFO Received uploads request method=POST path=/37722026/11/11 23:27:37 INFO Aborted multipart uploads count=0 kept=037732026/11/11 23:27:37 WARN Force mode enabled - objects will be deleted immediately without grace period37742026/11/11 23:27:37 INFO Received complete upload request method=POST path=/37752026/11/11 23:27:37 INFO Received push request method=POST path=/37762026/11/11 23:27:37 INFO Completed upload id=3 objects=24 duration=6.663917ms37772026/11/11 23:27:37 INFO Received push request method=POST path=/37782026/11/11 23:27:37 INFO Received complete upload request method=POST path=/37792026/11/11 23:27:37 INFO Received complete upload request method=POST path=/37802026/11/11 23:27:37 INFO Completed upload id=13 objects=8 duration=386.458µs37812026/11/11 23:27:37 INFO Completed upload id=4 objects=14 duration=980.541µs37822026/11/11 23:27:37 INFO Received complete upload request method=POST path=/37832026/11/11 23:27:37 INFO Completed upload id=11 objects=6 duration=953.75µs37842026/11/11 23:27:37 INFO Received uploads request method=POST path=/37852026/11/11 23:27:37 INFO Received push request method=POST path=/37862026/11/11 23:27:37 INFO Aborted multipart uploads count=0 kept=037872026/11/11 23:27:37 INFO Received uploads request method=POST path=/37882026/11/11 23:27:37 WARN Force mode enabled - objects will be deleted immediately without grace period37892026/11/11 23:27:37 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037902026/11/11 23:27:37 INFO Received push request method=POST path=/37912026/11/11 23:27:37 INFO Vacuumed table table=pending_closures37922026/11/11 23:27:37 INFO Vacuumed table table=pending_objects37932026/11/11 23:27:37 INFO Vacuumed table table=multipart_uploads37942026/11/11 23:27:37 INFO Vacuumed table table=closures37952026/11/11 23:27:37 INFO Received complete upload request method=POST path=/37962026/11/11 23:27:37 INFO Vacuumed table table=objects37972026/11/11 23:27:37 INFO Completed upload id=4 objects=7 duration=790.042µs37982026/11/11 23:27:37 INFO Received push request method=POST path=/37992026/11/11 23:27:37 INFO Received push request method=POST path=/38002026/11/11 23:27:37 INFO Received uploads request method=POST path=/38012026/11/11 23:27:37 INFO Received push request method=POST path=/38022026/11/11 23:27:37 INFO Received uploads request method=POST path=/38032026/11/11 23:27:37 INFO Aborted multipart uploads count=0 kept=038042026/11/11 23:27:37 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=038052026/11/11 23:27:37 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038062026/11/11 23:27:37 INFO Received push request method=POST path=/38072026/11/11 23:27:38 INFO Aborted multipart uploads count=0 kept=038082026/11/11 23:27:38 INFO Received complete push request method=POST path=/38092026/11/11 23:27:38 WARN Force mode enabled - objects will be deleted immediately without grace period38102026/11/11 23:27:38 INFO Vacuumed table table=pending_closures38112026/11/11 23:27:38 INFO Completed push id=12 objects=15 duration=777µs38122026/11/11 23:27:38 INFO Vacuumed table table=pending_closures38132026/11/11 23:27:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038142026/11/11 23:27:38 INFO Received push request method=POST path=/38152026/11/11 23:27:38 INFO Received complete push request method=POST path=/38162026/11/11 23:27:38 INFO Completed push id=7 objects=7 duration=666.834µs38172026/11/11 23:27:38 INFO Vacuumed table table=pending_objects38182026/11/11 23:27:38 INFO Vacuumed table table=multipart_uploads38192026/11/11 23:27:38 INFO Vacuumed table table=pending_objects38202026/11/11 23:27:38 INFO Received complete push request method=POST path=/38212026/11/11 23:27:38 INFO Vacuumed table table=closures38222026/11/11 23:27:38 INFO Received complete push request method=POST path=/38232026/11/11 23:27:38 INFO Vacuumed table table=multipart_uploads38242026/11/11 23:27:38 INFO Completed push id=28 objects=26 duration=589.833µs38252026/11/11 23:27:38 INFO Completed push id=12 objects=6 duration=668.459µs38262026/11/11 23:27:38 INFO Vacuumed table table=pending_closures38272026/11/11 23:27:38 INFO Vacuumed table table=objects38282026/11/11 23:27:38 INFO Vacuumed table table=closures38292026/11/11 23:27:38 INFO Received uploads request method=POST path=/38302026/11/11 23:27:38 INFO Received complete push request method=POST path=/38312026/11/11 23:27:38 INFO Vacuumed table table=objects38322026/11/11 23:27:38 INFO Received complete push request method=POST path=/38332026/11/11 23:27:38 INFO Completed push id=30 objects=3 duration=432µs38342026/11/11 23:27:38 INFO Vacuumed table table=pending_objects38352026/11/11 23:27:38 INFO Completed push id=28 objects=19 duration=6.343542ms38362026/11/11 23:27:38 INFO Vacuumed table table=multipart_uploads38372026/11/11 23:27:38 INFO Received uploads request method=POST path=/38382026/11/11 23:27:38 INFO Received complete push request method=POST path=/38392026/11/11 23:27:38 INFO Aborted multipart uploads count=0 kept=038402026/11/11 23:27:38 INFO Completed push id=32 objects=19 duration=400.125µs38412026/11/11 23:27:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038422026/11/11 23:27:38 INFO Received complete upload request method=POST path=/38432026/11/11 23:27:38 INFO Completed upload id=15 objects=6 duration=326.125µs38442026/11/11 23:27:38 INFO Vacuumed table table=pending_closures38452026/11/11 23:27:38 INFO Aborted multipart uploads count=0 kept=038462026/11/11 23:27:38 INFO Vacuumed table table=pending_objects38472026/11/11 23:27:38 INFO Received push request method=POST path=/38482026/11/11 23:27:38 INFO Vacuumed table table=closures38492026/11/11 23:27:38 INFO Vacuumed table table=multipart_uploads38502026/11/11 23:27:38 WARN Force mode enabled - objects will be deleted immediately without grace period38512026/11/11 23:27:38 INFO Vacuumed table table=closures38522026/11/11 23:27:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038532026/11/11 23:27:38 INFO Vacuumed table table=objects38542026/11/11 23:27:38 INFO Received complete push request method=POST path=/38552026/11/11 23:27:38 INFO Received push request method=POST path=/38562026/11/11 23:27:38 INFO Completed push id=5 objects=8 duration=5.707666ms38572026/11/11 23:27:38 INFO Vacuumed table table=objects38582026/11/11 23:27:38 INFO Vacuumed table table=pending_closures38592026/11/11 23:27:38 INFO Received push request method=POST path=/38602026/11/11 23:27:38 INFO Received uploads request method=POST path=/38612026/11/11 23:27:38 INFO Received complete push request method=POST path=/38622026/11/11 23:27:38 INFO Vacuumed table table=pending_objects38632026/11/11 23:27:38 INFO Received uploads request method=POST path=/38642026/11/11 23:27:38 INFO Aborted multipart uploads count=0 kept=038652026/11/11 23:27:38 INFO Vacuumed table table=multipart_uploads38662026/11/11 23:27:38 INFO Completed push id=8 objects=6 duration=1.135833ms38672026/11/11 23:27:38 INFO Received uploads request method=POST path=/38682026/11/11 23:27:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038692026/11/11 23:27:38 INFO Vacuumed table table=closures38702026/11/11 23:27:38 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."38712026/11/11 23:27:38 INFO Received uploads request method=POST path=/38722026/11/11 23:27:38 INFO Vacuumed table table=pending_closures38732026/11/11 23:27:38 INFO Received push request method=POST path=/38742026/11/11 23:27:38 INFO Received uploads request method=POST path=/38752026/11/11 23:27:38 INFO Received uploads request method=POST path=/38762026/11/11 23:27:38 INFO Vacuumed table table=objects38772026/11/11 23:27:38 INFO Vacuumed table table=pending_objects38782026/11/11 23:27:38 INFO Aborted multipart uploads count=0 kept=038792026/11/11 23:27:38 INFO Vacuumed table table=multipart_uploads38802026/11/11 23:27:38 INFO Vacuumed table table=closures38812026/11/11 23:27:38 INFO Vacuumed table table=objects38822026/11/11 23:27:38 WARN Force mode enabled - objects will be deleted immediately without grace period38832026/11/11 23:27:38 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=4 objects-failed-to-delete=038842026/11/11 23:27:38 INFO Received uploads request method=POST path=/38852026/11/11 23:27:38 INFO Vacuumed table table=pending_closures38862026/11/11 23:27:38 INFO Received uploads request method=POST path=/38872026/11/11 23:27:38 INFO Received push request method=POST path=/38882026/11/11 23:27:38 INFO Vacuumed table table=pending_objects38892026/11/11 23:27:38 INFO Vacuumed table table=multipart_uploads38902026/11/11 23:27:38 INFO Received complete upload request method=POST path=/38912026/11/11 23:27:38 INFO Vacuumed table table=closures38922026/11/11 23:27:38 INFO Completed upload id=8 objects=3 duration=10.746375ms38932026/11/11 23:27:38 INFO Received complete push request method=POST path=/38942026/11/11 23:27:38 INFO Vacuumed table table=objects38952026/11/11 23:27:38 INFO Completed push id=29 objects=10 duration=5.810291ms38962026/11/11 23:27:38 INFO Received uploads request method=POST path=/38972026/11/11 23:27:38 INFO Received complete upload request method=POST path=/38982026/11/11 23:27:38 INFO Completed upload id=10 objects=6 duration=886.292µs38992026/11/11 23:27:38 INFO Aborted multipart uploads count=0 kept=039002026/11/11 23:27:38 WARN Force mode enabled - objects will be deleted immediately without grace period39012026/11/11 23:27:38 INFO Received complete upload request method=POST path=/39022026/11/11 23:27:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039032026/11/11 23:27:38 INFO Vacuumed table table=pending_closures39042026/11/11 23:27:38 INFO Received complete push request method=POST path=/39052026/11/11 23:27:38 INFO Received complete push request method=POST path=/39062026/11/11 23:27:38 INFO Received complete push request method=POST path=/39072026/11/11 23:27:38 INFO Vacuumed table table=pending_objects39082026/11/11 23:27:38 INFO Completed push id=5 objects=28 duration=537.666µs39092026/11/11 23:27:38 INFO Completed push id=6 objects=6 duration=563µs39102026/11/11 23:27:38 INFO Completed upload id=1 objects=15 duration=30.443708ms39112026/11/11 23:27:38 INFO Completed push id=15 objects=4 duration=307.666µs39122026/11/11 23:27:38 INFO Vacuumed table table=multipart_uploads39132026/11/11 23:27:38 INFO Vacuumed table table=closures39142026/11/11 23:27:38 INFO Received push request method=POST path=/39152026/11/11 23:27:38 INFO Received complete push request method=POST path=/39162026/11/11 23:27:38 INFO Vacuumed table table=objects39172026/11/11 23:27:38 INFO Received uploads request method=POST path=/39182026/11/11 23:27:38 INFO Received push request method=POST path=/39192026/11/11 23:27:38 INFO Completed push id=10 objects=2 duration=494.5µs39202026/11/11 23:27:38 INFO Aborted multipart uploads count=0 kept=039212026/11/11 23:27:38 INFO Received uploads request method=POST path=/39222026/11/11 23:27:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039232026/11/11 23:27:38 INFO Vacuumed table table=pending_closures39242026/11/11 23:27:38 INFO Received complete upload request method=POST path=/39252026/11/11 23:27:38 INFO Aborted multipart uploads count=0 kept=039262026/11/11 23:27:38 INFO Completed upload id=30 objects=6 duration=422.333µs39272026/11/11 23:27:38 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."39282026/11/11 23:27:38 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."39292026/11/11 23:27:38 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."39302026/11/11 23:27:38 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."39312026/11/11 23:27:38 INFO Received complete upload request method=POST path=/39322026/11/11 23:27:38 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."39332026/11/11 23:27:38 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."39342026/11/11 23:27:38 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."39352026/11/11 23:27:38 INFO Vacuumed table table=pending_objects39362026/11/11 23:27:38 INFO Completed upload id=31 objects=5 duration=6.380083ms39372026/11/11 23:27:38 INFO Vacuumed table table=multipart_uploads39382026/11/11 23:27:38 INFO Aborted multipart uploads count=0 kept=039392026/11/11 23:27:38 INFO Vacuumed table table=closures39402026/11/11 23:27:38 INFO Received complete upload request method=POST path=/39412026/11/11 23:27:38 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."39422026/11/11 23:27:38 INFO Vacuumed table table=objects39432026/11/11 23:27:38 INFO Completed upload id=34 objects=3 duration=158.958µs39442026/11/11 23:27:38 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."39452026/11/11 23:27:38 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."39462026/11/11 23:27:38 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."39472026/11/11 23:27:38 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."39482026/11/11 23:27:38 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."39492026/11/11 23:27:38 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."39502026/11/11 23:27:38 INFO Received uploads request method=POST path=/39512026/11/11 23:27:38 INFO Aborted multipart uploads count=0 kept=039522026/11/11 23:27:38 WARN Force mode enabled - objects will be deleted immediately without grace period39532026/11/11 23:27:38 INFO Received push request method=POST path=/39542026/11/11 23:27:38 INFO Received complete push request method=POST path=/39552026/11/11 23:27:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039562026/11/11 23:27:38 INFO Received push request method=POST path=/39572026/11/11 23:27:38 INFO Received push request method=POST path=/39582026/11/11 23:27:38 INFO Completed push id=16 objects=27 duration=453.791µs39592026/11/11 23:27:38 INFO Aborted multipart uploads count=0 kept=039602026/11/11 23:27:38 INFO Received uploads request method=POST path=/39612026/11/11 23:27:38 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."39622026/11/11 23:27:38 INFO Received uploads request method=POST path=/39632026/11/11 23:27:38 INFO Received uploads request method=POST path=/39642026/11/11 23:27:38 INFO Received complete upload request method=POST path=/39652026/11/11 23:27:38 INFO Received push request method=POST path=/39662026/11/11 23:27:38 INFO Vacuumed table table=pending_closures39672026/11/11 23:27:38 INFO Completed upload id=17 objects=5 duration=590.542µs39682026/11/11 23:27:38 INFO Aborted multipart uploads count=0 kept=039692026/11/11 23:27:38 INFO Vacuumed table table=pending_objects39702026/11/11 23:27:38 INFO Vacuumed table table=multipart_uploads39712026/11/11 23:27:38 INFO Received complete push request method=POST path=/39722026/11/11 23:27:38 INFO Vacuumed table table=closures39732026/11/11 23:27:38 INFO Completed push id=10 objects=15 duration=558.917µs39742026/11/11 23:27:38 INFO Vacuumed table table=objects39752026/11/11 23:27:38 INFO Aborted multipart uploads count=0 kept=039762026/11/11 23:27:38 WARN Force mode enabled - objects will be deleted immediately without grace period39772026/11/11 23:27:38 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=039782026/11/11 23:27:38 INFO Received complete push request method=POST path=/39792026/11/11 23:27:38 INFO Completed push id=13 objects=22 duration=363.625µs39802026/11/11 23:27:38 INFO Received complete push request method=POST path=/39812026/11/11 23:27:38 INFO Completed push id=21 objects=6 duration=405.625µs39822026/11/11 23:27:38 INFO Received complete upload request method=POST path=/39832026/11/11 23:27:38 INFO Completed upload id=22 objects=7 duration=469.084µs39842026/11/11 23:27:38 INFO Vacuumed table table=pending_closures39852026/11/11 23:27:38 INFO Received complete push request method=POST path=/39862026/11/11 23:27:38 INFO Completed push id=24 objects=4 duration=275.292µs39872026/11/11 23:27:38 INFO Received complete push request method=POST path=/39882026/11/11 23:27:38 INFO Completed push id=19 objects=3 duration=440.25µs39892026/11/11 23:27:38 INFO Received complete push request method=POST path=/39902026/11/11 23:27:38 INFO Vacuumed table table=pending_objects39912026/11/11 23:27:38 INFO Completed push id=25 objects=9 duration=541.75µs39922026/11/11 23:27:38 INFO Vacuumed table table=multipart_uploads39932026/11/11 23:27:38 INFO Vacuumed table table=closures39942026/11/11 23:27:38 INFO Vacuumed table table=objects39952026/11/11 23:27:38 INFO Received complete upload request method=POST path=/39962026/11/11 23:27:38 INFO Completed upload id=26 objects=3 duration=281.042µs39972026/11/11 23:27:38 INFO Received push request method=POST path=/39982026/11/11 23:27:38 INFO Received complete push request method=POST path=/39992026/11/11 23:27:38 INFO Completed push id=17 objects=22 duration=646.667µs40002026/11/11 23:27:38 INFO Received complete push request method=POST path=/40012026/11/11 23:27:38 INFO Received complete upload request method=POST path=/40022026/11/11 23:27:38 INFO Completed push id=18 objects=10 duration=584.208µs40032026/11/11 23:27:38 INFO Completed upload id=6 objects=24 duration=1.124375ms40042026/11/11 23:27:38 INFO Received uploads request method=POST path=/40052026/11/11 23:27:38 INFO Received complete push request method=POST path=/40062026/11/11 23:27:38 INFO Received push request method=POST path=/40072026/11/11 23:27:38 INFO Aborted multipart uploads count=0 kept=040082026/11/11 23:27:38 INFO Completed push id=14 objects=3 duration=611.458µs40092026/11/11 23:27:38 WARN Force mode enabled - objects will be deleted immediately without grace period40102026/11/11 23:27:38 INFO Aborted multipart uploads count=0 kept=040112026/11/11 23:27:38 WARN Force mode enabled - objects will be deleted immediately without grace period40122026/11/11 23:27:38 INFO Garbage collection completed failed-uploads-deleted=5 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040132026/11/11 23:27:38 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."40142026/11/11 23:27:38 INFO Received complete upload request method=POST path=/40152026/11/11 23:27:38 INFO Completed upload id=20 objects=4 duration=376.084µs40162026/11/11 23:27:38 INFO Received complete upload request method=POST path=/40172026/11/11 23:27:38 INFO Completed upload id=27 objects=7 duration=225.583µs40182026/11/11 23:27:38 INFO Received complete upload request method=POST path=/40192026/11/11 23:27:38 INFO Received complete upload request method=POST path=/40202026/11/11 23:27:38 INFO Completed upload id=11 objects=6 duration=714.417µs40212026/11/11 23:27:38 INFO Completed upload id=28 objects=3 duration=165.791µs40222026/11/11 23:27:38 INFO Received uploads request method=POST path=/40232026/11/11 23:27:38 INFO Received complete push request method=POST path=/40242026/11/11 23:27:38 INFO Vacuumed table table=pending_closures40252026/11/11 23:27:38 INFO Completed push id=7 objects=20 duration=10.767542ms40262026/11/11 23:27:39 INFO Received uploads request method=POST path=/40272026/11/11 23:27:39 INFO Received complete push request method=POST path=/40282026/11/11 23:27:39 INFO Received complete push request method=POST path=/40292026/11/11 23:27:39 INFO Received push request method=POST path=/40302026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=040312026/11/11 23:27:39 INFO Completed push id=9 objects=24 duration=350.708µs40322026/11/11 23:27:39 INFO Completed push id=20 objects=4 duration=444.166µs40332026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040342026/11/11 23:27:39 INFO Received uploads request method=POST path=/40352026/11/11 23:27:39 INFO Received complete upload request method=POST path=/40362026/11/11 23:27:39 INFO Received complete push request method=POST path=/40372026/11/11 23:27:39 INFO Vacuumed table table=pending_objects40382026/11/11 23:27:39 INFO Vacuumed table table=pending_closures40392026/11/11 23:27:39 INFO Received complete push request method=POST path=/40402026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads40412026/11/11 23:27:39 INFO Completed push id=9 objects=5 duration=603.458µs40422026/11/11 23:27:39 INFO Completed push id=8 objects=6 duration=735.291µs40432026/11/11 23:27:39 INFO Completed upload id=14 objects=8 duration=1.0325ms40442026/11/11 23:27:39 INFO Received uploads request method=POST path=/40452026/11/11 23:27:39 INFO Received complete push request method=POST path=/40462026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=040472026/11/11 23:27:39 INFO Vacuumed table table=pending_objects40482026/11/11 23:27:39 INFO Received complete upload request method=POST path=/40492026/11/11 23:27:39 INFO Received uploads request method=POST path=/40502026/11/11 23:27:39 INFO Completed push id=29 objects=23 duration=350.583µs40512026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads40522026/11/11 23:27:39 INFO Completed upload id=10 objects=10 duration=338.084µs40532026/11/11 23:27:39 WARN Force mode enabled - objects will be deleted immediately without grace period40542026/11/11 23:27:39 INFO Vacuumed table table=closures40552026/11/11 23:27:39 INFO Vacuumed table table=closures40562026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=040572026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040582026/11/11 23:27:39 INFO Received complete push request method=POST path=/40592026/11/11 23:27:39 INFO Received uploads request method=POST path=/40602026/11/11 23:27:39 WARN Force mode enabled - objects will be deleted immediately without grace period40612026/11/11 23:27:39 INFO Vacuumed table table=pending_closures40622026/11/11 23:27:39 INFO Vacuumed table table=objects40632026/11/11 23:27:39 INFO Completed push id=30 objects=6 duration=7.169791ms40642026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=040652026/11/11 23:27:39 INFO Received push request method=POST path=/40662026/11/11 23:27:39 INFO Vacuumed table table=pending_objects40672026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads40682026/11/11 23:27:39 INFO Received push request method=POST path=/40692026/11/11 23:27:39 INFO Vacuumed table table=closures40702026/11/11 23:27:39 INFO Vacuumed table table=objects40712026/11/11 23:27:39 INFO Vacuumed table table=pending_closures40722026/11/11 23:27:39 INFO Vacuumed table table=objects40732026/11/11 23:27:39 INFO Received uploads request method=POST path=/40742026/11/11 23:27:39 INFO Received push request method=POST path=/40752026/11/11 23:27:39 INFO Vacuumed table table=pending_objects40762026/11/11 23:27:39 INFO Received complete upload request method=POST path=/40772026/11/11 23:27:39 INFO Completed upload id=9 objects=3 duration=524.667µs40782026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads40792026/11/11 23:27:39 INFO Received complete push request method=POST path=/40802026/11/11 23:27:39 INFO Vacuumed table table=closures40812026/11/11 23:27:39 INFO Completed push id=14 objects=3 duration=384.708µs40822026/11/11 23:27:39 INFO Vacuumed table table=objects40832026/11/11 23:27:39 INFO Received push request method=POST path=/40842026/11/11 23:27:39 INFO Received push request method=POST path=/40852026/11/11 23:27:39 INFO Received push request method=POST path=/40862026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=040872026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=040882026/11/11 23:27:39 INFO Vacuumed table table=pending_closures40892026/11/11 23:27:39 INFO Received push request method=POST path=/40902026/11/11 23:27:39 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."40912026/11/11 23:27:39 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."40922026/11/11 23:27:39 INFO Vacuumed table table=pending_objects40932026/11/11 23:27:39 INFO Received push request method=POST path=/40942026/11/11 23:27:39 INFO Received uploads request method=POST path=/40952026/11/11 23:27:39 INFO Received complete push request method=POST path=/40962026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=040972026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads40982026/11/11 23:27:39 WARN Force mode enabled - objects will be deleted immediately without grace period40992026/11/11 23:27:39 INFO Completed push id=5 objects=3 duration=1.31925ms41002026/11/11 23:27:39 INFO Received complete push request method=POST path=/41012026/11/11 23:27:39 INFO Received uploads request method=POST path=/41022026/11/11 23:27:39 INFO Received push request method=POST path=/41032026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041042026/11/11 23:27:39 INFO Completed push id=33 objects=3 duration=264.709µs41052026/11/11 23:27:39 INFO Received complete upload request method=POST path=/41062026/11/11 23:27:39 INFO Received complete push request method=POST path=/41072026/11/11 23:27:39 INFO Received uploads request method=POST path=/41082026/11/11 23:27:39 INFO Received push request method=POST path=/41092026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=041102026/11/11 23:27:39 INFO Completed push id=16 objects=5 duration=290.375µs41112026/11/11 23:27:39 INFO Completed upload id=22 objects=8 duration=403.167µs41122026/11/11 23:27:39 INFO Received complete push request method=POST path=/41132026/11/11 23:27:39 INFO Vacuumed table table=closures41142026/11/11 23:27:39 INFO Received complete push request method=POST path=/41152026/11/11 23:27:39 INFO Completed push id=12 objects=3 duration=658.125µs41162026/11/11 23:27:39 INFO Received complete push request method=POST path=/41172026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041182026/11/11 23:27:39 INFO Received uploads request method=POST path=/41192026/11/11 23:27:39 INFO Completed push id=34 objects=6 duration=325µs41202026/11/11 23:27:39 INFO Vacuumed table table=pending_closures41212026/11/11 23:27:39 INFO Completed push id=8 objects=6 duration=878.833µs41222026/11/11 23:27:39 INFO Vacuumed table table=objects41232026/11/11 23:27:39 INFO Received complete push request method=POST path=/41242026/11/11 23:27:39 INFO Received uploads request method=POST path=/41252026/11/11 23:27:39 INFO Received complete push request method=POST path=/41262026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=041272026/11/11 23:27:39 INFO Vacuumed table table=pending_closures41282026/11/11 23:27:39 INFO Completed push id=23 objects=5 duration=481.25µs41292026/11/11 23:27:39 INFO Completed push id=35 objects=8 duration=440.416µs41302026/11/11 23:27:39 INFO Vacuumed table table=pending_objects41312026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=041322026/11/11 23:27:39 INFO Received push request method=POST path=/41332026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads41342026/11/11 23:27:39 INFO Vacuumed table table=pending_closures41352026/11/11 23:27:39 INFO Received uploads request method=POST path=/41362026/11/11 23:27:39 INFO Vacuumed table table=pending_objects41372026/11/11 23:27:39 INFO Vacuumed table table=closures41382026/11/11 23:27:39 INFO Vacuumed table table=pending_objects41392026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads41402026/11/11 23:27:39 INFO Received uploads request method=POST path=/41412026/11/11 23:27:39 INFO Received complete upload request method=POST path=/41422026/11/11 23:27:39 INFO Received complete upload request method=POST path=/41432026/11/11 23:27:39 INFO Vacuumed table table=closures41442026/11/11 23:27:39 INFO Completed upload id=36 objects=9 duration=292.375µs41452026/11/11 23:27:39 INFO Completed upload id=24 objects=8 duration=406.292µs41462026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads41472026/11/11 23:27:39 INFO Vacuumed table table=objects41482026/11/11 23:27:39 INFO Received push request method=POST path=/41492026/11/11 23:27:39 INFO Vacuumed table table=objects41502026/11/11 23:27:39 INFO Vacuumed table table=closures41512026/11/11 23:27:39 INFO Received push request method=POST path=/41522026/11/11 23:27:39 INFO Received complete upload request method=POST path=/41532026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=041542026/11/11 23:27:39 INFO Completed upload id=28 objects=5 duration=315.791µs41552026/11/11 23:27:39 WARN Force mode enabled - objects will be deleted immediately without grace period41562026/11/11 23:27:39 INFO Received push request method=POST path=/41572026/11/11 23:27:39 INFO Vacuumed table table=objects41582026/11/11 23:27:39 INFO Received complete upload request method=POST path=/41592026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041602026/11/11 23:27:39 INFO Completed upload id=16 objects=17 duration=707.917µs41612026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=041622026/11/11 23:27:39 INFO Received uploads request method=POST path=/41632026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=041642026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041652026/11/11 23:27:39 INFO Received uploads request method=POST path=/41662026/11/11 23:27:39 INFO Received uploads request method=POST path=/41672026/11/11 23:27:39 INFO Vacuumed table table=pending_closures41682026/11/11 23:27:39 WARN Force mode enabled - objects will be deleted immediately without grace period41692026/11/11 23:27:39 INFO Vacuumed table table=pending_closures41702026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=041712026/11/11 23:27:39 INFO Vacuumed table table=pending_objects41722026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads41732026/11/11 23:27:39 INFO Received complete upload request method=POST path=/41742026/11/11 23:27:39 WARN Force mode enabled - objects will be deleted immediately without grace period41752026/11/11 23:27:39 INFO Received uploads request method=POST path=/41762026/11/11 23:27:39 INFO Vacuumed table table=closures41772026/11/11 23:27:39 INFO Completed upload id=16 objects=23 duration=313.25µs41782026/11/11 23:27:39 INFO Vacuumed table table=pending_objects41792026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041802026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads41812026/11/11 23:27:39 INFO Vacuumed table table=objects41822026/11/11 23:27:39 INFO Received complete upload request method=POST path=/41832026/11/11 23:27:39 INFO Vacuumed table table=pending_closures41842026/11/11 23:27:39 INFO Vacuumed table table=closures41852026/11/11 23:27:39 INFO Received uploads request method=POST path=/41862026/11/11 23:27:39 INFO Completed upload id=15 objects=6 duration=427.958µs41872026/11/11 23:27:39 INFO Received complete upload request method=POST path=/41882026/11/11 23:27:39 INFO Vacuumed table table=objects41892026/11/11 23:27:39 INFO Vacuumed table table=pending_objects41902026/11/11 23:27:39 INFO Received uploads request method=POST path=/41912026/11/11 23:27:39 INFO Completed upload id=30 objects=5 duration=461.375µs41922026/11/11 23:27:39 INFO Received uploads request method=POST path=/41932026/11/11 23:27:39 INFO Received uploads request method=POST path=/41942026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads41952026/11/11 23:27:39 INFO Received complete upload request method=POST path=/41962026/11/11 23:27:39 INFO Received push request method=POST path=/41972026/11/11 23:27:39 INFO Vacuumed table table=closures41982026/11/11 23:27:39 INFO Completed upload id=23 objects=18 duration=687.667µs41992026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=042002026/11/11 23:27:39 INFO Vacuumed table table=objects42012026/11/11 23:27:39 INFO Received complete upload request method=POST path=/42022026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042032026/11/11 23:27:39 INFO Received uploads request method=POST path=/42042026/11/11 23:27:39 INFO Vacuumed table table=pending_closures42052026/11/11 23:27:39 INFO Completed upload id=10 objects=17 duration=4.337ms42062026/11/11 23:27:39 INFO Received push request method=POST path=/42072026/11/11 23:27:39 INFO Received uploads request method=POST path=/42082026/11/11 23:27:39 INFO Vacuumed table table=pending_objects42092026/11/11 23:27:39 INFO Received complete upload request method=POST path=/42102026/11/11 23:27:39 INFO Received push request method=POST path=/42112026/11/11 23:27:39 INFO Received uploads request method=POST path=/42122026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads42132026/11/11 23:27:39 INFO Completed upload id=22 objects=22 duration=269.209µs42142026/11/11 23:27:39 INFO Vacuumed table table=closures42152026/11/11 23:27:39 INFO Received push request method=POST path=/42162026/11/11 23:27:39 INFO Vacuumed table table=objects42172026/11/11 23:27:39 INFO Received push request method=POST path=/42182026/11/11 23:27:39 INFO Received uploads request method=POST path=/42192026/11/11 23:27:39 INFO Received complete push request method=POST path=/42202026/11/11 23:27:39 INFO Received complete push request method=POST path=/42212026/11/11 23:27:39 INFO Completed push id=31 objects=3 duration=372.041µs42222026/11/11 23:27:39 INFO Completed push id=14 objects=6 duration=5.489167ms42232026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=042242026/11/11 23:27:39 INFO Received complete upload request method=POST path=/42252026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042262026/11/11 23:27:39 INFO Received complete push request method=POST path=/42272026/11/11 23:27:39 INFO Received push request method=POST path=/42282026/11/11 23:27:39 INFO Vacuumed table table=pending_closures42292026/11/11 23:27:39 INFO Completed upload id=17 objects=7 duration=378.542µs42302026/11/11 23:27:39 INFO Completed push id=26 objects=5 duration=469.5µs42312026/11/11 23:27:39 INFO Received uploads request method=POST path=/42322026/11/11 23:27:39 INFO Received push request method=POST path=/42332026/11/11 23:27:39 INFO Vacuumed table table=pending_objects42342026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads42352026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=042362026/11/11 23:27:39 INFO Received complete upload request method=POST path=/42372026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=042382026/11/11 23:27:39 INFO Vacuumed table table=closures42392026/11/11 23:27:39 INFO Completed upload id=24 objects=14 duration=6.811083ms42402026/11/11 23:27:39 INFO Vacuumed table table=objects42412026/11/11 23:27:39 INFO Received complete push request method=POST path=/42422026/11/11 23:27:39 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."42432026/11/11 23:27:39 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."42442026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=042452026/11/11 23:27:39 INFO Received complete push request method=POST path=/42462026/11/11 23:27:39 INFO Completed push id=19 objects=3 duration=474.292µs42472026/11/11 23:27:39 INFO Completed push id=18 objects=6 duration=6.227958ms42482026/11/11 23:27:39 INFO Vacuumed table table=pending_closures42492026/11/11 23:27:39 INFO Vacuumed table table=pending_objects42502026/11/11 23:27:39 INFO Received push request method=POST path=/42512026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads42522026/11/11 23:27:39 INFO Received complete upload request method=POST path=/42532026/11/11 23:27:39 INFO Received uploads request method=POST path=/42542026/11/11 23:27:39 INFO Received push request method=POST path=/42552026/11/11 23:27:39 INFO Completed upload id=23 objects=9 duration=325µs42562026/11/11 23:27:39 INFO Vacuumed table table=closures42572026/11/11 23:27:39 INFO Received uploads request method=POST path=/42582026/11/11 23:27:39 INFO Received push request method=POST path=/42592026/11/11 23:27:39 INFO Vacuumed table table=objects42602026/11/11 23:27:39 INFO Received uploads request method=POST path=/42612026/11/11 23:27:39 INFO Received push request method=POST path=/42622026/11/11 23:27:39 INFO Received uploads request method=POST path=/42632026/11/11 23:27:39 INFO Received push request method=POST path=/42642026/11/11 23:27:39 INFO Received uploads request method=POST path=/42652026/11/11 23:27:39 INFO Received complete upload request method=POST path=/42662026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=042672026/11/11 23:27:39 INFO Completed upload id=25 objects=4 duration=411.083µs42682026/11/11 23:27:39 INFO Received complete upload request method=POST path=/42692026/11/11 23:27:39 INFO Received uploads request method=POST path=/42702026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042712026/11/11 23:27:39 INFO Received push request method=POST path=/42722026/11/11 23:27:39 INFO Received push request method=POST path=/42732026/11/11 23:27:39 INFO Completed upload id=14 objects=6 duration=279.25µs42742026/11/11 23:27:39 INFO Vacuumed table table=pending_closures42752026/11/11 23:27:39 INFO Received complete push request method=POST path=/42762026/11/11 23:27:39 INFO Vacuumed table table=pending_objects42772026/11/11 23:27:39 INFO Completed push id=12 objects=2 duration=861.833µs42782026/11/11 23:27:39 INFO Received complete push request method=POST path=/42792026/11/11 23:27:39 INFO Received push request method=POST path=/42802026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads42812026/11/11 23:27:39 INFO Received complete push request method=POST path=/42822026/11/11 23:27:39 INFO Received uploads request method=POST path=/42832026/11/11 23:27:39 INFO Completed push id=20 objects=8 duration=277.917µs42842026/11/11 23:27:39 INFO Vacuumed table table=closures42852026/11/11 23:27:39 INFO Received complete push request method=POST path=/42862026/11/11 23:27:39 INFO Completed push id=16 objects=26 duration=508.833µs42872026/11/11 23:27:39 INFO Completed push id=32 objects=6 duration=327.917µs42882026/11/11 23:27:39 INFO Vacuumed table table=objects42892026/11/11 23:27:39 INFO Received uploads request method=POST path=/42902026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=042912026/11/11 23:27:39 INFO Received complete upload request method=POST path=/42922026/11/11 23:27:39 INFO Received uploads request method=POST path=/42932026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=042942026/11/11 23:27:39 INFO Received push request method=POST path=/42952026/11/11 23:27:39 WARN Force mode enabled - objects will be deleted immediately without grace period42962026/11/11 23:27:39 INFO Completed upload id=33 objects=7 duration=382.25µs42972026/11/11 23:27:39 WARN Force mode enabled - objects will be deleted immediately without grace period42982026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042992026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=043002026/11/11 23:27:39 INFO Received complete push request method=POST path=/43012026/11/11 23:27:39 INFO Received complete upload request method=POST path=/43022026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043032026/11/11 23:27:39 INFO Vacuumed table table=pending_closures43042026/11/11 23:27:39 INFO Completed upload id=34 objects=3 duration=256.666µs43052026/11/11 23:27:39 INFO Vacuumed table table=pending_objects43062026/11/11 23:27:39 INFO Vacuumed table table=pending_closures43072026/11/11 23:27:39 INFO Completed push id=27 objects=11 duration=5.677667ms43082026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads43092026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043102026/11/11 23:27:39 INFO Vacuumed table table=closures43112026/11/11 23:27:39 INFO Vacuumed table table=pending_closures43122026/11/11 23:27:39 INFO Vacuumed table table=pending_objects43132026/11/11 23:27:39 INFO Received complete push request method=POST path=/43142026/11/11 23:27:39 INFO Vacuumed table table=objects43152026/11/11 23:27:39 INFO Received complete upload request method=POST path=/43162026/11/11 23:27:39 INFO Completed push id=35 objects=11 duration=320.834µs43172026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads43182026/11/11 23:27:39 INFO Vacuumed table table=pending_objects43192026/11/11 23:27:39 INFO Received complete push request method=POST path=/43202026/11/11 23:27:39 INFO Completed upload id=29 objects=26 duration=294.667µs43212026/11/11 23:27:39 INFO Vacuumed table table=closures43222026/11/11 23:27:39 INFO Received complete upload request method=POST path=/43232026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads43242026/11/11 23:27:39 INFO Completed push id=27 objects=6 duration=1.030166ms43252026/11/11 23:27:39 INFO Received complete push request method=POST path=/43262026/11/11 23:27:39 INFO Vacuumed table table=objects43272026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=043282026/11/11 23:27:39 INFO Completed upload id=7 objects=3 duration=267.291µs43292026/11/11 23:27:39 INFO Vacuumed table table=closures43302026/11/11 23:27:39 INFO Completed push id=30 objects=8 duration=310.25µs43312026/11/11 23:27:39 INFO Vacuumed table table=objects43322026/11/11 23:27:39 WARN Force mode enabled - objects will be deleted immediately without grace period43332026/11/11 23:27:39 INFO Received complete upload request method=POST path=/43342026/11/11 23:27:39 INFO Received complete upload request method=POST path=/43352026/11/11 23:27:39 INFO Received complete upload request method=POST path=/43362026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=6 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043372026/11/11 23:27:39 INFO Completed upload id=19 objects=2 duration=727µs43382026/11/11 23:27:39 INFO Received push request method=POST path=/43392026/11/11 23:27:39 INFO Received complete upload request method=POST path=/43402026/11/11 23:27:39 INFO Completed upload id=15 objects=2 duration=500.458µs43412026/11/11 23:27:39 INFO Completed upload id=7 objects=10 duration=496.541µs43422026/11/11 23:27:39 INFO Received uploads request method=POST path=/43432026/11/11 23:27:39 INFO Received uploads request method=POST path=/43442026/11/11 23:27:39 INFO Completed upload id=31 objects=5 duration=229.542µs43452026/11/11 23:27:39 INFO Vacuumed table table=pending_closures43462026/11/11 23:27:39 INFO Received complete push request method=POST path=/43472026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=043482026/11/11 23:27:39 INFO Completed push id=33 objects=7 duration=646.583µs43492026/11/11 23:27:39 INFO Received complete push request method=POST path=/43502026/11/11 23:27:39 INFO Received push request method=POST path=/43512026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043522026/11/11 23:27:39 INFO Completed push id=32 objects=3 duration=282.083µs43532026/11/11 23:27:39 INFO Received complete upload request method=POST path=/43542026/11/11 23:27:39 INFO Completed upload id=26 objects=3 duration=292.708µs43552026/11/11 23:27:39 INFO Vacuumed table table=pending_objects43562026/11/11 23:27:39 INFO Vacuumed table table=pending_closures43572026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads43582026/11/11 23:27:39 INFO Vacuumed table table=pending_objects43592026/11/11 23:27:39 INFO Vacuumed table table=closures43602026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=043612026/11/11 23:27:39 INFO Aborted multipart uploads count=0 kept=043622026/11/11 23:27:39 WARN Force mode enabled - objects will be deleted immediately without grace period43632026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads43642026/11/11 23:27:39 INFO Vacuumed table table=objects43652026/11/11 23:27:39 INFO Vacuumed table table=closures43662026/11/11 23:27:39 WARN Force mode enabled - objects will be deleted immediately without grace period43672026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043682026/11/11 23:27:39 INFO Received uploads request method=POST path=/43692026/11/11 23:27:39 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043702026/11/11 23:27:39 INFO Vacuumed table table=objects43712026/11/11 23:27:39 INFO Vacuumed table table=pending_closures43722026/11/11 23:27:39 INFO Received uploads request method=POST path=/43732026/11/11 23:27:39 INFO Vacuumed table table=pending_objects43742026/11/11 23:27:39 INFO Vacuumed table table=multipart_uploads43752026/11/11 23:27:39 INFO Vacuumed table table=pending_closures43762026/11/11 23:27:39 INFO Received complete upload request method=POST path=/43772026/11/11 23:27:39 INFO Received complete push request method=POST path=/43782026/11/11 23:27:39 INFO Vacuumed table table=closures43792026/11/11 23:27:39 INFO Completed upload id=26 objects=8 duration=565.416µs43802026/11/11 23:27:39 INFO Received complete push request method=POST path=/43812026/11/11 23:27:39 INFO Vacuumed table table=objects43822026/11/11 23:27:39 INFO Completed push id=22 objects=3 duration=1.332917ms43832026/11/11 23:27:39 INFO Completed push id=18 objects=23 duration=630.959µs43842026/11/11 23:27:39 INFO Vacuumed table table=pending_objects43852026/11/11 23:27:40 INFO Received complete push request method=POST path=/43862026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads43872026/11/11 23:27:40 INFO Received uploads request method=POST path=/43882026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=043892026/11/11 23:27:40 INFO Vacuumed table table=closures43902026/11/11 23:27:40 INFO Completed push id=28 objects=14 duration=7.219958ms43912026/11/11 23:27:40 INFO Vacuumed table table=objects43922026/11/11 23:27:40 WARN Force mode enabled - objects will be deleted immediately without grace period43932026/11/11 23:27:40 INFO Received push request method=POST path=/43942026/11/11 23:27:40 INFO Received uploads request method=POST path=/43952026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=043962026/11/11 23:27:40 WARN Force mode enabled - objects will be deleted immediately without grace period43972026/11/11 23:27:40 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043982026/11/11 23:27:40 INFO Vacuumed table table=pending_closures43992026/11/11 23:27:40 INFO Received complete push request method=POST path=/44002026/11/11 23:27:40 INFO Vacuumed table table=pending_objects44012026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads44022026/11/11 23:27:40 INFO Completed push id=5 objects=9 duration=10.753666ms44032026/11/11 23:27:40 INFO Vacuumed table table=closures44042026/11/11 23:27:40 INFO Received push request method=POST path=/44052026/11/11 23:27:40 INFO Vacuumed table table=objects44062026/11/11 23:27:40 INFO Received push request method=POST path=/44072026/11/11 23:27:40 INFO Received uploads request method=POST path=/44082026/11/11 23:27:40 INFO Received uploads request method=POST path=/44092026/11/11 23:27:40 INFO Received complete push request method=POST path=/44102026/11/11 23:27:40 INFO Completed push id=11 objects=9 duration=582.833µs44112026/11/11 23:27:40 INFO Received uploads request method=POST path=/44122026/11/11 23:27:40 INFO Received complete upload request method=POST path=/44132026/11/11 23:27:40 INFO Received push request method=POST path=/44142026/11/11 23:27:40 INFO Completed upload id=21 objects=9 duration=990.166µs44152026/11/11 23:27:40 INFO Received push request method=POST path=/44162026/11/11 23:27:40 INFO Received push request method=POST path=/44172026/11/11 23:27:40 INFO Received uploads request method=POST path=/44182026/11/11 23:27:40 INFO Received complete push request method=POST path=/44192026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=044202026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=044212026/11/11 23:27:40 INFO Completed push id=25 objects=10 duration=487.167µs44222026/11/11 23:27:40 WARN Force mode enabled - objects will be deleted immediately without grace period44232026/11/11 23:27:40 WARN Force mode enabled - objects will be deleted immediately without grace period44242026/11/11 23:27:40 INFO Received complete upload request method=POST path=/44252026/11/11 23:27:40 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044262026/11/11 23:27:40 INFO Completed upload id=29 objects=5 duration=307.458µs44272026/11/11 23:27:40 INFO Vacuumed table table=pending_closures44282026/11/11 23:27:40 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044292026/11/11 23:27:40 INFO Vacuumed table table=pending_objects44302026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads44312026/11/11 23:27:40 INFO Vacuumed table table=pending_closures44322026/11/11 23:27:40 INFO Received complete upload request method=POST path=/44332026/11/11 23:27:40 INFO Completed upload id=30 objects=2 duration=229.208µs44342026/11/11 23:27:40 INFO Vacuumed table table=pending_objects44352026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads44362026/11/11 23:27:40 INFO Received complete push request method=POST path=/44372026/11/11 23:27:40 INFO Vacuumed table table=closures44382026/11/11 23:27:40 INFO Completed push id=31 objects=20 duration=340.459µs44392026/11/11 23:27:40 INFO Vacuumed table table=objects44402026/11/11 23:27:40 INFO Vacuumed table table=closures44412026/11/11 23:27:40 INFO Received complete upload request method=POST path=/44422026/11/11 23:27:40 INFO Received uploads request method=POST path=/44432026/11/11 23:27:40 INFO Vacuumed table table=objects44442026/11/11 23:27:40 INFO Completed upload id=32 objects=5 duration=251.292µs44452026/11/11 23:27:40 INFO Received complete upload request method=POST path=/44462026/11/11 23:27:40 INFO Completed upload id=33 objects=3 duration=183.791µs44472026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=044482026/11/11 23:27:40 WARN Force mode enabled - objects will be deleted immediately without grace period44492026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=044502026/11/11 23:27:40 INFO Received push request method=POST path=/44512026/11/11 23:27:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044522026/11/11 23:27:40 WARN Force mode enabled - objects will be deleted immediately without grace period44532026/11/11 23:27:40 INFO Received complete upload request method=POST path=/44542026/11/11 23:27:40 INFO Vacuumed table table=pending_closures44552026/11/11 23:27:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044562026/11/11 23:27:40 INFO Completed upload id=23 objects=15 duration=6.380584ms44572026/11/11 23:27:40 INFO Vacuumed table table=pending_objects44582026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads44592026/11/11 23:27:40 INFO Vacuumed table table=closures44602026/11/11 23:27:40 INFO Vacuumed table table=objects44612026/11/11 23:27:40 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."44622026/11/11 23:27:40 INFO Vacuumed table table=pending_closures44632026/11/11 23:27:40 INFO Received uploads request method=POST path=/44642026/11/11 23:27:40 INFO Received uploads request method=POST path=/44652026/11/11 23:27:40 INFO Vacuumed table table=pending_objects44662026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads44672026/11/11 23:27:40 INFO Vacuumed table table=closures44682026/11/11 23:27:40 INFO Vacuumed table table=objects44692026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=044702026/11/11 23:27:40 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."44712026/11/11 23:27:40 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."44722026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=044732026/11/11 23:27:40 WARN Force mode enabled - objects will be deleted immediately without grace period44742026/11/11 23:27:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=044752026/11/11 23:27:40 INFO Vacuumed table table=pending_closures44762026/11/11 23:27:40 INFO Received complete upload request method=POST path=/44772026/11/11 23:27:40 INFO Vacuumed table table=pending_objects44782026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads44792026/11/11 23:27:40 INFO Completed upload id=19 objects=6 duration=10.411709ms44802026/11/11 23:27:40 INFO Vacuumed table table=closures44812026/11/11 23:27:40 INFO Vacuumed table table=objects44822026/11/11 23:27:40 INFO Received complete upload request method=POST path=/44832026/11/11 23:27:40 INFO Completed upload id=21 objects=7 duration=1.271875ms44842026/11/11 23:27:40 INFO Received push request method=POST path=/44852026/11/11 23:27:40 INFO Received uploads request method=POST path=/44862026/11/11 23:27:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=044872026/11/11 23:27:40 INFO Vacuumed table table=pending_closures44882026/11/11 23:27:40 INFO Received uploads request method=POST path=/44892026/11/11 23:27:40 INFO Vacuumed table table=pending_objects44902026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads44912026/11/11 23:27:40 INFO Received complete upload request method=POST path=/44922026/11/11 23:27:40 INFO Vacuumed table table=closures44932026/11/11 23:27:40 INFO Received complete upload request method=POST path=/44942026/11/11 23:27:40 INFO Completed upload id=27 objects=4 duration=319.375µs44952026/11/11 23:27:40 INFO Completed upload id=11 objects=7 duration=5.39275ms44962026/11/11 23:27:40 INFO Vacuumed table table=objects44972026/11/11 23:27:40 INFO Received complete push request method=POST path=/44982026/11/11 23:27:40 INFO Received complete push request method=POST path=/44992026/11/11 23:27:40 INFO Completed push id=13 objects=11 duration=537.375µs45002026/11/11 23:27:40 INFO Received uploads request method=POST path=/45012026/11/11 23:27:40 INFO Received complete upload request method=POST path=/45022026/11/11 23:27:40 INFO Completed push id=11 objects=11 duration=1.822583ms45032026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=045042026/11/11 23:27:40 INFO Completed upload id=22 objects=7 duration=343.5µs45052026/11/11 23:27:40 WARN Force mode enabled - objects will be deleted immediately without grace period45062026/11/11 23:27:40 INFO Received uploads request method=POST path=/45072026/11/11 23:27:40 INFO Received uploads request method=POST path=/45082026/11/11 23:27:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045092026/11/11 23:27:40 INFO Vacuumed table table=pending_closures45102026/11/11 23:27:40 INFO Vacuumed table table=pending_objects45112026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads45122026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=045132026/11/11 23:27:40 INFO Vacuumed table table=closures45142026/11/11 23:27:40 INFO Received complete push request method=POST path=/45152026/11/11 23:27:40 INFO Vacuumed table table=objects45162026/11/11 23:27:40 INFO Completed push id=24 objects=9 duration=5.695208ms45172026/11/11 23:27:40 INFO Received complete push request method=POST path=/45182026/11/11 23:27:40 WARN Force mode enabled - objects will be deleted immediately without grace period45192026/11/11 23:27:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045202026/11/11 23:27:40 INFO Completed push id=15 objects=24 duration=2.409292ms45212026/11/11 23:27:40 INFO Received complete upload request method=POST path=/45222026/11/11 23:27:40 INFO Received push request method=POST path=/45232026/11/11 23:27:40 INFO Vacuumed table table=pending_closures45242026/11/11 23:27:40 INFO Completed upload id=15 objects=3 duration=540.291µs45252026/11/11 23:27:40 INFO Received uploads request method=POST path=/45262026/11/11 23:27:40 INFO Received push request method=POST path=/45272026/11/11 23:27:40 INFO Vacuumed table table=pending_objects45282026/11/11 23:27:40 INFO Received uploads request method=POST path=/45292026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads45302026/11/11 23:27:40 INFO Vacuumed table table=closures45312026/11/11 23:27:40 INFO Vacuumed table table=objects45322026/11/11 23:27:40 INFO Received uploads request method=POST path=/45332026/11/11 23:27:40 INFO Received push request method=POST path=/45342026/11/11 23:27:40 INFO Received uploads request method=POST path=/45352026/11/11 23:27:40 INFO Received uploads request method=POST path=/45362026/11/11 23:27:40 INFO Received complete push request method=POST path=/45372026/11/11 23:27:40 INFO Completed push id=21 objects=3 duration=412.708µs45382026/11/11 23:27:40 INFO Received complete push request method=POST path=/45392026/11/11 23:27:40 INFO Received uploads request method=POST path=/45402026/11/11 23:27:40 INFO Completed push id=9 objects=9 duration=6.138583ms45412026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=045422026/11/11 23:27:40 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=3 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045432026/11/11 23:27:40 INFO Received complete push request method=POST path=/45442026/11/11 23:27:40 INFO Vacuumed table table=pending_closures45452026/11/11 23:27:40 INFO Vacuumed table table=pending_objects45462026/11/11 23:27:40 INFO Completed push id=16 objects=14 duration=6.072708ms45472026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads45482026/11/11 23:27:40 INFO Received complete upload request method=POST path=/45492026/11/11 23:27:40 INFO Vacuumed table table=closures45502026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=045512026/11/11 23:27:40 INFO Completed upload id=6 objects=6 duration=206.708µs45522026/11/11 23:27:40 WARN Force mode enabled - objects will be deleted immediately without grace period45532026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=045542026/11/11 23:27:40 INFO Vacuumed table table=objects45552026/11/11 23:27:40 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045562026/11/11 23:27:40 WARN Force mode enabled - objects will be deleted immediately without grace period45572026/11/11 23:27:40 INFO Vacuumed table table=pending_closures45582026/11/11 23:27:40 INFO Received complete upload request method=POST path=/45592026/11/11 23:27:40 INFO Vacuumed table table=pending_objects45602026/11/11 23:27:40 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045612026/11/11 23:27:40 INFO Received push request method=POST path=/45622026/11/11 23:27:40 INFO Completed upload id=21 objects=8 duration=242.75µs45632026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads45642026/11/11 23:27:40 INFO Vacuumed table table=closures45652026/11/11 23:27:40 INFO Vacuumed table table=pending_closures45662026/11/11 23:27:40 INFO Vacuumed table table=objects45672026/11/11 23:27:40 INFO Vacuumed table table=pending_objects45682026/11/11 23:27:40 INFO Received complete push request method=POST path=/45692026/11/11 23:27:40 INFO Received push request method=POST path=/45702026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads45712026/11/11 23:27:40 INFO Completed push id=22 objects=6 duration=381.708µs45722026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=045732026/11/11 23:27:40 INFO Vacuumed table table=closures45742026/11/11 23:27:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045752026/11/11 23:27:40 INFO Received complete push request method=POST path=/45762026/11/11 23:27:40 INFO Vacuumed table table=pending_closures45772026/11/11 23:27:40 INFO Completed push id=24 objects=8 duration=920.75µs45782026/11/11 23:27:40 INFO Received complete push request method=POST path=/45792026/11/11 23:27:40 INFO Vacuumed table table=objects45802026/11/11 23:27:40 INFO Vacuumed table table=pending_objects45812026/11/11 23:27:40 INFO Completed push id=30 objects=4 duration=6.686708ms45822026/11/11 23:27:40 INFO Received complete upload request method=POST path=/45832026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads45842026/11/11 23:27:40 INFO Completed upload id=25 objects=9 duration=235µs45852026/11/11 23:27:40 INFO Received push request method=POST path=/45862026/11/11 23:27:40 INFO Vacuumed table table=closures45872026/11/11 23:27:40 INFO Received uploads request method=POST path=/45882026/11/11 23:27:40 INFO Received complete upload request method=POST path=/45892026/11/11 23:27:40 INFO Completed upload id=27 objects=3 duration=192.917µs45902026/11/11 23:27:40 INFO Vacuumed table table=objects45912026/11/11 23:27:40 INFO Received uploads request method=POST path=/45922026/11/11 23:27:40 INFO Received complete upload request method=POST path=/45932026/11/11 23:27:40 INFO Received push request method=POST path=/45942026/11/11 23:27:40 INFO Received push request method=POST path=/45952026/11/11 23:27:40 INFO Completed upload id=28 objects=6 duration=154.208µs45962026/11/11 23:27:40 INFO Received complete push request method=POST path=/45972026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=045982026/11/11 23:27:40 INFO Received uploads request method=POST path=/45992026/11/11 23:27:40 INFO Completed push id=30 objects=4 duration=600.208µs46002026/11/11 23:27:40 WARN Force mode enabled - objects will be deleted immediately without grace period46012026/11/11 23:27:40 INFO Received push request method=POST path=/46022026/11/11 23:27:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046032026/11/11 23:27:40 INFO Vacuumed table table=pending_closures46042026/11/11 23:27:40 INFO Received complete push request method=POST path=/46052026/11/11 23:27:40 INFO Received complete push request method=POST path=/46062026/11/11 23:27:40 INFO Received complete upload request method=POST path=/46072026/11/11 23:27:40 INFO Completed push id=31 objects=7 duration=237µs46082026/11/11 23:27:40 INFO Vacuumed table table=pending_objects46092026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads46102026/11/11 23:27:40 INFO Completed upload id=25 objects=6 duration=6.458916ms46112026/11/11 23:27:40 INFO Completed push id=32 objects=7 duration=6.51325ms46122026/11/11 23:27:40 INFO Vacuumed table table=closures46132026/11/11 23:27:40 INFO Received complete upload request method=POST path=/46142026/11/11 23:27:40 INFO Vacuumed table table=objects46152026/11/11 23:27:40 INFO Completed upload id=32 objects=5 duration=311.875µs46162026/11/11 23:27:40 INFO Received complete push request method=POST path=/46172026/11/11 23:27:40 INFO Received push request method=POST path=/46182026/11/11 23:27:40 INFO Received complete push request method=POST path=/46192026/11/11 23:27:40 INFO Completed push id=26 objects=4 duration=489.75µs46202026/11/11 23:27:40 INFO Received uploads request method=POST path=/46212026/11/11 23:27:40 INFO Received push request method=POST path=/46222026/11/11 23:27:40 INFO Completed push id=4 objects=6 duration=688.042µs46232026/11/11 23:27:40 INFO Received complete upload request method=POST path=/46242026/11/11 23:27:40 INFO Received complete push request method=POST path=/46252026/11/11 23:27:40 INFO Received uploads request method=POST path=/46262026/11/11 23:27:40 INFO Completed upload id=28 objects=13 duration=343.583µs46272026/11/11 23:27:40 INFO Completed push id=11 objects=9 duration=398.542µs46282026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=046292026/11/11 23:27:40 WARN Force mode enabled - objects will be deleted immediately without grace period46302026/11/11 23:27:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046312026/11/11 23:27:40 INFO Received complete push request method=POST path=/46322026/11/11 23:27:40 INFO Vacuumed table table=pending_closures46332026/11/11 23:27:40 INFO Completed push id=29 objects=3 duration=353.459µs46342026/11/11 23:27:40 INFO Vacuumed table table=pending_objects46352026/11/11 23:27:40 INFO Vacuumed table table=multipart_uploads46362026/11/11 23:27:40 INFO Vacuumed table table=closures46372026/11/11 23:27:40 INFO Received complete upload request method=POST path=/46382026/11/11 23:27:40 INFO Vacuumed table table=objects46392026/11/11 23:27:40 INFO Completed upload id=31 objects=2 duration=3.85325ms46402026/11/11 23:27:40 INFO Received complete push request method=POST path=/46412026/11/11 23:27:40 INFO Aborted multipart uploads count=0 kept=046422026/11/11 23:27:40 INFO Completed push id=33 objects=6 duration=220.625µs46432026/11/11 23:27:41 WARN Force mode enabled - objects will be deleted immediately without grace period46442026/11/11 23:27:41 INFO Received complete push request method=POST path=/46452026/11/11 23:27:41 INFO Aborted multipart uploads count=0 kept=046462026/11/11 23:27:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046472026/11/11 23:27:41 INFO Completed push id=15 objects=24 duration=375.791µs46482026/11/11 23:27:41 INFO Vacuumed table table=pending_closures46492026/11/11 23:27:41 WARN Force mode enabled - objects will be deleted immediately without grace period46502026/11/11 23:27:41 INFO Vacuumed table table=pending_objects46512026/11/11 23:27:41 INFO Received complete upload request method=POST path=/46522026/11/11 23:27:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046532026/11/11 23:27:41 INFO Vacuumed table table=multipart_uploads46542026/11/11 23:27:41 INFO Completed upload id=19 objects=2 duration=4.005959ms46552026/11/11 23:27:41 INFO Vacuumed table table=closures46562026/11/11 23:27:41 INFO Received complete push request method=POST path=/46572026/11/11 23:27:41 INFO Vacuumed table table=pending_closures46582026/11/11 23:27:41 INFO Vacuumed table table=objects46592026/11/11 23:27:41 INFO Received complete upload request method=POST path=/46602026/11/11 23:27:41 INFO Completed upload id=24 objects=7 duration=388.791µs46612026/11/11 23:27:41 INFO Completed push id=17 objects=22 duration=837.291µs46622026/11/11 23:27:41 INFO Received push request method=POST path=/46632026/11/11 23:27:41 INFO Received uploads request method=POST path=/46642026/11/11 23:27:41 INFO Received complete push request method=POST path=/46652026/11/11 23:27:41 INFO Vacuumed table table=pending_objects46662026/11/11 23:27:41 INFO Vacuumed table table=multipart_uploads46672026/11/11 23:27:41 INFO Completed push id=26 objects=3 duration=253.459µs46682026/11/11 23:27:41 INFO Vacuumed table table=closures46692026/11/11 23:27:41 INFO Received complete upload request method=POST path=/46702026/11/11 23:27:41 INFO Completed upload id=28 objects=8 duration=211.584µs46712026/11/11 23:27:41 INFO Vacuumed table table=objects46722026/11/11 23:27:41 INFO Received complete upload request method=POST path=/46732026/11/11 23:27:41 INFO Received complete upload request method=POST path=/46742026/11/11 23:27:41 INFO Received push request method=POST path=/46752026/11/11 23:27:41 INFO Completed upload id=29 objects=10 duration=431.375µs46762026/11/11 23:27:41 INFO Completed upload id=9 objects=2 duration=600.333µs46772026/11/11 23:27:41 INFO Received complete push request method=POST path=/46782026/11/11 23:27:41 INFO Completed push id=19 objects=14 duration=525.417µs46792026/11/11 23:27:41 INFO Received complete push request method=POST path=/46802026/11/11 23:27:41 INFO Received complete upload request method=POST path=/46812026/11/11 23:27:41 INFO Completed upload id=30 objects=20 duration=345.916µs46822026/11/11 23:27:41 INFO Completed push id=8 objects=3 duration=536.25µs46832026/11/11 23:27:41 INFO Received uploads request method=POST path=/46842026/11/11 23:27:41 INFO Received complete push request method=POST path=/46852026/11/11 23:27:41 INFO Received push request method=POST path=/46862026/11/11 23:27:41 INFO Completed push id=31 objects=11 duration=310.417µs46872026/11/11 23:27:41 INFO Received uploads request method=POST path=/46882026/11/11 23:27:41 INFO Received complete upload request method=POST path=/46892026/11/11 23:27:41 INFO Received complete upload request method=POST path=/46902026/11/11 23:27:41 INFO Completed upload id=13 objects=10 duration=6.430292ms46912026/11/11 23:27:41 INFO Completed upload id=32 objects=2 duration=197.792µs46922026/11/11 23:27:41 INFO Aborted multipart uploads count=0 kept=046932026/11/11 23:27:41 WARN Force mode enabled - objects will be deleted immediately without grace period46942026/11/11 23:27:41 INFO Received complete push request method=POST path=/46952026/11/11 23:27:41 INFO Completed push id=33 objects=3 duration=230.584µs46962026/11/11 23:27:41 INFO Received complete upload request method=POST path=/46972026/11/11 23:27:41 INFO Completed upload id=34 objects=3 duration=188.833µs46982026/11/11 23:27:41 INFO Aborted multipart uploads count=0 kept=046992026/11/11 23:27:41 WARN Force mode enabled - objects will be deleted immediately without grace period47002026/11/11 23:27:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047012026/11/11 23:27:41 INFO Received complete upload request method=POST path=/47022026/11/11 23:27:41 INFO Vacuumed table table=pending_closures47032026/11/11 23:27:41 INFO Completed upload id=20 objects=6 duration=223.208µs47042026/11/11 23:27:41 INFO Received uploads request method=POST path=/47052026/11/11 23:27:41 INFO Vacuumed table table=pending_objects47062026/11/11 23:27:41 INFO Vacuumed table table=multipart_uploads47072026/11/11 23:27:41 INFO Received complete upload request method=POST path=/47082026/11/11 23:27:41 INFO Received complete upload request method=POST path=/47092026/11/11 23:27:41 INFO Completed upload id=21 objects=2 duration=170.875µs47102026/11/11 23:27:41 INFO Vacuumed table table=closures47112026/11/11 23:27:41 INFO Completed upload id=23 objects=16 duration=4.325083ms47122026/11/11 23:27:41 INFO Vacuumed table table=objects47132026/11/11 23:27:41 INFO Received uploads request method=POST path=/47142026/11/11 23:27:41 INFO Aborted multipart uploads count=0 kept=047152026/11/11 23:27:41 WARN Force mode enabled - objects will be deleted immediately without grace period47162026/11/11 23:27:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047172026/11/11 23:27:41 INFO Vacuumed table table=pending_closures47182026/11/11 23:27:41 INFO Received complete upload request method=POST path=/47192026/11/11 23:27:41 INFO Vacuumed table table=pending_objects47202026/11/11 23:27:41 INFO Completed upload id=34 objects=24 duration=6.055ms47212026/11/11 23:27:41 INFO Vacuumed table table=multipart_uploads47222026/11/11 23:27:41 INFO Aborted multipart uploads count=0 kept=047232026/11/11 23:27:41 INFO Vacuumed table table=closures47242026/11/11 23:27:41 WARN Force mode enabled - objects will be deleted immediately without grace period47252026/11/11 23:27:41 INFO Vacuumed table table=objects47262026/11/11 23:27:41 INFO Aborted multipart uploads count=0 kept=047272026/11/11 23:27:41 WARN Force mode enabled - objects will be deleted immediately without grace period47282026/11/11 23:27:41 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047292026/11/11 23:27:41 INFO Received uploads request method=POST path=/47302026/11/11 23:27:41 INFO Vacuumed table table=pending_closures47312026/11/11 23:27:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047322026/11/11 23:27:41 INFO Vacuumed table table=pending_objects47332026/11/11 23:27:41 INFO Vacuumed table table=multipart_uploads47342026/11/11 23:27:41 INFO Vacuumed table table=closures47352026/11/11 23:27:41 INFO Vacuumed table table=pending_closures47362026/11/11 23:27:41 INFO Vacuumed table table=objects47372026/11/11 23:27:41 INFO Vacuumed table table=pending_objects47382026/11/11 23:27:41 INFO Vacuumed table table=multipart_uploads47392026/11/11 23:27:41 INFO Vacuumed table table=closures47402026/11/11 23:27:41 INFO Vacuumed table table=objects47412026/11/11 23:27:41 INFO Received uploads request method=POST path=/47422026/11/11 23:27:41 INFO Received push request method=POST path=/47432026/11/11 23:27:41 INFO Received complete push request method=POST path=/47442026/11/11 23:27:41 INFO Completed push id=24 objects=24 duration=864.125µs47452026/11/11 23:27:41 INFO Aborted multipart uploads count=0 kept=047462026/11/11 23:27:41 WARN Force mode enabled - objects will be deleted immediately without grace period47472026/11/11 23:27:41 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047482026/11/11 23:27:41 INFO Vacuumed table table=pending_closures47492026/11/11 23:27:41 INFO Vacuumed table table=pending_objects47502026/11/11 23:27:41 INFO Vacuumed table table=multipart_uploads47512026/11/11 23:27:41 INFO Vacuumed table table=closures47522026/11/11 23:27:41 INFO Vacuumed table table=objects47532026/11/11 23:27:41 INFO Received uploads request method=POST path=/47542026/11/11 23:27:41 INFO Received uploads request method=POST path=/47552026/11/11 23:27:41 INFO Received push request method=POST path=/47562026/11/11 23:27:41 INFO Received uploads request method=POST path=/47572026/11/11 23:27:41 INFO Aborted multipart uploads count=0 kept=047582026/11/11 23:27:41 WARN Force mode enabled - objects will be deleted immediately without grace period47592026/11/11 23:27:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047602026/11/11 23:27:41 INFO Vacuumed table table=pending_closures47612026/11/11 23:27:41 INFO Vacuumed table table=pending_objects47622026/11/11 23:27:41 INFO Vacuumed table table=multipart_uploads47632026/11/11 23:27:41 INFO Vacuumed table table=closures47642026/11/11 23:27:41 INFO Vacuumed table table=objects47652026/11/11 23:27:41 INFO Received push request method=POST path=/47662026/11/11 23:27:41 INFO Received complete upload request method=POST path=/47672026/11/11 23:27:41 INFO Completed upload id=23 objects=7 duration=677.542µs47682026/11/11 23:27:41 INFO Received complete upload request method=POST path=/47692026/11/11 23:27:41 INFO Completed upload id=26 objects=5 duration=271.5µs47702026/11/11 23:27:41 INFO Received complete push request method=POST path=/47712026/11/11 23:27:41 INFO Completed push id=27 objects=3 duration=378.166µs47722026/11/11 23:27:41 INFO Received complete upload request method=POST path=/47732026/11/11 23:27:41 INFO Completed upload id=28 objects=5 duration=170.666µs47742026/11/11 23:27:41 INFO Received complete push request method=POST path=/47752026/11/11 23:27:41 INFO Completed push id=29 objects=17 duration=226.875µs47762026/11/11 23:27:41 INFO Aborted multipart uploads count=0 kept=047772026/11/11 23:27:41 WARN Force mode enabled - objects will be deleted immediately without grace period47782026/11/11 23:27:41 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=11 objects-failed-to-delete=047792026/11/11 23:27:41 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047802026/11/11 23:27:41 INFO Vacuumed table table=pending_closures47812026/11/11 23:27:41 INFO Received complete push request method=POST path=/47822026/11/11 23:27:41 INFO Vacuumed table table=pending_closures47832026/11/11 23:27:41 INFO Vacuumed table table=pending_objects47842026/11/11 23:27:41 INFO Vacuumed table table=multipart_uploads47852026/11/11 23:27:41 INFO Completed push id=13 objects=15 duration=380µs47862026/11/11 23:27:41 INFO Vacuumed table table=closures47872026/11/11 23:27:41 INFO Vacuumed table table=pending_objects47882026/11/11 23:27:41 INFO Vacuumed table table=objects47892026/11/11 23:27:41 INFO Vacuumed table table=multipart_uploads47902026/11/11 23:27:41 INFO Vacuumed table table=closures47912026/11/11 23:27:41 INFO Vacuumed table table=objects47922026/11/11 23:27:41 INFO Received push request method=POST path=/47932026/11/11 23:27:41 INFO Aborted multipart uploads count=0 kept=047942026/11/11 23:27:41 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."47952026/11/11 23:27:41 INFO Received push request method=POST path=/47962026/11/11 23:27:41 INFO Received complete push request method=POST path=/47972026/11/11 23:27:41 INFO Completed push id=14 objects=4 duration=800.334µs47982026/11/11 23:27:41 INFO Received push request method=POST path=/47992026/11/11 23:27:41 INFO Received uploads request method=POST path=/48002026/11/11 23:27:41 INFO Aborted multipart uploads count=0 kept=048012026/11/11 23:27:41 WARN Force mode enabled - objects will be deleted immediately without grace period48022026/11/11 23:27:41 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048032026/11/11 23:27:41 INFO Vacuumed table table=pending_closures48042026/11/11 23:27:41 INFO Vacuumed table table=pending_objects48052026/11/11 23:27:41 INFO Vacuumed table table=multipart_uploads48062026/11/11 23:27:41 INFO Vacuumed table table=closures48072026/11/11 23:27:41 INFO Vacuumed table table=objects48082026/11/11 23:27:41 INFO Received complete push request method=POST path=/48092026/11/11 23:27:41 INFO Completed push id=7 objects=15 duration=692.5µs48102026/11/11 23:27:41 INFO Received complete push request method=POST path=/48112026/11/11 23:27:41 INFO Completed push id=20 objects=3 duration=364.208µs48122026/11/11 23:27:41 INFO Received push request method=POST path=/48132026/11/11 23:27:41 INFO Received push request method=POST path=/48142026/11/11 23:27:41 INFO Received push request method=POST path=/48152026/11/11 23:27:41 INFO Received push request method=POST path=/48162026/11/11 23:27:41 INFO Received push request method=POST path=/48172026/11/11 23:27:41 INFO Received complete upload request method=POST path=/48182026/11/11 23:27:41 INFO Completed upload id=22 objects=6 duration=191.417µs48192026/11/11 23:27:41 INFO Received uploads request method=POST path=/48202026/11/11 23:27:41 INFO Received uploads request method=POST path=/48212026/11/11 23:27:41 INFO Received push request method=POST path=/48222026/11/11 23:27:41 INFO Received push request method=POST path=/48232026/11/11 23:27:41 INFO Received uploads request method=POST path=/48242026/11/11 23:27:41 INFO Received uploads request method=POST path=/48252026/11/11 23:27:41 INFO Received uploads request method=POST path=/48262026/11/11 23:27:41 INFO Received uploads request method=POST path=/48272026/11/11 23:27:41 INFO Received uploads request method=POST path=/48282026/11/11 23:27:41 INFO Received push request method=POST path=/48292026/11/11 23:27:41 INFO Received complete push request method=POST path=/48302026/11/11 23:27:41 INFO Completed push id=16 objects=16 duration=1.054625ms48312026/11/11 23:27:41 INFO Aborted multipart uploads count=0 kept=048322026/11/11 23:27:41 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."48332026/11/11 23:27:41 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."48342026/11/11 23:27:41 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."48352026/11/11 23:27:41 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."48362026/11/11 23:27:41 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."48372026/11/11 23:27:41 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."48382026/11/11 23:27:41 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."48392026/11/11 23:27:41 INFO Received uploads request method=POST path=/48402026/11/11 23:27:41 INFO Received uploads request method=POST path=/48412026/11/11 23:27:41 INFO Received push request method=POST path=/48422026/11/11 23:27:41 INFO Received uploads request method=POST path=/48432026/11/11 23:27:41 INFO Aborted multipart uploads count=0 kept=048442026/11/11 23:27:41 WARN Force mode enabled - objects will be deleted immediately without grace period48452026/11/11 23:27:41 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."48462026/11/11 23:27:41 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."48472026/11/11 23:27:41 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."48482026/11/11 23:27:41 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."48492026/11/11 23:27:41 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."48502026/11/11 23:27:41 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."48512026/11/11 23:27:41 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."48522026/11/11 23:27:41 INFO Received complete upload request method=POST path=/48532026/11/11 23:27:41 INFO Completed upload id=21 objects=3 duration=215.541µs48542026/11/11 23:27:41 INFO Aborted multipart uploads count=0 kept=048552026/11/11 23:27:41 WARN Force mode enabled - objects will be deleted immediately without grace period48562026/11/11 23:27:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=7 objects-failed-to-delete=048572026/11/11 23:27:41 INFO Vacuumed table table=pending_closures48582026/11/11 23:27:41 INFO Vacuumed table table=pending_objects48592026/11/11 23:27:41 INFO Vacuumed table table=multipart_uploads48602026/11/11 23:27:41 INFO Vacuumed table table=closures48612026/11/11 23:27:41 INFO Vacuumed table table=objects48622026/11/11 23:27:41 INFO Aborted multipart uploads count=0 kept=048632026/11/11 23:27:41 WARN Force mode enabled - objects will be deleted immediately without grace period48642026/11/11 23:27:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048652026/11/11 23:27:41 INFO Vacuumed table table=pending_closures48662026/11/11 23:27:41 INFO Vacuumed table table=pending_objects48672026/11/11 23:27:41 INFO Vacuumed table table=multipart_uploads48682026/11/11 23:27:41 INFO Vacuumed table table=closures48692026/11/11 23:27:41 INFO Vacuumed table table=objects48702026/11/11 23:27:41 INFO Received push request method=POST path=/48712026/11/11 23:27:42 INFO Received complete push request method=POST path=/48722026/11/11 23:27:42 INFO Completed push id=22 objects=3 duration=627.041µs48732026/11/11 23:27:42 INFO Received uploads request method=POST path=/48742026/11/11 23:27:42 INFO Received push request method=POST path=/48752026/11/11 23:27:42 INFO Received complete push request method=POST path=/48762026/11/11 23:27:42 INFO Completed push id=24 objects=3 duration=270.5µs48772026/11/11 23:27:42 INFO Received push request method=POST path=/48782026/11/11 23:27:42 INFO Aborted multipart uploads count=0 kept=048792026/11/11 23:27:42 INFO Received complete push request method=POST path=/48802026/11/11 23:27:42 WARN Force mode enabled - objects will be deleted immediately without grace period48812026/11/11 23:27:42 INFO Completed push id=19 objects=14 duration=411.458µs48822026/11/11 23:27:42 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048832026/11/11 23:27:42 INFO Vacuumed table table=pending_closures48842026/11/11 23:27:42 INFO Vacuumed table table=pending_objects48852026/11/11 23:27:42 INFO Vacuumed table table=multipart_uploads48862026/11/11 23:27:42 INFO Vacuumed table table=closures48872026/11/11 23:27:42 INFO Vacuumed table table=objects48882026/11/11 23:27:42 INFO Aborted multipart uploads count=0 kept=048892026/11/11 23:27:42 WARN Force mode enabled - objects will be deleted immediately without grace period48902026/11/11 23:27:42 INFO Aborted multipart uploads count=0 kept=048912026/11/11 23:27:42 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=048922026/11/11 23:27:42 INFO Vacuumed table table=pending_closures48932026/11/11 23:27:42 INFO Received complete push request method=POST path=/48942026/11/11 23:27:42 INFO Vacuumed table table=pending_objects48952026/11/11 23:27:42 INFO Vacuumed table table=multipart_uploads48962026/11/11 23:27:42 INFO Vacuumed table table=closures48972026/11/11 23:27:42 INFO Completed push id=16 objects=7 duration=412.292µs48982026/11/11 23:27:42 INFO Vacuumed table table=objects48992026/11/11 23:27:42 INFO Received uploads request method=POST path=/49002026/11/11 23:27:42 INFO Received complete upload request method=POST path=/49012026/11/11 23:27:42 INFO Completed upload id=26 objects=2 duration=398.125µs49022026/11/11 23:27:42 INFO Aborted multipart uploads count=0 kept=049032026/11/11 23:27:42 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049042026/11/11 23:27:42 INFO Vacuumed table table=pending_closures49052026/11/11 23:27:42 INFO Vacuumed table table=pending_objects49062026/11/11 23:27:42 INFO Vacuumed table table=multipart_uploads49072026/11/11 23:27:42 INFO Vacuumed table table=closures49082026/11/11 23:27:42 INFO Vacuumed table table=objects49092026/11/11 23:27:42 INFO Received push request method=POST path=/49102026/11/11 23:27:42 INFO Aborted multipart uploads count=0 kept=049112026/11/11 23:27:42 INFO Received complete push request method=POST path=/49122026/11/11 23:27:42 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."49132026/11/11 23:27:42 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."49142026/11/11 23:27:42 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."49152026/11/11 23:27:42 INFO Completed push id=23 objects=6 duration=271.375µs49162026/11/11 23:27:42 INFO Received complete push request method=POST path=/49172026/11/11 23:27:42 INFO Received push request method=POST path=/49182026/11/11 23:27:42 INFO Completed push id=24 objects=6 duration=231.209µs49192026/11/11 23:27:42 INFO Received push request method=POST path=/49202026/11/11 23:27:42 INFO Received complete push request method=POST path=/49212026/11/11 23:27:42 INFO Completed push id=28 objects=11 duration=529.334µs49222026/11/11 23:27:42 INFO Received complete push request method=POST path=/49232026/11/11 23:27:42 INFO Completed push id=25 objects=7 duration=334.875µs49242026/11/11 23:27:42 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=049252026/11/11 23:27:42 INFO Vacuumed table table=pending_closures49262026-11-11 23:27:42.353 UTC [78886] LOG: skipping analyze of "pending_objects" --- lock not available49272026/11/11 23:27:42 INFO Vacuumed table table=pending_objects49282026/11/11 23:27:42 INFO Vacuumed table table=multipart_uploads49292026/11/11 23:27:42 INFO Vacuumed table table=closures49302026/11/11 23:27:42 INFO Vacuumed table table=objects49312026/11/11 23:27:42 INFO Received push request method=POST path=/49322026/11/11 23:27:42 INFO Received uploads request method=POST path=/49332026/11/11 23:27:42 INFO Aborted multipart uploads count=0 kept=049342026/11/11 23:27:42 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049352026/11/11 23:27:42 INFO Vacuumed table table=pending_closures49362026/11/11 23:27:42 INFO Vacuumed table table=pending_objects49372026/11/11 23:27:42 INFO Vacuumed table table=multipart_uploads49382026/11/11 23:27:42 INFO Vacuumed table table=closures49392026/11/11 23:27:42 INFO Received complete push request method=POST path=/49402026/11/11 23:27:42 INFO Vacuumed table table=objects49412026/11/11 23:27:42 INFO Completed push id=29 objects=24 duration=4.947625ms49422026/11/11 23:27:42 INFO Aborted multipart uploads count=0 kept=049432026/11/11 23:27:42 WARN Force mode enabled - objects will be deleted immediately without grace period49442026/11/11 23:27:42 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=049452026/11/11 23:27:42 INFO Vacuumed table table=pending_closures49462026/11/11 23:27:42 INFO Vacuumed table table=pending_objects49472026/11/11 23:27:42 INFO Vacuumed table table=multipart_uploads49482026/11/11 23:27:42 INFO Vacuumed table table=closures49492026/11/11 23:27:42 INFO Received complete upload request method=POST path=/49502026/11/11 23:27:42 INFO Vacuumed table table=objects49512026/11/11 23:27:42 INFO Completed upload id=26 objects=24 duration=463.5µs49522026/11/11 23:27:42 INFO Received complete upload request method=POST path=/49532026/11/11 23:27:42 INFO Completed upload id=27 objects=26 duration=1.009584ms49542026/11/11 23:27:42 INFO Received complete push request method=POST path=/49552026/11/11 23:27:42 INFO Completed push id=28 objects=6 duration=518.834µs49562026/11/11 23:27:42 INFO Received complete push request method=POST path=/49572026/11/11 23:27:42 INFO Completed push id=29 objects=6 duration=429.125µs49582026/11/11 23:27:42 INFO Received complete upload request method=POST path=/49592026/11/11 23:27:42 INFO Completed upload id=13 objects=14 duration=2.712084ms49602026/11/11 23:27:42 INFO Aborted multipart uploads count=0 kept=049612026/11/11 23:27:42 INFO Received complete upload request method=POST path=/49622026/11/11 23:27:42 INFO Completed upload id=30 objects=8 duration=390.125µs49632026/11/11 23:27:42 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049642026/11/11 23:27:42 INFO Vacuumed table table=pending_closures49652026/11/11 23:27:42 INFO Vacuumed table table=pending_objects49662026/11/11 23:27:42 INFO Vacuumed table table=multipart_uploads49672026/11/11 23:27:42 INFO Vacuumed table table=closures49682026/11/11 23:27:42 INFO Vacuumed table table=objects49692026/11/11 23:27:42 INFO Received complete push request method=POST path=/49702026/11/11 23:27:42 INFO Completed push id=31 objects=6 duration=1.649ms49712026/11/11 23:27:42 INFO Received complete upload request method=POST path=/49722026/11/11 23:27:42 INFO Completed upload id=31 objects=6 duration=281.5µs49732026/11/11 23:27:42 INFO Received uploads request method=POST path=/49742026/11/11 23:27:42 INFO Received complete upload request method=POST path=/49752026/11/11 23:27:42 INFO Completed upload id=32 objects=4 duration=232.458µs49762026/11/11 23:27:42 INFO Received complete upload request method=POST path=/49772026/11/11 23:27:42 INFO Completed upload id=33 objects=10 duration=302.958µs49782026/11/11 23:27:42 INFO Received complete upload request method=POST path=/49792026/11/11 23:27:42 INFO Completed upload id=34 objects=23 duration=376µs49802026/11/11 23:27:42 INFO Received complete push request method=POST path=/49812026/11/11 23:27:42 INFO Completed push id=35 objects=6 duration=1.00525ms49822026/11/11 23:27:42 INFO Aborted multipart uploads count=0 kept=049832026/11/11 23:27:42 WARN Force mode enabled - objects will be deleted immediately without grace period49842026/11/11 23:27:42 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=049852026/11/11 23:27:42 INFO Vacuumed table table=pending_closures49862026/11/11 23:27:42 INFO Vacuumed table table=pending_objects49872026/11/11 23:27:42 INFO Vacuumed table table=multipart_uploads49882026/11/11 23:27:42 INFO Vacuumed table table=closures49892026/11/11 23:27:42 INFO Vacuumed table table=objects49902026/11/11 23:27:43 INFO Received complete push request method=POST path=/49912026/11/11 23:27:43 INFO Completed push id=1 objects=9 duration=1.07025ms49922026/11/11 23:27:43 INFO Received complete upload request method=POST path=/49932026/11/11 23:27:43 INFO Completed upload id=2 objects=7 duration=1.381875ms49942026/11/11 23:27:43 INFO Received complete upload request method=POST path=/49952026/11/11 23:27:43 INFO Completed upload id=8 objects=6 duration=1.403666ms49962026/11/11 23:27:43 INFO Received complete upload request method=POST path=/49972026/11/11 23:27:43 INFO Completed upload id=12 objects=9 duration=1.101542ms49982026/11/11 23:27:43 INFO Received complete upload request method=POST path=/49992026/11/11 23:27:43 INFO Completed upload id=18 objects=22 duration=1.480708ms50002026/11/11 23:27:43 INFO Received complete upload request method=POST path=/50012026/11/11 23:27:43 INFO Completed upload id=21 objects=8 duration=664.292µs50022026/11/11 23:27:43 INFO Received complete upload request method=POST path=/50032026/11/11 23:27:43 INFO Completed upload id=22 objects=7 duration=602.75µs50042026/11/11 23:27:43 INFO Received complete push request method=POST path=/50052026/11/11 23:27:43 INFO Completed push id=24 objects=5 duration=534.25µs50062026/11/11 23:27:43 INFO Received complete upload request method=POST path=/50072026/11/11 23:27:43 INFO Completed upload id=25 objects=8 duration=321.292µs50082026/11/11 23:27:43 INFO Received complete push request method=POST path=/50092026/11/11 23:27:43 INFO Completed push id=26 objects=4 duration=361.042µs50102026/11/11 23:27:43 INFO Received complete push request method=POST path=/50112026/11/11 23:27:43 INFO Completed push id=30 objects=2 duration=284.5µs50122026/11/11 23:27:43 INFO Received complete push request method=POST path=/50132026/11/11 23:27:43 INFO Completed push id=32 objects=7 duration=315µs50142026/11/11 23:27:43 INFO Received complete upload request method=POST path=/50152026/11/11 23:27:43 INFO Completed upload id=33 objects=6 duration=201.125µs50162026/11/11 23:27:43 INFO Received complete upload request method=POST path=/50172026/11/11 23:27:43 INFO Completed upload id=34 objects=3 duration=177.417µs50182026/11/11 23:27:44 INFO Aborted multipart uploads count=0 kept=050192026/11/11 23:27:44 WARN Force mode enabled - objects will be deleted immediately without grace period50202026/11/11 23:27:44 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050212026/11/11 23:27:44 INFO Vacuumed table table=pending_closures50222026/11/11 23:27:44 INFO Vacuumed table table=pending_objects50232026/11/11 23:27:44 INFO Vacuumed table table=multipart_uploads50242026/11/11 23:27:44 INFO Vacuumed table table=closures50252026/11/11 23:27:44 INFO Vacuumed table table=objects5026--- PASS: FuzzPushGC (0.00s)5027 --- PASS: FuzzPushGC/seed#0 (8.10s)5028 --- PASS: FuzzPushGC/seed#11 (9.88s)5029 --- PASS: FuzzPushGC/seed#15 (9.95s)5030 --- PASS: FuzzPushGC/seed#7 (10.86s)5031 --- PASS: FuzzPushGC/seed#13 (11.26s)5032 --- PASS: FuzzPushGC/seed#4 (12.00s)5033 --- PASS: FuzzPushGC/seed#8 (12.49s)5034 --- PASS: FuzzPushGC/seed#3 (12.59s)5035 --- PASS: FuzzPushGC/seed#6 (13.03s)5036 --- PASS: FuzzPushGC/seed#14 (13.64s)5037 --- PASS: FuzzPushGC/seed#5 (13.81s)5038 --- PASS: FuzzPushGC/seed#9 (13.87s)5039 --- PASS: FuzzPushGC/seed#1 (14.17s)5040 --- PASS: FuzzPushGC/seed#2 (15.01s)5041 --- PASS: FuzzPushGC/seed#12 (15.54s)5042 --- PASS: FuzzPushGC/seed#10 (16.57s)5043PASS5044{"timestamp":"2026-11-11T23:27:44.011211Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:58288","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(9)"}5045{"timestamp":"2026-11-11T23:27:44.011239Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:58306","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(9)"}5046{"timestamp":"2026-11-11T23:27:44.011263Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:58314","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(7)"}5047{"timestamp":"2026-11-11T23:27:44.011501Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:58110","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(8)"}50482026-11-11 23:27:44.159 UTC [78319] LOG: received smart shutdown request50492026-11-11 23:27:44.160 UTC [78319] LOG: background worker "logical replication launcher" (PID 78329) exited with exit code 150502026-11-11 23:27:45.126 UTC [78324] LOG: checkpoint complete: wrote 3380 buffers (20.6%), wrote 5 SLRU buffers; 0 WAL file(s) added, 0 removed, 33 recycled; write=28.423 s, sync=0.836 s, total=29.308 s; sync files=42365, longest=0.020 s, average=0.001 s; distance=533468 kB, estimate=533468 kB; lsn=0/278BDFC8, redo lsn=0/220841A850512026-11-11 23:27:45.128 UTC [78324] LOG: shutting down50522026-11-11 23:27:45.128 UTC [78324] LOG: checkpoint starting: shutdown immediate50532026-11-11 23:27:45.624 UTC [78324] LOG: checkpoint complete: wrote 9052 buffers (55.2%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 5 recycled; write=0.442 s, sync=0.047 s, total=0.497 s; sync files=2975, longest=0.001 s, average=0.001 s; distance=90343 kB, estimate=489156 kB; lsn=0/278BE058, redo lsn=0/278BE05850542026-11-11 23:27:45.631 UTC [78319] LOG: database system is shut down5055Running OIDC tests...5056=== RUN TestAudienceForIssuer5057=== PAUSE TestAudienceForIssuer5058=== RUN TestGlobMatch5059=== PAUSE TestGlobMatch5060=== RUN TestValidateToken_ValidToken5061=== PAUSE TestValidateToken_ValidToken5062=== RUN TestValidateToken_WrongAudience5063=== PAUSE TestValidateToken_WrongAudience5064=== RUN TestValidateToken_Expired5065=== PAUSE TestValidateToken_Expired5066=== RUN TestValidateToken_BoundClaimsMismatch5067=== PAUSE TestValidateToken_BoundClaimsMismatch5068=== RUN TestValidateToken_BoundSubjectMismatch5069=== PAUSE TestValidateToken_BoundSubjectMismatch5070=== RUN TestValidateToken_MultipleProviders5071=== PAUSE TestValidateToken_MultipleProviders5072=== RUN TestValidateToken_NoMatchingProvider5073=== PAUSE TestValidateToken_NoMatchingProvider5074=== RUN TestValidateToken_KubernetesServiceAccount5075=== PAUSE TestValidateToken_KubernetesServiceAccount5076=== RUN TestNewValidator_KubernetesRequiresCA5077=== PAUSE TestNewValidator_KubernetesRequiresCA5078=== RUN TestValidateToken_KubernetesIssuerFromOwnToken5079=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken5080=== RUN TestPins_ReservedForMatchingRule5081=== PAUSE TestPins_ReservedForMatchingRule5082=== RUN TestPins_TopLevelShorthand5083=== PAUSE TestPins_TopLevelShorthand5084=== RUN TestPins_ConfigValidation5085=== PAUSE TestPins_ConfigValidation5086=== RUN TestScopes_LegacyProviderDefaultsToWrite5087=== PAUSE TestScopes_LegacyProviderDefaultsToWrite5088=== RUN TestScopes_Rules5089=== PAUSE TestScopes_Rules5090=== RUN TestScopes_ConfigValidation5091=== PAUSE TestScopes_ConfigValidation5092=== CONT TestAudienceForIssuer5093--- PASS: TestAudienceForIssuer (0.00s)5094=== CONT TestValidateToken_Expired5095=== CONT TestValidateToken_WrongAudience5096=== CONT TestValidateToken_KubernetesServiceAccount5097=== CONT TestValidateToken_BoundClaimsMismatch5098=== CONT TestPins_ConfigValidation5099=== CONT TestPins_ReservedForMatchingRule5100=== CONT TestScopes_ConfigValidation5101--- PASS: TestPins_ConfigValidation (0.00s)5102=== CONT TestScopes_Rules5103--- PASS: TestScopes_ConfigValidation (0.00s)5104=== CONT TestScopes_LegacyProviderDefaultsToWrite5105=== CONT TestValidateToken_ValidToken5106=== CONT TestPins_TopLevelShorthand5107=== CONT TestValidateToken_MultipleProviders5108=== CONT TestValidateToken_NoMatchingProvider5109=== CONT TestValidateToken_BoundSubjectMismatch5110=== CONT TestGlobMatch5111=== RUN TestGlobMatch/foo_foo5112=== PAUSE TestGlobMatch/foo_foo5113=== RUN TestGlobMatch/foo_bar5114=== PAUSE TestGlobMatch/foo_bar5115=== RUN TestGlobMatch/*_5116=== PAUSE TestGlobMatch/*_5117=== RUN TestGlobMatch/*_anything5118=== PAUSE TestGlobMatch/*_anything5119=== RUN TestGlobMatch/foo*_foo5120=== PAUSE TestGlobMatch/foo*_foo5121=== RUN TestGlobMatch/foo*_foobar5122=== PAUSE TestGlobMatch/foo*_foobar5123=== RUN TestGlobMatch/foo*_bar5124=== PAUSE TestGlobMatch/foo*_bar5125=== CONT TestValidateToken_KubernetesIssuerFromOwnToken5126=== CONT TestNewValidator_KubernetesRequiresCA5127=== RUN TestGlobMatch/*bar_bar5128=== PAUSE TestGlobMatch/*bar_bar5129=== RUN TestGlobMatch/*bar_foobar5130=== PAUSE TestGlobMatch/*bar_foobar5131=== RUN TestGlobMatch/*bar_foo5132=== PAUSE TestGlobMatch/*bar_foo5133=== RUN TestGlobMatch/foo*bar_foobar5134=== PAUSE TestGlobMatch/foo*bar_foobar5135=== RUN TestGlobMatch/foo*bar_foo123bar5136=== PAUSE TestGlobMatch/foo*bar_foo123bar5137=== RUN TestGlobMatch/foo*bar_foobarbaz5138=== PAUSE TestGlobMatch/foo*bar_foobarbaz5139=== RUN TestGlobMatch/*/*_foo/bar5140=== PAUSE TestGlobMatch/*/*_foo/bar5141=== RUN TestGlobMatch/*/*_foo5142=== PAUSE TestGlobMatch/*/*_foo5143=== RUN TestGlobMatch/refs/heads/*_refs/heads/main5144=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main5145=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.05146=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.05147=== RUN TestGlobMatch/refs/*/main_refs/heads/main5148=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main5149=== RUN TestGlobMatch/fo?_foo5150=== PAUSE TestGlobMatch/fo?_foo5151=== RUN TestGlobMatch/fo?_fo5152=== PAUSE TestGlobMatch/fo?_fo5153=== RUN TestGlobMatch/fo?_fooo5154=== PAUSE TestGlobMatch/fo?_fooo5155=== RUN TestGlobMatch/?oo_foo5156=== PAUSE TestGlobMatch/?oo_foo5157=== RUN TestGlobMatch/?oo_boo5158=== PAUSE TestGlobMatch/?oo_boo5159=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5160=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5161=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5162=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5163=== CONT TestGlobMatch/foo_foo5164=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5165=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5166=== CONT TestGlobMatch/?oo_boo5167=== CONT TestGlobMatch/?oo_foo5168=== CONT TestGlobMatch/fo?_fooo5169=== CONT TestGlobMatch/fo?_fo5170=== CONT TestGlobMatch/fo?_foo5171=== CONT TestGlobMatch/refs/*/main_refs/heads/main5172=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.05173=== CONT TestGlobMatch/refs/heads/*_refs/heads/main5174=== CONT TestGlobMatch/*/*_foo5175=== CONT TestGlobMatch/*/*_foo/bar5176=== CONT TestGlobMatch/foo*bar_foobarbaz5177=== CONT TestGlobMatch/foo*bar_foo123bar5178=== CONT TestGlobMatch/foo*bar_foobar5179=== CONT TestGlobMatch/*bar_foo5180=== CONT TestGlobMatch/*bar_foobar5181=== CONT TestGlobMatch/*bar_bar5182=== CONT TestGlobMatch/foo*_bar5183=== CONT TestGlobMatch/foo*_foobar5184=== CONT TestGlobMatch/foo*_foo5185=== CONT TestGlobMatch/*_anything5186=== CONT TestGlobMatch/*_5187=== CONT TestGlobMatch/foo_bar5188--- PASS: TestGlobMatch (0.00s)5189 --- PASS: TestGlobMatch/foo_foo (0.00s)5190 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)5191 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)5192 --- PASS: TestGlobMatch/?oo_boo (0.00s)5193 --- PASS: TestGlobMatch/?oo_foo (0.00s)5194 --- PASS: TestGlobMatch/fo?_fooo (0.00s)5195 --- PASS: TestGlobMatch/fo?_fo (0.00s)5196 --- PASS: TestGlobMatch/fo?_foo (0.00s)5197 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)5198 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)5199 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)5200 --- PASS: TestGlobMatch/*/*_foo (0.00s)5201 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)5202 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)5203 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)5204 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)5205 --- PASS: TestGlobMatch/*bar_foo (0.00s)5206 --- PASS: TestGlobMatch/*bar_foobar (0.00s)5207 --- PASS: TestGlobMatch/*bar_bar (0.00s)5208 --- PASS: TestGlobMatch/foo*_bar (0.00s)5209 --- PASS: TestGlobMatch/foo*_foobar (0.00s)5210 --- PASS: TestGlobMatch/foo*_foo (0.00s)5211 --- PASS: TestGlobMatch/*_anything (0.00s)5212 --- PASS: TestGlobMatch/*_ (0.00s)5213 --- PASS: TestGlobMatch/foo_bar (0.00s)52142026/11/11 23:27:47 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58696/oidc5215--- PASS: TestValidateToken_BoundClaimsMismatch (0.05s)52162026/11/11 23:27:47 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58698/oidc5217--- PASS: TestScopes_Rules (0.08s)52182026/11/11 23:27:47 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58701/oidc5219--- PASS: TestValidateToken_ValidToken (0.08s)52202026/11/11 23:27:47 http: TLS handshake error from 127.0.0.1:58705: read tcp 127.0.0.1:58704->127.0.0.1:58705: use of closed network connection5221--- PASS: TestNewValidator_KubernetesRequiresCA (0.08s)52222026/11/11 23:27:47 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58706/oidc52232026/11/11 23:27:47 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC1235224--- PASS: TestValidateToken_WrongAudience (0.10s)52252026/11/11 23:27:47 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58710/oidc5226--- PASS: TestPins_TopLevelShorthand (0.09s)5227--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (0.09s)52282026/11/11 23:27:47 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58712/oidc5229--- PASS: TestPins_ReservedForMatchingRule (0.11s)52302026/11/11 23:27:47 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58716/oidc52312026/11/11 23:27:47 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:587145232--- PASS: TestScopes_LegacyProviderDefaultsToWrite (0.11s)5233--- PASS: TestValidateToken_KubernetesServiceAccount (0.12s)52342026/11/11 23:27:47 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:58703/oidc52352026/11/11 23:27:47 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:58718/oidc5236--- PASS: TestValidateToken_MultipleProviders (0.12s)52372026/11/11 23:27:47 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:58699/oidc5238--- PASS: TestValidateToken_NoMatchingProvider (0.14s)52392026/11/11 23:27:47 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58723/oidc5240--- PASS: TestValidateToken_Expired (0.16s)52412026/11/11 23:27:47 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58725/oidc5242--- PASS: TestValidateToken_BoundSubjectMismatch (0.16s)5243PASS5244Running hook tests...5245=== RUN TestSendPathsEmpty5246=== PAUSE TestSendPathsEmpty5247=== RUN TestQueueEnqueueAndFetch5248=== PAUSE TestQueueEnqueueAndFetch5249=== RUN TestQueueDeduplication5250=== PAUSE TestQueueDeduplication5251=== RUN TestQueueRemove5252=== PAUSE TestQueueRemove5253=== RUN TestQueueFetchBatchLimit5254=== PAUSE TestQueueFetchBatchLimit5255=== RUN TestQueueRetryMovesToBack5256=== PAUSE TestQueueRetryMovesToBack5257=== RUN TestQueueFetchRemoveLifecycle5258=== PAUSE TestQueueFetchRemoveLifecycle5259=== RUN TestQueueConcurrentWriters5260=== PAUSE TestQueueConcurrentWriters5261=== RUN TestQueueRemoveLargeClosure5262=== PAUSE TestQueueRemoveLargeClosure5263=== RUN TestServerClientIntegration5264--- PASS: TestServerClientIntegration (0.00s)5265=== RUN TestServerQueueError52662026/11/11 23:27:48 ERROR Failed to queue paths error="permission denied" count=15267--- PASS: TestServerQueueError (0.00s)5268=== RUN TestGetListenerSocketActivation5269 server_test.go:146: systemd socket activation does not exist on darwin5270--- SKIP: TestGetListenerSocketActivation (0.00s)5271=== RUN TestServerStalledClientDoesNotBlockShutdown5272=== PAUSE TestServerStalledClientDoesNotBlockShutdown5273=== RUN TestServerBacksOffOnAcceptErrors5274=== PAUSE TestServerBacksOffOnAcceptErrors5275=== RUN TestDrainIsolatesPoisonPath5276=== PAUSE TestDrainIsolatesPoisonPath5277=== RUN TestRunNotBlockedByPoisonHead5278=== PAUSE TestRunNotBlockedByPoisonHead5279=== RUN TestDrainGivesUpWhenServerDown5280=== PAUSE TestDrainGivesUpWhenServerDown5281=== RUN TestFailedPathPrunedByLaterClosure5282=== PAUSE TestFailedPathPrunedByLaterClosure5283=== RUN TestWorkerUploadsAndRemoves5284=== PAUSE TestWorkerUploadsAndRemoves5285=== RUN TestWorkerSkipsGCdPaths5286=== PAUSE TestWorkerSkipsGCdPaths5287=== RUN TestWorkerPrunesClosureDeps5288=== PAUSE TestWorkerPrunesClosureDeps5289=== RUN TestDrainTimeout5290=== PAUSE TestDrainTimeout5291=== RUN TestWorkerRemoveFailureIsNotProgress5292=== PAUSE TestWorkerRemoveFailureIsNotProgress5293=== RUN TestWorkerKeepsPathItCannotStat5294=== PAUSE TestWorkerKeepsPathItCannotStat5295=== CONT TestSendPathsEmpty5296=== CONT TestFailedPathPrunedByLaterClosure5297--- PASS: TestSendPathsEmpty (0.00s)5298=== CONT TestWorkerKeepsPathItCannotStat5299=== CONT TestWorkerRemoveFailureIsNotProgress5300=== CONT TestDrainTimeout5301=== CONT TestWorkerPrunesClosureDeps5302=== CONT TestWorkerSkipsGCdPaths5303=== CONT TestWorkerUploadsAndRemoves5304=== CONT TestDrainIsolatesPoisonPath5305=== CONT TestQueueRetryMovesToBack5306=== CONT TestQueueFetchBatchLimit5307=== CONT TestQueueRemove5308=== CONT TestQueueDeduplication5309=== CONT TestQueueEnqueueAndFetch5310=== CONT TestQueueFetchRemoveLifecycle5311=== CONT TestServerStalledClientDoesNotBlockShutdown5312=== CONT TestServerBacksOffOnAcceptErrors5313=== CONT TestQueueRemoveLargeClosure53142026/11/11 23:27:48 ERROR Accept failed error="too many open files"5315=== CONT TestDrainGivesUpWhenServerDown5316=== CONT TestQueueConcurrentWriters5317=== CONT TestRunNotBlockedByPoisonHead53182026/11/11 23:27:48 ERROR Accept failed error="too many open files"53192026/11/11 23:27:48 INFO Upload queue status pending=253202026/11/11 23:27:48 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestWorkerSkipsGCdPaths682373829/002/nonexistent5321--- PASS: TestQueueFetchBatchLimit (0.01s)53222026/11/11 23:27:48 INFO Upload queue status pending=253232026/11/11 23:27:48 INFO Uploading batch count=253242026/11/11 23:27:48 WARN Cannot stat store path, will retry later path=/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestWorkerKeepsPathItCannotStat4003392372/002/locked/aaa error="stat /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestWorkerKeepsPathItCannotStat4003392372/002/locked/aaa: permission denied"53252026/11/11 23:27:48 INFO Uploading batch count=153262026/11/11 23:27:48 INFO Uploading batch count=253272026/11/11 23:27:48 INFO Upload queue status pending=253282026/11/11 23:27:48 INFO Uploading batch count=153292026/11/11 23:27:48 INFO Uploading batch count=453302026/11/11 23:27:48 ERROR Upload failed error="upload failed" count=45331--- PASS: TestQueueEnqueueAndFetch (0.01s)53322026/11/11 23:27:48 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestDrainIsolatesPoisonPath1097401429/002/bbb53332026/11/11 23:27:48 INFO Uploading batch count=153342026/11/11 23:27:48 ERROR Upload failed error="upload failed" count=153352026/11/11 23:27:48 WARN Cannot stat store path, will retry later path=/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestWorkerKeepsPathItCannotStat4003392372/002/locked/aaa error="stat /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestWorkerKeepsPathItCannotStat4003392372/002/locked/aaa: permission denied"5336--- PASS: TestQueueFetchRemoveLifecycle (0.01s)53372026/11/11 23:27:48 INFO Upload queue status pending=353382026/11/11 23:27:48 INFO Uploading batch count=153392026/11/11 23:27:48 ERROR Upload failed error="upload failed" count=153402026/11/11 23:27:48 INFO Uploading batch count=15341--- PASS: TestQueueDeduplication (0.01s)53422026/11/11 23:27:48 INFO Uploading batch count=153432026/11/11 23:27:48 INFO Uploading batch count=253442026/11/11 23:27:48 ERROR Upload failed error="upload failed" count=253452026/11/11 23:27:48 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestDrainGivesUpWhenServerDown1820196297/002/a53462026/11/11 23:27:48 ERROR Upload failed error="upload failed" count=153472026/11/11 23:27:48 WARN Cannot stat store path, will retry later path=/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestWorkerKeepsPathItCannotStat4003392372/002/locked/aaa error="stat /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestWorkerKeepsPathItCannotStat4003392372/002/locked/aaa: permission denied"53482026/11/11 23:27:48 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestWorkerRemoveFailureIsNotProgress3522841835/002/nonexistent53492026/11/11 23:27:48 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=153502026/11/11 23:27:48 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestWorkerRemoveFailureIsNotProgress3522841835/002/nonexistent53512026/11/11 23:27:48 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestDrainGivesUpWhenServerDown1820196297/002/b53522026/11/11 23:27:48 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=153532026/11/11 23:27:48 ERROR Drain finished with paths left in queue remaining=15354--- PASS: TestQueueRemove (0.01s)53552026/11/11 23:27:48 INFO Uploading batch count=153562026/11/11 23:27:48 INFO Uploading batch count=153572026/11/11 23:27:48 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestWorkerRemoveFailureIsNotProgress3522841835/002/nonexistent53582026/11/11 23:27:48 ERROR Upload failed error="upload failed" count=153592026/11/11 23:27:48 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=153602026/11/11 23:27:48 ERROR Drain finished with paths left in queue remaining=15361--- PASS: TestQueueRetryMovesToBack (0.02s)53622026/11/11 23:27:48 INFO Uploading batch count=253632026/11/11 23:27:48 ERROR Upload failed error="upload failed" count=253642026/11/11 23:27:48 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestDrainGivesUpWhenServerDown1820196297/002/c53652026/11/11 23:27:48 INFO Uploading batch count=153662026/11/11 23:27:48 ERROR Upload failed error="upload failed" count=153672026/11/11 23:27:48 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestDrainGivesUpWhenServerDown1820196297/002/d53682026/11/11 23:27:48 ERROR Drain finished with paths left in queue remaining=15369--- PASS: TestWorkerRemoveFailureIsNotProgress (0.02s)5370--- PASS: TestWorkerKeepsPathItCannotStat (0.02s)53712026/11/11 23:27:48 ERROR Accept failed error="too many open files"53722026/11/11 23:27:48 INFO Uploading batch count=253732026/11/11 23:27:48 ERROR Upload failed error="upload failed" count=253742026/11/11 23:27:48 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestDrainGivesUpWhenServerDown1820196297/002/e5375--- PASS: TestFailedPathPrunedByLaterClosure (0.02s)53762026/11/11 23:27:48 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestDrainGivesUpWhenServerDown1820196297/002/f53772026/11/11 23:27:48 ERROR Drain finished with paths left in queue remaining=105378--- PASS: TestDrainIsolatesPoisonPath (0.02s)5379--- PASS: TestDrainGivesUpWhenServerDown (0.02s)5380--- PASS: TestWorkerSkipsGCdPaths (0.03s)5381--- PASS: TestWorkerPrunesClosureDeps (0.03s)5382--- PASS: TestWorkerUploadsAndRemoves (0.03s)53832026/11/11 23:27:48 ERROR Accept failed error="too many open files"5384--- PASS: TestQueueRemoveLargeClosure (0.05s)5385--- PASS: TestQueueConcurrentWriters (0.05s)53862026/11/11 23:27:48 ERROR Accept failed error="too many open files"53872026/11/11 23:27:48 ERROR Accept failed error="too many open files"53882026/11/11 23:27:48 ERROR Failed to decode request error="read unix /private/var/tb/5/scratch/f4d39195ead3f86e/build/hook1464123147/test.sock->: i/o timeout"53892026/11/11 23:27:48 ERROR Failed to write response error="write unix /private/var/tb/5/scratch/f4d39195ead3f86e/build/hook1464123147/test.sock->: i/o timeout"5390--- PASS: TestServerStalledClientDoesNotBlockShutdown (0.20s)53912026/11/11 23:27:48 ERROR Upload failed error="context deadline exceeded" count=253922026/11/11 23:27:48 ERROR Drain finished with paths left in queue remaining=45393--- PASS: TestDrainTimeout (0.22s)53942026/11/11 23:27:48 ERROR Accept failed error="too many open files"53952026/11/11 23:27:48 ERROR Accept failed error="too many open files"5396--- PASS: TestServerBacksOffOnAcceptErrors (0.64s)53972026/11/11 23:27:49 INFO Uploading batch count=153982026/11/11 23:27:49 INFO Uploading batch count=153992026/11/11 23:27:49 INFO Uploading batch count=154002026/11/11 23:27:49 ERROR Upload failed error="upload failed" count=154012026/11/11 23:27:49 INFO Uploading batch count=154022026/11/11 23:27:49 ERROR Upload failed error="upload failed" count=154032026/11/11 23:27:49 INFO Uploading batch count=154042026/11/11 23:27:49 ERROR Upload failed error="upload failed" count=154052026/11/11 23:27:49 INFO Uploading batch count=154062026/11/11 23:27:49 ERROR Upload failed error="upload failed" count=154072026/11/11 23:27:49 ERROR Drain finished with paths left in queue remaining=15408--- PASS: TestRunNotBlockedByPoisonHead (1.03s)5409PASS5410Running signing tests...5411=== RUN TestExternalSigner5412=== PAUSE TestExternalSigner5413=== RUN TestExternalSignerWrongPublicKey5414=== PAUSE TestExternalSignerWrongPublicKey5415=== RUN TestExternalSignerErrors5416=== PAUSE TestExternalSignerErrors5417=== RUN TestExternalSignerRestartsAfterCrash5418=== PAUSE TestExternalSignerRestartsAfterCrash5419=== RUN TestExternalSignerProgramErrorKeepsProcess5420=== PAUSE TestExternalSignerProgramErrorKeepsProcess5421=== RUN TestExternalSignerClose5422=== PAUSE TestExternalSignerClose5423=== RUN TestExternalSignerRejectsInvalidPublicKey5424=== PAUSE TestExternalSignerRejectsInvalidPublicKey5425=== RUN TestExternalSignerCancellation5426=== PAUSE TestExternalSignerCancellation5427=== RUN TestGenerateFingerprint5428=== PAUSE TestGenerateFingerprint5429=== RUN TestParseSigningKey5430=== PAUSE TestParseSigningKey5431=== RUN TestSign5432=== PAUSE TestSign5433=== RUN TestSignBatch5434=== PAUSE TestSignBatch5435=== RUN TestNilKeyPublicKey5436=== PAUSE TestNilKeyPublicKey5437=== RUN TestSignErrors5438=== PAUSE TestSignErrors5439=== CONT TestExternalSigner5440=== CONT TestSignBatch5441=== CONT TestExternalSignerCancellation5442=== CONT TestSignErrors5443=== CONT TestSign5444=== CONT TestParseSigningKey5445=== RUN TestParseSigningKey/valid_32-byte_key5446=== CONT TestGenerateFingerprint5447=== RUN TestGenerateFingerprint/basic_with_references5448=== CONT TestNilKeyPublicKey5449--- PASS: TestNilKeyPublicKey (0.00s)5450=== PAUSE TestGenerateFingerprint/basic_with_references5451=== RUN TestGenerateFingerprint/no_references5452=== CONT TestExternalSignerClose5453=== PAUSE TestGenerateFingerprint/no_references5454=== RUN TestGenerateFingerprint/unsorted_references_get_sorted5455=== PAUSE TestGenerateFingerprint/unsorted_references_get_sorted5456=== RUN TestGenerateFingerprint/invalid_nar_hash_prefix5457=== PAUSE TestGenerateFingerprint/invalid_nar_hash_prefix5458=== CONT TestExternalSignerRestartsAfterCrash5459=== CONT TestExternalSignerWrongPublicKey5460=== PAUSE TestParseSigningKey/valid_32-byte_key5461=== RUN TestParseSigningKey/valid_32-byte_key_with_different_name5462=== PAUSE TestParseSigningKey/valid_32-byte_key_with_different_name5463=== RUN TestParseSigningKey/no_colon5464=== PAUSE TestParseSigningKey/no_colon5465=== RUN TestParseSigningKey/empty_name5466=== CONT TestExternalSignerRejectsInvalidPublicKey5467=== RUN TestExternalSignerRejectsInvalidPublicKey/#005468=== PAUSE TestExternalSignerRejectsInvalidPublicKey/#005469=== RUN TestExternalSignerRejectsInvalidPublicKey/no-colon5470=== PAUSE TestExternalSignerRejectsInvalidPublicKey/no-colon5471=== CONT TestExternalSignerProgramErrorKeepsProcess5472=== RUN TestGenerateFingerprint/invalid_nar_hash_length5473=== PAUSE TestGenerateFingerprint/invalid_nar_hash_length5474=== RUN TestGenerateFingerprint/invalid_store_path_prefix5475=== PAUSE TestGenerateFingerprint/invalid_store_path_prefix5476=== RUN TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5477=== PAUSE TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5478=== RUN TestGenerateFingerprint/invalid_reference_prefix5479=== PAUSE TestGenerateFingerprint/invalid_reference_prefix5480=== CONT TestGenerateFingerprint/basic_with_references5481=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5482=== CONT TestGenerateFingerprint/invalid_nar_hash_prefix5483=== CONT TestGenerateFingerprint/invalid_nar_hash_length5484=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5485=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5486=== CONT TestExternalSignerErrors5487=== CONT TestGenerateFingerprint/unsorted_references_get_sorted5488=== CONT TestGenerateFingerprint/invalid_reference_prefix5489=== PAUSE TestParseSigningKey/empty_name5490=== RUN TestExternalSignerErrors/invalid_JSON5491=== CONT TestGenerateFingerprint/invalid_store_path_prefix5492=== PAUSE TestExternalSignerErrors/invalid_JSON5493=== CONT TestGenerateFingerprint/no_references5494=== RUN TestExternalSignerErrors/null5495=== PAUSE TestExternalSignerErrors/null5496=== RUN TestExternalSignerErrors/too_few_signatures5497--- PASS: TestGenerateFingerprint (0.00s)5498 --- PASS: TestGenerateFingerprint/basic_with_references (0.00s)5499 --- PASS: TestGenerateFingerprint/invalid_nar_hash_prefix (0.00s)5500 --- PASS: TestGenerateFingerprint/invalid_nar_hash_length (0.00s)5501 --- PASS: TestGenerateFingerprint/unsorted_references_get_sorted (0.00s)5502 --- PASS: TestGenerateFingerprint/invalid_reference_prefix (0.00s)5503 --- PASS: TestGenerateFingerprint/invalid_store_path_prefix (0.00s)5504 --- PASS: TestGenerateFingerprint/no_references (0.00s)5505=== PAUSE TestExternalSignerErrors/too_few_signatures5506=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5507=== CONT TestExternalSignerRejectsInvalidPublicKey/#005508=== RUN TestParseSigningKey/invalid_base645509=== PAUSE TestParseSigningKey/invalid_base645510=== CONT TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5511=== RUN TestParseSigningKey/wrong_length5512=== PAUSE TestParseSigningKey/wrong_length5513=== CONT TestExternalSignerRejectsInvalidPublicKey/no-colon5514=== RUN TestExternalSignerErrors/program_error5515=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5516=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5517--- PASS: TestExternalSignerRejectsInvalidPublicKey (0.00s)5518 --- PASS: TestExternalSignerRejectsInvalidPublicKey/#00 (0.00s)5519 --- PASS: TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= (0.00s)5520 --- PASS: TestExternalSignerRejectsInvalidPublicKey/no-colon (0.00s)5521 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:YQ== (0.00s)5522 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:invalid! (0.00s)5523=== CONT TestParseSigningKey/valid_32-byte_key5524=== CONT TestParseSigningKey/wrong_length5525=== CONT TestParseSigningKey/invalid_base645526=== CONT TestParseSigningKey/empty_name5527=== CONT TestParseSigningKey/no_colon5528=== CONT TestParseSigningKey/valid_32-byte_key_with_different_name5529=== PAUSE TestExternalSignerErrors/program_error5530=== RUN TestExternalSignerErrors/wrong_shape5531=== PAUSE TestExternalSignerErrors/wrong_shape5532=== RUN TestExternalSignerErrors/missing_signatures5533=== PAUSE TestExternalSignerErrors/missing_signatures5534=== RUN TestExternalSignerErrors/null_signatures5535=== PAUSE TestExternalSignerErrors/null_signatures5536=== RUN TestExternalSignerErrors/unrequested_key5537=== PAUSE TestExternalSignerErrors/unrequested_key5538=== RUN TestExternalSignerErrors/missing_signature_name5539=== PAUSE TestExternalSignerErrors/missing_signature_name5540=== RUN TestExternalSignerErrors/wrong_signature_name5541=== PAUSE TestExternalSignerErrors/wrong_signature_name5542=== RUN TestExternalSignerErrors/invalid_signature_base645543=== PAUSE TestExternalSignerErrors/invalid_signature_base645544=== RUN TestExternalSignerErrors/wrong_signature_length5545=== PAUSE TestExternalSignerErrors/wrong_signature_length5546=== RUN TestExternalSignerErrors/wrong_id5547=== PAUSE TestExternalSignerErrors/wrong_id5548=== RUN TestExternalSignerErrors/wrong_signatures_shape5549=== PAUSE TestExternalSignerErrors/wrong_signatures_shape5550=== RUN TestExternalSignerErrors/exit_at_startup5551=== PAUSE TestExternalSignerErrors/exit_at_startup5552=== RUN TestExternalSignerErrors/exit_on_request5553=== PAUSE TestExternalSignerErrors/exit_on_request5554=== RUN TestSignErrors/nil_key5555=== CONT TestExternalSignerErrors/invalid_JSON5556=== PAUSE TestSignErrors/nil_key5557=== CONT TestExternalSignerErrors/missing_signature_name5558--- PASS: TestSign (0.00s)5559=== CONT TestExternalSignerErrors/program_error5560=== CONT TestExternalSignerErrors/missing_signatures5561=== CONT TestExternalSignerErrors/wrong_signatures_shape5562--- PASS: TestParseSigningKey (0.00s)5563 --- PASS: TestParseSigningKey/wrong_length (0.00s)5564 --- PASS: TestParseSigningKey/invalid_base64 (0.00s)5565 --- PASS: TestParseSigningKey/empty_name (0.00s)5566 --- PASS: TestParseSigningKey/no_colon (0.00s)5567 --- PASS: TestParseSigningKey/valid_32-byte_key (0.00s)5568 --- PASS: TestParseSigningKey/valid_32-byte_key_with_different_name (0.00s)5569=== CONT TestExternalSignerErrors/wrong_signature_length5570=== CONT TestExternalSignerErrors/too_few_signatures5571=== CONT TestExternalSignerErrors/invalid_signature_base645572=== CONT TestExternalSignerErrors/wrong_signature_name5573=== CONT TestExternalSignerErrors/wrong_shape5574=== CONT TestExternalSignerErrors/wrong_id5575--- PASS: TestSignBatch (0.00s)5576=== CONT TestExternalSignerErrors/null5577=== RUN TestSignErrors/nil_narinfo5578=== PAUSE TestSignErrors/nil_narinfo5579=== RUN TestSignErrors/invalid_narinfo5580=== CONT TestExternalSignerErrors/unrequested_key5581=== CONT TestExternalSignerErrors/exit_on_request5582=== CONT TestExternalSignerErrors/exit_at_startup5583=== CONT TestExternalSignerErrors/null_signatures5584=== PAUSE TestSignErrors/invalid_narinfo5585=== RUN TestSignErrors/canceled5586=== PAUSE TestSignErrors/canceled5587=== CONT TestSignErrors/nil_key5588=== CONT TestSignErrors/canceled5589=== CONT TestSignErrors/invalid_narinfo5590=== CONT TestSignErrors/nil_narinfo5591--- PASS: TestSignErrors (0.01s)5592 --- PASS: TestSignErrors/nil_key (0.00s)5593 --- PASS: TestSignErrors/canceled (0.00s)5594 --- PASS: TestSignErrors/invalid_narinfo (0.00s)5595 --- PASS: TestSignErrors/nil_narinfo (0.00s)5596--- PASS: TestExternalSigner (0.23s)5597--- PASS: TestExternalSignerClose (0.41s)5598--- PASS: TestExternalSignerWrongPublicKey (0.60s)5599--- PASS: TestExternalSignerCancellation (1.36s)5600--- PASS: TestExternalSignerRestartsAfterCrash (2.12s)5601/private/var/tb/5/scratch/f4d39195ead3f86e/build/TestExternalSignerRestartsAfterCrash1211254223/001/sign program: line 7: /private/var/tb/5/scratch/f4d39195ead3f86e/build/TestExternalSignerRestartsAfterCrash1211254223/001/sign program.starts: No such file or directory5602--- PASS: TestExternalSignerProgramErrorKeepsProcess (3.05s)5603--- PASS: TestExternalSignerErrors (0.00s)5604 --- PASS: TestExternalSignerErrors/wrong_signature_length (0.78s)5605 --- PASS: TestExternalSignerErrors/missing_signatures (1.03s)5606 --- PASS: TestExternalSignerErrors/unrequested_key (1.15s)5607 --- PASS: TestExternalSignerErrors/exit_on_request (1.53s)5608 --- PASS: TestExternalSignerErrors/exit_at_startup (1.72s)5609 --- PASS: TestExternalSignerErrors/wrong_signatures_shape (1.91s)5610 --- PASS: TestExternalSignerErrors/wrong_signature_name (2.28s)5611 --- PASS: TestExternalSignerErrors/invalid_JSON (2.49s)5612 --- PASS: TestExternalSignerErrors/wrong_shape (2.67s)5613 --- PASS: TestExternalSignerErrors/program_error (2.86s)5614 --- PASS: TestExternalSignerErrors/null (3.23s)5615 --- PASS: TestExternalSignerErrors/missing_signature_name (3.43s)5616 --- PASS: TestExternalSignerErrors/wrong_id (3.62s)5617 --- PASS: TestExternalSignerErrors/null_signatures (3.81s)5618 --- PASS: TestExternalSignerErrors/invalid_signature_base64 (4.00s)5619 --- PASS: TestExternalSignerErrors/too_few_signatures (4.19s)5620PASS5621Running OpenBao signer tests...5622=== RUN TestSign5623=== PAUSE TestSign5624=== RUN TestSignErrors5625=== PAUSE TestSignErrors5626=== RUN TestServe5627=== PAUSE TestServe5628=== RUN TestServeRejectsGarbage5629=== PAUSE TestServeRejectsGarbage5630=== CONT TestSign5631=== CONT TestServeRejectsGarbage5632=== CONT TestServe5633=== CONT TestSignErrors5634=== RUN TestSignErrors/http_error5635--- PASS: TestServeRejectsGarbage (0.00s)5636=== PAUSE TestSignErrors/http_error5637=== RUN TestSignErrors/invalid_json5638=== PAUSE TestSignErrors/invalid_json5639=== RUN TestSignErrors/no_prefix5640=== PAUSE TestSignErrors/no_prefix5641=== RUN TestSignErrors/invalid_base645642=== PAUSE TestSignErrors/invalid_base645643=== CONT TestSignErrors/http_error5644=== CONT TestSignErrors/no_prefix5645=== CONT TestSignErrors/invalid_base645646=== CONT TestSignErrors/invalid_json5647--- PASS: TestServe (0.00s)5648--- PASS: TestSign (0.00s)5649--- PASS: TestSignErrors (0.00s)5650 --- PASS: TestSignErrors/http_error (0.00s)5651 --- PASS: TestSignErrors/no_prefix (0.00s)5652 --- PASS: TestSignErrors/invalid_json (0.00s)5653 --- PASS: TestSignErrors/invalid_base64 (0.00s)5654PASS

succeeded niks3-go-unit-tests checks.aarch64-darwin.go-unit-tests · build #489 · raw

succeeded `niks3-go-unit-tests` `checks.aarch64-darwin.go-unit-tests` · build #489 · raw