niks3-go-unit-tests
checks.aarch64-darwin.go-unit-tests
· build #495
· raw
1tribuchet: building on mac022Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestUploadBuildLog_FileBodyReplayedOnRetry6=== PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry7=== RUN TestRegisterUploadedObjectReusesConnections8=== PAUSE TestRegisterUploadedObjectReusesConnections9=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica10=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica11=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun12=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun13=== RUN TestCaseHackSuffix14=== PAUSE TestCaseHackSuffix15=== RUN TestFilterOversizedClosures16=== PAUSE TestFilterOversizedClosures17=== RUN TestUploadMultipart_PartsInParallel18=== PAUSE TestUploadMultipart_PartsInParallel19=== RUN TestUploadMultipart_ReturnsAfterDroppedBody20=== PAUSE TestUploadMultipart_ReturnsAfterDroppedBody21=== RUN TestPartSizeForNAR22=== PAUSE TestPartSizeForNAR23=== RUN TestDumpPathCaseHackMatchesNix24--- PASS: TestDumpPathCaseHackMatchesNix (0.05s)25=== RUN TestDumpPathCaseHackCollision26--- PASS: TestDumpPathCaseHackCollision (0.00s)27=== RUN TestDumpPathMatchesNix28=== PAUSE TestDumpPathMatchesNix29=== RUN TestDumpPathSingleFile30=== PAUSE TestDumpPathSingleFile31=== RUN TestDumpPathWriterError32=== PAUSE TestDumpPathWriterError33=== RUN TestEncodeNixBase3234=== PAUSE TestEncodeNixBase3235=== RUN TestEncodeNixBase32WithRealHash36=== PAUSE TestEncodeNixBase32WithRealHash37=== RUN TestConvertHashToNix3238=== PAUSE TestConvertHashToNix3239=== RUN TestGetStorePathHash40=== PAUSE TestGetStorePathHash41=== RUN TestPathInfoHashCompatibility42=== PAUSE TestPathInfoHashCompatibility43=== RUN TestParsePathInfoJSON44=== PAUSE TestParsePathInfoJSON45=== RUN TestParsePathInfoJSONMultiplePaths46=== PAUSE TestParsePathInfoJSONMultiplePaths47=== RUN TestPathInfoCACompatibility48=== PAUSE TestPathInfoCACompatibility49=== RUN TestRateLimiterFeedback50=== PAUSE TestRateLimiterFeedback51=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess52=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess53=== RUN TestRegisterUploadedObject_BoundedAgainstSilentServer54=== PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer55=== RUN TestRegisterUploadedObject_OutlivesPush56=== PAUSE TestRegisterUploadedObject_OutlivesPush57=== RUN TestResolveStorePath58=== PAUSE TestResolveStorePath59=== RUN TestDoWithRetry_BodyReplayedViaGetBody60=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody61=== RUN TestRetryTimeoutOutlastsMaxRetries62=== PAUSE TestRetryTimeoutOutlastsMaxRetries63=== RUN TestDoWithRetry_FinalResponseBodyReadable64=== PAUSE TestDoWithRetry_FinalResponseBodyReadable65=== RUN TestShellSplit66=== PAUSE TestShellSplit67=== RUN TestShellSplitErrors68=== PAUSE TestShellSplitErrors69=== RUN TestStreamPushReportsEveryPath70=== PAUSE TestStreamPushReportsEveryPath71=== RUN TestStreamPushBatchesUnderLoad72=== PAUSE TestStreamPushBatchesUnderLoad73=== RUN TestStreamPushIsolatesFailures74=== PAUSE TestStreamPushIsolatesFailures75=== RUN TestStreamPushGivesUpOnDeadServer76=== PAUSE TestStreamPushGivesUpOnDeadServer77=== RUN TestStreamPushRequestLine78=== PAUSE TestStreamPushRequestLine79=== RUN TestStreamPushReportsSignatures80=== PAUSE TestStreamPushReportsSignatures81=== RUN TestStreamPushReportsSkippedPaths82=== PAUSE TestStreamPushReportsSkippedPaths83=== RUN TestClientSignaturesByStorePath84=== PAUSE TestClientSignaturesByStorePath85=== RUN TestStreamPushStopsWhenCanceled86=== PAUSE TestStreamPushStopsWhenCanceled87=== RUN TestSetClientTLS88=== PAUSE TestSetClientTLS89=== RUN TestSetClientTLSDoesNotMutateDefaultTransport90=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport91=== RUN TestSetClientTLSErrors92=== PAUSE TestSetClientTLSErrors93=== RUN TestStaticToken94=== PAUSE TestStaticToken95=== RUN TestFileTokenReadsAndCaches96=== PAUSE TestFileTokenReadsAndCaches97=== RUN TestFileTokenMissing98=== PAUSE TestFileTokenMissing99=== RUN TestFileTokenEmpty100=== PAUSE TestFileTokenEmpty101=== RUN TestScriptTokenNoExpiryRerunsEveryCall102=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall103=== RUN TestScriptTokenCachesUntilRefresh104=== PAUSE TestScriptTokenCachesUntilRefresh105=== RUN TestScriptTokenEmptyToken106=== PAUSE TestScriptTokenEmptyToken107=== RUN TestScriptTokenBadJSON108=== PAUSE TestScriptTokenBadJSON109=== RUN TestScriptTokenScriptFails110=== PAUSE TestScriptTokenScriptFails111=== RUN TestScriptTokenEmptyCommand112=== PAUSE TestScriptTokenEmptyCommand113=== CONT TestDoServerRequestAttachesToken114=== CONT TestRetryTimeoutOutlastsMaxRetries115=== CONT TestSetClientTLSDoesNotMutateDefaultTransport116=== CONT TestStreamPushReportsSignatures117=== CONT TestDoWithRetry_FinalResponseBodyReadable118=== CONT TestScriptTokenEmptyCommand119--- PASS: TestScriptTokenEmptyCommand (0.00s)120=== CONT TestResolveStorePath121=== CONT TestScriptTokenScriptFails122=== CONT TestSetClientTLS123=== CONT TestScriptTokenBadJSON124=== CONT TestScriptTokenEmptyToken125=== CONT TestScriptTokenCachesUntilRefresh126=== CONT TestScriptTokenNoExpiryRerunsEveryCall127=== CONT TestFileTokenMissing1282026/11/12 00:09:29 ERROR Upload failed error=boom count=1129=== CONT TestFileTokenReadsAndCaches1302026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:62947131=== CONT TestRegisterUploadedObject_BoundedAgainstSilentServer132=== CONT TestStaticToken133=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess134=== CONT TestSetClientTLSErrors135=== CONT TestStreamPushGivesUpOnDeadServer1362026/11/12 00:09:29 WARN Rate limiter enabled after throttle name=server-test rate=51372026/11/12 00:09:29 WARN Rate limiter enabled after throttle name=server-test rate=51382026/11/12 00:09:29 ERROR Upload failed error="connection refused" count=201392026/11/12 00:09:29 ERROR Server seems unavailable, giving up on batch untried=171402026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:629481412026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:62947142=== CONT TestStreamPushStopsWhenCanceled143=== CONT TestParsePathInfoJSONMultiplePaths144=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths145=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths146=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths147=== CONT TestParsePathInfoJSON148=== RUN TestParsePathInfoJSON/Nix_format149=== PAUSE TestParsePathInfoJSON/Nix_format1502026/11/12 00:09:29 WARN Rate limiter backed off name=server-test rate=5151=== CONT TestGetStorePathHash152=== RUN TestGetStorePathHash/valid_store_path153=== PAUSE TestGetStorePathHash/valid_store_path154=== RUN TestGetStorePathHash/basename_without_hyphen_should_error155=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error156=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error157=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error158=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error159=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths160=== CONT TestStreamPushReportsSkippedPaths161=== CONT TestPathInfoHashCompatibility162=== CONT TestEncodeNixBase32WithRealHash163=== CONT TestEncodeNixBase32164=== CONT TestDoWithRetry_BodyReplayedViaGetBody165=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)166=== CONT TestStreamPushReportsEveryPath167=== CONT TestFileTokenEmpty168--- PASS: TestStreamPushReportsSignatures (0.00s)169=== CONT TestRegisterUploadedObject_OutlivesPush170=== CONT TestPathInfoCACompatibility171=== CONT TestClientSignaturesByStorePath172=== CONT TestRateLimiterFeedback173=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error174=== CONT TestPartSizeForNAR175=== CONT TestShellSplitErrors176=== CONT TestDumpPathMatchesNix177=== CONT TestUploadMultipart_ReturnsAfterDroppedBody178=== RUN TestPartSizeForNAR/zero_stays_at_minimum179=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum180=== RUN TestPartSizeForNAR/small_stays_at_minimum181=== PAUSE TestPartSizeForNAR/small_stays_at_minimum182=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum183=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum184=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts185=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts186=== RUN TestPartSizeForNAR/1_TiB187=== PAUSE TestPartSizeForNAR/1_TiB188=== RUN TestPartSizeForNAR/5_TiB_S3_max_object189=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object190=== RUN TestPartSizeForNAR/capped_at_5_GiB191=== PAUSE TestPartSizeForNAR/capped_at_5_GiB192=== RUN TestParsePathInfoJSON/Lix_format193=== PAUSE TestParsePathInfoJSON/Lix_format194=== RUN TestParsePathInfoJSON/empty_input195=== PAUSE TestParsePathInfoJSON/empty_input196=== RUN TestParsePathInfoJSON/whitespace_only197=== PAUSE TestParsePathInfoJSON/whitespace_only198=== RUN TestParsePathInfoJSON/invalid_JSON199=== PAUSE TestParsePathInfoJSON/invalid_JSON200=== CONT TestStreamPushIsolatesFailures201=== RUN TestSetClientTLSErrors/missing_cert_file202=== PAUSE TestSetClientTLSErrors/missing_cert_file203=== RUN TestSetClientTLSErrors/missing_key_file204=== PAUSE TestSetClientTLSErrors/missing_key_file205=== RUN TestSetClientTLSErrors/missing_ca_file206=== PAUSE TestSetClientTLSErrors/missing_ca_file207=== RUN TestSetClientTLSErrors/invalid_ca_file208=== PAUSE TestSetClientTLSErrors/invalid_ca_file209=== CONT TestStreamPushBatchesUnderLoad210=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=false211=== CONT TestConvertHashToNix32212=== RUN TestSetClientTLS/rejects_connection_without_client_cert213=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert214=== RUN TestConvertHashToNix32/SRI_format_to_Nix32215=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32216=== RUN TestRateLimiterFeedback/429_enables_limiter217=== CONT TestUploadMultipart_PartsInParallel218=== CONT TestDumpPathWriterError219=== RUN TestEncodeNixBase32/test_string_hash2202026/11/12 00:09:29 WARN Rate limiter enabled after throttle name=server-test rate=52212026/11/12 00:09:29 ERROR Upload failed error="bad path" count=32222026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:629592232026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:62947224=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)225=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon226--- PASS: TestFileTokenMissing (0.00s)227--- PASS: TestStaticToken (0.00s)228=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon229--- PASS: TestResolveStorePath (0.00s)2302026/11/12 00:09:29 WARN Rate limiter backed off name=server-test rate=52312026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:62959232=== RUN TestPathInfoCACompatibility/null_ca_field233=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=false234=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=true235=== CONT TestStreamPushRequestLine236=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA237=== RUN TestConvertHashToNix32/already_Nix32_format238=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun239=== CONT TestFilterOversizedClosures240=== PAUSE TestRateLimiterFeedback/429_enables_limiter241=== CONT TestCaseHackSuffix242=== PAUSE TestEncodeNixBase32/test_string_hash243=== CONT TestDumpPathSingleFile244=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI245--- PASS: TestDoServerRequestAttachesToken (0.01s)246=== CONT TestRegisterUploadedObjectReusesConnections2472026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:62947248=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica249=== CONT TestUploadBuildLog_FileBodyReplayedOnRetry250=== PAUSE TestConvertHashToNix32/already_Nix32_format251=== RUN TestConvertHashToNix32/invalid_format252=== PAUSE TestPathInfoCACompatibility/null_ca_field253=== RUN TestPathInfoCACompatibility/old_string_format_-_text254=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text255=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive256=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive257=== RUN TestPathInfoCACompatibility/new_structured_format_-_text258=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text259=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method260=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method261=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA262=== RUN TestSetClientTLS/preserves_debug_logging_transport263=== PAUSE TestSetClientTLS/preserves_debug_logging_transport264=== CONT TestShellSplit265--- PASS: TestShellSplit (0.00s)266=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths267=== CONT TestGetStorePathHash/valid_store_path268=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=true269=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths270--- PASS: TestParsePathInfoJSONMultiplePaths (0.00s)271 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)272 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)273=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error274=== CONT TestPartSizeForNAR/zero_stays_at_minimum275=== PAUSE TestConvertHashToNix32/invalid_format276=== CONT TestGetStorePathHash/basename_without_hyphen_should_error277=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error278--- PASS: TestGetStorePathHash (0.00s)279 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)280 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)281 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)282 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)283=== CONT TestParsePathInfoJSON/Nix_format284=== CONT TestPartSizeForNAR/capped_at_5_GiB285=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum286=== CONT TestPartSizeForNAR/small_stays_at_minimum287=== CONT TestPartSizeForNAR/5_TiB_S3_max_object288=== CONT TestSetClientTLSErrors/invalid_ca_file2892026/11/12 00:09:29 ERROR Upload failed error=boom count=1290=== CONT TestSetClientTLSErrors/missing_ca_file291=== CONT TestSetClientTLSErrors/missing_key_file292=== CONT TestParsePathInfoJSON/whitespace_only293=== CONT TestParsePathInfoJSON/invalid_JSON294=== CONT TestParsePathInfoJSON/empty_input295=== CONT TestParsePathInfoJSON/Lix_format296--- PASS: TestParsePathInfoJSON (0.00s)297 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)298 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)299 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)300 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)301 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)302=== CONT TestPartSizeForNAR/1_TiB303=== CONT TestSetClientTLS/rejects_connection_without_client_cert304=== RUN TestFilterOversizedClosures/no_limit_keeps_everything305=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything306=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped307=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped308=== RUN TestFilterOversizedClosures/all_closures_skipped309=== PAUSE TestFilterOversizedClosures/all_closures_skipped310=== RUN TestRateLimiterFeedback/503_enables_limiter311=== PAUSE TestRateLimiterFeedback/503_enables_limiter312=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter313=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter314=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter315=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter316=== CONT TestSetClientTLSErrors/missing_cert_file317=== CONT TestPathInfoCACompatibility/null_ca_field318=== CONT TestPathInfoCACompatibility/old_string_format_-_text319=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=false320=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts321--- PASS: TestPartSizeForNAR (0.00s)322 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)323 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)324 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)325 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)326 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)327 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)328 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)329=== CONT TestSetClientTLS/preserves_debug_logging_transport330--- PASS: TestSetClientTLSErrors (0.01s)331 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)332 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)333 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)334 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)335=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive336=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA337=== CONT TestConvertHashToNix32/SRI_format_to_Nix32338=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=true339--- PASS: TestStreamPushBatchesUnderLoad (0.01s)340 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=false (0.00s)341 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=true (0.00s)342=== CONT TestConvertHashToNix32/invalid_format343=== CONT TestConvertHashToNix32/already_Nix32_format344--- PASS: TestConvertHashToNix32 (0.01s)345 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)346 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)347 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)348=== CONT TestFilterOversizedClosures/no_limit_keeps_everything349=== CONT TestRateLimiterFeedback/429_enables_limiter350=== RUN TestEncodeNixBase32/empty_input351=== PAUSE TestEncodeNixBase32/empty_input352=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI353=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512354=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512355--- PASS: TestStreamPushGivesUpOnDeadServer (0.00s)356--- PASS: TestStreamPushStopsWhenCanceled (0.00s)357--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.01s)358--- PASS: TestScriptTokenScriptFails (0.01s)359--- PASS: TestDoWithRetry_FinalResponseBodyReadable (0.01s)360--- PASS: TestStreamPushReportsSkippedPaths (0.00s)361--- PASS: TestEncodeNixBase32WithRealHash (0.00s)362--- PASS: TestFileTokenReadsAndCaches (0.00s)363--- PASS: TestStreamPushReportsEveryPath (0.00s)364--- PASS: TestClientSignaturesByStorePath (0.00s)365--- PASS: TestShellSplitErrors (0.00s)366--- PASS: TestFileTokenEmpty (0.00s)367--- PASS: TestStreamPushIsolatesFailures (0.00s)368--- PASS: TestRegisterUploadedObject_OutlivesPush (0.00s)369--- PASS: TestScriptTokenEmptyToken (0.01s)370--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.01s)371--- PASS: TestScriptTokenBadJSON (0.01s)372=== CONT TestEncodeNixBase32/test_string_hash373=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)374=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method375=== CONT TestPathInfoCACompatibility/new_structured_format_-_text376--- PASS: TestPathInfoCACompatibility (0.01s)377 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)378 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)379 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)380 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)381 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)382=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter383=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon384=== CONT TestFilterOversizedClosures/all_closures_skipped3852026/11/12 00:09:29 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50386=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped3872026/11/12 00:09:29 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000388--- PASS: TestFilterOversizedClosures (0.00s)389 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)390 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)391 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)392=== CONT TestEncodeNixBase32/empty_input393--- PASS: TestEncodeNixBase32 (0.01s)394 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)395 --- PASS: TestEncodeNixBase32/empty_input (0.00s)396=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter397=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI398=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512399--- PASS: TestPathInfoHashCompatibility (0.01s)400 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)401 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)402 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)403 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)404=== CONT TestRateLimiterFeedback/503_enables_limiter4052026/11/12 00:09:29 WARN Rate limiter enabled after throttle name=server-test rate=54062026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:630094072026/11/12 00:09:29 WARN Rate limiter backed off name=server-test rate=54082026/11/12 00:09:29 INFO Garbage collection started4092026/11/12 00:09:29 INFO Garbage collection started4102026/11/12 00:09:29 WARN Rate limiter enabled after throttle name=s3-test rate=54112026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:63016412--- PASS: TestUploadMultipart_ReturnsAfterDroppedBody (0.01s)4132026/11/12 00:09:29 WARN Rate limiter enabled after throttle name=server-test rate=54142026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:630414152026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:629474162026/11/12 00:09:29 WARN Rate limiter backed off name=server-test rate=5417--- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (0.01s)4182026/11/12 00:09:29 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=0419--- PASS: TestRateLimiterFeedback (0.01s)420 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.00s)421 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.00s)422 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s)423 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s)424--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (0.01s)4252026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:629474262026/11/12 00:09:29 WARN Garbage collection finished on another replica. Its statistics are not available here427--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (0.02s)428--- PASS: TestStreamPushRequestLine (0.02s)4292026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:629474302026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:629474312026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:629474322026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:629474332026/11/12 00:09:29 http: TLS handshake error from 127.0.0.1:63047: read tcp 127.0.0.1:62956->127.0.0.1:63047: use of closed network connection434--- PASS: TestSetClientTLS (0.01s)435 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.01s)436 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.02s)437 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.03s)4382026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:629474392026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:629474402026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:629474412026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:629474422026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:62947443--- PASS: TestRegisterUploadedObjectReusesConnections (0.04s)4442026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:629474452026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:62947446--- PASS: TestScriptTokenCachesUntilRefresh (0.05s)4472026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:629474482026/11/12 00:09:29 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:62947449--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.06s)450--- PASS: TestRetryTimeoutOutlastsMaxRetries (0.06s)451--- PASS: TestDumpPathWriterError (0.05s)452--- PASS: TestCaseHackSuffix (0.05s)453--- PASS: TestDumpPathSingleFile (0.05s)454--- PASS: TestDumpPathMatchesNix (0.08s)4552026/11/12 00:09:29 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:62952/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:62952/api/uploads/complete4562026/11/12 00:09:29 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:62952/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:62952/api/uploads/complete4572026/11/12 00:09:29 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4582026/11/12 00:09:29 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4592026/11/12 00:09:29 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:62952/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:62952/api/uploads/complete4602026/11/12 00:09:29 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"461--- PASS: TestRegisterUploadedObject_BoundedAgainstSilentServer (0.11s)462--- PASS: TestUploadMultipart_PartsInParallel (0.62s)463--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)464PASS465Running server tests...466The files belonging to this database system will be owned by user "_tribuchetbld7".467This user must also own the server process.468469The database cluster will be initialized with locale "C".470The default database encoding has accordingly been set to "SQL_ASCII".471The default text search configuration will be set to "english".472473Data page checksums are enabled.474475creating directory /private/var/tb/7/scratch/b827b2ae36b0e69e/build/postgres634965743/data ... ok476creating subdirectories ... ok477selecting dynamic shared memory implementation ... posix478selecting default "max_connections" ... 100479selecting default "shared_buffers" ... 128MB480selecting default time zone ... UTC481creating configuration files ... ok482running bootstrap script ... ok483performing post-bootstrap initialization ... ok484syncing data to disk ... ok485486initdb: warning: enabling "trust" authentication for local connections487initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.488489Success. You can now start the database server using:490491 pg_ctl -D /private/var/tb/7/scratch/b827b2ae36b0e69e/build/postgres634965743/data -l logfile start492493/private/var/tb/7/scratch/b827b2ae36b0e69e/build/postgres634965743:5432 - no response4942026-11-12 00:09:31.380 UTC [10471] LOG: starting PostgreSQL 18.6 on aarch64-apple-darwin25.6.0, compiled by clang version 21.1.8, 64-bit4952026-11-12 00:09:31.380 UTC [10471] LOG: listening on Unix socket "/private/var/tb/7/scratch/b827b2ae36b0e69e/build/postgres634965743/.s.PGSQL.5432"4962026-11-12 00:09:31.382 UTC [10478] LOG: database system was shut down at 2026-11-12 00:09:31 UTC4972026-11-12 00:09:31.383 UTC [10471] LOG: database system is ready to accept connections498/private/var/tb/7/scratch/b827b2ae36b0e69e/build/postgres634965743:5432 - accepting connections499{"timestamp":"2026-11-12T00:09:31.59863Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"d663ea76-85fe-466d-bbb3-a84a7d23a1cd","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(5)"}500=== RUN TestAbortSucceeded501=== PAUSE TestAbortSucceeded502=== RUN TestService_AuthMiddleware503=== PAUSE TestService_AuthMiddleware504=== RUN TestService_AuthMiddleware_MTLSProxyHeader505=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader506=== RUN TestService_AuthMiddleware_MTLSBoundSubjects507=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects508=== RUN TestService_ReadAuthMiddleware509=== PAUSE TestService_ReadAuthMiddleware510=== RUN TestService_AuthMiddleware_OIDC511=== PAUSE TestService_AuthMiddleware_OIDC512=== RUN TestService_RequireScope_OIDC513=== PAUSE TestService_RequireScope_OIDC514=== RUN TestService_ReadScope_PublicByDefault515=== PAUSE TestService_ReadScope_PublicByDefault516=== RUN TestCacheConfigHandler517=== PAUSE TestCacheConfigHandler518=== RUN TestCacheStatsHandler519=== PAUSE TestCacheStatsHandler520=== RUN TestClientCADerivations521=== PAUSE TestClientCADerivations522=== RUN TestClientErrorHandling523=== PAUSE TestClientErrorHandling524=== RUN TestClientIntegration525=== PAUSE TestClientIntegration526=== RUN TestClientMultipleUploads527=== PAUSE TestClientMultipleUploads528=== RUN TestClientWithDependencies529=== PAUSE TestClientWithDependencies530=== RUN TestClientSharedPathCommittedMidPush531=== PAUSE TestClientSharedPathCommittedMidPush532=== RUN TestPinProtectsFromGC533=== PAUSE TestPinProtectsFromGC534=== RUN TestClientPushesUseOnePush535=== PAUSE TestClientPushesUseOnePush536=== RUN TestClientFallsBackToClosures537=== PAUSE TestClientFallsBackToClosures538=== RUN TestDeleteClosuresRacingPins539=== PAUSE TestDeleteClosuresRacingPins540=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock541=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock542=== RUN TestResolveDBConnectionString543=== PAUSE TestResolveDBConnectionString544=== RUN TestConnectWaitsForAPeerMigration545=== PAUSE TestConnectWaitsForAPeerMigration546=== RUN TestLeadElectsOneAndHandsOver547=== PAUSE TestLeadElectsOneAndHandsOver548=== RUN TestLeadIncumbentWinsAfterRestart5492026/11/12 00:09:31 INFO lead: acquired remote=192.0.2.1:1234 generation=15502026/11/12 00:09:32 INFO lead: released remote=192.0.2.1:1234 generation=15512026/11/12 00:09:32 INFO lead: acquired remote=192.0.2.1:1234 generation=25522026/11/12 00:09:33 INFO lead: released remote=192.0.2.1:1234 generation=2553--- PASS: TestLeadIncumbentWinsAfterRestart (1.93s)554=== RUN TestLeadUnannouncedStandbyYieldsToIncumbent555=== PAUSE TestLeadUnannouncedStandbyYieldsToIncumbent556=== RUN TestLeadIncumbentAnnouncesAtOnce557=== PAUSE TestLeadIncumbentAnnouncesAtOnce558=== RUN TestLeadGenerationGrowsWithEveryGrant559=== PAUSE TestLeadGenerationGrowsWithEveryGrant560=== RUN TestLeadEndsOnShutdown561=== PAUSE TestLeadEndsOnShutdown562=== RUN TestLeadEndsWhenItsConnectionHangs5632026/11/12 00:09:33 INFO lead: acquired remote=192.0.2.1:1234 generation=15642026-11-12 00:09:35.203 UTC [10527] FATAL: terminating connection due to administrator command5652026/11/12 00:09:35 INFO lead: acquired remote=192.0.2.1:1234 generation=25662026/11/12 00:09:36 WARN lead: lock connection lost error="timeout: context deadline exceeded"5672026/11/12 00:09:36 INFO lead: released remote=192.0.2.1:1234 generation=15682026/11/12 00:09:36 INFO lead: released remote=192.0.2.1:1234 generation=2569--- PASS: TestLeadEndsWhenItsConnectionHangs (2.97s)570=== RUN TestGCAdvisoryLockBlocksConcurrentRun571=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun572=== RUN TestGCBugBareHashReferences573=== PAUSE TestGCBugBareHashReferences574=== RUN TestGCMetrics575=== PAUSE TestGCMetrics576=== RUN TestPushDedupSurvivesConcurrentGC577=== PAUSE TestPushDedupSurvivesConcurrentGC578=== RUN TestDeduplicatedObjectsRecordedAsPending579=== PAUSE TestDeduplicatedObjectsRecordedAsPending580=== RUN TestGCSweepSkipsPendingObjects581=== PAUSE TestGCSweepSkipsPendingObjects582=== RUN TestTombstonedObjectOfferedWithoutWaiting583=== PAUSE TestTombstonedObjectOfferedWithoutWaiting584=== RUN TestGCSweepDeliversEachKeyOnce585=== PAUSE TestGCSweepDeliversEachKeyOnce586=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection587=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection588=== RUN TestForceGCDuringPushOffersSweptObject589=== PAUSE TestForceGCDuringPushOffersSweptObject590=== RUN TestFailedS3DeleteKeepsTombstone591=== PAUSE TestFailedS3DeleteKeepsTombstone592=== RUN TestSweepSkipsRowsLockedByPush593=== PAUSE TestSweepSkipsRowsLockedByPush594=== RUN TestRegisterCompletedObjectNeedsPendingRow595=== PAUSE TestRegisterCompletedObjectNeedsPendingRow596=== RUN TestGetExistingObjectsLocksRows597=== PAUSE TestGetExistingObjectsLocksRows598=== RUN TestSweepRechecksPendingAfterLocking599=== PAUSE TestSweepRechecksPendingAfterLocking600=== RUN TestPushWaitsForSweepHoldingRow601=== PAUSE TestPushWaitsForSweepHoldingRow602=== RUN TestCommitRacingPendingCleanupKeepsObjects603=== PAUSE TestCommitRacingPendingCleanupKeepsObjects604=== RUN TestGCEndsOnShutdown605=== PAUSE TestGCEndsOnShutdown606=== RUN TestGCTaskStore_StartNew607=== PAUSE TestGCTaskStore_StartNew608=== RUN TestGCTaskStore_DeduplicateSameParams609=== PAUSE TestGCTaskStore_DeduplicateSameParams610=== RUN TestGCTaskStore_ConflictDifferentParams611=== PAUSE TestGCTaskStore_ConflictDifferentParams612=== RUN TestGCTaskStore_GetEmpty613=== PAUSE TestGCTaskStore_GetEmpty614=== RUN TestGCTaskStore_GetReturnsLatest615=== PAUSE TestGCTaskStore_GetReturnsLatest616=== RUN TestGCTaskStore_CompletedAllowsNewTask617=== PAUSE TestGCTaskStore_CompletedAllowsNewTask618=== RUN TestGCTaskStore_PhaseUpdates619=== PAUSE TestGCTaskStore_PhaseUpdates620=== RUN TestGCTaskStore_Fail621=== PAUSE TestGCTaskStore_Fail622=== RUN TestGracefulShutdownDrainsInflight623=== PAUSE TestGracefulShutdownDrainsInflight624=== RUN TestService_healthCheckHandler625=== PAUSE TestService_healthCheckHandler626=== RUN TestService_readinessHandler627=== PAUSE TestService_readinessHandler628=== RUN TestGenerateLandingPage629=== PAUSE TestGenerateLandingPage630=== RUN TestEnsureAbortUploadsRule631=== PAUSE TestEnsureAbortUploadsRule632=== RUN TestCacheConfigHandlerMaxNarSize633=== PAUSE TestCacheConfigHandlerMaxNarSize634=== RUN TestCreatePendingClosureRejectsOversizedNAR635=== PAUSE TestCreatePendingClosureRejectsOversizedNAR636=== RUN TestNARDeduplicationMetadataUploadBug637=== PAUSE TestNARDeduplicationMetadataUploadBug638=== RUN TestMetricsInventory639=== PAUSE TestMetricsInventory640=== RUN TestMetricsCommit641=== PAUSE TestMetricsCommit642=== RUN TestService_NativeMTLS643=== PAUSE TestService_NativeMTLS644=== RUN TestServerTLSConfig645=== PAUSE TestServerTLSConfig646=== RUN TestMultipartCleanup647=== PAUSE TestMultipartCleanup648=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded649=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded650=== RUN TestPendingCleanupUsesOneCutoff651=== PAUSE TestPendingCleanupUsesOneCutoff652=== RUN TestObjectStatsTrigger653=== PAUSE TestObjectStatsTrigger654=== RUN TestObjectStatsUpdatedOncePerCommit655=== PAUSE TestObjectStatsUpdatedOncePerCommit656=== RUN TestOrphanedObjectsGC657=== PAUSE TestOrphanedObjectsGC658=== RUN TestOrphanedObjectsGCStressTest659=== PAUSE TestOrphanedObjectsGCStressTest660=== RUN TestCreatePin_ReservedPins661=== PAUSE TestCreatePin_ReservedPins662=== RUN TestPresentReportsOnlyClosureRoots663=== PAUSE TestPresentReportsOnlyClosureRoots664=== RUN TestPresentRefreshesOnlyStaleClosures665=== PAUSE TestPresentRefreshesOnlyStaleClosures666=== RUN TestPresentNotReportedWhileGCDeletesClosure667=== PAUSE TestPresentNotReportedWhileGCDeletesClosure668=== RUN TestParseSingleRange669=== PAUSE TestParseSingleRange670=== RUN TestProxyHeadersOnlyTrustedOnSocket6712026/11/12 00:09:36 INFO Starting HTTP server address=proxy.sock6722026/11/12 00:09:36 INFO Starting HTTP server address=127.0.0.1:630706732026/11/12 00:09:36 WARN mTLS auth: subject not in bound subjects subject="CN=someone"6742026/11/12 00:09:36 INFO Shutdown signal received, draining in-flight requests timeout=10s675--- PASS: TestProxyHeadersOnlyTrustedOnSocket (0.16s)676=== RUN TestIsValidCachePath677=== PAUSE TestIsValidCachePath678=== RUN TestReadProxyNarinfo679=== PAUSE TestReadProxyNarinfo680=== RUN TestReadProxyNarinfoAlreadyDecompressed681=== PAUSE TestReadProxyNarinfoAlreadyDecompressed682=== RUN TestReadProxyContentNegotiation683=== PAUSE TestReadProxyContentNegotiation684=== RUN TestReadProxyNarStreaming685=== PAUSE TestReadProxyNarStreaming686=== RUN TestReadProxy404687=== PAUSE TestReadProxy404688=== RUN TestReadProxyInvalidPath689=== PAUSE TestReadProxyInvalidPath690=== RUN TestReadProxyHead691=== PAUSE TestReadProxyHead692=== RUN TestReadProxyOutlastsServerWriteTimeout693=== PAUSE TestReadProxyOutlastsServerWriteTimeout694=== RUN TestReadProxyConditionalGet695=== PAUSE TestReadProxyConditionalGet696=== RUN TestReadProxyRootRedirectsToIndexHTML697=== PAUSE TestReadProxyRootRedirectsToIndexHTML698=== RUN TestReadProxyDisabled699=== PAUSE TestReadProxyDisabled700=== RUN TestReadRedirectNar701=== PAUSE TestReadRedirectNar702=== RUN TestReadRedirectKeepsNarinfoProxied703=== PAUSE TestReadRedirectKeepsNarinfoProxied704=== RUN TestReadProxyRangeRequest705=== PAUSE TestReadProxyRangeRequest706=== RUN TestReadRedirectUsesPublicS3URL707=== PAUSE TestReadRedirectUsesPublicS3URL708=== RUN TestPush_OverlappingRootsStoreOneRowPerKey709=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey710=== RUN TestPush_CompleteCommitsEveryRoot711=== PAUSE TestPush_CompleteCommitsEveryRoot712=== RUN TestPush_RepeatedRootCommitsOnce713=== PAUSE TestPush_RepeatedRootCommitsOnce714=== RUN TestPush_CompleteWaitsForCleanupOfThePush715=== PAUSE TestPush_CompleteWaitsForCleanupOfThePush716=== RUN TestPush_RootsAreLockedInByteOrder717=== PAUSE TestPush_RootsAreLockedInByteOrder718=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit719=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit720=== RUN TestPush_RejectsBadRequests721=== PAUSE TestPush_RejectsBadRequests722=== RUN TestPush_SignsNarinfosOfItsPendingObjects723=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects724=== RUN TestSharedNarBothUploadsComplete725=== PAUSE TestSharedNarBothUploadsComplete726=== RUN TestCompleteMultipartUpload_ErrorButObjectExists727=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists728=== RUN TestCompletedNarNotReofferedAcrossClosures729=== PAUSE TestCompletedNarNotReofferedAcrossClosures730=== RUN TestPresignedUploadRegisteredBeforeCommit731=== PAUSE TestPresignedUploadRegisteredBeforeCommit732=== RUN TestService_Rustfstest733=== PAUSE TestService_Rustfstest734=== RUN TestParseSize735=== PAUSE TestParseSize736=== RUN TestSkippedUploadsHandler737=== PAUSE TestSkippedUploadsHandler738=== RUN TestSystemdListenerNotActivated739--- PASS: TestSystemdListenerNotActivated (0.00s)740=== RUN TestWatchdogBeatsWhenHealthy741--- PASS: TestWatchdogBeatsWhenHealthy (0.03s)742=== RUN TestWatchdogSkipsWhenUnhealthy7432026/11/12 00:09:36 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7442026/11/12 00:09:36 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7452026/11/12 00:09:36 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7462026/11/12 00:09:36 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7472026/11/12 00:09:36 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7482026/11/12 00:09:36 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7492026/11/12 00:09:36 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7502026/11/12 00:09:36 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7512026/11/12 00:09:36 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7522026/11/12 00:09:37 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"753--- PASS: TestWatchdogSkipsWhenUnhealthy (0.21s)754=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle755=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle756=== RUN TestProxyWriteTimeout757=== PAUSE TestProxyWriteTimeout758=== RUN TestPendingClosureWriteTimeout759=== PAUSE TestPendingClosureWriteTimeout760=== RUN TestIsValidUploadKey761=== PAUSE TestIsValidUploadKey762=== RUN TestUploadHandlersRejectInvalidKeys763=== PAUSE TestUploadHandlersRejectInvalidKeys764=== RUN TestUploadHandlersRejectOversizedBody765=== PAUSE TestUploadHandlersRejectOversizedBody766=== RUN TestService_cleanupPendingClosuresHandler767=== PAUSE TestService_cleanupPendingClosuresHandler768=== RUN TestService_createPendingClosureHandler769=== PAUSE TestService_createPendingClosureHandler770=== RUN TestService_verifyS3Integrity771=== PAUSE TestService_verifyS3Integrity772=== RUN TestService_verifyS3ReuploadsStaleNarinfo773=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo774=== RUN TestCompleteMultipartUnregistered775=== PAUSE TestCompleteMultipartUnregistered776=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT777=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT778=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit779=== CONT TestGCTaskStore_DeduplicateSameParams780=== CONT TestOrphanedObjectsGCStressTest781=== CONT TestReadProxyInvalidPath782=== CONT TestPushWaitsForSweepHoldingRow783=== CONT TestGCTaskStore_GetEmpty784=== CONT TestReadRedirectUsesPublicS3URL785--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)786--- PASS: TestGCTaskStore_GetEmpty (0.00s)787=== CONT TestPush_RootsAreLockedInByteOrder788=== CONT TestGCTaskStore_ConflictDifferentParams789--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)790=== CONT TestReadProxyRangeRequest791=== CONT TestCreatePin_ReservedPins792=== CONT TestGCTaskStore_StartNew793--- PASS: TestGCTaskStore_StartNew (0.00s)794=== CONT TestForceGCDuringPushOffersSweptObject795=== CONT TestGCEndsOnShutdown796=== CONT TestPresentReportsOnlyClosureRoots797=== CONT TestPush_CompleteWaitsForCleanupOfThePush798=== RUN TestGCEndsOnShutdown/before_the_run799=== PAUSE TestGCEndsOnShutdown/before_the_run800=== CONT TestReadProxyHead801=== CONT TestCommitRacingPendingCleanupKeepsObjects802=== CONT TestReadProxyOutlastsServerWriteTimeout803=== CONT TestSweepRechecksPendingAfterLocking804=== CONT TestPush_RepeatedRootCommitsOnce805=== CONT TestGetExistingObjectsLocksRows806=== CONT TestRegisterCompletedObjectNeedsPendingRow807=== CONT TestPush_CompleteCommitsEveryRoot808=== CONT TestSweepSkipsRowsLockedByPush809=== CONT TestPush_OverlappingRootsStoreOneRowPerKey810=== CONT TestFailedS3DeleteKeepsTombstone811=== CONT TestAbortSucceeded812=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows813=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete814=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows815--- PASS: TestAbortSucceeded (0.00s)816=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete817=== CONT TestReadRedirectKeepsNarinfoProxied818=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check819=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark820=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check821=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark822=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection823=== CONT TestReadRedirectNar8242026/11/12 00:09:37 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:63072/oidc8252026/11/12 00:09:38 INFO Received push request method=POST path=/api/pushes826--- PASS: TestReadRedirectUsesPublicS3URL (1.37s)827=== CONT TestGCSweepDeliversEachKeyOnce8282026-11-12 00:09:38.464 UTC [10626] ERROR: Push does not exist: id=18292026-11-12 00:09:38.464 UTC [10626] CONTEXT: PL/pgSQL function commit_push(bigint) line 11 at RAISE8302026-11-12 00:09:38.464 UTC [10626] STATEMENT: SELECT commit_push($1)831--- PASS: TestPush_CompleteWaitsForCleanupOfThePush (1.45s)832=== CONT TestReadProxyDisabled833--- PASS: TestGetExistingObjectsLocksRows (1.63s)834=== CONT TestTombstonedObjectOfferedWithoutWaiting835--- PASS: TestReadProxyHead (2.12s)836=== CONT TestReadProxyRootRedirectsToIndexHTML8372026/11/12 00:09:39 INFO Aborted multipart uploads count=0 kept=08382026/11/12 00:09:39 WARN Force mode enabled - objects will be deleted immediately without grace period8392026/11/12 00:09:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=08402026/11/12 00:09:39 INFO Vacuumed table table=pending_closures8412026/11/12 00:09:39 INFO Vacuumed table table=pending_objects8422026/11/12 00:09:39 INFO Vacuumed table table=multipart_uploads8432026/11/12 00:09:39 INFO Vacuumed table table=closures8442026/11/12 00:09:39 INFO Vacuumed table table=objects845--- PASS: TestSweepRechecksPendingAfterLocking (2.41s)846=== CONT TestGCSweepSkipsPendingObjects8472026/11/12 00:09:39 INFO Received push request method=POST path=/api/pushes8482026/11/12 00:09:39 INFO Received complete push request method=POST path=/api/pushes/1/complete8492026/11/12 00:09:39 INFO Completed push id=1 objects=4 duration=23.383792ms850--- PASS: TestPush_RepeatedRootCommitsOnce (2.58s)851=== CONT TestReadProxyConditionalGet852--- PASS: TestPresentReportsOnlyClosureRoots (2.74s)853=== CONT TestReadProxyNarinfo8542026/11/12 00:09:40 INFO Received push request method=POST path=/api/pushes8552026/11/12 00:09:40 INFO Received complete push request method=POST path=/api/pushes/1/complete8562026/11/12 00:09:40 INFO Completed push id=1 objects=4 duration=44.520208ms8572026/11/12 00:09:40 INFO Received push request method=POST path=/api/pushes8582026/11/12 00:09:40 INFO Aborted multipart uploads count=0 kept=08592026/11/12 00:09:40 WARN Force mode enabled - objects will be deleted immediately without grace period8602026/11/12 00:09:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=08612026/11/12 00:09:40 INFO Vacuumed table table=pending_closures8622026/11/12 00:09:40 INFO Vacuumed table table=pending_objects8632026/11/12 00:09:40 INFO Vacuumed table table=multipart_uploads8642026/11/12 00:09:40 INFO Vacuumed table table=closures8652026/11/12 00:09:40 INFO Vacuumed table table=objects8662026/11/12 00:09:40 INFO Received complete push request method=POST path=/api/pushes/2/complete8672026/11/12 00:09:40 INFO Aborted multipart uploads count=0 kept=08682026/11/12 00:09:40 INFO Completed push id=2 objects=4 duration=10.28525ms869--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (3.44s)870=== CONT TestDeduplicatedObjectsRecordedAsPending8712026/11/12 00:09:40 WARN Force mode enabled - objects will be deleted immediately without grace period8722026/11/12 00:09:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=08732026/11/12 00:09:40 INFO Vacuumed table table=pending_closures8742026/11/12 00:09:40 INFO Vacuumed table table=pending_objects8752026/11/12 00:09:40 INFO Vacuumed table table=multipart_uploads8762026/11/12 00:09:40 INFO Vacuumed table table=closures8772026/11/12 00:09:40 INFO Vacuumed table table=objects8782026/11/12 00:09:40 INFO Aborted multipart uploads count=0 kept=08792026/11/12 00:09:40 WARN Force mode enabled - objects will be deleted immediately without grace period8802026/11/12 00:09:40 INFO Received uploads request method=POST path=/api/pending_closures8812026/11/12 00:09:40 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=08822026/11/12 00:09:40 INFO Vacuumed table table=pending_closures8832026/11/12 00:09:40 INFO Vacuumed table table=pending_objects8842026/11/12 00:09:40 INFO Vacuumed table table=multipart_uploads8852026/11/12 00:09:40 INFO Vacuumed table table=closures8862026/11/12 00:09:40 INFO Vacuumed table table=objects887--- PASS: TestSweepSkipsRowsLockedByPush (3.82s)888=== CONT TestReadProxy4048892026/11/12 00:09:40 INFO Aborted multipart uploads count=0 kept=08902026/11/12 00:09:40 WARN Force mode enabled - objects will be deleted immediately without grace period8912026/11/12 00:09:40 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."8922026/11/12 00:09:40 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."893--- PASS: TestFailedS3DeleteKeepsTombstone (3.89s)894=== CONT TestReadProxyNarStreaming8952026/11/12 00:09:40 INFO Received uploads request method=POST path=/api/pending_closures896--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (4.20s)8972026/11/12 00:09:41 INFO Received push request method=POST path=/api/pushes898=== CONT TestReadProxyContentNegotiation8992026-11-12 00:09:41.387 UTC [10672] ERROR: could not obtain lock on row in relation "closures"9002026-11-12 00:09:41.387 UTC [10672] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9012026-11-12 00:09:41.394 UTC [10672] ERROR: could not obtain lock on row in relation "closures"9022026-11-12 00:09:41.394 UTC [10672] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9032026-11-12 00:09:41.397 UTC [10672] ERROR: could not obtain lock on row in relation "closures"9042026-11-12 00:09:41.397 UTC [10672] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9052026-11-12 00:09:41.399 UTC [10672] ERROR: could not obtain lock on row in relation "closures"9062026-11-12 00:09:41.399 UTC [10672] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT9072026-11-12 00:09:41.400 UTC [10672] ERROR: could not obtain lock on row in relation "closures"9082026-11-12 00:09:41.400 UTC [10672] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT909--- PASS: TestPush_RootsAreLockedInByteOrder (4.41s)910=== CONT TestPinProtectsFromGC9112026/11/12 00:09:41 INFO Received push request method=POST path=/api/pushes912--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (4.61s)913=== CONT TestReadProxyNarinfoAlreadyDecompressed914--- PASS: TestRegisterCompletedObjectNeedsPendingRow (4.81s)915=== CONT TestClientSharedPathCommittedMidPush916--- PASS: TestReadRedirectNar (5.14s)917=== CONT TestPushDedupSurvivesConcurrentGC918--- PASS: TestReadProxyRangeRequest (5.42s)919=== CONT TestGCMetrics920--- PASS: TestReadProxyInvalidPath (5.63s)921=== CONT TestGCBugBareHashReferences922--- PASS: TestReadRedirectKeepsNarinfoProxied (5.98s)923=== CONT TestGCAdvisoryLockBlocksConcurrentRun9242026/11/12 00:09:43 INFO Aborted multipart uploads count=0 kept=09252026/11/12 00:09:43 WARN Force mode enabled - objects will be deleted immediately without grace period9262026/11/12 00:09:43 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=09272026/11/12 00:09:43 INFO Vacuumed table table=pending_closures9282026/11/12 00:09:43 INFO Vacuumed table table=pending_objects9292026/11/12 00:09:43 INFO Received push request method=POST path=/api/pushes9302026/11/12 00:09:43 INFO Vacuumed table table=multipart_uploads9312026/11/12 00:09:44 INFO Vacuumed table table=closures9322026/11/12 00:09:44 INFO Vacuumed table table=objects933--- PASS: TestCommitRacingPendingCleanupKeepsObjects (7.08s)934=== CONT TestLeadEndsOnShutdown9352026/11/12 00:09:44 INFO Received complete push request method=POST path=/api/pushes/1/complete9362026/11/12 00:09:44 INFO Completed push id=1 objects=6 duration=57.282333ms937--- PASS: TestPush_CompleteCommitsEveryRoot (7.17s)938=== CONT TestService_ReadScope_PublicByDefault9392026/11/12 00:09:44 INFO Received uploads request method=POST path=/api/pending_closures9402026/11/12 00:09:44 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux9412026/11/12 00:09:44 WARN Refused reserved pin name=worker-x86_64-linux9422026/11/12 00:09:44 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux9432026/11/12 00:09:44 INFO Received create pin request method=POST path=/api/pins/my-app9442026/11/12 00:09:44 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux945--- PASS: TestCreatePin_ReservedPins (7.65s)946=== CONT TestService_RequireScope_OIDC9472026/11/12 00:09:44 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:63131/oidc948--- PASS: TestPushWaitsForSweepHoldingRow (7.86s)949=== CONT TestService_AuthMiddleware_OIDC950--- PASS: TestReadProxyDisabled (6.42s)951=== CONT TestCacheConfigHandler952=== RUN TestCacheConfigHandler/full_config,_no_issuer953=== PAUSE TestCacheConfigHandler/full_config,_no_issuer954=== RUN TestCacheConfigHandler/no_cache_url_configured955=== PAUSE TestCacheConfigHandler/no_cache_url_configured956=== RUN TestCacheConfigHandler/no_signing_keys957=== PAUSE TestCacheConfigHandler/no_signing_keys958=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator959=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator960=== CONT TestClientWithDependencies9612026/11/12 00:09:44 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:63137/oidc9622026/11/12 00:09:45 INFO Received uploads request method=POST path=/api/pending_closures963--- PASS: TestTombstonedObjectOfferedWithoutWaiting (6.53s)964=== CONT TestClientMultipleUploads9652026/11/12 00:09:45 INFO Aborted multipart uploads count=0 kept=09662026/11/12 00:09:45 WARN Force mode enabled - objects will be deleted immediately without grace period967--- PASS: TestReadProxyRootRedirectsToIndexHTML (6.64s)968=== CONT TestClientIntegration9692026/11/12 00:09:46 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=09702026/11/12 00:09:46 INFO Vacuumed table table=pending_closures9712026/11/12 00:09:46 INFO Vacuumed table table=pending_objects9722026/11/12 00:09:46 INFO Vacuumed table table=multipart_uploads9732026/11/12 00:09:46 INFO Vacuumed table table=closures9742026/11/12 00:09:46 INFO Received uploads request method=POST path=/api/pending_closures9752026/11/12 00:09:46 INFO Vacuumed table table=objects976--- PASS: TestGCSweepDeliversEachKeyOnce (7.77s)977=== CONT TestClientErrorHandling978=== RUN TestClientErrorHandling/InvalidStorePath979=== PAUSE TestClientErrorHandling/InvalidStorePath980=== RUN TestClientErrorHandling/InvalidAuthToken981=== PAUSE TestClientErrorHandling/InvalidAuthToken982=== RUN TestClientErrorHandling/ServerNotAvailable983=== PAUSE TestClientErrorHandling/ServerNotAvailable984=== CONT TestClientCADerivations9852026/11/12 00:09:46 INFO Aborted multipart uploads count=0 kept=09862026/11/12 00:09:46 WARN Force mode enabled - objects will be deleted immediately without grace period9872026/11/12 00:09:46 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=09882026/11/12 00:09:46 INFO Vacuumed table table=pending_closures9892026/11/12 00:09:46 INFO Vacuumed table table=pending_objects9902026/11/12 00:09:46 INFO Vacuumed table table=multipart_uploads9912026/11/12 00:09:46 INFO Vacuumed table table=closures9922026/11/12 00:09:46 INFO Vacuumed table table=objects993--- PASS: TestGCSweepSkipsPendingObjects (7.04s)994=== CONT TestCacheStatsHandler995--- PASS: TestReadProxyConditionalGet (6.89s)996=== CONT TestLeadGenerationGrowsWithEveryGrant9972026/11/12 00:09:46 ERROR Failed to decode narinfo key=3hcdxyjf9yiq7qf3i4548drb6sjmwa1v.narinfo limit=16777216 error=<nil>998--- PASS: TestReadProxyNarinfo (7.12s)999=== CONT TestNARDeduplicationMetadataUploadBug10002026/11/12 00:09:47 INFO Received uploads request method=POST path=/api/pending_closures1001--- PASS: TestDeduplicatedObjectsRecordedAsPending (6.70s)1002=== CONT TestLeadIncumbentAnnouncesAtOnce1003--- PASS: TestReadProxyOutlastsServerWriteTimeout (10.56s)1004=== CONT TestOrphanedObjectsGC1005--- PASS: TestReadProxyNarStreaming (6.96s)1006=== CONT TestLeadUnannouncedStandbyYieldsToIncumbent1007--- PASS: TestReadProxy404 (7.22s)1008=== CONT TestObjectStatsUpdatedOncePerCommit1009--- PASS: TestReadProxyContentNegotiation (7.16s)1010=== CONT TestLeadElectsOneAndHandsOver1011--- PASS: TestReadProxyNarinfoAlreadyDecompressed (7.18s)1012=== CONT TestConnectWaitsForAPeerMigration1013=== NAME TestPinProtectsFromGC1014 client_integration_test.go:779: Pinned store path: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestPinProtectsFromGC2573990013/001/store/h6z518smx0w0qbgq3lk619x381lkfxyh-pinned-file.txt1015 client_integration_test.go:780: Unpinned store path: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestPinProtectsFromGC2573990013/001/store/pyjvajavhm8dksj8cyzkz49fj868lwqg-unpinned-file.txt10162026/11/12 00:09:48 INFO Received push request method=POST path=/api/pushes10172026/11/12 00:09:48 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)10182026/11/12 00:09:49 INFO Uploading h6z518smx0w0qbgq3lk619x381lkfxyh-pinned-file.txt (128B)10192026/11/12 00:09:49 WARN Failed to register uploaded object key=h6z518smx0w0qbgq3lk619x381lkfxyh.ls error="server returned 404: 404 page not found\n"10202026/11/12 00:09:49 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"10212026/11/12 00:09:49 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign10222026/11/12 00:09:49 INFO Signed narinfos id=1 count=110232026/11/12 00:09:49 INFO Uploading 1 narinfos10242026/11/12 00:09:49 INFO Received complete push request method=POST path=/api/pushes/1/complete10252026/11/12 00:09:49 WARN Failed to register uploaded object key=h6z518smx0w0qbgq3lk619x381lkfxyh.narinfo error="server returned 404: 404 page not found\n"10262026/11/12 00:09:49 INFO Completed push id=1 objects=3 duration=28.051625ms10272026/11/12 00:09:49 INFO Upload complete. (262ms)10282026/11/12 00:09:49 INFO Received push request method=POST path=/api/pushes10292026/11/12 00:09:49 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)10302026/11/12 00:09:49 INFO Uploading pyjvajavhm8dksj8cyzkz49fj868lwqg-unpinned-file.txt (128B)10312026/11/12 00:09:49 WARN Failed to register uploaded object key=pyjvajavhm8dksj8cyzkz49fj868lwqg.ls error="server returned 404: 404 page not found\n"10322026/11/12 00:09:49 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign10332026/11/12 00:09:49 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"10342026/11/12 00:09:49 INFO Signed narinfos id=2 count=110352026/11/12 00:09:49 INFO Uploading 1 narinfos10362026/11/12 00:09:49 INFO Received complete push request method=POST path=/api/pushes/2/complete10372026/11/12 00:09:49 WARN Failed to register uploaded object key=pyjvajavhm8dksj8cyzkz49fj868lwqg.narinfo error="server returned 404: 404 page not found\n"10382026/11/12 00:09:49 INFO Completed push id=2 objects=3 duration=852.916µs10392026/11/12 00:09:49 INFO Upload complete. (174ms)10402026/11/12 00:09:49 INFO Starting cleanup of old closures method=DELETE path=/api/closures10412026/11/12 00:09:49 INFO Garbage collection started10422026/11/12 00:09:49 INFO Aborted multipart uploads count=0 kept=010432026/11/12 00:09:49 WARN Force mode enabled - objects will be deleted immediately without grace period10442026/11/12 00:09:49 INFO Received uploads request method=POST path=/api/pending_closures10452026/11/12 00:09:49 INFO Received push request method=POST path=/api/pushes10462026/11/12 00:09:49 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)10472026/11/12 00:09:49 INFO Uploading skwppfnzi2ff6ch8p3dqfl6rh1p5v573-shared-dep (136B)10482026/11/12 00:09:49 INFO Uploading giar3gc73xqvsiyj39v7c26fjivjlh1q-top (264B)10492026/11/12 00:09:49 WARN Failed to register uploaded object key=skwppfnzi2ff6ch8p3dqfl6rh1p5v573.ls error="server returned 404: 404 page not found\n"10502026/11/12 00:09:49 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"10512026/11/12 00:09:49 WARN Failed to register uploaded object key=giar3gc73xqvsiyj39v7c26fjivjlh1q.ls error="server returned 404: 404 page not found\n"10522026/11/12 00:09:49 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign10532026/11/12 00:09:49 WARN Failed to register uploaded object key=nar/1i09p96mm60h55qn5g7jhz8qa8rfzlhmry0cy8l7dp4a4n55mnjn.nar.zst error="server returned 404: 404 page not found\n"10542026/11/12 00:09:49 INFO Signed narinfos id=1 count=210552026/11/12 00:09:49 INFO Aborted multipart uploads count=0 kept=010562026/11/12 00:09:49 INFO Uploading 2 narinfos10572026/11/12 00:09:49 INFO Aborted multipart uploads count=0 kept=010582026/11/12 00:09:49 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010592026/11/12 00:09:49 WARN Force mode enabled - objects will be deleted immediately without grace period10602026/11/12 00:09:49 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010612026/11/12 00:09:49 INFO Vacuumed table table=pending_closures10622026/11/12 00:09:49 INFO Vacuumed table table=pending_objects10632026/11/12 00:09:49 WARN Failed to register uploaded object key=giar3gc73xqvsiyj39v7c26fjivjlh1q.narinfo error="server returned 404: 404 page not found\n"10642026/11/12 00:09:49 INFO Received complete push request method=POST path=/api/pushes/1/complete10652026/11/12 00:09:49 INFO Vacuumed table table=multipart_uploads10662026/11/12 00:09:49 WARN Failed to register uploaded object key=skwppfnzi2ff6ch8p3dqfl6rh1p5v573.narinfo error="server returned 404: 404 page not found\n"10672026/11/12 00:09:49 INFO Vacuumed table table=closures10682026/11/12 00:09:49 INFO Vacuumed table table=objects1069--- PASS: TestGCMetrics (7.43s)1070=== CONT TestObjectStatsTrigger10712026/11/12 00:09:49 INFO Vacuumed table table=pending_closures10722026/11/12 00:09:49 INFO Completed push id=1 objects=6 duration=56.679625ms10732026/11/12 00:09:49 INFO Upload complete. (398ms)1074=== NAME TestClientSharedPathCommittedMidPush1075 client_integration_test.go:728: Retrieved narinfo from S3:1076 StorePath: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientSharedPathCommittedMidPush2920505365/001/store/skwppfnzi2ff6ch8p3dqfl6rh1p5v573-shared-dep1077 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1078 Compression: zstd1079 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821080 NarSize: 1361081 References: 1082 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n10832026/11/12 00:09:49 INFO Vacuumed table table=pending_objects1084 client_integration_test.go:728: Retrieved narinfo from S3:1085 StorePath: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientSharedPathCommittedMidPush2920505365/001/store/giar3gc73xqvsiyj39v7c26fjivjlh1q-top1086 URL: nar/1i09p96mm60h55qn5g7jhz8qa8rfzlhmry0cy8l7dp4a4n55mnjn.nar.zst1087 Compression: zstd1088 NarHash: sha256:1i09p96mm60h55qn5g7jhz8qa8rfzlhmry0cy8l7dp4a4n55mnjn10892026/11/12 00:09:49 INFO Vacuumed table table=multipart_uploads1090 NarSize: 2641091 References: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientSharedPathCommittedMidPush2920505365/001/store/skwppfnzi2ff6ch8p3dqfl6rh1p5v573-shared-dep1092 CA: text:sha256:1j5871f8n538c2bbai4rn1b91fgvysrnq6cd5217raqnpxirk3k810932026/11/12 00:09:49 INFO Vacuumed table table=closures10942026/11/12 00:09:49 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=010952026/11/12 00:09:49 INFO Vacuumed table table=objects10962026/11/12 00:09:50 INFO Vacuumed table table=pending_closures10972026/11/12 00:09:50 INFO Aborted multipart uploads count=0 kept=01098--- PASS: TestClientSharedPathCommittedMidPush (8.19s)1099=== CONT TestResolveDBConnectionString11002026/11/12 00:09:50 WARN Force mode enabled - objects will be deleted immediately without grace period11012026/11/12 00:09:50 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=01102=== RUN TestResolveDBConnectionString/flag_wins1103=== PAUSE TestResolveDBConnectionString/flag_wins1104=== RUN TestResolveDBConnectionString/file_when_flag_empty1105=== PAUSE TestResolveDBConnectionString/file_when_flag_empty1106=== RUN TestResolveDBConnectionString/missing_file_is_an_error1107=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error1108=== RUN TestResolveDBConnectionString/PGHOST_allows_empty1109=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty1110=== RUN TestResolveDBConnectionString/nothing_configured1111=== PAUSE TestResolveDBConnectionString/nothing_configured1112=== CONT TestPendingCleanupUsesOneCutoff11132026/11/12 00:09:50 INFO Vacuumed table table=pending_objects11142026/11/12 00:09:50 INFO Vacuumed table table=pending_closures11152026/11/12 00:09:50 INFO Vacuumed table table=multipart_uploads11162026/11/12 00:09:50 INFO Vacuumed table table=pending_objects11172026/11/12 00:09:50 INFO Vacuumed table table=multipart_uploads11182026/11/12 00:09:50 INFO Vacuumed table table=closures11192026/11/12 00:09:50 INFO Vacuumed table table=closures11202026/11/12 00:09:50 INFO Vacuumed table table=objects11212026/11/12 00:09:50 INFO Vacuumed table table=objects1122--- PASS: TestPushDedupSurvivesConcurrentGC (7.97s)1123=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock1124=== NAME TestOrphanedObjectsGCStressTest1125 orphaned_objects_gc_test.go:422: Created 10 active closures, 5 to-delete closures, 20 orphaned chains1126 orphaned_objects_gc_test.go:440: Marked 210 objects for deletion1127--- PASS: TestGCBugBareHashReferences (7.79s)1128=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded11292026/11/12 00:09:50 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=_tribuchetbld7 database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"1130--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (7.47s)1131=== CONT TestDeleteClosuresRacingPins11322026/11/12 00:09:50 INFO lead: acquired remote=192.0.2.1:1234 generation=11133--- PASS: TestService_ReadScope_PublicByDefault (7.03s)1134=== CONT TestService_ReadAuthMiddleware11352026/11/12 00:09:51 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=011362026/11/12 00:09:51 INFO Received create pin request method=POST path=/api/pins/myapp1137=== RUN TestService_RequireScope_OIDC/builder_may_write1138=== PAUSE TestService_RequireScope_OIDC/builder_may_write1139=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1140=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1141=== RUN TestService_RequireScope_OIDC/ops_may_admin1142=== PAUSE TestService_RequireScope_OIDC/ops_may_admin1143=== RUN TestService_RequireScope_OIDC/ops_may_not_write1144=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1145=== RUN TestService_RequireScope_OIDC/reader_may_not_write1146=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1147=== RUN TestService_RequireScope_OIDC/static_token_may_admin11482026/11/12 00:09:51 INFO Created/updated pin name=myapp store_path=/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestPinProtectsFromGC2573990013/001/store/h6z518smx0w0qbgq3lk619x381lkfxyh-pinned-file.txt narinfo_key=h6z518smx0w0qbgq3lk619x381lkfxyh.narinfo1149=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1150=== RUN TestService_RequireScope_OIDC/static_token_may_write1151=== PAUSE TestService_RequireScope_OIDC/static_token_may_write1152=== RUN TestService_RequireScope_OIDC/reader_may_read1153=== PAUSE TestService_RequireScope_OIDC/reader_may_read1154=== RUN TestService_RequireScope_OIDC/writer_implies_read11552026/11/12 00:09:51 INFO Starting cleanup of old closures method=DELETE path=/api/closures1156=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1157=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1158=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read1159=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc1160=== PAUSE TestService_RequireScope_OIDC/reader_may_read_with_netrc1161=== RUN TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1162=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1163=== RUN TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1164=== PAUSE TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1165=== RUN TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1166=== PAUSE TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1167=== CONT TestMultipartCleanup11682026/11/12 00:09:51 INFO Garbage collection started11692026/11/12 00:09:51 INFO Aborted multipart uploads count=0 kept=011702026/11/12 00:09:51 WARN Force mode enabled - objects will be deleted immediately without grace period11712026/11/12 00:09:51 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011722026/11/12 00:09:51 INFO Vacuumed table table=pending_closures11732026/11/12 00:09:51 INFO Vacuumed table table=pending_objects11742026/11/12 00:09:51 INFO Vacuumed table table=multipart_uploads11752026/11/12 00:09:51 INFO Vacuumed table table=closures11762026/11/12 00:09:51 INFO Vacuumed table table=objects11772026/11/12 00:09:51 INFO lead: released remote=192.0.2.1:1234 generation=11178--- PASS: TestLeadEndsOnShutdown (7.81s)1179=== CONT TestClientFallsBackToClosures1180=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token1181=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token1182=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1183=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1184=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1185=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1186=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1187=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1188=== CONT TestService_AuthMiddleware_MTLSBoundSubjects1189=== NAME TestClientWithDependencies1190 client_integration_test.go:661: Built derivation: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientWithDependencies509333623/001/store/vbl02gsjvbg5xwjhabgya7s26sf2rb25-test-script1191 client_integration_test.go:663: Found 1 dependencies (including self)11922026/11/12 00:09:52 INFO Received push request method=POST path=/api/pushes11932026/11/12 00:09:52 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)11942026/11/12 00:09:52 INFO Uploading vbl02gsjvbg5xwjhabgya7s26sf2rb25-test-script (136B)11952026/11/12 00:09:52 WARN Failed to register uploaded object key=vbl02gsjvbg5xwjhabgya7s26sf2rb25.ls error="server returned 404: 404 page not found\n"11962026/11/12 00:09:52 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"11972026/11/12 00:09:52 WARN Failed to register uploaded object key=log/ngw9jk6q00j1pnvddilysddg3nm21wis-test-script.drv error="server returned 404: 404 page not found\n"11982026/11/12 00:09:52 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign11992026/11/12 00:09:52 INFO Signed narinfos id=1 count=112002026/11/12 00:09:52 INFO Uploading 1 narinfos12012026/11/12 00:09:52 INFO Received complete push request method=POST path=/api/pushes/1/complete12022026/11/12 00:09:52 WARN Failed to register uploaded object key=vbl02gsjvbg5xwjhabgya7s26sf2rb25.narinfo error="server returned 404: 404 page not found\n"12032026/11/12 00:09:52 INFO Completed push id=1 objects=4 duration=36.902459ms12042026/11/12 00:09:52 INFO Upload complete. (302ms)1205 client_integration_test.go:665: Skipping nix copy test - isolated store (/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientWithDependencies509333623/001/store) requires matching store prefix1206=== NAME TestClientMultipleUploads1207 client_integration_test.go:406: Created store path 0: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientMultipleUploads653843906/001/store/7x4fbd10ipxvw8v1wcdaii81qrjczyi9-test-file-0.txt1208--- PASS: TestClientWithDependencies (8.04s)1209=== CONT TestClientPushesUseOnePush1210=== NAME TestClientMultipleUploads1211 client_integration_test.go:406: Created store path 1: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientMultipleUploads653843906/001/store/dr6mhi5xkzz01iivial56i7dry5s64d1-test-file-1.txt1212=== NAME TestClientIntegration1213 client_integration_test.go:292: Created store path: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientIntegration1213674460/002/store/a7xj8hblaq5z6apfhhbw1srsghr8xd7v-test-file.txt1214=== NAME TestClientMultipleUploads1215 client_integration_test.go:406: Created store path 2: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientMultipleUploads653843906/001/store/b8638kl2zyhn2fl4ypz5nql2wy2bgby5-test-file-2.txt12162026/11/12 00:09:53 INFO Received push request method=POST path=/api/pushes12172026/11/12 00:09:53 INFO Received push request method=POST path=/api/pushes12182026/11/12 00:09:53 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)12192026/11/12 00:09:53 INFO Uploading a7xj8hblaq5z6apfhhbw1srsghr8xd7v-test-file.txt (152B)12202026/11/12 00:09:53 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)12212026/11/12 00:09:53 INFO Uploading b8638kl2zyhn2fl4ypz5nql2wy2bgby5-test-file-2.txt (160B)12222026/11/12 00:09:53 INFO Uploading 7x4fbd10ipxvw8v1wcdaii81qrjczyi9-test-file-0.txt (160B)12232026/11/12 00:09:53 INFO Uploading dr6mhi5xkzz01iivial56i7dry5s64d1-test-file-1.txt (160B)12242026/11/12 00:09:53 WARN Failed to register uploaded object key=a7xj8hblaq5z6apfhhbw1srsghr8xd7v.ls error="server returned 404: 404 page not found\n"12252026/11/12 00:09:53 WARN Failed to register uploaded object key=b8638kl2zyhn2fl4ypz5nql2wy2bgby5.ls error="server returned 404: 404 page not found\n"12262026/11/12 00:09:53 WARN Failed to register uploaded object key=dr6mhi5xkzz01iivial56i7dry5s64d1.ls error="server returned 404: 404 page not found\n"12272026/11/12 00:09:53 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"12282026/11/12 00:09:53 WARN Failed to register uploaded object key=7x4fbd10ipxvw8v1wcdaii81qrjczyi9.ls error="server returned 404: 404 page not found\n"12292026/11/12 00:09:53 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"12302026/11/12 00:09:53 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12312026/11/12 00:09:53 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"12322026/11/12 00:09:53 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12332026/11/12 00:09:53 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"12342026/11/12 00:09:53 INFO Signed narinfos id=1 count=112352026/11/12 00:09:53 INFO Signed narinfos id=1 count=312362026/11/12 00:09:53 INFO Uploading 1 narinfos12372026/11/12 00:09:53 INFO Uploading 3 narinfos12382026/11/12 00:09:53 INFO Received complete push request method=POST path=/api/pushes/1/complete12392026/11/12 00:09:53 WARN Failed to register uploaded object key=a7xj8hblaq5z6apfhhbw1srsghr8xd7v.narinfo error="server returned 404: 404 page not found\n"12402026/11/12 00:09:53 WARN Failed to register uploaded object key=dr6mhi5xkzz01iivial56i7dry5s64d1.narinfo error="server returned 404: 404 page not found\n"12412026/11/12 00:09:53 WARN Failed to register uploaded object key=b8638kl2zyhn2fl4ypz5nql2wy2bgby5.narinfo error="server returned 404: 404 page not found\n"12422026/11/12 00:09:53 INFO Received complete push request method=POST path=/api/pushes/1/complete12432026/11/12 00:09:53 WARN Failed to register uploaded object key=7x4fbd10ipxvw8v1wcdaii81qrjczyi9.narinfo error="server returned 404: 404 page not found\n"12442026/11/12 00:09:53 INFO Completed push id=1 objects=3 duration=40.33175ms12452026/11/12 00:09:53 INFO Upload complete. (349ms)12462026/11/12 00:09:53 INFO Completed push id=1 objects=9 duration=18.953958ms12472026/11/12 00:09:53 INFO Upload complete. (350ms)1248 client_integration_test.go:417: Uploaded 3 paths in 391.413333ms12492026/11/12 00:09:53 INFO lead: acquired remote=192.0.2.1:1234 generation=112502026/11/12 00:09:53 INFO All 1 paths already cached1251=== NAME TestClientIntegration1252 client_integration_test.go:318: Retrieved narinfo from S3:1253 StorePath: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientIntegration1213674460/002/store/a7xj8hblaq5z6apfhhbw1srsghr8xd7v-test-file.txt1254 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1255 Compression: zstd1256 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11257 NarSize: 1521258 References: 1259 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11260 client_integration_test.go:319: Retrieved .ls file from S3 (compressed size: 77 bytes)1261 client_integration_test.go:319: Decompressed .ls content (64 bytes):1262 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}1263--- PASS: TestClientMultipleUploads (8.41s)1264=== CONT TestService_AuthMiddleware_MTLSProxyHeader1265=== NAME TestPinProtectsFromGC1266 client_integration_test.go:888: Pin successfully protected closure from garbage collection1267--- PASS: TestPinProtectsFromGC (12.29s)1268=== CONT TestService_AuthMiddleware1269=== NAME TestClientCADerivations1270 client_ca_test.go:136: Built CA derivation: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientCADerivations840017147/001/store/f3rvcm0hms2i8mhvjn6d030hsrv2j1i5-ca-test1271 client_ca_test.go:139: Found 1 dependencies (including self)12722026/11/12 00:09:53 INFO Received push request method=POST path=/api/pushes12732026/11/12 00:09:53 INFO Object in database but missing from S3 key=a7xj8hblaq5z6apfhhbw1srsghr8xd7v.ls12742026/11/12 00:09:53 WARN Found objects in DB but missing from S3, will re-upload count=112752026/11/12 00:09:53 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)1276--- PASS: TestCacheStatsHandler (7.36s)1277=== CONT TestParseSingleRange1278=== RUN TestParseSingleRange/none1279=== PAUSE TestParseSingleRange/none1280=== RUN TestParseSingleRange/unknown_unit1281=== PAUSE TestParseSingleRange/unknown_unit1282=== RUN TestParseSingleRange/multi-range_ignored1283=== PAUSE TestParseSingleRange/multi-range_ignored1284=== RUN TestParseSingleRange/malformed_no_dash1285=== PAUSE TestParseSingleRange/malformed_no_dash12862026/11/12 00:09:53 INFO Received complete push request method=POST path=/api/pushes/2/complete1287=== RUN TestParseSingleRange/malformed_both_empty12882026/11/12 00:09:53 WARN Failed to register uploaded object key=a7xj8hblaq5z6apfhhbw1srsghr8xd7v.ls error="server returned 404: 404 page not found\n"1289=== PAUSE TestParseSingleRange/malformed_both_empty12902026/11/12 00:09:53 INFO Completed push id=2 objects=3 duration=514.75µs1291=== RUN TestParseSingleRange/malformed_end_before_start1292=== PAUSE TestParseSingleRange/malformed_end_before_start12932026/11/12 00:09:53 INFO Upload complete. (135ms)1294=== RUN TestParseSingleRange/closed1295=== NAME TestClientIntegration1296 client_integration_test.go:336: Retrieved .ls file from S3 (compressed size: 62 bytes)1297=== PAUSE TestParseSingleRange/closed1298=== RUN TestParseSingleRange/open-ended1299=== PAUSE TestParseSingleRange/open-ended1300=== RUN TestParseSingleRange/end_clamped_to_size1301=== PAUSE TestParseSingleRange/end_clamped_to_size1302=== RUN TestParseSingleRange/suffix1303=== PAUSE TestParseSingleRange/suffix1304=== RUN TestParseSingleRange/suffix_exceeds_size1305=== PAUSE TestParseSingleRange/suffix_exceeds_size1306=== RUN TestParseSingleRange/single_byte1307=== PAUSE TestParseSingleRange/single_byte1308=== RUN TestParseSingleRange/start_past_EOF1309=== PAUSE TestParseSingleRange/start_past_EOF1310=== NAME TestClientIntegration1311 client_integration_test.go:336: Decompressed .ls content (49 bytes):1312 {"version":1,"root":{"type":"regular","size":39}}1313=== RUN TestParseSingleRange/start_far_past_EOF1314=== PAUSE TestParseSingleRange/start_far_past_EOF1315=== CONT TestServerTLSConfig1316=== RUN TestServerTLSConfig/no_client_CA1317=== PAUSE TestServerTLSConfig/no_client_CA1318=== RUN TestServerTLSConfig/missing_CA_file1319=== PAUSE TestServerTLSConfig/missing_CA_file1320=== RUN TestServerTLSConfig/not_a_PEM_file1321=== PAUSE TestServerTLSConfig/not_a_PEM_file1322=== CONT TestMetricsCommit13232026/11/12 00:09:53 INFO Received push request method=POST path=/api/pushes13242026/11/12 00:09:53 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)13252026/11/12 00:09:53 INFO Uploading f3rvcm0hms2i8mhvjn6d030hsrv2j1i5-ca-test (144B)13262026/11/12 00:09:54 WARN Failed to register uploaded object key=f3rvcm0hms2i8mhvjn6d030hsrv2j1i5.ls error="server returned 404: 404 page not found\n"13272026/11/12 00:09:54 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"13282026/11/12 00:09:54 WARN Failed to register uploaded object key=log/lrvjp9vmm0b4fgbbyadavbnqmnf4s21l-ca-test.drv error="server returned 404: 404 page not found\n"13292026/11/12 00:09:54 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign13302026/11/12 00:09:54 INFO Signed narinfos id=1 count=113312026/11/12 00:09:54 INFO Uploading 1 narinfos13322026/11/12 00:09:54 INFO Received complete push request method=POST path=/api/pushes/1/complete13332026/11/12 00:09:54 WARN Failed to register uploaded object key=f3rvcm0hms2i8mhvjn6d030hsrv2j1i5.narinfo error="server returned 404: 404 page not found\n"13342026/11/12 00:09:54 INFO Completed push id=1 objects=4 duration=71.86875ms13352026/11/12 00:09:54 INFO Upload complete. (388ms)1336=== NAME TestClientCADerivations1337 client_ca_test.go:180: Narinfo contains CA field: StorePath: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientCADerivations840017147/001/store/f3rvcm0hms2i8mhvjn6d030hsrv2j1i5-ca-test1338 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1339 Compression: zstd1340 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1341 NarSize: 1441342 References: 1343 Deriver: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientCADerivations840017147/001/store/lrvjp9vmm0b4fgbbyadavbnqmnf4s21l-ca-test.drv1344 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1345 client_ca_test.go:185: Checking for realisation files in S3...1346 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1347 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache13482026/11/12 00:09:54 INFO Received push request method=POST path=/api/pushes13492026/11/12 00:09:54 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)1350 client_ca_test.go:258: nix copy output: error: binary cache 's3://bucket53?endpoint=http://localhost:63054®ion=eu-west-1' is for Nix stores with prefix '/nix/store', not '/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientCADerivations840017147/001/store'1351 client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 113522026/11/12 00:09:54 INFO Received complete push request method=POST path=/api/pushes/3/complete13532026/11/12 00:09:54 WARN Failed to register uploaded object key=a7xj8hblaq5z6apfhhbw1srsghr8xd7v.ls error="server returned 404: 404 page not found\n"13542026/11/12 00:09:54 INFO Completed push id=3 objects=3 duration=686.125µs13552026/11/12 00:09:54 INFO Upload complete. (104ms)1356=== NAME TestClientIntegration1357 client_integration_test.go:353: Retrieved .ls file from S3 (compressed size: 62 bytes)1358 client_integration_test.go:353: Decompressed .ls content (49 bytes):1359 {"version":1,"root":{"type":"regular","size":39}}1360 client_integration_test.go:364: Testing garbage collection...13612026/11/12 00:09:54 INFO Starting cleanup of old closures method=DELETE path=/api/closures13622026/11/12 00:09:54 INFO Garbage collection started1363=== NAME TestNARDeduplicationMetadataUploadBug1364 metadata_upload_test.go:48: First store path: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestNARDeduplicationMetadataUploadBug3061516300/001/store/nkh4728fgglmgdlbqv5jdj0hng0hq0ff-file1.txt13652026/11/12 00:09:54 INFO Aborted multipart uploads count=0 kept=013662026/11/12 00:09:54 INFO lead: acquired remote=192.0.2.1:1234 generation=113672026/11/12 00:09:54 INFO lead: released remote=192.0.2.1:1234 generation=113682026/11/12 00:09:54 WARN Force mode enabled - objects will be deleted immediately without grace period1369--- PASS: TestLeadIncumbentAnnouncesAtOnce (7.25s)1370=== CONT TestIsValidCachePath1371=== RUN TestIsValidCachePath/narinfo1372=== PAUSE TestIsValidCachePath/narinfo1373=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars1374--- PASS: TestClientCADerivations (8.26s)1375=== CONT TestService_NativeMTLS1376=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars1377=== RUN TestIsValidCachePath/nar_zst1378=== PAUSE TestIsValidCachePath/nar_zst1379=== RUN TestIsValidCachePath/nar_xz1380=== PAUSE TestIsValidCachePath/nar_xz1381=== RUN TestIsValidCachePath/nar_bz21382=== PAUSE TestIsValidCachePath/nar_bz21383=== RUN TestIsValidCachePath/nar_uncompressed1384=== PAUSE TestIsValidCachePath/nar_uncompressed1385=== RUN TestIsValidCachePath/ls1386=== PAUSE TestIsValidCachePath/ls1387=== RUN TestIsValidCachePath/log1388=== PAUSE TestIsValidCachePath/log1389=== RUN TestIsValidCachePath/realisation1390=== PAUSE TestIsValidCachePath/realisation1391=== RUN TestIsValidCachePath/nix-cache-info1392=== PAUSE TestIsValidCachePath/nix-cache-info1393=== RUN TestIsValidCachePath/index.html1394=== PAUSE TestIsValidCachePath/index.html1395=== RUN TestIsValidCachePath/traversal_parent1396=== PAUSE TestIsValidCachePath/traversal_parent1397=== RUN TestIsValidCachePath/traversal_in_middle1398=== PAUSE TestIsValidCachePath/traversal_in_middle1399=== RUN TestIsValidCachePath/invalid_char_e1400=== PAUSE TestIsValidCachePath/invalid_char_e1401=== RUN TestIsValidCachePath/invalid_char_u1402=== PAUSE TestIsValidCachePath/invalid_char_u1403=== RUN TestIsValidCachePath/random_path1404=== PAUSE TestIsValidCachePath/random_path1405=== RUN TestIsValidCachePath/empty1406=== PAUSE TestIsValidCachePath/empty1407=== RUN TestIsValidCachePath/leading_slash1408=== PAUSE TestIsValidCachePath/leading_slash1409=== RUN TestIsValidCachePath/wrong_extension1410=== PAUSE TestIsValidCachePath/wrong_extension1411=== RUN TestIsValidCachePath/short_hash1412=== PAUSE TestIsValidCachePath/short_hash1413=== CONT TestMetricsInventory14142026/11/12 00:09:54 INFO Received push request method=POST path=/api/pushes14152026/11/12 00:09:54 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)14162026/11/12 00:09:54 INFO Uploading nkh4728fgglmgdlbqv5jdj0hng0hq0ff-file1.txt (160B)14172026/11/12 00:09:54 INFO lead: released remote=192.0.2.1:1234 generation=114182026/11/12 00:09:54 WARN Failed to register uploaded object key=nkh4728fgglmgdlbqv5jdj0hng0hq0ff.ls error="server returned 404: 404 page not found\n"14192026/11/12 00:09:54 INFO lead: acquired remote=192.0.2.1:1234 generation=214202026/11/12 00:09:54 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign14212026/11/12 00:09:54 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"14222026/11/12 00:09:54 INFO Signed narinfos id=1 count=114232026/11/12 00:09:54 INFO Uploading 1 narinfos14242026/11/12 00:09:54 INFO Received complete push request method=POST path=/api/pushes/1/complete14252026/11/12 00:09:54 WARN Failed to register uploaded object key=nkh4728fgglmgdlbqv5jdj0hng0hq0ff.narinfo error="server returned 404: 404 page not found\n"14262026/11/12 00:09:54 INFO Completed push id=1 objects=3 duration=48.63075ms14272026/11/12 00:09:54 INFO Upload complete. (330ms)1428=== NAME TestNARDeduplicationMetadataUploadBug1429 metadata_upload_test.go:54: Retrieved narinfo from S3:1430 StorePath: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestNARDeduplicationMetadataUploadBug3061516300/001/store/nkh4728fgglmgdlbqv5jdj0hng0hq0ff-file1.txt1431 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1432 Compression: zstd1433 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1434 NarSize: 1601435 References: 1436 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1437 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)1438 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):1439 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}14402026/11/12 00:09:54 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=014412026/11/12 00:09:54 INFO Vacuumed table table=pending_closures14422026/11/12 00:09:54 INFO Vacuumed table table=pending_objects14432026/11/12 00:09:54 INFO Vacuumed table table=multipart_uploads1444 metadata_upload_test.go:64: Second store path (same content): /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestNARDeduplicationMetadataUploadBug3061516300/001/store/qz1gzp3l3gh99db2h9w60j66v530z7g9-file2.txt14452026/11/12 00:09:54 INFO Vacuumed table table=closures14462026/11/12 00:09:55 INFO Vacuumed table table=objects14472026/11/12 00:09:55 INFO lead: acquired remote=192.0.2.1:1234 generation=114482026/11/12 00:09:55 INFO Received push request method=POST path=/api/pushes14492026/11/12 00:09:55 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)14502026/11/12 00:09:55 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign14512026/11/12 00:09:55 WARN Failed to register uploaded object key=qz1gzp3l3gh99db2h9w60j66v530z7g9.ls error="server returned 404: 404 page not found\n"14522026/11/12 00:09:55 INFO Signed narinfos id=2 count=114532026/11/12 00:09:55 INFO Uploading 1 narinfos14542026/11/12 00:09:55 INFO Received complete push request method=POST path=/api/pushes/2/complete14552026/11/12 00:09:55 WARN Failed to register uploaded object key=qz1gzp3l3gh99db2h9w60j66v530z7g9.narinfo error="server returned 404: 404 page not found\n"14562026/11/12 00:09:55 INFO Completed push id=2 objects=3 duration=847.292µs14572026/11/12 00:09:55 INFO Upload complete. (169ms)1458 metadata_upload_test.go:76: Retrieved narinfo from S3:1459 StorePath: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestNARDeduplicationMetadataUploadBug3061516300/001/store/qz1gzp3l3gh99db2h9w60j66v530z7g9-file2.txt1460 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1461 Compression: zstd1462 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1463 NarSize: 1601464 References: 1465 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1466 metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes)1467 metadata_upload_test.go:77: Decompressed .ls content (49 bytes):1468 {"version":1,"root":{"type":"regular","size":44}}1469--- PASS: TestNARDeduplicationMetadataUploadBug (8.48s)1470=== CONT TestProxyWriteTimeout1471=== RUN TestProxyWriteTimeout/narinfo1472=== PAUSE TestProxyWriteTimeout/narinfo1473=== RUN TestProxyWriteTimeout/1_GiB_nar1474=== PAUSE TestProxyWriteTimeout/1_GiB_nar1475=== RUN TestProxyWriteTimeout/10_GiB_nar1476=== PAUSE TestProxyWriteTimeout/10_GiB_nar1477=== RUN TestProxyWriteTimeout/unknown_size1478=== PAUSE TestProxyWriteTimeout/unknown_size1479=== CONT TestPresentNotReportedWhileGCDeletesClosure1480=== RUN TestPresentNotReportedWhileGCDeletesClosure/fresh1481=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/fresh1482=== RUN TestPresentNotReportedWhileGCDeletesClosure/stale1483=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/stale1484=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT14852026/11/12 00:09:55 INFO lead: released remote=192.0.2.1:1234 generation=214862026/11/12 00:09:55 INFO lead: acquired remote=192.0.2.1:1234 generation=314872026/11/12 00:09:55 INFO lead: released remote=192.0.2.1:1234 generation=31488--- PASS: TestLeadGenerationGrowsWithEveryGrant (9.27s)1489=== CONT TestService_healthCheckHandler1490--- PASS: TestObjectStatsUpdatedOncePerCommit (7.73s)1491=== CONT TestCompleteMultipartUnregistered14922026/11/12 00:09:55 INFO lead: acquired remote=192.0.2.1:1234 generation=114932026/11/12 00:09:56 INFO lead: released remote=192.0.2.1:1234 generation=114942026/11/12 00:09:56 INFO lead: acquired remote=192.0.2.1:1234 generation=214952026/11/12 00:09:56 INFO lead: given back to the incumbent remote=192.0.2.1:1234 generation=214962026/11/12 00:09:56 INFO lead: acquired remote=192.0.2.1:1234 generation=31497--- PASS: TestObjectStatsTrigger (6.45s)1498=== CONT TestService_cleanupPendingClosuresHandler14992026/11/12 00:09:56 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=01500=== NAME TestClientIntegration1501 client_integration_test.go:371: Objects in database after GC:1502 client_integration_test.go:371: Successfully deleted all objects with GC --force1503=== CONT TestUploadHandlersRejectOversizedBody1504--- PASS: TestClientIntegration (10.83s)15052026/11/12 00:09:56 INFO Received uploads request method=POST path=/api/pending_closures1506=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure1507=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure1508=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart1509=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart1510=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts1511=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts1512=== CONT TestCreatePendingClosureRejectsOversizedNAR15132026/11/12 00:09:56 INFO Received uploads request method=POST path=/api/pending_closures1514--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.01s)1515=== CONT TestUploadHandlersRejectInvalidKeys1516=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1517=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1518=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal1519=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal1520=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key1521=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key1522=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key1523=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key1524=== CONT TestService_verifyS3ReuploadsStaleNarinfo1525=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1526=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1527=== RUN TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1528=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1529=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1530=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1531=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1532=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1533=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1534=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1535=== CONT TestIsValidUploadKey1536=== RUN TestIsValidUploadKey/narinfo1537=== PAUSE TestIsValidUploadKey/narinfo1538=== RUN TestIsValidUploadKey/nar_zst1539=== PAUSE TestIsValidUploadKey/nar_zst1540=== RUN TestIsValidUploadKey/nar_xz1541=== PAUSE TestIsValidUploadKey/nar_xz1542=== RUN TestIsValidUploadKey/nar_plain1543=== PAUSE TestIsValidUploadKey/nar_plain1544=== RUN TestIsValidUploadKey/listing1545=== PAUSE TestIsValidUploadKey/listing1546=== RUN TestIsValidUploadKey/build_log1547=== PAUSE TestIsValidUploadKey/build_log1548=== RUN TestIsValidUploadKey/build_log_home-manager_file1549=== PAUSE TestIsValidUploadKey/build_log_home-manager_file1550=== RUN TestIsValidUploadKey/build_log_plus_in_name1551=== PAUSE TestIsValidUploadKey/build_log_plus_in_name1552=== RUN TestIsValidUploadKey/build_log_question_mark1553=== PAUSE TestIsValidUploadKey/build_log_question_mark1554=== RUN TestIsValidUploadKey/build_log_equals1555=== PAUSE TestIsValidUploadKey/build_log_equals15562026/11/12 00:09:56 INFO Received uploads request method=POST path=/api/pending_closures1557=== RUN TestIsValidUploadKey/realisation1558=== PAUSE TestIsValidUploadKey/realisation1559=== RUN TestIsValidUploadKey/realisation_plus_in_output1560=== PAUSE TestIsValidUploadKey/realisation_plus_in_output1561=== RUN TestIsValidUploadKey/nix-cache-info1562=== PAUSE TestIsValidUploadKey/nix-cache-info1563=== RUN TestIsValidUploadKey/index.html1564=== PAUSE TestIsValidUploadKey/index.html1565=== RUN TestIsValidUploadKey/narinfo_key,_nar_type1566=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type1567=== RUN TestIsValidUploadKey/nar_key,_narinfo_type1568=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type1569=== RUN TestIsValidUploadKey/listing_key,_narinfo_type1570=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type1571=== RUN TestIsValidUploadKey/traversal1572=== PAUSE TestIsValidUploadKey/traversal1573=== RUN TestIsValidUploadKey/traversal_nar1574=== PAUSE TestIsValidUploadKey/traversal_nar15752026/11/12 00:09:56 INFO Received cleanup request method=DELETE path=/api/pending_closures1576=== RUN TestIsValidUploadKey/absolute1577=== PAUSE TestIsValidUploadKey/absolute1578=== RUN TestIsValidUploadKey/empty_key1579=== PAUSE TestIsValidUploadKey/empty_key1580=== RUN TestIsValidUploadKey/unknown_type1581=== PAUSE TestIsValidUploadKey/unknown_type1582=== CONT TestCacheConfigHandlerMaxNarSize1583--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)1584=== CONT TestPendingClosureWriteTimeout1585=== RUN TestPendingClosureWriteTimeout/empty1586=== PAUSE TestPendingClosureWriteTimeout/empty1587=== RUN TestPendingClosureWriteTimeout/400_objects15882026/11/12 00:09:57 INFO lead: released remote=192.0.2.1:1234 generation=11589=== PAUSE TestPendingClosureWriteTimeout/400_objects1590=== RUN TestPendingClosureWriteTimeout/670k_objects1591=== PAUSE TestPendingClosureWriteTimeout/670k_objects1592=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1593=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1594=== CONT TestService_verifyS3Integrity15952026/11/12 00:09:57 INFO lead: acquired remote=192.0.2.1:1234 generation=21596--- PASS: TestConnectWaitsForAPeerMigration (8.39s)1597=== CONT TestService_createPendingClosureHandler1598=== NAME TestOrphanedObjectsGC1599 orphaned_objects_gc_test.go:287: GC Test Summary:1600 orphaned_objects_gc_test.go:288: - Kept: 2 objects from closure A1601 orphaned_objects_gc_test.go:289: - Deleted: 2 objects from closure B1602 orphaned_objects_gc_test.go:290: - Deleted: 6 orphaned chain objects (X1->X2->X3)1603 orphaned_objects_gc_test.go:291: - Deleted: 2 orphaned single objects (Y)1604 orphaned_objects_gc_test.go:292: - Total deleted: 10 objects1605--- PASS: TestOrphanedObjectsGC (9.66s)1606=== CONT TestEnsureAbortUploadsRule16072026/11/12 00:09:57 INFO Received uploads request method=POST path=/api/pending_closures1608--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (7.55s)1609=== CONT TestGCTaskStore_PhaseUpdates1610--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)1611=== CONT TestGCTaskStore_CompletedAllowsNewTask1612--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)1613=== CONT TestGenerateLandingPage1614--- PASS: TestGenerateLandingPage (0.00s)1615=== CONT TestGracefulShutdownDrainsInflight16162026/11/12 00:09:58 INFO Starting HTTP server address=127.0.0.1:6325916172026/11/12 00:09:58 INFO Shutdown signal received, draining in-flight requests timeout=10s1618--- PASS: TestGracefulShutdownDrainsInflight (0.07s)1619=== CONT TestService_readinessHandler16202026/11/12 00:09:58 INFO lead: released remote=192.0.2.1:1234 generation=31621--- PASS: TestLeadUnannouncedStandbyYieldsToIncumbent (10.29s)1622=== CONT TestGCTaskStore_Fail1623--- PASS: TestGCTaskStore_Fail (0.00s)1624=== CONT TestPresignedUploadRegisteredBeforeCommit16252026/11/12 00:09:58 INFO lead: released remote=192.0.2.1:1234 generation=21626--- PASS: TestLeadElectsOneAndHandsOver (9.82s)1627=== CONT TestSharedNarBothUploadsComplete1628--- PASS: TestService_ReadAuthMiddleware (7.39s)1629=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle16302026/11/12 00:09:59 INFO Received uploads request method=POST path=/api/pending_closures16312026/11/12 00:09:59 INFO Received cleanup request method=DELETE path=/api/pending_closures16322026/11/12 00:09:59 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket68/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""16332026/11/12 00:09:59 INFO Aborted multipart uploads count=0 kept=116342026/11/12 00:09:59 INFO Received cleanup request method=DELETE path=/api/pending_closures16352026/11/12 00:09:59 INFO Aborted multipart uploads count=1 kept=01636--- PASS: TestMultipartCleanup (7.84s)1637=== CONT TestPresentRefreshesOnlyStaleClosures1638--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (9.35s)1639=== CONT TestCompletedNarNotReofferedAcrossClosures16402026/11/12 00:09:59 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"16412026/11/12 00:09:59 WARN mTLS auth: bound subjects configured but subject DN unavailable16422026/11/12 00:09:59 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1643--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (7.77s)1644=== CONT TestSkippedUploadsHandler16452026/11/12 00:09:59 INFO Client skipped oversized paths paths=3 nar_bytes=50000000001646--- PASS: TestSkippedUploadsHandler (0.00s)1647=== CONT TestPush_SignsNarinfosOfItsPendingObjects16482026/11/12 00:10:00 INFO Received uploads request method=POST path=/api/pending_closures16492026/11/12 00:10:00 INFO Received uploads request method=POST path=/api/pending_closures16502026/11/12 00:10:00 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)16512026/11/12 00:10:00 INFO Uploading 60q8wjj5zl8xyxa39zaiv684a3cafmkk-b (256B)16522026/11/12 00:10:00 INFO Uploading vimr5lbsgzj0rszj2mhz3b7kk8gpqcbr-shared-dep (136B)16532026/11/12 00:10:00 WARN Failed to register uploaded object key=4gf3ym3y1ai320j4yjaccarbs24qa8k6.ls error="server returned 404: 404 page not found\n"16542026/11/12 00:10:00 WARN Failed to register uploaded object key=60q8wjj5zl8xyxa39zaiv684a3cafmkk.ls error="server returned 404: 404 page not found\n"16552026/11/12 00:10:00 WARN Failed to register uploaded object key=nar/165jl7n1kvlf5znyps7pj7hpy0v1m5xxg2af2cpxsg5bprbpiysv.nar.zst error="server returned 404: 404 page not found\n"16562026/11/12 00:10:00 WARN Failed to register uploaded object key=vimr5lbsgzj0rszj2mhz3b7kk8gpqcbr.ls error="server returned 404: 404 page not found\n"16572026/11/12 00:10:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign16582026/11/12 00:10:00 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"16592026/11/12 00:10:00 INFO Signed narinfos id=1 count=216602026/11/12 00:10:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign16612026/11/12 00:10:00 INFO Signed narinfos id=2 count=216622026/11/12 00:10:00 INFO Uploading 4 narinfos16632026/11/12 00:10:00 WARN Failed to register uploaded object key=vimr5lbsgzj0rszj2mhz3b7kk8gpqcbr.narinfo error="server returned 404: 404 page not found\n"16642026/11/12 00:10:00 WARN Failed to register uploaded object key=60q8wjj5zl8xyxa39zaiv684a3cafmkk.narinfo error="server returned 404: 404 page not found\n"16652026/11/12 00:10:00 WARN Failed to register uploaded object key=4gf3ym3y1ai320j4yjaccarbs24qa8k6.narinfo error="server returned 404: 404 page not found\n"16662026/11/12 00:10:00 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete16672026/11/12 00:10:00 WARN Failed to register uploaded object key=vimr5lbsgzj0rszj2mhz3b7kk8gpqcbr.narinfo error="server returned 404: 404 page not found\n"1668--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (7.15s)1669=== CONT TestCompleteMultipartUpload_ErrorButObjectExists16702026/11/12 00:10:00 INFO Completed upload id=1 objects=6 duration=61.733833ms16712026/11/12 00:10:00 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete16722026/11/12 00:10:00 INFO Completed upload id=2 objects=6 duration=3.271042ms16732026/11/12 00:10:00 INFO Upload complete. (471ms)1674=== NAME TestClientFallsBackToClosures1675 client_pushes_test.go:112: Retrieved narinfo from S3:1676 StorePath: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientFallsBackToClosures469620883/001/store/vimr5lbsgzj0rszj2mhz3b7kk8gpqcbr-shared-dep1677 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1678 Compression: zstd1679 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821680 NarSize: 1361681 References: 1682 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1683 client_pushes_test.go:112: Retrieved narinfo from S3:1684 StorePath: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientFallsBackToClosures469620883/001/store/4gf3ym3y1ai320j4yjaccarbs24qa8k6-a1685 URL: nar/165jl7n1kvlf5znyps7pj7hpy0v1m5xxg2af2cpxsg5bprbpiysv.nar.zst1686 Compression: zstd1687 NarHash: sha256:165jl7n1kvlf5znyps7pj7hpy0v1m5xxg2af2cpxsg5bprbpiysv1688 NarSize: 2561689 References: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientFallsBackToClosures469620883/001/store/vimr5lbsgzj0rszj2mhz3b7kk8gpqcbr-shared-dep1690 CA: text:sha256:0zs41xs2d4vlfps49yndkw97vffhi01nhdz7d6xb370lbjljvclb1691 client_pushes_test.go:112: Retrieved narinfo from S3:1692 StorePath: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientFallsBackToClosures469620883/001/store/60q8wjj5zl8xyxa39zaiv684a3cafmkk-b1693 URL: nar/165jl7n1kvlf5znyps7pj7hpy0v1m5xxg2af2cpxsg5bprbpiysv.nar.zst1694 Compression: zstd1695 NarHash: sha256:165jl7n1kvlf5znyps7pj7hpy0v1m5xxg2af2cpxsg5bprbpiysv1696 NarSize: 2561697 References: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientFallsBackToClosures469620883/001/store/vimr5lbsgzj0rszj2mhz3b7kk8gpqcbr-shared-dep1698 CA: text:sha256:0zs41xs2d4vlfps49yndkw97vffhi01nhdz7d6xb370lbjljvclb1699--- PASS: TestClientFallsBackToClosures (9.06s)1700=== CONT TestParseSize1701--- PASS: TestParseSize (0.00s)1702=== CONT TestService_Rustfstest17032026/11/12 00:10:01 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"1704--- PASS: TestService_AuthMiddleware (7.39s)1705=== CONT TestGCTaskStore_GetReturnsLatest1706--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)1707=== CONT TestPush_RejectsBadRequests17082026/11/12 00:10:01 INFO Received push request method=POST path=/api/pushes17092026/11/12 00:10:01 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)17102026/11/12 00:10:01 INFO Uploading srja9508fqid9mb7hfp04hljim0gql98-shared-dep (136B)17112026/11/12 00:10:01 INFO Uploading 3h0jwpmg8l7ds4g73rbla3dxs89zpqq8-a (256B)17122026/11/12 00:10:01 WARN Failed to register uploaded object key=8vqcz4yalr190dxzskm8w6pysisvn8jf.ls error="server returned 404: 404 page not found\n"17132026/11/12 00:10:01 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"17142026/11/12 00:10:01 WARN Failed to register uploaded object key=srja9508fqid9mb7hfp04hljim0gql98.ls error="server returned 404: 404 page not found\n"17152026/11/12 00:10:01 WARN Failed to register uploaded object key=3h0jwpmg8l7ds4g73rbla3dxs89zpqq8.ls error="server returned 404: 404 page not found\n"17162026/11/12 00:10:01 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign17172026/11/12 00:10:01 WARN Failed to register uploaded object key=nar/0bl39d0pk7dlbmwh3jhvnx9qsa13yvqmf69khk9qv2wxby9r6bw5.nar.zst error="server returned 404: 404 page not found\n"17182026/11/12 00:10:01 INFO Signed narinfos id=1 count=317192026/11/12 00:10:01 INFO Uploading 3 narinfos17202026/11/12 00:10:01 WARN Failed to register uploaded object key=srja9508fqid9mb7hfp04hljim0gql98.narinfo error="server returned 404: 404 page not found\n"17212026/11/12 00:10:01 WARN Failed to register uploaded object key=3h0jwpmg8l7ds4g73rbla3dxs89zpqq8.narinfo error="server returned 404: 404 page not found\n"17222026/11/12 00:10:01 INFO Received complete push request method=POST path=/api/pushes/1/complete17232026/11/12 00:10:01 WARN Failed to register uploaded object key=8vqcz4yalr190dxzskm8w6pysisvn8jf.narinfo error="server returned 404: 404 page not found\n"17242026/11/12 00:10:01 INFO Completed push id=1 objects=8 duration=44.736959ms17252026/11/12 00:10:01 INFO Upload complete. (368ms)1726=== NAME TestClientPushesUseOnePush1727 client_pushes_test.go:97: Retrieved narinfo from S3:1728 StorePath: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientPushesUseOnePush2163357009/001/store/srja9508fqid9mb7hfp04hljim0gql98-shared-dep1729 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1730 Compression: zstd1731 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821732 NarSize: 1361733 References: 1734 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1735 client_pushes_test.go:97: Retrieved narinfo from S3:1736 StorePath: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientPushesUseOnePush2163357009/001/store/3h0jwpmg8l7ds4g73rbla3dxs89zpqq8-a1737 URL: nar/0bl39d0pk7dlbmwh3jhvnx9qsa13yvqmf69khk9qv2wxby9r6bw5.nar.zst1738 Compression: zstd1739 NarHash: sha256:0bl39d0pk7dlbmwh3jhvnx9qsa13yvqmf69khk9qv2wxby9r6bw51740 NarSize: 2561741 References: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientPushesUseOnePush2163357009/001/store/srja9508fqid9mb7hfp04hljim0gql98-shared-dep1742 CA: text:sha256:0fw4zaag8bab1br1x62kc7dbp19l77hpvb3943m8cr9wr0wg8d631743 client_pushes_test.go:97: Retrieved narinfo from S3:1744 StorePath: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientPushesUseOnePush2163357009/001/store/8vqcz4yalr190dxzskm8w6pysisvn8jf-b1745 URL: nar/0bl39d0pk7dlbmwh3jhvnx9qsa13yvqmf69khk9qv2wxby9r6bw5.nar.zst1746 Compression: zstd1747 NarHash: sha256:0bl39d0pk7dlbmwh3jhvnx9qsa13yvqmf69khk9qv2wxby9r6bw51748 NarSize: 2561749 References: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestClientPushesUseOnePush2163357009/001/store/srja9508fqid9mb7hfp04hljim0gql98-shared-dep1750 CA: text:sha256:0fw4zaag8bab1br1x62kc7dbp19l77hpvb3943m8cr9wr0wg8d6317512026/11/12 00:10:01 INFO Received push request method=POST path=/api/pushes1752--- PASS: TestClientPushesUseOnePush (8.70s)1753=== CONT TestForceGCDuringPushOffersSweptObject/before_pending_rows17542026/11/12 00:10:01 INFO Received complete push request method=POST path=/api/pushes/1/complete17552026/11/12 00:10:01 INFO Completed push id=1 objects=4 duration=37.606416ms17562026/11/12 00:10:01 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete17572026/11/12 00:10:01 INFO Completed upload id=2 objects=2 duration=788.709µs1758--- PASS: TestMetricsCommit (7.85s)1759=== CONT TestGCEndsOnShutdown/before_the_run17602026/11/12 00:10:01 WARN mTLS auth: subject not in bound subjects subject="CN=reader"17612026/11/12 00:10:01 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1762--- PASS: TestService_NativeMTLS (7.49s)1763=== CONT TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1764--- PASS: TestMetricsInventory (8.00s)1765=== CONT TestForceGCDuringPushOffersSweptObject/after_presence_check17662026/11/12 00:10:02 INFO Received uploads request method=POST path=/api/pending_closures1767--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (7.48s)1768=== CONT TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete17692026/11/12 00:10:03 INFO Aborted multipart uploads count=1 kept=017702026/11/12 00:10:03 INFO Received complete multipart upload request method=POST path=/api/multipart/complete17712026/11/12 00:10:03 INFO Received cleanup request method=DELETE path=/api/pending_closures17722026/11/12 00:10:03 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst1773--- PASS: TestCompleteMultipartUnregistered (7.40s)17742026/11/12 00:10:03 INFO Aborted multipart uploads count=1 kept=01775=== CONT TestCacheConfigHandler/full_config,_no_issuer1776=== CONT TestCacheConfigHandler/no_signing_keys1777=== CONT TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1778--- PASS: TestPendingCleanupUsesOneCutoff (13.16s)1779=== CONT TestCacheConfigHandler/no_cache_url_configured1780--- PASS: TestCacheConfigHandler (0.01s)1781 --- PASS: TestCacheConfigHandler/full_config,_no_issuer (0.00s)1782 --- PASS: TestCacheConfigHandler/no_signing_keys (0.00s)1783 --- PASS: TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator (0.00s)1784 --- PASS: TestCacheConfigHandler/no_cache_url_configured (0.00s)1785=== CONT TestClientErrorHandling/InvalidStorePath1786=== CONT TestClientErrorHandling/ServerNotAvailable17872026/11/12 00:10:03 INFO Received cleanup request method=DELETE path=/api/pending_closures17882026/11/12 00:10:03 INFO Aborted multipart uploads count=0 kept=017892026/11/12 00:10:03 INFO Received uploads request method=POST path=/api/pending_closures17902026/11/12 00:10:03 INFO Received cleanup request method=DELETE path=/api/pending_closures17912026/11/12 00:10:03 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present17922026/11/12 00:10:03 INFO Aborted multipart uploads count=1 kept=017932026/11/12 00:10:03 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete17942026-11-12 00:10:03.632 UTC [10920] ERROR: Closure does not exist: id=117952026-11-12 00:10:03.632 UTC [10920] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 13 at RAISE17962026-11-12 00:10:03.632 UTC [10920] STATEMENT: -- name: CommitPendingClosure :one1797 SELECT commit_pending_closure($1::bigint)::bigint1798 1799--- PASS: TestService_cleanupPendingClosuresHandler (7.31s)1800=== CONT TestClientErrorHandling/InvalidAuthToken18012026/11/12 00:10:03 WARN Request failed, retrying attempt=2 backoff=205.721019ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present1802--- PASS: TestService_healthCheckHandler (7.98s)1803=== CONT TestResolveDBConnectionString/flag_wins1804=== CONT TestResolveDBConnectionString/PGHOST_allows_empty1805=== CONT TestResolveDBConnectionString/nothing_configured1806=== CONT TestResolveDBConnectionString/missing_file_is_an_error1807=== CONT TestResolveDBConnectionString/file_when_flag_empty1808=== CONT TestService_RequireScope_OIDC/builder_may_write1809=== CONT TestService_RequireScope_OIDC/reader_may_read1810=== CONT TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1811--- PASS: TestResolveDBConnectionString (0.04s)1812 --- PASS: TestResolveDBConnectionString/flag_wins (0.00s)1813 --- PASS: TestResolveDBConnectionString/PGHOST_allows_empty (0.00s)1814 --- PASS: TestResolveDBConnectionString/nothing_configured (0.00s)1815 --- PASS: TestResolveDBConnectionString/missing_file_is_an_error (0.00s)1816 --- PASS: TestResolveDBConnectionString/file_when_flag_empty (0.00s)1817=== CONT TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read18182026/11/12 00:10:03 WARN Authentication failed token_preview=wrong token_length=5 oidc_error="not a JWT" oidc_provider=""1819=== CONT TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1820=== CONT TestService_RequireScope_OIDC/reader_may_read_with_netrc1821=== CONT TestService_RequireScope_OIDC/anonymous_may_not_read1822=== CONT TestService_RequireScope_OIDC/writer_implies_read1823=== CONT TestService_RequireScope_OIDC/reader_may_not_write1824=== CONT TestService_RequireScope_OIDC/static_token_may_write1825=== CONT TestService_RequireScope_OIDC/static_token_may_admin1826=== CONT TestService_RequireScope_OIDC/ops_may_admin1827=== CONT TestService_RequireScope_OIDC/ops_may_not_write1828=== CONT TestService_RequireScope_OIDC/builder_may_not_admin1829=== CONT TestService_AuthMiddleware_OIDC/valid_OIDC_token1830=== CONT TestService_AuthMiddleware_OIDC/malformed_token_rejected18312026/11/12 00:10:03 WARN Authentication failed token_preview=not-a-valid-jwt token_length=15 oidc_error="not a JWT" oidc_provider=""1832=== CONT TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1833=== CONT TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1834--- PASS: TestService_RequireScope_OIDC (6.96s)1835 --- PASS: TestService_RequireScope_OIDC/builder_may_write (0.00s)1836 --- PASS: TestService_RequireScope_OIDC/reader_may_read (0.00s)1837 --- PASS: TestService_RequireScope_OIDC/empty_netrc_password_may_not_read (0.00s)1838 --- PASS: TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read (0.00s)1839 --- PASS: TestService_RequireScope_OIDC/static_token_may_admin_with_netrc (0.00s)1840 --- PASS: TestService_RequireScope_OIDC/reader_may_read_with_netrc (0.00s)1841 --- PASS: TestService_RequireScope_OIDC/anonymous_may_not_read (0.00s)1842 --- PASS: TestService_RequireScope_OIDC/writer_implies_read (0.00s)1843 --- PASS: TestService_RequireScope_OIDC/reader_may_not_write (0.00s)1844 --- PASS: TestService_RequireScope_OIDC/static_token_may_write (0.00s)1845 --- PASS: TestService_RequireScope_OIDC/static_token_may_admin (0.00s)1846 --- PASS: TestService_RequireScope_OIDC/ops_may_admin (0.00s)1847 --- PASS: TestService_RequireScope_OIDC/ops_may_not_write (0.00s)1848 --- PASS: TestService_RequireScope_OIDC/builder_may_not_admin (0.00s)18492026/11/12 00:10:03 WARN Authentication failed token_preview=eyJhbGciOi...B8B0mD_qlg token_length=701 oidc_error="no rule matched: claim \"repository_owner\" value [otherorg] not in allowed patterns [myorg]" oidc_provider=test1850=== CONT TestParseSingleRange/none1851=== CONT TestParseSingleRange/open-ended1852=== CONT TestParseSingleRange/start_far_past_EOF1853=== CONT TestParseSingleRange/start_past_EOF1854=== CONT TestParseSingleRange/single_byte1855=== CONT TestParseSingleRange/suffix_exceeds_size1856=== CONT TestParseSingleRange/suffix1857=== CONT TestParseSingleRange/end_clamped_to_size1858--- PASS: TestService_AuthMiddleware_OIDC (7.34s)1859 --- PASS: TestService_AuthMiddleware_OIDC/valid_OIDC_token (0.00s)1860 --- PASS: TestService_AuthMiddleware_OIDC/malformed_token_rejected (0.01s)1861 --- PASS: TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured (0.00s)1862 --- PASS: TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected (0.01s)1863=== CONT TestParseSingleRange/malformed_both_empty1864=== CONT TestParseSingleRange/closed1865=== CONT TestParseSingleRange/malformed_end_before_start1866=== CONT TestParseSingleRange/multi-range_ignored1867=== CONT TestParseSingleRange/malformed_no_dash1868=== CONT TestParseSingleRange/unknown_unit1869--- PASS: TestParseSingleRange (0.06s)1870 --- PASS: TestParseSingleRange/none (0.00s)1871 --- PASS: TestParseSingleRange/open-ended (0.00s)1872 --- PASS: TestParseSingleRange/start_far_past_EOF (0.00s)1873 --- PASS: TestParseSingleRange/start_past_EOF (0.00s)1874 --- PASS: TestParseSingleRange/single_byte (0.00s)1875 --- PASS: TestParseSingleRange/suffix_exceeds_size (0.00s)1876 --- PASS: TestParseSingleRange/suffix (0.00s)1877 --- PASS: TestParseSingleRange/end_clamped_to_size (0.00s)1878 --- PASS: TestParseSingleRange/malformed_both_empty (0.00s)1879 --- PASS: TestParseSingleRange/closed (0.00s)1880 --- PASS: TestParseSingleRange/malformed_end_before_start (0.00s)1881 --- PASS: TestParseSingleRange/multi-range_ignored (0.00s)1882 --- PASS: TestParseSingleRange/malformed_no_dash (0.00s)1883 --- PASS: TestParseSingleRange/unknown_unit (0.00s)1884=== CONT TestServerTLSConfig/no_client_CA1885=== CONT TestServerTLSConfig/not_a_PEM_file1886=== CONT TestServerTLSConfig/missing_CA_file1887--- PASS: TestServerTLSConfig (0.01s)1888 --- PASS: TestServerTLSConfig/no_client_CA (0.00s)1889 --- PASS: TestServerTLSConfig/not_a_PEM_file (0.03s)1890 --- PASS: TestServerTLSConfig/missing_CA_file (0.00s)1891=== CONT TestIsValidCachePath/narinfo1892=== CONT TestIsValidCachePath/index.html1893=== CONT TestIsValidCachePath/short_hash1894=== CONT TestIsValidCachePath/wrong_extension1895=== CONT TestIsValidCachePath/leading_slash1896=== CONT TestIsValidCachePath/empty1897=== CONT TestIsValidCachePath/random_path1898=== CONT TestIsValidCachePath/invalid_char_u1899=== CONT TestIsValidCachePath/invalid_char_e1900=== CONT TestIsValidCachePath/traversal_in_middle1901=== CONT TestIsValidCachePath/traversal_parent1902=== CONT TestIsValidCachePath/nar_uncompressed1903=== CONT TestIsValidCachePath/nix-cache-info1904=== CONT TestIsValidCachePath/realisation1905=== CONT TestIsValidCachePath/log1906=== CONT TestIsValidCachePath/ls1907=== CONT TestIsValidCachePath/nar_xz1908=== CONT TestIsValidCachePath/nar_bz21909=== CONT TestIsValidCachePath/nar_zst1910=== CONT TestIsValidCachePath/narinfo_all_nix_base32_chars1911--- PASS: TestIsValidCachePath (0.04s)1912 --- PASS: TestIsValidCachePath/narinfo (0.00s)1913 --- PASS: TestIsValidCachePath/index.html (0.00s)1914 --- PASS: TestIsValidCachePath/short_hash (0.00s)1915 --- PASS: TestIsValidCachePath/wrong_extension (0.00s)1916 --- PASS: TestIsValidCachePath/leading_slash (0.00s)1917 --- PASS: TestIsValidCachePath/empty (0.00s)1918 --- PASS: TestIsValidCachePath/random_path (0.00s)1919 --- PASS: TestIsValidCachePath/invalid_char_u (0.00s)1920 --- PASS: TestIsValidCachePath/invalid_char_e (0.00s)1921 --- PASS: TestIsValidCachePath/traversal_in_middle (0.00s)1922 --- PASS: TestIsValidCachePath/traversal_parent (0.00s)1923 --- PASS: TestIsValidCachePath/nar_uncompressed (0.00s)1924 --- PASS: TestIsValidCachePath/nix-cache-info (0.00s)1925 --- PASS: TestIsValidCachePath/realisation (0.00s)1926 --- PASS: TestIsValidCachePath/log (0.00s)1927 --- PASS: TestIsValidCachePath/ls (0.00s)1928 --- PASS: TestIsValidCachePath/nar_xz (0.00s)1929 --- PASS: TestIsValidCachePath/nar_bz2 (0.00s)1930 --- PASS: TestIsValidCachePath/nar_zst (0.00s)1931 --- PASS: TestIsValidCachePath/narinfo_all_nix_base32_chars (0.00s)1932=== CONT TestProxyWriteTimeout/narinfo1933=== CONT TestPresentNotReportedWhileGCDeletesClosure/fresh19342026/11/12 00:10:03 WARN Request failed, retrying attempt=3 backoff=373.086843ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present19352026/11/12 00:10:04 WARN Request failed, retrying attempt=4 backoff=838.807778ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present19362026/11/12 00:10:04 INFO Installed lifecycle rule aborting incomplete NAR uploads bucket=bucket81 days=21937--- PASS: TestEnsureAbortUploadsRule (7.09s)1938=== CONT TestProxyWriteTimeout/unknown_size1939=== CONT TestProxyWriteTimeout/10_GiB_nar1940=== CONT TestProxyWriteTimeout/1_GiB_nar1941=== CONT TestPresentNotReportedWhileGCDeletesClosure/stale1942--- PASS: TestProxyWriteTimeout (0.01s)1943 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1944 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1945 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1946 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)19472026/11/12 00:10:04 INFO Received uploads request method=POST path=/api/pending_closures19482026/11/12 00:10:04 INFO Received uploads request method=POST path=/api/pending_closures19492026/11/12 00:10:04 INFO Received uploads request method=POST path=/api/pending_closures1950--- PASS: TestDeleteClosuresRacingPins (14.12s)1951=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure19522026/11/12 00:10:04 INFO Received uploads request method=POST path=/19532026/11/12 00:10:04 INFO Received uploads request method=POST path=/api/pending_closures1954=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts19552026/11/12 00:10:04 INFO Received request for more parts method=POST path=/1956=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart19572026/11/12 00:10:04 INFO Received complete multipart upload request method=POST path=/1958--- PASS: TestUploadHandlersRejectOversizedBody (0.07s)1959 --- PASS: TestUploadHandlersRejectOversizedBody/create_pending_closure (0.28s)1960 --- PASS: TestUploadHandlersRejectOversizedBody/request_more_parts (0.02s)1961 --- PASS: TestUploadHandlersRejectOversizedBody/complete_multipart (0.02s)1962=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info19632026/11/12 00:10:04 INFO Received uploads request method=POST path=/1964=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key19652026/11/12 00:10:04 INFO Received complete multipart upload request method=POST path=/1966=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key19672026/11/12 00:10:04 INFO Received request for more parts method=POST path=/1968=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal19692026/11/12 00:10:04 INFO Received uploads request method=POST path=/1970--- PASS: TestUploadHandlersRejectInvalidKeys (0.02s)1971 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.01s)1972 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1973 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.01s)1974 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.01s)1975=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed19762026/11/12 00:10:05 WARN Request failed, retrying attempt=5 backoff=1.750951763s error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present19772026/11/12 00:10:05 INFO Received uploads request method=POST path=/api/pending_closures19782026/11/12 00:10:05 INFO Received uploads request method=POST path=/api/pending_closures19792026/11/12 00:10:05 INFO Received uploads request method=POST path=/api/pending_closures19802026/11/12 00:10:05 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst19812026/11/12 00:10:05 INFO Received uploads request method=POST path=/api/pending_closures1982--- PASS: TestPresignedUploadRegisteredBeforeCommit (7.64s)1983=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed19842026/11/12 00:10:06 WARN readiness check failed error="closed pool"1985--- PASS: TestService_readinessHandler (8.03s)1986=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed19872026/11/12 00:10:06 INFO Received uploads request method=POST path=/api/pending_closures19882026/11/12 00:10:06 INFO Received complete multipart upload request method=POST path=/api/multipart/complete19892026/11/12 00:10:06 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=NzBkM2MzNDQtNjRhZi00YTc4LTg4OTQtNTJiZDdlN2RlNmVmLjI2MWFhMmE2LTcxMzItNDM1OC1iYWFkLWNkMDQxODg4NjBiNXgxNzk0NDQyMjA0NTM5NTY3MDAw parts=1019902026/11/12 00:10:06 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete19912026/11/12 00:10:06 INFO Completed upload id=1 objects=2 duration=31.973833ms19922026/11/12 00:10:06 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000019932026/11/12 00:10:06 INFO Received uploads request method=POST path=/api/pending_closures19942026/11/12 00:10:06 INFO Starting cleanup of old closures method=DELETE path=/api/closures19952026/11/12 00:10:06 INFO Aborted multipart uploads count=0 kept=019962026/11/12 00:10:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=019972026/11/12 00:10:06 INFO Vacuumed table table=pending_closures19982026/11/12 00:10:06 INFO Vacuumed table table=pending_objects19992026/11/12 00:10:06 WARN Request failed, retrying attempt=1 backoff=100ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config20002026/11/12 00:10:07 INFO Vacuumed table table=multipart_uploads
loading 662 hidden lines…
26632026/11/12 00:10:25 INFO Received push request method=POST path=/26642026/11/12 00:10:25 INFO Received push request method=POST path=/26652026/11/12 00:10:25 INFO Received uploads request method=POST path=/26662026/11/12 00:10:25 INFO Received uploads request method=POST path=/26672026/11/12 00:10:25 INFO Received push request method=POST path=/26682026/11/12 00:10:25 INFO Received uploads request method=POST path=/26692026/11/12 00:10:25 INFO Aborted multipart uploads count=0 kept=026702026/11/12 00:10:25 WARN Force mode enabled - objects will be deleted immediately without grace period26712026/11/12 00:10:25 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=026722026/11/12 00:10:25 INFO Vacuumed table table=pending_closures26732026/11/12 00:10:25 INFO Vacuumed table table=pending_closures26742026/11/12 00:10:25 INFO Received complete push request method=POST path=/26752026/11/12 00:10:25 INFO Completed push id=9 objects=4 duration=836.917µs26762026/11/12 00:10:25 INFO Vacuumed table table=pending_objects26772026/11/12 00:10:25 INFO Vacuumed table table=pending_objects26782026/11/12 00:10:25 INFO Vacuumed table table=multipart_uploads26792026/11/12 00:10:25 INFO Vacuumed table table=multipart_uploads26802026/11/12 00:10:25 INFO Vacuumed table table=closures26812026/11/12 00:10:25 INFO Vacuumed table table=closures26822026/11/12 00:10:25 INFO Received complete push request method=POST path=/26832026/11/12 00:10:25 INFO Vacuumed table table=objects26842026/11/12 00:10:25 INFO Vacuumed table table=objects26852026/11/12 00:10:25 INFO Aborted multipart uploads count=0 kept=026862026/11/12 00:10:25 WARN Force mode enabled - objects will be deleted immediately without grace period26872026/11/12 00:10:25 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=026882026/11/12 00:10:25 INFO Vacuumed table table=pending_closures26892026/11/12 00:10:25 INFO Completed push id=1 objects=8 duration=31.031209ms26902026/11/12 00:10:25 INFO Received uploads request method=POST path=/26912026/11/12 00:10:25 INFO Received push request method=POST path=/26922026/11/12 00:10:25 INFO Received push request method=POST path=/26932026/11/12 00:10:25 INFO Vacuumed table table=pending_objects26942026/11/12 00:10:26 INFO Vacuumed table table=multipart_uploads26952026/11/12 00:10:26 INFO Vacuumed table table=closures26962026/11/12 00:10:26 INFO Vacuumed table table=objects26972026/11/12 00:10:26 INFO Received push request method=POST path=/26982026/11/12 00:10:26 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."26992026/11/12 00:10:26 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."27002026/11/12 00:10:26 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."27012026/11/12 00:10:26 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."27022026/11/12 00:10:26 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."27032026/11/12 00:10:26 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."27042026/11/12 00:10:26 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."27052026/11/12 00:10:26 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."27062026/11/12 00:10:26 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."27072026/11/12 00:10:26 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."27082026/11/12 00:10:26 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."27092026/11/12 00:10:26 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."27102026/11/12 00:10:26 INFO Received uploads request method=POST path=/27112026/11/12 00:10:26 INFO Received push request method=POST path=/27122026/11/12 00:10:26 INFO Received uploads request method=POST path=/27132026/11/12 00:10:26 INFO Received push request method=POST path=/27142026/11/12 00:10:26 INFO Received push request method=POST path=/27152026/11/12 00:10:26 INFO Received complete upload request method=POST path=/27162026/11/12 00:10:26 INFO Aborted multipart uploads count=0 kept=027172026/11/12 00:10:26 INFO Completed upload id=10 objects=6 duration=1.396584ms27182026/11/12 00:10:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=027192026/11/12 00:10:26 INFO Vacuumed table table=pending_closures27202026/11/12 00:10:26 INFO Vacuumed table table=pending_objects27212026/11/12 00:10:26 INFO Vacuumed table table=multipart_uploads27222026/11/12 00:10:26 INFO Received push request method=POST path=/27232026/11/12 00:10:26 INFO Aborted multipart uploads count=0 kept=027242026/11/12 00:10:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027252026/11/12 00:10:26 INFO Received complete push request method=POST path=/27262026/11/12 00:10:26 INFO Vacuumed table table=pending_closures27272026/11/12 00:10:26 INFO Vacuumed table table=closures27282026/11/12 00:10:26 INFO Completed push id=8 objects=6 duration=668.666µs27292026/11/12 00:10:26 INFO Vacuumed table table=pending_objects27302026/11/12 00:10:26 INFO Received complete push request method=POST path=/27312026/11/12 00:10:26 INFO Vacuumed table table=objects27322026/11/12 00:10:26 INFO Vacuumed table table=multipart_uploads27332026/11/12 00:10:26 INFO Completed push id=3 objects=21 duration=5.78775ms27342026/11/12 00:10:26 INFO Vacuumed table table=closures27352026/11/12 00:10:26 INFO Vacuumed table table=objects27362026/11/12 00:10:26 INFO Received push request method=POST path=/27372026/11/12 00:10:26 INFO Received uploads request method=POST path=/27382026/11/12 00:10:26 INFO Received uploads request method=POST path=/27392026/11/12 00:10:26 INFO Received push request method=POST path=/27402026/11/12 00:10:26 INFO Received complete upload request method=POST path=/27412026/11/12 00:10:26 INFO Received uploads request method=POST path=/27422026/11/12 00:10:26 INFO Completed upload id=10 objects=2 duration=10.44425ms27432026/11/12 00:10:26 INFO Received push request method=POST path=/27442026/11/12 00:10:26 INFO Received uploads request method=POST path=/27452026/11/12 00:10:26 INFO Aborted multipart uploads count=0 kept=027462026/11/12 00:10:26 INFO Received push request method=POST path=/27472026/11/12 00:10:26 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=027482026/11/12 00:10:26 INFO Aborted multipart uploads count=0 kept=027492026/11/12 00:10:26 INFO Vacuumed table table=pending_closures27502026/11/12 00:10:26 INFO Received complete push request method=POST path=/27512026/11/12 00:10:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027522026/11/12 00:10:26 INFO Vacuumed table table=pending_objects27532026/11/12 00:10:26 INFO Completed push id=15 objects=5 duration=599.334µs27542026/11/12 00:10:26 INFO Received complete push request method=POST path=/27552026/11/12 00:10:26 INFO Vacuumed table table=multipart_uploads27562026/11/12 00:10:26 INFO Completed push id=12 objects=7 duration=521.375µs27572026/11/12 00:10:26 INFO Received uploads request method=POST path=/27582026/11/12 00:10:26 INFO Aborted multipart uploads count=0 kept=027592026/11/12 00:10:26 INFO Vacuumed table table=closures27602026/11/12 00:10:26 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=027612026/11/12 00:10:26 INFO Received push request method=POST path=/27622026/11/12 00:10:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=027632026/11/12 00:10:26 INFO Vacuumed table table=objects27642026/11/12 00:10:26 INFO Vacuumed table table=pending_closures27652026/11/12 00:10:26 INFO Received uploads request method=POST path=/27662026/11/12 00:10:26 INFO Received push request method=POST path=/27672026/11/12 00:10:26 INFO Vacuumed table table=pending_closures27682026/11/12 00:10:26 INFO Vacuumed table table=pending_closures27692026/11/12 00:10:26 INFO Vacuumed table table=pending_objects27702026/11/12 00:10:26 INFO Aborted multipart uploads count=0 kept=027712026/11/12 00:10:26 INFO Vacuumed table table=multipart_uploads27722026/11/12 00:10:26 INFO Vacuumed table table=closures27732026/11/12 00:10:26 INFO Vacuumed table table=pending_objects27742026/11/12 00:10:26 WARN Force mode enabled - objects will be deleted immediately without grace period27752026/11/12 00:10:26 INFO Vacuumed table table=objects27762026/11/12 00:10:26 INFO Vacuumed table table=multipart_uploads27772026/11/12 00:10:26 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027782026/11/12 00:10:26 INFO Vacuumed table table=closures27792026/11/12 00:10:26 INFO Vacuumed table table=pending_closures27802026/11/12 00:10:26 INFO Vacuumed table table=pending_objects27812026/11/12 00:10:26 INFO Vacuumed table table=pending_objects27822026/11/12 00:10:26 INFO Vacuumed table table=objects27832026/11/12 00:10:26 INFO Vacuumed table table=multipart_uploads27842026/11/12 00:10:26 INFO Vacuumed table table=multipart_uploads27852026/11/12 00:10:26 INFO Vacuumed table table=closures27862026/11/12 00:10:26 INFO Vacuumed table table=objects27872026/11/12 00:10:26 INFO Vacuumed table table=closures27882026/11/12 00:10:26 INFO Vacuumed table table=objects27892026/11/12 00:10:26 INFO Aborted multipart uploads count=0 kept=027902026/11/12 00:10:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027912026/11/12 00:10:26 INFO Vacuumed table table=pending_closures27922026/11/12 00:10:26 INFO Received uploads request method=POST path=/27932026/11/12 00:10:26 INFO Vacuumed table table=pending_objects27942026/11/12 00:10:26 INFO Vacuumed table table=multipart_uploads27952026/11/12 00:10:26 INFO Received uploads request method=POST path=/27962026/11/12 00:10:26 INFO Vacuumed table table=closures27972026/11/12 00:10:26 INFO Received uploads request method=POST path=/27982026/11/12 00:10:26 INFO Received complete push request method=POST path=/27992026/11/12 00:10:26 INFO Vacuumed table table=objects28002026/11/12 00:10:26 INFO Received uploads request method=POST path=/28012026/11/12 00:10:26 INFO Received push request method=POST path=/28022026/11/12 00:10:26 INFO Received uploads request method=POST path=/28032026/11/12 00:10:26 INFO Completed push id=1 objects=24 duration=29.589875ms28042026/11/12 00:10:26 INFO Received complete push request method=POST path=/28052026/11/12 00:10:26 INFO Completed push id=8 objects=6 duration=1.143666ms28062026/11/12 00:10:26 INFO Received push request method=POST path=/28072026/11/12 00:10:26 INFO Received complete push request method=POST path=/28082026/11/12 00:10:26 INFO Received push request method=POST path=/28092026/11/12 00:10:26 INFO Received uploads request method=POST path=/28102026/11/12 00:10:26 INFO Completed push id=6 objects=26 duration=1.1625ms28112026/11/12 00:10:26 INFO Aborted multipart uploads count=0 kept=028122026/11/12 00:10:26 INFO Received complete push request method=POST path=/28132026/11/12 00:10:26 INFO Received push request method=POST path=/28142026/11/12 00:10:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028152026/11/12 00:10:26 INFO Aborted multipart uploads count=0 kept=028162026/11/12 00:10:26 INFO Completed push id=14 objects=3 duration=956.958µs28172026/11/12 00:10:26 INFO Aborted multipart uploads count=0 kept=028182026/11/12 00:10:26 WARN Force mode enabled - objects will be deleted immediately without grace period28192026/11/12 00:10:26 WARN Force mode enabled - objects will be deleted immediately without grace period28202026/11/12 00:10:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028212026/11/12 00:10:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028222026/11/12 00:10:26 INFO Vacuumed table table=pending_closures28232026/11/12 00:10:26 INFO Vacuumed table table=pending_objects28242026/11/12 00:10:26 INFO Vacuumed table table=multipart_uploads28252026/11/12 00:10:26 INFO Vacuumed table table=pending_closures28262026/11/12 00:10:26 INFO Vacuumed table table=pending_closures28272026/11/12 00:10:26 INFO Vacuumed table table=closures28282026/11/12 00:10:26 INFO Vacuumed table table=objects28292026/11/12 00:10:26 INFO Vacuumed table table=pending_objects28302026/11/12 00:10:26 INFO Received uploads request method=POST path=/28312026/11/12 00:10:26 INFO Vacuumed table table=multipart_uploads28322026/11/12 00:10:26 INFO Vacuumed table table=closures28332026/11/12 00:10:27 INFO Vacuumed table table=objects28342026/11/12 00:10:27 INFO Received complete push request method=POST path=/28352026/11/12 00:10:27 INFO Vacuumed table table=pending_objects28362026/11/12 00:10:27 INFO Completed push id=21 objects=9 duration=6.252541ms28372026/11/12 00:10:27 INFO Vacuumed table table=multipart_uploads28382026/11/12 00:10:27 INFO Aborted multipart uploads count=0 kept=028392026/11/12 00:10:27 INFO Received complete push request method=POST path=/28402026/11/12 00:10:27 INFO Received push request method=POST path=/28412026/11/12 00:10:27 WARN Force mode enabled - objects will be deleted immediately without grace period28422026/11/12 00:10:27 INFO Completed push id=11 objects=8 duration=965.25µs28432026/11/12 00:10:27 INFO Received uploads request method=POST path=/28442026/11/12 00:10:27 INFO Aborted multipart uploads count=0 kept=028452026/11/12 00:10:27 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028462026/11/12 00:10:27 INFO Received push request method=POST path=/28472026/11/12 00:10:27 INFO Vacuumed table table=pending_closures28482026/11/12 00:10:27 INFO Vacuumed table table=pending_objects28492026/11/12 00:10:27 INFO Vacuumed table table=closures28502026/11/12 00:10:27 INFO Vacuumed table table=multipart_uploads28512026/11/12 00:10:27 INFO Received push request method=POST path=/28522026/11/12 00:10:27 INFO Vacuumed table table=closures28532026/11/12 00:10:27 INFO Vacuumed table table=objects28542026/11/12 00:10:27 INFO Received complete upload request method=POST path=/28552026/11/12 00:10:27 INFO Vacuumed table table=objects28562026/11/12 00:10:27 INFO Completed upload id=13 objects=8 duration=10.529625ms28572026/11/12 00:10:27 INFO Received uploads request method=POST path=/28582026/11/12 00:10:27 INFO Received uploads request method=POST path=/28592026/11/12 00:10:27 INFO Received push request method=POST path=/28602026/11/12 00:10:27 INFO Received push request method=POST path=/28612026/11/12 00:10:27 INFO Received uploads request method=POST path=/28622026/11/12 00:10:27 INFO Received push request method=POST path=/28632026/11/12 00:10:27 INFO Received uploads request method=POST path=/28642026/11/12 00:10:27 INFO Received uploads request method=POST path=/28652026/11/12 00:10:27 INFO Received complete push request method=POST path=/28662026/11/12 00:10:27 INFO Completed push id=19 objects=8 duration=1.255083ms28672026/11/12 00:10:27 INFO Received uploads request method=POST path=/28682026/11/12 00:10:27 INFO Aborted multipart uploads count=0 kept=028692026/11/12 00:10:27 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028702026/11/12 00:10:27 INFO Vacuumed table table=pending_closures28712026/11/12 00:10:27 INFO Vacuumed table table=pending_objects28722026/11/12 00:10:27 INFO Vacuumed table table=multipart_uploads28732026/11/12 00:10:27 INFO Vacuumed table table=closures28742026/11/12 00:10:27 INFO Vacuumed table table=objects28752026/11/12 00:10:27 INFO Aborted multipart uploads count=0 kept=028762026/11/12 00:10:27 WARN Force mode enabled - objects will be deleted immediately without grace period28772026/11/12 00:10:27 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028782026/11/12 00:10:27 INFO Vacuumed table table=pending_closures28792026/11/12 00:10:27 INFO Vacuumed table table=pending_objects28802026/11/12 00:10:27 INFO Vacuumed table table=multipart_uploads28812026/11/12 00:10:27 INFO Vacuumed table table=closures28822026/11/12 00:10:27 INFO Vacuumed table table=objects28832026/11/12 00:10:27 INFO Received complete upload request method=POST path=/28842026/11/12 00:10:27 INFO Completed upload id=5 objects=7 duration=1.211666ms28852026/11/12 00:10:27 INFO Received push request method=POST path=/28862026/11/12 00:10:27 INFO Received uploads request method=POST path=/28872026/11/12 00:10:27 INFO Received complete push request method=POST path=/28882026/11/12 00:10:27 INFO Completed push id=13 objects=6 duration=1.979667ms28892026/11/12 00:10:27 INFO Received uploads request method=POST path=/28902026/11/12 00:10:27 INFO Received push request method=POST path=/28912026/11/12 00:10:27 INFO Received uploads request method=POST path=/28922026/11/12 00:10:27 INFO Received push request method=POST path=/28932026/11/12 00:10:27 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."28942026/11/12 00:10:27 INFO Received push request method=POST path=/28952026/11/12 00:10:27 INFO Received uploads request method=POST path=/28962026/11/12 00:10:27 INFO Received complete push request method=POST path=/28972026/11/12 00:10:27 INFO Completed push id=9 objects=24 duration=487.292µs28982026/11/12 00:10:27 INFO Received complete upload request method=POST path=/28992026/11/12 00:10:27 INFO Received uploads request method=POST path=/29002026/11/12 00:10:27 INFO Completed upload id=6 objects=9 duration=685.208µs29012026/11/12 00:10:27 INFO Received uploads request method=POST path=/29022026/11/12 00:10:27 INFO Received uploads request method=POST path=/29032026/11/12 00:10:27 INFO Received uploads request method=POST path=/29042026/11/12 00:10:27 INFO Received complete upload request method=POST path=/29052026/11/12 00:10:27 INFO Completed upload id=10 objects=10 duration=478.417µs29062026/11/12 00:10:27 INFO Received push request method=POST path=/29072026/11/12 00:10:27 INFO Received complete upload request method=POST path=/29082026/11/12 00:10:27 INFO Received complete push request method=POST path=/29092026/11/12 00:10:27 INFO Completed upload id=3 objects=5 duration=687.458µs29102026/11/12 00:10:27 INFO Received push request method=POST path=/29112026/11/12 00:10:27 INFO Received uploads request method=POST path=/29122026/11/12 00:10:27 INFO Aborted multipart uploads count=0 kept=029132026/11/12 00:10:27 INFO Received push request method=POST path=/29142026/11/12 00:10:27 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=029152026/11/12 00:10:27 INFO Received push request method=POST path=/29162026/11/12 00:10:27 INFO Completed push id=1 objects=5 duration=45.155417ms29172026/11/12 00:10:27 INFO Vacuumed table table=pending_closures29182026/11/12 00:10:27 INFO Received push request method=POST path=/29192026/11/12 00:10:27 INFO Received uploads request method=POST path=/29202026/11/12 00:10:27 INFO Vacuumed table table=pending_objects29212026/11/12 00:10:27 INFO Vacuumed table table=multipart_uploads29222026/11/12 00:10:27 INFO Received complete push request method=POST path=/29232026/11/12 00:10:27 INFO Vacuumed table table=closures29242026/11/12 00:10:27 INFO Completed push id=14 objects=3 duration=436.75µs29252026/11/12 00:10:27 INFO Vacuumed table table=objects29262026/11/12 00:10:27 INFO Received push request method=POST path=/29272026/11/12 00:10:27 INFO Received complete push request method=POST path=/29282026/11/12 00:10:27 INFO Completed push id=5 objects=28 duration=680.291µs29292026/11/12 00:10:27 INFO Received push request method=POST path=/29302026/11/12 00:10:27 INFO Received complete push request method=POST path=/29312026/11/12 00:10:27 INFO Completed push id=10 objects=2 duration=553.5µs29322026/11/12 00:10:27 INFO Aborted multipart uploads count=0 kept=029332026/11/12 00:10:27 WARN Force mode enabled - objects will be deleted immediately without grace period29342026/11/12 00:10:27 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029352026/11/12 00:10:27 INFO Vacuumed table table=pending_closures29362026/11/12 00:10:27 INFO Vacuumed table table=pending_objects29372026/11/12 00:10:27 INFO Vacuumed table table=multipart_uploads29382026/11/12 00:10:27 INFO Vacuumed table table=closures29392026/11/12 00:10:27 INFO Vacuumed table table=objects29402026/11/12 00:10:27 INFO Received complete push request method=POST path=/29412026/11/12 00:10:27 INFO Completed push id=2 objects=10 duration=1.607ms29422026/11/12 00:10:27 INFO Received push request method=POST path=/29432026/11/12 00:10:27 INFO Received uploads request method=POST path=/29442026/11/12 00:10:27 INFO Received push request method=POST path=/29452026/11/12 00:10:27 INFO Received uploads request method=POST path=/29462026/11/12 00:10:27 INFO Received complete upload request method=POST path=/29472026/11/12 00:10:27 INFO Received complete push request method=POST path=/29482026/11/12 00:10:27 INFO Aborted multipart uploads count=0 kept=029492026/11/12 00:10:27 INFO Completed upload id=17 objects=5 duration=540.25µs29502026/11/12 00:10:27 INFO Completed push id=8 objects=7 duration=528.833µs29512026/11/12 00:10:27 WARN Force mode enabled - objects will be deleted immediately without grace period29522026/11/12 00:10:27 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029532026/11/12 00:10:27 INFO Vacuumed table table=pending_closures29542026/11/12 00:10:27 INFO Vacuumed table table=pending_objects29552026/11/12 00:10:27 INFO Received uploads request method=POST path=/29562026/11/12 00:10:27 INFO Vacuumed table table=multipart_uploads29572026/11/12 00:10:27 INFO Vacuumed table table=closures29582026/11/12 00:10:27 INFO Vacuumed table table=objects29592026/11/12 00:10:28 INFO Received complete push request method=POST path=/29602026/11/12 00:10:28 INFO Received uploads request method=POST path=/29612026/11/12 00:10:28 INFO Completed push id=17 objects=22 duration=716µs29622026/11/12 00:10:28 INFO Received uploads request method=POST path=/29632026/11/12 00:10:28 INFO Received uploads request method=POST path=/29642026/11/12 00:10:28 INFO Received uploads request method=POST path=/29652026/11/12 00:10:28 INFO Received complete push request method=POST path=/29662026/11/12 00:10:28 INFO Completed push id=18 objects=6 duration=875.958µs29672026/11/12 00:10:28 INFO Received complete push request method=POST path=/29682026/11/12 00:10:28 INFO Received complete upload request method=POST path=/29692026/11/12 00:10:28 INFO Completed upload id=23 objects=9 duration=525.083µs29702026/11/12 00:10:28 INFO Completed push id=19 objects=3 duration=6.709667ms29712026/11/12 00:10:28 INFO Received complete push request method=POST path=/29722026/11/12 00:10:28 INFO Completed push id=17 objects=6 duration=819.583µs29732026/11/12 00:10:28 INFO Received push request method=POST path=/29742026/11/12 00:10:28 INFO Received uploads request method=POST path=/29752026/11/12 00:10:28 INFO Received push request method=POST path=/29762026/11/12 00:10:28 INFO Received complete push request method=POST path=/29772026/11/12 00:10:28 INFO Received complete upload request method=POST path=/29782026/11/12 00:10:28 INFO Received complete push request method=POST path=/29792026/11/12 00:10:28 INFO Completed push id=16 objects=26 duration=429.5µs29802026/11/12 00:10:28 INFO Completed push id=2 objects=4 duration=498.209µs29812026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=029822026/11/12 00:10:28 INFO Received push request method=POST path=/29832026/11/12 00:10:28 WARN Force mode enabled - objects will be deleted immediately without grace period29842026/11/12 00:10:28 INFO Received uploads request method=POST path=/29852026/11/12 00:10:28 INFO Completed upload id=1 objects=15 duration=42.133167ms29862026/11/12 00:10:28 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029872026/11/12 00:10:28 INFO Received complete upload request method=POST path=/29882026/11/12 00:10:28 INFO Received uploads request method=POST path=/29892026/11/12 00:10:28 INFO Received uploads request method=POST path=/29902026/11/12 00:10:28 INFO Received complete push request method=POST path=/29912026/11/12 00:10:28 INFO Vacuumed table table=pending_closures29922026/11/12 00:10:28 INFO Completed upload id=9 objects=16 duration=839.25µs29932026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=029942026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=029952026/11/12 00:10:28 INFO Vacuumed table table=pending_objects29962026/11/12 00:10:28 INFO Received push request method=POST path=/29972026/11/12 00:10:28 INFO Received push request method=POST path=/29982026/11/12 00:10:28 INFO Vacuumed table table=multipart_uploads29992026/11/12 00:10:28 WARN Force mode enabled - objects will be deleted immediately without grace period30002026/11/12 00:10:28 INFO Received push request method=POST path=/30012026/11/12 00:10:28 INFO Vacuumed table table=closures30022026/11/12 00:10:28 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."30032026/11/12 00:10:28 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."30042026/11/12 00:10:28 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."30052026/11/12 00:10:28 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."30062026/11/12 00:10:28 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."30072026/11/12 00:10:28 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."30082026/11/12 00:10:28 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."30092026/11/12 00:10:28 INFO Completed push id=3 objects=22 duration=35.56175ms30102026/11/12 00:10:28 INFO Vacuumed table table=objects30112026/11/12 00:10:28 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030122026/11/12 00:10:28 INFO Received push request method=POST path=/30132026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=030142026/11/12 00:10:28 INFO Received complete push request method=POST path=/30152026/11/12 00:10:28 INFO Received complete push request method=POST path=/30162026/11/12 00:10:28 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."30172026/11/12 00:10:28 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."30182026/11/12 00:10:28 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."30192026/11/12 00:10:28 INFO Completed push id=10 objects=3 duration=807.916µs30202026/11/12 00:10:28 INFO Completed push id=27 objects=6 duration=1.021834ms30212026/11/12 00:10:28 INFO Received push request method=POST path=/30222026/11/12 00:10:28 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."30232026/11/12 00:10:28 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."30242026/11/12 00:10:28 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."30252026/11/12 00:10:28 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."30262026/11/12 00:10:28 INFO Vacuumed table table=pending_closures30272026/11/12 00:10:28 INFO Received uploads request method=POST path=/30282026/11/12 00:10:28 INFO Received complete upload request method=POST path=/30292026/11/12 00:10:28 INFO Received push request method=POST path=/30302026/11/12 00:10:28 INFO Received complete upload request method=POST path=/30312026/11/12 00:10:28 INFO Received complete upload request method=POST path=/30322026/11/12 00:10:28 INFO Received push request method=POST path=/30332026/11/12 00:10:28 INFO Completed upload id=14 objects=16 duration=713µs30342026/11/12 00:10:28 INFO Received push request method=POST path=/30352026/11/12 00:10:28 INFO Completed upload id=19 objects=25 duration=693.5µs30362026/11/12 00:10:28 INFO Completed upload id=15 objects=2 duration=276.709µs30372026/11/12 00:10:28 INFO Received complete push request method=POST path=/30382026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=030392026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=030402026/11/12 00:10:28 INFO Vacuumed table table=pending_objects30412026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=030422026/11/12 00:10:28 INFO Completed push id=20 objects=4 duration=357.125µs30432026/11/12 00:10:28 INFO Vacuumed table table=multipart_uploads30442026/11/12 00:10:28 WARN Force mode enabled - objects will be deleted immediately without grace period30452026/11/12 00:10:28 WARN Force mode enabled - objects will be deleted immediately without grace period30462026/11/12 00:10:28 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."30472026/11/12 00:10:28 INFO Received complete push request method=POST path=/30482026/11/12 00:10:28 INFO Received push request method=POST path=/30492026/11/12 00:10:28 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030502026/11/12 00:10:28 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=030512026/11/12 00:10:28 INFO Received uploads request method=POST path=/30522026/11/12 00:10:28 INFO Vacuumed table table=pending_closures30532026/11/12 00:10:28 INFO Vacuumed table table=pending_closures30542026/11/12 00:10:28 INFO Received uploads request method=POST path=/30552026/11/12 00:10:28 INFO Vacuumed table table=pending_objects30562026/11/12 00:10:28 INFO Vacuumed table table=pending_objects30572026/11/12 00:10:28 INFO Received uploads request method=POST path=/30582026/11/12 00:10:28 INFO Vacuumed table table=multipart_uploads30592026/11/12 00:10:28 INFO Vacuumed table table=multipart_uploads30602026/11/12 00:10:28 INFO Vacuumed table table=closures30612026/11/12 00:10:28 INFO Vacuumed table table=closures30622026/11/12 00:10:28 INFO Vacuumed table table=closures30632026/11/12 00:10:28 INFO Vacuumed table table=objects30642026/11/12 00:10:28 INFO Vacuumed table table=objects30652026/11/12 00:10:28 INFO Completed push id=3 objects=8 duration=35.603917ms30662026/11/12 00:10:28 INFO Received complete upload request method=POST path=/30672026/11/12 00:10:28 INFO Received push request method=POST path=/30682026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=030692026/11/12 00:10:28 INFO Completed upload id=26 objects=8 duration=373.458µs30702026/11/12 00:10:28 INFO Received uploads request method=POST path=/30712026/11/12 00:10:28 INFO Received uploads request method=POST path=/30722026/11/12 00:10:28 INFO Received push request method=POST path=/30732026/11/12 00:10:28 INFO Vacuumed table table=objects30742026/11/12 00:10:28 INFO Received complete push request method=POST path=/30752026/11/12 00:10:28 INFO Received complete push request method=POST path=/30762026/11/12 00:10:28 INFO Received uploads request method=POST path=/30772026/11/12 00:10:28 INFO Received uploads request method=POST path=/30782026/11/12 00:10:28 INFO Received complete push request method=POST path=/30792026/11/12 00:10:28 INFO Completed push id=28 objects=14 duration=433.875µs30802026/11/12 00:10:28 INFO Completed push id=4 objects=7 duration=539.75µs30812026/11/12 00:10:28 INFO Received push request method=POST path=/30822026/11/12 00:10:28 INFO Received push request method=POST path=/30832026/11/12 00:10:28 INFO Completed push id=4 objects=7 duration=560.334µs30842026/11/12 00:10:28 INFO Received uploads request method=POST path=/30852026/11/12 00:10:28 INFO Received push request method=POST path=/30862026/11/12 00:10:28 INFO Received uploads request method=POST path=/30872026/11/12 00:10:28 INFO Received complete upload request method=POST path=/30882026/11/12 00:10:28 INFO Received push request method=POST path=/30892026/11/12 00:10:28 INFO Received push request method=POST path=/30902026/11/12 00:10:28 INFO Received uploads request method=POST path=/30912026/11/12 00:10:28 INFO Received uploads request method=POST path=/30922026/11/12 00:10:28 INFO Received uploads request method=POST path=/30932026/11/12 00:10:28 INFO Completed upload id=2 objects=2 duration=458.958µs30942026/11/12 00:10:28 INFO Received uploads request method=POST path=/30952026/11/12 00:10:28 INFO Received uploads request method=POST path=/30962026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=030972026/11/12 00:10:28 INFO Received uploads request method=POST path=/30982026/11/12 00:10:28 WARN Force mode enabled - objects will be deleted immediately without grace period30992026/11/12 00:10:28 INFO Received complete push request method=POST path=/31002026/11/12 00:10:28 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031012026/11/12 00:10:28 INFO Completed push id=16 objects=4 duration=704.75µs31022026/11/12 00:10:28 INFO Received push request method=POST path=/31032026/11/12 00:10:28 INFO Vacuumed table table=pending_closures31042026/11/12 00:10:28 INFO Received uploads request method=POST path=/31052026/11/12 00:10:28 INFO Received complete upload request method=POST path=/31062026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=031072026/11/12 00:10:28 INFO Received complete upload request method=POST path=/31082026/11/12 00:10:28 INFO Completed upload id=22 objects=8 duration=290.083µs31092026/11/12 00:10:28 INFO Vacuumed table table=pending_objects31102026/11/12 00:10:28 INFO Vacuumed table table=multipart_uploads31112026/11/12 00:10:28 INFO Completed upload id=20 objects=5 duration=386.833µs31122026/11/12 00:10:28 INFO Received complete push request method=POST path=/31132026/11/12 00:10:28 INFO Vacuumed table table=closures31142026/11/12 00:10:28 INFO Completed push id=23 objects=5 duration=5.793791ms31152026/11/12 00:10:28 INFO Vacuumed table table=objects31162026/11/12 00:10:28 INFO Received uploads request method=POST path=/31172026/11/12 00:10:28 WARN Force mode enabled - objects will be deleted immediately without grace period31182026/11/12 00:10:28 INFO Received complete upload request method=POST path=/31192026/11/12 00:10:28 INFO Received complete upload request method=POST path=/31202026/11/12 00:10:28 INFO Received uploads request method=POST path=/31212026/11/12 00:10:28 INFO Received complete push request method=POST path=/31222026/11/12 00:10:28 INFO Received push request method=POST path=/31232026/11/12 00:10:28 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."31242026/11/12 00:10:28 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."31252026/11/12 00:10:28 INFO Completed upload id=24 objects=8 duration=303.625µs31262026/11/12 00:10:28 INFO Completed upload id=17 objects=6 duration=669.833µs31272026/11/12 00:10:28 INFO Received complete upload request method=POST path=/31282026/11/12 00:10:28 INFO Completed push id=6 objects=5 duration=446.084µs31292026/11/12 00:10:28 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."31302026/11/12 00:10:28 INFO Received complete upload request method=POST path=/31312026/11/12 00:10:28 INFO Received push request method=POST path=/31322026/11/12 00:10:28 INFO Received push request method=POST path=/31332026/11/12 00:10:28 INFO Completed upload id=21 objects=9 duration=565.875µs31342026/11/12 00:10:28 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."31352026/11/12 00:10:28 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."31362026/11/12 00:10:28 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."31372026/11/12 00:10:28 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."31382026/11/12 00:10:28 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."31392026/11/12 00:10:28 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."31402026/11/12 00:10:28 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."31412026/11/12 00:10:28 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."31422026/11/12 00:10:28 INFO Completed upload id=28 objects=5 duration=327.083µs31432026/11/12 00:10:28 INFO Received push request method=POST path=/31442026/11/12 00:10:28 INFO Received push request method=POST path=/31452026/11/12 00:10:28 INFO Received complete push request method=POST path=/31462026/11/12 00:10:28 INFO Received push request method=POST path=/31472026/11/12 00:10:28 INFO Received complete push request method=POST path=/31482026/11/12 00:10:28 INFO Received push request method=POST path=/31492026/11/12 00:10:28 INFO Received push request method=POST path=/31502026/11/12 00:10:28 INFO Received complete push request method=POST path=/31512026/11/12 00:10:28 INFO Completed push id=25 objects=10 duration=593.792µs31522026/11/12 00:10:28 INFO Completed push id=17 objects=11 duration=490.75µs31532026/11/12 00:10:28 INFO Received uploads request method=POST path=/31542026/11/12 00:10:28 INFO Received push request method=POST path=/31552026/11/12 00:10:28 INFO Completed push id=8 objects=2 duration=786.625µs31562026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=031572026/11/12 00:10:28 INFO Received push request method=POST path=/31582026/11/12 00:10:28 INFO Received complete upload request method=POST path=/31592026/11/12 00:10:28 INFO Received uploads request method=POST path=/31602026/11/12 00:10:28 INFO Completed upload id=29 objects=5 duration=487.667µs31612026/11/12 00:10:28 WARN Force mode enabled - objects will be deleted immediately without grace period31622026/11/12 00:10:28 INFO Received push request method=POST path=/31632026/11/12 00:10:28 INFO Received push request method=POST path=/31642026/11/12 00:10:28 INFO Received uploads request method=POST path=/31652026/11/12 00:10:28 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031662026/11/12 00:10:28 INFO Received complete upload request method=POST path=/31672026/11/12 00:10:28 INFO Received uploads request method=POST path=/31682026/11/12 00:10:28 INFO Received uploads request method=POST path=/31692026/11/12 00:10:28 INFO Vacuumed table table=pending_closures31702026/11/12 00:10:28 INFO Received push request method=POST path=/31712026/11/12 00:10:28 INFO Completed upload id=30 objects=2 duration=493.75µs31722026/11/12 00:10:28 INFO Vacuumed table table=pending_objects31732026/11/12 00:10:28 INFO Received complete upload request method=POST path=/31742026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=031752026/11/12 00:10:28 INFO Received complete push request method=POST path=/31762026/11/12 00:10:28 INFO Received complete upload request method=POST path=/31772026/11/12 00:10:28 INFO Vacuumed table table=multipart_uploads31782026/11/12 00:10:28 INFO Received push request method=POST path=/31792026/11/12 00:10:28 INFO Received complete push request method=POST path=/31802026/11/12 00:10:28 INFO Completed upload id=15 objects=9 duration=306.333µs31812026/11/12 00:10:28 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031822026/11/12 00:10:28 INFO Completed push id=31 objects=20 duration=298.333µs31832026/11/12 00:10:28 INFO Completed upload id=26 objects=6 duration=240.792µs31842026/11/12 00:10:28 INFO Vacuumed table table=closures31852026/11/12 00:10:28 INFO Received complete upload request method=POST path=/31862026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=031872026/11/12 00:10:28 INFO Received complete upload request method=POST path=/31882026/11/12 00:10:28 INFO Completed upload id=3 objects=25 duration=604.417µs31892026/11/12 00:10:28 INFO Received uploads request method=POST path=/31902026/11/12 00:10:28 INFO Vacuumed table table=objects31912026/11/12 00:10:28 INFO Received complete upload request method=POST path=/31922026/11/12 00:10:28 INFO Completed upload id=30 objects=5 duration=300.584µs31932026/11/12 00:10:28 INFO Completed push id=1 objects=6 duration=24.710125ms31942026/11/12 00:10:28 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031952026/11/12 00:10:28 INFO Received complete upload request method=POST path=/31962026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=031972026/11/12 00:10:28 INFO Received complete upload request method=POST path=/31982026/11/12 00:10:28 INFO Completed upload id=32 objects=5 duration=527.25µs31992026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=032002026/11/12 00:10:28 INFO Vacuumed table table=pending_closures32012026/11/12 00:10:28 INFO Completed upload id=29 objects=5 duration=456.208µs32022026/11/12 00:10:28 INFO Vacuumed table table=pending_closures32032026/11/12 00:10:28 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032042026/11/12 00:10:28 INFO Completed upload id=6 objects=7 duration=1.077583ms32052026/11/12 00:10:28 INFO Received complete upload request method=POST path=/32062026/11/12 00:10:28 WARN Force mode enabled - objects will be deleted immediately without grace period32072026/11/12 00:10:28 INFO Completed upload id=33 objects=3 duration=356.583µs32082026/11/12 00:10:28 INFO Vacuumed table table=pending_closures32092026/11/12 00:10:28 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=032102026/11/12 00:10:28 INFO Received push request method=POST path=/32112026/11/12 00:10:28 INFO Vacuumed table table=pending_objects32122026/11/12 00:10:28 INFO Vacuumed table table=pending_objects32132026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=032142026/11/12 00:10:28 INFO Aborted multipart uploads count=0 kept=032152026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads32162026/11/12 00:10:29 INFO Received complete push request method=POST path=/32172026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads32182026/11/12 00:10:29 WARN Force mode enabled - objects will be deleted immediately without grace period32192026/11/12 00:10:29 WARN Force mode enabled - objects will be deleted immediately without grace period32202026/11/12 00:10:29 INFO Completed push id=21 objects=17 duration=619µs32212026/11/12 00:10:29 INFO Vacuumed table table=closures32222026/11/12 00:10:29 INFO Vacuumed table table=pending_objects32232026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032242026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032252026/11/12 00:10:29 INFO Received complete push request method=POST path=/32262026/11/12 00:10:29 INFO Vacuumed table table=objects32272026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads32282026/11/12 00:10:29 INFO Vacuumed table table=pending_closures32292026/11/12 00:10:29 INFO Vacuumed table table=pending_closures32302026/11/12 00:10:29 INFO Vacuumed table table=closures32312026/11/12 00:10:29 INFO Completed push id=19 objects=7 duration=466.583µs32322026/11/12 00:10:29 INFO Vacuumed table table=closures32332026/11/12 00:10:29 INFO Vacuumed table table=pending_objects32342026/11/12 00:10:29 INFO Received push request method=POST path=/32352026/11/12 00:10:29 INFO Received push request method=POST path=/32362026/11/12 00:10:29 INFO Vacuumed table table=objects32372026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads32382026/11/12 00:10:29 INFO Vacuumed table table=pending_closures32392026/11/12 00:10:29 INFO Received complete push request method=POST path=/32402026/11/12 00:10:29 INFO Vacuumed table table=closures32412026/11/12 00:10:29 INFO Received push request method=POST path=/32422026/11/12 00:10:29 INFO Completed push id=31 objects=3 duration=542.333µs32432026/11/12 00:10:29 INFO Vacuumed table table=objects32442026/11/12 00:10:29 INFO Vacuumed table table=objects32452026/11/12 00:10:29 INFO Vacuumed table table=pending_objects32462026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=032472026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=032482026/11/12 00:10:29 INFO Received push request method=POST path=/32492026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads32502026/11/12 00:10:29 INFO Vacuumed table table=pending_objects32512026/11/12 00:10:29 INFO Vacuumed table table=pending_closures32522026/11/12 00:10:29 INFO Received push request method=POST path=/32532026/11/12 00:10:29 WARN Force mode enabled - objects will be deleted immediately without grace period32542026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=032552026/11/12 00:10:29 INFO Received uploads request method=POST path=/32562026/11/12 00:10:29 INFO Received uploads request method=POST path=/32572026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads32582026/11/12 00:10:29 WARN Force mode enabled - objects will be deleted immediately without grace period32592026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032602026/11/12 00:10:29 INFO Vacuumed table table=pending_objects32612026/11/12 00:10:29 INFO Received uploads request method=POST path=/32622026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=032632026/11/12 00:10:29 INFO Vacuumed table table=closures32642026/11/12 00:10:29 INFO Vacuumed table table=closures32652026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads32662026/11/12 00:10:29 INFO Received complete upload request method=POST path=/32672026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032682026/11/12 00:10:29 INFO Vacuumed table table=pending_closures32692026/11/12 00:10:29 WARN Force mode enabled - objects will be deleted immediately without grace period32702026/11/12 00:10:29 INFO Received complete upload request method=POST path=/32712026/11/12 00:10:29 INFO Vacuumed table table=objects32722026/11/12 00:10:29 INFO Completed upload id=7 objects=10 duration=453.958µs32732026/11/12 00:10:29 INFO Vacuumed table table=pending_objects32742026/11/12 00:10:29 INFO Vacuumed table table=pending_closures32752026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032762026/11/12 00:10:29 INFO Completed upload id=31 objects=3 duration=600.625µs32772026/11/12 00:10:29 INFO Vacuumed table table=objects32782026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads32792026/11/12 00:10:29 INFO Vacuumed table table=pending_closures32802026/11/12 00:10:29 INFO Vacuumed table table=closures32812026/11/12 00:10:29 INFO Received complete upload request method=POST path=/32822026/11/12 00:10:29 INFO Completed upload id=22 objects=2 duration=480.708µs32832026/11/12 00:10:29 INFO Vacuumed table table=pending_objects32842026/11/12 00:10:29 INFO Vacuumed table table=closures32852026/11/12 00:10:29 INFO Vacuumed table table=pending_objects32862026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=032872026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads32882026/11/12 00:10:29 INFO Received uploads request method=POST path=/32892026/11/12 00:10:29 INFO Vacuumed table table=objects32902026/11/12 00:10:29 INFO Vacuumed table table=closures32912026/11/12 00:10:29 INFO Vacuumed table table=objects32922026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads32932026/11/12 00:10:29 INFO Vacuumed table table=objects32942026/11/12 00:10:29 WARN Force mode enabled - objects will be deleted immediately without grace period32952026/11/12 00:10:29 INFO Received push request method=POST path=/32962026/11/12 00:10:29 INFO Vacuumed table table=closures32972026/11/12 00:10:29 INFO Received uploads request method=POST path=/32982026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032992026/11/12 00:10:29 INFO Received push request method=POST path=/33002026/11/12 00:10:29 INFO Vacuumed table table=objects33012026/11/12 00:10:29 INFO Vacuumed table table=pending_closures33022026/11/12 00:10:29 INFO Received push request method=POST path=/33032026/11/12 00:10:29 INFO Received push request method=POST path=/33042026/11/12 00:10:29 INFO Vacuumed table table=pending_objects33052026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=033062026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads33072026/11/12 00:10:29 INFO Received complete push request method=POST path=/33082026/11/12 00:10:29 INFO Vacuumed table table=closures33092026/11/12 00:10:29 INFO Completed push id=32 objects=6 duration=9.403125ms33102026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033112026/11/12 00:10:29 INFO Vacuumed table table=objects33122026/11/12 00:10:29 INFO Received complete push request method=POST path=/33132026/11/12 00:10:29 INFO Received complete upload request method=POST path=/33142026/11/12 00:10:29 INFO Completed upload id=33 objects=7 duration=239.583µs33152026/11/12 00:10:29 INFO Received push request method=POST path=/33162026/11/12 00:10:29 INFO Vacuumed table table=pending_closures33172026/11/12 00:10:29 INFO Completed push id=31 objects=14 duration=849.583µs33182026/11/12 00:10:29 INFO Received complete upload request method=POST path=/33192026/11/12 00:10:29 INFO Received complete upload request method=POST path=/33202026/11/12 00:10:29 INFO Completed upload id=34 objects=3 duration=268.042µs33212026/11/12 00:10:29 INFO Completed upload id=18 objects=2 duration=7.01975ms33222026/11/12 00:10:29 INFO Vacuumed table table=pending_objects33232026/11/12 00:10:29 INFO Received complete push request method=POST path=/33242026/11/12 00:10:29 INFO Received complete push request method=POST path=/33252026/11/12 00:10:29 INFO Received uploads request method=POST path=/33262026/11/12 00:10:29 INFO Received push request method=POST path=/33272026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads33282026/11/12 00:10:29 INFO Completed push id=35 objects=11 duration=454.541µs33292026/11/12 00:10:29 INFO Completed push id=7 objects=7 duration=515.291µs33302026/11/12 00:10:29 INFO Received uploads request method=POST path=/33312026/11/12 00:10:29 INFO Vacuumed table table=closures33322026/11/12 00:10:29 INFO Received complete push request method=POST path=/33332026/11/12 00:10:29 INFO Received uploads request method=POST path=/33342026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=033352026/11/12 00:10:29 INFO Received complete push request method=POST path=/33362026/11/12 00:10:29 INFO Vacuumed table table=objects33372026/11/12 00:10:29 WARN Force mode enabled - objects will be deleted immediately without grace period33382026/11/12 00:10:29 INFO Completed push id=6 objects=8 duration=987.542µs33392026/11/12 00:10:29 INFO Received push request method=POST path=/33402026/11/12 00:10:29 INFO Completed push id=2 objects=4 duration=593.25µs33412026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=6 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033422026/11/12 00:10:29 INFO Received push request method=POST path=/33432026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=033442026/11/12 00:10:29 INFO Received uploads request method=POST path=/33452026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=033462026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=033472026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033482026/11/12 00:10:29 INFO Vacuumed table table=pending_closures33492026/11/12 00:10:29 INFO Vacuumed table table=pending_closures33502026/11/12 00:10:29 INFO Received uploads request method=POST path=/33512026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033522026/11/12 00:10:29 INFO Received uploads request method=POST path=/33532026/11/12 00:10:29 INFO Vacuumed table table=pending_objects33542026/11/12 00:10:29 WARN Force mode enabled - objects will be deleted immediately without grace period33552026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads33562026/11/12 00:10:29 INFO Vacuumed table table=pending_objects33572026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=5 objects-failed-to-delete=033582026/11/12 00:10:29 INFO Received complete upload request method=POST path=/33592026/11/12 00:10:29 INFO Vacuumed table table=closures33602026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads33612026/11/12 00:10:29 INFO Completed upload id=7 objects=4 duration=311.166µs33622026/11/12 00:10:29 INFO Vacuumed table table=objects33632026/11/12 00:10:29 INFO Vacuumed table table=closures33642026/11/12 00:10:29 INFO Received push request method=POST path=/33652026/11/12 00:10:29 INFO Received push request method=POST path=/33662026/11/12 00:10:29 INFO Received complete upload request method=POST path=/33672026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=033682026/11/12 00:10:29 INFO Vacuumed table table=objects33692026/11/12 00:10:29 INFO Received uploads request method=POST path=/33702026/11/12 00:10:29 INFO Vacuumed table table=pending_closures33712026/11/12 00:10:29 INFO Completed upload id=12 objects=26 duration=1.008458ms33722026/11/12 00:10:29 WARN Force mode enabled - objects will be deleted immediately without grace period33732026/11/12 00:10:29 INFO Vacuumed table table=pending_closures33742026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=033752026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033762026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033772026/11/12 00:10:29 INFO Vacuumed table table=pending_closures33782026/11/12 00:10:29 INFO Received uploads request method=POST path=/33792026/11/12 00:10:29 INFO Vacuumed table table=pending_closures33802026/11/12 00:10:29 INFO Vacuumed table table=pending_objects33812026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads33822026/11/12 00:10:29 INFO Vacuumed table table=pending_objects33832026/11/12 00:10:29 INFO Vacuumed table table=pending_objects33842026/11/12 00:10:29 INFO Vacuumed table table=closures33852026/11/12 00:10:29 INFO Received complete push request method=POST path=/33862026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads33872026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads33882026/11/12 00:10:29 INFO Completed push id=16 objects=8 duration=6.23925ms33892026/11/12 00:10:29 INFO Vacuumed table table=objects33902026/11/12 00:10:29 INFO Vacuumed table table=closures33912026/11/12 00:10:29 INFO Vacuumed table table=closures33922026/11/12 00:10:29 INFO Vacuumed table table=pending_objects33932026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=033942026/11/12 00:10:29 INFO Vacuumed table table=objects33952026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads33962026/11/12 00:10:29 WARN Force mode enabled - objects will be deleted immediately without grace period33972026/11/12 00:10:29 INFO Received complete upload request method=POST path=/33982026/11/12 00:10:29 INFO Received uploads request method=POST path=/33992026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034002026/11/12 00:10:29 INFO Completed upload id=13 objects=7 duration=379.375µs34012026/11/12 00:10:29 INFO Vacuumed table table=pending_closures34022026/11/12 00:10:29 INFO Received complete upload request method=POST path=/34032026/11/12 00:10:29 INFO Vacuumed table table=pending_objects34042026/11/12 00:10:29 INFO Completed upload id=20 objects=4 duration=386.292µs34052026/11/12 00:10:29 INFO Vacuumed table table=objects34062026/11/12 00:10:29 INFO Received complete upload request method=POST path=/34072026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads34082026/11/12 00:10:29 INFO Vacuumed table table=closures34092026/11/12 00:10:29 INFO Received complete upload request method=POST path=/34102026/11/12 00:10:29 INFO Completed upload id=9 objects=7 duration=702.708µs34112026/11/12 00:10:29 INFO Completed upload id=23 objects=7 duration=424.542µs34122026/11/12 00:10:29 INFO Vacuumed table table=closures34132026/11/12 00:10:29 INFO Vacuumed table table=objects34142026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=034152026/11/12 00:10:29 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."34162026/11/12 00:10:29 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."34172026/11/12 00:10:29 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."34182026/11/12 00:10:29 INFO Vacuumed table table=objects34192026/11/12 00:10:29 INFO Received complete upload request method=POST path=/34202026/11/12 00:10:29 INFO Received uploads request method=POST path=/34212026/11/12 00:10:29 INFO Received complete push request method=POST path=/34222026/11/12 00:10:29 INFO Completed push id=10 objects=2 duration=891.333µs34232026/11/12 00:10:29 INFO Received push request method=POST path=/34242026/11/12 00:10:29 INFO Received complete upload request method=POST path=/34252026/11/12 00:10:29 INFO Completed upload id=3 objects=6 duration=34.310542ms34262026/11/12 00:10:29 INFO Completed upload id=18 objects=11 duration=428.958µs34272026/11/12 00:10:29 INFO Received complete push request method=POST path=/34282026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=034292026/11/12 00:10:29 INFO Received uploads request method=POST path=/34302026/11/12 00:10:29 INFO Completed push id=9 objects=7 duration=1.314083ms34312026/11/12 00:10:29 INFO Received push request method=POST path=/34322026/11/12 00:10:29 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo error="Access Denied."34332026/11/12 00:10:29 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."34342026/11/12 00:10:29 INFO Received push request method=POST path=/34352026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=034362026/11/12 00:10:29 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."34372026/11/12 00:10:29 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."34382026/11/12 00:10:29 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."34392026/11/12 00:10:29 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."34402026/11/12 00:10:29 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."34412026/11/12 00:10:29 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."34422026/11/12 00:10:29 ERROR failed to remove object object=nar/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.nar.zst error="Access Denied."34432026/11/12 00:10:29 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."34442026/11/12 00:10:29 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."34452026/11/12 00:10:29 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."34462026/11/12 00:10:29 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."34472026/11/12 00:10:29 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."34482026/11/12 00:10:29 WARN Force mode enabled - objects will be deleted immediately without grace period34492026/11/12 00:10:29 INFO Received uploads request method=POST path=/34502026/11/12 00:10:29 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."34512026/11/12 00:10:29 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."34522026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=034532026/11/12 00:10:29 INFO Received push request method=POST path=/34542026/11/12 00:10:29 INFO Received complete push request method=POST path=/34552026/11/12 00:10:29 INFO Received complete push request method=POST path=/34562026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=034572026/11/12 00:10:29 INFO Completed push id=6 objects=7 duration=1.11275ms34582026/11/12 00:10:29 INFO Aborted multipart uploads count=0 kept=034592026/11/12 00:10:29 INFO Vacuumed table table=pending_closures34602026/11/12 00:10:29 INFO Completed push id=11 objects=6 duration=1.005666ms34612026/11/12 00:10:29 INFO Vacuumed table table=pending_objects34622026/11/12 00:10:29 INFO Received uploads request method=POST path=/34632026/11/12 00:10:29 WARN Force mode enabled - objects will be deleted immediately without grace period34642026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads34652026/11/12 00:10:29 INFO Received push request method=POST path=/34662026/11/12 00:10:29 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034672026/11/12 00:10:29 INFO Received complete upload request method=POST path=/34682026/11/12 00:10:29 INFO Received uploads request method=POST path=/34692026/11/12 00:10:29 INFO Vacuumed table table=pending_closures34702026/11/12 00:10:29 INFO Completed upload id=12 objects=21 duration=659.667µs34712026/11/12 00:10:29 INFO Vacuumed table table=pending_objects34722026/11/12 00:10:29 INFO Vacuumed table table=multipart_uploads34732026/11/12 00:10:30 INFO Received complete upload request method=POST path=/34742026/11/12 00:10:30 INFO Vacuumed table table=closures34752026/11/12 00:10:30 INFO Received push request method=POST path=/34762026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=034772026/11/12 00:10:30 INFO Vacuumed table table=closures34782026/11/12 00:10:30 INFO Completed upload id=11 objects=7 duration=454.791µs34792026/11/12 00:10:30 INFO Vacuumed table table=objects34802026/11/12 00:10:30 WARN Force mode enabled - objects will be deleted immediately without grace period34812026/11/12 00:10:30 INFO Received uploads request method=POST path=/34822026/11/12 00:10:30 INFO Received push request method=POST path=/34832026/11/12 00:10:30 INFO Received uploads request method=POST path=/34842026/11/12 00:10:30 INFO Vacuumed table table=objects34852026/11/12 00:10:30 INFO Received complete push request method=POST path=/34862026/11/12 00:10:30 INFO Received complete push request method=POST path=/34872026/11/12 00:10:30 INFO Received uploads request method=POST path=/34882026/11/12 00:10:30 INFO Received complete upload request method=POST path=/34892026/11/12 00:10:30 INFO Completed push id=7 objects=7 duration=548.541µs34902026/11/12 00:10:30 INFO Completed push id=18 objects=6 duration=412.542µs34912026/11/12 00:10:30 INFO Completed upload id=10 objects=7 duration=6.25875ms34922026/11/12 00:10:30 INFO Received complete push request method=POST path=/34932026/11/12 00:10:30 INFO Completed push id=13 objects=23 duration=786.5µs34942026/11/12 00:10:30 INFO Received push request method=POST path=/34952026/11/12 00:10:30 INFO Received uploads request method=POST path=/34962026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=034972026/11/12 00:10:30 INFO Received complete push request method=POST path=/34982026/11/12 00:10:30 INFO Received uploads request method=POST path=/34992026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=035002026/11/12 00:10:30 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035012026/11/12 00:10:30 INFO Received uploads request method=POST path=/35022026/11/12 00:10:30 INFO Completed push id=12 objects=6 duration=294µs35032026/11/12 00:10:30 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=035042026/11/12 00:10:30 INFO Vacuumed table table=pending_closures35052026/11/12 00:10:30 INFO Received complete upload request method=POST path=/35062026/11/12 00:10:30 INFO Received complete upload request method=POST path=/35072026/11/12 00:10:30 INFO Received complete push request method=POST path=/35082026/11/12 00:10:30 INFO Completed upload id=18 objects=4 duration=473.916µs35092026/11/12 00:10:30 INFO Vacuumed table table=pending_objects35102026/11/12 00:10:30 INFO Vacuumed table table=multipart_uploads35112026/11/12 00:10:30 INFO Completed upload id=10 objects=23 duration=6.566625ms35122026/11/12 00:10:30 INFO Vacuumed table table=closures35132026/11/12 00:10:30 INFO Vacuumed table table=pending_closures35142026/11/12 00:10:30 INFO Vacuumed table table=objects35152026/11/12 00:10:30 INFO Completed push id=1 objects=5 duration=22.733375ms35162026/11/12 00:10:30 INFO Received uploads request method=POST path=/35172026/11/12 00:10:30 INFO Received complete upload request method=POST path=/35182026/11/12 00:10:30 INFO Received push request method=POST path=/35192026/11/12 00:10:30 INFO Completed upload id=20 objects=3 duration=991.541µs35202026/11/12 00:10:30 INFO Vacuumed table table=pending_objects35212026/11/12 00:10:30 INFO Received uploads request method=POST path=/35222026/11/12 00:10:30 INFO Received push request method=POST path=/35232026/11/12 00:10:30 INFO Vacuumed table table=multipart_uploads35242026/11/12 00:10:30 INFO Vacuumed table table=closures35252026/11/12 00:10:30 INFO Vacuumed table table=objects35262026/11/12 00:10:30 INFO Received complete upload request method=POST path=/35272026/11/12 00:10:30 INFO Received push request method=POST path=/35282026/11/12 00:10:30 INFO Completed upload id=24 objects=7 duration=4.807166ms35292026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=035302026/11/12 00:10:30 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035312026/11/12 00:10:30 INFO Vacuumed table table=pending_closures35322026/11/12 00:10:30 INFO Received uploads request method=POST path=/35332026/11/12 00:10:30 INFO Vacuumed table table=pending_objects35342026/11/12 00:10:30 INFO Vacuumed table table=multipart_uploads35352026/11/12 00:10:30 INFO Received uploads request method=POST path=/35362026/11/12 00:10:30 INFO Vacuumed table table=closures35372026/11/12 00:10:30 INFO Received complete upload request method=POST path=/35382026/11/12 00:10:30 INFO Completed upload id=15 objects=4 duration=619.875µs35392026/11/12 00:10:30 INFO Vacuumed table table=objects35402026/11/12 00:10:30 INFO Received complete upload request method=POST path=/35412026/11/12 00:10:30 INFO Received push request method=POST path=/35422026/11/12 00:10:30 INFO Completed upload id=11 objects=6 duration=547.167µs35432026/11/12 00:10:30 INFO Received uploads request method=POST path=/35442026/11/12 00:10:30 INFO Received push request method=POST path=/35452026/11/12 00:10:30 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."35462026/11/12 00:10:30 INFO Received uploads request method=POST path=/35472026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=035482026/11/12 00:10:30 INFO Received uploads request method=POST path=/35492026/11/12 00:10:30 INFO Received complete upload request method=POST path=/35502026/11/12 00:10:30 WARN Force mode enabled - objects will be deleted immediately without grace period35512026/11/12 00:10:30 INFO Completed upload id=16 objects=3 duration=377.042µs35522026/11/12 00:10:30 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035532026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=035542026/11/12 00:10:30 INFO Vacuumed table table=pending_closures35552026/11/12 00:10:30 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035562026/11/12 00:10:30 INFO Vacuumed table table=pending_objects35572026/11/12 00:10:30 INFO Vacuumed table table=multipart_uploads35582026/11/12 00:10:30 INFO Vacuumed table table=pending_closures35592026/11/12 00:10:30 INFO Vacuumed table table=closures35602026/11/12 00:10:30 INFO Received complete upload request method=POST path=/35612026/11/12 00:10:30 INFO Vacuumed table table=objects35622026/11/12 00:10:30 INFO Completed upload id=24 objects=18 duration=392.875µs35632026/11/12 00:10:30 INFO Received complete upload request method=POST path=/35642026/11/12 00:10:30 INFO Received complete push request method=POST path=/35652026/11/12 00:10:30 INFO Completed upload id=3 objects=24 duration=524.958µs35662026/11/12 00:10:30 INFO Vacuumed table table=pending_objects35672026/11/12 00:10:30 INFO Vacuumed table table=multipart_uploads35682026/11/12 00:10:30 INFO Completed push id=25 objects=2 duration=636.167µs35692026/11/12 00:10:30 INFO Received complete push request method=POST path=/35702026/11/12 00:10:30 INFO Vacuumed table table=closures35712026/11/12 00:10:30 INFO Vacuumed table table=objects35722026/11/12 00:10:30 INFO Completed push id=5 objects=9 duration=7.055542ms35732026/11/12 00:10:30 INFO Received complete push request method=POST path=/35742026/11/12 00:10:30 INFO Completed push id=27 objects=4 duration=379.916µs35752026/11/12 00:10:30 INFO Received push request method=POST path=/35762026/11/12 00:10:30 INFO Received uploads request method=POST path=/35772026/11/12 00:10:30 INFO Received push request method=POST path=/35782026/11/12 00:10:30 INFO Received uploads request method=POST path=/35792026/11/12 00:10:30 INFO Received complete upload request method=POST path=/35802026/11/12 00:10:30 INFO Received complete push request method=POST path=/35812026/11/12 00:10:30 INFO Completed upload id=18 objects=4 duration=1.2155ms35822026/11/12 00:10:30 INFO Received uploads request method=POST path=/35832026/11/12 00:10:30 INFO Received complete push request method=POST path=/35842026/11/12 00:10:30 INFO Completed push id=12 objects=15 duration=661.5µs35852026/11/12 00:10:30 INFO Received complete push request method=POST path=/35862026/11/12 00:10:30 INFO Received push request method=POST path=/35872026/11/12 00:10:30 INFO Received complete push request method=POST path=/35882026/11/12 00:10:30 INFO Received push request method=POST path=/35892026/11/12 00:10:30 INFO Completed push id=15 objects=4 duration=625.083µs35902026/11/12 00:10:30 INFO Completed push id=3 objects=5 duration=26.475542ms35912026/11/12 00:10:30 INFO Received complete push request method=POST path=/35922026/11/12 00:10:30 INFO Completed push id=17 objects=6 duration=695µs35932026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=035942026/11/12 00:10:30 INFO Received push request method=POST path=/35952026/11/12 00:10:30 INFO Completed push id=23 objects=28 duration=971.458µs35962026/11/12 00:10:30 WARN Force mode enabled - objects will be deleted immediately without grace period35972026/11/12 00:10:30 INFO Received uploads request method=POST path=/35982026/11/12 00:10:30 INFO Received uploads request method=POST path=/35992026/11/12 00:10:30 INFO Received push request method=POST path=/36002026/11/12 00:10:30 INFO Received complete upload request method=POST path=/36012026/11/12 00:10:30 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036022026/11/12 00:10:30 INFO Completed upload id=27 objects=8 duration=493.458µs36032026/11/12 00:10:30 INFO Vacuumed table table=pending_closures36042026/11/12 00:10:30 INFO Received uploads request method=POST path=/36052026/11/12 00:10:30 INFO Vacuumed table table=pending_objects36062026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=036072026/11/12 00:10:30 INFO Received push request method=POST path=/36082026/11/12 00:10:30 INFO Vacuumed table table=multipart_uploads36092026/11/12 00:10:30 INFO Received complete upload request method=POST path=/36102026/11/12 00:10:30 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036112026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=036122026/11/12 00:10:30 INFO Completed upload id=15 objects=6 duration=358.125µs36132026/11/12 00:10:30 INFO Received complete push request method=POST path=/36142026/11/12 00:10:30 INFO Vacuumed table table=pending_closures36152026/11/12 00:10:30 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036162026/11/12 00:10:30 INFO Completed push id=13 objects=22 duration=329µs36172026/11/12 00:10:30 INFO Received push request method=POST path=/36182026/11/12 00:10:30 INFO Vacuumed table table=pending_objects36192026/11/12 00:10:30 INFO Vacuumed table table=pending_closures36202026/11/12 00:10:30 INFO Vacuumed table table=multipart_uploads36212026/11/12 00:10:30 INFO Received complete upload request method=POST path=/36222026/11/12 00:10:30 INFO Received complete push request method=POST path=/36232026/11/12 00:10:30 INFO Vacuumed table table=pending_objects36242026/11/12 00:10:30 INFO Vacuumed table table=multipart_uploads36252026/11/12 00:10:30 INFO Completed push id=21 objects=6 duration=11.173583ms36262026/11/12 00:10:30 INFO Vacuumed table table=closures36272026/11/12 00:10:30 INFO Completed upload id=5 objects=5 duration=11.188834ms36282026/11/12 00:10:30 INFO Vacuumed table table=closures36292026/11/12 00:10:30 INFO Vacuumed table table=closures36302026/11/12 00:10:30 INFO Received complete push request method=POST path=/36312026/11/12 00:10:30 INFO Received complete upload request method=POST path=/36322026/11/12 00:10:30 INFO Received uploads request method=POST path=/36332026/11/12 00:10:30 INFO Vacuumed table table=objects36342026/11/12 00:10:30 INFO Vacuumed table table=objects36352026/11/12 00:10:30 INFO Completed push id=28 objects=26 duration=558.042µs36362026/11/12 00:10:30 INFO Vacuumed table table=objects36372026/11/12 00:10:30 INFO Completed upload id=22 objects=7 duration=581.208µs36382026/11/12 00:10:30 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."36392026/11/12 00:10:30 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."36402026/11/12 00:10:30 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."36412026/11/12 00:10:30 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."36422026/11/12 00:10:30 INFO Received complete push request method=POST path=/36432026/11/12 00:10:30 INFO Received complete push request method=POST path=/36442026/11/12 00:10:30 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."36452026/11/12 00:10:30 INFO Received uploads request method=POST path=/36462026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=036472026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=036482026/11/12 00:10:30 INFO Completed push id=30 objects=3 duration=289.041µs36492026/11/12 00:10:30 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."36502026/11/12 00:10:30 INFO Completed push id=24 objects=4 duration=316.792µs36512026/11/12 00:10:30 INFO Received push request method=POST path=/36522026/11/12 00:10:30 INFO Received push request method=POST path=/36532026/11/12 00:10:30 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036542026/11/12 00:10:30 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."36552026/11/12 00:10:30 WARN Force mode enabled - objects will be deleted immediately without grace period36562026/11/12 00:10:30 INFO Received complete push request method=POST path=/36572026/11/12 00:10:30 INFO Received complete push request method=POST path=/36582026/11/12 00:10:30 INFO Received uploads request method=POST path=/36592026/11/12 00:10:30 INFO Received push request method=POST path=/36602026/11/12 00:10:30 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."36612026/11/12 00:10:30 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."36622026/11/12 00:10:30 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."36632026/11/12 00:10:30 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."36642026/11/12 00:10:30 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."36652026/11/12 00:10:30 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."36662026/11/12 00:10:30 INFO Completed push id=25 objects=9 duration=446.667µs36672026/11/12 00:10:30 INFO Vacuumed table table=pending_closures36682026/11/12 00:10:30 INFO Completed push id=32 objects=19 duration=564.541µs36692026/11/12 00:10:30 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036702026/11/12 00:10:30 INFO Received uploads request method=POST path=/36712026/11/12 00:10:30 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."36722026/11/12 00:10:30 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."36732026/11/12 00:10:30 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."36742026/11/12 00:10:30 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."36752026/11/12 00:10:30 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."36762026/11/12 00:10:30 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."36772026/11/12 00:10:30 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."36782026/11/12 00:10:30 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."36792026/11/12 00:10:30 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."36802026/11/12 00:10:30 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."36812026/11/12 00:10:30 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."36822026/11/12 00:10:30 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."36832026/11/12 00:10:30 INFO Received push request method=POST path=/36842026/11/12 00:10:30 INFO Vacuumed table table=pending_objects36852026/11/12 00:10:30 INFO Vacuumed table table=multipart_uploads36862026/11/12 00:10:30 INFO Vacuumed table table=pending_closures36872026/11/12 00:10:30 INFO Vacuumed table table=closures36882026/11/12 00:10:30 INFO Received complete upload request method=POST path=/36892026/11/12 00:10:30 INFO Received uploads request method=POST path=/36902026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=036912026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=036922026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=036932026/11/12 00:10:30 INFO Completed upload id=26 objects=3 duration=220.542µs36942026/11/12 00:10:30 INFO Vacuumed table table=pending_objects36952026/11/12 00:10:30 INFO Vacuumed table table=objects36962026/11/12 00:10:30 INFO Received push request method=POST path=/36972026/11/12 00:10:30 WARN Force mode enabled - objects will be deleted immediately without grace period36982026/11/12 00:10:30 INFO Vacuumed table table=multipart_uploads36992026/11/12 00:10:30 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037002026/11/12 00:10:30 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037012026/11/12 00:10:30 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=037022026/11/12 00:10:30 INFO Received uploads request method=POST path=/37032026/11/12 00:10:30 INFO Vacuumed table table=closures37042026/11/12 00:10:30 INFO Received complete upload request method=POST path=/37052026/11/12 00:10:30 INFO Vacuumed table table=objects37062026/11/12 00:10:30 INFO Vacuumed table table=pending_closures37072026/11/12 00:10:30 INFO Vacuumed table table=pending_closures37082026/11/12 00:10:30 INFO Completed upload id=25 objects=16 duration=1.043583ms37092026/11/12 00:10:30 INFO Received push request method=POST path=/37102026/11/12 00:10:30 INFO Received push request method=POST path=/37112026/11/12 00:10:30 INFO Vacuumed table table=pending_objects37122026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=037132026/11/12 00:10:30 INFO Vacuumed table table=multipart_uploads37142026/11/12 00:10:30 INFO Aborted multipart uploads count=0 kept=037152026/11/12 00:10:30 INFO Vacuumed table table=pending_objects37162026/11/12 00:10:30 INFO Vacuumed table table=pending_closures37172026/11/12 00:10:31 WARN Force mode enabled - objects will be deleted immediately without grace period37182026/11/12 00:10:31 INFO Received complete push request method=POST path=/37192026/11/12 00:10:31 WARN Force mode enabled - objects will be deleted immediately without grace period37202026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads37212026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037222026/11/12 00:10:31 INFO Completed push id=2 objects=7 duration=1.626917ms37232026/11/12 00:10:31 INFO Vacuumed table table=closures37242026/11/12 00:10:31 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."37252026/11/12 00:10:31 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."37262026/11/12 00:10:31 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."37272026/11/12 00:10:31 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."37282026/11/12 00:10:31 INFO Vacuumed table table=pending_closures37292026/11/12 00:10:31 INFO Vacuumed table table=closures37302026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=037312026/11/12 00:10:31 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."37322026/11/12 00:10:31 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."37332026/11/12 00:10:31 INFO Vacuumed table table=objects37342026/11/12 00:10:31 INFO Vacuumed table table=objects37352026/11/12 00:10:31 INFO Vacuumed table table=pending_objects37362026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037372026/11/12 00:10:31 INFO Vacuumed table table=pending_objects37382026/11/12 00:10:31 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."37392026/11/12 00:10:31 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."37402026/11/12 00:10:31 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."37412026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads37422026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads37432026/11/12 00:10:31 INFO Vacuumed table table=closures37442026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=037452026/11/12 00:10:31 INFO Received push request method=POST path=/37462026/11/12 00:10:31 INFO Vacuumed table table=closures37472026/11/12 00:10:31 INFO Received complete upload request method=POST path=/37482026/11/12 00:10:31 WARN Force mode enabled - objects will be deleted immediately without grace period37492026/11/12 00:10:31 INFO Vacuumed table table=objects37502026/11/12 00:10:31 INFO Completed upload id=27 objects=7 duration=482.083µs37512026/11/12 00:10:31 INFO Vacuumed table table=objects37522026/11/12 00:10:31 INFO Received uploads request method=POST path=/37532026/11/12 00:10:31 INFO Vacuumed table table=pending_closures37542026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=9 objects-failed-to-delete=037552026/11/12 00:10:31 INFO Received complete upload request method=POST path=/37562026/11/12 00:10:31 INFO Received push request method=POST path=/37572026/11/12 00:10:31 INFO Received push request method=POST path=/37582026/11/12 00:10:31 INFO Completed upload id=28 objects=3 duration=206.709µs37592026/11/12 00:10:31 INFO Received push request method=POST path=/37602026/11/12 00:10:31 INFO Vacuumed table table=pending_objects37612026/11/12 00:10:31 INFO Vacuumed table table=pending_closures37622026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads37632026/11/12 00:10:31 INFO Received complete push request method=POST path=/37642026/11/12 00:10:31 INFO Received uploads request method=POST path=/37652026/11/12 00:10:31 INFO Completed push id=26 objects=14 duration=910.542µs37662026/11/12 00:10:31 INFO Received complete push request method=POST path=/37672026/11/12 00:10:31 INFO Received complete upload request method=POST path=/37682026/11/12 00:10:31 INFO Vacuumed table table=closures37692026/11/12 00:10:31 INFO Completed upload id=22 objects=5 duration=16.217958ms37702026/11/12 00:10:31 INFO Vacuumed table table=pending_objects37712026/11/12 00:10:31 INFO Completed push id=16 objects=27 duration=16.274583ms37722026/11/12 00:10:31 INFO Received complete push request method=POST path=/37732026/11/12 00:10:31 INFO Received push request method=POST path=/37742026/11/12 00:10:31 INFO Completed push id=14 objects=3 duration=448.583µs37752026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads37762026/11/12 00:10:31 INFO Vacuumed table table=objects37772026/11/12 00:10:31 INFO Vacuumed table table=closures37782026/11/12 00:10:31 INFO Received complete push request method=POST path=/37792026/11/12 00:10:31 INFO Received uploads request method=POST path=/37802026/11/12 00:10:31 INFO Completed push id=28 objects=3 duration=754.375µs37812026/11/12 00:10:31 INFO Received complete upload request method=POST path=/37822026/11/12 00:10:31 INFO Vacuumed table table=objects37832026/11/12 00:10:31 INFO Received push request method=POST path=/37842026/11/12 00:10:31 INFO Received uploads request method=POST path=/37852026/11/12 00:10:31 INFO Received complete push request method=POST path=/37862026/11/12 00:10:31 INFO Completed push id=29 objects=15 duration=260.875µs37872026/11/12 00:10:31 INFO Received complete push request method=POST path=/37882026/11/12 00:10:31 INFO Completed upload id=20 objects=4 duration=6.577542ms37892026/11/12 00:10:31 INFO Completed push id=29 objects=23 duration=460.292µs37902026/11/12 00:10:31 INFO Received uploads request method=POST path=/37912026/11/12 00:10:31 INFO Received complete push request method=POST path=/37922026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=037932026/11/12 00:10:31 INFO Completed push id=30 objects=6 duration=316.833µs37942026/11/12 00:10:31 INFO Received push request method=POST path=/37952026/11/12 00:10:31 INFO Received complete push request method=POST path=/37962026/11/12 00:10:31 INFO Received complete push request method=POST path=/37972026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037982026/11/12 00:10:31 INFO Received complete upload request method=POST path=/37992026/11/12 00:10:31 INFO Received uploads request method=POST path=/38002026/11/12 00:10:31 INFO Completed push id=30 objects=6 duration=317.125µs38012026/11/12 00:10:31 INFO Completed push id=11 objects=11 duration=1.005958ms38022026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=038032026/11/12 00:10:31 INFO Vacuumed table table=pending_closures38042026/11/12 00:10:31 INFO Completed upload id=31 objects=3 duration=187.833µs38052026/11/12 00:10:31 INFO Vacuumed table table=pending_objects38062026/11/12 00:10:31 WARN Force mode enabled - objects will be deleted immediately without grace period38072026/11/12 00:10:31 INFO Received push request method=POST path=/38082026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=038092026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads38102026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038112026/11/12 00:10:31 INFO Received uploads request method=POST path=/38122026/11/12 00:10:31 WARN Force mode enabled - objects will be deleted immediately without grace period38132026/11/12 00:10:31 INFO Vacuumed table table=closures38142026/11/12 00:10:31 INFO Vacuumed table table=objects38152026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038162026/11/12 00:10:31 INFO Vacuumed table table=pending_closures38172026/11/12 00:10:31 INFO Received push request method=POST path=/38182026/11/12 00:10:31 INFO Received push request method=POST path=/38192026/11/12 00:10:31 INFO Received push request method=POST path=/38202026/11/12 00:10:31 INFO Received push request method=POST path=/38212026/11/12 00:10:31 INFO Vacuumed table table=pending_objects38222026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads38232026/11/12 00:10:31 INFO Vacuumed table table=pending_closures38242026/11/12 00:10:31 INFO Vacuumed table table=closures38252026/11/12 00:10:31 INFO Vacuumed table table=objects38262026/11/12 00:10:31 INFO Received push request method=POST path=/38272026/11/12 00:10:31 INFO Vacuumed table table=pending_objects38282026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads38292026/11/12 00:10:31 INFO Vacuumed table table=closures38302026/11/12 00:10:31 INFO Vacuumed table table=objects38312026/11/12 00:10:31 INFO Received complete push request method=POST path=/38322026/11/12 00:10:31 INFO Completed push id=33 objects=3 duration=392.208µs38332026/11/12 00:10:31 INFO Received complete push request method=POST path=/38342026/11/12 00:10:31 INFO Completed push id=34 objects=6 duration=255.5µs38352026/11/12 00:10:31 INFO Received complete push request method=POST path=/38362026/11/12 00:10:31 INFO Received complete push request method=POST path=/38372026/11/12 00:10:31 INFO Completed push id=35 objects=8 duration=331.542µs38382026/11/12 00:10:31 INFO Completed push id=18 objects=10 duration=393.375µs38392026/11/12 00:10:31 INFO Received complete upload request method=POST path=/38402026/11/12 00:10:31 INFO Completed upload id=36 objects=9 duration=349.375µs38412026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=038422026/11/12 00:10:31 WARN Force mode enabled - objects will be deleted immediately without grace period38432026/11/12 00:10:31 INFO Received complete push request method=POST path=/38442026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038452026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=038462026/11/12 00:10:31 WARN Force mode enabled - objects will be deleted immediately without grace period38472026/11/12 00:10:31 INFO Completed push id=27 objects=25 duration=6.593959ms38482026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=5 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038492026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=038502026/11/12 00:10:31 INFO Vacuumed table table=pending_closures38512026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038522026/11/12 00:10:31 INFO Vacuumed table table=pending_closures38532026/11/12 00:10:31 INFO Vacuumed table table=pending_objects38542026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads38552026/11/12 00:10:31 INFO Vacuumed table table=pending_objects38562026/11/12 00:10:31 INFO Vacuumed table table=closures38572026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads38582026/11/12 00:10:31 INFO Vacuumed table table=objects38592026/11/12 00:10:31 INFO Vacuumed table table=closures38602026/11/12 00:10:31 INFO Vacuumed table table=objects38612026/11/12 00:10:31 INFO Received complete push request method=POST path=/38622026/11/12 00:10:31 INFO Completed push id=24 objects=5 duration=1.012875ms38632026/11/12 00:10:31 INFO Received complete push request method=POST path=/38642026/11/12 00:10:31 INFO Vacuumed table table=pending_closures38652026/11/12 00:10:31 INFO Completed push id=1 objects=11 duration=667.25µs38662026/11/12 00:10:31 INFO Received uploads request method=POST path=/38672026/11/12 00:10:31 INFO Received complete push request method=POST path=/38682026/11/12 00:10:31 INFO Completed push id=16 objects=5 duration=447.166µs38692026/11/12 00:10:31 INFO Received complete upload request method=POST path=/38702026/11/12 00:10:31 INFO Received uploads request method=POST path=/38712026/11/12 00:10:31 INFO Completed upload id=4 objects=7 duration=708.792µs38722026/11/12 00:10:31 INFO Vacuumed table table=pending_objects38732026/11/12 00:10:31 INFO Received push request method=POST path=/38742026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads38752026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=038762026/11/12 00:10:31 INFO Received uploads request method=POST path=/38772026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=038782026/11/12 00:10:31 INFO Received push request method=POST path=/38792026/11/12 00:10:31 INFO Vacuumed table table=pending_closures38802026/11/12 00:10:31 INFO Received uploads request method=POST path=/38812026/11/12 00:10:31 INFO Vacuumed table table=pending_objects38822026/11/12 00:10:31 INFO Vacuumed table table=closures38832026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads38842026/11/12 00:10:31 INFO Received push request method=POST path=/38852026/11/12 00:10:31 INFO Vacuumed table table=closures38862026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=038872026/11/12 00:10:31 INFO Received complete upload request method=POST path=/38882026/11/12 00:10:31 WARN Force mode enabled - objects will be deleted immediately without grace period38892026/11/12 00:10:31 INFO Vacuumed table table=objects38902026/11/12 00:10:31 INFO Completed upload id=16 objects=3 duration=337.917µs38912026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038922026/11/12 00:10:31 INFO Received complete push request method=POST path=/38932026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=038942026/11/12 00:10:31 INFO Vacuumed table table=objects38952026/11/12 00:10:31 INFO Completed push id=4 objects=6 duration=845.5µs38962026/11/12 00:10:31 INFO Received push request method=POST path=/38972026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038982026/11/12 00:10:31 INFO Vacuumed table table=pending_closures38992026/11/12 00:10:31 INFO Received push request method=POST path=/39002026/11/12 00:10:31 INFO Received complete push request method=POST path=/39012026/11/12 00:10:31 INFO Vacuumed table table=pending_objects39022026/11/12 00:10:31 INFO Received push request method=POST path=/39032026/11/12 00:10:31 INFO Completed push id=7 objects=20 duration=1.311167ms39042026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads39052026/11/12 00:10:31 INFO Vacuumed table table=pending_closures39062026/11/12 00:10:31 INFO Vacuumed table table=closures39072026/11/12 00:10:31 INFO Vacuumed table table=objects39082026/11/12 00:10:31 INFO Vacuumed table table=pending_objects39092026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=039102026/11/12 00:10:31 INFO Received uploads request method=POST path=/39112026/11/12 00:10:31 INFO Received push request method=POST path=/39122026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads39132026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039142026/11/12 00:10:31 INFO Received uploads request method=POST path=/39152026/11/12 00:10:31 INFO Received push request method=POST path=/39162026/11/12 00:10:31 INFO Vacuumed table table=pending_closures39172026/11/12 00:10:31 INFO Received push request method=POST path=/39182026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=039192026/11/12 00:10:31 INFO Received complete push request method=POST path=/39202026/11/12 00:10:31 INFO Vacuumed table table=pending_objects39212026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039222026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=039232026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads39242026/11/12 00:10:31 INFO Completed push id=21 objects=26 duration=492.375µs39252026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039262026/11/12 00:10:31 INFO Vacuumed table table=closures39272026/11/12 00:10:31 INFO Vacuumed table table=closures39282026/11/12 00:10:31 INFO Vacuumed table table=pending_closures39292026/11/12 00:10:31 INFO Vacuumed table table=pending_closures39302026/11/12 00:10:31 INFO Vacuumed table table=objects39312026/11/12 00:10:31 INFO Vacuumed table table=pending_objects39322026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads39332026/11/12 00:10:31 INFO Vacuumed table table=objects39342026/11/12 00:10:31 INFO Vacuumed table table=closures39352026/11/12 00:10:31 INFO Vacuumed table table=pending_objects39362026/11/12 00:10:31 INFO Vacuumed table table=objects39372026/11/12 00:10:31 INFO Received uploads request method=POST path=/39382026/11/12 00:10:31 INFO Vacuumed table table=multipart_uploads39392026/11/12 00:10:31 INFO Vacuumed table table=closures39402026/11/12 00:10:31 INFO Received uploads request method=POST path=/39412026/11/12 00:10:31 INFO Received complete push request method=POST path=/39422026/11/12 00:10:31 INFO Received uploads request method=POST path=/39432026/11/12 00:10:31 INFO Vacuumed table table=objects39442026/11/12 00:10:31 INFO Received complete upload request method=POST path=/39452026/11/12 00:10:31 INFO Completed push id=13 objects=15 duration=1.46175ms39462026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=039472026/11/12 00:10:31 INFO Completed upload id=9 objects=2 duration=674.375µs39482026/11/12 00:10:31 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039492026/11/12 00:10:31 INFO Received uploads request method=POST path=/39502026/11/12 00:10:31 INFO Aborted multipart uploads count=0 kept=039512026/11/12 00:10:31 INFO Vacuumed table table=pending_closures39522026/11/12 00:10:32 INFO Received complete push request method=POST path=/39532026/11/12 00:10:32 INFO Received complete push request method=POST path=/39542026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=039552026/11/12 00:10:32 INFO Received complete upload request method=POST path=/39562026/11/12 00:10:32 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."39572026/11/12 00:10:32 INFO Completed upload id=28 objects=17 duration=417.167µs39582026/11/12 00:10:32 INFO Vacuumed table table=pending_objects39592026/11/12 00:10:32 INFO Completed push id=8 objects=3 duration=5.767125ms39602026/11/12 00:10:32 INFO Completed push id=5 objects=3 duration=5.676708ms39612026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period39622026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads39632026/11/12 00:10:32 INFO Received complete push request method=POST path=/39642026/11/12 00:10:32 INFO Received uploads request method=POST path=/39652026/11/12 00:10:32 INFO Completed push id=29 objects=2 duration=707.791µs39662026/11/12 00:10:32 INFO Vacuumed table table=closures39672026/11/12 00:10:32 INFO Received push request method=POST path=/39682026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=4 objects-failed-to-delete=039692026/11/12 00:10:32 INFO Vacuumed table table=objects39702026/11/12 00:10:32 INFO Received complete push request method=POST path=/39712026/11/12 00:10:32 INFO Received uploads request method=POST path=/39722026/11/12 00:10:32 INFO Received push request method=POST path=/39732026/11/12 00:10:32 INFO Vacuumed table table=pending_closures39742026/11/12 00:10:32 INFO Completed push id=31 objects=6 duration=636.75µs39752026/11/12 00:10:32 INFO Received uploads request method=POST path=/39762026/11/12 00:10:32 INFO Received push request method=POST path=/39772026/11/12 00:10:32 INFO Received complete upload request method=POST path=/39782026/11/12 00:10:32 INFO Received push request method=POST path=/39792026/11/12 00:10:32 INFO Completed upload id=13 objects=10 duration=675.875µs39802026/11/12 00:10:32 INFO Received complete push request method=POST path=/39812026/11/12 00:10:32 INFO Received complete push request method=POST path=/39822026/11/12 00:10:32 INFO Received uploads request method=POST path=/39832026/11/12 00:10:32 INFO Received uploads request method=POST path=/39842026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=039852026/11/12 00:10:32 INFO Completed push id=32 objects=9 duration=270.125µs39862026/11/12 00:10:32 INFO Vacuumed table table=pending_objects39872026/11/12 00:10:32 INFO Completed push id=8 objects=6 duration=589.292µs39882026/11/12 00:10:32 INFO Received push request method=POST path=/39892026/11/12 00:10:32 INFO Received complete upload request method=POST path=/39902026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period39912026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads39922026/11/12 00:10:32 INFO Received push request method=POST path=/39932026/11/12 00:10:32 INFO Completed upload id=23 objects=18 duration=676.834µs39942026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=039952026/11/12 00:10:32 INFO Received complete push request method=POST path=/39962026/11/12 00:10:32 INFO Vacuumed table table=closures39972026/11/12 00:10:32 INFO Received uploads request method=POST path=/39982026/11/12 00:10:32 INFO Vacuumed table table=objects39992026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040002026/11/12 00:10:32 INFO Completed push id=13 objects=12 duration=950.875µs40012026/11/12 00:10:32 INFO Received uploads request method=POST path=/40022026/11/12 00:10:32 INFO Received push request method=POST path=/40032026/11/12 00:10:32 INFO Received complete upload request method=POST path=/40042026/11/12 00:10:32 INFO Vacuumed table table=pending_closures40052026/11/12 00:10:32 INFO Completed upload id=10 objects=6 duration=970.166µs40062026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=040072026/11/12 00:10:32 INFO Received push request method=POST path=/40082026/11/12 00:10:32 INFO Vacuumed table table=pending_objects40092026/11/12 00:10:32 INFO Received push request method=POST path=/40102026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=040112026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=040122026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads40132026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period40142026/11/12 00:10:32 INFO Vacuumed table table=pending_closures40152026/11/12 00:10:32 INFO Received complete upload request method=POST path=/40162026/11/12 00:10:32 INFO Vacuumed table table=closures40172026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040182026/11/12 00:10:32 INFO Vacuumed table table=pending_objects40192026/11/12 00:10:32 INFO Completed upload id=22 objects=22 duration=234.958µs40202026/11/12 00:10:32 INFO Vacuumed table table=objects40212026/11/12 00:10:32 INFO Received complete upload request method=POST path=/40222026/11/12 00:10:32 INFO Vacuumed table table=pending_closures40232026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads40242026/11/12 00:10:32 INFO Completed upload id=33 objects=9 duration=348.25µs40252026/11/12 00:10:32 INFO Vacuumed table table=pending_objects40262026/11/12 00:10:32 INFO Vacuumed table table=closures40272026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=040282026/11/12 00:10:32 INFO Received push request method=POST path=/40292026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads40302026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period40312026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=040322026/11/12 00:10:32 INFO Received complete push request method=POST path=/40332026/11/12 00:10:32 INFO Vacuumed table table=objects40342026/11/12 00:10:32 INFO Vacuumed table table=closures40352026/11/12 00:10:32 INFO Completed push id=26 objects=5 duration=6.126333ms40362026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period40372026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040382026/11/12 00:10:32 INFO Received complete upload request method=POST path=/40392026/11/12 00:10:32 INFO Received push request method=POST path=/40402026/11/12 00:10:32 INFO Vacuumed table table=objects40412026/11/12 00:10:32 INFO Completed upload id=24 objects=14 duration=286.208µs40422026/11/12 00:10:32 INFO Vacuumed table table=pending_closures40432026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040442026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=040452026/11/12 00:10:32 INFO Received complete push request method=POST path=/40462026/11/12 00:10:32 INFO Vacuumed table table=pending_objects40472026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads40482026/11/12 00:10:32 INFO Completed push id=19 objects=3 duration=9.85525ms40492026/11/12 00:10:32 INFO Vacuumed table table=closures40502026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040512026/11/12 00:10:32 INFO Vacuumed table table=pending_closures40522026/11/12 00:10:32 INFO Received uploads request method=POST path=/40532026/11/12 00:10:32 INFO Vacuumed table table=objects40542026/11/12 00:10:32 INFO Vacuumed table table=pending_objects40552026/11/12 00:10:32 INFO Vacuumed table table=pending_closures40562026/11/12 00:10:32 INFO Received uploads request method=POST path=/40572026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads40582026/11/12 00:10:32 INFO Received complete upload request method=POST path=/40592026/11/12 00:10:32 INFO Vacuumed table table=closures40602026/11/12 00:10:32 INFO Received complete upload request method=POST path=/40612026/11/12 00:10:32 INFO Completed upload id=19 objects=2 duration=768.125µs40622026/11/12 00:10:32 INFO Received complete upload request method=POST path=/40632026/11/12 00:10:32 INFO Vacuumed table table=objects40642026/11/12 00:10:32 INFO Completed upload id=25 objects=4 duration=474.083µs40652026/11/12 00:10:32 INFO Vacuumed table table=pending_objects40662026/11/12 00:10:32 INFO Received uploads request method=POST path=/40672026/11/12 00:10:32 INFO Completed upload id=10 objects=17 duration=393.042µs40682026/11/12 00:10:32 INFO Received uploads request method=POST path=/40692026/11/12 00:10:32 INFO Received push request method=POST path=/40702026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads40712026/11/12 00:10:32 INFO Received complete push request method=POST path=/40722026/11/12 00:10:32 INFO Received push request method=POST path=/40732026/11/12 00:10:32 INFO Received push request method=POST path=/40742026/11/12 00:10:32 INFO Completed push id=33 objects=7 duration=632.125µs40752026/11/12 00:10:32 INFO Vacuumed table table=closures40762026/11/12 00:10:32 INFO Received complete push request method=POST path=/40772026/11/12 00:10:32 INFO Received complete upload request method=POST path=/40782026/11/12 00:10:32 INFO Completed upload id=26 objects=3 duration=419.583µs40792026/11/12 00:10:32 INFO Completed push id=20 objects=8 duration=511.084µs40802026/11/12 00:10:32 INFO Vacuumed table table=objects40812026/11/12 00:10:32 INFO Received push request method=POST path=/40822026/11/12 00:10:32 INFO Received uploads request method=POST path=/40832026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=040842026/11/12 00:10:32 INFO Received uploads request method=POST path=/40852026/11/12 00:10:32 INFO Received complete push request method=POST path=/40862026/11/12 00:10:32 INFO Received complete push request method=POST path=/40872026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period40882026/11/12 00:10:32 INFO Completed push id=20 objects=3 duration=554.75µs40892026/11/12 00:10:32 INFO Received complete upload request method=POST path=/40902026/11/12 00:10:32 INFO Completed push id=14 objects=6 duration=831.333µs40912026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040922026/11/12 00:10:32 INFO Received push request method=POST path=/40932026/11/12 00:10:32 INFO Completed upload id=6 objects=24 duration=1.268166ms40942026/11/12 00:10:32 INFO Received push request method=POST path=/40952026/11/12 00:10:32 INFO Vacuumed table table=pending_closures40962026/11/12 00:10:32 INFO Vacuumed table table=pending_objects40972026/11/12 00:10:32 INFO Received push request method=POST path=/40982026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads40992026/11/12 00:10:32 INFO Received complete push request method=POST path=/41002026/11/12 00:10:32 INFO Received push request method=POST path=/41012026/11/12 00:10:32 INFO Received push request method=POST path=/41022026/11/12 00:10:32 INFO Received push request method=POST path=/41032026/11/12 00:10:32 INFO Received complete push request method=POST path=/41042026/11/12 00:10:32 INFO Vacuumed table table=closures41052026/11/12 00:10:32 INFO Completed push id=4 objects=6 duration=607.833µs41062026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=041072026/11/12 00:10:32 INFO Received push request method=POST path=/41082026/11/12 00:10:32 INFO Completed push id=27 objects=11 duration=475.459µs41092026/11/12 00:10:32 INFO Vacuumed table table=objects41102026/11/12 00:10:32 INFO Received uploads request method=POST path=/41112026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=041122026/11/12 00:10:32 INFO Received complete upload request method=POST path=/41132026/11/12 00:10:32 INFO Received complete upload request method=POST path=/41142026/11/12 00:10:32 INFO Received push request method=POST path=/41152026/11/12 00:10:32 INFO Vacuumed table table=pending_closures41162026/11/12 00:10:32 INFO Completed upload id=29 objects=26 duration=240.625µs41172026/11/12 00:10:32 INFO Completed upload id=22 objects=6 duration=416.208µs41182026/11/12 00:10:32 INFO Vacuumed table table=pending_objects41192026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=041202026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads41212026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period41222026/11/12 00:10:32 INFO Received uploads request method=POST path=/41232026/11/12 00:10:32 INFO Vacuumed table table=closures41242026/11/12 00:10:32 INFO Received complete push request method=POST path=/41252026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041262026/11/12 00:10:32 INFO Completed push id=30 objects=8 duration=375.125µs41272026/11/12 00:10:32 INFO Vacuumed table table=objects41282026/11/12 00:10:32 INFO Vacuumed table table=pending_closures41292026/11/12 00:10:32 INFO Received uploads request method=POST path=/41302026/11/12 00:10:32 INFO Received complete upload request method=POST path=/41312026/11/12 00:10:32 INFO Vacuumed table table=pending_objects41322026/11/12 00:10:32 INFO Received push request method=POST path=/41332026/11/12 00:10:32 INFO Received complete push request method=POST path=/41342026/11/12 00:10:32 INFO Received push request method=POST path=/41352026/11/12 00:10:32 INFO Completed upload id=31 objects=5 duration=261.792µs41362026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads41372026/11/12 00:10:32 INFO Completed push id=8 objects=6 duration=507.75µs41382026/11/12 00:10:32 INFO Received push request method=POST path=/41392026/11/12 00:10:32 INFO Received uploads request method=POST path=/41402026/11/12 00:10:32 INFO Received complete upload request method=POST path=/41412026/11/12 00:10:32 INFO Vacuumed table table=closures41422026/11/12 00:10:32 INFO Received complete push request method=POST path=/41432026/11/12 00:10:32 INFO Received uploads request method=POST path=/41442026/11/12 00:10:32 INFO Received uploads request method=POST path=/41452026/11/12 00:10:32 INFO Completed upload id=4 objects=14 duration=970µs41462026/11/12 00:10:32 INFO Completed push id=32 objects=3 duration=287.709µs41472026/11/12 00:10:32 INFO Received uploads request method=POST path=/41482026/11/12 00:10:32 INFO Vacuumed table table=objects41492026/11/12 00:10:32 INFO Received complete push request method=POST path=/41502026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=041512026/11/12 00:10:32 INFO Completed push id=12 objects=2 duration=1.352333ms41522026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=041532026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period41542026/11/12 00:10:32 INFO Received uploads request method=POST path=/41552026/11/12 00:10:32 INFO Received uploads request method=POST path=/41562026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period41572026/11/12 00:10:32 INFO Received uploads request method=POST path=/41582026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041592026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041602026/11/12 00:10:32 INFO Received uploads request method=POST path=/41612026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=041622026/11/12 00:10:32 INFO Received uploads request method=POST path=/41632026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period41642026/11/12 00:10:32 INFO Received uploads request method=POST path=/41652026/11/12 00:10:32 INFO Vacuumed table table=pending_closures41662026/11/12 00:10:32 INFO Vacuumed table table=pending_closures41672026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041682026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=041692026/11/12 00:10:32 INFO Received push request method=POST path=/41702026/11/12 00:10:32 INFO Vacuumed table table=pending_closures41712026/11/12 00:10:32 INFO Vacuumed table table=pending_objects41722026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041732026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads41742026/11/12 00:10:32 INFO Received complete upload request method=POST path=/41752026/11/12 00:10:32 INFO Received complete upload request method=POST path=/41762026/11/12 00:10:32 INFO Vacuumed table table=pending_closures41772026/11/12 00:10:32 INFO Vacuumed table table=pending_objects41782026/11/12 00:10:32 INFO Vacuumed table table=closures41792026/11/12 00:10:32 INFO Vacuumed table table=pending_objects41802026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads41812026/11/12 00:10:32 INFO Completed upload id=9 objects=3 duration=5.433709ms41822026/11/12 00:10:32 INFO Completed upload id=11 objects=6 duration=5.451416ms41832026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads41842026/11/12 00:10:32 INFO Vacuumed table table=objects41852026/11/12 00:10:32 INFO Vacuumed table table=closures41862026/11/12 00:10:32 INFO Vacuumed table table=pending_objects41872026/11/12 00:10:32 INFO Vacuumed table table=closures41882026/11/12 00:10:32 INFO Received uploads request method=POST path=/41892026/11/12 00:10:32 INFO Received push request method=POST path=/41902026/11/12 00:10:32 INFO Vacuumed table table=objects41912026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads41922026/11/12 00:10:32 INFO Received complete push request method=POST path=/41932026/11/12 00:10:32 INFO Vacuumed table table=objects41942026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=041952026/11/12 00:10:32 INFO Received uploads request method=POST path=/41962026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=041972026/11/12 00:10:32 INFO Vacuumed table table=closures41982026/11/12 00:10:32 INFO Completed push id=19 objects=6 duration=927.958µs41992026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period42002026/11/12 00:10:32 INFO Received push request method=POST path=/42012026/11/12 00:10:32 INFO Received push request method=POST path=/42022026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042032026/11/12 00:10:32 INFO Vacuumed table table=objects42042026/11/12 00:10:32 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."42052026/11/12 00:10:32 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."42062026/11/12 00:10:32 INFO Vacuumed table table=pending_closures42072026/11/12 00:10:32 INFO Received complete push request method=POST path=/42082026/11/12 00:10:32 INFO Vacuumed table table=pending_objects42092026/11/12 00:10:32 INFO Received push request method=POST path=/42102026/11/12 00:10:32 INFO Received uploads request method=POST path=/42112026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=042122026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=042132026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads42142026/11/12 00:10:32 INFO Completed push id=9 objects=5 duration=630.708µs42152026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period42162026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042172026/11/12 00:10:32 INFO Received uploads request method=POST path=/42182026/11/12 00:10:32 INFO Received uploads request method=POST path=/42192026/11/12 00:10:32 INFO Vacuumed table table=closures42202026/11/12 00:10:32 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."42212026/11/12 00:10:32 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."42222026/11/12 00:10:32 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."42232026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=042242026/11/12 00:10:32 INFO Received complete push request method=POST path=/42252026/11/12 00:10:32 INFO Vacuumed table table=objects42262026/11/12 00:10:32 INFO Vacuumed table table=pending_closures42272026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=042282026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=042292026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period42302026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period42312026/11/12 00:10:32 INFO Completed push id=12 objects=3 duration=731.583µs42322026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042332026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=11 objects-failed-to-delete=042342026/11/12 00:10:32 INFO Vacuumed table table=pending_closures42352026/11/12 00:10:32 INFO Vacuumed table table=pending_closures42362026/11/12 00:10:32 INFO Vacuumed table table=pending_objects42372026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042382026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads42392026/11/12 00:10:32 INFO Vacuumed table table=closures42402026/11/12 00:10:32 INFO Vacuumed table table=pending_closures42412026/11/12 00:10:32 INFO Vacuumed table table=pending_objects42422026/11/12 00:10:32 INFO Received uploads request method=POST path=/42432026/11/12 00:10:32 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."42442026/11/12 00:10:32 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."42452026/11/12 00:10:32 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."42462026/11/12 00:10:32 INFO Vacuumed table table=pending_objects42472026/11/12 00:10:32 INFO Vacuumed table table=objects42482026/11/12 00:10:32 INFO Vacuumed table table=pending_objects42492026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads42502026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads42512026/11/12 00:10:32 INFO Received uploads request method=POST path=/42522026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads42532026/11/12 00:10:32 INFO Vacuumed table table=closures42542026/11/12 00:10:32 INFO Vacuumed table table=closures42552026/11/12 00:10:32 INFO Vacuumed table table=closures42562026/11/12 00:10:32 INFO Vacuumed table table=objects42572026/11/12 00:10:32 INFO Vacuumed table table=objects42582026/11/12 00:10:32 INFO Received push request method=POST path=/42592026/11/12 00:10:32 INFO Vacuumed table table=objects42602026/11/12 00:10:32 INFO Received push request method=POST path=/42612026/11/12 00:10:32 INFO Received complete push request method=POST path=/42622026/11/12 00:10:32 INFO Completed push id=22 objects=3 duration=918.667µs42632026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=042642026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period42652026/11/12 00:10:32 INFO Received push request method=POST path=/42662026/11/12 00:10:32 INFO Received complete push request method=POST path=/42672026/11/12 00:10:32 INFO Completed push id=5 objects=8 duration=992.916µs42682026/11/12 00:10:32 INFO Received push request method=POST path=/42692026/11/12 00:10:32 INFO Received uploads request method=POST path=/42702026/11/12 00:10:32 INFO Received complete upload request method=POST path=/42712026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=042722026/11/12 00:10:32 INFO Received complete push request method=POST path=/42732026/11/12 00:10:32 INFO Completed upload id=16 objects=23 duration=282.041µs42742026/11/12 00:10:32 INFO Received complete upload request method=POST path=/42752026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042762026/11/12 00:10:32 INFO Completed push id=14 objects=4 duration=988.125µs42772026/11/12 00:10:32 INFO Received complete upload request method=POST path=/42782026/11/12 00:10:32 INFO Completed upload id=17 objects=25 duration=991.75µs42792026/11/12 00:10:32 INFO Vacuumed table table=pending_closures42802026/11/12 00:10:32 INFO Completed upload id=15 objects=6 duration=370.417µs42812026/11/12 00:10:32 INFO Vacuumed table table=pending_objects42822026/11/12 00:10:32 INFO Received push request method=POST path=/42832026/11/12 00:10:32 INFO Received push request method=POST path=/42842026/11/12 00:10:32 INFO Received uploads request method=POST path=/42852026/11/12 00:10:32 INFO Vacuumed table table=multipart_uploads42862026/11/12 00:10:32 INFO Received complete upload request method=POST path=/42872026/11/12 00:10:32 INFO Completed upload id=21 objects=2 duration=269.5µs42882026/11/12 00:10:32 INFO Vacuumed table table=closures42892026/11/12 00:10:32 INFO Received uploads request method=POST path=/42902026/11/12 00:10:32 INFO Vacuumed table table=objects42912026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=042922026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period42932026/11/12 00:10:32 INFO Received uploads request method=POST path=/42942026/11/12 00:10:32 INFO Aborted multipart uploads count=0 kept=042952026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042962026/11/12 00:10:32 WARN Force mode enabled - objects will be deleted immediately without grace period42972026/11/12 00:10:32 INFO Received uploads request method=POST path=/42982026/11/12 00:10:32 INFO Vacuumed table table=pending_closures42992026/11/12 00:10:32 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043002026/11/12 00:10:33 INFO Received push request method=POST path=/43012026/11/12 00:10:33 INFO Vacuumed table table=pending_closures43022026/11/12 00:10:33 INFO Vacuumed table table=pending_objects43032026/11/12 00:10:33 INFO Received complete upload request method=POST path=/43042026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads43052026/11/12 00:10:33 INFO Vacuumed table table=pending_objects43062026/11/12 00:10:33 INFO Completed upload id=8 objects=3 duration=681.917µs43072026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads43082026/11/12 00:10:33 INFO Received complete upload request method=POST path=/43092026/11/12 00:10:33 INFO Received uploads request method=POST path=/43102026/11/12 00:10:33 INFO Vacuumed table table=closures43112026/11/12 00:10:33 INFO Completed upload id=17 objects=7 duration=343.25µs43122026/11/12 00:10:33 INFO Vacuumed table table=objects43132026/11/12 00:10:33 INFO Received complete push request method=POST path=/43142026/11/12 00:10:33 INFO Vacuumed table table=closures43152026/11/12 00:10:33 INFO Completed push id=6 objects=6 duration=571.667µs43162026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=043172026/11/12 00:10:33 INFO Vacuumed table table=objects43182026/11/12 00:10:33 INFO Received push request method=POST path=/43192026/11/12 00:10:33 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."43202026/11/12 00:10:33 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."43212026/11/12 00:10:33 INFO Received complete push request method=POST path=/43222026/11/12 00:10:33 INFO Completed push id=9 objects=9 duration=1.233333ms43232026/11/12 00:10:33 INFO Received complete upload request method=POST path=/43242026/11/12 00:10:33 INFO Completed upload id=14 objects=6 duration=244.25µs43252026/11/12 00:10:33 INFO Received complete upload request method=POST path=/43262026/11/12 00:10:33 INFO Received push request method=POST path=/43272026/11/12 00:10:33 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."43282026/11/12 00:10:33 INFO Completed upload id=21 objects=8 duration=656.25µs43292026/11/12 00:10:33 INFO Received complete push request method=POST path=/43302026/11/12 00:10:33 INFO Received uploads request method=POST path=/43312026/11/12 00:10:33 INFO Completed push id=22 objects=6 duration=10.757292ms43322026/11/12 00:10:33 INFO Received complete push request method=POST path=/43332026/11/12 00:10:33 INFO Completed push id=24 objects=8 duration=228.208µs43342026/11/12 00:10:33 INFO Received complete push request method=POST path=/43352026/11/12 00:10:33 INFO Received uploads request method=POST path=/43362026/11/12 00:10:33 INFO Received complete upload request method=POST path=/43372026/11/12 00:10:33 INFO Completed push id=10 objects=15 duration=545.458µs43382026/11/12 00:10:33 INFO Completed upload id=25 objects=9 duration=205.458µs43392026/11/12 00:10:33 INFO Received complete upload request method=POST path=/43402026/11/12 00:10:33 INFO Completed upload id=27 objects=3 duration=489.208µs43412026/11/12 00:10:33 INFO Received complete upload request method=POST path=/43422026/11/12 00:10:33 INFO Completed upload id=28 objects=6 duration=316.333µs43432026/11/12 00:10:33 INFO Received complete push request method=POST path=/43442026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=043452026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=043462026/11/12 00:10:33 INFO Completed push id=16 objects=7 duration=545.208µs43472026/11/12 00:10:33 WARN Force mode enabled - objects will be deleted immediately without grace period43482026/11/12 00:10:33 INFO Received complete push request method=POST path=/43492026/11/12 00:10:33 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."43502026/11/12 00:10:33 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."43512026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=043522026/11/12 00:10:33 INFO Completed push id=30 objects=4 duration=291.375µs43532026/11/12 00:10:33 INFO Received complete upload request method=POST path=/43542026/11/12 00:10:33 INFO Completed upload id=7 objects=3 duration=406.083µs43552026/11/12 00:10:33 INFO Vacuumed table table=pending_closures43562026/11/12 00:10:33 INFO Received complete push request method=POST path=/43572026/11/12 00:10:33 INFO Completed push id=31 objects=7 duration=282.875µs43582026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=043592026/11/12 00:10:33 WARN Force mode enabled - objects will be deleted immediately without grace period43602026/11/12 00:10:33 INFO Vacuumed table table=pending_objects43612026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads43622026/11/12 00:10:33 INFO Received complete upload request method=POST path=/43632026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=043642026/11/12 00:10:33 INFO Vacuumed table table=closures43652026/11/12 00:10:33 INFO Completed upload id=32 objects=5 duration=175.75µs43662026/11/12 00:10:33 INFO Vacuumed table table=pending_closures43672026/11/12 00:10:33 INFO Vacuumed table table=objects43682026/11/12 00:10:33 INFO Vacuumed table table=pending_objects43692026/11/12 00:10:33 INFO Received push request method=POST path=/43702026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads43712026/11/12 00:10:33 INFO Vacuumed table table=closures43722026/11/12 00:10:33 INFO Vacuumed table table=objects43732026/11/12 00:10:33 INFO Received complete upload request method=POST path=/43742026/11/12 00:10:33 INFO Completed upload id=21 objects=7 duration=740.334µs43752026/11/12 00:10:33 INFO Received uploads request method=POST path=/43762026/11/12 00:10:33 INFO Received push request method=POST path=/43772026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=043782026/11/12 00:10:33 WARN Force mode enabled - objects will be deleted immediately without grace period43792026/11/12 00:10:33 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."43802026/11/12 00:10:33 INFO Received uploads request method=POST path=/43812026/11/12 00:10:33 INFO Received complete push request method=POST path=/43822026/11/12 00:10:33 INFO Received complete push request method=POST path=/43832026/11/12 00:10:33 INFO Completed push id=16 objects=16 duration=1.634375ms43842026/11/12 00:10:33 INFO Received uploads request method=POST path=/43852026/11/12 00:10:33 INFO Received complete push request method=POST path=/43862026/11/12 00:10:33 INFO Completed push id=18 objects=23 duration=449.917µs43872026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=043882026/11/12 00:10:33 INFO Received complete upload request method=POST path=/43892026/11/12 00:10:33 INFO Received complete push request method=POST path=/43902026/11/12 00:10:33 INFO Completed upload id=27 objects=4 duration=427.084µs43912026/11/12 00:10:33 INFO Completed push id=15 objects=7 duration=542.875µs43922026/11/12 00:10:33 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."43932026/11/12 00:10:33 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."43942026/11/12 00:10:33 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."43952026/11/12 00:10:33 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."43962026/11/12 00:10:33 INFO Received uploads request method=POST path=/43972026/11/12 00:10:33 INFO Completed push id=23 objects=6 duration=267.584µs43982026/11/12 00:10:33 INFO Received complete push request method=POST path=/43992026/11/12 00:10:33 INFO Received complete push request method=POST path=/44002026/11/12 00:10:33 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."44012026/11/12 00:10:33 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."44022026/11/12 00:10:33 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."44032026/11/12 00:10:33 INFO Completed push id=13 objects=11 duration=877.083µs44042026/11/12 00:10:33 INFO Received uploads request method=POST path=/44052026/11/12 00:10:33 INFO Completed push id=7 objects=24 duration=1.750875ms44062026/11/12 00:10:33 INFO Received complete push request method=POST path=/44072026/11/12 00:10:33 INFO Received uploads request method=POST path=/44082026/11/12 00:10:33 INFO Completed push id=24 objects=6 duration=499.292µs44092026/11/12 00:10:33 INFO Received uploads request method=POST path=/44102026/11/12 00:10:33 INFO Received uploads request method=POST path=/44112026/11/12 00:10:33 INFO Received uploads request method=POST path=/44122026/11/12 00:10:33 INFO Received complete upload request method=POST path=/44132026/11/12 00:10:33 INFO Completed upload id=14 objects=8 duration=1.009417ms44142026/11/12 00:10:33 INFO Received uploads request method=POST path=/44152026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=044162026/11/12 00:10:33 INFO Received uploads request method=POST path=/44172026/11/12 00:10:33 WARN Force mode enabled - objects will be deleted immediately without grace period44182026/11/12 00:10:33 INFO Received push request method=POST path=/44192026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044202026/11/12 00:10:33 INFO Received push request method=POST path=/44212026/11/12 00:10:33 INFO Received uploads request method=POST path=/44222026/11/12 00:10:33 INFO Received uploads request method=POST path=/44232026/11/12 00:10:33 INFO Vacuumed table table=pending_closures44242026/11/12 00:10:33 INFO Received uploads request method=POST path=/44252026/11/12 00:10:33 INFO Vacuumed table table=pending_objects44262026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads44272026/11/12 00:10:33 INFO Received complete upload request method=POST path=/44282026/11/12 00:10:33 INFO Vacuumed table table=closures44292026/11/12 00:10:33 INFO Received push request method=POST path=/44302026/11/12 00:10:33 INFO Completed upload id=22 objects=2 duration=523.25µs44312026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=044322026/11/12 00:10:33 INFO Received uploads request method=POST path=/44332026/11/12 00:10:33 INFO Vacuumed table table=objects44342026/11/12 00:10:33 WARN Force mode enabled - objects will be deleted immediately without grace period44352026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=044362026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=044372026/11/12 00:10:33 INFO Received complete push request method=POST path=/44382026/11/12 00:10:33 INFO Received uploads request method=POST path=/44392026/11/12 00:10:33 INFO Completed push id=13 objects=2 duration=349.042µs44402026/11/12 00:10:33 WARN Force mode enabled - objects will be deleted immediately without grace period44412026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=044422026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=044432026/11/12 00:10:33 INFO Received complete push request method=POST path=/44442026/11/12 00:10:33 INFO Vacuumed table table=pending_closures44452026/11/12 00:10:33 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."44462026/11/12 00:10:33 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."44472026/11/12 00:10:33 INFO Vacuumed table table=pending_objects44482026/11/12 00:10:33 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."44492026/11/12 00:10:33 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."44502026/11/12 00:10:33 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."44512026/11/12 00:10:33 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."44522026/11/12 00:10:33 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."44532026/11/12 00:10:33 INFO Received complete upload request method=POST path=/44542026/11/12 00:10:33 INFO Completed push id=11 objects=9 duration=443.292µs44552026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads44562026/11/12 00:10:33 INFO Completed upload id=27 objects=13 duration=251.792µs44572026/11/12 00:10:33 INFO Vacuumed table table=pending_closures44582026/11/12 00:10:33 INFO Received push request method=POST path=/44592026/11/12 00:10:33 INFO Received complete upload request method=POST path=/44602026/11/12 00:10:33 INFO Vacuumed table table=closures44612026/11/12 00:10:33 INFO Completed upload id=21 objects=3 duration=6.055667ms44622026/11/12 00:10:33 INFO Vacuumed table table=pending_objects44632026/11/12 00:10:33 INFO Vacuumed table table=objects44642026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=044652026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads44662026/11/12 00:10:33 INFO Received push request method=POST path=/44672026/11/12 00:10:33 INFO Vacuumed table table=closures44682026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=044692026/11/12 00:10:33 INFO Received complete push request method=POST path=/44702026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044712026/11/12 00:10:33 INFO Received uploads request method=POST path=/44722026/11/12 00:10:33 WARN Force mode enabled - objects will be deleted immediately without grace period44732026/11/12 00:10:33 INFO Received complete push request method=POST path=/44742026/11/12 00:10:33 INFO Vacuumed table table=pending_closures44752026/11/12 00:10:33 INFO Completed push id=25 objects=7 duration=522.041µs44762026/11/12 00:10:33 INFO Received uploads request method=POST path=/44772026/11/12 00:10:33 INFO Vacuumed table table=objects44782026/11/12 00:10:33 INFO Vacuumed table table=pending_objects44792026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads44802026/11/12 00:10:33 INFO Completed push id=16 objects=14 duration=804.667µs44812026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=7 objects-failed-to-delete=044822026/11/12 00:10:33 INFO Received push request method=POST path=/44832026/11/12 00:10:33 INFO Vacuumed table table=closures44842026/11/12 00:10:33 INFO Vacuumed table table=objects44852026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=044862026/11/12 00:10:33 INFO Received push request method=POST path=/44872026/11/12 00:10:33 WARN Force mode enabled - objects will be deleted immediately without grace period44882026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=044892026/11/12 00:10:33 INFO Received push request method=POST path=/44902026/11/12 00:10:33 INFO Received uploads request method=POST path=/44912026/11/12 00:10:33 INFO Received uploads request method=POST path=/44922026/11/12 00:10:33 INFO Vacuumed table table=pending_closures44932026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044942026/11/12 00:10:33 WARN Force mode enabled - objects will be deleted immediately without grace period44952026/11/12 00:10:33 INFO Vacuumed table table=pending_closures44962026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044972026/11/12 00:10:33 INFO Vacuumed table table=pending_objects44982026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads44992026/11/12 00:10:33 INFO Vacuumed table table=pending_closures45002026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=045012026/11/12 00:10:33 INFO Received push request method=POST path=/45022026/11/12 00:10:33 INFO Vacuumed table table=pending_objects45032026/11/12 00:10:33 INFO Vacuumed table table=pending_objects45042026/11/12 00:10:33 INFO Vacuumed table table=closures45052026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045062026/11/12 00:10:33 INFO Received push request method=POST path=/45072026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads45082026/11/12 00:10:33 INFO Vacuumed table table=pending_closures45092026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads45102026/11/12 00:10:33 INFO Vacuumed table table=closures45112026/11/12 00:10:33 INFO Vacuumed table table=objects45122026/11/12 00:10:33 INFO Vacuumed table table=closures45132026/11/12 00:10:33 INFO Vacuumed table table=objects45142026/11/12 00:10:33 INFO Received uploads request method=POST path=/45152026/11/12 00:10:33 INFO Vacuumed table table=pending_objects45162026/11/12 00:10:33 INFO Vacuumed table table=objects45172026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=045182026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads45192026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=045202026/11/12 00:10:33 INFO Received uploads request method=POST path=/45212026/11/12 00:10:33 INFO Vacuumed table table=closures45222026/11/12 00:10:33 WARN Force mode enabled - objects will be deleted immediately without grace period45232026/11/12 00:10:33 INFO Received uploads request method=POST path=/45242026/11/12 00:10:33 INFO Vacuumed table table=objects45252026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045262026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045272026/11/12 00:10:33 INFO Vacuumed table table=pending_closures45282026/11/12 00:10:33 INFO Received complete upload request method=POST path=/45292026/11/12 00:10:33 INFO Vacuumed table table=pending_closures45302026/11/12 00:10:33 INFO Received push request method=POST path=/45312026/11/12 00:10:33 INFO Vacuumed table table=pending_objects45322026/11/12 00:10:33 INFO Completed upload id=29 objects=4 duration=525.542µs45332026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads45342026/11/12 00:10:33 INFO Vacuumed table table=pending_objects45352026/11/12 00:10:33 INFO Received push request method=POST path=/45362026/11/12 00:10:33 INFO Received uploads request method=POST path=/45372026/11/12 00:10:33 INFO Vacuumed table table=closures45382026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads45392026/11/12 00:10:33 INFO Vacuumed table table=closures45402026/11/12 00:10:33 INFO Vacuumed table table=objects45412026/11/12 00:10:33 INFO Received push request method=POST path=/45422026/11/12 00:10:33 INFO Received uploads request method=POST path=/45432026/11/12 00:10:33 INFO Received complete upload request method=POST path=/45442026/11/12 00:10:33 INFO Vacuumed table table=objects45452026/11/12 00:10:33 INFO Completed upload id=23 objects=15 duration=5.813667ms45462026/11/12 00:10:33 INFO Received complete upload request method=POST path=/45472026/11/12 00:10:33 INFO Received push request method=POST path=/45482026/11/12 00:10:33 INFO Received complete upload request method=POST path=/45492026/11/12 00:10:33 INFO Completed upload id=34 objects=24 duration=451.041µs45502026/11/12 00:10:33 INFO Received push request method=POST path=/45512026/11/12 00:10:33 INFO Completed upload id=16 objects=17 duration=771.834µs45522026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=045532026/11/12 00:10:33 WARN Force mode enabled - objects will be deleted immediately without grace period45542026/11/12 00:10:33 INFO Received complete push request method=POST path=/45552026/11/12 00:10:33 INFO Received uploads request method=POST path=/45562026/11/12 00:10:33 INFO Received uploads request method=POST path=/45572026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045582026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=045592026/11/12 00:10:33 INFO Completed push id=22 objects=3 duration=895.833µs45602026/11/12 00:10:33 INFO Received complete upload request method=POST path=/45612026/11/12 00:10:33 INFO Completed upload id=25 objects=6 duration=395.375µs45622026/11/12 00:10:33 WARN Force mode enabled - objects will be deleted immediately without grace period45632026/11/12 00:10:33 INFO Received uploads request method=POST path=/45642026/11/12 00:10:33 INFO Vacuumed table table=pending_closures45652026/11/12 00:10:33 INFO Vacuumed table table=pending_objects45662026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads45672026/11/12 00:10:33 INFO Received uploads request method=POST path=/45682026/11/12 00:10:33 INFO Received push request method=POST path=/45692026/11/12 00:10:33 INFO Received push request method=POST path=/45702026/11/12 00:10:33 INFO Vacuumed table table=closures45712026/11/12 00:10:33 INFO Received complete upload request method=POST path=/45722026/11/12 00:10:33 INFO Received complete push request method=POST path=/45732026/11/12 00:10:33 INFO Vacuumed table table=objects45742026/11/12 00:10:33 INFO Received push request method=POST path=/45752026/11/12 00:10:33 INFO Completed upload id=19 objects=6 duration=711.041µs45762026/11/12 00:10:33 INFO Completed push id=11 objects=6 duration=1.37775ms45772026/11/12 00:10:33 INFO Received complete push request method=POST path=/45782026/11/12 00:10:33 INFO Received uploads request method=POST path=/45792026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=045802026/11/12 00:10:33 INFO Completed push id=24 objects=3 duration=469.209µs45812026/11/12 00:10:33 WARN Force mode enabled - objects will be deleted immediately without grace period45822026/11/12 00:10:33 INFO Received complete push request method=POST path=/45832026/11/12 00:10:33 INFO Received push request method=POST path=/45842026/11/12 00:10:33 INFO Completed push id=11 objects=9 duration=470.542µs45852026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045862026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=045872026/11/12 00:10:33 INFO Received complete upload request method=POST path=/45882026/11/12 00:10:33 WARN Force mode enabled - objects will be deleted immediately without grace period45892026/11/12 00:10:33 INFO Vacuumed table table=pending_closures45902026/11/12 00:10:33 INFO Completed upload id=26 objects=24 duration=500.958µs45912026/11/12 00:10:33 INFO Vacuumed table table=pending_objects45922026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads45932026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045942026/11/12 00:10:33 INFO Vacuumed table table=closures45952026/11/12 00:10:33 INFO Vacuumed table table=objects45962026/11/12 00:10:33 INFO Vacuumed table table=pending_closures45972026/11/12 00:10:33 INFO Received uploads request method=POST path=/45982026/11/12 00:10:33 INFO Received complete push request method=POST path=/45992026/11/12 00:10:33 INFO Vacuumed table table=pending_objects46002026/11/12 00:10:33 INFO Received complete push request method=POST path=/46012026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=046022026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads46032026/11/12 00:10:33 INFO Completed push id=20 objects=13 duration=399.5µs46042026/11/12 00:10:33 INFO Completed push id=15 objects=24 duration=397.875µs46052026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046062026/11/12 00:10:33 INFO Vacuumed table table=closures46072026/11/12 00:10:33 INFO Received complete upload request method=POST path=/46082026/11/12 00:10:33 INFO Vacuumed table table=pending_closures46092026/11/12 00:10:33 INFO Received complete upload request method=POST path=/46102026/11/12 00:10:33 INFO Received complete push request method=POST path=/46112026/11/12 00:10:33 INFO Vacuumed table table=objects46122026/11/12 00:10:33 INFO Completed upload id=22 objects=7 duration=220.583µs46132026/11/12 00:10:33 INFO Completed push id=22 objects=4 duration=227.292µs46142026/11/12 00:10:33 INFO Completed upload id=19 objects=2 duration=336.125µs46152026/11/12 00:10:33 INFO Vacuumed table table=pending_objects46162026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads46172026/11/12 00:10:33 INFO Vacuumed table table=closures46182026/11/12 00:10:33 INFO Received complete upload request method=POST path=/46192026/11/12 00:10:33 INFO Vacuumed table table=objects46202026/11/12 00:10:33 INFO Received complete push request method=POST path=/46212026/11/12 00:10:33 INFO Completed upload id=24 objects=7 duration=260.917µs46222026/11/12 00:10:33 INFO Completed push id=24 objects=9 duration=4.2975ms46232026/11/12 00:10:33 INFO Aborted multipart uploads count=0 kept=046242026/11/12 00:10:33 INFO Received push request method=POST path=/46252026/11/12 00:10:33 INFO Received complete push request method=POST path=/46262026/11/12 00:10:33 INFO Completed push id=26 objects=3 duration=545.375µs46272026/11/12 00:10:33 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=046282026/11/12 00:10:33 INFO Received push request method=POST path=/46292026/11/12 00:10:33 INFO Received complete upload request method=POST path=/46302026/11/12 00:10:33 INFO Completed upload id=28 objects=8 duration=253.917µs46312026/11/12 00:10:33 INFO Vacuumed table table=pending_closures46322026/11/12 00:10:33 INFO Received complete upload request method=POST path=/46332026/11/12 00:10:33 INFO Received uploads request method=POST path=/46342026/11/12 00:10:33 INFO Completed upload id=29 objects=10 duration=167.042µs46352026/11/12 00:10:33 INFO Received complete upload request method=POST path=/46362026/11/12 00:10:33 INFO Completed upload id=30 objects=20 duration=228.833µs46372026/11/12 00:10:33 INFO Vacuumed table table=pending_objects46382026/11/12 00:10:33 INFO Received complete push request method=POST path=/46392026/11/12 00:10:33 INFO Vacuumed table table=multipart_uploads46402026/11/12 00:10:33 INFO Received complete push request method=POST path=/46412026/11/12 00:10:33 INFO Vacuumed table table=closures46422026/11/12 00:10:33 INFO Completed push id=31 objects=11 duration=5.96375ms46432026/11/12 00:10:33 INFO Completed push id=21 objects=3 duration=6.025958ms46442026/11/12 00:10:33 INFO Received complete upload request method=POST path=/46452026/11/12 00:10:34 INFO Vacuumed table table=objects46462026/11/12 00:10:34 INFO Completed upload id=23 objects=7 duration=706.5µs46472026/11/12 00:10:34 INFO Received complete upload request method=POST path=/46482026/11/12 00:10:34 INFO Received uploads request method=POST path=/46492026/11/12 00:10:34 INFO Completed upload id=32 objects=2 duration=210.208µs46502026/11/12 00:10:34 INFO Received complete push request method=POST path=/46512026/11/12 00:10:34 INFO Received uploads request method=POST path=/46522026/11/12 00:10:34 INFO Received complete push request method=POST path=/46532026/11/12 00:10:34 INFO Aborted multipart uploads count=0 kept=046542026/11/12 00:10:34 INFO Received complete upload request method=POST path=/46552026/11/12 00:10:34 INFO Completed push id=24 objects=5 duration=268.458µs46562026/11/12 00:10:34 INFO Completed push id=33 objects=3 duration=326.208µs46572026/11/12 00:10:34 INFO Completed upload id=33 objects=5 duration=5.972583ms46582026/11/12 00:10:34 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=3 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046592026/11/12 00:10:34 INFO Received complete upload request method=POST path=/46602026/11/12 00:10:34 INFO Completed upload id=25 objects=2 duration=464.625µs46612026/11/12 00:10:34 INFO Received complete upload request method=POST path=/46622026/11/12 00:10:34 INFO Received complete upload request method=POST path=/46632026/11/12 00:10:34 INFO Vacuumed table table=pending_closures46642026/11/12 00:10:34 INFO Completed upload id=34 objects=3 duration=230.958µs46652026/11/12 00:10:34 INFO Completed upload id=26 objects=2 duration=6.32725ms46662026/11/12 00:10:34 INFO Received complete upload request method=POST path=/46672026/11/12 00:10:34 INFO Received complete push request method=POST path=/46682026/11/12 00:10:34 INFO Aborted multipart uploads count=0 kept=046692026/11/12 00:10:34 INFO Completed upload id=26 objects=20 duration=246.041µs46702026/11/12 00:10:34 INFO Vacuumed table table=pending_objects46712026/11/12 00:10:34 INFO Aborted multipart uploads count=0 kept=046722026/11/12 00:10:34 INFO Completed push id=15 objects=24 duration=918.125µs46732026/11/12 00:10:34 WARN Force mode enabled - objects will be deleted immediately without grace period46742026/11/12 00:10:34 INFO Vacuumed table table=multipart_uploads46752026/11/12 00:10:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046762026/11/12 00:10:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046772026/11/12 00:10:34 INFO Vacuumed table table=closures46782026/11/12 00:10:34 INFO Received uploads request method=POST path=/46792026/11/12 00:10:34 INFO Vacuumed table table=objects46802026/11/12 00:10:34 INFO Received uploads request method=POST path=/46812026/11/12 00:10:34 INFO Vacuumed table table=pending_closures46822026/11/12 00:10:34 INFO Received push request method=POST path=/46832026/11/12 00:10:34 INFO Vacuumed table table=pending_closures46842026/11/12 00:10:34 INFO Received uploads request method=POST path=/46852026/11/12 00:10:34 INFO Received push request method=POST path=/46862026/11/12 00:10:34 INFO Received complete push request method=POST path=/46872026/11/12 00:10:34 INFO Vacuumed table table=pending_objects46882026/11/12 00:10:34 INFO Completed push id=30 objects=4 duration=6.087333ms46892026/11/12 00:10:34 INFO Vacuumed table table=multipart_uploads46902026/11/12 00:10:34 INFO Received uploads request method=POST path=/46912026/11/12 00:10:34 INFO Vacuumed table table=closures46922026/11/12 00:10:34 INFO Received push request method=POST path=/46932026/11/12 00:10:34 INFO Vacuumed table table=objects46942026/11/12 00:10:34 INFO Vacuumed table table=pending_objects46952026/11/12 00:10:34 INFO Received push request method=POST path=/46962026/11/12 00:10:34 INFO Vacuumed table table=multipart_uploads46972026/11/12 00:10:34 INFO Received complete upload request method=POST path=/46982026/11/12 00:10:34 INFO Received complete push request method=POST path=/46992026/11/12 00:10:34 INFO Vacuumed table table=closures47002026/11/12 00:10:34 INFO Vacuumed table table=objects47012026/11/12 00:10:34 INFO Completed upload id=27 objects=26 duration=4.406584ms47022026/11/12 00:10:34 INFO Completed push id=32 objects=7 duration=4.403792ms47032026/11/12 00:10:34 INFO Received complete push request method=POST path=/47042026/11/12 00:10:34 INFO Completed push id=28 objects=7 duration=805.375µs47052026/11/12 00:10:34 INFO Received complete upload request method=POST path=/47062026/11/12 00:10:34 INFO Completed upload id=6 objects=6 duration=335.125µs47072026/11/12 00:10:34 INFO Received complete push request method=POST path=/47082026/11/12 00:10:34 INFO Received complete push request method=POST path=/47092026/11/12 00:10:34 INFO Completed push id=26 objects=4 duration=274.208µs47102026/11/12 00:10:34 INFO Received push request method=POST path=/47112026/11/12 00:10:34 INFO Aborted multipart uploads count=0 kept=047122026/11/12 00:10:34 INFO Completed push id=28 objects=6 duration=351.75µs47132026/11/12 00:10:34 WARN Force mode enabled - objects will be deleted immediately without grace period47142026/11/12 00:10:34 INFO Received complete upload request method=POST path=/47152026/11/12 00:10:34 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047162026/11/12 00:10:34 INFO Received complete push request method=POST path=/47172026/11/12 00:10:34 INFO Completed upload id=28 objects=13 duration=262µs47182026/11/12 00:10:34 INFO Aborted multipart uploads count=0 kept=047192026/11/12 00:10:34 INFO Completed push id=29 objects=6 duration=216.416µs47202026/11/12 00:10:34 INFO Vacuumed table table=pending_closures47212026/11/12 00:10:34 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."47222026/11/12 00:10:34 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."47232026/11/12 00:10:34 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."47242026/11/12 00:10:34 INFO Received complete push request method=POST path=/47252026/11/12 00:10:34 INFO Completed push id=29 objects=3 duration=235.5µs47262026/11/12 00:10:34 INFO Vacuumed table table=pending_objects47272026/11/12 00:10:34 INFO Vacuumed table table=multipart_uploads47282026/11/12 00:10:34 INFO Vacuumed table table=closures47292026/11/12 00:10:34 INFO Received push request method=POST path=/47302026/11/12 00:10:34 INFO Received complete upload request method=POST path=/47312026/11/12 00:10:34 INFO Completed upload id=31 objects=2 duration=273.834µs47322026/11/12 00:10:34 INFO Vacuumed table table=objects47332026/11/12 00:10:34 INFO Received complete push request method=POST path=/47342026/11/12 00:10:34 INFO Completed push id=33 objects=6 duration=250.458µs47352026/11/12 00:10:34 INFO Received push request method=POST path=/47362026/11/12 00:10:34 INFO Received push request method=POST path=/47372026/11/12 00:10:34 INFO Received complete upload request method=POST path=/47382026/11/12 00:10:34 INFO Received complete push request method=POST path=/47392026/11/12 00:10:34 INFO Completed upload id=30 objects=6 duration=948.167µs47402026/11/12 00:10:34 INFO Received uploads request method=POST path=/47412026/11/12 00:10:34 INFO Aborted multipart uploads count=0 kept=047422026/11/12 00:10:34 WARN Force mode enabled - objects will be deleted immediately without grace period47432026/11/12 00:10:34 INFO Completed push id=28 objects=11 duration=4.61875ms47442026/11/12 00:10:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047452026/11/12 00:10:34 INFO Received complete push request method=POST path=/47462026/11/12 00:10:34 INFO Completed push id=31 objects=5 duration=221.792µs47472026/11/12 00:10:34 INFO Aborted multipart uploads count=0 kept=047482026/11/12 00:10:34 INFO Vacuumed table table=pending_closures47492026/11/12 00:10:34 WARN Force mode enabled - objects will be deleted immediately without grace period47502026/11/12 00:10:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047512026/11/12 00:10:34 INFO Vacuumed table table=pending_closures47522026/11/12 00:10:34 INFO Vacuumed table table=pending_objects47532026/11/12 00:10:34 INFO Vacuumed table table=multipart_uploads47542026/11/12 00:10:34 INFO Vacuumed table table=pending_objects47552026/11/12 00:10:34 INFO Vacuumed table table=multipart_uploads47562026/11/12 00:10:34 INFO Vacuumed table table=closures47572026/11/12 00:10:34 INFO Vacuumed table table=closures47582026/11/12 00:10:34 INFO Vacuumed table table=objects47592026/11/12 00:10:34 INFO Vacuumed table table=objects47602026/11/12 00:10:34 INFO Received push request method=POST path=/47612026/11/12 00:10:34 INFO Received uploads request method=POST path=/47622026/11/12 00:10:34 INFO Aborted multipart uploads count=0 kept=047632026/11/12 00:10:34 WARN Force mode enabled - objects will be deleted immediately without grace period47642026/11/12 00:10:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047652026/11/12 00:10:34 INFO Vacuumed table table=pending_closures47662026/11/12 00:10:34 INFO Vacuumed table table=pending_objects47672026/11/12 00:10:34 INFO Vacuumed table table=multipart_uploads47682026/11/12 00:10:34 INFO Vacuumed table table=closures47692026/11/12 00:10:34 INFO Vacuumed table table=objects47702026/11/12 00:10:34 INFO Aborted multipart uploads count=0 kept=047712026/11/12 00:10:34 WARN Force mode enabled - objects will be deleted immediately without grace period47722026/11/12 00:10:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047732026/11/12 00:10:34 INFO Vacuumed table table=pending_closures47742026/11/12 00:10:34 INFO Vacuumed table table=pending_objects47752026/11/12 00:10:34 INFO Vacuumed table table=multipart_uploads47762026/11/12 00:10:34 INFO Received complete upload request method=POST path=/47772026/11/12 00:10:34 INFO Vacuumed table table=closures47782026/11/12 00:10:34 INFO Received complete push request method=POST path=/47792026/11/12 00:10:34 INFO Completed upload id=21 objects=25 duration=441.459µs47802026/11/12 00:10:34 INFO Vacuumed table table=objects47812026/11/12 00:10:34 INFO Received complete upload request method=POST path=/47822026/11/12 00:10:34 INFO Completed push id=32 objects=26 duration=520.75µs47832026/11/12 00:10:34 INFO Received uploads request method=POST path=/47842026/11/12 00:10:34 INFO Completed upload id=30 objects=8 duration=385.75µs47852026/11/12 00:10:34 INFO Received push request method=POST path=/47862026/11/12 00:10:34 INFO Received complete upload request method=POST path=/47872026/11/12 00:10:34 INFO Completed upload id=33 objects=2 duration=297.916µs47882026/11/12 00:10:34 INFO Received complete upload request method=POST path=/47892026/11/12 00:10:34 INFO Completed upload id=34 objects=5 duration=268.167µs47902026/11/12 00:10:34 INFO Received complete upload request method=POST path=/47912026/11/12 00:10:34 INFO Completed upload id=35 objects=11 duration=482.875µs47922026/11/12 00:10:34 INFO Received complete upload request method=POST path=/47932026/11/12 00:10:34 INFO Completed upload id=36 objects=3 duration=288.417µs47942026/11/12 00:10:34 INFO Aborted multipart uploads count=0 kept=047952026/11/12 00:10:34 WARN Force mode enabled - objects will be deleted immediately without grace period47962026/11/12 00:10:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047972026/11/12 00:10:34 INFO Vacuumed table table=pending_closures47982026/11/12 00:10:34 INFO Received complete upload request method=POST path=/47992026/11/12 00:10:34 INFO Completed upload id=31 objects=6 duration=330.708µs48002026/11/12 00:10:34 INFO Vacuumed table table=pending_objects48012026/11/12 00:10:34 INFO Vacuumed table table=multipart_uploads48022026/11/12 00:10:34 INFO Received complete upload request method=POST path=/48032026/11/12 00:10:34 INFO Vacuumed table table=closures48042026/11/12 00:10:34 INFO Completed upload id=32 objects=4 duration=365.875µs48052026/11/12 00:10:34 INFO Received complete push request method=POST path=/48062026/11/12 00:10:34 INFO Vacuumed table table=objects48072026/11/12 00:10:34 INFO Completed push id=29 objects=24 duration=1.186166ms48082026/11/12 00:10:34 INFO Aborted multipart uploads count=0 kept=048092026/11/12 00:10:34 WARN Force mode enabled - objects will be deleted immediately without grace period48102026/11/12 00:10:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=048112026/11/12 00:10:34 INFO Vacuumed table table=pending_closures48122026/11/12 00:10:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=048132026/11/12 00:10:34 INFO Vacuumed table table=pending_closures48142026/11/12 00:10:34 INFO Vacuumed table table=pending_objects48152026/11/12 00:10:34 INFO Vacuumed table table=multipart_uploads48162026/11/12 00:10:34 INFO Vacuumed table table=closures48172026/11/12 00:10:34 INFO Vacuumed table table=pending_objects48182026/11/12 00:10:34 INFO Vacuumed table table=multipart_uploads48192026/11/12 00:10:34 INFO Vacuumed table table=objects48202026/11/12 00:10:34 INFO Received complete upload request method=POST path=/48212026/11/12 00:10:34 INFO Vacuumed table table=closures48222026/11/12 00:10:34 INFO Completed upload id=33 objects=10 duration=399.083µs48232026/11/12 00:10:34 INFO Received complete upload request method=POST path=/48242026/11/12 00:10:34 INFO Vacuumed table table=objects48252026/11/12 00:10:34 INFO Completed upload id=23 objects=16 duration=2.04375ms48262026/11/12 00:10:34 INFO Aborted multipart uploads count=0 kept=048272026/11/12 00:10:34 WARN Force mode enabled - objects will be deleted immediately without grace period48282026/11/12 00:10:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048292026/11/12 00:10:34 INFO Vacuumed table table=pending_closures48302026/11/12 00:10:34 INFO Aborted multipart uploads count=0 kept=048312026/11/12 00:10:34 INFO Vacuumed table table=pending_objects48322026/11/12 00:10:34 WARN Force mode enabled - objects will be deleted immediately without grace period48332026/11/12 00:10:34 INFO Vacuumed table table=multipart_uploads48342026/11/12 00:10:34 INFO Vacuumed table table=closures48352026/11/12 00:10:34 INFO Vacuumed table table=objects48362026/11/12 00:10:34 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048372026/11/12 00:10:34 INFO Vacuumed table table=pending_closures48382026/11/12 00:10:34 INFO Received complete upload request method=POST path=/48392026/11/12 00:10:34 INFO Vacuumed table table=pending_objects48402026/11/12 00:10:34 INFO Vacuumed table table=multipart_uploads48412026/11/12 00:10:34 INFO Completed upload id=15 objects=3 duration=5.616792ms48422026/11/12 00:10:34 INFO Vacuumed table table=closures48432026/11/12 00:10:34 INFO Vacuumed table table=objects48442026/11/12 00:10:34 INFO Received push request method=POST path=/48452026/11/12 00:10:34 INFO Received uploads request method=POST path=/48462026/11/12 00:10:34 INFO Received complete upload request method=POST path=/48472026/11/12 00:10:34 INFO Completed upload id=12 objects=6 duration=375.875µs48482026/11/12 00:10:34 INFO Received complete push request method=POST path=/48492026/11/12 00:10:34 INFO Completed push id=17 objects=3 duration=549.333µs48502026/11/12 00:10:34 INFO Received complete upload request method=POST path=/48512026/11/12 00:10:34 INFO Completed upload id=34 objects=23 duration=339.75µs48522026/11/12 00:10:34 INFO Received complete push request method=POST path=/48532026/11/12 00:10:34 INFO Completed push id=23 objects=5 duration=302.166µs48542026/11/12 00:10:34 INFO Received complete upload request method=POST path=/48552026/11/12 00:10:34 INFO Completed upload id=24 objects=3 duration=202.5µs48562026/11/12 00:10:34 INFO Received complete push request method=POST path=/48572026/11/12 00:10:34 INFO Completed push id=25 objects=4 duration=506.875µs48582026/11/12 00:10:34 INFO Received complete push request method=POST path=/48592026/11/12 00:10:34 INFO Completed push id=35 objects=6 duration=245.75µs48602026/11/12 00:10:34 INFO Aborted multipart uploads count=0 kept=048612026/11/12 00:10:34 WARN Force mode enabled - objects will be deleted immediately without grace period48622026/11/12 00:10:34 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=048632026/11/12 00:10:34 INFO Vacuumed table table=pending_closures48642026/11/12 00:10:34 INFO Vacuumed table table=pending_objects48652026/11/12 00:10:34 INFO Vacuumed table table=multipart_uploads48662026/11/12 00:10:34 INFO Vacuumed table table=closures48672026/11/12 00:10:34 INFO Vacuumed table table=objects48682026/11/12 00:10:35 INFO Received complete push request method=POST path=/48692026/11/12 00:10:35 INFO Completed push id=17 objects=22 duration=874.125µs48702026/11/12 00:10:35 INFO Received uploads request method=POST path=/48712026/11/12 00:10:35 INFO Received push request method=POST path=/48722026/11/12 00:10:35 INFO Received complete push request method=POST path=/48732026/11/12 00:10:35 INFO Completed push id=19 objects=14 duration=429.625µs48742026/11/12 00:10:35 INFO Received uploads request method=POST path=/48752026/11/12 00:10:35 INFO Received complete push request method=POST path=/48762026/11/12 00:10:35 INFO Completed push id=26 objects=23 duration=373.083µs48772026/11/12 00:10:35 INFO Received complete push request method=POST path=/48782026/11/12 00:10:35 INFO Completed push id=7 objects=15 duration=1.000209ms48792026/11/12 00:10:35 INFO Received complete upload request method=POST path=/48802026/11/12 00:10:35 INFO Completed upload id=20 objects=6 duration=267.334µs48812026/11/12 00:10:35 INFO Received uploads request method=POST path=/48822026/11/12 00:10:35 INFO Received push request method=POST path=/48832026/11/12 00:10:35 INFO Received push request method=POST path=/48842026/11/12 00:10:35 INFO Received complete upload request method=POST path=/48852026/11/12 00:10:35 INFO Completed upload id=21 objects=2 duration=295.958µs48862026/11/12 00:10:35 INFO Received uploads request method=POST path=/48872026/11/12 00:10:35 INFO Aborted multipart uploads count=0 kept=048882026/11/12 00:10:35 WARN Force mode enabled - objects will be deleted immediately without grace period48892026/11/12 00:10:35 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048902026/11/12 00:10:35 INFO Vacuumed table table=pending_closures48912026/11/12 00:10:35 INFO Vacuumed table table=pending_objects48922026/11/12 00:10:35 INFO Vacuumed table table=multipart_uploads48932026/11/12 00:10:35 INFO Vacuumed table table=closures48942026/11/12 00:10:35 INFO Vacuumed table table=objects48952026/11/12 00:10:35 INFO Received uploads request method=POST path=/48962026/11/12 00:10:35 INFO Received complete push request method=POST path=/48972026/11/12 00:10:35 INFO Received push request method=POST path=/48982026/11/12 00:10:35 INFO Completed push id=27 objects=15 duration=248.292µs48992026/11/12 00:10:35 INFO Received complete push request method=POST path=/49002026/11/12 00:10:35 INFO Completed push id=24 objects=24 duration=904.333µs49012026/11/12 00:10:35 INFO Aborted multipart uploads count=0 kept=049022026/11/12 00:10:35 WARN Force mode enabled - objects will be deleted immediately without grace period49032026/11/12 00:10:35 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049042026/11/12 00:10:35 INFO Vacuumed table table=pending_closures49052026/11/12 00:10:35 INFO Vacuumed table table=pending_objects49062026/11/12 00:10:35 INFO Vacuumed table table=multipart_uploads49072026/11/12 00:10:35 INFO Vacuumed table table=closures49082026/11/12 00:10:35 INFO Vacuumed table table=objects49092026/11/12 00:10:35 INFO Received uploads request method=POST path=/49102026/11/12 00:10:35 INFO Received uploads request method=POST path=/49112026/11/12 00:10:35 INFO Received push request method=POST path=/49122026/11/12 00:10:35 INFO Received uploads request method=POST path=/49132026/11/12 00:10:35 INFO Aborted multipart uploads count=0 kept=049142026/11/12 00:10:35 WARN Force mode enabled - objects will be deleted immediately without grace period49152026/11/12 00:10:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049162026/11/12 00:10:35 INFO Vacuumed table table=pending_closures49172026/11/12 00:10:35 INFO Vacuumed table table=pending_objects49182026/11/12 00:10:35 INFO Vacuumed table table=multipart_uploads49192026/11/12 00:10:35 INFO Vacuumed table table=closures49202026/11/12 00:10:35 INFO Vacuumed table table=objects49212026/11/12 00:10:35 INFO Received push request method=POST path=/49222026/11/12 00:10:35 INFO Received complete upload request method=POST path=/49232026/11/12 00:10:35 INFO Completed upload id=23 objects=7 duration=844.583µs49242026/11/12 00:10:35 INFO Received complete upload request method=POST path=/49252026/11/12 00:10:35 INFO Completed upload id=26 objects=5 duration=218.833µs49262026/11/12 00:10:35 INFO Received complete push request method=POST path=/49272026/11/12 00:10:35 INFO Completed push id=27 objects=3 duration=395.292µs49282026/11/12 00:10:35 INFO Received complete upload request method=POST path=/49292026/11/12 00:10:35 INFO Completed upload id=28 objects=5 duration=201.166µs49302026/11/12 00:10:35 INFO Received complete push request method=POST path=/49312026/11/12 00:10:35 INFO Completed push id=29 objects=17 duration=219.667µs49322026/11/12 00:10:35 INFO Aborted multipart uploads count=0 kept=049332026/11/12 00:10:35 INFO Received complete push request method=POST path=/49342026/11/12 00:10:35 WARN Force mode enabled - objects will be deleted immediately without grace period49352026/11/12 00:10:35 INFO Completed push id=28 objects=19 duration=257.209µs49362026/11/12 00:10:35 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049372026/11/12 00:10:35 INFO Vacuumed table table=pending_closures49382026/11/12 00:10:35 INFO Received complete push request method=POST path=/49392026/11/12 00:10:35 INFO Completed push id=19 objects=14 duration=396.75µs49402026/11/12 00:10:35 INFO Vacuumed table table=pending_objects49412026/11/12 00:10:35 INFO Vacuumed table table=multipart_uploads49422026/11/12 00:10:35 INFO Vacuumed table table=closures49432026/11/12 00:10:35 INFO Vacuumed table table=objects49442026/11/12 00:10:35 INFO Aborted multipart uploads count=0 kept=049452026/11/12 00:10:35 WARN Force mode enabled - objects will be deleted immediately without grace period49462026/11/12 00:10:35 INFO Received complete push request method=POST path=/49472026/11/12 00:10:35 INFO Completed push id=29 objects=10 duration=251.458µs49482026/11/12 00:10:35 INFO Received complete upload request method=POST path=/49492026/11/12 00:10:35 INFO Completed upload id=30 objects=6 duration=200.25µs49502026/11/12 00:10:35 INFO Received complete upload request method=POST path=/49512026/11/12 00:10:35 INFO Completed upload id=31 objects=5 duration=333.5µs49522026/11/12 00:10:35 INFO Received complete upload request method=POST path=/49532026/11/12 00:10:35 INFO Completed upload id=34 objects=3 duration=175.75µs49542026/11/12 00:10:35 INFO Aborted multipart uploads count=0 kept=049552026/11/12 00:10:35 WARN Force mode enabled - objects will be deleted immediately without grace period49562026/11/12 00:10:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049572026/11/12 00:10:35 INFO Vacuumed table table=pending_closures49582026/11/12 00:10:35 INFO Vacuumed table table=pending_objects49592026/11/12 00:10:35 INFO Vacuumed table table=multipart_uploads49602026/11/12 00:10:35 INFO Vacuumed table table=closures49612026/11/12 00:10:35 INFO Vacuumed table table=objects49622026/11/12 00:10:35 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=049632026/11/12 00:10:35 INFO Vacuumed table table=pending_closures49642026-11-12 00:10:35.741 UTC [11087] LOG: skipping analyze of "pending_objects" --- lock not available49652026/11/12 00:10:35 INFO Vacuumed table table=pending_objects49662026/11/12 00:10:35 INFO Vacuumed table table=multipart_uploads49672026/11/12 00:10:35 INFO Vacuumed table table=closures49682026/11/12 00:10:35 INFO Vacuumed table table=objects49692026/11/12 00:10:35 INFO Received push request method=POST path=/49702026/11/12 00:10:35 INFO Received uploads request method=POST path=/49712026/11/12 00:10:35 INFO Aborted multipart uploads count=0 kept=049722026/11/12 00:10:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049732026/11/12 00:10:35 INFO Vacuumed table table=pending_closures49742026/11/12 00:10:35 INFO Vacuumed table table=pending_objects49752026/11/12 00:10:35 INFO Vacuumed table table=multipart_uploads49762026/11/12 00:10:35 INFO Vacuumed table table=closures49772026/11/12 00:10:35 INFO Vacuumed table table=objects49782026/11/12 00:10:35 INFO Received complete upload request method=POST path=/49792026/11/12 00:10:35 INFO Completed upload id=13 objects=14 duration=4.241625ms49802026/11/12 00:10:35 INFO Aborted multipart uploads count=0 kept=049812026/11/12 00:10:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049822026/11/12 00:10:35 INFO Vacuumed table table=pending_closures49832026/11/12 00:10:36 INFO Vacuumed table table=pending_objects49842026/11/12 00:10:36 INFO Vacuumed table table=multipart_uploads49852026/11/12 00:10:36 INFO Vacuumed table table=closures49862026/11/12 00:10:36 INFO Vacuumed table table=objects49872026/11/12 00:10:36 INFO Received complete push request method=POST path=/49882026/11/12 00:10:36 INFO Completed push id=31 objects=6 duration=2.053ms49892026/11/12 00:10:36 INFO Received uploads request method=POST path=/49902026/11/12 00:10:36 INFO Received complete push request method=POST path=/49912026/11/12 00:10:36 INFO Completed push id=1 objects=9 duration=1.678041ms49922026/11/12 00:10:36 INFO Received complete upload request method=POST path=/49932026/11/12 00:10:36 INFO Completed upload id=2 objects=7 duration=1.368875ms49942026/11/12 00:10:36 INFO Received complete upload request method=POST path=/49952026/11/12 00:10:36 INFO Completed upload id=8 objects=6 duration=1.322209ms49962026/11/12 00:10:36 INFO Received complete upload request method=POST path=/49972026/11/12 00:10:36 INFO Completed upload id=12 objects=9 duration=1.298417ms49982026/11/12 00:10:37 INFO Received complete upload request method=POST path=/49992026/11/12 00:10:37 INFO Completed upload id=18 objects=22 duration=1.454666ms50002026/11/12 00:10:37 INFO Received complete upload request method=POST path=/50012026/11/12 00:10:37 INFO Completed upload id=21 objects=8 duration=681.125µs50022026/11/12 00:10:37 INFO Received complete upload request method=POST path=/50032026/11/12 00:10:37 INFO Completed upload id=22 objects=7 duration=751.5µs50042026/11/12 00:10:37 INFO Received complete push request method=POST path=/50052026/11/12 00:10:37 INFO Completed push id=24 objects=5 duration=484.875µs50062026/11/12 00:10:37 INFO Received complete upload request method=POST path=/50072026/11/12 00:10:37 INFO Completed upload id=25 objects=8 duration=409.875µs50082026/11/12 00:10:37 INFO Received complete push request method=POST path=/50092026/11/12 00:10:37 INFO Completed push id=26 objects=4 duration=333.125µs50102026/11/12 00:10:37 INFO Received complete push request method=POST path=/50112026/11/12 00:10:37 INFO Completed push id=30 objects=2 duration=357.333µs50122026/11/12 00:10:37 INFO Received complete push request method=POST path=/50132026/11/12 00:10:37 INFO Completed push id=32 objects=7 duration=322.583µs50142026/11/12 00:10:37 INFO Received complete upload request method=POST path=/50152026/11/12 00:10:37 INFO Completed upload id=33 objects=6 duration=205.042µs50162026/11/12 00:10:37 INFO Received complete upload request method=POST path=/50172026/11/12 00:10:37 INFO Completed upload id=34 objects=3 duration=180.542µs50182026/11/12 00:10:37 INFO Aborted multipart uploads count=0 kept=050192026/11/12 00:10:37 WARN Force mode enabled - objects will be deleted immediately without grace period50202026/11/12 00:10:37 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050212026/11/12 00:10:37 INFO Vacuumed table table=pending_closures50222026/11/12 00:10:37 INFO Vacuumed table table=pending_objects50232026/11/12 00:10:37 INFO Vacuumed table table=multipart_uploads50242026/11/12 00:10:37 INFO Vacuumed table table=closures50252026/11/12 00:10:37 INFO Vacuumed table table=objects5026--- PASS: FuzzPushGC (0.00s)5027 --- PASS: FuzzPushGC/seed#6 (8.67s)5028 --- PASS: FuzzPushGC/seed#8 (9.01s)5029 --- PASS: FuzzPushGC/seed#7 (10.52s)5030 --- PASS: FuzzPushGC/seed#15 (10.82s)5031 --- PASS: FuzzPushGC/seed#4 (11.02s)5032 --- PASS: FuzzPushGC/seed#0 (11.81s)5033 --- PASS: FuzzPushGC/seed#3 (12.17s)5034 --- PASS: FuzzPushGC/seed#9 (13.24s)5035 --- PASS: FuzzPushGC/seed#5 (13.59s)5036 --- PASS: FuzzPushGC/seed#14 (13.78s)5037 --- PASS: FuzzPushGC/seed#11 (14.09s)5038 --- PASS: FuzzPushGC/seed#2 (14.18s)5039 --- PASS: FuzzPushGC/seed#12 (14.42s)5040 --- PASS: FuzzPushGC/seed#1 (14.92s)5041 --- PASS: FuzzPushGC/seed#13 (15.09s)5042 --- PASS: FuzzPushGC/seed#10 (16.72s)5043PASS5044{"timestamp":"2026-11-12T00:10:37.281682Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:63394","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(9)"}5045{"timestamp":"2026-11-12T00:10:37.281989Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:63224","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(4)"}5046{"timestamp":"2026-11-12T00:10:37.282203Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:63766","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(4)"}50472026-11-12 00:10:37.432 UTC [10471] LOG: received smart shutdown request50482026-11-12 00:10:37.433 UTC [10471] LOG: background worker "logical replication launcher" (PID 10481) exited with exit code 150492026-11-12 00:10:38.433 UTC [10476] LOG: checkpoint complete: wrote 3488 buffers (21.3%), wrote 5 SLRU buffers; 0 WAL file(s) added, 0 removed, 33 recycled; write=25.485 s, sync=0.862 s, total=26.730 s; sync files=42369, longest=0.002 s, average=0.001 s; distance=533104 kB, estimate=533104 kB; lsn=0/27947918, redo lsn=0/22028FA850502026-11-12 00:10:38.434 UTC [10476] LOG: shutting down50512026-11-12 00:10:38.434 UTC [10476] LOG: checkpoint starting: shutdown immediate50522026-11-12 00:10:38.955 UTC [10476] LOG: checkpoint complete: wrote 8843 buffers (54.0%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 5 recycled; write=0.425 s, sync=0.092 s, total=0.521 s; sync files=2940, longest=0.022 s, average=0.001 s; distance=91258 kB, estimate=488919 kB; lsn=0/27947990, redo lsn=0/2794799050532026-11-12 00:10:38.962 UTC [10471] LOG: database system is shut down5054Running OIDC tests...5055=== RUN TestAudienceForIssuer5056=== PAUSE TestAudienceForIssuer5057=== RUN TestProviderClientGivesUpOnSilentIssuer5058=== PAUSE TestProviderClientGivesUpOnSilentIssuer5059=== RUN TestGlobMatch5060=== PAUSE TestGlobMatch5061=== RUN TestValidateToken_ValidToken5062=== PAUSE TestValidateToken_ValidToken5063=== RUN TestValidateToken_WrongAudience5064=== PAUSE TestValidateToken_WrongAudience5065=== RUN TestValidateToken_Expired5066=== PAUSE TestValidateToken_Expired5067=== RUN TestValidateToken_BoundClaimsMismatch5068=== PAUSE TestValidateToken_BoundClaimsMismatch5069=== RUN TestValidateToken_BoundSubjectMismatch5070=== PAUSE TestValidateToken_BoundSubjectMismatch5071=== RUN TestValidateToken_MultipleProviders5072=== PAUSE TestValidateToken_MultipleProviders5073=== RUN TestValidateToken_UnknownIssuerFetchesNoKeys5074=== PAUSE TestValidateToken_UnknownIssuerFetchesNoKeys5075=== RUN TestValidateToken_GoogleSchemelessIssuer5076=== PAUSE TestValidateToken_GoogleSchemelessIssuer5077=== RUN TestValidateToken_KubernetesServiceAccount5078=== PAUSE TestValidateToken_KubernetesServiceAccount5079=== RUN TestNewValidator_KubernetesRequiresCA5080=== PAUSE TestNewValidator_KubernetesRequiresCA5081=== RUN TestValidateToken_KubernetesIssuerFromOwnToken5082=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken5083=== RUN TestPins_ReservedForMatchingRule5084=== PAUSE TestPins_ReservedForMatchingRule5085=== RUN TestPins_TopLevelShorthand5086=== PAUSE TestPins_TopLevelShorthand5087=== RUN TestPins_ConfigValidation5088=== PAUSE TestPins_ConfigValidation5089=== RUN TestScopes_LegacyProviderDefaultsToWrite5090=== PAUSE TestScopes_LegacyProviderDefaultsToWrite5091=== RUN TestScopes_Rules5092=== PAUSE TestScopes_Rules5093=== RUN TestScopes_ConfigValidation5094=== PAUSE TestScopes_ConfigValidation5095=== CONT TestAudienceForIssuer5096--- PASS: TestAudienceForIssuer (0.00s)5097=== CONT TestValidateToken_WrongAudience5098=== CONT TestValidateToken_Expired5099=== CONT TestPins_TopLevelShorthand5100=== CONT TestValidateToken_GoogleSchemelessIssuer5101=== CONT TestPins_ReservedForMatchingRule5102=== CONT TestValidateToken_KubernetesIssuerFromOwnToken5103=== CONT TestNewValidator_KubernetesRequiresCA5104=== CONT TestValidateToken_KubernetesServiceAccount5105=== CONT TestGlobMatch5106=== RUN TestGlobMatch/foo_foo5107=== PAUSE TestGlobMatch/foo_foo5108=== RUN TestGlobMatch/foo_bar5109=== PAUSE TestGlobMatch/foo_bar5110=== RUN TestGlobMatch/*_5111=== CONT TestValidateToken_ValidToken5112=== CONT TestProviderClientGivesUpOnSilentIssuer5113=== CONT TestScopes_Rules5114--- PASS: TestProviderClientGivesUpOnSilentIssuer (0.00s)5115=== CONT TestScopes_ConfigValidation5116--- PASS: TestScopes_ConfigValidation (0.00s)5117=== CONT TestScopes_LegacyProviderDefaultsToWrite5118=== CONT TestValidateToken_MultipleProviders5119=== CONT TestValidateToken_UnknownIssuerFetchesNoKeys51202026/11/12 00:10:40 INFO OIDC provider initialized name=known issuer=https://known.example5121=== CONT TestPins_ConfigValidation5122=== CONT TestValidateToken_BoundSubjectMismatch5123--- PASS: TestPins_ConfigValidation (0.00s)5124=== CONT TestValidateToken_BoundClaimsMismatch5125=== PAUSE TestGlobMatch/*_5126=== RUN TestGlobMatch/*_anything5127=== PAUSE TestGlobMatch/*_anything5128=== RUN TestGlobMatch/foo*_foo5129=== PAUSE TestGlobMatch/foo*_foo5130=== RUN TestGlobMatch/foo*_foobar5131=== PAUSE TestGlobMatch/foo*_foobar5132=== RUN TestGlobMatch/foo*_bar5133=== PAUSE TestGlobMatch/foo*_bar5134=== RUN TestGlobMatch/*bar_bar5135=== PAUSE TestGlobMatch/*bar_bar5136=== RUN TestGlobMatch/*bar_foobar5137=== PAUSE TestGlobMatch/*bar_foobar5138=== RUN TestGlobMatch/*bar_foo5139=== PAUSE TestGlobMatch/*bar_foo5140=== RUN TestGlobMatch/foo*bar_foobar5141=== PAUSE TestGlobMatch/foo*bar_foobar5142=== RUN TestGlobMatch/foo*bar_foo123bar5143=== PAUSE TestGlobMatch/foo*bar_foo123bar5144=== RUN TestGlobMatch/foo*bar_foobarbaz5145=== PAUSE TestGlobMatch/foo*bar_foobarbaz5146=== RUN TestGlobMatch/*/*_foo/bar5147=== PAUSE TestGlobMatch/*/*_foo/bar5148=== RUN TestGlobMatch/*/*_foo5149=== PAUSE TestGlobMatch/*/*_foo5150=== RUN TestGlobMatch/refs/heads/*_refs/heads/main5151=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main5152=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.05153=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.05154=== RUN TestGlobMatch/refs/*/main_refs/heads/main5155=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main5156=== RUN TestGlobMatch/fo?_foo5157=== PAUSE TestGlobMatch/fo?_foo5158=== RUN TestGlobMatch/fo?_fo5159=== PAUSE TestGlobMatch/fo?_fo5160=== RUN TestGlobMatch/fo?_fooo5161=== PAUSE TestGlobMatch/fo?_fooo5162=== RUN TestGlobMatch/?oo_foo5163=== PAUSE TestGlobMatch/?oo_foo5164=== RUN TestGlobMatch/?oo_boo5165=== PAUSE TestGlobMatch/?oo_boo5166=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5167=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5168=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5169=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5170=== CONT TestGlobMatch/foo_foo5171=== CONT TestGlobMatch/*/*_foo/bar5172=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5173=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5174=== CONT TestGlobMatch/?oo_boo5175=== CONT TestGlobMatch/?oo_foo5176=== CONT TestGlobMatch/fo?_fooo5177=== CONT TestGlobMatch/fo?_fo5178=== CONT TestGlobMatch/fo?_foo5179=== CONT TestGlobMatch/refs/*/main_refs/heads/main5180=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.05181=== CONT TestGlobMatch/refs/heads/*_refs/heads/main5182=== CONT TestGlobMatch/*/*_foo5183=== CONT TestGlobMatch/*bar_bar5184=== CONT TestGlobMatch/foo*bar_foobarbaz5185=== CONT TestGlobMatch/foo*bar_foo123bar5186=== CONT TestGlobMatch/foo*bar_foobar5187=== CONT TestGlobMatch/*bar_foo5188=== CONT TestGlobMatch/*bar_foobar5189=== CONT TestGlobMatch/foo*_foo5190=== CONT TestGlobMatch/foo*_bar5191=== CONT TestGlobMatch/foo*_foobar5192=== CONT TestGlobMatch/*_5193=== CONT TestGlobMatch/*_anything5194=== CONT TestGlobMatch/foo_bar5195--- PASS: TestGlobMatch (0.02s)5196 --- PASS: TestGlobMatch/foo_foo (0.00s)5197 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)5198 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)5199 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)5200 --- PASS: TestGlobMatch/?oo_boo (0.00s)5201 --- PASS: TestGlobMatch/?oo_foo (0.00s)5202 --- PASS: TestGlobMatch/fo?_fooo (0.00s)5203 --- PASS: TestGlobMatch/fo?_fo (0.00s)5204 --- PASS: TestGlobMatch/fo?_foo (0.00s)5205 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)5206 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)5207 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)5208 --- PASS: TestGlobMatch/*/*_foo (0.00s)5209 --- PASS: TestGlobMatch/*bar_bar (0.00s)5210 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)5211 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)5212 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)5213 --- PASS: TestGlobMatch/*bar_foo (0.00s)5214 --- PASS: TestGlobMatch/*bar_foobar (0.00s)5215 --- PASS: TestGlobMatch/foo*_foo (0.00s)5216 --- PASS: TestGlobMatch/foo*_bar (0.00s)5217 --- PASS: TestGlobMatch/foo*_foobar (0.00s)5218 --- PASS: TestGlobMatch/*_ (0.00s)5219 --- PASS: TestGlobMatch/*_anything (0.00s)5220 --- PASS: TestGlobMatch/foo_bar (0.00s)52212026/11/12 00:10:40 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:63809/oidc52222026/11/12 00:10:40 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:63808/oidc5223--- PASS: TestValidateToken_ValidToken (0.04s)5224--- PASS: TestValidateToken_WrongAudience (0.04s)52252026/11/12 00:10:40 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:63812/oidc5226--- PASS: TestValidateToken_BoundSubjectMismatch (0.03s)52272026/11/12 00:10:41 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:63816/oidc5228--- PASS: TestValidateToken_Expired (0.09s)5229--- PASS: TestValidateToken_UnknownIssuerFetchesNoKeys (0.08s)52302026/11/12 00:10:41 http: TLS handshake error from 127.0.0.1:63815: read tcp 127.0.0.1:63814->127.0.0.1:63815: use of closed network connection5231--- PASS: TestNewValidator_KubernetesRequiresCA (0.10s)52322026/11/12 00:10:41 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:6381952332026/11/12 00:10:41 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:63821/oidc5234--- PASS: TestValidateToken_KubernetesServiceAccount (0.10s)5235--- PASS: TestScopes_LegacyProviderDefaultsToWrite (0.10s)52362026/11/12 00:10:41 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:63824/oidc5237--- PASS: TestPins_TopLevelShorthand (0.11s)52382026/11/12 00:10:41 INFO OIDC provider initialized name=google issuer=https://accounts.google.com5239--- PASS: TestValidateToken_GoogleSchemelessIssuer (0.12s)52402026/11/12 00:10:41 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:63829/oidc52412026/11/12 00:10:41 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC1235242--- PASS: TestScopes_Rules (0.12s)5243--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (0.13s)52442026/11/12 00:10:41 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:63833/oidc5245--- PASS: TestPins_ReservedForMatchingRule (0.15s)52462026/11/12 00:10:41 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:63835/oidc5247--- PASS: TestValidateToken_BoundClaimsMismatch (0.16s)52482026/11/12 00:10:41 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:63828/oidc52492026/11/12 00:10:41 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:63837/oidc5250--- PASS: TestValidateToken_MultipleProviders (0.17s)5251PASS5252Running hook tests...5253=== RUN TestSendPathsEmpty5254=== PAUSE TestSendPathsEmpty5255=== RUN TestQueueEnqueueAndFetch5256=== PAUSE TestQueueEnqueueAndFetch5257=== RUN TestQueueDeduplication5258=== PAUSE TestQueueDeduplication5259=== RUN TestQueueRemove5260=== PAUSE TestQueueRemove5261=== RUN TestQueueFetchBatchLimit5262=== PAUSE TestQueueFetchBatchLimit5263=== RUN TestQueueRetryMovesToBack5264=== PAUSE TestQueueRetryMovesToBack5265=== RUN TestQueueFetchRemoveLifecycle5266=== PAUSE TestQueueFetchRemoveLifecycle5267=== RUN TestQueueConcurrentWriters5268=== PAUSE TestQueueConcurrentWriters5269=== RUN TestQueueRemoveLargeClosure5270=== PAUSE TestQueueRemoveLargeClosure5271=== RUN TestServerClientIntegration5272--- PASS: TestServerClientIntegration (0.00s)5273=== RUN TestServerQueueError52742026/11/12 00:10:41 ERROR Failed to queue paths error="permission denied" count=15275--- PASS: TestServerQueueError (0.00s)5276=== RUN TestGetListenerSocketActivation5277 server_test.go:147: systemd socket activation does not exist on darwin5278--- SKIP: TestGetListenerSocketActivation (0.00s)5279=== RUN TestServerStalledClientDoesNotBlockShutdown5280=== PAUSE TestServerStalledClientDoesNotBlockShutdown5281=== RUN TestServerBacksOffOnAcceptErrors5282=== PAUSE TestServerBacksOffOnAcceptErrors5283=== RUN TestDrainIsolatesPoisonPath5284=== PAUSE TestDrainIsolatesPoisonPath5285=== RUN TestRunNotBlockedByPoisonHead5286=== PAUSE TestRunNotBlockedByPoisonHead5287=== RUN TestDrainGivesUpWhenServerDown5288=== PAUSE TestDrainGivesUpWhenServerDown5289=== RUN TestFailedPathPrunedByLaterClosure5290=== PAUSE TestFailedPathPrunedByLaterClosure5291=== RUN TestWorkerUploadsAndRemoves5292=== PAUSE TestWorkerUploadsAndRemoves5293=== RUN TestWorkerSkipsGCdPaths5294=== PAUSE TestWorkerSkipsGCdPaths5295=== RUN TestWorkerPrunesClosureDeps5296=== PAUSE TestWorkerPrunesClosureDeps5297=== RUN TestDrainTimeout5298=== PAUSE TestDrainTimeout5299=== RUN TestWorkerRemoveFailureIsNotProgress5300=== PAUSE TestWorkerRemoveFailureIsNotProgress5301=== RUN TestWorkerKeepsPathItCannotStat5302=== PAUSE TestWorkerKeepsPathItCannotStat5303=== CONT TestSendPathsEmpty5304=== CONT TestDrainIsolatesPoisonPath5305--- PASS: TestSendPathsEmpty (0.00s)5306=== CONT TestServerBacksOffOnAcceptErrors5307=== CONT TestServerStalledClientDoesNotBlockShutdown53082000/01/01 00:00:00 ERROR Accept failed error="too many open files"5309=== CONT TestQueueRemoveLargeClosure53102000/01/01 00:00:00 ERROR Accept failed error="too many open files"53112000/01/01 00:00:00 ERROR Accept failed error="too many open files"53122000/01/01 00:00:00 ERROR Accept failed error="too many open files"53132000/01/01 00:00:00 ERROR Accept failed error="too many open files"53142000/01/01 00:00:00 ERROR Accept failed error="too many open files"5315=== CONT TestQueueConcurrentWriters53162000/01/01 00:00:00 ERROR Accept failed error="too many open files"53172000/01/01 00:00:00 ERROR Accept failed error="too many open files"5318--- PASS: TestServerBacksOffOnAcceptErrors (0.00s)5319=== CONT TestQueueFetchRemoveLifecycle5320=== CONT TestQueueRetryMovesToBack5321=== CONT TestQueueFetchBatchLimit5322=== CONT TestQueueRemove5323=== CONT TestQueueDeduplication5324=== CONT TestQueueEnqueueAndFetch5325=== CONT TestWorkerRemoveFailureIsNotProgress5326=== CONT TestWorkerKeepsPathItCannotStat5327=== CONT TestFailedPathPrunedByLaterClosure5328=== CONT TestWorkerSkipsGCdPaths5329=== CONT TestWorkerUploadsAndRemoves5330=== CONT TestDrainTimeout5331=== CONT TestWorkerPrunesClosureDeps5332=== CONT TestDrainGivesUpWhenServerDown5333=== CONT TestRunNotBlockedByPoisonHead53342026/11/12 00:10:41 INFO Upload queue status pending=253352026/11/12 00:10:41 INFO Uploading batch count=253362026/11/12 00:10:41 INFO Upload queue status pending=253372026/11/12 00:10:41 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestWorkerSkipsGCdPaths157240446/002/nonexistent53382026/11/12 00:10:41 INFO Uploading batch count=153392026/11/12 00:10:41 INFO Uploading batch count=453402026/11/12 00:10:41 ERROR Upload failed error="upload failed" count=453412026/11/12 00:10:41 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestDrainIsolatesPoisonPath2031048725/002/bbb5342--- PASS: TestQueueEnqueueAndFetch (0.02s)53432000/01/01 00:00:00 INFO Uploading batch count=253442000/01/01 00:00:00 ERROR Upload failed error="context deadline exceeded" count=253452000/01/01 00:00:00 ERROR Drain finished with paths left in queue remaining=453462026/11/12 00:10:41 INFO Uploading batch count=153472026/11/12 00:10:41 ERROR Upload failed error="upload failed" count=15348--- PASS: TestQueueDeduplication (0.02s)53492026/11/12 00:10:41 INFO Uploading batch count=153502026/11/12 00:10:41 ERROR Upload failed error="upload failed" count=153512026/11/12 00:10:41 WARN Cannot stat store path, will retry later path=/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestWorkerKeepsPathItCannotStat1481068437/002/locked/aaa error="stat /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestWorkerKeepsPathItCannotStat1481068437/002/locked/aaa: permission denied"53522026/11/12 00:10:41 INFO Uploading batch count=253532026/11/12 00:10:41 ERROR Upload failed error="upload failed" count=253542026/11/12 00:10:41 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestDrainGivesUpWhenServerDown350823666/002/a5355--- PASS: TestQueueFetchBatchLimit (0.02s)53562026/11/12 00:10:41 WARN Cannot stat store path, will retry later path=/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestWorkerKeepsPathItCannotStat1481068437/002/locked/aaa error="stat /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestWorkerKeepsPathItCannotStat1481068437/002/locked/aaa: permission denied"53572026/11/12 00:10:41 INFO Uploading batch count=153582026/11/12 00:10:41 INFO Uploading batch count=153592026/11/12 00:10:41 INFO Upload queue status pending=25360--- PASS: TestDrainTimeout (0.02s)53612026/11/12 00:10:41 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestDrainGivesUpWhenServerDown350823666/002/b53622026/11/12 00:10:41 INFO Uploading batch count=153632026/11/12 00:10:41 ERROR Upload failed error="upload failed" count=153642026/11/12 00:10:41 INFO Uploading batch count=153652026/11/12 00:10:41 INFO Upload queue status pending=35366--- PASS: TestQueueRemove (0.02s)53672026/11/12 00:10:41 INFO Uploading batch count=253682026/11/12 00:10:41 ERROR Upload failed error="upload failed" count=253692026/11/12 00:10:41 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestDrainGivesUpWhenServerDown350823666/002/c53702026/11/12 00:10:41 INFO Uploading batch count=153712026/11/12 00:10:41 ERROR Upload failed error="upload failed" count=153722026/11/12 00:10:41 INFO Uploading batch count=153732026/11/12 00:10:41 ERROR Upload failed error="upload failed" count=153742026/11/12 00:10:41 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestWorkerRemoveFailureIsNotProgress3417904394/002/nonexistent53752026/11/12 00:10:41 WARN Cannot stat store path, will retry later path=/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestWorkerKeepsPathItCannotStat1481068437/002/locked/aaa error="stat /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestWorkerKeepsPathItCannotStat1481068437/002/locked/aaa: permission denied"53762026/11/12 00:10:41 ERROR Drain finished with paths left in queue remaining=153772026/11/12 00:10:41 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=153782026/11/12 00:10:41 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestDrainGivesUpWhenServerDown350823666/002/d5379--- PASS: TestQueueRetryMovesToBack (0.02s)53802026/11/12 00:10:41 ERROR Drain finished with paths left in queue remaining=153812026/11/12 00:10:41 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestWorkerRemoveFailureIsNotProgress3417904394/002/nonexistent53822026/11/12 00:10:41 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=153832026/11/12 00:10:41 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestWorkerRemoveFailureIsNotProgress3417904394/002/nonexistent53842026/11/12 00:10:41 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=153852026/11/12 00:10:41 ERROR Drain finished with paths left in queue remaining=153862026/11/12 00:10:41 INFO Uploading batch count=253872026/11/12 00:10:41 ERROR Upload failed error="upload failed" count=253882026/11/12 00:10:41 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestDrainGivesUpWhenServerDown350823666/002/e5389--- PASS: TestQueueFetchRemoveLifecycle (0.02s)5390--- PASS: TestFailedPathPrunedByLaterClosure (0.02s)53912026/11/12 00:10:41 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestDrainGivesUpWhenServerDown350823666/002/f53922026/11/12 00:10:41 ERROR Drain finished with paths left in queue remaining=105393--- PASS: TestDrainIsolatesPoisonPath (0.02s)5394--- PASS: TestWorkerRemoveFailureIsNotProgress (0.02s)5395--- PASS: TestWorkerKeepsPathItCannotStat (0.02s)5396--- PASS: TestDrainGivesUpWhenServerDown (0.02s)5397--- PASS: TestWorkerSkipsGCdPaths (0.04s)5398--- PASS: TestWorkerUploadsAndRemoves (0.04s)5399--- PASS: TestQueueRemoveLargeClosure (0.05s)5400--- PASS: TestWorkerPrunesClosureDeps (0.05s)5401--- PASS: TestQueueConcurrentWriters (0.07s)54022026/11/12 00:10:41 ERROR Failed to decode request error="read unix /private/var/tb/7/scratch/b827b2ae36b0e69e/build/hook2246717395/test.sock->: i/o timeout"54032026/11/12 00:10:41 ERROR Failed to write response error="write unix /private/var/tb/7/scratch/b827b2ae36b0e69e/build/hook2246717395/test.sock->: i/o timeout"5404--- PASS: TestServerStalledClientDoesNotBlockShutdown (0.20s)54052026/11/12 00:10:42 INFO Uploading batch count=154062026/11/12 00:10:42 INFO Uploading batch count=154072026/11/12 00:10:42 INFO Uploading batch count=154082026/11/12 00:10:42 ERROR Upload failed error="upload failed" count=154092026/11/12 00:10:42 INFO Uploading batch count=154102026/11/12 00:10:42 ERROR Upload failed error="upload failed" count=154112026/11/12 00:10:42 INFO Uploading batch count=154122026/11/12 00:10:42 ERROR Upload failed error="upload failed" count=154132026/11/12 00:10:42 INFO Uploading batch count=154142026/11/12 00:10:42 ERROR Upload failed error="upload failed" count=154152026/11/12 00:10:42 ERROR Drain finished with paths left in queue remaining=15416--- PASS: TestRunNotBlockedByPoisonHead (1.04s)5417PASS5418Running signing tests...5419=== RUN TestExternalSigner5420=== PAUSE TestExternalSigner5421=== RUN TestExternalSignerWrongPublicKey5422=== PAUSE TestExternalSignerWrongPublicKey5423=== RUN TestExternalSignerErrors5424=== PAUSE TestExternalSignerErrors5425=== RUN TestExternalSignerRestartsAfterCrash5426=== PAUSE TestExternalSignerRestartsAfterCrash5427=== RUN TestExternalSignerProgramErrorKeepsProcess5428=== PAUSE TestExternalSignerProgramErrorKeepsProcess5429=== RUN TestExternalSignerClose5430=== PAUSE TestExternalSignerClose5431=== RUN TestExternalSignerRejectsInvalidPublicKey5432=== PAUSE TestExternalSignerRejectsInvalidPublicKey5433=== RUN TestExternalSignerCancellation5434=== PAUSE TestExternalSignerCancellation5435=== RUN TestGenerateFingerprint5436=== PAUSE TestGenerateFingerprint5437=== RUN TestParseSigningKey5438=== PAUSE TestParseSigningKey5439=== RUN TestSign5440=== PAUSE TestSign5441=== RUN TestSignBatch5442=== PAUSE TestSignBatch5443=== RUN TestNilKeyPublicKey5444=== PAUSE TestNilKeyPublicKey5445=== RUN TestSignErrors5446=== PAUSE TestSignErrors5447=== CONT TestExternalSigner5448=== CONT TestExternalSignerCancellation5449=== CONT TestSignBatch5450=== CONT TestParseSigningKey5451=== RUN TestParseSigningKey/valid_32-byte_key5452=== CONT TestSignErrors5453=== CONT TestGenerateFingerprint5454=== PAUSE TestParseSigningKey/valid_32-byte_key5455=== CONT TestExternalSignerWrongPublicKey5456=== RUN TestParseSigningKey/valid_32-byte_key_with_different_name5457=== PAUSE TestParseSigningKey/valid_32-byte_key_with_different_name5458=== RUN TestParseSigningKey/no_colon5459=== PAUSE TestParseSigningKey/no_colon5460=== RUN TestParseSigningKey/empty_name5461=== PAUSE TestParseSigningKey/empty_name5462=== RUN TestParseSigningKey/invalid_base645463=== PAUSE TestParseSigningKey/invalid_base645464=== RUN TestParseSigningKey/wrong_length5465=== PAUSE TestParseSigningKey/wrong_length5466=== CONT TestParseSigningKey/valid_32-byte_key5467=== CONT TestSign5468=== RUN TestGenerateFingerprint/basic_with_references5469=== PAUSE TestGenerateFingerprint/basic_with_references5470=== RUN TestGenerateFingerprint/no_references5471=== PAUSE TestGenerateFingerprint/no_references5472=== RUN TestGenerateFingerprint/unsorted_references_get_sorted5473=== PAUSE TestGenerateFingerprint/unsorted_references_get_sorted5474=== RUN TestGenerateFingerprint/invalid_nar_hash_prefix5475=== PAUSE TestGenerateFingerprint/invalid_nar_hash_prefix5476=== RUN TestGenerateFingerprint/invalid_nar_hash_length5477=== PAUSE TestGenerateFingerprint/invalid_nar_hash_length5478=== RUN TestGenerateFingerprint/invalid_store_path_prefix5479=== PAUSE TestGenerateFingerprint/invalid_store_path_prefix5480=== RUN TestGenerateFingerprint/invalid_reference_prefix5481=== PAUSE TestGenerateFingerprint/invalid_reference_prefix5482=== CONT TestGenerateFingerprint/basic_with_references5483=== CONT TestParseSigningKey/wrong_length5484=== CONT TestGenerateFingerprint/invalid_store_path_prefix5485=== CONT TestGenerateFingerprint/invalid_nar_hash_length5486=== CONT TestGenerateFingerprint/invalid_nar_hash_prefix5487=== CONT TestGenerateFingerprint/unsorted_references_get_sorted5488=== CONT TestGenerateFingerprint/no_references5489=== CONT TestGenerateFingerprint/invalid_reference_prefix5490--- PASS: TestGenerateFingerprint (0.00s)5491 --- PASS: TestGenerateFingerprint/basic_with_references (0.00s)5492 --- PASS: TestGenerateFingerprint/invalid_store_path_prefix (0.00s)5493 --- PASS: TestGenerateFingerprint/invalid_nar_hash_length (0.00s)5494 --- PASS: TestGenerateFingerprint/invalid_nar_hash_prefix (0.00s)5495 --- PASS: TestGenerateFingerprint/unsorted_references_get_sorted (0.00s)5496 --- PASS: TestGenerateFingerprint/no_references (0.00s)5497 --- PASS: TestGenerateFingerprint/invalid_reference_prefix (0.00s)5498=== CONT TestParseSigningKey/no_colon5499=== CONT TestExternalSignerRejectsInvalidPublicKey5500=== RUN TestExternalSignerRejectsInvalidPublicKey/#005501=== PAUSE TestExternalSignerRejectsInvalidPublicKey/#005502=== RUN TestExternalSignerRejectsInvalidPublicKey/no-colon5503=== PAUSE TestExternalSignerRejectsInvalidPublicKey/no-colon5504=== RUN TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5505=== PAUSE TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5506=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5507=== CONT TestExternalSignerClose5508=== CONT TestNilKeyPublicKey5509--- PASS: TestNilKeyPublicKey (0.00s)5510=== CONT TestExternalSignerErrors5511=== RUN TestExternalSignerErrors/exit_at_startup5512=== CONT TestExternalSignerRestartsAfterCrash5513=== PAUSE TestExternalSignerErrors/exit_at_startup5514=== RUN TestExternalSignerErrors/wrong_shape5515=== PAUSE TestExternalSignerErrors/wrong_shape5516=== CONT TestParseSigningKey/empty_name5517=== RUN TestExternalSignerErrors/wrong_id5518=== PAUSE TestExternalSignerErrors/wrong_id5519=== RUN TestExternalSignerErrors/missing_signature_name5520=== PAUSE TestExternalSignerErrors/missing_signature_name5521=== RUN TestExternalSignerErrors/null5522=== CONT TestParseSigningKey/valid_32-byte_key_with_different_name5523=== CONT TestExternalSignerProgramErrorKeepsProcess5524=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5525=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5526=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5527=== CONT TestExternalSignerRejectsInvalidPublicKey/#005528=== CONT TestParseSigningKey/invalid_base645529=== CONT TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5530=== PAUSE TestExternalSignerErrors/null5531=== RUN TestExternalSignerErrors/exit_on_request5532=== CONT TestExternalSignerRejectsInvalidPublicKey/no-colon5533=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5534=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5535--- PASS: TestExternalSignerRejectsInvalidPublicKey (0.00s)5536 --- PASS: TestExternalSignerRejectsInvalidPublicKey/#00 (0.00s)5537 --- PASS: TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= (0.00s)5538 --- PASS: TestExternalSignerRejectsInvalidPublicKey/no-colon (0.00s)5539 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:invalid! (0.00s)5540 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:YQ== (0.00s)5541=== PAUSE TestExternalSignerErrors/exit_on_request5542=== RUN TestExternalSignerErrors/invalid_JSON5543=== PAUSE TestExternalSignerErrors/invalid_JSON5544=== RUN TestExternalSignerErrors/missing_signatures5545=== PAUSE TestExternalSignerErrors/missing_signatures5546=== RUN TestExternalSignerErrors/too_few_signatures5547=== PAUSE TestExternalSignerErrors/too_few_signatures5548=== RUN TestExternalSignerErrors/unrequested_key5549=== PAUSE TestExternalSignerErrors/unrequested_key5550=== RUN TestExternalSignerErrors/wrong_signature_name5551=== PAUSE TestExternalSignerErrors/wrong_signature_name5552=== RUN TestExternalSignerErrors/invalid_signature_base645553=== PAUSE TestExternalSignerErrors/invalid_signature_base645554=== RUN TestExternalSignerErrors/wrong_signature_length5555=== PAUSE TestExternalSignerErrors/wrong_signature_length5556=== RUN TestExternalSignerErrors/null_signatures5557=== PAUSE TestExternalSignerErrors/null_signatures5558=== RUN TestExternalSignerErrors/wrong_signatures_shape5559=== PAUSE TestExternalSignerErrors/wrong_signatures_shape5560=== RUN TestExternalSignerErrors/program_error5561=== PAUSE TestExternalSignerErrors/program_error5562=== CONT TestExternalSignerErrors/exit_at_startup5563=== CONT TestExternalSignerErrors/program_error5564=== CONT TestExternalSignerErrors/too_few_signatures5565=== CONT TestExternalSignerErrors/null5566=== CONT TestExternalSignerErrors/missing_signatures5567=== CONT TestExternalSignerErrors/invalid_JSON5568=== CONT TestExternalSignerErrors/exit_on_request5569=== CONT TestExternalSignerErrors/wrong_signature_length5570=== CONT TestExternalSignerErrors/wrong_signatures_shape5571=== RUN TestSignErrors/nil_key5572=== PAUSE TestSignErrors/nil_key5573=== RUN TestSignErrors/nil_narinfo5574=== PAUSE TestSignErrors/nil_narinfo5575=== RUN TestSignErrors/invalid_narinfo5576=== PAUSE TestSignErrors/invalid_narinfo5577=== RUN TestSignErrors/canceled5578=== PAUSE TestSignErrors/canceled5579=== CONT TestExternalSignerErrors/wrong_shape5580--- PASS: TestSign (0.00s)5581=== CONT TestExternalSignerErrors/unrequested_key5582--- PASS: TestSignBatch (0.00s)5583=== CONT TestSignErrors/nil_key5584=== CONT TestExternalSignerErrors/null_signatures5585=== CONT TestExternalSignerErrors/wrong_id5586=== CONT TestExternalSignerErrors/missing_signature_name5587=== CONT TestExternalSignerErrors/wrong_signature_name5588=== CONT TestExternalSignerErrors/invalid_signature_base645589--- PASS: TestParseSigningKey (0.00s)5590 --- PASS: TestParseSigningKey/wrong_length (0.00s)5591 --- PASS: TestParseSigningKey/no_colon (0.00s)5592 --- PASS: TestParseSigningKey/empty_name (0.00s)5593 --- PASS: TestParseSigningKey/invalid_base64 (0.00s)5594 --- PASS: TestParseSigningKey/valid_32-byte_key (0.00s)5595 --- PASS: TestParseSigningKey/valid_32-byte_key_with_different_name (0.00s)5596=== CONT TestSignErrors/invalid_narinfo5597=== CONT TestSignErrors/canceled5598=== CONT TestSignErrors/nil_narinfo5599--- PASS: TestSignErrors (0.00s)5600 --- PASS: TestSignErrors/invalid_narinfo (0.00s)5601 --- PASS: TestSignErrors/canceled (0.00s)5602 --- PASS: TestSignErrors/nil_narinfo (0.00s)5603 --- PASS: TestSignErrors/nil_key (0.00s)5604--- PASS: TestExternalSignerWrongPublicKey (0.20s)5605--- PASS: TestExternalSignerProgramErrorKeepsProcess (1.51s)5606--- PASS: TestExternalSigner (1.91s)5607--- PASS: TestExternalSignerCancellation (2.47s)5608--- PASS: TestExternalSignerClose (2.83s)5609--- PASS: TestExternalSignerRestartsAfterCrash (3.41s)5610/private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestExternalSignerRestartsAfterCrash143486297/001/sign program: line 7: /private/var/tb/7/scratch/b827b2ae36b0e69e/build/TestExternalSignerRestartsAfterCrash143486297/001/sign program.starts: No such file or directory5611--- PASS: TestExternalSignerErrors (0.00s)5612 --- PASS: TestExternalSignerErrors/wrong_signatures_shape (0.38s)5613 --- PASS: TestExternalSignerErrors/exit_on_request (0.58s)5614 --- PASS: TestExternalSignerErrors/wrong_signature_length (0.76s)5615 --- PASS: TestExternalSignerErrors/program_error (0.95s)5616 --- PASS: TestExternalSignerErrors/too_few_signatures (1.14s)5617 --- PASS: TestExternalSignerErrors/missing_signature_name (1.33s)5618 --- PASS: TestExternalSignerErrors/missing_signatures (1.70s)5619 --- PASS: TestExternalSignerErrors/wrong_shape (2.08s)5620 --- PASS: TestExternalSignerErrors/unrequested_key (2.26s)5621 --- PASS: TestExternalSignerErrors/null (2.64s)5622 --- PASS: TestExternalSignerErrors/null_signatures (3.01s)5623 --- PASS: TestExternalSignerErrors/wrong_signature_name (3.20s)5624 --- PASS: TestExternalSignerErrors/wrong_id (3.59s)5625 --- PASS: TestExternalSignerErrors/invalid_JSON (3.78s)5626 --- PASS: TestExternalSignerErrors/exit_at_startup (3.97s)5627 --- PASS: TestExternalSignerErrors/invalid_signature_base64 (4.16s)5628PASS5629Running OpenBao signer tests...5630=== RUN TestSign5631=== PAUSE TestSign5632=== RUN TestSignErrors5633=== PAUSE TestSignErrors5634=== RUN TestServe5635=== PAUSE TestServe5636=== RUN TestServeRejectsGarbage5637=== PAUSE TestServeRejectsGarbage5638=== CONT TestSign5639=== CONT TestServeRejectsGarbage5640=== CONT TestSignErrors5641=== CONT TestServe5642=== RUN TestSignErrors/http_error5643--- PASS: TestServeRejectsGarbage (0.00s)5644=== PAUSE TestSignErrors/http_error5645=== RUN TestSignErrors/invalid_json5646=== PAUSE TestSignErrors/invalid_json5647=== RUN TestSignErrors/no_prefix5648=== PAUSE TestSignErrors/no_prefix5649=== RUN TestSignErrors/invalid_base645650=== PAUSE TestSignErrors/invalid_base645651=== CONT TestSignErrors/http_error5652=== CONT TestSignErrors/no_prefix5653=== CONT TestSignErrors/invalid_json5654=== CONT TestSignErrors/invalid_base645655--- PASS: TestServe (0.00s)5656--- PASS: TestSign (0.00s)5657--- PASS: TestSignErrors (0.00s)5658 --- PASS: TestSignErrors/invalid_base64 (0.00s)5659 --- PASS: TestSignErrors/http_error (0.00s)5660 --- PASS: TestSignErrors/no_prefix (0.00s)5661 --- PASS: TestSignErrors/invalid_json (0.00s)5662PASS