niks3-go-unit-tests log

failed `niks3-go-unit-tests` `checks.aarch64-linux.go-unit-tests` · build #487 · raw

1tribuchet: building on eliza2Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestUploadBuildLog_FileBodyReplayedOnRetry6=== PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry7=== RUN TestRegisterUploadedObjectReusesConnections8=== PAUSE TestRegisterUploadedObjectReusesConnections9=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica10=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica11=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun12=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun13=== RUN TestCaseHackSuffix14=== PAUSE TestCaseHackSuffix15=== RUN TestFilterOversizedClosures16=== PAUSE TestFilterOversizedClosures17=== RUN TestUploadMultipart_PartsInParallel18=== PAUSE TestUploadMultipart_PartsInParallel19=== RUN TestUploadMultipart_ReturnsAfterDroppedBody20=== PAUSE TestUploadMultipart_ReturnsAfterDroppedBody21=== RUN TestPartSizeForNAR22=== PAUSE TestPartSizeForNAR23=== RUN TestDumpPathCaseHackMatchesNix24--- PASS: TestDumpPathCaseHackMatchesNix (8.95s)25=== RUN TestDumpPathCaseHackCollision26--- PASS: TestDumpPathCaseHackCollision (0.00s)27=== RUN TestDumpPathMatchesNix28=== PAUSE TestDumpPathMatchesNix29=== RUN TestDumpPathSingleFile30=== PAUSE TestDumpPathSingleFile31=== RUN TestDumpPathWriterError32=== PAUSE TestDumpPathWriterError33=== RUN TestEncodeNixBase3234=== PAUSE TestEncodeNixBase3235=== RUN TestEncodeNixBase32WithRealHash36=== PAUSE TestEncodeNixBase32WithRealHash37=== RUN TestConvertHashToNix3238=== PAUSE TestConvertHashToNix3239=== RUN TestGetStorePathHash40=== PAUSE TestGetStorePathHash41=== RUN TestPathInfoHashCompatibility42=== PAUSE TestPathInfoHashCompatibility43=== RUN TestParsePathInfoJSON44=== PAUSE TestParsePathInfoJSON45=== RUN TestParsePathInfoJSONMultiplePaths46=== PAUSE TestParsePathInfoJSONMultiplePaths47=== RUN TestPathInfoCACompatibility48=== PAUSE TestPathInfoCACompatibility49=== RUN TestRateLimiterFeedback50=== PAUSE TestRateLimiterFeedback51=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess52=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess53=== RUN TestRegisterUploadedObject_BoundedAgainstSilentServer54=== PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer55=== RUN TestRegisterUploadedObject_OutlivesPush56=== PAUSE TestRegisterUploadedObject_OutlivesPush57=== RUN TestResolveStorePath58=== PAUSE TestResolveStorePath59=== RUN TestDoWithRetry_BodyReplayedViaGetBody60=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody61=== RUN TestRetryTimeoutOutlastsMaxRetries62=== PAUSE TestRetryTimeoutOutlastsMaxRetries63=== RUN TestDoWithRetry_FinalResponseBodyReadable64=== PAUSE TestDoWithRetry_FinalResponseBodyReadable65=== RUN TestShellSplit66=== PAUSE TestShellSplit67=== RUN TestShellSplitErrors68=== PAUSE TestShellSplitErrors69=== RUN TestStreamPushReportsEveryPath70=== PAUSE TestStreamPushReportsEveryPath71=== RUN TestStreamPushBatchesUnderLoad72=== PAUSE TestStreamPushBatchesUnderLoad73=== RUN TestStreamPushIsolatesFailures74=== PAUSE TestStreamPushIsolatesFailures75=== RUN TestStreamPushGivesUpOnDeadServer76=== PAUSE TestStreamPushGivesUpOnDeadServer77=== RUN TestStreamPushRequestLine78=== PAUSE TestStreamPushRequestLine79=== RUN TestStreamPushReportsSignatures80=== PAUSE TestStreamPushReportsSignatures81=== RUN TestStreamPushReportsSkippedPaths82=== PAUSE TestStreamPushReportsSkippedPaths83=== RUN TestClientSignaturesByStorePath84=== PAUSE TestClientSignaturesByStorePath85=== RUN TestStreamPushStopsWhenCanceled86=== PAUSE TestStreamPushStopsWhenCanceled87=== RUN TestSetClientTLS88=== PAUSE TestSetClientTLS89=== RUN TestSetClientTLSDoesNotMutateDefaultTransport90=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport91=== RUN TestSetClientTLSErrors92=== PAUSE TestSetClientTLSErrors93=== RUN TestStaticToken94=== PAUSE TestStaticToken95=== RUN TestFileTokenReadsAndCaches96=== PAUSE TestFileTokenReadsAndCaches97=== RUN TestFileTokenMissing98=== PAUSE TestFileTokenMissing99=== RUN TestFileTokenEmpty100=== PAUSE TestFileTokenEmpty101=== RUN TestScriptTokenNoExpiryRerunsEveryCall102=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall103=== RUN TestScriptTokenCachesUntilRefresh104=== PAUSE TestScriptTokenCachesUntilRefresh105=== RUN TestScriptTokenEmptyToken106=== PAUSE TestScriptTokenEmptyToken107=== RUN TestScriptTokenBadJSON108=== PAUSE TestScriptTokenBadJSON109=== RUN TestScriptTokenScriptFails110=== PAUSE TestScriptTokenScriptFails111=== RUN TestScriptTokenEmptyCommand112=== PAUSE TestScriptTokenEmptyCommand113=== CONT TestStreamPushGivesUpOnDeadServer114=== CONT TestScriptTokenEmptyToken115=== CONT TestFileTokenReadsAndCaches116=== CONT TestStaticToken117--- PASS: TestStaticToken (0.00s)118=== CONT TestSetClientTLSErrors119=== CONT TestScriptTokenNoExpiryRerunsEveryCall120--- PASS: TestFileTokenReadsAndCaches (0.00s)121=== CONT TestFileTokenEmpty122--- PASS: TestFileTokenEmpty (0.00s)123=== CONT TestFileTokenMissing124--- PASS: TestFileTokenMissing (0.00s)125=== CONT TestScriptTokenScriptFails126=== CONT TestDoServerRequestAttachesToken127=== CONT TestStreamPushIsolatesFailures128=== RUN TestSetClientTLSErrors/missing_cert_file129=== PAUSE TestSetClientTLSErrors/missing_cert_file130=== RUN TestSetClientTLSErrors/missing_key_file131=== PAUSE TestSetClientTLSErrors/missing_key_file132=== CONT TestStreamPushBatchesUnderLoad133=== CONT TestScriptTokenEmptyCommand134=== CONT TestSetClientTLS135--- PASS: TestScriptTokenScriptFails (1.02s)136=== CONT TestStreamPushStopsWhenCanceled137=== CONT TestShellSplit138=== CONT TestRegisterUploadedObject_BoundedAgainstSilentServer139=== CONT TestStreamPushReportsSkippedPaths1402026/11/11 23:09:49 ERROR Upload failed error="connection refused" count=20141=== CONT TestStreamPushReportsSignatures142=== CONT TestStreamPushRequestLine1432026/11/11 23:09:49 ERROR Server seems unavailable, giving up on batch untried=17144=== CONT TestScriptTokenBadJSON145=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun146=== CONT TestScriptTokenCachesUntilRefresh147=== RUN TestSetClientTLSErrors/missing_ca_file148=== PAUSE TestSetClientTLSErrors/missing_ca_file149=== RUN TestSetClientTLSErrors/invalid_ca_file150=== PAUSE TestSetClientTLSErrors/invalid_ca_file151=== CONT TestEncodeNixBase32WithRealHash152=== CONT TestDumpPathWriterError153=== CONT TestDumpPathSingleFile1542026/11/11 23:09:49 ERROR Upload failed error=boom count=1155=== CONT TestEncodeNixBase32156=== RUN TestEncodeNixBase32/test_string_hash157=== PAUSE TestEncodeNixBase32/test_string_hash158=== RUN TestEncodeNixBase32/empty_input159=== PAUSE TestEncodeNixBase32/empty_input1602026/11/11 23:09:49 ERROR Upload failed error=boom count=1161=== CONT TestPartSizeForNAR162=== RUN TestPartSizeForNAR/zero_stays_at_minimum163=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum164=== RUN TestPartSizeForNAR/small_stays_at_minimum165=== PAUSE TestPartSizeForNAR/small_stays_at_minimum166=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum167=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum168=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts169=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts170=== RUN TestPartSizeForNAR/1_TiB171=== PAUSE TestPartSizeForNAR/1_TiB172=== RUN TestPartSizeForNAR/5_TiB_S3_max_object173=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object174=== RUN TestPartSizeForNAR/capped_at_5_GiB175=== PAUSE TestPartSizeForNAR/capped_at_5_GiB176=== CONT TestStreamPushReportsEveryPath177=== CONT TestDumpPathMatchesNix178=== CONT TestUploadMultipart_ReturnsAfterDroppedBody179=== CONT TestShellSplitErrors180=== CONT TestUploadMultipart_PartsInParallel181=== CONT TestFilterOversizedClosures182=== RUN TestFilterOversizedClosures/no_limit_keeps_everything183=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything184=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped185=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped186=== RUN TestFilterOversizedClosures/all_closures_skipped187=== PAUSE TestFilterOversizedClosures/all_closures_skipped188--- PASS: TestScriptTokenEmptyToken (1.02s)189--- PASS: TestStreamPushStopsWhenCanceled (0.00s)190--- PASS: TestScriptTokenEmptyCommand (0.00s)191--- PASS: TestShellSplit (0.00s)192--- PASS: TestStreamPushReportsSkippedPaths (0.00s)193--- PASS: TestEncodeNixBase32WithRealHash (0.00s)194--- PASS: TestStreamPushGivesUpOnDeadServer (1.02s)195--- PASS: TestStreamPushReportsSignatures (0.00s)196--- PASS: TestShellSplitErrors (0.00s)197--- PASS: TestStreamPushReportsEveryPath (0.00s)198=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica199--- PASS: TestScriptTokenBadJSON (0.01s)200=== CONT TestDoWithRetry_FinalResponseBodyReadable201=== CONT TestRetryTimeoutOutlastsMaxRetries202=== CONT TestDoWithRetry_BodyReplayedViaGetBody203=== CONT TestSetClientTLSDoesNotMutateDefaultTransport204=== CONT TestResolveStorePath205--- PASS: TestResolveStorePath (0.00s)206=== CONT TestRegisterUploadedObjectReusesConnections207=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=false208=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=false209=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=true210=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=true211=== CONT TestRegisterUploadedObject_OutlivesPush212=== CONT TestUploadBuildLog_FileBodyReplayedOnRetry213=== CONT TestClientSignaturesByStorePath214--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (1.03s)215--- PASS: TestClientSignaturesByStorePath (0.00s)216=== CONT TestRateLimiterFeedback217=== RUN TestRateLimiterFeedback/429_enables_limiter218=== PAUSE TestRateLimiterFeedback/429_enables_limiter219=== RUN TestRateLimiterFeedback/503_enables_limiter220=== PAUSE TestRateLimiterFeedback/503_enables_limiter221=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter222=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter223=== CONT TestCaseHackSuffix224=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess2252026/11/11 23:09:49 WARN Rate limiter enabled after throttle name=server-test rate=5226=== CONT TestParsePathInfoJSON227=== RUN TestParsePathInfoJSON/Nix_format228=== PAUSE TestParsePathInfoJSON/Nix_format229=== RUN TestParsePathInfoJSON/Lix_format230=== PAUSE TestParsePathInfoJSON/Lix_format231=== RUN TestParsePathInfoJSON/empty_input232=== PAUSE TestParsePathInfoJSON/empty_input233=== RUN TestParsePathInfoJSON/whitespace_only234=== PAUSE TestParsePathInfoJSON/whitespace_only235=== RUN TestParsePathInfoJSON/invalid_JSON236=== PAUSE TestParsePathInfoJSON/invalid_JSON237=== CONT TestPathInfoCACompatibility238=== RUN TestPathInfoCACompatibility/null_ca_field239=== PAUSE TestPathInfoCACompatibility/null_ca_field240=== RUN TestPathInfoCACompatibility/old_string_format_-_text241=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text242=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive243=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive244=== RUN TestPathInfoCACompatibility/new_structured_format_-_text245=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text246=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method247=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter248=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method249=== CONT TestParsePathInfoJSONMultiplePaths250=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter251=== CONT TestGetStorePathHash252=== RUN TestGetStorePathHash/valid_store_path253=== PAUSE TestGetStorePathHash/valid_store_path254=== RUN TestGetStorePathHash/basename_without_hyphen_should_error255=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error256=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error257=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error258=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error259=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error260=== CONT TestPathInfoHashCompatibility261=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)262=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)263=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon264=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon265=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI266=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI267=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512268=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512269=== CONT TestConvertHashToNix32270=== RUN TestConvertHashToNix32/SRI_format_to_Nix322712026/11/11 23:09:52 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:37187/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:37187/api/uploads/complete272=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix322732026/11/11 23:09:52 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"274=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths275=== RUN TestConvertHashToNix32/already_Nix32_format276=== PAUSE TestConvertHashToNix32/already_Nix32_format277=== RUN TestConvertHashToNix32/invalid_format278=== PAUSE TestConvertHashToNix32/invalid_format279=== CONT TestSetClientTLSErrors/missing_cert_file280=== CONT TestEncodeNixBase32/test_string_hash281=== CONT TestSetClientTLSErrors/invalid_ca_file282=== CONT TestSetClientTLSErrors/missing_ca_file2832026/11/11 23:09:52 ERROR Upload failed error="bad path" count=3284=== RUN TestSetClientTLS/rejects_connection_without_client_cert285=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert286=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA287--- PASS: TestStreamPushIsolatesFailures (2.65s)288=== CONT TestSetClientTLSErrors/missing_key_file289=== CONT TestPartSizeForNAR/zero_stays_at_minimum290=== CONT TestEncodeNixBase32/empty_input2912026/11/11 23:09:52 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:37187/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:37187/api/uploads/complete292--- PASS: TestSetClientTLSErrors (1.02s)293 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)294 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)295 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)296 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)297=== CONT TestPartSizeForNAR/5_TiB_S3_max_object2982026/11/11 23:09:52 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"299=== CONT TestPartSizeForNAR/1_TiB3002026/11/11 23:09:52 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:37187/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:37187/api/uploads/complete301=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts3022026/11/11 23:09:52 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"303=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum304--- PASS: TestEncodeNixBase32 (0.00s)305 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)306 --- PASS: TestEncodeNixBase32/empty_input (0.00s)307=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA308=== RUN TestSetClientTLS/preserves_debug_logging_transport309=== PAUSE TestSetClientTLS/preserves_debug_logging_transport310=== CONT TestPartSizeForNAR/capped_at_5_GiB311=== CONT TestFilterOversizedClosures/all_closures_skipped3122026/11/11 23:09:52 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50313=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped3142026/11/11 23:09:52 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000315=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=false316=== CONT TestFilterOversizedClosures/no_limit_keeps_everything317--- PASS: TestFilterOversizedClosures (0.00s)318 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)319 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)320 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)321=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths322=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths323=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths324=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=true3252026/11/11 23:09:52 WARN Rate limiter enabled after throttle name=server-test rate=5326=== CONT TestParsePathInfoJSON/Nix_format3272026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:466353282026/11/11 23:09:52 INFO Garbage collection started3292026/11/11 23:09:52 INFO Garbage collection started3302026/11/11 23:09:52 WARN Rate limiter enabled after throttle name=server-test rate=53312026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:44989332--- PASS: TestRegisterUploadedObject_OutlivesPush (2.64s)333=== CONT TestParsePathInfoJSON/invalid_JSON334=== CONT TestParsePathInfoJSON/whitespace_only3352026/11/11 23:09:52 WARN Rate limiter backed off name=server-test rate=5336=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive337=== CONT TestPathInfoCACompatibility/old_string_format_-_text3382026/11/11 23:09:52 WARN Rate limiter backed off name=server-test rate=53392026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:44989340=== CONT TestRateLimiterFeedback/429_enables_limiter341=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter3422026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:35183343=== CONT TestRateLimiterFeedback/503_enables_limiter3442026/11/11 23:09:52 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=0345=== CONT TestGetStorePathHash/valid_store_path346=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)347=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter348=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error349=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error350=== CONT TestGetStorePathHash/basename_without_hyphen_should_error351=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha5123522026/11/11 23:09:52 WARN Rate limiter enabled after throttle name=server-test rate=53532026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:46263354=== CONT TestConvertHashToNix32/SRI_format_to_Nix323552026/11/11 23:09:52 WARN Rate limiter backed off name=server-test rate=5356=== CONT TestConvertHashToNix32/already_Nix32_format3572026/11/11 23:09:52 WARN Rate limiter enabled after throttle name=s3-test rate=53582026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:38399359=== CONT TestConvertHashToNix32/invalid_format360=== CONT TestSetClientTLS/rejects_connection_without_client_cert361=== CONT TestSetClientTLS/preserves_debug_logging_transport3622026/11/11 23:09:52 WARN Garbage collection finished on another replica. Its statistics are not available here363=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA364=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths3652026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:35183366=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths367=== CONT TestParsePathInfoJSON/empty_input368=== CONT TestPathInfoCACompatibility/null_ca_field369=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method370=== CONT TestPathInfoCACompatibility/new_structured_format_-_text371=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI372=== CONT TestPartSizeForNAR/small_stays_at_minimum3732026/11/11 23:09:52 WARN Rate limiter enabled after throttle name=server-test rate=5374--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (2.64s)3752026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:37077376=== CONT TestParsePathInfoJSON/Lix_format377--- PASS: TestDoServerRequestAttachesToken (3.66s)378=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon379--- PASS: TestStreamPushBatchesUnderLoad (0.01s)380 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=false (0.00s)381 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=true (0.00s)3822026/11/11 23:09:52 WARN Rate limiter backed off name=server-test rate=5383--- PASS: TestRegisterUploadedObject_BoundedAgainstSilentServer (2.65s)384--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (2.65s)385--- PASS: TestUploadMultipart_ReturnsAfterDroppedBody (2.64s)386--- PASS: TestDoWithRetry_FinalResponseBodyReadable (2.64s)387--- PASS: TestDumpPathSingleFile (2.65s)388--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (2.64s)389--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (2.64s)390--- PASS: TestGetStorePathHash (0.00s)391 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)392 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)393 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)394 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)395--- PASS: TestRateLimiterFeedback (0.01s)396 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.00s)397 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s)398 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.00s)399 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s)400--- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (2.64s)401--- PASS: TestConvertHashToNix32 (2.62s)402 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)403 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)404 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)405--- PASS: TestParsePathInfoJSONMultiplePaths (2.63s)406 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)407 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)408--- PASS: TestScriptTokenCachesUntilRefresh (2.65s)409--- PASS: TestPathInfoHashCompatibility (0.00s)410 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)411 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)412 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)413 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)414--- PASS: TestPartSizeForNAR (0.00s)415 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)416 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)417 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)418 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)419 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)420 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)421 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)422--- PASS: TestParsePathInfoJSON (0.00s)423 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)424 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)425 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)426 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)427 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)4282026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:35183429--- PASS: TestPathInfoCACompatibility (0.00s)430 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)431 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)432 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)433 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)434 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)4352026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:35183436--- PASS: TestCaseHackSuffix (3.02s)4372026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:351834382026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:35183439--- PASS: TestStreamPushRequestLine (3.05s)4402026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:351834412026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:351834422026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:351834432026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:351834442026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:35183445--- PASS: TestRegisterUploadedObjectReusesConnections (3.05s)4462026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:351834472026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:351834482026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:351834492026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:351834502026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:351834512026/11/11 23:09:52 http: TLS handshake error from 127.0.0.1:41198: remote error: tls: bad certificate452--- PASS: TestDumpPathWriterError (3.07s)4532026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:35183454--- PASS: TestSetClientTLS (2.65s)455 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.40s)456 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.40s)457 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.42s)4582026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:351834592026/11/11 23:09:52 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:35183460--- PASS: TestRetryTimeoutOutlastsMaxRetries (3.07s)461--- PASS: TestDumpPathMatchesNix (3.35s)462--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (3.44s)463--- PASS: TestUploadMultipart_PartsInParallel (3.65s)464PASS465Running server tests...466The files belonging to this database system will be owned by user "nixbld".467This user must also own the server process.468469The database cluster will be initialized with locale "C".470The default database encoding has accordingly been set to "SQL_ASCII".471The default text search configuration will be set to "english".472473Data page checksums are enabled.474475creating directory /build/postgres3193581781/data ... ok476creating subdirectories ... ok477selecting dynamic shared memory implementation ... posix478selecting default "max_connections" ... 100479selecting default "shared_buffers" ... 128MB480selecting default time zone ... UTC481creating configuration files ... ok482running bootstrap script ... ok483performing post-bootstrap initialization ... ok484syncing data to disk ... ok485486initdb: warning: enabling "trust" authentication for local connections487initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.488489Success. You can now start the database server using:490491 pg_ctl -D /build/postgres3193581781/data -l logfile start492493/build/postgres3193581781:5432 - no response494/build/postgres3193581781:5432 - no response495/build/postgres3193581781:5432 - no response496/build/postgres3193581781:5432 - no response497/build/postgres3193581781:5432 - no response498/build/postgres3193581781:5432 - no response499/build/postgres3193581781:5432 - no response500/build/postgres3193581781:5432 - no response501/build/postgres3193581781:5432 - no response502/build/postgres3193581781:5432 - no response503/build/postgres3193581781:5432 - no response504/build/postgres3193581781:5432 - no response505/build/postgres3193581781:5432 - no response506/build/postgres3193581781:5432 - no response507/build/postgres3193581781:5432 - no response508/build/postgres3193581781:5432 - no response509/build/postgres3193581781:5432 - no response510/build/postgres3193581781:5432 - no response511/build/postgres3193581781:5432 - no response512/build/postgres3193581781:5432 - no response513/build/postgres3193581781:5432 - no response514/build/postgres3193581781:5432 - no response515/build/postgres3193581781:5432 - no response516/build/postgres3193581781:5432 - no response517/build/postgres3193581781:5432 - no response518/build/postgres3193581781:5432 - no response519/build/postgres3193581781:5432 - no response520/build/postgres3193581781:5432 - no response521/build/postgres3193581781:5432 - no response522/build/postgres3193581781:5432 - no response523/build/postgres3193581781:5432 - no response524/build/postgres3193581781:5432 - no response525/build/postgres3193581781:5432 - no response526/build/postgres3193581781:5432 - no response527/build/postgres3193581781:5432 - no response528/build/postgres3193581781:5432 - no response529/build/postgres3193581781:5432 - no response530/build/postgres3193581781:5432 - no response531/build/postgres3193581781:5432 - no response532/build/postgres3193581781:5432 - no response533/build/postgres3193581781:5432 - no response534/build/postgres3193581781:5432 - no response535/build/postgres3193581781:5432 - no response536/build/postgres3193581781:5432 - no response537/build/postgres3193581781:5432 - no response538/build/postgres3193581781:5432 - no response539/build/postgres3193581781:5432 - no response540/build/postgres3193581781:5432 - no response541/build/postgres3193581781:5432 - no response542/build/postgres3193581781:5432 - no response543/build/postgres3193581781:5432 - no response544/build/postgres3193581781:5432 - no response545/build/postgres3193581781:5432 - no response546/build/postgres3193581781:5432 - no response547/build/postgres3193581781:5432 - no response548/build/postgres3193581781:5432 - no response549/build/postgres3193581781:5432 - no response550/build/postgres3193581781:5432 - no response551/build/postgres3193581781:5432 - no response552/build/postgres3193581781:5432 - no response553/build/postgres3193581781:5432 - no response554/build/postgres3193581781:5432 - no response555/build/postgres3193581781:5432 - no response556/build/postgres3193581781:5432 - no response557/build/postgres3193581781:5432 - no response558/build/postgres3193581781:5432 - no response559/build/postgres3193581781:5432 - no response560/build/postgres3193581781:5432 - no response5612026-11-11 23:10:14.194 UTC [129] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit5622026-11-11 23:10:14.195 UTC [129] LOG: listening on Unix socket "/build/postgres3193581781/.s.PGSQL.5432"5632026-11-11 23:10:14.205 UTC [203] LOG: database system was shut down at 2026-11-11 23:10:06 UTC5642026-11-11 23:10:14.212 UTC [129] LOG: database system is ready to accept connections565/build/postgres3193581781:5432 - accepting connections566{"timestamp":"2026-11-11T23:10:37.637809838Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"dc6b431b-9a98-4045-96d7-e99981d3354c","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":33,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(194)"}567{"timestamp":"2026-11-11T23:10:37.760886188Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"25426ff3-eaeb-4891-8b0a-9d2757cacfab","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(195)"}568{"timestamp":"2026-11-11T23:10:37.883877558Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8a20ee25-b33f-49b2-98bc-8cacd96f49bc","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(194)"}569{"timestamp":"2026-11-11T23:10:37.988949398Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"9b766e43-d107-4619-83db-0c535bec8475","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(194)"}570{"timestamp":"2026-11-11T23:10:38.124744969Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f075ccd9-fc44-4bfb-ae56-75428b9c7568","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(195)"}571{"timestamp":"2026-11-11T23:10:38.227713188Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c7a09de6-e5d7-481d-801e-6b2f200dd05a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(195)"}572{"timestamp":"2026-11-11T23:10:38.329182454Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8d3d0ccb-6bd0-4ceb-a873-e20bf131778a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(203)"}573{"timestamp":"2026-11-11T23:10:38.434566016Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2aea7cb3-5679-4dba-bf3d-417bb6f0b213","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(206)"}574{"timestamp":"2026-11-11T23:10:38.544223459Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c8177fe0-7d07-42fb-9b85-d56ac7507ae9","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(203)"}575=== RUN TestAbortSucceeded576=== PAUSE TestAbortSucceeded577=== RUN TestService_AuthMiddleware578=== PAUSE TestService_AuthMiddleware579=== RUN TestService_AuthMiddleware_MTLSProxyHeader580=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader581=== RUN TestService_AuthMiddleware_MTLSBoundSubjects582=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects583=== RUN TestService_ReadAuthMiddleware584=== PAUSE TestService_ReadAuthMiddleware585=== RUN TestService_AuthMiddleware_OIDC586=== PAUSE TestService_AuthMiddleware_OIDC587=== RUN TestService_RequireScope_OIDC588=== PAUSE TestService_RequireScope_OIDC589=== RUN TestService_ReadScope_PublicByDefault590=== PAUSE TestService_ReadScope_PublicByDefault591=== RUN TestCacheConfigHandler592=== PAUSE TestCacheConfigHandler593=== RUN TestCacheStatsHandler594=== PAUSE TestCacheStatsHandler595=== RUN TestClientCADerivations596=== PAUSE TestClientCADerivations597=== RUN TestClientErrorHandling598=== PAUSE TestClientErrorHandling599=== RUN TestClientIntegration600=== PAUSE TestClientIntegration601=== RUN TestClientMultipleUploads602=== PAUSE TestClientMultipleUploads603=== RUN TestClientWithDependencies604=== PAUSE TestClientWithDependencies605=== RUN TestClientSharedPathCommittedMidPush606=== PAUSE TestClientSharedPathCommittedMidPush607=== RUN TestPinProtectsFromGC608=== PAUSE TestPinProtectsFromGC609=== RUN TestClientPushesUseOnePush610=== PAUSE TestClientPushesUseOnePush611=== RUN TestClientFallsBackToClosures612=== PAUSE TestClientFallsBackToClosures613=== RUN TestDeleteClosuresRacingPins614=== PAUSE TestDeleteClosuresRacingPins615=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock616=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock617=== RUN TestResolveDBConnectionString618=== PAUSE TestResolveDBConnectionString619=== RUN TestConnectWaitsForAPeerMigration620=== PAUSE TestConnectWaitsForAPeerMigration621=== RUN TestLeadElectsOneAndHandsOver622=== PAUSE TestLeadElectsOneAndHandsOver623=== RUN TestLeadIncumbentWinsAfterRestart6242026/11/11 23:10:39 INFO lead: acquired remote=192.0.2.1:1234 generation=16252026/11/11 23:10:39 INFO lead: released remote=192.0.2.1:1234 generation=16262026/11/11 23:10:39 INFO lead: acquired remote=192.0.2.1:1234 generation=26272026/11/11 23:10:40 INFO lead: released remote=192.0.2.1:1234 generation=2628--- PASS: TestLeadIncumbentWinsAfterRestart (2.26s)629=== RUN TestLeadUnannouncedStandbyYieldsToIncumbent630=== PAUSE TestLeadUnannouncedStandbyYieldsToIncumbent631=== RUN TestLeadIncumbentAnnouncesAtOnce632=== PAUSE TestLeadIncumbentAnnouncesAtOnce633=== RUN TestLeadGenerationGrowsWithEveryGrant634=== PAUSE TestLeadGenerationGrowsWithEveryGrant635=== RUN TestLeadEndsOnShutdown636=== PAUSE TestLeadEndsOnShutdown637=== RUN TestLeadEndsWhenItsConnectionHangs6382026/11/11 23:10:42 INFO lead: acquired remote=192.0.2.1:1234 generation=16392026-11-11 23:10:44.092 UTC [452] FATAL: terminating connection due to administrator command6402026/11/11 23:10:44 INFO lead: acquired remote=192.0.2.1:1234 generation=26412026/11/11 23:10:45 WARN lead: lock connection lost error="timeout: context deadline exceeded"6422026/11/11 23:10:45 INFO lead: released remote=192.0.2.1:1234 generation=16432026/11/11 23:10:45 INFO lead: released remote=192.0.2.1:1234 generation=2644--- PASS: TestLeadEndsWhenItsConnectionHangs (4.77s)645=== RUN TestGCAdvisoryLockBlocksConcurrentRun646=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun647=== RUN TestGCBugBareHashReferences648=== PAUSE TestGCBugBareHashReferences649=== RUN TestGCMetrics650=== PAUSE TestGCMetrics651=== RUN TestPushDedupSurvivesConcurrentGC652=== PAUSE TestPushDedupSurvivesConcurrentGC653=== RUN TestDeduplicatedObjectsRecordedAsPending654=== PAUSE TestDeduplicatedObjectsRecordedAsPending655=== RUN TestGCSweepSkipsPendingObjects656=== PAUSE TestGCSweepSkipsPendingObjects657=== RUN TestTombstonedObjectOfferedWithoutWaiting658=== PAUSE TestTombstonedObjectOfferedWithoutWaiting659=== RUN TestGCSweepDeliversEachKeyOnce660=== PAUSE TestGCSweepDeliversEachKeyOnce661=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection662=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection663=== RUN TestForceGCDuringPushOffersSweptObject664=== PAUSE TestForceGCDuringPushOffersSweptObject665=== RUN TestFailedS3DeleteKeepsTombstone666=== PAUSE TestFailedS3DeleteKeepsTombstone667=== RUN TestSweepSkipsRowsLockedByPush668=== PAUSE TestSweepSkipsRowsLockedByPush669=== RUN TestRegisterCompletedObjectNeedsPendingRow670=== PAUSE TestRegisterCompletedObjectNeedsPendingRow671=== RUN TestGetExistingObjectsLocksRows672=== PAUSE TestGetExistingObjectsLocksRows673=== RUN TestSweepRechecksPendingAfterLocking674=== PAUSE TestSweepRechecksPendingAfterLocking675=== RUN TestPushWaitsForSweepHoldingRow676=== PAUSE TestPushWaitsForSweepHoldingRow677=== RUN TestCommitRacingPendingCleanupKeepsObjects678=== PAUSE TestCommitRacingPendingCleanupKeepsObjects679=== RUN TestGCEndsOnShutdown680=== PAUSE TestGCEndsOnShutdown681=== RUN TestGCTaskStore_StartNew682=== PAUSE TestGCTaskStore_StartNew683=== RUN TestGCTaskStore_DeduplicateSameParams684=== PAUSE TestGCTaskStore_DeduplicateSameParams685=== RUN TestGCTaskStore_ConflictDifferentParams686=== PAUSE TestGCTaskStore_ConflictDifferentParams687=== RUN TestGCTaskStore_GetEmpty688=== PAUSE TestGCTaskStore_GetEmpty689=== RUN TestGCTaskStore_GetReturnsLatest690=== PAUSE TestGCTaskStore_GetReturnsLatest691=== RUN TestGCTaskStore_CompletedAllowsNewTask692=== PAUSE TestGCTaskStore_CompletedAllowsNewTask693=== RUN TestGCTaskStore_PhaseUpdates694=== PAUSE TestGCTaskStore_PhaseUpdates695=== RUN TestGCTaskStore_Fail696=== PAUSE TestGCTaskStore_Fail697=== RUN TestGracefulShutdownDrainsInflight698=== PAUSE TestGracefulShutdownDrainsInflight699=== RUN TestService_healthCheckHandler700=== PAUSE TestService_healthCheckHandler701=== RUN TestService_readinessHandler702=== PAUSE TestService_readinessHandler703=== RUN TestGenerateLandingPage704=== PAUSE TestGenerateLandingPage705=== RUN TestEnsureAbortUploadsRule706=== PAUSE TestEnsureAbortUploadsRule707=== RUN TestCacheConfigHandlerMaxNarSize708=== PAUSE TestCacheConfigHandlerMaxNarSize709=== RUN TestCreatePendingClosureRejectsOversizedNAR710=== PAUSE TestCreatePendingClosureRejectsOversizedNAR711=== RUN TestNARDeduplicationMetadataUploadBug712=== PAUSE TestNARDeduplicationMetadataUploadBug713=== RUN TestMetricsInventory714=== PAUSE TestMetricsInventory715=== RUN TestMetricsCommit716=== PAUSE TestMetricsCommit717=== RUN TestService_NativeMTLS718=== PAUSE TestService_NativeMTLS719=== RUN TestServerTLSConfig720=== PAUSE TestServerTLSConfig721=== RUN TestMultipartCleanup722=== PAUSE TestMultipartCleanup723=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded724=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded725=== RUN TestPendingCleanupUsesOneCutoff726=== PAUSE TestPendingCleanupUsesOneCutoff727=== RUN TestObjectStatsTrigger728=== PAUSE TestObjectStatsTrigger729=== RUN TestObjectStatsUpdatedOncePerCommit730=== PAUSE TestObjectStatsUpdatedOncePerCommit731=== RUN TestOrphanedObjectsGC732=== PAUSE TestOrphanedObjectsGC733=== RUN TestOrphanedObjectsGCStressTest734=== PAUSE TestOrphanedObjectsGCStressTest735=== RUN TestCreatePin_ReservedPins736=== PAUSE TestCreatePin_ReservedPins737=== RUN TestPresentReportsOnlyClosureRoots738=== PAUSE TestPresentReportsOnlyClosureRoots739=== RUN TestPresentRefreshesOnlyStaleClosures740=== PAUSE TestPresentRefreshesOnlyStaleClosures741=== RUN TestPresentNotReportedWhileGCDeletesClosure742=== PAUSE TestPresentNotReportedWhileGCDeletesClosure743=== RUN TestParseSingleRange744=== PAUSE TestParseSingleRange745=== RUN TestProxyHeadersOnlyTrustedOnSocket7462026/11/11 23:10:49 INFO Starting HTTP server address=127.0.0.1:467477472026/11/11 23:10:49 INFO Starting HTTP server address=proxy.sock7482026/11/11 23:10:49 WARN mTLS auth: subject not in bound subjects subject="CN=someone"7492026/11/11 23:10:49 INFO Shutdown signal received, draining in-flight requests timeout=10s750--- PASS: TestProxyHeadersOnlyTrustedOnSocket (3.97s)751=== RUN TestIsValidCachePath752=== PAUSE TestIsValidCachePath753=== RUN TestReadProxyNarinfo754=== PAUSE TestReadProxyNarinfo755=== RUN TestReadProxyNarinfoAlreadyDecompressed756=== PAUSE TestReadProxyNarinfoAlreadyDecompressed757=== RUN TestReadProxyContentNegotiation758=== PAUSE TestReadProxyContentNegotiation759=== RUN TestReadProxyNarStreaming760=== PAUSE TestReadProxyNarStreaming761=== RUN TestReadProxy404762=== PAUSE TestReadProxy404763=== RUN TestReadProxyInvalidPath764=== PAUSE TestReadProxyInvalidPath765=== RUN TestReadProxyHead766=== PAUSE TestReadProxyHead767=== RUN TestReadProxyOutlastsServerWriteTimeout768=== PAUSE TestReadProxyOutlastsServerWriteTimeout769=== RUN TestReadProxyConditionalGet770=== PAUSE TestReadProxyConditionalGet771=== RUN TestReadProxyRootRedirectsToIndexHTML772=== PAUSE TestReadProxyRootRedirectsToIndexHTML773=== RUN TestReadProxyDisabled774=== PAUSE TestReadProxyDisabled775=== RUN TestReadRedirectNar776=== PAUSE TestReadRedirectNar777=== RUN TestReadRedirectKeepsNarinfoProxied778=== PAUSE TestReadRedirectKeepsNarinfoProxied779=== RUN TestReadProxyRangeRequest780=== PAUSE TestReadProxyRangeRequest781=== RUN TestReadRedirectUsesPublicS3URL782=== PAUSE TestReadRedirectUsesPublicS3URL783=== RUN TestPush_OverlappingRootsStoreOneRowPerKey784=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey785=== RUN TestPush_CompleteCommitsEveryRoot786=== PAUSE TestPush_CompleteCommitsEveryRoot787=== RUN TestPush_RepeatedRootCommitsOnce788=== PAUSE TestPush_RepeatedRootCommitsOnce789=== RUN TestPush_CompleteWaitsForCleanupOfThePush790=== PAUSE TestPush_CompleteWaitsForCleanupOfThePush791=== RUN TestPush_RootsAreLockedInByteOrder792=== PAUSE TestPush_RootsAreLockedInByteOrder793=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit794=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit795=== RUN TestPush_RejectsBadRequests796=== PAUSE TestPush_RejectsBadRequests797=== RUN TestPush_SignsNarinfosOfItsPendingObjects798=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects799=== RUN TestSharedNarBothUploadsComplete800=== PAUSE TestSharedNarBothUploadsComplete801=== RUN TestCompleteMultipartUpload_ErrorButObjectExists802=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists803=== RUN TestCompletedNarNotReofferedAcrossClosures804=== PAUSE TestCompletedNarNotReofferedAcrossClosures805=== RUN TestPresignedUploadRegisteredBeforeCommit806=== PAUSE TestPresignedUploadRegisteredBeforeCommit807=== RUN TestService_Rustfstest808=== PAUSE TestService_Rustfstest809=== RUN TestParseSize810=== PAUSE TestParseSize811=== RUN TestSkippedUploadsHandler812=== PAUSE TestSkippedUploadsHandler813=== RUN TestSystemdListenerNotActivated814--- PASS: TestSystemdListenerNotActivated (0.00s)815=== RUN TestWatchdogBeatsWhenHealthy816--- PASS: TestWatchdogBeatsWhenHealthy (0.02s)817=== RUN TestWatchdogSkipsWhenUnhealthy8182026/11/11 23:10:49 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8192026/11/11 23:10:49 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8202026/11/11 23:10:49 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8212026/11/11 23:10:49 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8222026/11/11 23:10:49 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8232026/11/11 23:10:49 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8242026/11/11 23:10:49 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8252026/11/11 23:10:49 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8262026/11/11 23:10:49 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"8272026/11/11 23:10:49 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"828--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)829=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle830=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle831=== RUN TestProxyWriteTimeout832=== PAUSE TestProxyWriteTimeout833=== RUN TestPendingClosureWriteTimeout834=== PAUSE TestPendingClosureWriteTimeout835=== RUN TestIsValidUploadKey836=== PAUSE TestIsValidUploadKey837=== RUN TestUploadHandlersRejectInvalidKeys838=== PAUSE TestUploadHandlersRejectInvalidKeys839=== RUN TestUploadHandlersRejectOversizedBody840=== PAUSE TestUploadHandlersRejectOversizedBody841=== RUN TestService_cleanupPendingClosuresHandler842=== PAUSE TestService_cleanupPendingClosuresHandler843=== RUN TestService_createPendingClosureHandler844=== PAUSE TestService_createPendingClosureHandler845=== RUN TestService_verifyS3Integrity846=== PAUSE TestService_verifyS3Integrity847=== RUN TestService_verifyS3ReuploadsStaleNarinfo848=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo849=== RUN TestCompleteMultipartUnregistered850=== PAUSE TestCompleteMultipartUnregistered851=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT852=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT853=== CONT TestPendingClosureWriteTimeout854=== CONT TestCreatePin_ReservedPins855=== CONT TestPush_SignsNarinfosOfItsPendingObjects856=== CONT TestAbortSucceeded857--- PASS: TestAbortSucceeded (0.00s)858=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle859=== RUN TestPendingClosureWriteTimeout/empty860=== PAUSE TestPendingClosureWriteTimeout/empty861=== RUN TestPendingClosureWriteTimeout/400_objects862=== PAUSE TestPendingClosureWriteTimeout/400_objects863=== RUN TestPendingClosureWriteTimeout/670k_objects864=== PAUSE TestPendingClosureWriteTimeout/670k_objects865=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline866=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline867=== CONT TestServerTLSConfig868=== RUN TestServerTLSConfig/no_client_CA869=== PAUSE TestServerTLSConfig/no_client_CA870=== RUN TestServerTLSConfig/missing_CA_file871=== PAUSE TestServerTLSConfig/missing_CA_file872=== RUN TestServerTLSConfig/not_a_PEM_file873=== PAUSE TestServerTLSConfig/not_a_PEM_file874=== CONT TestService_NativeMTLS875=== CONT TestForceGCDuringPushOffersSweptObject876=== CONT TestMultipartCleanup877=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows878=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows879=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check880=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check881=== CONT TestPresentNotReportedWhileGCDeletesClosure882=== RUN TestPresentNotReportedWhileGCDeletesClosure/stale883=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/stale884=== RUN TestPresentNotReportedWhileGCDeletesClosure/fresh885=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/fresh886=== CONT TestMetricsCommit887=== CONT TestParseSingleRange888=== CONT TestMetricsInventory889=== RUN TestParseSingleRange/none890=== CONT TestPresentRefreshesOnlyStaleClosures891=== CONT TestProxyWriteTimeout892=== RUN TestProxyWriteTimeout/narinfo893=== PAUSE TestProxyWriteTimeout/narinfo894=== RUN TestProxyWriteTimeout/1_GiB_nar895=== PAUSE TestProxyWriteTimeout/1_GiB_nar896=== RUN TestProxyWriteTimeout/10_GiB_nar897=== PAUSE TestProxyWriteTimeout/10_GiB_nar898=== RUN TestProxyWriteTimeout/unknown_size899=== PAUSE TestProxyWriteTimeout/unknown_size900=== CONT TestPresentReportsOnlyClosureRoots901=== CONT TestNARDeduplicationMetadataUploadBug902=== CONT TestSkippedUploadsHandler903=== CONT TestParseSize904=== CONT TestService_Rustfstest905=== CONT TestPresignedUploadRegisteredBeforeCommit906--- PASS: TestParseSize (9.15s)907=== CONT TestCreatePendingClosureRejectsOversizedNAR9082026/11/11 23:10:59 INFO Client skipped oversized paths paths=3 nar_bytes=5000000000909=== CONT TestCompletedNarNotReofferedAcrossClosures910=== CONT TestCompleteMultipartUpload_ErrorButObjectExists911=== CONT TestSharedNarBothUploadsComplete912=== CONT TestOrphanedObjectsGCStressTest913=== CONT TestOrphanedObjectsGC914=== CONT TestObjectStatsUpdatedOncePerCommit915=== CONT TestObjectStatsTrigger9162026/11/11 23:10:59 INFO Received uploads request method=POST path=/api/pending_closures917=== CONT TestPendingCleanupUsesOneCutoff918=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded919=== PAUSE TestParseSingleRange/none920=== RUN TestParseSingleRange/unknown_unit921=== PAUSE TestParseSingleRange/unknown_unit922=== RUN TestParseSingleRange/multi-range_ignored923=== PAUSE TestParseSingleRange/multi-range_ignored924=== RUN TestParseSingleRange/malformed_no_dash925=== PAUSE TestParseSingleRange/malformed_no_dash926=== RUN TestParseSingleRange/malformed_both_empty927=== PAUSE TestParseSingleRange/malformed_both_empty928=== RUN TestParseSingleRange/malformed_end_before_start929=== PAUSE TestParseSingleRange/malformed_end_before_start930=== RUN TestParseSingleRange/closed931=== PAUSE TestParseSingleRange/closed932=== RUN TestParseSingleRange/open-ended933=== PAUSE TestParseSingleRange/open-ended934=== RUN TestParseSingleRange/end_clamped_to_size935=== PAUSE TestParseSingleRange/end_clamped_to_size936=== RUN TestParseSingleRange/suffix937=== PAUSE TestParseSingleRange/suffix938=== RUN TestParseSingleRange/suffix_exceeds_size939=== PAUSE TestParseSingleRange/suffix_exceeds_size940=== RUN TestParseSingleRange/single_byte941=== PAUSE TestParseSingleRange/single_byte942=== RUN TestParseSingleRange/start_past_EOF943=== PAUSE TestParseSingleRange/start_past_EOF944=== RUN TestParseSingleRange/start_far_past_EOF945=== PAUSE TestParseSingleRange/start_far_past_EOF946=== CONT TestCacheConfigHandlerMaxNarSize947--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)948=== CONT TestEnsureAbortUploadsRule949--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.06s)950=== CONT TestGenerateLandingPage951--- PASS: TestSkippedUploadsHandler (9.40s)952=== CONT TestService_readinessHandler953--- PASS: TestGenerateLandingPage (0.01s)954=== CONT TestService_healthCheckHandler9552026/11/11 23:10:59 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:35993/oidc9562026/11/11 23:10:59 INFO Received uploads request method=POST path=/api/pending_closures9572026/11/11 23:10:59 INFO Received push request method=POST path=/api/pushes958--- PASS: TestMetricsInventory (9.62s)959=== CONT TestGracefulShutdownDrainsInflight9602026/11/11 23:10:59 INFO Starting HTTP server address=127.0.0.1:371219612026/11/11 23:10:59 INFO Shutdown signal received, draining in-flight requests timeout=10s9622026/11/11 23:10:59 INFO Received uploads request method=POST path=/api/pending_closures9632026/11/11 23:10:59 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9642026/11/11 23:10:59 INFO Signed narinfos id=1 count=19652026/11/11 23:10:59 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9662026/11/11 23:10:59 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9672026/11/11 23:10:59 ERROR Failed to sign narinfos signer="index 1" error="signing key cannot be nil"9682026/11/11 23:10:59 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9692026/11/11 23:10:59 ERROR Failed to sign narinfos signer=niks3-test-1 error="signer failed"9702026/11/11 23:10:59 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9712026/11/11 23:10:59 ERROR Signer returned unexpected signature count signer=niks3-test-1 got=0 want=19722026/11/11 23:10:59 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9732026/11/11 23:10:59 ERROR Signer returned unexpected object key signer=niks3-test-1 object_key=unexpected.narinfo974--- PASS: TestPush_SignsNarinfosOfItsPendingObjects (9.66s)975=== CONT TestGCTaskStore_Fail976--- PASS: TestGCTaskStore_Fail (0.00s)977=== CONT TestGCTaskStore_PhaseUpdates978--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)979=== CONT TestGCTaskStore_CompletedAllowsNewTask980--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)981=== CONT TestGCTaskStore_GetReturnsLatest982--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)983=== CONT TestGCTaskStore_GetEmpty984--- PASS: TestGCTaskStore_GetEmpty (0.00s)985=== CONT TestGCTaskStore_ConflictDifferentParams986--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)987=== CONT TestGCTaskStore_StartNew988--- PASS: TestGCTaskStore_StartNew (0.00s)989=== CONT TestGCTaskStore_DeduplicateSameParams990--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)991=== CONT TestGCEndsOnShutdown992=== RUN TestGCEndsOnShutdown/before_the_run993=== PAUSE TestGCEndsOnShutdown/before_the_run994=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete995=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete996=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark997=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark998=== CONT TestCommitRacingPendingCleanupKeepsObjects999--- PASS: TestGracefulShutdownDrainsInflight (0.07s)1000=== CONT TestPushWaitsForSweepHoldingRow1001--- PASS: TestPresentRefreshesOnlyStaleClosures (9.70s)1002=== CONT TestSweepRechecksPendingAfterLocking10032026/11/11 23:10:59 WARN mTLS auth: subject not in bound subjects subject="CN=reader"10042026/11/11 23:10:59 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1005--- PASS: TestService_NativeMTLS (9.71s)1006=== CONT TestReadProxyDisabled10072026/11/11 23:10:59 INFO Received cleanup request method=DELETE path=/api/pending_closures10082026/11/11 23:10:59 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket4/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""10092026/11/11 23:10:59 INFO Aborted multipart uploads count=0 kept=110102026/11/11 23:10:59 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10112026/11/11 23:10:59 INFO Received cleanup request method=DELETE path=/api/pending_closures10122026/11/11 23:10:59 INFO Aborted multipart uploads count=1 kept=01013--- PASS: TestMultipartCleanup (9.79s)1014=== CONT TestGetExistingObjectsLocksRows10152026/11/11 23:11:04 WARN Rate limiter enabled after throttle name=s3-test rate=510162026/11/11 23:11:04 WARN S3 rate limit hit operation="complete multipart upload" error="Please reduce your request rate."1017=== NAME TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle1018 throttle_test.go:213: Proxy stats: total=13, throttled=10, completeMultipart=101019 throttle_test.go:215: Rate limiter: enabled=true, rate=5.001020--- PASS: TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle (14.17s)1021=== CONT TestRegisterCompletedObjectNeedsPendingRow1022=== NAME TestNARDeduplicationMetadataUploadBug1023 metadata_upload_test.go:48: First store path: /build/TestNARDeduplicationMetadataUploadBug3751931310/001/store/vq04cxh2vagz41ky15bpp591zabckkkq-file1.txt1024--- PASS: TestReadProxyDisabled (10.61s)1025=== CONT TestPush_RejectsBadRequests1026--- PASS: TestRegisterCompletedObjectNeedsPendingRow (6.17s)1027=== CONT TestSweepSkipsRowsLockedByPush1028--- PASS: TestObjectStatsTrigger (11.03s)1029=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit10302026/11/11 23:11:10 INFO Received uploads request method=POST path=/api/pending_closures10312026/11/11 23:11:10 INFO Received uploads request method=POST path=/api/pending_closures10322026/11/11 23:11:10 INFO Received uploads request method=POST path=/api/pending_closures10332026/11/11 23:11:10 INFO Received cleanup request method=DELETE path=/api/pending_closures10342026/11/11 23:11:10 INFO Aborted multipart uploads count=0 kept=010352026/11/11 23:11:10 WARN Force mode enabled - objects will be deleted immediately without grace period1036--- PASS: TestService_healthCheckHandler (11.07s)1037=== CONT TestFailedS3DeleteKeepsTombstone10382026/11/11 23:11:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010392026/11/11 23:11:10 INFO Vacuumed table table=pending_closures10402026/11/11 23:11:10 INFO Vacuumed table table=pending_objects10412026/11/11 23:11:10 INFO Vacuumed table table=multipart_uploads10422026/11/11 23:11:10 INFO Vacuumed table table=closures10432026/11/11 23:11:10 INFO Vacuumed table table=objects1044--- PASS: TestService_Rustfstest (20.31s)1045=== CONT TestPush_RootsAreLockedInByteOrder1046--- PASS: TestCommitRacingPendingCleanupKeepsObjects (10.88s)1047=== CONT TestPush_CompleteWaitsForCleanupOfThePush10482026/11/11 23:11:10 INFO Received uploads request method=POST path=/api/pending_closures10492026/11/11 23:11:10 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux10502026/11/11 23:11:10 WARN Refused reserved pin name=worker-x86_64-linux10512026/11/11 23:11:10 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux10522026/11/11 23:11:10 INFO Received create pin request method=POST path=/api/pins/my-app10532026/11/11 23:11:10 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux1054--- PASS: TestCreatePin_ReservedPins (20.60s)1055=== CONT TestReadProxy40410562026/11/11 23:11:10 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10572026/11/11 23:11:10 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=ZjU3OTAwOWMtYzllMi00YmVmLWJmMjktZWUxZjJiOWIyODJkLjMwNzRlYmM4LWZkZTAtNDY3My04OTUxLWZjMjk2ZWM0YmUzY3gxNzk0NDM4NjcwNDYzMTI1OTcz10582026/11/11 23:11:10 INFO Installed lifecycle rule aborting incomplete NAR uploads bucket=bucket21 days=21059--- PASS: TestEnsureAbortUploadsRule (11.27s)1060=== CONT TestPush_RepeatedRootCommitsOnce10612026/11/11 23:11:10 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=ZjU3OTAwOWMtYzllMi00YmVmLWJmMjktZWUxZjJiOWIyODJkLjMwNzRlYmM4LWZkZTAtNDY3My04OTUxLWZjMjk2ZWM0YmUzY3gxNzk0NDM4NjcwNDYzMTI1OTcz parts=11062--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (11.31s)1063=== CONT TestReadProxyRootRedirectsToIndexHTML10642026/11/11 23:11:10 INFO Aborted multipart uploads count=0 kept=010652026/11/11 23:11:10 WARN Force mode enabled - objects will be deleted immediately without grace period10662026/11/11 23:11:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010672026/11/11 23:11:10 INFO Vacuumed table table=pending_closures10682026/11/11 23:11:10 INFO Vacuumed table table=pending_objects10692026/11/11 23:11:10 INFO Vacuumed table table=multipart_uploads10702026/11/11 23:11:10 INFO Vacuumed table table=closures10712026/11/11 23:11:10 INFO Vacuumed table table=objects1072--- PASS: TestSweepRechecksPendingAfterLocking (11.16s)1073=== CONT TestPush_CompleteCommitsEveryRoot10742026/11/11 23:11:10 INFO Received uploads request method=POST path=/api/pending_closures1075--- PASS: TestObjectStatsUpdatedOncePerCommit (11.52s)1076=== CONT TestReadProxyConditionalGet1077--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (11.54s)1078=== CONT TestPush_OverlappingRootsStoreOneRowPerKey10792026/11/11 23:11:10 INFO Received uploads request method=POST path=/api/pending_closures1080--- PASS: TestPushWaitsForSweepHoldingRow (11.23s)1081=== CONT TestReadProxyOutlastsServerWriteTimeout10822026/11/11 23:11:10 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst10832026/11/11 23:11:10 INFO Received uploads request method=POST path=/api/pending_closures1084--- PASS: TestPresignedUploadRegisteredBeforeCommit (20.73s)1085=== CONT TestReadRedirectUsesPublicS3URL10862026/11/11 23:11:10 INFO Received push request method=POST path=/api/pushes10872026/11/11 23:11:10 INFO Received complete push request method=POST path=/api/pushes/1/complete10882026/11/11 23:11:10 INFO Completed push id=1 objects=4 duration=17.997051ms1089--- PASS: TestGetExistingObjectsLocksRows (11.23s)1090=== CONT TestReadProxyHead10912026/11/11 23:11:10 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete10922026/11/11 23:11:10 INFO Completed upload id=2 objects=2 duration=4.619024ms1093--- PASS: TestMetricsCommit (21.03s)1094=== CONT TestReadProxyRangeRequest10952026/11/11 23:11:11 WARN readiness check failed error="closed pool"1096--- PASS: TestService_readinessHandler (11.77s)1097=== CONT TestReadProxyInvalidPath1098--- PASS: TestPresentReportsOnlyClosureRoots (21.25s)1099=== CONT TestReadRedirectKeepsNarinfoProxied11002026/11/11 23:11:11 INFO Received uploads request method=POST path=/api/pending_closures11012026/11/11 23:11:11 INFO Received uploads request method=POST path=/api/pending_closures11022026/11/11 23:11:11 INFO Received uploads request method=POST path=/api/pending_closures1103=== RUN TestPush_RejectsBadRequests/no_roots1104=== PAUSE TestPush_RejectsBadRequests/no_roots1105=== RUN TestPush_RejectsBadRequests/no_objects1106=== PAUSE TestPush_RejectsBadRequests/no_objects1107=== RUN TestPush_RejectsBadRequests/bad_root1108=== PAUSE TestPush_RejectsBadRequests/bad_root1109=== RUN TestPush_RejectsBadRequests/root_not_in_objects1110=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects1111=== CONT TestReadRedirectNar11122026/11/11 23:11:11 INFO Received push request method=POST path=/api/pushes11132026/11/11 23:11:11 INFO Aborted multipart uploads count=0 kept=011142026/11/11 23:11:11 INFO Received uploads request method=POST path=/api/pending_closures11152026/11/11 23:11:11 INFO Received complete push request method=POST path=/api/pushes/1/complete1116=== NAME TestOrphanedObjectsGC1117 orphaned_objects_gc_test.go:287: GC Test Summary:1118 orphaned_objects_gc_test.go:288: - Kept: 2 objects from closure A1119 orphaned_objects_gc_test.go:289: - Deleted: 2 objects from closure B1120 orphaned_objects_gc_test.go:290: - Deleted: 6 orphaned chain objects (X1->X2->X3)1121 orphaned_objects_gc_test.go:291: - Deleted: 2 orphaned single objects (Y)1122 orphaned_objects_gc_test.go:292: - Total deleted: 10 objects1123--- PASS: TestOrphanedObjectsGC (12.11s)1124=== CONT TestService_createPendingClosureHandler11252026/11/11 23:11:11 INFO Received push request method=POST path=/api/pushes11262026/11/11 23:11:11 WARN Force mode enabled - objects will be deleted immediately without grace period11272026/11/11 23:11:11 INFO Completed push id=1 objects=4 duration=54.569899ms11282026/11/11 23:11:11 INFO Received push request method=POST path=/api/pushes11292026/11/11 23:11:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011302026/11/11 23:11:11 INFO Vacuumed table table=pending_closures11312026/11/11 23:11:11 INFO Vacuumed table table=pending_objects11322026/11/11 23:11:11 INFO Vacuumed table table=multipart_uploads11332026/11/11 23:11:11 INFO Vacuumed table table=closures11342026/11/11 23:11:11 INFO Vacuumed table table=objects11352026/11/11 23:11:11 INFO Aborted multipart uploads count=0 kept=011362026/11/11 23:11:11 INFO Aborted multipart uploads count=0 kept=011372026/11/11 23:11:11 INFO Received push request method=POST path=/api/pushes11382026/11/11 23:11:11 INFO Aborted multipart uploads count=0 kept=011392026/11/11 23:11:11 WARN Force mode enabled - objects will be deleted immediately without grace period11402026/11/11 23:11:11 INFO Received complete push request method=POST path=/api/pushes/1/complete11412026/11/11 23:11:11 WARN Force mode enabled - objects will be deleted immediately without grace period11422026/11/11 23:11:11 WARN Force mode enabled - objects will be deleted immediately without grace period11432026/11/11 23:11:11 INFO Received push request method=POST path=/api/pushes11442026/11/11 23:11:11 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."11452026/11/11 23:11:11 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."1146--- PASS: TestFailedS3DeleteKeepsTombstone (1.14s)1147=== CONT TestGCAdvisoryLockBlocksConcurrentRun1148--- PASS: TestReadProxy404 (1.03s)1149=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection11502026/11/11 23:11:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=011512026/11/11 23:11:11 INFO Vacuumed table table=pending_closures11522026/11/11 23:11:11 INFO Vacuumed table table=pending_objects11532026/11/11 23:11:11 INFO Vacuumed table table=multipart_uploads11542026/11/11 23:11:11 INFO Vacuumed table table=closures11552026/11/11 23:11:11 INFO Vacuumed table table=objects1156--- PASS: TestSweepSkipsRowsLockedByPush (1.31s)1157=== CONT TestDeleteClosuresRacingPins1158--- PASS: TestReadProxyRootRedirectsToIndexHTML (0.98s)1159=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT11602026/11/11 23:11:11 INFO Completed push id=1 objects=4 duration=70.40193ms11612026/11/11 23:11:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=01162--- PASS: TestPush_RepeatedRootCommitsOnce (1.04s)1163=== CONT TestGCSweepDeliversEachKeyOnce11642026/11/11 23:11:11 INFO Vacuumed table table=pending_closures11652026/11/11 23:11:11 INFO Vacuumed table table=pending_objects11662026/11/11 23:11:11 INFO Vacuumed table table=multipart_uploads11672026/11/11 23:11:11 INFO Vacuumed table table=closures11682026/11/11 23:11:11 INFO Vacuumed table table=objects1169--- PASS: TestReadRedirectUsesPublicS3URL (0.80s)1170=== CONT TestLeadEndsOnShutdown11712026/11/11 23:11:11 INFO Received complete push request method=POST path=/api/pushes/2/complete11722026/11/11 23:11:11 INFO Completed push id=2 objects=4 duration=7.208768ms1173--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (1.36s)1174=== CONT TestCompleteMultipartUnregistered11752026/11/11 23:11:11 INFO Received push request method=POST path=/api/pushes11762026/11/11 23:11:11 INFO Received complete push request method=POST path=/api/pushes/1/complete11772026-11-11 23:11:11.742 UTC [685] ERROR: Push does not exist: id=111782026-11-11 23:11:11.742 UTC [685] CONTEXT: PL/pgSQL function commit_push(bigint) line 11 at RAISE11792026-11-11 23:11:11.742 UTC [685] STATEMENT: SELECT commit_push($1)1180--- PASS: TestPush_CompleteWaitsForCleanupOfThePush (1.31s)1181=== CONT TestTombstonedObjectOfferedWithoutWaiting11822026/11/11 23:11:11 INFO Received push request method=POST path=/api/pushes1183--- PASS: TestReadProxyConditionalGet (0.95s)1184=== CONT TestService_verifyS3ReuploadsStaleNarinfo1185=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1186=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1187=== RUN TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1188=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1189=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1190=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost1191=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1192=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1193=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1194=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1195=== CONT TestLeadGenerationGrowsWithEveryGrant11962026-11-11 23:11:11.747 UTC [695] ERROR: could not obtain lock on row in relation "closures"11972026-11-11 23:11:11.747 UTC [695] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT11982026-11-11 23:11:11.749 UTC [695] ERROR: could not obtain lock on row in relation "closures"11992026-11-11 23:11:11.749 UTC [695] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT12002026-11-11 23:11:11.750 UTC [695] ERROR: could not obtain lock on row in relation "closures"12012026-11-11 23:11:11.750 UTC [695] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT12022026-11-11 23:11:11.750 UTC [695] ERROR: could not obtain lock on row in relation "closures"12032026-11-11 23:11:11.750 UTC [695] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT12042026-11-11 23:11:11.750 UTC [695] ERROR: could not obtain lock on row in relation "closures"12052026-11-11 23:11:11.750 UTC [695] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT1206--- PASS: TestPush_RootsAreLockedInByteOrder (1.35s)1207=== CONT TestGCSweepSkipsPendingObjects12082026/11/11 23:11:11 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)12092026/11/11 23:11:11 INFO Uploading vq04cxh2vagz41ky15bpp591zabckkkq-file1.txt (160B)12102026/11/11 23:11:11 INFO Received complete multipart upload request method=POST path=/api/multipart/complete12112026/11/11 23:11:11 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"12122026/11/11 23:11:11 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign1213--- PASS: TestReadProxyInvalidPath (0.71s)1214=== CONT TestService_verifyS3Integrity12152026/11/11 23:11:11 WARN Failed to register uploaded object key=vq04cxh2vagz41ky15bpp591zabckkkq.ls error="server returned 404: 404 page not found\n"12162026/11/11 23:11:11 INFO Signed narinfos id=1 count=112172026/11/11 23:11:11 INFO Uploading 1 narinfos12182026/11/11 23:11:11 INFO Received complete push request method=POST path=/api/pushes/1/complete12192026/11/11 23:11:11 WARN Failed to register uploaded object key=vq04cxh2vagz41ky15bpp591zabckkkq.narinfo error="server returned 404: 404 page not found\n"12202026/11/11 23:11:11 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=ZjU3OTAwOWMtYzllMi00YmVmLWJmMjktZWUxZjJiOWIyODJkLmI0ZTBhYmZkLTVjNzgtNDA2NC1iNmRmLWU0MTE1YzdlMTBhYngxNzk0NDM4NjcxMTgxNjM4Njg4 parts=1212212026/11/11 23:11:11 INFO Received uploads request method=POST path=/api/pending_closures12222026/11/11 23:11:11 INFO Completed push id=1 objects=3 duration=16.838801ms12232026/11/11 23:11:11 INFO Upload complete. (379ms)12242026/11/11 23:11:11 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1225=== NAME TestNARDeduplicationMetadataUploadBug1226 metadata_upload_test.go:54: Retrieved narinfo from S3:1227 StorePath: /build/TestNARDeduplicationMetadataUploadBug3751931310/001/store/vq04cxh2vagz41ky15bpp591zabckkkq-file1.txt1228 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1229 Compression: zstd1230 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1231 NarSize: 1601232 References: 1233 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1234--- PASS: TestCompletedNarNotReofferedAcrossClosures (12.57s)1235=== CONT TestLeadIncumbentAnnouncesAtOnce1236=== NAME TestNARDeduplicationMetadataUploadBug1237 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)1238 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):1239 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}12402026/11/11 23:11:11 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=ZjU3OTAwOWMtYzllMi00YmVmLWJmMjktZWUxZjJiOWIyODJkLjBmODA1MzdjLWNlNTAtNGE0MC04MjI3LWEyYWUwNDIxOWMwN3gxNzk0NDM4NjcxMjE1MzI4NjI4 parts=121241--- PASS: TestReadProxyHead (0.95s)1242=== CONT TestDeduplicatedObjectsRecordedAsPending1243--- PASS: TestReadProxyRangeRequest (1.04s)1244=== CONT TestGCMetrics12452026/11/11 23:11:11 INFO Completed push id=1 objects=6 duration=271.963688ms1246--- PASS: TestPush_CompleteCommitsEveryRoot (1.23s)1247=== CONT TestLeadUnannouncedStandbyYieldsToIncumbent1248=== NAME TestNARDeduplicationMetadataUploadBug1249 metadata_upload_test.go:64: Second store path (same content): /build/TestNARDeduplicationMetadataUploadBug3751931310/001/store/5qfxlhw4d9srayb54yazx0rnda43k98z-file2.txt12502026/11/11 23:11:12 INFO Received push request method=POST path=/api/pushes1251--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (1.25s)1252=== CONT TestGCBugBareHashReferences1253--- PASS: TestReadRedirectKeepsNarinfoProxied (0.92s)1254=== CONT TestPushDedupSurvivesConcurrentGC1255--- PASS: TestReadRedirectNar (0.89s)1256=== CONT TestLeadElectsOneAndHandsOver12572026/11/11 23:11:12 INFO Received uploads request method=POST path=/api/pending_closures1258--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (0.64s)1259=== CONT TestReadProxyNarinfoAlreadyDecompressed12602026/11/11 23:11:12 INFO Received uploads request method=POST path=/api/pending_closures12612026/11/11 23:11:12 INFO Received uploads request method=POST path=/api/pending_closures12622026/11/11 23:11:12 INFO Received uploads request method=POST path=/api/pending_closures12632026/11/11 23:11:12 INFO Received uploads request method=POST path=/api/pending_closures12642026/11/11 23:11:12 INFO lead: acquired remote=192.0.2.1:1234 generation=112652026/11/11 23:11:12 INFO Received complete multipart upload request method=POST path=/api/multipart/complete12662026/11/11 23:11:12 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=nixbld database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"1267--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (0.83s)1268=== CONT TestConnectWaitsForAPeerMigration12692026/11/11 23:11:12 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=ZjU3OTAwOWMtYzllMi00YmVmLWJmMjktZWUxZjJiOWIyODJkLjRiYTMwMzMwLTgwMjMtNGI1Zi1iZjhiLTNjZWZhNDc5YWUxNXgxNzk0NDM4NjcxMjI3NjE0Njg1 parts=121270--- PASS: TestSharedNarBothUploadsComplete (13.11s)1271=== CONT TestReadProxyNarStreaming12722026/11/11 23:11:12 INFO Received uploads request method=POST path=/api/pending_closures1273--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (0.90s)1274=== CONT TestResolveDBConnectionString1275=== RUN TestResolveDBConnectionString/flag_wins1276=== PAUSE TestResolveDBConnectionString/flag_wins1277=== RUN TestResolveDBConnectionString/file_when_flag_empty1278=== PAUSE TestResolveDBConnectionString/file_when_flag_empty1279=== RUN TestResolveDBConnectionString/missing_file_is_an_error1280=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error1281=== RUN TestResolveDBConnectionString/PGHOST_allows_empty1282=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty1283=== RUN TestResolveDBConnectionString/nothing_configured1284=== PAUSE TestResolveDBConnectionString/nothing_configured1285=== CONT TestReadProxyContentNegotiation12862026/11/11 23:11:12 INFO Received complete multipart upload request method=POST path=/api/multipart/complete12872026/11/11 23:11:12 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst1288--- PASS: TestCompleteMultipartUnregistered (0.80s)1289=== CONT TestUploadHandlersRejectOversizedBody12902026/11/11 23:11:12 INFO lead: acquired remote=192.0.2.1:1234 generation=112912026/11/11 23:11:12 INFO lead: released remote=192.0.2.1:1234 generation=11292--- PASS: TestLeadIncumbentAnnouncesAtOnce (0.69s)1293=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock12942026/11/11 23:11:12 INFO Received uploads request method=POST path=/api/pending_closures1295--- PASS: TestTombstonedObjectOfferedWithoutWaiting (0.86s)1296=== CONT TestService_cleanupPendingClosuresHandler1297=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure1298=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure1299=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart1300=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart13012026/11/11 23:11:12 INFO Aborted multipart uploads count=0 kept=01302=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts1303=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts1304=== CONT TestClientCADerivations13052026/11/11 23:11:12 INFO lead: acquired remote=192.0.2.1:1234 generation=113062026/11/11 23:11:12 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1307--- PASS: TestReadProxyNarinfoAlreadyDecompressed (0.53s)1308=== CONT TestClientSharedPathCommittedMidPush13092026/11/11 23:11:12 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=ZjU3OTAwOWMtYzllMi00YmVmLWJmMjktZWUxZjJiOWIyODJkLjY5ZmI5NDg3LTQzMGItNDNkNC1hYTQzLWJiNmJjZjZjY2QyMHgxNzk0NDM4NjcyMjc4MDQzMjc4 parts=1013102026/11/11 23:11:12 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete13112026/11/11 23:11:12 INFO Completed upload id=1 objects=2 duration=12.006575ms13122026/11/11 23:11:12 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000013132026/11/11 23:11:12 INFO Received uploads request method=POST path=/api/pending_closures13142026/11/11 23:11:12 INFO Starting cleanup of old closures method=DELETE path=/api/closures13152026/11/11 23:11:12 INFO Aborted multipart uploads count=0 kept=013162026/11/11 23:11:12 INFO Aborted multipart uploads count=0 kept=013172026/11/11 23:11:12 INFO Received complete multipart upload request method=POST path=/api/multipart/complete13182026/11/11 23:11:12 INFO Received uploads request method=POST path=/api/pending_closures13192026/11/11 23:11:12 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=ZjU3OTAwOWMtYzllMi00YmVmLWJmMjktZWUxZjJiOWIyODJkLmUwOGUxM2JiLThiY2QtNGZmYi04YjYwLTljMjlhOGUxYTU3MngxNzk0NDM4NjcyNDI0NDQzNDcx parts=101320--- PASS: TestDeduplicatedObjectsRecordedAsPending (0.98s)1321=== CONT TestReadProxyNarinfo13222026/11/11 23:11:12 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete13232026/11/11 23:11:12 INFO Completed upload id=1 objects=2 duration=5.889376ms13242026/11/11 23:11:12 INFO Received uploads request method=POST path=/api/pending_closures13252026/11/11 23:11:12 WARN Force mode enabled - objects will be deleted immediately without grace period13262026/11/11 23:11:12 WARN Force mode enabled - objects will be deleted immediately without grace period13272026/11/11 23:11:12 INFO Received uploads request method=POST path=/api/pending_closures13282026/11/11 23:11:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013292026/11/11 23:11:12 INFO Vacuumed table table=pending_closures13302026/11/11 23:11:12 INFO Vacuumed table table=pending_objects13312026/11/11 23:11:12 INFO Vacuumed table table=multipart_uploads13322026/11/11 23:11:12 INFO Vacuumed table table=closures13332026/11/11 23:11:12 INFO Vacuumed table table=objects1334--- PASS: TestGCMetrics (0.91s)1335=== CONT TestClientWithDependencies13362026/11/11 23:11:12 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo13372026/11/11 23:11:12 WARN Found objects in DB but missing from S3, will re-upload count=113382026/11/11 23:11:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=01339--- PASS: TestService_verifyS3Integrity (1.08s)1340=== CONT TestClientFallsBackToClosures13412026/11/11 23:11:12 INFO Received uploads request method=POST path=/api/pending_closures13422026/11/11 23:11:12 INFO Vacuumed table table=pending_closures13432026/11/11 23:11:12 INFO Vacuumed table table=pending_objects13442026/11/11 23:11:12 INFO Vacuumed table table=multipart_uploads13452026/11/11 23:11:12 INFO Vacuumed table table=closures13462026/11/11 23:11:12 INFO Vacuumed table table=objects13472026/11/11 23:11:12 INFO Aborted multipart uploads count=0 kept=013482026/11/11 23:11:12 INFO lead: acquired remote=192.0.2.1:1234 generation=113492026/11/11 23:11:12 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000013502026/11/11 23:11:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=01351--- PASS: TestService_createPendingClosureHandler (1.56s)1352=== CONT TestClientMultipleUploads13532026/11/11 23:11:12 INFO Vacuumed table table=pending_closures13542026/11/11 23:11:12 INFO Vacuumed table table=pending_objects13552026/11/11 23:11:12 INFO Vacuumed table table=multipart_uploads13562026/11/11 23:11:12 INFO Vacuumed table table=closures13572026/11/11 23:11:12 INFO Vacuumed table table=objects1358--- PASS: TestGCSweepDeliversEachKeyOnce (1.38s)1359=== CONT TestService_AuthMiddleware_OIDC13602026/11/11 23:11:12 WARN Force mode enabled - objects will be deleted immediately without grace period13612026/11/11 23:11:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013622026/11/11 23:11:12 INFO Vacuumed table table=pending_closures13632026/11/11 23:11:12 INFO Vacuumed table table=pending_objects13642026/11/11 23:11:12 INFO Vacuumed table table=multipart_uploads13652026/11/11 23:11:12 INFO Vacuumed table table=closures13662026/11/11 23:11:12 INFO Vacuumed table table=objects1367--- PASS: TestGCSweepSkipsPendingObjects (1.26s)1368=== CONT TestClientIntegration13692026/11/11 23:11:13 INFO Received cleanup request method=DELETE path=/api/pending_closures13702026/11/11 23:11:13 INFO Aborted multipart uploads count=0 kept=013712026/11/11 23:11:13 INFO Received uploads request method=POST path=/api/pending_closures13722026/11/11 23:11:13 INFO Received cleanup request method=DELETE path=/api/pending_closures13732026/11/11 23:11:13 INFO Aborted multipart uploads count=1 kept=013742026/11/11 23:11:13 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete13752026-11-11 23:11:13.067 UTC [856] ERROR: Closure does not exist: id=113762026-11-11 23:11:13.067 UTC [856] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 13 at RAISE13772026-11-11 23:11:13.067 UTC [856] STATEMENT: -- name: CommitPendingClosure :one1378 SELECT commit_pending_closure($1::bigint)::bigint1379 1380--- PASS: TestService_cleanupPendingClosuresHandler (0.47s)1381=== CONT TestCacheStatsHandler13822026/11/11 23:11:13 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:39039/oidc13832026/11/11 23:11:13 INFO Received uploads request method=POST path=/api/pending_closures13842026/11/11 23:11:13 INFO lead: acquired remote=192.0.2.1:1234 generation=113852026/11/11 23:11:13 INFO Aborted multipart uploads count=0 kept=013862026/11/11 23:11:13 ERROR Failed to decode narinfo key=3hcdxyjf9yiq7qf3i4548drb6sjmwa1v.narinfo limit=16777216 error=<nil>1387--- PASS: TestReadProxyNarinfo (0.43s)1388=== CONT TestClientPushesUseOnePush13892026/11/11 23:11:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013902026/11/11 23:11:13 INFO Vacuumed table table=pending_closures13912026/11/11 23:11:13 INFO Received push request method=POST path=/api/pushes13922026/11/11 23:11:13 INFO Vacuumed table table=pending_objects13932026/11/11 23:11:13 INFO Vacuumed table table=multipart_uploads13942026/11/11 23:11:13 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)13952026/11/11 23:11:13 INFO Vacuumed table table=closures13962026/11/11 23:11:13 INFO Vacuumed table table=objects13972026/11/11 23:11:13 INFO Aborted multipart uploads count=0 kept=013982026/11/11 23:11:13 WARN Force mode enabled - objects will be deleted immediately without grace period13992026/11/11 23:11:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=014002026/11/11 23:11:13 INFO Vacuumed table table=pending_closures14012026/11/11 23:11:13 INFO Vacuumed table table=pending_objects14022026/11/11 23:11:13 INFO Vacuumed table table=multipart_uploads14032026/11/11 23:11:13 INFO Vacuumed table table=closures14042026/11/11 23:11:13 INFO Vacuumed table table=objects14052026/11/11 23:11:13 INFO lead: released remote=192.0.2.1:1234 generation=11406--- PASS: TestLeadEndsOnShutdown (1.75s)1407=== CONT TestCacheConfigHandler1408=== RUN TestCacheConfigHandler/full_config,_no_issuer1409=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1410=== RUN TestCacheConfigHandler/no_cache_url_configured1411=== PAUSE TestCacheConfigHandler/no_cache_url_configured1412=== RUN TestCacheConfigHandler/no_signing_keys1413=== PAUSE TestCacheConfigHandler/no_signing_keys1414=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1415=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1416=== CONT TestPinProtectsFromGC1417--- PASS: TestPushDedupSurvivesConcurrentGC (1.46s)1418=== CONT TestService_ReadScope_PublicByDefault14192026/11/11 23:11:13 WARN Failed to register uploaded object key=5qfxlhw4d9srayb54yazx0rnda43k98z.ls error="server returned 404: 404 page not found\n"14202026/11/11 23:11:13 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign14212026/11/11 23:11:13 INFO Signed narinfos id=2 count=114222026/11/11 23:11:13 INFO Uploading 1 narinfos14232026/11/11 23:11:13 INFO Received complete push request method=POST path=/api/pushes/2/complete14242026/11/11 23:11:13 WARN Failed to register uploaded object key=5qfxlhw4d9srayb54yazx0rnda43k98z.narinfo error="server returned 404: 404 page not found\n"14252026/11/11 23:11:13 INFO Completed push id=2 objects=3 duration=7.685593ms14262026/11/11 23:11:13 INFO Upload complete. (1.174s)1427=== NAME TestNARDeduplicationMetadataUploadBug1428 metadata_upload_test.go:76: Retrieved narinfo from S3:1429 StorePath: /build/TestNARDeduplicationMetadataUploadBug3751931310/001/store/5qfxlhw4d9srayb54yazx0rnda43k98z-file2.txt1430 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1431 Compression: zstd1432 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1433 NarSize: 1601434 References: 1435 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1436 metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes)1437 metadata_upload_test.go:77: Decompressed .ls content (49 bytes):1438 {"version":1,"root":{"type":"regular","size":44}}1439--- PASS: TestReadProxyNarStreaming (1.19s)1440=== CONT TestClientErrorHandling1441=== RUN TestClientErrorHandling/InvalidStorePath1442=== PAUSE TestClientErrorHandling/InvalidStorePath1443=== RUN TestClientErrorHandling/InvalidAuthToken1444=== PAUSE TestClientErrorHandling/InvalidAuthToken1445=== RUN TestClientErrorHandling/ServerNotAvailable1446=== PAUSE TestClientErrorHandling/ServerNotAvailable1447=== CONT TestService_RequireScope_OIDC1448--- PASS: TestNARDeduplicationMetadataUploadBug (23.66s)1449=== CONT TestIsValidCachePath1450=== RUN TestIsValidCachePath/narinfo1451=== PAUSE TestIsValidCachePath/narinfo1452=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars1453=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars1454=== RUN TestIsValidCachePath/nar_zst1455=== PAUSE TestIsValidCachePath/nar_zst1456=== RUN TestIsValidCachePath/nar_xz1457=== PAUSE TestIsValidCachePath/nar_xz1458=== RUN TestIsValidCachePath/nar_bz21459=== PAUSE TestIsValidCachePath/nar_bz21460=== RUN TestIsValidCachePath/nar_uncompressed1461=== PAUSE TestIsValidCachePath/nar_uncompressed1462=== RUN TestIsValidCachePath/ls1463=== PAUSE TestIsValidCachePath/ls1464=== RUN TestIsValidCachePath/log1465=== PAUSE TestIsValidCachePath/log1466=== RUN TestIsValidCachePath/realisation1467=== PAUSE TestIsValidCachePath/realisation1468=== RUN TestIsValidCachePath/nix-cache-info1469=== PAUSE TestIsValidCachePath/nix-cache-info1470=== RUN TestIsValidCachePath/index.html1471=== PAUSE TestIsValidCachePath/index.html1472=== RUN TestIsValidCachePath/traversal_parent1473=== PAUSE TestIsValidCachePath/traversal_parent1474=== RUN TestIsValidCachePath/traversal_in_middle1475=== PAUSE TestIsValidCachePath/traversal_in_middle1476=== RUN TestIsValidCachePath/invalid_char_e1477=== PAUSE TestIsValidCachePath/invalid_char_e1478=== RUN TestIsValidCachePath/invalid_char_u1479=== PAUSE TestIsValidCachePath/invalid_char_u1480=== RUN TestIsValidCachePath/random_path1481=== PAUSE TestIsValidCachePath/random_path1482=== RUN TestIsValidCachePath/empty1483=== PAUSE TestIsValidCachePath/empty1484=== RUN TestIsValidCachePath/leading_slash1485=== PAUSE TestIsValidCachePath/leading_slash1486=== RUN TestIsValidCachePath/wrong_extension1487=== PAUSE TestIsValidCachePath/wrong_extension1488=== RUN TestIsValidCachePath/short_hash1489=== PAUSE TestIsValidCachePath/short_hash1490=== CONT TestService_AuthMiddleware_MTLSProxyHeader1491--- PASS: TestReadProxyContentNegotiation (1.22s)1492=== CONT TestService_AuthMiddleware_MTLSBoundSubjects14932026/11/11 23:11:13 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:41985/oidc1494--- PASS: TestGCBugBareHashReferences (1.63s)1495=== CONT TestUploadHandlersRejectInvalidKeys1496=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1497=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1498=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal1499=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal1500=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key1501=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key1502=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key1503=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key1504=== CONT TestService_ReadAuthMiddleware1505=== NAME TestClientIntegration1506 client_integration_test.go:291: Created store path: /build/TestClientIntegration1783177883/002/store/bcir67hn40hd56il4kd0xfcjz0vym1dm-test-file.txt1507--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (0.23s)1508=== CONT TestService_AuthMiddleware1509=== NAME TestOrphanedObjectsGCStressTest1510 orphaned_objects_gc_test.go:422: Created 10 active closures, 5 to-delete closures, 20 orphaned chains15112026/11/11 23:11:13 INFO lead: released remote=192.0.2.1:1234 generation=115122026/11/11 23:11:13 INFO lead: acquired remote=192.0.2.1:1234 generation=21513=== NAME TestClientMultipleUploads1514 client_integration_test.go:405: Created store path 0: /build/TestClientMultipleUploads3942683860/001/store/qarxz46zjbn5b7959fyf2dcqmyh466x4-test-file-0.txt1515=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token1516=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token1517=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1518=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1519=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1520=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1521=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1522=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1523=== CONT TestIsValidUploadKey1524=== RUN TestIsValidUploadKey/narinfo1525=== PAUSE TestIsValidUploadKey/narinfo1526=== RUN TestIsValidUploadKey/nar_zst1527=== PAUSE TestIsValidUploadKey/nar_zst1528=== RUN TestIsValidUploadKey/nar_xz1529=== PAUSE TestIsValidUploadKey/nar_xz1530=== RUN TestIsValidUploadKey/nar_plain1531=== PAUSE TestIsValidUploadKey/nar_plain1532=== RUN TestIsValidUploadKey/listing1533=== PAUSE TestIsValidUploadKey/listing1534=== RUN TestIsValidUploadKey/build_log1535=== PAUSE TestIsValidUploadKey/build_log1536=== RUN TestIsValidUploadKey/build_log_home-manager_file1537=== PAUSE TestIsValidUploadKey/build_log_home-manager_file1538=== RUN TestIsValidUploadKey/build_log_plus_in_name1539=== PAUSE TestIsValidUploadKey/build_log_plus_in_name1540=== RUN TestIsValidUploadKey/build_log_question_mark1541=== PAUSE TestIsValidUploadKey/build_log_question_mark1542=== RUN TestIsValidUploadKey/build_log_equals1543=== PAUSE TestIsValidUploadKey/build_log_equals1544=== RUN TestIsValidUploadKey/realisation1545=== PAUSE TestIsValidUploadKey/realisation1546=== RUN TestIsValidUploadKey/realisation_plus_in_output1547=== PAUSE TestIsValidUploadKey/realisation_plus_in_output1548=== RUN TestIsValidUploadKey/nix-cache-info1549=== PAUSE TestIsValidUploadKey/nix-cache-info1550=== RUN TestIsValidUploadKey/index.html1551=== PAUSE TestIsValidUploadKey/index.html1552=== RUN TestIsValidUploadKey/narinfo_key,_nar_type1553=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type1554=== RUN TestIsValidUploadKey/nar_key,_narinfo_type1555=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type1556=== RUN TestIsValidUploadKey/listing_key,_narinfo_type1557=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type1558=== RUN TestIsValidUploadKey/traversal1559=== PAUSE TestIsValidUploadKey/traversal1560=== RUN TestIsValidUploadKey/traversal_nar1561=== PAUSE TestIsValidUploadKey/traversal_nar1562=== RUN TestIsValidUploadKey/absolute1563=== PAUSE TestIsValidUploadKey/absolute1564=== RUN TestIsValidUploadKey/empty_key1565=== PAUSE TestIsValidUploadKey/empty_key1566=== RUN TestIsValidUploadKey/unknown_type1567=== PAUSE TestIsValidUploadKey/unknown_type1568=== CONT TestPendingClosureWriteTimeout/empty1569=== CONT TestServerTLSConfig/no_client_CA1570=== CONT TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1571=== NAME TestOrphanedObjectsGCStressTest1572 orphaned_objects_gc_test.go:440: Marked 210 objects for deletion1573--- PASS: TestService_ReadAuthMiddleware (0.27s)1574=== CONT TestPendingClosureWriteTimeout/670k_objects1575=== CONT TestPendingClosureWriteTimeout/400_objects1576=== CONT TestServerTLSConfig/not_a_PEM_file1577=== CONT TestServerTLSConfig/missing_CA_file1578=== CONT TestForceGCDuringPushOffersSweptObject/before_pending_rows1579--- PASS: TestServerTLSConfig (0.00s)1580 --- PASS: TestServerTLSConfig/no_client_CA (0.00s)1581 --- PASS: TestServerTLSConfig/not_a_PEM_file (0.00s)1582 --- PASS: TestServerTLSConfig/missing_CA_file (0.00s)1583--- PASS: TestCacheStatsHandler (0.96s)1584=== CONT TestPresentNotReportedWhileGCDeletesClosure/stale1585=== NAME TestClientCADerivations1586 client_ca_test.go:136: Built CA derivation: /build/TestClientCADerivations1352219559/001/store/g0p76b5vxfl3l634xn2xgmab6v9xvald-ca-test15872026/11/11 23:11:14 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"15882026/11/11 23:11:14 WARN mTLS auth: bound subjects configured but subject DN unavailable15892026/11/11 23:11:14 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1590--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (0.52s)1591=== CONT TestPresentNotReportedWhileGCDeletesClosure/fresh15922026/11/11 23:11:14 INFO lead: released remote=192.0.2.1:1234 generation=11593=== NAME TestPinProtectsFromGC1594 client_integration_test.go:778: Pinned store path: /build/TestPinProtectsFromGC3998447353/001/store/pfyz5xn6c923fri3zy6flwiwkzlx7vf5-pinned-file.txt1595 client_integration_test.go:779: Unpinned store path: /build/TestPinProtectsFromGC3998447353/001/store/gcbq1zlbhbh1311xwam2xbg9bbavhrmc-unpinned-file.txt1596=== NAME TestClientMultipleUploads1597 client_integration_test.go:405: Created store path 1: /build/TestClientMultipleUploads3942683860/001/store/1z4zvyvma1wns5jixj73gswic1v1wf39-test-file-1.txt1598--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (1.67s)1599=== CONT TestForceGCDuringPushOffersSweptObject/after_presence_check16002026/11/11 23:11:14 INFO Received uploads request method=POST path=/api/pending_closures16012026/11/11 23:11:14 INFO lead: acquired remote=192.0.2.1:1234 generation=216022026/11/11 23:11:14 INFO lead: released remote=192.0.2.1:1234 generation=11603=== RUN TestService_RequireScope_OIDC/builder_may_write1604=== PAUSE TestService_RequireScope_OIDC/builder_may_write1605=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1606=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1607=== RUN TestService_RequireScope_OIDC/ops_may_admin1608=== PAUSE TestService_RequireScope_OIDC/ops_may_admin1609=== RUN TestService_RequireScope_OIDC/ops_may_not_write1610=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1611=== RUN TestService_RequireScope_OIDC/reader_may_not_write1612=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1613=== RUN TestService_RequireScope_OIDC/static_token_may_admin1614=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1615=== RUN TestService_RequireScope_OIDC/static_token_may_write1616=== PAUSE TestService_RequireScope_OIDC/static_token_may_write1617=== RUN TestService_RequireScope_OIDC/reader_may_read1618=== PAUSE TestService_RequireScope_OIDC/reader_may_read1619=== RUN TestService_RequireScope_OIDC/writer_implies_read1620=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1621=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1622=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read1623=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc1624=== PAUSE TestService_RequireScope_OIDC/reader_may_read_with_netrc1625=== RUN TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1626=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1627=== RUN TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1628=== PAUSE TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1629=== RUN TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1630=== PAUSE TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1631=== CONT TestProxyWriteTimeout/narinfo1632=== CONT TestProxyWriteTimeout/unknown_size1633=== CONT TestProxyWriteTimeout/10_GiB_nar1634=== CONT TestProxyWriteTimeout/1_GiB_nar1635--- PASS: TestProxyWriteTimeout (0.00s)1636 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1637 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1638 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1639 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)1640=== CONT TestParseSingleRange/none1641=== CONT TestParseSingleRange/open-ended1642=== CONT TestParseSingleRange/start_far_past_EOF1643=== CONT TestParseSingleRange/start_past_EOF1644=== CONT TestParseSingleRange/single_byte1645=== CONT TestParseSingleRange/suffix_exceeds_size1646=== CONT TestParseSingleRange/suffix1647=== CONT TestParseSingleRange/end_clamped_to_size1648=== CONT TestParseSingleRange/malformed_both_empty1649=== CONT TestParseSingleRange/closed1650=== CONT TestParseSingleRange/malformed_end_before_start1651=== CONT TestParseSingleRange/unknown_unit1652=== CONT TestParseSingleRange/malformed_no_dash1653=== CONT TestParseSingleRange/multi-range_ignored1654--- PASS: TestParseSingleRange (9.38s)1655 --- PASS: TestParseSingleRange/none (0.00s)1656 --- PASS: TestParseSingleRange/open-ended (0.00s)1657 --- PASS: TestParseSingleRange/start_far_past_EOF (0.00s)1658 --- PASS: TestParseSingleRange/start_past_EOF (0.00s)1659 --- PASS: TestParseSingleRange/single_byte (0.00s)1660 --- PASS: TestParseSingleRange/suffix_exceeds_size (0.00s)1661 --- PASS: TestParseSingleRange/suffix (0.00s)1662 --- PASS: TestParseSingleRange/end_clamped_to_size (0.00s)1663 --- PASS: TestParseSingleRange/malformed_both_empty (0.00s)1664 --- PASS: TestParseSingleRange/closed (0.00s)1665 --- PASS: TestParseSingleRange/malformed_end_before_start (0.00s)1666 --- PASS: TestParseSingleRange/unknown_unit (0.00s)1667 --- PASS: TestParseSingleRange/malformed_no_dash (0.00s)1668 --- PASS: TestParseSingleRange/multi-range_ignored (0.00s)1669=== CONT TestGCEndsOnShutdown/before_the_run1670--- PASS: TestService_ReadScope_PublicByDefault (0.75s)1671=== CONT TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark16722026/11/11 23:11:14 INFO lead: acquired remote=192.0.2.1:1234 generation=216732026/11/11 23:11:14 INFO Received uploads request method=POST path=/api/pending_closures16742026/11/11 23:11:14 INFO Aborted multipart uploads count=0 kept=016752026/11/11 23:11:14 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"1676--- PASS: TestService_AuthMiddleware (0.54s)1677=== CONT TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete16782026/11/11 23:11:14 INFO lead: given back to the incumbent remote=192.0.2.1:1234 generation=216792026/11/11 23:11:14 INFO lead: acquired remote=192.0.2.1:1234 generation=316802026/11/11 23:11:14 INFO Received push request method=POST path=/api/pushes16812026/11/11 23:11:14 WARN Force mode enabled - objects will be deleted immediately without grace period16822026/11/11 23:11:14 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)16832026/11/11 23:11:14 INFO Uploading bcir67hn40hd56il4kd0xfcjz0vym1dm-test-file.txt (152B)16842026/11/11 23:11:14 WARN Failed to register uploaded object key=bcir67hn40hd56il4kd0xfcjz0vym1dm.ls error="server returned 404: 404 page not found\n"16852026/11/11 23:11:14 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16862026/11/11 23:11:14 INFO Signed narinfos id=1 count=116872026/11/11 23:11:14 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"16882026/11/11 23:11:14 INFO Uploading 1 narinfos16892026/11/11 23:11:14 INFO Received complete push request method=POST path=/api/pushes/1/complete16902026/11/11 23:11:14 WARN Failed to register uploaded object key=bcir67hn40hd56il4kd0xfcjz0vym1dm.narinfo error="server returned 404: 404 page not found\n"16912026/11/11 23:11:14 INFO Completed push id=1 objects=3 duration=13.434088ms16922026/11/11 23:11:14 INFO Upload complete. (361ms)16932026/11/11 23:11:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=1 objects-deleted-after-grace-period=1 objects-failed-to-delete=016942026/11/11 23:11:14 INFO Vacuumed table table=pending_closures16952026/11/11 23:11:14 INFO Vacuumed table table=pending_objects16962026/11/11 23:11:14 INFO Vacuumed table table=multipart_uploads16972026/11/11 23:11:14 INFO Vacuumed table table=closures16982026/11/11 23:11:14 INFO Vacuumed table table=objects1699=== CONT TestPush_RejectsBadRequests/no_roots17002026/11/11 23:11:14 INFO Received push request method=POST path=/api/pushes1701=== CONT TestPush_RejectsBadRequests/bad_root17022026/11/11 23:11:14 INFO Received push request method=POST path=/api/pushes1703=== CONT TestPush_RejectsBadRequests/root_not_in_objects17042026/11/11 23:11:14 INFO Received push request method=POST path=/api/pushes1705=== CONT TestPush_RejectsBadRequests/no_objects17062026/11/11 23:11:14 INFO Received push request method=POST path=/api/pushes1707=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed1708--- PASS: TestPush_RejectsBadRequests (1.03s)1709 --- PASS: TestPush_RejectsBadRequests/no_roots (0.00s)1710 --- PASS: TestPush_RejectsBadRequests/bad_root (0.00s)1711 --- PASS: TestPush_RejectsBadRequests/root_not_in_objects (0.00s)1712 --- PASS: TestPush_RejectsBadRequests/no_objects (0.00s)1713=== NAME TestClientMultipleUploads1714 client_integration_test.go:405: Created store path 2: /build/TestClientMultipleUploads3942683860/001/store/d5almv2i2ilp4l41yd0yn75z9j0kmj5n-test-file-2.txt1715=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost17162026/11/11 23:11:14 INFO Aborted multipart uploads count=0 kept=01717=== NAME TestClientWithDependencies1718 client_integration_test.go:660: Built derivation: /build/TestClientWithDependencies1839425905/001/store/dbck54bps854w5y7vn0gwkbxyn2l2qw2-test-script1719=== NAME TestClientCADerivations1720 client_ca_test.go:139: Found 1 dependencies (including self)17212026/11/11 23:11:14 INFO Received uploads request method=POST path=/api/pending_closures17222026/11/11 23:11:14 INFO Aborted multipart uploads count=0 kept=017232026/11/11 23:11:14 INFO All 1 paths already cached1724=== NAME TestClientIntegration1725 client_integration_test.go:317: Retrieved narinfo from S3:1726 StorePath: /build/TestClientIntegration1783177883/002/store/bcir67hn40hd56il4kd0xfcjz0vym1dm-test-file.txt1727 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1728 Compression: zstd1729 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11730 NarSize: 1521731 References: 1732 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11733 client_integration_test.go:318: Retrieved .ls file from S3 (compressed size: 77 bytes)1734 client_integration_test.go:318: Decompressed .ls content (64 bytes):1735 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}17362026/11/11 23:11:14 INFO lead: released remote=192.0.2.1:1234 generation=217372026/11/11 23:11:14 INFO Received uploads request method=POST path=/api/pending_closures17382026/11/11 23:11:14 INFO lead: acquired remote=192.0.2.1:1234 generation=317392026/11/11 23:11:14 INFO lead: released remote=192.0.2.1:1234 generation=31740--- PASS: TestLeadGenerationGrowsWithEveryGrant (3.20s)1741=== CONT TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1742=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed1743=== NAME TestOrphanedObjectsGCStressTest1744 orphaned_objects_gc_test.go:503: Stress test completed successfully:1745 orphaned_objects_gc_test.go:504: - Active objects preserved: 201746 orphaned_objects_gc_test.go:505: - Objects deleted: 2101747 orphaned_objects_gc_test.go:506: - Total GC'd: 2101748--- PASS: TestOrphanedObjectsGCStressTest (15.76s)1749=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed1750--- PASS: TestPendingClosureWriteTimeout (0.00s)1751 --- PASS: TestPendingClosureWriteTimeout/empty (0.00s)1752 --- PASS: TestPendingClosureWriteTimeout/670k_objects (0.00s)1753 --- PASS: TestPendingClosureWriteTimeout/400_objects (0.00s)1754 --- PASS: TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline (1.18s)1755=== CONT TestResolveDBConnectionString/flag_wins1756=== CONT TestResolveDBConnectionString/PGHOST_allows_empty1757=== CONT TestResolveDBConnectionString/nothing_configured1758=== CONT TestResolveDBConnectionString/missing_file_is_an_error1759=== CONT TestResolveDBConnectionString/file_when_flag_empty1760--- PASS: TestResolveDBConnectionString (0.00s)1761 --- PASS: TestResolveDBConnectionString/flag_wins (0.00s)1762 --- PASS: TestResolveDBConnectionString/PGHOST_allows_empty (0.00s)1763 --- PASS: TestResolveDBConnectionString/nothing_configured (0.00s)1764 --- PASS: TestResolveDBConnectionString/missing_file_is_an_error (0.00s)1765 --- PASS: TestResolveDBConnectionString/file_when_flag_empty (0.00s)1766=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure17672026/11/11 23:11:15 INFO Received uploads request method=POST path=/1768--- PASS: TestPresentNotReportedWhileGCDeletesClosure (0.00s)1769 --- PASS: TestPresentNotReportedWhileGCDeletesClosure/fresh (0.85s)1770 --- PASS: TestPresentNotReportedWhileGCDeletesClosure/stale (1.02s)1771=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts17722026/11/11 23:11:15 INFO Received request for more parts method=POST path=/1773=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart17742026/11/11 23:11:15 INFO Received complete multipart upload request method=POST path=/1775=== CONT TestCacheConfigHandler/full_config,_no_issuer1776=== CONT TestCacheConfigHandler/no_signing_keys1777=== CONT TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1778=== CONT TestCacheConfigHandler/no_cache_url_configured1779--- PASS: TestCacheConfigHandler (0.00s)1780 --- PASS: TestCacheConfigHandler/full_config,_no_issuer (0.00s)1781 --- PASS: TestCacheConfigHandler/no_signing_keys (0.00s)1782 --- PASS: TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator (0.00s)1783 --- PASS: TestCacheConfigHandler/no_cache_url_configured (0.00s)1784=== CONT TestClientErrorHandling/InvalidStorePath17852026/11/11 23:11:15 INFO Received push request method=POST path=/api/pushes17862026/11/11 23:11:15 WARN Force mode enabled - objects will be deleted immediately without grace period17872026/11/11 23:11:15 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)17882026/11/11 23:11:15 INFO Uploading pfyz5xn6c923fri3zy6flwiwkzlx7vf5-pinned-file.txt (128B)1789=== NAME TestClientWithDependencies1790 client_integration_test.go:662: Found 1 dependencies (including self)17912026/11/11 23:11:15 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"17922026/11/11 23:11:15 WARN Failed to register uploaded object key=pfyz5xn6c923fri3zy6flwiwkzlx7vf5.ls error="server returned 404: 404 page not found\n"17932026/11/11 23:11:15 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign17942026/11/11 23:11:15 ERROR failed to remove object object=nar/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.nar.zst error="Post \"http://localhost:42089/bucket99/?delete=\": context canceled"17952026/11/11 23:11:15 INFO Signed narinfos id=1 count=117962026/11/11 23:11:15 INFO Uploading 1 narinfos1797=== CONT TestClientErrorHandling/ServerNotAvailable17982026/11/11 23:11:15 INFO Received complete push request method=POST path=/api/pushes/1/complete17992026/11/11 23:11:15 WARN Failed to register uploaded object key=pfyz5xn6c923fri3zy6flwiwkzlx7vf5.narinfo error="server returned 404: 404 page not found\n"18002026/11/11 23:11:15 INFO Completed push id=1 objects=3 duration=6.978886ms18012026/11/11 23:11:15 INFO Upload complete. (699ms)18022026/11/11 23:11:15 INFO Received uploads request method=POST path=/api/pending_closures18032026/11/11 23:11:15 INFO Received uploads request method=POST path=/api/pending_closures18042026/11/11 23:11:15 INFO Aborted multipart uploads count=0 kept=018052026/11/11 23:11:15 INFO Received complete multipart upload request method=POST path=/api/multipart/complete18062026/11/11 23:11:15 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000100000000000000000000.nar.zst upload_id=ZjU3OTAwOWMtYzllMi00YmVmLWJmMjktZWUxZjJiOWIyODJkLjQ0ZTQ4OTgxLTY1ZjMtNDU2Ny1iNmE3LTVlZjMxOTA3N2E5YXgxNzk0NDM4Njc0Nzc1ODMwMDQw parts=1018072026/11/11 23:11:15 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete18082026/11/11 23:11:15 INFO Completed upload id=1 objects=3 duration=4.458963ms18092026/11/11 23:11:15 INFO lead: released remote=192.0.2.1:1234 generation=21810--- PASS: TestLeadElectsOneAndHandsOver (3.21s)1811=== CONT TestClientErrorHandling/InvalidAuthToken18122026/11/11 23:11:15 INFO Received uploads request method=POST path=/api/pending_closures18132026/11/11 23:11:15 INFO Object in database but missing from S3 key=nar/dadb44fdadb44fdadb44fdadb44f000100000000000000000000.nar.zst18142026/11/11 23:11:15 WARN Narinfo references a NAR missing from S3, will re-upload narinfo=dadb44fdadb44fdadb44fdadb44f0001.narinfo1815=== CONT TestIsValidCachePath/narinfo1816=== CONT TestIsValidCachePath/index.html1817=== CONT TestIsValidCachePath/short_hash1818=== CONT TestIsValidCachePath/wrong_extension1819=== CONT TestIsValidCachePath/leading_slash1820=== CONT TestIsValidCachePath/empty1821=== CONT TestIsValidCachePath/random_path1822=== CONT TestIsValidCachePath/invalid_char_u1823=== CONT TestIsValidCachePath/invalid_char_e1824=== CONT TestIsValidCachePath/traversal_in_middle1825=== CONT TestIsValidCachePath/traversal_parent1826=== CONT TestIsValidCachePath/nar_uncompressed1827=== CONT TestIsValidCachePath/nix-cache-info1828=== CONT TestIsValidCachePath/realisation1829=== CONT TestIsValidCachePath/log1830=== CONT TestIsValidCachePath/ls1831=== CONT TestIsValidCachePath/nar_xz1832=== CONT TestIsValidCachePath/nar_bz21833=== CONT TestIsValidCachePath/nar_zst1834=== CONT TestIsValidCachePath/narinfo_all_nix_base32_chars1835--- PASS: TestIsValidCachePath (0.00s)1836 --- PASS: TestIsValidCachePath/narinfo (0.00s)1837 --- PASS: TestIsValidCachePath/index.html (0.00s)1838 --- PASS: TestIsValidCachePath/short_hash (0.00s)1839 --- PASS: TestIsValidCachePath/wrong_extension (0.00s)1840 --- PASS: TestIsValidCachePath/leading_slash (0.00s)1841 --- PASS: TestIsValidCachePath/empty (0.00s)1842 --- PASS: TestIsValidCachePath/random_path (0.00s)1843 --- PASS: TestIsValidCachePath/invalid_char_u (0.00s)1844 --- PASS: TestIsValidCachePath/invalid_char_e (0.00s)1845 --- PASS: TestIsValidCachePath/traversal_in_middle (0.00s)1846 --- PASS: TestIsValidCachePath/traversal_parent (0.00s)1847 --- PASS: TestIsValidCachePath/nar_uncompressed (0.00s)1848 --- PASS: TestIsValidCachePath/nix-cache-info (0.00s)1849 --- PASS: TestIsValidCachePath/realisation (0.00s)1850 --- PASS: TestIsValidCachePath/log (0.00s)1851 --- PASS: TestIsValidCachePath/ls (0.00s)1852 --- PASS: TestIsValidCachePath/nar_xz (0.00s)1853 --- PASS: TestIsValidCachePath/nar_bz2 (0.00s)1854 --- PASS: TestIsValidCachePath/nar_zst (0.00s)1855 --- PASS: TestIsValidCachePath/narinfo_all_nix_base32_chars (0.00s)1856=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info18572026/11/11 23:11:15 INFO Received uploads request method=POST path=/1858=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key18592026/11/11 23:11:15 INFO Received complete multipart upload request method=POST path=/1860=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key18612026/11/11 23:11:15 INFO Received request for more parts method=POST path=/1862=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal18632026/11/11 23:11:15 INFO Received uploads request method=POST path=/1864--- PASS: TestUploadHandlersRejectInvalidKeys (0.00s)1865 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)1866 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1867 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1868 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s)1869=== CONT TestService_AuthMiddleware_OIDC/valid_OIDC_token1870=== CONT TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1871=== CONT TestService_AuthMiddleware_OIDC/malformed_token_rejected18722026/11/11 23:11:15 WARN Authentication failed token_preview=not-a-valid-jwt token_length=15 oidc_error="no provider could verify the token (signature or issuer mismatch)" oidc_provider="" tried_providers=[test]1873=== CONT TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected18742026/11/11 23:11:15 INFO Received complete multipart upload request method=POST path=/api/multipart/complete18752026/11/11 23:11:15 WARN Authentication failed token_preview=eyJhbGciOi...Vd8Gl4FZMg token_length=701 oidc_error="no rule matched: claim \"repository_owner\" value [otherorg] not in allowed patterns [myorg]" oidc_provider=test tried_providers=[test]1876=== CONT TestIsValidUploadKey/narinfo1877=== CONT TestIsValidUploadKey/realisation_plus_in_output1878=== CONT TestIsValidUploadKey/realisation1879=== CONT TestIsValidUploadKey/build_log_equals1880=== CONT TestIsValidUploadKey/build_log_question_mark1881=== CONT TestIsValidUploadKey/build_log_plus_in_name1882=== CONT TestIsValidUploadKey/build_log_home-manager_file1883=== CONT TestIsValidUploadKey/build_log1884=== CONT TestIsValidUploadKey/listing1885=== CONT TestIsValidUploadKey/nar_plain1886=== CONT TestIsValidUploadKey/nar_xz1887=== CONT TestIsValidUploadKey/nar_zst1888=== CONT TestIsValidUploadKey/traversal_nar1889=== CONT TestIsValidUploadKey/nix-cache-info1890=== CONT TestIsValidUploadKey/traversal1891=== CONT TestIsValidUploadKey/listing_key,_narinfo_type1892=== CONT TestIsValidUploadKey/nar_key,_narinfo_type1893=== CONT TestIsValidUploadKey/narinfo_key,_nar_type1894=== CONT TestIsValidUploadKey/index.html1895=== CONT TestIsValidUploadKey/empty_key1896=== CONT TestIsValidUploadKey/unknown_type1897=== CONT TestIsValidUploadKey/absolute1898--- PASS: TestIsValidUploadKey (0.00s)1899 --- PASS: TestIsValidUploadKey/narinfo (0.00s)1900 --- PASS: TestIsValidUploadKey/realisation_plus_in_output (0.00s)1901 --- PASS: TestIsValidUploadKey/realisation (0.00s)1902 --- PASS: TestIsValidUploadKey/build_log_equals (0.00s)1903 --- PASS: TestIsValidUploadKey/build_log_question_mark (0.00s)1904 --- PASS: TestIsValidUploadKey/build_log_plus_in_name (0.00s)1905 --- PASS: TestIsValidUploadKey/build_log_home-manager_file (0.00s)1906 --- PASS: TestIsValidUploadKey/build_log (0.00s)1907 --- PASS: TestIsValidUploadKey/listing (0.00s)1908 --- PASS: TestIsValidUploadKey/nar_plain (0.00s)1909 --- PASS: TestIsValidUploadKey/nar_xz (0.00s)1910 --- PASS: TestIsValidUploadKey/nar_zst (0.00s)1911 --- PASS: TestIsValidUploadKey/traversal_nar (0.00s)1912 --- PASS: TestIsValidUploadKey/nix-cache-info (0.00s)1913 --- PASS: TestIsValidUploadKey/traversal (0.00s)1914 --- PASS: TestIsValidUploadKey/listing_key,_narinfo_type (0.00s)1915 --- PASS: TestIsValidUploadKey/nar_key,_narinfo_type (0.00s)1916 --- PASS: TestIsValidUploadKey/narinfo_key,_nar_type (0.00s)1917 --- PASS: TestIsValidUploadKey/index.html (0.00s)1918 --- PASS: TestIsValidUploadKey/empty_key (0.00s)1919 --- PASS: TestIsValidUploadKey/unknown_type (0.00s)1920 --- PASS: TestIsValidUploadKey/absolute (0.00s)1921--- PASS: TestService_AuthMiddleware_OIDC (0.90s)1922 --- PASS: TestService_AuthMiddleware_OIDC/valid_OIDC_token (0.01s)1923 --- PASS: TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured (0.00s)1924 --- PASS: TestService_AuthMiddleware_OIDC/malformed_token_rejected (0.00s)1925 --- PASS: TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected (0.00s)1926=== CONT TestService_RequireScope_OIDC/builder_may_write1927=== CONT TestService_RequireScope_OIDC/reader_may_read1928=== CONT TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1929=== CONT TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read19302026/11/11 23:11:15 WARN Authentication failed token_preview=wrong token_length=5 oidc_error="no provider could verify the token (signature or issuer mismatch)" oidc_provider="" tried_providers=[test]1931=== CONT TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1932=== CONT TestService_RequireScope_OIDC/reader_may_read_with_netrc1933=== CONT TestService_RequireScope_OIDC/anonymous_may_not_read1934=== CONT TestService_RequireScope_OIDC/writer_implies_read1935=== CONT TestService_RequireScope_OIDC/reader_may_not_write1936=== CONT TestService_RequireScope_OIDC/static_token_may_write1937=== CONT TestService_RequireScope_OIDC/static_token_may_admin1938=== CONT TestService_RequireScope_OIDC/ops_may_admin1939=== CONT TestService_RequireScope_OIDC/ops_may_not_write1940=== CONT TestService_RequireScope_OIDC/builder_may_not_admin1941--- PASS: TestService_RequireScope_OIDC (0.71s)1942 --- PASS: TestService_RequireScope_OIDC/builder_may_write (0.00s)1943 --- PASS: TestService_RequireScope_OIDC/reader_may_read (0.00s)1944 --- PASS: TestService_RequireScope_OIDC/empty_netrc_password_may_not_read (0.00s)1945 --- PASS: TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read (0.00s)1946 --- PASS: TestService_RequireScope_OIDC/static_token_may_admin_with_netrc (0.00s)1947 --- PASS: TestService_RequireScope_OIDC/reader_may_read_with_netrc (0.00s)1948 --- PASS: TestService_RequireScope_OIDC/anonymous_may_not_read (0.00s)1949 --- PASS: TestService_RequireScope_OIDC/writer_implies_read (0.00s)1950 --- PASS: TestService_RequireScope_OIDC/reader_may_not_write (0.00s)1951 --- PASS: TestService_RequireScope_OIDC/static_token_may_write (0.00s)1952 --- PASS: TestService_RequireScope_OIDC/static_token_may_admin (0.00s)1953 --- PASS: TestService_RequireScope_OIDC/ops_may_admin (0.00s)1954 --- PASS: TestService_RequireScope_OIDC/ops_may_not_write (0.00s)1955 --- PASS: TestService_RequireScope_OIDC/builder_may_not_admin (0.00s)19562026/11/11 23:11:15 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000300000000000000000000.nar.zst upload_id=ZjU3OTAwOWMtYzllMi00YmVmLWJmMjktZWUxZjJiOWIyODJkLmZjYmQwNGFmLTA5ZWUtNDk4Ni1hOWNkLTYzYTM1MDE3OTRlN3gxNzk0NDM4Njc0OTUwNzUzNjg0 parts=1019572026/11/11 23:11:15 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete19582026/11/11 23:11:15 INFO Completed upload id=1 objects=3 duration=4.104739ms19592026/11/11 23:11:15 INFO Received uploads request method=POST path=/api/pending_closures19602026/11/11 23:11:15 INFO Received complete multipart upload request method=POST path=/api/multipart/complete19612026/11/11 23:11:15 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000200000000000000000000.nar.zst upload_id=ZjU3OTAwOWMtYzllMi00YmVmLWJmMjktZWUxZjJiOWIyODJkLmYxZjNlOGMxLTBjMzgtNDM3Mi05ZGM1LTEzNjgyY2E4MGViY3gxNzk0NDM4Njc1MjU3NzM4MTQ0 parts=1019622026/11/11 23:11:15 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete19632026/11/11 23:11:15 INFO Completed upload id=1 objects=3 duration=3.312451ms19642026/11/11 23:11:15 INFO Received uploads request method=POST path=/api/pending_closures19652026/11/11 23:11:15 WARN Force mode enabled - objects will be deleted immediately without grace period19662026/11/11 23:11:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=019672026/11/11 23:11:15 INFO Vacuumed table table=pending_closures19682026/11/11 23:11:15 INFO Vacuumed table table=pending_objects19692026/11/11 23:11:15 INFO Vacuumed table table=multipart_uploads19702026/11/11 23:11:15 INFO Vacuumed table table=closures19712026/11/11 23:11:15 INFO Vacuumed table table=objects19722026/11/11 23:11:15 INFO Received push request method=POST path=/api/pushes19732026/11/11 23:11:15 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)19742026/11/11 23:11:15 INFO Uploading skp34x97905dgmqrvp7iyhkpw2w0cr6v-top (224B)19752026/11/11 23:11:15 INFO Uploading pyzzfrpbgkj1qsp0vdfkzkzppclgk58k-shared-dep (136B)19762026/11/11 23:11:15 WARN Failed to register uploaded object key=pyzzfrpbgkj1qsp0vdfkzkzppclgk58k.ls error="server returned 404: 404 page not found\n"1977--- PASS: TestForceGCDuringPushOffersSweptObject (0.00s)1978 --- PASS: TestForceGCDuringPushOffersSweptObject/before_pending_rows (0.52s)1979 --- PASS: TestForceGCDuringPushOffersSweptObject/after_presence_check (1.56s)19802026/11/11 23:11:15 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"19812026/11/11 23:11:15 WARN Failed to register uploaded object key=nar/18f9hy2qvqv2yanfrz05kb8clba8aj6iyxc3xsqhsq46f7fb8c29.nar.zst error="server returned 404: 404 page not found\n"19822026/11/11 23:11:15 WARN Failed to register uploaded object key=skp34x97905dgmqrvp7iyhkpw2w0cr6v.ls error="server returned 404: 404 page not found\n"19832026/11/11 23:11:15 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign19842026/11/11 23:11:15 INFO Signed narinfos id=1 count=219852026/11/11 23:11:15 INFO Uploading 2 narinfos19862026/11/11 23:11:15 INFO Received complete push request method=POST path=/api/pushes/1/complete19872026/11/11 23:11:15 WARN Failed to register uploaded object key=skp34x97905dgmqrvp7iyhkpw2w0cr6v.narinfo error="server returned 404: 404 page not found\n"19882026/11/11 23:11:15 WARN Failed to register uploaded object key=pyzzfrpbgkj1qsp0vdfkzkzppclgk58k.narinfo error="server returned 404: 404 page not found\n"19892026/11/11 23:11:15 INFO Completed push id=1 objects=6 duration=11.122966ms19902026/11/11 23:11:15 INFO Upload complete. (1.178s)1991=== NAME TestClientSharedPathCommittedMidPush1992 client_integration_test.go:727: Retrieved narinfo from S3:1993 StorePath: /build/TestClientSharedPathCommittedMidPush3727865505/001/store/pyzzfrpbgkj1qsp0vdfkzkzppclgk58k-shared-dep1994 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1995 Compression: zstd1996 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821997 NarSize: 1361998 References: 1999 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n2000 client_integration_test.go:727: Retrieved narinfo from S3:

loading 286 hidden lines…

22872026/11/11 23:11:20 INFO Created/updated pin name=myapp store_path=/build/TestPinProtectsFromGC3998447353/001/store/pfyz5xn6c923fri3zy6flwiwkzlx7vf5-pinned-file.txt narinfo_key=pfyz5xn6c923fri3zy6flwiwkzlx7vf5.narinfo22882026/11/11 23:11:20 INFO Starting cleanup of old closures method=DELETE path=/api/closures22892026/11/11 23:11:20 INFO Garbage collection started22902026/11/11 23:11:20 INFO Aborted multipart uploads count=0 kept=022912026/11/11 23:11:21 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=02292 client_integration_test.go:370: Objects in database after GC:2293 client_integration_test.go:370: Successfully deleted all objects with GC --force2294--- PASS: TestClientIntegration (8.80s)22952026/11/11 23:11:22 WARN Request failed, retrying attempt=1 backoff=100ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config22962026/11/11 23:11:22 WARN Force mode enabled - objects will be deleted immediately without grace period22972026/11/11 23:11:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=022982026/11/11 23:11:22 INFO Vacuumed table table=pending_closures22992026/11/11 23:11:22 INFO Vacuumed table table=pending_objects23002026/11/11 23:11:22 INFO Vacuumed table table=multipart_uploads23012026/11/11 23:11:22 INFO Vacuumed table table=closures23022026/11/11 23:11:22 INFO Vacuumed table table=objects23032026/11/11 23:11:22 WARN Request failed, retrying attempt=2 backoff=188.079618ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config2304=== NAME TestPinProtectsFromGC2305 client_integration_test.go:887: Pin successfully protected closure from garbage collection2306--- PASS: TestPinProtectsFromGC (9.40s)23072026/11/11 23:11:22 WARN Request failed, retrying attempt=3 backoff=399.540445ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config23082026/11/11 23:11:23 WARN Request failed, retrying attempt=4 backoff=864.891448ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config23092026/11/11 23:11:24 WARN Request failed, retrying attempt=5 backoff=1.543136312s error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config23102026/11/11 23:11:25 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: sending request: request failed after retries: Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused"23112026/11/11 23:11:25 WARN Request failed, retrying attempt=1 backoff=100ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config23122026/11/11 23:11:25 WARN Request failed, retrying attempt=2 backoff=205.134582ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config23132026/11/11 23:11:25 WARN Request failed, retrying attempt=3 backoff=383.117131ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config23142026/11/11 23:11:26 WARN Request failed, retrying attempt=4 backoff=874.637125ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config23152026/11/11 23:11:27 WARN Request failed, retrying attempt=5 backoff=1.739141421s error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config23162026/11/11 23:11:28 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures23172026/11/11 23:11:29 WARN Request failed, retrying attempt=2 backoff=197.016354ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures23182026/11/11 23:11:29 WARN Request failed, retrying attempt=3 backoff=361.38945ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures23192026/11/11 23:11:29 WARN Request failed, retrying attempt=4 backoff=751.837736ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures23202026/11/11 23:11:30 WARN Request failed, retrying attempt=5 backoff=1.482795445s error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures2321--- PASS: TestClientErrorHandling (0.00s)2322 --- PASS: TestClientErrorHandling/InvalidStorePath (3.54s)2323 --- PASS: TestClientErrorHandling/InvalidAuthToken (4.47s)2324 --- PASS: TestClientErrorHandling/ServerNotAvailable (16.69s)2325=== NAME TestClientCADerivations2326 client_ca_test.go:258: nix copy output: warning: you don't have Internet access; disabling some network-dependent features2327 warning: failed to create TLS context for AWS credential providers; SSO, STS WebIdentity, and ECS container authentication will be unavailable2328 error: binary cache 's3://bucket73?endpoint=http://localhost:42089&region=eu-west-1' is for Nix stores with prefix '/nix/store', not '/build/TestClientCADerivations1352219559/001/store'2329 client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 12330--- PASS: TestClientCADerivations (35.04s)2331=== RUN FuzzPushGC2332=== RUN FuzzPushGC/seed#02333=== PAUSE FuzzPushGC/seed#02334=== RUN FuzzPushGC/seed#12335=== PAUSE FuzzPushGC/seed#12336=== RUN FuzzPushGC/seed#22337=== PAUSE FuzzPushGC/seed#22338=== RUN FuzzPushGC/seed#32339=== PAUSE FuzzPushGC/seed#32340=== RUN FuzzPushGC/seed#42341=== PAUSE FuzzPushGC/seed#42342=== RUN FuzzPushGC/seed#52343=== PAUSE FuzzPushGC/seed#52344=== RUN FuzzPushGC/seed#62345=== PAUSE FuzzPushGC/seed#62346=== RUN FuzzPushGC/seed#72347=== PAUSE FuzzPushGC/seed#72348=== RUN FuzzPushGC/seed#82349=== PAUSE FuzzPushGC/seed#82350=== RUN FuzzPushGC/seed#92351=== PAUSE FuzzPushGC/seed#92352=== RUN FuzzPushGC/seed#102353=== PAUSE FuzzPushGC/seed#102354=== RUN FuzzPushGC/seed#112355=== PAUSE FuzzPushGC/seed#112356=== RUN FuzzPushGC/seed#122357=== PAUSE FuzzPushGC/seed#122358=== RUN FuzzPushGC/seed#132359=== PAUSE FuzzPushGC/seed#132360=== RUN FuzzPushGC/seed#142361=== PAUSE FuzzPushGC/seed#142362=== RUN FuzzPushGC/seed#152363=== PAUSE FuzzPushGC/seed#152364=== CONT FuzzPushGC/seed#02365=== CONT FuzzPushGC/seed#152366=== CONT FuzzPushGC/seed#142367=== CONT FuzzPushGC/seed#72368=== CONT FuzzPushGC/seed#132369=== CONT FuzzPushGC/seed#122370=== CONT FuzzPushGC/seed#112371=== CONT FuzzPushGC/seed#92372=== CONT FuzzPushGC/seed#82373=== CONT FuzzPushGC/seed#42374=== CONT FuzzPushGC/seed#52375=== CONT FuzzPushGC/seed#22376=== CONT FuzzPushGC/seed#32377=== CONT FuzzPushGC/seed#12378=== CONT FuzzPushGC/seed#62379=== CONT FuzzPushGC/seed#1023802026-11-11 23:11:47.734 UTC [201] LOG: checkpoint starting: wal23812026/11/11 23:11:49 INFO Received push request method=POST path=/23822026/11/11 23:11:49 INFO Received push request method=POST path=/23832026/11/11 23:11:49 INFO Received push request method=POST path=/23842026/11/11 23:11:49 INFO Received push request method=POST path=/23852026/11/11 23:11:49 INFO Received uploads request method=POST path=/23862026/11/11 23:11:49 INFO Received push request method=POST path=/23872026/11/11 23:11:49 INFO Received uploads request method=POST path=/23882026/11/11 23:11:49 INFO Received push request method=POST path=/23892026/11/11 23:11:49 INFO Aborted multipart uploads count=0 kept=023902026/11/11 23:11:49 INFO Received complete push request method=POST path=/23912026/11/11 23:11:49 INFO Received push request method=POST path=/23922026/11/11 23:11:49 INFO Completed push id=1 objects=8 duration=8.123237ms23932026/11/11 23:11:49 INFO Received complete push request method=POST path=/23942026/11/11 23:11:49 INFO Received push request method=POST path=/23952026/11/11 23:11:49 INFO Completed push id=1 objects=10 duration=5.798336ms23962026/11/11 23:11:50 INFO Received uploads request method=POST path=/23972026/11/11 23:11:50 INFO Received uploads request method=POST path=/23982026/11/11 23:11:50 INFO Received push request method=POST path=/23992026/11/11 23:11:50 INFO Received push request method=POST path=/24002026/11/11 23:11:50 INFO Received complete push request method=POST path=/24012026/11/11 23:11:50 INFO Received push request method=POST path=/24022026/11/11 23:11:50 INFO Completed push id=1 objects=5 duration=6.919886ms24032026/11/11 23:11:50 INFO Received complete push request method=POST path=/24042026/11/11 23:11:50 INFO Received uploads request method=POST path=/24052026/11/11 23:11:50 INFO Received push request method=POST path=/24062026/11/11 23:11:50 INFO Completed push id=1 objects=6 duration=5.233469ms24072026/11/11 23:11:50 INFO Received complete upload request method=POST path=/24082026/11/11 23:11:50 INFO Completed upload id=3 objects=5 duration=1.499934ms24092026/11/11 23:11:50 INFO Received push request method=POST path=/24102026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=024112026/11/11 23:11:50 INFO Received push request method=POST path=/24122026/11/11 23:11:50 INFO Received uploads request method=POST path=/24132026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=024142026/11/11 23:11:50 INFO Received complete push request method=POST path=/24152026/11/11 23:11:50 INFO Received complete push request method=POST path=/24162026/11/11 23:11:50 INFO Completed push id=2 objects=4 duration=1.964159ms24172026/11/11 23:11:50 INFO Completed push id=2 objects=4 duration=2.436223ms24182026/11/11 23:11:50 INFO Received push request method=POST path=/24192026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=024202026/11/11 23:11:50 INFO Received complete push request method=POST path=/24212026/11/11 23:11:50 INFO Completed push id=4 objects=3 duration=2.074179ms24222026/11/11 23:11:50 INFO Received push request method=POST path=/24232026/11/11 23:11:50 INFO Received uploads request method=POST path=/24242026/11/11 23:11:50 INFO Received push request method=POST path=/24252026/11/11 23:11:50 INFO Received complete upload request method=POST path=/24262026/11/11 23:11:50 INFO Completed upload id=3 objects=25 duration=2.08892ms24272026/11/11 23:11:50 INFO Received uploads request method=POST path=/24282026/11/11 23:11:50 INFO Received push request method=POST path=/24292026/11/11 23:11:50 INFO Received complete push request method=POST path=/24302026/11/11 23:11:50 INFO Received push request method=POST path=/24312026/11/11 23:11:50 INFO Completed push id=3 objects=8 duration=12.119736ms24322026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=024332026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=024342026/11/11 23:11:50 INFO Received uploads request method=POST path=/24352026/11/11 23:11:50 INFO Received push request method=POST path=/24362026/11/11 23:11:50 INFO Received complete push request method=POST path=/24372026/11/11 23:11:50 INFO Completed push id=4 objects=7 duration=2.262062ms24382026/11/11 23:11:50 INFO Received complete push request method=POST path=/24392026/11/11 23:11:50 INFO Completed push id=8 objects=13 duration=1.985839ms24402026/11/11 23:11:50 INFO Received uploads request method=POST path=/24412026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=024422026/11/11 23:11:50 INFO Received uploads request method=POST path=/24432026/11/11 23:11:50 INFO Received push request method=POST path=/24442026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=024452026/11/11 23:11:50 WARN Force mode enabled - objects will be deleted immediately without grace period24462026/11/11 23:11:50 WARN Force mode enabled - objects will be deleted immediately without grace period24472026/11/11 23:11:50 WARN Force mode enabled - objects will be deleted immediately without grace period24482026/11/11 23:11:50 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=024492026/11/11 23:11:50 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=024502026/11/11 23:11:50 INFO Vacuumed table table=pending_closures24512026/11/11 23:11:50 INFO Vacuumed table table=pending_objects24522026/11/11 23:11:50 INFO Vacuumed table table=multipart_uploads24532026/11/11 23:11:50 INFO Vacuumed table table=closures24542026/11/11 23:11:50 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=024552026/11/11 23:11:50 INFO Vacuumed table table=objects24562026/11/11 23:11:50 INFO Vacuumed table table=pending_closures24572026/11/11 23:11:50 INFO Vacuumed table table=pending_closures24582026/11/11 23:11:50 INFO Vacuumed table table=pending_objects24592026/11/11 23:11:50 INFO Vacuumed table table=multipart_uploads24602026/11/11 23:11:50 INFO Vacuumed table table=closures24612026/11/11 23:11:50 INFO Vacuumed table table=objects24622026/11/11 23:11:50 INFO Vacuumed table table=pending_objects24632026/11/11 23:11:50 INFO Vacuumed table table=multipart_uploads24642026/11/11 23:11:50 INFO Vacuumed table table=closures24652026/11/11 23:11:50 INFO Vacuumed table table=objects24662026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=024672026/11/11 23:11:50 WARN Force mode enabled - objects will be deleted immediately without grace period24682026/11/11 23:11:50 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=024692026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=024702026/11/11 23:11:50 INFO Vacuumed table table=pending_closures24712026/11/11 23:11:50 WARN Force mode enabled - objects will be deleted immediately without grace period24722026/11/11 23:11:50 INFO Vacuumed table table=pending_objects24732026/11/11 23:11:50 INFO Vacuumed table table=multipart_uploads24742026/11/11 23:11:50 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=024752026/11/11 23:11:50 INFO Vacuumed table table=closures24762026/11/11 23:11:50 INFO Vacuumed table table=objects24772026/11/11 23:11:50 INFO Vacuumed table table=pending_closures24782026/11/11 23:11:50 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."24792026/11/11 23:11:50 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."24802026/11/11 23:11:50 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."24812026/11/11 23:11:50 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."24822026/11/11 23:11:50 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."24832026/11/11 23:11:50 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."24842026/11/11 23:11:50 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."24852026/11/11 23:11:50 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."24862026/11/11 23:11:50 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."24872026/11/11 23:11:50 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."24882026/11/11 23:11:50 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."24892026/11/11 23:11:50 INFO Vacuumed table table=pending_objects24902026/11/11 23:11:50 INFO Vacuumed table table=multipart_uploads24912026/11/11 23:11:50 INFO Vacuumed table table=closures24922026/11/11 23:11:50 INFO Vacuumed table table=objects24932026/11/11 23:11:50 INFO Received uploads request method=POST path=/24942026/11/11 23:11:50 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=5 objects-failed-to-delete=024952026/11/11 23:11:50 INFO Received push request method=POST path=/24962026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=024972026/11/11 23:11:50 INFO Vacuumed table table=pending_closures24982026/11/11 23:11:50 WARN Force mode enabled - objects will be deleted immediately without grace period24992026/11/11 23:11:50 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=025002026/11/11 23:11:50 INFO Received push request method=POST path=/25012026/11/11 23:11:50 INFO Vacuumed table table=pending_objects25022026/11/11 23:11:50 INFO Vacuumed table table=pending_closures25032026/11/11 23:11:50 INFO Vacuumed table table=multipart_uploads25042026/11/11 23:11:50 INFO Vacuumed table table=closures25052026/11/11 23:11:50 INFO Vacuumed table table=pending_objects25062026/11/11 23:11:50 INFO Vacuumed table table=multipart_uploads25072026/11/11 23:11:50 INFO Vacuumed table table=closures25082026/11/11 23:11:50 INFO Vacuumed table table=objects25092026/11/11 23:11:50 INFO Vacuumed table table=objects25102026/11/11 23:11:50 INFO Received complete upload request method=POST path=/25112026/11/11 23:11:50 INFO Completed upload id=10 objects=4 duration=2.342963ms25122026/11/11 23:11:50 INFO Received complete push request method=POST path=/25132026/11/11 23:11:50 INFO Completed push id=8 objects=2 duration=1.847857ms25142026/11/11 23:11:50 INFO Received push request method=POST path=/25152026/11/11 23:11:50 INFO Received uploads request method=POST path=/25162026/11/11 23:11:50 INFO Received push request method=POST path=/25172026/11/11 23:11:50 INFO Received complete push request method=POST path=/25182026/11/11 23:11:50 INFO Received complete upload request method=POST path=/25192026/11/11 23:11:50 INFO Received push request method=POST path=/25202026/11/11 23:11:50 INFO Completed push id=1 objects=7 duration=6.152058ms25212026/11/11 23:11:50 INFO Received complete push request method=POST path=/25222026/11/11 23:11:50 INFO Completed upload id=2 objects=7 duration=5.423431ms25232026/11/11 23:11:50 INFO Completed push id=1 objects=5 duration=7.024747ms25242026/11/11 23:11:50 INFO Received uploads request method=POST path=/25252026/11/11 23:11:50 INFO Received push request method=POST path=/25262026/11/11 23:11:50 INFO Received push request method=POST path=/25272026/11/11 23:11:50 INFO Received complete push request method=POST path=/25282026/11/11 23:11:50 INFO Received push request method=POST path=/25292026/11/11 23:11:50 INFO Received uploads request method=POST path=/25302026/11/11 23:11:50 INFO Completed push id=1 objects=2 duration=3.290351ms25312026/11/11 23:11:50 INFO Received uploads request method=POST path=/25322026/11/11 23:11:50 INFO Received complete push request method=POST path=/25332026/11/11 23:11:50 INFO Received uploads request method=POST path=/25342026/11/11 23:11:50 INFO Completed push id=1 objects=7 duration=4.730185ms25352026/11/11 23:11:50 INFO Received uploads request method=POST path=/25362026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=025372026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=025382026/11/11 23:11:50 INFO Received complete upload request method=POST path=/25392026/11/11 23:11:50 INFO Completed upload id=2 objects=10 duration=2.417083ms25402026/11/11 23:11:50 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=025412026/11/11 23:11:50 INFO Vacuumed table table=pending_closures25422026/11/11 23:11:50 INFO Received complete upload request method=POST path=/25432026/11/11 23:11:50 INFO Vacuumed table table=pending_objects25442026/11/11 23:11:50 INFO Vacuumed table table=multipart_uploads25452026/11/11 23:11:50 INFO Completed upload id=11 objects=14 duration=1.917179ms25462026/11/11 23:11:50 INFO Vacuumed table table=closures25472026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=025482026/11/11 23:11:50 INFO Vacuumed table table=objects25492026/11/11 23:11:50 INFO Received push request method=POST path=/25502026/11/11 23:11:50 INFO Received push request method=POST path=/25512026/11/11 23:11:50 INFO Received push request method=POST path=/25522026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=025532026/11/11 23:11:50 INFO Received uploads request method=POST path=/25542026/11/11 23:11:50 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=025552026/11/11 23:11:50 INFO Vacuumed table table=pending_closures25562026/11/11 23:11:50 INFO Vacuumed table table=pending_objects25572026/11/11 23:11:50 INFO Vacuumed table table=multipart_uploads25582026/11/11 23:11:50 INFO Received complete upload request method=POST path=/25592026/11/11 23:11:50 INFO Vacuumed table table=closures25602026/11/11 23:11:50 INFO Vacuumed table table=objects25612026/11/11 23:11:50 INFO Completed upload id=5 objects=5 duration=2.09688ms25622026/11/11 23:11:50 INFO Received uploads request method=POST path=/25632026/11/11 23:11:50 INFO Received push request method=POST path=/25642026/11/11 23:11:50 INFO Received uploads request method=POST path=/25652026/11/11 23:11:50 INFO Received uploads request method=POST path=/25662026/11/11 23:11:50 INFO Received push request method=POST path=/25672026/11/11 23:11:50 INFO Received complete push request method=POST path=/25682026/11/11 23:11:50 INFO Received push request method=POST path=/25692026/11/11 23:11:50 INFO Completed push id=3 objects=5 duration=1.551855ms25702026/11/11 23:11:50 INFO Received push request method=POST path=/25712026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=025722026/11/11 23:11:50 WARN Force mode enabled - objects will be deleted immediately without grace period25732026/11/11 23:11:50 INFO Received uploads request method=POST path=/25742026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=025752026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=025762026/11/11 23:11:50 INFO Received push request method=POST path=/25772026/11/11 23:11:50 WARN Force mode enabled - objects will be deleted immediately without grace period25782026/11/11 23:11:50 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=025792026/11/11 23:11:50 INFO Vacuumed table table=pending_closures25802026/11/11 23:11:50 INFO Received push request method=POST path=/25812026/11/11 23:11:50 INFO Vacuumed table table=pending_objects25822026/11/11 23:11:50 INFO Vacuumed table table=multipart_uploads25832026/11/11 23:11:50 INFO Vacuumed table table=closures25842026/11/11 23:11:50 INFO Received complete upload request method=POST path=/25852026/11/11 23:11:50 INFO Received push request method=POST path=/25862026/11/11 23:11:50 INFO Vacuumed table table=objects25872026/11/11 23:11:50 INFO Completed upload id=10 objects=23 duration=1.821337ms25882026/11/11 23:11:50 INFO Received push request method=POST path=/25892026/11/11 23:11:50 INFO Received uploads request method=POST path=/25902026/11/11 23:11:50 INFO Received uploads request method=POST path=/25912026/11/11 23:11:50 INFO Received push request method=POST path=/25922026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=025932026/11/11 23:11:50 INFO Received uploads request method=POST path=/25942026/11/11 23:11:50 INFO Received uploads request method=POST path=/25952026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=025962026/11/11 23:11:50 INFO Received uploads request method=POST path=/25972026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=025982026/11/11 23:11:50 INFO Received complete push request method=POST path=/25992026/11/11 23:11:50 INFO Completed push id=4 objects=4 duration=1.758217ms26002026/11/11 23:11:50 INFO Received complete push request method=POST path=/26012026/11/11 23:11:50 INFO Completed push id=14 objects=3 duration=3.072249ms26022026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=026032026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=026042026/11/11 23:11:50 INFO Received complete upload request method=POST path=/26052026/11/11 23:11:50 INFO Completed upload id=11 objects=6 duration=1.187011ms26062026/11/11 23:11:50 INFO Received push request method=POST path=/26072026/11/11 23:11:50 INFO Received complete push request method=POST path=/26082026/11/11 23:11:50 INFO Received complete push request method=POST path=/26092026/11/11 23:11:50 INFO Completed push id=5 objects=6 duration=1.799397ms26102026/11/11 23:11:50 INFO Completed push id=1 objects=24 duration=7.435011ms26112026/11/11 23:11:50 INFO Received push request method=POST path=/26122026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=026132026/11/11 23:11:50 INFO Received complete push request method=POST path=/26142026/11/11 23:11:50 INFO Completed push id=12 objects=15 duration=2.439183ms26152026/11/11 23:11:50 WARN Force mode enabled - objects will be deleted immediately without grace period26162026/11/11 23:11:50 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=026172026/11/11 23:11:50 INFO Vacuumed table table=pending_closures26182026/11/11 23:11:50 INFO Vacuumed table table=pending_objects26192026/11/11 23:11:50 INFO Vacuumed table table=multipart_uploads26202026/11/11 23:11:50 INFO Vacuumed table table=closures26212026/11/11 23:11:50 INFO Vacuumed table table=objects26222026/11/11 23:11:50 INFO Received push request method=POST path=/26232026/11/11 23:11:50 INFO Received complete push request method=POST path=/26242026/11/11 23:11:50 INFO Completed push id=5 objects=3 duration=1.648736ms26252026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=026262026/11/11 23:11:50 INFO Aborted multipart uploads count=0 kept=026272026/11/11 23:11:50 INFO Received uploads request method=POST path=/26282026/11/11 23:11:50 INFO Received uploads request method=POST path=/26292026/11/11 23:11:51 INFO Received complete push request method=POST path=/26302026/11/11 23:11:51 INFO Completed push id=6 objects=26 duration=3.891697ms26312026/11/11 23:11:51 INFO Aborted multipart uploads count=0 kept=026322026/11/11 23:11:51 WARN Force mode enabled - objects will be deleted immediately without grace period26332026/11/11 23:11:51 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=026342026/11/11 23:11:51 INFO Vacuumed table table=pending_closures26352026/11/11 23:11:51 INFO Vacuumed table table=pending_objects26362026/11/11 23:11:51 INFO Vacuumed table table=multipart_uploads26372026/11/11 23:11:51 INFO Vacuumed table table=closures26382026/11/11 23:11:51 INFO Vacuumed table table=objects26392026/11/11 23:11:51 INFO Received complete upload request method=POST path=/26402026/11/11 23:11:51 INFO Completed upload id=15 objects=6 duration=930.089µs26412026/11/11 23:11:51 INFO Received uploads request method=POST path=/26422026/11/11 23:11:51 INFO Received push request method=POST path=/26432026/11/11 23:11:51 INFO Aborted multipart uploads count=0 kept=026442026/11/11 23:11:51 WARN Force mode enabled - objects will be deleted immediately without grace period26452026/11/11 23:11:51 INFO Received complete push request method=POST path=/26462026/11/11 23:11:51 INFO Completed push id=16 objects=27 duration=2.005079ms26472026/11/11 23:11:51 INFO Received uploads request method=POST path=/26482026/11/11 23:11:51 INFO Received push request method=POST path=/26492026/11/11 23:11:51 INFO Received complete push request method=POST path=/26502026/11/11 23:11:51 INFO Completed push id=18 objects=10 duration=2.387423ms26512026/11/11 23:11:51 INFO Aborted multipart uploads count=0 kept=026522026/11/11 23:11:51 WARN Force mode enabled - objects will be deleted immediately without grace period26532026/11/11 23:11:53 WARN Force mode enabled - objects will be deleted immediately without grace period26542026/11/11 23:11:53 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=026552026/11/11 23:11:53 INFO Vacuumed table table=pending_closures26562026/11/11 23:11:53 INFO Vacuumed table table=pending_objects26572026/11/11 23:11:53 INFO Vacuumed table table=multipart_uploads26582026/11/11 23:11:53 INFO Vacuumed table table=closures26592026/11/11 23:11:53 INFO Vacuumed table table=objects26602026/11/11 23:11:53 INFO Received push request method=POST path=/26612026/11/11 23:11:53 INFO Received push request method=POST path=/26622026/11/11 23:11:54 INFO Received complete push request method=POST path=/26632026/11/11 23:11:54 INFO Completed push id=3 objects=22 duration=5.906416ms26642026/11/11 23:11:54 INFO Received push request method=POST path=/26652026/11/11 23:11:54 INFO Received push request method=POST path=/26662026/11/11 23:11:54 INFO Received complete upload request method=POST path=/26672026/11/11 23:11:54 INFO Completed upload id=2 objects=2 duration=1.865018ms26682026/11/11 23:11:54 INFO Received complete push request method=POST path=/26692026/11/11 23:11:54 INFO Completed push id=6 objects=5 duration=2.288742ms26702026/11/11 23:11:54 INFO Received push request method=POST path=/26712026/11/11 23:11:54 INFO Received push request method=POST path=/26722026/11/11 23:11:54 INFO Received push request method=POST path=/26732026/11/11 23:11:54 INFO Received uploads request method=POST path=/26742026/11/11 23:11:54 INFO Received push request method=POST path=/26752026/11/11 23:11:54 INFO Aborted multipart uploads count=0 kept=026762026/11/11 23:11:54 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=026772026/11/11 23:11:54 INFO Vacuumed table table=pending_closures26782026/11/11 23:11:54 INFO Vacuumed table table=pending_objects26792026/11/11 23:11:54 INFO Vacuumed table table=multipart_uploads26802026/11/11 23:11:54 INFO Vacuumed table table=closures26812026/11/11 23:11:54 INFO Vacuumed table table=objects26822026/11/11 23:11:54 INFO Received uploads request method=POST path=/26832026/11/11 23:11:54 INFO Aborted multipart uploads count=0 kept=026842026/11/11 23:11:54 WARN Force mode enabled - objects will be deleted immediately without grace period26852026/11/11 23:11:54 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=026862026/11/11 23:11:54 INFO Vacuumed table table=pending_closures26872026/11/11 23:11:54 INFO Vacuumed table table=pending_objects26882026/11/11 23:11:54 INFO Vacuumed table table=multipart_uploads26892026/11/11 23:11:54 INFO Vacuumed table table=closures26902026/11/11 23:11:54 INFO Vacuumed table table=objects26912026/11/11 23:11:54 INFO Received push request method=POST path=/26922026/11/11 23:11:54 INFO Aborted multipart uploads count=0 kept=026932026/11/11 23:11:54 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=026942026/11/11 23:11:54 INFO Vacuumed table table=pending_closures26952026/11/11 23:11:54 INFO Vacuumed table table=pending_objects26962026/11/11 23:11:54 INFO Vacuumed table table=multipart_uploads26972026/11/11 23:11:54 INFO Vacuumed table table=closures26982026/11/11 23:11:54 INFO Vacuumed table table=objects26992026/11/11 23:11:54 INFO Received push request method=POST path=/27002026/11/11 23:11:54 INFO Aborted multipart uploads count=0 kept=027012026/11/11 23:11:54 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027022026/11/11 23:11:54 INFO Vacuumed table table=pending_closures27032026/11/11 23:11:54 INFO Vacuumed table table=pending_objects27042026/11/11 23:11:54 INFO Vacuumed table table=multipart_uploads27052026/11/11 23:11:54 INFO Vacuumed table table=closures27062026/11/11 23:11:54 INFO Vacuumed table table=objects27072026/11/11 23:11:54 INFO Received push request method=POST path=/27082026/11/11 23:11:54 INFO Received uploads request method=POST path=/27092026/11/11 23:11:54 INFO Aborted multipart uploads count=0 kept=027102026/11/11 23:11:54 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027112026/11/11 23:11:54 INFO Vacuumed table table=pending_closures27122026/11/11 23:11:54 INFO Vacuumed table table=pending_objects27132026/11/11 23:11:54 INFO Vacuumed table table=multipart_uploads27142026/11/11 23:11:54 INFO Vacuumed table table=closures27152026/11/11 23:11:54 INFO Vacuumed table table=objects27162026/11/11 23:11:54 INFO Received complete push request method=POST path=/27172026/11/11 23:11:54 INFO Completed push id=9 objects=7 duration=3.503313ms27182026/11/11 23:11:54 INFO Received complete upload request method=POST path=/27192026/11/11 23:11:54 INFO Completed upload id=10 objects=7 duration=1.576975ms27202026/11/11 23:11:54 INFO Received uploads request method=POST path=/27212026/11/11 23:11:54 INFO Received uploads request method=POST path=/27222026/11/11 23:11:54 INFO Received complete upload request method=POST path=/27232026/11/11 23:11:54 INFO Completed upload id=18 objects=4 duration=1.637816ms27242026/11/11 23:11:54 INFO Received complete push request method=POST path=/27252026/11/11 23:11:54 INFO Completed push id=1 objects=11 duration=2.261562ms27262026/11/11 23:11:54 INFO Received complete upload request method=POST path=/27272026/11/11 23:11:54 INFO Completed upload id=16 objects=3 duration=1.159311ms27282026/11/11 23:11:54 INFO Received push request method=POST path=/27292026/11/11 23:11:54 INFO Received push request method=POST path=/27302026/11/11 23:11:54 INFO Received complete push request method=POST path=/27312026/11/11 23:11:54 INFO Completed push id=4 objects=6 duration=2.142981ms27322026/11/11 23:11:54 INFO Received uploads request method=POST path=/27332026/11/11 23:11:54 INFO Received push request method=POST path=/27342026/11/11 23:11:54 INFO Aborted multipart uploads count=0 kept=027352026/11/11 23:11:54 WARN Force mode enabled - objects will be deleted immediately without grace period27362026/11/11 23:11:55 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027372026/11/11 23:11:55 INFO Vacuumed table table=pending_closures27382026/11/11 23:11:55 INFO Vacuumed table table=pending_objects27392026/11/11 23:11:55 INFO Vacuumed table table=multipart_uploads27402026/11/11 23:11:55 INFO Vacuumed table table=closures27412026/11/11 23:11:55 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027422026/11/11 23:11:55 INFO Vacuumed table table=objects27432026/11/11 23:11:55 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027442026/11/11 23:11:55 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027452026/11/11 23:11:55 INFO Vacuumed table table=pending_closures27462026/11/11 23:11:55 INFO Vacuumed table table=pending_closures27472026/11/11 23:11:55 INFO Vacuumed table table=pending_objects27482026/11/11 23:11:55 INFO Vacuumed table table=pending_closures27492026/11/11 23:11:55 INFO Vacuumed table table=multipart_uploads27502026/11/11 23:11:55 INFO Vacuumed table table=pending_objects27512026/11/11 23:11:55 INFO Vacuumed table table=multipart_uploads27522026/11/11 23:11:55 INFO Vacuumed table table=closures27532026/11/11 23:11:55 INFO Vacuumed table table=objects27542026/11/11 23:11:55 INFO Vacuumed table table=pending_objects27552026/11/11 23:11:55 INFO Vacuumed table table=closures27562026/11/11 23:11:55 INFO Vacuumed table table=multipart_uploads27572026/11/11 23:11:55 INFO Vacuumed table table=objects27582026/11/11 23:11:55 WARN Force mode enabled - objects will be deleted immediately without grace period27592026/11/11 23:11:55 INFO Vacuumed table table=closures27602026/11/11 23:11:55 INFO Vacuumed table table=objects27612026/11/11 23:11:55 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027622026/11/11 23:11:55 INFO Vacuumed table table=pending_closures27632026/11/11 23:11:55 INFO Vacuumed table table=pending_objects27642026/11/11 23:11:55 INFO Vacuumed table table=multipart_uploads27652026/11/11 23:11:55 INFO Vacuumed table table=closures27662026/11/11 23:11:55 INFO Vacuumed table table=objects27672026/11/11 23:11:55 INFO Received uploads request method=POST path=/27682026/11/11 23:11:55 INFO Aborted multipart uploads count=0 kept=027692026/11/11 23:11:55 WARN Force mode enabled - objects will be deleted immediately without grace period27702026/11/11 23:11:55 INFO Received uploads request method=POST path=/27712026/11/11 23:11:55 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027722026/11/11 23:11:55 INFO Vacuumed table table=pending_closures27732026/11/11 23:11:55 INFO Vacuumed table table=pending_objects27742026/11/11 23:11:55 INFO Vacuumed table table=multipart_uploads27752026/11/11 23:11:55 INFO Received push request method=POST path=/27762026/11/11 23:11:55 INFO Vacuumed table table=closures27772026/11/11 23:11:55 INFO Received push request method=POST path=/27782026/11/11 23:11:55 INFO Vacuumed table table=objects27792026/11/11 23:11:55 INFO Received uploads request method=POST path=/27802026/11/11 23:11:55 INFO Received push request method=POST path=/27812026/11/11 23:11:55 INFO Aborted multipart uploads count=0 kept=027822026/11/11 23:11:55 WARN Force mode enabled - objects will be deleted immediately without grace period27832026/11/11 23:11:55 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."27842026/11/11 23:11:55 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=027852026/11/11 23:11:55 INFO Vacuumed table table=pending_closures27862026/11/11 23:11:55 INFO Vacuumed table table=pending_objects27872026/11/11 23:11:55 INFO Vacuumed table table=multipart_uploads27882026/11/11 23:11:55 INFO Vacuumed table table=closures27892026/11/11 23:11:55 INFO Vacuumed table table=objects27902026/11/11 23:11:55 INFO Received uploads request method=POST path=/27912026/11/11 23:11:55 INFO Received push request method=POST path=/27922026/11/11 23:11:55 INFO Received push request method=POST path=/27932026/11/11 23:11:55 INFO Received complete upload request method=POST path=/27942026/11/11 23:11:55 INFO Received complete upload request method=POST path=/27952026/11/11 23:11:55 INFO Received complete upload request method=POST path=/27962026/11/11 23:11:55 INFO Completed upload id=3 objects=6 duration=5.341091ms27972026/11/11 23:11:55 INFO Completed upload id=17 objects=25 duration=2.876667ms27982026/11/11 23:11:55 INFO Received uploads request method=POST path=/27992026/11/11 23:11:55 INFO Completed upload id=1 objects=9 duration=6.043957ms28002026/11/11 23:11:55 INFO Aborted multipart uploads count=0 kept=028012026/11/11 23:11:55 WARN Force mode enabled - objects will be deleted immediately without grace period28022026/11/11 23:11:55 INFO Received uploads request method=POST path=/28032026/11/11 23:11:55 INFO Received push request method=POST path=/28042026/11/11 23:11:55 INFO Received push request method=POST path=/28052026/11/11 23:11:55 INFO Received push request method=POST path=/28062026/11/11 23:11:55 INFO Received complete upload request method=POST path=/28072026/11/11 23:11:55 INFO Completed upload id=19 objects=25 duration=2.705046ms28082026/11/11 23:11:55 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=028092026/11/11 23:11:55 INFO Received uploads request method=POST path=/28102026/11/11 23:11:55 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo error="Access Denied."28112026/11/11 23:11:55 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."28122026/11/11 23:11:55 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."28132026/11/11 23:11:55 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."28142026/11/11 23:11:55 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."28152026/11/11 23:11:55 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."28162026/11/11 23:11:55 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."28172026/11/11 23:11:55 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."28182026/11/11 23:11:55 ERROR failed to remove object object=nar/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.nar.zst error="Access Denied."28192026/11/11 23:11:55 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."28202026/11/11 23:11:55 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."28212026/11/11 23:11:55 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."28222026/11/11 23:11:55 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."28232026/11/11 23:11:55 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."28242026/11/11 23:11:55 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."28252026/11/11 23:11:55 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."28262026/11/11 23:11:55 INFO Received complete upload request method=POST path=/28272026/11/11 23:11:55 INFO Completed upload id=21 objects=2 duration=1.154331ms28282026/11/11 23:11:55 INFO Received complete push request method=POST path=/28292026/11/11 23:11:55 INFO Aborted multipart uploads count=0 kept=028302026/11/11 23:11:55 WARN Force mode enabled - objects will be deleted immediately without grace period28312026/11/11 23:11:55 INFO Aborted multipart uploads count=0 kept=028322026/11/11 23:11:55 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=028332026/11/11 23:11:55 INFO Vacuumed table table=pending_closures28342026/11/11 23:11:55 INFO Vacuumed table table=pending_objects28352026/11/11 23:11:55 INFO Aborted multipart uploads count=0 kept=028362026/11/11 23:11:55 INFO Vacuumed table table=multipart_uploads28372026/11/11 23:11:55 WARN Force mode enabled - objects will be deleted immediately without grace period28382026/11/11 23:11:55 INFO Vacuumed table table=closures28392026/11/11 23:11:55 INFO Vacuumed table table=objects28402026/11/11 23:11:55 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028412026/11/11 23:11:55 INFO Vacuumed table table=pending_closures28422026/11/11 23:11:55 INFO Vacuumed table table=pending_closures28432026/11/11 23:11:55 INFO Vacuumed table table=pending_objects28442026/11/11 23:11:55 INFO Completed push id=6 objects=7 duration=17.647608ms28452026/11/11 23:11:55 INFO Vacuumed table table=multipart_uploads28462026/11/11 23:11:55 INFO Vacuumed table table=closures28472026/11/11 23:11:55 INFO Vacuumed table table=objects28482026/11/11 23:11:55 INFO Vacuumed table table=pending_objects28492026/11/11 23:11:55 INFO Vacuumed table table=multipart_uploads28502026/11/11 23:11:55 INFO Vacuumed table table=closures28512026/11/11 23:11:55 INFO Vacuumed table table=objects28522026/11/11 23:11:55 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=028532026/11/11 23:11:55 INFO Received uploads request method=POST path=/28542026/11/11 23:11:55 INFO Received uploads request method=POST path=/28552026/11/11 23:11:55 INFO Received uploads request method=POST path=/28562026/11/11 23:11:55 INFO Received push request method=POST path=/28572026/11/11 23:11:55 INFO Vacuumed table table=pending_closures28582026/11/11 23:11:55 INFO Vacuumed table table=pending_objects28592026/11/11 23:11:55 INFO Received push request method=POST path=/28602026/11/11 23:11:55 INFO Vacuumed table table=multipart_uploads28612026/11/11 23:11:55 INFO Received uploads request method=POST path=/28622026/11/11 23:11:55 INFO Vacuumed table table=closures28632026/11/11 23:11:55 INFO Vacuumed table table=objects28642026/11/11 23:11:55 INFO Received complete push request method=POST path=/28652026/11/11 23:11:55 INFO Completed push id=7 objects=24 duration=4.298881ms28662026/11/11 23:11:55 INFO Received complete upload request method=POST path=/28672026/11/11 23:11:55 INFO Completed upload id=5 objects=7 duration=2.761846ms28682026/11/11 23:11:56 INFO Received uploads request method=POST path=/28692026/11/11 23:11:56 INFO Received complete push request method=POST path=/28702026/11/11 23:11:56 INFO Received push request method=POST path=/28712026/11/11 23:11:56 INFO Completed push id=2 objects=25 duration=3.076929ms28722026/11/11 23:11:56 INFO Received complete upload request method=POST path=/28732026/11/11 23:11:56 INFO Completed upload id=6 objects=9 duration=1.352913ms28742026/11/11 23:11:56 INFO Received uploads request method=POST path=/28752026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=4 objects-deleted-after-grace-period=0 objects-failed-to-delete=028762026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=5 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028772026/11/11 23:11:56 INFO Received uploads request method=POST path=/28782026/11/11 23:11:56 INFO Received uploads request method=POST path=/28792026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period28802026/11/11 23:11:56 INFO Vacuumed table table=pending_closures28812026/11/11 23:11:56 INFO Vacuumed table table=pending_closures28822026/11/11 23:11:56 INFO Vacuumed table table=pending_objects28832026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads28842026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=028852026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=028862026/11/11 23:11:56 INFO Vacuumed table table=closures28872026/11/11 23:11:56 INFO Vacuumed table table=pending_closures28882026/11/11 23:11:56 INFO Vacuumed table table=pending_closures28892026/11/11 23:11:56 INFO Vacuumed table table=pending_objects28902026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period28912026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads28922026/11/11 23:11:56 INFO Vacuumed table table=objects28932026/11/11 23:11:56 INFO Vacuumed table table=pending_objects28942026/11/11 23:11:56 INFO Vacuumed table table=pending_objects28952026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads28962026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads28972026/11/11 23:11:56 INFO Received complete push request method=POST path=/28982026/11/11 23:11:56 INFO Vacuumed table table=closures28992026/11/11 23:11:56 INFO Vacuumed table table=objects29002026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029012026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period29022026/11/11 23:11:56 INFO Completed push id=6 objects=8 duration=6.231519ms29032026/11/11 23:11:56 INFO Vacuumed table table=closures29042026/11/11 23:11:56 INFO Vacuumed table table=closures29052026/11/11 23:11:56 INFO Vacuumed table table=pending_closures29062026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029072026/11/11 23:11:56 INFO Vacuumed table table=objects29082026/11/11 23:11:56 INFO Vacuumed table table=pending_closures29092026/11/11 23:11:56 INFO Vacuumed table table=pending_objects29102026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads29112026/11/11 23:11:56 INFO Vacuumed table table=closures29122026/11/11 23:11:56 INFO Vacuumed table table=objects29132026/11/11 23:11:56 INFO Vacuumed table table=pending_objects29142026/11/11 23:11:56 INFO Vacuumed table table=objects29152026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029162026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029172026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads29182026/11/11 23:11:56 INFO Vacuumed table table=closures29192026/11/11 23:11:56 INFO Vacuumed table table=objects29202026/11/11 23:11:56 INFO Vacuumed table table=pending_closures29212026/11/11 23:11:56 INFO Vacuumed table table=pending_objects29222026/11/11 23:11:56 INFO Vacuumed table table=pending_closures29232026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads29242026/11/11 23:11:56 INFO Vacuumed table table=closures29252026/11/11 23:11:56 INFO Vacuumed table table=pending_objects29262026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads29272026/11/11 23:11:56 INFO Vacuumed table table=objects29282026/11/11 23:11:56 INFO Vacuumed table table=closures29292026/11/11 23:11:56 INFO Received complete push request method=POST path=/29302026/11/11 23:11:56 INFO Vacuumed table table=objects29312026/11/11 23:11:56 INFO Received uploads request method=POST path=/29322026/11/11 23:11:56 INFO Completed push id=2 objects=7 duration=5.581334ms29332026/11/11 23:11:56 INFO Received push request method=POST path=/29342026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=029352026/11/11 23:11:56 INFO Received uploads request method=POST path=/29362026/11/11 23:11:56 INFO Received push request method=POST path=/29372026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period29382026/11/11 23:11:56 INFO Received uploads request method=POST path=/29392026/11/11 23:11:56 INFO Received complete push request method=POST path=/29402026/11/11 23:11:56 INFO Received push request method=POST path=/29412026/11/11 23:11:56 INFO Received uploads request method=POST path=/29422026/11/11 23:11:56 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."29432026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."29442026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=029452026/11/11 23:11:56 INFO Completed push id=6 objects=5 duration=4.096079ms29462026/11/11 23:11:56 INFO Received complete upload request method=POST path=/29472026/11/11 23:11:56 INFO Completed upload id=3 objects=5 duration=1.09607ms29482026/11/11 23:11:56 INFO Received push request method=POST path=/29492026/11/11 23:11:56 INFO Received push request method=POST path=/29502026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=029512026/11/11 23:11:56 INFO Received push request method=POST path=/29522026/11/11 23:11:56 INFO Received push request method=POST path=/29532026/11/11 23:11:56 INFO Received uploads request method=POST path=/29542026/11/11 23:11:56 INFO Received push request method=POST path=/29552026/11/11 23:11:56 INFO Received push request method=POST path=/29562026/11/11 23:11:56 INFO Received uploads request method=POST path=/29572026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029582026/11/11 23:11:56 INFO Received uploads request method=POST path=/29592026/11/11 23:11:56 INFO Vacuumed table table=pending_closures29602026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=029612026/11/11 23:11:56 INFO Vacuumed table table=pending_objects29622026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads29632026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029642026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=029652026/11/11 23:11:56 INFO Vacuumed table table=closures29662026/11/11 23:11:56 INFO Received complete upload request method=POST path=/29672026/11/11 23:11:56 INFO Vacuumed table table=objects29682026/11/11 23:11:56 INFO Vacuumed table table=pending_closures29692026/11/11 23:11:56 INFO Received uploads request method=POST path=/29702026/11/11 23:11:56 INFO Vacuumed table table=pending_objects29712026/11/11 23:11:56 INFO Completed upload id=3 objects=8 duration=2.741226ms29722026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads29732026/11/11 23:11:56 INFO Vacuumed table table=closures29742026/11/11 23:11:56 INFO Received uploads request method=POST path=/29752026/11/11 23:11:56 INFO Vacuumed table table=objects29762026/11/11 23:11:56 INFO Received push request method=POST path=/29772026/11/11 23:11:56 INFO Received push request method=POST path=/29782026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=029792026/11/11 23:11:56 INFO Received push request method=POST path=/29802026/11/11 23:11:56 INFO Vacuumed table table=pending_closures29812026/11/11 23:11:56 INFO Vacuumed table table=pending_objects29822026/11/11 23:11:56 INFO Received uploads request method=POST path=/29832026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads29842026/11/11 23:11:56 INFO Vacuumed table table=closures29852026/11/11 23:11:56 INFO Received uploads request method=POST path=/29862026/11/11 23:11:56 INFO Received push request method=POST path=/29872026/11/11 23:11:56 INFO Received complete push request method=POST path=/29882026/11/11 23:11:56 INFO Vacuumed table table=objects29892026/11/11 23:11:56 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."29902026/11/11 23:11:56 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."29912026/11/11 23:11:56 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."29922026/11/11 23:11:56 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."29932026/11/11 23:11:56 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."29942026/11/11 23:11:56 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."29952026/11/11 23:11:56 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."29962026/11/11 23:11:56 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."29972026/11/11 23:11:56 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."29982026/11/11 23:11:56 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."29992026/11/11 23:11:56 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."30002026/11/11 23:11:56 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."30012026/11/11 23:11:56 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."30022026/11/11 23:11:56 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."30032026/11/11 23:11:56 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."30042026/11/11 23:11:56 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."30052026/11/11 23:11:56 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."30062026/11/11 23:11:56 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."30072026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."30082026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."30092026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."30102026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."30112026/11/11 23:11:56 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."30122026/11/11 23:11:56 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."30132026/11/11 23:11:56 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."30142026/11/11 23:11:56 INFO Completed push id=13 objects=2 duration=1.332293ms30152026/11/11 23:11:56 INFO Received complete upload request method=POST path=/30162026/11/11 23:11:56 INFO Completed upload id=7 objects=4 duration=1.11051ms30172026/11/11 23:11:56 INFO Received push request method=POST path=/30182026/11/11 23:11:56 INFO Received uploads request method=POST path=/30192026/11/11 23:11:56 INFO Received complete upload request method=POST path=/30202026/11/11 23:11:56 INFO Received uploads request method=POST path=/30212026/11/11 23:11:56 INFO Completed upload id=6 objects=7 duration=2.556844ms30222026/11/11 23:11:56 INFO Received complete push request method=POST path=/30232026/11/11 23:11:56 INFO Received complete upload request method=POST path=/30242026/11/11 23:11:56 INFO Received push request method=POST path=/30252026/11/11 23:11:56 INFO Completed upload id=27 objects=13 duration=1.06835ms30262026/11/11 23:11:56 INFO Completed push id=16 objects=4 duration=2.214901ms30272026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=030282026/11/11 23:11:56 INFO Received push request method=POST path=/30292026/11/11 23:11:56 INFO Received uploads request method=POST path=/30302026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period30312026/11/11 23:11:56 INFO Received push request method=POST path=/30322026/11/11 23:11:56 INFO Received complete push request method=POST path=/30332026/11/11 23:11:56 INFO Received uploads request method=POST path=/30342026/11/11 23:11:56 INFO Completed push id=3 objects=5 duration=4.860527ms30352026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=030362026/11/11 23:11:56 INFO Received complete upload request method=POST path=/30372026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=030382026/11/11 23:11:56 INFO Received complete push request method=POST path=/30392026/11/11 23:11:56 INFO Received push request method=POST path=/30402026/11/11 23:11:56 INFO Completed upload id=6 objects=24 duration=2.963068ms30412026/11/11 23:11:56 INFO Completed push id=2 objects=10 duration=2.305722ms30422026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period30432026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030442026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030452026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=030462026/11/11 23:11:56 INFO Vacuumed table table=pending_closures30472026/11/11 23:11:56 INFO Vacuumed table table=pending_closures30482026/11/11 23:11:56 INFO Vacuumed table table=pending_objects30492026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads30502026/11/11 23:11:56 INFO Vacuumed table table=pending_objects30512026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads30522026/11/11 23:11:56 INFO Vacuumed table table=closures30532026/11/11 23:11:56 INFO Vacuumed table table=closures30542026/11/11 23:11:56 INFO Received push request method=POST path=/30552026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=030562026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030572026/11/11 23:11:56 INFO Vacuumed table table=objects30582026/11/11 23:11:56 INFO Vacuumed table table=objects30592026/11/11 23:11:56 INFO Vacuumed table table=pending_closures30602026/11/11 23:11:56 INFO Vacuumed table table=pending_objects30612026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=030622026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads30632026/11/11 23:11:56 INFO Vacuumed table table=closures30642026/11/11 23:11:56 INFO Vacuumed table table=objects30652026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030662026/11/11 23:11:56 INFO Received complete upload request method=POST path=/30672026/11/11 23:11:56 INFO Received push request method=POST path=/30682026/11/11 23:11:56 INFO Vacuumed table table=pending_closures30692026/11/11 23:11:56 INFO Completed upload id=20 objects=5 duration=1.00341ms30702026/11/11 23:11:56 INFO Vacuumed table table=pending_objects30712026/11/11 23:11:56 INFO Received uploads request method=POST path=/30722026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads30732026/11/11 23:11:56 INFO Vacuumed table table=closures30742026/11/11 23:11:56 INFO Vacuumed table table=objects30752026/11/11 23:11:56 INFO Received uploads request method=POST path=/30762026/11/11 23:11:56 INFO Received uploads request method=POST path=/30772026/11/11 23:11:56 INFO Received complete upload request method=POST path=/30782026/11/11 23:11:56 INFO Received push request method=POST path=/30792026/11/11 23:11:56 INFO Received uploads request method=POST path=/30802026/11/11 23:11:56 INFO Received uploads request method=POST path=/30812026/11/11 23:11:56 INFO Received push request method=POST path=/30822026/11/11 23:11:56 INFO Received complete upload request method=POST path=/30832026/11/11 23:11:56 INFO Received complete push request method=POST path=/30842026/11/11 23:11:56 INFO Completed upload id=2 objects=26 duration=4.415002ms30852026/11/11 23:11:56 INFO Completed push id=4 objects=4 duration=2.255681ms30862026/11/11 23:11:56 INFO Received complete push request method=POST path=/30872026/11/11 23:11:56 INFO Received push request method=POST path=/30882026/11/11 23:11:56 INFO Completed upload id=1 objects=15 duration=6.148399ms30892026/11/11 23:11:56 INFO Completed push id=8 objects=6 duration=2.422803ms30902026/11/11 23:11:56 INFO Received push request method=POST path=/30912026/11/11 23:11:56 INFO Received complete upload request method=POST path=/30922026/11/11 23:11:56 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."30932026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."30942026/11/11 23:11:56 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."30952026/11/11 23:11:56 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."30962026/11/11 23:11:56 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."30972026/11/11 23:11:56 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."30982026/11/11 23:11:56 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."30992026/11/11 23:11:56 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."31002026/11/11 23:11:56 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."31012026/11/11 23:11:56 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."31022026/11/11 23:11:56 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."31032026/11/11 23:11:56 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."31042026/11/11 23:11:56 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."31052026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:aaaaaaaa!out.doi error="Access Denied."31062026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."31072026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."31082026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."31092026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."31102026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."31112026/11/11 23:11:56 INFO Received push request method=POST path=/31122026/11/11 23:11:56 INFO Received complete upload request method=POST path=/31132026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."31142026/11/11 23:11:56 INFO Completed upload id=4 objects=7 duration=2.195641ms31152026/11/11 23:11:56 INFO Received push request method=POST path=/31162026/11/11 23:11:56 INFO Completed upload id=9 objects=7 duration=2.366522ms31172026/11/11 23:11:56 INFO Received uploads request method=POST path=/31182026/11/11 23:11:56 INFO Received complete push request method=POST path=/31192026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=031202026/11/11 23:11:56 INFO Completed push id=8 objects=7 duration=1.663735ms31212026/11/11 23:11:56 INFO Received push request method=POST path=/31222026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period31232026/11/11 23:11:56 INFO Received uploads request method=POST path=/31242026/11/11 23:11:56 INFO Received push request method=POST path=/31252026/11/11 23:11:56 INFO Received complete push request method=POST path=/31262026/11/11 23:11:56 INFO Received complete upload request method=POST path=/31272026/11/11 23:11:56 INFO Completed upload id=9 objects=3 duration=1.251452ms31282026/11/11 23:11:56 INFO Completed push id=6 objects=6 duration=1.858678ms31292026/11/11 23:11:56 INFO Received push request method=POST path=/31302026/11/11 23:11:56 INFO Received uploads request method=POST path=/31312026/11/11 23:11:56 INFO Received push request method=POST path=/31322026/11/11 23:11:56 INFO Received uploads request method=POST path=/31332026/11/11 23:11:56 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."31342026/11/11 23:11:56 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."31352026/11/11 23:11:56 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."31362026/11/11 23:11:56 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."31372026/11/11 23:11:56 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."31382026/11/11 23:11:56 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."31392026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."31402026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."31412026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."31422026/11/11 23:11:56 INFO Received uploads request method=POST path=/31432026/11/11 23:11:56 INFO Received uploads request method=POST path=/31442026/11/11 23:11:56 INFO Received uploads request method=POST path=/31452026/11/11 23:11:56 INFO Received push request method=POST path=/31462026/11/11 23:11:56 INFO Received push request method=POST path=/31472026/11/11 23:11:56 INFO Received uploads request method=POST path=/31482026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=031492026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period31502026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=031512026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period31522026/11/11 23:11:56 INFO Received complete upload request method=POST path=/31532026/11/11 23:11:56 INFO Completed upload id=3 objects=24 duration=1.209412ms31542026/11/11 23:11:56 INFO Received push request method=POST path=/31552026/11/11 23:11:56 INFO Received uploads request method=POST path=/31562026/11/11 23:11:56 INFO Received push request method=POST path=/31572026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=031582026/11/11 23:11:56 INFO Received push request method=POST path=/31592026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=031602026/11/11 23:11:56 INFO Received complete upload request method=POST path=/31612026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=9 objects-failed-to-delete=031622026/11/11 23:11:56 INFO Received uploads request method=POST path=/31632026/11/11 23:11:56 INFO Completed upload id=9 objects=16 duration=943.289µs31642026/11/11 23:11:56 INFO Vacuumed table table=pending_closures31652026/11/11 23:11:56 INFO Received uploads request method=POST path=/31662026/11/11 23:11:56 INFO Vacuumed table table=pending_objects31672026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads31682026/11/11 23:11:56 INFO Vacuumed table table=closures31692026/11/11 23:11:56 INFO Received uploads request method=POST path=/31702026/11/11 23:11:56 INFO Vacuumed table table=objects31712026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=16 objects-failed-to-delete=031722026/11/11 23:11:56 INFO Vacuumed table table=pending_closures31732026/11/11 23:11:56 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."31742026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."31752026/11/11 23:11:56 INFO Vacuumed table table=pending_objects31762026/11/11 23:11:56 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."31772026/11/11 23:11:56 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."31782026/11/11 23:11:56 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."31792026/11/11 23:11:56 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."31802026/11/11 23:11:56 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."31812026/11/11 23:11:56 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."31822026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."31832026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads31842026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=031852026/11/11 23:11:56 INFO Received push request method=POST path=/31862026/11/11 23:11:56 INFO Vacuumed table table=closures31872026/11/11 23:11:56 INFO Vacuumed table table=objects31882026/11/11 23:11:56 INFO Received push request method=POST path=/31892026/11/11 23:11:56 INFO Received push request method=POST path=/31902026/11/11 23:11:56 INFO Received complete upload request method=POST path=/31912026/11/11 23:11:56 INFO Completed upload id=29 objects=4 duration=1.276312ms31922026/11/11 23:11:56 INFO Received complete push request method=POST path=/31932026/11/11 23:11:56 INFO Completed push id=7 objects=27 duration=1.405734ms31942026/11/11 23:11:56 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."31952026/11/11 23:11:56 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."31962026/11/11 23:11:56 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."31972026/11/11 23:11:56 INFO Received complete upload request method=POST path=/31982026/11/11 23:11:56 INFO Received push request method=POST path=/31992026/11/11 23:11:56 INFO Received uploads request method=POST path=/32002026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=032012026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=032022026/11/11 23:11:56 INFO Completed upload id=23 objects=18 duration=2.11126ms32032026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period32042026/11/11 23:11:56 INFO Received uploads request method=POST path=/32052026/11/11 23:11:56 INFO Received push request method=POST path=/32062026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032072026/11/11 23:11:56 INFO Received push request method=POST path=/32082026/11/11 23:11:56 INFO Vacuumed table table=pending_closures32092026/11/11 23:11:56 INFO Received uploads request method=POST path=/32102026/11/11 23:11:56 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."32112026/11/11 23:11:56 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."32122026/11/11 23:11:56 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."32132026/11/11 23:11:56 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."32142026/11/11 23:11:56 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."32152026/11/11 23:11:56 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."32162026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."32172026/11/11 23:11:56 INFO Vacuumed table table=pending_objects32182026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads32192026/11/11 23:11:56 INFO Vacuumed table table=closures32202026/11/11 23:11:56 INFO Vacuumed table table=objects32212026/11/11 23:11:56 INFO Received uploads request method=POST path=/32222026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=032232026/11/11 23:11:56 INFO Received push request method=POST path=/32242026/11/11 23:11:56 INFO Received uploads request method=POST path=/32252026/11/11 23:11:56 INFO Received complete upload request method=POST path=/32262026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period32272026/11/11 23:11:56 INFO Completed upload id=10 objects=6 duration=2.042859ms32282026/11/11 23:11:56 INFO Received complete upload request method=POST path=/32292026/11/11 23:11:56 INFO Received push request method=POST path=/32302026/11/11 23:11:56 INFO Completed upload id=3 objects=9 duration=1.241932ms32312026/11/11 23:11:56 INFO Received complete push request method=POST path=/32322026/11/11 23:11:56 INFO Received uploads request method=POST path=/32332026/11/11 23:11:56 INFO Completed push id=10 objects=3 duration=1.184331ms32342026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032352026/11/11 23:11:56 INFO Vacuumed table table=pending_closures32362026/11/11 23:11:56 INFO Received uploads request method=POST path=/32372026/11/11 23:11:56 INFO Vacuumed table table=pending_objects32382026/11/11 23:11:56 INFO Received uploads request method=POST path=/32392026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads32402026/11/11 23:11:56 INFO Vacuumed table table=closures32412026/11/11 23:11:56 INFO Received complete push request method=POST path=/32422026/11/11 23:11:56 INFO Vacuumed table table=objects32432026/11/11 23:11:56 INFO Received complete push request method=POST path=/32442026/11/11 23:11:56 INFO Completed push id=12 objects=3 duration=1.377613ms32452026/11/11 23:11:56 INFO Completed push id=7 objects=20 duration=2.549285ms32462026/11/11 23:11:56 INFO Received push request method=POST path=/32472026/11/11 23:11:56 INFO Received push request method=POST path=/32482026/11/11 23:11:56 INFO Received uploads request method=POST path=/32492026/11/11 23:11:56 INFO Received push request method=POST path=/32502026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=032512026/11/11 23:11:56 INFO Received uploads request method=POST path=/32522026/11/11 23:11:56 INFO Received uploads request method=POST path=/32532026/11/11 23:11:56 INFO Received complete push request method=POST path=/32542026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=032552026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=032562026/11/11 23:11:56 INFO Received complete push request method=POST path=/32572026/11/11 23:11:56 INFO Received push request method=POST path=/32582026/11/11 23:11:56 INFO Vacuumed table table=pending_closures32592026/11/11 23:11:56 INFO Completed push id=10 objects=2 duration=1.895798ms32602026/11/11 23:11:56 INFO Completed push id=11 objects=6 duration=1.603815ms32612026/11/11 23:11:56 INFO Received uploads request method=POST path=/32622026/11/11 23:11:56 INFO Vacuumed table table=pending_objects32632026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads32642026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032652026/11/11 23:11:56 INFO Vacuumed table table=pending_closures32662026/11/11 23:11:56 INFO Vacuumed table table=closures32672026/11/11 23:11:56 INFO Vacuumed table table=pending_objects32682026/11/11 23:11:56 INFO Vacuumed table table=objects32692026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads32702026/11/11 23:11:56 INFO Vacuumed table table=closures32712026/11/11 23:11:56 INFO Vacuumed table table=objects32722026/11/11 23:11:56 INFO Received complete upload request method=POST path=/32732026/11/11 23:11:56 INFO Completed upload id=22 objects=22 duration=991.609µs32742026/11/11 23:11:56 INFO Received uploads request method=POST path=/32752026/11/11 23:11:56 INFO Received complete push request method=POST path=/32762026/11/11 23:11:56 INFO Received push request method=POST path=/32772026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=032782026/11/11 23:11:56 INFO Completed push id=5 objects=6 duration=1.706976ms32792026/11/11 23:11:56 INFO Received push request method=POST path=/32802026/11/11 23:11:56 INFO Received push request method=POST path=/32812026/11/11 23:11:56 INFO Received uploads request method=POST path=/32822026/11/11 23:11:56 INFO Received push request method=POST path=/32832026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032842026/11/11 23:11:56 INFO Vacuumed table table=pending_closures32852026/11/11 23:11:56 INFO Received complete upload request method=POST path=/32862026/11/11 23:11:56 INFO Vacuumed table table=pending_objects32872026/11/11 23:11:56 INFO Received complete push request method=POST path=/32882026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads32892026/11/11 23:11:56 INFO Completed upload id=12 objects=21 duration=1.890358ms32902026/11/11 23:11:56 INFO Vacuumed table table=closures32912026/11/11 23:11:56 INFO Received push request method=POST path=/32922026/11/11 23:11:56 INFO Completed push id=20 objects=13 duration=1.196812ms32932026/11/11 23:11:56 INFO Received push request method=POST path=/32942026/11/11 23:11:56 INFO Vacuumed table table=objects32952026/11/11 23:11:56 INFO Received push request method=POST path=/32962026/11/11 23:11:56 INFO Received uploads request method=POST path=/32972026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=032982026/11/11 23:11:56 INFO Received uploads request method=POST path=/32992026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period33002026/11/11 23:11:56 INFO Received complete push request method=POST path=/33012026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=033022026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=033032026/11/11 23:11:56 INFO Completed push id=8 objects=6 duration=2.921288ms33042026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period33052026/11/11 23:11:56 INFO Received push request method=POST path=/33062026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033072026/11/11 23:11:56 INFO Vacuumed table table=pending_closures33082026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."33092026/11/11 23:11:56 INFO Vacuumed table table=pending_objects33102026/11/11 23:11:56 INFO Received push request method=POST path=/33112026/11/11 23:11:56 INFO Received push request method=POST path=/33122026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads33132026/11/11 23:11:56 INFO Received complete push request method=POST path=/33142026/11/11 23:11:56 INFO Vacuumed table table=closures33152026/11/11 23:11:56 INFO Completed push id=22 objects=4 duration=1.04277ms33162026/11/11 23:11:56 INFO Vacuumed table table=objects33172026/11/11 23:11:56 INFO Received complete push request method=POST path=/33182026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=033192026/11/11 23:11:56 INFO Completed push id=26 objects=5 duration=3.14841ms33202026/11/11 23:11:56 INFO Received complete push request method=POST path=/33212026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period33222026/11/11 23:11:56 INFO Completed push id=5 objects=3 duration=9.171307ms33232026/11/11 23:11:56 INFO Received complete push request method=POST path=/33242026/11/11 23:11:56 INFO Received uploads request method=POST path=/33252026/11/11 23:11:56 INFO Completed push id=12 objects=7 duration=2.10212ms33262026/11/11 23:11:56 INFO Received push request method=POST path=/33272026/11/11 23:11:56 INFO Received push request method=POST path=/33282026/11/11 23:11:56 INFO Received push request method=POST path=/33292026/11/11 23:11:56 INFO Received complete push request method=POST path=/33302026/11/11 23:11:56 INFO Completed push id=13 objects=23 duration=2.15402ms33312026/11/11 23:11:56 INFO Received complete upload request method=POST path=/33322026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=4 objects-failed-to-delete=033332026/11/11 23:11:56 INFO Completed upload id=24 objects=14 duration=1.019669ms33342026/11/11 23:11:56 INFO Received uploads request method=POST path=/33352026/11/11 23:11:56 INFO Vacuumed table table=pending_closures33362026/11/11 23:11:56 INFO Received complete upload request method=POST path=/33372026/11/11 23:11:56 INFO Vacuumed table table=pending_objects33382026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads33392026/11/11 23:11:56 INFO Received complete upload request method=POST path=/33402026/11/11 23:11:56 INFO Received complete push request method=POST path=/33412026/11/11 23:11:56 INFO Vacuumed table table=closures33422026/11/11 23:11:56 INFO Completed upload id=23 objects=7 duration=1.293152ms33432026/11/11 23:11:56 INFO Completed upload id=16 objects=23 duration=1.264932ms33442026/11/11 23:11:56 INFO Vacuumed table table=objects33452026/11/11 23:11:56 INFO Received uploads request method=POST path=/33462026/11/11 23:11:56 INFO Completed push id=13 objects=12 duration=3.299591ms33472026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=033482026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=033492026/11/11 23:11:56 INFO Received complete upload request method=POST path=/33502026/11/11 23:11:56 INFO Completed upload id=10 objects=6 duration=2.13286ms33512026/11/11 23:11:56 INFO Received complete upload request method=POST path=/33522026/11/11 23:11:56 INFO Completed upload id=15 objects=6 duration=866.389µs33532026/11/11 23:11:56 INFO Received complete push request method=POST path=/33542026/11/11 23:11:56 INFO Completed push id=19 objects=3 duration=1.818377ms33552026/11/11 23:11:56 INFO Received complete push request method=POST path=/33562026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033572026/11/11 23:11:56 INFO Completed push id=24 objects=5 duration=1.08837ms33582026/11/11 23:11:56 INFO Vacuumed table table=pending_closures33592026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=033602026/11/11 23:11:56 INFO Vacuumed table table=pending_objects33612026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads33622026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=033632026/11/11 23:11:56 INFO Vacuumed table table=closures33642026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=033652026/11/11 23:11:56 INFO Vacuumed table table=pending_closures33662026/11/11 23:11:56 INFO Vacuumed table table=objects33672026/11/11 23:11:56 INFO Vacuumed table table=pending_closures33682026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period33692026/11/11 23:11:56 INFO Vacuumed table table=pending_objects33702026/11/11 23:11:56 INFO Vacuumed table table=pending_objects33712026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads33722026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=033732026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads33742026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033752026/11/11 23:11:56 INFO Vacuumed table table=closures33762026/11/11 23:11:56 INFO Vacuumed table table=closures33772026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=033782026/11/11 23:11:56 INFO Vacuumed table table=pending_closures33792026/11/11 23:11:56 INFO Vacuumed table table=objects33802026/11/11 23:11:56 INFO Vacuumed table table=objects33812026/11/11 23:11:56 INFO Vacuumed table table=pending_objects33822026/11/11 23:11:56 INFO Received push request method=POST path=/33832026/11/11 23:11:56 INFO Received push request method=POST path=/33842026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads33852026/11/11 23:11:56 INFO Received uploads request method=POST path=/33862026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period33872026/11/11 23:11:56 INFO Vacuumed table table=closures33882026/11/11 23:11:56 INFO Vacuumed table table=objects33892026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033902026/11/11 23:11:56 INFO Vacuumed table table=pending_closures33912026/11/11 23:11:56 INFO Received uploads request method=POST path=/33922026/11/11 23:11:56 INFO Vacuumed table table=pending_objects33932026/11/11 23:11:56 INFO Received complete upload request method=POST path=/33942026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=033952026/11/11 23:11:56 INFO Received uploads request method=POST path=/33962026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads33972026/11/11 23:11:56 INFO Vacuumed table table=pending_closures33982026/11/11 23:11:56 INFO Completed upload id=25 objects=2 duration=3.005909ms33992026/11/11 23:11:56 INFO Vacuumed table table=closures34002026/11/11 23:11:56 INFO Vacuumed table table=objects34012026/11/11 23:11:56 INFO Vacuumed table table=pending_objects34022026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads34032026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=034042026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=034052026/11/11 23:11:56 INFO Vacuumed table table=closures34062026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034072026/11/11 23:11:56 INFO Vacuumed table table=objects34082026/11/11 23:11:56 INFO Vacuumed table table=pending_closures34092026/11/11 23:11:56 INFO Received complete push request method=POST path=/34102026/11/11 23:11:56 INFO Vacuumed table table=pending_objects34112026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads34122026/11/11 23:11:56 INFO Vacuumed table table=closures34132026/11/11 23:11:56 INFO Completed push id=8 objects=6 duration=1.629055ms34142026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=034152026/11/11 23:11:56 INFO Vacuumed table table=objects34162026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period34172026/11/11 23:11:56 INFO Received complete upload request method=POST path=/34182026/11/11 23:11:56 INFO Received uploads request method=POST path=/34192026/11/11 23:11:56 INFO Completed upload id=17 objects=7 duration=1.210272ms34202026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034212026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=034222026/11/11 23:11:56 INFO Received push request method=POST path=/34232026/11/11 23:11:56 INFO Received complete upload request method=POST path=/34242026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=034252026/11/11 23:11:56 INFO Vacuumed table table=pending_closures34262026/11/11 23:11:56 INFO Completed upload id=26 objects=20 duration=1.03283ms34272026/11/11 23:11:56 INFO Vacuumed table table=pending_closures34282026/11/11 23:11:56 INFO Vacuumed table table=pending_objects34292026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads34302026/11/11 23:11:56 INFO Vacuumed table table=pending_objects34312026/11/11 23:11:56 INFO Vacuumed table table=closures34322026/11/11 23:11:56 INFO Received uploads request method=POST path=/34332026/11/11 23:11:56 INFO Vacuumed table table=objects34342026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads34352026/11/11 23:11:56 INFO Vacuumed table table=closures34362026/11/11 23:11:56 INFO Vacuumed table table=objects34372026/11/11 23:11:56 INFO Received push request method=POST path=/34382026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=034392026/11/11 23:11:56 INFO Vacuumed table table=pending_closures34402026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=034412026/11/11 23:11:56 INFO Vacuumed table table=pending_objects34422026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads34432026/11/11 23:11:56 INFO Vacuumed table table=closures34442026/11/11 23:11:56 INFO Vacuumed table table=objects34452026/11/11 23:11:56 INFO Received push request method=POST path=/34462026/11/11 23:11:56 INFO Received uploads request method=POST path=/34472026/11/11 23:11:56 INFO Received push request method=POST path=/34482026/11/11 23:11:56 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."34492026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."34502026/11/11 23:11:56 INFO Received push request method=POST path=/34512026/11/11 23:11:56 INFO Received complete push request method=POST path=/34522026/11/11 23:11:56 INFO Received push request method=POST path=/34532026/11/11 23:11:56 INFO Completed push id=28 objects=7 duration=1.04317ms34542026/11/11 23:11:56 INFO Received push request method=POST path=/34552026/11/11 23:11:56 INFO Received push request method=POST path=/34562026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=034572026/11/11 23:11:56 INFO Received push request method=POST path=/34582026/11/11 23:11:56 INFO Received uploads request method=POST path=/34592026/11/11 23:11:56 INFO Received push request method=POST path=/34602026/11/11 23:11:56 INFO Received uploads request method=POST path=/34612026/11/11 23:11:56 INFO Received uploads request method=POST path=/34622026/11/11 23:11:56 INFO Received uploads request method=POST path=/34632026/11/11 23:11:56 INFO Received push request method=POST path=/34642026/11/11 23:11:56 INFO Received uploads request method=POST path=/34652026/11/11 23:11:56 INFO Received complete upload request method=POST path=/34662026/11/11 23:11:56 INFO Completed upload id=30 objects=6 duration=1.165571ms34672026/11/11 23:11:56 INFO Received complete push request method=POST path=/34682026/11/11 23:11:56 INFO Received uploads request method=POST path=/34692026/11/11 23:11:56 INFO Received uploads request method=POST path=/34702026/11/11 23:11:56 INFO Received complete push request method=POST path=/34712026/11/11 23:11:56 INFO Completed push id=31 objects=14 duration=2.490344ms34722026/11/11 23:11:56 INFO Received complete upload request method=POST path=/34732026/11/11 23:11:56 INFO Completed push id=5 objects=9 duration=5.143369ms34742026/11/11 23:11:56 INFO Completed upload id=25 objects=4 duration=1.385073ms34752026/11/11 23:11:56 INFO Received uploads request method=POST path=/34762026/11/11 23:11:56 INFO Received uploads request method=POST path=/34772026/11/11 23:11:56 INFO Received push request method=POST path=/34782026/11/11 23:11:56 INFO Received complete upload request method=POST path=/34792026/11/11 23:11:56 INFO Completed upload id=14 objects=6 duration=844.608µs34802026/11/11 23:11:56 INFO Received complete push request method=POST path=/34812026/11/11 23:11:56 INFO Completed push id=31 objects=5 duration=902.749µs34822026/11/11 23:11:56 INFO Received push request method=POST path=/34832026/11/11 23:11:56 INFO Received uploads request method=POST path=/34842026/11/11 23:11:56 INFO Received uploads request method=POST path=/34852026/11/11 23:11:56 INFO Received complete upload request method=POST path=/34862026/11/11 23:11:56 INFO Received uploads request method=POST path=/34872026/11/11 23:11:56 INFO Completed upload id=11 objects=6 duration=1.834357ms34882026/11/11 23:11:56 INFO Received uploads request method=POST path=/34892026/11/11 23:11:56 INFO Received push request method=POST path=/34902026/11/11 23:11:56 INFO Received push request method=POST path=/34912026/11/11 23:11:56 INFO Received push request method=POST path=/34922026/11/11 23:11:56 INFO Received push request method=POST path=/34932026/11/11 23:11:56 INFO Received complete upload request method=POST path=/34942026/11/11 23:11:56 INFO Completed upload id=4 objects=14 duration=2.259621ms34952026/11/11 23:11:56 INFO Received complete upload request method=POST path=/34962026/11/11 23:11:56 INFO Received uploads request method=POST path=/34972026/11/11 23:11:56 INFO Received push request method=POST path=/34982026/11/11 23:11:56 INFO Received uploads request method=POST path=/34992026/11/11 23:11:56 INFO Received push request method=POST path=/35002026/11/11 23:11:56 INFO Completed upload id=17 objects=6 duration=2.870688ms35012026/11/11 23:11:56 INFO Received complete push request method=POST path=/35022026/11/11 23:11:56 INFO Completed push id=32 objects=26 duration=1.352653ms35032026/11/11 23:11:56 INFO Received uploads request method=POST path=/35042026/11/11 23:11:56 INFO Received complete upload request method=POST path=/35052026/11/11 23:11:56 INFO Completed upload id=15 objects=4 duration=1.965299ms35062026/11/11 23:11:56 INFO Received uploads request method=POST path=/35072026/11/11 23:11:56 INFO Received complete upload request method=POST path=/35082026/11/11 23:11:56 INFO Received push request method=POST path=/35092026/11/11 23:11:56 INFO Completed upload id=7 objects=3 duration=765.227µs35102026/11/11 23:11:56 INFO Received complete upload request method=POST path=/35112026/11/11 23:11:56 INFO Completed upload id=10 objects=17 duration=1.256932ms35122026/11/11 23:11:56 INFO Received uploads request method=POST path=/35132026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=035142026/11/11 23:11:56 INFO Received complete upload request method=POST path=/35152026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period35162026/11/11 23:11:56 INFO Completed upload id=33 objects=2 duration=693.966µs35172026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=035182026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035192026/11/11 23:11:56 INFO Vacuumed table table=pending_closures35202026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period35212026/11/11 23:11:56 INFO Received complete push request method=POST path=/35222026/11/11 23:11:56 INFO Received push request method=POST path=/35232026/11/11 23:11:56 INFO Vacuumed table table=pending_objects35242026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035252026/11/11 23:11:56 INFO Completed push id=20 objects=8 duration=1.07053ms35262026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads35272026/11/11 23:11:56 INFO Vacuumed table table=closures35282026/11/11 23:11:56 INFO Received push request method=POST path=/35292026/11/11 23:11:56 INFO Received push request method=POST path=/35302026/11/11 23:11:56 INFO Received uploads request method=POST path=/35312026/11/11 23:11:56 INFO Received push request method=POST path=/35322026/11/11 23:11:56 INFO Vacuumed table table=objects35332026/11/11 23:11:56 INFO Vacuumed table table=pending_closures35342026/11/11 23:11:56 INFO Received push request method=POST path=/35352026/11/11 23:11:56 INFO Received complete upload request method=POST path=/35362026/11/11 23:11:56 INFO Vacuumed table table=pending_objects35372026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads35382026/11/11 23:11:56 INFO Completed upload id=34 objects=5 duration=978.629µs35392026/11/11 23:11:56 INFO Vacuumed table table=closures35402026/11/11 23:11:56 INFO Vacuumed table table=objects35412026/11/11 23:11:56 INFO Received complete push request method=POST path=/35422026/11/11 23:11:56 INFO Received uploads request method=POST path=/35432026/11/11 23:11:56 INFO Completed push id=18 objects=23 duration=2.686206ms35442026/11/11 23:11:56 INFO Received uploads request method=POST path=/35452026/11/11 23:11:56 INFO Received complete push request method=POST path=/35462026/11/11 23:11:56 INFO Received push request method=POST path=/35472026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=035482026/11/11 23:11:56 INFO Completed push id=9 objects=5 duration=2.226681ms35492026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=035502026/11/11 23:11:56 INFO Received push request method=POST path=/35512026/11/11 23:11:56 INFO Received uploads request method=POST path=/35522026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035532026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period35542026/11/11 23:11:56 INFO Vacuumed table table=pending_closures35552026/11/11 23:11:56 INFO Received complete push request method=POST path=/35562026/11/11 23:11:56 INFO Received complete upload request method=POST path=/35572026/11/11 23:11:56 INFO Vacuumed table table=pending_objects35582026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads35592026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035602026/11/11 23:11:56 INFO Completed push id=7 objects=7 duration=1.803277ms35612026/11/11 23:11:56 INFO Completed upload id=35 objects=11 duration=1.811437ms35622026/11/11 23:11:56 INFO Vacuumed table table=closures35632026/11/11 23:11:56 INFO Vacuumed table table=pending_closures35642026/11/11 23:11:56 INFO Vacuumed table table=objects35652026/11/11 23:11:56 INFO Vacuumed table table=pending_objects35662026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads35672026/11/11 23:11:56 INFO Received uploads request method=POST path=/35682026/11/11 23:11:56 INFO Vacuumed table table=closures35692026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=035702026/11/11 23:11:56 INFO Vacuumed table table=objects35712026/11/11 23:11:56 INFO Received push request method=POST path=/35722026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period35732026/11/11 23:11:56 INFO Received complete upload request method=POST path=/35742026/11/11 23:11:56 INFO Completed upload id=16 objects=3 duration=927.129µs35752026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035762026/11/11 23:11:56 INFO Vacuumed table table=pending_closures35772026/11/11 23:11:56 INFO Received push request method=POST path=/35782026/11/11 23:11:56 INFO Received uploads request method=POST path=/35792026/11/11 23:11:56 INFO Vacuumed table table=pending_objects35802026/11/11 23:11:56 INFO Received push request method=POST path=/35812026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads35822026/11/11 23:11:56 INFO Vacuumed table table=closures35832026/11/11 23:11:56 INFO Received push request method=POST path=/35842026/11/11 23:11:56 INFO Received complete push request method=POST path=/35852026/11/11 23:11:56 INFO Received complete upload request method=POST path=/35862026/11/11 23:11:56 INFO Vacuumed table table=objects35872026/11/11 23:11:56 INFO Completed push id=12 objects=6 duration=1.06777ms35882026/11/11 23:11:56 INFO Completed upload id=36 objects=3 duration=642.366µs35892026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=035902026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period35912026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=035922026/11/11 23:11:56 INFO Received complete push request method=POST path=/35932026/11/11 23:11:56 INFO Completed push id=11 objects=9 duration=1.208852ms35942026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035952026/11/11 23:11:56 INFO Received push request method=POST path=/35962026/11/11 23:11:56 INFO Received complete push request method=POST path=/35972026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=035982026/11/11 23:11:56 INFO Received push request method=POST path=/35992026/11/11 23:11:56 INFO Vacuumed table table=pending_closures36002026/11/11 23:11:56 INFO Vacuumed table table=pending_objects36012026/11/11 23:11:56 INFO Completed push id=14 objects=6 duration=1.817197ms36022026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads36032026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=036042026/11/11 23:11:56 INFO Received complete push request method=POST path=/36052026/11/11 23:11:56 INFO Vacuumed table table=closures36062026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period36072026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036082026/11/11 23:11:56 INFO Vacuumed table table=objects36092026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036102026/11/11 23:11:56 INFO Vacuumed table table=pending_closures36112026/11/11 23:11:56 INFO Completed push id=19 objects=8 duration=2.954008ms36122026/11/11 23:11:56 INFO Vacuumed table table=pending_objects36132026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads36142026/11/11 23:11:56 INFO Vacuumed table table=pending_closures36152026/11/11 23:11:56 INFO Vacuumed table table=closures36162026/11/11 23:11:56 INFO Vacuumed table table=objects36172026/11/11 23:11:56 INFO Received push request method=POST path=/36182026/11/11 23:11:56 INFO Vacuumed table table=pending_objects36192026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads36202026/11/11 23:11:56 INFO Vacuumed table table=closures36212026/11/11 23:11:56 INFO Vacuumed table table=objects36222026/11/11 23:11:56 INFO Received push request method=POST path=/36232026/11/11 23:11:56 INFO Received complete push request method=POST path=/36242026/11/11 23:11:56 INFO Completed push id=27 objects=11 duration=932.109µs36252026/11/11 23:11:56 INFO Received push request method=POST path=/36262026/11/11 23:11:56 INFO Received push request method=POST path=/36272026/11/11 23:11:56 INFO Received uploads request method=POST path=/36282026/11/11 23:11:56 INFO Received uploads request method=POST path=/36292026/11/11 23:11:56 INFO Received uploads request method=POST path=/36302026/11/11 23:11:56 INFO Received complete push request method=POST path=/36312026/11/11 23:11:56 INFO Completed push id=9 objects=4 duration=2.441603ms36322026/11/11 23:11:56 INFO Received complete push request method=POST path=/36332026/11/11 23:11:56 INFO Received complete push request method=POST path=/36342026/11/11 23:11:56 INFO Received complete upload request method=POST path=/36352026/11/11 23:11:56 INFO Completed push id=5 objects=8 duration=3.501233ms36362026/11/11 23:11:56 INFO Completed upload id=29 objects=26 duration=1.314533ms36372026/11/11 23:11:56 INFO Received push request method=POST path=/36382026/11/11 23:11:56 INFO Completed push id=13 objects=22 duration=1.528175ms36392026/11/11 23:11:56 INFO Received push request method=POST path=/36402026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=036412026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=036422026/11/11 23:11:56 INFO Received push request method=POST path=/36432026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period36442026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036452026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036462026/11/11 23:11:56 INFO Vacuumed table table=pending_closures36472026/11/11 23:11:56 INFO Vacuumed table table=pending_closures36482026/11/11 23:11:56 INFO Vacuumed table table=pending_objects36492026/11/11 23:11:56 INFO Vacuumed table table=pending_objects36502026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads36512026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads36522026/11/11 23:11:56 INFO Vacuumed table table=closures36532026/11/11 23:11:56 INFO Vacuumed table table=closures36542026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=036552026/11/11 23:11:56 INFO Vacuumed table table=objects36562026/11/11 23:11:56 INFO Vacuumed table table=objects36572026/11/11 23:11:56 INFO Received push request method=POST path=/36582026/11/11 23:11:56 INFO Vacuumed table table=pending_closures36592026/11/11 23:11:56 INFO Vacuumed table table=pending_objects36602026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads36612026/11/11 23:11:56 INFO Vacuumed table table=closures36622026/11/11 23:11:56 INFO Received complete push request method=POST path=/36632026/11/11 23:11:56 INFO Vacuumed table table=objects36642026/11/11 23:11:56 INFO Completed push id=30 objects=8 duration=963.29µs36652026/11/11 23:11:56 INFO Received uploads request method=POST path=/36662026/11/11 23:11:56 INFO Received complete push request method=POST path=/36672026/11/11 23:11:56 INFO Completed push id=21 objects=6 duration=1.247852ms36682026/11/11 23:11:56 INFO Received complete upload request method=POST path=/36692026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=036702026/11/11 23:11:56 INFO Completed upload id=18 objects=4 duration=1.739777ms36712026/11/11 23:11:56 INFO Received uploads request method=POST path=/36722026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=036732026/11/11 23:11:56 INFO Vacuumed table table=pending_closures36742026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=036752026/11/11 23:11:56 INFO Received uploads request method=POST path=/36762026/11/11 23:11:56 INFO Vacuumed table table=pending_objects36772026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads36782026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period36792026/11/11 23:11:56 INFO Vacuumed table table=closures36802026/11/11 23:11:56 INFO Vacuumed table table=objects36812026/11/11 23:11:56 INFO Received complete upload request method=POST path=/36822026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036832026/11/11 23:11:56 INFO Completed upload id=31 objects=5 duration=815.308µs36842026/11/11 23:11:56 INFO Vacuumed table table=pending_closures36852026/11/11 23:11:56 INFO Vacuumed table table=pending_objects36862026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads36872026/11/11 23:11:56 INFO Vacuumed table table=closures36882026/11/11 23:11:56 INFO Received complete upload request method=POST path=/36892026/11/11 23:11:56 INFO Vacuumed table table=objects36902026/11/11 23:11:56 INFO Completed upload id=22 objects=7 duration=824.308µs36912026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=036922026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period36932026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036942026/11/11 23:11:56 INFO Received complete upload request method=POST path=/36952026/11/11 23:11:56 INFO Vacuumed table table=pending_closures36962026/11/11 23:11:56 INFO Received push request method=POST path=/36972026/11/11 23:11:56 INFO Vacuumed table table=pending_objects36982026/11/11 23:11:56 INFO Completed upload id=15 objects=9 duration=1.07759ms36992026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads37002026/11/11 23:11:56 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."37012026/11/11 23:11:56 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."37022026/11/11 23:11:56 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."37032026/11/11 23:11:56 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."37042026/11/11 23:11:56 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."37052026/11/11 23:11:56 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."37062026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."37072026/11/11 23:11:56 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."37082026/11/11 23:11:56 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."37092026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."37102026/11/11 23:11:56 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."37112026/11/11 23:11:56 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."37122026/11/11 23:11:56 INFO Vacuumed table table=closures37132026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=037142026/11/11 23:11:56 INFO Vacuumed table table=objects37152026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=037162026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037172026/11/11 23:11:56 INFO Vacuumed table table=pending_closures37182026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period37192026/11/11 23:11:56 INFO Received complete push request method=POST path=/37202026/11/11 23:11:56 INFO Vacuumed table table=pending_objects37212026/11/11 23:11:56 INFO Completed push id=32 objects=3 duration=855.708µs37222026/11/11 23:11:56 INFO Received push request method=POST path=/37232026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads37242026/11/11 23:11:56 INFO Vacuumed table table=closures37252026/11/11 23:11:56 INFO Received complete push request method=POST path=/37262026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037272026/11/11 23:11:56 INFO Vacuumed table table=objects37282026/11/11 23:11:56 INFO Received complete push request method=POST path=/37292026/11/11 23:11:56 INFO Completed push id=24 objects=4 duration=949.869µs37302026/11/11 23:11:56 INFO Vacuumed table table=pending_closures37312026/11/11 23:11:56 INFO Completed push id=17 objects=6 duration=1.695976ms37322026/11/11 23:11:56 INFO Vacuumed table table=pending_objects37332026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads37342026/11/11 23:11:56 INFO Received uploads request method=POST path=/37352026/11/11 23:11:56 INFO Vacuumed table table=closures37362026/11/11 23:11:56 INFO Vacuumed table table=objects37372026/11/11 23:11:56 INFO Received uploads request method=POST path=/37382026/11/11 23:11:56 INFO Received push request method=POST path=/37392026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=037402026/11/11 23:11:56 INFO Received uploads request method=POST path=/37412026/11/11 23:11:56 INFO Received push request method=POST path=/37422026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period37432026/11/11 23:11:56 INFO Received complete push request method=POST path=/37442026/11/11 23:11:56 INFO Completed push id=25 objects=9 duration=1.458854ms37452026/11/11 23:11:56 INFO Received push request method=POST path=/37462026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037472026/11/11 23:11:56 INFO Vacuumed table table=pending_closures37482026/11/11 23:11:56 INFO Vacuumed table table=pending_objects37492026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads37502026/11/11 23:11:56 INFO Vacuumed table table=closures37512026/11/11 23:11:56 INFO Vacuumed table table=objects37522026/11/11 23:11:56 INFO Received uploads request method=POST path=/37532026/11/11 23:11:56 INFO Received uploads request method=POST path=/37542026/11/11 23:11:56 INFO Received complete push request method=POST path=/37552026/11/11 23:11:56 INFO Completed push id=21 objects=17 duration=1.610735ms37562026/11/11 23:11:56 INFO Received uploads request method=POST path=/37572026/11/11 23:11:56 INFO Received uploads request method=POST path=/37582026/11/11 23:11:56 INFO Received push request method=POST path=/37592026/11/11 23:11:56 INFO Received complete upload request method=POST path=/37602026/11/11 23:11:56 INFO Received push request method=POST path=/37612026/11/11 23:11:56 INFO Completed upload id=26 objects=3 duration=989.229µs37622026/11/11 23:11:56 INFO Received complete push request method=POST path=/37632026/11/11 23:11:56 INFO Completed push id=19 objects=6 duration=3.15393ms37642026/11/11 23:11:56 INFO Received uploads request method=POST path=/37652026/11/11 23:11:56 INFO Received push request method=POST path=/37662026/11/11 23:11:56 INFO Received uploads request method=POST path=/37672026/11/11 23:11:56 INFO Received uploads request method=POST path=/37682026/11/11 23:11:56 INFO Received complete push request method=POST path=/37692026/11/11 23:11:56 INFO Received push request method=POST path=/37702026/11/11 23:11:56 INFO Completed push id=19 objects=7 duration=1.600235ms37712026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=037722026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037732026/11/11 23:11:56 INFO Vacuumed table table=pending_closures37742026/11/11 23:11:56 INFO Received complete upload request method=POST path=/37752026/11/11 23:11:56 INFO Vacuumed table table=pending_objects37762026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads37772026/11/11 23:11:56 INFO Completed upload id=27 objects=7 duration=1.09857ms37782026/11/11 23:11:56 INFO Vacuumed table table=closures37792026/11/11 23:11:56 INFO Received complete push request method=POST path=/37802026/11/11 23:11:56 INFO Vacuumed table table=objects37812026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=037822026/11/11 23:11:56 INFO Received complete push request method=POST path=/37832026/11/11 23:11:56 INFO Completed push id=12 objects=2 duration=2.741926ms37842026/11/11 23:11:56 INFO Received push request method=POST path=/37852026/11/11 23:11:56 INFO Received complete upload request method=POST path=/37862026/11/11 23:11:56 INFO Received push request method=POST path=/37872026/11/11 23:11:56 INFO Completed push id=15 objects=24 duration=4.18468ms37882026/11/11 23:11:56 INFO Completed upload id=23 objects=15 duration=2.441784ms37892026/11/11 23:11:56 INFO Received push request method=POST path=/37902026/11/11 23:11:56 INFO Received complete push request method=POST path=/37912026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=037922026/11/11 23:11:56 INFO Completed push id=17 objects=22 duration=1.843718ms37932026/11/11 23:11:56 INFO Received complete upload request method=POST path=/37942026/11/11 23:11:56 INFO Vacuumed table table=pending_closures37952026/11/11 23:11:56 INFO Completed upload id=28 objects=3 duration=744.347µs37962026/11/11 23:11:56 INFO Vacuumed table table=pending_objects37972026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads37982026/11/11 23:11:56 INFO Received uploads request method=POST path=/37992026/11/11 23:11:56 INFO Vacuumed table table=closures38002026/11/11 23:11:56 INFO Vacuumed table table=objects38012026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=038022026/11/11 23:11:56 INFO Received complete upload request method=POST path=/38032026/11/11 23:11:56 INFO Received uploads request method=POST path=/38042026/11/11 23:11:56 INFO Received complete push request method=POST path=/38052026/11/11 23:11:56 INFO Completed upload id=8 objects=3 duration=3.515614ms38062026/11/11 23:11:56 INFO Received uploads request method=POST path=/38072026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=038082026/11/11 23:11:56 INFO Received push request method=POST path=/38092026/11/11 23:11:56 INFO Completed push id=3 objects=21 duration=4.972427ms38102026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period38112026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period38122026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038132026/11/11 23:11:56 INFO Vacuumed table table=pending_closures38142026/11/11 23:11:56 INFO Vacuumed table table=pending_objects38152026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads38162026/11/11 23:11:56 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."38172026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."38182026/11/11 23:11:56 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."38192026/11/11 23:11:56 INFO Vacuumed table table=closures38202026/11/11 23:11:56 INFO Received complete push request method=POST path=/38212026/11/11 23:11:56 INFO Vacuumed table table=objects38222026/11/11 23:11:56 INFO Completed push id=29 objects=23 duration=2.834927ms38232026/11/11 23:11:56 INFO Received uploads request method=POST path=/38242026/11/11 23:11:56 INFO Received complete upload request method=POST path=/38252026/11/11 23:11:56 INFO Received uploads request method=POST path=/38262026/11/11 23:11:56 INFO Received uploads request method=POST path=/38272026/11/11 23:11:56 INFO Completed upload id=19 objects=6 duration=1.225732ms38282026/11/11 23:11:56 INFO Received complete upload request method=POST path=/38292026/11/11 23:11:56 INFO Received uploads request method=POST path=/38302026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=038312026/11/11 23:11:56 INFO Completed upload id=10 objects=2 duration=2.609285ms38322026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period38332026/11/11 23:11:56 INFO Received complete push request method=POST path=/38342026/11/11 23:11:56 INFO Received uploads request method=POST path=/38352026/11/11 23:11:56 INFO Completed push id=30 objects=6 duration=971.429µs38362026/11/11 23:11:56 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."38372026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."38382026/11/11 23:11:56 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."38392026/11/11 23:11:56 INFO Received push request method=POST path=/38402026/11/11 23:11:56 INFO Received push request method=POST path=/38412026/11/11 23:11:56 INFO Received complete upload request method=POST path=/38422026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=038432026/11/11 23:11:56 INFO Completed upload id=22 objects=7 duration=967.989µs38442026/11/11 23:11:56 INFO Received uploads request method=POST path=/38452026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=038462026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038472026/11/11 23:11:56 INFO Received uploads request method=POST path=/38482026/11/11 23:11:56 INFO Vacuumed table table=pending_closures38492026/11/11 23:11:56 INFO Vacuumed table table=pending_objects38502026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads38512026/11/11 23:11:56 INFO Received complete push request method=POST path=/38522026/11/11 23:11:56 INFO Vacuumed table table=closures38532026/11/11 23:11:56 INFO Received complete push request method=POST path=/38542026/11/11 23:11:56 INFO Completed push id=6 objects=6 duration=7.246829ms38552026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=038562026/11/11 23:11:56 INFO Received push request method=POST path=/38572026/11/11 23:11:56 INFO Completed push id=33 objects=3 duration=5.899676ms38582026/11/11 23:11:56 INFO Vacuumed table table=objects38592026/11/11 23:11:56 INFO Vacuumed table table=pending_closures38602026/11/11 23:11:56 INFO Received uploads request method=POST path=/38612026/11/11 23:11:56 INFO Received push request method=POST path=/38622026/11/11 23:11:56 INFO Received complete upload request method=POST path=/38632026/11/11 23:11:56 INFO Vacuumed table table=pending_objects38642026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads38652026/11/11 23:11:56 INFO Received complete upload request method=POST path=/38662026/11/11 23:11:56 INFO Received push request method=POST path=/38672026/11/11 23:11:56 INFO Received complete push request method=POST path=/38682026/11/11 23:11:56 INFO Completed push id=34 objects=6 duration=3.342311ms38692026/11/11 23:11:56 INFO Vacuumed table table=closures38702026/11/11 23:11:56 INFO Completed upload id=6 objects=6 duration=4.779066ms38712026/11/11 23:11:56 INFO Received complete upload request method=POST path=/38722026/11/11 23:11:56 INFO Completed upload id=12 objects=26 duration=7.710854ms38732026/11/11 23:11:56 INFO Vacuumed table table=objects38742026/11/11 23:11:56 INFO Completed upload id=7 objects=10 duration=1.496634ms38752026/11/11 23:11:56 INFO Received complete push request method=POST path=/38762026/11/11 23:11:56 INFO Completed push id=24 objects=9 duration=6.498622ms38772026/11/11 23:11:56 INFO Received complete upload request method=POST path=/38782026/11/11 23:11:56 INFO Received complete push request method=POST path=/38792026/11/11 23:11:56 INFO Completed upload id=14 objects=16 duration=1.943658ms38802026/11/11 23:11:56 INFO Completed push id=35 objects=8 duration=1.095391ms38812026/11/11 23:11:56 INFO Received uploads request method=POST path=/38822026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=038832026/11/11 23:11:56 INFO Received push request method=POST path=/38842026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period38852026/11/11 23:11:56 INFO Received complete push request method=POST path=/38862026/11/11 23:11:56 INFO Received push request method=POST path=/38872026/11/11 23:11:56 INFO Completed push id=15 objects=7 duration=1.553915ms38882026/11/11 23:11:56 INFO Received push request method=POST path=/38892026/11/11 23:11:56 INFO Received complete upload request method=POST path=/38902026/11/11 23:11:56 INFO Received uploads request method=POST path=/38912026/11/11 23:11:56 INFO Received complete push request method=POST path=/38922026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=038932026/11/11 23:11:56 INFO Completed upload id=36 objects=9 duration=953.969µs38942026/11/11 23:11:56 INFO Received complete push request method=POST path=/38952026/11/11 23:11:56 INFO Completed push id=15 objects=5 duration=2.418743ms38962026/11/11 23:11:56 INFO Completed push id=10 objects=15 duration=1.540715ms38972026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038982026/11/11 23:11:56 INFO Vacuumed table table=pending_closures38992026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039002026/11/11 23:11:56 INFO Vacuumed table table=pending_closures39012026/11/11 23:11:56 INFO Vacuumed table table=pending_objects39022026/11/11 23:11:56 INFO Received uploads request method=POST path=/39032026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads39042026/11/11 23:11:56 INFO Vacuumed table table=pending_objects39052026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads39062026/11/11 23:11:56 INFO Vacuumed table table=closures39072026/11/11 23:11:56 INFO Vacuumed table table=closures39082026/11/11 23:11:56 INFO Vacuumed table table=objects39092026/11/11 23:11:56 INFO Vacuumed table table=objects39102026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039112026/11/11 23:11:56 INFO Received push request method=POST path=/39122026/11/11 23:11:56 INFO Vacuumed table table=pending_closures39132026/11/11 23:11:56 INFO Vacuumed table table=pending_objects39142026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads39152026/11/11 23:11:56 INFO Vacuumed table table=closures39162026/11/11 23:11:56 INFO Received push request method=POST path=/39172026/11/11 23:11:56 INFO Vacuumed table table=objects39182026/11/11 23:11:56 INFO Received complete push request method=POST path=/39192026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=039202026/11/11 23:11:56 INFO Completed push id=16 objects=8 duration=1.740317ms39212026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period39222026/11/11 23:11:56 INFO Received uploads request method=POST path=/39232026/11/11 23:11:56 INFO Received complete push request method=POST path=/39242026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039252026/11/11 23:11:56 INFO Vacuumed table table=pending_closures39262026/11/11 23:11:56 INFO Received uploads request method=POST path=/39272026/11/11 23:11:56 INFO Vacuumed table table=pending_objects39282026/11/11 23:11:56 INFO Completed push id=11 objects=8 duration=2.780646ms39292026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=039302026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads39312026/11/11 23:11:56 INFO Vacuumed table table=closures39322026/11/11 23:11:56 INFO Vacuumed table table=objects39332026/11/11 23:11:56 INFO Received complete push request method=POST path=/39342026/11/11 23:11:56 INFO Received complete push request method=POST path=/39352026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period39362026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039372026/11/11 23:11:56 INFO Completed push id=17 objects=11 duration=1.815178ms39382026/11/11 23:11:56 INFO Vacuumed table table=pending_closures39392026/11/11 23:11:56 INFO Received push request method=POST path=/39402026/11/11 23:11:56 INFO Vacuumed table table=pending_objects39412026/11/11 23:11:56 INFO Completed push id=22 objects=3 duration=3.674275ms39422026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=039432026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads39442026/11/11 23:11:56 INFO Vacuumed table table=closures39452026/11/11 23:11:56 INFO Vacuumed table table=objects39462026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039472026/11/11 23:11:56 INFO Vacuumed table table=pending_closures39482026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=039492026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=039502026/11/11 23:11:56 INFO Vacuumed table table=pending_objects39512026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads39522026/11/11 23:11:56 INFO Vacuumed table table=closures39532026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period39542026/11/11 23:11:56 INFO Vacuumed table table=objects39552026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039562026/11/11 23:11:56 INFO Vacuumed table table=pending_closures39572026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039582026/11/11 23:11:56 INFO Vacuumed table table=pending_objects39592026/11/11 23:11:56 INFO Vacuumed table table=pending_closures39602026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads39612026/11/11 23:11:56 INFO Vacuumed table table=closures39622026/11/11 23:11:56 INFO Received push request method=POST path=/39632026/11/11 23:11:56 INFO Vacuumed table table=pending_objects39642026/11/11 23:11:56 INFO Vacuumed table table=objects39652026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads39662026/11/11 23:11:56 INFO Vacuumed table table=closures39672026/11/11 23:11:56 INFO Vacuumed table table=objects39682026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=039692026/11/11 23:11:56 INFO Received complete push request method=POST path=/39702026/11/11 23:11:56 INFO Received uploads request method=POST path=/39712026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period39722026/11/11 23:11:56 INFO Completed push id=8 objects=6 duration=2.814947ms39732026/11/11 23:11:56 INFO Received complete upload request method=POST path=/39742026/11/11 23:11:56 INFO Received push request method=POST path=/39752026/11/11 23:11:56 INFO Completed upload id=11 objects=7 duration=1.911678ms39762026/11/11 23:11:56 INFO Received push request method=POST path=/39772026/11/11 23:11:56 INFO Received push request method=POST path=/39782026/11/11 23:11:56 INFO Received push request method=POST path=/39792026/11/11 23:11:56 INFO Received complete push request method=POST path=/39802026/11/11 23:11:56 INFO Received uploads request method=POST path=/39812026/11/11 23:11:56 INFO Received complete push request method=POST path=/39822026/11/11 23:11:56 INFO Received uploads request method=POST path=/39832026/11/11 23:11:56 INFO Completed push id=11 objects=6 duration=2.739226ms39842026/11/11 23:11:56 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."39852026/11/11 23:11:56 INFO Completed push id=13 objects=6 duration=3.267371ms39862026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=039872026/11/11 23:11:56 INFO Received complete push request method=POST path=/39882026/11/11 23:11:56 INFO Received push request method=POST path=/39892026/11/11 23:11:56 WARN Force mode enabled - objects will be deleted immediately without grace period39902026/11/11 23:11:56 INFO Received complete upload request method=POST path=/39912026/11/11 23:11:56 INFO Completed upload id=22 objects=2 duration=1.252712ms39922026/11/11 23:11:56 INFO Received uploads request method=POST path=/39932026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039942026/11/11 23:11:56 INFO Completed push id=11 objects=11 duration=5.856555ms39952026/11/11 23:11:56 INFO Vacuumed table table=pending_closures39962026/11/11 23:11:56 INFO Vacuumed table table=pending_objects39972026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads39982026/11/11 23:11:56 INFO Received push request method=POST path=/39992026/11/11 23:11:56 INFO Vacuumed table table=closures40002026/11/11 23:11:56 INFO Vacuumed table table=objects40012026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=040022026/11/11 23:11:56 INFO Received push request method=POST path=/40032026/11/11 23:11:56 INFO Received uploads request method=POST path=/40042026/11/11 23:11:56 INFO Received complete upload request method=POST path=/40052026/11/11 23:11:56 INFO Vacuumed table table=pending_closures40062026/11/11 23:11:56 INFO Completed upload id=26 objects=6 duration=988.669µs40072026/11/11 23:11:56 INFO Vacuumed table table=pending_objects40082026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads40092026/11/11 23:11:56 INFO Vacuumed table table=closures40102026/11/11 23:11:56 INFO Vacuumed table table=objects40112026/11/11 23:11:56 INFO Received complete push request method=POST path=/40122026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=040132026/11/11 23:11:56 INFO Completed push id=18 objects=6 duration=1.419954ms40142026/11/11 23:11:56 INFO Received push request method=POST path=/40152026/11/11 23:11:56 INFO Received uploads request method=POST path=/40162026/11/11 23:11:56 INFO Received push request method=POST path=/40172026/11/11 23:11:56 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=040182026/11/11 23:11:56 INFO Received uploads request method=POST path=/40192026/11/11 23:11:56 INFO Vacuumed table table=pending_closures40202026/11/11 23:11:56 INFO Received complete push request method=POST path=/40212026/11/11 23:11:56 INFO Vacuumed table table=pending_objects40222026/11/11 23:11:56 INFO Vacuumed table table=multipart_uploads40232026/11/11 23:11:56 INFO Vacuumed table table=closures40242026/11/11 23:11:56 INFO Received push request method=POST path=/40252026/11/11 23:11:56 INFO Completed push id=26 objects=14 duration=3.050669ms40262026/11/11 23:11:56 INFO Vacuumed table table=objects40272026/11/11 23:11:56 INFO Received complete push request method=POST path=/40282026/11/11 23:11:56 INFO Aborted multipart uploads count=0 kept=040292026/11/11 23:11:56 INFO Completed push id=21 objects=3 duration=1.366713ms40302026/11/11 23:11:56 INFO Received uploads request method=POST path=/40312026/11/11 23:11:56 INFO Received complete upload request method=POST path=/40322026/11/11 23:11:56 INFO Received uploads request method=POST path=/40332026/11/11 23:11:56 INFO Completed upload id=29 objects=5 duration=1.026109ms40342026/11/11 23:11:56 INFO Received uploads request method=POST path=/40352026/11/11 23:11:56 INFO Received complete push request method=POST path=/40362026/11/11 23:11:57 INFO Received push request method=POST path=/40372026/11/11 23:11:57 INFO Completed push id=4 objects=6 duration=1.863338ms40382026/11/11 23:11:57 INFO Received uploads request method=POST path=/40392026/11/11 23:11:57 INFO Received complete push request method=POST path=/40402026/11/11 23:11:57 INFO Received uploads request method=POST path=/40412026/11/11 23:11:57 INFO Received complete push request method=POST path=/40422026/11/11 23:11:57 INFO Completed push id=9 objects=24 duration=1.357613ms40432026/11/11 23:11:57 INFO Completed push id=17 objects=6 duration=2.04584ms40442026/11/11 23:11:57 INFO Received uploads request method=POST path=/40452026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=040462026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=040472026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=040482026/11/11 23:11:57 INFO Received push request method=POST path=/40492026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period40502026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period40512026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040522026/11/11 23:11:57 INFO Received uploads request method=POST path=/40532026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040542026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=040552026/11/11 23:11:57 INFO Vacuumed table table=pending_closures40562026/11/11 23:11:57 INFO Vacuumed table table=pending_closures40572026/11/11 23:11:57 INFO Vacuumed table table=pending_objects40582026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=040592026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads40602026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=040612026/11/11 23:11:57 INFO Vacuumed table table=pending_objects40622026/11/11 23:11:57 INFO Vacuumed table table=closures40632026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period40642026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads40652026/11/11 23:11:57 INFO Vacuumed table table=objects40662026/11/11 23:11:57 INFO Vacuumed table table=closures40672026/11/11 23:11:57 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."40682026/11/11 23:11:57 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."40692026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period40702026/11/11 23:11:57 INFO Vacuumed table table=objects40712026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040722026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040732026/11/11 23:11:57 INFO Vacuumed table table=pending_closures40742026/11/11 23:11:57 INFO Received complete upload request method=POST path=/40752026/11/11 23:11:57 INFO Vacuumed table table=pending_closures40762026/11/11 23:11:57 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."40772026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=040782026/11/11 23:11:57 INFO Completed upload id=9 objects=2 duration=1.732216ms40792026/11/11 23:11:57 INFO Vacuumed table table=pending_objects40802026/11/11 23:11:57 INFO Vacuumed table table=pending_objects40812026/11/11 23:11:57 INFO Received push request method=POST path=/40822026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads40832026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads40842026/11/11 23:11:57 INFO Vacuumed table table=closures40852026/11/11 23:11:57 INFO Vacuumed table table=closures40862026/11/11 23:11:57 INFO Vacuumed table table=objects40872026/11/11 23:11:57 INFO Vacuumed table table=objects40882026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=3 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040892026/11/11 23:11:57 INFO Vacuumed table table=pending_closures40902026/11/11 23:11:57 INFO Received complete push request method=POST path=/40912026/11/11 23:11:57 INFO Received uploads request method=POST path=/40922026/11/11 23:11:57 INFO Vacuumed table table=pending_objects40932026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads40942026/11/11 23:11:57 INFO Completed push id=4 objects=7 duration=2.588164ms40952026/11/11 23:11:57 INFO Vacuumed table table=closures40962026/11/11 23:11:57 INFO Vacuumed table table=objects40972026/11/11 23:11:57 INFO Received push request method=POST path=/40982026/11/11 23:11:57 INFO Received complete push request method=POST path=/40992026/11/11 23:11:57 INFO Received complete push request method=POST path=/41002026/11/11 23:11:57 INFO Received push request method=POST path=/41012026/11/11 23:11:57 INFO Completed push id=13 objects=15 duration=1.792477ms41022026/11/11 23:11:57 INFO Completed push id=8 objects=3 duration=1.616875ms41032026/11/11 23:11:57 INFO Received push request method=POST path=/41042026/11/11 23:11:57 INFO Received uploads request method=POST path=/41052026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=041062026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period41072026/11/11 23:11:57 INFO Received push request method=POST path=/41082026/11/11 23:11:57 INFO Received complete upload request method=POST path=/41092026/11/11 23:11:57 INFO Received uploads request method=POST path=/41102026/11/11 23:11:57 INFO Received push request method=POST path=/41112026/11/11 23:11:57 INFO Completed upload id=10 objects=10 duration=923.168µs41122026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=041132026/11/11 23:11:57 INFO Received push request method=POST path=/41142026/11/11 23:11:57 INFO Received uploads request method=POST path=/41152026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=041162026/11/11 23:11:57 INFO Received complete upload request method=POST path=/41172026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=041182026/11/11 23:11:57 INFO Received uploads request method=POST path=/41192026/11/11 23:11:57 INFO Vacuumed table table=pending_closures41202026/11/11 23:11:57 INFO Completed upload id=14 objects=8 duration=1.943678ms41212026/11/11 23:11:57 INFO Vacuumed table table=pending_objects41222026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041232026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads41242026/11/11 23:11:57 INFO Received uploads request method=POST path=/41252026/11/11 23:11:57 INFO Vacuumed table table=pending_closures41262026/11/11 23:11:57 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."41272026/11/11 23:11:57 INFO Vacuumed table table=closures41282026/11/11 23:11:57 INFO Vacuumed table table=pending_objects41292026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads41302026/11/11 23:11:57 INFO Vacuumed table table=objects41312026/11/11 23:11:57 INFO Vacuumed table table=closures41322026/11/11 23:11:57 INFO Vacuumed table table=objects41332026/11/11 23:11:57 INFO Received uploads request method=POST path=/41342026/11/11 23:11:57 INFO Received push request method=POST path=/41352026/11/11 23:11:57 INFO Received push request method=POST path=/41362026/11/11 23:11:57 INFO Received complete push request method=POST path=/41372026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=041382026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period41392026/11/11 23:11:57 INFO Received push request method=POST path=/41402026/11/11 23:11:57 INFO Completed push id=14 objects=3 duration=3.732996ms41412026/11/11 23:11:57 INFO Received complete upload request method=POST path=/41422026/11/11 23:11:57 INFO Received complete upload request method=POST path=/41432026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041442026/11/11 23:11:57 INFO Received uploads request method=POST path=/41452026/11/11 23:11:57 INFO Completed upload id=13 objects=10 duration=1.417474ms41462026/11/11 23:11:57 INFO Vacuumed table table=pending_closures41472026/11/11 23:11:57 INFO Completed upload id=21 objects=7 duration=2.194561ms41482026/11/11 23:11:57 INFO Vacuumed table table=pending_objects41492026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads41502026/11/11 23:11:57 INFO Received complete push request method=POST path=/41512026/11/11 23:11:57 INFO Vacuumed table table=closures41522026/11/11 23:11:57 INFO Vacuumed table table=objects41532026/11/11 23:11:57 INFO Received push request method=POST path=/41542026/11/11 23:11:57 INFO Completed push id=30 objects=4 duration=3.246971ms41552026/11/11 23:11:57 INFO Received complete upload request method=POST path=/41562026/11/11 23:11:57 INFO Received complete push request method=POST path=/41572026/11/11 23:11:57 INFO Received push request method=POST path=/41582026/11/11 23:11:57 INFO Completed upload id=31 objects=3 duration=2.609565ms41592026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=041602026/11/11 23:11:57 INFO Completed push id=20 objects=3 duration=1.909098ms41612026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period41622026/11/11 23:11:57 INFO Received push request method=POST path=/41632026/11/11 23:11:57 INFO Received complete push request method=POST path=/41642026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=041652026/11/11 23:11:57 INFO Completed push id=21 objects=9 duration=1.850897ms41662026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period41672026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041682026/11/11 23:11:57 INFO Received uploads request method=POST path=/41692026/11/11 23:11:57 INFO Received uploads request method=POST path=/41702026/11/11 23:11:57 INFO Vacuumed table table=pending_closures41712026/11/11 23:11:57 INFO Vacuumed table table=pending_objects41722026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads41732026/11/11 23:11:57 INFO Received complete push request method=POST path=/41742026/11/11 23:11:57 INFO Vacuumed table table=closures41752026/11/11 23:11:57 INFO Completed push id=7 objects=7 duration=1.444514ms41762026/11/11 23:11:57 INFO Vacuumed table table=objects41772026/11/11 23:11:57 INFO Received complete upload request method=POST path=/41782026/11/11 23:11:57 INFO Completed upload id=22 objects=2 duration=978.469µs41792026/11/11 23:11:57 INFO Received push request method=POST path=/41802026/11/11 23:11:57 INFO Received push request method=POST path=/41812026/11/11 23:11:57 INFO Received uploads request method=POST path=/41822026/11/11 23:11:57 INFO Received push request method=POST path=/41832026/11/11 23:11:57 INFO Received uploads request method=POST path=/41842026/11/11 23:11:57 INFO Received push request method=POST path=/41852026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=041862026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period41872026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041882026/11/11 23:11:57 INFO Received uploads request method=POST path=/41892026/11/11 23:11:57 INFO Vacuumed table table=pending_closures41902026/11/11 23:11:57 INFO Vacuumed table table=pending_objects41912026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads41922026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=041932026/11/11 23:11:57 INFO Vacuumed table table=closures41942026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period41952026/11/11 23:11:57 INFO Vacuumed table table=objects41962026/11/11 23:11:57 INFO Received push request method=POST path=/41972026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=11 objects-failed-to-delete=041982026/11/11 23:11:57 INFO Vacuumed table table=pending_closures41992026/11/11 23:11:57 INFO Received uploads request method=POST path=/42002026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=042012026/11/11 23:11:57 INFO Vacuumed table table=pending_objects42022026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads42032026/11/11 23:11:57 INFO Vacuumed table table=closures42042026/11/11 23:11:57 INFO Vacuumed table table=pending_closures42052026/11/11 23:11:57 INFO Vacuumed table table=objects42062026/11/11 23:11:57 INFO Vacuumed table table=pending_objects42072026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads42082026/11/11 23:11:57 INFO Vacuumed table table=closures42092026/11/11 23:11:57 INFO Received uploads request method=POST path=/42102026/11/11 23:11:57 INFO Vacuumed table table=objects42112026/11/11 23:11:57 INFO Received uploads request method=POST path=/42122026/11/11 23:11:57 INFO Received push request method=POST path=/42132026/11/11 23:11:57 INFO Received complete upload request method=POST path=/42142026/11/11 23:11:57 INFO Received push request method=POST path=/42152026/11/11 23:11:57 INFO Received complete push request method=POST path=/42162026/11/11 23:11:57 INFO Completed push id=32 objects=7 duration=6.27934ms42172026/11/11 23:11:57 INFO Completed upload id=18 objects=2 duration=8.067777ms42182026/11/11 23:11:57 INFO Received complete upload request method=POST path=/42192026/11/11 23:11:57 INFO Completed upload id=18 objects=11 duration=1.461274ms42202026/11/11 23:11:57 INFO Received uploads request method=POST path=/42212026/11/11 23:11:57 INFO Received push request method=POST path=/42222026/11/11 23:11:57 INFO Received push request method=POST path=/42232026/11/11 23:11:57 INFO Received uploads request method=POST path=/42242026/11/11 23:11:57 INFO Received push request method=POST path=/42252026/11/11 23:11:57 INFO Received push request method=POST path=/42262026/11/11 23:11:57 INFO Received complete push request method=POST path=/42272026/11/11 23:11:57 INFO Received complete push request method=POST path=/42282026/11/11 23:11:57 INFO Received complete upload request method=POST path=/42292026/11/11 23:11:57 INFO Completed push id=26 objects=4 duration=936.729µs42302026/11/11 23:11:57 INFO Completed push id=14 objects=3 duration=1.188792ms42312026/11/11 23:11:57 INFO Received complete upload request method=POST path=/42322026/11/11 23:11:57 INFO Completed upload id=23 objects=16 duration=2.12284ms42332026/11/11 23:11:57 INFO Received push request method=POST path=/42342026/11/11 23:11:57 INFO Completed upload id=27 objects=4 duration=1.01891ms42352026/11/11 23:11:57 INFO Received complete push request method=POST path=/42362026/11/11 23:11:57 INFO Completed push id=27 objects=25 duration=3.15747ms42372026/11/11 23:11:57 INFO Received uploads request method=POST path=/42382026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=042392026/11/11 23:11:57 INFO Received push request method=POST path=/42402026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period42412026/11/11 23:11:57 INFO Received push request method=POST path=/42422026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042432026/11/11 23:11:57 INFO Received push request method=POST path=/42442026/11/11 23:11:57 INFO Vacuumed table table=pending_closures42452026/11/11 23:11:57 INFO Vacuumed table table=pending_objects42462026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads42472026/11/11 23:11:57 INFO Received complete upload request method=POST path=/42482026/11/11 23:11:57 INFO Vacuumed table table=closures42492026/11/11 23:11:57 INFO Received push request method=POST path=/42502026/11/11 23:11:57 INFO Completed upload id=28 objects=13 duration=952.709µs42512026/11/11 23:11:57 INFO Received complete push request method=POST path=/42522026/11/11 23:11:57 INFO Vacuumed table table=objects42532026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=042542026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period42552026/11/11 23:11:57 INFO Completed push id=13 objects=11 duration=1.543235ms42562026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042572026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=042582026/11/11 23:11:57 INFO Received complete push request method=POST path=/42592026/11/11 23:11:57 INFO Vacuumed table table=pending_closures42602026/11/11 23:11:57 INFO Vacuumed table table=pending_objects42612026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads42622026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042632026/11/11 23:11:57 INFO Vacuumed table table=closures42642026/11/11 23:11:57 INFO Vacuumed table table=pending_closures42652026/11/11 23:11:57 INFO Completed push id=14 objects=4 duration=2.626165ms42662026/11/11 23:11:57 INFO Vacuumed table table=objects42672026/11/11 23:11:57 INFO Received complete upload request method=POST path=/42682026/11/11 23:11:57 INFO Vacuumed table table=pending_objects42692026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads42702026/11/11 23:11:57 INFO Completed upload id=13 objects=7 duration=1.06117ms42712026/11/11 23:11:57 INFO Received complete upload request method=POST path=/42722026/11/11 23:11:57 INFO Vacuumed table table=closures42732026/11/11 23:11:57 INFO Completed upload id=22 objects=6 duration=872.048µs42742026/11/11 23:11:57 INFO Vacuumed table table=objects42752026/11/11 23:11:57 INFO Received push request method=POST path=/42762026/11/11 23:11:57 INFO Received complete push request method=POST path=/42772026/11/11 23:11:57 INFO Completed push id=29 objects=3 duration=868.468µs42782026/11/11 23:11:57 INFO Received complete upload request method=POST path=/42792026/11/11 23:11:57 INFO Received uploads request method=POST path=/42802026/11/11 23:11:57 INFO Completed upload id=16 objects=17 duration=2.290321ms42812026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=042822026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period42832026/11/11 23:11:57 INFO Received complete upload request method=POST path=/42842026/11/11 23:11:57 INFO Received uploads request method=POST path=/42852026/11/11 23:11:57 INFO Completed upload id=20 objects=4 duration=1.163231ms42862026/11/11 23:11:57 INFO Received uploads request method=POST path=/42872026/11/11 23:11:57 INFO Received uploads request method=POST path=/42882026/11/11 23:11:57 INFO Received complete push request method=POST path=/42892026/11/11 23:11:57 INFO Completed push id=24 objects=5 duration=2.557544ms42902026/11/11 23:11:57 INFO Received push request method=POST path=/42912026/11/11 23:11:57 INFO Received push request method=POST path=/42922026/11/11 23:11:57 INFO Received complete upload request method=POST path=/42932026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=042942026/11/11 23:11:57 INFO Completed upload id=31 objects=2 duration=1.089871ms42952026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=042962026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period42972026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period42982026/11/11 23:11:57 INFO Received complete upload request method=POST path=/42992026/11/11 23:11:57 INFO Completed upload id=23 objects=7 duration=3.009508ms43002026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043012026/11/11 23:11:57 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."43022026/11/11 23:11:57 INFO Vacuumed table table=pending_closures43032026/11/11 23:11:57 INFO Received uploads request method=POST path=/43042026/11/11 23:11:57 INFO Vacuumed table table=pending_objects43052026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads43062026/11/11 23:11:57 INFO Vacuumed table table=closures43072026/11/11 23:11:57 INFO Vacuumed table table=objects43082026/11/11 23:11:57 INFO Received complete push request method=POST path=/43092026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=043102026/11/11 23:11:57 INFO Completed push id=33 objects=6 duration=903.649µs43112026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period43122026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043132026/11/11 23:11:57 INFO Vacuumed table table=pending_closures43142026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=043152026/11/11 23:11:57 INFO Received uploads request method=POST path=/43162026/11/11 23:11:57 INFO Vacuumed table table=pending_objects43172026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads43182026/11/11 23:11:57 INFO Received uploads request method=POST path=/43192026/11/11 23:11:57 INFO Vacuumed table table=closures43202026/11/11 23:11:57 INFO Vacuumed table table=objects43212026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043222026/11/11 23:11:57 INFO Received uploads request method=POST path=/43232026/11/11 23:11:57 INFO Vacuumed table table=pending_closures43242026/11/11 23:11:57 INFO Vacuumed table table=pending_objects43252026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads43262026/11/11 23:11:57 INFO Vacuumed table table=closures43272026/11/11 23:11:57 INFO Vacuumed table table=objects43282026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=043292026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period43302026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043312026/11/11 23:11:57 INFO Vacuumed table table=pending_closures43322026/11/11 23:11:57 INFO Received complete push request method=POST path=/43332026/11/11 23:11:57 INFO Received uploads request method=POST path=/43342026/11/11 23:11:57 INFO Vacuumed table table=pending_objects43352026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads43362026/11/11 23:11:57 INFO Completed push id=15 objects=4 duration=1.800317ms43372026/11/11 23:11:57 INFO Vacuumed table table=closures43382026/11/11 23:11:57 INFO Vacuumed table table=objects43392026/11/11 23:11:57 INFO Received push request method=POST path=/43402026/11/11 23:11:57 INFO Received push request method=POST path=/43412026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=043422026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period43432026/11/11 23:11:57 INFO Received push request method=POST path=/43442026/11/11 23:11:57 INFO Received complete upload request method=POST path=/43452026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043462026/11/11 23:11:57 INFO Completed upload id=24 objects=18 duration=1.869157ms43472026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=043482026/11/11 23:11:57 INFO Received push request method=POST path=/43492026/11/11 23:11:57 INFO Vacuumed table table=pending_closures43502026/11/11 23:11:57 INFO Vacuumed table table=pending_closures43512026/11/11 23:11:57 INFO Vacuumed table table=pending_objects43522026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads43532026/11/11 23:11:57 INFO Vacuumed table table=closures43542026/11/11 23:11:57 INFO Received uploads request method=POST path=/43552026/11/11 23:11:57 INFO Vacuumed table table=pending_objects43562026/11/11 23:11:57 INFO Received complete upload request method=POST path=/43572026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads43582026/11/11 23:11:57 INFO Received complete push request method=POST path=/43592026/11/11 23:11:57 INFO Vacuumed table table=objects43602026/11/11 23:11:57 INFO Vacuumed table table=closures43612026/11/11 23:11:57 INFO Completed upload id=33 objects=5 duration=2.413703ms43622026/11/11 23:11:57 INFO Vacuumed table table=objects43632026/11/11 23:11:57 INFO Received complete push request method=POST path=/43642026/11/11 23:11:57 INFO Completed push id=21 objects=26 duration=2.172361ms43652026/11/11 23:11:57 INFO Completed push id=16 objects=14 duration=4.590384ms43662026/11/11 23:11:57 INFO Received complete push request method=POST path=/43672026/11/11 23:11:57 INFO Received uploads request method=POST path=/43682026/11/11 23:11:57 INFO Completed push id=25 objects=2 duration=2.123721ms43692026/11/11 23:11:57 INFO Received complete upload request method=POST path=/43702026/11/11 23:11:57 INFO Received uploads request method=POST path=/43712026/11/11 23:11:57 INFO Completed upload id=13 objects=8 duration=950.029µs43722026/11/11 23:11:57 INFO Received uploads request method=POST path=/43732026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=043742026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period43752026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043762026/11/11 23:11:57 INFO Vacuumed table table=pending_closures43772026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=043782026/11/11 23:11:57 INFO Received complete push request method=POST path=/43792026/11/11 23:11:57 INFO Vacuumed table table=pending_objects43802026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads43812026/11/11 23:11:57 INFO Received complete upload request method=POST path=/43822026/11/11 23:11:57 INFO Received complete push request method=POST path=/43832026/11/11 23:11:57 INFO Completed push id=27 objects=4 duration=925.049µs43842026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period43852026/11/11 23:11:57 INFO Completed upload id=28 objects=17 duration=945.549µs43862026/11/11 23:11:57 INFO Vacuumed table table=closures43872026/11/11 23:11:57 INFO Received uploads request method=POST path=/43882026/11/11 23:11:57 INFO Received uploads request method=POST path=/43892026/11/11 23:11:57 INFO Vacuumed table table=objects43902026/11/11 23:11:57 INFO Completed push id=7 objects=15 duration=2.11104ms43912026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043922026/11/11 23:11:57 INFO Vacuumed table table=pending_closures43932026/11/11 23:11:57 INFO Vacuumed table table=pending_objects43942026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads43952026/11/11 23:11:57 INFO Vacuumed table table=closures43962026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=043972026/11/11 23:11:57 INFO Vacuumed table table=objects43982026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period43992026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044002026/11/11 23:11:57 INFO Vacuumed table table=pending_closures44012026/11/11 23:11:57 INFO Received complete push request method=POST path=/44022026/11/11 23:11:57 INFO Vacuumed table table=pending_objects44032026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads44042026/11/11 23:11:57 INFO Completed push id=14 objects=3 duration=1.146971ms44052026/11/11 23:11:57 INFO Vacuumed table table=closures44062026/11/11 23:11:57 INFO Received complete push request method=POST path=/44072026/11/11 23:11:57 INFO Vacuumed table table=objects44082026/11/11 23:11:57 INFO Completed push id=29 objects=2 duration=1.00351ms44092026/11/11 23:11:57 INFO Received uploads request method=POST path=/44102026/11/11 23:11:57 INFO Received uploads request method=POST path=/44112026/11/11 23:11:57 INFO Received complete push request method=POST path=/44122026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=044132026/11/11 23:11:57 INFO Received complete upload request method=POST path=/44142026/11/11 23:11:57 INFO Completed upload id=15 objects=3 duration=1.818877ms44152026/11/11 23:11:57 INFO Completed push id=16 objects=16 duration=3.034209ms44162026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044172026/11/11 23:11:57 INFO Vacuumed table table=pending_closures44182026/11/11 23:11:57 INFO Received complete upload request method=POST path=/44192026/11/11 23:11:57 INFO Vacuumed table table=pending_objects44202026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads44212026/11/11 23:11:57 INFO Vacuumed table table=closures44222026/11/11 23:11:57 INFO Completed upload id=21 objects=25 duration=1.389693ms44232026/11/11 23:11:57 INFO Vacuumed table table=objects44242026/11/11 23:11:57 INFO Received complete push request method=POST path=/44252026/11/11 23:11:57 INFO Completed push id=28 objects=26 duration=1.226192ms44262026/11/11 23:11:57 INFO Received complete push request method=POST path=/44272026/11/11 23:11:57 INFO Completed push id=31 objects=6 duration=1.07823ms44282026/11/11 23:11:57 INFO Received push request method=POST path=/44292026/11/11 23:11:57 INFO Received uploads request method=POST path=/44302026/11/11 23:11:57 INFO Received push request method=POST path=/44312026/11/11 23:11:57 INFO Received uploads request method=POST path=/44322026/11/11 23:11:57 INFO Received push request method=POST path=/44332026/11/11 23:11:57 INFO Received complete push request method=POST path=/44342026/11/11 23:11:57 INFO Received push request method=POST path=/44352026/11/11 23:11:57 INFO Completed push id=30 objects=3 duration=968.429µs44362026/11/11 23:11:57 INFO Received uploads request method=POST path=/44372026/11/11 23:11:57 INFO Received complete push request method=POST path=/44382026/11/11 23:11:57 INFO Completed push id=32 objects=9 duration=1.494354ms44392026/11/11 23:11:57 INFO Received uploads request method=POST path=/44402026/11/11 23:11:57 INFO Received push request method=POST path=/44412026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=044422026/11/11 23:11:57 INFO Received complete upload request method=POST path=/44432026/11/11 23:11:57 INFO Completed upload id=20 objects=4 duration=1.003269ms44442026/11/11 23:11:57 INFO Received uploads request method=POST path=/44452026/11/11 23:11:57 INFO Received complete push request method=POST path=/44462026/11/11 23:11:57 INFO Completed push id=32 objects=19 duration=1.201311ms44472026/11/11 23:11:57 INFO Received uploads request method=POST path=/44482026/11/11 23:11:57 INFO Received push request method=POST path=/44492026/11/11 23:11:57 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."44502026/11/11 23:11:57 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."44512026/11/11 23:11:57 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."44522026/11/11 23:11:57 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."44532026/11/11 23:11:57 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."44542026/11/11 23:11:57 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."44552026/11/11 23:11:57 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."44562026/11/11 23:11:57 INFO Received complete upload request method=POST path=/44572026/11/11 23:11:57 INFO Completed upload id=33 objects=9 duration=994.25µs44582026/11/11 23:11:57 INFO Received complete upload request method=POST path=/44592026/11/11 23:11:57 INFO Completed upload id=25 objects=6 duration=7.386111ms44602026/11/11 23:11:57 INFO Received uploads request method=POST path=/44612026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=044622026/11/11 23:11:57 INFO Received complete push request method=POST path=/44632026/11/11 23:11:57 INFO Received complete upload request method=POST path=/44642026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period44652026/11/11 23:11:57 INFO Completed upload id=12 objects=6 duration=1.357753ms44662026/11/11 23:11:57 INFO Completed push id=5 objects=28 duration=1.900938ms44672026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044682026/11/11 23:11:57 INFO Received uploads request method=POST path=/44692026/11/11 23:11:57 INFO Vacuumed table table=pending_closures44702026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=044712026/11/11 23:11:57 INFO Vacuumed table table=pending_objects44722026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads44732026/11/11 23:11:57 INFO Received complete push request method=POST path=/44742026/11/11 23:11:57 INFO Completed push id=16 objects=5 duration=1.298932ms44752026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period44762026/11/11 23:11:57 INFO Vacuumed table table=closures44772026/11/11 23:11:57 INFO Vacuumed table table=objects44782026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044792026/11/11 23:11:57 INFO Received complete push request method=POST path=/44802026/11/11 23:11:57 INFO Vacuumed table table=pending_closures44812026/11/11 23:11:57 INFO Completed push id=17 objects=22 duration=2.13518ms44822026/11/11 23:11:57 INFO Received complete push request method=POST path=/44832026/11/11 23:11:57 INFO Vacuumed table table=pending_objects44842026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads44852026/11/11 23:11:57 INFO Vacuumed table table=closures44862026/11/11 23:11:57 INFO Received complete push request method=POST path=/44872026/11/11 23:11:57 INFO Completed push id=17 objects=3 duration=1.665016ms44882026/11/11 23:11:57 INFO Vacuumed table table=objects44892026/11/11 23:11:57 INFO Completed push id=10 objects=2 duration=1.684676ms44902026/11/11 23:11:57 INFO Received push request method=POST path=/44912026/11/11 23:11:57 INFO Received uploads request method=POST path=/44922026/11/11 23:11:57 INFO Received uploads request method=POST path=/44932026/11/11 23:11:57 INFO Received uploads request method=POST path=/44942026/11/11 23:11:57 INFO Received uploads request method=POST path=/44952026/11/11 23:11:57 INFO Received complete push request method=POST path=/44962026/11/11 23:11:57 INFO Received complete upload request method=POST path=/44972026/11/11 23:11:57 INFO Received uploads request method=POST path=/44982026/11/11 23:11:57 INFO Received push request method=POST path=/44992026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045002026/11/11 23:11:57 INFO Completed push id=23 objects=5 duration=1.222192ms45012026/11/11 23:11:57 INFO Completed upload id=17 objects=5 duration=1.436774ms45022026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=045032026/11/11 23:11:57 INFO Vacuumed table table=pending_closures45042026/11/11 23:11:57 INFO Vacuumed table table=pending_objects45052026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads45062026/11/11 23:11:57 INFO Vacuumed table table=closures45072026/11/11 23:11:57 INFO Vacuumed table table=objects45082026/11/11 23:11:57 INFO Received complete push request method=POST path=/45092026/11/11 23:11:57 INFO Completed push id=16 objects=7 duration=1.233492ms45102026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=045112026/11/11 23:11:57 INFO Vacuumed table table=pending_closures45122026/11/11 23:11:57 INFO Vacuumed table table=pending_objects45132026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads45142026/11/11 23:11:57 INFO Vacuumed table table=closures45152026/11/11 23:11:57 INFO Received complete push request method=POST path=/45162026/11/11 23:11:57 INFO Vacuumed table table=objects45172026/11/11 23:11:57 INFO Received complete push request method=POST path=/45182026/11/11 23:11:57 INFO Received complete upload request method=POST path=/45192026/11/11 23:11:57 INFO Received complete upload request method=POST path=/45202026/11/11 23:11:57 INFO Completed push id=11 objects=9 duration=1.470014ms45212026/11/11 23:11:57 INFO Completed upload id=24 objects=3 duration=924.548µs45222026/11/11 23:11:57 INFO Completed push id=19 objects=14 duration=2.294382ms45232026/11/11 23:11:57 INFO Completed upload id=20 objects=3 duration=1.453994ms45242026/11/11 23:11:57 INFO Received complete push request method=POST path=/45252026/11/11 23:11:57 INFO Completed push id=18 objects=6 duration=2.459024ms45262026/11/11 23:11:57 INFO Received complete push request method=POST path=/45272026/11/11 23:11:57 INFO Completed push id=19 objects=3 duration=1.105411ms45282026/11/11 23:11:57 INFO Received uploads request method=POST path=/45292026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=045302026/11/11 23:11:57 INFO Received complete push request method=POST path=/45312026/11/11 23:11:57 INFO Completed push id=23 objects=6 duration=1.05727ms45322026/11/11 23:11:57 INFO Received complete push request method=POST path=/45332026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045342026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=045352026/11/11 23:11:57 INFO Vacuumed table table=pending_closures45362026/11/11 23:11:57 INFO Completed push id=25 objects=4 duration=1.203012ms45372026/11/11 23:11:57 INFO Received push request method=POST path=/45382026/11/11 23:11:57 INFO Vacuumed table table=pending_objects45392026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads45402026/11/11 23:11:57 INFO Received complete upload request method=POST path=/45412026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period45422026/11/11 23:11:57 INFO Vacuumed table table=closures45432026/11/11 23:11:57 INFO Received complete push request method=POST path=/45442026/11/11 23:11:57 INFO Vacuumed table table=objects45452026/11/11 23:11:57 INFO Completed upload id=23 objects=9 duration=1.302812ms45462026/11/11 23:11:57 INFO Completed push id=15 objects=24 duration=1.423874ms45472026/11/11 23:11:57 INFO Received push request method=POST path=/45482026/11/11 23:11:57 INFO Received complete push request method=POST path=/45492026/11/11 23:11:57 INFO Completed push id=24 objects=6 duration=1.06445ms45502026/11/11 23:11:57 INFO Received uploads request method=POST path=/45512026/11/11 23:11:57 INFO Received complete push request method=POST path=/45522026/11/11 23:11:57 INFO Completed push id=20 objects=4 duration=933.529µs45532026/11/11 23:11:57 INFO Received complete upload request method=POST path=/45542026/11/11 23:11:57 INFO Completed upload id=19 objects=2 duration=885.388µs45552026/11/11 23:11:57 INFO Received complete push request method=POST path=/45562026/11/11 23:11:57 INFO Completed push id=19 objects=14 duration=1.366873ms45572026/11/11 23:11:57 INFO Received uploads request method=POST path=/45582026/11/11 23:11:57 INFO Received push request method=POST path=/45592026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=045602026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=045612026/11/11 23:11:57 INFO Received complete upload request method=POST path=/45622026/11/11 23:11:57 INFO Vacuumed table table=pending_closures45632026/11/11 23:11:57 INFO Completed upload id=24 objects=7 duration=889.349µs45642026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period45652026/11/11 23:11:57 INFO Received complete push request method=POST path=/45662026/11/11 23:11:57 INFO Vacuumed table table=pending_objects45672026/11/11 23:11:57 INFO Completed push id=25 objects=7 duration=1.216532ms45682026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads45692026/11/11 23:11:57 INFO Received complete upload request method=POST path=/45702026/11/11 23:11:57 INFO Vacuumed table table=closures45712026/11/11 23:11:57 INFO Completed upload id=24 objects=7 duration=1.03937ms45722026/11/11 23:11:57 INFO Vacuumed table table=objects45732026/11/11 23:11:57 INFO Received uploads request method=POST path=/45742026/11/11 23:11:57 INFO Received complete upload request method=POST path=/45752026/11/11 23:11:57 INFO Received push request method=POST path=/45762026/11/11 23:11:57 INFO Completed upload id=22 objects=8 duration=911.849µs45772026/11/11 23:11:57 INFO Received complete push request method=POST path=/45782026/11/11 23:11:57 INFO Completed push id=26 objects=23 duration=1.171311ms45792026/11/11 23:11:57 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."45802026/11/11 23:11:57 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."45812026/11/11 23:11:57 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."45822026/11/11 23:11:57 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."45832026/11/11 23:11:57 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."45842026/11/11 23:11:57 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."45852026/11/11 23:11:57 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."45862026/11/11 23:11:57 INFO Received complete push request method=POST path=/45872026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=045882026/11/11 23:11:57 INFO Received uploads request method=POST path=/45892026/11/11 23:11:57 INFO Received push request method=POST path=/45902026/11/11 23:11:57 INFO Completed push id=26 objects=3 duration=1.873797ms45912026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045922026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=045932026/11/11 23:11:57 INFO Vacuumed table table=pending_closures45942026/11/11 23:11:57 INFO Received complete push request method=POST path=/45952026/11/11 23:11:57 INFO Vacuumed table table=pending_objects45962026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads45972026/11/11 23:11:57 INFO Completed push id=23 objects=5 duration=1.219991ms45982026/11/11 23:11:57 INFO Vacuumed table table=closures45992026/11/11 23:11:57 INFO Vacuumed table table=objects46002026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046012026/11/11 23:11:57 INFO Vacuumed table table=pending_closures46022026/11/11 23:11:57 INFO Received complete upload request method=POST path=/46032026/11/11 23:11:57 INFO Vacuumed table table=pending_objects46042026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads46052026/11/11 23:11:57 INFO Vacuumed table table=closures46062026/11/11 23:11:57 INFO Completed upload id=20 objects=6 duration=1.213371ms46072026/11/11 23:11:57 INFO Vacuumed table table=objects46082026/11/11 23:11:57 INFO Received complete upload request method=POST path=/46092026/11/11 23:11:57 INFO Completed upload id=28 objects=8 duration=814.467µs46102026/11/11 23:11:57 INFO Received uploads request method=POST path=/46112026/11/11 23:11:57 INFO Received complete upload request method=POST path=/46122026/11/11 23:11:57 INFO Completed upload id=26 objects=24 duration=1.252412ms46132026/11/11 23:11:57 INFO Received complete upload request method=POST path=/46142026/11/11 23:11:57 INFO Completed upload id=21 objects=3 duration=1.05103ms46152026/11/11 23:11:57 INFO Received complete upload request method=POST path=/46162026/11/11 23:11:57 INFO Received uploads request method=POST path=/46172026/11/11 23:11:57 INFO Received complete push request method=POST path=/46182026/11/11 23:11:57 INFO Completed upload id=24 objects=8 duration=1.227132ms46192026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=046202026/11/11 23:11:57 INFO Completed push id=27 objects=15 duration=1.05487ms46212026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046222026/11/11 23:11:57 INFO Vacuumed table table=pending_closures46232026/11/11 23:11:57 INFO Received complete upload request method=POST path=/46242026/11/11 23:11:57 INFO Vacuumed table table=pending_objects46252026/11/11 23:11:57 INFO Received push request method=POST path=/46262026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads46272026/11/11 23:11:57 INFO Completed upload id=29 objects=10 duration=1.01459ms46282026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=046292026/11/11 23:11:57 INFO Vacuumed table table=closures46302026/11/11 23:11:57 INFO Vacuumed table table=objects46312026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=046322026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period46332026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046342026/11/11 23:11:57 INFO Vacuumed table table=pending_closures46352026/11/11 23:11:57 INFO Received uploads request method=POST path=/46362026/11/11 23:11:57 INFO Vacuumed table table=pending_objects46372026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads46382026/11/11 23:11:57 INFO Vacuumed table table=closures46392026/11/11 23:11:57 INFO Received complete upload request method=POST path=/46402026/11/11 23:11:57 INFO Vacuumed table table=objects46412026/11/11 23:11:57 INFO Completed upload id=28 objects=5 duration=821.888µs46422026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=046432026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period46442026/11/11 23:11:57 INFO Received complete upload request method=POST path=/46452026/11/11 23:11:57 INFO Completed upload id=30 objects=20 duration=973.049µs46462026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046472026/11/11 23:11:57 INFO Vacuumed table table=pending_closures46482026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=7 objects-failed-to-delete=046492026/11/11 23:11:57 INFO Vacuumed table table=pending_objects46502026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads46512026/11/11 23:11:57 INFO Vacuumed table table=closures46522026/11/11 23:11:57 INFO Vacuumed table table=pending_closures46532026/11/11 23:11:57 INFO Vacuumed table table=objects46542026/11/11 23:11:57 INFO Vacuumed table table=pending_objects46552026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads46562026/11/11 23:11:57 INFO Vacuumed table table=closures46572026/11/11 23:11:57 INFO Received complete upload request method=POST path=/46582026/11/11 23:11:57 INFO Received push request method=POST path=/46592026/11/11 23:11:57 INFO Vacuumed table table=objects46602026/11/11 23:11:57 INFO Completed upload id=27 objects=26 duration=1.064491ms46612026/11/11 23:11:57 INFO Received complete push request method=POST path=/46622026/11/11 23:11:57 INFO Completed push id=31 objects=11 duration=1.05817ms46632026/11/11 23:11:57 INFO Received complete upload request method=POST path=/46642026/11/11 23:11:57 INFO Completed upload id=30 objects=5 duration=871.568µs46652026/11/11 23:11:57 INFO Received complete push request method=POST path=/46662026/11/11 23:11:57 INFO Received complete upload request method=POST path=/46672026/11/11 23:11:57 INFO Completed push id=16 objects=26 duration=2.316822ms46682026/11/11 23:11:57 INFO Completed upload id=21 objects=2 duration=1.05873ms46692026/11/11 23:11:57 INFO Received push request method=POST path=/46702026/11/11 23:11:57 INFO Received complete push request method=POST path=/46712026/11/11 23:11:57 INFO Received complete push request method=POST path=/46722026/11/11 23:11:57 INFO Completed push id=28 objects=6 duration=1.09869ms46732026/11/11 23:11:57 INFO Completed push id=28 objects=19 duration=1.372113ms46742026/11/11 23:11:57 INFO Received complete push request method=POST path=/46752026/11/11 23:11:57 INFO Received complete upload request method=POST path=/46762026/11/11 23:11:57 INFO Completed push id=23 objects=28 duration=2.440083ms46772026/11/11 23:11:57 INFO Completed upload id=32 objects=2 duration=975.129µs46782026/11/11 23:11:57 INFO Received complete push request method=POST path=/46792026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=046802026/11/11 23:11:57 INFO Completed push id=31 objects=3 duration=1.009629ms46812026/11/11 23:11:57 INFO Received uploads request method=POST path=/46822026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period46832026/11/11 23:11:57 INFO Received uploads request method=POST path=/46842026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=046852026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046862026/11/11 23:11:57 INFO Received complete push request method=POST path=/46872026/11/11 23:11:57 INFO Vacuumed table table=pending_closures46882026/11/11 23:11:57 INFO Vacuumed table table=pending_objects46892026/11/11 23:11:57 INFO Completed push id=29 objects=6 duration=1.00259ms46902026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads46912026/11/11 23:11:57 INFO Vacuumed table table=closures46922026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period46932026/11/11 23:11:57 INFO Vacuumed table table=objects46942026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046952026/11/11 23:11:57 INFO Received complete push request method=POST path=/46962026/11/11 23:11:57 INFO Vacuumed table table=pending_closures46972026/11/11 23:11:57 INFO Completed push id=33 objects=3 duration=833.308µs46982026/11/11 23:11:57 INFO Vacuumed table table=pending_objects46992026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads47002026/11/11 23:11:57 INFO Vacuumed table table=closures47012026/11/11 23:11:57 INFO Vacuumed table table=objects47022026/11/11 23:11:57 INFO Received push request method=POST path=/47032026/11/11 23:11:57 INFO Received complete push request method=POST path=/47042026/11/11 23:11:57 INFO Completed push id=29 objects=10 duration=1.160911ms47052026/11/11 23:11:57 INFO Received push request method=POST path=/47062026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=047072026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period47082026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047092026/11/11 23:11:57 INFO Received complete upload request method=POST path=/47102026/11/11 23:11:57 INFO Received complete upload request method=POST path=/47112026/11/11 23:11:57 INFO Vacuumed table table=pending_closures47122026/11/11 23:11:57 INFO Completed upload id=34 objects=3 duration=965.549µs47132026/11/11 23:11:57 INFO Completed upload id=27 objects=8 duration=851.248µs47142026/11/11 23:11:57 INFO Vacuumed table table=pending_objects47152026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads47162026/11/11 23:11:57 INFO Vacuumed table table=closures47172026/11/11 23:11:57 INFO Vacuumed table table=objects47182026/11/11 23:11:57 INFO Received complete push request method=POST path=/47192026/11/11 23:11:57 INFO Received complete push request method=POST path=/47202026/11/11 23:11:57 INFO Received complete upload request method=POST path=/47212026/11/11 23:11:57 INFO Completed push id=27 objects=6 duration=2.424963ms47222026/11/11 23:11:57 INFO Completed push id=32 objects=6 duration=1.432773ms47232026/11/11 23:11:57 INFO Completed upload id=13 objects=14 duration=3.066808ms47242026/11/11 23:11:57 INFO Received complete push request method=POST path=/47252026/11/11 23:11:57 INFO Received complete upload request method=POST path=/47262026/11/11 23:11:57 INFO Completed push id=22 objects=3 duration=3.413672ms47272026/11/11 23:11:57 INFO Completed upload id=30 objects=8 duration=2.242602ms47282026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=047292026/11/11 23:11:57 INFO Received complete upload request method=POST path=/47302026/11/11 23:11:57 INFO Received push request method=POST path=/47312026/11/11 23:11:57 INFO Completed upload id=30 objects=6 duration=1.212712ms47322026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period47332026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047342026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=047352026/11/11 23:11:57 INFO Received complete upload request method=POST path=/47362026/11/11 23:11:57 INFO Completed upload id=33 objects=7 duration=933.729µs47372026/11/11 23:11:57 INFO Vacuumed table table=pending_closures47382026/11/11 23:11:57 INFO Vacuumed table table=pending_objects47392026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads47402026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047412026/11/11 23:11:57 INFO Vacuumed table table=closures47422026/11/11 23:11:57 INFO Vacuumed table table=objects47432026/11/11 23:11:57 INFO Vacuumed table table=pending_closures47442026/11/11 23:11:57 INFO Received uploads request method=POST path=/47452026/11/11 23:11:57 INFO Vacuumed table table=pending_objects47462026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads47472026/11/11 23:11:57 INFO Vacuumed table table=closures47482026/11/11 23:11:57 INFO Vacuumed table table=objects47492026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=047502026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047512026/11/11 23:11:57 INFO Vacuumed table table=pending_closures47522026/11/11 23:11:57 INFO Received uploads request method=POST path=/47532026/11/11 23:11:57 INFO Received complete upload request method=POST path=/47542026/11/11 23:11:57 INFO Vacuumed table table=pending_objects47552026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads47562026/11/11 23:11:57 INFO Received complete upload request method=POST path=/47572026/11/11 23:11:57 INFO Completed upload id=31 objects=5 duration=911.709µs47582026/11/11 23:11:57 INFO Vacuumed table table=closures47592026/11/11 23:11:57 INFO Completed upload id=31 objects=6 duration=1.01565ms47602026/11/11 23:11:57 INFO Received complete upload request method=POST path=/47612026/11/11 23:11:57 INFO Vacuumed table table=objects47622026/11/11 23:11:57 INFO Received complete upload request method=POST path=/47632026/11/11 23:11:57 INFO Completed upload id=34 objects=3 duration=1.314472ms47642026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=047652026/11/11 23:11:57 INFO Completed upload id=15 objects=2 duration=1.156031ms47662026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=047672026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period47682026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047692026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047702026/11/11 23:11:57 INFO Vacuumed table table=pending_closures47712026/11/11 23:11:57 INFO Vacuumed table table=pending_closures47722026/11/11 23:11:57 INFO Vacuumed table table=pending_objects47732026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads47742026/11/11 23:11:57 INFO Vacuumed table table=closures47752026/11/11 23:11:57 INFO Vacuumed table table=pending_objects47762026/11/11 23:11:57 INFO Vacuumed table table=objects47772026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads47782026/11/11 23:11:57 INFO Vacuumed table table=closures47792026/11/11 23:11:57 INFO Vacuumed table table=objects47802026/11/11 23:11:57 INFO Received complete upload request method=POST path=/47812026/11/11 23:11:57 INFO Received complete upload request method=POST path=/47822026/11/11 23:11:57 INFO Completed upload id=34 objects=3 duration=825.728µs47832026/11/11 23:11:57 INFO Completed upload id=32 objects=4 duration=827.408µs47842026/11/11 23:11:57 INFO Received push request method=POST path=/47852026/11/11 23:11:57 INFO Received push request method=POST path=/47862026/11/11 23:11:57 INFO Received complete push request method=POST path=/47872026/11/11 23:11:57 INFO Completed push id=35 objects=11 duration=1.147411ms47882026/11/11 23:11:57 INFO Received complete push request method=POST path=/47892026/11/11 23:11:57 INFO Received complete upload request method=POST path=/47902026/11/11 23:11:57 INFO Completed upload id=19 objects=2 duration=1.639136ms47912026/11/11 23:11:57 INFO Completed push id=31 objects=6 duration=2.705506ms47922026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=047932026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047942026/11/11 23:11:57 INFO Vacuumed table table=pending_closures47952026/11/11 23:11:57 INFO Received complete push request method=POST path=/47962026/11/11 23:11:57 INFO Vacuumed table table=pending_objects47972026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=047982026/11/11 23:11:57 INFO Completed push id=24 objects=3 duration=889.288µs47992026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads48002026/11/11 23:11:57 INFO Vacuumed table table=closures48012026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period48022026/11/11 23:11:57 INFO Vacuumed table table=objects48032026/11/11 23:11:57 INFO Received complete push request method=POST path=/48042026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048052026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=048062026/11/11 23:11:57 INFO Received uploads request method=POST path=/48072026/11/11 23:11:57 INFO Received complete upload request method=POST path=/48082026/11/11 23:11:57 INFO Vacuumed table table=pending_closures48092026/11/11 23:11:57 INFO Completed upload id=33 objects=10 duration=1.943919ms48102026/11/11 23:11:57 INFO Received uploads request method=POST path=/48112026/11/11 23:11:57 INFO Completed push id=24 objects=24 duration=4.029958ms48122026/11/11 23:11:57 INFO Vacuumed table table=pending_objects48132026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads48142026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period48152026/11/11 23:11:57 INFO Vacuumed table table=closures48162026/11/11 23:11:57 INFO Vacuumed table table=objects48172026/11/11 23:11:57 INFO Received push request method=POST path=/48182026/11/11 23:11:57 INFO Received complete upload request method=POST path=/48192026/11/11 23:11:57 INFO Completed upload id=25 objects=16 duration=2.004479ms48202026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=048212026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period48222026/11/11 23:11:57 INFO Received complete push request method=POST path=/48232026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048242026/11/11 23:11:57 INFO Completed push id=33 objects=7 duration=1.517675ms48252026/11/11 23:11:57 INFO Vacuumed table table=pending_closures48262026/11/11 23:11:57 INFO Vacuumed table table=pending_objects48272026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads48282026/11/11 23:11:57 INFO Received complete upload request method=POST path=/48292026/11/11 23:11:57 INFO Received push request method=POST path=/48302026/11/11 23:11:57 INFO Vacuumed table table=closures48312026/11/11 23:11:57 INFO Completed upload id=34 objects=23 duration=1.270792ms48322026/11/11 23:11:57 INFO Vacuumed table table=objects48332026/11/11 23:11:57 INFO Received complete upload request method=POST path=/48342026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=048352026/11/11 23:11:57 INFO Completed upload id=26 objects=3 duration=913.589µs48362026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period48372026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048382026/11/11 23:11:57 INFO Vacuumed table table=pending_closures48392026/11/11 23:11:57 INFO Vacuumed table table=pending_objects48402026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads48412026/11/11 23:11:57 INFO Vacuumed table table=closures48422026/11/11 23:11:57 INFO Vacuumed table table=objects48432026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=048442026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period48452026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048462026/11/11 23:11:57 INFO Received uploads request method=POST path=/48472026/11/11 23:11:57 INFO Received uploads request method=POST path=/48482026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=048492026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period48502026/11/11 23:11:57 INFO Vacuumed table table=pending_closures48512026/11/11 23:11:57 INFO Vacuumed table table=pending_objects48522026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048532026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads48542026/11/11 23:11:57 INFO Received complete push request method=POST path=/48552026/11/11 23:11:57 INFO Vacuumed table table=pending_closures48562026/11/11 23:11:57 INFO Vacuumed table table=closures48572026/11/11 23:11:57 INFO Completed push id=35 objects=6 duration=913.288µs48582026/11/11 23:11:57 INFO Vacuumed table table=objects48592026/11/11 23:11:57 INFO Vacuumed table table=pending_objects48602026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads48612026/11/11 23:11:57 INFO Vacuumed table table=closures48622026/11/11 23:11:57 INFO Vacuumed table table=objects48632026/11/11 23:11:57 INFO Received push request method=POST path=/48642026/11/11 23:11:57 INFO Received uploads request method=POST path=/48652026/11/11 23:11:57 INFO Received uploads request method=POST path=/48662026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=048672026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period48682026/11/11 23:11:57 INFO Received complete push request method=POST path=/48692026/11/11 23:11:57 INFO Completed push id=1 objects=9 duration=1.237472ms48702026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=048712026/11/11 23:11:57 INFO Vacuumed table table=pending_closures48722026/11/11 23:11:57 INFO Received push request method=POST path=/48732026/11/11 23:11:57 INFO Vacuumed table table=pending_objects48742026/11/11 23:11:57 INFO Received complete upload request method=POST path=/48752026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads48762026/11/11 23:11:57 INFO Vacuumed table table=closures48772026/11/11 23:11:57 INFO Completed upload id=26 objects=8 duration=1.08811ms48782026/11/11 23:11:57 INFO Vacuumed table table=objects48792026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=048802026/11/11 23:11:57 INFO Received push request method=POST path=/48812026/11/11 23:11:57 INFO Received complete push request method=POST path=/48822026/11/11 23:11:57 INFO Completed push id=28 objects=14 duration=1.225392ms48832026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=048842026/11/11 23:11:57 INFO Vacuumed table table=pending_closures48852026/11/11 23:11:57 INFO Vacuumed table table=pending_objects48862026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads48872026/11/11 23:11:57 INFO Received complete push request method=POST path=/48882026/11/11 23:11:57 INFO Vacuumed table table=closures48892026/11/11 23:11:57 INFO Vacuumed table table=objects48902026/11/11 23:11:57 INFO Received uploads request method=POST path=/48912026/11/11 23:11:57 INFO Completed push id=9 objects=9 duration=2.778267ms48922026/11/11 23:11:57 INFO Received push request method=POST path=/48932026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=048942026/11/11 23:11:57 INFO Received uploads request method=POST path=/48952026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period48962026/11/11 23:11:57 INFO Received complete upload request method=POST path=/48972026/11/11 23:11:57 INFO Received complete upload request method=POST path=/48982026/11/11 23:11:57 INFO Completed upload id=21 objects=8 duration=991.25µs48992026/11/11 23:11:57 INFO Completed upload id=2 objects=7 duration=930.869µs49002026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049012026/11/11 23:11:57 INFO Received uploads request method=POST path=/49022026/11/11 23:11:57 INFO Vacuumed table table=pending_closures49032026/11/11 23:11:57 INFO Vacuumed table table=pending_objects49042026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads49052026/11/11 23:11:57 INFO Vacuumed table table=closures49062026/11/11 23:11:57 INFO Vacuumed table table=objects49072026/11/11 23:11:57 INFO Received uploads request method=POST path=/49082026/11/11 23:11:57 INFO Received complete push request method=POST path=/49092026/11/11 23:11:57 INFO Completed push id=22 objects=6 duration=1.147531ms49102026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49112026/11/11 23:11:57 INFO Completed upload id=22 objects=5 duration=1.584635ms49122026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=049132026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49142026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period49152026/11/11 23:11:57 INFO Completed upload id=26 objects=2 duration=1.676736ms49162026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049172026/11/11 23:11:57 INFO Vacuumed table table=pending_closures49182026/11/11 23:11:57 INFO Received complete push request method=POST path=/49192026/11/11 23:11:57 INFO Completed push id=24 objects=8 duration=1.06095ms49202026/11/11 23:11:57 INFO Vacuumed table table=pending_objects49212026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads49222026/11/11 23:11:57 INFO Vacuumed table table=closures49232026/11/11 23:11:57 INFO Vacuumed table table=objects49242026/11/11 23:11:57 INFO Received complete push request method=POST path=/49252026/11/11 23:11:57 INFO Received push request method=POST path=/49262026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=049272026/11/11 23:11:57 INFO Completed push id=28 objects=3 duration=1.727396ms49282026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49292026/11/11 23:11:57 INFO Completed upload id=8 objects=6 duration=1.005629ms49302026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49312026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049322026/11/11 23:11:57 INFO Completed upload id=25 objects=9 duration=862.269µs49332026/11/11 23:11:57 INFO Vacuumed table table=pending_closures49342026/11/11 23:11:57 INFO Vacuumed table table=pending_objects49352026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads49362026/11/11 23:11:57 INFO Vacuumed table table=closures49372026/11/11 23:11:57 INFO Received uploads request method=POST path=/49382026/11/11 23:11:57 INFO Vacuumed table table=objects49392026/11/11 23:11:57 INFO Received complete push request method=POST path=/49402026/11/11 23:11:57 INFO Completed push id=29 objects=15 duration=1.04341ms49412026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49422026/11/11 23:11:57 INFO Completed upload id=27 objects=3 duration=906.069µs49432026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49442026/11/11 23:11:57 INFO Completed upload id=23 objects=7 duration=2.805046ms49452026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49462026/11/11 23:11:57 INFO Completed upload id=21 objects=9 duration=1.474454ms49472026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49482026/11/11 23:11:57 INFO Completed upload id=28 objects=6 duration=614.446µs49492026/11/11 23:11:57 INFO Received push request method=POST path=/49502026/11/11 23:11:57 INFO Received complete push request method=POST path=/49512026/11/11 23:11:57 INFO Completed push id=30 objects=6 duration=2.013099ms49522026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49532026/11/11 23:11:57 INFO Completed upload id=26 objects=5 duration=822.628µs49542026/11/11 23:11:57 INFO Received complete push request method=POST path=/49552026/11/11 23:11:57 INFO Received complete push request method=POST path=/49562026/11/11 23:11:57 INFO Completed push id=30 objects=4 duration=906.849µs49572026/11/11 23:11:57 INFO Completed push id=25 objects=10 duration=1.786117ms49582026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49592026/11/11 23:11:57 INFO Completed upload id=12 objects=9 duration=882.389µs49602026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49612026/11/11 23:11:57 INFO Completed upload id=31 objects=3 duration=732.227µs49622026/11/11 23:11:57 INFO Received complete push request method=POST path=/49632026/11/11 23:11:57 INFO Completed push id=27 objects=3 duration=1.515854ms49642026/11/11 23:11:57 INFO Received complete push request method=POST path=/49652026/11/11 23:11:57 INFO Completed push id=31 objects=7 duration=1.293573ms49662026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49672026/11/11 23:11:57 INFO Completed upload id=29 objects=5 duration=998.03µs49682026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=049692026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=049702026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49712026/11/11 23:11:57 INFO Completed upload id=28 objects=5 duration=848.388µs49722026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49732026/11/11 23:11:57 INFO Completed upload id=32 objects=5 duration=909.048µs49742026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period49752026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049762026/11/11 23:11:57 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."49772026/11/11 23:11:57 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."49782026/11/11 23:11:57 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."49792026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49802026/11/11 23:11:57 INFO Vacuumed table table=pending_closures49812026/11/11 23:11:57 INFO Completed upload id=30 objects=2 duration=1.306992ms49822026/11/11 23:11:57 INFO Vacuumed table table=pending_objects49832026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads49842026/11/11 23:11:57 INFO Vacuumed table table=closures49852026/11/11 23:11:57 INFO Vacuumed table table=objects49862026/11/11 23:11:57 INFO Received complete push request method=POST path=/49872026/11/11 23:11:57 INFO Completed push id=29 objects=17 duration=1.02711ms49882026/11/11 23:11:57 INFO Received push request method=POST path=/49892026/11/11 23:11:57 INFO Received complete push request method=POST path=/49902026/11/11 23:11:57 INFO Completed push id=31 objects=20 duration=1.096511ms49912026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=049922026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period49932026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049942026/11/11 23:11:57 INFO Vacuumed table table=pending_closures49952026/11/11 23:11:57 INFO Vacuumed table table=pending_objects49962026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads49972026/11/11 23:11:57 INFO Received complete upload request method=POST path=/49982026/11/11 23:11:57 INFO Vacuumed table table=closures49992026/11/11 23:11:57 INFO Completed upload id=32 objects=5 duration=936.109µs50002026/11/11 23:11:57 INFO Vacuumed table table=objects50012026/11/11 23:11:57 INFO Received push request method=POST path=/50022026/11/11 23:11:57 INFO Received complete upload request method=POST path=/50032026/11/11 23:11:57 INFO Completed upload id=34 objects=24 duration=1.10425ms50042026/11/11 23:11:57 INFO Received complete upload request method=POST path=/50052026/11/11 23:11:57 INFO Completed upload id=33 objects=3 duration=799.947µs50062026/11/11 23:11:57 INFO Received complete push request method=POST path=/50072026/11/11 23:11:57 INFO Completed push id=28 objects=11 duration=2.0838ms50082026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=050092026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period50102026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050112026/11/11 23:11:57 INFO Garbage collection completed failed-uploads-deleted=6 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050122026/11/11 23:11:57 INFO Vacuumed table table=pending_closures50132026/11/11 23:11:57 INFO Vacuumed table table=pending_closures50142026/11/11 23:11:57 INFO Aborted multipart uploads count=0 kept=050152026/11/11 23:11:57 INFO Vacuumed table table=pending_objects50162026/11/11 23:11:57 INFO Vacuumed table table=pending_objects50172026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads50182026/11/11 23:11:57 INFO Vacuumed table table=multipart_uploads50192026/11/11 23:11:57 WARN Force mode enabled - objects will be deleted immediately without grace period50202026/11/11 23:11:57 INFO Vacuumed table table=closures50212026/11/11 23:11:57 INFO Vacuumed table table=closures50222026/11/11 23:11:57 INFO Vacuumed table table=objects50232026/11/11 23:11:57 INFO Vacuumed table table=objects50242026/11/11 23:11:58 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050252026/11/11 23:11:58 INFO Received complete upload request method=POST path=/50262026/11/11 23:11:58 INFO Vacuumed table table=pending_closures50272026/11/11 23:11:58 INFO Completed upload id=18 objects=22 duration=1.03819ms50282026/11/11 23:11:58 INFO Vacuumed table table=pending_objects50292026/11/11 23:11:58 INFO Vacuumed table table=multipart_uploads50302026/11/11 23:11:58 INFO Vacuumed table table=closures50312026/11/11 23:11:58 INFO Vacuumed table table=objects50322026/11/11 23:11:58 INFO Received complete upload request method=POST path=/50332026/11/11 23:11:58 INFO Completed upload id=21 objects=8 duration=895.069µs50342026/11/11 23:11:58 INFO Received complete push request method=POST path=/50352026/11/11 23:11:58 INFO Completed push id=29 objects=24 duration=1.502774ms50362026/11/11 23:11:58 INFO Received complete upload request method=POST path=/50372026/11/11 23:11:58 INFO Completed upload id=22 objects=7 duration=769.527µs50382026/11/11 23:11:58 INFO Aborted multipart uploads count=0 kept=050392026/11/11 23:11:58 WARN Force mode enabled - objects will be deleted immediately without grace period50402026/11/11 23:11:58 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=050412026/11/11 23:11:58 INFO Vacuumed table table=pending_closures50422026/11/11 23:11:58 INFO Vacuumed table table=pending_objects50432026/11/11 23:11:58 INFO Vacuumed table table=multipart_uploads50442026/11/11 23:11:58 INFO Vacuumed table table=closures50452026/11/11 23:11:58 INFO Received complete push request method=POST path=/50462026/11/11 23:11:58 INFO Vacuumed table table=objects50472026/11/11 23:11:58 INFO Completed push id=24 objects=5 duration=971.109µs50482026/11/11 23:11:58 INFO Received complete upload request method=POST path=/50492026/11/11 23:11:58 INFO Completed upload id=25 objects=8 duration=874.969µs50502026/11/11 23:11:58 INFO Received complete push request method=POST path=/50512026/11/11 23:11:58 INFO Completed push id=26 objects=4 duration=773.927µs50522026/11/11 23:11:58 INFO Received complete push request method=POST path=/50532026/11/11 23:11:58 INFO Completed push id=30 objects=2 duration=612.805µs50542026/11/11 23:11:58 INFO Received complete push request method=POST path=/50552026/11/11 23:11:58 INFO Completed push id=32 objects=7 duration=805.028µs50562026/11/11 23:11:58 INFO Received complete upload request method=POST path=/50572026/11/11 23:11:58 INFO Completed upload id=33 objects=6 duration=579.445µs50582026/11/11 23:11:58 INFO Received complete upload request method=POST path=/50592026/11/11 23:11:58 INFO Completed upload id=34 objects=3 duration=541.765µs50602026/11/11 23:11:58 INFO Aborted multipart uploads count=0 kept=050612026/11/11 23:11:58 WARN Force mode enabled - objects will be deleted immediately without grace period50622026/11/11 23:11:58 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050632026/11/11 23:11:58 INFO Vacuumed table table=pending_closures50642026/11/11 23:11:58 INFO Vacuumed table table=pending_objects50652026/11/11 23:11:58 INFO Vacuumed table table=multipart_uploads50662026/11/11 23:11:58 INFO Vacuumed table table=closures50672026/11/11 23:11:58 INFO Vacuumed table table=objects5068--- PASS: FuzzPushGC (0.01s)5069 --- PASS: FuzzPushGC/seed#11 (8.94s)5070 --- PASS: FuzzPushGC/seed#3 (9.02s)5071 --- PASS: FuzzPushGC/seed#4 (9.24s)5072 --- PASS: FuzzPushGC/seed#14 (9.60s)5073 --- PASS: FuzzPushGC/seed#7 (9.75s)5074 --- PASS: FuzzPushGC/seed#0 (9.75s)5075 --- PASS: FuzzPushGC/seed#5 (10.00s)5076 --- PASS: FuzzPushGC/seed#13 (10.05s)5077 --- PASS: FuzzPushGC/seed#12 (10.12s)5078 --- PASS: FuzzPushGC/seed#15 (10.27s)5079 --- PASS: FuzzPushGC/seed#1 (10.30s)5080 --- PASS: FuzzPushGC/seed#8 (10.33s)5081 --- PASS: FuzzPushGC/seed#9 (10.33s)5082 --- PASS: FuzzPushGC/seed#6 (10.33s)5083 --- PASS: FuzzPushGC/seed#2 (10.38s)5084 --- PASS: FuzzPushGC/seed#10 (10.50s)5085PASS50862026-11-11 23:11:58.742 UTC [129] LOG: received smart shutdown request50872026-11-11 23:11:58.748 UTC [129] LOG: background worker "logical replication launcher" (PID 206) exited with exit code 150882026-11-11 23:11:59.964 UTC [201] LOG: checkpoint complete: wrote 3426 buffers (20.9%), wrote 5 SLRU buffers; 0 WAL file(s) added, 0 removed, 33 recycled; write=11.126 s, sync=0.486 s, total=12.237 s; sync files=42369, longest=0.003 s, average=0.001 s; distance=536755 kB, estimate=536755 kB; lsn=0/26DE6718, redo lsn=0/22392B8050892026-11-11 23:11:59.970 UTC [201] LOG: shutting down50902026-11-11 23:11:59.970 UTC [201] LOG: checkpoint starting: shutdown immediate50912026-11-11 23:12:00.711 UTC [201] LOG: checkpoint complete: wrote 8452 buffers (51.6%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 4 recycled; write=0.208 s, sync=0.522 s, total=0.741 s; sync files=2543, longest=0.020 s, average=0.001 s; distance=76111 kB, estimate=490691 kB; lsn=0/26DE6790, redo lsn=0/26DE679050922026-11-11 23:12:00.891 UTC [129] LOG: database system is shut down5093Running OIDC tests...5094=== RUN TestAudienceForIssuer5095=== PAUSE TestAudienceForIssuer5096=== RUN TestGlobMatch5097=== PAUSE TestGlobMatch5098=== RUN TestValidateToken_ValidToken5099=== PAUSE TestValidateToken_ValidToken5100=== RUN TestValidateToken_WrongAudience5101=== PAUSE TestValidateToken_WrongAudience5102=== RUN TestValidateToken_Expired5103=== PAUSE TestValidateToken_Expired5104=== RUN TestValidateToken_BoundClaimsMismatch5105=== PAUSE TestValidateToken_BoundClaimsMismatch5106=== RUN TestValidateToken_BoundSubjectMismatch5107=== PAUSE TestValidateToken_BoundSubjectMismatch5108=== RUN TestValidateToken_MultipleProviders5109=== PAUSE TestValidateToken_MultipleProviders5110=== RUN TestValidateToken_NoMatchingProvider5111=== PAUSE TestValidateToken_NoMatchingProvider5112=== RUN TestValidateToken_KubernetesServiceAccount5113=== PAUSE TestValidateToken_KubernetesServiceAccount5114=== RUN TestNewValidator_KubernetesRequiresCA5115=== PAUSE TestNewValidator_KubernetesRequiresCA5116=== RUN TestValidateToken_KubernetesIssuerFromOwnToken5117=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken5118=== RUN TestPins_ReservedForMatchingRule5119=== PAUSE TestPins_ReservedForMatchingRule5120=== RUN TestPins_TopLevelShorthand5121=== PAUSE TestPins_TopLevelShorthand5122=== RUN TestPins_ConfigValidation5123=== PAUSE TestPins_ConfigValidation5124=== RUN TestScopes_LegacyProviderDefaultsToWrite5125=== PAUSE TestScopes_LegacyProviderDefaultsToWrite5126=== RUN TestScopes_Rules5127=== PAUSE TestScopes_Rules5128=== RUN TestScopes_ConfigValidation5129=== PAUSE TestScopes_ConfigValidation5130=== CONT TestAudienceForIssuer5131--- PASS: TestAudienceForIssuer (0.00s)5132=== CONT TestScopes_ConfigValidation5133--- PASS: TestScopes_ConfigValidation (0.00s)5134=== CONT TestScopes_Rules5135=== CONT TestValidateToken_NoMatchingProvider5136=== CONT TestPins_ReservedForMatchingRule5137=== CONT TestValidateToken_MultipleProviders5138=== CONT TestScopes_LegacyProviderDefaultsToWrite5139=== CONT TestValidateToken_BoundSubjectMismatch5140=== CONT TestValidateToken_KubernetesIssuerFromOwnToken5141=== CONT TestValidateToken_BoundClaimsMismatch5142=== CONT TestValidateToken_Expired5143=== CONT TestValidateToken_WrongAudience5144=== CONT TestNewValidator_KubernetesRequiresCA5145=== CONT TestValidateToken_KubernetesServiceAccount5146=== CONT TestPins_ConfigValidation5147=== CONT TestPins_TopLevelShorthand5148=== CONT TestValidateToken_ValidToken5149=== CONT TestGlobMatch5150=== RUN TestGlobMatch/foo_foo5151=== PAUSE TestGlobMatch/foo_foo5152=== RUN TestGlobMatch/foo_bar5153=== PAUSE TestGlobMatch/foo_bar5154=== RUN TestGlobMatch/*_5155=== PAUSE TestGlobMatch/*_5156=== RUN TestGlobMatch/*_anything5157=== PAUSE TestGlobMatch/*_anything5158=== RUN TestGlobMatch/foo*_foo5159=== PAUSE TestGlobMatch/foo*_foo5160=== RUN TestGlobMatch/foo*_foobar5161=== PAUSE TestGlobMatch/foo*_foobar5162=== RUN TestGlobMatch/foo*_bar5163=== PAUSE TestGlobMatch/foo*_bar5164=== RUN TestGlobMatch/*bar_bar5165=== PAUSE TestGlobMatch/*bar_bar5166=== RUN TestGlobMatch/*bar_foobar5167=== PAUSE TestGlobMatch/*bar_foobar5168=== RUN TestGlobMatch/*bar_foo5169=== PAUSE TestGlobMatch/*bar_foo5170=== RUN TestGlobMatch/foo*bar_foobar5171=== PAUSE TestGlobMatch/foo*bar_foobar5172=== RUN TestGlobMatch/foo*bar_foo123bar5173=== PAUSE TestGlobMatch/foo*bar_foo123bar5174=== RUN TestGlobMatch/foo*bar_foobarbaz5175--- PASS: TestPins_ConfigValidation (0.00s)5176=== PAUSE TestGlobMatch/foo*bar_foobarbaz5177=== RUN TestGlobMatch/*/*_foo/bar5178=== PAUSE TestGlobMatch/*/*_foo/bar5179=== RUN TestGlobMatch/*/*_foo5180=== PAUSE TestGlobMatch/*/*_foo5181=== RUN TestGlobMatch/refs/heads/*_refs/heads/main5182=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main5183=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.05184=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.05185=== RUN TestGlobMatch/refs/*/main_refs/heads/main5186=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main5187=== RUN TestGlobMatch/fo?_foo5188=== PAUSE TestGlobMatch/fo?_foo5189=== RUN TestGlobMatch/fo?_fo5190=== PAUSE TestGlobMatch/fo?_fo5191=== RUN TestGlobMatch/fo?_fooo5192=== PAUSE TestGlobMatch/fo?_fooo5193=== RUN TestGlobMatch/?oo_foo5194=== PAUSE TestGlobMatch/?oo_foo5195=== RUN TestGlobMatch/?oo_boo5196=== PAUSE TestGlobMatch/?oo_boo5197=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5198=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5199=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5200=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5201=== CONT TestGlobMatch/foo_foo5202=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5203=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5204=== CONT TestGlobMatch/?oo_boo5205=== CONT TestGlobMatch/?oo_foo5206=== CONT TestGlobMatch/fo?_fooo5207=== CONT TestGlobMatch/fo?_fo5208=== CONT TestGlobMatch/fo?_foo5209=== CONT TestGlobMatch/refs/*/main_refs/heads/main5210=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.05211=== CONT TestGlobMatch/refs/heads/*_refs/heads/main5212=== CONT TestGlobMatch/*/*_foo5213=== CONT TestGlobMatch/*/*_foo/bar5214=== CONT TestGlobMatch/foo*bar_foobarbaz5215=== CONT TestGlobMatch/foo*bar_foo123bar5216=== CONT TestGlobMatch/foo*bar_foobar5217=== CONT TestGlobMatch/*bar_foo5218=== CONT TestGlobMatch/*bar_foobar5219=== CONT TestGlobMatch/*bar_bar5220=== CONT TestGlobMatch/foo*_bar5221=== CONT TestGlobMatch/foo*_foobar5222=== CONT TestGlobMatch/foo*_foo5223=== CONT TestGlobMatch/*_anything5224=== CONT TestGlobMatch/*_5225=== CONT TestGlobMatch/foo_bar5226--- PASS: TestGlobMatch (0.00s)5227 --- PASS: TestGlobMatch/foo_foo (0.00s)5228 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)5229 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)5230 --- PASS: TestGlobMatch/?oo_boo (0.00s)5231 --- PASS: TestGlobMatch/?oo_foo (0.00s)5232 --- PASS: TestGlobMatch/fo?_fooo (0.00s)5233 --- PASS: TestGlobMatch/fo?_fo (0.00s)5234 --- PASS: TestGlobMatch/fo?_foo (0.00s)5235 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)5236 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)5237 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)5238 --- PASS: TestGlobMatch/*/*_foo (0.00s)5239 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)5240 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)5241 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)5242 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)5243 --- PASS: TestGlobMatch/*bar_foo (0.00s)5244 --- PASS: TestGlobMatch/*bar_foobar (0.00s)5245 --- PASS: TestGlobMatch/*bar_bar (0.00s)5246 --- PASS: TestGlobMatch/foo*_bar (0.00s)5247 --- PASS: TestGlobMatch/foo*_foobar (0.00s)5248 --- PASS: TestGlobMatch/foo*_foo (0.00s)5249 --- PASS: TestGlobMatch/*_anything (0.00s)5250 --- PASS: TestGlobMatch/*_ (0.00s)5251 --- PASS: TestGlobMatch/foo_bar (0.00s)52522026/11/11 23:12:05 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:40719/oidc52532026/11/11 23:12:05 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:33039/oidc52542026/11/11 23:12:15 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:45333/oidc5255=== NAME TestValidateToken_MultipleProviders5256 integration_test.go:190: failed to create validator: failed to initialize OIDC provider for http://127.0.0.1:38257/oidc: Get "http://127.0.0.1:38257/oidc/.well-known/openid-configuration": context deadline exceeded5257=== NAME TestPins_ReservedForMatchingRule5258 pins_test.go:37: OIDC validation failed: no provider could verify the token (signature or issuer mismatch)52592026/11/11 23:12:15 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:33701/oidc5260=== NAME TestScopes_Rules5261 scopes_test.go:74: map[aud:client-fa717375a00f4287b8038f05 exp:1794442335 iat:1794438735 iss:http://127.0.0.1:45333/oidc sub:sa:ci:builder]: OIDC validation failed: no provider could verify the token (signature or issuer mismatch)5262--- FAIL: TestValidateToken_MultipleProviders (11.70s)5263--- FAIL: TestPins_ReservedForMatchingRule (11.72s)5264--- PASS: TestValidateToken_NoMatchingProvider (11.72s)52652026/11/11 23:12:15 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC1235266--- FAIL: TestScopes_Rules (11.77s)5267--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (11.61s)52682026/11/11 23:12:15 http: TLS handshake error from 127.0.0.1:34442: remote error: tls: bad certificate5269--- PASS: TestNewValidator_KubernetesRequiresCA (10.37s)52702026/11/11 23:12:15 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:44959/oidc5271--- PASS: TestValidateToken_Expired (10.78s)52722026/11/11 23:12:15 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:44537/oidc52732026/11/11 23:12:15 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:41673/oidc5274--- PASS: TestValidateToken_BoundSubjectMismatch (11.70s)5275--- PASS: TestValidateToken_WrongAudience (10.40s)52762026/11/11 23:12:15 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:36861/oidc5277--- PASS: TestValidateToken_ValidToken (9.76s)52782026/11/11 23:12:15 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:396075279--- PASS: TestValidateToken_KubernetesServiceAccount (10.48s)52802026/11/11 23:12:15 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:45255/oidc5281--- PASS: TestPins_TopLevelShorthand (9.83s)52822026/11/11 23:12:15 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:34159/oidc5283--- PASS: TestScopes_LegacyProviderDefaultsToWrite (11.89s)52842026/11/11 23:12:15 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:46181/oidc5285--- PASS: TestValidateToken_BoundClaimsMismatch (11.01s)5286FAIL

failed niks3-go-unit-tests checks.aarch64-linux.go-unit-tests · build #487 · raw

failed `niks3-go-unit-tests` `checks.aarch64-linux.go-unit-tests` · build #487 · raw