this derivation will be built: /nix/store/fv7dxf6zapvyjimvqvr8bmb1f91nvq5x-niks3-go-unit-tests.drv Failed to find a machine for remote build! derivation: fv7dxf6zapvyjimvqvr8bmb1f91nvq5x-niks3-go-unit-tests.drv required (system, features): (aarch64-linux, []) 1 available machines: (systems, maxjobs, supportedFeatures, mandatoryFeatures) ([aarch64-darwin, x86_64-darwin], 8, [big-parallel, recursive-nix], []) building '/nix/store/fv7dxf6zapvyjimvqvr8bmb1f91nvq5x-niks3-go-unit-tests.drv' niks3-go-unit-tests> tribuchet: building on eliza niks3-go-unit-tests> Running client tests... niks3-go-unit-tests> === RUN TestDoServerRequestAttachesToken niks3-go-unit-tests> === PAUSE TestDoServerRequestAttachesToken niks3-go-unit-tests> === RUN TestCaseHackSuffix niks3-go-unit-tests> === PAUSE TestCaseHackSuffix niks3-go-unit-tests> === RUN TestFilterOversizedClosures niks3-go-unit-tests> === PAUSE TestFilterOversizedClosures niks3-go-unit-tests> === RUN TestPartSizeForNAR niks3-go-unit-tests> === PAUSE TestPartSizeForNAR niks3-go-unit-tests> === RUN TestUploadMultipart_SupersededByPeer niks3-go-unit-tests> === PAUSE TestUploadMultipart_SupersededByPeer niks3-go-unit-tests> === RUN TestDumpPathMatchesNix niks3-go-unit-tests> === PAUSE TestDumpPathMatchesNix niks3-go-unit-tests> === RUN TestDumpPathSingleFile niks3-go-unit-tests> === PAUSE TestDumpPathSingleFile niks3-go-unit-tests> === RUN TestDumpPathWriterError niks3-go-unit-tests> === PAUSE TestDumpPathWriterError niks3-go-unit-tests> === RUN TestEncodeNixBase32 niks3-go-unit-tests> === PAUSE TestEncodeNixBase32 niks3-go-unit-tests> === RUN TestEncodeNixBase32WithRealHash niks3-go-unit-tests> === PAUSE TestEncodeNixBase32WithRealHash niks3-go-unit-tests> === RUN TestConvertHashToNix32 niks3-go-unit-tests> === PAUSE TestConvertHashToNix32 niks3-go-unit-tests> === RUN TestGetStorePathHash niks3-go-unit-tests> === PAUSE TestGetStorePathHash niks3-go-unit-tests> === RUN TestPathInfoHashCompatibility niks3-go-unit-tests> === PAUSE TestPathInfoHashCompatibility niks3-go-unit-tests> === RUN TestParsePathInfoJSON niks3-go-unit-tests> === PAUSE TestParsePathInfoJSON niks3-go-unit-tests> === RUN TestParsePathInfoJSONMultiplePaths niks3-go-unit-tests> === PAUSE TestParsePathInfoJSONMultiplePaths niks3-go-unit-tests> === RUN TestPathInfoCACompatibility niks3-go-unit-tests> === PAUSE TestPathInfoCACompatibility niks3-go-unit-tests> === RUN TestRateLimiterFeedback niks3-go-unit-tests> === PAUSE TestRateLimiterFeedback niks3-go-unit-tests> === RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess niks3-go-unit-tests> === PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess niks3-go-unit-tests> === RUN TestResolveStorePath niks3-go-unit-tests> === PAUSE TestResolveStorePath niks3-go-unit-tests> === RUN TestDoWithRetry_BodyReplayedViaGetBody niks3-go-unit-tests> === PAUSE TestDoWithRetry_BodyReplayedViaGetBody niks3-go-unit-tests> === RUN TestShellSplit niks3-go-unit-tests> === PAUSE TestShellSplit niks3-go-unit-tests> === RUN TestShellSplitErrors niks3-go-unit-tests> === PAUSE TestShellSplitErrors niks3-go-unit-tests> === RUN TestSetClientTLS niks3-go-unit-tests> === PAUSE TestSetClientTLS niks3-go-unit-tests> === RUN TestSetClientTLSDoesNotMutateDefaultTransport niks3-go-unit-tests> === PAUSE TestSetClientTLSDoesNotMutateDefaultTransport niks3-go-unit-tests> === RUN TestSetClientTLSErrors niks3-go-unit-tests> === PAUSE TestSetClientTLSErrors niks3-go-unit-tests> === RUN TestStaticToken niks3-go-unit-tests> === PAUSE TestStaticToken niks3-go-unit-tests> === RUN TestFileTokenReadsAndCaches niks3-go-unit-tests> === PAUSE TestFileTokenReadsAndCaches niks3-go-unit-tests> === RUN TestFileTokenMissing niks3-go-unit-tests> === PAUSE TestFileTokenMissing niks3-go-unit-tests> === RUN TestFileTokenEmpty niks3-go-unit-tests> === PAUSE TestFileTokenEmpty niks3-go-unit-tests> === RUN TestScriptTokenNoExpiryRerunsEveryCall niks3-go-unit-tests> === PAUSE TestScriptTokenNoExpiryRerunsEveryCall niks3-go-unit-tests> === RUN TestScriptTokenCachesUntilRefresh niks3-go-unit-tests> === PAUSE TestScriptTokenCachesUntilRefresh niks3-go-unit-tests> === RUN TestScriptTokenEmptyToken niks3-go-unit-tests> === PAUSE TestScriptTokenEmptyToken niks3-go-unit-tests> === RUN TestScriptTokenBadJSON niks3-go-unit-tests> === PAUSE TestScriptTokenBadJSON niks3-go-unit-tests> === RUN TestScriptTokenScriptFails niks3-go-unit-tests> === PAUSE TestScriptTokenScriptFails niks3-go-unit-tests> === RUN TestScriptTokenEmptyCommand niks3-go-unit-tests> === PAUSE TestScriptTokenEmptyCommand niks3-go-unit-tests> === CONT TestDoWithRetry_BodyReplayedViaGetBody niks3-go-unit-tests> === CONT TestDoServerRequestAttachesToken niks3-go-unit-tests> === CONT TestGetStorePathHash niks3-go-unit-tests> === CONT TestResolveStorePath niks3-go-unit-tests> === CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess niks3-go-unit-tests> === CONT TestRateLimiterFeedback niks3-go-unit-tests> === CONT TestPathInfoCACompatibility niks3-go-unit-tests> === CONT TestParsePathInfoJSONMultiplePaths niks3-go-unit-tests> === CONT TestParsePathInfoJSON niks3-go-unit-tests> === CONT TestPathInfoHashCompatibility niks3-go-unit-tests> === CONT TestDumpPathSingleFile niks3-go-unit-tests> === CONT TestConvertHashToNix32 niks3-go-unit-tests> === CONT TestEncodeNixBase32WithRealHash niks3-go-unit-tests> === CONT TestEncodeNixBase32 niks3-go-unit-tests> === CONT TestDumpPathWriterError niks3-go-unit-tests> === CONT TestFileTokenMissing niks3-go-unit-tests> === CONT TestScriptTokenEmptyCommand niks3-go-unit-tests> === CONT TestScriptTokenScriptFails niks3-go-unit-tests> === CONT TestScriptTokenBadJSON niks3-go-unit-tests> === CONT TestScriptTokenEmptyToken niks3-go-unit-tests> === CONT TestScriptTokenCachesUntilRefresh niks3-go-unit-tests> === CONT TestScriptTokenNoExpiryRerunsEveryCall niks3-go-unit-tests> === CONT TestFileTokenEmpty niks3-go-unit-tests> === CONT TestPartSizeForNAR niks3-go-unit-tests> === RUN TestGetStorePathHash/valid_store_path niks3-go-unit-tests> === PAUSE TestGetStorePathHash/valid_store_path niks3-go-unit-tests> === RUN TestGetStorePathHash/basename_without_hyphen_should_error niks3-go-unit-tests> === PAUSE TestGetStorePathHash/basename_without_hyphen_should_error niks3-go-unit-tests> === RUN TestGetStorePathHash/hash_with_invalid_characters_should_error niks3-go-unit-tests> === PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error niks3-go-unit-tests> === RUN TestGetStorePathHash/hash_with_wrong_length_should_error niks3-go-unit-tests> === PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error niks3-go-unit-tests> --- PASS: TestResolveStorePath (0.00s) niks3-go-unit-tests> === CONT TestUploadMultipart_SupersededByPeer niks3-go-unit-tests> === RUN TestUploadMultipart_SupersededByPeer/exists niks3-go-unit-tests> === PAUSE TestUploadMultipart_SupersededByPeer/exists niks3-go-unit-tests> === RUN TestUploadMultipart_SupersededByPeer/missing niks3-go-unit-tests> === PAUSE TestUploadMultipart_SupersededByPeer/missing niks3-go-unit-tests> === CONT TestDumpPathMatchesNix niks3-go-unit-tests> === CONT TestFilterOversizedClosures niks3-go-unit-tests> === RUN TestFilterOversizedClosures/no_limit_keeps_everything niks3-go-unit-tests> === PAUSE TestFilterOversizedClosures/no_limit_keeps_everything niks3-go-unit-tests> === RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped niks3-go-unit-tests> === PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped niks3-go-unit-tests> === RUN TestFilterOversizedClosures/all_closures_skipped niks3-go-unit-tests> === PAUSE TestFilterOversizedClosures/all_closures_skipped niks3-go-unit-tests> === CONT TestSetClientTLSDoesNotMutateDefaultTransport niks3-go-unit-tests> 2026/08/27 09:40:13 WARN Rate limiter enabled after throttle name=server-test rate=5 niks3-go-unit-tests> === RUN TestRateLimiterFeedback/429_enables_limiter niks3-go-unit-tests> === PAUSE TestRateLimiterFeedback/429_enables_limiter niks3-go-unit-tests> === RUN TestRateLimiterFeedback/503_enables_limiter niks3-go-unit-tests> === PAUSE TestRateLimiterFeedback/503_enables_limiter niks3-go-unit-tests> === RUN TestRateLimiterFeedback/200_does_not_enable_limiter niks3-go-unit-tests> === PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter niks3-go-unit-tests> === RUN TestRateLimiterFeedback/400_does_not_enable_limiter niks3-go-unit-tests> === PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter niks3-go-unit-tests> === RUN TestPathInfoCACompatibility/null_ca_field niks3-go-unit-tests> === PAUSE TestPathInfoCACompatibility/null_ca_field niks3-go-unit-tests> === CONT TestFileTokenReadsAndCaches niks3-go-unit-tests> === RUN TestConvertHashToNix32/SRI_format_to_Nix32 niks3-go-unit-tests> === PAUSE TestConvertHashToNix32/SRI_format_to_Nix32 niks3-go-unit-tests> === RUN TestConvertHashToNix32/already_Nix32_format niks3-go-unit-tests> === PAUSE TestConvertHashToNix32/already_Nix32_format niks3-go-unit-tests> === RUN TestConvertHashToNix32/invalid_format niks3-go-unit-tests> === PAUSE TestConvertHashToNix32/invalid_format niks3-go-unit-tests> --- PASS: TestEncodeNixBase32WithRealHash (0.00s) niks3-go-unit-tests> --- PASS: TestFileTokenReadsAndCaches (0.00s) niks3-go-unit-tests> === RUN TestPathInfoCACompatibility/old_string_format_-_text niks3-go-unit-tests> === PAUSE TestPathInfoCACompatibility/old_string_format_-_text niks3-go-unit-tests> === RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive niks3-go-unit-tests> === PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive niks3-go-unit-tests> === RUN TestPathInfoCACompatibility/new_structured_format_-_text niks3-go-unit-tests> === PAUSE TestPathInfoCACompatibility/new_structured_format_-_text niks3-go-unit-tests> === RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method niks3-go-unit-tests> === PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method niks3-go-unit-tests> === CONT TestCaseHackSuffix niks3-go-unit-tests> === CONT TestSetClientTLSErrors niks3-go-unit-tests> === RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths niks3-go-unit-tests> === PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths niks3-go-unit-tests> === RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths niks3-go-unit-tests> === PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths niks3-go-unit-tests> === CONT TestGetStorePathHash/valid_store_path niks3-go-unit-tests> === RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) niks3-go-unit-tests> === CONT TestUploadMultipart_SupersededByPeer/exists niks3-go-unit-tests> === CONT TestStaticToken niks3-go-unit-tests> === RUN TestEncodeNixBase32/test_string_hash niks3-go-unit-tests> === RUN TestParsePathInfoJSON/Nix_format niks3-go-unit-tests> === CONT TestShellSplitErrors niks3-go-unit-tests> === RUN TestSetClientTLSErrors/missing_cert_file niks3-go-unit-tests> --- PASS: TestScriptTokenEmptyCommand (0.00s) niks3-go-unit-tests> === CONT TestSetClientTLS niks3-go-unit-tests> === CONT TestShellSplit niks3-go-unit-tests> === CONT TestGetStorePathHash/basename_without_hyphen_should_error niks3-go-unit-tests> === PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) niks3-go-unit-tests> === RUN TestPathInfoHashCompatibility/old_string_format_with_colon niks3-go-unit-tests> === PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon niks3-go-unit-tests> === RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI niks3-go-unit-tests> --- PASS: TestFileTokenMissing (0.00s) niks3-go-unit-tests> 2026/08/27 09:40:13 WARN Rate limiter enabled after throttle name=server-test rate=5 niks3-go-unit-tests> 2026/08/27 09:40:13 WARN Request returned retryable status, retrying attempt=1 max_attempts=6 backoff=0s status=503 url=http://127.0.0.1:36383 niks3-go-unit-tests> === CONT TestGetStorePathHash/hash_with_wrong_length_should_error niks3-go-unit-tests> === CONT TestGetStorePathHash/hash_with_invalid_characters_should_error niks3-go-unit-tests> === PAUSE TestEncodeNixBase32/test_string_hash niks3-go-unit-tests> === RUN TestEncodeNixBase32/empty_input niks3-go-unit-tests> --- PASS: TestScriptTokenScriptFails (0.01s) niks3-go-unit-tests> --- PASS: TestStaticToken (0.00s) niks3-go-unit-tests> 2026/08/27 09:40:13 WARN Rate limiter backed off name=server-test rate=5 niks3-go-unit-tests> 2026/08/27 09:40:13 WARN Request returned retryable status, retrying attempt=2 max_attempts=6 backoff=0s status=503 url=http://127.0.0.1:36383 niks3-go-unit-tests> === CONT TestFilterOversizedClosures/no_limit_keeps_everything niks3-go-unit-tests> === CONT TestFilterOversizedClosures/all_closures_skipped niks3-go-unit-tests> === CONT TestUploadMultipart_SupersededByPeer/missing niks3-go-unit-tests> === CONT TestRateLimiterFeedback/429_enables_limiter niks3-go-unit-tests> --- PASS: TestShellSplitErrors (0.00s) niks3-go-unit-tests> --- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.04s) niks3-go-unit-tests> --- PASS: TestDoServerRequestAttachesToken (0.04s) niks3-go-unit-tests> === CONT TestRateLimiterFeedback/200_does_not_enable_limiter niks3-go-unit-tests> --- PASS: TestFileTokenEmpty (0.00s) niks3-go-unit-tests> --- PASS: TestDumpPathSingleFile (0.05s) niks3-go-unit-tests> === RUN TestPartSizeForNAR/zero_stays_at_minimum niks3-go-unit-tests> === PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI niks3-go-unit-tests> --- PASS: TestScriptTokenEmptyToken (0.04s) niks3-go-unit-tests> === CONT TestRateLimiterFeedback/503_enables_limiter niks3-go-unit-tests> === PAUSE TestParsePathInfoJSON/Nix_format niks3-go-unit-tests> === PAUSE TestEncodeNixBase32/empty_input niks3-go-unit-tests> 2026/08/27 09:40:13 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50 niks3-go-unit-tests> === CONT TestRateLimiterFeedback/400_does_not_enable_limiter niks3-go-unit-tests> === PAUSE TestPartSizeForNAR/zero_stays_at_minimum niks3-go-unit-tests> === CONT TestConvertHashToNix32/invalid_format niks3-go-unit-tests> === CONT TestPathInfoCACompatibility/null_ca_field niks3-go-unit-tests> === CONT TestConvertHashToNix32/already_Nix32_format niks3-go-unit-tests> === CONT TestPathInfoCACompatibility/old_string_format_-_text niks3-go-unit-tests> --- PASS: TestScriptTokenBadJSON (0.04s) niks3-go-unit-tests> === CONT TestPathInfoCACompatibility/new_structured_format_-_text niks3-go-unit-tests> === CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive niks3-go-unit-tests> === CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped niks3-go-unit-tests> 2026/08/27 09:40:13 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000 niks3-go-unit-tests> --- PASS: TestFilterOversizedClosures (0.00s) niks3-go-unit-tests> --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s) niks3-go-unit-tests> --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s) niks3-go-unit-tests> --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s) niks3-go-unit-tests> === CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method niks3-go-unit-tests> --- PASS: TestPathInfoCACompatibility (0.00s) niks3-go-unit-tests> --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s) niks3-go-unit-tests> --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s) niks3-go-unit-tests> --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s) niks3-go-unit-tests> --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s) niks3-go-unit-tests> --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s) niks3-go-unit-tests> === CONT TestConvertHashToNix32/SRI_format_to_Nix32 niks3-go-unit-tests> --- PASS: TestConvertHashToNix32 (0.00s) niks3-go-unit-tests> --- PASS: TestConvertHashToNix32/invalid_format (0.00s) niks3-go-unit-tests> --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s) niks3-go-unit-tests> --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s) niks3-go-unit-tests> === CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths niks3-go-unit-tests> === CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths niks3-go-unit-tests> --- PASS: TestParsePathInfoJSONMultiplePaths (0.01s) niks3-go-unit-tests> --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s) niks3-go-unit-tests> --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s) niks3-go-unit-tests> === CONT TestEncodeNixBase32/empty_input niks3-go-unit-tests> 2026/08/27 09:40:13 WARN Rate limiter enabled after throttle name=server-test rate=5 niks3-go-unit-tests> === CONT TestEncodeNixBase32/test_string_hash niks3-go-unit-tests> 2026/08/27 09:40:13 WARN Request returned retryable status, retrying attempt=1 max_attempts=2 backoff=0s status=429 url=http://127.0.0.1:45099 niks3-go-unit-tests> === RUN TestSetClientTLS/rejects_connection_without_client_cert niks3-go-unit-tests> === PAUSE TestSetClientTLS/rejects_connection_without_client_cert niks3-go-unit-tests> === RUN TestPartSizeForNAR/small_stays_at_minimum niks3-go-unit-tests> === PAUSE TestPartSizeForNAR/small_stays_at_minimum niks3-go-unit-tests> === RUN TestPartSizeForNAR/80_GiB_fits_at_minimum niks3-go-unit-tests> === PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum niks3-go-unit-tests> === RUN TestPartSizeForNAR/115_GiB_needs_larger_parts niks3-go-unit-tests> === PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts niks3-go-unit-tests> 2026/08/27 09:40:13 WARN Rate limiter backed off name=server-test rate=5 niks3-go-unit-tests> === RUN TestPartSizeForNAR/1_TiB niks3-go-unit-tests> === PAUSE TestPartSizeForNAR/1_TiB niks3-go-unit-tests> === RUN TestPartSizeForNAR/5_TiB_S3_max_object niks3-go-unit-tests> === PAUSE TestPartSizeForNAR/5_TiB_S3_max_object niks3-go-unit-tests> === RUN TestPartSizeForNAR/capped_at_5_GiB niks3-go-unit-tests> === PAUSE TestPartSizeForNAR/capped_at_5_GiB niks3-go-unit-tests> === CONT TestPartSizeForNAR/zero_stays_at_minimum niks3-go-unit-tests> === RUN TestParsePathInfoJSON/Lix_format niks3-go-unit-tests> === PAUSE TestParsePathInfoJSON/Lix_format niks3-go-unit-tests> === RUN TestParsePathInfoJSON/empty_input niks3-go-unit-tests> === PAUSE TestParsePathInfoJSON/empty_input niks3-go-unit-tests> === RUN TestParsePathInfoJSON/whitespace_only niks3-go-unit-tests> === PAUSE TestParsePathInfoJSON/whitespace_only niks3-go-unit-tests> === RUN TestParsePathInfoJSON/invalid_JSON niks3-go-unit-tests> === PAUSE TestParsePathInfoJSON/invalid_JSON niks3-go-unit-tests> === CONT TestParsePathInfoJSON/Nix_format niks3-go-unit-tests> === CONT TestPartSizeForNAR/capped_at_5_GiB niks3-go-unit-tests> === CONT TestPartSizeForNAR/5_TiB_S3_max_object niks3-go-unit-tests> === CONT TestPartSizeForNAR/1_TiB niks3-go-unit-tests> === CONT TestPartSizeForNAR/115_GiB_needs_larger_parts niks3-go-unit-tests> === CONT TestPartSizeForNAR/80_GiB_fits_at_minimum niks3-go-unit-tests> === CONT TestPartSizeForNAR/small_stays_at_minimum niks3-go-unit-tests> === PAUSE TestSetClientTLSErrors/missing_cert_file niks3-go-unit-tests> === RUN TestSetClientTLSErrors/missing_key_file niks3-go-unit-tests> === PAUSE TestSetClientTLSErrors/missing_key_file niks3-go-unit-tests> === RUN TestSetClientTLSErrors/missing_ca_file niks3-go-unit-tests> === PAUSE TestSetClientTLSErrors/missing_ca_file niks3-go-unit-tests> === RUN TestSetClientTLSErrors/invalid_ca_file niks3-go-unit-tests> === PAUSE TestSetClientTLSErrors/invalid_ca_file niks3-go-unit-tests> === CONT TestSetClientTLSErrors/missing_cert_file niks3-go-unit-tests> === CONT TestParsePathInfoJSON/invalid_JSON niks3-go-unit-tests> === CONT TestParsePathInfoJSON/whitespace_only niks3-go-unit-tests> === CONT TestParsePathInfoJSON/empty_input niks3-go-unit-tests> === CONT TestParsePathInfoJSON/Lix_format niks3-go-unit-tests> === RUN TestSetClientTLS/succeeds_with_client_cert_and_CA niks3-go-unit-tests> === PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA niks3-go-unit-tests> 2026/08/27 09:40:13 WARN Rate limiter enabled after throttle name=server-test rate=5 niks3-go-unit-tests> === RUN TestSetClientTLS/preserves_debug_logging_transport niks3-go-unit-tests> 2026/08/27 09:40:13 WARN Request returned retryable status, retrying attempt=1 max_attempts=2 backoff=0s status=503 url=http://127.0.0.1:36563 niks3-go-unit-tests> === PAUSE TestSetClientTLS/preserves_debug_logging_transport niks3-go-unit-tests> === CONT TestSetClientTLS/rejects_connection_without_client_cert niks3-go-unit-tests> === CONT TestSetClientTLS/preserves_debug_logging_transport niks3-go-unit-tests> === CONT TestSetClientTLSErrors/missing_ca_file niks3-go-unit-tests> 2026/08/27 09:40:13 WARN Rate limiter backed off name=server-test rate=5 niks3-go-unit-tests> === CONT TestSetClientTLS/succeeds_with_client_cert_and_CA niks3-go-unit-tests> --- PASS: TestGetStorePathHash (0.00s) niks3-go-unit-tests> --- PASS: TestGetStorePathHash/valid_store_path (0.00s) niks3-go-unit-tests> --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.02s) niks3-go-unit-tests> --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s) niks3-go-unit-tests> --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.01s) niks3-go-unit-tests> --- PASS: TestEncodeNixBase32 (0.06s) niks3-go-unit-tests> --- PASS: TestEncodeNixBase32/empty_input (0.00s) niks3-go-unit-tests> --- PASS: TestEncodeNixBase32/test_string_hash (0.00s) niks3-go-unit-tests> === CONT TestSetClientTLSErrors/invalid_ca_file niks3-go-unit-tests> === CONT TestSetClientTLSErrors/missing_key_file niks3-go-unit-tests> === RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512 niks3-go-unit-tests> === PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512 niks3-go-unit-tests> === CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) niks3-go-unit-tests> --- PASS: TestShellSplit (0.01s) niks3-go-unit-tests> === CONT TestPathInfoHashCompatibility/old_string_format_with_colon niks3-go-unit-tests> === CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI niks3-go-unit-tests> === CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512 niks3-go-unit-tests> --- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.07s) niks3-go-unit-tests> --- PASS: TestParsePathInfoJSON (0.06s) niks3-go-unit-tests> --- PASS: TestParsePathInfoJSON/Nix_format (0.00s) niks3-go-unit-tests> --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s) niks3-go-unit-tests> --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s) niks3-go-unit-tests> --- PASS: TestParsePathInfoJSON/empty_input (0.00s) niks3-go-unit-tests> --- PASS: TestParsePathInfoJSON/Lix_format (0.00s) niks3-go-unit-tests> --- PASS: TestUploadMultipart_SupersededByPeer (0.00s) niks3-go-unit-tests> --- PASS: TestUploadMultipart_SupersededByPeer/missing (0.01s) niks3-go-unit-tests> --- PASS: TestUploadMultipart_SupersededByPeer/exists (0.03s) niks3-go-unit-tests> --- PASS: TestRateLimiterFeedback (0.00s) niks3-go-unit-tests> --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s) niks3-go-unit-tests> --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.00s) niks3-go-unit-tests> --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.01s) niks3-go-unit-tests> --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s) niks3-go-unit-tests> --- PASS: TestSetClientTLSErrors (0.06s) niks3-go-unit-tests> --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s) niks3-go-unit-tests> --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s) niks3-go-unit-tests> --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s) niks3-go-unit-tests> --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s) niks3-go-unit-tests> --- PASS: TestPathInfoHashCompatibility (0.06s) niks3-go-unit-tests> --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s) niks3-go-unit-tests> --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s) niks3-go-unit-tests> --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s) niks3-go-unit-tests> --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s) niks3-go-unit-tests> --- PASS: TestScriptTokenCachesUntilRefresh (0.03s) niks3-go-unit-tests> --- PASS: TestPartSizeForNAR (0.01s) niks3-go-unit-tests> --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s) niks3-go-unit-tests> --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s) niks3-go-unit-tests> --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s) niks3-go-unit-tests> --- PASS: TestPartSizeForNAR/1_TiB (0.00s) niks3-go-unit-tests> --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s) niks3-go-unit-tests> --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s) niks3-go-unit-tests> --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s) niks3-go-unit-tests> --- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.04s) niks3-go-unit-tests> 2026/08/27 09:40:13 http: TLS handshake error from 127.0.0.1:41414: remote error: tls: bad certificate niks3-go-unit-tests> --- PASS: TestSetClientTLS (0.04s) niks3-go-unit-tests> --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.01s) niks3-go-unit-tests> --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.01s) niks3-go-unit-tests> --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.02s) niks3-go-unit-tests> --- PASS: TestDumpPathWriterError (0.08s) niks3-go-unit-tests> --- PASS: TestCaseHackSuffix (0.08s) niks3-go-unit-tests> --- PASS: TestDumpPathMatchesNix (0.11s) niks3-go-unit-tests> --- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s) niks3-go-unit-tests> PASS niks3-go-unit-tests> Running server tests... niks3-go-unit-tests> The files belonging to this database system will be owned by user "nixbld". niks3-go-unit-tests> This user must also own the server process. niks3-go-unit-tests> niks3-go-unit-tests> The database cluster will be initialized with locale "C". niks3-go-unit-tests> The default database encoding has accordingly been set to "SQL_ASCII". niks3-go-unit-tests> The default text search configuration will be set to "english". niks3-go-unit-tests> niks3-go-unit-tests> Data page checksums are enabled. niks3-go-unit-tests> niks3-go-unit-tests> creating directory /build/postgres1380383068/data ... ok niks3-go-unit-tests> creating subdirectories ... ok niks3-go-unit-tests> selecting dynamic shared memory implementation ... posix niks3-go-unit-tests> selecting default "max_connections" ... 100 niks3-go-unit-tests> selecting default "shared_buffers" ... 128MB niks3-go-unit-tests> selecting default time zone ... UTC niks3-go-unit-tests> creating configuration files ... ok niks3-go-unit-tests> running bootstrap script ... ok niks3-go-unit-tests> performing post-bootstrap initialization ... ok niks3-go-unit-tests> syncing data to disk ... ok niks3-go-unit-tests> niks3-go-unit-tests> initdb: warning: enabling "trust" authentication for local connections niks3-go-unit-tests> initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. niks3-go-unit-tests> niks3-go-unit-tests> Success. You can now start the database server using: niks3-go-unit-tests> niks3-go-unit-tests> pg_ctl -D /build/postgres1380383068/data -l logfile start niks3-go-unit-tests> niks3-go-unit-tests> /build/postgres1380383068:5432 - no response niks3-go-unit-tests> 2026-08-27 09:40:15.674 UTC [108] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit niks3-go-unit-tests> 2026-08-27 09:40:15.678 UTC [108] LOG: listening on Unix socket "/build/postgres1380383068/.s.PGSQL.5432" niks3-go-unit-tests> 2026-08-27 09:40:15.684 UTC [115] LOG: database system was shut down at 2026-08-27 09:40:15 UTC niks3-go-unit-tests> 2026-08-27 09:40:15.687 UTC [108] LOG: database system is ready to accept connections niks3-go-unit-tests> /build/postgres1380383068:5432 - accepting connections niks3-go-unit-tests> === RUN TestService_AuthMiddleware niks3-go-unit-tests> === PAUSE TestService_AuthMiddleware niks3-go-unit-tests> === RUN TestService_AuthMiddleware_MTLSProxyHeader niks3-go-unit-tests> === PAUSE TestService_AuthMiddleware_MTLSProxyHeader niks3-go-unit-tests> === RUN TestService_AuthMiddleware_MTLSBoundSubjects niks3-go-unit-tests> === PAUSE TestService_AuthMiddleware_MTLSBoundSubjects niks3-go-unit-tests> === RUN TestService_ReadAuthMiddleware niks3-go-unit-tests> === PAUSE TestService_ReadAuthMiddleware niks3-go-unit-tests> === RUN TestService_AuthMiddleware_OIDC niks3-go-unit-tests> === PAUSE TestService_AuthMiddleware_OIDC niks3-go-unit-tests> === RUN TestCacheConfigHandler niks3-go-unit-tests> === PAUSE TestCacheConfigHandler niks3-go-unit-tests> === RUN TestCacheStatsHandler niks3-go-unit-tests> === PAUSE TestCacheStatsHandler niks3-go-unit-tests> === RUN TestClientCADerivations niks3-go-unit-tests> === PAUSE TestClientCADerivations niks3-go-unit-tests> === RUN TestClientErrorHandling niks3-go-unit-tests> === PAUSE TestClientErrorHandling niks3-go-unit-tests> === RUN TestClientIntegration niks3-go-unit-tests> === PAUSE TestClientIntegration niks3-go-unit-tests> === RUN TestClientMultipleUploads niks3-go-unit-tests> === PAUSE TestClientMultipleUploads niks3-go-unit-tests> === RUN TestClientWithDependencies niks3-go-unit-tests> === PAUSE TestClientWithDependencies niks3-go-unit-tests> === RUN TestPinProtectsFromGC niks3-go-unit-tests> === PAUSE TestPinProtectsFromGC niks3-go-unit-tests> === RUN TestGCAdvisoryLockBlocksConcurrentRun niks3-go-unit-tests> 2026-08-27 09:40:28.973 UTC [517] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:28.973 UTC [517] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:28 OK 20241026095416_initial_model.sql (9.18ms) niks3-go-unit-tests> 2026/08/27 09:40:28 OK 20251210153512_drop_unused_gin_index.sql (2.16ms) niks3-go-unit-tests> 2026/08/27 09:40:28 OK 20251218171726_add_pins.sql (2.98ms) niks3-go-unit-tests> 2026/08/27 09:40:29 OK 20260628120000_add_object_size_and_stats.sql (3.16ms) niks3-go-unit-tests> 2026/08/27 09:40:29 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:29 OK 1_commit_pending_closure.sql (1.9ms) niks3-go-unit-tests> 2026/08/27 09:40:29 OK 2_object_stats_trigger.sql (997.71µs) niks3-go-unit-tests> 2026/08/27 09:40:29 goose: up to current file version: 2 niks3-go-unit-tests> --- PASS: TestGCAdvisoryLockBlocksConcurrentRun (0.69s) niks3-go-unit-tests> === RUN TestGCBugBareHashReferences niks3-go-unit-tests> === PAUSE TestGCBugBareHashReferences niks3-go-unit-tests> === RUN TestGCMetrics niks3-go-unit-tests> === PAUSE TestGCMetrics niks3-go-unit-tests> === RUN TestGCTaskStore_StartNew niks3-go-unit-tests> === PAUSE TestGCTaskStore_StartNew niks3-go-unit-tests> === RUN TestGCTaskStore_DeduplicateSameParams niks3-go-unit-tests> === PAUSE TestGCTaskStore_DeduplicateSameParams niks3-go-unit-tests> === RUN TestGCTaskStore_ConflictDifferentParams niks3-go-unit-tests> === PAUSE TestGCTaskStore_ConflictDifferentParams niks3-go-unit-tests> === RUN TestGCTaskStore_GetEmpty niks3-go-unit-tests> === PAUSE TestGCTaskStore_GetEmpty niks3-go-unit-tests> === RUN TestGCTaskStore_GetReturnsLatest niks3-go-unit-tests> === PAUSE TestGCTaskStore_GetReturnsLatest niks3-go-unit-tests> === RUN TestGCTaskStore_CompletedAllowsNewTask niks3-go-unit-tests> === PAUSE TestGCTaskStore_CompletedAllowsNewTask niks3-go-unit-tests> === RUN TestGCTaskStore_PhaseUpdates niks3-go-unit-tests> === PAUSE TestGCTaskStore_PhaseUpdates niks3-go-unit-tests> === RUN TestGCTaskStore_Fail niks3-go-unit-tests> === PAUSE TestGCTaskStore_Fail niks3-go-unit-tests> === RUN TestGracefulShutdownDrainsInflight niks3-go-unit-tests> === PAUSE TestGracefulShutdownDrainsInflight niks3-go-unit-tests> === RUN TestService_healthCheckHandler niks3-go-unit-tests> === PAUSE TestService_healthCheckHandler niks3-go-unit-tests> === RUN TestGenerateLandingPage niks3-go-unit-tests> === PAUSE TestGenerateLandingPage niks3-go-unit-tests> === RUN TestCacheConfigHandlerMaxNarSize niks3-go-unit-tests> === PAUSE TestCacheConfigHandlerMaxNarSize niks3-go-unit-tests> === RUN TestCreatePendingClosureRejectsOversizedNAR niks3-go-unit-tests> === PAUSE TestCreatePendingClosureRejectsOversizedNAR niks3-go-unit-tests> === RUN TestNARDeduplicationMetadataUploadBug niks3-go-unit-tests> === PAUSE TestNARDeduplicationMetadataUploadBug niks3-go-unit-tests> === RUN TestMetricsInventory niks3-go-unit-tests> === PAUSE TestMetricsInventory niks3-go-unit-tests> === RUN TestService_NativeMTLS niks3-go-unit-tests> === PAUSE TestService_NativeMTLS niks3-go-unit-tests> === RUN TestServerTLSConfig niks3-go-unit-tests> === PAUSE TestServerTLSConfig niks3-go-unit-tests> === RUN TestMultipartCleanup niks3-go-unit-tests> === PAUSE TestMultipartCleanup niks3-go-unit-tests> === RUN TestObjectStatsTrigger niks3-go-unit-tests> === PAUSE TestObjectStatsTrigger niks3-go-unit-tests> === RUN TestOrphanedObjectsGC niks3-go-unit-tests> === PAUSE TestOrphanedObjectsGC niks3-go-unit-tests> === RUN TestOrphanedObjectsGCStressTest niks3-go-unit-tests> === PAUSE TestOrphanedObjectsGCStressTest niks3-go-unit-tests> === RUN TestResurrectedObjectNotDeleted niks3-go-unit-tests> === PAUSE TestResurrectedObjectNotDeleted niks3-go-unit-tests> === RUN TestParseSingleRange niks3-go-unit-tests> === PAUSE TestParseSingleRange niks3-go-unit-tests> === RUN TestIsValidCachePath niks3-go-unit-tests> === PAUSE TestIsValidCachePath niks3-go-unit-tests> === RUN TestReadProxyNarinfo niks3-go-unit-tests> === PAUSE TestReadProxyNarinfo niks3-go-unit-tests> === RUN TestReadProxyNarinfoAlreadyDecompressed niks3-go-unit-tests> === PAUSE TestReadProxyNarinfoAlreadyDecompressed niks3-go-unit-tests> === RUN TestReadProxyNarStreaming niks3-go-unit-tests> === PAUSE TestReadProxyNarStreaming niks3-go-unit-tests> === RUN TestReadProxy404 niks3-go-unit-tests> === PAUSE TestReadProxy404 niks3-go-unit-tests> === RUN TestReadProxyInvalidPath niks3-go-unit-tests> === PAUSE TestReadProxyInvalidPath niks3-go-unit-tests> === RUN TestReadProxyHead niks3-go-unit-tests> === PAUSE TestReadProxyHead niks3-go-unit-tests> === RUN TestReadProxyConditionalGet niks3-go-unit-tests> === PAUSE TestReadProxyConditionalGet niks3-go-unit-tests> === RUN TestReadProxyRootRedirectsToIndexHTML niks3-go-unit-tests> === PAUSE TestReadProxyRootRedirectsToIndexHTML niks3-go-unit-tests> === RUN TestReadProxyDisabled niks3-go-unit-tests> === PAUSE TestReadProxyDisabled niks3-go-unit-tests> === RUN TestReadProxyRangeRequest niks3-go-unit-tests> === PAUSE TestReadProxyRangeRequest niks3-go-unit-tests> === RUN TestRedundantMultipartUpload niks3-go-unit-tests> === PAUSE TestRedundantMultipartUpload niks3-go-unit-tests> === RUN TestCompleteMultipartUpload_ErrorButObjectExists niks3-go-unit-tests> === PAUSE TestCompleteMultipartUpload_ErrorButObjectExists niks3-go-unit-tests> === RUN TestCompletedNarNotReofferedAcrossClosures niks3-go-unit-tests> === PAUSE TestCompletedNarNotReofferedAcrossClosures niks3-go-unit-tests> === RUN TestPresignedUploadRegisteredBeforeCommit niks3-go-unit-tests> === PAUSE TestPresignedUploadRegisteredBeforeCommit niks3-go-unit-tests> === RUN TestService_Rustfstest niks3-go-unit-tests> === PAUSE TestService_Rustfstest niks3-go-unit-tests> === RUN TestParseSize niks3-go-unit-tests> === PAUSE TestParseSize niks3-go-unit-tests> === RUN TestSkippedUploadsHandler niks3-go-unit-tests> === PAUSE TestSkippedUploadsHandler niks3-go-unit-tests> === RUN TestSystemdListenerNotActivated niks3-go-unit-tests> --- PASS: TestSystemdListenerNotActivated (0.00s) niks3-go-unit-tests> === RUN TestWatchdogBeatsWhenHealthy niks3-go-unit-tests> --- PASS: TestWatchdogBeatsWhenHealthy (0.02s) niks3-go-unit-tests> === RUN TestWatchdogSkipsWhenUnhealthy niks3-go-unit-tests> 2026/08/27 09:40:29 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded" niks3-go-unit-tests> 2026/08/27 09:40:29 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded" niks3-go-unit-tests> 2026/08/27 09:40:29 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded" niks3-go-unit-tests> 2026/08/27 09:40:29 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded" niks3-go-unit-tests> 2026/08/27 09:40:29 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded" niks3-go-unit-tests> 2026/08/27 09:40:29 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded" niks3-go-unit-tests> 2026/08/27 09:40:29 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded" niks3-go-unit-tests> 2026/08/27 09:40:29 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded" niks3-go-unit-tests> 2026/08/27 09:40:29 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded" niks3-go-unit-tests> 2026/08/27 09:40:29 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded" niks3-go-unit-tests> --- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s) niks3-go-unit-tests> === RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle niks3-go-unit-tests> === PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle niks3-go-unit-tests> === RUN TestProxyWriteTimeout niks3-go-unit-tests> === PAUSE TestProxyWriteTimeout niks3-go-unit-tests> === RUN TestIsValidUploadKey niks3-go-unit-tests> === PAUSE TestIsValidUploadKey niks3-go-unit-tests> === RUN TestUploadHandlersRejectInvalidKeys niks3-go-unit-tests> === PAUSE TestUploadHandlersRejectInvalidKeys niks3-go-unit-tests> === RUN TestUploadHandlersRejectOversizedBody niks3-go-unit-tests> === PAUSE TestUploadHandlersRejectOversizedBody niks3-go-unit-tests> === RUN TestService_cleanupPendingClosuresHandler niks3-go-unit-tests> === PAUSE TestService_cleanupPendingClosuresHandler niks3-go-unit-tests> === RUN TestService_createPendingClosureHandler niks3-go-unit-tests> === PAUSE TestService_createPendingClosureHandler niks3-go-unit-tests> === RUN TestService_verifyS3Integrity niks3-go-unit-tests> === PAUSE TestService_verifyS3Integrity niks3-go-unit-tests> === RUN TestCompleteMultipartUnregistered niks3-go-unit-tests> === PAUSE TestCompleteMultipartUnregistered niks3-go-unit-tests> === RUN TestCreatePendingClosure_SmallNARUsesSimplePUT niks3-go-unit-tests> === PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT niks3-go-unit-tests> === CONT TestService_AuthMiddleware niks3-go-unit-tests> === CONT TestObjectStatsTrigger niks3-go-unit-tests> === CONT TestCompleteMultipartUpload_ErrorButObjectExists niks3-go-unit-tests> === CONT TestReadProxyRootRedirectsToIndexHTML niks3-go-unit-tests> === CONT TestParseSize niks3-go-unit-tests> --- PASS: TestParseSize (0.00s) niks3-go-unit-tests> === CONT TestProxyWriteTimeout niks3-go-unit-tests> === RUN TestProxyWriteTimeout/narinfo niks3-go-unit-tests> === PAUSE TestProxyWriteTimeout/narinfo niks3-go-unit-tests> === RUN TestProxyWriteTimeout/1_GiB_nar niks3-go-unit-tests> === PAUSE TestProxyWriteTimeout/1_GiB_nar niks3-go-unit-tests> === RUN TestProxyWriteTimeout/10_GiB_nar niks3-go-unit-tests> === PAUSE TestProxyWriteTimeout/10_GiB_nar niks3-go-unit-tests> === RUN TestProxyWriteTimeout/unknown_size niks3-go-unit-tests> === PAUSE TestProxyWriteTimeout/unknown_size niks3-go-unit-tests> === CONT TestProxyWriteTimeout/narinfo niks3-go-unit-tests> === CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle niks3-go-unit-tests> === CONT TestGCTaskStore_ConflictDifferentParams niks3-go-unit-tests> --- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s) niks3-go-unit-tests> === CONT TestMultipartCleanup niks3-go-unit-tests> === CONT TestGenerateLandingPage niks3-go-unit-tests> === CONT TestIsValidUploadKey niks3-go-unit-tests> === RUN TestIsValidUploadKey/narinfo niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/narinfo niks3-go-unit-tests> === RUN TestIsValidUploadKey/nar_zst niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/nar_zst niks3-go-unit-tests> === RUN TestIsValidUploadKey/nar_xz niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/nar_xz niks3-go-unit-tests> === RUN TestIsValidUploadKey/nar_plain niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/nar_plain niks3-go-unit-tests> === RUN TestIsValidUploadKey/listing niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/listing niks3-go-unit-tests> === RUN TestIsValidUploadKey/build_log niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/build_log niks3-go-unit-tests> === RUN TestIsValidUploadKey/build_log_home-manager_file niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/build_log_home-manager_file niks3-go-unit-tests> === CONT TestProxyWriteTimeout/10_GiB_nar niks3-go-unit-tests> === RUN TestIsValidUploadKey/build_log_plus_in_name niks3-go-unit-tests> === CONT TestReadProxyNarinfo niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/build_log_plus_in_name niks3-go-unit-tests> === CONT TestCompleteMultipartUnregistered niks3-go-unit-tests> === RUN TestIsValidUploadKey/build_log_question_mark niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/build_log_question_mark niks3-go-unit-tests> === RUN TestIsValidUploadKey/build_log_equals niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/build_log_equals niks3-go-unit-tests> === RUN TestIsValidUploadKey/realisation niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/realisation niks3-go-unit-tests> === RUN TestIsValidUploadKey/realisation_plus_in_output niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/realisation_plus_in_output niks3-go-unit-tests> === RUN TestIsValidUploadKey/nix-cache-info niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/nix-cache-info niks3-go-unit-tests> === RUN TestIsValidUploadKey/index.html niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/index.html niks3-go-unit-tests> === CONT TestService_verifyS3Integrity niks3-go-unit-tests> === CONT TestService_createPendingClosureHandler niks3-go-unit-tests> === CONT TestService_cleanupPendingClosuresHandler niks3-go-unit-tests> === CONT TestUploadHandlersRejectOversizedBody niks3-go-unit-tests> === CONT TestUploadHandlersRejectInvalidKeys niks3-go-unit-tests> === CONT TestMetricsInventory niks3-go-unit-tests> === CONT TestServerTLSConfig niks3-go-unit-tests> === RUN TestServerTLSConfig/no_client_CA niks3-go-unit-tests> === PAUSE TestServerTLSConfig/no_client_CA niks3-go-unit-tests> === RUN TestServerTLSConfig/missing_CA_file niks3-go-unit-tests> === PAUSE TestServerTLSConfig/missing_CA_file niks3-go-unit-tests> === RUN TestServerTLSConfig/not_a_PEM_file niks3-go-unit-tests> === PAUSE TestServerTLSConfig/not_a_PEM_file niks3-go-unit-tests> === CONT TestService_NativeMTLS niks3-go-unit-tests> === CONT TestReadProxyHead niks3-go-unit-tests> === CONT TestReadProxyConditionalGet niks3-go-unit-tests> --- PASS: TestGenerateLandingPage (0.01s) niks3-go-unit-tests> === CONT TestParseSingleRange niks3-go-unit-tests> === RUN TestParseSingleRange/none niks3-go-unit-tests> === CONT TestIsValidCachePath niks3-go-unit-tests> === RUN TestIsValidCachePath/narinfo niks3-go-unit-tests> === PAUSE TestParseSingleRange/none niks3-go-unit-tests> === PAUSE TestIsValidCachePath/narinfo niks3-go-unit-tests> === RUN TestIsValidCachePath/narinfo_all_nix_base32_chars niks3-go-unit-tests> === CONT TestReadProxyNarinfoAlreadyDecompressed niks3-go-unit-tests> === CONT TestCreatePendingClosure_SmallNARUsesSimplePUT niks3-go-unit-tests> === RUN TestIsValidUploadKey/narinfo_key,_nar_type niks3-go-unit-tests> === CONT TestResurrectedObjectNotDeleted niks3-go-unit-tests> === CONT TestReadProxyNarStreaming niks3-go-unit-tests> === RUN TestParseSingleRange/unknown_unit niks3-go-unit-tests> === RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info niks3-go-unit-tests> === PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info niks3-go-unit-tests> === RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal niks3-go-unit-tests> === PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal niks3-go-unit-tests> === RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key niks3-go-unit-tests> === PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key niks3-go-unit-tests> === RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key niks3-go-unit-tests> === PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key niks3-go-unit-tests> === CONT TestPresignedUploadRegisteredBeforeCommit niks3-go-unit-tests> === PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars niks3-go-unit-tests> === RUN TestIsValidCachePath/nar_zst niks3-go-unit-tests> === PAUSE TestIsValidCachePath/nar_zst niks3-go-unit-tests> === RUN TestIsValidCachePath/nar_xz niks3-go-unit-tests> === PAUSE TestIsValidCachePath/nar_xz niks3-go-unit-tests> === RUN TestIsValidCachePath/nar_bz2 niks3-go-unit-tests> === PAUSE TestIsValidCachePath/nar_bz2 niks3-go-unit-tests> === RUN TestIsValidCachePath/nar_uncompressed niks3-go-unit-tests> === PAUSE TestIsValidCachePath/nar_uncompressed niks3-go-unit-tests> === RUN TestIsValidCachePath/ls niks3-go-unit-tests> === PAUSE TestIsValidCachePath/ls niks3-go-unit-tests> === RUN TestIsValidCachePath/log niks3-go-unit-tests> === PAUSE TestIsValidCachePath/log niks3-go-unit-tests> === RUN TestIsValidCachePath/realisation niks3-go-unit-tests> === PAUSE TestIsValidCachePath/realisation niks3-go-unit-tests> === RUN TestIsValidCachePath/nix-cache-info niks3-go-unit-tests> === PAUSE TestIsValidCachePath/nix-cache-info niks3-go-unit-tests> === RUN TestIsValidCachePath/index.html niks3-go-unit-tests> === PAUSE TestIsValidCachePath/index.html niks3-go-unit-tests> === RUN TestIsValidCachePath/traversal_parent niks3-go-unit-tests> === PAUSE TestIsValidCachePath/traversal_parent niks3-go-unit-tests> === RUN TestIsValidCachePath/traversal_in_middle niks3-go-unit-tests> === PAUSE TestIsValidCachePath/traversal_in_middle niks3-go-unit-tests> === RUN TestIsValidCachePath/invalid_char_e niks3-go-unit-tests> === PAUSE TestIsValidCachePath/invalid_char_e niks3-go-unit-tests> === RUN TestIsValidCachePath/invalid_char_u niks3-go-unit-tests> === PAUSE TestIsValidCachePath/invalid_char_u niks3-go-unit-tests> === RUN TestIsValidCachePath/random_path niks3-go-unit-tests> === PAUSE TestIsValidCachePath/random_path niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/narinfo_key,_nar_type niks3-go-unit-tests> === RUN TestIsValidUploadKey/nar_key,_narinfo_type niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/nar_key,_narinfo_type niks3-go-unit-tests> === RUN TestIsValidUploadKey/listing_key,_narinfo_type niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/listing_key,_narinfo_type niks3-go-unit-tests> === RUN TestIsValidUploadKey/traversal niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/traversal niks3-go-unit-tests> === RUN TestIsValidUploadKey/traversal_nar niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/traversal_nar niks3-go-unit-tests> === RUN TestIsValidUploadKey/absolute niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/absolute niks3-go-unit-tests> === RUN TestIsValidUploadKey/empty_key niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/empty_key niks3-go-unit-tests> === RUN TestIsValidUploadKey/unknown_type niks3-go-unit-tests> === PAUSE TestIsValidUploadKey/unknown_type niks3-go-unit-tests> === PAUSE TestParseSingleRange/unknown_unit niks3-go-unit-tests> === RUN TestParseSingleRange/multi-range_ignored niks3-go-unit-tests> === PAUSE TestParseSingleRange/multi-range_ignored niks3-go-unit-tests> === RUN TestParseSingleRange/malformed_no_dash niks3-go-unit-tests> === PAUSE TestParseSingleRange/malformed_no_dash niks3-go-unit-tests> === RUN TestParseSingleRange/malformed_both_empty niks3-go-unit-tests> === PAUSE TestParseSingleRange/malformed_both_empty niks3-go-unit-tests> === RUN TestParseSingleRange/malformed_end_before_start niks3-go-unit-tests> === PAUSE TestParseSingleRange/malformed_end_before_start niks3-go-unit-tests> === RUN TestParseSingleRange/closed niks3-go-unit-tests> === PAUSE TestParseSingleRange/closed niks3-go-unit-tests> === RUN TestParseSingleRange/open-ended niks3-go-unit-tests> === PAUSE TestParseSingleRange/open-ended niks3-go-unit-tests> === RUN TestParseSingleRange/end_clamped_to_size niks3-go-unit-tests> === PAUSE TestParseSingleRange/end_clamped_to_size niks3-go-unit-tests> === RUN TestParseSingleRange/suffix niks3-go-unit-tests> === PAUSE TestParseSingleRange/suffix niks3-go-unit-tests> === RUN TestParseSingleRange/suffix_exceeds_size niks3-go-unit-tests> === PAUSE TestParseSingleRange/suffix_exceeds_size niks3-go-unit-tests> === RUN TestParseSingleRange/single_byte niks3-go-unit-tests> === PAUSE TestParseSingleRange/single_byte niks3-go-unit-tests> === RUN TestParseSingleRange/start_past_EOF niks3-go-unit-tests> === PAUSE TestParseSingleRange/start_past_EOF niks3-go-unit-tests> === RUN TestParseSingleRange/start_far_past_EOF niks3-go-unit-tests> === PAUSE TestParseSingleRange/start_far_past_EOF niks3-go-unit-tests> === CONT TestCreatePendingClosureRejectsOversizedNAR niks3-go-unit-tests> === RUN TestIsValidCachePath/empty niks3-go-unit-tests> === PAUSE TestIsValidCachePath/empty niks3-go-unit-tests> === RUN TestIsValidCachePath/leading_slash niks3-go-unit-tests> === PAUSE TestIsValidCachePath/leading_slash niks3-go-unit-tests> === RUN TestIsValidCachePath/wrong_extension niks3-go-unit-tests> === PAUSE TestIsValidCachePath/wrong_extension niks3-go-unit-tests> === RUN TestIsValidCachePath/short_hash niks3-go-unit-tests> === PAUSE TestIsValidCachePath/short_hash niks3-go-unit-tests> === CONT TestService_Rustfstest niks3-go-unit-tests> 2026/08/27 09:40:29 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> --- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.00s) niks3-go-unit-tests> === CONT TestCompletedNarNotReofferedAcrossClosures niks3-go-unit-tests> === CONT TestNARDeduplicationMetadataUploadBug niks3-go-unit-tests> 2026-08-27 09:40:30.116 UTC [593] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.116 UTC [593] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026-08-27 09:40:30.116 UTC [596] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.116 UTC [596] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (76.88ms) niks3-go-unit-tests> === RUN TestUploadHandlersRejectOversizedBody/create_pending_closure niks3-go-unit-tests> === PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure niks3-go-unit-tests> === RUN TestUploadHandlersRejectOversizedBody/complete_multipart niks3-go-unit-tests> === PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart niks3-go-unit-tests> 2026-08-27 09:40:30.330 UTC [597] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.330 UTC [597] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026-08-27 09:40:30.330 UTC [595] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.330 UTC [595] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> === RUN TestUploadHandlersRejectOversizedBody/request_more_parts niks3-go-unit-tests> === PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts niks3-go-unit-tests> === CONT TestSkippedUploadsHandler niks3-go-unit-tests> 2026/08/27 09:40:30 INFO Client skipped oversized paths paths=3 nar_bytes=5000000000 niks3-go-unit-tests> --- PASS: TestSkippedUploadsHandler (0.01s) niks3-go-unit-tests> === CONT TestGCTaskStore_PhaseUpdates niks3-go-unit-tests> --- PASS: TestGCTaskStore_PhaseUpdates (0.00s) niks3-go-unit-tests> === CONT TestService_healthCheckHandler niks3-go-unit-tests> 2026-08-27 09:40:30.355 UTC [594] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.355 UTC [594] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (108.18ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (8.03ms) niks3-go-unit-tests> 2026-08-27 09:40:30.365 UTC [598] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.365 UTC [598] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026-08-27 09:40:30.367 UTC [604] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.367 UTC [604] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026-08-27 09:40:30.367 UTC [592] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.367 UTC [592] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026-08-27 09:40:30.368 UTC [601] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.368 UTC [601] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026-08-27 09:40:30.372 UTC [600] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.372 UTC [600] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026-08-27 09:40:30.372 UTC [599] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.372 UTC [599] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026-08-27 09:40:30.373 UTC [591] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.373 UTC [591] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (28.47ms) niks3-go-unit-tests> 2026-08-27 09:40:30.391 UTC [605] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.391 UTC [605] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (25.83ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (60.22ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (30.96ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (98.38ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (17.75ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (32.92ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (51.31ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (122.92ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (139.58ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (35ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (19.63ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (265.01ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (70.61ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (278.17ms) niks3-go-unit-tests> 2026-08-27 09:40:30.717 UTC [606] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.717 UTC [606] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026-08-27 09:40:30.725 UTC [607] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.725 UTC [607] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (244.76ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (341.88ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (231.1ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (298.04ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (340.14ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (211.41ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (64.71ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (250.87ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (73.12ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (14.76ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (14.72ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (224.79ms) niks3-go-unit-tests> 2026-08-27 09:40:30.778 UTC [609] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.778 UTC [609] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026-08-27 09:40:30.779 UTC [608] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.779 UTC [608] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (192.2ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (17.32ms) niks3-go-unit-tests> 2026-08-27 09:40:30.780 UTC [610] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.780 UTC [610] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026-08-27 09:40:30.793 UTC [611] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.793 UTC [611] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (35.56ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (35.66ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (44.21ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (29.26ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (31.64ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (45.34ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (45.74ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (52.56ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (20.73ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (41.98ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (21.12ms) niks3-go-unit-tests> 2026-08-27 09:40:30.829 UTC [612] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.829 UTC [612] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (56.03ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (76.02ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (26.26ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (36.33ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (27.29ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (29.23ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (71.86ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (41.44ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (15.44ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (325.3ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (47.89ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (30.66ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (46.22ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (49.34ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (29.08ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (28.92ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (38.77ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (52.29ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (12.73ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (24.58ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (53.14ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (94.81ms) niks3-go-unit-tests> 2026-08-27 09:40:30.882 UTC [613] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.882 UTC [613] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (9.66ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (9.99ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (10.42ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (21.09ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (22.33ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (41.96ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (24.67ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026-08-27 09:40:30.893 UTC [614] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.893 UTC [614] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (20.14ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (11.51ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (29.12ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (21.29ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (10.51ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (24.22ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (13.57ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (24.31ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (65.18ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (8.05ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (28.58ms) niks3-go-unit-tests> 2026-08-27 09:40:30.904 UTC [615] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.904 UTC [615] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026-08-27 09:40:30.909 UTC [616] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:30.909 UTC [616] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (13.88ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (17.35ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (18.86ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (12.95ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (16.3ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (31.43ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (2.72ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (15.05ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (17.93ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (20.2ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (14.4ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (3.3ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (18.84ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.918074215Z","level":"ERROR","duration":"644.186µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (19.98ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.918901122Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"fc034b8d-624a-4499-9a49-c521d43be807","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.919592488Z","level":"ERROR","duration":"2.122558ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(370)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.919624329Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"d5b3d83f-64ef-4d82-83a3-e2b1adf00b4e","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":2,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(370)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.922838257Z","level":"ERROR","duration":"175.242µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.922890158Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ee4d4642-7e70-4c63-862a-b1145d867a61","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (9.48ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (8.4ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (8.57ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (28.04ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (8.86ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (13.11ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (12.67ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (9.99ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.930216062Z","level":"ERROR","duration":"505.465µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.930283782Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"99ee5a73-7f13-438b-ad95-e99885886892","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket13/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (6.98ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (24.66ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (7.38ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (21.59ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (9.74ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (7.25ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (9.84ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.938285712Z","level":"ERROR","duration":"290.382µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.938346173Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6c25a644-842b-48a7-a245-7fc2ef9de13b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket15/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.939393442Z","level":"ERROR","duration":"1.726495ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.939440103Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2647dcd6-7e45-4f4c-b3b4-4f012ac40b58","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket14/","status_code":503,"duration_ms":1,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (19.2ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (15.01ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (8.17ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (9.64ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (28.26ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (3.85ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (3.87ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (23.55ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (8.98ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (11.58ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.949495591Z","level":"ERROR","duration":"243.602µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.949595132Z","level":"ERROR","duration":"514.745µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.949649752Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"5a86a9c2-f307-4a63-9c78-69455cb2daf6","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.949793554Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"82b6b3f7-6aa2-4a32-ab77-e5027abc9739","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket17/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.950115236Z","level":"ERROR","duration":"1.384852ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.950145397Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6ac5efb4-e171-46f4-a753-51656a6acfaa","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket19/","status_code":503,"duration_ms":1,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (6.46ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.953659928Z","level":"ERROR","duration":"170.922µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.953787729Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"407bcec0-0a3e-4195-bf77-d5356c88b2c7","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (5.7ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (8.22ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (8.18ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.956319411Z","level":"ERROR","duration":"586.605µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.956375592Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c423bce9-95d9-4c12-93f5-6bfdf83db590","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (23.27ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (5.84ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (12.2ms) niks3-go-unit-tests> --- PASS: TestObjectStatsTrigger (1.18s) niks3-go-unit-tests> === CONT TestGracefulShutdownDrainsInflight niks3-go-unit-tests> 2026/08/27 09:40:30 INFO Starting HTTP server address=127.0.0.1:42557 niks3-go-unit-tests> 2026/08/27 09:40:30 INFO Shutdown signal received, draining in-flight requests timeout=10s niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (8.16ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.96991653Z","level":"ERROR","duration":"1.879956ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.969981411Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"38bbee5e-f2d1-4e73-9ba3-cccf6d7e32c2","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":6,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20241026095416_initial_model.sql (26.93ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.971465244Z","level":"ERROR","duration":"174.262µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.971491524Z","level":"ERROR","duration":"175.602µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.971535604Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c96bc788-75a7-4174-bcf4-257a273480b2","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.971540884Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a4637e67-8abc-499e-8a22-af056bb77628","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket17/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (4.97ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (17.36ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.975396038Z","level":"ERROR","duration":"2.445721ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.975444959Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"d772284a-de48-47eb-a9c0-19c9f75b13be","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket22/","status_code":503,"duration_ms":2,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (12.71ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251210153512_drop_unused_gin_index.sql (7.14ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.98239794Z","level":"ERROR","duration":"160.242µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.9824545Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8a54eeba-2855-41d8-ae10-6f12cf11175a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket19/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (10.58ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.984066454Z","level":"ERROR","duration":"160.741µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.984112475Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4b5a93d6-2e3b-4453-9a14-573a7df95ef8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket22/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (9.96ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20251218171726_add_pins.sql (8.48ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (3.5ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (8.47ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 20260628120000_add_object_size_and_stats.sql (4.27ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (2.82ms) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:30 OK 1_commit_pending_closure.sql (3.43ms) niks3-go-unit-tests> 2026/08/27 09:40:30 OK 2_object_stats_trigger.sql (965.63µs) niks3-go-unit-tests> 2026/08/27 09:40:30 goose: up to current file version: 2 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.99492763Z","level":"ERROR","duration":"3.013787ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.99500125Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0ad31e5c-72c9-40aa-8569-43453a7bd551","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket23/","status_code":503,"duration_ms":3,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.997542793Z","level":"ERROR","duration":"533.605µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.997593393Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"5f689161-9b11-4861-bd79-3dd90f3fbf82","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket24/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> --- PASS: TestResurrectedObjectNotDeleted (1.16s) niks3-go-unit-tests> === CONT TestGCTaskStore_Fail niks3-go-unit-tests> --- PASS: TestGCTaskStore_Fail (0.00s) niks3-go-unit-tests> === CONT TestProxyWriteTimeout/1_GiB_nar niks3-go-unit-tests> === CONT TestReadProxyRangeRequest niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.999286228Z","level":"ERROR","duration":"873.468µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:30.999335548Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a52d489f-cba8-4747-9f93-11ee42ad2d94","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket25/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.024836012Z","level":"ERROR","duration":"140.041µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.024885553Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f7f577e1-b443-49fa-956c-5d5bb44cdbae","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.025090734Z","level":"ERROR","duration":"48.18µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.025105835Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ca20b13d-bfbf-492e-96f2-f4a16ac34cf0","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> --- PASS: TestGracefulShutdownDrainsInflight (0.07s) niks3-go-unit-tests> === CONT TestRedundantMultipartUpload niks3-go-unit-tests> --- PASS: TestReadProxyConditionalGet (1.24s) niks3-go-unit-tests> === CONT TestGCTaskStore_GetReturnsLatest niks3-go-unit-tests> --- PASS: TestGCTaskStore_GetReturnsLatest (0.00s) niks3-go-unit-tests> === CONT TestGCTaskStore_CompletedAllowsNewTask niks3-go-unit-tests> --- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s) niks3-go-unit-tests> === CONT TestReadProxyInvalidPath niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.059528497Z","level":"ERROR","duration":"188.482µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(370)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.059585257Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"bd5c8507-5b92-4fad-898e-5fabc08bc78a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(370)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.059813219Z","level":"ERROR","duration":"74.76µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(370)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.059836379Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"85fde4fc-5ca6-48ba-985d-b15ce128d38d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(370)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.071687463Z","level":"ERROR","duration":"173.661µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.071744284Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"5d4029e8-b3ff-4a15-8b1b-55bd01b30593","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket17/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> 2026-08-27 09:40:31.083 UTC [623] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:31.083 UTC [623] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20241026095416_initial_model.sql (8.79ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251210153512_drop_unused_gin_index.sql (6.47ms) niks3-go-unit-tests> 2026/08/27 09:40:31 INFO Received complete multipart upload request method=POST path=/api/multipart/complete niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.10771418Z","level":"ERROR","duration":"237.262µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.10777992Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6dcfd6f1-6f58-4d2b-9f99-bb4c1fdbf9fb","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> 2026/08/27 09:40:31 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:31 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst niks3-go-unit-tests> --- PASS: TestCompleteMultipartUnregistered (1.32s) niks3-go-unit-tests> === CONT TestReadProxyDisabled niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251218171726_add_pins.sql (22.7ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.133048662Z","level":"ERROR","duration":"186.762µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.133122763Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a3ccd21e-eb51-46ac-a398-5408d4c6c737","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket15/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> 2026/08/27 09:40:31 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:31 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:31 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:31 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026-08-27 09:40:31.166 UTC [625] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:31.166 UTC [625] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:31 WARN mTLS auth: subject not in bound subjects subject="CN=reader" niks3-go-unit-tests> 2026/08/27 09:40:31 WARN mTLS auth: subject not in bound subjects subject="CN=writer" niks3-go-unit-tests> --- PASS: TestService_NativeMTLS (1.39s) niks3-go-unit-tests> === CONT TestOrphanedObjectsGCStressTest niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20260628120000_add_object_size_and_stats.sql (61.22ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:31 OK 1_commit_pending_closure.sql (4.84ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 2_object_stats_trigger.sql (1.13ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: up to current file version: 2 niks3-go-unit-tests> --- PASS: TestReadProxyNarinfo (1.41s) niks3-go-unit-tests> === CONT TestGCTaskStore_GetEmpty niks3-go-unit-tests> --- PASS: TestGCTaskStore_GetEmpty (0.00s) niks3-go-unit-tests> === CONT TestOrphanedObjectsGC niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.207354234Z","level":"ERROR","duration":"3.158327ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.207507356Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e07a8d0c-9040-4f48-830b-516b2d07c218","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":3,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20241026095416_initial_model.sql (23.19ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251210153512_drop_unused_gin_index.sql (11.6ms) niks3-go-unit-tests> 2026/08/27 09:40:31 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst niks3-go-unit-tests> 2026/08/27 09:40:31 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251218171726_add_pins.sql (29.43ms) niks3-go-unit-tests> --- PASS: TestReadProxyHead (1.47s) niks3-go-unit-tests> === CONT TestClientIntegration niks3-go-unit-tests> --- PASS: TestPresignedUploadRegisteredBeforeCommit (1.43s) niks3-go-unit-tests> === CONT TestGCTaskStore_DeduplicateSameParams niks3-go-unit-tests> --- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s) niks3-go-unit-tests> === CONT TestGCTaskStore_StartNew niks3-go-unit-tests> --- PASS: TestGCTaskStore_StartNew (0.00s) niks3-go-unit-tests> === CONT TestGCMetrics niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20260628120000_add_object_size_and_stats.sql (18.03ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:31 OK 1_commit_pending_closure.sql (14.95ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 2_object_stats_trigger.sql (3.78ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: up to current file version: 2 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.300460452Z","level":"ERROR","duration":"462.065µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.300743154Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"89696fd2-8957-4aa8-b30d-4f01b76da222","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.309564212Z","level":"ERROR","duration":"204.701µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.309622572Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"57abb50b-8773-4559-9887-3ed15a8329b3","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> 2026/08/27 09:40:31 INFO Received cleanup request method=DELETE path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:31 INFO Aborted multipart uploads count=1 niks3-go-unit-tests> --- PASS: TestMultipartCleanup (1.56s) niks3-go-unit-tests> === CONT TestGCBugBareHashReferences niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.359023626Z","level":"ERROR","duration":"163.422µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.359080446Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"81bf6dbd-e7b9-462b-acb2-607a3d8cf49d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket15/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.362296714Z","level":"ERROR","duration":"118.021µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.362351035Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ed229176-996c-4409-8794-92238011334f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.395121002Z","level":"ERROR","duration":"167.621µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.395181803Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"b055f231-6094-4667-abb7-cb819616188d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> 2026-08-27 09:40:31.411 UTC [648] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:31.411 UTC [648] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026-08-27 09:40:31.449 UTC [649] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:31.449 UTC [649] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.477730788Z","level":"ERROR","duration":"172.782µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.477795668Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"7f08c8bf-eb86-4b01-a37a-7f868ec69c35","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> 2026-08-27 09:40:31.485 UTC [650] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:31.485 UTC [650] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.512943557Z","level":"ERROR","duration":"173.622µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.513007597Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"10752563-d154-4a4a-a09d-a65d2f6f7892","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> 2026-08-27 09:40:31.516 UTC [651] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:31.516 UTC [651] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.523674211Z","level":"ERROR","duration":"194.302µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.523733691Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"62539178-424b-426a-a21d-a86dfe057e58","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.533988302Z","level":"ERROR","duration":"164.802µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.534065362Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"3f2c9f62-a495-4d6c-ad9d-9c74ae2e3b48","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20241026095416_initial_model.sql (57.76ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20241026095416_initial_model.sql (52.8ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251210153512_drop_unused_gin_index.sql (11.5ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251210153512_drop_unused_gin_index.sql (13.94ms) niks3-go-unit-tests> 2026-08-27 09:40:31.563 UTC [652] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:31.563 UTC [652] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251218171726_add_pins.sql (18.7ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20241026095416_initial_model.sql (25.61ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20241026095416_initial_model.sql (86.34ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251218171726_add_pins.sql (20.34ms) niks3-go-unit-tests> 2026-08-27 09:40:31.590 UTC [653] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:31.590 UTC [653] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.592236313Z","level":"ERROR","duration":"256.643µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.592308073Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f5a60385-24bf-447c-b766-b849c85bc1e3","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251210153512_drop_unused_gin_index.sql (11.57ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251210153512_drop_unused_gin_index.sql (14.9ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20260628120000_add_object_size_and_stats.sql (18.36ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20260628120000_add_object_size_and_stats.sql (17.53ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251218171726_add_pins.sql (8.3ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251218171726_add_pins.sql (5.81ms) niks3-go-unit-tests> 2026-08-27 09:40:31.603 UTC [654] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:31.603 UTC [654] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:31 OK 1_commit_pending_closure.sql (11.1ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 1_commit_pending_closure.sql (15.68ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 2_object_stats_trigger.sql (11.77ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: up to current file version: 2 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.625348363Z","level":"ERROR","duration":"1.574734ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.625419244Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"350dc3e7-a551-4738-9a50-932ee7a2fb27","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":1,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> 2026/08/27 09:40:31 OK 2_object_stats_trigger.sql (10.19ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: up to current file version: 2 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.62837605Z","level":"ERROR","duration":"471.444µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.62843903Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e9f4303d-5a40-495a-9afa-637ddecaa53c","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20260628120000_add_object_size_and_stats.sql (26.06ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20260628120000_add_object_size_and_stats.sql (27.58ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20241026095416_initial_model.sql (49.61ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20241026095416_initial_model.sql (17.23ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20241026095416_initial_model.sql (31.95ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 1_commit_pending_closure.sql (6.61ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 1_commit_pending_closure.sql (13.42ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251210153512_drop_unused_gin_index.sql (6.74ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251210153512_drop_unused_gin_index.sql (8.25ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251210153512_drop_unused_gin_index.sql (10.71ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 2_object_stats_trigger.sql (6.44ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: up to current file version: 2 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.645207698Z","level":"ERROR","duration":"516.865µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.645280018Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"51f2dc0c-c766-48d1-ab9e-da600fb9292a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> 2026/08/27 09:40:31 OK 2_object_stats_trigger.sql (9.83ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251218171726_add_pins.sql (11.77ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.655925392Z","level":"ERROR","duration":"2.084238ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.655983232Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a2118b98-f01b-482a-9242-2e81cbf900be","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket31/","status_code":503,"duration_ms":2,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251218171726_add_pins.sql (15.48ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20251218171726_add_pins.sql (14.79ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.662319948Z","level":"ERROR","duration":"172.842µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.662378148Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"82e23eb5-b19f-4490-9ac7-d2df32c51b36","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket31/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20260628120000_add_object_size_and_stats.sql (7.2ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20260628120000_add_object_size_and_stats.sql (5.52ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:31 OK 20260628120000_add_object_size_and_stats.sql (6.93ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:31 OK 1_commit_pending_closure.sql (8.53ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 1_commit_pending_closure.sql (8.5ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 1_commit_pending_closure.sql (9ms) niks3-go-unit-tests> 2026/08/27 09:40:31 OK 2_object_stats_trigger.sql (6.42ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:31 OK 2_object_stats_trigger.sql (8.03ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:31 OK 2_object_stats_trigger.sql (6.59ms) niks3-go-unit-tests> 2026/08/27 09:40:31 goose: up to current file version: 2 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.684470782Z","level":"ERROR","duration":"788.647µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.684534063Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"5d9aac7c-8f86-4538-b9f4-9b75ecd71a5d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket32/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.687108186Z","level":"ERROR","duration":"4.364559ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.687174906Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"bcedfa90-8eea-4a24-ba59-631c95b84386","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":4,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.688173535Z","level":"ERROR","duration":"355.743µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.688218455Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e2ffb8a7-e661-450e-bb9e-2147307e634c","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.712835951Z","level":"ERROR","duration":"163.121µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.712896332Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2c7ae4b0-d1e4-46ac-82b5-dd5c9a6eb655","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.738665758Z","level":"ERROR","duration":"231.402µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.738721559Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6e5afcbd-71b5-4f22-9505-325c575505c3","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.746524487Z","level":"ERROR","duration":"2.050478ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.746589287Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8505f72a-2c21-4c96-95a4-13f67fa87582","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":2,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.748474224Z","level":"ERROR","duration":"165.861µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.748532385Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"441a78c1-5410-4fc0-addb-0c01607e850d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket32/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.754412816Z","level":"ERROR","duration":"159.981µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.754514337Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"3b474a61-75e3-4330-999a-220e92578261","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.787305945Z","level":"ERROR","duration":"223.342µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.787306105Z","level":"ERROR","duration":"169.182µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.787374685Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"200e5596-fd09-46aa-918a-c54f8d0bff9a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.787518787Z","level":"ERROR","duration":"97.981µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.787548307Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f7c6c70d-2155-4b46-af28-bed0b302fa6a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.788493675Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"34a42e1f-a605-4d99-a7e2-e9f3f4a1859d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":1,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.798558284Z","level":"ERROR","duration":"204.822µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.798609464Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"32ffc1c1-0c2a-4947-bb2e-700d335f19ef","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.808144548Z","level":"ERROR","duration":"164.322µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.808205668Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c34ee7da-9fcf-479b-879c-ea792a4eb701","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.84484157Z","level":"ERROR","duration":"184.802µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.844907111Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"163a775b-7b0b-40b5-96db-095f236d1c96","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket32/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.846200462Z","level":"ERROR","duration":"148.661µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.846255162Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0f94f744-9574-4b6c-8334-78ebf1402187","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.883376468Z","level":"ERROR","duration":"176.221µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.883434549Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"bd4a1487-72b0-4fde-b617-f008ad549a37","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.890998615Z","level":"ERROR","duration":"158.121µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.891057496Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ab79d628-14d0-482f-b021-8afb752aa1ac","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.895702076Z","level":"ERROR","duration":"249.102µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.895758857Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"cbd693e6-5a42-4f95-9f19-bf6b9f85439b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket15/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.948017236Z","level":"ERROR","duration":"174.802µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:31.948077696Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f9987d6d-5bc6-4019-959d-733cfa0aab48","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> 2026/08/27 09:40:32 INFO Received complete multipart upload request method=POST path=/api/multipart/complete niks3-go-unit-tests> 2026/08/27 09:40:32 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=MmJjZDQzOTUtNDQ2Yi00YzUxLWJiNjQtZDExYTRhM2YxYjNjLmViZTQ3YWE0LTRmMjgtNDQ0Ny1iYjIxLTIxZWM1ODlmMjY2NngxNzg3ODIzNjMxMTk2Njg5ODIx parts=10 niks3-go-unit-tests> 2026/08/27 09:40:32 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete niks3-go-unit-tests> 2026/08/27 09:40:32 INFO Completed upload id=1 niks3-go-unit-tests> 2026/08/27 09:40:32 INFO Received get closure request method=GET path=/api/closures/00000000000000000000000000000000 niks3-go-unit-tests> 2026/08/27 09:40:32 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:32 INFO Starting cleanup of old closures method=DELETE path=/api/closures niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.039844222Z","level":"ERROR","duration":"161.802µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.039904282Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"cc82ee3a-9c19-4d0f-8f99-eea5aa5b00a8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket31/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> 2026/08/27 09:40:32 INFO Aborted multipart uploads count=0 niks3-go-unit-tests> 2026/08/27 09:40:32 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=0 objects-failed-to-delete=0 niks3-go-unit-tests> 2026/08/27 09:40:32 INFO Vacuumed table table=pending_closures niks3-go-unit-tests> 2026/08/27 09:40:32 INFO Vacuumed table table=pending_objects niks3-go-unit-tests> 2026/08/27 09:40:32 INFO Vacuumed table table=multipart_uploads niks3-go-unit-tests> 2026/08/27 09:40:32 INFO Vacuumed table table=closures niks3-go-unit-tests> 2026/08/27 09:40:32 INFO Vacuumed table table=objects niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.09550077Z","level":"ERROR","duration":"181.302µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.095564171Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8abdeaea-49ea-454b-be94-81526a0c43a9","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.104146126Z","level":"ERROR","duration":"170.642µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.104205106Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"05dea3e9-229b-45b2-9918-323405f538c8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.114000032Z","level":"ERROR","duration":"152.421µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.114053093Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"da80bf4e-69af-45a4-8403-ad75e7d866a7","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.114596158Z","level":"ERROR","duration":"141.762µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.114625698Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"b6c92574-8856-4849-b17d-601d1f33db98","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> 2026/08/27 09:40:32 INFO Received get closure request method=GET path=/api/closures/00000000000000000000000000000000 niks3-go-unit-tests> --- PASS: TestService_createPendingClosureHandler (2.35s) niks3-go-unit-tests> === CONT TestPinProtectsFromGC niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.144869083Z","level":"ERROR","duration":"163.021µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.144928104Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"45412508-f173-4ea6-94e9-7c982ecde223","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.17408618Z","level":"ERROR","duration":"166.181µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.174143941Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"aa8568fc-f3f6-46da-a90b-f10c03a1be0e","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.177084046Z","level":"ERROR","duration":"166.601µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.177141147Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"54a045f4-6952-40fa-84fb-88ef61c6e4e2","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.205139333Z","level":"ERROR","duration":"233.022µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.205198653Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4aa53a00-9384-4846-a81b-124581b1ecad","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> 2026-08-27 09:40:32.212 UTC [659] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:32.212 UTC [659] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.225976115Z","level":"ERROR","duration":"164.501µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.226034576Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"5e1e569b-4a4b-4dc9-b6a7-9f50299ae967","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> 2026/08/27 09:40:32 OK 20241026095416_initial_model.sql (8.32ms) niks3-go-unit-tests> 2026/08/27 09:40:32 OK 20251210153512_drop_unused_gin_index.sql (1.54ms) niks3-go-unit-tests> 2026/08/27 09:40:32 OK 20251218171726_add_pins.sql (3.84ms) niks3-go-unit-tests> 2026/08/27 09:40:32 OK 20260628120000_add_object_size_and_stats.sql (3.09ms) niks3-go-unit-tests> 2026/08/27 09:40:32 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:32 OK 1_commit_pending_closure.sql (1.82ms) niks3-go-unit-tests> 2026/08/27 09:40:32 OK 2_object_stats_trigger.sql (795.07µs) niks3-go-unit-tests> 2026/08/27 09:40:32 goose: up to current file version: 2 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.242136617Z","level":"ERROR","duration":"434.563µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.242188578Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0d557119-6a4a-482e-affd-775aeb778ff5","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.269242695Z","level":"ERROR","duration":"158.821µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.269297456Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"5a119063-257f-4a05-a102-de9423516736","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.336400205Z","level":"ERROR","duration":"174.422µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.336463045Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"3411b9e7-337b-4231-8128-01cb3c34feff","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.353441155Z","level":"ERROR","duration":"165.022µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.353500695Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"eac7b430-1eee-43d2-bbfc-08050d746423","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.392420557Z","level":"ERROR","duration":"164.421µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.392475797Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"68f59675-b696-4a5a-b0e0-a27aa7983668","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.402786867Z","level":"ERROR","duration":"92.7µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.402842328Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8682d566-4c5e-4816-9127-9bcc45648711","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.40310213Z","level":"ERROR","duration":"771.227µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.403161971Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"3d2862bc-2a10-4466-bfde-7cd91dc9979f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.411952368Z","level":"ERROR","duration":"150.261µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.412005309Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ef432076-04be-4002-9d35-07bf39e3190f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket31/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.412496633Z","level":"ERROR","duration":"86.001µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.412524193Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4256fff6-2663-4247-be6b-348d3f33e65b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.431393079Z","level":"ERROR","duration":"156.442µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.431447859Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8343d8ae-1880-4608-93a2-cac14fe842e3","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.446500631Z","level":"ERROR","duration":"172.161µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.446555632Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"1cb150a6-d391-4ef6-84e9-668ae1f9ffd2","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.543634084Z","level":"ERROR","duration":"227.582µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.543692145Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"5fab57a3-04b5-43c7-a721-fb929ea1b3c5","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket15/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.572622439Z","level":"ERROR","duration":"159.921µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.572680379Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4ca3ff92-5a66-46b6-b5d4-048d83b75bda","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket32/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.657739246Z","level":"ERROR","duration":"163.042µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.657796026Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"aa5a16f9-2793-4fca-b23f-aecd23c48423","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.659512321Z","level":"ERROR","duration":"147.801µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.659562342Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"7e8684a5-190a-4e1a-bc43-c43078509126","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.661300597Z","level":"ERROR","duration":"133.541µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.661348497Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"fb1dde28-8c4c-47cd-b4aa-b8aad21bd0cd","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.669216746Z","level":"ERROR","duration":"149.561µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.669268547Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"20559144-9d8e-48e1-8b9f-f3c28a2412bc","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.68220202Z","level":"ERROR","duration":"156.121µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.682261521Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"edba74ae-5a46-4307-a301-adc5ca9f59ed","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.700217958Z","level":"ERROR","duration":"155.321µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.700272699Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"448dbb64-c77e-4088-b43b-841437ca1f33","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.719370207Z","level":"ERROR","duration":"157.462µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.719428127Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e4e81b82-1f89-464e-a949-64242d03ec78","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.721133622Z","level":"ERROR","duration":"136.561µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.721184363Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"84e9ba78-446c-46b0-88a6-2cb245feb59a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.778145462Z","level":"ERROR","duration":"159.801µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.778200923Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ed21e4a2-6d00-42df-bc97-65932ee57ffc","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.78015308Z","level":"ERROR","duration":"136.221µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.780202541Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"cc30ba4e-f1b2-48af-8c0d-570bfeec3a19","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket32/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.792341127Z","level":"ERROR","duration":"167.241µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.792402848Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a4ce13cd-c77f-4064-bd2a-3e95e61eeb7c","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket15/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.805485963Z","level":"ERROR","duration":"279.903µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.805540463Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"b66bd204-1fee-4cec-abb0-db0c74bec4b5","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.809377657Z","level":"ERROR","duration":"163.762µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.809426857Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"9a6c351b-2a34-4d87-97c1-f38240977cd1","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.821301781Z","level":"ERROR","duration":"149.441µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.821353822Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"30a12ef9-63cb-4587-869e-f915c08fa445","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket31/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.837308422Z","level":"ERROR","duration":"153.181µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.837363342Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"da95c240-3921-4208-89ec-bb3b520da006","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.879349151Z","level":"ERROR","duration":"228.782µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.879404931Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8cbcdd8b-6f9d-498b-88f4-544b18b6b2ef","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.945580712Z","level":"ERROR","duration":"160.501µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.945640753Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"db05c947-2571-4689-953b-ffe5996335d6","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.987758402Z","level":"ERROR","duration":"168.181µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.987816523Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"7a50676d-4421-49cc-84d5-ea21a77829eb","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.988211186Z","level":"ERROR","duration":"76.32µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:32.988237287Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"95963ce1-ee95-401e-898e-843809421e8d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.030066654Z","level":"ERROR","duration":"170.682µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.030132174Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0d4cff77-f5ba-445b-b49c-1e1b48ea5d4d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.037090116Z","level":"ERROR","duration":"164.942µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.037144676Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"da9bb002-d1b4-499f-9b3d-96d7b03dddbe","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.046085675Z","level":"ERROR","duration":"154.142µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.046139535Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0cabe1c8-45dc-466c-88b7-ec1dbf83564b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.081076802Z","level":"ERROR","duration":"160.582µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.081139982Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"b8191b84-d610-45e4-97e5-e670354f97b5","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.081372644Z","level":"ERROR","duration":"53.34µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.081392945Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e09c6ead-bc94-4fa7-9744-cc3ba6afd6a8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.090095081Z","level":"ERROR","duration":"151.661µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.090158281Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"46bbb916-f870-4837-9f49-24f3ab98f01a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.106124322Z","level":"ERROR","duration":"159.222µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.106183242Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"56c8bc4b-cc2e-4bf4-997f-6f3c5656da25","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket31/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.143419409Z","level":"ERROR","duration":"164.242µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.143476909Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f6ee689f-a91b-45bf-b657-104eb49f98e4","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.257549791Z","level":"ERROR","duration":"227.162µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.257711372Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"00f6e79e-c0f5-4912-93da-17ac4fd05f14","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.266671871Z","level":"ERROR","duration":"165.162µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.266729311Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"7a7449dd-b33c-467a-90e6-30e0b0274e24","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.348710771Z","level":"ERROR","duration":"160.841µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.348765692Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"cdf82a7a-1936-4ab3-ad3b-de43b51de47e","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.354619403Z","level":"ERROR","duration":"148.361µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.354664323Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4cfe7902-37f8-41b0-91b3-bffebc9dfc3a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.409501125Z","level":"ERROR","duration":"159.022µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.409555665Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"38cbb2bc-4595-4984-8e36-29dff9bdfabe","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.467417993Z","level":"ERROR","duration":"161.361µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.467648875Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2d05bfcb-30a2-4ae4-a847-44d3b10ff647","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.502617942Z","level":"ERROR","duration":"206.962µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.502799364Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ce20b235-6307-4e0b-a38d-1df903d1cba4","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.551866835Z","level":"ERROR","duration":"153.202µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.552003596Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c47dbe1f-5d11-4887-b22a-7d3018d669e0","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket15/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.565920518Z","level":"ERROR","duration":"198.862µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.565967178Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"303fb34b-e2b0-433d-b87b-534272f44016","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.594977153Z","level":"ERROR","duration":"166.901µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.595137375Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"53ae92dd-1bb7-499d-8ad5-e5e9355eac03","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket32/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.670367335Z","level":"ERROR","duration":"203.582µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.670533216Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f7965d8d-7961-40b0-9e17-94198fdef99e","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.722703514Z","level":"ERROR","duration":"155.681µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.722919736Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"d410b317-0d46-42e1-9612-ddace47a1e0c","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.761701137Z","level":"ERROR","duration":"300.543µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.761762737Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"d8df5adf-a0a3-451f-803c-3209c559d075","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.762789406Z","level":"ERROR","duration":"123.081µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.762824307Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"edc309fe-fae5-43b9-8cfa-85e782cfabbe","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket31/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.778983068Z","level":"ERROR","duration":"240.742µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.77914239Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"93c6424b-92a8-429a-bc8c-a793f7af36fa","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.779932717Z","level":"ERROR","duration":"109.061µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.779962557Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"65efa1b8-d9af-433f-9da4-91c78a5007ff","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.79278347Z","level":"ERROR","duration":"149.522µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.79285521Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"1fbbc5a1-57ee-4a3f-a82f-215d69c4a799","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.795611735Z","level":"ERROR","duration":"140.482µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.795658795Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8bca0c84-74e5-40e4-8ae7-84eb9b36e153","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.801379625Z","level":"ERROR","duration":"132.001µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.801427926Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"254be516-66b8-43e1-974b-f9e9fb3b8342","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.81218038Z","level":"ERROR","duration":"149.081µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.812318901Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"88b61a9d-a0a1-4a46-b166-c05c91eb41b0","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.851158182Z","level":"ERROR","duration":"152.541µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.851212803Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"bdc7c5fc-00f1-4822-9e21-dca0bb62d3b5","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.862172959Z","level":"ERROR","duration":"135.561µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.862215979Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e55da123-738b-47f2-b8b5-c8b7459092fb","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.901114521Z","level":"ERROR","duration":"210.362µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.901163761Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"19f1e43d-626a-4143-8c41-4f8b6d88b648","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket15/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.936176628Z","level":"ERROR","duration":"151.101µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.936228789Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"5bd93148-c10b-4f70-8e96-d1afc2890d5f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.962838543Z","level":"ERROR","duration":"162.802µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.962905763Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"dd101357-1f0b-4583-ba3e-27a78edbc851","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.973158473Z","level":"ERROR","duration":"158.321µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:33.973351275Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"92e8e9c9-9d4a-4cd3-b312-5ce55b95e4c2","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.030413176Z","level":"ERROR","duration":"154.762µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.030466656Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e321aadb-2767-4e5f-a495-3c452d094ad8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.056539065Z","level":"ERROR","duration":"156.481µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.056599346Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"07694f9f-bd84-4248-b50c-be55b26a6f02","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.077417409Z","level":"ERROR","duration":"158.902µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.077472209Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8d24aa95-6421-45c7-a766-3096d73862e0","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.078094615Z","level":"ERROR","duration":"82.201µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.078119595Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a81610f1-b69a-4495-b0e3-a3a7c10baff9","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.081854707Z","level":"ERROR","duration":"126.661µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.081902688Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"5566183f-4f37-4573-9175-d2eda1a6c3f9","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.128783779Z","level":"ERROR","duration":"156.801µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.12884046Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"827d774b-d8d5-4323-8dc9-777ee3dcd4c8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.137761158Z","level":"ERROR","duration":"146.781µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.137823999Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"d6c61a60-7686-4138-aa45-49da383e99eb","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.146734957Z","level":"ERROR","duration":"168.801µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.146896519Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c3343420-b6c1-4191-a4ff-f4387da3e6bc","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.283302116Z","level":"ERROR","duration":"221.542µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.283372177Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c9cc91ea-a7ce-41d6-9cfa-64e8b167a34f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.296391891Z","level":"ERROR","duration":"154.321µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.296446911Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4baa571a-fc39-491a-8ab7-228f598b874a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.303261391Z","level":"ERROR","duration":"149.901µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.303315372Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"b9da51da-c1d5-451c-815c-9df0178f6bdc","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.341179324Z","level":"ERROR","duration":"150.841µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.341232824Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"152694b7-089d-45ea-9b6b-83a9a2692ecd","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.347098796Z","level":"ERROR","duration":"153.881µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.347157637Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"08385653-17f6-417c-85fe-680e9fb4e65c","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.350165743Z","level":"ERROR","duration":"145.341µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.350213804Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0236c22c-f5f8-4fb9-82ea-00cafafa8bf2","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.397024354Z","level":"ERROR","duration":"161.961µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.397080915Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4e27bcdb-7955-4c17-9bf0-62007ca805a2","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.419033988Z","level":"ERROR","duration":"158.622µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.419090788Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"5c094443-3d04-407c-a053-11fd91791fb5","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket31/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.443924046Z","level":"ERROR","duration":"152.421µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.444141868Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"005f9f79-1b46-4bd4-a518-986fbe8f4415","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.45002608Z","level":"ERROR","duration":"149.242µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.450209581Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c7e985e6-328c-4c71-97ac-7d6a70cc386a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.485259049Z","level":"ERROR","duration":"155.561µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.485438331Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"b4eaa58e-402d-4e9b-a420-c0d787863cb1","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket32/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.50930324Z","level":"ERROR","duration":"151.401µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.50935804Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"3bcebbd8-9794-44a5-a61b-548fbe4e8804","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.544368128Z","level":"ERROR","duration":"153.441µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.544422488Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"3c45aae1-db7a-45f2-92d8-30e74bd9dc41","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.555316144Z","level":"ERROR","duration":"150.061µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.555473226Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c3508a4f-7b16-4069-8641-9792a1d3a9e1","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket15/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.639594064Z","level":"ERROR","duration":"211.562µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.639782586Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4a806374-9700-4493-8ff5-5517ead869ae","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.664667664Z","level":"ERROR","duration":"151.401µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.664858346Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"847adce7-5353-4657-ad31-3710eecb883e","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.673636743Z","level":"ERROR","duration":"147.542µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.673691963Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"651a92dc-5366-4d97-8806-41ac9e730ab1","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.765939353Z","level":"ERROR","duration":"153.461µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.766111315Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"30c112c0-fc73-416f-bd1a-9e99c8191b57","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.831022224Z","level":"ERROR","duration":"154.821µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.831292187Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c1eea610-6a5c-4ed4-95ed-d911fc96ce7c","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.862343059Z","level":"ERROR","duration":"158.021µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.862574321Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"bd4e9e85-eca6-4c9f-9be1-aea435cb74f3","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.906562428Z","level":"ERROR","duration":"205.142µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.906711269Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ca87c10c-f447-4332-ab08-fcd7aaa6e9f8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.911585852Z","level":"ERROR","duration":"149.742µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.911754193Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"5bd7ef83-7cc5-429d-b0e2-77944dc56a71","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket32/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.920613451Z","level":"ERROR","duration":"153.261µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.920776312Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"b0737d2b-5a45-49d7-9aa3-4c5dc60018d5","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.924621266Z","level":"ERROR","duration":"141.141µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.924671967Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e2faf12f-68b1-4101-8177-b5f319fb5eeb","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.939499357Z","level":"ERROR","duration":"147.101µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.939564938Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"da0a0ea6-7ff5-4161-a2ac-07fb175fe0c8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.94444884Z","level":"ERROR","duration":"200.162µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.944500681Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a3633009-e3d1-447e-99d0-98006561fd7a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.975311831Z","level":"ERROR","duration":"148.841µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.975366172Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8c913811-0748-49be-bae3-d87b701d3871","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.978172596Z","level":"ERROR","duration":"146.801µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:34.978229757Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"00adfd93-99bb-432b-8596-c06cce10b681","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.012217075Z","level":"ERROR","duration":"151.101µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.012270396Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a0876367-8d91-47f5-850b-22e1742fb296","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.02416632Z","level":"ERROR","duration":"149.901µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.024223041Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"abe425e2-c3d0-4a26-a1a4-45ebd0286d2b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.057046289Z","level":"ERROR","duration":"155.261µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.057102189Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4a3c6e9d-12a8-4f78-a431-83e3c4b833d5","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.06396881Z","level":"ERROR","duration":"148.702µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.064106591Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"152e8035-febf-43a2-b6b3-4620d11769dc","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.075928355Z","level":"ERROR","duration":"149.742µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.075983155Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"3186b52e-2d8d-45b7-9a0f-98db289f7fac","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.103815939Z","level":"ERROR","duration":"154.441µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.10387208Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"580f9015-c56e-4699-98ab-7e413f866d6a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.113703486Z","level":"ERROR","duration":"147.181µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.113757147Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"d33599e4-54a7-43c5-aaff-72cb83a0b58f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.161758028Z","level":"ERROR","duration":"153.021µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.16195457Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"5586ef8d-1728-45bb-b928-c9e84506f8f7","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.166992214Z","level":"ERROR","duration":"145.681µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.167043795Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2c5c88f5-37f8-4149-8677-e10cc45e9055","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket31/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.25311175Z","level":"ERROR","duration":"215.922µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.253169751Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"068d8538-5454-476d-8886-4bf86e23ac7e","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.258001813Z","level":"ERROR","duration":"148.401µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.258158575Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e955e5f7-0941-4476-9ea9-0446bed8d2a7","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket15/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.280554991Z","level":"ERROR","duration":"152.021µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.28149828Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"02453dff-235d-47a2-97b6-5f79041745eb","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":1,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.2883801Z","level":"ERROR","duration":"148.001µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.288560442Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"b731dd79-bf89-4380-84c0-e65bac19796f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.292406855Z","level":"ERROR","duration":"144.701µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.292456696Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"9f56843d-3bfc-4c91-8cad-cd74c1206587","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.314427309Z","level":"ERROR","duration":"146.282µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.31452995Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c5c14275-ce0d-4ef3-80a0-c24979eb566f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.316258765Z","level":"ERROR","duration":"129.861µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.316302445Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"dc0eae3e-8b52-452e-bfaa-2be3a6817290","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket31/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.354146417Z","level":"ERROR","duration":"155.061µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.354198178Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8641d3a2-82db-4d4e-a39c-88a031fbafdf","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.381076654Z","level":"ERROR","duration":"154.582µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.381130994Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"80057391-3d5d-402a-8f29-09c10b21e2f1","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.382813789Z","level":"ERROR","duration":"132.161µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.38286127Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"cadca9c1-696f-42e1-8d29-f553b5a7bc50","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.478883453Z","level":"ERROR","duration":"154.922µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.478944173Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"486e74de-4883-4fed-8f11-f867839e50e9","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket32/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.489633247Z","level":"ERROR","duration":"150.161µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.489685468Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f7e77df2-488d-4ee1-a197-279e80f5df3f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.495505838Z","level":"ERROR","duration":"149.241µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.495560139Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"926ee3a7-4340-4c2d-9aab-4a1f8e1f2f34","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.514313503Z","level":"ERROR","duration":"152.661µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.514531265Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"20896f63-bfc9-45b0-b20f-99a6a05dfd0a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.65969636Z","level":"ERROR","duration":"219.382µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.659862082Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"37c3e442-7a45-462f-b4a9-52bb979d4aa8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.667769771Z","level":"ERROR","duration":"151.461µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.667818631Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"13757a74-d1e7-4306-9f59-3c6a1c466e1d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.696653485Z","level":"ERROR","duration":"170.482µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.696793846Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"24ac43a4-6a37-4ec2-abb6-6424cbc27954","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.724714331Z","level":"ERROR","duration":"157.502µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.724773991Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c77e9796-b903-46f3-bbb0-e934080bd21f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.795595533Z","level":"ERROR","duration":"155.201µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.795652454Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"d79ff5bb-eb42-49e9-af28-ec1a4183bddc","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.813577851Z","level":"ERROR","duration":"150.141µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.813717832Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"aa8a3383-c442-4c8d-ad30-0bba7f3d20dc","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.905622559Z","level":"ERROR","duration":"157.521µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.90567944Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2c11677f-0c15-4edb-923d-8ce2022f8546","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.91256074Z","level":"ERROR","duration":"151.701µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.912612761Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c7631fce-6c7c-4638-aaa5-b3d6a578357a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.919444601Z","level":"ERROR","duration":"144.602µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.919497301Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2f2c803c-ea91-49d5-82ba-fbe29d1261e1","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.994610781Z","level":"ERROR","duration":"272.863µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.994679201Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6d4de2d3-dba1-44ae-9aa8-fb7d08441835","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.997459866Z","level":"ERROR","duration":"137.642µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:35.997511046Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"10c1a81a-b7c3-4e19-a7d8-c42439ca0e3a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket32/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.013276344Z","level":"ERROR","duration":"152.561µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.013462386Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"28f8da13-7c88-4fd7-817c-7340d3d55e0a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.054579447Z","level":"ERROR","duration":"250.682µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.054721849Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"1839e27f-c962-4573-a7bb-b2b08b28eed4","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.101822782Z","level":"ERROR","duration":"196.502µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.101961143Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6c436665-6a35-4e13-9ced-ad06d9d9de65","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.114973437Z","level":"ERROR","duration":"153.141µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.115035018Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4adb8318-f0df-4842-a13b-df730fe768b3","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.177066263Z","level":"ERROR","duration":"167.942µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.177120903Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"46272035-f44d-4814-9a0d-c0e52d891803","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.198923815Z","level":"ERROR","duration":"5.003670527s","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"ServiceUnavailableThe service is unavailable. Please retry.\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.198987635Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"17ed4143-ad1b-49d5-95f8-110b36ddc395","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket23/","status_code":503,"duration_ms":5003,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.210632697Z","level":"ERROR","duration":"5.008819151s","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"ServiceUnavailableThe service is unavailable. Please retry.\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.210699278Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a62c7122-7aff-4620-8383-f8a82336f523","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket25/","status_code":503,"duration_ms":5008,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.234951011Z","level":"ERROR","duration":"183.082µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.235069752Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6610a7ff-18d3-4fb3-9fce-2c6ca8d27820","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket25/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.258136074Z","level":"ERROR","duration":"180.921µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.258278556Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"b26475f9-ae33-42b4-ab03-5892d7bad51c","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.258845401Z","level":"ERROR","duration":"92.541µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.258872101Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"bf113fc9-b43a-4954-92ee-30e94b798ce2","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket23/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.262456893Z","level":"ERROR","duration":"113.761µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.262562354Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"526feb7c-16fc-4eeb-a90b-ed256e659dbd","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.283292855Z","level":"ERROR","duration":"5.009076934s","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"ServiceUnavailableThe service is unavailable. Please retry.\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.283358756Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"cc8f84b7-0e5c-459a-a9f9-2a1ded637615","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket17/","status_code":503,"duration_ms":5009,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.298677171Z","level":"ERROR","duration":"105.661µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.299189355Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4bb1fae4-02fa-469d-92da-065a59d73eb1","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.349070673Z","level":"ERROR","duration":"167.862µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.349193754Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2f43c298-3b92-4c13-9e0a-a233bce2bcc0","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket25/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.353985156Z","level":"ERROR","duration":"149.121µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.354040457Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ff0490a0-9f6a-47e6-adf4-53f3e4540a36","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket15/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.370969345Z","level":"ERROR","duration":"159.501µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.371029786Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"442d5fcb-777d-4a0e-a043-1758a84ab79b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.406016053Z","level":"ERROR","duration":"161.882µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.406143694Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0c068583-de0e-44d7-ac71-269ef83ec856","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket34/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.406673299Z","level":"ERROR","duration":"94.421µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.406696539Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6085bbc4-7b61-4d7f-8c7b-53cd36b99032","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket23/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.419473231Z","level":"ERROR","duration":"145.141µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.419525652Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e2ea6ef4-9372-4982-8d1b-53473897e5e3","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.475584024Z","level":"ERROR","duration":"156.682µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.475726105Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"62687ae0-003a-411f-ab01-57b8fcc849e7","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket15/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.515802117Z","level":"ERROR","duration":"179.942µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.515942858Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8edd78f2-8253-45d4-a5d8-bf62d73e7eb4","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.516329402Z","level":"ERROR","duration":"73.981µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.516346962Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e85344fb-73fa-4a64-9cf7-b212af7cb631","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.517964356Z","level":"ERROR","duration":"120.881µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.518007916Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"cc691be1-bab9-42fb-a9f8-d61800279e93","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.62324238Z","level":"ERROR","duration":"230.022µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.623315641Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6ba7438c-9647-4b78-9b2b-931f94e9f675","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.623676504Z","level":"ERROR","duration":"92.601µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.623696444Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"51cb9a32-408f-45ec-aeb1-24f43ca8dcf8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.692735911Z","level":"ERROR","duration":"155.542µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.692873252Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ba716ca5-dffe-4c10-bffa-9adbd3b46190","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket15/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.693376656Z","level":"ERROR","duration":"86.3µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.693399176Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8909b0b1-5c1c-4f90-9b38-655f89fd413d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.699154687Z","level":"ERROR","duration":"140.321µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.699202467Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a540b265-9b2d-4916-9898-c4a38a36b82b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.69955323Z","level":"ERROR","duration":"58.46µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.699579991Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ac0eca1a-a452-4dc5-a355-41b81ec2244e","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.733408268Z","level":"ERROR","duration":"152.481µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.733465368Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f91aff49-da42-4fff-a8a4-b512a42cb7c2","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.738282091Z","level":"ERROR","duration":"145.361µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.738436952Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8bb9d39b-8dfb-44b1-93ba-8107772a3e10","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket32/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.759354615Z","level":"ERROR","duration":"146.601µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.759474777Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4cdcdedf-96de-4594-9e4d-8be2a6d263ba","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket23/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.767165544Z","level":"ERROR","duration":"114.841µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.767264365Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"75e4db69-49b4-4650-8cab-2f0fa1c49390","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.780947845Z","level":"ERROR","duration":"130.841µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.780985525Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"fcdd8468-6edf-4fce-b572-6968b585bf76","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket25/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.845781555Z","level":"ERROR","duration":"143.021µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.845905876Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2a0fd6eb-8996-43bb-8289-031eed5146c5","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.886849975Z","level":"ERROR","duration":"154.721µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.886954156Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c15c43cb-f140-451d-8442-968a0d368d8d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket32/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.901821206Z","level":"ERROR","duration":"148.421µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.901872507Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ad87ba5b-0aa8-4a2b-892e-d29382aa5f08","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.937329598Z","level":"ERROR","message":"Scanner data usage save failed","event":"scanner_persist_state","component":"scanner","subsystem":"runtime","path":"buckets/.usage.v2.json","state":"save_failed","error":"Lock error: Lock acquisition timeout for resource '.rustfs.sys/buckets/.usage.v2.json@latest' after 5s","target":"rustfs::scanner","filename":"crates/scanner/src/scanner.rs","line_number":3872,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:36.93752766Z","level":"ERROR","message":"Scanner cycle completed without a durable data usage snapshot","event":"scanner_persist_state","component":"scanner","subsystem":"runtime","cycle":5,"state":"usage_not_durable","target":"rustfs::scanner","filename":"crates/scanner/src/scanner.rs","line_number":2763,"threadName":"rustfs-worker","threadId":"ThreadId(368)"} niks3-go-unit-tests> --- PASS: TestReadProxyNarStreaming (7.13s) niks3-go-unit-tests> === CONT TestClientWithDependencies niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.02406968Z","level":"ERROR","duration":"164.362µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.024138Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4c01991f-2bc3-4b9d-99cb-2ccc4cfd7c0f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket35/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.03204353Z","level":"ERROR","duration":"190.582µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.032534454Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"112ac107-843b-40b3-bc3e-05481e1b2fd9","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> 2026-08-27 09:40:37.062 UTC [662] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:37.062 UTC [662] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20241026095416_initial_model.sql (7.83ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20251210153512_drop_unused_gin_index.sql (1.67ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20251218171726_add_pins.sql (3.69ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20260628120000_add_object_size_and_stats.sql (2.73ms) niks3-go-unit-tests> 2026/08/27 09:40:37 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:37 OK 1_commit_pending_closure.sql (1.91ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 2_object_stats_trigger.sql (905.23µs) niks3-go-unit-tests> 2026/08/27 09:40:37 goose: up to current file version: 2 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.088166763Z","level":"ERROR","duration":"622.886µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.088232683Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"64ed84f4-a609-48d9-a257-4377a2be91bb","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket36/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.141257689Z","level":"ERROR","duration":"169.782µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.141321949Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ae3a15bd-312a-4817-8c5e-942946108a45","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket32/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.156393202Z","level":"ERROR","duration":"182.902µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.156710244Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"cc0885b8-5885-46df-91f8-74a9e78d5c96","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket17/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> --- PASS: TestMetricsInventory (7.35s) niks3-go-unit-tests> === CONT TestClientMultipleUploads niks3-go-unit-tests> --- PASS: TestReadProxyInvalidPath (6.17s) niks3-go-unit-tests> === CONT TestCacheConfigHandlerMaxNarSize niks3-go-unit-tests> --- PASS: TestCacheConfigHandlerMaxNarSize (0.00s) niks3-go-unit-tests> === CONT TestCacheConfigHandler niks3-go-unit-tests> === RUN TestCacheConfigHandler/full_config,_no_issuer niks3-go-unit-tests> === PAUSE TestCacheConfigHandler/full_config,_no_issuer niks3-go-unit-tests> === RUN TestCacheConfigHandler/no_cache_url_configured niks3-go-unit-tests> === PAUSE TestCacheConfigHandler/no_cache_url_configured niks3-go-unit-tests> === RUN TestCacheConfigHandler/no_signing_keys niks3-go-unit-tests> === PAUSE TestCacheConfigHandler/no_signing_keys niks3-go-unit-tests> === RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator niks3-go-unit-tests> === PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator niks3-go-unit-tests> === CONT TestClientErrorHandling niks3-go-unit-tests> === RUN TestClientErrorHandling/InvalidStorePath niks3-go-unit-tests> === PAUSE TestClientErrorHandling/InvalidStorePath niks3-go-unit-tests> === RUN TestClientErrorHandling/InvalidAuthToken niks3-go-unit-tests> === PAUSE TestClientErrorHandling/InvalidAuthToken niks3-go-unit-tests> === RUN TestClientErrorHandling/ServerNotAvailable niks3-go-unit-tests> === PAUSE TestClientErrorHandling/ServerNotAvailable niks3-go-unit-tests> === CONT TestClientCADerivations niks3-go-unit-tests> 2026-08-27 09:40:37.263 UTC [667] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:37.263 UTC [667] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20241026095416_initial_model.sql (18.36ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20251210153512_drop_unused_gin_index.sql (1.76ms) niks3-go-unit-tests> 2026-08-27 09:40:37.310 UTC [668] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:37.310 UTC [668] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20251218171726_add_pins.sql (15.68ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20260628120000_add_object_size_and_stats.sql (10.18ms) niks3-go-unit-tests> 2026/08/27 09:40:37 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:37 OK 1_commit_pending_closure.sql (4.78ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 2_object_stats_trigger.sql (1.86ms) niks3-go-unit-tests> 2026/08/27 09:40:37 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Created nix-cache-info in bucket bucket=bucket36 niks3-go-unit-tests> --- PASS: TestService_Rustfstest (7.49s) niks3-go-unit-tests> === CONT TestCacheStatsHandler niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20241026095416_initial_model.sql (29.16ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20251210153512_drop_unused_gin_index.sql (3.64ms) niks3-go-unit-tests> --- PASS: TestReadProxyNarinfoAlreadyDecompressed (7.58s) niks3-go-unit-tests> === CONT TestService_ReadAuthMiddleware niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20251218171726_add_pins.sql (76.34ms) niks3-go-unit-tests> --- PASS: TestService_healthCheckHandler (7.09s) niks3-go-unit-tests> === CONT TestService_AuthMiddleware_OIDC niks3-go-unit-tests> 2026/08/27 09:40:37 INFO OIDC provider initialized name=test niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20260628120000_add_object_size_and_stats.sql (18.81ms) niks3-go-unit-tests> 2026/08/27 09:40:37 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:37 OK 1_commit_pending_closure.sql (4.37ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 2_object_stats_trigger.sql (5.11ms) niks3-go-unit-tests> 2026/08/27 09:40:37 goose: up to current file version: 2 niks3-go-unit-tests> === NAME TestClientWithDependencies niks3-go-unit-tests> client_integration_test.go:593: Built derivation: /build/TestClientWithDependencies154482307/001/store/41k02xvhb5h8xpj6jnb2lpxi1fa78702-test-script niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Created nix-cache-info in bucket bucket=bucket37 niks3-go-unit-tests> 2026-08-27 09:40:37.480 UTC [710] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:37.480 UTC [710] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Created nix-cache-info in bucket bucket=bucket25 niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20241026095416_initial_model.sql (39.34ms) niks3-go-unit-tests> === NAME TestNARDeduplicationMetadataUploadBug niks3-go-unit-tests> metadata_upload_test.go:48: First store path: /build/TestNARDeduplicationMetadataUploadBug2938406037/001/store/qsdgmb38d92vzm9ipxy0gzhlvklakyhq-file1.txt niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Aborted multipart uploads count=0 niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20251210153512_drop_unused_gin_index.sql (17.99ms) niks3-go-unit-tests> === NAME TestClientWithDependencies niks3-go-unit-tests> client_integration_test.go:595: Found 1 dependencies (including self) niks3-go-unit-tests> 2026-08-27 09:40:37.576 UTC [748] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:37.576 UTC [748] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:37 WARN Force mode enabled - objects will be deleted immediately without grace period niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20251218171726_add_pins.sql (25.44ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20260628120000_add_object_size_and_stats.sql (18.27ms) niks3-go-unit-tests> 2026/08/27 09:40:37 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026-08-27 09:40:37.620 UTC [750] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:37.620 UTC [750] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:37 OK 1_commit_pending_closure.sql (11.78ms) niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=0 niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Vacuumed table table=pending_closures niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Vacuumed table table=pending_objects niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Vacuumed table table=multipart_uploads niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.640909536Z","level":"ERROR","duration":"169.221µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.640993917Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"36d545d2-917d-48fb-adf6-1bf655a94f9f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Vacuumed table table=closures niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Vacuumed table table=objects niks3-go-unit-tests> --- PASS: TestGCMetrics (6.38s) niks3-go-unit-tests> === CONT TestService_AuthMiddleware_MTLSBoundSubjects niks3-go-unit-tests> 2026/08/27 09:40:37 OK 2_object_stats_trigger.sql (26.37ms) niks3-go-unit-tests> 2026/08/27 09:40:37 goose: up to current file version: 2 niks3-go-unit-tests> === NAME TestClientMultipleUploads niks3-go-unit-tests> client_integration_test.go:338: Created store path 0: /build/TestClientMultipleUploads2416235048/001/store/4n5icqg8z6wmwnyaj13kqrsmdjiwpp40-test-file-0.txt niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20241026095416_initial_model.sql (27.15ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20251210153512_drop_unused_gin_index.sql (6.03ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.645731959Z","level":"ERROR","duration":"164.362µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.645783759Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"65bdc445-ee00-4297-bab5-bd0f249c8d6b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.646336704Z","level":"ERROR","duration":"94.761µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.646362364Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4460456f-f107-49d7-a7a6-58788e6eefb7","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.660244566Z","level":"ERROR","duration":"539.865µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.660304647Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"17a896a9-3fed-41a3-8113-fc6030c9bb40","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket39/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.667327628Z","level":"ERROR","duration":"171.061µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.667375589Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"67174812-a6f1-448d-b6fc-034c24984881","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket39/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20241026095416_initial_model.sql (72.1ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20251218171726_add_pins.sql (6.37ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20251210153512_drop_unused_gin_index.sql (14.53ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20260628120000_add_object_size_and_stats.sql (21.02ms) niks3-go-unit-tests> 2026/08/27 09:40:37 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:37 OK 1_commit_pending_closure.sql (9.19ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20251218171726_add_pins.sql (20.92ms) niks3-go-unit-tests> 2026/08/27 09:40:37 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:37 OK 2_object_stats_trigger.sql (4.91ms) niks3-go-unit-tests> 2026/08/27 09:40:37 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20260628120000_add_object_size_and_stats.sql (16.05ms) niks3-go-unit-tests> 2026/08/27 09:40:37 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.717371507Z","level":"ERROR","duration":"694.646µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.722774675Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"3a3c4f45-c406-473b-8470-cd22e96bf827","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":6,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> 2026/08/27 09:40:37 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n" niks3-go-unit-tests> client_integration_test.go:338: Created store path 1: /build/TestClientMultipleUploads2416235048/001/store/10zb25c9c60qilk1mwswardjd7hplz2k-test-file-1.txt niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Uploading 1 paths to 127.0.0.1 (0 already cached) niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Uploading 41k02xvhb5h8xpj6jnb2lpxi1fa78702-test-script (136B) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.763627554Z","level":"ERROR","duration":"194.482µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.763691374Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"1644b0f0-6d31-4f46-9dbe-bf9d3772efeb","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> 2026/08/27 09:40:37 OK 1_commit_pending_closure.sql (41.21ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.803178061Z","level":"ERROR","duration":"190.702µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.803241682Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"d9c7a393-fa66-487a-8526-ec17829305ff","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.81218934Z","level":"ERROR","duration":"165.001µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.812236681Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2f099b08-b03e-4a02-90b3-31176731c305","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> 2026/08/27 09:40:37 WARN Failed to register uploaded object key=log/bjbnm291kzaiwcy90waikk284nkq8ha7-test-script.drv error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:37 OK 2_object_stats_trigger.sql (89.65ms) niks3-go-unit-tests> 2026/08/27 09:40:37 goose: up to current file version: 2 niks3-go-unit-tests> client_integration_test.go:338: Created store path 2: /build/TestClientMultipleUploads2416235048/001/store/yn8dx9xm0g02bwvcm2rzkmbg0xckmc9l-test-file-2.txt niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.871754563Z","level":"ERROR","duration":"150.561µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.871815804Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2a5b352d-2f35-45b0-988b-e30dffe0fb0a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.883140223Z","level":"ERROR","duration":"173.621µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.883304125Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ac3dc2b8-3e03-4854-b255-98162a6b8780","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.89638438Z","level":"ERROR","duration":"167.962µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.89644512Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"7394dc41-14cf-48d3-a01e-d191c9fda5df","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.921669381Z","level":"ERROR","duration":"2.642723ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:37.921734122Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"75c31ecd-4f5d-450f-b874-a86ae19a7544","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":2,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> 2026-08-27 09:40:37.926 UTC [891] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:37.926 UTC [891] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Uploading 1 paths to 127.0.0.1 (0 already cached) niks3-go-unit-tests> 2026/08/27 09:40:37 INFO Uploading qsdgmb38d92vzm9ipxy0gzhlvklakyhq-file1.txt (160B) niks3-go-unit-tests> 2026/08/27 09:40:37 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:37 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20241026095416_initial_model.sql (17.18ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20251210153512_drop_unused_gin_index.sql (2ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20251218171726_add_pins.sql (3.44ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 20260628120000_add_object_size_and_stats.sql (4.29ms) niks3-go-unit-tests> 2026/08/27 09:40:37 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:37 OK 1_commit_pending_closure.sql (2.74ms) niks3-go-unit-tests> 2026/08/27 09:40:37 OK 2_object_stats_trigger.sql (1.82ms) niks3-go-unit-tests> 2026/08/27 09:40:37 goose: up to current file version: 2 niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Uploading 3 paths to 127.0.0.1 (0 already cached) niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Uploading 4n5icqg8z6wmwnyaj13kqrsmdjiwpp40-test-file-0.txt (160B) niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Uploading 10zb25c9c60qilk1mwswardjd7hplz2k-test-file-1.txt (160B) niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Uploading yn8dx9xm0g02bwvcm2rzkmbg0xckmc9l-test-file-2.txt (160B) niks3-go-unit-tests> 2026/08/27 09:40:38 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:38 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:38 WARN Failed to register uploaded object key=10zb25c9c60qilk1mwswardjd7hplz2k.ls error="server returned 404: 404 page not found\n" niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.071183655Z","level":"ERROR","duration":"157.421µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.071241775Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f5c9d4b9-c0c4-4f9f-a989-ae52ad220ea8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket39/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.141459652Z","level":"ERROR","duration":"166.442µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.141522772Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"39c12260-34ff-4346-b494-d20bd777b3c4","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.155075791Z","level":"ERROR","duration":"161.861µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.155127952Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"454cffa6-ca0b-46f9-9e6a-6a35bb33405b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket17/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.366657629Z","level":"ERROR","duration":"293.402µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.36672043Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"7bfa0f55-af18-44ae-874f-032e3b0f0d37","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.367240034Z","level":"ERROR","duration":"101.461µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.367270955Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f042c3a5-79c9-484a-b4bf-60161fd6ab6f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.379496522Z","level":"ERROR","duration":"166.042µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.39636071Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f6b6a109-3e1c-43cd-8b20-844f709aaad7","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket17/","status_code":503,"duration_ms":17,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.397052536Z","level":"ERROR","duration":"155.681µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.397095756Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"3c20c45d-5562-4011-b274-c16bee44416d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.42712922Z","level":"ERROR","duration":"171.621µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.427185781Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"fa0eadf7-8e0d-440c-93b9-9b846e0db226","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.435014909Z","level":"ERROR","duration":"161.701µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.43507221Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"682a600c-3b21-40d9-8d47-5d6c373cad76","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> 2026/08/27 09:40:38 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:38 WARN Failed to register uploaded object key=4n5icqg8z6wmwnyaj13kqrsmdjiwpp40.ls error="server returned 404: 404 page not found\n" niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.591191281Z","level":"ERROR","duration":"159.702µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.591257041Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0b750d45-8bdc-4507-a421-3bc0985d3eb5","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.591623625Z","level":"ERROR","duration":"98.741µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.591648805Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"720bfd9b-896c-4398-b253-43cf6d05a2e1","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.733098107Z","level":"ERROR","duration":"357.303µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(378)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.733277309Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e1967a65-403f-40cf-b66c-8f820f68368f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":90,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(378)"} niks3-go-unit-tests> 2026/08/27 09:40:38 WARN Failed to register uploaded object key=yn8dx9xm0g02bwvcm2rzkmbg0xckmc9l.ls error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign niks3-go-unit-tests> 2026/08/27 09:40:38 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:38 WARN Failed to register uploaded object key=qsdgmb38d92vzm9ipxy0gzhlvklakyhq.ls error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Signed narinfos id=1 count=1 niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Uploading 1 narinfos niks3-go-unit-tests> 2026/08/27 09:40:38 WARN Failed to register uploaded object key=qsdgmb38d92vzm9ipxy0gzhlvklakyhq.narinfo error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:38 WARN Failed to register uploaded object key=41k02xvhb5h8xpj6jnb2lpxi1fa78702.ls error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Signed narinfos id=1 count=1 niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Uploading 1 narinfos niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.779797037Z","level":"ERROR","duration":"915.959583ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(375)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.779889638Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"bfe7d0d8-cbec-4dae-82b3-5810174de18e","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket41/","status_code":503,"duration_ms":916,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(375)"} niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Signed narinfos id=2 count=1 niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Signed narinfos id=3 count=1 niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Signed narinfos id=1 count=1 niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Uploading 3 narinfos niks3-go-unit-tests> 2026/08/27 09:40:38 WARN Failed to register uploaded object key=41k02xvhb5h8xpj6jnb2lpxi1fa78702.narinfo error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete niks3-go-unit-tests> 2026/08/27 09:40:38 WARN Failed to register uploaded object key=10zb25c9c60qilk1mwswardjd7hplz2k.narinfo error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Completed upload id=1 niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Upload complete. (1.169s) niks3-go-unit-tests> 2026/08/27 09:40:38 WARN Failed to register uploaded object key=4n5icqg8z6wmwnyaj13kqrsmdjiwpp40.narinfo error="server returned 404: 404 page not found\n" niks3-go-unit-tests> === NAME TestNARDeduplicationMetadataUploadBug niks3-go-unit-tests> metadata_upload_test.go:54: Retrieved narinfo from S3: niks3-go-unit-tests> StorePath: /build/TestNARDeduplicationMetadataUploadBug2938406037/001/store/qsdgmb38d92vzm9ipxy0gzhlvklakyhq-file1.txt niks3-go-unit-tests> URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst niks3-go-unit-tests> Compression: zstd niks3-go-unit-tests> NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf niks3-go-unit-tests> NarSize: 160 niks3-go-unit-tests> References: niks3-go-unit-tests> CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf niks3-go-unit-tests> 2026/08/27 09:40:38 WARN Failed to register uploaded object key=yn8dx9xm0g02bwvcm2rzkmbg0xckmc9l.narinfo error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received complete multipart upload request method=POST path=/api/multipart/complete niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.803124122Z","level":"ERROR","duration":"432.244µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.803175543Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"37956a70-ab3c-4e21-a787-c61ff4eac71a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes) niks3-go-unit-tests> metadata_upload_test.go:55: Decompressed .ls content (64 bytes): niks3-go-unit-tests> {"version":1,"root":{"type":"regular","size":44,"narOffset":96}} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.810016743Z","level":"ERROR","duration":"827.999432ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(380)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.810087063Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f04ede35-69ad-4e29-bdd6-6a06a8ea1e88","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket42/","status_code":503,"duration_ms":828,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(380)"} niks3-go-unit-tests> 2026/08/27 09:40:38 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MmJjZDQzOTUtNDQ2Yi00YzUxLWJiNjQtZDExYTRhM2YxYjNjLjY4N2ZmY2ViLTAxMGUtNDdhOC05MjM2LWI2YjQ3NDJkODQzYXgxNzg3ODIzNjM3NTkwNzQ5MDU2 niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Completed upload id=1 niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Upload complete. (1.17s) niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Completed upload id=1 niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete niks3-go-unit-tests> === NAME TestClientWithDependencies niks3-go-unit-tests> client_integration_test.go:597: Skipping nix copy test - isolated store (/build/TestClientWithDependencies154482307/001/store) requires matching store prefix niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Completed upload id=2 niks3-go-unit-tests> --- PASS: TestClientWithDependencies (1.85s) niks3-go-unit-tests> === CONT TestService_AuthMiddleware_MTLSProxyHeader niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MmJjZDQzOTUtNDQ2Yi00YzUxLWJiNjQtZDExYTRhM2YxYjNjLjY4N2ZmY2ViLTAxMGUtNDdhOC05MjM2LWI2YjQ3NDJkODQzYXgxNzg3ODIzNjM3NTkwNzQ5MDU2 parts=1 niks3-go-unit-tests> --- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (9.05s) niks3-go-unit-tests> === CONT TestProxyWriteTimeout/unknown_size niks3-go-unit-tests> --- PASS: TestProxyWriteTimeout (0.00s) niks3-go-unit-tests> --- PASS: TestProxyWriteTimeout/narinfo (0.00s) niks3-go-unit-tests> --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s) niks3-go-unit-tests> --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s) niks3-go-unit-tests> --- PASS: TestProxyWriteTimeout/unknown_size (0.00s) niks3-go-unit-tests> === CONT TestReadProxy404 niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Completed upload id=3 niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Upload complete. (921ms) niks3-go-unit-tests> === NAME TestClientMultipleUploads niks3-go-unit-tests> client_integration_test.go:349: Uploaded 3 paths in 982.013703ms niks3-go-unit-tests> --- PASS: TestClientMultipleUploads (1.67s) niks3-go-unit-tests> === CONT TestServerTLSConfig/no_client_CA niks3-go-unit-tests> === CONT TestServerTLSConfig/missing_CA_file niks3-go-unit-tests> === CONT TestServerTLSConfig/not_a_PEM_file niks3-go-unit-tests> --- PASS: TestServerTLSConfig (0.00s) niks3-go-unit-tests> --- PASS: TestServerTLSConfig/no_client_CA (0.00s) niks3-go-unit-tests> --- PASS: TestServerTLSConfig/missing_CA_file (0.00s) niks3-go-unit-tests> --- PASS: TestServerTLSConfig/not_a_PEM_file (0.00s) niks3-go-unit-tests> === CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.86434796Z","level":"ERROR","duration":"162.942µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.86440958Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e01f5479-a0d9-41ad-8c52-e80bb47acdd1","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket39/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received uploads request method=POST path=/ niks3-go-unit-tests> === CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received complete multipart upload request method=POST path=/ niks3-go-unit-tests> === CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received request for more parts method=POST path=/ niks3-go-unit-tests> === CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received uploads request method=POST path=/ niks3-go-unit-tests> --- PASS: TestUploadHandlersRejectInvalidKeys (0.05s) niks3-go-unit-tests> --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s) niks3-go-unit-tests> --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s) niks3-go-unit-tests> --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s) niks3-go-unit-tests> --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s) niks3-go-unit-tests> === CONT TestIsValidUploadKey/narinfo niks3-go-unit-tests> === CONT TestIsValidUploadKey/realisation_plus_in_output niks3-go-unit-tests> === CONT TestIsValidUploadKey/realisation niks3-go-unit-tests> === CONT TestIsValidUploadKey/build_log_equals niks3-go-unit-tests> === CONT TestIsValidUploadKey/build_log_question_mark niks3-go-unit-tests> === CONT TestIsValidUploadKey/build_log_plus_in_name niks3-go-unit-tests> === CONT TestIsValidUploadKey/build_log_home-manager_file niks3-go-unit-tests> === CONT TestIsValidUploadKey/build_log niks3-go-unit-tests> === CONT TestIsValidUploadKey/listing niks3-go-unit-tests> === CONT TestIsValidUploadKey/nar_plain niks3-go-unit-tests> === CONT TestIsValidUploadKey/nar_xz niks3-go-unit-tests> === CONT TestIsValidUploadKey/nar_zst niks3-go-unit-tests> === CONT TestParseSingleRange/none niks3-go-unit-tests> === CONT TestIsValidUploadKey/nix-cache-info niks3-go-unit-tests> === CONT TestIsValidCachePath/narinfo niks3-go-unit-tests> === CONT TestParseSingleRange/start_far_past_EOF niks3-go-unit-tests> === CONT TestParseSingleRange/start_past_EOF niks3-go-unit-tests> === CONT TestParseSingleRange/single_byte niks3-go-unit-tests> === CONT TestParseSingleRange/suffix_exceeds_size niks3-go-unit-tests> === CONT TestParseSingleRange/suffix niks3-go-unit-tests> === CONT TestParseSingleRange/end_clamped_to_size niks3-go-unit-tests> === CONT TestParseSingleRange/open-ended niks3-go-unit-tests> === CONT TestParseSingleRange/closed niks3-go-unit-tests> === CONT TestParseSingleRange/malformed_end_before_start niks3-go-unit-tests> === CONT TestParseSingleRange/malformed_both_empty niks3-go-unit-tests> === CONT TestParseSingleRange/malformed_no_dash niks3-go-unit-tests> === CONT TestParseSingleRange/multi-range_ignored niks3-go-unit-tests> === CONT TestParseSingleRange/unknown_unit niks3-go-unit-tests> --- PASS: TestParseSingleRange (0.05s) niks3-go-unit-tests> --- PASS: TestParseSingleRange/none (0.00s) niks3-go-unit-tests> --- PASS: TestParseSingleRange/start_far_past_EOF (0.00s) niks3-go-unit-tests> --- PASS: TestParseSingleRange/start_past_EOF (0.00s) niks3-go-unit-tests> --- PASS: TestParseSingleRange/single_byte (0.00s) niks3-go-unit-tests> --- PASS: TestParseSingleRange/suffix_exceeds_size (0.00s) niks3-go-unit-tests> --- PASS: TestParseSingleRange/suffix (0.00s) niks3-go-unit-tests> --- PASS: TestParseSingleRange/end_clamped_to_size (0.00s) niks3-go-unit-tests> --- PASS: TestParseSingleRange/open-ended (0.00s) niks3-go-unit-tests> --- PASS: TestParseSingleRange/closed (0.00s) niks3-go-unit-tests> --- PASS: TestParseSingleRange/malformed_end_before_start (0.00s) niks3-go-unit-tests> --- PASS: TestParseSingleRange/malformed_both_empty (0.00s) niks3-go-unit-tests> --- PASS: TestParseSingleRange/malformed_no_dash (0.00s) niks3-go-unit-tests> --- PASS: TestParseSingleRange/multi-range_ignored (0.00s) niks3-go-unit-tests> --- PASS: TestParseSingleRange/unknown_unit (0.00s) niks3-go-unit-tests> === CONT TestIsValidCachePath/random_path niks3-go-unit-tests> === CONT TestIsValidUploadKey/unknown_type niks3-go-unit-tests> === CONT TestIsValidUploadKey/empty_key niks3-go-unit-tests> === CONT TestIsValidUploadKey/absolute niks3-go-unit-tests> === CONT TestIsValidUploadKey/traversal_nar niks3-go-unit-tests> === CONT TestIsValidUploadKey/traversal niks3-go-unit-tests> === CONT TestIsValidUploadKey/listing_key,_narinfo_type niks3-go-unit-tests> === CONT TestIsValidUploadKey/nar_key,_narinfo_type niks3-go-unit-tests> === CONT TestIsValidUploadKey/narinfo_key,_nar_type niks3-go-unit-tests> === CONT TestIsValidUploadKey/index.html niks3-go-unit-tests> --- PASS: TestIsValidUploadKey (0.06s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/narinfo (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/realisation_plus_in_output (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/realisation (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/build_log_equals (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/build_log_question_mark (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/build_log_plus_in_name (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/build_log_home-manager_file (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/build_log (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/listing (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/nar_plain (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/nar_xz (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/nar_zst (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/nix-cache-info (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/unknown_type (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/empty_key (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/absolute (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/traversal_nar (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/traversal (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/listing_key,_narinfo_type (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/nar_key,_narinfo_type (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/narinfo_key,_nar_type (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidUploadKey/index.html (0.00s) niks3-go-unit-tests> === CONT TestIsValidCachePath/realisation niks3-go-unit-tests> === CONT TestIsValidCachePath/invalid_char_u niks3-go-unit-tests> === CONT TestIsValidCachePath/invalid_char_e niks3-go-unit-tests> === CONT TestIsValidCachePath/traversal_in_middle niks3-go-unit-tests> === CONT TestIsValidCachePath/traversal_parent niks3-go-unit-tests> === CONT TestIsValidCachePath/index.html niks3-go-unit-tests> === CONT TestIsValidCachePath/nix-cache-info niks3-go-unit-tests> === CONT TestIsValidCachePath/nar_bz2 niks3-go-unit-tests> === CONT TestIsValidCachePath/log niks3-go-unit-tests> === CONT TestIsValidCachePath/ls niks3-go-unit-tests> === CONT TestIsValidCachePath/nar_uncompressed niks3-go-unit-tests> === CONT TestIsValidCachePath/wrong_extension niks3-go-unit-tests> === CONT TestIsValidCachePath/short_hash niks3-go-unit-tests> === CONT TestIsValidCachePath/leading_slash niks3-go-unit-tests> === CONT TestIsValidCachePath/empty niks3-go-unit-tests> === CONT TestIsValidCachePath/nar_zst niks3-go-unit-tests> === CONT TestIsValidCachePath/nar_xz niks3-go-unit-tests> === CONT TestIsValidCachePath/narinfo_all_nix_base32_chars niks3-go-unit-tests> --- PASS: TestIsValidCachePath (0.05s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/narinfo (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/random_path (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/realisation (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/invalid_char_u (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/invalid_char_e (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/traversal_in_middle (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/traversal_parent (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/index.html (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/nix-cache-info (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/nar_bz2 (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/log (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/ls (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/nar_uncompressed (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/wrong_extension (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/short_hash (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/leading_slash (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/empty (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/nar_zst (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/nar_xz (0.00s) niks3-go-unit-tests> --- PASS: TestIsValidCachePath/narinfo_all_nix_base32_chars (0.00s) niks3-go-unit-tests> === CONT TestUploadHandlersRejectOversizedBody/create_pending_closure niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received uploads request method=POST path=/ niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.89398022Z","level":"ERROR","duration":"181.102µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.89404772Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a4c55e12-7b8b-4f7e-ba99-b6bf6cd7a338","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket42/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> 2026-08-27 09:40:38.897 UTC [956] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:38.897 UTC [956] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:38 INFO Received complete multipart upload request method=POST path=/api/multipart/complete niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.901501866Z","level":"ERROR","duration":"163.082µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.901556546Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"fdc910e6-5bd5-43b3-bbd8-2b7a000bdf69","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket41/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(195)"} niks3-go-unit-tests> === NAME TestNARDeduplicationMetadataUploadBug niks3-go-unit-tests> metadata_upload_test.go:64: Second store path (same content): /build/TestNARDeduplicationMetadataUploadBug2938406037/001/store/km730175iqsbq36b5dhx6q861kfcpvmx-file2.txt niks3-go-unit-tests> 2026/08/27 09:40:38 OK 20241026095416_initial_model.sql (48.22ms) niks3-go-unit-tests> 2026/08/27 09:40:38 OK 20251210153512_drop_unused_gin_index.sql (4.53ms) niks3-go-unit-tests> 2026/08/27 09:40:38 OK 20251218171726_add_pins.sql (3.26ms) niks3-go-unit-tests> 2026-08-27 09:40:38.967 UTC [973] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:38.967 UTC [973] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:38 OK 20260628120000_add_object_size_and_stats.sql (4.66ms) niks3-go-unit-tests> 2026/08/27 09:40:38 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:38 OK 1_commit_pending_closure.sql (4.57ms) niks3-go-unit-tests> 2026/08/27 09:40:38 OK 2_object_stats_trigger.sql (2.86ms) niks3-go-unit-tests> 2026/08/27 09:40:38 goose: up to current file version: 2 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.986615593Z","level":"ERROR","duration":"2.101538ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:38.986689414Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"83c74900-0753-47f0-9fee-46461cc48236","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket43/","status_code":503,"duration_ms":2,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> 2026/08/27 09:40:38 OK 20241026095416_initial_model.sql (11.59ms) niks3-go-unit-tests> 2026/08/27 09:40:38 OK 20251210153512_drop_unused_gin_index.sql (2.15ms) niks3-go-unit-tests> 2026/08/27 09:40:38 OK 20251218171726_add_pins.sql (2.87ms) niks3-go-unit-tests> 2026/08/27 09:40:38 OK 20260628120000_add_object_size_and_stats.sql (4.09ms) niks3-go-unit-tests> 2026/08/27 09:40:38 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:39 OK 1_commit_pending_closure.sql (6.75ms) niks3-go-unit-tests> 2026/08/27 09:40:39 OK 2_object_stats_trigger.sql (1.12ms) niks3-go-unit-tests> 2026/08/27 09:40:39 goose: up to current file version: 2 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.00787678Z","level":"ERROR","duration":"380.483µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.007987801Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"3a717d91-48d2-40e5-b8f2-a01b01fa9a8c","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket44/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.050395073Z","level":"ERROR","duration":"234.942µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.050457834Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"1b26dadc-7000-4a49-a180-1969c7bb428b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> 2026/08/27 09:40:39 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n" niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.057838019Z","level":"ERROR","duration":"164.622µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.0580063Z","level":"ERROR","duration":"70.38µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.058047501Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"bcb6d2d6-30c0-49a8-bc82-b1e74f89737a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket42/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.058136301Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"7075e61a-d022-4bf4-b0bb-da789210d8d9","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket44/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.083956368Z","level":"ERROR","duration":"549.085µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.084015609Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2e2dba49-83fb-42e0-809a-49abf3cc49d6","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket43/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Uploading 0 paths to 127.0.0.1 (1 already cached) niks3-go-unit-tests> 2026/08/27 09:40:39 WARN Failed to register uploaded object key=km730175iqsbq36b5dhx6q861kfcpvmx.ls error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Signed narinfos id=2 count=1 niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Uploading 1 narinfos niks3-go-unit-tests> 2026/08/27 09:40:39 WARN Failed to register uploaded object key=km730175iqsbq36b5dhx6q861kfcpvmx.narinfo error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.129525868Z","level":"ERROR","duration":"175.941µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.129587249Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0f34a711-7cb6-49e2-9b16-8591b296de41","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Completed upload id=2 niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Upload complete. (119ms) niks3-go-unit-tests> metadata_upload_test.go:76: Retrieved narinfo from S3: niks3-go-unit-tests> StorePath: /build/TestNARDeduplicationMetadataUploadBug2938406037/001/store/km730175iqsbq36b5dhx6q861kfcpvmx-file2.txt niks3-go-unit-tests> URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst niks3-go-unit-tests> Compression: zstd niks3-go-unit-tests> NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf niks3-go-unit-tests> NarSize: 160 niks3-go-unit-tests> References: niks3-go-unit-tests> CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf niks3-go-unit-tests> metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes) niks3-go-unit-tests> metadata_upload_test.go:77: Decompressed .ls content (49 bytes): niks3-go-unit-tests> {"version":1,"root":{"type":"regular","size":44}} niks3-go-unit-tests> --- PASS: TestNARDeduplicationMetadataUploadBug (9.29s) niks3-go-unit-tests> === CONT TestUploadHandlersRejectOversizedBody/request_more_parts niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Received request for more parts method=POST path=/ niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.163599908Z","level":"ERROR","duration":"321.483µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.163660228Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6645377e-30ef-4f53-88dd-23cc1ee09713","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket17/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> === CONT TestUploadHandlersRejectOversizedBody/complete_multipart niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Received complete multipart upload request method=POST path=/ niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.218224987Z","level":"ERROR","duration":"1.13445ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.218289768Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"589a59a7-d6bc-4e0d-b5d1-59f2fe65e174","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":1,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> === CONT TestCacheConfigHandler/full_config,_no_issuer niks3-go-unit-tests> === CONT TestClientErrorHandling/InvalidStorePath niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.253021073Z","level":"ERROR","duration":"162.801µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.253085014Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"588a32ae-c0f1-462c-8c14-266004b37e74","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.269433277Z","level":"ERROR","duration":"187.822µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.269489058Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c57e8326-c8b9-4f4e-86e9-f4c498216120","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.294489977Z","level":"ERROR","duration":"193.502µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.294611598Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0edb94b5-e41f-4a63-bb19-084b011aead7","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket41/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> 2026-08-27 09:40:39.340 UTC [1029] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:39.340 UTC [1029] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.343098624Z","level":"ERROR","duration":"202.202µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.343157065Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"092cbad5-f827-4a35-83e5-12cd4056e90b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket39/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> 2026/08/27 09:40:39 OK 20241026095416_initial_model.sql (17.31ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.363396242Z","level":"ERROR","duration":"173.781µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.363460463Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4aeb2292-e01a-408e-936d-6b6c0c4279c4","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket43/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> 2026/08/27 09:40:39 OK 20251210153512_drop_unused_gin_index.sql (2.71ms) niks3-go-unit-tests> 2026/08/27 09:40:39 OK 20251218171726_add_pins.sql (2.64ms) niks3-go-unit-tests> 2026/08/27 09:40:39 OK 20260628120000_add_object_size_and_stats.sql (2.93ms) niks3-go-unit-tests> 2026/08/27 09:40:39 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:39 OK 1_commit_pending_closure.sql (1.81ms) niks3-go-unit-tests> 2026/08/27 09:40:39 OK 2_object_stats_trigger.sql (797.55µs) niks3-go-unit-tests> 2026/08/27 09:40:39 goose: up to current file version: 2 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.37796575Z","level":"ERROR","duration":"2.492322ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.378025551Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0faffdc9-4e8e-4a02-a8cf-c42e5ef06081","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket45/","status_code":503,"duration_ms":2,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.406718543Z","level":"ERROR","duration":"153.721µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.406803484Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"7a7172d5-58db-4846-a748-a3ce33899fed","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket44/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.415015416Z","level":"ERROR","duration":"155.122µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.415064236Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"72a1f710-029e-4246-a6be-5a25b74462ca","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket45/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.474815681Z","level":"ERROR","duration":"164.362µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.474872441Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"7ad55554-0612-4a24-a42f-4d5a01b20202","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket45/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(384)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.541207744Z","level":"ERROR","duration":"178.362µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(380)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.541261985Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"62be47ee-491f-4bcc-ae24-e8f76e54f795","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(380)"} niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Received complete multipart upload request method=POST path=/api/multipart/complete niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Received complete multipart upload request method=POST path=/api/multipart/complete niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=MmJjZDQzOTUtNDQ2Yi00YzUxLWJiNjQtZDExYTRhM2YxYjNjLjA5NjgxYzY5LTIzMjktNGYyYS1hZWU3LWM4NzUzNjE0NzI4MHgxNzg3ODIzNjM3MTc5OTEyMzQ4 parts=12 niks3-go-unit-tests> --- PASS: TestRedundantMultipartUpload (8.54s) niks3-go-unit-tests> === CONT TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator niks3-go-unit-tests> === CONT TestCacheConfigHandler/no_signing_keys niks3-go-unit-tests> === CONT TestCacheConfigHandler/no_cache_url_configured niks3-go-unit-tests> --- PASS: TestCacheConfigHandler (0.00s) niks3-go-unit-tests> --- PASS: TestCacheConfigHandler/full_config,_no_issuer (0.00s) niks3-go-unit-tests> --- PASS: TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator (0.00s) niks3-go-unit-tests> --- PASS: TestCacheConfigHandler/no_signing_keys (0.00s) niks3-go-unit-tests> --- PASS: TestCacheConfigHandler/no_cache_url_configured (0.00s) niks3-go-unit-tests> === CONT TestClientErrorHandling/ServerNotAvailable niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=MmJjZDQzOTUtNDQ2Yi00YzUxLWJiNjQtZDExYTRhM2YxYjNjLjYxN2E0ZTg1LTNlNWYtNGE1Yy1hYmE1LTE0MDg0NjE5YjYwY3gxNzg3ODIzNjM3MzIzNTcyNjEw parts=10 niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Completed upload id=1 niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:39 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo niks3-go-unit-tests> 2026/08/27 09:40:39 WARN Found objects in DB but missing from S3, will re-upload count=1 niks3-go-unit-tests> --- PASS: TestService_verifyS3Integrity (9.80s) niks3-go-unit-tests> === CONT TestClientErrorHandling/InvalidAuthToken niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.599127333Z","level":"ERROR","duration":"169.722µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.599189373Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"42e07a3d-6fd1-45dc-948d-ea66c36d32de","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.630179105Z","level":"ERROR","duration":"184.361µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.630242566Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"407b8eab-6a0e-45b2-a8ee-441d71b3e40a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.646262967Z","level":"ERROR","duration":"162.181µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.646326087Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"01bcee72-e535-46ee-bcc8-2d8b88642db3","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.646944573Z","level":"ERROR","duration":"111.041µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.646970653Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"364f4d9b-4435-4dd6-b38f-7584815b8157","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket41/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.65917164Z","level":"ERROR","duration":"163.102µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.659297641Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f8203fd9-47f7-4032-b8aa-6c8b484a8475","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket17/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.661649782Z","level":"ERROR","duration":"169.282µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.661700402Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6b13c96f-9951-41a7-b3c8-36bfd796a4c9","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket45/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.664123063Z","level":"ERROR","duration":"250.702µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.664160164Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e22de144-809d-4fd8-8f5b-ac3f3879b5c6","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> 2026-08-27 09:40:39.687 UTC [1068] ERROR: relation "goose_db_version" does not exist at character 36 niks3-go-unit-tests> 2026-08-27 09:40:39.687 UTC [1068] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC niks3-go-unit-tests> 2026/08/27 09:40:39 WARN Request failed, retrying attempt=1 max_attempts=6 backoff=100ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config niks3-go-unit-tests> 2026/08/27 09:40:39 OK 20241026095416_initial_model.sql (10.9ms) niks3-go-unit-tests> 2026/08/27 09:40:39 OK 20251210153512_drop_unused_gin_index.sql (3.66ms) niks3-go-unit-tests> 2026/08/27 09:40:39 OK 20251218171726_add_pins.sql (7.2ms) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.725818305Z","level":"ERROR","duration":"189.502µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.725883906Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2795e157-309c-4a60-85f2-c96261241d7a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket43/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> 2026/08/27 09:40:39 OK 20260628120000_add_object_size_and_stats.sql (3.51ms) niks3-go-unit-tests> 2026/08/27 09:40:39 goose: successfully migrated database to version: 20260628120000 niks3-go-unit-tests> 2026/08/27 09:40:39 OK 1_commit_pending_closure.sql (1.98ms) niks3-go-unit-tests> 2026/08/27 09:40:39 OK 2_object_stats_trigger.sql (957.95µs) niks3-go-unit-tests> 2026/08/27 09:40:39 goose: up to current file version: 2 niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.733908496Z","level":"ERROR","duration":"688.646µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.733972457Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"54365d10-5273-47da-94db-872ea4a51ad1","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket46/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> --- PASS: TestUploadHandlersRejectOversizedBody (0.54s) niks3-go-unit-tests> --- PASS: TestUploadHandlersRejectOversizedBody/request_more_parts (0.04s) niks3-go-unit-tests> --- PASS: TestUploadHandlersRejectOversizedBody/complete_multipart (0.05s) niks3-go-unit-tests> --- PASS: TestUploadHandlersRejectOversizedBody/create_pending_closure (0.88s) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.766391101Z","level":"ERROR","duration":"171.221µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.766559923Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"52b66ed7-ae29-424a-ae75-41730e170303","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.790921357Z","level":"ERROR","duration":"226.582µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.791145579Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2ae636e6-bd47-4208-9679-21ef6aace85d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket42/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.791471442Z","level":"ERROR","duration":"123.361µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.791498122Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a349d3d8-e614-4bef-887b-39c44e66b9e3","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> 2026/08/27 09:40:39 WARN Request failed, retrying attempt=2 max_attempts=6 backoff=186.804437ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.818095356Z","level":"ERROR","duration":"186.862µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.818296857Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"21d3aa77-f4dc-4b67-aa3e-98995b7ea8e9","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.83453562Z","level":"ERROR","duration":"199.582µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.834590941Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"cdce7348-c043-45d9-b027-2ec9c2f0cce9","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.852467257Z","level":"ERROR","duration":"161.561µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.852516918Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"1a9637a3-5ea6-497b-ad9f-a70b2cc239bb","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.886653898Z","level":"ERROR","duration":"203.762µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.886789819Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a6bf8915-2d2a-464d-ae18-e501e234b518","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket46/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.906747154Z","level":"ERROR","duration":"167.561µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.906818755Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0d45095b-5365-4b80-b863-aba920f6b2d6","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket46/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.940792573Z","level":"ERROR","duration":"168.501µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.941739722Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4d793932-d3cc-474e-885c-e5d75c5f9707","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket46/","status_code":503,"duration_ms":1,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.970789076Z","level":"ERROR","duration":"402.583µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:39.971115219Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"9d1390e9-803a-4842-867e-107a09ec0f96","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket42/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> 2026/08/27 09:40:39 WARN Request failed, retrying attempt=3 max_attempts=6 backoff=430.662214ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config niks3-go-unit-tests> === NAME TestService_AuthMiddleware niks3-go-unit-tests> auth_test.go:16: unexpected error: context deadline exceeded niks3-go-unit-tests> niks3-go-unit-tests> --- FAIL: TestService_AuthMiddleware (10.21s) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.055213378Z","level":"ERROR","duration":"175.002µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.055399339Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"49c78772-2c39-4dd3-b91d-f47ccb769401","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.132418976Z","level":"ERROR","duration":"175.142µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.132612958Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"5cdf3c41-c993-48a7-b372-66a6693179cf","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> === NAME TestService_cleanupPendingClosuresHandler niks3-go-unit-tests> uploads_test.go:108: unexpected error: Put "http://localhost:40529/bucket11/": context deadline exceeded niks3-go-unit-tests> niks3-go-unit-tests> --- FAIL: TestService_cleanupPendingClosuresHandler (10.35s) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.158931889Z","level":"ERROR","duration":"171.582µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.15905155Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6666ce9d-f7bd-4c90-83fb-12e0dddbae86","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket39/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.16702676Z","level":"ERROR","duration":"161.302µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.167183141Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"962f383b-df36-4f0c-9c46-2e829b498f76","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket44/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.234250751Z","level":"ERROR","duration":"163.241µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.234430912Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"30b16ac1-cdb5-481a-a8d4-0cb2b7709094","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket41/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.265485145Z","level":"ERROR","duration":"152.762µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.265647906Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"9e79ea5a-68d0-4aa0-b284-abf3a62f8cce","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.276835724Z","level":"ERROR","duration":"214.641µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.277007086Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"edb7afdd-1d91-465b-b3b3-3564fc1e81ae","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket42/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.297294764Z","level":"ERROR","duration":"164.241µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.297431525Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"550a5d56-df5b-49fa-b786-c93190674b33","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.36746108Z","level":"ERROR","duration":"180.501µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.367789443Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"1c50dbb3-392b-4fe2-b85e-f5109770dcef","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.403737579Z","level":"ERROR","duration":"162.542µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.40388864Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"b11c4ab7-01c7-4cc9-844e-cce0cde86e7b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket41/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> 2026/08/27 09:40:40 WARN Request failed, retrying attempt=4 max_attempts=6 backoff=842.946909ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.511775468Z","level":"ERROR","duration":"151.921µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.511924569Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4c8b13a1-6271-472f-84b5-9fd222bfd1ca","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket39/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.514807714Z","level":"ERROR","duration":"138.641µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.514856375Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"76459111-9414-4b75-9c94-94301c90d4ef","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket46/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.517532838Z","level":"ERROR","duration":"130.301µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.517578979Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c624ce33-45a0-473c-a668-6c6daf28f28f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket45/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.53138242Z","level":"ERROR","duration":"167.161µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.531517701Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"139e4035-1f67-4edb-aac8-fc5e2abf27d4","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket44/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.557891053Z","level":"ERROR","duration":"163.901µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.558508519Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"90961430-b229-4f1c-9c07-70d39f7eea43","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket43/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> === NAME TestCreatePendingClosure_SmallNARUsesSimplePUT niks3-go-unit-tests> uploads_test.go:458: unexpected error: Put "http://localhost:40529/bucket18/": context deadline exceeded niks3-go-unit-tests> niks3-go-unit-tests> --- FAIL: TestCreatePendingClosure_SmallNARUsesSimplePUT (10.92s) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.771119725Z","level":"ERROR","duration":"227.102µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.771238867Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"7dbcb388-c855-4c6a-8d18-f906ba35d922","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket44/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.828234367Z","level":"ERROR","duration":"152.101µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.828366749Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"3f976354-a77f-47ee-ad6d-9eedc3a7d19b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.832182202Z","level":"ERROR","duration":"142.721µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.832298983Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f7973311-f325-4fea-8302-a66390b5af8d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket39/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.846578668Z","level":"ERROR","duration":"198.261µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.846701389Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"dca92c47-3896-4895-9e24-08f311263299","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> === NAME TestReadProxyRootRedirectsToIndexHTML niks3-go-unit-tests> proxy_test.go:349: unexpected error: context deadline exceeded niks3-go-unit-tests> niks3-go-unit-tests> --- FAIL: TestReadProxyRootRedirectsToIndexHTML (11.09s) niks3-go-unit-tests> === NAME TestCompletedNarNotReofferedAcrossClosures niks3-go-unit-tests> reupload_uncommitted_test.go:23: unexpected error: Put "http://localhost:40529/bucket23/": context deadline exceeded niks3-go-unit-tests> niks3-go-unit-tests> --- FAIL: TestCompletedNarNotReofferedAcrossClosures (11.04s) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.889825448Z","level":"ERROR","duration":"161.601µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.88996353Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"1577cd5e-222b-4a93-adcd-ac2d0376979f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket46/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.911005814Z","level":"ERROR","duration":"182.041µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.911133875Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8918b211-5ac4-438e-8c9f-a3c974178b1c","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket44/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.92301942Z","level":"ERROR","duration":"140.562µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.923116041Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"41405ffb-49e9-400b-b8af-e3a87c3b2bbc","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.992921814Z","level":"ERROR","duration":"150.682µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.992980894Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"229f334a-8991-4396-8260-20a650596c38","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.996644366Z","level":"ERROR","duration":"132.121µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:40.996733527Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f5f36947-8577-4171-8a02-85c455cd14db","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.014762506Z","level":"ERROR","duration":"173.962µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.014971007Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"d4a23db3-24ec-4093-aff1-b82ca94df7b8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> === NAME TestReadProxyRangeRequest niks3-go-unit-tests> proxy_test.go:406: unexpected error: Put "http://localhost:40529/bucket26/": context deadline exceeded niks3-go-unit-tests> niks3-go-unit-tests> --- FAIL: TestReadProxyRangeRequest (10.07s) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.088188411Z","level":"ERROR","duration":"230.183µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.088248651Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"af3ab4a0-a9e3-4427-ab4d-a2d32007e80a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.094019422Z","level":"ERROR","duration":"144.862µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.094073862Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"5136fa93-f3fb-4442-9364-cf76d3a26cc1","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket46/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.112919308Z","level":"ERROR","duration":"159.542µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.112974348Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f4d2c716-4a1c-44db-9cd1-ed2677f76833","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket42/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.120693396Z","level":"ERROR","duration":"132.821µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.120740376Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"fdc39a21-5f51-450b-81c9-1ce9f6c2399b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket39/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.126455087Z","level":"ERROR","duration":"126.321µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.126548828Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"b6a1438d-817d-49e0-bad3-a1c618aed893","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket46/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.151951991Z","level":"ERROR","duration":"177.762µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.152017011Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e25bfde4-f660-4ada-935a-dac57c59bf9a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket45/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> 2026/08/27 09:40:41 WARN Request failed, retrying attempt=5 max_attempts=6 backoff=1.591194882s error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.280100796Z","level":"ERROR","duration":"179.121µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.280309818Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"33319c66-027c-4606-8077-bb2a28175f66","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.308381304Z","level":"ERROR","duration":"162.861µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.308506965Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"30f200ec-7d44-4cc5-bac4-5115c17a0605","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.395632731Z","level":"ERROR","duration":"225.622µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.395830372Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f37adf0b-d631-42c6-87e4-76643ff26bc0","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket41/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.402686493Z","level":"ERROR","duration":"135.381µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.402733393Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"3c231950-dc77-4c77-b0ad-124fa62168df","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.433597484Z","level":"ERROR","duration":"169.081µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.433664505Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"96ba76f9-214c-4a9d-8473-babed436c5ab","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket29/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> === NAME TestOrphanedObjectsGCStressTest niks3-go-unit-tests> orphaned_objects_gc_test.go:307: unexpected error: Put "http://localhost:40529/bucket28/": context deadline exceeded niks3-go-unit-tests> niks3-go-unit-tests> --- FAIL: TestOrphanedObjectsGCStressTest (10.26s) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.47072015Z","level":"ERROR","duration":"156.221µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.470872112Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e7d2ca8b-d0df-4d54-be9b-ccb53d90fdb0","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket43/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> === NAME TestOrphanedObjectsGC niks3-go-unit-tests> orphaned_objects_gc_test.go:98: unexpected error: context deadline exceeded niks3-go-unit-tests> niks3-go-unit-tests> --- FAIL: TestOrphanedObjectsGC (10.27s) niks3-go-unit-tests> === NAME TestReadProxyDisabled niks3-go-unit-tests> proxy_test.go:392: unexpected error: context deadline exceeded niks3-go-unit-tests> niks3-go-unit-tests> --- FAIL: TestReadProxyDisabled (10.39s) niks3-go-unit-tests> === NAME TestClientIntegration niks3-go-unit-tests> client_integration_test.go:252: unexpected error: context deadline exceeded niks3-go-unit-tests> niks3-go-unit-tests> --- FAIL: TestClientIntegration (10.31s) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.578690519Z","level":"ERROR","duration":"169.762µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.578891881Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8ff65f92-c8c1-41f7-94d5-8b559854a51d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> === NAME TestGCBugBareHashReferences niks3-go-unit-tests> gc_bug_test.go:18: unexpected error: Put "http://localhost:40529/bucket34/": context deadline exceeded niks3-go-unit-tests> niks3-go-unit-tests> --- FAIL: TestGCBugBareHashReferences (10.24s) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.686336144Z","level":"ERROR","duration":"169.101µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.686406405Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"58b6baa3-d031-4cec-a275-dc2a0ae2743c","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.728743097Z","level":"ERROR","duration":"219.702µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.728800257Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"3f129cd5-c1a3-46a4-9af9-c7691efbf8af","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket44/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.802602706Z","level":"ERROR","duration":"215.421µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.802750747Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6c492585-89ad-4b4d-bfae-620631456cad","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket43/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.804585703Z","level":"ERROR","duration":"134.101µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.804636583Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"640d8255-1411-4fde-8061-b352cf00c1f5","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket44/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.967820637Z","level":"ERROR","duration":"159.122µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.967927958Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"bbd98615-e48b-4b8d-9c10-36a0e23ad136","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket46/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.971978733Z","level":"ERROR","duration":"149.141µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:41.972066054Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"690458d7-ae22-4bd6-8ac3-26bafff82a63","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket42/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:42.039139283Z","level":"ERROR","duration":"274.222µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:42.039196084Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"7f303745-164c-40ad-9dff-184c1c689879","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket44/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:42.054804981Z","level":"ERROR","duration":"159.402µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:42.055030043Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"9310f20e-3043-4d52-849f-372f07030441","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket45/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:42.089177243Z","level":"ERROR","duration":"158.542µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:42.089291824Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0def7dcf-bb46-42b1-a33d-232b98fef1a9","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket39/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> === NAME TestPinProtectsFromGC niks3-go-unit-tests> client_integration_test.go:603: unexpected error: Put "http://localhost:40529/bucket35/": context deadline exceeded niks3-go-unit-tests> niks3-go-unit-tests> --- FAIL: TestPinProtectsFromGC (10.07s) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:42.237578086Z","level":"ERROR","duration":"155.281µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:42.237725587Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"95f9ea18-fb97-4846-bf69-7c307c2286ad","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:42.262701767Z","level":"ERROR","duration":"161.202µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:42.262764767Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c468eddc-c442-4544-9262-40dec653c4fe","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket43/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:42.337697105Z","level":"ERROR","duration":"155.681µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:42.337838567Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"beef6a03-0c0e-48c9-a407-6a75d92912ec","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket41/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:42.368849839Z","level":"ERROR","duration":"215.302µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:42.369004801Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"1d929ef4-b021-4690-9034-47f9da618d42","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(379)"} niks3-go-unit-tests> 2026/08/27 09:40:42 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: sending request: request failed after retries: Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" niks3-go-unit-tests> 2026/08/27 09:40:42 WARN Request failed, retrying attempt=1 max_attempts=6 backoff=100ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:43 WARN Request failed, retrying attempt=2 max_attempts=6 backoff=181.427703ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:43.013860045Z","level":"ERROR","duration":"162.602µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:43.014030906Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6da3933f-cf38-46aa-ae0e-264487c6db9e","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket43/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:43.052948848Z","level":"ERROR","duration":"159.381µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:43.053089749Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6a28a050-40a2-4904-b340-dce3f711fe03","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket43/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(376)"} niks3-go-unit-tests> 2026/08/27 09:40:43 WARN Request failed, retrying attempt=3 max_attempts=6 backoff=401.014363ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:43.301500712Z","level":"ERROR","duration":"228.302µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:43.301645433Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"25da5ce8-a9bd-4783-b3e3-af52ad35c432","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> 2026/08/27 09:40:43 WARN Request failed, retrying attempt=4 max_attempts=6 backoff=870.710479ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:43.734858518Z","level":"ERROR","duration":"244.202µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:43.73504818Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8a3b220f-11b2-41c2-9c27-d089027c7ec2","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:43.794036098Z","level":"ERROR","duration":"6.332695279s","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"ServiceUnavailableThe service is unavailable. Please retry.\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:43.794100199Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"b09d799f-9d6a-4167-bc68-d798b5b62081","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket38/","status_code":503,"duration_ms":6332,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> 2026/08/27 09:40:43 WARN Rate limiter enabled after throttle name=s3-test rate=5 niks3-go-unit-tests> 2026/08/27 09:40:43 WARN S3 rate limit hit operation="complete multipart upload" error="Please reduce your request rate." niks3-go-unit-tests> === NAME TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle niks3-go-unit-tests> throttle_test.go:213: Proxy stats: total=15, throttled=10, completeMultipart=10 niks3-go-unit-tests> throttle_test.go:215: Rate limiter: enabled=true, rate=5.00 niks3-go-unit-tests> --- PASS: TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle (14.05s) niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:43.931174743Z","level":"ERROR","duration":"171.042µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:43.931248884Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e4c67cbf-c3df-4d1c-a61c-f42acbc8bbbe","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket43/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:44.132545992Z","level":"ERROR","duration":"290.523µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:44.132751254Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"314f1948-9297-4664-940e-082d761198fd","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket43/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:44.369652155Z","level":"ERROR","duration":"152.481µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:44.369706156Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"019ff021-67a5-4d39-b433-8f798da640a9","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket43/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> 2026/08/27 09:40:44 WARN Request failed, retrying attempt=5 max_attempts=6 backoff=1.469676692s error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:44.683545933Z","level":"ERROR","duration":"216.122µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:44.683705674Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"d6442e97-7599-4dac-9f26-0fcf57997b08","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:44.870715937Z","level":"ERROR","duration":"153.422µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:44.870770337Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"10cea995-5d87-4598-a6a7-3d39c65c953c","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket43/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:45.549297198Z","level":"ERROR","duration":"213.902µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:45.54947476Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8a0be1f1-83d7-4d9e-a35b-66a2d0898c73","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket43/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(382)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:45.565961885Z","level":"ERROR","message":"Scanner data usage save failed","event":"scanner_persist_state","component":"scanner","subsystem":"runtime","path":"buckets/.usage.v2.json","state":"save_failed","error":"Lock error: Lock acquisition timeout for resource '.rustfs.sys/buckets/.usage.v2.json@latest' after 5s","target":"rustfs::scanner","filename":"crates/scanner/src/scanner.rs","line_number":3872,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:45.566079446Z","level":"ERROR","message":"Scanner cycle completed without a durable data usage snapshot","event":"scanner_persist_state","component":"scanner","subsystem":"runtime","cycle":7,"state":"usage_not_durable","target":"rustfs::scanner","filename":"crates/scanner/src/scanner.rs","line_number":2763,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:45.619371074Z","level":"ERROR","duration":"156.341µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"SlowDownbucket creation concurrency limit reached; retry later\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> {"timestamp":"2026-08-27T09:40:45.619423735Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2315eedd-3792-44e0-be03-133d51b782a8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket40/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(383)"} niks3-go-unit-tests> 2026/08/27 09:40:45 WARN mTLS auth: subject not in bound subjects subject="CN=writer" niks3-go-unit-tests> --- PASS: TestService_ReadAuthMiddleware (8.22s) niks3-go-unit-tests> --- PASS: TestCacheStatsHandler (8.34s) niks3-go-unit-tests> 2026/08/27 09:40:45 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other" niks3-go-unit-tests> 2026/08/27 09:40:45 WARN mTLS auth: bound subjects configured but subject DN unavailable niks3-go-unit-tests> 2026/08/27 09:40:45 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted" niks3-go-unit-tests> --- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (8.04s) niks3-go-unit-tests> 2026/08/27 09:40:45 INFO Created nix-cache-info in bucket bucket=bucket38 niks3-go-unit-tests> 2026/08/27 09:40:45 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n" niks3-go-unit-tests> --- PASS: TestReadProxy404 (6.97s) niks3-go-unit-tests> 2026/08/27 09:40:45 WARN Authentication failed token_preview=invalid-token token_length=13 reason="static API token mismatch" niks3-go-unit-tests> --- PASS: TestService_AuthMiddleware_MTLSProxyHeader (7.10s) niks3-go-unit-tests> --- PASS: TestClientErrorHandling (0.00s) niks3-go-unit-tests> --- PASS: TestClientErrorHandling/InvalidStorePath (6.47s) niks3-go-unit-tests> --- PASS: TestClientErrorHandling/InvalidAuthToken (6.24s) niks3-go-unit-tests> --- PASS: TestClientErrorHandling/ServerNotAvailable (6.36s) niks3-go-unit-tests> === NAME TestClientCADerivations niks3-go-unit-tests> client_ca_test.go:136: Built CA derivation: /build/TestClientCADerivations2674995104/001/store/w7f17766q4slqpcszi4hx0yq49j16m41-ca-test niks3-go-unit-tests> client_ca_test.go:139: Found 1 dependencies (including self) niks3-go-unit-tests> 2026/08/27 09:40:46 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:46 INFO Received uploads request method=POST path=/api/pending_closures niks3-go-unit-tests> 2026/08/27 09:40:46 INFO Uploading 1 paths to 127.0.0.1 (0 already cached) niks3-go-unit-tests> 2026/08/27 09:40:46 INFO Uploading w7f17766q4slqpcszi4hx0yq49j16m41-ca-test (144B) niks3-go-unit-tests> 2026/08/27 09:40:46 WARN Failed to register uploaded object key=log/k650mpfq7j1mdmvyli264kr7xz5j0hnn-ca-test.drv error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:46 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:46 WARN Failed to register uploaded object key=w7f17766q4slqpcszi4hx0yq49j16m41.ls error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign niks3-go-unit-tests> 2026/08/27 09:40:46 INFO Signed narinfos id=1 count=1 niks3-go-unit-tests> 2026/08/27 09:40:46 INFO Uploading 1 narinfos niks3-go-unit-tests> 2026/08/27 09:40:46 WARN Failed to register uploaded object key=w7f17766q4slqpcszi4hx0yq49j16m41.narinfo error="server returned 404: 404 page not found\n" niks3-go-unit-tests> 2026/08/27 09:40:46 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete niks3-go-unit-tests> 2026/08/27 09:40:46 INFO Completed upload id=1 niks3-go-unit-tests> 2026/08/27 09:40:46 INFO Upload complete. (181ms) niks3-go-unit-tests> client_ca_test.go:180: Narinfo contains CA field: StorePath: /build/TestClientCADerivations2674995104/001/store/w7f17766q4slqpcszi4hx0yq49j16m41-ca-test niks3-go-unit-tests> URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst niks3-go-unit-tests> Compression: zstd niks3-go-unit-tests> NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n niks3-go-unit-tests> NarSize: 144 niks3-go-unit-tests> References: niks3-go-unit-tests> Deriver: /build/TestClientCADerivations2674995104/001/store/k650mpfq7j1mdmvyli264kr7xz5j0hnn-ca-test.drv niks3-go-unit-tests> CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n niks3-go-unit-tests> client_ca_test.go:185: Checking for realisation files in S3... niks3-go-unit-tests> client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations niks3-go-unit-tests> client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache niks3-go-unit-tests> client_ca_test.go:258: nix copy output: warning: you don't have Internet access; disabling some network-dependent features niks3-go-unit-tests> warning: failed to create TLS context for AWS credential providers; SSO, STS WebIdentity, and ECS container authentication will be unavailable niks3-go-unit-tests> error: binary cache 's3://bucket38?endpoint=http://localhost:40529®ion=eu-west-1' is for Nix stores with prefix '/nix/store', not '/build/TestClientCADerivations2674995104/001/store' niks3-go-unit-tests> client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 1 niks3-go-unit-tests> --- PASS: TestClientCADerivations (9.26s) niks3-go-unit-tests> === RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token niks3-go-unit-tests> === PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token niks3-go-unit-tests> === RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected niks3-go-unit-tests> === PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected niks3-go-unit-tests> === RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected niks3-go-unit-tests> === PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected niks3-go-unit-tests> === RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured niks3-go-unit-tests> === PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured niks3-go-unit-tests> === CONT TestService_AuthMiddleware_OIDC/valid_OIDC_token niks3-go-unit-tests> === CONT TestService_AuthMiddleware_OIDC/malformed_token_rejected niks3-go-unit-tests> 2026/08/27 09:40:46 WARN Authentication failed token_preview=not-a-valid-jwt token_length=15 oidc_error="no provider could verify the token (signature or issuer mismatch)" oidc_provider="" tried_providers=[test] niks3-go-unit-tests> === CONT TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected niks3-go-unit-tests> === CONT TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured niks3-go-unit-tests> 2026/08/27 09:40:46 INFO OIDC auth successful provider=test niks3-go-unit-tests> 2026/08/27 09:40:46 WARN Authentication failed token_preview=eyJhbGciOi...p6qH10D55A token_length=702 oidc_error="bound claims validation failed: claim \"repository_owner\" value [otherorg] not in allowed patterns [myorg]" oidc_provider=test tried_providers=[test] niks3-go-unit-tests> --- PASS: TestService_AuthMiddleware_OIDC (9.13s) niks3-go-unit-tests> --- PASS: TestService_AuthMiddleware_OIDC/malformed_token_rejected (0.00s) niks3-go-unit-tests> --- PASS: TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured (0.00s) niks3-go-unit-tests> --- PASS: TestService_AuthMiddleware_OIDC/valid_OIDC_token (0.01s) niks3-go-unit-tests> --- PASS: TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected (0.01s) niks3-go-unit-tests> FAIL niks3-go-unit-tests> 2026-08-27 09:40:46.721 UTC [108] LOG: received smart shutdown request niks3-go-unit-tests> 2026/08/27 09:40:56 ERROR failed to kill rustfs error="no such process" niks3-go-unit-tests> 2026/08/27 09:40:56 INFO killed rustfs niks3-go-unit-tests> 2026-08-27 09:40:56.724 UTC [592] FATAL: terminating connection due to unexpected postmaster exit niks3-go-unit-tests> 2026-08-27 09:40:56.724 UTC [659] FATAL: terminating connection due to unexpected postmaster exit niks3-go-unit-tests> 2026-08-27 09:40:56.724 UTC [610] FATAL: terminating connection due to unexpected postmaster exit niks3-go-unit-tests> 2026-08-27 09:40:56.724 UTC [613] FATAL: terminating connection due to unexpected postmaster exit niks3-go-unit-tests> 2026-08-27 09:40:56.730 UTC [651] FATAL: terminating connection due to unexpected postmaster exit niks3-go-unit-tests> 2026-08-27 09:40:56.732 UTC [654] FATAL: terminating connection due to unexpected postmaster exit niks3-go-unit-tests> 2026-08-27 09:40:56.733 UTC [650] FATAL: terminating connection due to unexpected postmaster exit niks3-go-unit-tests> 2026-08-27 09:40:56.734 UTC [653] FATAL: terminating connection due to unexpected postmaster exit niks3-go-unit-tests> 2026-08-27 09:40:56.738 UTC [649] FATAL: terminating connection due to unexpected postmaster exit niks3-go-unit-tests> 2026-08-27 09:40:56.738 UTC [598] FATAL: terminating connection due to unexpected postmaster exit niks3-go-unit-tests> 2026-08-27 09:40:56.741 UTC [623] FATAL: terminating connection due to unexpected postmaster exit niks3-go-unit-tests> 2026-08-27 09:40:56.742 UTC [614] FATAL: terminating connection due to unexpected postmaster exit niks3-go-unit-tests> 2026/08/27 09:40:56 ERROR failed to wait for rustfs error="signal: killed" error: Cannot build '/nix/store/fv7dxf6zapvyjimvqvr8bmb1f91nvq5x-niks3-go-unit-tests.drv'. Reason: builder failed with exit code 1. Output paths: /nix/store/n224qyghx58lcv82w7xykck8c3snp2j4-niks3-go-unit-tests Last 25 log lines: > === CONT TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured > 2026/08/27 09:40:46 INFO OIDC auth successful provider=test > 2026/08/27 09:40:46 WARN Authentication failed token_preview=eyJhbGciOi...p6qH10D55A token_length=702 oidc_error="bound claims validation failed: claim \"repository_owner\" value [otherorg] not in allowed patterns [myorg]" oidc_provider=test tried_providers=[test] > --- PASS: TestService_AuthMiddleware_OIDC (9.13s) > --- PASS: TestService_AuthMiddleware_OIDC/malformed_token_rejected (0.00s) > --- PASS: TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured (0.00s) > --- PASS: TestService_AuthMiddleware_OIDC/valid_OIDC_token (0.01s) > --- PASS: TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected (0.01s) > FAIL > 2026-08-27 09:40:46.721 UTC [108] LOG: received smart shutdown request > 2026/08/27 09:40:56 ERROR failed to kill rustfs error="no such process" > 2026/08/27 09:40:56 INFO killed rustfs > 2026-08-27 09:40:56.724 UTC [592] FATAL: terminating connection due to unexpected postmaster exit > 2026-08-27 09:40:56.724 UTC [659] FATAL: terminating connection due to unexpected postmaster exit > 2026-08-27 09:40:56.724 UTC [610] FATAL: terminating connection due to unexpected postmaster exit > 2026-08-27 09:40:56.724 UTC [613] FATAL: terminating connection due to unexpected postmaster exit > 2026-08-27 09:40:56.730 UTC [651] FATAL: terminating connection due to unexpected postmaster exit > 2026-08-27 09:40:56.732 UTC [654] FATAL: terminating connection due to unexpected postmaster exit > 2026-08-27 09:40:56.733 UTC [650] FATAL: terminating connection due to unexpected postmaster exit > 2026-08-27 09:40:56.734 UTC [653] FATAL: terminating connection due to unexpected postmaster exit > 2026-08-27 09:40:56.738 UTC [649] FATAL: terminating connection due to unexpected postmaster exit > 2026-08-27 09:40:56.738 UTC [598] FATAL: terminating connection due to unexpected postmaster exit > 2026-08-27 09:40:56.741 UTC [623] FATAL: terminating connection due to unexpected postmaster exit > 2026-08-27 09:40:56.742 UTC [614] FATAL: terminating connection due to unexpected postmaster exit > 2026/08/27 09:40:56 ERROR failed to wait for rustfs error="signal: killed" For full logs, run: nix log /nix/store/fv7dxf6zapvyjimvqvr8bmb1f91nvq5x-niks3-go-unit-tests.drv