niks3-go-unit-tests log

succeeded `niks3-go-unit-tests` `checks.aarch64-linux.go-unit-tests` · build #142 · raw

1tribuchet: building on eliza2Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestCaseHackSuffix6=== PAUSE TestCaseHackSuffix7=== RUN TestFilterOversizedClosures8=== PAUSE TestFilterOversizedClosures9=== RUN TestPartSizeForNAR10=== PAUSE TestPartSizeForNAR11=== RUN TestUploadMultipart_SupersededByPeer12=== PAUSE TestUploadMultipart_SupersededByPeer13=== RUN TestDumpPathMatchesNix14=== PAUSE TestDumpPathMatchesNix15=== RUN TestDumpPathSingleFile16=== PAUSE TestDumpPathSingleFile17=== RUN TestDumpPathWriterError18=== PAUSE TestDumpPathWriterError19=== RUN TestEncodeNixBase3220=== PAUSE TestEncodeNixBase3221=== RUN TestEncodeNixBase32WithRealHash22=== PAUSE TestEncodeNixBase32WithRealHash23=== RUN TestConvertHashToNix3224=== PAUSE TestConvertHashToNix3225=== RUN TestGetStorePathHash26=== PAUSE TestGetStorePathHash27=== RUN TestPathInfoHashCompatibility28=== PAUSE TestPathInfoHashCompatibility29=== RUN TestParsePathInfoJSON30=== PAUSE TestParsePathInfoJSON31=== RUN TestParsePathInfoJSONMultiplePaths32=== PAUSE TestParsePathInfoJSONMultiplePaths33=== RUN TestPathInfoCACompatibility34=== PAUSE TestPathInfoCACompatibility35=== RUN TestRateLimiterFeedback36=== PAUSE TestRateLimiterFeedback37=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess38=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess39=== RUN TestResolveStorePath40=== PAUSE TestResolveStorePath41=== RUN TestDoWithRetry_BodyReplayedViaGetBody42=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody43=== RUN TestShellSplit44=== PAUSE TestShellSplit45=== RUN TestShellSplitErrors46=== PAUSE TestShellSplitErrors47=== RUN TestSetClientTLS48=== PAUSE TestSetClientTLS49=== RUN TestSetClientTLSDoesNotMutateDefaultTransport50=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport51=== RUN TestSetClientTLSErrors52=== PAUSE TestSetClientTLSErrors53=== RUN TestStaticToken54=== PAUSE TestStaticToken55=== RUN TestFileTokenReadsAndCaches56=== PAUSE TestFileTokenReadsAndCaches57=== RUN TestFileTokenMissing58=== PAUSE TestFileTokenMissing59=== RUN TestFileTokenEmpty60=== PAUSE TestFileTokenEmpty61=== RUN TestScriptTokenNoExpiryRerunsEveryCall62=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall63=== RUN TestScriptTokenCachesUntilRefresh64=== PAUSE TestScriptTokenCachesUntilRefresh65=== RUN TestScriptTokenEmptyToken66=== PAUSE TestScriptTokenEmptyToken67=== RUN TestScriptTokenBadJSON68=== PAUSE TestScriptTokenBadJSON69=== RUN TestScriptTokenScriptFails70=== PAUSE TestScriptTokenScriptFails71=== RUN TestScriptTokenEmptyCommand72=== PAUSE TestScriptTokenEmptyCommand73=== CONT TestDoServerRequestAttachesToken74=== CONT TestFileTokenMissing75=== CONT TestResolveStorePath76=== CONT TestScriptTokenEmptyToken77=== CONT TestScriptTokenScriptFails78=== CONT TestSetClientTLSDoesNotMutateDefaultTransport79--- PASS: TestFileTokenMissing (0.00s)80=== CONT TestShellSplitErrors81=== CONT TestShellSplit82--- PASS: TestShellSplitErrors (0.00s)83=== CONT TestConvertHashToNix3284=== CONT TestScriptTokenCachesUntilRefresh85=== CONT TestScriptTokenNoExpiryRerunsEveryCall86=== CONT TestDumpPathWriterError87=== RUN TestConvertHashToNix32/SRI_format_to_Nix3288=== CONT TestScriptTokenBadJSON89=== CONT TestFileTokenEmpty90=== CONT TestScriptTokenEmptyCommand91=== CONT TestDumpPathSingleFile92=== CONT TestEncodeNixBase32WithRealHash93=== CONT TestSetClientTLSErrors94=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess95=== CONT TestRateLimiterFeedback96=== CONT TestPathInfoCACompatibility97=== CONT TestParsePathInfoJSONMultiplePaths98=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths99=== CONT TestParsePathInfoJSON100=== CONT TestPathInfoHashCompatibility101=== CONT TestGetStorePathHash102=== CONT TestSetClientTLS103=== CONT TestDoWithRetry_BodyReplayedViaGetBody104=== CONT TestFileTokenReadsAndCaches105=== CONT TestDumpPathMatchesNix106--- PASS: TestResolveStorePath (0.00s)107=== CONT TestEncodeNixBase32108=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32109--- PASS: TestShellSplit (0.00s)110--- PASS: TestScriptTokenEmptyCommand (0.00s)111--- PASS: TestEncodeNixBase32WithRealHash (0.00s)112=== CONT TestPartSizeForNAR113--- PASS: TestFileTokenEmpty (0.00s)114=== RUN TestRateLimiterFeedback/429_enables_limiter115=== RUN TestPartSizeForNAR/zero_stays_at_minimum116=== RUN TestPathInfoCACompatibility/null_ca_field117=== RUN TestGetStorePathHash/valid_store_path118=== RUN TestParsePathInfoJSON/Nix_format119--- PASS: TestScriptTokenScriptFails (0.00s)1202026/08/27 09:31:27 WARN Rate limiter enabled after throttle name=server-test rate=5121=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)122=== PAUSE TestParsePathInfoJSON/Nix_format123=== RUN TestParsePathInfoJSON/Lix_format124=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths125=== CONT TestUploadMultipart_SupersededByPeer126=== RUN TestUploadMultipart_SupersededByPeer/exists127=== RUN TestEncodeNixBase32/test_string_hash128=== PAUSE TestEncodeNixBase32/test_string_hash129=== RUN TestEncodeNixBase32/empty_input130=== PAUSE TestEncodeNixBase32/empty_input131=== RUN TestConvertHashToNix32/already_Nix32_format132=== PAUSE TestConvertHashToNix32/already_Nix32_format133=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths134=== PAUSE TestPathInfoCACompatibility/null_ca_field135=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths136=== PAUSE TestUploadMultipart_SupersededByPeer/exists137=== RUN TestUploadMultipart_SupersededByPeer/missing138=== CONT TestCaseHackSuffix139=== RUN TestPathInfoCACompatibility/old_string_format_-_text140=== CONT TestFilterOversizedClosures141=== PAUSE TestRateLimiterFeedback/429_enables_limiter142=== PAUSE TestUploadMultipart_SupersededByPeer/missing143=== RUN TestConvertHashToNix32/invalid_format144=== CONT TestStaticToken145=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)146--- PASS: TestFileTokenReadsAndCaches (0.00s)147--- PASS: TestScriptTokenBadJSON (0.00s)148--- PASS: TestStaticToken (0.00s)149=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon150=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text151--- PASS: TestScriptTokenEmptyToken (0.01s)152=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths153=== PAUSE TestGetStorePathHash/valid_store_path154=== PAUSE TestParsePathInfoJSON/Lix_format155=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon156=== CONT TestEncodeNixBase32/test_string_hash157=== RUN TestSetClientTLSErrors/missing_cert_file158=== RUN TestGetStorePathHash/basename_without_hyphen_should_error159=== PAUSE TestSetClientTLSErrors/missing_cert_file160=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error1612026/08/27 09:31:27 WARN Rate limiter enabled after throttle name=server-test rate=5162=== CONT TestEncodeNixBase32/empty_input1632026/08/27 09:31:27 WARN Request returned retryable status, retrying attempt=1 max_attempts=6 backoff=0s status=503 url=http://127.0.0.1:37625164=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths165=== CONT TestUploadMultipart_SupersededByPeer/exists166=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error167=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive168=== RUN TestParsePathInfoJSON/empty_input169=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum170=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI171=== RUN TestSetClientTLSErrors/missing_key_file172=== RUN TestFilterOversizedClosures/no_limit_keeps_everything173=== RUN TestRateLimiterFeedback/503_enables_limiter174=== PAUSE TestRateLimiterFeedback/503_enables_limiter175=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter176=== CONT TestUploadMultipart_SupersededByPeer/missing177=== PAUSE TestConvertHashToNix32/invalid_format178--- PASS: TestEncodeNixBase32 (0.00s)179 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)180 --- PASS: TestEncodeNixBase32/empty_input (0.00s)181=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error182=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive183=== RUN TestPartSizeForNAR/small_stays_at_minimum184=== PAUSE TestParsePathInfoJSON/empty_input185=== PAUSE TestSetClientTLSErrors/missing_key_file186=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything187=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error188=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped189=== RUN TestSetClientTLSErrors/missing_ca_file190=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped191=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI1922026/08/27 09:31:27 WARN Rate limiter backed off name=server-test rate=51932026/08/27 09:31:27 WARN Request returned retryable status, retrying attempt=2 max_attempts=6 backoff=0s status=503 url=http://127.0.0.1:37625194=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error195=== CONT TestGetStorePathHash/valid_store_path196=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error197=== CONT TestConvertHashToNix32/already_Nix32_format198=== CONT TestConvertHashToNix32/invalid_format199--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.01s)200=== RUN TestPathInfoCACompatibility/new_structured_format_-_text201=== PAUSE TestPartSizeForNAR/small_stays_at_minimum202=== RUN TestParsePathInfoJSON/whitespace_only203=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error204=== CONT TestConvertHashToNix32/SRI_format_to_Nix32205=== PAUSE TestSetClientTLSErrors/missing_ca_file206=== RUN TestFilterOversizedClosures/all_closures_skipped207=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512208=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512209=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)210=== PAUSE TestFilterOversizedClosures/all_closures_skipped211=== CONT TestFilterOversizedClosures/no_limit_keeps_everything212=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512213=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum214=== CONT TestFilterOversizedClosures/all_closures_skipped2152026/08/27 09:31:27 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50216--- PASS: TestDoServerRequestAttachesToken (0.01s)217=== CONT TestGetStorePathHash/basename_without_hyphen_should_error218=== RUN TestSetClientTLS/rejects_connection_without_client_cert219=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert220=== PAUSE TestParsePathInfoJSON/whitespace_only221=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter222=== RUN TestParsePathInfoJSON/invalid_JSON223=== PAUSE TestParsePathInfoJSON/invalid_JSON224=== CONT TestParsePathInfoJSON/Nix_format225=== CONT TestParsePathInfoJSON/whitespace_only226=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI227=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon228=== RUN TestSetClientTLSErrors/invalid_ca_file229=== PAUSE TestSetClientTLSErrors/invalid_ca_file230=== CONT TestSetClientTLSErrors/missing_cert_file231=== CONT TestSetClientTLSErrors/invalid_ca_file232=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped233=== CONT TestSetClientTLSErrors/missing_ca_file234=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum2352026/08/27 09:31:27 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000236=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text237--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.01s)238=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA239=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter240=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter241=== CONT TestRateLimiterFeedback/429_enables_limiter242=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter243=== CONT TestParsePathInfoJSON/empty_input244=== CONT TestParsePathInfoJSON/Lix_format245=== CONT TestParsePathInfoJSON/invalid_JSON246=== CONT TestSetClientTLSErrors/missing_key_file247=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts248=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method249=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA250=== RUN TestSetClientTLS/preserves_debug_logging_transport251=== PAUSE TestSetClientTLS/preserves_debug_logging_transport252=== CONT TestSetClientTLS/rejects_connection_without_client_cert253=== CONT TestSetClientTLS/preserves_debug_logging_transport2542026/08/27 09:31:27 WARN Rate limiter enabled after throttle name=server-test rate=5255--- PASS: TestParsePathInfoJSONMultiplePaths (0.00s)256 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)257 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)2582026/08/27 09:31:27 WARN Request returned retryable status, retrying attempt=1 max_attempts=2 backoff=0s status=429 url=http://127.0.0.1:37733259=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter260--- PASS: TestScriptTokenCachesUntilRefresh (0.01s)261=== CONT TestRateLimiterFeedback/503_enables_limiter262--- PASS: TestGetStorePathHash (0.01s)263 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)264 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)265 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)266 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)2672026/08/27 09:31:27 WARN Rate limiter backed off name=server-test rate=5268--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.01s)269=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts270=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method271=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA272=== CONT TestPathInfoCACompatibility/new_structured_format_-_text273=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive274=== RUN TestPartSizeForNAR/1_TiB275=== PAUSE TestPartSizeForNAR/1_TiB276=== RUN TestPartSizeForNAR/5_TiB_S3_max_object277=== CONT TestPathInfoCACompatibility/null_ca_field2782026/08/27 09:31:27 WARN Rate limiter enabled after throttle name=server-test rate=5279=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method2802026/08/27 09:31:27 WARN Request returned retryable status, retrying attempt=1 max_attempts=2 backoff=0s status=503 url=http://127.0.0.1:39925281=== CONT TestPathInfoCACompatibility/old_string_format_-_text282--- PASS: TestPathInfoHashCompatibility (0.01s)283 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)284 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)285 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)286 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)287=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object2882026/08/27 09:31:27 WARN Rate limiter backed off name=server-test rate=5289=== RUN TestPartSizeForNAR/capped_at_5_GiB290--- PASS: TestFilterOversizedClosures (0.01s)291 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)292 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)293 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)294--- PASS: TestConvertHashToNix32 (0.01s)295 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)296 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)297 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)298=== PAUSE TestPartSizeForNAR/capped_at_5_GiB299=== CONT TestPartSizeForNAR/zero_stays_at_minimum300--- PASS: TestParsePathInfoJSON (0.01s)301 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)302 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)303 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)304 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)305 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)306=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts307=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum308=== CONT TestPartSizeForNAR/1_TiB309=== CONT TestPartSizeForNAR/small_stays_at_minimum310=== CONT TestPartSizeForNAR/capped_at_5_GiB311=== CONT TestPartSizeForNAR/5_TiB_S3_max_object312--- PASS: TestRateLimiterFeedback (0.01s)313 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s)314 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.00s)315 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.00s)316 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s)317--- PASS: TestPathInfoCACompatibility (0.01s)318 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)319 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)320 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)321 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)322 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)323--- PASS: TestUploadMultipart_SupersededByPeer (0.00s)324 --- PASS: TestUploadMultipart_SupersededByPeer/exists (0.00s)325 --- PASS: TestUploadMultipart_SupersededByPeer/missing (0.00s)326--- PASS: TestPartSizeForNAR (0.01s)327 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)328 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)329 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)330 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)331 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)332 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)333 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)334--- PASS: TestSetClientTLSErrors (0.01s)335 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)336 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)337 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)338 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)3392026/08/27 09:31:27 http: TLS handshake error from 127.0.0.1:52842: remote error: tls: bad certificate340--- PASS: TestSetClientTLS (0.01s)341 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.00s)342 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.00s)343 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.01s)344--- PASS: TestDumpPathSingleFile (0.04s)345--- PASS: TestCaseHackSuffix (0.03s)346--- PASS: TestDumpPathWriterError (0.05s)347--- PASS: TestDumpPathMatchesNix (0.08s)348--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)349PASS350Running server tests...351The files belonging to this database system will be owned by user "nixbld".352This user must also own the server process.353354The database cluster will be initialized with locale "C".355The default database encoding has accordingly been set to "SQL_ASCII".356The default text search configuration will be set to "english".357358Data page checksums are enabled.359360creating directory /build/postgres1945238980/data ... ok361creating subdirectories ... ok362selecting dynamic shared memory implementation ... posix363selecting default "max_connections" ... 100364selecting default "shared_buffers" ... 128MB365selecting default time zone ... UTC366creating configuration files ... ok367running bootstrap script ... ok368performing post-bootstrap initialization ... ok369syncing data to disk ... ok370371initdb: warning: enabling "trust" authentication for local connections372initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.373374Success. You can now start the database server using:375376 pg_ctl -D /build/postgres1945238980/data -l logfile start377378/build/postgres1945238980:5432 - no response3792026-08-27 09:31:29.134 UTC [112] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit3802026-08-27 09:31:29.135 UTC [112] LOG: listening on Unix socket "/build/postgres1945238980/.s.PGSQL.5432"3812026-08-27 09:31:29.140 UTC [119] LOG: database system was shut down at 2026-08-27 09:31:28 UTC3822026-08-27 09:31:29.144 UTC [112] LOG: database system is ready to accept connections383/build/postgres1945238980:5432 - accepting connections384=== RUN TestService_AuthMiddleware385=== PAUSE TestService_AuthMiddleware386=== RUN TestService_AuthMiddleware_MTLSProxyHeader387=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader388=== RUN TestService_AuthMiddleware_MTLSBoundSubjects389=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects390=== RUN TestService_ReadAuthMiddleware391=== PAUSE TestService_ReadAuthMiddleware392=== RUN TestService_AuthMiddleware_OIDC393=== PAUSE TestService_AuthMiddleware_OIDC394=== RUN TestCacheConfigHandler395=== PAUSE TestCacheConfigHandler396=== RUN TestCacheStatsHandler397=== PAUSE TestCacheStatsHandler398=== RUN TestClientCADerivations399=== PAUSE TestClientCADerivations400=== RUN TestClientErrorHandling401=== PAUSE TestClientErrorHandling402=== RUN TestClientIntegration403=== PAUSE TestClientIntegration404=== RUN TestClientMultipleUploads405=== PAUSE TestClientMultipleUploads406=== RUN TestClientWithDependencies407=== PAUSE TestClientWithDependencies408=== RUN TestPinProtectsFromGC409=== PAUSE TestPinProtectsFromGC410=== RUN TestGCAdvisoryLockBlocksConcurrentRun4112026-08-27 09:31:33.249 UTC [519] ERROR: relation "goose_db_version" does not exist at character 364122026-08-27 09:31:33.249 UTC [519] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4132026/08/27 09:31:33 OK 20241026095416_initial_model.sql (9.88ms)4142026/08/27 09:31:33 OK 20251210153512_drop_unused_gin_index.sql (1.57ms)4152026/08/27 09:31:33 OK 20251218171726_add_pins.sql (2.98ms)4162026/08/27 09:31:33 OK 20260628120000_add_object_size_and_stats.sql (2.8ms)4172026/08/27 09:31:33 goose: successfully migrated database to version: 202606281200004182026/08/27 09:31:33 OK 1_commit_pending_closure.sql (1.92ms)4192026/08/27 09:31:33 OK 2_object_stats_trigger.sql (724.69µs)4202026/08/27 09:31:33 goose: up to current file version: 2421--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (0.42s)422=== RUN TestGCBugBareHashReferences423=== PAUSE TestGCBugBareHashReferences424=== RUN TestGCMetrics425=== PAUSE TestGCMetrics426=== RUN TestGCTaskStore_StartNew427=== PAUSE TestGCTaskStore_StartNew428=== RUN TestGCTaskStore_DeduplicateSameParams429=== PAUSE TestGCTaskStore_DeduplicateSameParams430=== RUN TestGCTaskStore_ConflictDifferentParams431=== PAUSE TestGCTaskStore_ConflictDifferentParams432=== RUN TestGCTaskStore_GetEmpty433=== PAUSE TestGCTaskStore_GetEmpty434=== RUN TestGCTaskStore_GetReturnsLatest435=== PAUSE TestGCTaskStore_GetReturnsLatest436=== RUN TestGCTaskStore_CompletedAllowsNewTask437=== PAUSE TestGCTaskStore_CompletedAllowsNewTask438=== RUN TestGCTaskStore_PhaseUpdates439=== PAUSE TestGCTaskStore_PhaseUpdates440=== RUN TestGCTaskStore_Fail441=== PAUSE TestGCTaskStore_Fail442=== RUN TestGracefulShutdownDrainsInflight443=== PAUSE TestGracefulShutdownDrainsInflight444=== RUN TestService_healthCheckHandler445=== PAUSE TestService_healthCheckHandler446=== RUN TestGenerateLandingPage447=== PAUSE TestGenerateLandingPage448=== RUN TestCacheConfigHandlerMaxNarSize449=== PAUSE TestCacheConfigHandlerMaxNarSize450=== RUN TestCreatePendingClosureRejectsOversizedNAR451=== PAUSE TestCreatePendingClosureRejectsOversizedNAR452=== RUN TestNARDeduplicationMetadataUploadBug453=== PAUSE TestNARDeduplicationMetadataUploadBug454=== RUN TestMetricsInventory455=== PAUSE TestMetricsInventory456=== RUN TestService_NativeMTLS457=== PAUSE TestService_NativeMTLS458=== RUN TestServerTLSConfig459=== PAUSE TestServerTLSConfig460=== RUN TestMultipartCleanup461=== PAUSE TestMultipartCleanup462=== RUN TestObjectStatsTrigger463=== PAUSE TestObjectStatsTrigger464=== RUN TestOrphanedObjectsGC465=== PAUSE TestOrphanedObjectsGC466=== RUN TestOrphanedObjectsGCStressTest467=== PAUSE TestOrphanedObjectsGCStressTest468=== RUN TestResurrectedObjectNotDeleted469=== PAUSE TestResurrectedObjectNotDeleted470=== RUN TestParseSingleRange471=== PAUSE TestParseSingleRange472=== RUN TestIsValidCachePath473=== PAUSE TestIsValidCachePath474=== RUN TestReadProxyNarinfo475=== PAUSE TestReadProxyNarinfo476=== RUN TestReadProxyNarinfoAlreadyDecompressed477=== PAUSE TestReadProxyNarinfoAlreadyDecompressed478=== RUN TestReadProxyNarStreaming479=== PAUSE TestReadProxyNarStreaming480=== RUN TestReadProxy404481=== PAUSE TestReadProxy404482=== RUN TestReadProxyInvalidPath483=== PAUSE TestReadProxyInvalidPath484=== RUN TestReadProxyHead485=== PAUSE TestReadProxyHead486=== RUN TestReadProxyConditionalGet487=== PAUSE TestReadProxyConditionalGet488=== RUN TestReadProxyRootRedirectsToIndexHTML489=== PAUSE TestReadProxyRootRedirectsToIndexHTML490=== RUN TestReadProxyDisabled491=== PAUSE TestReadProxyDisabled492=== RUN TestReadProxyRangeRequest493=== PAUSE TestReadProxyRangeRequest494=== RUN TestRedundantMultipartUpload495=== PAUSE TestRedundantMultipartUpload496=== RUN TestCompleteMultipartUpload_ErrorButObjectExists497=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists498=== RUN TestCompletedNarNotReofferedAcrossClosures499=== PAUSE TestCompletedNarNotReofferedAcrossClosures500=== RUN TestPresignedUploadRegisteredBeforeCommit501=== PAUSE TestPresignedUploadRegisteredBeforeCommit502=== RUN TestService_Rustfstest503=== PAUSE TestService_Rustfstest504=== RUN TestParseSize505=== PAUSE TestParseSize506=== RUN TestSkippedUploadsHandler507=== PAUSE TestSkippedUploadsHandler508=== RUN TestSystemdListenerNotActivated509--- PASS: TestSystemdListenerNotActivated (0.00s)510=== RUN TestWatchdogBeatsWhenHealthy511--- PASS: TestWatchdogBeatsWhenHealthy (0.02s)512=== RUN TestWatchdogSkipsWhenUnhealthy5132026/08/27 09:31:33 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5142026/08/27 09:31:33 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5152026/08/27 09:31:33 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5162026/08/27 09:31:33 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5172026/08/27 09:31:33 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5182026/08/27 09:31:33 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5192026/08/27 09:31:33 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5202026/08/27 09:31:33 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5212026/08/27 09:31:33 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5222026/08/27 09:31:33 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"523--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)524=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle525=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle526=== RUN TestProxyWriteTimeout527=== PAUSE TestProxyWriteTimeout528=== RUN TestIsValidUploadKey529=== PAUSE TestIsValidUploadKey530=== RUN TestUploadHandlersRejectInvalidKeys531=== PAUSE TestUploadHandlersRejectInvalidKeys532=== RUN TestUploadHandlersRejectOversizedBody533=== PAUSE TestUploadHandlersRejectOversizedBody534=== RUN TestService_cleanupPendingClosuresHandler535=== PAUSE TestService_cleanupPendingClosuresHandler536=== RUN TestService_createPendingClosureHandler537=== PAUSE TestService_createPendingClosureHandler538=== RUN TestService_verifyS3Integrity539=== PAUSE TestService_verifyS3Integrity540=== RUN TestCompleteMultipartUnregistered541=== PAUSE TestCompleteMultipartUnregistered542=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT543=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT544=== CONT TestService_createPendingClosureHandler545=== CONT TestService_verifyS3Integrity546=== CONT TestParseSize547=== CONT TestService_cleanupPendingClosuresHandler548=== CONT TestUploadHandlersRejectOversizedBody549=== CONT TestUploadHandlersRejectInvalidKeys550=== CONT TestIsValidUploadKey551=== CONT TestProxyWriteTimeout552=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle553=== CONT TestSkippedUploadsHandler554=== CONT TestCompleteMultipartUnregistered555=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT556=== CONT TestService_Rustfstest557=== CONT TestPresignedUploadRegisteredBeforeCommit558=== CONT TestCompletedNarNotReofferedAcrossClosures5592026/08/27 09:31:33 INFO Client skipped oversized paths paths=3 nar_bytes=5000000000560=== CONT TestCompleteMultipartUpload_ErrorButObjectExists561=== CONT TestRedundantMultipartUpload562=== CONT TestReadProxyRangeRequest563=== CONT TestReadProxyDisabled564=== CONT TestGracefulShutdownDrainsInflight565=== CONT TestGCTaskStore_Fail566=== CONT TestService_healthCheckHandler5672026/08/27 09:31:33 INFO Starting HTTP server address=127.0.0.1:45509568=== CONT TestGCTaskStore_PhaseUpdates569=== CONT TestGCTaskStore_ConflictDifferentParams570=== CONT TestReadProxyRootRedirectsToIndexHTML571=== CONT TestGCTaskStore_CompletedAllowsNewTask572=== CONT TestGCTaskStore_DeduplicateSameParams573=== CONT TestService_AuthMiddleware574--- PASS: TestParseSize (0.00s)575=== CONT TestGCTaskStore_GetReturnsLatest576=== CONT TestReadProxyHead577=== CONT TestGCTaskStore_GetEmpty578=== CONT TestGCTaskStore_StartNew579=== CONT TestReadProxyInvalidPath580=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info581=== RUN TestIsValidUploadKey/narinfo5822026/08/27 09:31:33 INFO Shutdown signal received, draining in-flight requests timeout=10s583=== PAUSE TestIsValidUploadKey/narinfo584=== RUN TestProxyWriteTimeout/narinfo585=== CONT TestReadProxyConditionalGet586=== PAUSE TestProxyWriteTimeout/narinfo587=== RUN TestProxyWriteTimeout/1_GiB_nar588=== PAUSE TestProxyWriteTimeout/1_GiB_nar589=== RUN TestProxyWriteTimeout/10_GiB_nar590=== PAUSE TestProxyWriteTimeout/10_GiB_nar591--- PASS: TestGCTaskStore_Fail (0.00s)592=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info593=== RUN TestIsValidUploadKey/nar_zst594=== PAUSE TestIsValidUploadKey/nar_zst595=== RUN TestIsValidUploadKey/nar_xz596=== PAUSE TestIsValidUploadKey/nar_xz597=== RUN TestIsValidUploadKey/nar_plain598=== PAUSE TestIsValidUploadKey/nar_plain599=== RUN TestProxyWriteTimeout/unknown_size600--- PASS: TestSkippedUploadsHandler (0.01s)601=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal602=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal603=== RUN TestIsValidUploadKey/listing604=== PAUSE TestIsValidUploadKey/listing605=== PAUSE TestProxyWriteTimeout/unknown_size606--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)607--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)608--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)609--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)610--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)611--- PASS: TestGCTaskStore_GetEmpty (0.00s)612--- PASS: TestGCTaskStore_StartNew (0.00s)613=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key614=== CONT TestGCMetrics615=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key616=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key617=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key618=== CONT TestReadProxy404619=== RUN TestIsValidUploadKey/build_log620=== PAUSE TestIsValidUploadKey/build_log621=== RUN TestIsValidUploadKey/build_log_home-manager_file622=== PAUSE TestIsValidUploadKey/build_log_home-manager_file623=== RUN TestIsValidUploadKey/build_log_plus_in_name624=== PAUSE TestIsValidUploadKey/build_log_plus_in_name625=== RUN TestIsValidUploadKey/build_log_question_mark626=== PAUSE TestIsValidUploadKey/build_log_question_mark627=== RUN TestIsValidUploadKey/build_log_equals628=== PAUSE TestIsValidUploadKey/build_log_equals629=== RUN TestIsValidUploadKey/realisation630=== PAUSE TestIsValidUploadKey/realisation631=== RUN TestIsValidUploadKey/realisation_plus_in_output632=== PAUSE TestIsValidUploadKey/realisation_plus_in_output633=== RUN TestIsValidUploadKey/nix-cache-info634=== PAUSE TestIsValidUploadKey/nix-cache-info635=== RUN TestIsValidUploadKey/index.html636=== PAUSE TestIsValidUploadKey/index.html637=== RUN TestIsValidUploadKey/narinfo_key,_nar_type638=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type639=== RUN TestIsValidUploadKey/nar_key,_narinfo_type640=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type641=== RUN TestIsValidUploadKey/listing_key,_narinfo_type642=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type643=== RUN TestIsValidUploadKey/traversal644=== PAUSE TestIsValidUploadKey/traversal645=== RUN TestIsValidUploadKey/traversal_nar646=== PAUSE TestIsValidUploadKey/traversal_nar647=== RUN TestIsValidUploadKey/absolute648=== PAUSE TestIsValidUploadKey/absolute649=== RUN TestIsValidUploadKey/empty_key650=== PAUSE TestIsValidUploadKey/empty_key651=== RUN TestIsValidUploadKey/unknown_type652=== PAUSE TestIsValidUploadKey/unknown_type653=== CONT TestGCBugBareHashReferences6542026-08-27 09:31:33.942 UTC [572] ERROR: relation "goose_db_version" does not exist at character 366552026-08-27 09:31:33.942 UTC [572] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6562026-08-27 09:31:33.943 UTC [575] ERROR: relation "goose_db_version" does not exist at character 366572026-08-27 09:31:33.943 UTC [575] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6582026-08-27 09:31:33.943 UTC [578] ERROR: relation "goose_db_version" does not exist at character 366592026-08-27 09:31:33.943 UTC [578] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6602026-08-27 09:31:33.953 UTC [579] ERROR: relation "goose_db_version" does not exist at character 366612026-08-27 09:31:33.953 UTC [579] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC662=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure663=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure664=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart665=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart666=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts667=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts668=== CONT TestReadProxyNarStreaming6692026-08-27 09:31:34.068 UTC [597] ERROR: relation "goose_db_version" does not exist at character 366702026-08-27 09:31:34.068 UTC [597] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6712026-08-27 09:31:34.069 UTC [596] ERROR: relation "goose_db_version" does not exist at character 366722026-08-27 09:31:34.069 UTC [596] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6732026-08-27 09:31:34.069 UTC [598] ERROR: relation "goose_db_version" does not exist at character 366742026-08-27 09:31:34.069 UTC [598] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6752026-08-27 09:31:34.069 UTC [599] ERROR: relation "goose_db_version" does not exist at character 366762026-08-27 09:31:34.069 UTC [599] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6772026/08/27 09:31:34 OK 20241026095416_initial_model.sql (106.89ms)6782026-08-27 09:31:34.070 UTC [600] ERROR: relation "goose_db_version" does not exist at character 366792026-08-27 09:31:34.070 UTC [600] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6802026-08-27 09:31:34.071 UTC [603] ERROR: relation "goose_db_version" does not exist at character 366812026-08-27 09:31:34.071 UTC [603] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6822026/08/27 09:31:34 OK 20241026095416_initial_model.sql (109.69ms)6832026-08-27 09:31:34.071 UTC [601] ERROR: relation "goose_db_version" does not exist at character 366842026-08-27 09:31:34.071 UTC [601] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6852026-08-27 09:31:34.072 UTC [602] ERROR: relation "goose_db_version" does not exist at character 366862026-08-27 09:31:34.072 UTC [602] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6872026-08-27 09:31:34.072 UTC [604] ERROR: relation "goose_db_version" does not exist at character 366882026-08-27 09:31:34.072 UTC [604] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6892026-08-27 09:31:34.073 UTC [605] ERROR: relation "goose_db_version" does not exist at character 366902026-08-27 09:31:34.073 UTC [605] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6912026/08/27 09:31:34 OK 20241026095416_initial_model.sql (110.39ms)6922026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (4.2ms)6932026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (4.87ms)6942026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (4.09ms)695--- PASS: TestGracefulShutdownDrainsInflight (0.27s)696=== CONT TestPinProtectsFromGC6972026/08/27 09:31:34 OK 20251218171726_add_pins.sql (16.34ms)6982026/08/27 09:31:34 OK 20251218171726_add_pins.sql (18.81ms)6992026/08/27 09:31:34 OK 20241026095416_initial_model.sql (37.65ms)7002026/08/27 09:31:34 OK 20251218171726_add_pins.sql (19.01ms)7012026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (4.37ms)7022026/08/27 09:31:34 OK 20241026095416_initial_model.sql (27.89ms)7032026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (9.45ms)7042026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (11.27ms)7052026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200007062026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200007072026/08/27 09:31:34 OK 20241026095416_initial_model.sql (29.4ms)7082026/08/27 09:31:34 OK 20241026095416_initial_model.sql (28.67ms)7092026/08/27 09:31:34 OK 20241026095416_initial_model.sql (29.67ms)7102026/08/27 09:31:34 OK 20241026095416_initial_model.sql (30.62ms)7112026/08/27 09:31:34 OK 20251218171726_add_pins.sql (7.1ms)7122026/08/27 09:31:34 OK 20241026095416_initial_model.sql (29.67ms)7132026/08/27 09:31:34 OK 20241026095416_initial_model.sql (30.14ms)7142026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (9.72ms)7152026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200007162026/08/27 09:31:34 OK 20241026095416_initial_model.sql (31.05ms)7172026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (4.53ms)7182026/08/27 09:31:34 OK 20241026095416_initial_model.sql (32.21ms)7192026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (4.33ms)7202026/08/27 09:31:34 OK 20241026095416_initial_model.sql (32.35ms)7212026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (3.27ms)7222026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (3.3ms)7232026/08/27 09:31:34 OK 1_commit_pending_closure.sql (5.12ms)7242026/08/27 09:31:34 OK 1_commit_pending_closure.sql (5.45ms)7252026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (2.83ms)7262026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (4.39ms)7272026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (3.37ms)7282026/08/27 09:31:34 OK 1_commit_pending_closure.sql (3.85ms)7292026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (4.19ms)7302026/08/27 09:31:34 OK 2_object_stats_trigger.sql (2.75ms)7312026/08/27 09:31:34 goose: up to current file version: 27322026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (3.98ms)7332026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (4.29ms)7342026/08/27 09:31:34 OK 2_object_stats_trigger.sql (4.48ms)7352026/08/27 09:31:34 OK 20251218171726_add_pins.sql (6.83ms)7362026/08/27 09:31:34 OK 20251218171726_add_pins.sql (5.3ms)7372026/08/27 09:31:34 goose: up to current file version: 27382026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (7.25ms)7392026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200007402026/08/27 09:31:34 OK 2_object_stats_trigger.sql (4.3ms)7412026/08/27 09:31:34 goose: up to current file version: 27422026/08/27 09:31:34 OK 20251218171726_add_pins.sql (6.31ms)7432026/08/27 09:31:34 OK 20251218171726_add_pins.sql (7.68ms)7442026/08/27 09:31:34 OK 20251218171726_add_pins.sql (6.91ms)7452026/08/27 09:31:34 OK 20251218171726_add_pins.sql (7.08ms)7462026/08/27 09:31:34 OK 20251218171726_add_pins.sql (5.6ms)7472026/08/27 09:31:34 OK 20251218171726_add_pins.sql (6.73ms)7482026/08/27 09:31:34 OK 20251218171726_add_pins.sql (5.65ms)7492026/08/27 09:31:34 OK 20251218171726_add_pins.sql (5.75ms)7502026/08/27 09:31:34 OK 1_commit_pending_closure.sql (5.74ms)7512026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (6.25ms)7522026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200007532026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (6.08ms)7542026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200007552026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (7.78ms)7562026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200007572026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (6.72ms)7582026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200007592026/08/27 09:31:34 OK 2_object_stats_trigger.sql (3.23ms)7602026/08/27 09:31:34 goose: up to current file version: 27612026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (6.63ms)7622026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200007632026/08/27 09:31:34 OK 1_commit_pending_closure.sql (4.24ms)7642026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (6.39ms)7652026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200007662026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (8.21ms)7672026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200007682026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (8.18ms)7692026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200007702026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (8.25ms)7712026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200007722026/08/27 09:31:34 OK 1_commit_pending_closure.sql (4.35ms)7732026/08/27 09:31:34 OK 1_commit_pending_closure.sql (4.26ms)7742026/08/27 09:31:34 OK 1_commit_pending_closure.sql (3.99ms)7752026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (9.02ms)7762026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200007772026/08/27 09:31:34 OK 2_object_stats_trigger.sql (3.98ms)7782026/08/27 09:31:34 goose: up to current file version: 27792026/08/27 09:31:34 OK 1_commit_pending_closure.sql (3.89ms)7802026/08/27 09:31:34 OK 1_commit_pending_closure.sql (5.46ms)7812026/08/27 09:31:34 OK 2_object_stats_trigger.sql (4.1ms)7822026/08/27 09:31:34 goose: up to current file version: 27832026/08/27 09:31:34 OK 2_object_stats_trigger.sql (4.04ms)7842026/08/27 09:31:34 goose: up to current file version: 27852026/08/27 09:31:34 OK 2_object_stats_trigger.sql (2.83ms)7862026/08/27 09:31:34 goose: up to current file version: 27872026/08/27 09:31:34 OK 1_commit_pending_closure.sql (4.94ms)7882026/08/27 09:31:34 OK 1_commit_pending_closure.sql (4.84ms)7892026/08/27 09:31:34 OK 1_commit_pending_closure.sql (5.15ms)7902026-08-27 09:31:34.135 UTC [609] ERROR: relation "goose_db_version" does not exist at character 367912026-08-27 09:31:34.135 UTC [609] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7922026/08/27 09:31:34 OK 1_commit_pending_closure.sql (3.65ms)7932026/08/27 09:31:34 OK 2_object_stats_trigger.sql (2.11ms)7942026/08/27 09:31:34 goose: up to current file version: 27952026-08-27 09:31:34.137 UTC [610] ERROR: relation "goose_db_version" does not exist at character 367962026-08-27 09:31:34.137 UTC [610] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC797{"timestamp":"2026-08-27T09:31:34.137142688Z","level":"ERROR","duration":"381.363µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}798{"timestamp":"2026-08-27T09:31:34.137503072Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e0fb4eb2-812c-4218-a11d-f88249e15c6a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket10/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}7992026/08/27 09:31:34 OK 2_object_stats_trigger.sql (2.85ms)8002026/08/27 09:31:34 goose: up to current file version: 28012026/08/27 09:31:34 OK 2_object_stats_trigger.sql (3.1ms)8022026/08/27 09:31:34 goose: up to current file version: 28032026/08/27 09:31:34 OK 2_object_stats_trigger.sql (3.78ms)8042026/08/27 09:31:34 goose: up to current file version: 28052026/08/27 09:31:34 OK 2_object_stats_trigger.sql (3.87ms)8062026/08/27 09:31:34 goose: up to current file version: 28072026-08-27 09:31:34.139 UTC [612] ERROR: relation "goose_db_version" does not exist at character 368082026-08-27 09:31:34.139 UTC [612] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC8092026-08-27 09:31:34.140 UTC [611] ERROR: relation "goose_db_version" does not exist at character 368102026-08-27 09:31:34.140 UTC [611] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC8112026-08-27 09:31:34.140 UTC [613] ERROR: relation "goose_db_version" does not exist at character 368122026-08-27 09:31:34.140 UTC [613] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC8132026/08/27 09:31:34 OK 2_object_stats_trigger.sql (7.49ms)8142026/08/27 09:31:34 goose: up to current file version: 2815{"timestamp":"2026-08-27T09:31:34.141239326Z","level":"ERROR","duration":"297.863µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}816{"timestamp":"2026-08-27T09:31:34.141274006Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"1f0fbcd0-b793-49c6-9f56-ecc37a9134e1","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket13/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}817{"timestamp":"2026-08-27T09:31:34.141411107Z","level":"ERROR","duration":"429.904µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(232)"}818{"timestamp":"2026-08-27T09:31:34.141470388Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ca7e0209-fb30-4e70-9d47-e4230738baef","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket14/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(232)"}8192026-08-27 09:31:34.141 UTC [614] ERROR: relation "goose_db_version" does not exist at character 368202026-08-27 09:31:34.141 UTC [614] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC821{"timestamp":"2026-08-27T09:31:34.142864981Z","level":"ERROR","duration":"3.639654ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}822{"timestamp":"2026-08-27T09:31:34.142937161Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2b9c18fc-16b1-4f60-aca1-2b169d457a99","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":3,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}823{"timestamp":"2026-08-27T09:31:34.142712099Z","level":"ERROR","duration":"1.754256ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"}824{"timestamp":"2026-08-27T09:31:34.143580087Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f0b8ba12-1d46-4ea6-83b1-508ac1152ce5","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket12/","status_code":503,"duration_ms":2,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"}825{"timestamp":"2026-08-27T09:31:34.145041801Z","level":"ERROR","duration":"1.601675ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}826{"timestamp":"2026-08-27T09:31:34.145120241Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"cc13b040-7432-4b33-99b4-2cc20461f9bd","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket15/","status_code":503,"duration_ms":1,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}8272026/08/27 09:31:34 INFO Received cleanup request method=DELETE path=/api/pending_closures8282026-08-27 09:31:34.149 UTC [615] ERROR: relation "goose_db_version" does not exist at character 368292026-08-27 09:31:34.149 UTC [615] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC8302026/08/27 09:31:34 INFO Aborted multipart uploads count=08312026/08/27 09:31:34 INFO Received uploads request method=POST path=/api/pending_closures8322026/08/27 09:31:34 INFO Received uploads request method=POST path=/api/pending_closures8332026-08-27 09:31:34.156 UTC [616] ERROR: relation "goose_db_version" does not exist at character 368342026-08-27 09:31:34.156 UTC [616] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC8352026/08/27 09:31:34 OK 20241026095416_initial_model.sql (14.93ms)8362026/08/27 09:31:34 INFO Received uploads request method=POST path=/api/pending_closures8372026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (4.07ms)8382026/08/27 09:31:34 OK 20251218171726_add_pins.sql (4.51ms)8392026/08/27 09:31:34 OK 20241026095416_initial_model.sql (12.13ms)8402026/08/27 09:31:34 OK 20241026095416_initial_model.sql (13.48ms)8412026/08/27 09:31:34 OK 20241026095416_initial_model.sql (15.64ms)8422026/08/27 09:31:34 OK 20241026095416_initial_model.sql (14.77ms)8432026/08/27 09:31:34 OK 20241026095416_initial_model.sql (14.66ms)8442026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (3.05ms)845{"timestamp":"2026-08-27T09:31:34.170785936Z","level":"ERROR","duration":"243.702µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(381)"}846{"timestamp":"2026-08-27T09:31:34.170877617Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"5ac15110-fb77-45a2-8674-a567da9f0d0e","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket11/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(381)"}8472026/08/27 09:31:34 OK 20241026095416_initial_model.sql (15.62ms)8482026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (3.65ms)8492026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (3.61ms)8502026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (7.44ms)8512026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200008522026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (4.55ms)8532026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (4.75ms)8542026/08/27 09:31:34 INFO Received complete multipart upload request method=POST path=/api/multipart/complete8552026/08/27 09:31:34 OK 20251218171726_add_pins.sql (5.58ms)8562026/08/27 09:31:34 INFO Received cleanup request method=DELETE path=/api/pending_closures8572026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (4.08ms)858--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (0.36s)859=== CONT TestReadProxyNarinfoAlreadyDecompressed8602026/08/27 09:31:34 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst8612026/08/27 09:31:34 OK 20241026095416_initial_model.sql (11.71ms)862--- PASS: TestCompleteMultipartUnregistered (0.36s)8632026/08/27 09:31:34 OK 20251218171726_add_pins.sql (5.25ms)864=== CONT TestClientWithDependencies8652026/08/27 09:31:34 OK 1_commit_pending_closure.sql (3.83ms)8662026/08/27 09:31:34 OK 20251218171726_add_pins.sql (6.2ms)8672026/08/27 09:31:34 INFO Aborted multipart uploads count=18682026/08/27 09:31:34 OK 20251218171726_add_pins.sql (6.41ms)8692026-08-27 09:31:34.182 UTC [621] ERROR: relation "goose_db_version" does not exist at character 368702026-08-27 09:31:34.182 UTC [621] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC8712026-08-27 09:31:34.183 UTC [620] ERROR: relation "goose_db_version" does not exist at character 368722026-08-27 09:31:34.183 UTC [620] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC8732026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (7.35ms)8742026/08/27 09:31:34 OK 20251218171726_add_pins.sql (8.41ms)8752026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200008762026/08/27 09:31:34 OK 20251218171726_add_pins.sql (7.01ms)8772026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (6ms)8782026/08/27 09:31:34 OK 2_object_stats_trigger.sql (4.71ms)8792026/08/27 09:31:34 goose: up to current file version: 28802026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (6.03ms)8812026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200008822026/08/27 09:31:34 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete8832026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (7.35ms)8842026/08/27 09:31:34 goose: successfully migrated database to version: 20260628120000885{"timestamp":"2026-08-27T09:31:34.186377139Z","level":"ERROR","duration":"640.106µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}8862026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (5.47ms)887{"timestamp":"2026-08-27T09:31:34.187445689Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"affed9b7-3e91-4581-b378-e2dfc7e190fe","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket16/","status_code":503,"duration_ms":1,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}8882026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200008892026-08-27 09:31:34.187 UTC [578] ERROR: Closure does not exist: id=18902026-08-27 09:31:34.187 UTC [578] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 16 at RAISE8912026-08-27 09:31:34.187 UTC [578] STATEMENT: -- name: CommitPendingClosure :exec892 SELECT commit_pending_closure($1::bigint)893 894--- PASS: TestService_cleanupPendingClosuresHandler (0.37s)895=== CONT TestReadProxyNarinfo8962026/08/27 09:31:34 OK 1_commit_pending_closure.sql (5.31ms)8972026/08/27 09:31:34 OK 1_commit_pending_closure.sql (4.76ms)8982026/08/27 09:31:34 OK 20251218171726_add_pins.sql (7.44ms)8992026/08/27 09:31:34 OK 1_commit_pending_closure.sql (4.06ms)9002026/08/27 09:31:34 OK 2_object_stats_trigger.sql (2.2ms)9012026/08/27 09:31:34 goose: up to current file version: 29022026/08/27 09:31:34 OK 1_commit_pending_closure.sql (6.06ms)9032026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (7.24ms)9042026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200009052026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (7.52ms)9062026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200009072026/08/27 09:31:34 OK 2_object_stats_trigger.sql (1.11ms)9082026/08/27 09:31:34 goose: up to current file version: 29092026/08/27 09:31:34 OK 2_object_stats_trigger.sql (2.53ms)9102026/08/27 09:31:34 goose: up to current file version: 2911{"timestamp":"2026-08-27T09:31:34.194429093Z","level":"ERROR","duration":"566.445µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}912{"timestamp":"2026-08-27T09:31:34.194562654Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2eb9a96d-be25-44c1-a1d4-673558d06935","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket17/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}9132026/08/27 09:31:34 OK 2_object_stats_trigger.sql (3.2ms)914{"timestamp":"2026-08-27T09:31:34.194646975Z","level":"ERROR","duration":"232.322µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(299)"}9152026/08/27 09:31:34 goose: up to current file version: 2916{"timestamp":"2026-08-27T09:31:34.194704036Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"928fcf4e-5d07-4c6d-96ef-d84b3fd29ab4","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket16/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(299)"}917{"timestamp":"2026-08-27T09:31:34.195349121Z","level":"ERROR","duration":"480.744µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(232)"}918{"timestamp":"2026-08-27T09:31:34.195418442Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8c8d4441-9648-45a0-802f-a57f337612c0","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(232)"}919{"timestamp":"2026-08-27T09:31:34.19624607Z","level":"ERROR","duration":"596.806µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}920{"timestamp":"2026-08-27T09:31:34.196336471Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"90682167-3f9f-4608-aaed-e7751f6f51db","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket19/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}921{"timestamp":"2026-08-27T09:31:34.196395191Z","level":"ERROR","duration":"473.624µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(299)"}922{"timestamp":"2026-08-27T09:31:34.196456272Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"37bbe931-1edf-4680-a94e-dfb4e03f45ec","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(299)"}9232026/08/27 09:31:34 OK 1_commit_pending_closure.sql (4.36ms)9242026/08/27 09:31:34 OK 1_commit_pending_closure.sql (6.06ms)9252026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (7.58ms)9262026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200009272026/08/27 09:31:34 OK 2_object_stats_trigger.sql (4.41ms)9282026/08/27 09:31:34 goose: up to current file version: 29292026/08/27 09:31:34 OK 2_object_stats_trigger.sql (5.11ms)9302026/08/27 09:31:34 goose: up to current file version: 29312026/08/27 09:31:34 OK 1_commit_pending_closure.sql (4.97ms)932{"timestamp":"2026-08-27T09:31:34.205754457Z","level":"ERROR","duration":"613.766µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}933{"timestamp":"2026-08-27T09:31:34.205841278Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"01099e46-230a-4c11-8409-779c8fd800e0","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}934{"timestamp":"2026-08-27T09:31:34.206390703Z","level":"ERROR","duration":"579.866µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}935{"timestamp":"2026-08-27T09:31:34.206511344Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8eec05ae-a7a9-4cef-bd78-52572089713c","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket22/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}936{"timestamp":"2026-08-27T09:31:34.206613105Z","level":"ERROR","duration":"193.282µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}937{"timestamp":"2026-08-27T09:31:34.206669705Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"7b20a397-0cbe-4b31-80f6-8324b2dd084a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}9382026/08/27 09:31:34 OK 2_object_stats_trigger.sql (4.17ms)9392026/08/27 09:31:34 goose: up to current file version: 2940{"timestamp":"2026-08-27T09:31:34.210767783Z","level":"ERROR","duration":"444.265µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}941{"timestamp":"2026-08-27T09:31:34.210821403Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"b807337f-9284-4924-b22b-139646ef4f7f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket23/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}9422026/08/27 09:31:34 OK 20241026095416_initial_model.sql (17.77ms)9432026/08/27 09:31:34 OK 20241026095416_initial_model.sql (18.91ms)9442026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (5.01ms)9452026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (7.92ms)9462026/08/27 09:31:34 OK 20251218171726_add_pins.sql (6.75ms)9472026/08/27 09:31:34 OK 20251218171726_add_pins.sql (6.64ms)9482026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (6.91ms)9492026/08/27 09:31:34 goose: successfully migrated database to version: 20260628120000950{"timestamp":"2026-08-27T09:31:34.234644182Z","level":"ERROR","duration":"208.822µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}951{"timestamp":"2026-08-27T09:31:34.234701902Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"174bbe4b-66a9-4ab7-b81e-a5c5afca3e58","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket17/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}9522026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (8.56ms)9532026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200009542026/08/27 09:31:34 OK 1_commit_pending_closure.sql (6.43ms)9552026/08/27 09:31:34 OK 1_commit_pending_closure.sql (4.75ms)956{"timestamp":"2026-08-27T09:31:34.241467024Z","level":"ERROR","duration":"225.962µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}957{"timestamp":"2026-08-27T09:31:34.241559185Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"9c77039a-b9bc-4f10-ac6d-0e8d1378c950","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket19/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}9582026/08/27 09:31:34 OK 2_object_stats_trigger.sql (4.3ms)9592026/08/27 09:31:34 goose: up to current file version: 2960{"timestamp":"2026-08-27T09:31:34.242303872Z","level":"ERROR","duration":"180.722µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}961{"timestamp":"2026-08-27T09:31:34.242372552Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"956f7a29-cb13-41ee-a797-60ed7318ae71","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket14/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}9622026/08/27 09:31:34 OK 2_object_stats_trigger.sql (3.15ms)9632026/08/27 09:31:34 goose: up to current file version: 2964{"timestamp":"2026-08-27T09:31:34.24434159Z","level":"ERROR","duration":"435.744µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}965{"timestamp":"2026-08-27T09:31:34.244397351Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"d144a19b-aa7e-49d7-95ba-276e061b8563","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket24/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}966{"timestamp":"2026-08-27T09:31:34.246226288Z","level":"ERROR","duration":"453.305µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}967{"timestamp":"2026-08-27T09:31:34.246284948Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"c8642599-cccc-4e20-ae5c-a60b3ee58ba5","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket25/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}968{"timestamp":"2026-08-27T09:31:34.250969071Z","level":"ERROR","duration":"135.921µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}969{"timestamp":"2026-08-27T09:31:34.251007811Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a7aa3f5c-8d08-4c0a-887b-8c1cda9a3d32","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}9702026-08-27 09:31:34.266 UTC [628] ERROR: relation "goose_db_version" does not exist at character 369712026-08-27 09:31:34.266 UTC [628] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC9722026-08-27 09:31:34.267 UTC [629] ERROR: relation "goose_db_version" does not exist at character 369732026-08-27 09:31:34.267 UTC [629] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC9742026-08-27 09:31:34.283 UTC [630] ERROR: relation "goose_db_version" does not exist at character 369752026-08-27 09:31:34.283 UTC [630] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC9762026/08/27 09:31:34 OK 20241026095416_initial_model.sql (12.41ms)9772026/08/27 09:31:34 OK 20241026095416_initial_model.sql (13.26ms)9782026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (2.13ms)9792026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (1.67ms)9802026/08/27 09:31:34 OK 20251218171726_add_pins.sql (4.4ms)9812026/08/27 09:31:34 OK 20251218171726_add_pins.sql (7.95ms)982{"timestamp":"2026-08-27T09:31:34.297571798Z","level":"ERROR","duration":"268.443µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}983{"timestamp":"2026-08-27T09:31:34.297664179Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"720168f2-930a-4b47-b21a-f17ae47d293a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket25/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}9842026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (5.9ms)9852026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200009862026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (4.54ms)9872026/08/27 09:31:34 goose: successfully migrated database to version: 202606281200009882026/08/27 09:31:34 OK 1_commit_pending_closure.sql (3.55ms)9892026/08/27 09:31:34 OK 20241026095416_initial_model.sql (12.44ms)9902026/08/27 09:31:34 OK 2_object_stats_trigger.sql (1.18ms)9912026/08/27 09:31:34 goose: up to current file version: 29922026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (1.7ms)9932026/08/27 09:31:34 OK 1_commit_pending_closure.sql (2.28ms)9942026/08/27 09:31:34 OK 2_object_stats_trigger.sql (1.2ms)995{"timestamp":"2026-08-27T09:31:34.305009626Z","level":"ERROR","duration":"690.026µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}9962026/08/27 09:31:34 goose: up to current file version: 2997{"timestamp":"2026-08-27T09:31:34.305107687Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4570554c-e5b7-4d8c-9318-29930f14d661","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}998{"timestamp":"2026-08-27T09:31:34.306088996Z","level":"ERROR","duration":"182.542µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}999{"timestamp":"2026-08-27T09:31:34.306165517Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"209bd8cc-0805-468d-9714-9136da5832ad","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket17/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}10002026/08/27 09:31:34 OK 20251218171726_add_pins.sql (3.21ms)1001{"timestamp":"2026-08-27T09:31:34.306878963Z","level":"ERROR","duration":"582.045µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1002{"timestamp":"2026-08-27T09:31:34.306952144Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4203ad5c-501c-436d-86dc-3d7850bd494a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}10032026/08/27 09:31:34 OK 20260628120000_add_object_size_and_stats.sql (3.37ms)10042026/08/27 09:31:34 goose: successfully migrated database to version: 2026062812000010052026/08/27 09:31:34 OK 1_commit_pending_closure.sql (2.23ms)10062026/08/27 09:31:34 OK 2_object_stats_trigger.sql (998.55µs)10072026/08/27 09:31:34 goose: up to current file version: 21008{"timestamp":"2026-08-27T09:31:34.315426181Z","level":"ERROR","duration":"416.184µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1009{"timestamp":"2026-08-27T09:31:34.315474702Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a7afed48-c4ff-4594-9cc4-58fc9b11c252","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1010{"timestamp":"2026-08-27T09:31:34.336671116Z","level":"ERROR","duration":"236.722µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1011{"timestamp":"2026-08-27T09:31:34.336774037Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"9fa432de-7f64-4ce4-9e0d-5ce51e35fb16","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket13/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1012{"timestamp":"2026-08-27T09:31:34.34149496Z","level":"ERROR","duration":"146.001µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1013{"timestamp":"2026-08-27T09:31:34.341550961Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"6613d60a-d2f3-4565-aed1-c8201f62f26e","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket23/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1014{"timestamp":"2026-08-27T09:31:34.348196302Z","level":"ERROR","duration":"128.462µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1015{"timestamp":"2026-08-27T09:31:34.348253742Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"62d9b090-94a5-4f37-9aef-090cabd7b50e","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1016{"timestamp":"2026-08-27T09:31:34.348506024Z","level":"ERROR","duration":"97.22µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1017{"timestamp":"2026-08-27T09:31:34.348556705Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ae3f2be9-339b-4211-b760-f279648a57de","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1018{"timestamp":"2026-08-27T09:31:34.350055619Z","level":"ERROR","duration":"99.161µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1019{"timestamp":"2026-08-27T09:31:34.350099619Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"095a6260-64c7-47bf-92ea-54aa4b41e23a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket22/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1020{"timestamp":"2026-08-27T09:31:34.352821904Z","level":"ERROR","duration":"212.282µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1021{"timestamp":"2026-08-27T09:31:34.352897945Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"91fefc82-ebf5-4515-96ae-ee1d95a94a27","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1022{"timestamp":"2026-08-27T09:31:34.358447995Z","level":"ERROR","duration":"128.421µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1023{"timestamp":"2026-08-27T09:31:34.358571396Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"1fd2bfc2-8c35-444c-8c94-e377ec7fe206","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket24/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1024{"timestamp":"2026-08-27T09:31:34.375623813Z","level":"ERROR","duration":"225.342µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1025{"timestamp":"2026-08-27T09:31:34.375717534Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"575c6dc4-11d6-4692-9644-a2d47afd1876","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1026{"timestamp":"2026-08-27T09:31:34.420817707Z","level":"ERROR","duration":"291.603µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1027{"timestamp":"2026-08-27T09:31:34.420888787Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4f0b5ed7-1c6b-4668-a7ee-acee7504981e","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket24/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1028{"timestamp":"2026-08-27T09:31:34.464039883Z","level":"ERROR","duration":"242.063µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1029{"timestamp":"2026-08-27T09:31:34.464145484Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"2ef019db-7f37-4c6a-8765-2d58c421a2be","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket16/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1030{"timestamp":"2026-08-27T09:31:34.46487481Z","level":"ERROR","duration":"137.521µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1031{"timestamp":"2026-08-27T09:31:34.464930971Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"96815650-b035-482e-9d4e-d9af6b2d20a4","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket14/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1032{"timestamp":"2026-08-27T09:31:34.466697747Z","level":"ERROR","duration":"135.121µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1033{"timestamp":"2026-08-27T09:31:34.466751847Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0f68d620-e051-43cd-8784-294763bf14ff","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket18/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1034{"timestamp":"2026-08-27T09:31:34.511738399Z","level":"ERROR","duration":"157.641µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1035{"timestamp":"2026-08-27T09:31:34.51180344Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"524cc3e1-4d84-47c0-8ead-da9b0ffcaf86","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1036{"timestamp":"2026-08-27T09:31:34.512028502Z","level":"ERROR","duration":"61.6µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1037{"timestamp":"2026-08-27T09:31:34.512054642Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"d4429547-547f-4be1-895c-b31363197e00","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket25/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1038{"timestamp":"2026-08-27T09:31:34.528923577Z","level":"ERROR","duration":"149.541µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1039{"timestamp":"2026-08-27T09:31:34.528973377Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"87cb3588-6908-4257-a619-41d9dc4a85a4","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket17/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1040{"timestamp":"2026-08-27T09:31:34.570191075Z","level":"ERROR","duration":"233.562µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1041{"timestamp":"2026-08-27T09:31:34.570277536Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0131e7ee-2098-4be8-878f-44b9c1c63ae8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket22/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1042{"timestamp":"2026-08-27T09:31:34.572054712Z","level":"ERROR","duration":"120.801µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1043{"timestamp":"2026-08-27T09:31:34.572106013Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e9657cc8-8b1e-447c-84e4-43a2f5b5397b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket13/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1044{"timestamp":"2026-08-27T09:31:34.578873914Z","level":"ERROR","duration":"104.201µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1045{"timestamp":"2026-08-27T09:31:34.578919255Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"d6a0cadb-d94a-49f5-badd-c9f9cf5c29d0","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket17/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1046{"timestamp":"2026-08-27T09:31:34.583595958Z","level":"ERROR","duration":"139.942µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1047{"timestamp":"2026-08-27T09:31:34.583667758Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"29825a2d-4b6c-4398-9216-a8e35775ae04","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1048{"timestamp":"2026-08-27T09:31:34.628780351Z","level":"ERROR","duration":"231.702µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1049{"timestamp":"2026-08-27T09:31:34.628871892Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"9988e131-7faa-472b-bbb1-c30df6366497","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket19/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1050{"timestamp":"2026-08-27T09:31:34.663895913Z","level":"ERROR","duration":"233.002µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1051{"timestamp":"2026-08-27T09:31:34.663994194Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"811feee1-4233-43d2-a06e-e203665f5fdb","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1052{"timestamp":"2026-08-27T09:31:34.66791037Z","level":"ERROR","duration":"211.442µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1053{"timestamp":"2026-08-27T09:31:34.667995271Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"51cad47b-436e-4ea0-9ac8-fad83322b424","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket27/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1054{"timestamp":"2026-08-27T09:31:34.680486865Z","level":"ERROR","duration":"546.725µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1055{"timestamp":"2026-08-27T09:31:34.680574426Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4f643aaa-fcef-4649-9545-735e5073af9d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket23/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1056{"timestamp":"2026-08-27T09:31:34.689653429Z","level":"ERROR","duration":"241.682µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1057{"timestamp":"2026-08-27T09:31:34.68974163Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"d8fb0f86-f5e3-42a7-8151-1f1232adc0fb","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket28/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1058{"timestamp":"2026-08-27T09:31:34.775211193Z","level":"ERROR","duration":"263.683µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}1059{"timestamp":"2026-08-27T09:31:34.775296933Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4ed600b8-ac54-4893-a167-efa7002ea010","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(319)"}10602026/08/27 09:31:34 INFO Received uploads request method=POST path=/api/pending_closures10612026/08/27 09:31:34 INFO Received uploads request method=POST path=/api/pending_closures1062--- PASS: TestReadProxyRangeRequest (1.08s)1063=== CONT TestClientMultipleUploads10642026/08/27 09:31:34 INFO Received uploads request method=POST path=/api/pending_closures10652026/08/27 09:31:34 INFO Received uploads request method=POST path=/api/pending_closures10662026/08/27 09:31:34 INFO Received uploads request method=POST path=/api/pending_closures10672026/08/27 09:31:34 INFO Received uploads request method=POST path=/api/pending_closures10682026/08/27 09:31:34 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst10692026/08/27 09:31:34 INFO Received uploads request method=POST path=/api/pending_closures1070--- PASS: TestPresignedUploadRegisteredBeforeCommit (1.13s)1071=== CONT TestIsValidCachePath1072=== RUN TestIsValidCachePath/narinfo1073=== PAUSE TestIsValidCachePath/narinfo1074=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars1075=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars1076=== RUN TestIsValidCachePath/nar_zst1077=== PAUSE TestIsValidCachePath/nar_zst1078=== RUN TestIsValidCachePath/nar_xz1079=== PAUSE TestIsValidCachePath/nar_xz1080=== RUN TestIsValidCachePath/nar_bz21081=== PAUSE TestIsValidCachePath/nar_bz21082=== RUN TestIsValidCachePath/nar_uncompressed1083=== PAUSE TestIsValidCachePath/nar_uncompressed1084=== RUN TestIsValidCachePath/ls1085=== PAUSE TestIsValidCachePath/ls1086=== RUN TestIsValidCachePath/log1087=== PAUSE TestIsValidCachePath/log1088=== RUN TestIsValidCachePath/realisation1089=== PAUSE TestIsValidCachePath/realisation1090=== RUN TestIsValidCachePath/nix-cache-info1091=== PAUSE TestIsValidCachePath/nix-cache-info1092=== RUN TestIsValidCachePath/index.html1093=== PAUSE TestIsValidCachePath/index.html1094=== RUN TestIsValidCachePath/traversal_parent1095=== PAUSE TestIsValidCachePath/traversal_parent1096=== RUN TestIsValidCachePath/traversal_in_middle1097=== PAUSE TestIsValidCachePath/traversal_in_middle1098=== RUN TestIsValidCachePath/invalid_char_e1099=== PAUSE TestIsValidCachePath/invalid_char_e1100=== RUN TestIsValidCachePath/invalid_char_u1101=== PAUSE TestIsValidCachePath/invalid_char_u1102=== RUN TestIsValidCachePath/random_path1103=== PAUSE TestIsValidCachePath/random_path1104=== RUN TestIsValidCachePath/empty1105=== PAUSE TestIsValidCachePath/empty1106=== RUN TestIsValidCachePath/leading_slash1107=== PAUSE TestIsValidCachePath/leading_slash1108=== RUN TestIsValidCachePath/wrong_extension1109=== PAUSE TestIsValidCachePath/wrong_extension1110=== RUN TestIsValidCachePath/short_hash1111=== PAUSE TestIsValidCachePath/short_hash1112=== CONT TestClientIntegration1113--- PASS: TestReadProxyDisabled (1.13s)1114=== CONT TestService_NativeMTLS11152026/08/27 09:31:34 INFO Received uploads request method=POST path=/api/pending_closures11162026/08/27 09:31:34 INFO Received uploads request method=POST path=/api/pending_closures11172026-08-27 09:31:34.967 UTC [637] ERROR: relation "goose_db_version" does not exist at character 3611182026-08-27 09:31:34.967 UTC [637] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC11192026/08/27 09:31:34 OK 20241026095416_initial_model.sql (15.35ms)11202026/08/27 09:31:34 OK 20251210153512_drop_unused_gin_index.sql (2.94ms)11212026/08/27 09:31:35 OK 20251218171726_add_pins.sql (6.41ms)11222026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (7.13ms)11232026/08/27 09:31:35 goose: successfully migrated database to version: 2026062812000011242026/08/27 09:31:35 INFO Received complete multipart upload request method=POST path=/api/multipart/complete11252026/08/27 09:31:35 OK 1_commit_pending_closure.sql (3.42ms)11262026/08/27 09:31:35 OK 2_object_stats_trigger.sql (3.5ms)11272026/08/27 09:31:35 goose: up to current file version: 211282026/08/27 09:31:35 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MzE5NzViNTQtNmY5Ni00YmI4LTk2YWQtY2JlYjY5ZDUyNjk0LjZkMGIyOWFhLTJiY2EtNGNiYy05NGE0LTYyODFhMTA2NTEzOHgxNzg3ODIzMDk0OTc0MjQ1ODM211292026/08/27 09:31:35 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MzE5NzViNTQtNmY5Ni00YmI4LTk2YWQtY2JlYjY5ZDUyNjk0LjZkMGIyOWFhLTJiY2EtNGNiYy05NGE0LTYyODFhMTA2NTEzOHgxNzg3ODIzMDk0OTc0MjQ1ODM2 parts=11130--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (1.21s)1131=== CONT TestClientErrorHandling1132=== RUN TestClientErrorHandling/InvalidStorePath1133=== PAUSE TestClientErrorHandling/InvalidStorePath1134=== RUN TestClientErrorHandling/InvalidAuthToken1135=== PAUSE TestClientErrorHandling/InvalidAuthToken1136=== RUN TestClientErrorHandling/ServerNotAvailable1137=== PAUSE TestClientErrorHandling/ServerNotAvailable1138=== CONT TestMetricsInventory11392026-08-27 09:31:35.043 UTC [639] ERROR: relation "goose_db_version" does not exist at character 3611402026-08-27 09:31:35.043 UTC [639] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC11412026-08-27 09:31:35.043 UTC [640] ERROR: relation "goose_db_version" does not exist at character 3611422026-08-27 09:31:35.043 UTC [640] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC11432026/08/27 09:31:35 OK 20241026095416_initial_model.sql (14.08ms)11442026/08/27 09:31:35 OK 20241026095416_initial_model.sql (14.01ms)11452026/08/27 09:31:35 OK 20251210153512_drop_unused_gin_index.sql (2.83ms)11462026/08/27 09:31:35 OK 20251210153512_drop_unused_gin_index.sql (1.92ms)11472026/08/27 09:31:35 OK 20251218171726_add_pins.sql (4.49ms)11482026/08/27 09:31:35 OK 20251218171726_add_pins.sql (5.52ms)11492026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (5.05ms)11502026/08/27 09:31:35 goose: successfully migrated database to version: 2026062812000011512026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (4.9ms)11522026/08/27 09:31:35 goose: successfully migrated database to version: 2026062812000011532026/08/27 09:31:35 OK 1_commit_pending_closure.sql (3.18ms)11542026/08/27 09:31:35 OK 1_commit_pending_closure.sql (3.09ms)11552026/08/27 09:31:35 OK 2_object_stats_trigger.sql (1.8ms)11562026/08/27 09:31:35 goose: up to current file version: 211572026/08/27 09:31:35 OK 2_object_stats_trigger.sql (2.04ms)11582026/08/27 09:31:35 goose: up to current file version: 21159{"timestamp":"2026-08-27T09:31:35.085275953Z","level":"ERROR","duration":"644.046µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1160{"timestamp":"2026-08-27T09:31:35.085375754Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"4f9351ad-d284-4866-86fd-1439b80ffde4","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1161{"timestamp":"2026-08-27T09:31:35.0860926Z","level":"ERROR","duration":"420.743µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1162{"timestamp":"2026-08-27T09:31:35.086153561Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"239c2acc-b1cd-43c4-ae7b-4939611a9b77","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket31/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1163{"timestamp":"2026-08-27T09:31:35.088101539Z","level":"ERROR","duration":"237.902µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1164{"timestamp":"2026-08-27T09:31:35.08818766Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0d2764e7-8ff3-4442-baee-99e4cb500b5d","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket14/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}11652026-08-27 09:31:35.105 UTC [642] ERROR: relation "goose_db_version" does not exist at character 3611662026-08-27 09:31:35.105 UTC [642] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC1167{"timestamp":"2026-08-27T09:31:35.109689577Z","level":"ERROR","duration":"224.242µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1168{"timestamp":"2026-08-27T09:31:35.109777517Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"777b8b07-c8ff-437f-a5d6-c90bb5e715fe","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket24/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1169{"timestamp":"2026-08-27T09:31:35.110173601Z","level":"ERROR","duration":"136.861µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1170{"timestamp":"2026-08-27T09:31:35.110226782Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f42eb7a7-4e2c-48fb-83b6-d2adce0c4cbf","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket31/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}11712026/08/27 09:31:35 OK 20241026095416_initial_model.sql (9.67ms)11722026/08/27 09:31:35 OK 20251210153512_drop_unused_gin_index.sql (1.51ms)11732026/08/27 09:31:35 OK 20251218171726_add_pins.sql (3.57ms)11742026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (3.44ms)11752026/08/27 09:31:35 goose: successfully migrated database to version: 2026062812000011762026/08/27 09:31:35 OK 1_commit_pending_closure.sql (1.94ms)11772026/08/27 09:31:35 OK 2_object_stats_trigger.sql (854.87µs)11782026/08/27 09:31:35 goose: up to current file version: 21179{"timestamp":"2026-08-27T09:31:35.135110009Z","level":"ERROR","duration":"779.407µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1180{"timestamp":"2026-08-27T09:31:35.13520397Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"91f922c8-61bb-42e1-8094-4b861a84f899","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket32/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(312)"}1181{"timestamp":"2026-08-27T09:31:35.167551646Z","level":"ERROR","duration":"281.262µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(269)"}1182{"timestamp":"2026-08-27T09:31:35.167650227Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"98f14379-6e01-495c-95b1-92ea861a3cd7","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket23/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(269)"}1183{"timestamp":"2026-08-27T09:31:35.21274872Z","level":"ERROR","duration":"231.922µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(269)"}1184{"timestamp":"2026-08-27T09:31:35.212855221Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"732b798d-1a4a-43c0-9be5-8cbc37dc6e8f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket14/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(269)"}1185--- PASS: TestGCBugBareHashReferences (1.28s)1186=== CONT TestNARDeduplicationMetadataUploadBug1187{"timestamp":"2026-08-27T09:31:35.221714443Z","level":"ERROR","duration":"1.290331ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1188{"timestamp":"2026-08-27T09:31:35.221791823Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"151d0558-5369-429d-b0c4-c514f963f2ee","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket32/","status_code":503,"duration_ms":1,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1189{"timestamp":"2026-08-27T09:31:35.228586406Z","level":"ERROR","duration":"166.722µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1190{"timestamp":"2026-08-27T09:31:35.228649046Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"fc97e6fc-cace-46e3-a868-f29fce9ccd93","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket13/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1191{"timestamp":"2026-08-27T09:31:35.23123967Z","level":"ERROR","duration":"133.081µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1192{"timestamp":"2026-08-27T09:31:35.23129299Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"797d960e-cba1-4364-919c-3ebf0c9065d8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket25/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1193{"timestamp":"2026-08-27T09:31:35.236041614Z","level":"ERROR","duration":"119.461µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1194{"timestamp":"2026-08-27T09:31:35.236090054Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"84db39b4-cb9e-4249-8712-0d9f9b93bf75","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket24/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1195{"timestamp":"2026-08-27T09:31:35.278125659Z","level":"ERROR","duration":"227.642µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(269)"}1196{"timestamp":"2026-08-27T09:31:35.27822258Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e3f5f825-bfd7-4b82-b6e3-0e1a04304830","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(269)"}1197{"timestamp":"2026-08-27T09:31:35.282080676Z","level":"ERROR","duration":"157.842µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(269)"}1198{"timestamp":"2026-08-27T09:31:35.282146756Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0385f616-5f0b-44ab-9843-adb51d17a4e2","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket30/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(269)"}11992026-08-27 09:31:35.290 UTC [645] ERROR: relation "goose_db_version" does not exist at character 3612002026-08-27 09:31:35.290 UTC [645] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC12012026/08/27 09:31:35 OK 20241026095416_initial_model.sql (11.44ms)12022026/08/27 09:31:35 OK 20251210153512_drop_unused_gin_index.sql (1.5ms)12032026/08/27 09:31:35 OK 20251218171726_add_pins.sql (3.46ms)12042026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (3.16ms)12052026/08/27 09:31:35 goose: successfully migrated database to version: 2026062812000012062026/08/27 09:31:35 OK 1_commit_pending_closure.sql (2.01ms)12072026/08/27 09:31:35 OK 2_object_stats_trigger.sql (934.21µs)12082026/08/27 09:31:35 goose: up to current file version: 21209{"timestamp":"2026-08-27T09:31:35.32292025Z","level":"ERROR","duration":"675.467µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(269)"}1210{"timestamp":"2026-08-27T09:31:35.32300647Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e67c6d5c-74aa-4ed4-b75a-32f1dfb5b999","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(269)"}1211{"timestamp":"2026-08-27T09:31:35.366268427Z","level":"ERROR","duration":"231.642µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1212{"timestamp":"2026-08-27T09:31:35.366352807Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"b86b3886-cce5-480d-82dd-77103a99ed91","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket33/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(207)"}1213{"timestamp":"2026-08-27T09:31:35.424409979Z","level":"ERROR","duration":"198.962µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}1214{"timestamp":"2026-08-27T09:31:35.4244579Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a15e4783-d20f-44fe-ae56-0e9537cd02c8","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket26/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(387)"}1215--- PASS: TestReadProxyNarinfo (1.27s)1216=== CONT TestClientCADerivations12172026/08/27 09:31:35 INFO Created nix-cache-info in bucket bucket=bucket2712182026/08/27 09:31:35 INFO Aborted multipart uploads count=012192026/08/27 09:31:35 WARN Force mode enabled - objects will be deleted immediately without grace period12202026/08/27 09:31:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012212026/08/27 09:31:35 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"1222--- PASS: TestService_AuthMiddleware (1.54s)1223=== CONT TestCreatePendingClosureRejectsOversizedNAR12242026/08/27 09:31:35 INFO Vacuumed table table=pending_closures12252026/08/27 09:31:35 INFO Received uploads request method=POST path=/api/pending_closures1226--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.00s)1227=== CONT TestCacheStatsHandler12282026/08/27 09:31:35 INFO Vacuumed table table=pending_objects12292026/08/27 09:31:35 INFO Vacuumed table table=multipart_uploads12302026/08/27 09:31:35 INFO Vacuumed table table=closures12312026/08/27 09:31:35 INFO Vacuumed table table=objects1232--- PASS: TestGCMetrics (1.55s)1233=== CONT TestCacheConfigHandlerMaxNarSize1234--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)1235=== CONT TestGenerateLandingPage12362026/08/27 09:31:35 INFO Created nix-cache-info in bucket bucket=bucket291237--- PASS: TestGenerateLandingPage (0.01s)1238=== CONT TestCacheConfigHandler1239=== RUN TestCacheConfigHandler/full_config,_no_issuer1240=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1241=== RUN TestCacheConfigHandler/no_cache_url_configured1242=== PAUSE TestCacheConfigHandler/no_cache_url_configured1243=== RUN TestCacheConfigHandler/no_signing_keys1244=== PAUSE TestCacheConfigHandler/no_signing_keys1245=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1246=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1247=== CONT TestServerTLSConfig1248=== RUN TestServerTLSConfig/no_client_CA1249=== PAUSE TestServerTLSConfig/no_client_CA1250=== RUN TestServerTLSConfig/missing_CA_file1251=== PAUSE TestServerTLSConfig/missing_CA_file1252=== RUN TestServerTLSConfig/not_a_PEM_file1253=== PAUSE TestServerTLSConfig/not_a_PEM_file1254=== CONT TestOrphanedObjectsGCStressTest12552026/08/27 09:31:35 WARN mTLS auth: subject not in bound subjects subject="CN=reader"12562026/08/27 09:31:35 WARN mTLS auth: subject not in bound subjects subject="CN=writer"1257--- PASS: TestService_NativeMTLS (0.57s)1258=== CONT TestService_AuthMiddleware_OIDC12592026/08/27 09:31:35 INFO OIDC provider initialized name=test1260--- PASS: TestReadProxy404 (1.59s)1261=== CONT TestOrphanedObjectsGC1262=== NAME TestClientMultipleUploads1263 client_integration_test.go:338: Created store path 0: /build/TestClientMultipleUploads2341389745/001/store/dg7ac5w20skkcvmpb4yc8ssyw98bvkdm-test-file-0.txt12642026-08-27 09:31:35.550 UTC [708] ERROR: relation "goose_db_version" does not exist at character 3612652026-08-27 09:31:35.550 UTC [708] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC1266=== NAME TestClientWithDependencies1267 client_integration_test.go:593: Built derivation: /build/TestClientWithDependencies2318163371/001/store/qlz3hrcm85b6dkldwm0i5f36w2fassvg-test-script1268--- PASS: TestMetricsInventory (0.52s)1269=== CONT TestParseSingleRange1270=== RUN TestParseSingleRange/none1271=== PAUSE TestParseSingleRange/none1272=== RUN TestParseSingleRange/unknown_unit1273=== PAUSE TestParseSingleRange/unknown_unit1274=== RUN TestParseSingleRange/multi-range_ignored1275=== PAUSE TestParseSingleRange/multi-range_ignored1276=== RUN TestParseSingleRange/malformed_no_dash1277=== PAUSE TestParseSingleRange/malformed_no_dash1278=== RUN TestParseSingleRange/malformed_both_empty1279=== PAUSE TestParseSingleRange/malformed_both_empty1280=== RUN TestParseSingleRange/malformed_end_before_start1281=== PAUSE TestParseSingleRange/malformed_end_before_start1282=== RUN TestParseSingleRange/closed1283=== PAUSE TestParseSingleRange/closed1284=== RUN TestParseSingleRange/open-ended1285=== PAUSE TestParseSingleRange/open-ended1286=== RUN TestParseSingleRange/end_clamped_to_size1287=== PAUSE TestParseSingleRange/end_clamped_to_size1288=== RUN TestParseSingleRange/suffix1289=== PAUSE TestParseSingleRange/suffix1290=== RUN TestParseSingleRange/suffix_exceeds_size1291=== PAUSE TestParseSingleRange/suffix_exceeds_size1292=== RUN TestParseSingleRange/single_byte1293=== PAUSE TestParseSingleRange/single_byte1294=== RUN TestParseSingleRange/start_past_EOF1295=== PAUSE TestParseSingleRange/start_past_EOF1296=== RUN TestParseSingleRange/start_far_past_EOF1297=== PAUSE TestParseSingleRange/start_far_past_EOF1298=== CONT TestService_ReadAuthMiddleware12992026/08/27 09:31:35 INFO Created nix-cache-info in bucket bucket=bucket331300--- PASS: TestReadProxyRootRedirectsToIndexHTML (1.64s)1301=== CONT TestObjectStatsTrigger13022026/08/27 09:31:35 OK 20241026095416_initial_model.sql (25.77ms)1303=== NAME TestClientMultipleUploads1304 client_integration_test.go:338: Created store path 1: /build/TestClientMultipleUploads2341389745/001/store/nyczl96zhiajvzpp6v0npbz6rscily4l-test-file-1.txt13052026/08/27 09:31:35 INFO Created nix-cache-info in bucket bucket=bucket3013062026/08/27 09:31:35 OK 20251210153512_drop_unused_gin_index.sql (3.93ms)1307=== NAME TestClientWithDependencies1308 client_integration_test.go:595: Found 1 dependencies (including self)13092026/08/27 09:31:35 OK 20251218171726_add_pins.sql (5.88ms)13102026-08-27 09:31:35.595 UTC [733] ERROR: relation "goose_db_version" does not exist at character 3613112026-08-27 09:31:35.595 UTC [733] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13122026-08-27 09:31:35.604 UTC [769] ERROR: relation "goose_db_version" does not exist at character 3613132026-08-27 09:31:35.604 UTC [769] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC1314=== NAME TestNARDeduplicationMetadataUploadBug1315 metadata_upload_test.go:48: First store path: /build/TestNARDeduplicationMetadataUploadBug2991872231/001/store/5sgmaw02r1dsd8567qaxsjanxp1kxnyh-file1.txt13162026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (10.76ms)13172026/08/27 09:31:35 goose: successfully migrated database to version: 202606281200001318--- PASS: TestReadProxyHead (1.68s)1319=== CONT TestResurrectedObjectNotDeleted13202026/08/27 09:31:35 OK 1_commit_pending_closure.sql (5.34ms)13212026/08/27 09:31:35 OK 2_object_stats_trigger.sql (17.6ms)13222026/08/27 09:31:35 goose: up to current file version: 213232026/08/27 09:31:35 OK 20241026095416_initial_model.sql (24.19ms)1324=== NAME TestClientMultipleUploads1325 client_integration_test.go:338: Created store path 2: /build/TestClientMultipleUploads2341389745/001/store/1jlxv03aa3fxfypsx8vv7jfaxkgxj8mr-test-file-2.txt13262026/08/27 09:31:35 OK 20251210153512_drop_unused_gin_index.sql (3.92ms)13272026/08/27 09:31:35 OK 20241026095416_initial_model.sql (12.15ms)1328--- PASS: TestService_Rustfstest (1.82s)1329=== CONT TestService_AuthMiddleware_MTLSBoundSubjects13302026/08/27 09:31:35 OK 20251210153512_drop_unused_gin_index.sql (4.32ms)13312026/08/27 09:31:35 OK 20251218171726_add_pins.sql (6.19ms)1332=== NAME TestClientIntegration1333 client_integration_test.go:276: Created store path: /build/TestClientIntegration3067807208/002/store/lk51sl6024f99520swjhhgngard718mg-test-file.txt13342026-08-27 09:31:35.646 UTC [825] ERROR: relation "goose_db_version" does not exist at character 3613352026-08-27 09:31:35.646 UTC [825] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13362026-08-27 09:31:35.646 UTC [823] ERROR: relation "goose_db_version" does not exist at character 3613372026-08-27 09:31:35.646 UTC [823] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13382026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (7.79ms)13392026/08/27 09:31:35 goose: successfully migrated database to version: 2026062812000013402026/08/27 09:31:35 OK 20251218171726_add_pins.sql (9.93ms)13412026-08-27 09:31:35.651 UTC [828] ERROR: relation "goose_db_version" does not exist at character 3613422026-08-27 09:31:35.651 UTC [828] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13432026/08/27 09:31:35 OK 1_commit_pending_closure.sql (5.81ms)13442026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (5.96ms)13452026/08/27 09:31:35 goose: successfully migrated database to version: 2026062812000013462026/08/27 09:31:35 OK 2_object_stats_trigger.sql (2.61ms)13472026/08/27 09:31:35 goose: up to current file version: 213482026/08/27 09:31:35 OK 1_commit_pending_closure.sql (3.41ms)13492026/08/27 09:31:35 OK 2_object_stats_trigger.sql (2.86ms)13502026/08/27 09:31:35 goose: up to current file version: 213512026/08/27 09:31:35 INFO Created nix-cache-info in bucket bucket=bucket3413522026/08/27 09:31:35 OK 20241026095416_initial_model.sql (12.38ms)13532026/08/27 09:31:35 OK 20251210153512_drop_unused_gin_index.sql (3.3ms)13542026/08/27 09:31:35 OK 20241026095416_initial_model.sql (12.94ms)13552026/08/27 09:31:35 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"13562026/08/27 09:31:35 OK 20251218171726_add_pins.sql (5.04ms)13572026/08/27 09:31:35 INFO Received uploads request method=POST path=/api/pending_closures13582026-08-27 09:31:35.676 UTC [867] ERROR: relation "goose_db_version" does not exist at character 3613592026-08-27 09:31:35.676 UTC [867] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13602026/08/27 09:31:35 OK 20241026095416_initial_model.sql (19.32ms)13612026/08/27 09:31:35 OK 20251210153512_drop_unused_gin_index.sql (3.51ms)13622026/08/27 09:31:35 OK 20251210153512_drop_unused_gin_index.sql (3.29ms)13632026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (6.45ms)13642026/08/27 09:31:35 goose: successfully migrated database to version: 2026062812000013652026/08/27 09:31:35 OK 20251218171726_add_pins.sql (5.37ms)13662026/08/27 09:31:35 OK 1_commit_pending_closure.sql (4.48ms)13672026/08/27 09:31:35 OK 20251218171726_add_pins.sql (6.07ms)13682026/08/27 09:31:35 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)13692026/08/27 09:31:35 INFO Uploading qlz3hrcm85b6dkldwm0i5f36w2fassvg-test-script (136B)13702026/08/27 09:31:35 OK 2_object_stats_trigger.sql (2.99ms)13712026/08/27 09:31:35 goose: up to current file version: 213722026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (7.12ms)13732026/08/27 09:31:35 goose: successfully migrated database to version: 2026062812000013742026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (5.25ms)13752026/08/27 09:31:35 goose: successfully migrated database to version: 2026062812000013762026/08/27 09:31:35 OK 1_commit_pending_closure.sql (3.37ms)13772026/08/27 09:31:35 OK 2_object_stats_trigger.sql (1.95ms)13782026/08/27 09:31:35 goose: up to current file version: 213792026/08/27 09:31:35 OK 1_commit_pending_closure.sql (3.37ms)13802026/08/27 09:31:35 WARN Failed to register uploaded object key=log/sr2smdiim9s56xrcs1sacganbx4j0d02-test-script.drv error="server returned 404: 404 page not found\n"13812026/08/27 09:31:35 OK 20241026095416_initial_model.sql (11.87ms)13822026/08/27 09:31:35 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"13832026/08/27 09:31:35 OK 2_object_stats_trigger.sql (2.05ms)13842026/08/27 09:31:35 goose: up to current file version: 213852026/08/27 09:31:35 INFO Received uploads request method=POST path=/api/pending_closures13862026/08/27 09:31:35 OK 20251210153512_drop_unused_gin_index.sql (2.12ms)13872026/08/27 09:31:35 WARN Failed to register uploaded object key=qlz3hrcm85b6dkldwm0i5f36w2fassvg.ls error="server returned 404: 404 page not found\n"13882026-08-27 09:31:35.708 UTC [924] ERROR: relation "goose_db_version" does not exist at character 3613892026-08-27 09:31:35.708 UTC [924] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13902026/08/27 09:31:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign13912026/08/27 09:31:35 INFO Signed narinfos id=1 count=113922026/08/27 09:31:35 INFO Uploading 1 narinfos13932026/08/27 09:31:35 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"13942026/08/27 09:31:35 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"13952026/08/27 09:31:35 OK 20251218171726_add_pins.sql (15.61ms)13962026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (4.52ms)13972026/08/27 09:31:35 goose: successfully migrated database to version: 2026062812000013982026/08/27 09:31:35 OK 1_commit_pending_closure.sql (2.37ms)13992026/08/27 09:31:35 OK 2_object_stats_trigger.sql (2.78ms)14002026/08/27 09:31:35 goose: up to current file version: 214012026/08/27 09:31:35 OK 20241026095416_initial_model.sql (9.2ms)1402--- PASS: TestReadProxyConditionalGet (1.79s)1403=== CONT TestMultipartCleanup14042026-08-27 09:31:35.729 UTC [961] ERROR: relation "goose_db_version" does not exist at character 3614052026-08-27 09:31:35.729 UTC [961] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC14062026/08/27 09:31:35 OK 20251210153512_drop_unused_gin_index.sql (3.48ms)1407--- PASS: TestCacheStatsHandler (0.25s)1408=== CONT TestService_AuthMiddleware_MTLSProxyHeader14092026/08/27 09:31:35 OK 20251218171726_add_pins.sql (3.71ms)14102026/08/27 09:31:35 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"14112026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (7.89ms)14122026/08/27 09:31:35 goose: successfully migrated database to version: 2026062812000014132026/08/27 09:31:35 OK 20241026095416_initial_model.sql (8.41ms)14142026/08/27 09:31:35 OK 1_commit_pending_closure.sql (2.97ms)14152026/08/27 09:31:35 OK 2_object_stats_trigger.sql (1.02ms)14162026/08/27 09:31:35 goose: up to current file version: 214172026/08/27 09:31:35 OK 20251210153512_drop_unused_gin_index.sql (2.37ms)14182026/08/27 09:31:35 INFO Received uploads request method=POST path=/api/pending_closures14192026/08/27 09:31:35 OK 20251218171726_add_pins.sql (4.24ms)14202026/08/27 09:31:35 INFO Received uploads request method=POST path=/api/pending_closures14212026/08/27 09:31:35 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)14222026/08/27 09:31:35 INFO Uploading 5sgmaw02r1dsd8567qaxsjanxp1kxnyh-file1.txt (160B)14232026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (5.85ms)14242026/08/27 09:31:35 goose: successfully migrated database to version: 2026062812000014252026/08/27 09:31:35 OK 1_commit_pending_closure.sql (5.39ms)14262026/08/27 09:31:35 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)14272026/08/27 09:31:35 INFO Uploading lk51sl6024f99520swjhhgngard718mg-test-file.txt (152B)14282026/08/27 09:31:35 OK 2_object_stats_trigger.sql (3.48ms)14292026/08/27 09:31:35 goose: up to current file version: 214302026/08/27 09:31:35 INFO Received uploads request method=POST path=/api/pending_closures1431=== NAME TestClientCADerivations1432 client_ca_test.go:136: Built CA derivation: /build/TestClientCADerivations2356769355/001/store/1f7nl8ygvlcbjhp3psc85pj4hlviazhn-ca-test14332026/08/27 09:31:35 INFO Received uploads request method=POST path=/api/pending_closures14342026/08/27 09:31:35 INFO Received uploads request method=POST path=/api/pending_closures14352026/08/27 09:31:35 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)14362026/08/27 09:31:35 INFO Uploading 1jlxv03aa3fxfypsx8vv7jfaxkgxj8mr-test-file-2.txt (160B)14372026/08/27 09:31:35 INFO Uploading nyczl96zhiajvzpp6v0npbz6rscily4l-test-file-1.txt (160B)14382026/08/27 09:31:35 INFO Uploading dg7ac5w20skkcvmpb4yc8ssyw98bvkdm-test-file-0.txt (160B)14392026-08-27 09:31:35.821 UTC [1057] ERROR: relation "goose_db_version" does not exist at character 3614402026-08-27 09:31:35.821 UTC [1057] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC14412026-08-27 09:31:35.821 UTC [1058] ERROR: relation "goose_db_version" does not exist at character 3614422026-08-27 09:31:35.821 UTC [1058] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC1443 client_ca_test.go:139: Found 1 dependencies (including self)14442026/08/27 09:31:35 OK 20241026095416_initial_model.sql (10.58ms)14452026/08/27 09:31:35 OK 20241026095416_initial_model.sql (11.39ms)14462026/08/27 09:31:35 OK 20251210153512_drop_unused_gin_index.sql (1.61ms)14472026/08/27 09:31:35 OK 20251210153512_drop_unused_gin_index.sql (1.36ms)14482026/08/27 09:31:35 OK 20251218171726_add_pins.sql (3.96ms)14492026/08/27 09:31:35 OK 20251218171726_add_pins.sql (4.34ms)14502026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (3.77ms)14512026/08/27 09:31:35 goose: successfully migrated database to version: 2026062812000014522026/08/27 09:31:35 OK 20260628120000_add_object_size_and_stats.sql (3.95ms)14532026/08/27 09:31:35 goose: successfully migrated database to version: 2026062812000014542026/08/27 09:31:35 OK 1_commit_pending_closure.sql (2.05ms)14552026/08/27 09:31:35 OK 1_commit_pending_closure.sql (2.22ms)14562026/08/27 09:31:35 OK 2_object_stats_trigger.sql (917.65µs)14572026/08/27 09:31:35 goose: up to current file version: 214582026/08/27 09:31:35 OK 2_object_stats_trigger.sql (1.32ms)14592026/08/27 09:31:35 goose: up to current file version: 214602026/08/27 09:31:35 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"14612026/08/27 09:31:35 INFO Received uploads request method=POST path=/api/pending_closures14622026/08/27 09:31:35 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)14632026/08/27 09:31:35 INFO Uploading 1f7nl8ygvlcbjhp3psc85pj4hlviazhn-ca-test (144B)1464{"timestamp":"2026-08-27T09:31:36.770784731Z","level":"ERROR","duration":"801.167µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(366)"}1465{"timestamp":"2026-08-27T09:31:36.770837952Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"8899a01f-27c9-47ce-83f4-e10bf75090eb","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket44/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(366)"}1466{"timestamp":"2026-08-27T09:31:36.771363297Z","level":"ERROR","duration":"1.147931ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(329)"}1467{"timestamp":"2026-08-27T09:31:36.771409037Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"ecba3a47-4f3d-457a-9c87-7fc975e59bd9","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket42/","status_code":503,"duration_ms":1,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(329)"}14682026/08/27 09:31:36 WARN Failed to register uploaded object key=qlz3hrcm85b6dkldwm0i5f36w2fassvg.narinfo error="server returned 404: 404 page not found\n"14692026/08/27 09:31:36 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"14702026/08/27 09:31:36 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete14712026/08/27 09:31:36 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"14722026/08/27 09:31:36 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"14732026/08/27 09:31:36 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"14742026/08/27 09:31:36 WARN Failed to register uploaded object key=log/iyd7l0ygjkhdmw87zq5kk85cks07f9w2-ca-test.drv error="server returned 404: 404 page not found\n"14752026/08/27 09:31:36 WARN Failed to register uploaded object key=5sgmaw02r1dsd8567qaxsjanxp1kxnyh.ls error="server returned 404: 404 page not found\n"14762026/08/27 09:31:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign14772026/08/27 09:31:36 WARN Failed to register uploaded object key=1jlxv03aa3fxfypsx8vv7jfaxkgxj8mr.ls error="server returned 404: 404 page not found\n"14782026/08/27 09:31:36 WARN Failed to register uploaded object key=lk51sl6024f99520swjhhgngard718mg.ls error="server returned 404: 404 page not found\n"14792026/08/27 09:31:36 WARN Failed to register uploaded object key=dg7ac5w20skkcvmpb4yc8ssyw98bvkdm.ls error="server returned 404: 404 page not found\n"14802026/08/27 09:31:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign14812026/08/27 09:31:36 INFO Signed narinfos id=1 count=114822026/08/27 09:31:36 INFO Uploading 1 narinfos14832026/08/27 09:31:36 INFO Signed narinfos id=1 count=114842026/08/27 09:31:36 INFO Uploading 1 narinfos14852026/08/27 09:31:36 INFO Completed upload id=114862026/08/27 09:31:36 INFO Upload complete. (1.139s)14872026/08/27 09:31:36 WARN Failed to register uploaded object key=lk51sl6024f99520swjhhgngard718mg.narinfo error="server returned 404: 404 page not found\n"14882026/08/27 09:31:36 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete14892026/08/27 09:31:36 WARN Failed to register uploaded object key=5sgmaw02r1dsd8567qaxsjanxp1kxnyh.narinfo error="server returned 404: 404 page not found\n"14902026/08/27 09:31:36 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete1491=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token1492=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token1493=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1494=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1495=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1496=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1497=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1498=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1499=== CONT TestProxyWriteTimeout/narinfo1500=== CONT TestProxyWriteTimeout/unknown_size1501=== CONT TestProxyWriteTimeout/10_GiB_nar1502=== CONT TestProxyWriteTimeout/1_GiB_nar1503--- PASS: TestProxyWriteTimeout (0.12s)1504 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1505 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1506 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1507 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)1508=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1509=== NAME TestClientWithDependencies1510 client_integration_test.go:597: Skipping nix copy test - isolated store (/build/TestClientWithDependencies2318163371/001/store) requires matching store prefix15112026/08/27 09:31:36 INFO Received uploads request method=POST path=/1512=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key15132026/08/27 09:31:36 INFO Received complete multipart upload request method=POST path=/1514=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key15152026/08/27 09:31:36 INFO Received request for more parts method=POST path=/1516=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal15172026/08/27 09:31:36 INFO Received uploads request method=POST path=/1518--- PASS: TestUploadHandlersRejectInvalidKeys (0.12s)1519 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)1520 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1521 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1522 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s)1523=== CONT TestIsValidUploadKey/narinfo1524=== CONT TestIsValidUploadKey/unknown_type1525=== CONT TestIsValidUploadKey/empty_key1526=== CONT TestIsValidUploadKey/absolute1527=== CONT TestIsValidUploadKey/traversal_nar1528=== CONT TestIsValidUploadKey/traversal1529=== CONT TestIsValidUploadKey/listing_key,_narinfo_type1530=== CONT TestIsValidUploadKey/nar_key,_narinfo_type1531=== CONT TestIsValidUploadKey/narinfo_key,_nar_type1532=== CONT TestIsValidUploadKey/index.html1533=== CONT TestIsValidUploadKey/nix-cache-info1534=== CONT TestIsValidUploadKey/realisation_plus_in_output1535=== CONT TestIsValidUploadKey/realisation1536=== CONT TestIsValidUploadKey/build_log_equals1537=== CONT TestIsValidUploadKey/build_log_question_mark1538=== CONT TestIsValidUploadKey/build_log_plus_in_name1539=== CONT TestIsValidUploadKey/build_log_home-manager_file1540=== CONT TestIsValidUploadKey/build_log1541=== CONT TestIsValidUploadKey/listing1542=== CONT TestIsValidUploadKey/nar_plain1543=== CONT TestIsValidUploadKey/nar_xz1544=== CONT TestIsValidUploadKey/nar_zst1545--- PASS: TestIsValidUploadKey (0.12s)1546 --- PASS: TestIsValidUploadKey/narinfo (0.00s)1547 --- PASS: TestIsValidUploadKey/unknown_type (0.00s)1548 --- PASS: TestIsValidUploadKey/empty_key (0.00s)1549 --- PASS: TestIsValidUploadKey/absolute (0.00s)1550 --- PASS: TestIsValidUploadKey/traversal_nar (0.00s)1551 --- PASS: TestIsValidUploadKey/traversal (0.00s)1552 --- PASS: TestIsValidUploadKey/listing_key,_narinfo_type (0.00s)1553 --- PASS: TestIsValidUploadKey/nar_key,_narinfo_type (0.00s)1554 --- PASS: TestIsValidUploadKey/narinfo_key,_nar_type (0.00s)1555 --- PASS: TestIsValidUploadKey/index.html (0.00s)1556 --- PASS: TestIsValidUploadKey/nix-cache-info (0.00s)1557 --- PASS: TestIsValidUploadKey/realisation_plus_in_output (0.00s)1558 --- PASS: TestIsValidUploadKey/realisation (0.00s)1559 --- PASS: TestIsValidUploadKey/build_log_equals (0.00s)1560 --- PASS: TestIsValidUploadKey/build_log_question_mark (0.00s)1561 --- PASS: TestIsValidUploadKey/build_log_plus_in_name (0.00s)1562 --- PASS: TestIsValidUploadKey/build_log_home-manager_file (0.00s)1563 --- PASS: TestIsValidUploadKey/build_log (0.00s)1564 --- PASS: TestIsValidUploadKey/listing (0.00s)1565 --- PASS: TestIsValidUploadKey/nar_plain (0.00s)1566 --- PASS: TestIsValidUploadKey/nar_xz (0.00s)1567 --- PASS: TestIsValidUploadKey/nar_zst (0.00s)1568=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure15692026/08/27 09:31:36 INFO Received uploads request method=POST path=/15702026/08/27 09:31:36 INFO Completed upload id=115712026/08/27 09:31:36 INFO Completed upload id=115722026/08/27 09:31:36 INFO Upload complete. (1.115s)15732026/08/27 09:31:36 INFO Upload complete. (1.124s)1574--- PASS: TestClientWithDependencies (2.61s)1575=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts15762026/08/27 09:31:36 INFO Received request for more parts method=POST path=/15772026/08/27 09:31:36 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"1578=== NAME TestClientIntegration1579 client_integration_test.go:292: Retrieved narinfo from S3:1580 StorePath: /build/TestClientIntegration3067807208/002/store/lk51sl6024f99520swjhhgngard718mg-test-file.txt1581 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1582 Compression: zstd1583 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11584 NarSize: 1521585 References: 1586 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11587=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart15882026/08/27 09:31:36 INFO Received complete multipart upload request method=POST path=/1589--- PASS: TestObjectStatsTrigger (1.22s)1590=== NAME TestNARDeduplicationMetadataUploadBug1591 metadata_upload_test.go:54: Retrieved narinfo from S3:1592 StorePath: /build/TestNARDeduplicationMetadataUploadBug2991872231/001/store/5sgmaw02r1dsd8567qaxsjanxp1kxnyh-file1.txt1593 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1594 Compression: zstd1595 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1596 NarSize: 1601597 References: 1598 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf15992026/08/27 09:31:36 WARN Failed to register uploaded object key=1f7nl8ygvlcbjhp3psc85pj4hlviazhn.ls error="server returned 404: 404 page not found\n"16002026/08/27 09:31:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign16012026/08/27 09:31:36 WARN mTLS auth: subject not in bound subjects subject="CN=writer"1602--- PASS: TestService_ReadAuthMiddleware (1.25s)1603=== CONT TestIsValidCachePath/narinfo1604=== CONT TestIsValidCachePath/empty1605=== CONT TestIsValidCachePath/random_path1606=== CONT TestIsValidCachePath/invalid_char_u1607=== CONT TestIsValidCachePath/invalid_char_e1608=== CONT TestIsValidCachePath/leading_slash1609=== CONT TestIsValidCachePath/traversal_in_middle1610=== CONT TestIsValidCachePath/traversal_parent1611=== CONT TestIsValidCachePath/index.html1612=== CONT TestIsValidCachePath/nix-cache-info1613=== CONT TestIsValidCachePath/realisation1614=== CONT TestIsValidCachePath/log1615=== CONT TestIsValidCachePath/ls1616=== CONT TestIsValidCachePath/nar_uncompressed1617=== CONT TestIsValidCachePath/nar_bz21618=== CONT TestIsValidCachePath/nar_xz1619=== CONT TestIsValidCachePath/nar_zst1620=== CONT TestIsValidCachePath/narinfo_all_nix_base32_chars1621=== CONT TestIsValidCachePath/short_hash1622=== CONT TestIsValidCachePath/wrong_extension1623--- PASS: TestIsValidCachePath (0.00s)1624 --- PASS: TestIsValidCachePath/narinfo (0.00s)1625 --- PASS: TestIsValidCachePath/empty (0.00s)1626 --- PASS: TestIsValidCachePath/random_path (0.00s)1627 --- PASS: TestIsValidCachePath/invalid_char_u (0.00s)1628 --- PASS: TestIsValidCachePath/invalid_char_e (0.00s)1629 --- PASS: TestIsValidCachePath/leading_slash (0.00s)1630 --- PASS: TestIsValidCachePath/traversal_in_middle (0.00s)1631 --- PASS: TestIsValidCachePath/traversal_parent (0.00s)1632 --- PASS: TestIsValidCachePath/index.html (0.00s)1633 --- PASS: TestIsValidCachePath/nix-cache-info (0.00s)1634 --- PASS: TestIsValidCachePath/realisation (0.00s)1635 --- PASS: TestIsValidCachePath/log (0.00s)1636 --- PASS: TestIsValidCachePath/ls (0.00s)1637 --- PASS: TestIsValidCachePath/nar_uncompressed (0.00s)1638 --- PASS: TestIsValidCachePath/nar_bz2 (0.00s)1639 --- PASS: TestIsValidCachePath/nar_xz (0.00s)1640 --- PASS: TestIsValidCachePath/nar_zst (0.00s)1641 --- PASS: TestIsValidCachePath/narinfo_all_nix_base32_chars (0.00s)1642 --- PASS: TestIsValidCachePath/short_hash (0.00s)1643 --- PASS: TestIsValidCachePath/wrong_extension (0.00s)1644=== CONT TestClientErrorHandling/InvalidStorePath16452026/08/27 09:31:36 INFO Signed narinfos id=1 count=116462026/08/27 09:31:36 INFO Uploading 1 narinfos1647=== NAME TestClientIntegration1648 client_integration_test.go:293: Retrieved .ls file from S3 (compressed size: 77 bytes)1649 client_integration_test.go:293: Decompressed .ls content (64 bytes):1650 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}1651 client_integration_test.go:296: Testing garbage collection...1652=== NAME TestNARDeduplicationMetadataUploadBug1653 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)1654 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):1655 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}16562026/08/27 09:31:36 WARN Failed to register uploaded object key=1f7nl8ygvlcbjhp3psc85pj4hlviazhn.narinfo error="server returned 404: 404 page not found\n"16572026/08/27 09:31:36 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete16582026/08/27 09:31:36 INFO Completed upload id=116592026/08/27 09:31:36 INFO Upload complete. (956ms)1660--- PASS: TestReadProxyNarStreaming (2.76s)1661=== CONT TestClientErrorHandling/InvalidAuthToken16622026/08/27 09:31:36 INFO Starting cleanup of old closures method=DELETE path=/api/closures16632026/08/27 09:31:36 INFO Garbage collection started16642026/08/27 09:31:36 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"1665=== NAME TestNARDeduplicationMetadataUploadBug1666 metadata_upload_test.go:64: Second store path (same content): /build/TestNARDeduplicationMetadataUploadBug2991872231/001/store/vldp0b2p4qa6zhd9hqn3gg8wcyi6a0wd-file2.txt1667=== NAME TestClientCADerivations1668 client_ca_test.go:180: Narinfo contains CA field: StorePath: /build/TestClientCADerivations2356769355/001/store/1f7nl8ygvlcbjhp3psc85pj4hlviazhn-ca-test1669 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1670 Compression: zstd1671 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1672 NarSize: 1441673 References: 1674 Deriver: /build/TestClientCADerivations2356769355/001/store/iyd7l0ygjkhdmw87zq5kk85cks07f9w2-ca-test.drv1675 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1676 client_ca_test.go:185: Checking for realisation files in S3...1677 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1678 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache16792026/08/27 09:31:36 WARN Failed to register uploaded object key=nyczl96zhiajvzpp6v0npbz6rscily4l.ls error="server returned 404: 404 page not found\n"16802026/08/27 09:31:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign16812026/08/27 09:31:36 INFO Signed narinfos id=1 count=116822026/08/27 09:31:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign16832026/08/27 09:31:36 INFO Signed narinfos id=2 count=116842026/08/27 09:31:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign1685--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (1.11s)1686=== CONT TestClientErrorHandling/ServerNotAvailable16872026/08/27 09:31:36 INFO Signed narinfos id=3 count=116882026/08/27 09:31:36 INFO Uploading 3 narinfos16892026/08/27 09:31:36 INFO Aborted multipart uploads count=016902026/08/27 09:31:36 WARN Failed to register uploaded object key=1jlxv03aa3fxfypsx8vv7jfaxkgxj8mr.narinfo error="server returned 404: 404 page not found\n"16912026/08/27 09:31:36 WARN Force mode enabled - objects will be deleted immediately without grace period16922026/08/27 09:31:36 WARN Failed to register uploaded object key=nyczl96zhiajvzpp6v0npbz6rscily4l.narinfo error="server returned 404: 404 page not found\n"16932026/08/27 09:31:36 WARN Failed to register uploaded object key=dg7ac5w20skkcvmpb4yc8ssyw98bvkdm.narinfo error="server returned 404: 404 page not found\n"16942026/08/27 09:31:36 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete16952026/08/27 09:31:36 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1696--- PASS: TestReadProxyInvalidPath (2.93s)1697=== CONT TestCacheConfigHandler/full_config,_no_issuer16982026/08/27 09:31:36 INFO Completed upload id=116992026/08/27 09:31:36 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete1700=== CONT TestCacheConfigHandler/no_signing_keys1701=== CONT TestCacheConfigHandler/no_cache_url_configured1702=== CONT TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1703--- PASS: TestCacheConfigHandler (0.00s)1704 --- PASS: TestCacheConfigHandler/full_config,_no_issuer (0.00s)1705 --- PASS: TestCacheConfigHandler/no_signing_keys (0.00s)1706 --- PASS: TestCacheConfigHandler/no_cache_url_configured (0.00s)1707 --- PASS: TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator (0.00s)1708=== CONT TestServerTLSConfig/no_client_CA1709=== CONT TestServerTLSConfig/not_a_PEM_file1710=== CONT TestServerTLSConfig/missing_CA_file1711--- PASS: TestServerTLSConfig (0.00s)1712 --- PASS: TestServerTLSConfig/no_client_CA (0.00s)1713 --- PASS: TestServerTLSConfig/not_a_PEM_file (0.00s)1714 --- PASS: TestServerTLSConfig/missing_CA_file (0.00s)1715=== CONT TestParseSingleRange/none1716=== CONT TestParseSingleRange/open-ended1717=== CONT TestParseSingleRange/start_far_past_EOF1718=== CONT TestParseSingleRange/start_past_EOF1719=== CONT TestParseSingleRange/single_byte1720=== CONT TestParseSingleRange/suffix_exceeds_size1721=== CONT TestParseSingleRange/suffix1722=== CONT TestParseSingleRange/end_clamped_to_size1723=== CONT TestParseSingleRange/malformed_both_empty1724=== CONT TestParseSingleRange/closed1725=== CONT TestParseSingleRange/malformed_end_before_start1726=== CONT TestParseSingleRange/multi-range_ignored1727=== CONT TestParseSingleRange/unknown_unit1728=== CONT TestParseSingleRange/malformed_no_dash1729--- PASS: TestParseSingleRange (0.00s)1730 --- PASS: TestParseSingleRange/none (0.00s)1731 --- PASS: TestParseSingleRange/open-ended (0.00s)1732 --- PASS: TestParseSingleRange/start_far_past_EOF (0.00s)1733 --- PASS: TestParseSingleRange/start_past_EOF (0.00s)1734 --- PASS: TestParseSingleRange/single_byte (0.00s)1735 --- PASS: TestParseSingleRange/suffix_exceeds_size (0.00s)1736 --- PASS: TestParseSingleRange/suffix (0.00s)1737 --- PASS: TestParseSingleRange/end_clamped_to_size (0.00s)1738 --- PASS: TestParseSingleRange/malformed_both_empty (0.00s)1739 --- PASS: TestParseSingleRange/closed (0.00s)1740 --- PASS: TestParseSingleRange/malformed_end_before_start (0.00s)1741 --- PASS: TestParseSingleRange/multi-range_ignored (0.00s)1742 --- PASS: TestParseSingleRange/unknown_unit (0.00s)1743 --- PASS: TestParseSingleRange/malformed_no_dash (0.00s)1744=== CONT TestService_AuthMiddleware_OIDC/valid_OIDC_token17452026/08/27 09:31:36 INFO Completed upload id=217462026/08/27 09:31:36 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete17472026/08/27 09:31:36 INFO OIDC auth successful provider=test1748=== CONT TestService_AuthMiddleware_OIDC/malformed_token_rejected17492026/08/27 09:31:36 WARN Authentication failed token_preview=not-a-valid-jwt token_length=15 oidc_error="no provider could verify the token (signature or issuer mismatch)" oidc_provider="" tried_providers=[test]1750=== CONT TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1751=== CONT TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected17522026/08/27 09:31:36 INFO Completed upload id=317532026/08/27 09:31:36 INFO Upload complete. (1.204s)1754=== NAME TestClientMultipleUploads1755 client_integration_test.go:349: Uploaded 3 paths in 1.242882904s17562026/08/27 09:31:36 WARN Authentication failed token_preview=eyJhbGciOi...U2-pZuep9A token_length=702 oidc_error="bound claims validation failed: claim \"repository_owner\" value [otherorg] not in allowed patterns [myorg]" oidc_provider=test tried_providers=[test]1757--- PASS: TestService_AuthMiddleware_OIDC (1.27s)1758 --- PASS: TestService_AuthMiddleware_OIDC/valid_OIDC_token (0.01s)1759 --- PASS: TestService_AuthMiddleware_OIDC/malformed_token_rejected (0.00s)1760 --- PASS: TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured (0.00s)1761 --- PASS: TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected (0.00s)1762--- PASS: TestService_healthCheckHandler (3.06s)1763--- PASS: TestClientMultipleUploads (1.99s)17642026-08-27 09:31:36.893 UTC [1224] ERROR: relation "goose_db_version" does not exist at character 3617652026-08-27 09:31:36.893 UTC [1224] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC17662026/08/27 09:31:36 INFO Received uploads request method=POST path=/api/pending_closures1767--- PASS: TestReadProxyNarinfoAlreadyDecompressed (2.73s)17682026/08/27 09:31:36 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"1769--- PASS: TestResurrectedObjectNotDeleted (1.30s)17702026-08-27 09:31:36.924 UTC [1271] ERROR: relation "goose_db_version" does not exist at character 3617712026-08-27 09:31:36.924 UTC [1271] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC17722026/08/27 09:31:36 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"17732026/08/27 09:31:36 WARN mTLS auth: bound subjects configured but subject DN unavailable17742026/08/27 09:31:36 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1775--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (1.29s)17762026/08/27 09:31:36 OK 20241026095416_initial_model.sql (16.45ms)17772026/08/27 09:31:36 OK 20251210153512_drop_unused_gin_index.sql (1.56ms)17782026/08/27 09:31:36 OK 20251218171726_add_pins.sql (7.94ms)17792026/08/27 09:31:36 OK 20241026095416_initial_model.sql (11.09ms)17802026/08/27 09:31:36 INFO Created nix-cache-info in bucket bucket=bucket2517812026/08/27 09:31:36 OK 20260628120000_add_object_size_and_stats.sql (3.62ms)17822026/08/27 09:31:36 goose: successfully migrated database to version: 2026062812000017832026/08/27 09:31:36 OK 20251210153512_drop_unused_gin_index.sql (2.02ms)17842026/08/27 09:31:36 OK 1_commit_pending_closure.sql (1.82ms)17852026/08/27 09:31:36 OK 2_object_stats_trigger.sql (712.87µs)17862026/08/27 09:31:36 goose: up to current file version: 217872026/08/27 09:31:36 OK 20251218171726_add_pins.sql (2.62ms)17882026/08/27 09:31:36 INFO Received uploads request method=POST path=/api/pending_closures17892026/08/27 09:31:36 OK 20260628120000_add_object_size_and_stats.sql (2.74ms)17902026/08/27 09:31:36 goose: successfully migrated database to version: 2026062812000017912026/08/27 09:31:36 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)17922026/08/27 09:31:36 OK 1_commit_pending_closure.sql (1.79ms)17932026/08/27 09:31:36 OK 2_object_stats_trigger.sql (1.73ms)17942026/08/27 09:31:36 goose: up to current file version: 217952026/08/27 09:31:36 WARN Failed to register uploaded object key=vldp0b2p4qa6zhd9hqn3gg8wcyi6a0wd.ls error="server returned 404: 404 page not found\n"17962026/08/27 09:31:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign17972026/08/27 09:31:36 INFO Signed narinfos id=2 count=117982026/08/27 09:31:36 INFO Received complete multipart upload request method=POST path=/api/multipart/complete17992026/08/27 09:31:36 INFO Uploading 1 narinfos18002026/08/27 09:31:36 WARN Failed to register uploaded object key=vldp0b2p4qa6zhd9hqn3gg8wcyi6a0wd.narinfo error="server returned 404: 404 page not found\n"18012026/08/27 09:31:36 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete18022026/08/27 09:31:36 INFO Completed upload id=218032026/08/27 09:31:36 INFO Upload complete. (94ms)1804=== NAME TestNARDeduplicationMetadataUploadBug1805 metadata_upload_test.go:76: Retrieved narinfo from S3:1806 StorePath: /build/TestNARDeduplicationMetadataUploadBug2991872231/001/store/vldp0b2p4qa6zhd9hqn3gg8wcyi6a0wd-file2.txt1807 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1808 Compression: zstd1809 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1810 NarSize: 1601811 References: 1812 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf18132026/08/27 09:31:36 WARN Request failed, retrying attempt=1 max_attempts=6 backoff=100ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config18142026/08/27 09:31:36 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=MzE5NzViNTQtNmY5Ni00YmI4LTk2YWQtY2JlYjY5ZDUyNjk0LmE3MzgyZWU4LWMzNDQtNDc0ZC05NTQ2LTg0MzFlNTZlODY3OHgxNzg3ODIzMDk0MTY3NjI1NDQ3 parts=1018152026/08/27 09:31:36 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete18162026/08/27 09:31:36 INFO Completed upload id=118172026/08/27 09:31:36 INFO Received uploads request method=POST path=/api/pending_closures18182026/08/27 09:31:36 INFO Received uploads request method=POST path=/api/pending_closures18192026/08/27 09:31:36 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo18202026/08/27 09:31:36 WARN Found objects in DB but missing from S3, will re-upload count=11821--- PASS: TestService_verifyS3Integrity (3.18s)1822=== NAME TestPinProtectsFromGC1823 client_integration_test.go:646: Pinned store path: /build/TestPinProtectsFromGC1685997211/001/store/gdlsda2g2n8bkyx8zjm6vji0qkam5070-pinned-file.txt1824 client_integration_test.go:647: Unpinned store path: /build/TestPinProtectsFromGC1685997211/001/store/y4crfn21zmw84x4b0n4fpxhzx5dg64q4-unpinned-file.txt18252026/08/27 09:31:37 INFO Received cleanup request method=DELETE path=/api/pending_closures18262026/08/27 09:31:37 INFO Aborted multipart uploads count=118272026/08/27 09:31:37 WARN Request failed, retrying attempt=2 max_attempts=6 backoff=210.030287ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config18282026/08/27 09:31:37 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"18292026/08/27 09:31:37 INFO Received uploads request method=POST path=/api/pending_closures18302026/08/27 09:31:37 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)18312026/08/27 09:31:37 INFO Uploading gdlsda2g2n8bkyx8zjm6vji0qkam5070-pinned-file.txt (128B)18322026/08/27 09:31:37 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"18332026/08/27 09:31:37 WARN Failed to register uploaded object key=gdlsda2g2n8bkyx8zjm6vji0qkam5070.ls error="server returned 404: 404 page not found\n"18342026/08/27 09:31:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign18352026/08/27 09:31:37 INFO Signed narinfos id=1 count=118362026/08/27 09:31:37 INFO Uploading 1 narinfos18372026/08/27 09:31:37 WARN Failed to register uploaded object key=gdlsda2g2n8bkyx8zjm6vji0qkam5070.narinfo error="server returned 404: 404 page not found\n"18382026/08/27 09:31:37 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete18392026/08/27 09:31:37 INFO Completed upload id=118402026/08/27 09:31:37 INFO Upload complete. (205ms)1841=== NAME TestClientCADerivations1842 client_ca_test.go:258: nix copy output: warning: you don't have Internet access; disabling some network-dependent features1843 warning: failed to create TLS context for AWS credential providers; SSO, STS WebIdentity, and ECS container authentication will be unavailable1844 error: binary cache 's3://bucket34?endpoint=http://localhost:38273&region=eu-west-1' is for Nix stores with prefix '/nix/store', not '/build/TestClientCADerivations2356769355/001/store'1845 client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 11846--- PASS: TestClientCADerivations (1.81s)18472026/08/27 09:31:37 WARN Request failed, retrying attempt=3 max_attempts=6 backoff=362.266771ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config18482026/08/27 09:31:37 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"18492026/08/27 09:31:37 INFO Received uploads request method=POST path=/api/pending_closures18502026/08/27 09:31:37 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)18512026/08/27 09:31:37 INFO Uploading y4crfn21zmw84x4b0n4fpxhzx5dg64q4-unpinned-file.txt (128B)18522026/08/27 09:31:37 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"18532026/08/27 09:31:37 WARN Failed to register uploaded object key=y4crfn21zmw84x4b0n4fpxhzx5dg64q4.ls error="server returned 404: 404 page not found\n"18542026/08/27 09:31:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign18552026/08/27 09:31:37 INFO Signed narinfos id=2 count=118562026/08/27 09:31:37 INFO Uploading 1 narinfos18572026/08/27 09:31:37 WARN Failed to register uploaded object key=y4crfn21zmw84x4b0n4fpxhzx5dg64q4.narinfo error="server returned 404: 404 page not found\n"18582026/08/27 09:31:37 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete18592026/08/27 09:31:37 INFO Completed upload id=218602026/08/27 09:31:37 INFO Upload complete. (82ms)18612026/08/27 09:31:37 INFO Received create pin request method=POST path=/api/pins/myapp18622026/08/27 09:31:37 INFO Created/updated pin name=myapp store_path=/build/TestPinProtectsFromGC1685997211/001/store/gdlsda2g2n8bkyx8zjm6vji0qkam5070-pinned-file.txt narinfo_key=gdlsda2g2n8bkyx8zjm6vji0qkam5070.narinfo18632026/08/27 09:31:37 INFO Starting cleanup of old closures method=DELETE path=/api/closures18642026/08/27 09:31:37 INFO Garbage collection started18652026/08/27 09:31:37 INFO Aborted multipart uploads count=018662026/08/27 09:31:37 WARN Force mode enabled - objects will be deleted immediately without grace period18672026/08/27 09:31:37 WARN Request failed, retrying attempt=4 max_attempts=6 backoff=867.340754ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config1868--- PASS: TestMultipartCleanup (2.01s)1869=== NAME TestNARDeduplicationMetadataUploadBug1870 metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes)1871 metadata_upload_test.go:77: Decompressed .ls content (49 bytes):1872 {"version":1,"root":{"type":"regular","size":44}}1873--- PASS: TestNARDeduplicationMetadataUploadBug (2.52s)18742026/08/27 09:31:37 INFO Received complete multipart upload request method=POST path=/api/multipart/complete18752026/08/27 09:31:37 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=MzE5NzViNTQtNmY5Ni00YmI4LTk2YWQtY2JlYjY5ZDUyNjk0LjNmZDRlODg2LWFiZTYtNGRiOC04MWJlLTM4NmEzZTM5MDc5ZHgxNzg3ODIzMDk0OTUwOTY0NTYz parts=1018762026/08/27 09:31:37 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete18772026/08/27 09:31:37 INFO Completed upload id=118782026/08/27 09:31:37 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000018792026/08/27 09:31:37 INFO Received uploads request method=POST path=/api/pending_closures18802026/08/27 09:31:37 INFO Starting cleanup of old closures method=DELETE path=/api/closures18812026/08/27 09:31:37 INFO Aborted multipart uploads count=018822026/08/27 09:31:37 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=0 objects-failed-to-delete=018832026/08/27 09:31:37 INFO Vacuumed table table=pending_closures1884=== NAME TestOrphanedObjectsGC1885 orphaned_objects_gc_test.go:290: GC Test Summary:1886 orphaned_objects_gc_test.go:291: - Kept: 2 objects from closure A1887 orphaned_objects_gc_test.go:292: - Deleted: 2 objects from closure B1888 orphaned_objects_gc_test.go:293: - Deleted: 6 orphaned chain objects (X1->X2->X3)1889 orphaned_objects_gc_test.go:294: - Deleted: 2 orphaned single objects (Y)1890 orphaned_objects_gc_test.go:295: - Total deleted: 10 objects1891--- PASS: TestOrphanedObjectsGC (2.30s)18922026/08/27 09:31:37 INFO Vacuumed table table=pending_objects18932026/08/27 09:31:37 INFO Vacuumed table table=multipart_uploads18942026/08/27 09:31:37 INFO Received complete multipart upload request method=POST path=/api/multipart/complete18952026/08/27 09:31:37 INFO Vacuumed table table=closures18962026/08/27 09:31:37 INFO Vacuumed table table=objects18972026/08/27 09:31:37 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"18982026/08/27 09:31:37 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=MzE5NzViNTQtNmY5Ni00YmI4LTk2YWQtY2JlYjY5ZDUyNjk0LmI4MDc4NzMzLWVlNzItNDFiZC1iNDk1LWE1OTQ0YTZmODRjOXgxNzg3ODIzMDk0ODg5OTkwODg0 parts=1218992026/08/27 09:31:37 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001900--- PASS: TestRedundantMultipartUpload (4.04s)1901--- PASS: TestService_createPendingClosureHandler (4.04s)19022026/08/27 09:31:37 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1903--- PASS: TestUploadHandlersRejectOversizedBody (0.24s)1904 --- PASS: TestUploadHandlersRejectOversizedBody/request_more_parts (0.09s)1905 --- PASS: TestUploadHandlersRejectOversizedBody/complete_multipart (0.10s)1906 --- PASS: TestUploadHandlersRejectOversizedBody/create_pending_closure (1.17s)19072026/08/27 09:31:37 WARN Authentication failed token_preview=invalid-token token_length=13 reason="static API token mismatch"19082026/08/27 09:31:37 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=2310 objects-failed-to-delete=019092026/08/27 09:31:37 INFO Vacuumed table table=pending_closures19102026/08/27 09:31:37 INFO Vacuumed table table=pending_objects19112026/08/27 09:31:37 INFO Vacuumed table table=multipart_uploads19122026/08/27 09:31:37 INFO Vacuumed table table=closures19132026/08/27 09:31:37 INFO Vacuumed table table=objects19142026/08/27 09:31:37 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=MzE5NzViNTQtNmY5Ni00YmI4LTk2YWQtY2JlYjY5ZDUyNjk0LjE5OTZjNjljLTIyYjktNGM2Yy1iMzhkLTZhYjI1NmFiYTUyZngxNzg3ODIzMDk0OTc2ODMxMTc5 parts=1219152026/08/27 09:31:37 INFO Received uploads request method=POST path=/api/pending_closures1916--- PASS: TestCompletedNarNotReofferedAcrossClosures (4.16s)1917=== NAME TestOrphanedObjectsGCStressTest1918 orphaned_objects_gc_test.go:425: Created 10 active closures, 5 to-delete closures, 20 orphaned chains1919 orphaned_objects_gc_test.go:446: Marked 210 objects for deletion19202026/08/27 09:31:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=2001 objects-failed-to-delete=019212026/08/27 09:31:38 INFO Vacuumed table table=pending_closures19222026/08/27 09:31:38 INFO Vacuumed table table=pending_objects19232026/08/27 09:31:38 INFO Vacuumed table table=multipart_uploads19242026/08/27 09:31:38 INFO Vacuumed table table=closures19252026/08/27 09:31:38 INFO Vacuumed table table=objects1926 orphaned_objects_gc_test.go:509: Stress test completed successfully:1927 orphaned_objects_gc_test.go:510: - Active objects preserved: 201928 orphaned_objects_gc_test.go:511: - Objects deleted: 2101929 orphaned_objects_gc_test.go:512: - Total GC'd: 21019302026/08/27 09:31:38 WARN Request failed, retrying attempt=5 max_attempts=6 backoff=1.729771018s error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config1931--- PASS: TestOrphanedObjectsGCStressTest (3.03s)19322026/08/27 09:31:38 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=2001 objects_failed=01933=== NAME TestClientIntegration1934 client_integration_test.go:303: Objects in database after GC:1935 client_integration_test.go:303: Successfully deleted all objects with GC --force1936--- PASS: TestClientIntegration (3.90s)19372026/08/27 09:31:39 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=2310 objects_failed=01938=== NAME TestPinProtectsFromGC1939 client_integration_test.go:709: Pin successfully protected closure from garbage collection1940--- PASS: TestPinProtectsFromGC (5.33s)19412026/08/27 09:31:40 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: sending request: request failed after retries: Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused"19422026/08/27 09:31:40 WARN Request failed, retrying attempt=1 max_attempts=6 backoff=100ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures19432026/08/27 09:31:40 WARN Rate limiter enabled after throttle name=s3-test rate=519442026/08/27 09:31:40 WARN S3 rate limit hit operation="complete multipart upload" error="Please reduce your request rate."1945=== NAME TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle1946 throttle_test.go:213: Proxy stats: total=15, throttled=10, completeMultipart=1019472026/08/27 09:31:40 WARN Request failed, retrying attempt=2 max_attempts=6 backoff=211.940058ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures1948 throttle_test.go:215: Rate limiter: enabled=true, rate=5.001949--- PASS: TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle (6.57s)19502026/08/27 09:31:40 WARN Request failed, retrying attempt=3 max_attempts=6 backoff=437.235906ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures19512026/08/27 09:31:41 WARN Request failed, retrying attempt=4 max_attempts=6 backoff=758.598802ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures19522026/08/27 09:31:41 WARN Request failed, retrying attempt=5 max_attempts=6 backoff=1.723733941s error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures1953--- PASS: TestClientErrorHandling (0.00s)1954 --- PASS: TestClientErrorHandling/InvalidStorePath (0.98s)1955 --- PASS: TestClientErrorHandling/InvalidAuthToken (1.13s)1956 --- PASS: TestClientErrorHandling/ServerNotAvailable (6.69s)1957PASS19582026-08-27 09:31:43.830 UTC [112] LOG: received smart shutdown request19592026-08-27 09:31:43.836 UTC [112] LOG: background worker "logical replication launcher" (PID 122) exited with exit code 119602026-08-27 09:31:43.848 UTC [117] LOG: shutting down19612026-08-27 09:31:43.849 UTC [117] LOG: checkpoint starting: shutdown immediate19622026-08-27 09:31:44.997 UTC [117] LOG: checkpoint complete: wrote 12082 buffers (73.7%), wrote 3 SLRU buffers; 0 WAL file(s) added, 0 removed, 13 recycled; write=0.232 s, sync=0.895 s, total=1.150 s; sync files=15167, longest=0.006 s, average=0.001 s; distance=208933 kB, estimate=208933 kB; lsn=0/E36D280, redo lsn=0/E36D28019632026-08-27 09:31:45.091 UTC [112] LOG: database system is shut down1964Running OIDC tests...1965=== RUN TestGlobMatch1966=== PAUSE TestGlobMatch1967=== RUN TestAudienceForIssuer1968=== PAUSE TestAudienceForIssuer1969=== RUN TestValidateToken_ValidToken1970=== PAUSE TestValidateToken_ValidToken1971=== RUN TestValidateToken_WrongAudience1972=== PAUSE TestValidateToken_WrongAudience1973=== RUN TestValidateToken_Expired1974=== PAUSE TestValidateToken_Expired1975=== RUN TestValidateToken_BoundClaimsMismatch1976=== PAUSE TestValidateToken_BoundClaimsMismatch1977=== RUN TestValidateToken_BoundSubjectMismatch1978=== PAUSE TestValidateToken_BoundSubjectMismatch1979=== RUN TestValidateToken_MultipleProviders1980=== PAUSE TestValidateToken_MultipleProviders1981=== RUN TestValidateToken_NoMatchingProvider1982=== PAUSE TestValidateToken_NoMatchingProvider1983=== CONT TestGlobMatch1984=== CONT TestValidateToken_BoundClaimsMismatch1985=== CONT TestValidateToken_MultipleProviders1986=== CONT TestValidateToken_WrongAudience1987=== RUN TestGlobMatch/foo_foo1988=== PAUSE TestGlobMatch/foo_foo1989=== RUN TestGlobMatch/foo_bar1990=== PAUSE TestGlobMatch/foo_bar1991=== RUN TestGlobMatch/*_1992=== CONT TestValidateToken_ValidToken1993=== CONT TestAudienceForIssuer1994--- PASS: TestAudienceForIssuer (0.00s)1995=== CONT TestValidateToken_BoundSubjectMismatch1996=== CONT TestValidateToken_Expired1997=== CONT TestValidateToken_NoMatchingProvider1998=== PAUSE TestGlobMatch/*_1999=== RUN TestGlobMatch/*_anything2000=== PAUSE TestGlobMatch/*_anything2001=== RUN TestGlobMatch/foo*_foo2002=== PAUSE TestGlobMatch/foo*_foo2003=== RUN TestGlobMatch/foo*_foobar2004=== PAUSE TestGlobMatch/foo*_foobar2005=== RUN TestGlobMatch/foo*_bar2006=== PAUSE TestGlobMatch/foo*_bar2007=== RUN TestGlobMatch/*bar_bar2008=== PAUSE TestGlobMatch/*bar_bar2009=== RUN TestGlobMatch/*bar_foobar2010=== PAUSE TestGlobMatch/*bar_foobar2011=== RUN TestGlobMatch/*bar_foo2012=== PAUSE TestGlobMatch/*bar_foo2013=== RUN TestGlobMatch/foo*bar_foobar2014=== PAUSE TestGlobMatch/foo*bar_foobar2015=== RUN TestGlobMatch/foo*bar_foo123bar2016=== PAUSE TestGlobMatch/foo*bar_foo123bar2017=== RUN TestGlobMatch/foo*bar_foobarbaz2018=== PAUSE TestGlobMatch/foo*bar_foobarbaz2019=== RUN TestGlobMatch/*/*_foo/bar2020=== PAUSE TestGlobMatch/*/*_foo/bar2021=== RUN TestGlobMatch/*/*_foo2022=== PAUSE TestGlobMatch/*/*_foo2023=== RUN TestGlobMatch/refs/heads/*_refs/heads/main2024=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main2025=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.02026=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.02027=== RUN TestGlobMatch/refs/*/main_refs/heads/main2028=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main2029=== RUN TestGlobMatch/fo?_foo2030=== PAUSE TestGlobMatch/fo?_foo2031=== RUN TestGlobMatch/fo?_fo2032=== PAUSE TestGlobMatch/fo?_fo2033=== RUN TestGlobMatch/fo?_fooo2034=== PAUSE TestGlobMatch/fo?_fooo2035=== RUN TestGlobMatch/?oo_foo2036=== PAUSE TestGlobMatch/?oo_foo2037=== RUN TestGlobMatch/?oo_boo2038=== PAUSE TestGlobMatch/?oo_boo2039=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main2040=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main2041=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main2042=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main2043=== CONT TestGlobMatch/foo_foo2044=== CONT TestGlobMatch/*/*_foo/bar2045=== CONT TestGlobMatch/fo?_foo2046=== CONT TestGlobMatch/*_2047=== CONT TestGlobMatch/foo*_foo2048=== CONT TestGlobMatch/*_anything2049=== CONT TestGlobMatch/foo*_bar2050=== CONT TestGlobMatch/fo?_fo2051=== CONT TestGlobMatch/foo_bar2052=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main2053=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main2054=== CONT TestGlobMatch/?oo_boo2055=== CONT TestGlobMatch/?oo_foo2056=== CONT TestGlobMatch/fo?_fooo2057=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.02058=== CONT TestGlobMatch/refs/*/main_refs/heads/main2059=== CONT TestGlobMatch/refs/heads/*_refs/heads/main2060=== CONT TestGlobMatch/*/*_foo2061=== CONT TestGlobMatch/*bar_bar2062=== CONT TestGlobMatch/foo*bar_foobarbaz2063=== CONT TestGlobMatch/foo*bar_foo123bar2064=== CONT TestGlobMatch/foo*bar_foobar2065=== CONT TestGlobMatch/*bar_foo2066=== CONT TestGlobMatch/*bar_foobar2067=== CONT TestGlobMatch/foo*_foobar2068--- PASS: TestGlobMatch (0.00s)2069 --- PASS: TestGlobMatch/foo_foo (0.00s)2070 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)2071 --- PASS: TestGlobMatch/fo?_foo (0.00s)2072 --- PASS: TestGlobMatch/*_ (0.00s)2073 --- PASS: TestGlobMatch/foo*_foo (0.00s)2074 --- PASS: TestGlobMatch/*_anything (0.00s)2075 --- PASS: TestGlobMatch/foo*_bar (0.00s)2076 --- PASS: TestGlobMatch/fo?_fo (0.00s)2077 --- PASS: TestGlobMatch/foo_bar (0.00s)2078 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)2079 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)2080 --- PASS: TestGlobMatch/?oo_boo (0.00s)2081 --- PASS: TestGlobMatch/?oo_foo (0.00s)2082 --- PASS: TestGlobMatch/fo?_fooo (0.00s)2083 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)2084 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)2085 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)2086 --- PASS: TestGlobMatch/*/*_foo (0.00s)2087 --- PASS: TestGlobMatch/*bar_bar (0.00s)2088 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)2089 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)2090 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)2091 --- PASS: TestGlobMatch/*bar_foo (0.00s)2092 --- PASS: TestGlobMatch/*bar_foobar (0.00s)2093 --- PASS: TestGlobMatch/foo*_foobar (0.00s)20942026/08/27 09:31:45 INFO OIDC provider initialized name=test20952026/08/27 09:31:45 INFO OIDC provider initialized name=test20962026/08/27 09:31:45 INFO OIDC provider initialized name=provider120972026/08/27 09:31:45 INFO OIDC provider initialized name=provider120982026/08/27 09:31:45 INFO OIDC provider initialized name=test20992026/08/27 09:31:45 INFO OIDC provider initialized name=test21002026/08/27 09:31:45 INFO OIDC provider initialized name=test21012026/08/27 09:31:45 INFO OIDC provider initialized name=provider22102--- PASS: TestValidateToken_BoundClaimsMismatch (0.01s)2103--- PASS: TestValidateToken_BoundSubjectMismatch (0.01s)2104--- PASS: TestValidateToken_Expired (0.01s)2105--- PASS: TestValidateToken_ValidToken (0.01s)2106--- PASS: TestValidateToken_MultipleProviders (0.01s)2107--- PASS: TestValidateToken_NoMatchingProvider (0.01s)2108--- PASS: TestValidateToken_WrongAudience (0.01s)2109PASS2110Running hook tests...2111=== RUN TestSendPathsEmpty2112=== PAUSE TestSendPathsEmpty2113=== RUN TestQueueEnqueueAndFetch2114=== PAUSE TestQueueEnqueueAndFetch2115=== RUN TestQueueDeduplication2116=== PAUSE TestQueueDeduplication2117=== RUN TestQueueRemove2118=== PAUSE TestQueueRemove2119=== RUN TestQueueFetchBatchLimit2120=== PAUSE TestQueueFetchBatchLimit2121=== RUN TestQueueRetryMovesToBack2122=== PAUSE TestQueueRetryMovesToBack2123=== RUN TestQueueFetchRemoveLifecycle2124=== PAUSE TestQueueFetchRemoveLifecycle2125=== RUN TestQueueConcurrentWriters2126=== PAUSE TestQueueConcurrentWriters2127=== RUN TestQueueRemoveLargeClosure2128=== PAUSE TestQueueRemoveLargeClosure2129=== RUN TestServerClientIntegration2130=== PAUSE TestServerClientIntegration2131=== RUN TestServerQueueError2132=== PAUSE TestServerQueueError2133=== RUN TestGetListenerSocketActivation2134 server_test.go:210: === RUN TestGetListenerSocketActivation2135 --- PASS: TestGetListenerSocketActivation (0.00s)2136 PASS2137 2138--- PASS: TestGetListenerSocketActivation (0.01s)2139=== RUN TestDrainIsolatesPoisonPath2140=== PAUSE TestDrainIsolatesPoisonPath2141=== RUN TestRunNotBlockedByPoisonHead2142=== PAUSE TestRunNotBlockedByPoisonHead2143=== RUN TestDrainGivesUpWhenServerDown2144=== PAUSE TestDrainGivesUpWhenServerDown2145=== RUN TestFailedPathPrunedByLaterClosure2146=== PAUSE TestFailedPathPrunedByLaterClosure2147=== RUN TestWorkerUploadsAndRemoves2148=== PAUSE TestWorkerUploadsAndRemoves2149=== RUN TestWorkerSkipsGCdPaths2150=== PAUSE TestWorkerSkipsGCdPaths2151=== RUN TestWorkerPrunesClosureDeps2152=== PAUSE TestWorkerPrunesClosureDeps2153=== CONT TestSendPathsEmpty2154--- PASS: TestSendPathsEmpty (0.00s)2155=== CONT TestFailedPathPrunedByLaterClosure2156=== CONT TestQueueFetchBatchLimit2157=== CONT TestWorkerPrunesClosureDeps2158=== CONT TestWorkerSkipsGCdPaths2159=== CONT TestWorkerUploadsAndRemoves2160=== CONT TestQueueRemove2161=== CONT TestQueueDeduplication2162=== CONT TestQueueEnqueueAndFetch2163=== CONT TestServerClientIntegration2164=== CONT TestQueueRemoveLargeClosure2165=== CONT TestQueueFetchRemoveLifecycle2166=== CONT TestRunNotBlockedByPoisonHead2167=== CONT TestDrainGivesUpWhenServerDown2168=== CONT TestDrainIsolatesPoisonPath2169=== CONT TestServerQueueError2170=== CONT TestQueueConcurrentWriters2171=== CONT TestQueueRetryMovesToBack21722026/08/27 09:31:46 ERROR Failed to queue paths error="permission denied" count=12173--- PASS: TestServerQueueError (0.00s)2174--- PASS: TestServerClientIntegration (0.00s)21752026/08/27 09:31:46 INFO Uploading batch count=421762026/08/27 09:31:46 ERROR Upload failed error="upload failed" count=421772026/08/27 09:31:46 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainIsolatesPoisonPath4063758672/002/bbb21782026/08/27 09:31:46 INFO Uploading batch count=221792026/08/27 09:31:46 ERROR Upload failed error="upload failed" count=221802026/08/27 09:31:46 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1990454714/002/a21812026/08/27 09:31:46 INFO Upload queue status pending=221822026/08/27 09:31:46 INFO Uploading batch count=12183--- PASS: TestQueueRemove (0.02s)21842026/08/27 09:31:46 ERROR Upload failed error="upload failed" count=121852026/08/27 09:31:46 INFO Upload queue status pending=321862026/08/27 09:31:46 INFO Uploading batch count=121872026/08/27 09:31:46 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1990454714/002/b2188--- PASS: TestQueueFetchBatchLimit (0.02s)21892026/08/27 09:31:46 INFO Upload queue status pending=221902026/08/27 09:31:46 INFO Uploading batch count=12191--- PASS: TestQueueEnqueueAndFetch (0.02s)21922026/08/27 09:31:46 ERROR Upload failed error="upload failed" count=121932026/08/27 09:31:46 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerSkipsGCdPaths985221072/002/nonexistent21942026/08/27 09:31:46 INFO Uploading batch count=121952026/08/27 09:31:46 ERROR Upload failed error="upload failed" count=121962026/08/27 09:31:46 INFO Uploading batch count=121972026/08/27 09:31:46 INFO Upload queue status pending=22198--- PASS: TestQueueFetchRemoveLifecycle (0.02s)21992026/08/27 09:31:46 INFO Uploading batch count=222002026/08/27 09:31:46 ERROR Upload failed error="upload failed" count=222012026/08/27 09:31:46 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1990454714/002/c22022026/08/27 09:31:46 INFO Uploading batch count=222032026/08/27 09:31:46 INFO Uploading batch count=122042026/08/27 09:31:46 INFO Uploading batch count=122052026/08/27 09:31:46 ERROR Upload failed error="upload failed" count=122062026/08/27 09:31:46 INFO Uploading batch count=12207--- PASS: TestQueueRetryMovesToBack (0.02s)22082026/08/27 09:31:46 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1990454714/002/d22092026/08/27 09:31:46 INFO Uploading batch count=22210--- PASS: TestQueueDeduplication (0.02s)22112026/08/27 09:31:46 ERROR Upload failed error="upload failed" count=222122026/08/27 09:31:46 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1990454714/002/e22132026/08/27 09:31:46 INFO Uploading batch count=122142026/08/27 09:31:46 ERROR Upload failed error="upload failed" count=122152026/08/27 09:31:46 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown1990454714/002/f22162026/08/27 09:31:46 ERROR Drain finished with paths left in queue remaining=1022172026/08/27 09:31:46 ERROR Drain finished with paths left in queue remaining=12218--- PASS: TestDrainGivesUpWhenServerDown (0.02s)2219--- PASS: TestDrainIsolatesPoisonPath (0.02s)2220--- PASS: TestFailedPathPrunedByLaterClosure (0.03s)2221--- PASS: TestWorkerPrunesClosureDeps (0.04s)2222--- PASS: TestWorkerUploadsAndRemoves (0.04s)2223--- PASS: TestWorkerSkipsGCdPaths (0.04s)2224--- PASS: TestQueueRemoveLargeClosure (0.32s)2225--- PASS: TestQueueConcurrentWriters (0.37s)22262026/08/27 09:31:47 INFO Uploading batch count=122272026/08/27 09:31:47 INFO Uploading batch count=122282026/08/27 09:31:47 INFO Uploading batch count=122292026/08/27 09:31:47 ERROR Upload failed error="upload failed" count=122302026/08/27 09:31:47 INFO Uploading batch count=122312026/08/27 09:31:47 ERROR Upload failed error="upload failed" count=122322026/08/27 09:31:47 INFO Uploading batch count=122332026/08/27 09:31:47 ERROR Upload failed error="upload failed" count=122342026/08/27 09:31:47 INFO Uploading batch count=122352026/08/27 09:31:47 ERROR Upload failed error="upload failed" count=122362026/08/27 09:31:47 ERROR Drain finished with paths left in queue remaining=12237--- PASS: TestRunNotBlockedByPoisonHead (1.04s)2238PASS

succeeded niks3-go-unit-tests checks.aarch64-linux.go-unit-tests · build #142 · raw

succeeded `niks3-go-unit-tests` `checks.aarch64-linux.go-unit-tests` · build #142 · raw