niks3-go-unit-tests
checks.x86_64-linux.go-unit-tests
· build #148
· raw
1tribuchet: building on jamie2Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestCaseHackSuffix6=== PAUSE TestCaseHackSuffix7=== RUN TestFilterOversizedClosures8=== PAUSE TestFilterOversizedClosures9=== RUN TestPartSizeForNAR10=== PAUSE TestPartSizeForNAR11=== RUN TestUploadMultipart_SupersededByPeer12=== PAUSE TestUploadMultipart_SupersededByPeer13=== RUN TestDumpPathMatchesNix14=== PAUSE TestDumpPathMatchesNix15=== RUN TestDumpPathSingleFile16=== PAUSE TestDumpPathSingleFile17=== RUN TestDumpPathWriterError18=== PAUSE TestDumpPathWriterError19=== RUN TestEncodeNixBase3220=== PAUSE TestEncodeNixBase3221=== RUN TestEncodeNixBase32WithRealHash22=== PAUSE TestEncodeNixBase32WithRealHash23=== RUN TestConvertHashToNix3224=== PAUSE TestConvertHashToNix3225=== RUN TestGetStorePathHash26=== PAUSE TestGetStorePathHash27=== RUN TestPathInfoHashCompatibility28=== PAUSE TestPathInfoHashCompatibility29=== RUN TestParsePathInfoJSON30=== PAUSE TestParsePathInfoJSON31=== RUN TestParsePathInfoJSONMultiplePaths32=== PAUSE TestParsePathInfoJSONMultiplePaths33=== RUN TestPathInfoCACompatibility34=== PAUSE TestPathInfoCACompatibility35=== RUN TestRateLimiterFeedback36=== PAUSE TestRateLimiterFeedback37=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess38=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess39=== RUN TestResolveStorePath40=== PAUSE TestResolveStorePath41=== RUN TestDoWithRetry_BodyReplayedViaGetBody42=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody43=== RUN TestShellSplit44=== PAUSE TestShellSplit45=== RUN TestShellSplitErrors46=== PAUSE TestShellSplitErrors47=== RUN TestSetClientTLS48=== PAUSE TestSetClientTLS49=== RUN TestSetClientTLSDoesNotMutateDefaultTransport50=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport51=== RUN TestSetClientTLSErrors52=== PAUSE TestSetClientTLSErrors53=== RUN TestStaticToken54=== PAUSE TestStaticToken55=== RUN TestFileTokenReadsAndCaches56=== PAUSE TestFileTokenReadsAndCaches57=== RUN TestFileTokenMissing58=== PAUSE TestFileTokenMissing59=== RUN TestFileTokenEmpty60=== PAUSE TestFileTokenEmpty61=== RUN TestScriptTokenNoExpiryRerunsEveryCall62=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall63=== RUN TestScriptTokenCachesUntilRefresh64=== PAUSE TestScriptTokenCachesUntilRefresh65=== RUN TestScriptTokenEmptyToken66=== PAUSE TestScriptTokenEmptyToken67=== RUN TestScriptTokenBadJSON68=== PAUSE TestScriptTokenBadJSON69=== RUN TestScriptTokenScriptFails70=== PAUSE TestScriptTokenScriptFails71=== RUN TestScriptTokenEmptyCommand72=== PAUSE TestScriptTokenEmptyCommand73=== CONT TestDoServerRequestAttachesToken74=== CONT TestResolveStorePath75=== CONT TestScriptTokenEmptyCommand76=== CONT TestEncodeNixBase32WithRealHash77=== CONT TestPathInfoCACompatibility78=== CONT TestSetClientTLSDoesNotMutateDefaultTransport79=== RUN TestPathInfoCACompatibility/null_ca_field80=== PAUSE TestPathInfoCACompatibility/null_ca_field81=== RUN TestPathInfoCACompatibility/old_string_format_-_text82=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text83=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive84=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive85=== RUN TestPathInfoCACompatibility/new_structured_format_-_text86=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text87=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method88=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method89=== CONT TestPathInfoCACompatibility/null_ca_field90=== CONT TestStaticToken91=== CONT TestScriptTokenScriptFails92=== CONT TestFileTokenMissing93=== CONT TestShellSplitErrors94=== CONT TestScriptTokenEmptyToken95=== CONT TestSetClientTLS96=== CONT TestFileTokenReadsAndCaches97--- PASS: TestScriptTokenEmptyCommand (0.00s)98--- PASS: TestEncodeNixBase32WithRealHash (0.00s)99--- PASS: TestStaticToken (0.00s)100--- PASS: TestResolveStorePath (0.00s)101--- PASS: TestFileTokenMissing (0.00s)102--- PASS: TestShellSplitErrors (0.00s)103=== CONT TestParsePathInfoJSON104=== RUN TestParsePathInfoJSON/Nix_format105=== PAUSE TestParsePathInfoJSON/Nix_format106=== RUN TestParsePathInfoJSON/Lix_format107=== CONT TestScriptTokenCachesUntilRefresh108=== CONT TestScriptTokenBadJSON109=== CONT TestParsePathInfoJSONMultiplePaths110=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths111=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths112=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths113=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths114=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess115--- PASS: TestFileTokenReadsAndCaches (0.00s)116=== CONT TestDumpPathSingleFile1172026/08/27 09:40:48 WARN Rate limiter enabled after throttle name=server-test rate=5118=== CONT TestDumpPathWriterError119=== CONT TestRateLimiterFeedback120=== RUN TestRateLimiterFeedback/429_enables_limiter121=== PAUSE TestRateLimiterFeedback/429_enables_limiter122=== RUN TestRateLimiterFeedback/503_enables_limiter123=== PAUSE TestRateLimiterFeedback/503_enables_limiter124=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter125=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter126=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter127=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter128=== CONT TestPathInfoHashCompatibility129=== CONT TestDumpPathMatchesNix130=== CONT TestGetStorePathHash131=== CONT TestConvertHashToNix32132=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method133=== CONT TestPathInfoCACompatibility/new_structured_format_-_text134=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive135=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)136=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)137=== RUN TestGetStorePathHash/valid_store_path138=== RUN TestConvertHashToNix32/SRI_format_to_Nix32139=== CONT TestDoWithRetry_BodyReplayedViaGetBody140=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon141=== CONT TestSetClientTLSErrors142=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon143=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI144=== CONT TestUploadMultipart_SupersededByPeer145=== CONT TestEncodeNixBase32146=== PAUSE TestParsePathInfoJSON/Lix_format147=== CONT TestScriptTokenNoExpiryRerunsEveryCall148=== CONT TestFileTokenEmpty149=== CONT TestPathInfoCACompatibility/old_string_format_-_text150--- PASS: TestScriptTokenScriptFails (0.00s)151=== PAUSE TestGetStorePathHash/valid_store_path152=== CONT TestFilterOversizedClosures153=== RUN TestFilterOversizedClosures/no_limit_keeps_everything154=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything155=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped156=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped157=== RUN TestFilterOversizedClosures/all_closures_skipped158=== PAUSE TestFilterOversizedClosures/all_closures_skipped159=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths160=== CONT TestPartSizeForNAR161=== RUN TestPartSizeForNAR/zero_stays_at_minimum162=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths163=== RUN TestGetStorePathHash/basename_without_hyphen_should_error164=== CONT TestCaseHackSuffix1652026/08/27 09:40:48 WARN Rate limiter enabled after throttle name=server-test rate=5166=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI167=== RUN TestEncodeNixBase32/test_string_hash168=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha5121692026/08/27 09:40:48 WARN Request returned retryable status, retrying attempt=1 max_attempts=6 backoff=0s status=503 url=http://127.0.0.1:44949170=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512171=== RUN TestUploadMultipart_SupersededByPeer/exists172=== RUN TestParsePathInfoJSON/empty_input173=== CONT TestFilterOversizedClosures/all_closures_skipped1742026/08/27 09:40:48 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50175=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32176=== PAUSE TestParsePathInfoJSON/empty_input177=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped178=== RUN TestConvertHashToNix32/already_Nix32_format1792026/08/27 09:40:48 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000180=== CONT TestRateLimiterFeedback/429_enables_limiter181=== PAUSE TestConvertHashToNix32/already_Nix32_format182--- PASS: TestScriptTokenEmptyToken (0.01s)183--- PASS: TestScriptTokenBadJSON (0.00s)184--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.01s)185=== CONT TestShellSplit186=== CONT TestFilterOversizedClosures/no_limit_keeps_everything187=== RUN TestSetClientTLSErrors/missing_cert_file188=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter189=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter190=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error191=== PAUSE TestEncodeNixBase32/test_string_hash1922026/08/27 09:40:48 WARN Rate limiter enabled after throttle name=server-test rate=5193=== RUN TestEncodeNixBase32/empty_input194=== PAUSE TestUploadMultipart_SupersededByPeer/exists195=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum196=== RUN TestParsePathInfoJSON/whitespace_only197=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)198=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI199=== CONT TestRateLimiterFeedback/503_enables_limiter2002026/08/27 09:40:48 WARN Request returned retryable status, retrying attempt=1 max_attempts=2 backoff=0s status=429 url=http://127.0.0.1:42197201=== RUN TestConvertHashToNix32/invalid_format2022026/08/27 09:40:48 WARN Rate limiter backed off name=server-test rate=5203=== RUN TestSetClientTLS/rejects_connection_without_client_cert2042026/08/27 09:40:48 WARN Request returned retryable status, retrying attempt=2 max_attempts=6 backoff=0s status=503 url=http://127.0.0.1:44949205--- PASS: TestDoServerRequestAttachesToken (0.01s)206=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512207=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon208=== PAUSE TestSetClientTLSErrors/missing_cert_file209=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error210=== PAUSE TestEncodeNixBase32/empty_input211=== RUN TestUploadMultipart_SupersededByPeer/missing212=== CONT TestEncodeNixBase32/empty_input213=== PAUSE TestParsePathInfoJSON/whitespace_only214=== CONT TestEncodeNixBase32/test_string_hash215--- PASS: TestPathInfoCACompatibility (0.00s)216 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)217 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)218 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)219 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)220 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)221=== PAUSE TestConvertHashToNix32/invalid_format222=== CONT TestConvertHashToNix32/SRI_format_to_Nix32223=== CONT TestConvertHashToNix32/invalid_format224--- PASS: TestFileTokenEmpty (0.00s)225--- PASS: TestShellSplit (0.00s)226=== PAUSE TestUploadMultipart_SupersededByPeer/missing227=== CONT TestConvertHashToNix32/already_Nix32_format228=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert229--- PASS: TestParsePathInfoJSONMultiplePaths (0.00s)230 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)231 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)232--- PASS: TestFilterOversizedClosures (0.00s)233 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)234 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)235 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)236=== RUN TestPartSizeForNAR/small_stays_at_minimum237--- PASS: TestConvertHashToNix32 (0.01s)238 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)239 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)240 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)241=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error242--- PASS: TestScriptTokenCachesUntilRefresh (0.01s)243=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA244=== RUN TestSetClientTLSErrors/missing_key_file2452026/08/27 09:40:48 WARN Rate limiter backed off name=server-test rate=5246--- PASS: TestEncodeNixBase32 (0.01s)247 --- PASS: TestEncodeNixBase32/empty_input (0.00s)248 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)249=== CONT TestUploadMultipart_SupersededByPeer/missing250=== RUN TestParsePathInfoJSON/invalid_JSON251=== PAUSE TestPartSizeForNAR/small_stays_at_minimum252=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error253=== CONT TestUploadMultipart_SupersededByPeer/exists254=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum255--- PASS: TestPathInfoHashCompatibility (0.01s)256 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)257 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)258 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)259 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)260=== PAUSE TestParsePathInfoJSON/invalid_JSON2612026/08/27 09:40:48 WARN Rate limiter enabled after throttle name=server-test rate=5262=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error2632026/08/27 09:40:48 WARN Request returned retryable status, retrying attempt=1 max_attempts=2 backoff=0s status=503 url=http://127.0.0.1:33991264=== CONT TestParsePathInfoJSON/Nix_format265=== CONT TestParsePathInfoJSON/invalid_JSON266=== CONT TestParsePathInfoJSON/empty_input267=== CONT TestParsePathInfoJSON/Lix_format268=== CONT TestParsePathInfoJSON/whitespace_only269--- PASS: TestParsePathInfoJSON (0.01s)270 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)271 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)272 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)273 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)274 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)275=== CONT TestGetStorePathHash/valid_store_path276=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error277=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error278=== CONT TestGetStorePathHash/basename_without_hyphen_should_error279--- PASS: TestGetStorePathHash (0.01s)280 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)281 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)282 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)283 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)284=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA285--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.01s)286=== PAUSE TestSetClientTLSErrors/missing_key_file287=== RUN TestSetClientTLSErrors/missing_ca_file288=== PAUSE TestSetClientTLSErrors/missing_ca_file289=== RUN TestSetClientTLSErrors/invalid_ca_file290=== PAUSE TestSetClientTLSErrors/invalid_ca_file291=== CONT TestSetClientTLSErrors/missing_cert_file292=== CONT TestSetClientTLSErrors/missing_ca_file293=== RUN TestSetClientTLS/preserves_debug_logging_transport294=== PAUSE TestSetClientTLS/preserves_debug_logging_transport295=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum296=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts2972026/08/27 09:40:48 WARN Rate limiter backed off name=server-test rate=5298=== CONT TestSetClientTLSErrors/invalid_ca_file299=== CONT TestSetClientTLSErrors/missing_key_file300=== CONT TestSetClientTLS/rejects_connection_without_client_cert301=== CONT TestSetClientTLS/preserves_debug_logging_transport302=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA303=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts304--- PASS: TestRateLimiterFeedback (0.00s)305 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.00s)306 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s)307 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.01s)308 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s)309=== RUN TestPartSizeForNAR/1_TiB310=== PAUSE TestPartSizeForNAR/1_TiB311=== RUN TestPartSizeForNAR/5_TiB_S3_max_object312=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object313=== RUN TestPartSizeForNAR/capped_at_5_GiB314=== PAUSE TestPartSizeForNAR/capped_at_5_GiB315=== CONT TestPartSizeForNAR/zero_stays_at_minimum316=== CONT TestPartSizeForNAR/capped_at_5_GiB317=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts318=== CONT TestPartSizeForNAR/5_TiB_S3_max_object319=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum320--- PASS: TestSetClientTLSErrors (0.01s)321 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)322 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)323 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)324 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)325=== CONT TestPartSizeForNAR/1_TiB326=== CONT TestPartSizeForNAR/small_stays_at_minimum327--- PASS: TestUploadMultipart_SupersededByPeer (0.01s)328 --- PASS: TestUploadMultipart_SupersededByPeer/missing (0.00s)329 --- PASS: TestUploadMultipart_SupersededByPeer/exists (0.00s)330--- PASS: TestPartSizeForNAR (0.01s)331 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)332 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)333 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)334 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)335 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)336 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)337 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)338--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.01s)3392026/08/27 09:40:48 http: TLS handshake error from 127.0.0.1:36932: remote error: tls: bad certificate340--- PASS: TestSetClientTLS (0.02s)341 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.00s)342 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.00s)343 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.01s)344--- PASS: TestDumpPathSingleFile (0.03s)345--- PASS: TestCaseHackSuffix (0.03s)346--- PASS: TestDumpPathWriterError (0.04s)347--- PASS: TestDumpPathMatchesNix (0.07s)348--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)349PASS350Running server tests...351The files belonging to this database system will be owned by user "nixbld".352This user must also own the server process.353354The database cluster will be initialized with locale "C".355The default database encoding has accordingly been set to "SQL_ASCII".356The default text search configuration will be set to "english".357358Data page checksums are enabled.359360creating directory /build/postgres748268769/data ... ok361creating subdirectories ... ok362selecting dynamic shared memory implementation ... posix363selecting default "max_connections" ... 100364selecting default "shared_buffers" ... 128MB365selecting default time zone ... UTC366creating configuration files ... ok367running bootstrap script ... ok368performing post-bootstrap initialization ... ok369syncing data to disk ... ok370371initdb: warning: enabling "trust" authentication for local connections372initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.373374Success. You can now start the database server using:375376 pg_ctl -D /build/postgres748268769/data -l logfile start377378/build/postgres748268769:5432 - no response3792026-08-27 09:40:49.995 UTC [112] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit3802026-08-27 09:40:49.995 UTC [112] LOG: listening on Unix socket "/build/postgres748268769/.s.PGSQL.5432"3812026-08-27 09:40:50.000 UTC [119] LOG: database system was shut down at 2026-08-27 09:40:49 UTC3822026-08-27 09:40:50.004 UTC [112] LOG: database system is ready to accept connections383/build/postgres748268769:5432 - accepting connections384=== RUN TestService_AuthMiddleware385=== PAUSE TestService_AuthMiddleware386=== RUN TestService_AuthMiddleware_MTLSProxyHeader387=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader388=== RUN TestService_AuthMiddleware_MTLSBoundSubjects389=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects390=== RUN TestService_ReadAuthMiddleware391=== PAUSE TestService_ReadAuthMiddleware392=== RUN TestService_AuthMiddleware_OIDC393=== PAUSE TestService_AuthMiddleware_OIDC394=== RUN TestCacheConfigHandler395=== PAUSE TestCacheConfigHandler396=== RUN TestCacheStatsHandler397=== PAUSE TestCacheStatsHandler398=== RUN TestClientCADerivations399=== PAUSE TestClientCADerivations400=== RUN TestClientErrorHandling401=== PAUSE TestClientErrorHandling402=== RUN TestClientIntegration403=== PAUSE TestClientIntegration404=== RUN TestClientMultipleUploads405=== PAUSE TestClientMultipleUploads406=== RUN TestClientWithDependencies407=== PAUSE TestClientWithDependencies408=== RUN TestPinProtectsFromGC409=== PAUSE TestPinProtectsFromGC410=== RUN TestGCAdvisoryLockBlocksConcurrentRun4112026-08-27 09:40:50.488 UTC [906] ERROR: relation "goose_db_version" does not exist at character 364122026-08-27 09:40:50.488 UTC [906] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4132026/08/27 09:40:50 OK 20241026095416_initial_model.sql (7ms)4142026/08/27 09:40:50 OK 20251210153512_drop_unused_gin_index.sql (1.03ms)4152026/08/27 09:40:50 OK 20251218171726_add_pins.sql (1.79ms)4162026/08/27 09:40:50 OK 20260628120000_add_object_size_and_stats.sql (1.67ms)4172026/08/27 09:40:50 goose: successfully migrated database to version: 202606281200004182026/08/27 09:40:50 OK 1_commit_pending_closure.sql (1.25ms)4192026/08/27 09:40:50 OK 2_object_stats_trigger.sql (533.75µs)4202026/08/27 09:40:50 goose: up to current file version: 2421--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (0.12s)422=== RUN TestGCBugBareHashReferences423=== PAUSE TestGCBugBareHashReferences424=== RUN TestGCMetrics425=== PAUSE TestGCMetrics426=== RUN TestGCTaskStore_StartNew427=== PAUSE TestGCTaskStore_StartNew428=== RUN TestGCTaskStore_DeduplicateSameParams429=== PAUSE TestGCTaskStore_DeduplicateSameParams430=== RUN TestGCTaskStore_ConflictDifferentParams431=== PAUSE TestGCTaskStore_ConflictDifferentParams432=== RUN TestGCTaskStore_GetEmpty433=== PAUSE TestGCTaskStore_GetEmpty434=== RUN TestGCTaskStore_GetReturnsLatest435=== PAUSE TestGCTaskStore_GetReturnsLatest436=== RUN TestGCTaskStore_CompletedAllowsNewTask437=== PAUSE TestGCTaskStore_CompletedAllowsNewTask438=== RUN TestGCTaskStore_PhaseUpdates439=== PAUSE TestGCTaskStore_PhaseUpdates440=== RUN TestGCTaskStore_Fail441=== PAUSE TestGCTaskStore_Fail442=== RUN TestGracefulShutdownDrainsInflight443=== PAUSE TestGracefulShutdownDrainsInflight444=== RUN TestService_healthCheckHandler445=== PAUSE TestService_healthCheckHandler446=== RUN TestGenerateLandingPage447=== PAUSE TestGenerateLandingPage448=== RUN TestCacheConfigHandlerMaxNarSize449=== PAUSE TestCacheConfigHandlerMaxNarSize450=== RUN TestCreatePendingClosureRejectsOversizedNAR451=== PAUSE TestCreatePendingClosureRejectsOversizedNAR452=== RUN TestNARDeduplicationMetadataUploadBug453=== PAUSE TestNARDeduplicationMetadataUploadBug454=== RUN TestMetricsInventory455=== PAUSE TestMetricsInventory456=== RUN TestService_NativeMTLS457=== PAUSE TestService_NativeMTLS458=== RUN TestServerTLSConfig459=== PAUSE TestServerTLSConfig460=== RUN TestMultipartCleanup461=== PAUSE TestMultipartCleanup462=== RUN TestObjectStatsTrigger463=== PAUSE TestObjectStatsTrigger464=== RUN TestOrphanedObjectsGC465=== PAUSE TestOrphanedObjectsGC466=== RUN TestOrphanedObjectsGCStressTest467=== PAUSE TestOrphanedObjectsGCStressTest468=== RUN TestResurrectedObjectNotDeleted469=== PAUSE TestResurrectedObjectNotDeleted470=== RUN TestParseSingleRange471=== PAUSE TestParseSingleRange472=== RUN TestIsValidCachePath473=== PAUSE TestIsValidCachePath474=== RUN TestReadProxyNarinfo475=== PAUSE TestReadProxyNarinfo476=== RUN TestReadProxyNarinfoAlreadyDecompressed477=== PAUSE TestReadProxyNarinfoAlreadyDecompressed478=== RUN TestReadProxyNarStreaming479=== PAUSE TestReadProxyNarStreaming480=== RUN TestReadProxy404481=== PAUSE TestReadProxy404482=== RUN TestReadProxyInvalidPath483=== PAUSE TestReadProxyInvalidPath484=== RUN TestReadProxyHead485=== PAUSE TestReadProxyHead486=== RUN TestReadProxyConditionalGet487=== PAUSE TestReadProxyConditionalGet488=== RUN TestReadProxyRootRedirectsToIndexHTML489=== PAUSE TestReadProxyRootRedirectsToIndexHTML490=== RUN TestReadProxyDisabled491=== PAUSE TestReadProxyDisabled492=== RUN TestReadProxyRangeRequest493=== PAUSE TestReadProxyRangeRequest494=== RUN TestRedundantMultipartUpload495=== PAUSE TestRedundantMultipartUpload496=== RUN TestCompleteMultipartUpload_ErrorButObjectExists497=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists498=== RUN TestCompletedNarNotReofferedAcrossClosures499=== PAUSE TestCompletedNarNotReofferedAcrossClosures500=== RUN TestPresignedUploadRegisteredBeforeCommit501=== PAUSE TestPresignedUploadRegisteredBeforeCommit502=== RUN TestService_Rustfstest503=== PAUSE TestService_Rustfstest504=== RUN TestParseSize505=== PAUSE TestParseSize506=== RUN TestSkippedUploadsHandler507=== PAUSE TestSkippedUploadsHandler508=== RUN TestSystemdListenerNotActivated509--- PASS: TestSystemdListenerNotActivated (0.00s)510=== RUN TestWatchdogBeatsWhenHealthy511--- PASS: TestWatchdogBeatsWhenHealthy (0.02s)512=== RUN TestWatchdogSkipsWhenUnhealthy5132026/08/27 09:40:50 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5142026/08/27 09:40:50 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5152026/08/27 09:40:50 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5162026/08/27 09:40:50 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5172026/08/27 09:40:50 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5182026/08/27 09:40:50 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5192026/08/27 09:40:50 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5202026/08/27 09:40:50 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5212026/08/27 09:40:50 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"5222026/08/27 09:40:50 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"523--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)524=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle525=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle526=== RUN TestProxyWriteTimeout527=== PAUSE TestProxyWriteTimeout528=== RUN TestIsValidUploadKey529=== PAUSE TestIsValidUploadKey530=== RUN TestUploadHandlersRejectInvalidKeys531=== PAUSE TestUploadHandlersRejectInvalidKeys532=== RUN TestUploadHandlersRejectOversizedBody533=== PAUSE TestUploadHandlersRejectOversizedBody534=== RUN TestService_cleanupPendingClosuresHandler535=== PAUSE TestService_cleanupPendingClosuresHandler536=== RUN TestService_createPendingClosureHandler537=== PAUSE TestService_createPendingClosureHandler538=== RUN TestService_verifyS3Integrity539=== PAUSE TestService_verifyS3Integrity540=== RUN TestCompleteMultipartUnregistered541=== PAUSE TestCompleteMultipartUnregistered542=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT543=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT544=== CONT TestService_AuthMiddleware545=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT546=== CONT TestService_NativeMTLS547=== CONT TestGenerateLandingPage548=== CONT TestCompletedNarNotReofferedAcrossClosures549=== CONT TestGCTaskStore_GetReturnsLatest550--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)551=== CONT TestMetricsInventory552=== CONT TestResurrectedObjectNotDeleted553=== CONT TestOrphanedObjectsGCStressTest554=== CONT TestOrphanedObjectsGC555=== CONT TestObjectStatsTrigger556=== CONT TestMultipartCleanup557=== CONT TestServerTLSConfig558=== RUN TestServerTLSConfig/no_client_CA559=== CONT TestProxyWriteTimeout560=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle561=== CONT TestSkippedUploadsHandler562=== CONT TestParseSize563--- PASS: TestParseSize (0.00s)564=== CONT TestUploadHandlersRejectOversizedBody565=== CONT TestService_Rustfstest566=== CONT TestPresignedUploadRegisteredBeforeCommit567=== CONT TestService_createPendingClosureHandler568=== CONT TestCompleteMultipartUnregistered569=== CONT TestService_verifyS3Integrity570=== CONT TestNARDeduplicationMetadataUploadBug5712026/08/27 09:40:50 INFO Client skipped oversized paths paths=3 nar_bytes=5000000000572=== CONT TestIsValidUploadKey573=== RUN TestIsValidUploadKey/narinfo574=== PAUSE TestIsValidUploadKey/narinfo575=== RUN TestIsValidUploadKey/nar_zst576=== PAUSE TestIsValidUploadKey/nar_zst577=== CONT TestParseSingleRange578=== PAUSE TestServerTLSConfig/no_client_CA579=== RUN TestProxyWriteTimeout/narinfo580=== PAUSE TestProxyWriteTimeout/narinfo581=== RUN TestProxyWriteTimeout/1_GiB_nar582=== PAUSE TestProxyWriteTimeout/1_GiB_nar583=== RUN TestProxyWriteTimeout/10_GiB_nar584=== PAUSE TestProxyWriteTimeout/10_GiB_nar585=== RUN TestProxyWriteTimeout/unknown_size586=== PAUSE TestProxyWriteTimeout/unknown_size587=== RUN TestParseSingleRange/none588=== PAUSE TestParseSingleRange/none589=== RUN TestIsValidUploadKey/nar_xz590=== CONT TestService_cleanupPendingClosuresHandler591--- PASS: TestGenerateLandingPage (0.01s)592--- PASS: TestSkippedUploadsHandler (0.02s)593=== CONT TestReadProxyHead594=== RUN TestServerTLSConfig/missing_CA_file595=== PAUSE TestServerTLSConfig/missing_CA_file596=== RUN TestServerTLSConfig/not_a_PEM_file597=== PAUSE TestServerTLSConfig/not_a_PEM_file598=== PAUSE TestIsValidUploadKey/nar_xz599=== RUN TestIsValidUploadKey/nar_plain600=== PAUSE TestIsValidUploadKey/nar_plain601=== CONT TestUploadHandlersRejectInvalidKeys602=== RUN TestParseSingleRange/unknown_unit603=== PAUSE TestParseSingleRange/unknown_unit604=== RUN TestParseSingleRange/multi-range_ignored605=== PAUSE TestParseSingleRange/multi-range_ignored606=== RUN TestParseSingleRange/malformed_no_dash607=== PAUSE TestParseSingleRange/malformed_no_dash608=== CONT TestCompleteMultipartUpload_ErrorButObjectExists609=== RUN TestParseSingleRange/malformed_both_empty610=== PAUSE TestParseSingleRange/malformed_both_empty611=== RUN TestParseSingleRange/malformed_end_before_start612=== PAUSE TestParseSingleRange/malformed_end_before_start613=== RUN TestParseSingleRange/closed614=== PAUSE TestParseSingleRange/closed615=== RUN TestParseSingleRange/open-ended616=== RUN TestIsValidUploadKey/listing617=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info618=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info619=== PAUSE TestParseSingleRange/open-ended620=== PAUSE TestIsValidUploadKey/listing621=== RUN TestIsValidUploadKey/build_log622=== PAUSE TestIsValidUploadKey/build_log623=== RUN TestIsValidUploadKey/build_log_home-manager_file624=== PAUSE TestIsValidUploadKey/build_log_home-manager_file625=== RUN TestIsValidUploadKey/build_log_plus_in_name626=== PAUSE TestIsValidUploadKey/build_log_plus_in_name627=== RUN TestIsValidUploadKey/build_log_question_mark628=== PAUSE TestIsValidUploadKey/build_log_question_mark629=== RUN TestIsValidUploadKey/build_log_equals630=== PAUSE TestIsValidUploadKey/build_log_equals631=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal632=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal633=== RUN TestParseSingleRange/end_clamped_to_size634=== PAUSE TestParseSingleRange/end_clamped_to_size635=== RUN TestParseSingleRange/suffix636=== PAUSE TestParseSingleRange/suffix637=== RUN TestParseSingleRange/suffix_exceeds_size638=== PAUSE TestParseSingleRange/suffix_exceeds_size639=== RUN TestParseSingleRange/single_byte640=== PAUSE TestParseSingleRange/single_byte641=== RUN TestParseSingleRange/start_past_EOF642=== PAUSE TestParseSingleRange/start_past_EOF643=== RUN TestParseSingleRange/start_far_past_EOF644=== PAUSE TestParseSingleRange/start_far_past_EOF645=== RUN TestIsValidUploadKey/realisation646=== PAUSE TestIsValidUploadKey/realisation647=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key648=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key649=== CONT TestRedundantMultipartUpload650=== RUN TestIsValidUploadKey/realisation_plus_in_output651=== PAUSE TestIsValidUploadKey/realisation_plus_in_output652=== RUN TestIsValidUploadKey/nix-cache-info653=== PAUSE TestIsValidUploadKey/nix-cache-info654=== RUN TestIsValidUploadKey/index.html655=== PAUSE TestIsValidUploadKey/index.html656=== RUN TestIsValidUploadKey/narinfo_key,_nar_type657=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type658=== RUN TestIsValidUploadKey/nar_key,_narinfo_type659=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type660=== RUN TestIsValidUploadKey/listing_key,_narinfo_type661=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type662=== RUN TestIsValidUploadKey/traversal663=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key664=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key665=== PAUSE TestIsValidUploadKey/traversal666=== RUN TestIsValidUploadKey/traversal_nar667=== PAUSE TestIsValidUploadKey/traversal_nar668=== RUN TestIsValidUploadKey/absolute669=== PAUSE TestIsValidUploadKey/absolute670=== RUN TestIsValidUploadKey/empty_key671=== PAUSE TestIsValidUploadKey/empty_key672=== RUN TestIsValidUploadKey/unknown_type673=== PAUSE TestIsValidUploadKey/unknown_type674=== CONT TestReadProxyRangeRequest675=== CONT TestReadProxyDisabled6762026-08-27 09:40:50.930 UTC [977] ERROR: relation "goose_db_version" does not exist at character 366772026-08-27 09:40:50.930 UTC [977] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC678=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure679=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure680=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart681=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart682=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts683=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts684=== CONT TestReadProxyRootRedirectsToIndexHTML6852026-08-27 09:40:50.937 UTC [978] ERROR: relation "goose_db_version" does not exist at character 366862026-08-27 09:40:50.937 UTC [978] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6872026-08-27 09:40:50.943 UTC [979] ERROR: relation "goose_db_version" does not exist at character 366882026-08-27 09:40:50.943 UTC [979] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6892026-08-27 09:40:50.992 UTC [985] ERROR: relation "goose_db_version" does not exist at character 366902026-08-27 09:40:50.992 UTC [985] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6912026-08-27 09:40:51.006 UTC [986] ERROR: relation "goose_db_version" does not exist at character 366922026-08-27 09:40:51.006 UTC [986] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6932026-08-27 09:40:51.008 UTC [987] ERROR: relation "goose_db_version" does not exist at character 366942026-08-27 09:40:51.008 UTC [987] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6952026-08-27 09:40:51.032 UTC [988] ERROR: relation "goose_db_version" does not exist at character 366962026-08-27 09:40:51.032 UTC [988] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC6972026/08/27 09:40:51 OK 20241026095416_initial_model.sql (62.82ms)6982026/08/27 09:40:51 OK 20241026095416_initial_model.sql (63.88ms)6992026/08/27 09:40:51 OK 20241026095416_initial_model.sql (32.72ms)7002026/08/27 09:40:51 OK 20241026095416_initial_model.sql (85.58ms)7012026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.93ms)7022026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (5.34ms)7032026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (9.45ms)7042026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (5.49ms)7052026/08/27 09:40:51 OK 20241026095416_initial_model.sql (33.61ms)7062026/08/27 09:40:51 OK 20251218171726_add_pins.sql (11.73ms)7072026/08/27 09:40:51 OK 20251218171726_add_pins.sql (11.47ms)7082026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (5.37ms)7092026/08/27 09:40:51 OK 20241026095416_initial_model.sql (39.26ms)7102026/08/27 09:40:51 OK 20251218171726_add_pins.sql (13.51ms)7112026/08/27 09:40:51 OK 20251218171726_add_pins.sql (15.73ms)7122026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (4.46ms)7132026/08/27 09:40:51 OK 20251218171726_add_pins.sql (8.51ms)7142026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (11.78ms)7152026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200007162026-08-27 09:40:51.082 UTC [989] ERROR: relation "goose_db_version" does not exist at character 367172026-08-27 09:40:51.082 UTC [989] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7182026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (30.48ms)7192026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200007202026/08/27 09:40:51 OK 1_commit_pending_closure.sql (21.25ms)7212026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (24.62ms)7222026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200007232026/08/27 09:40:51 OK 20251218171726_add_pins.sql (28.62ms)7242026/08/27 09:40:51 OK 20241026095416_initial_model.sql (41.62ms)7252026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (31.25ms)7262026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200007272026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (28.83ms)7282026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200007292026/08/27 09:40:51 OK 1_commit_pending_closure.sql (7.82ms)7302026/08/27 09:40:51 OK 2_object_stats_trigger.sql (5.5ms)7312026/08/27 09:40:51 goose: up to current file version: 27322026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (5.38ms)7332026/08/27 09:40:51 OK 2_object_stats_trigger.sql (5.17ms)7342026/08/27 09:40:51 goose: up to current file version: 27352026/08/27 09:40:51 OK 1_commit_pending_closure.sql (7.04ms)7362026/08/27 09:40:51 OK 1_commit_pending_closure.sql (7.48ms)7372026/08/27 09:40:51 OK 1_commit_pending_closure.sql (7.96ms)7382026/08/27 09:40:51 OK 2_object_stats_trigger.sql (4.45ms)7392026/08/27 09:40:51 goose: up to current file version: 27402026-08-27 09:40:51.112 UTC [990] ERROR: relation "goose_db_version" does not exist at character 367412026-08-27 09:40:51.112 UTC [990] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7422026/08/27 09:40:51 OK 2_object_stats_trigger.sql (6.79ms)7432026/08/27 09:40:51 goose: up to current file version: 27442026/08/27 09:40:51 OK 2_object_stats_trigger.sql (7.01ms)7452026/08/27 09:40:51 goose: up to current file version: 27462026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (15.64ms)7472026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200007482026/08/27 09:40:51 OK 20251218171726_add_pins.sql (12.46ms)7492026/08/27 09:40:51 OK 1_commit_pending_closure.sql (5.96ms)7502026/08/27 09:40:51 OK 2_object_stats_trigger.sql (3.82ms)7512026/08/27 09:40:51 goose: up to current file version: 27522026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (11.63ms)7532026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200007542026-08-27 09:40:51.129 UTC [991] ERROR: relation "goose_db_version" does not exist at character 367552026-08-27 09:40:51.129 UTC [991] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7562026/08/27 09:40:51 OK 1_commit_pending_closure.sql (5.01ms)7572026-08-27 09:40:51.134 UTC [992] ERROR: relation "goose_db_version" does not exist at character 367582026-08-27 09:40:51.134 UTC [992] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7592026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures7602026-08-27 09:40:51.140 UTC [993] ERROR: relation "goose_db_version" does not exist at character 367612026-08-27 09:40:51.140 UTC [993] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7622026/08/27 09:40:51 WARN mTLS auth: subject not in bound subjects subject="CN=reader"7632026-08-27 09:40:51.142 UTC [994] ERROR: relation "goose_db_version" does not exist at character 367642026-08-27 09:40:51.142 UTC [994] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7652026/08/27 09:40:51 WARN mTLS auth: subject not in bound subjects subject="CN=writer"766--- PASS: TestService_NativeMTLS (0.38s)767=== CONT TestReadProxyConditionalGet7682026-08-27 09:40:51.145 UTC [996] ERROR: relation "goose_db_version" does not exist at character 367692026-08-27 09:40:51.145 UTC [996] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7702026-08-27 09:40:51.146 UTC [995] ERROR: relation "goose_db_version" does not exist at character 367712026-08-27 09:40:51.146 UTC [995] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7722026/08/27 09:40:51 OK 2_object_stats_trigger.sql (15.58ms)7732026/08/27 09:40:51 goose: up to current file version: 27742026/08/27 09:40:51 OK 20241026095416_initial_model.sql (24.72ms)7752026/08/27 09:40:51 OK 20241026095416_initial_model.sql (34.27ms)7762026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (3.06ms)7772026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures7782026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (4.09ms)7792026/08/27 09:40:51 OK 20241026095416_initial_model.sql (10.36ms)7802026/08/27 09:40:51 OK 20251218171726_add_pins.sql (4.49ms)7812026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (1.68ms)7822026-08-27 09:40:51.163 UTC [1000] ERROR: relation "goose_db_version" does not exist at character 367832026-08-27 09:40:51.163 UTC [1000] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7842026-08-27 09:40:51.163 UTC [999] ERROR: relation "goose_db_version" does not exist at character 367852026-08-27 09:40:51.163 UTC [999] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7862026/08/27 09:40:51 OK 20251218171726_add_pins.sql (6.54ms)7872026-08-27 09:40:51.165 UTC [1001] ERROR: relation "goose_db_version" does not exist at character 367882026-08-27 09:40:51.165 UTC [1001] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7892026/08/27 09:40:51 OK 20251218171726_add_pins.sql (3.23ms)7902026/08/27 09:40:51 OK 20241026095416_initial_model.sql (11.07ms)7912026/08/27 09:40:51 OK 20241026095416_initial_model.sql (11.03ms)7922026/08/27 09:40:51 OK 20241026095416_initial_model.sql (11.73ms)7932026/08/27 09:40:51 OK 20241026095416_initial_model.sql (11.69ms)7942026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (5.43ms)7952026-08-27 09:40:51.166 UTC [1002] ERROR: relation "goose_db_version" does not exist at character 367962026-08-27 09:40:51.166 UTC [1002] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7972026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200007982026/08/27 09:40:51 OK 20241026095416_initial_model.sql (10.6ms)7992026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.2ms)8002026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.18ms)8012026-08-27 09:40:51.168 UTC [1003] ERROR: relation "goose_db_version" does not exist at character 368022026-08-27 09:40:51.168 UTC [1003] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC8032026-08-27 09:40:51.168 UTC [1004] ERROR: relation "goose_db_version" does not exist at character 368042026-08-27 09:40:51.168 UTC [1004] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC8052026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.69ms)8062026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.9ms)8072026-08-27 09:40:51.169 UTC [1005] ERROR: relation "goose_db_version" does not exist at character 368082026-08-27 09:40:51.169 UTC [1005] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC8092026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (4.1ms)8102026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200008112026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.97ms)8122026-08-27 09:40:51.170 UTC [1006] ERROR: relation "goose_db_version" does not exist at character 368132026-08-27 09:40:51.170 UTC [1006] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC8142026/08/27 09:40:51 OK 1_commit_pending_closure.sql (4.02ms)8152026/08/27 09:40:51 OK 20251218171726_add_pins.sql (3.87ms)8162026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (7.31ms)8172026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200008182026/08/27 09:40:51 OK 20251218171726_add_pins.sql (4.08ms)8192026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.09ms)8202026/08/27 09:40:51 goose: up to current file version: 28212026/08/27 09:40:51 OK 20251218171726_add_pins.sql (4.36ms)8222026/08/27 09:40:51 OK 20251218171726_add_pins.sql (4.17ms)8232026/08/27 09:40:51 OK 1_commit_pending_closure.sql (3.3ms)8242026/08/27 09:40:51 OK 20251218171726_add_pins.sql (4.05ms)825--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (0.42s)826=== CONT TestReadProxyNarinfo8272026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.63ms)8282026/08/27 09:40:51 goose: up to current file version: 28292026/08/27 09:40:51 OK 1_commit_pending_closure.sql (4.23ms)8302026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (5.45ms)8312026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200008322026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (5.7ms)8332026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200008342026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (5.52ms)8352026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200008362026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.01ms)8372026/08/27 09:40:51 goose: up to current file version: 28382026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (5.37ms)8392026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200008402026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (5.57ms)8412026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200008422026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.32ms)8432026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.46ms)8442026-08-27 09:40:51.181 UTC [1008] ERROR: relation "goose_db_version" does not exist at character 368452026-08-27 09:40:51.181 UTC [1008] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC8462026/08/27 09:40:51 OK 20241026095416_initial_model.sql (10.49ms)8472026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.28ms)8482026/08/27 09:40:51 goose: up to current file version: 28492026/08/27 09:40:51 OK 1_commit_pending_closure.sql (3.97ms)8502026/08/27 09:40:51 OK 1_commit_pending_closure.sql (3.08ms)8512026/08/27 09:40:51 OK 1_commit_pending_closure.sql (3.65ms)8522026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.39ms)8532026/08/27 09:40:51 goose: up to current file version: 28542026/08/27 09:40:51 OK 20241026095416_initial_model.sql (11.43ms)8552026/08/27 09:40:51 OK 20241026095416_initial_model.sql (11.58ms)8562026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (1.73ms)8572026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.56ms)8582026/08/27 09:40:51 goose: up to current file version: 28592026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.3ms)8602026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.73ms)8612026/08/27 09:40:51 goose: up to current file version: 28622026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.2ms)8632026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.65ms)8642026/08/27 09:40:51 goose: up to current file version: 28652026/08/27 09:40:51 OK 20241026095416_initial_model.sql (11.53ms)8662026/08/27 09:40:51 OK 20251218171726_add_pins.sql (4.48ms)867{"timestamp":"2026-08-27T09:40:51.188628126Z","level":"ERROR","duration":"1.2218ms","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(585)"}8682026/08/27 09:40:51 OK 20241026095416_initial_model.sql (10.61ms)869{"timestamp":"2026-08-27T09:40:51.188817096Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"9414ec8f-3caf-4001-b412-04aea82f53f5","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket14/","status_code":503,"duration_ms":1,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(585)"}8702026/08/27 09:40:51 OK 20241026095416_initial_model.sql (10.33ms)871{"timestamp":"2026-08-27T09:40:51.188858146Z","level":"ERROR","duration":"780.7µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(764)"}872{"timestamp":"2026-08-27T09:40:51.188948776Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"78a50a3d-6c1f-4bba-abc8-9eaf8a889e83","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket16/","status_code":503,"duration_ms":1,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(764)"}8732026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (3.01ms)8742026/08/27 09:40:51 OK 20251218171726_add_pins.sql (4.25ms)8752026/08/27 09:40:51 OK 20241026095416_initial_model.sql (11.75ms)8762026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.23ms)8772026/08/27 09:40:51 OK 20251218171726_add_pins.sql (4.82ms)8782026/08/27 09:40:51 OK 20241026095416_initial_model.sql (13.06ms)8792026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.07ms)8802026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (3.89ms)8812026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200008822026/08/27 09:40:51 OK 20251218171726_add_pins.sql (3.29ms)8832026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.49ms)8842026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.22ms)8852026/08/27 09:40:51 OK 20251218171726_add_pins.sql (3.34ms)886--- PASS: TestMetricsInventory (0.44s)887=== CONT TestReadProxyNarinfoAlreadyDecompressed8882026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (4.34ms)8892026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200008902026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (4.79ms)8912026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200008922026/08/27 09:40:51 OK 1_commit_pending_closure.sql (3.08ms)8932026/08/27 09:40:51 OK 20251218171726_add_pins.sql (4.41ms)8942026/08/27 09:40:51 OK 20251218171726_add_pins.sql (3.62ms)8952026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.6ms)8962026/08/27 09:40:51 goose: up to current file version: 28972026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.73ms)8982026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.82ms)8992026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (5.06ms)9002026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200009012026/08/27 09:40:51 OK 20251218171726_add_pins.sql (4.75ms)9022026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (5.14ms)9032026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200009042026/08/27 09:40:51 OK 20241026095416_initial_model.sql (10.64ms)9052026/08/27 09:40:51 OK 2_object_stats_trigger.sql (1.93ms)9062026/08/27 09:40:51 goose: up to current file version: 29072026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (4.88ms)9082026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200009092026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.76ms)9102026/08/27 09:40:51 goose: up to current file version: 29112026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (4.46ms)912--- PASS: TestReadProxyRangeRequest (0.36s)9132026/08/27 09:40:51 goose: successfully migrated database to version: 20260628120000914=== CONT TestGCTaskStore_Fail915--- PASS: TestGCTaskStore_Fail (0.00s)9162026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.91ms)917=== CONT TestClientMultipleUploads9182026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.23ms)9192026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (1.89ms)9202026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures921{"timestamp":"2026-08-27T09:40:51.203122424Z","level":"ERROR","duration":"370.75µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(766)"}922{"timestamp":"2026-08-27T09:40:51.203206684Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a31386bb-4f73-432f-a347-c91d39de297b","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket19/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(766)"}9232026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (3.88ms)9242026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200009252026/08/27 09:40:51 OK 2_object_stats_trigger.sql (1.98ms)9262026/08/27 09:40:51 goose: up to current file version: 29272026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.31ms)9282026/08/27 09:40:51 goose: up to current file version: 29292026/08/27 09:40:51 OK 1_commit_pending_closure.sql (3.47ms)9302026/08/27 09:40:51 OK 20251218171726_add_pins.sql (3.02ms)931{"timestamp":"2026-08-27T09:40:51.205977653Z","level":"ERROR","duration":"189.71µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(765)"}9322026/08/27 09:40:51 OK 1_commit_pending_closure.sql (3.36ms)933{"timestamp":"2026-08-27T09:40:51.205952123Z","level":"ERROR","duration":"389.3µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(585)"}934{"timestamp":"2026-08-27T09:40:51.206063733Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"741e9fe5-cb2e-4cc1-93cf-632df6e6eed9","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket21/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(765)"}9352026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.83ms)936{"timestamp":"2026-08-27T09:40:51.206084903Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"a3de8af6-90de-47ee-8cd0-88f536990bd2","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket20/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(585)"}9372026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.34ms)9382026/08/27 09:40:51 goose: up to current file version: 2939{"timestamp":"2026-08-27T09:40:51.208719673Z","level":"ERROR","duration":"177.99µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(765)"}9402026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.43ms)941{"timestamp":"2026-08-27T09:40:51.208744093Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e84cc9e9-8fb1-4f1b-93f4-f2b2d7e2fb3a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket22/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(765)"}9422026/08/27 09:40:51 goose: up to current file version: 29432026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.14ms)9442026/08/27 09:40:51 goose: up to current file version: 29452026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (3.87ms)946{"timestamp":"2026-08-27T09:40:51.209978933Z","level":"ERROR","duration":"216.18µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(771)"}9472026/08/27 09:40:51 goose: successfully migrated database to version: 20260628120000948{"timestamp":"2026-08-27T09:40:51.210020543Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"7bfc1980-c534-4508-9400-29c69a6df076","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket23/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(771)"}949{"timestamp":"2026-08-27T09:40:51.210652143Z","level":"ERROR","duration":"336.02µs","resp":"Response { status: 503, version: HTTP/1.1, headers: {\"content-type\": \"application/xml\"}, body: Body { once: b\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>SlowDown</Code><Message>bucket creation concurrency limit reached; retry later</Message></Error>\" } }","target":"s3s::service","filename":"/build/rustfs-1.0.0-beta.12-vendor/source-git-1/s3s-0.14.1/src/service.rs","line_number":640,"threadName":"rustfs-worker","threadId":"ThreadId(587)"}950{"timestamp":"2026-08-27T09:40:51.210697823Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"46a474f9-2ba9-43b2-91a1-5f311fe1f71c","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"PUT","uri":"/bucket24/","status_code":503,"duration_ms":0,"result":"server_error","target":"rustfs::server::layer","filename":"rustfs/src/server/layer.rs","line_number":392,"threadName":"rustfs-worker","threadId":"ThreadId(587)"}9512026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.54ms)9522026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures9532026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.04ms)9542026/08/27 09:40:51 goose: up to current file version: 29552026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures956--- PASS: TestResurrectedObjectNotDeleted (0.47s)957=== CONT TestService_healthCheckHandler958--- PASS: TestReadProxyRootRedirectsToIndexHTML (0.29s)959=== CONT TestReadProxyNarStreaming9602026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures9612026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures9622026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures9632026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures9642026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures9652026/08/27 09:40:51 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"966--- PASS: TestService_AuthMiddleware (0.51s)967=== CONT TestGCTaskStore_GetEmpty968--- PASS: TestGCTaskStore_GetEmpty (0.00s)969=== CONT TestGracefulShutdownDrainsInflight9702026/08/27 09:40:51 INFO Starting HTTP server address=127.0.0.1:468399712026/08/27 09:40:51 INFO Shutdown signal received, draining in-flight requests timeout=10s9722026-08-27 09:40:51.275 UTC [1023] ERROR: relation "goose_db_version" does not exist at character 369732026-08-27 09:40:51.275 UTC [1023] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC974=== NAME TestNARDeduplicationMetadataUploadBug975 metadata_upload_test.go:48: First store path: /build/TestNARDeduplicationMetadataUploadBug3309597896/001/store/4vvchnhf743xz2b1zsm7maaakg086655-file1.txt9762026/08/27 09:40:51 OK 20241026095416_initial_model.sql (10.29ms)9772026-08-27 09:40:51.295 UTC [1040] ERROR: relation "goose_db_version" does not exist at character 369782026-08-27 09:40:51.295 UTC [1040] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC9792026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.43ms)9802026/08/27 09:40:51 OK 20251218171726_add_pins.sql (3.6ms)981--- PASS: TestReadProxyHead (0.52s)982=== CONT TestGCTaskStore_ConflictDifferentParams983--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)984=== CONT TestReadProxyInvalidPath9852026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (3.9ms)9862026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200009872026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.98ms)988--- PASS: TestObjectStatsTrigger (0.55s)989=== CONT TestCreatePendingClosureRejectsOversizedNAR9902026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures991--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.00s)992=== CONT TestGCTaskStore_DeduplicateSameParams993--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)994=== CONT TestPinProtectsFromGC9952026-08-27 09:40:51.308 UTC [1042] ERROR: relation "goose_db_version" does not exist at character 369962026-08-27 09:40:51.308 UTC [1042] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC9972026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.14ms)9982026/08/27 09:40:51 goose: up to current file version: 29992026/08/27 09:40:51 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10002026/08/27 09:40:51 OK 20241026095416_initial_model.sql (9.34ms)10012026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (1.61ms)10022026/08/27 09:40:51 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst1003--- PASS: TestCompleteMultipartUnregistered (0.55s)1004=== CONT TestGCBugBareHashReferences10052026-08-27 09:40:51.314 UTC [1046] ERROR: relation "goose_db_version" does not exist at character 3610062026-08-27 09:40:51.314 UTC [1046] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC10072026/08/27 09:40:51 OK 20251218171726_add_pins.sql (2.71ms)10082026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (3.43ms)10092026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000010102026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.15ms)10112026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.06ms)10122026/08/27 09:40:51 goose: up to current file version: 210132026/08/27 09:40:51 OK 20241026095416_initial_model.sql (9.72ms)1014--- PASS: TestService_Rustfstest (0.56s)1015=== CONT TestClientWithDependencies10162026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.35ms)10172026/08/27 09:40:51 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10182026/08/27 09:40:51 INFO Received cleanup request method=DELETE path=/api/pending_closures10192026/08/27 09:40:51 INFO Aborted multipart uploads count=11020--- PASS: TestGracefulShutdownDrainsInflight (0.07s)1021=== CONT TestGCTaskStore_StartNew1022--- PASS: TestGCTaskStore_StartNew (0.00s)1023=== CONT TestReadProxy40410242026/08/27 09:40:51 OK 20251218171726_add_pins.sql (13.91ms)10252026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures1026--- PASS: TestReadProxyConditionalGet (0.20s)1027=== CONT TestGCTaskStore_PhaseUpdates1028--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)1029=== CONT TestGCMetrics10302026/08/27 09:40:51 OK 20241026095416_initial_model.sql (23.04ms)1031--- PASS: TestMultipartCleanup (0.58s)1032=== CONT TestIsValidCachePath1033=== RUN TestIsValidCachePath/narinfo1034=== PAUSE TestIsValidCachePath/narinfo1035=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars1036=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars10372026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (4.58ms)1038=== RUN TestIsValidCachePath/nar_zst10392026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200001040=== PAUSE TestIsValidCachePath/nar_zst1041=== RUN TestIsValidCachePath/nar_xz1042=== PAUSE TestIsValidCachePath/nar_xz1043=== RUN TestIsValidCachePath/nar_bz21044=== PAUSE TestIsValidCachePath/nar_bz21045=== RUN TestIsValidCachePath/nar_uncompressed1046=== PAUSE TestIsValidCachePath/nar_uncompressed1047=== RUN TestIsValidCachePath/ls1048=== PAUSE TestIsValidCachePath/ls1049=== RUN TestIsValidCachePath/log1050=== PAUSE TestIsValidCachePath/log1051=== RUN TestIsValidCachePath/realisation1052=== PAUSE TestIsValidCachePath/realisation1053=== RUN TestIsValidCachePath/nix-cache-info1054=== PAUSE TestIsValidCachePath/nix-cache-info1055=== RUN TestIsValidCachePath/index.html1056=== PAUSE TestIsValidCachePath/index.html1057=== RUN TestIsValidCachePath/traversal_parent1058=== PAUSE TestIsValidCachePath/traversal_parent1059=== RUN TestIsValidCachePath/traversal_in_middle1060=== PAUSE TestIsValidCachePath/traversal_in_middle1061=== RUN TestIsValidCachePath/invalid_char_e1062=== PAUSE TestIsValidCachePath/invalid_char_e1063=== RUN TestIsValidCachePath/invalid_char_u10642026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.75ms)1065=== PAUSE TestIsValidCachePath/invalid_char_u1066=== RUN TestIsValidCachePath/random_path1067=== PAUSE TestIsValidCachePath/random_path1068=== RUN TestIsValidCachePath/empty1069=== PAUSE TestIsValidCachePath/empty1070=== RUN TestIsValidCachePath/leading_slash1071=== PAUSE TestIsValidCachePath/leading_slash1072=== RUN TestIsValidCachePath/wrong_extension1073=== PAUSE TestIsValidCachePath/wrong_extension1074=== RUN TestIsValidCachePath/short_hash1075=== PAUSE TestIsValidCachePath/short_hash1076=== CONT TestCacheConfigHandlerMaxNarSize1077--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)1078=== CONT TestGCTaskStore_CompletedAllowsNewTask1079--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)10802026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.41ms)1081=== CONT TestCacheConfigHandler1082=== RUN TestCacheConfigHandler/full_config,_no_issuer1083=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1084=== RUN TestCacheConfigHandler/no_cache_url_configured1085=== PAUSE TestCacheConfigHandler/no_cache_url_configured1086=== RUN TestCacheConfigHandler/no_signing_keys1087=== PAUSE TestCacheConfigHandler/no_signing_keys1088=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1089=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1090=== CONT TestService_AuthMiddleware_MTLSBoundSubjects10912026/08/27 09:40:51 OK 2_object_stats_trigger.sql (1.8ms)10922026/08/27 09:40:51 goose: up to current file version: 210932026/08/27 09:40:51 OK 20251218171726_add_pins.sql (5.54ms)10942026-08-27 09:40:51.357 UTC [1073] ERROR: relation "goose_db_version" does not exist at character 3610952026-08-27 09:40:51.357 UTC [1073] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC10962026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (7.59ms)10972026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000010982026-08-27 09:40:51.360 UTC [1074] ERROR: relation "goose_db_version" does not exist at character 3610992026-08-27 09:40:51.360 UTC [1074] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC11002026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.61ms)11012026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.99ms)11022026/08/27 09:40:51 goose: up to current file version: 211032026/08/27 09:40:51 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"1104--- PASS: TestReadProxyNarinfo (0.20s)1105=== CONT TestService_ReadAuthMiddleware11062026/08/27 09:40:51 INFO Received complete multipart upload request method=POST path=/api/multipart/complete11072026/08/27 09:40:51 OK 20241026095416_initial_model.sql (11.94ms)11082026/08/27 09:40:51 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MWJmZmFmNWYtOWM1Ni00ZWRmLTk0MWEtNjNlNzdjNWE1ZWZjLmEzMTgyYTA2LTJjZmYtNGI1NC1hYjNiLTkzZDE2OGRlY2FiNHgxNzg3ODIzNjUxMzUyMjMzMjEx1109--- PASS: TestReadProxyNarinfoAlreadyDecompressed (0.18s)1110=== CONT TestService_AuthMiddleware_OIDC11112026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.99ms)11122026/08/27 09:40:51 INFO OIDC provider initialized name=test11132026/08/27 09:40:51 OK 20241026095416_initial_model.sql (15.27ms)11142026/08/27 09:40:51 OK 20251218171726_add_pins.sql (6.41ms)11152026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (3.01ms)11162026/08/27 09:40:51 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MWJmZmFmNWYtOWM1Ni00ZWRmLTk0MWEtNjNlNzdjNWE1ZWZjLmEzMTgyYTA2LTJjZmYtNGI1NC1hYjNiLTkzZDE2OGRlY2FiNHgxNzg3ODIzNjUxMzUyMjMzMjEx parts=11117--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (0.56s)1118=== CONT TestClientIntegration11192026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (13.9ms)1120--- PASS: TestReadProxyDisabled (0.56s)11212026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200001122=== CONT TestClientCADerivations11232026/08/27 09:40:51 OK 20251218171726_add_pins.sql (16.14ms)11242026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures11252026/08/27 09:40:51 OK 1_commit_pending_closure.sql (6.23ms)11262026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.85ms)11272026/08/27 09:40:51 goose: up to current file version: 211282026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (6.85ms)11292026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000011302026/08/27 09:40:51 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)11312026/08/27 09:40:51 INFO Uploading 4vvchnhf743xz2b1zsm7maaakg086655-file1.txt (160B)11322026/08/27 09:40:51 OK 1_commit_pending_closure.sql (4.07ms)11332026/08/27 09:40:51 INFO Received cleanup request method=DELETE path=/api/pending_closures11342026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.6ms)11352026/08/27 09:40:51 goose: up to current file version: 211362026/08/27 09:40:51 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"11372026/08/27 09:40:51 WARN Failed to register uploaded object key=4vvchnhf743xz2b1zsm7maaakg086655.ls error="server returned 404: 404 page not found\n"11382026/08/27 09:40:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign11392026/08/27 09:40:51 INFO Signed narinfos id=1 count=111402026/08/27 09:40:51 INFO Uploading 1 narinfos11412026/08/27 09:40:51 INFO Aborted multipart uploads count=011422026-08-27 09:40:51.426 UTC [1124] ERROR: relation "goose_db_version" does not exist at character 3611432026-08-27 09:40:51.426 UTC [1124] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC11442026/08/27 09:40:51 WARN Failed to register uploaded object key=4vvchnhf743xz2b1zsm7maaakg086655.narinfo error="server returned 404: 404 page not found\n"11452026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures11462026/08/27 09:40:51 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete1147--- PASS: TestService_healthCheckHandler (0.20s)1148=== CONT TestCacheStatsHandler1149=== NAME TestClientMultipleUploads1150 client_integration_test.go:338: Created store path 0: /build/TestClientMultipleUploads2052934578/001/store/4cf17lmf39kaxc7f4ph7zxarcn7y1f0m-test-file-0.txt11512026-08-27 09:40:51.442 UTC [1142] ERROR: relation "goose_db_version" does not exist at character 3611522026-08-27 09:40:51.442 UTC [1142] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC11532026-08-27 09:40:51.443 UTC [1145] ERROR: relation "goose_db_version" does not exist at character 3611542026-08-27 09:40:51.443 UTC [1145] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC11552026/08/27 09:40:51 INFO Completed upload id=111562026/08/27 09:40:51 INFO Upload complete. (119ms)1157--- PASS: TestReadProxyNarStreaming (0.22s)1158=== CONT TestService_AuthMiddleware_MTLSProxyHeader1159=== NAME TestNARDeduplicationMetadataUploadBug1160 metadata_upload_test.go:54: Retrieved narinfo from S3:1161 StorePath: /build/TestNARDeduplicationMetadataUploadBug3309597896/001/store/4vvchnhf743xz2b1zsm7maaakg086655-file1.txt1162 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1163 Compression: zstd1164 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1165 NarSize: 1601166 References: 1167 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf11682026/08/27 09:40:51 INFO Received cleanup request method=DELETE path=/api/pending_closures1169 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)1170 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):1171 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}11722026/08/27 09:40:51 INFO Aborted multipart uploads count=111732026/08/27 09:40:51 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete11742026-08-27 09:40:51.465 UTC [1005] ERROR: Closure does not exist: id=111752026-08-27 09:40:51.465 UTC [1005] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 16 at RAISE11762026-08-27 09:40:51.465 UTC [1005] STATEMENT: -- name: CommitPendingClosure :exec1177 SELECT commit_pending_closure($1::bigint)1178 11792026-08-27 09:40:51.465 UTC [1150] ERROR: relation "goose_db_version" does not exist at character 3611802026-08-27 09:40:51.465 UTC [1150] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC1181--- PASS: TestService_cleanupPendingClosuresHandler (0.70s)1182=== CONT TestClientErrorHandling1183=== RUN TestClientErrorHandling/InvalidStorePath1184=== PAUSE TestClientErrorHandling/InvalidStorePath1185=== RUN TestClientErrorHandling/InvalidAuthToken1186=== PAUSE TestClientErrorHandling/InvalidAuthToken1187=== RUN TestClientErrorHandling/ServerNotAvailable1188=== PAUSE TestClientErrorHandling/ServerNotAvailable1189=== CONT TestProxyWriteTimeout/narinfo1190=== CONT TestProxyWriteTimeout/unknown_size1191=== CONT TestProxyWriteTimeout/10_GiB_nar1192=== CONT TestProxyWriteTimeout/1_GiB_nar1193--- PASS: TestProxyWriteTimeout (0.00s)1194 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1195 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1196 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1197 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)1198=== CONT TestServerTLSConfig/no_client_CA1199=== CONT TestServerTLSConfig/not_a_PEM_file1200=== CONT TestServerTLSConfig/missing_CA_file1201--- PASS: TestServerTLSConfig (0.07s)1202 --- PASS: TestServerTLSConfig/no_client_CA (0.00s)1203 --- PASS: TestServerTLSConfig/not_a_PEM_file (0.00s)1204 --- PASS: TestServerTLSConfig/missing_CA_file (0.00s)1205=== CONT TestParseSingleRange/none1206=== CONT TestParseSingleRange/suffix1207=== CONT TestParseSingleRange/end_clamped_to_size1208=== CONT TestParseSingleRange/open-ended1209=== CONT TestParseSingleRange/closed1210=== CONT TestParseSingleRange/malformed_end_before_start1211=== CONT TestParseSingleRange/suffix_exceeds_size1212=== CONT TestParseSingleRange/malformed_both_empty1213=== CONT TestParseSingleRange/malformed_no_dash1214=== CONT TestParseSingleRange/multi-range_ignored1215=== CONT TestParseSingleRange/unknown_unit1216=== CONT TestParseSingleRange/start_far_past_EOF1217=== CONT TestParseSingleRange/start_past_EOF1218=== CONT TestParseSingleRange/single_byte1219--- PASS: TestParseSingleRange (0.07s)1220 --- PASS: TestParseSingleRange/none (0.00s)1221 --- PASS: TestParseSingleRange/suffix (0.00s)1222 --- PASS: TestParseSingleRange/end_clamped_to_size (0.00s)1223 --- PASS: TestParseSingleRange/open-ended (0.00s)1224 --- PASS: TestParseSingleRange/closed (0.00s)1225 --- PASS: TestParseSingleRange/malformed_end_before_start (0.00s)1226 --- PASS: TestParseSingleRange/suffix_exceeds_size (0.00s)1227 --- PASS: TestParseSingleRange/malformed_both_empty (0.00s)1228 --- PASS: TestParseSingleRange/malformed_no_dash (0.00s)1229 --- PASS: TestParseSingleRange/multi-range_ignored (0.00s)1230 --- PASS: TestParseSingleRange/unknown_unit (0.00s)1231 --- PASS: TestParseSingleRange/start_far_past_EOF (0.00s)1232 --- PASS: TestParseSingleRange/start_past_EOF (0.00s)1233 --- PASS: TestParseSingleRange/single_byte (0.00s)1234=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info12352026/08/27 09:40:51 INFO Received uploads request method=POST path=/1236=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key12372026/08/27 09:40:51 INFO Received request for more parts method=POST path=/1238=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal12392026/08/27 09:40:51 INFO Received uploads request method=POST path=/1240=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key12412026/08/27 09:40:51 INFO Received complete multipart upload request method=POST path=/1242--- PASS: TestUploadHandlersRejectInvalidKeys (0.00s)1243 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)1244 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1245 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s)1246 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1247=== CONT TestIsValidUploadKey/narinfo1248=== CONT TestIsValidUploadKey/realisation_plus_in_output1249=== CONT TestIsValidUploadKey/realisation1250=== CONT TestIsValidUploadKey/build_log_equals1251=== CONT TestIsValidUploadKey/build_log_question_mark1252=== CONT TestIsValidUploadKey/build_log_plus_in_name1253=== CONT TestIsValidUploadKey/build_log_home-manager_file1254=== CONT TestIsValidUploadKey/build_log1255=== CONT TestIsValidUploadKey/nix-cache-info1256=== CONT TestIsValidUploadKey/listing1257=== CONT TestIsValidUploadKey/nar_plain1258=== CONT TestIsValidUploadKey/nar_xz1259=== CONT TestIsValidUploadKey/nar_zst1260=== CONT TestIsValidUploadKey/traversal_nar1261=== CONT TestIsValidUploadKey/absolute1262=== CONT TestIsValidUploadKey/unknown_type1263=== CONT TestIsValidUploadKey/nar_key,_narinfo_type1264=== CONT TestIsValidUploadKey/traversal1265=== CONT TestIsValidUploadKey/listing_key,_narinfo_type1266=== CONT TestIsValidUploadKey/narinfo_key,_nar_type1267=== CONT TestIsValidUploadKey/index.html1268=== CONT TestIsValidUploadKey/empty_key1269--- PASS: TestIsValidUploadKey (0.07s)1270 --- PASS: TestIsValidUploadKey/narinfo (0.00s)1271 --- PASS: TestIsValidUploadKey/realisation_plus_in_output (0.00s)1272 --- PASS: TestIsValidUploadKey/realisation (0.00s)1273 --- PASS: TestIsValidUploadKey/build_log_equals (0.00s)1274 --- PASS: TestIsValidUploadKey/build_log_question_mark (0.00s)1275 --- PASS: TestIsValidUploadKey/build_log_plus_in_name (0.00s)1276 --- PASS: TestIsValidUploadKey/build_log_home-manager_file (0.00s)1277 --- PASS: TestIsValidUploadKey/build_log (0.00s)1278 --- PASS: TestIsValidUploadKey/nix-cache-info (0.00s)1279 --- PASS: TestIsValidUploadKey/listing (0.00s)1280 --- PASS: TestIsValidUploadKey/nar_plain (0.00s)1281 --- PASS: TestIsValidUploadKey/nar_xz (0.00s)1282 --- PASS: TestIsValidUploadKey/nar_zst (0.00s)1283 --- PASS: TestIsValidUploadKey/traversal_nar (0.00s)1284 --- PASS: TestIsValidUploadKey/absolute (0.00s)1285 --- PASS: TestIsValidUploadKey/unknown_type (0.00s)1286 --- PASS: TestIsValidUploadKey/nar_key,_narinfo_type (0.00s)1287 --- PASS: TestIsValidUploadKey/traversal (0.00s)1288 --- PASS: TestIsValidUploadKey/listing_key,_narinfo_type (0.00s)1289 --- PASS: TestIsValidUploadKey/narinfo_key,_nar_type (0.00s)1290 --- PASS: TestIsValidUploadKey/index.html (0.00s)1291 --- PASS: TestIsValidUploadKey/empty_key (0.00s)1292=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure12932026/08/27 09:40:51 INFO Received uploads request method=POST path=/1294=== NAME TestClientMultipleUploads1295 client_integration_test.go:338: Created store path 1: /build/TestClientMultipleUploads2052934578/001/store/m68insw5l4nwlf93b9114hv4sh2w7a89-test-file-1.txt12962026/08/27 09:40:51 OK 20241026095416_initial_model.sql (12.82ms)12972026/08/27 09:40:51 OK 20241026095416_initial_model.sql (15.24ms)12982026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.93ms)12992026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.33ms)13002026-08-27 09:40:51.479 UTC [1167] ERROR: relation "goose_db_version" does not exist at character 3613012026-08-27 09:40:51.479 UTC [1167] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13022026/08/27 09:40:51 OK 20251218171726_add_pins.sql (4.11ms)13032026/08/27 09:40:51 OK 20251218171726_add_pins.sql (4.85ms)13042026/08/27 09:40:51 OK 20241026095416_initial_model.sql (22.38ms)13052026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (3.39ms)13062026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (4.57ms)13072026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000013082026-08-27 09:40:51.485 UTC [1168] ERROR: relation "goose_db_version" does not exist at character 3613092026-08-27 09:40:51.485 UTC [1168] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13102026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (6.38ms)13112026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000013122026/08/27 09:40:51 OK 20241026095416_initial_model.sql (13.11ms)13132026/08/27 09:40:51 OK 1_commit_pending_closure.sql (3.08ms)13142026/08/27 09:40:51 OK 20251218171726_add_pins.sql (5.53ms)13152026-08-27 09:40:51.489 UTC [1169] ERROR: relation "goose_db_version" does not exist at character 3613162026-08-27 09:40:51.489 UTC [1169] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13172026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.77ms)13182026/08/27 09:40:51 OK 1_commit_pending_closure.sql (3.71ms)13192026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.28ms)13202026/08/27 09:40:51 goose: up to current file version: 213212026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures13222026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.49ms)13232026/08/27 09:40:51 goose: up to current file version: 213242026/08/27 09:40:51 OK 20251218171726_add_pins.sql (4.52ms)1325=== NAME TestNARDeduplicationMetadataUploadBug1326 metadata_upload_test.go:64: Second store path (same content): /build/TestNARDeduplicationMetadataUploadBug3309597896/001/store/2pb1rrn6iaymvhlxbphi19a24cwx70fz-file2.txt1327=== NAME TestClientMultipleUploads1328 client_integration_test.go:338: Created store path 2: /build/TestClientMultipleUploads2052934578/001/store/fsnj1cqc000hcjrfapv8879k38gg14hm-test-file-2.txt13292026-08-27 09:40:51.506 UTC [1203] ERROR: relation "goose_db_version" does not exist at character 3613302026-08-27 09:40:51.506 UTC [1203] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13312026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (23.6ms)13322026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000013332026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (18.8ms)13342026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000013352026/08/27 09:40:51 OK 20241026095416_initial_model.sql (28.46ms)13362026/08/27 09:40:51 OK 20241026095416_initial_model.sql (18.55ms)13372026/08/27 09:40:51 OK 20241026095416_initial_model.sql (25.06ms)13382026/08/27 09:40:51 OK 1_commit_pending_closure.sql (4.42ms)13392026/08/27 09:40:51 OK 1_commit_pending_closure.sql (4.65ms)13402026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.09ms)13412026/08/27 09:40:51 goose: up to current file version: 213422026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (3.35ms)13432026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (3.44ms)13442026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (3.23ms)13452026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.9ms)13462026/08/27 09:40:51 goose: up to current file version: 213472026/08/27 09:40:51 OK 20251218171726_add_pins.sql (4.43ms)13482026/08/27 09:40:51 OK 20251218171726_add_pins.sql (5.6ms)13492026/08/27 09:40:51 OK 20251218171726_add_pins.sql (5.51ms)13502026/08/27 09:40:51 OK 20241026095416_initial_model.sql (10.19ms)13512026-08-27 09:40:51.529 UTC [1205] ERROR: relation "goose_db_version" does not exist at character 3613522026-08-27 09:40:51.529 UTC [1205] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13532026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (5.25ms)13542026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000013552026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (2.5ms)13562026/08/27 09:40:51 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst13572026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures13582026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (5ms)13592026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000013602026-08-27 09:40:51.532 UTC [1226] ERROR: relation "goose_db_version" does not exist at character 3613612026-08-27 09:40:51.532 UTC [1226] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13622026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (5.85ms)13632026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000013642026-08-27 09:40:51.533 UTC [1227] ERROR: relation "goose_db_version" does not exist at character 3613652026-08-27 09:40:51.533 UTC [1227] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC1366--- PASS: TestPresignedUploadRegisteredBeforeCommit (0.77s)1367=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts13682026/08/27 09:40:51 INFO Received request for more parts method=POST path=/13692026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.47ms)13702026/08/27 09:40:51 OK 20251218171726_add_pins.sql (3.4ms)13712026/08/27 09:40:51 OK 1_commit_pending_closure.sql (3.48ms)13722026/08/27 09:40:51 OK 2_object_stats_trigger.sql (3.15ms)13732026/08/27 09:40:51 goose: up to current file version: 213742026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.99ms)13752026/08/27 09:40:51 goose: up to current file version: 213762026/08/27 09:40:51 OK 1_commit_pending_closure.sql (5.01ms)13772026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (4.21ms)13782026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000013792026/08/27 09:40:51 OK 2_object_stats_trigger.sql (1.8ms)13802026/08/27 09:40:51 goose: up to current file version: 213812026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.56ms)13822026/08/27 09:40:51 OK 2_object_stats_trigger.sql (1.85ms)13832026/08/27 09:40:51 goose: up to current file version: 21384--- PASS: TestReadProxyInvalidPath (0.25s)1385=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart13862026/08/27 09:40:51 INFO Received complete multipart upload request method=POST path=/13872026/08/27 09:40:51 OK 20241026095416_initial_model.sql (10.74ms)13882026/08/27 09:40:51 OK 20241026095416_initial_model.sql (9.82ms)13892026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (1.97ms)13902026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (1.65ms)13912026/08/27 09:40:51 OK 20251218171726_add_pins.sql (2.75ms)13922026/08/27 09:40:51 OK 20251218171726_add_pins.sql (3.95ms)13932026/08/27 09:40:51 OK 20241026095416_initial_model.sql (14.3ms)13942026/08/27 09:40:51 WARN mTLS auth: subject not in bound subjects subject="CN=writer"1395--- PASS: TestService_ReadAuthMiddleware (0.19s)13962026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (2.66ms)1397=== CONT TestIsValidCachePath/narinfo13982026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200001399=== CONT TestIsValidCachePath/invalid_char_u1400=== CONT TestIsValidCachePath/invalid_char_e1401=== CONT TestIsValidCachePath/traversal_in_middle1402=== CONT TestIsValidCachePath/traversal_parent1403=== CONT TestIsValidCachePath/index.html1404=== CONT TestIsValidCachePath/nix-cache-info1405=== CONT TestIsValidCachePath/realisation1406=== CONT TestIsValidCachePath/log1407=== CONT TestIsValidCachePath/ls14082026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (1.75ms)1409=== CONT TestIsValidCachePath/nar_uncompressed1410=== CONT TestIsValidCachePath/nar_bz214112026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (2.76ms)1412=== CONT TestIsValidCachePath/nar_xz14132026/08/27 09:40:51 goose: successfully migrated database to version: 202606281200001414=== CONT TestIsValidCachePath/narinfo_all_nix_base32_chars1415=== CONT TestIsValidCachePath/nar_zst1416=== CONT TestIsValidCachePath/leading_slash1417=== CONT TestIsValidCachePath/short_hash1418=== CONT TestIsValidCachePath/empty1419=== CONT TestIsValidCachePath/wrong_extension1420=== CONT TestIsValidCachePath/random_path1421=== CONT TestCacheConfigHandler/full_config,_no_issuer1422--- PASS: TestIsValidCachePath (0.00s)1423 --- PASS: TestIsValidCachePath/narinfo (0.00s)1424 --- PASS: TestIsValidCachePath/invalid_char_u (0.00s)1425 --- PASS: TestIsValidCachePath/invalid_char_e (0.00s)1426 --- PASS: TestIsValidCachePath/traversal_in_middle (0.00s)1427 --- PASS: TestIsValidCachePath/traversal_parent (0.00s)1428 --- PASS: TestIsValidCachePath/index.html (0.00s)1429 --- PASS: TestIsValidCachePath/nix-cache-info (0.00s)1430 --- PASS: TestIsValidCachePath/realisation (0.00s)1431 --- PASS: TestIsValidCachePath/log (0.00s)1432 --- PASS: TestIsValidCachePath/ls (0.00s)1433 --- PASS: TestIsValidCachePath/nar_uncompressed (0.00s)1434 --- PASS: TestIsValidCachePath/nar_bz2 (0.00s)1435 --- PASS: TestIsValidCachePath/nar_xz (0.00s)1436 --- PASS: TestIsValidCachePath/narinfo_all_nix_base32_chars (0.00s)1437 --- PASS: TestIsValidCachePath/nar_zst (0.00s)1438 --- PASS: TestIsValidCachePath/leading_slash (0.00s)1439 --- PASS: TestIsValidCachePath/short_hash (0.00s)1440 --- PASS: TestIsValidCachePath/empty (0.00s)1441 --- PASS: TestIsValidCachePath/wrong_extension (0.00s)1442 --- PASS: TestIsValidCachePath/random_path (0.00s)14432026/08/27 09:40:51 OK 1_commit_pending_closure.sql (1.47ms)1444=== CONT TestCacheConfigHandler/no_signing_keys1445=== CONT TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1446=== CONT TestCacheConfigHandler/no_cache_url_configured1447--- PASS: TestCacheConfigHandler (0.00s)1448 --- PASS: TestCacheConfigHandler/full_config,_no_issuer (0.00s)1449 --- PASS: TestCacheConfigHandler/no_signing_keys (0.00s)1450 --- PASS: TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator (0.00s)1451 --- PASS: TestCacheConfigHandler/no_cache_url_configured (0.00s)1452=== CONT TestClientErrorHandling/InvalidStorePath14532026/08/27 09:40:51 OK 1_commit_pending_closure.sql (1.21ms)14542026/08/27 09:40:51 OK 2_object_stats_trigger.sql (734.8µs)14552026/08/27 09:40:51 goose: up to current file version: 214562026/08/27 09:40:51 OK 2_object_stats_trigger.sql (699.58µs)14572026/08/27 09:40:51 goose: up to current file version: 214582026/08/27 09:40:51 OK 20251218171726_add_pins.sql (2.88ms)14592026/08/27 09:40:51 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"14602026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (4.05ms)14612026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000014622026-08-27 09:40:51.567 UTC [1282] ERROR: relation "goose_db_version" does not exist at character 3614632026-08-27 09:40:51.567 UTC [1282] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC14642026-08-27 09:40:51.567 UTC [1288] ERROR: relation "goose_db_version" does not exist at character 3614652026-08-27 09:40:51.567 UTC [1288] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC14662026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.1ms)14672026/08/27 09:40:51 OK 2_object_stats_trigger.sql (1.13ms)14682026/08/27 09:40:51 goose: up to current file version: 214692026/08/27 09:40:51 INFO Aborted multipart uploads count=01470--- PASS: TestReadProxy404 (0.23s)1471=== CONT TestClientErrorHandling/ServerNotAvailable14722026/08/27 09:40:51 WARN Force mode enabled - objects will be deleted immediately without grace period1473=== CONT TestClientErrorHandling/InvalidAuthToken14742026/08/27 09:40:51 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"14752026/08/27 09:40:51 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=014762026/08/27 09:40:51 INFO Vacuumed table table=pending_closures14772026/08/27 09:40:51 INFO Vacuumed table table=pending_objects14782026/08/27 09:40:51 INFO Vacuumed table table=multipart_uploads14792026/08/27 09:40:51 INFO Vacuumed table table=closures14802026/08/27 09:40:51 INFO Vacuumed table table=objects1481--- PASS: TestGCMetrics (0.24s)14822026/08/27 09:40:51 OK 20241026095416_initial_model.sql (9.76ms)14832026/08/27 09:40:51 OK 20241026095416_initial_model.sql (8.95ms)14842026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (1.06ms)14852026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (1.18ms)1486=== NAME TestOrphanedObjectsGC1487 orphaned_objects_gc_test.go:290: GC Test Summary:1488 orphaned_objects_gc_test.go:291: - Kept: 2 objects from closure A1489 orphaned_objects_gc_test.go:292: - Deleted: 2 objects from closure B1490 orphaned_objects_gc_test.go:293: - Deleted: 6 orphaned chain objects (X1->X2->X3)1491 orphaned_objects_gc_test.go:294: - Deleted: 2 orphaned single objects (Y)1492 orphaned_objects_gc_test.go:295: - Total deleted: 10 objects1493--- PASS: TestOrphanedObjectsGC (0.83s)14942026/08/27 09:40:51 OK 20251218171726_add_pins.sql (3.27ms)14952026/08/27 09:40:51 OK 20251218171726_add_pins.sql (4.05ms)14962026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (3.57ms)14972026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000014982026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (3.6ms)14992026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000015002026/08/27 09:40:51 OK 1_commit_pending_closure.sql (3.14ms)15012026/08/27 09:40:51 OK 1_commit_pending_closure.sql (2.37ms)15022026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures15032026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.15ms)15042026/08/27 09:40:51 goose: up to current file version: 21505=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token1506=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token15072026/08/27 09:40:51 OK 2_object_stats_trigger.sql (2.13ms)1508=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected15092026/08/27 09:40:51 goose: up to current file version: 21510=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected15112026/08/27 09:40:51 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)1512=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1513=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1514=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1515=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1516=== CONT TestService_AuthMiddleware_OIDC/valid_OIDC_token1517=== CONT TestService_AuthMiddleware_OIDC/malformed_token_rejected1518=== CONT TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected15192026/08/27 09:40:51 WARN Authentication failed token_preview=not-a-valid-jwt token_length=15 oidc_error="no provider could verify the token (signature or issuer mismatch)" oidc_provider="" tried_providers=[test]1520=== CONT TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1521=== NAME TestPinProtectsFromGC1522 client_integration_test.go:646: Pinned store path: /build/TestPinProtectsFromGC1704933809/001/store/npg0b9rh5ina6gls3sx4yvfrsx0l0vln-pinned-file.txt1523 client_integration_test.go:647: Unpinned store path: /build/TestPinProtectsFromGC1704933809/001/store/p2r5waglmaz987dl98iaz43cl3an8z9r-unpinned-file.txt15242026/08/27 09:40:51 INFO OIDC auth successful provider=test15252026/08/27 09:40:51 WARN Authentication failed token_preview=eyJhbGciOi...WA2rPlISgA token_length=702 oidc_error="bound claims validation failed: claim \"repository_owner\" value [otherorg] not in allowed patterns [myorg]" oidc_provider=test tried_providers=[test]15262026/08/27 09:40:51 WARN Failed to register uploaded object key=2pb1rrn6iaymvhlxbphi19a24cwx70fz.ls error="server returned 404: 404 page not found\n"15272026/08/27 09:40:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign1528--- PASS: TestService_AuthMiddleware_OIDC (0.22s)1529 --- PASS: TestService_AuthMiddleware_OIDC/malformed_token_rejected (0.00s)1530 --- PASS: TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured (0.00s)1531 --- PASS: TestService_AuthMiddleware_OIDC/valid_OIDC_token (0.00s)1532 --- PASS: TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected (0.00s)15332026/08/27 09:40:51 INFO Signed narinfos id=2 count=115342026/08/27 09:40:51 INFO Uploading 1 narinfos15352026/08/27 09:40:51 WARN Failed to register uploaded object key=2pb1rrn6iaymvhlxbphi19a24cwx70fz.narinfo error="server returned 404: 404 page not found\n"15362026/08/27 09:40:51 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete15372026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures15382026/08/27 09:40:51 INFO Completed upload id=215392026/08/27 09:40:51 INFO Upload complete. (76ms)1540--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (0.15s)1541=== NAME TestNARDeduplicationMetadataUploadBug1542 metadata_upload_test.go:76: Retrieved narinfo from S3:1543 StorePath: /build/TestNARDeduplicationMetadataUploadBug3309597896/001/store/2pb1rrn6iaymvhlxbphi19a24cwx70fz-file2.txt1544 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1545 Compression: zstd1546 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1547 NarSize: 1601548 References: 1549 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1550 metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes)1551 metadata_upload_test.go:77: Decompressed .ls content (49 bytes):1552 {"version":1,"root":{"type":"regular","size":44}}15532026/08/27 09:40:51 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"15542026/08/27 09:40:51 WARN mTLS auth: bound subjects configured but subject DN unavailable15552026/08/27 09:40:51 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1556--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (0.27s)1557--- PASS: TestNARDeduplicationMetadataUploadBug (0.85s)15582026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures1559=== NAME TestClientWithDependencies1560 client_integration_test.go:593: Built derivation: /build/TestClientWithDependencies1609076985/001/store/hpwnvclb7f0r6402v45njkabrkfl4wsg-test-script15612026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures15622026/08/27 09:40:51 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)15632026/08/27 09:40:51 INFO Uploading m68insw5l4nwlf93b9114hv4sh2w7a89-test-file-1.txt (160B)15642026/08/27 09:40:51 INFO Uploading fsnj1cqc000hcjrfapv8879k38gg14hm-test-file-2.txt (160B)15652026/08/27 09:40:51 INFO Uploading 4cf17lmf39kaxc7f4ph7zxarcn7y1f0m-test-file-0.txt (160B)1566=== NAME TestClientIntegration1567 client_integration_test.go:276: Created store path: /build/TestClientIntegration3975591257/002/store/ag6608cwpim3bl9bcf226bpshf3sz3p8-test-file.txt15682026/08/27 09:40:51 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"15692026/08/27 09:40:51 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"15702026/08/27 09:40:51 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"15712026/08/27 09:40:51 WARN Failed to register uploaded object key=fsnj1cqc000hcjrfapv8879k38gg14hm.ls error="server returned 404: 404 page not found\n"15722026/08/27 09:40:51 WARN Failed to register uploaded object key=m68insw5l4nwlf93b9114hv4sh2w7a89.ls error="server returned 404: 404 page not found\n"15732026/08/27 09:40:51 WARN Failed to register uploaded object key=4cf17lmf39kaxc7f4ph7zxarcn7y1f0m.ls error="server returned 404: 404 page not found\n"15742026/08/27 09:40:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign15752026/08/27 09:40:51 INFO Signed narinfos id=2 count=115762026/08/27 09:40:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign15772026/08/27 09:40:51 INFO Signed narinfos id=3 count=115782026/08/27 09:40:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign15792026/08/27 09:40:51 INFO Signed narinfos id=1 count=115802026/08/27 09:40:51 INFO Uploading 3 narinfos15812026/08/27 09:40:51 WARN Failed to register uploaded object key=4cf17lmf39kaxc7f4ph7zxarcn7y1f0m.narinfo error="server returned 404: 404 page not found\n"15822026/08/27 09:40:51 WARN Failed to register uploaded object key=fsnj1cqc000hcjrfapv8879k38gg14hm.narinfo error="server returned 404: 404 page not found\n"15832026/08/27 09:40:51 WARN Failed to register uploaded object key=m68insw5l4nwlf93b9114hv4sh2w7a89.narinfo error="server returned 404: 404 page not found\n"15842026/08/27 09:40:51 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete15852026/08/27 09:40:51 INFO Completed upload id=11586=== NAME TestClientWithDependencies1587 client_integration_test.go:595: Found 1 dependencies (including self)15882026/08/27 09:40:51 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1589--- PASS: TestCacheStatsHandler (0.22s)15902026/08/27 09:40:51 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete15912026/08/27 09:40:51 INFO Completed upload id=215922026/08/27 09:40:51 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete15932026-08-27 09:40:51.659 UTC [1528] ERROR: relation "goose_db_version" does not exist at character 3615942026-08-27 09:40:51.659 UTC [1528] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC15952026/08/27 09:40:51 INFO Completed upload id=315962026/08/27 09:40:51 INFO Upload complete. (123ms)1597=== NAME TestClientMultipleUploads1598 client_integration_test.go:349: Uploaded 3 paths in 155.750207ms1599--- PASS: TestClientMultipleUploads (0.47s)16002026/08/27 09:40:51 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"16012026/08/27 09:40:51 OK 20241026095416_initial_model.sql (7.65ms)16022026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (974.81µs)16032026-08-27 09:40:51.674 UTC [1574] ERROR: relation "goose_db_version" does not exist at character 3616042026-08-27 09:40:51.674 UTC [1574] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC16052026/08/27 09:40:51 OK 20251218171726_add_pins.sql (1.85ms)16062026/08/27 09:40:51 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=MWJmZmFmNWYtOWM1Ni00ZWRmLTk0MWEtNjNlNzdjNWE1ZWZjLmJhZTE3MWM2LTEyMzUtNGE1Ni1hNzcwLWNjOGIzM2E1OTkyNHgxNzg3ODIzNjUxMjA4NjQ3MTUz parts=1016072026/08/27 09:40:51 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete16082026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (3.86ms)16092026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000016102026/08/27 09:40:51 INFO Completed upload id=11611=== NAME TestClientCADerivations1612 client_ca_test.go:136: Built CA derivation: /build/TestClientCADerivations3215544231/001/store/dmcsvwipqg4jrjrmlj1a9vwyh47m7hc5-ca-test16132026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures16142026/08/27 09:40:51 OK 1_commit_pending_closure.sql (1.52ms)16152026/08/27 09:40:51 OK 2_object_stats_trigger.sql (623.38µs)16162026/08/27 09:40:51 goose: up to current file version: 216172026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures16182026/08/27 09:40:51 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo16192026/08/27 09:40:51 WARN Found objects in DB but missing from S3, will re-upload count=11620--- PASS: TestService_verifyS3Integrity (0.92s)16212026/08/27 09:40:51 OK 20241026095416_initial_model.sql (6.92ms)16222026/08/27 09:40:51 INFO Received complete multipart upload request method=POST path=/api/multipart/complete16232026/08/27 09:40:51 OK 20251210153512_drop_unused_gin_index.sql (925.76µs)16242026/08/27 09:40:51 OK 20251218171726_add_pins.sql (2.31ms)16252026/08/27 09:40:51 INFO Received complete multipart upload request method=POST path=/api/multipart/complete16262026/08/27 09:40:51 OK 20260628120000_add_object_size_and_stats.sql (3.03ms)16272026/08/27 09:40:51 goose: successfully migrated database to version: 2026062812000016282026/08/27 09:40:51 WARN Request failed, retrying attempt=1 max_attempts=6 backoff=100ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config16292026/08/27 09:40:51 OK 1_commit_pending_closure.sql (1.57ms)16302026/08/27 09:40:51 OK 2_object_stats_trigger.sql (961.71µs)16312026/08/27 09:40:51 goose: up to current file version: 216322026/08/27 09:40:51 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"16332026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures16342026/08/27 09:40:51 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=MWJmZmFmNWYtOWM1Ni00ZWRmLTk0MWEtNjNlNzdjNWE1ZWZjLmRjOGVkYmYzLTVmMzEtNGY0Yy04ZjA4LWY3NGNjMGQ4ZWU0OXgxNzg3ODIzNjUxMjc0NjM1MTkz parts=1016352026/08/27 09:40:51 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete16362026/08/27 09:40:51 INFO Completed upload id=116372026/08/27 09:40:51 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001638=== NAME TestClientCADerivations1639 client_ca_test.go:139: Found 1 dependencies (including self)16402026/08/27 09:40:51 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=MWJmZmFmNWYtOWM1Ni00ZWRmLTk0MWEtNjNlNzdjNWE1ZWZjLjA3MjNkYTlhLTFlY2YtNDc0Ny1hNWZhLWRkZjRjZWUxOTFhNHgxNzg3ODIzNjUxMTU0Mzg3MTUx parts=1216412026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures16422026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures16432026/08/27 09:40:51 INFO Starting cleanup of old closures method=DELETE path=/api/closures16442026/08/27 09:40:51 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)16452026/08/27 09:40:51 INFO Uploading npg0b9rh5ina6gls3sx4yvfrsx0l0vln-pinned-file.txt (128B)1646--- PASS: TestCompletedNarNotReofferedAcrossClosures (0.96s)16472026/08/27 09:40:51 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"16482026/08/27 09:40:51 WARN Failed to register uploaded object key=npg0b9rh5ina6gls3sx4yvfrsx0l0vln.ls error="server returned 404: 404 page not found\n"16492026/08/27 09:40:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign16502026/08/27 09:40:51 INFO Signed narinfos id=1 count=116512026/08/27 09:40:51 INFO Uploading 1 narinfos16522026/08/27 09:40:51 WARN Failed to register uploaded object key=npg0b9rh5ina6gls3sx4yvfrsx0l0vln.narinfo error="server returned 404: 404 page not found\n"16532026/08/27 09:40:51 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete16542026/08/27 09:40:51 INFO Aborted multipart uploads count=016552026/08/27 09:40:51 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"16562026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures16572026/08/27 09:40:51 INFO Completed upload id=116582026/08/27 09:40:51 INFO Upload complete. (99ms)16592026/08/27 09:40:51 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=0 objects-failed-to-delete=016602026/08/27 09:40:51 INFO Vacuumed table table=pending_closures16612026/08/27 09:40:51 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)16622026/08/27 09:40:51 INFO Uploading hpwnvclb7f0r6402v45njkabrkfl4wsg-test-script (136B)16632026/08/27 09:40:51 INFO Vacuumed table table=pending_objects16642026/08/27 09:40:51 INFO Vacuumed table table=multipart_uploads16652026/08/27 09:40:51 WARN Failed to register uploaded object key=log/5n5qm0whqpnv6qrn3q72gliylv4zislg-test-script.drv error="server returned 404: 404 page not found\n"16662026/08/27 09:40:51 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"16672026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures16682026/08/27 09:40:51 INFO Vacuumed table table=closures16692026/08/27 09:40:51 WARN Failed to register uploaded object key=hpwnvclb7f0r6402v45njkabrkfl4wsg.ls error="server returned 404: 404 page not found\n"16702026/08/27 09:40:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign16712026/08/27 09:40:51 INFO Signed narinfos id=1 count=116722026/08/27 09:40:51 INFO Uploading 1 narinfos16732026/08/27 09:40:51 INFO Vacuumed table table=objects16742026/08/27 09:40:51 WARN Failed to register uploaded object key=hpwnvclb7f0r6402v45njkabrkfl4wsg.narinfo error="server returned 404: 404 page not found\n"16752026/08/27 09:40:51 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete16762026/08/27 09:40:51 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)16772026/08/27 09:40:51 INFO Uploading ag6608cwpim3bl9bcf226bpshf3sz3p8-test-file.txt (152B)16782026/08/27 09:40:51 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"16792026/08/27 09:40:51 INFO Completed upload id=116802026/08/27 09:40:51 INFO Upload complete. (64ms)16812026/08/27 09:40:51 WARN Failed to register uploaded object key=ag6608cwpim3bl9bcf226bpshf3sz3p8.ls error="server returned 404: 404 page not found\n"16822026/08/27 09:40:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign1683--- PASS: TestGCBugBareHashReferences (0.45s)16842026/08/27 09:40:51 INFO Signed narinfos id=1 count=116852026/08/27 09:40:51 INFO Uploading 1 narinfos1686=== NAME TestClientWithDependencies1687 client_integration_test.go:597: Skipping nix copy test - isolated store (/build/TestClientWithDependencies1609076985/001/store) requires matching store prefix16882026/08/27 09:40:51 WARN Failed to register uploaded object key=ag6608cwpim3bl9bcf226bpshf3sz3p8.narinfo error="server returned 404: 404 page not found\n"16892026/08/27 09:40:51 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete1690--- PASS: TestClientWithDependencies (0.44s)16912026/08/27 09:40:51 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000016922026/08/27 09:40:51 INFO Completed upload id=116932026/08/27 09:40:51 INFO Upload complete. (104ms)1694--- PASS: TestService_createPendingClosureHandler (1.00s)1695=== NAME TestClientIntegration1696 client_integration_test.go:292: Retrieved narinfo from S3:1697 StorePath: /build/TestClientIntegration3975591257/002/store/ag6608cwpim3bl9bcf226bpshf3sz3p8-test-file.txt1698 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1699 Compression: zstd1700 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11701 NarSize: 1521702 References: 1703 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11704 client_integration_test.go:293: Retrieved .ls file from S3 (compressed size: 77 bytes)1705 client_integration_test.go:293: Decompressed .ls content (64 bytes):1706 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}1707 client_integration_test.go:296: Testing garbage collection...17082026/08/27 09:40:51 INFO Received complete multipart upload request method=POST path=/api/multipart/complete17092026/08/27 09:40:51 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"17102026/08/27 09:40:51 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=MWJmZmFmNWYtOWM1Ni00ZWRmLTk0MWEtNjNlNzdjNWE1ZWZjLjIwNjJiNjk2LTZlYTItNDk1OS04MDdjLTgxNWFiZDgwYjA3MXgxNzg3ODIzNjUxMjY2MTI1Mjg0 parts=121711--- PASS: TestRedundantMultipartUpload (0.95s)17122026/08/27 09:40:51 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"17132026/08/27 09:40:51 WARN Request failed, retrying attempt=2 max_attempts=6 backoff=202.364095ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config17142026/08/27 09:40:51 INFO Starting cleanup of old closures method=DELETE path=/api/closures17152026/08/27 09:40:51 INFO Garbage collection started17162026/08/27 09:40:51 INFO Aborted multipart uploads count=017172026/08/27 09:40:51 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: server returned 404: 404 page not found\n"17182026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures17192026/08/27 09:40:51 WARN Force mode enabled - objects will be deleted immediately without grace period17202026/08/27 09:40:51 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)17212026/08/27 09:40:51 INFO Uploading dmcsvwipqg4jrjrmlj1a9vwyh47m7hc5-ca-test (144B)17222026/08/27 09:40:51 WARN Failed to register uploaded object key=log/c5wsbx6cprqw8sc1s5436qhj90arw3lg-ca-test.drv error="server returned 404: 404 page not found\n"17232026/08/27 09:40:51 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"17242026/08/27 09:40:51 WARN Failed to register uploaded object key=dmcsvwipqg4jrjrmlj1a9vwyh47m7hc5.ls error="server returned 404: 404 page not found\n"17252026/08/27 09:40:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign17262026/08/27 09:40:51 INFO Signed narinfos id=1 count=117272026/08/27 09:40:51 INFO Uploading 1 narinfos17282026/08/27 09:40:51 WARN Failed to register uploaded object key=dmcsvwipqg4jrjrmlj1a9vwyh47m7hc5.narinfo error="server returned 404: 404 page not found\n"17292026/08/27 09:40:51 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete17302026/08/27 09:40:51 INFO Received uploads request method=POST path=/api/pending_closures17312026/08/27 09:40:51 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)17322026/08/27 09:40:51 INFO Uploading p2r5waglmaz987dl98iaz43cl3an8z9r-unpinned-file.txt (128B)17332026/08/27 09:40:51 INFO Completed upload id=117342026/08/27 09:40:51 INFO Upload complete. (87ms)17352026/08/27 09:40:51 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"1736=== NAME TestClientCADerivations1737 client_ca_test.go:180: Narinfo contains CA field: StorePath: /build/TestClientCADerivations3215544231/001/store/dmcsvwipqg4jrjrmlj1a9vwyh47m7hc5-ca-test1738 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1739 Compression: zstd1740 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1741 NarSize: 1441742 References: 1743 Deriver: /build/TestClientCADerivations3215544231/001/store/c5wsbx6cprqw8sc1s5436qhj90arw3lg-ca-test.drv1744 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1745 client_ca_test.go:185: Checking for realisation files in S3...17462026/08/27 09:40:51 WARN Failed to register uploaded object key=p2r5waglmaz987dl98iaz43cl3an8z9r.ls error="server returned 404: 404 page not found\n"1747 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1748 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache17492026/08/27 09:40:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign17502026/08/27 09:40:51 INFO Signed narinfos id=2 count=117512026/08/27 09:40:51 INFO Uploading 1 narinfos17522026/08/27 09:40:51 WARN Failed to register uploaded object key=p2r5waglmaz987dl98iaz43cl3an8z9r.narinfo error="server returned 404: 404 page not found\n"17532026/08/27 09:40:51 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete17542026/08/27 09:40:51 INFO Completed upload id=217552026/08/27 09:40:51 INFO Upload complete. (77ms)17562026/08/27 09:40:51 WARN Authentication failed token_preview=invalid-token token_length=13 reason="static API token mismatch"17572026/08/27 09:40:51 INFO Received create pin request method=POST path=/api/pins/myapp17582026/08/27 09:40:51 INFO Created/updated pin name=myapp store_path=/build/TestPinProtectsFromGC1704933809/001/store/npg0b9rh5ina6gls3sx4yvfrsx0l0vln-pinned-file.txt narinfo_key=npg0b9rh5ina6gls3sx4yvfrsx0l0vln.narinfo17592026/08/27 09:40:51 INFO Starting cleanup of old closures method=DELETE path=/api/closures17602026/08/27 09:40:51 INFO Garbage collection started17612026/08/27 09:40:51 INFO Aborted multipart uploads count=017622026/08/27 09:40:51 WARN Force mode enabled - objects will be deleted immediately without grace period1763 client_ca_test.go:258: nix copy output: warning: you don't have Internet access; disabling some network-dependent features1764 warning: failed to create TLS context for AWS credential providers; SSO, STS WebIdentity, and ECS container authentication will be unavailable1765 error: binary cache 's3://bucket40?endpoint=http://localhost:42043®ion=eu-west-1' is for Nix stores with prefix '/nix/store', not '/build/TestClientCADerivations3215544231/001/store'1766 client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 11767--- PASS: TestClientCADerivations (0.58s)1768--- PASS: TestUploadHandlersRejectOversizedBody (0.18s)1769 --- PASS: TestUploadHandlersRejectOversizedBody/request_more_parts (0.04s)1770 --- PASS: TestUploadHandlersRejectOversizedBody/complete_multipart (0.07s)1771 --- PASS: TestUploadHandlersRejectOversizedBody/create_pending_closure (0.62s)17722026/08/27 09:40:52 WARN Request failed, retrying attempt=3 max_attempts=6 backoff=394.726876ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config17732026/08/27 09:40:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=2001 objects-failed-to-delete=017742026/08/27 09:40:52 INFO Vacuumed table table=pending_closures17752026/08/27 09:40:52 INFO Vacuumed table table=pending_objects17762026/08/27 09:40:52 INFO Vacuumed table table=multipart_uploads17772026/08/27 09:40:52 INFO Vacuumed table table=closures17782026/08/27 09:40:52 INFO Vacuumed table table=objects17792026/08/27 09:40:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=2001 objects-failed-to-delete=017802026/08/27 09:40:52 INFO Vacuumed table table=pending_closures17812026/08/27 09:40:52 INFO Vacuumed table table=pending_objects17822026/08/27 09:40:52 INFO Vacuumed table table=multipart_uploads17832026/08/27 09:40:52 INFO Vacuumed table table=closures17842026/08/27 09:40:52 INFO Vacuumed table table=objects1785=== NAME TestOrphanedObjectsGCStressTest1786 orphaned_objects_gc_test.go:425: Created 10 active closures, 5 to-delete closures, 20 orphaned chains1787 orphaned_objects_gc_test.go:446: Marked 210 objects for deletion17882026/08/27 09:40:52 WARN Request failed, retrying attempt=4 max_attempts=6 backoff=776.158737ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config1789 orphaned_objects_gc_test.go:509: Stress test completed successfully:1790 orphaned_objects_gc_test.go:510: - Active objects preserved: 201791 orphaned_objects_gc_test.go:511: - Objects deleted: 2101792 orphaned_objects_gc_test.go:512: - Total GC'd: 2101793--- PASS: TestOrphanedObjectsGCStressTest (1.88s)17942026/08/27 09:40:53 WARN Request failed, retrying attempt=5 max_attempts=6 backoff=1.495916735s error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config17952026/08/27 09:40:53 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=2001 objects_failed=01796=== NAME TestClientIntegration1797 client_integration_test.go:303: Objects in database after GC:1798 client_integration_test.go:303: Successfully deleted all objects with GC --force1799--- PASS: TestClientIntegration (2.42s)18002026/08/27 09:40:53 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=2001 objects_failed=01801=== NAME TestPinProtectsFromGC1802 client_integration_test.go:709: Pin successfully protected closure from garbage collection1803--- PASS: TestPinProtectsFromGC (2.58s)18042026/08/27 09:40:54 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: sending request: request failed after retries: Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused"18052026/08/27 09:40:54 WARN Request failed, retrying attempt=1 max_attempts=6 backoff=100ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures18062026/08/27 09:40:54 WARN Request failed, retrying attempt=2 max_attempts=6 backoff=209.619685ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures18072026/08/27 09:40:55 WARN Request failed, retrying attempt=3 max_attempts=6 backoff=429.9294ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures18082026/08/27 09:40:55 WARN Request failed, retrying attempt=4 max_attempts=6 backoff=874.878128ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures18092026/08/27 09:40:56 WARN Request failed, retrying attempt=5 max_attempts=6 backoff=1.505504326s error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures18102026/08/27 09:40:57 WARN Rate limiter enabled after throttle name=s3-test rate=518112026/08/27 09:40:57 WARN S3 rate limit hit operation="complete multipart upload" error="Please reduce your request rate."1812=== NAME TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle1813 throttle_test.go:213: Proxy stats: total=15, throttled=10, completeMultipart=101814 throttle_test.go:215: Rate limiter: enabled=true, rate=5.001815--- PASS: TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle (6.27s)1816--- PASS: TestClientErrorHandling (0.00s)1817 --- PASS: TestClientErrorHandling/InvalidStorePath (0.17s)1818 --- PASS: TestClientErrorHandling/InvalidAuthToken (0.27s)1819 --- PASS: TestClientErrorHandling/ServerNotAvailable (6.36s)1820PASS1821{"timestamp":"2026-08-27T09:40:57.93165668Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:36280","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":1836,"threadName":"rustfs-worker","threadId":"ThreadId(587)"}18222026-08-27 09:40:58.139 UTC [112] LOG: received smart shutdown request18232026-08-27 09:40:58.143 UTC [112] LOG: background worker "logical replication launcher" (PID 122) exited with exit code 118242026-08-27 09:40:58.148 UTC [117] LOG: shutting down18252026-08-27 09:40:58.149 UTC [117] LOG: checkpoint starting: shutdown immediate18262026-08-27 09:40:59.757 UTC [117] LOG: checkpoint complete: wrote 11287 buffers (68.9%), wrote 3 SLRU buffers; 0 WAL file(s) added, 0 removed, 13 recycled; write=0.264 s, sync=1.332 s, total=1.609 s; sync files=15167, longest=0.002 s, average=0.001 s; distance=208933 kB, estimate=208933 kB; lsn=0/E36D2F0, redo lsn=0/E36D2F018272026-08-27 09:40:59.843 UTC [112] LOG: database system is shut down1828Running OIDC tests...1829=== RUN TestGlobMatch1830=== PAUSE TestGlobMatch1831=== RUN TestAudienceForIssuer1832=== PAUSE TestAudienceForIssuer1833=== RUN TestValidateToken_ValidToken1834=== PAUSE TestValidateToken_ValidToken1835=== RUN TestValidateToken_WrongAudience1836=== PAUSE TestValidateToken_WrongAudience1837=== RUN TestValidateToken_Expired1838=== PAUSE TestValidateToken_Expired1839=== RUN TestValidateToken_BoundClaimsMismatch1840=== PAUSE TestValidateToken_BoundClaimsMismatch1841=== RUN TestValidateToken_BoundSubjectMismatch1842=== PAUSE TestValidateToken_BoundSubjectMismatch1843=== RUN TestValidateToken_MultipleProviders1844=== PAUSE TestValidateToken_MultipleProviders1845=== RUN TestValidateToken_NoMatchingProvider1846=== PAUSE TestValidateToken_NoMatchingProvider1847=== CONT TestGlobMatch1848=== CONT TestValidateToken_BoundClaimsMismatch1849=== RUN TestGlobMatch/foo_foo1850=== PAUSE TestGlobMatch/foo_foo1851=== RUN TestGlobMatch/foo_bar1852=== PAUSE TestGlobMatch/foo_bar1853=== CONT TestValidateToken_Expired1854=== CONT TestValidateToken_WrongAudience1855=== CONT TestValidateToken_ValidToken1856=== CONT TestAudienceForIssuer1857--- PASS: TestAudienceForIssuer (0.00s)1858=== CONT TestValidateToken_NoMatchingProvider1859=== CONT TestValidateToken_BoundSubjectMismatch1860=== CONT TestValidateToken_MultipleProviders1861=== RUN TestGlobMatch/*_1862=== PAUSE TestGlobMatch/*_1863=== RUN TestGlobMatch/*_anything1864=== PAUSE TestGlobMatch/*_anything1865=== RUN TestGlobMatch/foo*_foo1866=== PAUSE TestGlobMatch/foo*_foo1867=== RUN TestGlobMatch/foo*_foobar1868=== PAUSE TestGlobMatch/foo*_foobar1869=== RUN TestGlobMatch/foo*_bar1870=== PAUSE TestGlobMatch/foo*_bar1871=== RUN TestGlobMatch/*bar_bar1872=== PAUSE TestGlobMatch/*bar_bar1873=== RUN TestGlobMatch/*bar_foobar1874=== PAUSE TestGlobMatch/*bar_foobar1875=== RUN TestGlobMatch/*bar_foo1876=== PAUSE TestGlobMatch/*bar_foo1877=== RUN TestGlobMatch/foo*bar_foobar1878=== PAUSE TestGlobMatch/foo*bar_foobar1879=== RUN TestGlobMatch/foo*bar_foo123bar1880=== PAUSE TestGlobMatch/foo*bar_foo123bar1881=== RUN TestGlobMatch/foo*bar_foobarbaz1882=== PAUSE TestGlobMatch/foo*bar_foobarbaz1883=== RUN TestGlobMatch/*/*_foo/bar1884=== PAUSE TestGlobMatch/*/*_foo/bar1885=== RUN TestGlobMatch/*/*_foo1886=== PAUSE TestGlobMatch/*/*_foo1887=== RUN TestGlobMatch/refs/heads/*_refs/heads/main1888=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main1889=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.01890=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.01891=== RUN TestGlobMatch/refs/*/main_refs/heads/main1892=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main1893=== RUN TestGlobMatch/fo?_foo1894=== PAUSE TestGlobMatch/fo?_foo1895=== RUN TestGlobMatch/fo?_fo1896=== PAUSE TestGlobMatch/fo?_fo1897=== RUN TestGlobMatch/fo?_fooo1898=== PAUSE TestGlobMatch/fo?_fooo1899=== RUN TestGlobMatch/?oo_foo1900=== PAUSE TestGlobMatch/?oo_foo1901=== RUN TestGlobMatch/?oo_boo1902=== PAUSE TestGlobMatch/?oo_boo1903=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main1904=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main1905=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main1906=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main1907=== CONT TestGlobMatch/foo_foo1908=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main1909=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main1910=== CONT TestGlobMatch/?oo_boo1911=== CONT TestGlobMatch/?oo_foo1912=== CONT TestGlobMatch/fo?_fooo1913=== CONT TestGlobMatch/*/*_foo/bar1914=== CONT TestGlobMatch/foo*bar_foobarbaz1915=== CONT TestGlobMatch/foo*bar_foo123bar1916=== CONT TestGlobMatch/foo*bar_foobar1917=== CONT TestGlobMatch/fo?_fo1918=== CONT TestGlobMatch/fo?_foo1919=== CONT TestGlobMatch/refs/*/main_refs/heads/main1920=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.01921=== CONT TestGlobMatch/refs/heads/*_refs/heads/main1922=== CONT TestGlobMatch/*/*_foo1923=== CONT TestGlobMatch/*bar_foo1924=== CONT TestGlobMatch/*bar_foobar1925=== CONT TestGlobMatch/*bar_bar1926=== CONT TestGlobMatch/foo*_bar1927=== CONT TestGlobMatch/foo*_foobar1928=== CONT TestGlobMatch/foo*_foo1929=== CONT TestGlobMatch/*_anything1930=== CONT TestGlobMatch/*_1931=== CONT TestGlobMatch/foo_bar1932--- PASS: TestGlobMatch (0.00s)1933 --- PASS: TestGlobMatch/foo_foo (0.00s)1934 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)1935 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)1936 --- PASS: TestGlobMatch/?oo_boo (0.00s)1937 --- PASS: TestGlobMatch/?oo_foo (0.00s)1938 --- PASS: TestGlobMatch/fo?_fooo (0.00s)1939 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)1940 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)1941 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)1942 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)1943 --- PASS: TestGlobMatch/fo?_fo (0.00s)1944 --- PASS: TestGlobMatch/fo?_foo (0.00s)1945 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)1946 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)1947 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)1948 --- PASS: TestGlobMatch/*/*_foo (0.00s)1949 --- PASS: TestGlobMatch/*bar_foo (0.00s)1950 --- PASS: TestGlobMatch/*bar_foobar (0.00s)1951 --- PASS: TestGlobMatch/*bar_bar (0.00s)1952 --- PASS: TestGlobMatch/foo*_bar (0.00s)1953 --- PASS: TestGlobMatch/foo*_foobar (0.00s)1954 --- PASS: TestGlobMatch/foo*_foo (0.00s)1955 --- PASS: TestGlobMatch/*_anything (0.00s)1956 --- PASS: TestGlobMatch/*_ (0.00s)1957 --- PASS: TestGlobMatch/foo_bar (0.00s)19582026/08/27 09:41:00 INFO OIDC provider initialized name=test19592026/08/27 09:41:00 INFO OIDC provider initialized name=test19602026/08/27 09:41:00 INFO OIDC provider initialized name=provider119612026/08/27 09:41:00 INFO OIDC provider initialized name=test19622026/08/27 09:41:00 INFO OIDC provider initialized name=provider119632026/08/27 09:41:00 INFO OIDC provider initialized name=test19642026/08/27 09:41:00 INFO OIDC provider initialized name=test19652026/08/27 09:41:00 INFO OIDC provider initialized name=provider21966--- PASS: TestValidateToken_Expired (0.01s)1967--- PASS: TestValidateToken_BoundSubjectMismatch (0.01s)1968--- PASS: TestValidateToken_ValidToken (0.01s)1969--- PASS: TestValidateToken_WrongAudience (0.01s)1970--- PASS: TestValidateToken_BoundClaimsMismatch (0.01s)1971--- PASS: TestValidateToken_NoMatchingProvider (0.01s)1972--- PASS: TestValidateToken_MultipleProviders (0.01s)1973PASS1974Running hook tests...1975=== RUN TestSendPathsEmpty1976=== PAUSE TestSendPathsEmpty1977=== RUN TestQueueEnqueueAndFetch1978=== PAUSE TestQueueEnqueueAndFetch1979=== RUN TestQueueDeduplication1980=== PAUSE TestQueueDeduplication1981=== RUN TestQueueRemove1982=== PAUSE TestQueueRemove1983=== RUN TestQueueFetchBatchLimit1984=== PAUSE TestQueueFetchBatchLimit1985=== RUN TestQueueRetryMovesToBack1986=== PAUSE TestQueueRetryMovesToBack1987=== RUN TestQueueFetchRemoveLifecycle1988=== PAUSE TestQueueFetchRemoveLifecycle1989=== RUN TestQueueConcurrentWriters1990=== PAUSE TestQueueConcurrentWriters1991=== RUN TestQueueRemoveLargeClosure1992=== PAUSE TestQueueRemoveLargeClosure1993=== RUN TestServerClientIntegration1994=== PAUSE TestServerClientIntegration1995=== RUN TestServerQueueError1996=== PAUSE TestServerQueueError1997=== RUN TestGetListenerSocketActivation1998 server_test.go:210: === RUN TestGetListenerSocketActivation1999 --- PASS: TestGetListenerSocketActivation (0.00s)2000 PASS2001 2002--- PASS: TestGetListenerSocketActivation (0.01s)2003=== RUN TestDrainIsolatesPoisonPath2004=== PAUSE TestDrainIsolatesPoisonPath2005=== RUN TestRunNotBlockedByPoisonHead2006=== PAUSE TestRunNotBlockedByPoisonHead2007=== RUN TestDrainGivesUpWhenServerDown2008=== PAUSE TestDrainGivesUpWhenServerDown2009=== RUN TestFailedPathPrunedByLaterClosure2010=== PAUSE TestFailedPathPrunedByLaterClosure2011=== RUN TestWorkerUploadsAndRemoves2012=== PAUSE TestWorkerUploadsAndRemoves2013=== RUN TestWorkerSkipsGCdPaths2014=== PAUSE TestWorkerSkipsGCdPaths2015=== RUN TestWorkerPrunesClosureDeps2016=== PAUSE TestWorkerPrunesClosureDeps2017=== CONT TestSendPathsEmpty2018=== CONT TestServerClientIntegration2019--- PASS: TestSendPathsEmpty (0.00s)2020=== CONT TestQueueRemoveLargeClosure2021=== CONT TestQueueConcurrentWriters2022=== CONT TestQueueFetchRemoveLifecycle2023=== CONT TestQueueRetryMovesToBack2024=== CONT TestQueueFetchBatchLimit2025=== CONT TestQueueRemove2026=== CONT TestQueueDeduplication2027--- PASS: TestServerClientIntegration (0.00s)2028=== CONT TestQueueEnqueueAndFetch2029=== CONT TestRunNotBlockedByPoisonHead2030=== CONT TestDrainGivesUpWhenServerDown2031=== CONT TestDrainIsolatesPoisonPath2032=== CONT TestWorkerPrunesClosureDeps2033=== CONT TestServerQueueError2034=== CONT TestWorkerUploadsAndRemoves2035=== CONT TestFailedPathPrunedByLaterClosure2036=== CONT TestWorkerSkipsGCdPaths20372026/08/27 09:41:00 ERROR Failed to queue paths error="permission denied" count=12038--- PASS: TestServerQueueError (0.00s)20392026/08/27 09:41:00 INFO Uploading batch count=120402026/08/27 09:41:00 ERROR Upload failed error="upload failed" count=120412026/08/27 09:41:00 INFO Upload queue status pending=220422026/08/27 09:41:00 INFO Uploading batch count=420432026/08/27 09:41:00 ERROR Upload failed error="upload failed" count=420442026/08/27 09:41:00 INFO Uploading batch count=120452026/08/27 09:41:00 INFO Upload queue status pending=22046--- PASS: TestQueueFetchBatchLimit (0.02s)20472026/08/27 09:41:00 INFO Uploading batch count=120482026/08/27 09:41:00 INFO Uploading batch count=22049--- PASS: TestQueueRetryMovesToBack (0.02s)2050--- PASS: TestQueueEnqueueAndFetch (0.02s)20512026/08/27 09:41:00 INFO Uploading batch count=220522026/08/27 09:41:00 ERROR Upload failed error="upload failed" count=220532026/08/27 09:41:00 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown4074889915/002/a2054--- PASS: TestQueueDeduplication (0.02s)20552026/08/27 09:41:00 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainIsolatesPoisonPath855051476/002/bbb20562026/08/27 09:41:00 INFO Uploading batch count=12057--- PASS: TestQueueFetchRemoveLifecycle (0.02s)20582026/08/27 09:41:00 INFO Upload queue status pending=220592026/08/27 09:41:00 INFO Upload queue status pending=320602026/08/27 09:41:00 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown4074889915/002/b20612026/08/27 09:41:00 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerSkipsGCdPaths4013836147/002/nonexistent2062--- PASS: TestQueueRemove (0.02s)20632026/08/27 09:41:00 INFO Uploading batch count=120642026/08/27 09:41:00 ERROR Upload failed error="upload failed" count=120652026/08/27 09:41:00 INFO Uploading batch count=120662026/08/27 09:41:00 INFO Uploading batch count=220672026/08/27 09:41:00 ERROR Upload failed error="upload failed" count=220682026/08/27 09:41:00 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown4074889915/002/c2069--- PASS: TestFailedPathPrunedByLaterClosure (0.02s)20702026/08/27 09:41:00 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown4074889915/002/d20712026/08/27 09:41:00 INFO Uploading batch count=120722026/08/27 09:41:00 ERROR Upload failed error="upload failed" count=120732026/08/27 09:41:00 INFO Uploading batch count=220742026/08/27 09:41:00 ERROR Upload failed error="upload failed" count=220752026/08/27 09:41:00 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown4074889915/002/e20762026/08/27 09:41:00 INFO Uploading batch count=120772026/08/27 09:41:00 ERROR Upload failed error="upload failed" count=120782026/08/27 09:41:00 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown4074889915/002/f20792026/08/27 09:41:00 INFO Uploading batch count=120802026/08/27 09:41:00 ERROR Upload failed error="upload failed" count=120812026/08/27 09:41:00 ERROR Drain finished with paths left in queue remaining=1020822026/08/27 09:41:00 ERROR Drain finished with paths left in queue remaining=12083--- PASS: TestDrainGivesUpWhenServerDown (0.03s)2084--- PASS: TestDrainIsolatesPoisonPath (0.03s)2085--- PASS: TestWorkerSkipsGCdPaths (0.04s)2086--- PASS: TestWorkerPrunesClosureDeps (0.04s)2087--- PASS: TestWorkerUploadsAndRemoves (0.04s)2088--- PASS: TestQueueRemoveLargeClosure (0.15s)2089--- PASS: TestQueueConcurrentWriters (0.34s)20902026/08/27 09:41:01 INFO Uploading batch count=120912026/08/27 09:41:01 INFO Uploading batch count=120922026/08/27 09:41:01 INFO Uploading batch count=120932026/08/27 09:41:01 ERROR Upload failed error="upload failed" count=120942026/08/27 09:41:01 INFO Uploading batch count=120952026/08/27 09:41:01 ERROR Upload failed error="upload failed" count=120962026/08/27 09:41:01 INFO Uploading batch count=120972026/08/27 09:41:01 ERROR Upload failed error="upload failed" count=120982026/08/27 09:41:01 INFO Uploading batch count=120992026/08/27 09:41:01 ERROR Upload failed error="upload failed" count=121002026/08/27 09:41:01 ERROR Drain finished with paths left in queue remaining=12101--- PASS: TestRunNotBlockedByPoisonHead (1.04s)2102PASS