niks3-go-unit-tests
checks.aarch64-darwin.go-unit-tests
· build #410
· raw
1tribuchet: building on mac022Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestUploadBuildLog_FileBodyReplayedOnRetry6=== PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry7=== RUN TestRegisterUploadedObjectReusesConnections8=== PAUSE TestRegisterUploadedObjectReusesConnections9=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica10=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica11=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun12=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun13=== RUN TestCaseHackSuffix14=== PAUSE TestCaseHackSuffix15=== RUN TestFilterOversizedClosures16=== PAUSE TestFilterOversizedClosures17=== RUN TestUploadMultipart_PartsInParallel18=== PAUSE TestUploadMultipart_PartsInParallel19=== RUN TestPartSizeForNAR20=== PAUSE TestPartSizeForNAR21=== RUN TestUploadMultipart_SupersededByPeer22=== PAUSE TestUploadMultipart_SupersededByPeer23=== RUN TestDumpPathCaseHackMatchesNix24--- PASS: TestDumpPathCaseHackMatchesNix (0.06s)25=== RUN TestDumpPathCaseHackCollision26--- PASS: TestDumpPathCaseHackCollision (0.00s)27=== RUN TestSupersededNARStillUploadsListing28=== PAUSE TestSupersededNARStillUploadsListing29=== RUN TestDumpPathMatchesNix30=== PAUSE TestDumpPathMatchesNix31=== RUN TestDumpPathSingleFile32=== PAUSE TestDumpPathSingleFile33=== RUN TestDumpPathWriterError34=== PAUSE TestDumpPathWriterError35=== RUN TestEncodeNixBase3236=== PAUSE TestEncodeNixBase3237=== RUN TestEncodeNixBase32WithRealHash38=== PAUSE TestEncodeNixBase32WithRealHash39=== RUN TestConvertHashToNix3240=== PAUSE TestConvertHashToNix3241=== RUN TestGetStorePathHash42=== PAUSE TestGetStorePathHash43=== RUN TestPathInfoHashCompatibility44=== PAUSE TestPathInfoHashCompatibility45=== RUN TestParsePathInfoJSON46=== PAUSE TestParsePathInfoJSON47=== RUN TestParsePathInfoJSONMultiplePaths48=== PAUSE TestParsePathInfoJSONMultiplePaths49=== RUN TestPathInfoCACompatibility50=== PAUSE TestPathInfoCACompatibility51=== RUN TestRateLimiterFeedback52=== PAUSE TestRateLimiterFeedback53=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess54=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess55=== RUN TestRegisterUploadedObject_BoundedAgainstSilentServer56=== PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer57=== RUN TestResolveStorePath58=== PAUSE TestResolveStorePath59=== RUN TestDoWithRetry_BodyReplayedViaGetBody60=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody61=== RUN TestRetryTimeoutOutlastsMaxRetries62=== PAUSE TestRetryTimeoutOutlastsMaxRetries63=== RUN TestDoWithRetry_FinalResponseBodyReadable64=== PAUSE TestDoWithRetry_FinalResponseBodyReadable65=== RUN TestShellSplit66=== PAUSE TestShellSplit67=== RUN TestShellSplitErrors68=== PAUSE TestShellSplitErrors69=== RUN TestStreamPushReportsEveryPath70=== PAUSE TestStreamPushReportsEveryPath71=== RUN TestStreamPushBatchesUnderLoad72=== PAUSE TestStreamPushBatchesUnderLoad73=== RUN TestStreamPushIsolatesFailures74=== PAUSE TestStreamPushIsolatesFailures75=== RUN TestStreamPushGivesUpOnDeadServer76=== PAUSE TestStreamPushGivesUpOnDeadServer77=== RUN TestStreamPushRequestLine78=== PAUSE TestStreamPushRequestLine79=== RUN TestStreamPushReportsSignatures80=== PAUSE TestStreamPushReportsSignatures81=== RUN TestStreamPushReportsSkippedPaths82=== PAUSE TestStreamPushReportsSkippedPaths83=== RUN TestClientSignaturesByStorePath84=== PAUSE TestClientSignaturesByStorePath85=== RUN TestStreamPushStopsWhenCanceled86=== PAUSE TestStreamPushStopsWhenCanceled87=== RUN TestSetClientTLS88=== PAUSE TestSetClientTLS89=== RUN TestSetClientTLSDoesNotMutateDefaultTransport90=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport91=== RUN TestSetClientTLSErrors92=== PAUSE TestSetClientTLSErrors93=== RUN TestStaticToken94=== PAUSE TestStaticToken95=== RUN TestFileTokenReadsAndCaches96=== PAUSE TestFileTokenReadsAndCaches97=== RUN TestFileTokenMissing98=== PAUSE TestFileTokenMissing99=== RUN TestFileTokenEmpty100=== PAUSE TestFileTokenEmpty101=== RUN TestScriptTokenNoExpiryRerunsEveryCall102=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall103=== RUN TestScriptTokenCachesUntilRefresh104=== PAUSE TestScriptTokenCachesUntilRefresh105=== RUN TestScriptTokenEmptyToken106=== PAUSE TestScriptTokenEmptyToken107=== RUN TestScriptTokenBadJSON108=== PAUSE TestScriptTokenBadJSON109=== RUN TestScriptTokenScriptFails110=== PAUSE TestScriptTokenScriptFails111=== RUN TestScriptTokenEmptyCommand112=== PAUSE TestScriptTokenEmptyCommand113=== CONT TestDoServerRequestAttachesToken114=== CONT TestDoWithRetry_BodyReplayedViaGetBody115=== CONT TestUploadMultipart_PartsInParallel116=== CONT TestDumpPathWriterError117=== CONT TestResolveStorePath118=== CONT TestParsePathInfoJSON119=== RUN TestParsePathInfoJSON/Nix_format120=== PAUSE TestParsePathInfoJSON/Nix_format121=== RUN TestParsePathInfoJSON/Lix_format122=== PAUSE TestParsePathInfoJSON/Lix_format123=== RUN TestParsePathInfoJSON/empty_input124=== PAUSE TestParsePathInfoJSON/empty_input125=== RUN TestParsePathInfoJSON/whitespace_only126=== PAUSE TestParsePathInfoJSON/whitespace_only127=== RUN TestParsePathInfoJSON/invalid_JSON128=== PAUSE TestParsePathInfoJSON/invalid_JSON129=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess130=== CONT TestEncodeNixBase32131=== CONT TestRateLimiterFeedback132=== CONT TestPathInfoCACompatibility133=== CONT TestParsePathInfoJSONMultiplePaths134=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun135=== CONT TestFilterOversizedClosures136=== CONT TestCaseHackSuffix137=== CONT TestSupersededNARStillUploadsListing138=== CONT TestDumpPathSingleFile139=== RUN TestFilterOversizedClosures/no_limit_keeps_everything140=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything141=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped142=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped143=== RUN TestFilterOversizedClosures/all_closures_skipped144=== PAUSE TestFilterOversizedClosures/all_closures_skipped145=== CONT TestStreamPushStopsWhenCanceled146--- PASS: TestStreamPushStopsWhenCanceled (0.00s)147=== CONT TestScriptTokenScriptFails148--- PASS: TestResolveStorePath (0.00s)149=== CONT TestScriptTokenBadJSON150=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths1512026/10/11 05:54:55 WARN Rate limiter enabled after throttle name=server-test rate=5152=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths153=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths154=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths155=== CONT TestDumpPathMatchesNix156=== CONT TestConvertHashToNix32157=== CONT TestPathInfoHashCompatibility158=== CONT TestGetStorePathHash159=== CONT TestUploadMultipart_SupersededByPeer160=== CONT TestPartSizeForNAR161=== CONT TestEncodeNixBase32WithRealHash162=== CONT TestRegisterUploadedObject_BoundedAgainstSilentServer163=== RUN TestEncodeNixBase32/test_string_hash164=== CONT TestScriptTokenEmptyCommand165--- PASS: TestScriptTokenScriptFails (0.00s)166=== CONT TestScriptTokenCachesUntilRefresh167=== CONT TestScriptTokenEmptyToken168=== RUN TestRateLimiterFeedback/429_enables_limiter169=== RUN TestSupersededNARStillUploadsListing/small_NAR170=== PAUSE TestSupersededNARStillUploadsListing/small_NAR171=== RUN TestSupersededNARStillUploadsListing/dump_cut_short172=== PAUSE TestSupersededNARStillUploadsListing/dump_cut_short173=== RUN TestSupersededNARStillUploadsListing/listing_upload_fails174=== PAUSE TestRateLimiterFeedback/429_enables_limiter175=== RUN TestRateLimiterFeedback/503_enables_limiter176=== PAUSE TestRateLimiterFeedback/503_enables_limiter177=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter178=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter179=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter180=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter181=== PAUSE TestSupersededNARStillUploadsListing/listing_upload_fails182=== CONT TestFileTokenEmpty1832026/10/11 05:54:55 WARN Rate limiter enabled after throttle name=server-test rate=51842026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:586451852026/10/11 05:54:55 WARN Rate limiter backed off name=server-test rate=51862026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:586451872026/10/11 05:54:55 INFO Garbage collection started188=== CONT TestFileTokenReadsAndCaches189=== CONT TestFileTokenMissing190--- PASS: TestScriptTokenEmptyCommand (0.00s)191--- PASS: TestEncodeNixBase32WithRealHash (0.00s)192=== CONT TestScriptTokenNoExpiryRerunsEveryCall193=== RUN TestGetStorePathHash/valid_store_path194=== RUN TestPathInfoCACompatibility/null_ca_field195=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)196=== CONT TestStaticToken197=== CONT TestSetClientTLSErrors198=== PAUSE TestPathInfoCACompatibility/null_ca_field199=== PAUSE TestGetStorePathHash/valid_store_path200=== RUN TestGetStorePathHash/basename_without_hyphen_should_error201=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)202=== RUN TestPartSizeForNAR/zero_stays_at_minimum203=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum204=== RUN TestPartSizeForNAR/small_stays_at_minimum205=== PAUSE TestPartSizeForNAR/small_stays_at_minimum206=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum207=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum208=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts209=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts210=== RUN TestPartSizeForNAR/1_TiB211=== PAUSE TestPartSizeForNAR/1_TiB212=== RUN TestPartSizeForNAR/5_TiB_S3_max_object213=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object214=== RUN TestPartSizeForNAR/capped_at_5_GiB215=== PAUSE TestPartSizeForNAR/capped_at_5_GiB216=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error217=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error218=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error219=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error220=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error221=== CONT TestSetClientTLSDoesNotMutateDefaultTransport222=== PAUSE TestEncodeNixBase32/test_string_hash223=== RUN TestEncodeNixBase32/empty_input224=== PAUSE TestEncodeNixBase32/empty_input225=== RUN TestUploadMultipart_SupersededByPeer/exists226=== PAUSE TestUploadMultipart_SupersededByPeer/exists227=== RUN TestUploadMultipart_SupersededByPeer/missing228=== PAUSE TestUploadMultipart_SupersededByPeer/missing229=== CONT TestStreamPushIsolatesFailures230=== CONT TestClientSignaturesByStorePath231=== CONT TestStreamPushReportsSkippedPaths232--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.01s)233--- PASS: TestStaticToken (0.00s)234--- PASS: TestClientSignaturesByStorePath (0.00s)235--- PASS: TestStreamPushReportsSkippedPaths (0.00s)236=== CONT TestStreamPushReportsSignatures2372026/10/11 05:54:55 ERROR Upload failed error="bad path" count=3238=== RUN TestPathInfoCACompatibility/old_string_format_-_text239=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text240--- PASS: TestStreamPushIsolatesFailures (0.00s)241=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive242=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive243=== CONT TestStreamPushRequestLine244=== RUN TestPathInfoCACompatibility/new_structured_format_-_text245=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text246--- PASS: TestDoServerRequestAttachesToken (0.01s)247=== CONT TestStreamPushGivesUpOnDeadServer248=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method249=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method2502026/10/11 05:54:55 ERROR Upload failed error="connection refused" count=202512026/10/11 05:54:55 ERROR Server seems unavailable, giving up on batch untried=172522026/10/11 05:54:55 ERROR Upload failed error=boom count=1253--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (0.00s)254--- PASS: TestStreamPushGivesUpOnDeadServer (0.00s)255--- PASS: TestFileTokenMissing (0.00s)256=== CONT TestStreamPushReportsEveryPath257=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon258=== CONT TestSetClientTLS259=== RUN TestConvertHashToNix32/SRI_format_to_Nix32260=== CONT TestShellSplitErrors261=== CONT TestStreamPushBatchesUnderLoad262=== CONT TestDoWithRetry_FinalResponseBodyReadable263=== CONT TestShellSplit264=== RUN TestSetClientTLSErrors/missing_cert_file265=== PAUSE TestSetClientTLSErrors/missing_cert_file266=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica267=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32268=== RUN TestConvertHashToNix32/already_Nix32_format269=== PAUSE TestConvertHashToNix32/already_Nix32_format270=== RUN TestConvertHashToNix32/invalid_format271=== PAUSE TestConvertHashToNix32/invalid_format272=== CONT TestParsePathInfoJSON/Nix_format273=== RUN TestSetClientTLSErrors/missing_key_file2742026/10/11 05:54:55 ERROR Upload failed error=boom count=1275=== PAUSE TestSetClientTLSErrors/missing_key_file276=== CONT TestParsePathInfoJSON/Lix_format277=== RUN TestSetClientTLSErrors/missing_ca_file278=== PAUSE TestSetClientTLSErrors/missing_ca_file279=== RUN TestSetClientTLSErrors/invalid_ca_file280=== PAUSE TestSetClientTLSErrors/invalid_ca_file281--- PASS: TestFileTokenReadsAndCaches (0.00s)282--- PASS: TestScriptTokenBadJSON (0.01s)283--- PASS: TestFileTokenEmpty (0.00s)284--- PASS: TestStreamPushReportsEveryPath (0.00s)285--- PASS: TestShellSplitErrors (0.00s)286--- PASS: TestStreamPushReportsSignatures (0.00s)287=== CONT TestParsePathInfoJSON/empty_input288=== CONT TestFilterOversizedClosures/no_limit_keeps_everything289=== CONT TestRetryTimeoutOutlastsMaxRetries290=== CONT TestParsePathInfoJSON/invalid_JSON291=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths292=== CONT TestUploadBuildLog_FileBodyReplayedOnRetry293=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths294=== CONT TestRegisterUploadedObjectReusesConnections295--- PASS: TestShellSplit (0.00s)296--- PASS: TestParsePathInfoJSONMultiplePaths (0.00s)297 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)298 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)299=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped3002026/10/11 05:54:55 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000301=== CONT TestParsePathInfoJSON/whitespace_only302=== CONT TestRateLimiterFeedback/429_enables_limiter303=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter304=== CONT TestFilterOversizedClosures/all_closures_skipped3052026/10/11 05:54:55 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50306--- PASS: TestFilterOversizedClosures (0.00s)307 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)308 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)309 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)310=== CONT TestSupersededNARStillUploadsListing/small_NAR311--- PASS: TestParsePathInfoJSON (0.00s)312 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)313 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)314 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)315 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)316 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)317=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter318=== RUN TestSetClientTLS/rejects_connection_without_client_cert319=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert320=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA321=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA322=== RUN TestSetClientTLS/preserves_debug_logging_transport323=== PAUSE TestSetClientTLS/preserves_debug_logging_transport324=== CONT TestRateLimiterFeedback/503_enables_limiter3252026/10/11 05:54:55 INFO Uploading abc-hello (1.0GB)3262026/10/11 05:54:55 INFO Garbage collection started3272026/10/11 05:54:55 WARN Rate limiter enabled after throttle name=server-test rate=53282026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:58680329--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.01s)330=== CONT TestSupersededNARStillUploadsListing/listing_upload_fails331=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon332=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI333=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI334=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512335=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha5123362026/10/11 05:54:55 INFO Uploading abc-hello (1.0GB)337=== CONT TestSupersededNARStillUploadsListing/dump_cut_short3382026/10/11 05:54:55 WARN Rate limiter backed off name=server-test rate=5339=== CONT TestPartSizeForNAR/zero_stays_at_minimum340=== CONT TestGetStorePathHash/valid_store_path341=== CONT TestPartSizeForNAR/small_stays_at_minimum342=== CONT TestPartSizeForNAR/capped_at_5_GiB343=== CONT TestPartSizeForNAR/5_TiB_S3_max_object344=== CONT TestPartSizeForNAR/1_TiB345=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts346=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum347--- PASS: TestPartSizeForNAR (0.00s)348 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)349 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)350 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)351 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)352 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)353 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)354 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)355=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error356=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error357=== CONT TestGetStorePathHash/basename_without_hyphen_should_error358--- PASS: TestGetStorePathHash (0.00s)359 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)360 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)361 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)362 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)363=== CONT TestEncodeNixBase32/test_string_hash364=== CONT TestEncodeNixBase32/empty_input365--- PASS: TestEncodeNixBase32 (0.01s)366 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)367 --- PASS: TestEncodeNixBase32/empty_input (0.00s)368=== CONT TestUploadMultipart_SupersededByPeer/exists3692026/10/11 05:54:55 WARN Rate limiter enabled after throttle name=server-test rate=53702026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:58660371--- PASS: TestScriptTokenEmptyToken (0.01s)372=== CONT TestUploadMultipart_SupersededByPeer/missing373=== CONT TestPathInfoCACompatibility/null_ca_field374=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method375=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive376=== CONT TestPathInfoCACompatibility/new_structured_format_-_text3772026/10/11 05:54:55 WARN Rate limiter backed off name=server-test rate=5378=== CONT TestConvertHashToNix32/SRI_format_to_Nix32379=== CONT TestPathInfoCACompatibility/old_string_format_-_text380--- PASS: TestDoWithRetry_FinalResponseBodyReadable (0.01s)381=== CONT TestConvertHashToNix32/already_Nix32_format382--- PASS: TestPathInfoCACompatibility (0.00s)383 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)384 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)385 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)386 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)387 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)388=== CONT TestSetClientTLSErrors/invalid_ca_file3892026/10/11 05:54:55 WARN Rate limiter enabled after throttle name=server-test rate=53902026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:586643912026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:58662392=== CONT TestSetClientTLSErrors/missing_ca_file393=== CONT TestSetClientTLSErrors/missing_key_file3942026/10/11 05:54:55 WARN Rate limiter backed off name=server-test rate=5395=== CONT TestSetClientTLS/rejects_connection_without_client_cert3962026/10/11 05:54:55 WARN Rate limiter enabled after throttle name=s3-test rate=53972026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:58747398=== CONT TestSetClientTLS/preserves_debug_logging_transport399=== CONT TestConvertHashToNix32/invalid_format400--- PASS: TestConvertHashToNix32 (0.00s)401 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)402 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)403 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)404=== CONT TestSetClientTLSErrors/missing_cert_file405--- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (0.01s)406--- PASS: TestSetClientTLSErrors (0.00s)407 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)408 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)409 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)410 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)411=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA412--- PASS: TestUploadMultipart_SupersededByPeer (0.00s)413 --- PASS: TestUploadMultipart_SupersededByPeer/missing (0.00s)414 --- PASS: TestUploadMultipart_SupersededByPeer/exists (0.00s)415=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon416=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI417=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)4182026/10/11 05:54:55 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=0419--- PASS: TestRateLimiterFeedback (0.01s)420 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s)421 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.01s)422 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.01s)423 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.01s)424=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512425--- PASS: TestPathInfoHashCompatibility (0.01s)426 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)427 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)428 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)429 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)4302026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:586624312026/10/11 05:54:55 WARN Garbage collection finished on another replica. Its statistics are not available here432--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (0.01s)433--- PASS: TestStreamPushRequestLine (0.01s)4342026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:586624352026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:586624362026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:586624372026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:586624382026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:586624392026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:586624402026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:58662441--- PASS: TestRegisterUploadedObjectReusesConnections (0.02s)4422026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:586624432026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:586624442026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:586624452026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:586624462026/10/11 05:54:55 http: TLS handshake error from 127.0.0.1:58752: read tcp 127.0.0.1:58673->127.0.0.1:58752: use of closed network connection4472026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:58662448--- PASS: TestSetClientTLS (0.00s)449 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.01s)450 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.02s)451 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.03s)452--- PASS: TestScriptTokenCachesUntilRefresh (0.04s)4532026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:586624542026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:586624552026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:586624562026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:586624572026/10/11 05:54:55 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:58662458--- PASS: TestRetryTimeoutOutlastsMaxRetries (0.05s)459--- PASS: TestDumpPathWriterError (0.06s)460--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.06s)4612026/10/11 05:54:55 INFO Uploading abc-hello (1.0GB)462--- PASS: TestDumpPathSingleFile (0.08s)463--- PASS: TestCaseHackSuffix (0.07s)4642026/10/11 05:54:55 WARN Request failed, retrying attempt=1 backoff=100ms error="Put \"http://127.0.0.1:58730/part/2\": uploading part 1: server returned 404: no such upload\n" url=http://127.0.0.1:58730/part/24652026/10/11 05:54:55 WARN Request failed, retrying attempt=1 backoff=100ms error="Put \"http://127.0.0.1:58730/part/3\": uploading part 1: server returned 404: no such upload\n" url=http://127.0.0.1:58730/part/34662026/10/11 05:54:55 ERROR Failed to close zstd encoder error="io: read/write on closed pipe"467--- PASS: TestStreamPushBatchesUnderLoad (0.10s)468--- PASS: TestDumpPathMatchesNix (0.12s)4692026/10/11 05:54:55 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:58651/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:58651/api/uploads/complete4702026/10/11 05:54:55 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4712026/10/11 05:54:55 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:58651/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:58651/api/uploads/complete4722026/10/11 05:54:55 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4732026/10/11 05:54:55 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:58651/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:58651/api/uploads/complete4742026/10/11 05:54:55 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"475--- PASS: TestRegisterUploadedObject_BoundedAgainstSilentServer (0.20s)476--- PASS: TestSupersededNARStillUploadsListing (0.00s)477 --- PASS: TestSupersededNARStillUploadsListing/small_NAR (0.01s)478 --- PASS: TestSupersededNARStillUploadsListing/listing_upload_fails (0.01s)479 --- PASS: TestSupersededNARStillUploadsListing/dump_cut_short (0.56s)480--- PASS: TestUploadMultipart_PartsInParallel (0.62s)481--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)482PASS483Running server tests...484The files belonging to this database system will be owned by user "_tribuchetbld8".485This user must also own the server process.486487The database cluster will be initialized with locale "C".488The default database encoding has accordingly been set to "SQL_ASCII".489The default text search configuration will be set to "english".490491Data page checksums are enabled.492493creating directory /private/var/tb/8/scratch/59e19724d113dbbe/build/postgres1321795656/data ... ok494creating subdirectories ... ok495selecting dynamic shared memory implementation ... posix496selecting default "max_connections" ... 100497selecting default "shared_buffers" ... 128MB498selecting default time zone ... UTC499creating configuration files ... ok500running bootstrap script ... ok501performing post-bootstrap initialization ... ok502syncing data to disk ... ok503504initdb: warning: enabling "trust" authentication for local connections505initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.506507Success. You can now start the database server using:508509 pg_ctl -D /private/var/tb/8/scratch/59e19724d113dbbe/build/postgres1321795656/data -l logfile start510511/private/var/tb/8/scratch/59e19724d113dbbe/build/postgres1321795656:5432 - no response5122026-10-11 05:54:57.277 UTC [90579] LOG: starting PostgreSQL 18.6 on aarch64-apple-darwin25.6.0, compiled by clang version 21.1.8, 64-bit5132026-10-11 05:54:57.277 UTC [90579] LOG: listening on Unix socket "/private/var/tb/8/scratch/59e19724d113dbbe/build/postgres1321795656/.s.PGSQL.5432"5142026-10-11 05:54:57.281 UTC [90587] LOG: database system was shut down at 2026-10-11 05:54:57 UTC5152026-10-11 05:54:57.282 UTC [90579] LOG: database system is ready to accept connections516/private/var/tb/8/scratch/59e19724d113dbbe/build/postgres1321795656:5432 - accepting connections517{"timestamp":"2026-10-11T05:54:57.491299Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"07a1637a-586c-426c-9308-43d53364850e","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":501,"threadName":"rustfs-worker","threadId":"ThreadId(8)"}518{"timestamp":"2026-10-11T05:54:57.595061Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"f58dcc60-49fc-4509-9b97-300c86f0d6a3","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":501,"threadName":"rustfs-worker","threadId":"ThreadId(11)"}519=== RUN TestService_AuthMiddleware520=== PAUSE TestService_AuthMiddleware521=== RUN TestService_AuthMiddleware_MTLSProxyHeader522=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader523=== RUN TestService_AuthMiddleware_MTLSBoundSubjects524=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects525=== RUN TestService_ReadAuthMiddleware526=== PAUSE TestService_ReadAuthMiddleware527=== RUN TestService_AuthMiddleware_OIDC528=== PAUSE TestService_AuthMiddleware_OIDC529=== RUN TestService_RequireScope_OIDC530=== PAUSE TestService_RequireScope_OIDC531=== RUN TestService_ReadScope_PublicByDefault532=== PAUSE TestService_ReadScope_PublicByDefault533=== RUN TestCacheConfigHandler534=== PAUSE TestCacheConfigHandler535=== RUN TestCacheStatsHandler536=== PAUSE TestCacheStatsHandler537=== RUN TestClientCADerivations538=== PAUSE TestClientCADerivations539=== RUN TestClientErrorHandling540=== PAUSE TestClientErrorHandling541=== RUN TestClientIntegration542=== PAUSE TestClientIntegration543=== RUN TestClientMultipleUploads544=== PAUSE TestClientMultipleUploads545=== RUN TestClientWithDependencies546=== PAUSE TestClientWithDependencies547=== RUN TestClientSharedPathCommittedMidPush548=== PAUSE TestClientSharedPathCommittedMidPush549=== RUN TestPinProtectsFromGC550=== PAUSE TestPinProtectsFromGC551=== RUN TestClientPushesUseOnePush552=== PAUSE TestClientPushesUseOnePush553=== RUN TestClientFallsBackToClosures554=== PAUSE TestClientFallsBackToClosures555=== RUN TestDeleteClosuresRacingPins556=== PAUSE TestDeleteClosuresRacingPins557=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock558=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock559=== RUN TestResolveDBConnectionString560=== PAUSE TestResolveDBConnectionString561=== RUN TestConnectWaitsForAPeerMigration562=== PAUSE TestConnectWaitsForAPeerMigration563=== RUN TestLeadElectsOneAndHandsOver564=== PAUSE TestLeadElectsOneAndHandsOver565=== RUN TestLeadIncumbentWinsAfterRestart5662026/10/11 05:54:58 INFO lead: acquired remote=192.0.2.1:12345672026/10/11 05:54:58 INFO lead: released remote=192.0.2.1:12345682026/10/11 05:54:58 INFO lead: acquired remote=192.0.2.1:12345692026/10/11 05:54:59 INFO lead: released remote=192.0.2.1:1234570--- PASS: TestLeadIncumbentWinsAfterRestart (2.11s)571=== RUN TestLeadIncumbentAnnouncesAtOnce572=== PAUSE TestLeadIncumbentAnnouncesAtOnce573=== RUN TestLeadEndsOnShutdown574=== PAUSE TestLeadEndsOnShutdown575=== RUN TestLeadEndsWhenItsConnectionHangs5762026/10/11 05:55:00 INFO lead: acquired remote=192.0.2.1:12345772026-10-11 05:55:01.204 UTC [90630] FATAL: terminating connection due to administrator command5782026/10/11 05:55:01 INFO lead: acquired remote=192.0.2.1:12345792026/10/11 05:55:02 WARN lead: lock connection lost error="timeout: context deadline exceeded"5802026/10/11 05:55:02 INFO lead: released remote=192.0.2.1:12345812026/10/11 05:55:02 INFO lead: released remote=192.0.2.1:1234582--- PASS: TestLeadEndsWhenItsConnectionHangs (2.79s)583=== RUN TestGCAdvisoryLockBlocksConcurrentRun584=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun585=== RUN TestGCBugBareHashReferences586=== PAUSE TestGCBugBareHashReferences587=== RUN TestGCMetrics588=== PAUSE TestGCMetrics589=== RUN TestPushDedupSurvivesConcurrentGC590=== PAUSE TestPushDedupSurvivesConcurrentGC591=== RUN TestDeduplicatedObjectsRecordedAsPending592=== PAUSE TestDeduplicatedObjectsRecordedAsPending593=== RUN TestGCSweepSkipsPendingObjects594=== PAUSE TestGCSweepSkipsPendingObjects595=== RUN TestTombstonedObjectOfferedWithoutWaiting596=== PAUSE TestTombstonedObjectOfferedWithoutWaiting597=== RUN TestGCSweepDeliversEachKeyOnce598=== PAUSE TestGCSweepDeliversEachKeyOnce599=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection600=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection601=== RUN TestForceGCDuringPushOffersSweptObject602=== PAUSE TestForceGCDuringPushOffersSweptObject603=== RUN TestFailedS3DeleteKeepsTombstone604=== PAUSE TestFailedS3DeleteKeepsTombstone605=== RUN TestSweepSkipsRowsLockedByPush606=== PAUSE TestSweepSkipsRowsLockedByPush607=== RUN TestRegisterCompletedObjectNeedsPendingRow608=== PAUSE TestRegisterCompletedObjectNeedsPendingRow609=== RUN TestGetExistingObjectsLocksRows610=== PAUSE TestGetExistingObjectsLocksRows611=== RUN TestSweepRechecksPendingAfterLocking612=== PAUSE TestSweepRechecksPendingAfterLocking613=== RUN TestPushWaitsForSweepHoldingRow614=== PAUSE TestPushWaitsForSweepHoldingRow615=== RUN TestCommitRacingPendingCleanupKeepsObjects616=== PAUSE TestCommitRacingPendingCleanupKeepsObjects617=== RUN TestGCEndsOnShutdown618=== PAUSE TestGCEndsOnShutdown619=== RUN TestGCTaskStore_StartNew620=== PAUSE TestGCTaskStore_StartNew621=== RUN TestGCTaskStore_DeduplicateSameParams622=== PAUSE TestGCTaskStore_DeduplicateSameParams623=== RUN TestGCTaskStore_ConflictDifferentParams624=== PAUSE TestGCTaskStore_ConflictDifferentParams625=== RUN TestGCTaskStore_GetEmpty626=== PAUSE TestGCTaskStore_GetEmpty627=== RUN TestGCTaskStore_GetReturnsLatest628=== PAUSE TestGCTaskStore_GetReturnsLatest629=== RUN TestGCTaskStore_CompletedAllowsNewTask630=== PAUSE TestGCTaskStore_CompletedAllowsNewTask631=== RUN TestGCTaskStore_PhaseUpdates632=== PAUSE TestGCTaskStore_PhaseUpdates633=== RUN TestGCTaskStore_Fail634=== PAUSE TestGCTaskStore_Fail635=== RUN TestGracefulShutdownDrainsInflight636=== PAUSE TestGracefulShutdownDrainsInflight637=== RUN TestService_healthCheckHandler638=== PAUSE TestService_healthCheckHandler639=== RUN TestService_readinessHandler640=== PAUSE TestService_readinessHandler641=== RUN TestGenerateLandingPage642=== PAUSE TestGenerateLandingPage643=== RUN TestCacheConfigHandlerMaxNarSize644=== PAUSE TestCacheConfigHandlerMaxNarSize645=== RUN TestCreatePendingClosureRejectsOversizedNAR646=== PAUSE TestCreatePendingClosureRejectsOversizedNAR647=== RUN TestNARDeduplicationMetadataUploadBug648=== PAUSE TestNARDeduplicationMetadataUploadBug649=== RUN TestMetricsInventory650=== PAUSE TestMetricsInventory651=== RUN TestMetricsCommit652=== PAUSE TestMetricsCommit653=== RUN TestService_NativeMTLS654=== PAUSE TestService_NativeMTLS655=== RUN TestServerTLSConfig656=== PAUSE TestServerTLSConfig657=== RUN TestMultipartCleanup658=== PAUSE TestMultipartCleanup659=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded660=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded661=== RUN TestPendingCleanupUsesOneCutoff662=== PAUSE TestPendingCleanupUsesOneCutoff663=== RUN TestObjectStatsTrigger664=== PAUSE TestObjectStatsTrigger665=== RUN TestObjectStatsLargeCommit666=== PAUSE TestObjectStatsLargeCommit667=== RUN TestOrphanedObjectsGC668=== PAUSE TestOrphanedObjectsGC669=== RUN TestOrphanedObjectsGCStressTest670=== PAUSE TestOrphanedObjectsGCStressTest671=== RUN TestCreatePin_ReservedPins672=== PAUSE TestCreatePin_ReservedPins673=== RUN TestPresentReportsOnlyClosureRoots674=== PAUSE TestPresentReportsOnlyClosureRoots675=== RUN TestPresentRefreshesOnlyStaleClosures676=== PAUSE TestPresentRefreshesOnlyStaleClosures677=== RUN TestPresentNotReportedWhileGCDeletesClosure678=== PAUSE TestPresentNotReportedWhileGCDeletesClosure679=== RUN TestParseSingleRange680=== PAUSE TestParseSingleRange681=== RUN TestProxyHeadersOnlyTrustedOnSocket6822026/10/11 05:55:02 INFO Starting HTTP server address=127.0.0.1:587756832026/10/11 05:55:02 INFO Starting HTTP server address=proxy.sock6842026/10/11 05:55:02 WARN mTLS auth: subject not in bound subjects subject="CN=someone"6852026/10/11 05:55:02 INFO Shutdown signal received, draining in-flight requests timeout=10s686--- PASS: TestProxyHeadersOnlyTrustedOnSocket (0.20s)687=== RUN TestIsValidCachePath688=== PAUSE TestIsValidCachePath689=== RUN TestReadProxyNarinfo690=== PAUSE TestReadProxyNarinfo691=== RUN TestReadProxyNarinfoAlreadyDecompressed692=== PAUSE TestReadProxyNarinfoAlreadyDecompressed693=== RUN TestReadProxyContentNegotiation694=== PAUSE TestReadProxyContentNegotiation695=== RUN TestReadProxyNarStreaming696=== PAUSE TestReadProxyNarStreaming697=== RUN TestReadProxy404698=== PAUSE TestReadProxy404699=== RUN TestReadProxyInvalidPath700=== PAUSE TestReadProxyInvalidPath701=== RUN TestReadProxyHead702=== PAUSE TestReadProxyHead703=== RUN TestReadProxyOutlastsServerWriteTimeout704=== PAUSE TestReadProxyOutlastsServerWriteTimeout705=== RUN TestReadProxyConditionalGet706=== PAUSE TestReadProxyConditionalGet707=== RUN TestReadProxyRootRedirectsToIndexHTML708=== PAUSE TestReadProxyRootRedirectsToIndexHTML709=== RUN TestReadProxyDisabled710=== PAUSE TestReadProxyDisabled711=== RUN TestReadRedirectNar712=== PAUSE TestReadRedirectNar713=== RUN TestReadRedirectKeepsNarinfoProxied714=== PAUSE TestReadRedirectKeepsNarinfoProxied715=== RUN TestReadProxyRangeRequest716=== PAUSE TestReadProxyRangeRequest717=== RUN TestReadRedirectUsesPublicS3URL718=== PAUSE TestReadRedirectUsesPublicS3URL719=== RUN TestPush_OverlappingRootsStoreOneRowPerKey720=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey721=== RUN TestPush_CompleteCommitsEveryRoot722=== PAUSE TestPush_CompleteCommitsEveryRoot723=== RUN TestPush_RepeatedRootCommitsOnce724=== PAUSE TestPush_RepeatedRootCommitsOnce725=== RUN TestPush_CompleteWaitsForCleanupOfThePush726=== PAUSE TestPush_CompleteWaitsForCleanupOfThePush727=== RUN TestPush_RootsAreLockedInByteOrder728=== PAUSE TestPush_RootsAreLockedInByteOrder729=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit730=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit731=== RUN TestPush_RejectsBadRequests732=== PAUSE TestPush_RejectsBadRequests733=== RUN TestPush_SignsNarinfosOfItsPendingObjects734=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects735=== RUN TestRedundantMultipartUpload736=== PAUSE TestRedundantMultipartUpload737=== RUN TestCompleteMultipartUpload_ErrorButObjectExists738=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists739=== RUN TestCompletedNarNotReofferedAcrossClosures740=== PAUSE TestCompletedNarNotReofferedAcrossClosures741=== RUN TestPresignedUploadRegisteredBeforeCommit742=== PAUSE TestPresignedUploadRegisteredBeforeCommit743=== RUN TestService_Rustfstest744=== PAUSE TestService_Rustfstest745=== RUN TestParseSize746=== PAUSE TestParseSize747=== RUN TestSkippedUploadsHandler748=== PAUSE TestSkippedUploadsHandler749=== RUN TestSystemdListenerNotActivated750--- PASS: TestSystemdListenerNotActivated (0.00s)751=== RUN TestWatchdogBeatsWhenHealthy752--- PASS: TestWatchdogBeatsWhenHealthy (0.02s)753=== RUN TestWatchdogSkipsWhenUnhealthy7542026/10/11 05:55:02 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7552026/10/11 05:55:02 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7562026/10/11 05:55:02 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7572026/10/11 05:55:02 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7582026/10/11 05:55:02 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7592026/10/11 05:55:02 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7602026/10/11 05:55:02 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7612026/10/11 05:55:03 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7622026/10/11 05:55:03 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7632026/10/11 05:55:03 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"764--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)765=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle766=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle767=== RUN TestProxyWriteTimeout768=== PAUSE TestProxyWriteTimeout769=== RUN TestPendingClosureWriteTimeout770=== PAUSE TestPendingClosureWriteTimeout771=== RUN TestIsValidUploadKey772=== PAUSE TestIsValidUploadKey773=== RUN TestUploadHandlersRejectInvalidKeys774=== PAUSE TestUploadHandlersRejectInvalidKeys775=== RUN TestUploadHandlersRejectOversizedBody776=== PAUSE TestUploadHandlersRejectOversizedBody777=== RUN TestService_cleanupPendingClosuresHandler778=== PAUSE TestService_cleanupPendingClosuresHandler779=== RUN TestService_createPendingClosureHandler780=== PAUSE TestService_createPendingClosureHandler781=== RUN TestService_verifyS3Integrity782=== PAUSE TestService_verifyS3Integrity783=== RUN TestCompleteMultipartUnregistered784=== PAUSE TestCompleteMultipartUnregistered785=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT786=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT787=== CONT TestIsValidCachePath788=== CONT TestParseSingleRange789=== CONT TestParseSize790=== CONT TestCompletedNarNotReofferedAcrossClosures791--- PASS: TestParseSize (0.00s)792=== CONT TestPush_CompleteWaitsForCleanupOfThePush793=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit794=== RUN TestParseSingleRange/none795=== PAUSE TestParseSingleRange/none796=== RUN TestParseSingleRange/unknown_unit797=== PAUSE TestParseSingleRange/unknown_unit798=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT799=== CONT TestCompleteMultipartUnregistered800=== CONT TestService_verifyS3Integrity801=== CONT TestService_createPendingClosureHandler802=== CONT TestService_cleanupPendingClosuresHandler803=== CONT TestUploadHandlersRejectOversizedBody804=== CONT TestUploadHandlersRejectInvalidKeys805=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info806=== CONT TestIsValidUploadKey807=== CONT TestPendingClosureWriteTimeout808=== CONT TestProxyWriteTimeout809=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle810=== CONT TestSkippedUploadsHandler811=== CONT TestPresentNotReportedWhileGCDeletesClosure812=== CONT TestCompleteMultipartUpload_ErrorButObjectExists813=== RUN TestIsValidCachePath/narinfo814=== CONT TestService_AuthMiddleware815=== CONT TestService_Rustfstest816=== CONT TestPush_RootsAreLockedInByteOrder817=== CONT TestPresignedUploadRegisteredBeforeCommit818=== CONT TestPresentRefreshesOnlyStaleClosures819=== RUN TestParseSingleRange/multi-range_ignored820=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info821=== RUN TestIsValidUploadKey/narinfo822=== RUN TestProxyWriteTimeout/narinfo823=== RUN TestPendingClosureWriteTimeout/empty824=== RUN TestPresentNotReportedWhileGCDeletesClosure/stale825=== PAUSE TestPendingClosureWriteTimeout/empty826=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/stale827=== RUN TestPresentNotReportedWhileGCDeletesClosure/fresh828=== PAUSE TestIsValidCachePath/narinfo829=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars830=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/fresh831=== RUN TestPendingClosureWriteTimeout/400_objects8322026/10/11 05:55:03 INFO Client skipped oversized paths paths=3 nar_bytes=5000000000833=== PAUSE TestParseSingleRange/multi-range_ignored834=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal835=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal836=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key837=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key838=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key839=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key840=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars841=== RUN TestIsValidCachePath/nar_zst842=== PAUSE TestPendingClosureWriteTimeout/400_objects843=== RUN TestPendingClosureWriteTimeout/670k_objects844=== CONT TestPush_RepeatedRootCommitsOnce845=== PAUSE TestIsValidUploadKey/narinfo846=== RUN TestIsValidUploadKey/nar_zst847=== RUN TestParseSingleRange/malformed_no_dash848=== PAUSE TestPendingClosureWriteTimeout/670k_objects849=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline850=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline851=== CONT TestPush_OverlappingRootsStoreOneRowPerKey852=== PAUSE TestProxyWriteTimeout/narinfo853=== RUN TestProxyWriteTimeout/1_GiB_nar854=== PAUSE TestProxyWriteTimeout/1_GiB_nar855=== RUN TestProxyWriteTimeout/10_GiB_nar856=== PAUSE TestProxyWriteTimeout/10_GiB_nar857=== RUN TestProxyWriteTimeout/unknown_size858=== PAUSE TestProxyWriteTimeout/unknown_size859=== CONT TestReadRedirectUsesPublicS3URL860=== PAUSE TestIsValidCachePath/nar_zst861=== RUN TestIsValidCachePath/nar_xz862=== PAUSE TestIsValidCachePath/nar_xz863=== RUN TestIsValidCachePath/nar_bz2864=== PAUSE TestIsValidCachePath/nar_bz2865=== RUN TestIsValidCachePath/nar_uncompressed866=== PAUSE TestIsValidCachePath/nar_uncompressed867=== RUN TestIsValidCachePath/ls868=== PAUSE TestIsValidCachePath/ls869=== RUN TestIsValidCachePath/log870=== PAUSE TestIsValidCachePath/log871=== RUN TestIsValidCachePath/realisation872=== PAUSE TestIsValidCachePath/realisation873=== RUN TestIsValidCachePath/nix-cache-info874=== PAUSE TestIsValidCachePath/nix-cache-info875=== RUN TestIsValidCachePath/index.html876=== PAUSE TestIsValidCachePath/index.html877=== RUN TestIsValidCachePath/traversal_parent878=== PAUSE TestIsValidCachePath/traversal_parent879=== RUN TestIsValidCachePath/traversal_in_middle880=== PAUSE TestIsValidCachePath/traversal_in_middle881=== RUN TestIsValidCachePath/invalid_char_e882=== PAUSE TestIsValidCachePath/invalid_char_e883=== RUN TestIsValidCachePath/invalid_char_u884=== PAUSE TestIsValidCachePath/invalid_char_u885=== RUN TestIsValidCachePath/random_path886=== PAUSE TestIsValidCachePath/random_path887=== RUN TestIsValidCachePath/empty888=== PAUSE TestIsValidCachePath/empty889=== RUN TestIsValidCachePath/leading_slash890=== PAUSE TestIsValidCachePath/leading_slash891=== RUN TestIsValidCachePath/wrong_extension892=== PAUSE TestIsValidCachePath/wrong_extension893=== RUN TestIsValidCachePath/short_hash894=== PAUSE TestIsValidCachePath/short_hash895=== CONT TestPush_CompleteCommitsEveryRoot896--- PASS: TestSkippedUploadsHandler (0.05s)897=== CONT TestReadRedirectKeepsNarinfoProxied898=== CONT TestReadProxyRangeRequest899=== PAUSE TestIsValidUploadKey/nar_zst900=== RUN TestIsValidUploadKey/nar_xz901=== PAUSE TestIsValidUploadKey/nar_xz902=== RUN TestIsValidUploadKey/nar_plain903=== PAUSE TestIsValidUploadKey/nar_plain904=== RUN TestIsValidUploadKey/listing905=== PAUSE TestIsValidUploadKey/listing906=== RUN TestIsValidUploadKey/build_log907=== PAUSE TestIsValidUploadKey/build_log908=== RUN TestIsValidUploadKey/build_log_home-manager_file909=== PAUSE TestIsValidUploadKey/build_log_home-manager_file910=== RUN TestIsValidUploadKey/build_log_plus_in_name911=== PAUSE TestIsValidUploadKey/build_log_plus_in_name912=== RUN TestIsValidUploadKey/build_log_question_mark913=== PAUSE TestIsValidUploadKey/build_log_question_mark914=== RUN TestIsValidUploadKey/build_log_equals915=== PAUSE TestIsValidUploadKey/build_log_equals916=== RUN TestIsValidUploadKey/realisation917=== PAUSE TestIsValidUploadKey/realisation918=== RUN TestIsValidUploadKey/realisation_plus_in_output919=== PAUSE TestIsValidUploadKey/realisation_plus_in_output920=== RUN TestIsValidUploadKey/nix-cache-info921=== PAUSE TestIsValidUploadKey/nix-cache-info922=== RUN TestIsValidUploadKey/index.html923=== PAUSE TestIsValidUploadKey/index.html924=== RUN TestIsValidUploadKey/narinfo_key,_nar_type925=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type926=== RUN TestIsValidUploadKey/nar_key,_narinfo_type927=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type928=== RUN TestIsValidUploadKey/listing_key,_narinfo_type929=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type930=== RUN TestIsValidUploadKey/traversal931=== PAUSE TestIsValidUploadKey/traversal932=== RUN TestIsValidUploadKey/traversal_nar933=== PAUSE TestIsValidUploadKey/traversal_nar934=== RUN TestIsValidUploadKey/absolute935=== PAUSE TestIsValidUploadKey/absolute936=== RUN TestIsValidUploadKey/empty_key937=== PAUSE TestIsValidUploadKey/empty_key938=== RUN TestIsValidUploadKey/unknown_type939=== PAUSE TestIsValidUploadKey/unknown_type940=== PAUSE TestParseSingleRange/malformed_no_dash941=== RUN TestParseSingleRange/malformed_both_empty942=== PAUSE TestParseSingleRange/malformed_both_empty943=== RUN TestParseSingleRange/malformed_end_before_start944=== PAUSE TestParseSingleRange/malformed_end_before_start945=== RUN TestParseSingleRange/closed946=== PAUSE TestParseSingleRange/closed947=== RUN TestParseSingleRange/open-ended948=== PAUSE TestParseSingleRange/open-ended949=== RUN TestParseSingleRange/end_clamped_to_size950=== PAUSE TestParseSingleRange/end_clamped_to_size951=== RUN TestParseSingleRange/suffix952=== PAUSE TestParseSingleRange/suffix953=== RUN TestParseSingleRange/suffix_exceeds_size954=== PAUSE TestParseSingleRange/suffix_exceeds_size955=== RUN TestParseSingleRange/single_byte956=== PAUSE TestParseSingleRange/single_byte957=== RUN TestParseSingleRange/start_past_EOF958=== PAUSE TestParseSingleRange/start_past_EOF959=== RUN TestParseSingleRange/start_far_past_EOF960=== CONT TestReadRedirectNar961=== PAUSE TestParseSingleRange/start_far_past_EOF962=== CONT TestReadProxyDisabled963=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure964=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure965=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart966=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart967=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts968=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts969=== CONT TestReadProxyRootRedirectsToIndexHTML970--- PASS: TestPresentRefreshesOnlyStaleClosures (0.67s)971=== CONT TestReadProxyConditionalGet9722026/10/11 05:55:03 INFO Received uploads request method=POST path=/api/pending_closures9732026/10/11 05:55:04 INFO Received uploads request method=POST path=/api/pending_closures974--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (1.40s)975=== CONT TestReadProxyOutlastsServerWriteTimeout9762026/10/11 05:55:04 INFO Received uploads request method=POST path=/api/pending_closures9772026/10/11 05:55:05 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9782026/10/11 05:55:05 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=N2Q0OGEwM2MtOTBlZC00ZjAzLWEzMTctMjVjZTIzNGNlODg5LjBlNjBhZmFiLTA4YjUtNGUyMi05NzU0LTI4YThlZDY3YjAxNXgxNzkxNjk4MTA0ODA4NTEwMDAw9792026/10/11 05:55:05 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=N2Q0OGEwM2MtOTBlZC00ZjAzLWEzMTctMjVjZTIzNGNlODg5LjBlNjBhZmFiLTA4YjUtNGUyMi05NzU0LTI4YThlZDY3YjAxNXgxNzkxNjk4MTA0ODA4NTEwMDAw parts=1980--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (2.09s)981=== CONT TestReadProxyHead9822026/10/11 05:55:05 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9832026/10/11 05:55:05 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst984--- PASS: TestCompleteMultipartUnregistered (2.36s)985=== CONT TestReadProxyInvalidPath9862026/10/11 05:55:05 INFO Received push request method=POST path=/api/pushes9872026/10/11 05:55:05 INFO Received cleanup request method=DELETE path=/api/pending_closures9882026/10/11 05:55:05 INFO Aborted multipart uploads count=0 kept=09892026/10/11 05:55:05 INFO Received complete multipart upload request method=POST path=/api/multipart/complete9902026/10/11 05:55:06 INFO Received uploads request method=POST path=/api/pending_closures9912026-10-11 05:55:06.063 UTC [90829] ERROR: Push does not exist: id=19922026-10-11 05:55:06.063 UTC [90829] CONTEXT: PL/pgSQL function commit_push(bigint) line 11 at RAISE9932026-10-11 05:55:06.063 UTC [90829] STATEMENT: SELECT commit_push($1)994--- PASS: TestPush_CompleteWaitsForCleanupOfThePush (3.02s)9952026/10/11 05:55:06 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=N2Q0OGEwM2MtOTBlZC00ZjAzLWEzMTctMjVjZTIzNGNlODg5LjUzOGM0MjdlLTIyNzQtNDkzYS04NGVhLTU1NTFiYzVlMWE2NngxNzkxNjk4MTA0MDQ4NzY5MDAw parts=12996=== CONT TestReadProxy4049972026/10/11 05:55:06 INFO Received cleanup request method=DELETE path=/api/pending_closures9982026/10/11 05:55:06 INFO Received uploads request method=POST path=/api/pending_closures999=== CONT TestReadProxyNarStreaming1000--- PASS: TestCompletedNarNotReofferedAcrossClosures (3.04s)10012026/10/11 05:55:06 INFO Aborted multipart uploads count=1 kept=010022026/10/11 05:55:06 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10032026-10-11 05:55:06.112 UTC [90798] ERROR: Closure does not exist: id=110042026-10-11 05:55:06.112 UTC [90798] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 13 at RAISE10052026-10-11 05:55:06.112 UTC [90798] STATEMENT: -- name: CommitPendingClosure :one1006 SELECT commit_pending_closure($1::bigint)::bigint1007 1008--- PASS: TestService_cleanupPendingClosuresHandler (3.07s)1009=== CONT TestReadProxyContentNegotiation10102026/10/11 05:55:06 INFO Received push request method=POST path=/api/pushes10112026/10/11 05:55:06 INFO Received complete push request method=POST path=/api/pushes/1/complete10122026/10/11 05:55:06 INFO Completed push id=1 objects=4 duration=15.128708ms10132026/10/11 05:55:06 INFO Received push request method=POST path=/api/pushes10142026/10/11 05:55:06 INFO Aborted multipart uploads count=0 kept=010152026/10/11 05:55:06 WARN Force mode enabled - objects will be deleted immediately without grace period10162026/10/11 05:55:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=010172026/10/11 05:55:06 INFO Received push request method=POST path=/api/pushes10182026/10/11 05:55:06 INFO Vacuumed table table=pending_closures10192026/10/11 05:55:06 INFO Vacuumed table table=pending_objects10202026/10/11 05:55:06 INFO Vacuumed table table=multipart_uploads10212026/10/11 05:55:06 INFO Vacuumed table table=closures10222026/10/11 05:55:06 INFO Vacuumed table table=objects10232026/10/11 05:55:06 INFO Received complete push request method=POST path=/api/pushes/2/complete10242026/10/11 05:55:06 INFO Completed push id=2 objects=4 duration=6.624333ms10252026-10-11 05:55:06.662 UTC [90839] ERROR: could not obtain lock on row in relation "closures"10262026-10-11 05:55:06.662 UTC [90839] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT1027--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (3.62s)1028=== CONT TestReadProxyNarinfoAlreadyDecompressed10292026-10-11 05:55:06.662 UTC [90839] ERROR: could not obtain lock on row in relation "closures"10302026-10-11 05:55:06.662 UTC [90839] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT10312026-10-11 05:55:06.662 UTC [90839] ERROR: could not obtain lock on row in relation "closures"10322026-10-11 05:55:06.662 UTC [90839] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT10332026-10-11 05:55:06.663 UTC [90839] ERROR: could not obtain lock on row in relation "closures"10342026-10-11 05:55:06.663 UTC [90839] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT10352026-10-11 05:55:06.667 UTC [90839] ERROR: could not obtain lock on row in relation "closures"10362026-10-11 05:55:06.667 UTC [90839] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT1037--- PASS: TestPush_RootsAreLockedInByteOrder (3.63s)1038=== CONT TestReadProxyNarinfo10392026/10/11 05:55:06 INFO Received uploads request method=POST path=/api/pending_closures10402026/10/11 05:55:07 INFO Received uploads request method=POST path=/api/pending_closures10412026/10/11 05:55:07 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst10422026/10/11 05:55:07 INFO Received uploads request method=POST path=/api/pending_closures1043--- PASS: TestPresignedUploadRegisteredBeforeCommit (4.03s)1044=== CONT TestPush_SignsNarinfosOfItsPendingObjects10452026/10/11 05:55:07 INFO Received uploads request method=POST path=/api/pending_closures10462026/10/11 05:55:07 INFO Received uploads request method=POST path=/api/pending_closures10472026/10/11 05:55:07 INFO Received uploads request method=POST path=/api/pending_closures1048--- PASS: TestService_Rustfstest (4.55s)1049=== CONT TestRedundantMultipartUpload1050--- PASS: TestReadProxyDisabled (4.77s)1051=== CONT TestPush_RejectsBadRequests1052--- PASS: TestReadProxyRangeRequest (5.09s)1053=== CONT TestFailedS3DeleteKeepsTombstone10542026/10/11 05:55:08 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10552026/10/11 05:55:08 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=N2Q0OGEwM2MtOTBlZC00ZjAzLWEzMTctMjVjZTIzNGNlODg5LjRlZDUxZjg5LWJhOGYtNGY1OS04ZGY5LTZhMDZlZGJkNzI0YXgxNzkxNjk4MTA2ODEwNjUyMDAw parts=1010562026/10/11 05:55:08 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10572026/10/11 05:55:08 INFO Completed upload id=1 objects=2 duration=9.498541ms10582026/10/11 05:55:08 INFO Received uploads request method=POST path=/api/pending_closures10592026/10/11 05:55:08 INFO Received uploads request method=POST path=/api/pending_closures10602026/10/11 05:55:08 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo10612026/10/11 05:55:08 WARN Found objects in DB but missing from S3, will re-upload count=11062--- PASS: TestService_verifyS3Integrity (5.28s)1063=== CONT TestSweepSkipsRowsLockedByPush10642026/10/11 05:55:08 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"1065--- PASS: TestService_AuthMiddleware (5.38s)1066=== CONT TestForceGCDuringPushOffersSweptObject1067=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows1068=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows1069=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check1070=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check1071=== CONT TestGenerateLandingPage1072--- PASS: TestGenerateLandingPage (0.00s)1073=== CONT TestService_readinessHandler1074--- PASS: TestReadRedirectNar (5.58s)1075=== CONT TestService_healthCheckHandler10762026/10/11 05:55:08 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10772026/10/11 05:55:08 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=N2Q0OGEwM2MtOTBlZC00ZjAzLWEzMTctMjVjZTIzNGNlODg5LjAxYjI2MjUyLTljNDgtNDE5Yi04ZWQzLWI0MzE3MTFiZmQ1MHgxNzkxNjk4MTA3MzMxODMzMDAw parts=1010782026/10/11 05:55:08 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10792026/10/11 05:55:08 INFO Completed upload id=1 objects=2 duration=9.40675ms10802026/10/11 05:55:08 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000010812026/10/11 05:55:08 INFO Received uploads request method=POST path=/api/pending_closures10822026/10/11 05:55:08 INFO Starting cleanup of old closures method=DELETE path=/api/closures10832026/10/11 05:55:08 INFO Aborted multipart uploads count=0 kept=010842026/10/11 05:55:08 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=010852026/10/11 05:55:08 INFO Vacuumed table table=pending_closures10862026/10/11 05:55:08 INFO Vacuumed table table=pending_objects10872026/10/11 05:55:08 INFO Vacuumed table table=multipart_uploads10882026/10/11 05:55:08 INFO Received push request method=POST path=/api/pushes10892026/10/11 05:55:08 INFO Vacuumed table table=closures10902026/10/11 05:55:08 INFO Vacuumed table table=objects10912026/10/11 05:55:08 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001092--- PASS: TestService_createPendingClosureHandler (5.92s)1093=== CONT TestGracefulShutdownDrainsInflight10942026/10/11 05:55:08 INFO Starting HTTP server address=127.0.0.1:5893410952026/10/11 05:55:08 INFO Shutdown signal received, draining in-flight requests timeout=10s10962026/10/11 05:55:09 INFO Received complete push request method=POST path=/api/pushes/1/complete1097--- PASS: TestGracefulShutdownDrainsInflight (0.08s)1098=== CONT TestGCTaskStore_Fail1099--- PASS: TestGCTaskStore_Fail (0.00s)1100=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded11012026/10/11 05:55:09 INFO Completed push id=1 objects=4 duration=30.396958ms1102--- PASS: TestPush_RepeatedRootCommitsOnce (5.96s)1103=== CONT TestMultipartCleanup1104--- PASS: TestReadRedirectKeepsNarinfoProxied (6.09s)1105=== CONT TestGCTaskStore_PhaseUpdates1106--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)1107=== CONT TestPresentReportsOnlyClosureRoots11082026/10/11 05:55:09 INFO Received uploads request method=POST path=/api/pending_closures1109--- PASS: TestReadRedirectUsesPublicS3URL (6.67s)1110=== CONT TestNARDeduplicationMetadataUploadBug11112026/10/11 05:55:09 INFO Received complete multipart upload request method=POST path=/api/multipart/complete11122026/10/11 05:55:10 INFO Received push request method=POST path=/api/pushes1113--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (7.13s)1114=== CONT TestGCTaskStore_CompletedAllowsNewTask1115--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)1116=== CONT TestCreatePin_ReservedPins11172026/10/11 05:55:10 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58956/oidc1118--- PASS: TestReadProxyRootRedirectsToIndexHTML (7.37s)1119=== CONT TestCreatePendingClosureRejectsOversizedNAR11202026/10/11 05:55:10 INFO Received uploads request method=POST path=/api/pending_closures1121--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.01s)1122=== CONT TestGCTaskStore_GetReturnsLatest1123--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)1124=== CONT TestOrphanedObjectsGCStressTest11252026/10/11 05:55:10 INFO Received push request method=POST path=/api/pushes11262026/10/11 05:55:11 INFO Received complete push request method=POST path=/api/pushes/1/complete11272026/10/11 05:55:11 INFO Completed push id=1 objects=6 duration=11.110625ms1128--- PASS: TestPush_CompleteCommitsEveryRoot (7.97s)1129=== CONT TestCacheConfigHandlerMaxNarSize1130--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)1131=== CONT TestGCTaskStore_GetEmpty1132--- PASS: TestGCTaskStore_GetEmpty (0.00s)1133=== CONT TestOrphanedObjectsGC1134--- PASS: TestReadProxyConditionalGet (7.88s)1135=== CONT TestPushWaitsForSweepHoldingRow1136--- PASS: TestReadProxyHead (6.75s)1137=== CONT TestObjectStatsLargeCommit1138--- PASS: TestReadProxyInvalidPath (6.70s)1139=== CONT TestSweepRechecksPendingAfterLocking1140--- PASS: TestReadProxy404 (6.56s)1141=== CONT TestCommitRacingPendingCleanupKeepsObjects1142--- PASS: TestReadProxyNarStreaming (6.87s)1143=== CONT TestGetExistingObjectsLocksRows1144--- PASS: TestReadProxyContentNegotiation (7.22s)1145=== CONT TestGCTaskStore_ConflictDifferentParams1146--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)1147=== CONT TestGCTaskStore_DeduplicateSameParams1148--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)1149=== CONT TestGCTaskStore_StartNew1150--- PASS: TestGCTaskStore_StartNew (0.00s)1151=== CONT TestGCEndsOnShutdown1152=== RUN TestGCEndsOnShutdown/before_the_run1153=== PAUSE TestGCEndsOnShutdown/before_the_run1154=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1155=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1156=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1157=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1158=== CONT TestRegisterCompletedObjectNeedsPendingRow11592026/10/11 05:55:13 ERROR Failed to decode narinfo key=3hcdxyjf9yiq7qf3i4548drb6sjmwa1v.narinfo limit=16777216 error=<nil>1160--- PASS: TestReadProxyNarinfo (6.86s)1161=== CONT TestPendingCleanupUsesOneCutoff1162--- PASS: TestReadProxyNarinfoAlreadyDecompressed (7.08s)1163=== CONT TestServerTLSConfig1164=== RUN TestServerTLSConfig/no_client_CA1165=== PAUSE TestServerTLSConfig/no_client_CA1166=== RUN TestServerTLSConfig/missing_CA_file1167=== PAUSE TestServerTLSConfig/missing_CA_file1168=== RUN TestServerTLSConfig/not_a_PEM_file1169=== PAUSE TestServerTLSConfig/not_a_PEM_file1170=== CONT TestMetricsInventory11712026/10/11 05:55:13 INFO Received push request method=POST path=/api/pushes11722026/10/11 05:55:14 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign11732026/10/11 05:55:14 INFO Signed narinfos id=1 count=111742026/10/11 05:55:14 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign11752026/10/11 05:55:14 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign11762026/10/11 05:55:14 ERROR Failed to sign narinfos signer="index 1" error="signing key cannot be nil"11772026/10/11 05:55:14 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign11782026/10/11 05:55:14 ERROR Failed to sign narinfos signer=niks3-test-1 error="signer failed"11792026/10/11 05:55:14 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign11802026/10/11 05:55:14 ERROR Signer returned unexpected signature count signer=niks3-test-1 got=0 want=111812026/10/11 05:55:14 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign11822026/10/11 05:55:14 ERROR Signer returned unexpected object key signer=niks3-test-1 object_key=unexpected.narinfo1183--- PASS: TestPush_SignsNarinfosOfItsPendingObjects (6.93s)1184=== CONT TestMetricsCommit11852026/10/11 05:55:14 INFO Received uploads request method=POST path=/api/pending_closures11862026/10/11 05:55:14 INFO Received uploads request method=POST path=/api/pending_closures1187=== RUN TestPush_RejectsBadRequests/no_objects1188=== PAUSE TestPush_RejectsBadRequests/no_objects1189=== RUN TestPush_RejectsBadRequests/bad_root1190=== PAUSE TestPush_RejectsBadRequests/bad_root1191=== RUN TestPush_RejectsBadRequests/root_not_in_objects1192=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects1193=== RUN TestPush_RejectsBadRequests/no_roots1194=== PAUSE TestPush_RejectsBadRequests/no_roots1195=== CONT TestObjectStatsTrigger11962026/10/11 05:55:14 INFO Aborted multipart uploads count=0 kept=011972026/10/11 05:55:14 WARN Force mode enabled - objects will be deleted immediately without grace period11982026/10/11 05:55:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011992026/10/11 05:55:14 INFO Vacuumed table table=pending_closures12002026/10/11 05:55:14 INFO Vacuumed table table=pending_objects12012026/10/11 05:55:14 INFO Vacuumed table table=multipart_uploads12022026/10/11 05:55:14 INFO Vacuumed table table=closures12032026/10/11 05:55:14 INFO Vacuumed table table=objects12042026/10/11 05:55:14 INFO Aborted multipart uploads count=0 kept=012052026/10/11 05:55:14 WARN Force mode enabled - objects will be deleted immediately without grace period12062026/10/11 05:55:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=012072026/10/11 05:55:14 INFO Vacuumed table table=pending_closures12082026/10/11 05:55:14 INFO Vacuumed table table=pending_objects12092026/10/11 05:55:14 INFO Vacuumed table table=multipart_uploads12102026/10/11 05:55:14 INFO Vacuumed table table=closures12112026/10/11 05:55:14 INFO Vacuumed table table=objects1212--- PASS: TestSweepSkipsRowsLockedByPush (6.64s)1213=== CONT TestService_NativeMTLS12142026/10/11 05:55:15 WARN Rate limiter enabled after throttle name=s3-test rate=512152026/10/11 05:55:15 WARN S3 rate limit hit operation="complete multipart upload" error="Please reduce your request rate."1216=== NAME TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle1217 throttle_test.go:213: Proxy stats: total=13, throttled=10, completeMultipart=101218 throttle_test.go:215: Rate limiter: enabled=true, rate=5.001219--- PASS: TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle (11.98s)1220=== CONT TestDeduplicatedObjectsRecordedAsPending12212026/10/11 05:55:15 INFO Received uploads request method=POST path=/api/pending_closures12222026/10/11 05:55:15 INFO Aborted multipart uploads count=0 kept=012232026/10/11 05:55:15 WARN Force mode enabled - objects will be deleted immediately without grace period12242026/10/11 05:55:15 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."12252026/10/11 05:55:15 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."1226--- PASS: TestFailedS3DeleteKeepsTombstone (7.07s)1227=== CONT TestGCSweepDeliversEachKeyOnce12282026/10/11 05:55:15 WARN readiness check failed error="closed pool"1229--- PASS: TestService_readinessHandler (6.87s)1230=== CONT TestTombstonedObjectOfferedWithoutWaiting1231--- PASS: TestReadProxyOutlastsServerWriteTimeout (11.08s)1232=== CONT TestGCSweepSkipsPendingObjects1233--- PASS: TestService_healthCheckHandler (6.92s)1234=== CONT TestGCAdvisoryLockBlocksConcurrentRun12352026/10/11 05:55:15 INFO Received complete multipart upload request method=POST path=/api/multipart/complete12362026/10/11 05:55:15 INFO Received uploads request method=POST path=/api/pending_closures12372026/10/11 05:55:15 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=N2Q0OGEwM2MtOTBlZC00ZjAzLWEzMTctMjVjZTIzNGNlODg5LjI0ZmMyMjUwLWM0NjMtNDdlZi04OGIwLWZiNjY2NGY0NjU4ZXgxNzkxNjk4MTE0MjA1Mjk0MDAw parts=1212382026/10/11 05:55:15 INFO Received uploads request method=POST path=/api/pending_closures12392026/10/11 05:55:16 INFO Received uploads request method=POST path=/api/pending_closures1240--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (7.03s)1241=== CONT TestPushDedupSurvivesConcurrentGC12422026/10/11 05:55:16 INFO Received uploads request method=POST path=/api/pending_closures12432026/10/11 05:55:16 INFO Received cleanup request method=DELETE path=/api/pending_closures12442026/10/11 05:55:16 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket45/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""12452026/10/11 05:55:16 INFO Aborted multipart uploads count=0 kept=112462026/10/11 05:55:16 INFO Received cleanup request method=DELETE path=/api/pending_closures12472026/10/11 05:55:16 INFO Aborted multipart uploads count=1 kept=01248--- PASS: TestMultipartCleanup (7.67s)1249=== CONT TestGCMetrics1250--- PASS: TestPresentReportsOnlyClosureRoots (7.70s)1251=== CONT TestClientFallsBackToClosures1252=== NAME TestNARDeduplicationMetadataUploadBug1253 metadata_upload_test.go:48: First store path: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestNARDeduplicationMetadataUploadBug3447237600/001/store/ddm39k0ls6c764blid2g34w17gax70yd-file1.txt12542026/10/11 05:55:17 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux12552026/10/11 05:55:17 WARN Refused reserved pin name=worker-x86_64-linux12562026/10/11 05:55:17 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux12572026/10/11 05:55:17 INFO Received create pin request method=POST path=/api/pins/my-app12582026/10/11 05:55:17 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux1259--- PASS: TestCreatePin_ReservedPins (7.32s)1260=== CONT TestGCBugBareHashReferences12612026/10/11 05:55:17 INFO Received push request method=POST path=/api/pushes12622026/10/11 05:55:17 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)12632026/10/11 05:55:17 INFO Uploading ddm39k0ls6c764blid2g34w17gax70yd-file1.txt (160B)12642026/10/11 05:55:17 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"12652026/10/11 05:55:17 WARN Failed to register uploaded object key=ddm39k0ls6c764blid2g34w17gax70yd.ls error="server returned 404: 404 page not found\n"12662026/10/11 05:55:17 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12672026/10/11 05:55:17 INFO Signed narinfos id=1 count=112682026/10/11 05:55:17 INFO Uploading 1 narinfos12692026/10/11 05:55:17 INFO Received complete push request method=POST path=/api/pushes/1/complete12702026/10/11 05:55:17 WARN Failed to register uploaded object key=ddm39k0ls6c764blid2g34w17gax70yd.narinfo error="server returned 404: 404 page not found\n"12712026/10/11 05:55:17 INFO Completed push id=1 objects=3 duration=34.165458ms12722026/10/11 05:55:17 INFO Upload complete. (200ms)1273=== NAME TestNARDeduplicationMetadataUploadBug1274 metadata_upload_test.go:54: Retrieved narinfo from S3:1275 StorePath: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestNARDeduplicationMetadataUploadBug3447237600/001/store/ddm39k0ls6c764blid2g34w17gax70yd-file1.txt1276 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1277 Compression: zstd1278 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1279 NarSize: 1601280 References: 1281 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1282 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)1283 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):1284 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}1285 metadata_upload_test.go:64: Second store path (same content): /private/var/tb/8/scratch/59e19724d113dbbe/build/TestNARDeduplicationMetadataUploadBug3447237600/001/store/4m1jscg70j9v69wgljipypnj90a92ij0-file2.txt12862026/10/11 05:55:17 INFO Received push request method=POST path=/api/pushes12872026/10/11 05:55:17 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)12882026/10/11 05:55:17 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign12892026/10/11 05:55:17 WARN Failed to register uploaded object key=4m1jscg70j9v69wgljipypnj90a92ij0.ls error="server returned 404: 404 page not found\n"12902026/10/11 05:55:17 INFO Signed narinfos id=2 count=112912026/10/11 05:55:18 INFO Uploading 1 narinfos12922026/10/11 05:55:18 INFO Received complete push request method=POST path=/api/pushes/2/complete12932026/10/11 05:55:18 WARN Failed to register uploaded object key=4m1jscg70j9v69wgljipypnj90a92ij0.narinfo error="server returned 404: 404 page not found\n"12942026/10/11 05:55:18 INFO Completed push id=2 objects=3 duration=715.334µs12952026/10/11 05:55:18 INFO Upload complete. (125ms)1296 metadata_upload_test.go:76: Retrieved narinfo from S3:1297 StorePath: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestNARDeduplicationMetadataUploadBug3447237600/001/store/4m1jscg70j9v69wgljipypnj90a92ij0-file2.txt1298 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1299 Compression: zstd1300 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1301 NarSize: 1601302 References: 1303 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1304 metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes)1305 metadata_upload_test.go:77: Decompressed .ls content (49 bytes):1306 {"version":1,"root":{"type":"regular","size":44}}13072026/10/11 05:55:18 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1308=== CONT TestResolveDBConnectionString1309--- PASS: TestNARDeduplicationMetadataUploadBug (8.39s)1310=== RUN TestResolveDBConnectionString/flag_wins1311=== PAUSE TestResolveDBConnectionString/flag_wins1312=== RUN TestResolveDBConnectionString/file_when_flag_empty1313=== PAUSE TestResolveDBConnectionString/file_when_flag_empty1314=== RUN TestResolveDBConnectionString/missing_file_is_an_error1315=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error1316=== RUN TestResolveDBConnectionString/PGHOST_allows_empty1317=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty1318=== RUN TestResolveDBConnectionString/nothing_configured1319=== PAUSE TestResolveDBConnectionString/nothing_configured1320=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock13212026/10/11 05:55:18 WARN Failed to abort redundant multipart upload, keeping its row object_key=nar/gggggggggggggggggggggggggggggg0200000000000000000000.nar.zst upload_id=N2Q0OGEwM2MtOTBlZC00ZjAzLWEzMTctMjVjZTIzNGNlODg5LjBhNTViZjQyLWYwMWUtNGZmMC05NWUxLTFlMWNmNjdmMTVjOXgxNzkxNjk4MTE2MDMxMzk2MDAw error="Get \"http://127.0.0.1:1/bucket38/?location=\": dial tcp 127.0.0.1:1: connect: connection refused"13222026/10/11 05:55:18 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0200000000000000000000.nar.zst upload_id=N2Q0OGEwM2MtOTBlZC00ZjAzLWEzMTctMjVjZTIzNGNlODg5LmE0YTZiNTQ4LTA5YzQtNGZkMC04ZTc3LWM2NDUxYWUzYTUxMHgxNzkxNjk4MTE1OTY3NjkxMDAw parts=1213232026/10/11 05:55:18 INFO Received cleanup request method=DELETE path=/api/pending_closures13242026/10/11 05:55:18 INFO Aborted multipart uploads count=1 kept=01325--- PASS: TestRedundantMultipartUpload (10.68s)1326=== CONT TestDeleteClosuresRacingPins13272026/10/11 05:55:18 INFO Received uploads request method=POST path=/api/pending_closures1328--- PASS: TestPushWaitsForSweepHoldingRow (7.52s)1329=== CONT TestLeadEndsOnShutdown1330=== NAME TestOrphanedObjectsGC1331 orphaned_objects_gc_test.go:287: GC Test Summary:1332 orphaned_objects_gc_test.go:288: - Kept: 2 objects from closure A1333 orphaned_objects_gc_test.go:289: - Deleted: 2 objects from closure B1334 orphaned_objects_gc_test.go:290: - Deleted: 6 orphaned chain objects (X1->X2->X3)1335 orphaned_objects_gc_test.go:291: - Deleted: 2 orphaned single objects (Y)1336 orphaned_objects_gc_test.go:292: - Total deleted: 10 objects1337--- PASS: TestOrphanedObjectsGC (8.28s)1338=== CONT TestService_RequireScope_OIDC13392026/10/11 05:55:19 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:59041/oidc13402026/10/11 05:55:19 INFO Aborted multipart uploads count=0 kept=013412026/10/11 05:55:19 WARN Force mode enabled - objects will be deleted immediately without grace period13422026/10/11 05:55:19 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013432026/10/11 05:55:19 INFO Vacuumed table table=pending_closures13442026/10/11 05:55:19 INFO Vacuumed table table=pending_objects13452026/10/11 05:55:19 INFO Vacuumed table table=multipart_uploads13462026/10/11 05:55:19 INFO Vacuumed table table=closures13472026/10/11 05:55:19 INFO Vacuumed table table=objects1348--- PASS: TestSweepRechecksPendingAfterLocking (7.54s)1349=== CONT TestClientCADerivations13502026/10/11 05:55:20 INFO Aborted multipart uploads count=0 kept=013512026/10/11 05:55:20 WARN Force mode enabled - objects will be deleted immediately without grace period13522026/10/11 05:55:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013532026/10/11 05:55:20 INFO Vacuumed table table=pending_closures13542026/10/11 05:55:20 INFO Vacuumed table table=pending_objects13552026/10/11 05:55:20 INFO Vacuumed table table=multipart_uploads13562026/10/11 05:55:20 INFO Vacuumed table table=closures13572026/10/11 05:55:20 INFO Vacuumed table table=objects1358--- PASS: TestCommitRacingPendingCleanupKeepsObjects (7.59s)1359=== CONT TestConnectWaitsForAPeerMigration1360--- PASS: TestGetExistingObjectsLocksRows (7.29s)1361=== CONT TestLeadElectsOneAndHandsOver1362--- PASS: TestRegisterCompletedObjectNeedsPendingRow (7.29s)1363=== CONT TestLeadIncumbentAnnouncesAtOnce13642026/10/11 05:55:21 INFO Received uploads request method=POST path=/api/pending_closures13652026/10/11 05:55:21 INFO Received uploads request method=POST path=/api/pending_closures13662026/10/11 05:55:21 INFO Received cleanup request method=DELETE path=/api/pending_closures1367--- PASS: TestMetricsInventory (8.09s)1368=== CONT TestService_ReadAuthMiddleware13692026/10/11 05:55:22 INFO Received push request method=POST path=/api/pushes13702026/10/11 05:55:22 INFO Received complete push request method=POST path=/api/pushes/1/complete13712026/10/11 05:55:22 INFO Completed push id=1 objects=4 duration=200.355541ms13722026/10/11 05:55:22 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete13732026/10/11 05:55:22 INFO Completed upload id=2 objects=2 duration=1.791ms1374--- PASS: TestMetricsCommit (8.81s)1375=== CONT TestService_AuthMiddleware_OIDC13762026/10/11 05:55:22 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:59056/oidc1377--- PASS: TestObjectStatsTrigger (8.56s)1378=== CONT TestService_AuthMiddleware_MTLSBoundSubjects13792026/10/11 05:55:23 INFO Received uploads request method=POST path=/api/pending_closures1380--- PASS: TestDeduplicatedObjectsRecordedAsPending (8.46s)1381=== CONT TestCacheStatsHandler13822026/10/11 05:55:23 WARN mTLS auth: subject not in bound subjects subject="CN=reader"13832026/10/11 05:55:23 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1384--- PASS: TestService_NativeMTLS (8.88s)1385=== CONT TestPinProtectsFromGC13862026/10/11 05:55:24 INFO Aborted multipart uploads count=0 kept=013872026/10/11 05:55:24 WARN Force mode enabled - objects will be deleted immediately without grace period13882026/10/11 05:55:24 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=013892026/10/11 05:55:24 INFO Vacuumed table table=pending_closures13902026/10/11 05:55:24 INFO Vacuumed table table=pending_objects13912026/10/11 05:55:24 INFO Vacuumed table table=multipart_uploads13922026/10/11 05:55:24 INFO Vacuumed table table=closures13932026/10/11 05:55:24 INFO Vacuumed table table=objects1394--- PASS: TestGCSweepDeliversEachKeyOnce (9.70s)1395=== CONT TestClientPushesUseOnePush13962026/10/11 05:55:24 INFO Received uploads request method=POST path=/api/pending_closures1397--- PASS: TestTombstonedObjectOfferedWithoutWaiting (9.73s)1398=== CONT TestClientMultipleUploads13992026/10/11 05:55:25 INFO Received uploads request method=POST path=/api/pending_closures14002026/10/11 05:55:25 INFO Aborted multipart uploads count=0 kept=014012026/10/11 05:55:25 WARN Force mode enabled - objects will be deleted immediately without grace period14022026/10/11 05:55:25 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=014032026/10/11 05:55:25 INFO Vacuumed table table=pending_closures14042026/10/11 05:55:25 INFO Vacuumed table table=pending_objects14052026/10/11 05:55:25 INFO Vacuumed table table=multipart_uploads14062026/10/11 05:55:25 INFO Vacuumed table table=closures14072026/10/11 05:55:25 INFO Vacuumed table table=objects1408--- PASS: TestGCSweepSkipsPendingObjects (10.35s)1409=== CONT TestClientIntegration14102026/10/11 05:55:25 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=_tribuchetbld8 database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"1411--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (10.34s)1412=== CONT TestClientErrorHandling1413=== RUN TestClientErrorHandling/InvalidStorePath1414=== PAUSE TestClientErrorHandling/InvalidStorePath1415=== RUN TestClientErrorHandling/InvalidAuthToken1416=== PAUSE TestClientErrorHandling/InvalidAuthToken1417=== RUN TestClientErrorHandling/ServerNotAvailable1418=== PAUSE TestClientErrorHandling/ServerNotAvailable1419=== CONT TestClientSharedPathCommittedMidPush14202026/10/11 05:55:26 INFO Received uploads request method=POST path=/api/pending_closures14212026/10/11 05:55:26 INFO Aborted multipart uploads count=0 kept=014222026/10/11 05:55:26 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=014232026/10/11 05:55:26 INFO Vacuumed table table=pending_closures14242026/10/11 05:55:27 INFO Vacuumed table table=pending_objects14252026/10/11 05:55:27 INFO Vacuumed table table=multipart_uploads14262026/10/11 05:55:27 INFO Vacuumed table table=closures14272026/10/11 05:55:27 INFO Aborted multipart uploads count=0 kept=014282026/10/11 05:55:27 INFO Vacuumed table table=objects14292026/10/11 05:55:27 INFO Aborted multipart uploads count=0 kept=014302026/10/11 05:55:27 WARN Force mode enabled - objects will be deleted immediately without grace period14312026/10/11 05:55:27 WARN Force mode enabled - objects will be deleted immediately without grace period14322026/10/11 05:55:27 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=014332026/10/11 05:55:27 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=014342026/10/11 05:55:27 INFO Vacuumed table table=pending_closures14352026/10/11 05:55:27 INFO Vacuumed table table=pending_closures14362026/10/11 05:55:27 INFO Vacuumed table table=pending_objects14372026/10/11 05:55:27 INFO Vacuumed table table=multipart_uploads14382026/10/11 05:55:27 INFO Vacuumed table table=closures14392026/10/11 05:55:27 INFO Vacuumed table table=objects1440--- PASS: TestGCMetrics (10.50s)1441=== CONT TestCacheConfigHandler1442=== RUN TestCacheConfigHandler/full_config,_no_issuer1443=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1444=== RUN TestCacheConfigHandler/no_cache_url_configured1445=== PAUSE TestCacheConfigHandler/no_cache_url_configured1446=== RUN TestCacheConfigHandler/no_signing_keys1447=== PAUSE TestCacheConfigHandler/no_signing_keys1448=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1449=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1450=== CONT TestService_AuthMiddleware_MTLSProxyHeader14512026/10/11 05:55:27 INFO Vacuumed table table=pending_objects14522026/10/11 05:55:27 INFO Vacuumed table table=multipart_uploads14532026/10/11 05:55:27 INFO Vacuumed table table=closures14542026/10/11 05:55:27 INFO Vacuumed table table=objects1455--- PASS: TestPushDedupSurvivesConcurrentGC (11.26s)1456=== CONT TestService_ReadScope_PublicByDefault14572026/10/11 05:55:27 INFO Aborted multipart uploads count=1 kept=014582026/10/11 05:55:27 INFO Received cleanup request method=DELETE path=/api/pending_closures14592026/10/11 05:55:27 INFO Aborted multipart uploads count=1 kept=01460--- PASS: TestPendingCleanupUsesOneCutoff (13.94s)1461=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection1462--- PASS: TestGCBugBareHashReferences (10.76s)1463=== CONT TestClientWithDependencies14642026/10/11 05:55:28 INFO Received uploads request method=POST path=/api/pending_closures14652026/10/11 05:55:28 INFO Received uploads request method=POST path=/api/pending_closures14662026/10/11 05:55:28 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)14672026/10/11 05:55:28 INFO Uploading xrbjl67ddjyqbry1sxzkfijdd499psm7-shared-dep (136B)14682026/10/11 05:55:28 INFO Uploading x0zyy4aaxyvj4282pfd1d4jmqgdncawc-a (256B)14692026/10/11 05:55:28 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"14702026/10/11 05:55:28 WARN Failed to register uploaded object key=nar/166gjqlf4581824bkfdj1plblsl2wpnaz98sklzx09wvpz4sqkbx.nar.zst error="server returned 404: 404 page not found\n"1471=== NAME TestOrphanedObjectsGCStressTest1472 orphaned_objects_gc_test.go:422: Created 10 active closures, 5 to-delete closures, 20 orphaned chains14732026/10/11 05:55:28 WARN Failed to register uploaded object key=x0zyy4aaxyvj4282pfd1d4jmqgdncawc.ls error="server returned 404: 404 page not found\n"14742026/10/11 05:55:28 WARN Failed to register uploaded object key=xrbjl67ddjyqbry1sxzkfijdd499psm7.ls error="server returned 404: 404 page not found\n"14752026/10/11 05:55:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign14762026/10/11 05:55:28 WARN Failed to register uploaded object key=kq3i20qbn6whkqpwn2fyn2nh9awq7w59.ls error="server returned 404: 404 page not found\n"14772026/10/11 05:55:28 INFO Signed narinfos id=1 count=214782026/10/11 05:55:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign14792026/10/11 05:55:28 INFO Signed narinfos id=2 count=214802026/10/11 05:55:28 INFO Uploading 4 narinfos14812026/10/11 05:55:28 WARN Failed to register uploaded object key=x0zyy4aaxyvj4282pfd1d4jmqgdncawc.narinfo error="server returned 404: 404 page not found\n"14822026/10/11 05:55:28 WARN Failed to register uploaded object key=xrbjl67ddjyqbry1sxzkfijdd499psm7.narinfo error="server returned 404: 404 page not found\n"14832026/10/11 05:55:28 WARN Failed to register uploaded object key=kq3i20qbn6whkqpwn2fyn2nh9awq7w59.narinfo error="server returned 404: 404 page not found\n"1484 orphaned_objects_gc_test.go:440: Marked 210 objects for deletion14852026/10/11 05:55:29 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete14862026/10/11 05:55:29 WARN Failed to register uploaded object key=xrbjl67ddjyqbry1sxzkfijdd499psm7.narinfo error="server returned 404: 404 page not found\n"14872026/10/11 05:55:29 INFO Completed upload id=1 objects=6 duration=48.597ms14882026/10/11 05:55:29 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete14892026/10/11 05:55:29 INFO Completed upload id=2 objects=6 duration=2.9285ms14902026/10/11 05:55:29 INFO Upload complete. (515ms)1491=== NAME TestClientFallsBackToClosures1492 client_pushes_test.go:112: Retrieved narinfo from S3:1493 StorePath: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientFallsBackToClosures3861625839/001/store/xrbjl67ddjyqbry1sxzkfijdd499psm7-shared-dep1494 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1495 Compression: zstd1496 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821497 NarSize: 1361498 References: 1499 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1500 client_pushes_test.go:112: Retrieved narinfo from S3:1501 StorePath: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientFallsBackToClosures3861625839/001/store/x0zyy4aaxyvj4282pfd1d4jmqgdncawc-a1502 URL: nar/166gjqlf4581824bkfdj1plblsl2wpnaz98sklzx09wvpz4sqkbx.nar.zst1503 Compression: zstd1504 NarHash: sha256:166gjqlf4581824bkfdj1plblsl2wpnaz98sklzx09wvpz4sqkbx1505 NarSize: 2561506 References: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientFallsBackToClosures3861625839/001/store/xrbjl67ddjyqbry1sxzkfijdd499psm7-shared-dep1507 CA: text:sha256:1lqh3sal5q3l7srsbgbbwrxjl09kq0r93i4jr1x4n3cqd1a9qs9b1508 client_pushes_test.go:112: Retrieved narinfo from S3:1509 StorePath: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientFallsBackToClosures3861625839/001/store/kq3i20qbn6whkqpwn2fyn2nh9awq7w59-b1510 URL: nar/166gjqlf4581824bkfdj1plblsl2wpnaz98sklzx09wvpz4sqkbx.nar.zst1511 Compression: zstd1512 NarHash: sha256:166gjqlf4581824bkfdj1plblsl2wpnaz98sklzx09wvpz4sqkbx1513 NarSize: 2561514 References: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientFallsBackToClosures3861625839/001/store/xrbjl67ddjyqbry1sxzkfijdd499psm7-shared-dep1515 CA: text:sha256:1lqh3sal5q3l7srsbgbbwrxjl09kq0r93i4jr1x4n3cqd1a9qs9b1516--- PASS: TestClientFallsBackToClosures (12.42s)1517=== CONT TestPresentNotReportedWhileGCDeletesClosure/stale15182026/10/11 05:55:29 INFO lead: acquired remote=192.0.2.1:12341519=== RUN TestService_RequireScope_OIDC/builder_may_write1520=== PAUSE TestService_RequireScope_OIDC/builder_may_write1521=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1522=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1523=== RUN TestService_RequireScope_OIDC/ops_may_admin1524=== PAUSE TestService_RequireScope_OIDC/ops_may_admin1525=== RUN TestService_RequireScope_OIDC/ops_may_not_write1526=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1527=== RUN TestService_RequireScope_OIDC/reader_may_not_write1528=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1529=== RUN TestService_RequireScope_OIDC/static_token_may_admin1530=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1531=== RUN TestService_RequireScope_OIDC/static_token_may_write1532=== PAUSE TestService_RequireScope_OIDC/static_token_may_write1533=== RUN TestService_RequireScope_OIDC/reader_may_read1534=== PAUSE TestService_RequireScope_OIDC/reader_may_read1535=== RUN TestService_RequireScope_OIDC/writer_implies_read1536=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1537=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1538=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read1539=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc1540=== PAUSE TestService_RequireScope_OIDC/reader_may_read_with_netrc1541=== RUN TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1542=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1543=== RUN TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1544=== PAUSE TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1545=== RUN TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1546=== PAUSE TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1547=== CONT TestPresentNotReportedWhileGCDeletesClosure/fresh15482026/10/11 05:55:30 INFO lead: released remote=192.0.2.1:12341549--- PASS: TestLeadEndsOnShutdown (11.45s)1550=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info15512026/10/11 05:55:30 INFO Received uploads request method=POST path=/1552=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key15532026/10/11 05:55:30 INFO Received request for more parts method=POST path=/1554=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key15552026/10/11 05:55:30 INFO Received complete multipart upload request method=POST path=/1556=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal15572026/10/11 05:55:30 INFO Received uploads request method=POST path=/1558--- PASS: TestUploadHandlersRejectInvalidKeys (0.01s)1559 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)1560 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1561 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1562 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s)1563=== CONT TestPendingClosureWriteTimeout/empty1564=== CONT TestPendingClosureWriteTimeout/670k_objects1565=== CONT TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1566--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (12.60s)1567=== CONT TestPendingClosureWriteTimeout/400_objects1568=== CONT TestProxyWriteTimeout/narinfo1569=== CONT TestIsValidCachePath/narinfo1570=== CONT TestProxyWriteTimeout/10_GiB_nar1571=== CONT TestProxyWriteTimeout/unknown_size1572=== CONT TestIsValidCachePath/traversal_parent1573=== CONT TestIsValidCachePath/short_hash1574=== CONT TestIsValidCachePath/wrong_extension1575=== CONT TestIsValidCachePath/leading_slash1576=== CONT TestIsValidCachePath/empty1577=== CONT TestIsValidCachePath/random_path1578=== CONT TestIsValidCachePath/invalid_char_u1579=== CONT TestIsValidCachePath/invalid_char_e1580=== CONT TestIsValidCachePath/traversal_in_middle1581=== CONT TestIsValidCachePath/ls1582=== CONT TestIsValidCachePath/index.html1583=== CONT TestIsValidCachePath/nix-cache-info1584=== CONT TestIsValidCachePath/realisation1585=== CONT TestIsValidCachePath/log1586=== CONT TestProxyWriteTimeout/1_GiB_nar1587=== CONT TestIsValidCachePath/nar_xz1588=== CONT TestIsValidCachePath/nar_uncompressed1589=== CONT TestIsValidCachePath/nar_bz21590=== CONT TestIsValidCachePath/nar_zst1591=== CONT TestIsValidCachePath/narinfo_all_nix_base32_chars1592--- PASS: TestIsValidCachePath (0.05s)1593 --- PASS: TestIsValidCachePath/narinfo (0.00s)1594 --- PASS: TestIsValidCachePath/traversal_parent (0.00s)1595 --- PASS: TestIsValidCachePath/short_hash (0.00s)1596 --- PASS: TestIsValidCachePath/wrong_extension (0.00s)1597 --- PASS: TestIsValidCachePath/leading_slash (0.00s)1598 --- PASS: TestIsValidCachePath/empty (0.00s)1599 --- PASS: TestIsValidCachePath/random_path (0.00s)1600 --- PASS: TestIsValidCachePath/invalid_char_u (0.00s)1601 --- PASS: TestIsValidCachePath/invalid_char_e (0.00s)1602 --- PASS: TestIsValidCachePath/traversal_in_middle (0.00s)1603 --- PASS: TestIsValidCachePath/ls (0.00s)1604 --- PASS: TestIsValidCachePath/index.html (0.00s)1605 --- PASS: TestIsValidCachePath/nix-cache-info (0.00s)1606 --- PASS: TestIsValidCachePath/realisation (0.00s)1607 --- PASS: TestIsValidCachePath/log (0.00s)1608 --- PASS: TestIsValidCachePath/nar_xz (0.00s)1609 --- PASS: TestIsValidCachePath/nar_uncompressed (0.00s)1610 --- PASS: TestIsValidCachePath/nar_bz2 (0.00s)1611 --- PASS: TestIsValidCachePath/nar_zst (0.00s)1612 --- PASS: TestIsValidCachePath/narinfo_all_nix_base32_chars (0.00s)1613--- PASS: TestProxyWriteTimeout (0.05s)1614 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1615 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1616 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1617 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)1618=== CONT TestIsValidUploadKey/narinfo1619=== CONT TestIsValidUploadKey/realisation_plus_in_output1620=== CONT TestIsValidUploadKey/realisation1621=== CONT TestIsValidUploadKey/build_log_equals1622=== CONT TestIsValidUploadKey/build_log_question_mark1623=== CONT TestIsValidUploadKey/build_log_plus_in_name1624=== CONT TestIsValidUploadKey/build_log_home-manager_file1625=== CONT TestIsValidUploadKey/build_log1626=== CONT TestIsValidUploadKey/listing1627=== CONT TestIsValidUploadKey/nar_plain1628=== CONT TestIsValidUploadKey/nar_xz1629=== CONT TestIsValidUploadKey/nar_zst1630=== CONT TestIsValidUploadKey/traversal1631=== CONT TestIsValidUploadKey/unknown_type1632=== CONT TestIsValidUploadKey/empty_key1633=== CONT TestIsValidUploadKey/absolute1634=== CONT TestIsValidUploadKey/traversal_nar1635=== CONT TestIsValidUploadKey/narinfo_key,_nar_type1636=== CONT TestIsValidUploadKey/listing_key,_narinfo_type1637=== CONT TestIsValidUploadKey/nar_key,_narinfo_type1638=== CONT TestParseSingleRange/none1639=== CONT TestIsValidUploadKey/index.html1640=== CONT TestIsValidUploadKey/nix-cache-info1641--- PASS: TestIsValidUploadKey (0.05s)1642 --- PASS: TestIsValidUploadKey/narinfo (0.00s)1643 --- PASS: TestIsValidUploadKey/realisation_plus_in_output (0.00s)1644 --- PASS: TestIsValidUploadKey/realisation (0.00s)1645 --- PASS: TestIsValidUploadKey/build_log_equals (0.00s)1646 --- PASS: TestIsValidUploadKey/build_log_question_mark (0.00s)1647 --- PASS: TestIsValidUploadKey/build_log_plus_in_name (0.00s)1648 --- PASS: TestIsValidUploadKey/build_log_home-manager_file (0.00s)1649 --- PASS: TestIsValidUploadKey/build_log (0.00s)1650 --- PASS: TestIsValidUploadKey/listing (0.00s)1651 --- PASS: TestIsValidUploadKey/nar_plain (0.00s)1652 --- PASS: TestIsValidUploadKey/nar_xz (0.00s)1653 --- PASS: TestIsValidUploadKey/nar_zst (0.00s)1654 --- PASS: TestIsValidUploadKey/traversal (0.00s)1655 --- PASS: TestIsValidUploadKey/unknown_type (0.00s)1656 --- PASS: TestIsValidUploadKey/empty_key (0.00s)1657 --- PASS: TestIsValidUploadKey/absolute (0.00s)1658 --- PASS: TestIsValidUploadKey/traversal_nar (0.00s)1659 --- PASS: TestIsValidUploadKey/narinfo_key,_nar_type (0.00s)1660 --- PASS: TestIsValidUploadKey/listing_key,_narinfo_type (0.00s)1661 --- PASS: TestIsValidUploadKey/nar_key,_narinfo_type (0.00s)1662 --- PASS: TestIsValidUploadKey/index.html (0.00s)1663 --- PASS: TestIsValidUploadKey/nix-cache-info (0.00s)1664=== CONT TestParseSingleRange/open-ended1665=== CONT TestParseSingleRange/start_far_past_EOF1666=== CONT TestParseSingleRange/start_past_EOF1667=== CONT TestParseSingleRange/single_byte1668=== CONT TestParseSingleRange/suffix_exceeds_size1669=== CONT TestParseSingleRange/suffix1670=== CONT TestParseSingleRange/end_clamped_to_size1671=== CONT TestParseSingleRange/malformed_both_empty1672=== CONT TestParseSingleRange/closed1673=== CONT TestParseSingleRange/malformed_end_before_start1674=== CONT TestParseSingleRange/multi-range_ignored1675=== CONT TestParseSingleRange/malformed_no_dash1676=== CONT TestParseSingleRange/unknown_unit1677--- PASS: TestParseSingleRange (0.05s)1678 --- PASS: TestParseSingleRange/none (0.00s)1679 --- PASS: TestParseSingleRange/open-ended (0.00s)1680 --- PASS: TestParseSingleRange/start_far_past_EOF (0.00s)1681 --- PASS: TestParseSingleRange/start_past_EOF (0.00s)1682 --- PASS: TestParseSingleRange/single_byte (0.00s)1683 --- PASS: TestParseSingleRange/suffix_exceeds_size (0.00s)1684 --- PASS: TestParseSingleRange/suffix (0.00s)1685 --- PASS: TestParseSingleRange/end_clamped_to_size (0.00s)1686 --- PASS: TestParseSingleRange/malformed_both_empty (0.00s)1687 --- PASS: TestParseSingleRange/closed (0.00s)1688 --- PASS: TestParseSingleRange/malformed_end_before_start (0.00s)1689 --- PASS: TestParseSingleRange/multi-range_ignored (0.00s)1690 --- PASS: TestParseSingleRange/malformed_no_dash (0.00s)1691 --- PASS: TestParseSingleRange/unknown_unit (0.00s)1692=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure16932026/10/11 05:55:30 INFO Received uploads request method=POST path=/1694--- PASS: TestConnectWaitsForAPeerMigration (10.75s)1695=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts16962026/10/11 05:55:31 INFO Received request for more parts method=POST path=/1697=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart16982026/10/11 05:55:31 INFO Received complete multipart upload request method=POST path=/1699=== CONT TestForceGCDuringPushOffersSweptObject/before_pending_rows1700--- PASS: TestUploadHandlersRejectOversizedBody (0.08s)1701 --- PASS: TestUploadHandlersRejectOversizedBody/request_more_parts (0.03s)1702 --- PASS: TestUploadHandlersRejectOversizedBody/complete_multipart (0.02s)1703 --- PASS: TestUploadHandlersRejectOversizedBody/create_pending_closure (0.31s)1704=== CONT TestForceGCDuringPushOffersSweptObject/after_presence_check17052026/10/11 05:55:31 INFO lead: acquired remote=192.0.2.1:12341706=== NAME TestClientCADerivations1707 client_ca_test.go:136: Built CA derivation: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientCADerivations3670593593/001/store/mnf7f2l6ny1b7lkp8qzg57x72zqkx70q-ca-test17082026/10/11 05:55:31 INFO lead: acquired remote=192.0.2.1:123417092026/10/11 05:55:31 INFO lead: released remote=192.0.2.1:12341710--- PASS: TestLeadIncumbentAnnouncesAtOnce (11.02s)1711=== CONT TestGCEndsOnShutdown/before_the_run1712=== NAME TestClientCADerivations1713 client_ca_test.go:139: Found 1 dependencies (including self)17142026/10/11 05:55:31 INFO Received push request method=POST path=/api/pushes1715--- PASS: TestObjectStatsLargeCommit (20.13s)1716=== CONT TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark17172026/10/11 05:55:32 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)17182026/10/11 05:55:32 INFO Uploading mnf7f2l6ny1b7lkp8qzg57x72zqkx70q-ca-test (144B)17192026/10/11 05:55:32 WARN Failed to register uploaded object key=mnf7f2l6ny1b7lkp8qzg57x72zqkx70q.ls error="server returned 404: 404 page not found\n"17202026/10/11 05:55:32 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"1721--- PASS: TestService_ReadAuthMiddleware (10.28s)1722=== CONT TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete17232026/10/11 05:55:32 WARN Failed to register uploaded object key=log/1sfa4lk0i9rga03crxiizs0kmf57fd7i-ca-test.drv error="server returned 404: 404 page not found\n"17242026/10/11 05:55:32 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign17252026/10/11 05:55:32 INFO Signed narinfos id=1 count=117262026/10/11 05:55:32 INFO Uploading 1 narinfos17272026/10/11 05:55:32 INFO Received complete push request method=POST path=/api/pushes/1/complete17282026/10/11 05:55:32 WARN Failed to register uploaded object key=mnf7f2l6ny1b7lkp8qzg57x72zqkx70q.narinfo error="server returned 404: 404 page not found\n"17292026/10/11 05:55:32 INFO Completed push id=1 objects=4 duration=58.112834ms17302026/10/11 05:55:32 INFO Upload complete. (450ms)1731=== NAME TestClientCADerivations1732 client_ca_test.go:180: Narinfo contains CA field: StorePath: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientCADerivations3670593593/001/store/mnf7f2l6ny1b7lkp8qzg57x72zqkx70q-ca-test1733 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1734 Compression: zstd1735 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1736 NarSize: 1441737 References: 1738 Deriver: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientCADerivations3670593593/001/store/1sfa4lk0i9rga03crxiizs0kmf57fd7i-ca-test.drv1739 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1740 client_ca_test.go:185: Checking for realisation files in S3...1741 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1742 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache17432026/10/11 05:55:32 INFO lead: released remote=192.0.2.1:123417442026/10/11 05:55:32 INFO lead: acquired remote=192.0.2.1:12341745 client_ca_test.go:258: nix copy output: error: binary cache 's3://bucket75?endpoint=http://localhost:58759®ion=eu-west-1' is for Nix stores with prefix '/nix/store', not '/private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientCADerivations3670593593/001/store'1746 client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 11747=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token1748=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token1749=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1750=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1751=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1752=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1753=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1754=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1755=== CONT TestServerTLSConfig/no_client_CA1756=== CONT TestServerTLSConfig/not_a_PEM_file1757=== CONT TestServerTLSConfig/missing_CA_file1758--- PASS: TestServerTLSConfig (0.00s)1759 --- PASS: TestServerTLSConfig/no_client_CA (0.00s)1760 --- PASS: TestServerTLSConfig/not_a_PEM_file (0.05s)1761 --- PASS: TestServerTLSConfig/missing_CA_file (0.00s)1762=== CONT TestPush_RejectsBadRequests/no_objects17632026/10/11 05:55:32 INFO Received push request method=POST path=/api/pushes1764=== CONT TestPush_RejectsBadRequests/root_not_in_objects17652026/10/11 05:55:32 INFO Received push request method=POST path=/api/pushes1766=== CONT TestPush_RejectsBadRequests/no_roots17672026/10/11 05:55:32 INFO Received push request method=POST path=/api/pushes1768=== CONT TestPush_RejectsBadRequests/bad_root17692026/10/11 05:55:32 INFO Received push request method=POST path=/api/pushes1770--- PASS: TestPush_RejectsBadRequests (6.55s)1771 --- PASS: TestPush_RejectsBadRequests/no_objects (0.00s)1772 --- PASS: TestPush_RejectsBadRequests/root_not_in_objects (0.00s)1773 --- PASS: TestPush_RejectsBadRequests/no_roots (0.00s)1774 --- PASS: TestPush_RejectsBadRequests/bad_root (0.00s)1775=== CONT TestResolveDBConnectionString/flag_wins1776=== CONT TestResolveDBConnectionString/PGHOST_allows_empty1777=== CONT TestResolveDBConnectionString/nothing_configured1778=== CONT TestResolveDBConnectionString/missing_file_is_an_error1779=== CONT TestResolveDBConnectionString/file_when_flag_empty1780=== CONT TestClientErrorHandling/InvalidStorePath1781--- PASS: TestResolveDBConnectionString (0.05s)1782 --- PASS: TestResolveDBConnectionString/flag_wins (0.00s)1783 --- PASS: TestResolveDBConnectionString/PGHOST_allows_empty (0.00s)1784 --- PASS: TestResolveDBConnectionString/nothing_configured (0.00s)1785 --- PASS: TestResolveDBConnectionString/missing_file_is_an_error (0.00s)1786 --- PASS: TestResolveDBConnectionString/file_when_flag_empty (0.00s)1787--- PASS: TestClientCADerivations (13.13s)1788=== CONT TestClientErrorHandling/ServerNotAvailable17892026/10/11 05:55:33 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"17902026/10/11 05:55:33 WARN mTLS auth: bound subjects configured but subject DN unavailable17912026/10/11 05:55:33 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1792--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (10.11s)1793=== CONT TestClientErrorHandling/InvalidAuthToken17942026/10/11 05:55:33 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present17952026/10/11 05:55:33 INFO lead: released remote=192.0.2.1:12341796--- PASS: TestLeadElectsOneAndHandsOver (13.28s)1797=== CONT TestCacheConfigHandler/full_config,_no_issuer1798=== CONT TestCacheConfigHandler/no_signing_keys1799=== CONT TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1800=== CONT TestCacheConfigHandler/no_cache_url_configured1801--- PASS: TestCacheConfigHandler (0.00s)1802 --- PASS: TestCacheConfigHandler/full_config,_no_issuer (0.00s)1803 --- PASS: TestCacheConfigHandler/no_signing_keys (0.00s)1804 --- PASS: TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator (0.00s)1805 --- PASS: TestCacheConfigHandler/no_cache_url_configured (0.00s)1806=== CONT TestService_RequireScope_OIDC/builder_may_write1807=== CONT TestService_RequireScope_OIDC/reader_may_read1808=== CONT TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1809=== CONT TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read18102026/10/11 05:55:33 WARN Authentication failed token_preview=wrong token_length=5 oidc_error="no provider could verify the token (signature or issuer mismatch)" oidc_provider="" tried_providers=[test]1811=== CONT TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1812=== CONT TestService_RequireScope_OIDC/reader_may_read_with_netrc1813=== CONT TestService_RequireScope_OIDC/anonymous_may_not_read1814=== CONT TestService_RequireScope_OIDC/writer_implies_read1815=== CONT TestService_RequireScope_OIDC/reader_may_not_write1816=== CONT TestService_RequireScope_OIDC/static_token_may_write1817=== CONT TestService_RequireScope_OIDC/static_token_may_admin1818=== CONT TestService_RequireScope_OIDC/ops_may_admin1819=== CONT TestService_RequireScope_OIDC/ops_may_not_write1820=== CONT TestService_RequireScope_OIDC/builder_may_not_admin1821=== CONT TestService_AuthMiddleware_OIDC/valid_OIDC_token1822=== CONT TestService_AuthMiddleware_OIDC/malformed_token_rejected18232026/10/11 05:55:33 WARN Authentication failed token_preview=not-a-valid-jwt token_length=15 oidc_error="no provider could verify the token (signature or issuer mismatch)" oidc_provider="" tried_providers=[test]1824=== CONT TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured18252026/10/11 05:55:33 WARN Request failed, retrying attempt=2 backoff=191.010369ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present1826=== CONT TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1827--- PASS: TestService_RequireScope_OIDC (10.90s)1828 --- PASS: TestService_RequireScope_OIDC/builder_may_write (0.00s)1829 --- PASS: TestService_RequireScope_OIDC/reader_may_read (0.00s)1830 --- PASS: TestService_RequireScope_OIDC/empty_netrc_password_may_not_read (0.00s)1831 --- PASS: TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read (0.00s)1832 --- PASS: TestService_RequireScope_OIDC/static_token_may_admin_with_netrc (0.00s)1833 --- PASS: TestService_RequireScope_OIDC/reader_may_read_with_netrc (0.00s)1834 --- PASS: TestService_RequireScope_OIDC/anonymous_may_not_read (0.00s)1835 --- PASS: TestService_RequireScope_OIDC/writer_implies_read (0.00s)1836 --- PASS: TestService_RequireScope_OIDC/reader_may_not_write (0.00s)1837 --- PASS: TestService_RequireScope_OIDC/static_token_may_write (0.00s)1838 --- PASS: TestService_RequireScope_OIDC/static_token_may_admin (0.00s)1839 --- PASS: TestService_RequireScope_OIDC/ops_may_admin (0.00s)1840 --- PASS: TestService_RequireScope_OIDC/ops_may_not_write (0.00s)1841 --- PASS: TestService_RequireScope_OIDC/builder_may_not_admin (0.00s)18422026/10/11 05:55:33 WARN Authentication failed token_preview=eyJhbGciOi...HeDnVeDa_A token_length=701 oidc_error="no rule matched: claim \"repository_owner\" value [otherorg] not in allowed patterns [myorg]" oidc_provider=test tried_providers=[test]1843--- PASS: TestService_AuthMiddleware_OIDC (9.77s)1844 --- PASS: TestService_AuthMiddleware_OIDC/valid_OIDC_token (0.00s)1845 --- PASS: TestService_AuthMiddleware_OIDC/malformed_token_rejected (0.00s)1846 --- PASS: TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured (0.00s)1847 --- PASS: TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected (0.01s)1848--- PASS: TestCacheStatsHandler (10.25s)18492026/10/11 05:55:33 WARN Request failed, retrying attempt=3 backoff=422.476635ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present18502026/10/11 05:55:34 WARN Request failed, retrying attempt=4 backoff=829.780959ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present18512026/10/11 05:55:35 WARN Request failed, retrying attempt=5 backoff=1.567699007s error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present1852=== NAME TestPinProtectsFromGC1853 client_integration_test.go:731: Pinned store path: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestPinProtectsFromGC2188448925/001/store/04wck7jprc6msb0gslgizlb3kgg2gqs5-pinned-file.txt1854 client_integration_test.go:732: Unpinned store path: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestPinProtectsFromGC2188448925/001/store/7jrc9yfkxk3jrsdnffhyb9qhi31p3qiy-unpinned-file.txt1855=== NAME TestClientMultipleUploads1856 client_integration_test.go:358: Created store path 0: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientMultipleUploads731803428/001/store/6h488wczjcq8q7rx9mvhq64p5yd4qy25-test-file-0.txt18572026/10/11 05:55:35 INFO Received push request method=POST path=/api/pushes18582026/10/11 05:55:35 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)18592026/10/11 05:55:35 INFO Uploading 04wck7jprc6msb0gslgizlb3kgg2gqs5-pinned-file.txt (128B)18602026/10/11 05:55:35 WARN Failed to register uploaded object key=04wck7jprc6msb0gslgizlb3kgg2gqs5.ls error="server returned 404: 404 page not found\n"18612026/10/11 05:55:35 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign18622026/10/11 05:55:35 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"18632026/10/11 05:55:35 INFO Signed narinfos id=1 count=118642026/10/11 05:55:35 INFO Uploading 1 narinfos18652026/10/11 05:55:35 INFO Received complete push request method=POST path=/api/pushes/1/complete1866 client_integration_test.go:358: Created store path 1: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientMultipleUploads731803428/001/store/ghn2s550givas6ffkiwccvj6krir2pv4-test-file-1.txt18672026/10/11 05:55:35 WARN Failed to register uploaded object key=04wck7jprc6msb0gslgizlb3kgg2gqs5.narinfo error="server returned 404: 404 page not found\n"18682026/10/11 05:55:35 INFO Completed push id=1 objects=3 duration=181.487542ms18692026/10/11 05:55:35 INFO Upload complete. (605ms)1870 client_integration_test.go:358: Created store path 2: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientMultipleUploads731803428/001/store/3kczidkrxkc0hp6l027c8vkxsvmx7sgv-test-file-2.txt18712026/10/11 05:55:36 INFO Received push request method=POST path=/api/pushes18722026/10/11 05:55:36 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)18732026/10/11 05:55:36 INFO Uploading 7jrc9yfkxk3jrsdnffhyb9qhi31p3qiy-unpinned-file.txt (128B)18742026/10/11 05:55:36 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"18752026/10/11 05:55:36 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign18762026/10/11 05:55:36 WARN Failed to register uploaded object key=7jrc9yfkxk3jrsdnffhyb9qhi31p3qiy.ls error="server returned 404: 404 page not found\n"18772026/10/11 05:55:36 INFO Signed narinfos id=2 count=118782026/10/11 05:55:36 INFO Uploading 1 narinfos18792026/10/11 05:55:36 INFO Received push request method=POST path=/api/pushes18802026/10/11 05:55:36 INFO Received complete push request method=POST path=/api/pushes/2/complete18812026/10/11 05:55:36 WARN Failed to register uploaded object key=7jrc9yfkxk3jrsdnffhyb9qhi31p3qiy.narinfo error="server returned 404: 404 page not found\n"18822026/10/11 05:55:36 INFO Completed push id=2 objects=3 duration=1.040334ms18832026/10/11 05:55:36 INFO Upload complete. (273ms)18842026/10/11 05:55:36 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)18852026/10/11 05:55:36 INFO Uploading ghn2s550givas6ffkiwccvj6krir2pv4-test-file-1.txt (160B)18862026/10/11 05:55:36 INFO Uploading 3kczidkrxkc0hp6l027c8vkxsvmx7sgv-test-file-2.txt (160B)18872026/10/11 05:55:36 INFO Uploading 6h488wczjcq8q7rx9mvhq64p5yd4qy25-test-file-0.txt (160B)18882026/10/11 05:55:36 INFO Starting cleanup of old closures method=DELETE path=/api/closures18892026/10/11 05:55:36 INFO Garbage collection started18902026/10/11 05:55:36 INFO Aborted multipart uploads count=0 kept=018912026/10/11 05:55:36 WARN Failed to register uploaded object key=ghn2s550givas6ffkiwccvj6krir2pv4.ls error="server returned 404: 404 page not found\n"18922026/10/11 05:55:36 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"18932026/10/11 05:55:36 WARN Failed to register uploaded object key=6h488wczjcq8q7rx9mvhq64p5yd4qy25.ls error="server returned 404: 404 page not found\n"18942026/10/11 05:55:36 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"18952026/10/11 05:55:36 WARN Force mode enabled - objects will be deleted immediately without grace period18962026/10/11 05:55:36 WARN Failed to register uploaded object key=3kczidkrxkc0hp6l027c8vkxsvmx7sgv.ls error="server returned 404: 404 page not found\n"18972026/10/11 05:55:36 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign18982026/10/11 05:55:36 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"18992026/10/11 05:55:36 INFO Signed narinfos id=1 count=319002026/10/11 05:55:36 INFO Uploading 3 narinfos19012026/10/11 05:55:36 INFO Received complete push request method=POST path=/api/pushes/1/complete19022026/10/11 05:55:36 WARN Failed to register uploaded object key=ghn2s550givas6ffkiwccvj6krir2pv4.narinfo error="server returned 404: 404 page not found\n"19032026/10/11 05:55:36 WARN Failed to register uploaded object key=6h488wczjcq8q7rx9mvhq64p5yd4qy25.narinfo error="server returned 404: 404 page not found\n"19042026/10/11 05:55:36 WARN Failed to register uploaded object key=3kczidkrxkc0hp6l027c8vkxsvmx7sgv.narinfo error="server returned 404: 404 page not found\n"19052026/10/11 05:55:36 INFO Completed push id=1 objects=9 duration=81.960625ms19062026/10/11 05:55:36 INFO Upload complete. (502ms)1907 client_integration_test.go:369: Uploaded 3 paths in 551.392125ms19082026/10/11 05:55:36 WARN Request failed, retrying attempt=1 backoff=100ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config19092026/10/11 05:55:36 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=019102026/10/11 05:55:36 WARN Request failed, retrying attempt=2 backoff=195.270199ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config19112026-10-11 05:55:36.865 UTC [90585] LOG: checkpoint starting: wal1912--- PASS: TestClientMultipleUploads (11.84s)19132026/10/11 05:55:36 INFO Received push request method=POST path=/api/pushes19142026/10/11 05:55:37 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)19152026/10/11 05:55:37 INFO Uploading 9s2hgfn6iljfgn1p0wnjddpfkpdfzyf4-b (256B)19162026/10/11 05:55:37 WARN Request failed, retrying attempt=3 backoff=403.063086ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config19172026/10/11 05:55:37 INFO Uploading jhikky26nb10i9994wkbbab92kgdm8ky-shared-dep (136B)19182026/10/11 05:55:37 INFO Vacuumed table table=pending_closures19192026/10/11 05:55:37 INFO Received push request method=POST path=/api/pushes19202026/10/11 05:55:37 WARN Failed to register uploaded object key=nar/1k254lsgrpwzl67lyf1r8mkyny8r941zsy4ik15asd0g53d9s1pd.nar.zst error="server returned 404: 404 page not found\n"19212026/10/11 05:55:37 WARN Failed to register uploaded object key=jhikky26nb10i9994wkbbab92kgdm8ky.ls error="server returned 404: 404 page not found\n"19222026/10/11 05:55:37 WARN Failed to register uploaded object key=9s2hgfn6iljfgn1p0wnjddpfkpdfzyf4.ls error="server returned 404: 404 page not found\n"19232026/10/11 05:55:37 WARN Failed to register uploaded object key=5npm4ymc3jkjdswckf319wrfgb56rp7n.ls error="server returned 404: 404 page not found\n"19242026/10/11 05:55:37 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"19252026/10/11 05:55:37 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign19262026/10/11 05:55:37 INFO Signed narinfos id=1 count=31927--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (9.96s)19282026/10/11 05:55:37 INFO Uploading 3 narinfos19292026/10/11 05:55:37 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)19302026/10/11 05:55:37 INFO Uploading cwdlb7bp9qp8mkx2sg685c15prpp82md-top (264B)19312026/10/11 05:55:37 INFO Uploading 7fnj0v26n9xaj1f2mpsxlrkcflfann27-shared-dep (136B)19322026/10/11 05:55:37 INFO Vacuumed table table=pending_objects19332026/10/11 05:55:37 INFO Vacuumed table table=multipart_uploads19342026/10/11 05:55:37 WARN Failed to register uploaded object key=5npm4ymc3jkjdswckf319wrfgb56rp7n.narinfo error="server returned 404: 404 page not found\n"19352026/10/11 05:55:37 WARN Failed to register uploaded object key=9s2hgfn6iljfgn1p0wnjddpfkpdfzyf4.narinfo error="server returned 404: 404 page not found\n"19362026/10/11 05:55:37 WARN Failed to register uploaded object key=7fnj0v26n9xaj1f2mpsxlrkcflfann27.ls error="server returned 404: 404 page not found\n"1937=== NAME TestClientIntegration19382026/10/11 05:55:37 WARN Failed to register uploaded object key=cwdlb7bp9qp8mkx2sg685c15prpp82md.ls error="server returned 404: 404 page not found\n"19392026/10/11 05:55:37 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"19402026/10/11 05:55:37 INFO Received complete push request method=POST path=/api/pushes/1/complete1941 client_integration_test.go:286: Created store path: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientIntegration1447511405/002/store/xj4dxkrfincwd739k1x1y6g89jbbpds5-test-file.txt19422026/10/11 05:55:37 WARN Failed to register uploaded object key=jhikky26nb10i9994wkbbab92kgdm8ky.narinfo error="server returned 404: 404 page not found\n"19432026/10/11 05:55:37 INFO Vacuumed table table=closures19442026/10/11 05:55:37 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign19452026/10/11 05:55:37 WARN Failed to register uploaded object key=nar/0xnzy5g6c95n4qb4d7krygx5y22776mjp23mixsd70ydrfjhgf12.nar.zst error="server returned 404: 404 page not found\n"19462026/10/11 05:55:37 INFO Signed narinfos id=1 count=219472026/10/11 05:55:37 INFO Uploading 2 narinfos19482026/10/11 05:55:37 INFO Completed push id=1 objects=8 duration=60.960792ms19492026/10/11 05:55:37 INFO Upload complete. (777ms)19502026/10/11 05:55:37 INFO Vacuumed table table=objects1951=== NAME TestClientPushesUseOnePush1952 client_pushes_test.go:97: Retrieved narinfo from S3:1953 StorePath: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientPushesUseOnePush3625403492/001/store/jhikky26nb10i9994wkbbab92kgdm8ky-shared-dep1954 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1955 Compression: zstd1956 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821957 NarSize: 1361958 References: 1959 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1960 client_pushes_test.go:97: Retrieved narinfo from S3:1961 StorePath: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientPushesUseOnePush3625403492/001/store/5npm4ymc3jkjdswckf319wrfgb56rp7n-a1962 URL: nar/1k254lsgrpwzl67lyf1r8mkyny8r941zsy4ik15asd0g53d9s1pd.nar.zst1963 Compression: zstd1964 NarHash: sha256:1k254lsgrpwzl67lyf1r8mkyny8r941zsy4ik15asd0g53d9s1pd1965 NarSize: 2561966 References: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientPushesUseOnePush3625403492/001/store/jhikky26nb10i9994wkbbab92kgdm8ky-shared-dep1967 CA: text:sha256:1kiayibh1sg179xp2r11qi1ma7dcvmcyck29jzildcaibkadhg8019682026/10/11 05:55:37 WARN Failed to register uploaded object key=cwdlb7bp9qp8mkx2sg685c15prpp82md.narinfo error="server returned 404: 404 page not found\n"1969 client_pushes_test.go:97: Retrieved narinfo from S3:19702026/10/11 05:55:37 INFO Received complete push request method=POST path=/api/pushes/1/complete1971 StorePath: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientPushesUseOnePush3625403492/001/store/9s2hgfn6iljfgn1p0wnjddpfkpdfzyf4-b1972 URL: nar/1k254lsgrpwzl67lyf1r8mkyny8r941zsy4ik15asd0g53d9s1pd.nar.zst1973 Compression: zstd1974 NarHash: sha256:1k254lsgrpwzl67lyf1r8mkyny8r941zsy4ik15asd0g53d9s1pd1975 NarSize: 2561976 References: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientPushesUseOnePush3625403492/001/store/jhikky26nb10i9994wkbbab92kgdm8ky-shared-dep1977 CA: text:sha256:1kiayibh1sg179xp2r11qi1ma7dcvmcyck29jzildcaibkadhg8019782026/10/11 05:55:37 WARN Failed to register uploaded object key=7fnj0v26n9xaj1f2mpsxlrkcflfann27.narinfo error="server returned 404: 404 page not found\n"19792026/10/11 05:55:37 WARN Request failed, retrying attempt=4 backoff=810.975447ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config19802026/10/11 05:55:37 INFO Completed push id=1 objects=6 duration=70.084166ms19812026/10/11 05:55:37 INFO Received push request method=POST path=/api/pushes19822026/10/11 05:55:37 INFO Upload complete. (610ms)1983=== NAME TestClientSharedPathCommittedMidPush1984 client_integration_test.go:680: Retrieved narinfo from S3:1985 StorePath: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientSharedPathCommittedMidPush974349414/001/store/7fnj0v26n9xaj1f2mpsxlrkcflfann27-shared-dep1986 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1987 Compression: zstd1988 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821989 NarSize: 1361990 References: 1991 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1992 client_integration_test.go:680: Retrieved narinfo from S3:1993 StorePath: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientSharedPathCommittedMidPush974349414/001/store/cwdlb7bp9qp8mkx2sg685c15prpp82md-top1994 URL: nar/0xnzy5g6c95n4qb4d7krygx5y22776mjp23mixsd70ydrfjhgf12.nar.zst1995 Compression: zstd1996 NarHash: sha256:0xnzy5g6c95n4qb4d7krygx5y22776mjp23mixsd70ydrfjhgf121997 NarSize: 2641998 References: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientSharedPathCommittedMidPush974349414/001/store/7fnj0v26n9xaj1f2mpsxlrkcflfann27-shared-dep1999 CA: text:sha256:1nxvk0kxdqc6rf08s11gp547vl3wa9qawp7pj0hxv4bljwv50yfw2000--- PASS: TestClientPushesUseOnePush (12.60s)20012026/10/11 05:55:37 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)20022026/10/11 05:55:37 INFO Uploading xj4dxkrfincwd739k1x1y6g89jbbpds5-test-file.txt (152B)20032026/10/11 05:55:37 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"20042026/10/11 05:55:37 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign20052026/10/11 05:55:37 WARN Failed to register uploaded object key=xj4dxkrfincwd739k1x1y6g89jbbpds5.ls error="server returned 404: 404 page not found\n"20062026/10/11 05:55:37 INFO Signed narinfos id=1 count=120072026/10/11 05:55:37 INFO Uploading 1 narinfos20082026/10/11 05:55:37 INFO Received complete push request method=POST path=/api/pushes/1/complete20092026/10/11 05:55:37 WARN Failed to register uploaded object key=xj4dxkrfincwd739k1x1y6g89jbbpds5.narinfo error="server returned 404: 404 page not found\n"2010--- PASS: TestClientSharedPathCommittedMidPush (11.87s)2011=== NAME TestOrphanedObjectsGCStressTest2012 orphaned_objects_gc_test.go:503: Stress test completed successfully:2013 orphaned_objects_gc_test.go:504: - Active objects preserved: 202014 orphaned_objects_gc_test.go:505: - Objects deleted: 2102015 orphaned_objects_gc_test.go:506: - Total GC'd: 2102016--- PASS: TestOrphanedObjectsGCStressTest (27.33s)20172026/10/11 05:55:37 INFO Completed push id=1 objects=3 duration=55.079625ms20182026/10/11 05:55:37 INFO Upload complete. (490ms)20192026/10/11 05:55:37 INFO All 1 paths already cached2020--- PASS: TestService_ReadScope_PublicByDefault (10.55s)2021=== NAME TestClientIntegration2022 client_integration_test.go:312: Retrieved narinfo from S3:2023 StorePath: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientIntegration1447511405/002/store/xj4dxkrfincwd739k1x1y6g89jbbpds5-test-file.txt2024 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst2025 Compression: zstd2026 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk12027 NarSize: 1522028 References: 2029 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk12030 client_integration_test.go:313: Retrieved .ls file from S3 (compressed size: 77 bytes)2031 client_integration_test.go:313: Decompressed .ls content (64 bytes):2032 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}2033 client_integration_test.go:316: Testing garbage collection...20342026/10/11 05:55:37 INFO Starting cleanup of old closures method=DELETE path=/api/closures20352026/10/11 05:55:37 INFO Garbage collection started20362026/10/11 05:55:37 INFO Aborted multipart uploads count=0 kept=020372026/10/11 05:55:38 WARN Force mode enabled - objects will be deleted immediately without grace period2038--- PASS: TestDeleteClosuresRacingPins (19.76s)20392026/10/11 05:55:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=020402026/10/11 05:55:38 INFO Vacuumed table table=pending_closures20412026/10/11 05:55:38 WARN Request failed, retrying attempt=5 backoff=1.732902342s error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config20422026/10/11 05:55:38 INFO Vacuumed table table=pending_objects20432026/10/11 05:55:38 INFO Vacuumed table table=multipart_uploads20442026/10/11 05:55:38 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=020452026/10/11 05:55:38 INFO Received create pin request method=POST path=/api/pins/myapp20462026/10/11 05:55:38 INFO Vacuumed table table=closures20472026/10/11 05:55:38 INFO Created/updated pin name=myapp store_path=/private/var/tb/8/scratch/59e19724d113dbbe/build/TestPinProtectsFromGC2188448925/001/store/04wck7jprc6msb0gslgizlb3kgg2gqs5-pinned-file.txt narinfo_key=04wck7jprc6msb0gslgizlb3kgg2gqs5.narinfo20482026/10/11 05:55:38 INFO Starting cleanup of old closures method=DELETE path=/api/closures20492026/10/11 05:55:38 INFO Garbage collection started20502026/10/11 05:55:38 INFO Aborted multipart uploads count=0 kept=020512026/10/11 05:55:38 WARN Force mode enabled - objects will be deleted immediately without grace period20522026/10/11 05:55:38 INFO Vacuumed table table=objects20532026/10/11 05:55:38 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=020542026/10/11 05:55:38 INFO Vacuumed table table=pending_closures20552026/10/11 05:55:38 INFO Vacuumed table table=pending_objects20562026/10/11 05:55:38 INFO Vacuumed table table=multipart_uploads20572026/10/11 05:55:38 INFO Vacuumed table table=closures20582026/10/11 05:55:38 INFO Vacuumed table table=objects20592026/10/11 05:55:38 INFO Received uploads request method=POST path=/api/pending_closures2060--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (11.39s)20612026/10/11 05:55:39 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=02062=== NAME TestClientIntegration2063 client_integration_test.go:323: Objects in database after GC:2064 client_integration_test.go:323: Successfully deleted all objects with GC --force20652026/10/11 05:55:40 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: sending request: request failed after retries: Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused"20662026/10/11 05:55:40 WARN Request failed, retrying attempt=1 backoff=100ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config20672026/10/11 05:55:40 WARN Request failed, retrying attempt=2 backoff=204.501423ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config2068--- PASS: TestClientIntegration (14.30s)2069=== NAME TestClientWithDependencies2070 client_integration_test.go:613: Built derivation: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientWithDependencies2140888750/001/store/j7wyf506qg09sn2g5r71lm4f3ipg8fx2-test-script20712026/10/11 05:55:40 WARN Request failed, retrying attempt=3 backoff=374.422305ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config2072 client_integration_test.go:615: Found 1 dependencies (including self)2073=== NAME TestPinProtectsFromGC2074 client_integration_test.go:840: Pin successfully protected closure from garbage collection20752026/10/11 05:55:40 INFO Received push request method=POST path=/api/pushes20762026/10/11 05:55:40 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)20772026/10/11 05:55:40 INFO Uploading j7wyf506qg09sn2g5r71lm4f3ipg8fx2-test-script (136B)20782026/10/11 05:55:40 WARN Failed to register uploaded object key=j7wyf506qg09sn2g5r71lm4f3ipg8fx2.ls error="server returned 404: 404 page not found\n"20792026/10/11 05:55:40 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"20802026/10/11 05:55:40 WARN Failed to register uploaded object key=log/s85px8r4jv6z2nwlvlv5z3rrww92dacj-test-script.drv error="server returned 404: 404 page not found\n"20812026/10/11 05:55:40 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign2082--- PASS: TestPinProtectsFromGC (16.83s)20832026/10/11 05:55:40 INFO Signed narinfos id=1 count=120842026/10/11 05:55:40 INFO Uploading 1 narinfos20852026/10/11 05:55:40 WARN Request failed, retrying attempt=4 backoff=762.737001ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config20862026/10/11 05:55:40 INFO Received complete push request method=POST path=/api/pushes/1/complete20872026/10/11 05:55:40 WARN Failed to register uploaded object key=j7wyf506qg09sn2g5r71lm4f3ipg8fx2.narinfo error="server returned 404: 404 page not found\n"20882026/10/11 05:55:40 INFO Received uploads request method=POST path=/api/pending_closures2089--- PASS: TestPresentNotReportedWhileGCDeletesClosure (0.00s)2090 --- PASS: TestPresentNotReportedWhileGCDeletesClosure/stale (11.02s)2091 --- PASS: TestPresentNotReportedWhileGCDeletesClosure/fresh (10.55s)20922026/10/11 05:55:40 INFO Completed push id=1 objects=4 duration=35.023708ms20932026/10/11 05:55:40 INFO Upload complete. (363ms)2094=== NAME TestClientWithDependencies2095 client_integration_test.go:617: Skipping nix copy test - isolated store (/private/var/tb/8/scratch/59e19724d113dbbe/build/TestClientWithDependencies2140888750/001/store) requires matching store prefix2096--- PASS: TestClientWithDependencies (12.74s)20972026/10/11 05:55:41 WARN Request failed, retrying attempt=5 backoff=1.526337984s error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config20982026/10/11 05:55:41 INFO Received uploads request method=POST path=/api/pending_closures20992026/10/11 05:55:41 INFO Aborted multipart uploads count=0 kept=021002026/10/11 05:55:41 WARN Force mode enabled - objects will be deleted immediately without grace period21012026/10/11 05:55:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=02102--- PASS: TestPendingClosureWriteTimeout (0.05s)2103 --- PASS: TestPendingClosureWriteTimeout/empty (0.00s)2104 --- PASS: TestPendingClosureWriteTimeout/670k_objects (0.00s)2105 --- PASS: TestPendingClosureWriteTimeout/400_objects (0.00s)2106 --- PASS: TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline (11.05s)21072026/10/11 05:55:41 INFO Vacuumed table table=pending_closures21082026/10/11 05:55:41 INFO Vacuumed table table=pending_objects21092026/10/11 05:55:41 INFO Vacuumed table table=multipart_uploads21102026/10/11 05:55:41 INFO Vacuumed table table=closures21112026/10/11 05:55:41 INFO Vacuumed table table=objects21122026/10/11 05:55:42 INFO Received uploads request method=POST path=/api/pending_closures21132026/10/11 05:55:42 INFO Aborted multipart uploads count=0 kept=021142026/10/11 05:55:42 WARN Force mode enabled - objects will be deleted immediately without grace period21152026/10/11 05:55:42 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=1 objects-deleted-after-grace-period=1 objects-failed-to-delete=021162026/10/11 05:55:42 INFO Vacuumed table table=pending_closures21172026/10/11 05:55:42 INFO Vacuumed table table=pending_objects21182026/10/11 05:55:42 INFO Vacuumed table table=multipart_uploads21192026/10/11 05:55:42 INFO Vacuumed table table=closures21202026/10/11 05:55:42 INFO Vacuumed table table=objects2121--- PASS: TestForceGCDuringPushOffersSweptObject (0.00s)2122 --- PASS: TestForceGCDuringPushOffersSweptObject/after_presence_check (10.72s)2123 --- PASS: TestForceGCDuringPushOffersSweptObject/before_pending_rows (11.41s)21242026/10/11 05:55:43 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures21252026/10/11 05:55:43 WARN Request failed, retrying attempt=2 backoff=211.207842ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures21262026/10/11 05:55:43 INFO Aborted multipart uploads count=0 kept=021272026/10/11 05:55:43 WARN Force mode enabled - objects will be deleted immediately without grace period21282026-10-11 05:55:43.262 UTC [91405] ERROR: canceling statement due to user request21292026-10-11 05:55:43.262 UTC [91405] STATEMENT: -- name: MarkStaleObjects :execrows2130 WITH RECURSIVE ct AS (2131 SELECT timezone('UTC', now()) AS now2132 ),2133 closure_reach AS (2134 -- Start with all closure keys2135 SELECT o.key, o.refs2136 FROM objects o2137 INNER JOIN closures c ON o.key = c.key2138 UNION2139 -- Recursively add all referenced objects2140 SELECT o.key, o.refs2141 FROM objects o2142 INNER JOIN closure_reach cr ON o.key = ANY(cr.refs)2143 ),2144 reachable_objects AS (2145 SELECT DISTINCT key FROM closure_reach2146 ),2147 stale_objects AS (2148 SELECT o.key2149 FROM objects AS o, ct2150 WHERE2151 NOT EXISTS (2152 SELECT 12153 FROM reachable_objects ro2154 WHERE ro.key = o.key2155 )2156 AND NOT EXISTS (2157 SELECT 12158 FROM pending_objects AS po2159 WHERE po.key = o.key2160 )2161 AND o.deleted_at IS NULL -- Only mark fresh objects2162 ORDER BY o.key COLLATE "C" -- lock in key order, like commit_pending_closure2163 FOR UPDATE2164 )2165 UPDATE objects2166 SET2167 deleted_at = ct.now,2168 first_deleted_at = COALESCE(first_deleted_at, ct.now)2169 FROM stale_objects, ct2170 WHERE objects.key = stale_objects.key2171 21722026/10/11 05:55:43 WARN Request failed, retrying attempt=3 backoff=361.842425ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures21732026/10/11 05:55:43 INFO Aborted multipart uploads count=0 kept=021742026/10/11 05:55:43 WARN Force mode enabled - objects will be deleted immediately without grace period21752026/10/11 05:55:43 ERROR failed to remove object object=nar/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.nar.zst error="Post \"http://localhost:58759/bucket98/?delete=\": context canceled"2176--- PASS: TestGCEndsOnShutdown (0.00s)2177 --- PASS: TestGCEndsOnShutdown/before_the_run (10.93s)2178 --- PASS: TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark (11.23s)2179 --- PASS: TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete (11.57s)21802026/10/11 05:55:43 WARN Request failed, retrying attempt=4 backoff=824.14478ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures21812026/10/11 05:55:44 WARN Request failed, retrying attempt=5 backoff=1.557748812s error="Post \"http://localhost:19999/api/pending_closures\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures21822026/10/11 05:55:44 WARN Authentication failed token_preview=invalid-token token_length=13 reason="static API token mismatch"21832026/10/11 05:55:44 WARN Authentication failed token_preview=invalid-token token_length=13 reason="static API token mismatch"2184--- PASS: TestClientErrorHandling (0.00s)2185 --- PASS: TestClientErrorHandling/InvalidStorePath (11.43s)2186 --- PASS: TestClientErrorHandling/InvalidAuthToken (11.73s)2187 --- PASS: TestClientErrorHandling/ServerNotAvailable (13.37s)2188PASS2189{"timestamp":"2026-10-11T05:55:46.180869Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:58926","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2354,"threadName":"rustfs-worker","threadId":"ThreadId(10)"}21902026-10-11 05:55:48.705 UTC [90579] LOG: received smart shutdown request21912026-10-11 05:55:48.707 UTC [90579] LOG: background worker "logical replication launcher" (PID 90590) exited with exit code 121922026-10-11 05:55:58.660 UTC [90585] LOG: checkpoint complete: wrote 10927 buffers (66.7%), wrote 5 SLRU buffers; 0 WAL file(s) added, 0 removed, 33 recycled; write=14.963 s, sync=4.912 s, total=21.796 s; sync files=34045, longest=0.015 s, average=0.001 s; distance=532944 kB, estimate=532944 kB; lsn=0/22A682C8, redo lsn=0/22000FA821932026-10-11 05:55:58.673 UTC [90585] LOG: shutting down21942026-10-11 05:55:58.674 UTC [90585] LOG: checkpoint starting: shutdown immediate21952026/10/11 05:55:58 INFO killed process group pgid=9057921962026/10/11 05:55:58 ERROR failed to wait for process error="signal: killed"21972026-10-11 05:55:58.726 UTC [90585] PANIC: could not open file "pg_multixact/members": No such file or directory2198Running OIDC tests...2199=== RUN TestAudienceForIssuer2200=== PAUSE TestAudienceForIssuer2201=== RUN TestGlobMatch2202=== PAUSE TestGlobMatch2203=== RUN TestValidateToken_ValidToken2204=== PAUSE TestValidateToken_ValidToken2205=== RUN TestValidateToken_WrongAudience2206=== PAUSE TestValidateToken_WrongAudience2207=== RUN TestValidateToken_Expired2208=== PAUSE TestValidateToken_Expired2209=== RUN TestValidateToken_BoundClaimsMismatch2210=== PAUSE TestValidateToken_BoundClaimsMismatch2211=== RUN TestValidateToken_BoundSubjectMismatch2212=== PAUSE TestValidateToken_BoundSubjectMismatch2213=== RUN TestValidateToken_MultipleProviders2214=== PAUSE TestValidateToken_MultipleProviders2215=== RUN TestValidateToken_NoMatchingProvider2216=== PAUSE TestValidateToken_NoMatchingProvider2217=== RUN TestValidateToken_KubernetesServiceAccount2218=== PAUSE TestValidateToken_KubernetesServiceAccount2219=== RUN TestNewValidator_KubernetesRequiresCA2220=== PAUSE TestNewValidator_KubernetesRequiresCA2221=== RUN TestValidateToken_KubernetesIssuerFromOwnToken2222=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken2223=== RUN TestPins_ReservedForMatchingRule2224=== PAUSE TestPins_ReservedForMatchingRule2225=== RUN TestPins_TopLevelShorthand2226=== PAUSE TestPins_TopLevelShorthand2227=== RUN TestPins_ConfigValidation2228=== PAUSE TestPins_ConfigValidation2229=== RUN TestScopes_LegacyProviderDefaultsToWrite2230=== PAUSE TestScopes_LegacyProviderDefaultsToWrite2231=== RUN TestScopes_Rules2232=== PAUSE TestScopes_Rules2233=== RUN TestScopes_ConfigValidation2234=== PAUSE TestScopes_ConfigValidation2235=== CONT TestAudienceForIssuer2236--- PASS: TestAudienceForIssuer (0.00s)2237=== CONT TestPins_ConfigValidation2238=== CONT TestValidateToken_BoundClaimsMismatch2239=== CONT TestValidateToken_Expired2240=== CONT TestValidateToken_WrongAudience2241=== CONT TestValidateToken_ValidToken2242=== CONT TestGlobMatch2243=== RUN TestGlobMatch/foo_foo2244=== PAUSE TestGlobMatch/foo_foo2245=== RUN TestGlobMatch/foo_bar2246=== PAUSE TestGlobMatch/foo_bar2247=== RUN TestGlobMatch/*_2248=== PAUSE TestGlobMatch/*_2249=== RUN TestGlobMatch/*_anything2250=== PAUSE TestGlobMatch/*_anything2251=== RUN TestGlobMatch/foo*_foo2252=== PAUSE TestGlobMatch/foo*_foo2253=== RUN TestGlobMatch/foo*_foobar2254=== PAUSE TestGlobMatch/foo*_foobar2255=== RUN TestGlobMatch/foo*_bar2256=== PAUSE TestGlobMatch/foo*_bar2257=== RUN TestGlobMatch/*bar_bar2258=== PAUSE TestGlobMatch/*bar_bar2259=== RUN TestGlobMatch/*bar_foobar2260=== PAUSE TestGlobMatch/*bar_foobar2261=== RUN TestGlobMatch/*bar_foo2262=== PAUSE TestGlobMatch/*bar_foo2263=== RUN TestGlobMatch/foo*bar_foobar2264=== PAUSE TestGlobMatch/foo*bar_foobar2265=== RUN TestGlobMatch/foo*bar_foo123bar2266=== PAUSE TestGlobMatch/foo*bar_foo123bar2267=== RUN TestGlobMatch/foo*bar_foobarbaz2268=== PAUSE TestGlobMatch/foo*bar_foobarbaz2269=== RUN TestGlobMatch/*/*_foo/bar2270=== PAUSE TestGlobMatch/*/*_foo/bar2271=== CONT TestValidateToken_MultipleProviders2272--- PASS: TestPins_ConfigValidation (0.00s)2273=== CONT TestValidateToken_NoMatchingProvider2274=== CONT TestPins_ReservedForMatchingRule2275=== CONT TestPins_TopLevelShorthand2276=== CONT TestValidateToken_BoundSubjectMismatch2277=== CONT TestScopes_Rules2278=== CONT TestScopes_ConfigValidation2279--- PASS: TestScopes_ConfigValidation (0.00s)2280=== CONT TestValidateToken_KubernetesIssuerFromOwnToken2281=== CONT TestScopes_LegacyProviderDefaultsToWrite2282=== CONT TestNewValidator_KubernetesRequiresCA2283=== CONT TestValidateToken_KubernetesServiceAccount2284=== RUN TestGlobMatch/*/*_foo2285=== PAUSE TestGlobMatch/*/*_foo2286=== RUN TestGlobMatch/refs/heads/*_refs/heads/main2287=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main2288=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.02289=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.02290=== RUN TestGlobMatch/refs/*/main_refs/heads/main2291=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main2292=== RUN TestGlobMatch/fo?_foo2293=== PAUSE TestGlobMatch/fo?_foo2294=== RUN TestGlobMatch/fo?_fo2295=== PAUSE TestGlobMatch/fo?_fo2296=== RUN TestGlobMatch/fo?_fooo2297=== PAUSE TestGlobMatch/fo?_fooo2298=== RUN TestGlobMatch/?oo_foo2299=== PAUSE TestGlobMatch/?oo_foo2300=== RUN TestGlobMatch/?oo_boo2301=== PAUSE TestGlobMatch/?oo_boo2302=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main2303=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main2304=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main2305=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main2306=== CONT TestGlobMatch/foo_foo2307=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main2308=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main2309=== CONT TestGlobMatch/?oo_boo2310=== CONT TestGlobMatch/?oo_foo2311=== CONT TestGlobMatch/fo?_fooo2312=== CONT TestGlobMatch/fo?_fo2313=== CONT TestGlobMatch/fo?_foo2314=== CONT TestGlobMatch/refs/*/main_refs/heads/main2315=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.02316=== CONT TestGlobMatch/refs/heads/*_refs/heads/main2317=== CONT TestGlobMatch/*/*_foo2318=== CONT TestGlobMatch/*/*_foo/bar2319=== CONT TestGlobMatch/foo*bar_foobarbaz2320=== CONT TestGlobMatch/foo*bar_foo123bar2321=== CONT TestGlobMatch/foo*bar_foobar2322=== CONT TestGlobMatch/*bar_foo2323=== CONT TestGlobMatch/*bar_foobar2324=== CONT TestGlobMatch/*bar_bar2325=== CONT TestGlobMatch/foo*_bar2326=== CONT TestGlobMatch/foo*_foobar2327=== CONT TestGlobMatch/foo*_foo2328=== CONT TestGlobMatch/*_anything2329=== CONT TestGlobMatch/*_2330=== CONT TestGlobMatch/foo_bar2331--- PASS: TestGlobMatch (0.03s)2332 --- PASS: TestGlobMatch/foo_foo (0.00s)2333 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)2334 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)2335 --- PASS: TestGlobMatch/?oo_boo (0.00s)2336 --- PASS: TestGlobMatch/?oo_foo (0.00s)2337 --- PASS: TestGlobMatch/fo?_fooo (0.00s)2338 --- PASS: TestGlobMatch/fo?_fo (0.00s)2339 --- PASS: TestGlobMatch/fo?_foo (0.00s)2340 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)2341 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)2342 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)2343 --- PASS: TestGlobMatch/*/*_foo (0.00s)2344 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)2345 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)2346 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)2347 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)2348 --- PASS: TestGlobMatch/*bar_foo (0.00s)2349 --- PASS: TestGlobMatch/*bar_foobar (0.00s)2350 --- PASS: TestGlobMatch/*bar_bar (0.00s)2351 --- PASS: TestGlobMatch/foo*_bar (0.00s)2352 --- PASS: TestGlobMatch/foo*_foobar (0.00s)2353 --- PASS: TestGlobMatch/foo*_foo (0.00s)2354 --- PASS: TestGlobMatch/*_anything (0.00s)2355 --- PASS: TestGlobMatch/*_ (0.00s)2356 --- PASS: TestGlobMatch/foo_bar (0.00s)23572026/10/11 05:56:11 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:59521/oidc23582026/10/11 05:56:11 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:59527/oidc23592026/10/11 05:56:11 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:59524/oidc2360--- PASS: TestValidateToken_WrongAudience (0.08s)2361--- PASS: TestValidateToken_ValidToken (0.08s)23622026/10/11 05:56:11 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:5952523632026/10/11 05:56:11 http: TLS handshake error from 127.0.0.1:59523: remote error: tls: bad certificate2364--- PASS: TestNewValidator_KubernetesRequiresCA (0.07s)2365--- PASS: TestValidateToken_KubernetesServiceAccount (0.07s)2366--- PASS: TestPins_ReservedForMatchingRule (0.09s)23672026/10/11 05:56:11 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:59532/oidc23682026/10/11 05:56:11 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:59531/oidc2369--- PASS: TestPins_TopLevelShorthand (0.10s)2370--- PASS: TestScopes_Rules (0.10s)23712026/10/11 05:56:11 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:59535/oidc2372--- PASS: TestValidateToken_BoundSubjectMismatch (0.11s)23732026/10/11 05:56:11 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:59537/oidc23742026/10/11 05:56:11 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:59530/oidc2375--- PASS: TestValidateToken_MultipleProviders (0.13s)23762026/10/11 05:56:11 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC12323772026/10/11 05:56:11 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:59520/oidc2378--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (0.12s)2379--- PASS: TestValidateToken_NoMatchingProvider (0.13s)23802026/10/11 05:56:11 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:59545/oidc23812026/10/11 05:56:11 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:59547/oidc2382--- PASS: TestScopes_LegacyProviderDefaultsToWrite (0.13s)2383--- PASS: TestValidateToken_Expired (0.15s)23842026/10/11 05:56:11 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:59549/oidc2385--- PASS: TestValidateToken_BoundClaimsMismatch (0.17s)2386PASS2387Running hook tests...2388=== RUN TestSendPathsEmpty2389=== PAUSE TestSendPathsEmpty2390=== RUN TestQueueEnqueueAndFetch2391=== PAUSE TestQueueEnqueueAndFetch2392=== RUN TestQueueDeduplication2393=== PAUSE TestQueueDeduplication2394=== RUN TestQueueRemove2395=== PAUSE TestQueueRemove2396=== RUN TestQueueFetchBatchLimit2397=== PAUSE TestQueueFetchBatchLimit2398=== RUN TestQueueRetryMovesToBack2399=== PAUSE TestQueueRetryMovesToBack2400=== RUN TestQueueFetchRemoveLifecycle2401=== PAUSE TestQueueFetchRemoveLifecycle2402=== RUN TestQueueConcurrentWriters2403=== PAUSE TestQueueConcurrentWriters2404=== RUN TestQueueRemoveLargeClosure2405=== PAUSE TestQueueRemoveLargeClosure2406=== RUN TestServerClientIntegration2407--- PASS: TestServerClientIntegration (0.00s)2408=== RUN TestServerQueueError24092026/10/11 05:56:12 ERROR Failed to queue paths error="permission denied" count=12410--- PASS: TestServerQueueError (0.00s)2411=== RUN TestGetListenerSocketActivation2412 server_test.go:146: systemd socket activation does not exist on darwin2413--- SKIP: TestGetListenerSocketActivation (0.00s)2414=== RUN TestDrainIsolatesPoisonPath2415=== PAUSE TestDrainIsolatesPoisonPath2416=== RUN TestRunNotBlockedByPoisonHead2417=== PAUSE TestRunNotBlockedByPoisonHead2418=== RUN TestDrainGivesUpWhenServerDown2419=== PAUSE TestDrainGivesUpWhenServerDown2420=== RUN TestFailedPathPrunedByLaterClosure2421=== PAUSE TestFailedPathPrunedByLaterClosure2422=== RUN TestWorkerUploadsAndRemoves2423=== PAUSE TestWorkerUploadsAndRemoves2424=== RUN TestWorkerSkipsGCdPaths2425=== PAUSE TestWorkerSkipsGCdPaths2426=== RUN TestWorkerPrunesClosureDeps2427=== PAUSE TestWorkerPrunesClosureDeps2428=== RUN TestDrainTimeout2429=== PAUSE TestDrainTimeout2430=== CONT TestSendPathsEmpty2431=== CONT TestDrainTimeout2432--- PASS: TestSendPathsEmpty (0.00s)2433=== CONT TestDrainIsolatesPoisonPath2434=== CONT TestQueueRemoveLargeClosure2435=== CONT TestQueueConcurrentWriters2436=== CONT TestQueueFetchRemoveLifecycle2437=== CONT TestQueueDeduplication2438=== CONT TestQueueEnqueueAndFetch2439=== CONT TestQueueRetryMovesToBack2440=== CONT TestQueueRemove2441=== CONT TestQueueFetchBatchLimit2442=== CONT TestWorkerUploadsAndRemoves2443=== CONT TestWorkerPrunesClosureDeps2444=== CONT TestWorkerSkipsGCdPaths2445=== CONT TestDrainGivesUpWhenServerDown2446=== CONT TestFailedPathPrunedByLaterClosure2447=== CONT TestRunNotBlockedByPoisonHead24482026/10/11 05:56:12 INFO Upload queue status pending=324492026/10/11 05:56:12 INFO Uploading batch count=124502026/10/11 05:56:12 ERROR Upload failed error="upload failed" count=124512026/10/11 05:56:12 INFO Uploading batch count=224522026/10/11 05:56:12 INFO Upload queue status pending=224532026/10/11 05:56:12 INFO Uploading batch count=124542026/10/11 05:56:12 INFO Upload queue status pending=224552026/10/11 05:56:12 ERROR Upload failed error="upload failed" count=124562026/10/11 05:56:12 INFO Uploading batch count=124572026/10/11 05:56:12 INFO Upload queue status pending=224582026/10/11 05:56:12 INFO Uploading batch count=224592026/10/11 05:56:12 INFO Uploading batch count=12460--- PASS: TestQueueFetchBatchLimit (0.01s)24612026/10/11 05:56:12 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/8/scratch/59e19724d113dbbe/build/TestWorkerSkipsGCdPaths2740102576/002/nonexistent24622026/10/11 05:56:12 INFO Uploading batch count=424632026/10/11 05:56:12 ERROR Upload failed error="upload failed" count=424642026/10/11 05:56:12 INFO Uploading batch count=124652026/10/11 05:56:12 INFO Uploading batch count=124662026/10/11 05:56:12 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/8/scratch/59e19724d113dbbe/build/TestDrainIsolatesPoisonPath1141565269/002/bbb2467--- PASS: TestQueueEnqueueAndFetch (0.01s)2468--- PASS: TestQueueDeduplication (0.01s)2469--- PASS: TestQueueFetchRemoveLifecycle (0.02s)24702026/10/11 05:56:12 INFO Uploading batch count=224712026/10/11 05:56:12 ERROR Upload failed error="upload failed" count=224722026/10/11 05:56:12 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/8/scratch/59e19724d113dbbe/build/TestDrainGivesUpWhenServerDown294018489/002/a2473--- PASS: TestQueueRetryMovesToBack (0.01s)24742026/10/11 05:56:12 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/8/scratch/59e19724d113dbbe/build/TestDrainGivesUpWhenServerDown294018489/002/b24752026/10/11 05:56:12 INFO Uploading batch count=224762026/10/11 05:56:12 ERROR Upload failed error="upload failed" count=224772026/10/11 05:56:12 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/8/scratch/59e19724d113dbbe/build/TestDrainGivesUpWhenServerDown294018489/002/c24782026/10/11 05:56:12 INFO Uploading batch count=124792026/10/11 05:56:12 ERROR Upload failed error="upload failed" count=124802026/10/11 05:56:12 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/8/scratch/59e19724d113dbbe/build/TestDrainGivesUpWhenServerDown294018489/002/d2481--- PASS: TestQueueRemove (0.02s)24822026/10/11 05:56:12 INFO Uploading batch count=124832026/10/11 05:56:12 ERROR Upload failed error="upload failed" count=12484--- PASS: TestFailedPathPrunedByLaterClosure (0.02s)24852026/10/11 05:56:12 INFO Uploading batch count=224862026/10/11 05:56:12 ERROR Upload failed error="upload failed" count=224872026/10/11 05:56:12 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/8/scratch/59e19724d113dbbe/build/TestDrainGivesUpWhenServerDown294018489/002/e24882026/10/11 05:56:12 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/8/scratch/59e19724d113dbbe/build/TestDrainGivesUpWhenServerDown294018489/002/f24892026/10/11 05:56:12 INFO Uploading batch count=124902026/10/11 05:56:12 ERROR Upload failed error="upload failed" count=124912026/10/11 05:56:12 ERROR Drain finished with paths left in queue remaining=1024922026/10/11 05:56:12 ERROR Drain finished with paths left in queue remaining=12493--- PASS: TestDrainIsolatesPoisonPath (0.02s)2494--- PASS: TestDrainGivesUpWhenServerDown (0.02s)2495--- PASS: TestWorkerUploadsAndRemoves (0.03s)2496--- PASS: TestWorkerPrunesClosureDeps (0.03s)2497--- PASS: TestWorkerSkipsGCdPaths (0.03s)2498--- PASS: TestQueueRemoveLargeClosure (0.05s)2499--- PASS: TestQueueConcurrentWriters (0.13s)25002026/10/11 05:56:12 ERROR Upload failed error="context deadline exceeded" count=225012026/10/11 05:56:12 ERROR Drain finished with paths left in queue remaining=42502--- PASS: TestDrainTimeout (0.22s)25032026/10/11 05:56:13 INFO Uploading batch count=125042026/10/11 05:56:13 INFO Uploading batch count=125052026/10/11 05:56:13 INFO Uploading batch count=125062026/10/11 05:56:13 ERROR Upload failed error="upload failed" count=125072026/10/11 05:56:13 INFO Uploading batch count=125082026/10/11 05:56:13 ERROR Upload failed error="upload failed" count=125092026/10/11 05:56:13 INFO Uploading batch count=125102026/10/11 05:56:13 ERROR Upload failed error="upload failed" count=125112026/10/11 05:56:13 INFO Uploading batch count=125122026/10/11 05:56:13 ERROR Upload failed error="upload failed" count=125132026/10/11 05:56:13 ERROR Drain finished with paths left in queue remaining=12514--- PASS: TestRunNotBlockedByPoisonHead (1.04s)2515PASS2516Running signing tests...2517=== RUN TestExternalSigner2518=== PAUSE TestExternalSigner2519=== RUN TestExternalSignerWrongPublicKey2520=== PAUSE TestExternalSignerWrongPublicKey2521=== RUN TestExternalSignerErrors2522=== PAUSE TestExternalSignerErrors2523=== RUN TestExternalSignerRestartsAfterCrash2524=== PAUSE TestExternalSignerRestartsAfterCrash2525=== RUN TestExternalSignerProgramErrorKeepsProcess2526=== PAUSE TestExternalSignerProgramErrorKeepsProcess2527=== RUN TestExternalSignerClose2528=== PAUSE TestExternalSignerClose2529=== RUN TestExternalSignerRejectsInvalidPublicKey2530=== PAUSE TestExternalSignerRejectsInvalidPublicKey2531=== RUN TestExternalSignerCancellation2532=== PAUSE TestExternalSignerCancellation2533=== RUN TestGenerateFingerprint2534=== PAUSE TestGenerateFingerprint2535=== RUN TestParseSigningKey2536=== PAUSE TestParseSigningKey2537=== RUN TestSign2538=== PAUSE TestSign2539=== RUN TestSignBatch2540=== PAUSE TestSignBatch2541=== RUN TestNilKeyPublicKey2542=== PAUSE TestNilKeyPublicKey2543=== RUN TestSignErrors2544=== PAUSE TestSignErrors2545=== CONT TestExternalSigner2546=== CONT TestExternalSignerProgramErrorKeepsProcess2547=== CONT TestSignBatch2548=== CONT TestExternalSignerCancellation2549=== CONT TestSign2550=== CONT TestParseSigningKey2551=== RUN TestParseSigningKey/valid_32-byte_key2552=== PAUSE TestParseSigningKey/valid_32-byte_key2553=== RUN TestParseSigningKey/valid_32-byte_key_with_different_name2554=== PAUSE TestParseSigningKey/valid_32-byte_key_with_different_name2555=== RUN TestParseSigningKey/no_colon2556=== CONT TestGenerateFingerprint2557=== RUN TestGenerateFingerprint/basic_with_references2558=== PAUSE TestGenerateFingerprint/basic_with_references2559=== CONT TestExternalSignerErrors2560=== RUN TestExternalSignerErrors/missing_signatures2561=== CONT TestExternalSignerRestartsAfterCrash2562=== CONT TestExternalSignerWrongPublicKey2563=== CONT TestExternalSignerRejectsInvalidPublicKey2564=== RUN TestExternalSignerRejectsInvalidPublicKey/#002565=== PAUSE TestExternalSignerRejectsInvalidPublicKey/#002566=== CONT TestExternalSignerClose2567=== CONT TestSignErrors2568=== CONT TestNilKeyPublicKey2569=== PAUSE TestParseSigningKey/no_colon2570=== RUN TestParseSigningKey/empty_name2571=== PAUSE TestParseSigningKey/empty_name2572=== RUN TestParseSigningKey/invalid_base642573=== PAUSE TestParseSigningKey/invalid_base642574=== RUN TestExternalSignerRejectsInvalidPublicKey/no-colon2575=== PAUSE TestExternalSignerRejectsInvalidPublicKey/no-colon2576=== PAUSE TestExternalSignerErrors/missing_signatures2577=== RUN TestGenerateFingerprint/no_references2578=== PAUSE TestGenerateFingerprint/no_references2579=== RUN TestParseSigningKey/wrong_length2580=== PAUSE TestParseSigningKey/wrong_length2581=== CONT TestParseSigningKey/valid_32-byte_key2582=== CONT TestParseSigningKey/invalid_base642583--- PASS: TestNilKeyPublicKey (0.00s)2584=== RUN TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=2585=== RUN TestExternalSignerErrors/too_few_signatures2586=== RUN TestGenerateFingerprint/unsorted_references_get_sorted2587=== PAUSE TestGenerateFingerprint/unsorted_references_get_sorted2588=== CONT TestParseSigningKey/empty_name2589=== CONT TestParseSigningKey/no_colon2590=== CONT TestParseSigningKey/valid_32-byte_key_with_different_name2591=== CONT TestParseSigningKey/wrong_length2592=== PAUSE TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=2593=== PAUSE TestExternalSignerErrors/too_few_signatures2594=== RUN TestExternalSignerErrors/missing_signature_name2595=== PAUSE TestExternalSignerErrors/missing_signature_name2596=== RUN TestGenerateFingerprint/invalid_nar_hash_prefix2597=== PAUSE TestGenerateFingerprint/invalid_nar_hash_prefix2598=== RUN TestGenerateFingerprint/invalid_nar_hash_length2599=== PAUSE TestGenerateFingerprint/invalid_nar_hash_length2600=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!2601=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!2602=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==2603=== RUN TestExternalSignerErrors/program_error2604=== PAUSE TestExternalSignerErrors/program_error2605=== RUN TestGenerateFingerprint/invalid_store_path_prefix2606=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==2607=== CONT TestExternalSignerRejectsInvalidPublicKey/#002608=== CONT TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=2609=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!2610=== PAUSE TestGenerateFingerprint/invalid_store_path_prefix2611=== RUN TestGenerateFingerprint/invalid_reference_prefix2612=== RUN TestExternalSignerErrors/exit_at_startup2613=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==2614=== PAUSE TestExternalSignerErrors/exit_at_startup2615=== RUN TestExternalSignerErrors/unrequested_key2616=== CONT TestExternalSignerRejectsInvalidPublicKey/no-colon2617--- PASS: TestExternalSignerRejectsInvalidPublicKey (0.00s)2618 --- PASS: TestExternalSignerRejectsInvalidPublicKey/#00 (0.00s)2619 --- PASS: TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= (0.00s)2620 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:invalid! (0.00s)2621 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:YQ== (0.00s)2622 --- PASS: TestExternalSignerRejectsInvalidPublicKey/no-colon (0.00s)2623=== PAUSE TestGenerateFingerprint/invalid_reference_prefix2624=== CONT TestGenerateFingerprint/basic_with_references2625=== CONT TestGenerateFingerprint/invalid_nar_hash_prefix2626=== CONT TestGenerateFingerprint/invalid_nar_hash_length2627=== PAUSE TestExternalSignerErrors/unrequested_key2628=== CONT TestGenerateFingerprint/invalid_reference_prefix2629=== CONT TestGenerateFingerprint/invalid_store_path_prefix2630=== CONT TestGenerateFingerprint/unsorted_references_get_sorted2631=== CONT TestGenerateFingerprint/no_references2632--- PASS: TestGenerateFingerprint (0.00s)2633 --- PASS: TestGenerateFingerprint/basic_with_references (0.00s)2634 --- PASS: TestGenerateFingerprint/invalid_nar_hash_prefix (0.00s)2635 --- PASS: TestGenerateFingerprint/invalid_nar_hash_length (0.00s)2636 --- PASS: TestGenerateFingerprint/invalid_reference_prefix (0.00s)2637 --- PASS: TestGenerateFingerprint/invalid_store_path_prefix (0.00s)2638 --- PASS: TestGenerateFingerprint/unsorted_references_get_sorted (0.00s)2639 --- PASS: TestGenerateFingerprint/no_references (0.00s)2640=== RUN TestExternalSignerErrors/wrong_signature_length2641=== PAUSE TestExternalSignerErrors/wrong_signature_length2642=== RUN TestExternalSignerErrors/wrong_id2643=== PAUSE TestExternalSignerErrors/wrong_id2644=== RUN TestExternalSignerErrors/wrong_signatures_shape2645=== PAUSE TestExternalSignerErrors/wrong_signatures_shape2646=== RUN TestExternalSignerErrors/wrong_signature_name2647=== PAUSE TestExternalSignerErrors/wrong_signature_name2648=== RUN TestExternalSignerErrors/invalid_signature_base642649=== PAUSE TestExternalSignerErrors/invalid_signature_base642650=== RUN TestExternalSignerErrors/invalid_JSON2651=== PAUSE TestExternalSignerErrors/invalid_JSON2652=== RUN TestExternalSignerErrors/wrong_shape2653=== PAUSE TestExternalSignerErrors/wrong_shape2654=== RUN TestExternalSignerErrors/null2655=== PAUSE TestExternalSignerErrors/null2656=== RUN TestExternalSignerErrors/null_signatures2657=== PAUSE TestExternalSignerErrors/null_signatures2658=== RUN TestExternalSignerErrors/exit_on_request2659=== PAUSE TestExternalSignerErrors/exit_on_request2660=== CONT TestExternalSignerErrors/missing_signatures2661=== CONT TestExternalSignerErrors/wrong_id2662=== CONT TestExternalSignerErrors/wrong_signatures_shape2663=== CONT TestExternalSignerErrors/wrong_signature_length2664=== CONT TestExternalSignerErrors/exit_on_request2665=== CONT TestExternalSignerErrors/null2666=== CONT TestExternalSignerErrors/wrong_shape2667=== CONT TestExternalSignerErrors/invalid_JSON2668--- PASS: TestSign (0.00s)2669=== CONT TestExternalSignerErrors/invalid_signature_base642670=== CONT TestExternalSignerErrors/wrong_signature_name2671=== CONT TestExternalSignerErrors/exit_at_startup2672=== CONT TestExternalSignerErrors/unrequested_key2673=== CONT TestExternalSignerErrors/null_signatures2674=== CONT TestExternalSignerErrors/missing_signature_name2675--- PASS: TestParseSigningKey (0.00s)2676 --- PASS: TestParseSigningKey/invalid_base64 (0.00s)2677 --- PASS: TestParseSigningKey/empty_name (0.00s)2678 --- PASS: TestParseSigningKey/no_colon (0.00s)2679 --- PASS: TestParseSigningKey/wrong_length (0.00s)2680 --- PASS: TestParseSigningKey/valid_32-byte_key (0.00s)2681 --- PASS: TestParseSigningKey/valid_32-byte_key_with_different_name (0.00s)2682=== CONT TestExternalSignerErrors/program_error2683=== CONT TestExternalSignerErrors/too_few_signatures2684=== RUN TestSignErrors/nil_key2685=== PAUSE TestSignErrors/nil_key2686=== RUN TestSignErrors/nil_narinfo2687=== PAUSE TestSignErrors/nil_narinfo2688=== RUN TestSignErrors/invalid_narinfo2689=== PAUSE TestSignErrors/invalid_narinfo2690=== RUN TestSignErrors/canceled2691=== PAUSE TestSignErrors/canceled2692=== CONT TestSignErrors/nil_key2693=== CONT TestSignErrors/canceled2694=== CONT TestSignErrors/invalid_narinfo2695=== CONT TestSignErrors/nil_narinfo2696--- PASS: TestSignErrors (0.00s)2697 --- PASS: TestSignErrors/nil_key (0.00s)2698 --- PASS: TestSignErrors/canceled (0.00s)2699 --- PASS: TestSignErrors/invalid_narinfo (0.00s)2700 --- PASS: TestSignErrors/nil_narinfo (0.00s)2701--- PASS: TestSignBatch (0.01s)2702--- PASS: TestExternalSignerCancellation (0.24s)2703--- PASS: TestExternalSignerWrongPublicKey (0.44s)2704--- PASS: TestExternalSignerRestartsAfterCrash (1.21s)2705/private/var/tb/8/scratch/59e19724d113dbbe/build/TestExternalSignerRestartsAfterCrash2144667362/001/sign program: line 7: /private/var/tb/8/scratch/59e19724d113dbbe/build/TestExternalSignerRestartsAfterCrash2144667362/001/sign program.starts: No such file or directory2706--- PASS: TestExternalSigner (2.35s)2707--- PASS: TestExternalSignerProgramErrorKeepsProcess (2.52s)2708--- PASS: TestExternalSignerClose (3.10s)2709--- PASS: TestExternalSignerErrors (0.00s)2710 --- PASS: TestExternalSignerErrors/missing_signatures (0.62s)2711 --- PASS: TestExternalSignerErrors/null (0.81s)2712 --- PASS: TestExternalSignerErrors/wrong_signature_name (1.00s)2713 --- PASS: TestExternalSignerErrors/wrong_signature_length (1.37s)2714 --- PASS: TestExternalSignerErrors/exit_on_request (1.56s)2715 --- PASS: TestExternalSignerErrors/wrong_signatures_shape (1.76s)2716 --- PASS: TestExternalSignerErrors/invalid_JSON (1.94s)2717 --- PASS: TestExternalSignerErrors/wrong_id (2.14s)2718 --- PASS: TestExternalSignerErrors/too_few_signatures (2.70s)2719 --- PASS: TestExternalSignerErrors/wrong_shape (2.89s)2720 --- PASS: TestExternalSignerErrors/invalid_signature_base64 (3.34s)2721 --- PASS: TestExternalSignerErrors/missing_signature_name (3.53s)2722 --- PASS: TestExternalSignerErrors/exit_at_startup (3.71s)2723 --- PASS: TestExternalSignerErrors/unrequested_key (3.90s)2724 --- PASS: TestExternalSignerErrors/program_error (4.09s)2725 --- PASS: TestExternalSignerErrors/null_signatures (4.28s)2726PASS2727Running OpenBao signer tests...2728=== RUN TestSign2729=== PAUSE TestSign2730=== RUN TestSignErrors2731=== PAUSE TestSignErrors2732=== RUN TestServe2733=== PAUSE TestServe2734=== RUN TestServeRejectsGarbage2735=== PAUSE TestServeRejectsGarbage2736=== CONT TestSign2737=== CONT TestServe2738=== CONT TestSignErrors2739=== RUN TestSignErrors/http_error2740=== PAUSE TestSignErrors/http_error2741=== CONT TestServeRejectsGarbage2742=== RUN TestSignErrors/invalid_json2743=== PAUSE TestSignErrors/invalid_json2744--- PASS: TestServeRejectsGarbage (0.00s)2745=== RUN TestSignErrors/no_prefix2746=== PAUSE TestSignErrors/no_prefix2747=== RUN TestSignErrors/invalid_base642748=== PAUSE TestSignErrors/invalid_base642749=== CONT TestSignErrors/http_error2750=== CONT TestSignErrors/no_prefix2751=== CONT TestSignErrors/invalid_json2752--- PASS: TestServe (0.00s)2753=== CONT TestSignErrors/invalid_base642754--- PASS: TestSign (0.00s)2755--- PASS: TestSignErrors (0.00s)2756 --- PASS: TestSignErrors/http_error (0.00s)2757 --- PASS: TestSignErrors/invalid_json (0.00s)2758 --- PASS: TestSignErrors/no_prefix (0.00s)2759 --- PASS: TestSignErrors/invalid_base64 (0.00s)2760PASS