diff --git a/conf/proxy.toml b/conf/proxy.toml index fe93a686c..3b78cfff7 100644 --- a/conf/proxy.toml +++ b/conf/proxy.toml @@ -1,9 +1,13 @@ # workdir = "./work" -# Experimental out-of-process Rust SQL dataplane. When enabled, set -# enable-traffic-replay = false and start tiproxy-rs with the same absolute -# control socket, allowed UID, and TLS allowlist roots. -# enable-traffic-replay = false +# Experimental out-of-process Rust SQL dataplane. The Go default +# enable-traffic-replay = true is accepted by tiproxy-rs and SQL serving +# starts. Capture and replay are not implemented: the traffic API answers +# "unsupported" while the flag is true and "disabled" while it is false. +# The Go process still rejects rust-dataplane.enabled together with +# enable-traffic-replay. Start tiproxy-rs with the same absolute control +# socket, allowed UID, and TLS allowlist roots. +# enable-traffic-replay = true # [rust-dataplane] # enabled = true # control-socket = "/var/run/tiproxy/tiproxy-rust-control.sock" diff --git a/docs/design/rust-control-admin.md b/docs/design/rust-control-admin.md index 450272324..1714e0f0b 100644 --- a/docs/design/rust-control-admin.md +++ b/docs/design/rust-control-admin.md @@ -52,7 +52,7 @@ drains instead of continuing without an operator surface (Go only logs the | `DELETE /api/debug/health` | clears the override, `200 ""`. | | `GET /metrics`, `/metrics/`, `/api/metrics`, `/api/metrics/` | the B0 native exposition, `text/plain; version=0.0.4; charset=utf-8; escaping=underscores`. | | `GET /api/dataplane/status` | the Go key set in gin's sorted-map order; `desired_generation` and `sent_generation` are the latest composed generation (there is no transport hop in-process), `applied_generation`/`rejected_generation` come from SQL serving, `last_result_code` is `ERROR_CODE_INVALID_SNAPSHOT` when the latest rejection is newer than the latest apply, `ERROR_CODE_OK` after the first apply, `ERROR_CODE_UNSPECIFIED` before it. | -| `POST /api/traffic/{capture,replay,cancel}`, `GET /api/traffic/show` | `400 text/plain "traffic is disabled"`, the Go answer when `enable-traffic-replay = false`, which the Rust dataplane composition requires. | +| `POST /api/traffic/{capture,replay,cancel}`, `GET /api/traffic/show` | `400 text/plain`. The body is `traffic is unsupported` when `enable-traffic-replay` is true, and Go's `traffic is disabled` when it is false. The default is true and SQL serving still starts. | Health inputs: `closing` is the runtime lifecycle leaving `Ready`; the namespace-owner readiness is the config module's initial persistent view; @@ -288,7 +288,7 @@ certificate. every command's exit code and stdout: `health`, `config get`/`config set --input`, `namespace list`/`put`/`get`/`import`/`commit`/`del` including missing namespaces, and `traffic show`/`capture`/`replay`/`cancel`, which - both sides refuse with the disabled message while traffic replay is off. + both sides refuse. The evidence overlay sets `enable-traffic-replay = false`, so both answers are the disabled message. A default configuration answers `unsupported` on the native process. `config get` and `health` are compared semantically (the TOML renderers differ in layout and omitted zero fields); every other observation, including the CLI's `namespace list` failure on an empty set (the Go server diff --git a/docs/design/rust-control-config-namespace.md b/docs/design/rust-control-config-namespace.md index 06dd5322d..851e9dd91 100644 --- a/docs/design/rust-control-config-namespace.md +++ b/docs/design/rust-control-config-namespace.md @@ -123,9 +123,9 @@ old reader/election, and campaign with the new transport; owner-fenced writes remain unavailable throughout the handoff. Transport construction participates in prospective candidate validation, so an unsupported replacement retains the last-good source generation instead of publishing and then failing. In -particular, `cluster-tls.skip-ca = true` is rejected explicitly by the current -safe Rust etcd transport; it can never silently downgrade the owner connection -to plaintext. +particular, `cluster-tls.skip-ca = true` still opens a TLS connection and +does not require a CA. It does not downgrade the owner connection to +plaintext. Binary/build/process facts are not configuration: the composition root passes `TopologyRuntimeIdentity` directly to the CP-TOPO constructor. CP-TOPO combines @@ -208,10 +208,13 @@ would activate the value is rejected by the serving validator. | --- | --- | --- | | `workdir` | Restart | Modeled and checksummed; process/filesystem ownership is fixed at startup. | | `proxy.addr`, `proxy.advertise-addr`, `proxy.pd-addrs`, `proxy.port-range` | Restart | Modeled; SQL bind and topology identity are fixed at startup. | -| `api.addr`, `api.proxy-protocol` | Restart | Modeled; CP-ADMIN will own the HTTP listener. | +| `api.addr` | Restart | The native process binds it. `--admin-addr` overrides only that bind. | | `log.encoder`, `log.simple` | Restart | Modeled; encoder construction is fixed at startup. | -| `balance.routing-rule` | Restart | Modeled; listener-group construction is fixed at startup. | -| `ha.virtual-ip`, `ha.interface`, `ha.garp-burst-count`, `ha.garp-refresh-count` | Restart | Modeled; CP-HA consumes them at startup. | +| `api.proxy-protocol` | Restart, `v2` unsupported | Retained for the checksum. `v2` is rejected because the native admin listener does not apply PROXY protocol. Any other value is inert. | +| `balance.routing-rule` | Dynamic | Accepted for routers created after the reload. An existing router keeps the rule it was built with, matching the Go process. | +| `ha.virtual-ip` | Restart, applied | Excluded when the process selects an advertise address. | +| `ha.interface`, `ha.garp-burst-count`, `ha.garp-refresh-count` | Restart, inert | Retained. VIP ownership and GARP are not implemented. | +| `ha.virtual-ip` with `ha.interface` | Restart, unsupported | Go binds the VIP only when both are set. That activation is rejected. | | `metering.type`, `region`, `bucket`, `prefix`, `endpoint`, `shared-pool-id`; every field below `metering.aws`, `.oss`, `.cos`, `.azure`, `.localfs` | Restart | Modeled with secret-safe debug output; CP-METER consumes them at startup. | | `rust-dataplane.enabled`, `.control-socket`, `.routing-shadow-socket`, `.allowed-uid`, `.tls-allowed-roots`, `.metrics-owner-port` | Restart | Modeled; process/transport, TLS trust roots, observer socket, and metrics-owner bind cannot change online. A nonzero metrics-owner port must differ from every SQL/API listener port. | | `proxy.max-connections`, `proxy.high-memory-usage-reject-threshold`, `proxy.conn-buffer-size` | Dynamic | Applied by the Rust SQL serving snapshot. | @@ -219,16 +222,16 @@ would activate the value is rejected by the serving validator. | `proxy.proxy-protocol`, `.graceful-wait-before-shutdown`, `.graceful-close-conn-timeout`, `.public-endpoints` | Dynamic | Applied by the Rust SQL serving snapshot, including the latest graceful-close value at process drain. | | every `name`, `pd-addrs`, `ns-servers` below `proxy.backend-clusters` | Dynamic | Projected in stable order for CP-TOPO. | | `proxy.fail-backend-list`, `proxy.failover-timeout` | Dynamic | Retained for CP-ROUTE; no effect is claimed before #147. | -| `security.encryption-key-path` | Dynamic | Modeled and checksummed; CP-ADMIN consumes it. | +| `security.encryption-key-path` | Dynamic, inert | Retained, including a non-empty path, so a Go config still starts. Traffic capture is not implemented and does not read the key. | | `security.require-backend-tls` | Dynamic | Applied by the Rust SQL serving snapshot. | -| `cert`, `key`, `ca`, `min-tls-version`, `cert-allowed-cn`, `skip-ca` below each of `security.server-tls`, `.server-http-tls`, `.cluster-tls`, `.sql-tls` | Dynamic | All material is validated and content-watched atomically. SQL front/back is applied now; HTTP and cluster projections are ready for CP-ADMIN/CP-TOPO. `cluster-tls.skip-ca = true` is an explicit unsupported candidate while the Rust config owner uses the safe etcd transport. | +| `cert`, `key`, `ca`, `min-tls-version`, `cert-allowed-cn`, `skip-ca` below each of `security.server-tls`, `.server-http-tls`, `.cluster-tls`, `.sql-tls` | Dynamic | `Config.Check` accepts every minimum version, a relative path, and a certificate without its key. The effective minimum version follows Go `GetMinTLSVer`; the native handshake floor is TLS 1.2, and only a `1.3` suffix selects TLS 1.3 alone. A relative path is resolved against the process working directory. A configured `--tls-root` or `tls-allowed-roots` still confines the read; an empty allowlist does not add a jail. A certificate and key are still required together when a server identity is built. `cluster-tls.skip-ca = true` keeps the etcd connection on TLS and does not require a CA. | | `auto-certs` below global `server-tls` / `server-http-tls` and namespace `frontend.security` | Unsupported | Canonically round-tripped, then rejected explicitly because #146 does not generate server certificates. | | `auto-certs` below global `cluster-tls` / `sql-tls` and namespace `backend.security` | Dynamic, inert | Canonically round-tripped; the legacy client-side shape has no serving meaning. | | `rsa-key-size`, `autocert-expire-duration` below every global TLS block | Dynamic, inert unless auto-certs | Canonically round-tripped; because auto-certs is unsupported, these fields do not generate material in #146. | | `log.level`, every `filename`, `max-size`, `max-days`, `max-backups` below `log.log-file` | Dynamic | Level is applied by CP-001; file sink changes are retained for CP-ADMIN/log ownership. | | `balance.label-name`, `.policy`, `.routing-policy`; `migrations-per-second` below `.status`, `.health`, `.memory`, `.cpu`, `.location`, `.conn-count`; `.conn-count.count-ratio-threshold` | Dynamic | Validated and retained for CP-ROUTE. | | every `labels.` | Dynamic | Stable map projection for CP-TOPO/CP-ROUTE. | -| `enable-traffic-replay` | Dynamic, only `false` supported | Canonically modeled and reloadable, but enabling it is rejected by the serving validator because traffic capture remains M2 CP-CAPTURE #151. | +| `enable-traffic-replay` | Dynamic, inert | The Go default `true` is accepted and SQL serving starts. Capture and replay are not implemented: the traffic API answers `unsupported` when the flag is true and `disabled` when it is false. `rust-dataplane.enabled` together with this flag still fails `Config.Check`. | Namespace values have a separate, equally exact classification: @@ -320,12 +323,17 @@ where an unrelated first proxy update empties all namespaces. ## TLS rotation -TLS paths must be absolute and under configured allowed roots. Certificate and -key must be supplied together; minimum TLS is 1.2 or 1.3; allowed common names -are bounded and canonicalized. Candidate validation loads and parses every +A relative TLS path is resolved against the process working directory, the +same way Go `os.ReadFile` resolves it. When `--tls-root` or +`tls-allowed-roots` is non-empty, the canonical path must stay under one of +those roots. An empty allowlist has no extra jail. Minimum versions follow +Go `GetMinTLSVer`, with a native floor of TLS 1.2. A certificate without its +key is accepted by `Config.Check` and rejected when a server identity is +built, so it cannot become a plaintext listener. Allowed common names are +bounded and canonicalized. Candidate validation loads and parses every complete cert/key/CA set before publishing a source generation; the serving adapter then constructs and atomically swaps one complete immutable connection -factory. Partial, unreadable, invalid, or expired material is rejected without +factory. Unreadable, invalid, or expired material is rejected without publishing a partial set. A generation advances for an accepted config/namespace value change **or for a byte-level change to any referenced TLS material file**, even when every source revision and canonical config diff --git a/docs/design/rust-dataplane-parity.md b/docs/design/rust-dataplane-parity.md index e9029a0ba..5475c5861 100644 --- a/docs/design/rust-dataplane-parity.md +++ b/docs/design/rust-dataplane-parity.md @@ -121,7 +121,7 @@ configuration references in individual rows provide the executable evidence. | HS-006 | Preserve user, database, collation, zstd level, and connection attributes across backend authentication; allow the handshake hook to rewrite user/attributes and select a router. | `pkg/proxy/backend/authenticator.go`; `pkg/proxy/backend/handshake_handler.go`; `pkg/server/server.go:validateRustHandshakeHandler` | `pkg/proxy/backend/authenticator_test.go:TestCapability`, `TestCustomAuth`; `pkg/proxy/backend/backend_conn_mgr_test.go:TestConnAttrs`, `TestCustomHandshake`; corpus `handshake-response-native`, `handshake-response-modern`; Rust `rust/crates/dataplane/tests/session_engine.rs:compressed_client_zstd_roundtrips_end_to_end`, `change_user_success_commits_identity_clears_guard_and_migrates`; real-TiDB namespace matrix in `tests/dataplane/integration/run.sh`; `pkg/server/server_test.go:TestRustRouteOwnerRejectsCustomHandshakeHandler` | `session-core`, `control-proto`, `dataplane` | `PARITY-HS-006` | PARITY-VERIFIED (the supported production contract preserves user, database, collation, attributes, and the independently negotiated per-leg zstd level in the backend response; corpus and real-socket tests pin the encoded fields, and the real-TiDB matrix proves username-driven namespace selection and exact connection attribution in Rust mode. The remaining Go-only extension is not a Rust implementation gap: `RUST_ROUTE_OWNER` rejects every custom Go `HandshakeHandler` before listeners open, so its user/attribute mutation and arbitrary-router callback are explicitly outside the approved production boundary, as for HS-007. The native owner instead resolves the unmodified client username against its applied namespace snapshot and keeps that resolved namespace connection-stable.) | P0 | P0 | | HS-007 | Retry backend selection only when the handshake hook explicitly classifies a TiDB error as retryable; close and unregister each abandoned routed backend. | `pkg/proxy/backend/authenticator.go` (`RECONNECT`); `pkg/proxy/backend/backend_conn_mgr.go:abandonRoutedBackend`; `pkg/server/server.go:validateRustHandshakeHandler` | `pkg/proxy/backend/backend_conn_mgr_test.go:TestCustomHandshake`, `TestGetBackendIO`; `pkg/server/server_test.go:TestRustRouteOwnerRejectsCustomHandshakeHandler`; Rust `session_core::auth::tests::handler_reconnect_resets_the_relay`, `reconnect_swaps_backend_capability_for_compression` | `session-core`, `control-proto` | `PARITY-HS-007` | EXCLUDED (the production `RUST_ROUTE_OWNER` contract supports only the locally implemented default handshake policy and rejects a custom Go `HandshakeHandler` before listeners open; the default Go handler never approves `RECONNECT`, so no retry hook is reachable in Rust production. The pure Rust auth relay retains and tests the reference reconnect transition, but the runtime deliberately supplies `handler_reconnect=false`. Initial candidate dial retry and retirement remain covered by ADM-003; implementing a control-bridge callback solely for the unsupported custom-handler path would violate the approved route-owner boundary.) | P1 | P0 | | HS-008 | Translate handshake failures into the same client-visible MySQL errors and error-source labels, including capability, TLS, PROXY-v2, authentication, routing, and timeout failures. | `pkg/proxy/backend/error.go`; `pkg/proxy/backend/backend_conn_mgr.go:Connect`; `pkg/proxy/backend/authenticator.go:handleHandshakeError` | `pkg/proxy/backend/authenticator_test.go:TestUnsupportedCapability`, `TestRequireBackendTLS`, `TestProxyProtocol`, `TestAuthFail`; `pkg/proxy/backend/backend_conn_mgr_test.go:TestConnectFail`; Rust `session_core::error_source::tests::client_responses_pin_code_state_and_message`, `session_core::auth::tests::backend_tls_planning_matches_required_and_opportunistic_matrix`; `rust/crates/dataplane/tests/session_engine.rs:frontend_greeting_and_pre_read_handshake_limit_are_production_wired`, `frontend_missing_protocol41_is_rejected_before_backend_routing`, `backend_greeting_parser_and_required_capability_gate_are_production_wired`, `backend_tls_failures_use_go_client_errors_and_source_label`, `wrong_password_reports_using_password`, `no_backend_route_answer_refuses_the_client`; `dataplane::session_engine::error_classification_tests::route_timeout_preserves_go_last_failure_semantics` | `session-core`, `dataplane` | `PARITY-HS-008` | PARITY-VERIFIED (the production engine now routes all fixed Go `ErrToClient` handshake sentinels through the typed allowlist: pre-read oversize is 1153/08S01; frontend capability remains 1251/08004; backend capability, backend greeting/dial, required TLS, TLS/PROXY-v2, and no-backend use their exact 1105/HY000 vocabulary. Already-forwarded backend MySQL authentication errors are never duplicated. Each terminal also pins the matching Go quit label, including client-auth, backend-handshake, and proxy-no-backend; connect-budget expiry preserves Go's last-failure rule, while a control wait with no candidate remains a silent proxy error rather than a false no-backend label.) | P1 | P0 | -| HS-009 | Support direct backend authentication for replay/test clients with generated native or caching-SHA auth responses and optional initial database. This path remains test/topology support, not traffic capture. | `pkg/proxy/backend/authenticator.go:handshakeWithBackend`; `pkg/sqlreplay/conn/backend_conn.go:Connect`; `rust/crates/dataplane/src/server.rs:preflight_snapshot` | `pkg/proxy/backend/authenticator_test.go:TestHandshakeWithBackend`; `pkg/proxy/backend/backend_conn_mgr_test.go:TestConnectWithBackend`; Rust `dataplane::server::tests::capture_preflight_and_accept_backoff_are_explicit`; `rust/crates/control-proto/tests/snapshot_store.rs:expiry_ca_tls_policy_and_unsupported_configuration_are_rejected_atomically` | `session-core`, `dataplane`, `control-proto` | `PARITY-HS-009` | EXCLUDED (the only production caller is SQL traffic replay's synthetic `BackendConn`, and EXCL-001 explicitly excludes the entire capture/replay subsystem from Rust mode. Both snapshot validation and server preflight reject `traffic_replay_enabled` before a SQL listener can serve, so no Rust replay/test client can reach a direct-backend-auth path. Implementing password-based auth generation solely for that unreachable path would contradict the approved exclusion; ordinary traffic authentication remains covered by HS-003/004/005/006.) | P2 | P2 | +| HS-009 | Support direct backend authentication for replay/test clients with generated native or caching-SHA auth responses and optional initial database. This path remains test/topology support, not traffic capture. | `pkg/proxy/backend/authenticator.go:handshakeWithBackend`; `pkg/sqlreplay/conn/backend_conn.go:Connect`; `rust/crates/dataplane/src/server.rs:preflight_snapshot` | `pkg/proxy/backend/authenticator_test.go:TestHandshakeWithBackend`; `pkg/proxy/backend/backend_conn_mgr_test.go:TestConnectWithBackend`; Rust `dataplane::server::tests::capture_preflight_and_accept_backoff_are_explicit`; `rust/crates/control-proto/tests/snapshot_store.rs:expiry_ca_tls_policy_and_unsupported_configuration_are_rejected_atomically` | `session-core`, `dataplane`, `control-proto` | `PARITY-HS-009` | EXCLUDED (the only production caller is SQL traffic replay's synthetic `BackendConn`, and EXCL-001 explicitly excludes the entire capture/replay subsystem from Rust mode. The default `traffic_replay_enabled` value is accepted so a Go-compatible configuration can serve SQL. Capture and replay remain unimplemented, so no Rust replay client reaches the direct-backend-auth path. Implementing password-based auth generation solely for that unreachable path would contradict the approved exclusion; ordinary traffic authentication remains covered by HS-003/004/005/006.) | P2 | P2 | | HS-010 | Advertise the exact Go capability set: LONG_PASSWORD, FOUND_ROWS, LONG_FLAG, CONNECT_WITH_DB, ODBC, LOCAL_FILES, INTERACTIVE, PROTOCOL_41, SSL when configured, TRANSACTIONS, RESERVED, SECURE_CONNECTION, MULTI_STATEMENTS, MULTI_RESULTS, PLUGIN_AUTH, CONNECT_ATTRS, PLUGIN_AUTH_LENENC_CLIENT_DATA, DEPRECATE_EOF, COMPRESS, and ZSTD. Unsupported client flags are logged and masked. | `pkg/proxy/backend/authenticator.go:SupportedServerCapabilities` | `pkg/proxy/backend/authenticator_test.go:TestUnsupportedCapability`, `TestCapability`, `TestCompressProtocol`; `rust/crates/dataplane/tests/session_engine.rs:compressed_client_{zlib,zstd}_roundtrips_end_to_end` (COMPRESS + ZSTD are advertised and negotiate); `rust/crates/tiproxy-rs/src/main.rs:integration_capabilities_are_the_honest_contract` | `session-core` | `PARITY-HS-010` | GO-VERIFIED / RUST-IMPLEMENTED (WIRE-C: `proxy_capability_base` now advertises COMPRESS + ZSTD alongside the full set; unsupported client flags remain masked) | P0 | P0 | | HS-011 | Parse the backend greeting's version, connection ID, salt, capability halves, status, collation, and auth plugin before capability/TLS decisions; malformed/error greetings must not panic or allocate unbounded memory. | `pkg/proxy/net/mysql.go:ParseInitialHandshake`; `pkg/proxy/backend/authenticator.go:readInitialHandshake` | `pkg/proxy/net/mysql_test.go:TestHandshakeResp`; negative Go backend-greeting coverage is missing; Rust `initial_handshake_round_trip_and_borrowing`, `hostile_handshake_prefixes_never_panic`, `parity_hs_011_malformed_greeting_prefixes_are_safe`, `backend_greeting_parser_and_required_capability_gate_are_production_wired` | `mysql-wire`, `session-core` | `PARITY-HS-011` | GO-IMPLEMENTED / RUST-VERIFIED (the allocation-free parser validates protocol, filler, every fixed/variable field and every truncated prefix. A routed production dial accepts only a parsed greeting before capability/TLS decisions and fails closed on both a well-formed greeting missing a negotiated requirement and malformed protocol bytes; Go still lacks the row's negative greeting test, so this does not overstate Go-side verification) | P0 | P0 | @@ -224,7 +224,7 @@ RSP-001 through RSP-008 where applicable. | KA-002 | Apply conservative keepalive to healthy backends and aggressive keepalive to unhealthy backends; re-evaluate on health ticker and immediately after migration. | `pkg/proxy/backend/backend_conn_mgr.go:setKeepAlive`; `lib/config/proxy.go:DefaultKeepAlive` | `pkg/proxy/backend/backend_conn_mgr_test.go:TestKeepAlive`, `TestBackendStatusChange`; `rust/crates/dataplane/src/server.rs:existing_session_keeps_admission_snapshot_and_observes_live_generation`; `rust/crates/dataplane/src/session_engine.rs:live_backend_health_requires_the_exact_backend_identity`; `rust/crates/dataplane/tests/session_engine.rs:existing_session_reapplies_keepalive_on_backend_health_flip` | `dataplane`, `control-proto` | `PARITY-KA-002` | GO-VERIFIED / RUST-VERIFIED (the single socket owner consumes complete last-good generations, matches exact backend id/address/cluster, treats absence or mismatch as unhealthy, re-selects the immutable admission policy at command-safe boundaries and immediately after attach/swap, retries best-effort failures every five seconds, and exports a bounded success/failure counter) | P1 | P0 | | KA-003 | Probe an idle backend at `CheckBackendInterval`; if peer EOF is observed, classify backend-network exit and close the client. Do not probe concurrently with command I/O. | `pkg/proxy/net/packetio.go:IsPeerActive`; `pkg/proxy/backend/backend_conn_mgr.go:checkBackendActive` | `pkg/proxy/net/packetio_test.go:TestPeerActive`; `pkg/proxy/backend/backend_conn_mgr_test.go:TestBackendInactive`; `rust/crates/dataplane/tests/session_engine.rs:idle_probe_consumed_byte_is_counted_in_raw_backend_in`, `existing_session_reapplies_keepalive_on_backend_health_flip`; `rust/crates/dataplane/tests/session_loop.rs` | `proxy-io`, `dataplane` | `PARITY-KA-003` | GO-VERIFIED / RUST-VERIFIED (idle-safe probing and health-driven keepalive both run on the single socket owner, never concurrently with command I/O; peer EOF is classified as backend-network exit and live health changes preserve command framing) | P1 | P0 | | MTR-001 | After handshake and every completed/held command, account backend inbound/outbound bytes and physical packets by backend; count cross-location traffic in both directions. | `pkg/proxy/backend/backend_conn_mgr.go:updateTraffic`; `pkg/proxy/backend/metrics.go:addTraffic`; `pkg/metrics/traffic.go` | `pkg/proxy/backend/backend_conn_mgr_test.go:TestTrafficMetrics`; `rust/crates/dataplane/tests/session_engine.rs:session_path_emits_query_traffic_and_exact_quit_source`, `outbound_proxy_v2_header_counts_once_in_raw_backend_out`; `rust/crates/proxy-io/tests/counted.rs`; `rust/crates/proxy-io/tests/tls_matrix.rs:raw_counter_below_tls_counts_record_bytes_not_plaintext`; `dataplane::observability::histogram_wire_semantics_are_exact_and_cumulative`; `pkg/metrics:TestRustMetricsHistogramMergesExactBuckets` | `dataplane` | `PARITY-MTR-001` | GO-VERIFIED / RUST-VERIFIED (every clause is carried by the production path. **After handshake and after every completed or held command:** `Observation::HandshakeCompleted`, and `record_command`, which fires on all four `HoldFlow` branches (`Answered`, `Dropped`, `Fatal`, `Replay`) as well as the unheld path, so a held command is accounted once after it finally runs, as Go's `updateTraffic` does at `backend_conn_mgr.go:474` and `:519`. **Per-backend inbound/outbound bytes and physical packets:** `observability::traffic` emits the four `{backend}` counters. The byte/packet layering is Go's own, not a divergence: bytes are counted at the innermost `CountedIo` raw-socket seam as Go counts them in `basicReadWriter`, so TLS-record, PROXY and compression overhead is included, and physical packets at the framing layer as Go counts them in `packetIO` (`p.inPackets++` after a physical header). **Cross-location in both directions:** `if !local { cross_location_bytes += inbound + outbound }`, mirroring Go's `addTraffic`, fed from `assignment.local` through `backend.local`. Discriminating evidence: removing the cross-location emission, and dropping the `!local` guard so local traffic counts too, each fail `dataplane::observability::tests::native_exposition_matches_go_golden`, so the golden holds both a local and a non-local backend. **Scope:** Go calls `updateTraffic` from five sites and this row owns three. The other two, accounting the old socket on a migration swap and resetting counters for the new one (`backend_conn_mgr.go:715`, `:717`), are MIG-007's requirement and rest on MIG-007's evidence, not on this row) | P1 | P1 | -| MTR-002 | Meter per-keyspace response bytes, cross-AZ bytes, and public/private origin. The keyspace is connection-stable and deltas must remain correct across migration. | `pkg/proxy/backend/backend_conn_mgr.go:updateTraffic`; `pkg/proxy/backend/backend_conn_mgr.go:Meter`; `pkg/manager/meter/` | `pkg/proxy/backend/backend_conn_mgr_test.go:TestTrafficMetrics`; `pkg/manager/meter/meter_test.go:TestWrite`, `TestLoop`; `rust/crates/dataplane/src/metering.rs:native_sampler_delivers_final_batch_without_control_bridge`, `public_endpoint_uses_direct_peer_cidrs`, `unknown_keyspace_is_session_scoped_not_global_failure`; `rust/crates/dataplane/tests/session_engine.rs:metering_redirect_unknown_keeps_old_and_success_starts_new_generation`, `metering_unknown_initial_keyspace_rejects_before_billable_forwarding` | `dataplane`, `control-proto` | `PARITY-MTR-002` | GO-VERIFIED / RUST-VERIFIED (the three metered quantities match Go `updateTraffic` arithmetic exactly: the response-byte figure is the *inbound* delta (10), `cross_az_bytes` is inbound+outbound (30) and is charged only when the backend is not `local`, and the public/private split routes the same bytes to `public_response_bytes` by the direct accepted peer's CIDR class rather than by any forwarded header. The keyspace is a per-attribution label and an empty one is never billable -- Go skips `IncTraffic` under `len(keyspace) > 0`, Rust refuses to register the source at all and the refusal is session-scoped, not a global meter failure. **Across migration** the deltas do not merge: redirecting a session from `keyspace-a` to `keyspace-b` leaves the old generation's final source and the new generation's source coexisting until the WAL handoff, so neither generation's bytes land under the other's keyspace) | P1 | P0 | +| MTR-002 | Meter per-keyspace response bytes, cross-AZ bytes, and public/private origin. The keyspace is connection-stable and deltas must remain correct across migration. | `pkg/proxy/backend/backend_conn_mgr.go:updateTraffic`; `pkg/proxy/backend/backend_conn_mgr.go:Meter`; `pkg/manager/meter/` | `pkg/proxy/backend/backend_conn_mgr_test.go:TestTrafficMetrics`; `pkg/manager/meter/meter_test.go:TestWrite`, `TestLoop`; `rust/crates/dataplane/src/metering.rs:native_sampler_delivers_final_batch_without_control_bridge`, `public_endpoint_uses_direct_peer_cidrs`, `unknown_keyspace_is_session_scoped_not_global_failure`; `rust/crates/dataplane/tests/session_engine.rs:metering_redirect_unknown_keeps_old_and_success_starts_new_generation`, `metering_classic_empty_keyspace_serves_without_a_billable_source` | `dataplane`, `control-proto` | `PARITY-MTR-002` | GO-VERIFIED / RUST-VERIFIED (the three metered quantities match Go `updateTraffic` arithmetic exactly: the response-byte figure is the *inbound* delta (10), `cross_az_bytes` is inbound+outbound (30) and is charged only when the backend is not `local`, and the public/private split routes the same bytes to `public_response_bytes` by the direct accepted peer's CIDR class rather than by any forwarded header. The keyspace is a per-attribution label and an empty one is never billable -- Go skips `IncTraffic` under `len(keyspace) > 0` and still serves the session. Rust detaches metering for that classic topology, so the handshake succeeds and no source is registered. A direct register of an empty keyspace remains a session-scoped `UnknownAttribution`, not a global meter failure. **Across migration** the deltas do not merge: redirecting a session from `keyspace-a` to `keyspace-b` leaves the old generation's final source and the new generation's source coexisting until the WAL handoff, so neither generation's bytes land under the other's keyspace) | P1 | P0 | | MTR-003 | Preserve command-count and duration labels, handshake/backend-selection duration, connection age/lifetime, connection gauge/create/disconnect/reject counters, backend keepalive update outcomes, and error-source labels with their existing Prometheus names. | `pkg/proxy/backend/metrics.go`; `pkg/metrics/session.go`, `server.go`, `backend.go`, `metrics.go`, `rust_metrics.go`; `pkg/proxy/proxy.go`; `pkg/proxy/client/client_conn.go` | `pkg/proxy/backend/backend_conn_mgr_test.go:TestTrafficMetrics`; `rust/crates/dataplane/tests/session_engine.rs:session_path_emits_query_traffic_and_exact_quit_source`, `existing_session_reapplies_keepalive_on_backend_health_flip`; `pkg/metrics:TestRustMetricsCounterSequenceDedup`, `TestRustMetricsGaugeReconcilesAcrossEpoch`, `TestRustMetricsHistogramMergesExactBuckets`, `TestRustMetricsBackendKeepAliveUpdateMerges`; `dataplane::observability::queue_full_is_non_blocking_and_counted`, `backend_keepalive_update_metric_uses_bounded_labels`, `native_exposition_matches_go_golden`, `registry_is_the_cumulative_twin_of_the_bridge_deltas`; `tests/dataplane/metrics/gen` (Go `promhttp` oracle over `tests/dataplane/metrics/native-families.json`, gated by `make dataplane-metrics-parity`); `tiproxy-rs::metrics_http::routes_metrics_paths_and_rejects_others` | `dataplane`, `control-proto` | `PARITY-MTR-003` | GO-IMPLEMENTED / RUST-VERIFIED (the native Rust exposition on `--metrics-addr` (`/metrics`, `/api/metrics`) is the only production path. Checked against Go source on main `27689339`, in four layers: **family names** -- Go 35, Rust 32, the difference exactly MTR-006's three retirements and nothing added by Rust; **kinds** and **label sets** -- all 32 families match once Go's label constants are resolved (`LblMetricName`, `LblFactor`, `LblReason` ...) and label order is normalised, since declaration order only fixes `WithLabelValues` positions and not the exposed identity; **label values** -- closed by construction at the emission sites rather than by a validator: `migrate_res` and `res` come from a `bool` (`migrate_result`, Go `succeedToLabel`), and `reason` from an exhaustive enum whose seven values equal Go's seven factor `Name()`s, with the `test` redirect reason present on both sides (Go `group.go:RedirectConnections`). Discriminating evidence: `native_exposition_matches_go_golden` fails on renaming a family, on inverting `migrate_result`, and on swapping two factor names; the last also fails `control-router::ledger::redirect_tests::a_failed_migration_is_still_observed_and_a_self_redirect_reports_test`. **Historical:** the Go bridge (`pkg/metrics/rust_metrics.go:validateRustMetric`, called from `controlbridge/drain_metering.go`) validates a closed catalog strictly, but it serves the retired `MetricsBatch` wire contract. Since CP-ADMIN slice 5c Rust drops its per-tick deltas once the native registry has absorbed them, so that validator guards a channel that carries no traffic and is not this row's evidence) | P1 | P1 | | MTR-004 | Preserve migration success/failure/pending metrics and backend connection gauges, produced from the Rust dataplane's own routing lifecycle events. **Narrowed 2026-09-23 (owner decision):** this row used to require these families *as Go control-plane observations of Rust lifecycle events*. #223 Phase 2 removed that mechanism -- `ConnectionEvent` is a tombstone and no Rust terminal feeds a Go router -- so the families are still required and only their producer changed. | `pkg/metrics/balance.go`; `pkg/balance/router/` (the `pkg/controlbridge/router_adapter.go` producer was deleted at #223 Phase 2) | `pkg/balance/router/router_score_test.go:TestRebalance`, `TestRedirectFail`, `TestCloseRaceWithRedirectSucceed`, `TestCloseRaceWithRedirectFail`; `rust/crates/dataplane/tests/session_engine.rs:redirect_restores_candidate_and_swaps_atomically`, `candidate_failures_preserve_old_backend` | `control-proto`, `dataplane` | `PARITY-MTR-004` | GO-IMPLEMENTED / RUST-VERIFIED against the narrowed contract (all four families come from live Rust routing events in `control-router` and never cross `MetricsBatch` or the Go bridge. `pending_migrate`: `admit_redirect` -> `issue_migration` -> `MigrationHistory::remember`. `migrate_total` and `migrate_duration_seconds`: `finish_redirect` -> `settle_migration` -> `MigrationHistory::settle`, and `close` settles a migration interrupted mid-flight as a failure. `b_conn`: each router's `physical_connections()`, summed across routers into `MigrationSnapshot.backend_connections`. Event-to-value evidence, in `control-router::ledger::redirect_tests`: `a_settled_migration_reports_the_reason_frozen_at_issue_and_its_elapsed_time` pins the `Issued` observation at admission and `Settled { success: true, elapsed: 250ms }` at settlement, with a destructive drain; `review_close_settles_accepted_migration_once` pins exactly one failed terminal on `close`; `a_duration_exactly_on_a_bucket_bound_belongs_to_that_bucket` pins inclusive `le` bucket bounds -- before it existed, changing `seconds <= bound` to `seconds < bound` left every test in the workspace passing; `physical_connections_follow_gos_conn_list_exactly` pins the per-ledger count, and `control-router::tests` pins the exported cross-router total (`Some(2)` for two routers sharing an address). **`b_conn` is not Go's value:** Go writes it with `Set` from one namespace's router, so the last writer wins across namespaces sharing an address; Rust sums, and that difference is MTR-007's declared divergence rather than a claim of equality here. Family, kind and label evidence is MTR-003's) | P1 | P0 | | MTR-005 | Attach connection ID, listener address, real/proxy client address, namespace, backend address, capabilities, and quit source to the same lifecycle/error logs without auth payloads or secrets. | `pkg/proxy/proxy.go:onConn`; `pkg/proxy/backend/backend_conn_mgr.go:ConnInfo`; `pkg/proxy/backend/authenticator.go:ConnInfo` | `pkg/proxy/backend/backend_conn_mgr_test.go:TestConnID`, `TestConnAttrs`, `TestDisconnectLog`; `dataplane::observability::log_schema_is_closed_payload_free_and_bounded`; `dataplane::session_engine::proxy_client_log_address_prefers_inbound_inet_source_and_falls_back_to_peer`, `proxy_client_log_source_is_shared_with_the_force_close_owner`; `rust/crates/dataplane/tests/session_engine.rs:session_path_emits_query_traffic_and_exact_quit_source`, `inbound_proxy_v2_header_and_tlvs_are_forwarded_verbatim`; `tests/dataplane/integration/run.sh:MTR-005 lifecycle-log attribution` | `dataplane`, `control-proto` | `PARITY-MTR-005` | GO-VERIFIED / RUST-VERIFIED (lifecycle logs keep `client_addr` as the accepted TCP peer and use the parsed inbound PROXY inet source for `proxy_client_addr`, falling back to the peer for direct, LOCAL, UNSPEC, and Unix-address sessions; the immutable source is shared with the close owner so force-abort cannot regress attribution, while public/private metering remains keyed by Go-compatible TCP-peer `ProxyAddr()` semantics) | P2 | P1 | diff --git a/pkg/controlbridge/snapshot.go b/pkg/controlbridge/snapshot.go index d6a52b648..20ad01e65 100644 --- a/pkg/controlbridge/snapshot.go +++ b/pkg/controlbridge/snapshot.go @@ -8,7 +8,6 @@ import ( "fmt" "math" "net" - "net/netip" "os" "path/filepath" "slices" @@ -19,6 +18,7 @@ import ( "github.com/pingcap/tiproxy/lib/config" controlpb "github.com/pingcap/tiproxy/pkg/controlbridge/pb" pnet "github.com/pingcap/tiproxy/pkg/proxy/net" + "github.com/pingcap/tiproxy/pkg/util/netutil" "google.golang.org/protobuf/proto" ) @@ -96,18 +96,9 @@ func (builder *SnapshotBuilder) Build( return nil, ErrListenerRestartRequired } - frontendKeepalive, err := buildKeepalive("proxy.frontend-keepalive", normalized.Proxy.FrontendKeepalive) - if err != nil { - return nil, err - } - healthyKeepalive, err := buildKeepalive("proxy.backend-healthy-keepalive", normalized.Proxy.BackendHealthyKeepalive) - if err != nil { - return nil, err - } - unhealthyKeepalive, err := buildKeepalive("proxy.backend-unhealthy-keepalive", normalized.Proxy.BackendUnhealthyKeepalive) - if err != nil { - return nil, err - } + frontendKeepalive := buildKeepalive(normalized.Proxy.FrontendKeepalive) + healthyKeepalive := buildKeepalive(normalized.Proxy.BackendHealthyKeepalive) + unhealthyKeepalive := buildKeepalive(normalized.Proxy.BackendUnhealthyKeepalive) frontendTLS, err := builder.buildTLSPolicy("security.server-tls", normalized.Security.ServerSQLTLS) if err != nil { return nil, err @@ -116,10 +107,7 @@ func (builder *SnapshotBuilder) Build( if err != nil { return nil, err } - publicCIDRs, err := normalizeCIDRs(normalized.Proxy.PublicEndpoints) - if err != nil { - return nil, fmt.Errorf("validate proxy.public-endpoints: %w", err) - } + publicCIDRs := normalizeCIDRs(normalized.Proxy.PublicEndpoints) gracefulWait, err := secondsToMillis("proxy.graceful-wait-before-shutdown", normalized.Proxy.GracefulWaitBeforeShutdown) if err != nil { return nil, err @@ -200,39 +188,34 @@ func buildListeners(cfg *config.Config) ([]*controlpb.Listener, error) { return listeners, nil } -func buildKeepalive(field string, keepalive config.KeepAlive) (*controlpb.KeepalivePolicy, error) { - if keepalive.Idle < 0 || keepalive.Intvl < 0 || keepalive.Timeout < 0 || keepalive.Cnt < 0 { - return nil, fmt.Errorf("%s values must be nonnegative", field) - } - if keepalive.Cnt > math.MaxUint32 { - return nil, fmt.Errorf("%s probe count exceeds uint32", field) - } - idle, err := durationMillis(field+".idle", keepalive.Idle) - if err != nil { - return nil, err - } - interval, err := durationMillis(field+".intvl", keepalive.Intvl) - if err != nil { - return nil, err - } - timeout, err := durationMillis(field+".timeout", keepalive.Timeout) - if err != nil { - return nil, err - } +func buildKeepalive(keepalive config.KeepAlive) *controlpb.KeepalivePolicy { + // Go Config.Check does not reject keepalive numbers. The socket layer + // applies only a positive duration and a positive probe count, and a + // duration is truncated to whole milliseconds. return &controlpb.KeepalivePolicy{ Enabled: keepalive.Enabled, - IdleMillis: idle, - ProbeCount: uint32(keepalive.Cnt), - IntervalMillis: interval, - UserTimeoutMillis: timeout, - }, nil + IdleMillis: durationMillis(keepalive.Idle), + ProbeCount: probeCount(keepalive.Cnt), + IntervalMillis: durationMillis(keepalive.Intvl), + UserTimeoutMillis: durationMillis(keepalive.Timeout), + } +} + +func durationMillis(value time.Duration) uint64 { + if value < time.Millisecond { + return 0 + } + return uint64(value / time.Millisecond) } -func durationMillis(field string, value time.Duration) (uint64, error) { - if value > 0 && value < time.Millisecond { - return 0, fmt.Errorf("%s must be zero or at least one millisecond", field) +func probeCount(count int) uint32 { + if count <= 0 { + return 0 } - return uint64(value / time.Millisecond), nil + if uint64(count) > math.MaxUint32 { + return math.MaxUint32 + } + return uint32(count) } func secondsToMillis(field string, value int) (uint64, error) { @@ -250,12 +233,6 @@ func (builder *SnapshotBuilder) buildTLSPolicy(field string, cfg config.TLSConfi if cfg.AutoCerts { return nil, fmt.Errorf("%s.auto-certs is unsupported by the Rust dataplane; use shared certificate files", field) } - if (cfg.Cert == "") != (cfg.Key == "") { - return nil, fmt.Errorf("%s cert and key must be configured together", field) - } - if cfg.MinTLSVersion != "" && cfg.MinTLSVersion != "1.2" && cfg.MinTLSVersion != "1.3" { - return nil, fmt.Errorf("%s minimum TLS version must be 1.2 or 1.3", field) - } paths := []struct { name string value string @@ -289,20 +266,42 @@ func (builder *SnapshotBuilder) buildTLSPolicy(field string, cfg config.TLSConfi CertificatePath: resolved["cert"], PrivateKeyPath: resolved["key"], CaPath: resolved["ca"], - MinimumVersion: cfg.MinTLSVersion, + MinimumVersion: effectiveMinTLSVersion(cfg.MinTLSVersion), AllowedCommonNames: allowedCN, SkipCaVerification: cfg.SkipCA, }, nil } +func effectiveMinTLSVersion(value string) string { + switch { + case strings.HasSuffix(value, "1.0"): + return "1.0" + case strings.HasSuffix(value, "1.1"): + return "1.1" + case strings.HasSuffix(value, "1.2"): + return "1.2" + case strings.HasSuffix(value, "1.3"): + return "1.3" + default: + return "1.2" + } +} + func (builder *SnapshotBuilder) validateTLSPath(path string) (string, error) { if !filepath.IsAbs(path) { - return "", fmt.Errorf("path must be absolute") + absolute, err := filepath.Abs(path) + if err != nil { + return "", fmt.Errorf("resolve path: %w", err) + } + path = absolute } canonical, err := filepath.EvalSymlinks(filepath.Clean(path)) if err != nil { return "", fmt.Errorf("resolve path: %w", err) } + if len(builder.allowedTLSRoots) == 0 { + return canonical, nil + } for _, root := range builder.allowedTLSRoots { relative, err := filepath.Rel(root, canonical) if err == nil && relative != ".." && !strings.HasPrefix(relative, ".."+string(filepath.Separator)) { @@ -312,22 +311,15 @@ func (builder *SnapshotBuilder) validateTLSPath(path string) (string, error) { return "", fmt.Errorf("path is outside configured TLS roots") } -func normalizeCIDRs(values []string) ([]string, error) { - result := make([]string, 0, len(values)) - for _, value := range values { - value = strings.TrimSpace(value) - prefix, err := netip.ParsePrefix(value) - if err != nil { - address, addrErr := netip.ParseAddr(value) - if addrErr != nil { - return nil, fmt.Errorf("invalid IP or CIDR %q", value) - } - prefix = netip.PrefixFrom(address, address.BitLen()) - } - result = append(result, prefix.Masked().String()) +func normalizeCIDRs(values []string) []string { + // Same parser the SQL server uses: bare addresses become /32, invalid + // entries are skipped, and the survivors keep their order and duplicates. + cidrs, _ := netutil.ParseCIDRList(values) + result := make([]string, 0, len(cidrs)) + for _, cidr := range cidrs { + result = append(result, cidr.String()) } - slices.Sort(result) - return slices.Compact(result), nil + return result } func cloneListeners(listeners []*controlpb.Listener) []*controlpb.Listener { diff --git a/pkg/controlbridge/snapshot_test.go b/pkg/controlbridge/snapshot_test.go index 91d6a84e8..8cb603b6d 100644 --- a/pkg/controlbridge/snapshot_test.go +++ b/pkg/controlbridge/snapshot_test.go @@ -57,7 +57,7 @@ func TestSnapshotBuilderBuildsCompleteDeterministicSnapshot(t *testing.T) { require.Equal(t, controlpb.ProxyProtocolMode_PROXY_PROTOCOL_MODE_V2, actual.GetProxyProtocol()) require.Equal(t, uint64(7000), actual.GetGracefulWaitMillis()) require.Equal(t, uint64(11000), actual.GetGracefulCloseMillis()) - require.Equal(t, []string{"10.0.0.0/24", "10.0.0.9/32", "2001:db8::1/128"}, actual.GetPublicCidrs()) + require.Equal(t, []string{"2001:db8::/32", "10.0.0.9/32", "10.0.0.0/24", "10.0.0.0/24"}, actual.GetPublicCidrs()) require.Equal(t, uint32(0xaabb), actual.GetAdvertisedCapability()) require.Equal(t, "TiProxy-test", actual.GetServerVersion()) require.Equal(t, []string{"client-a", "client-b"}, actual.GetFrontendTls().GetAllowedCommonNames()) @@ -90,13 +90,16 @@ func TestSnapshotBuilderRejectsUnsafeOrRestartRequiredValues(t *testing.T) { invalidKeepalive := cfg.Clone() invalidKeepalive.Proxy.FrontendKeepalive.Idle = -time.Second - _, err = builder.Build(1, invalidKeepalive, 0, "test", nil, nil) - require.ErrorContains(t, err, "nonnegative") + invalidKeepalive.Security.ServerSQLTLS.MinTLSVersion = "1.0" + envelope, err := builder.Build(1, invalidKeepalive, 0, "test", nil, nil) + require.NoError(t, err) + require.Equal(t, uint64(0), envelope.GetStateSnapshot().GetConfig().GetFrontendKeepalive().GetIdleMillis()) + require.Equal(t, "1.0", envelope.GetStateSnapshot().GetConfig().GetFrontendTls().GetMinimumVersion()) partialTLS := cfg.Clone() partialTLS.Security.ServerSQLTLS.Cert = outside _, err = builder.Build(1, partialTLS, 0, "test", nil, nil) - require.ErrorContains(t, err, "configured together") + require.ErrorContains(t, err, "outside configured TLS roots") unsafeTLS := cfg.Clone() unsafeTLS.Security.ServerSQLTLS.Cert = outside @@ -119,8 +122,11 @@ func TestSnapshotBuilderRejectsInvalidAllowlistAndSubMillisecondKeepalive(t *tes builder, err := NewSnapshotBuilder(cfg, []string{t.TempDir()}) require.NoError(t, err) cfg.Proxy.FrontendKeepalive.Idle = time.Microsecond - _, err = builder.Build(1, cfg, 0, "test", nil, nil) - require.ErrorContains(t, err, "at least one millisecond") + cfg.Security.ServerSQLTLS.MinTLSVersion = "nope" + envelope, err := builder.Build(1, cfg, 0, "test", nil, nil) + require.NoError(t, err) + require.Equal(t, uint64(0), envelope.GetStateSnapshot().GetConfig().GetFrontendKeepalive().GetIdleMillis()) + require.Equal(t, "1.2", envelope.GetStateSnapshot().GetConfig().GetFrontendTls().GetMinimumVersion()) } func writeSnapshotTestFile(t *testing.T, directory, name string) string { diff --git a/pkg/manager/config/config.go b/pkg/manager/config/config.go index 87a8d0790..8391bf8a6 100644 --- a/pkg/manager/config/config.go +++ b/pkg/manager/config/config.go @@ -18,10 +18,13 @@ func (e *ConfigManager) reloadConfigFile(file string) error { if err != nil { return errors.WithStack(err) } - if bytes.Equal(content, e.fileContent) { + // An empty file is not equal to the initial nil buffer. The first read + // installs defaults, matching a process started without a config file. + if e.fileLoaded && bytes.Equal(content, e.fileContent) { return nil } e.fileContent = content + e.fileLoaded = true return e.SetTOMLConfig(content) } diff --git a/pkg/manager/config/config_test.go b/pkg/manager/config/config_test.go index 8569553d4..7d6eb5599 100644 --- a/pkg/manager/config/config_test.go +++ b/pkg/manager/config/config_test.go @@ -317,6 +317,16 @@ func TestFilePath(t *testing.T) { } } +func TestEmptyConfigFileInstallsDefaults(t *testing.T) { + path := filepath.Join(t.TempDir(), "cfg") + require.NoError(t, os.WriteFile(path, nil, 0o600)) + cfgmgr, _, _ := testConfigManager(t, path, "") + cfg := cfgmgr.GetConfig() + require.NotNil(t, cfg) + require.Equal(t, "0.0.0.0:6000", cfg.Proxy.Addr) + require.True(t, cfg.EnableTrafficReplay) +} + func TestChecksum(t *testing.T) { cfgmgr, _, _ := testConfigManager(t, "", "") diff --git a/pkg/manager/config/manager.go b/pkg/manager/config/manager.go index 1536dcea9..2478e4417 100644 --- a/pkg/manager/config/manager.go +++ b/pkg/manager/config/manager.go @@ -41,6 +41,7 @@ type ConfigManager struct { checkFileInterval time.Duration fileContent []byte // used to compare whether the config file has changed + fileLoaded bool // distinguishes the initial nil buffer from an empty file sts struct { sync.Mutex listeners []chan<- *config.Config diff --git a/rust/Cargo.lock b/rust/Cargo.lock index a39515ee2..2959707be 100644 --- a/rust/Cargo.lock +++ b/rust/Cargo.lock @@ -2,15 +2,6 @@ # It is not intended for manual editing. version = 4 -[[package]] -name = "addr2line" -version = "0.25.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1b5d307320b3181d6d7954e663bd7c774a838b8220fe0593c86d9fb09f498b4b" -dependencies = [ - "gimli", -] - [[package]] name = "adler2" version = "2.0.1" @@ -231,21 +222,6 @@ dependencies = [ "url", ] -[[package]] -name = "backtrace" -version = "0.3.76" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bb531853791a215d7c62a30daf0dde835f381ab5de4589cfe7c649d2cbe92bd6" -dependencies = [ - "addr2line", - "cfg-if", - "libc", - "miniz_oxide", - "object", - "rustc-demangle", - "windows-link", -] - [[package]] name = "base64" version = "0.22.1" @@ -722,9 +698,9 @@ dependencies = [ [[package]] name = "ctutils" -version = "0.4.2" +version = "0.4.8" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7d5515a3834141de9eafb9717ad39eea8247b5674e6066c404e8c4b365d2a29e" +checksum = "261af4dc7a06ea8a527a8717d8cbba3417530d6ea14e1edb3de6b67806be474f" dependencies = [ "cmov", ] @@ -1020,9 +996,9 @@ checksum = "53c0fa8157de1303bfffdaa1cc2a673bfffb60102f76b0ef4441659124373fed" [[package]] name = "futures-macro" -version = "0.3.31" +version = "0.3.32" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "162ee34ebcb7c64a8abebc059ce0fee27c2262618d7b60ed8faf72fef13c3650" +checksum = "e835b70203e41293343137df5c0664546da5745f82ec9b84d40be8336958447b" dependencies = [ "proc-macro2", "quote", @@ -1043,9 +1019,9 @@ checksum = "cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd" [[package]] name = "futures-util" -version = "0.3.31" +version = "0.3.32" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9fa08315bb612088cc391249efdc3bc77536f16c91f6cf495e6fbe85b20a4a81" +checksum = "389ca41296e6190b48053de0321d02a77f32f8a5d2461dd38762c0593805c6d6" dependencies = [ "futures-channel", "futures-core", @@ -1055,7 +1031,6 @@ dependencies = [ "futures-task", "memchr", "pin-project-lite", - "pin-utils", "slab", ] @@ -1098,12 +1073,6 @@ dependencies = [ "wasm-bindgen", ] -[[package]] -name = "gimli" -version = "0.32.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e629b9b98ef3dd8afe6ca2bd0f89306cec16d43d907889945bc5d6687f2f13c7" - [[package]] name = "h2" version = "0.4.19" @@ -1460,17 +1429,6 @@ dependencies = [ "hashbrown 0.17.1", ] -[[package]] -name = "io-uring" -version = "0.7.14" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d64d8ca234d152948ceaede1f419b6a83983a5ecccaac05fb337a809c96d3aa6" -dependencies = [ - "bitflags 2.13.1", - "cfg-if", - "libc", -] - [[package]] name = "ipnet" version = "2.12.1" @@ -1785,15 +1743,6 @@ dependencies = [ "autocfg", ] -[[package]] -name = "object" -version = "0.37.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ff76201f031d8863c38aa7f905eca4f53abbfa15f609db4277d44cd8938f33fe" -dependencies = [ - "memchr", -] - [[package]] name = "oid-registry" version = "0.8.1" @@ -1950,12 +1899,6 @@ version = "0.2.17" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" -[[package]] -name = "pin-utils" -version = "0.1.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184" - [[package]] name = "pkg-config" version = "0.3.34" @@ -2095,9 +2038,9 @@ dependencies = [ [[package]] name = "pulldown-cmark" -version = "0.13.4" +version = "0.13.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e9f068eba8e7071c5f9511831b44f32c740d5adf574e990f946ddb53db2f314e" +checksum = "3db2be46835f013dd3f4fa7f5f7402c537787b54ebb6fe2b94bbdd7e2669bb7b" dependencies = [ "bitflags 2.13.1", "memchr", @@ -2145,9 +2088,9 @@ dependencies = [ [[package]] name = "quinn-proto" -version = "0.11.17" +version = "0.11.19" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "04759210543be93709136e28212294a659ef5001836ff4eab4d663e4529bba83" +checksum = "0e750cca55fe4f0439a15d0bb529da9651e79993e8e72c61a899a36d462befbe" dependencies = [ "bytes", "getrandom 0.4.3", @@ -2452,12 +2395,6 @@ dependencies = [ "ordered-multimap", ] -[[package]] -name = "rustc-demangle" -version = "0.1.28" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b74b56ffa8bb2830709a538c2cbcae9aa062db0d2a42563bfb09bdaae44020eb" - [[package]] name = "rustc-hash" version = "2.1.3" @@ -2497,9 +2434,9 @@ dependencies = [ [[package]] name = "rustls" -version = "0.23.45" +version = "0.23.46" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0d41d731c7d2f962d1ccc364cec258de3c0e93b38c2fb3ba97ac74513048d634" +checksum = "48e13bd8c0e9365c43cfa5c9e8f9ad49d3c8444926c9aac819e0e4dc503c8fdf" dependencies = [ "log", "once_cell", @@ -2534,9 +2471,9 @@ dependencies = [ [[package]] name = "rustls-webpki" -version = "0.103.15" +version = "0.103.16" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f3c3cf1d8b1e7d4927e2d154c3fcb02979afb9939629c62cd9048d4f07b60ac2" +checksum = "4443b1b6d39d7c663139942cc4558262562d2c84123bb6282ef177009ca8b281" dependencies = [ "ring", "rustls-pki-types", @@ -2744,12 +2681,12 @@ checksum = "b9be42f50aa861c555654aa3a37f52f4b1074bacf4e48fe0ef7fa584e80f1f0f" [[package]] name = "socket2" -version = "0.6.1" +version = "0.6.5" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "17129e116933cf371d018bb80ae557e889637989d8638274fb25622827b03881" +checksum = "c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4" dependencies = [ "libc", - "windows-sys 0.60.2", + "windows-sys 0.61.2", ] [[package]] @@ -2946,28 +2883,25 @@ dependencies = [ [[package]] name = "tokio" -version = "1.47.1" +version = "1.54.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "89e49afdadebb872d3145a5638b59eb0691ea23e46ca484037cfab3b76b95038" +checksum = "b52fd3255258d26fac7264ab5a76b53c13367136dc011d3d6ad4f8ed070396a3" dependencies = [ - "backtrace", "bytes", - "io-uring", "libc", "mio", "pin-project-lite", "signal-hook-registry", - "slab", "socket2", "tokio-macros", - "windows-sys 0.59.0", + "windows-sys 0.61.2", ] [[package]] name = "tokio-macros" -version = "2.5.0" +version = "2.7.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6e06d43f1345a3bcd39f6a56dbb7dcab2ba47e68e8ac134855e7e2bdbaf8cab8" +checksum = "6328af13490e73a9b4694030fafd93f8c8c6a9dede33e821c3fc63eddf8042ba" dependencies = [ "proc-macro2", "quote", @@ -3532,7 +3466,7 @@ version = "0.52.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "282be5f36a8ce781fad8c8ae18fa3f9beff57ec1b52cb3de0789201425d9a33d" dependencies = [ - "windows-targets 0.52.6", + "windows-targets", ] [[package]] @@ -3541,16 +3475,7 @@ version = "0.59.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1e38bc4d79ed67fd075bcc251a1c39b32a1776bbe92e5bef1f0bf1f8c531853b" dependencies = [ - "windows-targets 0.52.6", -] - -[[package]] -name = "windows-sys" -version = "0.60.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f2f500e4d28234f72040990ec9d39e3a6b950f9f22d3dba18416c35882612bcb" -dependencies = [ - "windows-targets 0.53.5", + "windows-targets", ] [[package]] @@ -3568,31 +3493,14 @@ version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973" dependencies = [ - "windows_aarch64_gnullvm 0.52.6", - "windows_aarch64_msvc 0.52.6", - "windows_i686_gnu 0.52.6", - "windows_i686_gnullvm 0.52.6", - "windows_i686_msvc 0.52.6", - "windows_x86_64_gnu 0.52.6", - "windows_x86_64_gnullvm 0.52.6", - "windows_x86_64_msvc 0.52.6", -] - -[[package]] -name = "windows-targets" -version = "0.53.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4945f9f551b88e0d65f3db0bc25c33b8acea4d9e41163edf90dcd0b19f9069f3" -dependencies = [ - "windows-link", - "windows_aarch64_gnullvm 0.53.1", - "windows_aarch64_msvc 0.53.1", - "windows_i686_gnu 0.53.1", - "windows_i686_gnullvm 0.53.1", - "windows_i686_msvc 0.53.1", - "windows_x86_64_gnu 0.53.1", - "windows_x86_64_gnullvm 0.53.1", - "windows_x86_64_msvc 0.53.1", + "windows_aarch64_gnullvm", + "windows_aarch64_msvc", + "windows_i686_gnu", + "windows_i686_gnullvm", + "windows_i686_msvc", + "windows_x86_64_gnu", + "windows_x86_64_gnullvm", + "windows_x86_64_msvc", ] [[package]] @@ -3601,96 +3509,48 @@ version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3" -[[package]] -name = "windows_aarch64_gnullvm" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a9d8416fa8b42f5c947f8482c43e7d89e73a173cead56d044f6a56104a6d1b53" - [[package]] name = "windows_aarch64_msvc" version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469" -[[package]] -name = "windows_aarch64_msvc" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b9d782e804c2f632e395708e99a94275910eb9100b2114651e04744e9b125006" - [[package]] name = "windows_i686_gnu" version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b" -[[package]] -name = "windows_i686_gnu" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "960e6da069d81e09becb0ca57a65220ddff016ff2d6af6a223cf372a506593a3" - [[package]] name = "windows_i686_gnullvm" version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66" -[[package]] -name = "windows_i686_gnullvm" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "fa7359d10048f68ab8b09fa71c3daccfb0e9b559aed648a8f95469c27057180c" - [[package]] name = "windows_i686_msvc" version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66" -[[package]] -name = "windows_i686_msvc" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1e7ac75179f18232fe9c285163565a57ef8d3c89254a30685b57d83a38d326c2" - [[package]] name = "windows_x86_64_gnu" version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78" -[[package]] -name = "windows_x86_64_gnu" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9c3842cdd74a865a8066ab39c8a7a473c0778a3f29370b5fd6b4b9aa7df4a499" - [[package]] name = "windows_x86_64_gnullvm" version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d" -[[package]] -name = "windows_x86_64_gnullvm" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0ffa179e2d07eee8ad8f57493436566c7cc30ac536a3379fdf008f47f6bb7ae1" - [[package]] name = "windows_x86_64_msvc" version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec" -[[package]] -name = "windows_x86_64_msvc" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d6bbff5f0aada427a1e5a6da5f1f98158182f26556f345ac9e04d36d0ebed650" - [[package]] name = "winnow" version = "0.7.15" diff --git a/rust/Cargo.toml b/rust/Cargo.toml index 2baf77137..ff30689e2 100644 --- a/rust/Cargo.toml +++ b/rust/Cargo.toml @@ -66,7 +66,7 @@ dataplane = { path = "crates/dataplane", version = "0.1.0" } displaydoc = { version = "=0.2.5", default-features = false } etcd-client = { version = "=0.20.0", default-features = false, features = ["tls-ring"] } flate2 = { version = "=1.1.2", default-features = false, features = ["rust_backend"] } -futures-util = { version = "=0.3.31", default-features = false, features = ["std"] } +futures-util = { version = "=0.3.32", default-features = false, features = ["std"] } http = { version = "=1.3.1", default-features = false } if-addrs = { version = "=0.13.4", default-features = false } libc = { version = "=0.2.189", default-features = false } @@ -81,11 +81,11 @@ rcgen = { version = "=0.14.9", default-features = false, features = ["pem", "rin reqwest = { version = "=0.12.24", default-features = false, features = ["rustls-tls-manual-roots", "stream"] } regex = { version = "=1.13.1", default-features = false, features = ["std", "unicode", "perf"] } rustix = { version = "=0.38.44", default-features = false, features = ["fs", "std"] } -rustls = { version = "=0.23.45", default-features = false, features = ["ring", "std", "tls12"] } +rustls = { version = "=0.23.46", default-features = false, features = ["ring", "std", "tls12"] } rustls-pki-types = { version = "=1.14.1", default-features = false, features = ["alloc", "std"] } -socket2 = { version = "=0.6.1", default-features = false, features = ["all"] } +socket2 = { version = "=0.6.5", default-features = false, features = ["all"] } tokio-rustls = { version = "=0.26.4", default-features = false, features = ["ring", "tls12"] } -tokio = { version = "=1.47.1", default-features = false, features = ["io-util", "macros", "net", "rt", "rt-multi-thread", "sync", "time"] } +tokio = { version = "=1.54.1", default-features = false, features = ["io-util", "macros", "net", "rt", "rt-multi-thread", "sync", "time"] } thiserror = { version = "=2.0.17", default-features = false } toml = { version = "=0.8.23", default-features = false, features = ["display", "parse"] } tonic = { version = "=0.14.6", default-features = false, features = ["codegen", "server", "transport"] } diff --git a/rust/crates/control-admin/src/config.rs b/rust/crates/control-admin/src/config.rs index 3f75c0208..a36b78d94 100644 --- a/rust/crates/control-admin/src/config.rs +++ b/rust/crates/control-admin/src/config.rs @@ -64,6 +64,11 @@ pub trait ConfigAdmin: Send + Sync { /// Go `SetTOMLConfig`: partial document merged onto the current /// configuration, validated as a whole. fn put_config_toml(&self, data: Vec) -> AdminFuture<'_, Result<(), ConfigMutationError>>; + /// Whether the accepted configuration asks for traffic capture/replay. + /// + /// The native process does not capture. A true value makes the traffic + /// routes answer `unsupported`; false keeps Go's `disabled` answer. + fn traffic_replay_enabled(&self) -> bool; } /// In-memory [`ConfigAdmin`] with Go's per-process semantics, for tests and @@ -200,6 +205,13 @@ impl ConfigAdmin for MemoryConfigAdmin { Ok(()) }) } + + fn traffic_replay_enabled(&self) -> bool { + self.state + .lock() + .map(|state| state.config.traffic_replay_enabled()) + .unwrap_or(false) + } } /// Field kinds of the Go structs a body decodes into, so decoding can apply diff --git a/rust/crates/control-admin/src/router.rs b/rust/crates/control-admin/src/router.rs index c09c7745e..ba26d7836 100644 --- a/rust/crates/control-admin/src/router.rs +++ b/rust/crates/control-admin/src/router.rs @@ -861,11 +861,16 @@ async fn config_put(State(app): State>, request: Request) -> Respo fn traffic_disabled( verb: &'static str, -) -> impl Fn() -> std::future::Ready + Clone + Send + 'static { - move || { +) -> impl Fn(State>) -> std::future::Ready + Clone + Send + 'static { + move |State(app): State>| { + let status = if app.hooks.config.traffic_replay_enabled() { + "unsupported" + } else { + "disabled" + }; std::future::ready(text( StatusCode::BAD_REQUEST, - &format!("traffic {verb} is disabled"), + &format!("traffic {verb} is {status}"), )) } } @@ -1104,7 +1109,11 @@ mod tests { oneshot(router.clone(), request(method, path, "")).await; assert_eq!(status, StatusCode::BAD_REQUEST); assert_eq!(content_type, TEXT_CONTENT_TYPE); - assert_eq!(body, format!("traffic {verb} is disabled")); + assert_eq!( + body, + format!("traffic {verb} is unsupported"), + "the default configuration leaves enable-traffic-replay true" + ); } let (status, _, body) = oneshot(router.clone(), request("GET", "/api/debug/pprof/heap", "")).await; diff --git a/rust/crates/control-config/src/lib.rs b/rust/crates/control-config/src/lib.rs index 00d25d817..855415946 100644 --- a/rust/crates/control-config/src/lib.rs +++ b/rust/crates/control-config/src/lib.rs @@ -22,12 +22,13 @@ pub mod source; pub use model::{ AwsMeteringConfig, AzureMeteringConfig, BackendClusterConfig, ClientTlsConfig, - CloudMeteringConfig, ConfigError, ConfigPersistence, EffectiveConfig, HealthCheckConfig, - LocalFsMeteringConfig, LogOnlineConfig, MeteringConfig, NamespaceConfig, ProxyOnlineConfig, - RoutingBalancePolicy, RoutingConfig, RoutingConnectionFactorConfig, RoutingFactorConfig, - RoutingNamespace, RoutingRule, RoutingSelectionPolicy, ServingConfig, ServingKeepalive, - ServingListener, ServingNamespace, ServingTlsConfig, SourceRevision, TopologyConfig, - TopologyRuntimeIdentity, + CloudMeteringConfig, ConfigError, ConfigPersistence, EffectiveConfig, FieldContract, + FieldEffect, HealthCheckConfig, LocalFsMeteringConfig, LogOnlineConfig, MeteringConfig, + NamespaceConfig, ProxyOnlineConfig, ReloadClass, RoutingBalancePolicy, RoutingConfig, + RoutingConnectionFactorConfig, RoutingFactorConfig, RoutingNamespace, RoutingRule, + RoutingSelectionPolicy, ServingConfig, ServingKeepalive, ServingListener, ServingNamespace, + ServingTlsConfig, SourceRevision, TopologyConfig, TopologyRuntimeIdentity, field_contract, + field_contract_json, go_min_tls_version, }; pub use module::{ ConfigModule, ConfigModuleHandle, ConfigModuleOptions, ConfigMutationError, diff --git a/rust/crates/control-config/src/model.rs b/rust/crates/control-config/src/model.rs index 9649fc1b1..74538619b 100644 --- a/rust/crates/control-config/src/model.rs +++ b/rust/crates/control-config/src/model.rs @@ -16,7 +16,7 @@ use std::collections::{BTreeMap, BTreeSet}; use std::fmt; -use std::net::IpAddr; +use std::net::{IpAddr, Ipv4Addr, Ipv6Addr}; use std::path::{Path, PathBuf}; use std::sync::Arc; @@ -51,6 +51,231 @@ pub enum ConfigError { }, } +/// Whether a published field can change without restarting the process. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum ReloadClass { + /// Accepted and published for the next consumer. + Dynamic, + /// A post-startup change rejects the whole candidate. + Restart, +} + +/// What the native process does with a modeled Go field. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum FieldEffect { + /// The native process applies the same value Go applies. + Applied, + /// The value round-trips for checksum and storage, and has no runtime effect. + Inert, + /// Activating the value rejects the serving candidate before publication. + Unsupported, +} + +/// One machine-readable compatibility entry. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct FieldContract { + /// Stable field name. `a+b` is a pair that is unsupported together. + pub field: &'static str, + /// Reload class enforced by [`EffectiveConfig::check_reload_from`]. + pub reload: ReloadClass, + /// Native-process effect. This is the runtime policy, not the Go struct tag. + pub effect: FieldEffect, +} + +/// Compatibility contract for the fields whose Go and native-process behavior +/// is not obvious from the schema. Restart-required names match +/// [`EffectiveConfig::check_reload_from`]. `balance.routing-rule` stays +/// dynamic: existing routers keep the rule they were built with, and later +/// routers read the new value, which is also what the Go process does. +#[must_use] +pub fn field_contract() -> &'static [FieldContract] { + FIELD_CONTRACT +} + +/// JSON document printed by `tiproxy-rs --config-info json`. +#[must_use] +pub fn field_contract_json() -> String { + let mut out = String::from("{\"component\":\"TiProxy Server\",\"parameters\":["); + for (index, field) in FIELD_CONTRACT.iter().enumerate() { + if index > 0 { + out.push(','); + } + let reload = match field.reload { + ReloadClass::Dynamic => "dynamic", + ReloadClass::Restart => "restart", + }; + let effect = match field.effect { + FieldEffect::Applied => "applied", + FieldEffect::Inert => "inert", + FieldEffect::Unsupported => "unsupported", + }; + out.push_str("{\"key\":\""); + out.push_str(field.field); + out.push_str("\",\"reload\":\""); + out.push_str(reload); + out.push_str("\",\"effect\":\""); + out.push_str(effect); + out.push_str("\"}"); + } + out.push_str("]}"); + out +} + +const FIELD_CONTRACT: &[FieldContract] = &[ + FieldContract { + field: "workdir", + reload: ReloadClass::Restart, + effect: FieldEffect::Applied, + }, + FieldContract { + field: "proxy.addr", + reload: ReloadClass::Restart, + effect: FieldEffect::Applied, + }, + FieldContract { + field: "proxy.advertise-addr", + reload: ReloadClass::Restart, + effect: FieldEffect::Applied, + }, + FieldContract { + field: "proxy.pd-addrs", + reload: ReloadClass::Restart, + effect: FieldEffect::Applied, + }, + FieldContract { + field: "proxy.port-range", + reload: ReloadClass::Restart, + effect: FieldEffect::Applied, + }, + FieldContract { + field: "api", + reload: ReloadClass::Restart, + effect: FieldEffect::Applied, + }, + FieldContract { + field: "api.proxy-protocol", + reload: ReloadClass::Restart, + effect: FieldEffect::Unsupported, + }, + FieldContract { + field: "log.encoder", + reload: ReloadClass::Restart, + effect: FieldEffect::Applied, + }, + FieldContract { + field: "log.simple", + reload: ReloadClass::Restart, + effect: FieldEffect::Applied, + }, + FieldContract { + field: "ha", + reload: ReloadClass::Restart, + effect: FieldEffect::Applied, + }, + FieldContract { + field: "ha.virtual-ip", + reload: ReloadClass::Restart, + effect: FieldEffect::Applied, + }, + FieldContract { + field: "ha.interface", + reload: ReloadClass::Restart, + effect: FieldEffect::Inert, + }, + FieldContract { + field: "ha.garp-burst-count", + reload: ReloadClass::Restart, + effect: FieldEffect::Inert, + }, + FieldContract { + field: "ha.garp-refresh-count", + reload: ReloadClass::Restart, + effect: FieldEffect::Inert, + }, + FieldContract { + field: "ha.virtual-ip+ha.interface", + reload: ReloadClass::Restart, + effect: FieldEffect::Unsupported, + }, + FieldContract { + field: "metering", + reload: ReloadClass::Restart, + effect: FieldEffect::Applied, + }, + FieldContract { + field: "rust-dataplane", + reload: ReloadClass::Restart, + effect: FieldEffect::Applied, + }, + FieldContract { + field: "balance.routing-rule", + reload: ReloadClass::Dynamic, + effect: FieldEffect::Applied, + }, + FieldContract { + field: "enable-traffic-replay", + reload: ReloadClass::Dynamic, + effect: FieldEffect::Inert, + }, + FieldContract { + field: "security.server-tls.auto-certs", + reload: ReloadClass::Dynamic, + effect: FieldEffect::Unsupported, + }, + FieldContract { + field: "security.server-http-tls.auto-certs", + reload: ReloadClass::Dynamic, + effect: FieldEffect::Unsupported, + }, + FieldContract { + field: "security.cluster-tls.auto-certs", + reload: ReloadClass::Dynamic, + effect: FieldEffect::Inert, + }, + FieldContract { + field: "security.sql-tls.auto-certs", + reload: ReloadClass::Dynamic, + effect: FieldEffect::Inert, + }, + FieldContract { + field: "security.encryption-key-path", + reload: ReloadClass::Dynamic, + effect: FieldEffect::Inert, + }, + FieldContract { + field: "security.cluster-tls.skip-ca", + reload: ReloadClass::Dynamic, + effect: FieldEffect::Applied, + }, + FieldContract { + field: "proxy.public-endpoints", + reload: ReloadClass::Dynamic, + effect: FieldEffect::Applied, + }, + FieldContract { + field: "proxy.frontend-keepalive", + reload: ReloadClass::Dynamic, + effect: FieldEffect::Applied, + }, +]; + +/// Go `GetMinTLSVer`: a suffix of `1.0`, `1.1`, `1.2`, or `1.3` selects that +/// version, and an empty or unrecognized value falls back to `1.2`. +#[must_use] +pub fn go_min_tls_version(value: &str) -> &'static str { + if value.ends_with("1.0") { + "1.0" + } else if value.ends_with("1.1") { + "1.1" + } else if value.ends_with("1.2") { + "1.2" + } else if value.ends_with("1.3") { + "1.3" + } else { + "1.2" + } +} + /// Source lineages observed while constructing one immutable snapshot. #[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] pub struct SourceRevision { @@ -69,7 +294,10 @@ pub struct ClientTlsConfig { pub certificate_path: Option>, /// Optional client private-key path. pub private_key_path: Option>, - /// Minimum accepted TLS version (`1.2` or `1.3`). + /// Effective minimum TLS version (`1.0`, `1.1`, `1.2`, or `1.3`). + /// + /// This is Go's `GetMinTLSVer` result. The native TLS stack's floor is + /// 1.2, so `1.0` and `1.1` are accepted and then negotiated as 1.2. pub minimum_version: Arc, /// Whether server CA verification is disabled. pub skip_ca_verification: bool, @@ -298,7 +526,7 @@ pub struct ServingConfig { pub graceful_close_millis: u64, /// Restart-required SQL listeners. pub listeners: Arc<[ServingListener]>, - /// Canonical, sorted public endpoint CIDRs. + /// Public endpoint networks from Go `ParseCIDRList`, in input order. pub public_cidrs: Arc<[Arc]>, /// Frontend server TLS policy. pub frontend_tls: ServingTlsConfig, @@ -330,7 +558,7 @@ pub struct ServingListener { pub name: Arc, /// Host/IP without brackets. pub address: Arc, - /// Nonzero TCP port. + /// TCP port. Zero is the empty `proxy.addr` sentinel. pub port: u16, } @@ -604,7 +832,7 @@ impl EffectiveConfig { /// represented by the serving contract. pub fn serving(&self) -> Result { let listeners = serving_listeners(&self.proxy)?; - let public_cidrs = normalize_public_endpoints(&self.proxy.online.public_endpoints)?; + let public_cidrs = go_public_cidrs(&self.proxy.online.public_endpoints); Ok(ServingConfig { max_connections: self.proxy.online.max_connections, high_memory_reject_threshold: self.proxy.online.high_memory_usage_reject_threshold, @@ -618,18 +846,13 @@ impl EffectiveConfig { } })? }, - frontend_keepalive: serving_keepalive( - "proxy.frontend-keepalive", - &self.proxy.online.frontend_keepalive, - )?, + frontend_keepalive: serving_keepalive(&self.proxy.online.frontend_keepalive), healthy_backend_keepalive: serving_keepalive( - "proxy.backend-healthy-keepalive", &self.proxy.online.backend_healthy_keepalive, - )?, + ), unhealthy_backend_keepalive: serving_keepalive( - "proxy.backend-unhealthy-keepalive", &self.proxy.online.backend_unhealthy_keepalive, - )?, + ), proxy_protocol_v2: self.proxy.online.proxy_protocol == "v2", require_backend_tls: self.security.require_backend_tls, graceful_wait_millis: seconds_to_millis( @@ -779,6 +1002,33 @@ impl EffectiveConfig { &self.api.addr } + /// Returns `api.proxy-protocol`. Only `v2` has a Go runtime effect, and + /// the native admin listener rejects that activation. + #[must_use] + pub fn api_proxy_protocol(&self) -> &str { + &self.api.proxy_protocol + } + + /// Returns `security.encryption-key-path`. Go uses it for traffic capture. + #[must_use] + pub fn encryption_key_path(&self) -> &str { + &self.security.encryption_key_path + } + + /// Returns `ha.virtual-ip`. A non-empty value is excluded from advertise + /// address selection. Pairing it with [`Self::ha_interface`] asks Go to + /// own the VIP, which the native process rejects. + #[must_use] + pub fn ha_virtual_ip(&self) -> &str { + &self.ha.virtual_ip + } + + /// Returns `ha.interface`. + #[must_use] + pub fn ha_interface(&self) -> &str { + &self.ha.interface + } + /// Returns whether traffic replay is enabled. #[must_use] pub const fn traffic_replay_enabled(&self) -> bool { @@ -2216,17 +2466,9 @@ fn validate_balance(balance: &mut BalanceConfig) -> Result<(), ConfigError> { } fn validate_tls(field: &'static str, tls: &TlsConfig) -> Result<(), ConfigError> { - if tls.cert.is_empty() != tls.key.is_empty() { - return invalid(field, "partial_cert_key"); - } - if !tls.min_tls_version.is_empty() && !matches!(tls.min_tls_version.as_str(), "1.2" | "1.3") { - return invalid(field, "unsupported_minimum_version"); - } - for path in [&tls.cert, &tls.key, &tls.ca] { - if !path.is_empty() && !Path::new(path).is_absolute() { - return invalid(field, "relative_path"); - } - } + // Go `Config.Check` does not reject a partial certificate pair, a relative + // path, or a TLS version. Incomplete material fails later, when TLS state + // is built. Versions follow `GetMinTLSVer` in the serving projection. if tls .cert_allowed_cn .iter() @@ -2242,7 +2484,7 @@ fn client_tls(tls: &TlsConfig) -> ClientTlsConfig { ca_path: path_arc(&tls.ca), certificate_path: path_arc(&tls.cert), private_key_path: path_arc(&tls.key), - minimum_version: Arc::from(tls.min_tls_version.as_str()), + minimum_version: Arc::from(go_min_tls_version(&tls.min_tls_version)), skip_ca_verification: tls.skip_ca, allowed_common_names: normalized_common_names(&tls.cert_allowed_cn), } @@ -2253,7 +2495,7 @@ fn serving_tls(tls: &TlsConfig) -> ServingTlsConfig { certificate_path: path_arc(&tls.cert), private_key_path: path_arc(&tls.key), ca_path: path_arc(&tls.ca), - minimum_version: Arc::from(tls.min_tls_version.as_str()), + minimum_version: Arc::from(go_min_tls_version(&tls.min_tls_version)), allowed_common_names: normalized_common_names(&tls.cert_allowed_cn), skip_ca_verification: tls.skip_ca, } @@ -2269,50 +2511,49 @@ fn normalized_common_names(names: &[String]) -> Arc<[Arc]> { Arc::from(normalized) } -fn serving_keepalive( - field: &'static str, - keepalive: &KeepAliveConfig, -) -> Result { - if keepalive.idle < 0 || keepalive.count < 0 || keepalive.interval < 0 || keepalive.timeout < 0 - { - return invalid(field, "negative"); +fn serving_keepalive(keepalive: &KeepAliveConfig) -> ServingKeepalive { + // Go `Config.Check` accepts every keepalive number. A negative value is + // not applied by the socket layer. A positive duration is truncated to + // whole milliseconds, so a sub-millisecond value projects as zero instead + // of rejecting the candidate. + ServingKeepalive { + enabled: keepalive.enabled, + idle_millis: duration_millis(keepalive.idle), + probe_count: probe_count(keepalive.count), + interval_millis: duration_millis(keepalive.interval), + user_timeout_millis: duration_millis(keepalive.timeout), } - for value in [keepalive.idle, keepalive.interval, keepalive.timeout] { - if value > 0 && value < NANOS_PER_MILLISECOND { - return invalid(field, "sub_millisecond"); - } +} + +fn duration_millis(value: i64) -> u64 { + if value <= 0 { + 0 + } else { + u64::try_from(value / NANOS_PER_MILLISECOND).unwrap_or(u64::MAX) + } +} + +fn probe_count(count: i64) -> u32 { + if count <= 0 { + 0 + } else { + u32::try_from(count).unwrap_or(u32::MAX) } - Ok(ServingKeepalive { - enabled: keepalive.enabled, - idle_millis: u64::try_from(keepalive.idle / NANOS_PER_MILLISECOND).map_err(|_| { - ConfigError::InvalidField { - field, - class: "out_of_range", - } - })?, - probe_count: u32::try_from(keepalive.count).map_err(|_| ConfigError::InvalidField { - field, - class: "out_of_range", - })?, - interval_millis: u64::try_from(keepalive.interval / NANOS_PER_MILLISECOND).map_err( - |_| ConfigError::InvalidField { - field, - class: "out_of_range", - }, - )?, - user_timeout_millis: u64::try_from(keepalive.timeout / NANOS_PER_MILLISECOND).map_err( - |_| ConfigError::InvalidField { - field, - class: "out_of_range", - }, - )?, - }) } fn serving_listeners(proxy: &ProxyConfig) -> Result, ConfigError> { let addrs = split_addr_list(&proxy.addr); if addrs.is_empty() { - return invalid("proxy.addr", "empty_listener"); + if !proxy.port_range.is_empty() { + return invalid("proxy.addr", "empty_with_port_range"); + } + // Go `GetSQLAddrs` returns `[""]` and `net.Listen("tcp", "")` binds an + // ephemeral port. Port 0 is that sentinel for the listener layer. + return Ok(vec![ServingListener { + name: Arc::from("sql-0"), + address: Arc::from(""), + port: 0, + }]); } let addresses = if proxy.port_range.is_empty() { addrs @@ -2357,42 +2598,28 @@ fn seconds_to_millis(field: &'static str, seconds: i64) -> Result Result>, ConfigError> { - let mut result = values +/// Go `netutil.ParseCIDRList`: a value without `/` is parsed as `/32` (including +/// IPv6), invalid entries are skipped, and the surviving network addresses keep +/// their input order and duplicates. `Config.Check` accepts the raw list; the +/// SQL server logs a warning and keeps using the entries that parsed. +fn go_public_cidrs(values: &[String]) -> Vec> { + values .iter() - .map(|value| normalize_ip_prefix(value)) - .collect::, _>>()?; - result.sort(); - result.dedup(); - Ok(result.into_iter().map(Arc::::from).collect()) + .filter_map(|value| { + let spec = if value.contains('/') { + value.clone() + } else { + format!("{value}/32") + }; + mask_cidr(&spec).map(Arc::::from) + }) + .collect() } -fn normalize_ip_prefix(value: &str) -> Result { - let value = value.trim(); - let (address, prefix) = if let Some((address, prefix)) = value.split_once('/') { - let address = address - .parse::() - .map_err(|_| ConfigError::InvalidField { - field: "proxy.public-endpoints", - class: "invalid_ip_or_cidr", - })?; - let prefix = prefix - .parse::() - .map_err(|_| ConfigError::InvalidField { - field: "proxy.public-endpoints", - class: "invalid_ip_or_cidr", - })?; - (address, prefix) - } else { - let address = value - .parse::() - .map_err(|_| ConfigError::InvalidField { - field: "proxy.public-endpoints", - class: "invalid_ip_or_cidr", - })?; - let prefix = if address.is_ipv4() { 32 } else { 128 }; - (address, prefix) - }; +fn mask_cidr(spec: &str) -> Option { + let (address, prefix) = spec.rsplit_once('/')?; + let address = address.parse::().ok()?; + let prefix = prefix.parse::().ok()?; match address { IpAddr::V4(address) if prefix <= 32 => { let bits = u32::from(address); @@ -2401,11 +2628,7 @@ fn normalize_ip_prefix(value: &str) -> Result { } else { u32::MAX << (32 - prefix) }; - Ok(format!( - "{}/{}", - std::net::Ipv4Addr::from(bits & mask), - prefix - )) + Some(format!("{}/{}", Ipv4Addr::from(bits & mask), prefix)) } IpAddr::V6(address) if prefix <= 128 => { let bits = u128::from(address); @@ -2414,13 +2637,9 @@ fn normalize_ip_prefix(value: &str) -> Result { } else { u128::MAX << (128 - prefix) }; - Ok(format!( - "{}/{}", - std::net::Ipv6Addr::from(bits & mask), - prefix - )) + Some(format!("{}/{}", Ipv6Addr::from(bits & mask), prefix)) } - IpAddr::V4(_) | IpAddr::V6(_) => invalid("proxy.public-endpoints", "invalid_ip_or_cidr"), + IpAddr::V4(_) | IpAddr::V6(_) => None, } } @@ -2493,7 +2712,7 @@ fn normalize_ns_server(value: &str) -> Result { } // A bare IPv6 literal has no port (Go `SplitHostPort` fails), so Go falls back // to `net.JoinHostPort(server, "53")` → `[]:53`. - if value.parse::().is_ok() { + if value.parse::().is_ok() { return Ok(format!("[{value}]:53")); } // Plain `host:port` (hostname or IPv4). diff --git a/rust/crates/control-config/src/module.rs b/rust/crates/control-config/src/module.rs index 07b69142f..1cb0db7e3 100644 --- a/rust/crates/control-config/src/module.rs +++ b/rust/crates/control-config/src/module.rs @@ -808,14 +808,20 @@ async fn read_until_disconnect( ) -> Result<(), ReaderError> { let mut connection = connector.connect().await.map_err(|error| match error { control_external::EtcdConnectError::StaleOwner => ReaderError::StaleOwner, - control_external::EtcdConnectError::Dependency(_) => ReaderError::Dependency, + control_external::EtcdConnectError::Dependency(_) => { + eprintln!("tiproxy-rs etcd connect failed: {error:?}"); + ReaderError::Dependency + } })?; let response = connection .execute(|client| { Box::pin(client.get(CONFIG_PREFIX, Some(GetOptions::new().with_prefix()))) }) .await - .map_err(|error| map_reader_operation(&error))?; + .map_err(|error| { + eprintln!("tiproxy-rs etcd get /config failed: {error:?}"); + map_reader_operation(&error) + })?; let revision = response .header() .map_or(0, etcd_client::ResponseHeader::revision); diff --git a/rust/crates/control-config/tests/go_compat.rs b/rust/crates/control-config/tests/go_compat.rs new file mode 100644 index 000000000..9f40a6cc2 --- /dev/null +++ b/rust/crates/control-config/tests/go_compat.rs @@ -0,0 +1,181 @@ +// Copyright 2026 PingCAP, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +//! Acceptance timing shared with Go `Config.Check` and the SQL server. + +use std::path::Path; +use std::sync::Arc; + +use control_config::{ + ConfigError, ConfigNamespaceSource, ConfigNamespaceStore, FieldEffect, ReloadClass, + ServingConfig, StoreError, field_contract, +}; + +fn load(toml: &str) -> Result { + Ok( + ConfigNamespaceStore::from_toml(toml.as_bytes(), None, Path::new("/tmp"))? + .current() + .effective() + .serving()?, + ) +} + +#[test] +fn empty_document_serves_with_the_go_defaults() { + let serving = load("").unwrap_or_else(|error| unreachable!("empty document: {error}")); + assert!(serving.traffic_replay_enabled); + assert_eq!(serving.listeners.len(), 1); + assert_eq!(serving.listeners[0].address.as_ref(), "0.0.0.0"); + assert_eq!(serving.listeners[0].port, 6000); +} + +#[test] +fn tls_versions_relative_paths_and_partial_pairs_follow_go_check() { + let cases = [ + ("1.0", "1.0"), + ("1.1", "1.1"), + ("1.2", "1.2"), + ("1.3", "1.3"), + ("tls1.3", "1.3"), + ("nope", "1.2"), + ("", "1.2"), + ]; + for (input, effective) in cases { + let serving = load(&format!( + "[security.server-tls]\nmin-tls-version = \"{input}\"\ncert = \"relative.pem\"\n" + )) + .unwrap_or_else(|error| unreachable!("{input}: {error}")); + assert_eq!(serving.frontend_tls.minimum_version.as_ref(), effective); + assert_eq!( + serving + .frontend_tls + .certificate_path + .as_deref() + .and_then(|path| path.to_str()), + Some("relative.pem") + ); + assert!(serving.frontend_tls.private_key_path.is_none()); + } +} + +#[test] +fn public_endpoints_keep_go_parse_cidr_list_order() { + let serving = load( + r#" +[proxy] +public-endpoints = ["2001:db8::1", "10.0.0.9", "not-an-ip", "10.0.0.0/24", "10.0.0.0/24", "10.1.2.3/24"] +"#, + ) + .unwrap_or_else(|error| unreachable!("public endpoints: {error}")); + let cidrs = serving + .public_cidrs + .iter() + .map(Arc::as_ref) + .collect::>(); + assert_eq!( + cidrs, + [ + "2001:db8::/32", + "10.0.0.9/32", + "10.0.0.0/24", + "10.0.0.0/24", + "10.1.2.0/24", + ] + ); +} + +#[test] +fn keepalive_out_of_range_values_project_as_zero() { + let serving = load( + r#" +[proxy.frontend-keepalive] +idle = "999us" +cnt = -3 +intvl = "-1s" +"#, + ) + .unwrap_or_else(|error| unreachable!("keepalive: {error}")); + assert_eq!(serving.frontend_keepalive.idle_millis, 0); + assert_eq!(serving.frontend_keepalive.probe_count, 0); + assert_eq!(serving.frontend_keepalive.interval_millis, 0); +} + +#[test] +fn empty_sql_address_is_an_ephemeral_listener() { + let serving = + load("[proxy]\naddr = \"\"\n").unwrap_or_else(|error| unreachable!("empty addr: {error}")); + assert_eq!(serving.listeners.len(), 1); + assert_eq!(serving.listeners[0].name.as_ref(), "sql-0"); + assert_eq!(serving.listeners[0].address.as_ref(), ""); + assert_eq!(serving.listeners[0].port, 0); + assert!(matches!( + load("[proxy]\naddr = \"\"\nport-range = [6000, 6000]\n"), + Err(StoreError::Config(ConfigError::InvalidField { + field: "proxy.addr", + class: "empty_with_port_range", + })) + )); +} + +#[test] +fn routing_rule_reloads_and_restart_fields_match_the_contract() { + let store = ConfigNamespaceStore::from_toml(b"", None, Path::new("/tmp")) + .unwrap_or_else(|error| unreachable!("defaults: {error}")); + store + .apply_toml( + b"[balance]\nrouting-rule = \"port\"\n", + None, + 2, + Path::new("/tmp"), + ) + .unwrap_or_else(|error| unreachable!("routing-rule reload: {error}")); + assert!(matches!( + store.apply_toml( + b"[proxy]\naddr = \"127.0.0.1:1\"\n", + None, + 3, + Path::new("/tmp"), + ), + Err(StoreError::Config(ConfigError::RestartRequired { + field: "proxy.addr", + })) + )); + for name in [ + "workdir", + "proxy.addr", + "proxy.advertise-addr", + "proxy.pd-addrs", + "proxy.port-range", + "api", + "log.encoder", + "log.simple", + "ha", + "metering", + "rust-dataplane", + ] { + let entry = field_contract() + .iter() + .find(|entry| entry.field == name) + .unwrap_or_else(|| unreachable!("{name} is missing from the field contract")); + assert_eq!(entry.reload, ReloadClass::Restart, "{name}"); + } + let replay = field_contract() + .iter() + .find(|entry| entry.field == "enable-traffic-replay") + .unwrap_or_else(|| unreachable!("traffic replay contract")); + assert_eq!( + (replay.reload, replay.effect), + (ReloadClass::Dynamic, FieldEffect::Inert) + ); +} diff --git a/rust/crates/control-config/tests/source.rs b/rust/crates/control-config/tests/source.rs index ec98836d7..efa032091 100644 --- a/rust/crates/control-config/tests/source.rs +++ b/rust/crates/control-config/tests/source.rs @@ -993,7 +993,7 @@ timeout = "4.5s" } #[test] -fn malformed_or_sub_millisecond_go_duration_rejects_atomically() { +fn malformed_go_duration_rejects_and_sub_millisecond_projects_as_zero() { assert!(matches!( ConfigNamespaceStore::from_toml( br#" @@ -1005,20 +1005,23 @@ idle = "1fortnight" ), Err(StoreError::TomlDecode(_)) )); - assert!(matches!( - ConfigNamespaceStore::from_toml( - br#" + let store = ConfigNamespaceStore::from_toml( + br#" [proxy.frontend-keepalive] idle = "999us" +cnt = -1 "#, - None, - current_dir(), - ), - Err(StoreError::Config(ConfigError::InvalidField { - field: "proxy.frontend-keepalive", - class: "sub_millisecond" - })) - )); + None, + current_dir(), + ) + .unwrap_or_else(|error| unreachable!("sub-millisecond keepalive is accepted: {error}")); + let serving = store + .current() + .effective() + .serving() + .unwrap_or_else(|error| unreachable!("serving projection: {error}")); + assert_eq!(serving.frontend_keepalive.idle_millis, 0); + assert_eq!(serving.frontend_keepalive.probe_count, 0); } #[test] diff --git a/rust/crates/control-external/src/etcd.rs b/rust/crates/control-external/src/etcd.rs index 00ce7d41f..3214b1795 100644 --- a/rust/crates/control-external/src/etcd.rs +++ b/rust/crates/control-external/src/etcd.rs @@ -47,19 +47,31 @@ pub enum EtcdTlsVersion { } impl EtcdTlsVersion { - /// Parses the CP-CFG `minimum-version` value: `""` (no floor), `"1.2"`, or - /// `"1.3"`. + /// Parses a minimum version with Go `GetMinTLSVer` suffix rules. + /// + /// `""` keeps the default floor. A `1.3` suffix selects TLS 1.3 only. + /// `1.0` and `1.1` are accepted and floored at TLS 1.2, because rustls + /// cannot offer the older protocols. Any other non-empty value falls + /// back to TLS 1.2, matching Go's warning-and-default path. /// /// # Errors /// - /// Returns [`EtcdConfigError::UnsupportedTlsVersion`] for any other value. + /// This function does not reject a version string. The `Result` is kept + /// so existing callers can still propagate a configuration error. + #[allow(clippy::unnecessary_wraps)] pub fn parse(value: &str) -> Result, EtcdConfigError> { - match value { - "" => Ok(None), - "1.2" => Ok(Some(Self::V1_2)), - "1.3" => Ok(Some(Self::V1_3)), - _ => Err(EtcdConfigError::UnsupportedTlsVersion), + if value.is_empty() { + return Ok(None); + } + // Same suffix order as Go `GetMinTLSVer`. 1.0 and 1.1 cannot be + // offered by rustls, so they use the 1.2 floor. + if value.ends_with("1.0") || value.ends_with("1.1") || value.ends_with("1.2") { + return Ok(Some(Self::V1_2)); } + if value.ends_with("1.3") { + return Ok(Some(Self::V1_3)); + } + Ok(Some(Self::V1_2)) } } @@ -987,17 +999,29 @@ mod tests { } #[test] - fn minimum_tls_version_parses_only_the_frozen_values() { + fn minimum_tls_version_follows_go_suffix_rules() { assert_eq!(EtcdTlsVersion::parse(""), Ok(None)); assert_eq!(EtcdTlsVersion::parse("1.2"), Ok(Some(EtcdTlsVersion::V1_2))); assert_eq!(EtcdTlsVersion::parse("1.3"), Ok(Some(EtcdTlsVersion::V1_3))); + assert_eq!( + EtcdTlsVersion::parse("1.0"), + Ok(Some(EtcdTlsVersion::V1_2)), + "TLS 1.0 is accepted and floored at 1.2" + ); assert_eq!( EtcdTlsVersion::parse("1.1"), - Err(EtcdConfigError::UnsupportedTlsVersion) + Ok(Some(EtcdTlsVersion::V1_2)), + "TLS 1.1 is accepted and floored at 1.2" ); assert_eq!( EtcdTlsVersion::parse("tls1.3"), - Err(EtcdConfigError::UnsupportedTlsVersion) + Ok(Some(EtcdTlsVersion::V1_3)), + "Go matches the 1.3 suffix" + ); + assert_eq!( + EtcdTlsVersion::parse("nope"), + Ok(Some(EtcdTlsVersion::V1_2)), + "an unrecognized value falls back to TLS 1.2" ); } diff --git a/rust/crates/control-external/src/tls.rs b/rust/crates/control-external/src/tls.rs index 67a337df8..ae5d77d13 100644 --- a/rust/crates/control-external/src/tls.rs +++ b/rust/crates/control-external/src/tls.rs @@ -22,8 +22,8 @@ //! //! | `skip_ca` | common names | server certificate verification | //! |-----------|--------------|---------------------------------| -//! | false | empty | `WebPKI` chain + hostname | -//! | false | non-empty | `WebPKI` chain + hostname, then exact leaf Subject-CN pin | +//! | false | empty | CA chain only; hostname is not checked, matching Go | +//! | false | non-empty | CA chain only, then exact leaf Subject-CN pin | //! | true | empty | chain/hostname skipped, TLS `CertificateVerify` signature still checked | //! | true | non-empty | as above, then exact leaf Subject-CN pin | //! @@ -35,7 +35,6 @@ use std::collections::BTreeSet; use std::sync::Arc; -use rustls::client::WebPkiServerVerifier; use rustls::client::danger::{HandshakeSignatureValid, ServerCertVerified, ServerCertVerifier}; use rustls::crypto::CryptoProvider; use rustls::{ClientConfig, DigitallySignedStruct, RootCertStore, SignatureScheme}; @@ -73,7 +72,7 @@ pub(crate) fn build_client_config( .dangerous() .with_custom_certificate_verifier(verifier); - let config = match identity { + let mut config = match identity { Some((certificate_pem, key_pem)) => { let chain = parse_certificate_chain(certificate_pem)?; let key = parse_private_key(key_pem)?; @@ -83,6 +82,10 @@ pub(crate) fn build_client_config( } None => builder.with_no_client_auth(), }; + // PD/etcd serves gRPC and HTTP/1.1 on the same port. Without ALPN h2 the + // server answers HTTP/1.1, and the HTTP/2 framer reads "HTTP" as a frame + // longer than the default max frame size (FRAME_SIZE_ERROR). + config.alpn_protocols = vec![b"h2".to_vec()]; Ok(Arc::new(config)) } @@ -106,9 +109,7 @@ fn build_server_verifier( if roots.is_empty() { return Err(EtcdConfigError::EmptyCaCertificate); } - WebPkiServerVerifier::builder_with_provider(Arc::new(roots), Arc::clone(provider)) - .build() - .map_err(|_| EtcdConfigError::TlsSetup)? + Arc::new(CaOnlyServerVerification::new(roots, Arc::clone(provider))) }; if policy.allowed_common_names.is_empty() { return Ok(inner); @@ -144,6 +145,82 @@ fn parse_private_key(pem: &[u8]) -> Result, EtcdConfigErr Ok(key.clone_key()) } +/// Verifies the server certificate chains to the configured CA and is a TLS +/// server cert, and does not check the dialed hostname. +/// +/// Go `CertInfo.verifyCA` leaves `VerifyOptions.DNSName` empty. Operator +/// configs dial PD as `tc-pd:2379` while the serving certificate is issued +/// for the peer DNS name, so a hostname check refuses the connection. +#[derive(Debug)] +struct CaOnlyServerVerification { + roots: Arc, + provider: Arc, +} + +impl CaOnlyServerVerification { + fn new(roots: RootCertStore, provider: Arc) -> Self { + Self { + roots: Arc::new(roots), + provider, + } + } +} + +impl ServerCertVerifier for CaOnlyServerVerification { + fn verify_server_cert( + &self, + end_entity: &CertificateDer<'_>, + intermediates: &[CertificateDer<'_>], + _server_name: &ServerName<'_>, + _ocsp_response: &[u8], + now: UnixTime, + ) -> Result { + let certificate = rustls::server::ParsedCertificate::try_from(end_entity)?; + rustls::client::verify_server_cert_signed_by_trust_anchor( + &certificate, + &self.roots, + intermediates, + now, + self.provider.signature_verification_algorithms.all, + )?; + Ok(ServerCertVerified::assertion()) + } + + fn verify_tls12_signature( + &self, + message: &[u8], + cert: &CertificateDer<'_>, + dss: &DigitallySignedStruct, + ) -> Result { + rustls::crypto::verify_tls12_signature( + message, + cert, + dss, + &self.provider.signature_verification_algorithms, + ) + } + + fn verify_tls13_signature( + &self, + message: &[u8], + cert: &CertificateDer<'_>, + dss: &DigitallySignedStruct, + ) -> Result { + rustls::crypto::verify_tls13_signature( + message, + cert, + dss, + &self.provider.signature_verification_algorithms, + ) + } + + fn supported_verify_schemes(&self) -> Vec { + self.provider + .signature_verification_algorithms + .supported_schemes() + } +} + /// Skips certificate-chain and hostname trust while still validating the /// handshake `CertificateVerify` signature against the presented certificate, /// mirroring Go `skip-ca` exactly (the peer must still hold the key). @@ -603,11 +680,11 @@ mod tests { } None => builder.with_no_client_auth(), }; - Arc::new( - builder - .with_single_cert(parse_chain(cert_pem), parse_key(key_pem)) - .unwrap_or_else(|error| unreachable!("server cert: {error}")), - ) + let mut config = builder + .with_single_cert(parse_chain(cert_pem), parse_key(key_pem)) + .unwrap_or_else(|error| unreachable!("server cert: {error}")); + config.alpn_protocols = vec![b"h2".to_vec()]; + Arc::new(config) } /// Drives a full in-memory handshake, returning `Err` if either side @@ -694,15 +771,16 @@ mod tests { .is_ok(), "trusted CA and correct hostname must succeed" ); - // Correct CA but wrong hostname is rejected. + // Correct CA but a different dial name still succeeds. Go does not + // set VerifyOptions.DNSName for cluster-tls. assert!( do_handshake( client_config(Some(&trusted_ca), None, &policy), server_config(&server_cert, &server_key, None, rustls::ALL_VERSIONS), "wrong-host.example", ) - .is_err(), - "a hostname absent from the SAN must fail" + .is_ok(), + "a hostname absent from the SAN must still succeed when the CA matches" ); // Client trusts a different CA than signed the leaf. assert!( diff --git a/rust/crates/control-proto/src/snapshot.rs b/rust/crates/control-proto/src/snapshot.rs index 16d4406bd..e7094349b 100644 --- a/rust/crates/control-proto/src/snapshot.rs +++ b/rust/crates/control-proto/src/snapshot.rs @@ -15,6 +15,7 @@ //! Atomic validation and last-good storage for complete dataplane snapshots. use std::collections::BTreeSet; +use std::env; use std::fmt; use std::fs::{self, OpenOptions}; use std::io::Read; @@ -959,7 +960,9 @@ impl SnapshotStore { } let private_key = private_key .ok_or_else(|| SnapshotError::invalid(format!("{field} private key is required")))?; - let builder = if policy.minimum_version == "1.3" { + // Go `GetMinTLSVer` selects 1.3 only for a `1.3` suffix. 1.0 and 1.1 + // are accepted; rustls cannot offer them, so the handshake floor is 1.2. + let builder = if go_min_tls_version(&policy.minimum_version) == "1.3" { ServerConfig::builder_with_protocol_versions(&[&rustls::version::TLS13]) } else { ServerConfig::builder() @@ -980,17 +983,24 @@ impl SnapshotStore { fn read_tls_file(&self, path: &str, field: &str) -> Result, SnapshotError> { let path = Path::new(path); - if !path.is_absolute() { - return Err(SnapshotError::invalid(format!( - "{field} file path must be absolute" - ))); - } - let canonical = fs::canonicalize(path) + // Go opens certificate paths with `os.ReadFile`, which resolves a + // relative path against the process working directory. + let absolute = if path.is_absolute() { + path.to_path_buf() + } else { + env::current_dir() + .map_err(|_| SnapshotError::invalid(format!("{field} file is unavailable")))? + .join(path) + }; + let canonical = fs::canonicalize(&absolute) .map_err(|_| SnapshotError::invalid(format!("{field} file is unavailable")))?; - if !self - .allowed_tls_roots - .iter() - .any(|root| canonical.starts_with(root)) + // An empty allowlist matches Go: there is no extra root jail. A + // configured `--tls-root` or `tls-allowed-roots` still confines reads. + if !self.allowed_tls_roots.is_empty() + && !self + .allowed_tls_roots + .iter() + .any(|root| canonical.starts_with(root)) { return Err(SnapshotError::invalid(format!( "{field} file is outside configured TLS roots" @@ -1045,19 +1055,26 @@ fn build_client_verifier( })) } +/// Go `GetMinTLSVer` suffix match. Empty and unrecognized values are TLS 1.2. +fn go_min_tls_version(value: &str) -> &'static str { + if value.ends_with("1.0") { + "1.0" + } else if value.ends_with("1.1") { + "1.1" + } else if value.ends_with("1.2") { + "1.2" + } else if value.ends_with("1.3") { + "1.3" + } else { + "1.2" + } +} + fn validate_tls_policy( field: &str, policy: &TlsPolicy, frontend: bool, ) -> Result<(), SnapshotError> { - if !policy.minimum_version.is_empty() - && policy.minimum_version != "1.2" - && policy.minimum_version != "1.3" - { - return Err(SnapshotError::invalid(format!( - "{field}.minimum_version must be 1.2 or 1.3" - ))); - } if policy.certificate_path.is_empty() != policy.private_key_path.is_empty() { return Err(SnapshotError::invalid(format!( "{field} certificate and private key must be configured together" @@ -1092,11 +1109,9 @@ fn validate_tls_policy( } fn validate_config(config: &ConfigSnapshot) -> Result<(), SnapshotError> { - if config.traffic_replay_enabled { - return Err(SnapshotError::unsupported( - "traffic replay is unsupported by the Rust dataplane", - )); - } + // `traffic_replay_enabled` stays in the snapshot so the checksum matches + // Go. The native process does not capture or replay; that is reported by + // the traffic API, and it does not reject the serving generation. if !config.high_memory_reject_threshold.is_finite() || config.high_memory_reject_threshold < 0.0 || config.high_memory_reject_threshold > 1.0 @@ -1141,7 +1156,9 @@ fn validate_config(config: &ConfigSnapshot) -> Result<(), SnapshotError> { if listener.name.is_empty() || listener.name.len() > 128 { return Err(SnapshotError::invalid("listener name is invalid")); } - if listener.port == 0 || listener.port > u32::from(u16::MAX) { + // Port 0 is Go's empty `proxy.addr`: `net.Listen("tcp", "")` binds an + // ephemeral port instead of failing configuration validation. + if listener.port > u32::from(u16::MAX) { return Err(SnapshotError::invalid("listener port is invalid")); } validate_host(&listener.address, "listener address")?; diff --git a/rust/crates/control-proto/tests/snapshot_store.rs b/rust/crates/control-proto/tests/snapshot_store.rs index af3d00f1c..abe1d407e 100644 --- a/rust/crates/control-proto/tests/snapshot_store.rs +++ b/rust/crates/control-proto/tests/snapshot_store.rs @@ -242,12 +242,17 @@ fn expiry_ca_tls_policy_and_unsupported_configuration_are_rejected_atomically() .as_mut() .ok_or("missing config")? .traffic_replay_enabled = true; - let replay_error = store - .apply(4, replay, validation_time(), lineage_a()) - .err() - .ok_or("traffic replay unexpectedly applied")?; - assert_eq!(replay_error.kind(), SnapshotErrorKind::Unsupported); - assert_eq!(store.current()?.ok_or("missing last-good")?.generation(), 3); + let applied = store.apply(4, replay, validation_time(), lineage_a())?; + assert!( + applied + .snapshot + .raw() + .config + .as_ref() + .is_some_and(|config| config.traffic_replay_enabled), + "the Go default traffic-replay flag is retained and does not reject serving" + ); + assert_eq!(store.current()?.ok_or("missing last-good")?.generation(), 4); Ok(()) } @@ -588,11 +593,6 @@ fn listener_config_bound_cases() -> Vec { Box::new(|config: &mut ConfigSnapshot| config.listeners.clear()), "listeners must contain", ), - ( - "a listener port of zero", - Box::new(|config: &mut ConfigSnapshot| config.listeners[0].port = 0), - "listener port", - ), ( "a listener port beyond u16", Box::new(|config: &mut ConfigSnapshot| config.listeners[0].port = 70_000), @@ -701,7 +701,17 @@ fn config_bounds_are_rejected_without_disturbing_last_good() -> Result<(), Box Option { self.redirect_deadline diff --git a/rust/crates/dataplane/src/server.rs b/rust/crates/dataplane/src/server.rs index 2bfa0790f..c7c36a8f1 100644 --- a/rust/crates/dataplane/src/server.rs +++ b/rust/crates/dataplane/src/server.rs @@ -555,12 +555,12 @@ impl DataplaneServer { /// /// # Errors /// -/// Returns an explicit unsupported error for traffic capture/replay and a -/// configuration error when the complete config is absent. +/// Returns a configuration error when the complete config is absent. +/// `traffic_replay_enabled` does not fail preflight: the Go default is true, +/// and the native process serves SQL without implementing capture or replay. pub fn preflight_snapshot(snapshot: &StateSnapshot) -> Result<(), ServerError> { - let config = snapshot.config.as_ref().ok_or(ServerError::MissingConfig)?; - if config.traffic_replay_enabled { - return Err(ServerError::TrafficReplayUnsupported); + if snapshot.config.is_none() { + return Err(ServerError::MissingConfig); } Ok(()) } @@ -1722,9 +1722,13 @@ mod tests { }), ..Default::default() }; + assert!( + preflight_snapshot(&raw).is_ok(), + "the Go default enable-traffic-replay=true must still serve" + ); assert!(matches!( - preflight_snapshot(&raw), - Err(ServerError::TrafficReplayUnsupported) + preflight_snapshot(&StateSnapshot::default()), + Err(ServerError::MissingConfig) )); let mut backoff = AcceptBackoff::new(); assert_eq!(backoff.fail(), ACCEPT_BACKOFF_INITIAL); diff --git a/rust/crates/dataplane/src/session_engine.rs b/rust/crates/dataplane/src/session_engine.rs index 6add54773..8faf0fc62 100644 --- a/rust/crates/dataplane/src/session_engine.rs +++ b/rust/crates/dataplane/src/session_engine.rs @@ -987,14 +987,16 @@ async fn run_bound_session_observed( continue; } let assignment = redirect.to(); - let deadline = u64::try_from( - SystemTime::now() - .duration_since(UNIX_EPOCH) - .unwrap_or_default() - .saturating_add(budget) - .as_millis(), - ) - .unwrap_or(u64::MAX); + // `budget` was the queue TTL, already enforced above. + // Do not stamp it as an absolute handshake deadline. + // Go `tryRedirect` keeps `redirectInfo` with no timeout + // while `!finishedTxn()` and dials only after the next + // command leaves the transaction. An absolute + // acceptance-time deadline expires that wait (a + // transaction held for 30s) and the handshake then + // fails without migrating. Zero means + // `candidate_budget` applies the dial schedule when + // the handshake actually starts. let target = RedirectTarget { backend_id: assignment.backend_id.clone(), backend_address: assignment.backend_address.clone(), @@ -1002,7 +1004,7 @@ async fn run_bound_session_observed( keyspace: assignment.keyspace.clone(), backend_healthy: assignment.healthy, backend_local: assignment.local, - deadline_unix_millis: deadline, + deadline_unix_millis: 0, }; local_redirect = Some(envelope); // Atomic redirect pair: one operation permit spans both @@ -2182,17 +2184,23 @@ impl Engine { let backend_cluster = acquired.backend.cluster_name.clone(); let backend_keyspace = acquired.backend.keyspace.clone(); let backend_local = acquired.backend.local; - // A metered session must know its immutable source attribution before - // consuming or forwarding any backend protocol bytes. RouteEngine has - // only completed the TCP dial here: the greeting is still unread and - // the socket has not entered the counted/billable transport. Reject an - // incomplete assignment locally without poisoning the process-wide - // registry; this is the session-scoped fail-closed policy. - if self.metering.is_some() && (backend_id.is_empty() || backend_keyspace.is_empty()) { + // A metered session must know its backend id before consuming or + // forwarding any backend protocol bytes. RouteEngine has only + // completed the TCP dial here. Reject a missing backend id locally + // without poisoning the process-wide registry. + // + // An empty keyspace is the classic topology (`/topology/tidb/`). Go + // `BackendConnManager.updateTraffic` skips metering when `Keyspace()` + // is empty and still serves the session. Detach metering for that + // session so a later register does not fail the handshake. + if self.metering.is_some() && backend_id.is_empty() { self.quit_source = QuitSource::ProxyError; let _ = self.events.send(SessionEvent::BackendIoError).await; return Some(WireErrorSource::Proxy); } + if backend_keyspace.is_empty() { + self.metering = None; + } // Health-appropriate keepalive at dial time (KA-003 family): // the snapshot's healthy/unhealthy backend policy follows the // router-reported health of this assignment. Mid-session diff --git a/rust/crates/dataplane/tests/session_engine.rs b/rust/crates/dataplane/tests/session_engine.rs index 6467e4ad2..3cbce5a8d 100644 --- a/rust/crates/dataplane/tests/session_engine.rs +++ b/rust/crates/dataplane/tests/session_engine.rs @@ -3351,23 +3351,30 @@ async fn existing_session_reapplies_keepalive_on_backend_health_flip() { stack.server_task.abort(); } +/// An empty keyspace is classic `/topology/tidb/` topology. Go still proxies +/// the session and skips `IncTraffic` while `Keyspace()` is empty, so metering +/// is detached and no billable source is registered. #[tokio::test] -async fn metering_unknown_initial_keyspace_rejects_before_billable_forwarding() { +async fn metering_classic_empty_keyspace_serves_without_a_billable_source() { let Ok(registry) = MeteringSourceRegistry::new(7) else { unreachable!("metering registry") }; let stack = spawn_metered_stack(registry.clone()).await; spawn_route_answer(&stack, 1, 2); - let client = timeout(Duration::from_secs(5), MysqlClient::connect(stack.sql_port)) + let Some(mut client) = timeout(Duration::from_secs(5), MysqlClient::connect(stack.sql_port)) .await .ok() - .flatten(); - assert!(client.is_none(), "unknown keyspace must reject the session"); + .flatten() + else { + unreachable!("classic empty keyspace still serves the session") + }; + assert!(client.query_ok("SELECT classic").await); assert_eq!( metering_source_count(®istry), 0, - "an un-attributed backend never becomes a billable source" + "an empty keyspace is not a billable source" ); + client.quit().await; stack.dispatch_task.abort(); } @@ -5273,6 +5280,9 @@ async fn backend_to_client_destination_write_break_is_client_network_on_both_obs let Ok(stream) = reader.into_inner().reunite(writer.into_inner()) else { unreachable!("the client's split halves reunite") }; + // Tokio 1.51.6+ deprecates this because SO_LINGER can block the drop. The + // test still wants RST rather than a buffered FIN. + #[allow(deprecated)] let _ = stream.set_linger(Some(Duration::from_secs(0))); drop(stream); diff --git a/rust/crates/proxy-io/src/tls.rs b/rust/crates/proxy-io/src/tls.rs index d802622a6..9d71ceaec 100644 --- a/rust/crates/proxy-io/src/tls.rs +++ b/rust/crates/proxy-io/src/tls.rs @@ -304,6 +304,8 @@ where pub fn build_backend_config( policy: &ValidatedTlsPolicy, ) -> Result, TlsSetupError> { + // `1.3` is TLS 1.3 only. Go also accepts 1.0 and 1.1; rustls cannot offer + // them, so every other value uses the TLS 1.2 floor. let builder = if policy.minimum_version == "1.3" { ClientConfig::builder_with_protocol_versions(&[&rustls::version::TLS13]) } else { diff --git a/rust/crates/tiproxy-rs/src/config_composition.rs b/rust/crates/tiproxy-rs/src/config_composition.rs index 4b0d9b0b8..10e2128c2 100644 --- a/rust/crates/tiproxy-rs/src/config_composition.rs +++ b/rust/crates/tiproxy-rs/src/config_composition.rs @@ -337,6 +337,12 @@ impl CandidateValidator for ServingCandidateValidator { { return Err("server_auto_certs_unsupported"); } + if effective.api_proxy_protocol() == "v2" { + return Err("api_proxy_protocol_unsupported"); + } + if !effective.ha_virtual_ip().is_empty() && !effective.ha_interface().is_empty() { + return Err("ha_vip_unsupported"); + } let mut serving = effective.serving().map_err(|_| "serving_projection")?; apply_drain_override(&mut serving, self.drain_grace_override); let candidate = StateSnapshot { @@ -794,16 +800,23 @@ mod tests { -> Result<(), Box> { let snapshots = SnapshotStore::new([])?; let validator = Arc::new(ServingCandidateValidator::new(snapshots, None)); - let result = ConfigNamespaceStore::new_with_validator( + let store = ConfigNamespaceStore::new_with_validator( EffectiveConfig::default(), Vec::new(), SourceRevision::default(), Path::new("/tmp"), validator, + )?; + assert!( + store.current().effective().traffic_replay_enabled(), + "the Go default enable-traffic-replay=true must be accepted" ); assert!( - result.is_err(), - "traffic replay defaults true and Rust rejects it" + store + .current() + .effective() + .serving()? + .traffic_replay_enabled ); let server_auto = ConfigNamespaceStore::from_toml_with_validator( b"enable-traffic-replay = false\n[security.server-tls]\nauto-certs = true\n", @@ -820,6 +833,57 @@ mod tests { class: "serving_auto_certs_unsupported" }) )); + for (toml, class) in [ + ( + &b"api.proxy-protocol = \"v2\"\n"[..], + "api_proxy_protocol_unsupported", + ), + ( + &b"ha.virtual-ip = \"192.0.2.10\"\nha.interface = \"eth0\"\n"[..], + "ha_vip_unsupported", + ), + ] { + let rejected = ConfigNamespaceStore::from_toml_with_validator( + toml, + None, + Path::new("/tmp"), + Arc::new(ServingCandidateValidator::new( + SnapshotStore::new([])?, + None, + )), + ); + assert!( + matches!( + rejected, + Err(StoreError::CandidateRejected { class: got }) if got == class + ), + "{class} was not rejected" + ); + } + let vip_only = ConfigNamespaceStore::from_toml_with_validator( + b"ha.virtual-ip = \"192.0.2.10\"\n", + None, + Path::new("/tmp"), + Arc::new(ServingCandidateValidator::new( + SnapshotStore::new([])?, + None, + )), + )?; + assert_eq!(vip_only.current().effective().ha_virtual_ip(), "192.0.2.10"); + let encryption = ConfigNamespaceStore::from_toml_with_validator( + b"security.encryption-key-path = \"/tmp/key\"\n", + None, + Path::new("/tmp"), + Arc::new(ServingCandidateValidator::new( + SnapshotStore::new([])?, + None, + )), + )?; + assert_eq!( + encryption.current().effective().encryption_key_path(), + "/tmp/key", + "Go keeps security.encryption-key-path and still starts" + ); Ok(()) } diff --git a/rust/crates/tiproxy-rs/src/main.rs b/rust/crates/tiproxy-rs/src/main.rs index bf958f812..416c29b90 100644 --- a/rust/crates/tiproxy-rs/src/main.rs +++ b/rust/crates/tiproxy-rs/src/main.rs @@ -112,6 +112,8 @@ struct Options { metrics_addr: Option, admin_addr: Option, log_file: Option, + /// Go `--advertise-addr`. Overrides `proxy.advertise-addr` when non-empty. + advertise_addr: Option, } enum Command { @@ -119,6 +121,8 @@ enum Command { Version, Help, IntegrationCapabilities, + /// Prints the machine-readable field contract and exits, like Go `--config-info json`. + ConfigInfo, } /// The integration harness's capability contract (DPL-07): only what @@ -152,6 +156,10 @@ async fn main() -> ExitCode { println!("{INTEGRATION_CAPABILITIES}"); ExitCode::SUCCESS } + Command::ConfigInfo => { + println!("{}", control_config::field_contract_json()); + ExitCode::SUCCESS + } Command::Run(options) => match run(*options).await { Ok(()) => ExitCode::SUCCESS, Err(error) => { @@ -1584,7 +1592,7 @@ fn load_config_owner(options: &Options, process_id: &str) -> Result Result, String> { - if config.skip_ca_verification { - return Err("config persistence does not support skip-ca-verification".to_owned()); - } - let configured = config.ca_path.is_some() + // Same rule as topology cluster clients: skip-ca still performs the TLS + // handshake and does not require a CA. It must not fall back to plaintext. + let configured = config.skip_ca_verification + || config.ca_path.is_some() || config.certificate_path.is_some() || config.private_key_path.is_some(); if !configured { return Ok(None); } - let ca_path = config + let policy = control_external::EtcdTlsPolicy { + minimum_version: control_external::EtcdTlsVersion::parse(&config.minimum_version) + .map_err(|_| "config persistence TLS minimum version".to_owned())?, + allowed_common_names: config + .allowed_common_names + .iter() + .map(ToString::to_string) + .collect(), + skip_ca_verification: config.skip_ca_verification, + }; + let ca = config .ca_path .as_deref() - .ok_or_else(|| "config persistence TLS requires a CA".to_owned())?; - let ca = tls_material::read_tls_material(ca_path, allowed_tls_roots) + .map(|path| tls_material::read_tls_material(path, allowed_tls_roots)) + .transpose() .map_err(|_| "read config persistence TLS CA".to_owned())?; let certificate = read_optional_tls(config.certificate_path.as_deref(), allowed_tls_roots)?; let key = read_optional_tls(config.private_key_path.as_deref(), allowed_tls_roots)?; - EtcdTlsConfig::new( - Some(ca), - certificate, - key, - None, - control_external::EtcdTlsPolicy::default(), - ) - .map(Some) - .map_err(|error| format!("validate config persistence TLS: {error}")) + EtcdTlsConfig::new(ca, certificate, key, None, policy) + .map(Some) + .map_err(|error| format!("validate config persistence TLS: {error}")) } fn read_optional_tls( @@ -1983,6 +1995,14 @@ impl control_admin::ConfigAdmin for OwnerConfigAdmin { ) -> control_admin::AdminFuture<'_, Result<(), control_config::ConfigMutationError>> { Box::pin(self.handle.apply_local_toml(data)) } + + fn traffic_replay_enabled(&self) -> bool { + self.handle + .source() + .current() + .effective() + .traffic_replay_enabled() + } } /// `control_admin::DrainAdmin` over the dispatch owner: the operator label @@ -2178,6 +2198,7 @@ fn parse_options(arguments: impl IntoIterator) -> Result = None; let mut admin_addr: Option = None; let mut log_file: Option = None; + let mut advertise_addr: Option = None; let mut routing_shadow_socket = None; let mut standalone = false; let mut socket_argument = false; @@ -2186,6 +2207,28 @@ fn parse_options(arguments: impl IntoIterator) -> Result return Ok(Command::Version), "--help" | "-h" => return Ok(Command::Help), "--integration-capabilities" => return Ok(Command::IntegrationCapabilities), + "--config-info" => { + let format = arguments + .next() + .ok_or_else(|| "--config-info requires a format".to_owned())?; + if format != "json" { + return Err("only support json format".to_owned()); + } + return Ok(Command::ConfigInfo); + } + "--advertise-addr" => { + advertise_addr = Some( + arguments + .next() + .ok_or_else(|| "--advertise-addr requires an address".to_owned())?, + ); + } + // Go still accepts these flags and ignores their values. + "--log_encoder" | "--log_level" | "--log-encoder" | "--log-level" => { + arguments + .next() + .ok_or_else(|| format!("{argument} requires a value"))?; + } "--standalone" => standalone = true, "--config" => { config_file = Some(PathBuf::from( @@ -2313,6 +2356,7 @@ fn parse_options(arguments: impl IntoIterator) -> Result &'static str { "Usage: tiproxy-rs --config (--standalone | --control-socket --control-uid ) \ - [--tls-root ]... [--drain-grace-seconds ] [--health-port ] [--metrics-addr ] [--admin-addr ] [--log-file ] [--routing-shadow-socket ]\n\ + [--advertise-addr ] [--tls-root ]... [--drain-grace-seconds ] [--health-port ] [--metrics-addr ] [--admin-addr ] [--log-file ] [--routing-shadow-socket ]\n\ + tiproxy-rs --config-info json\n\ + --advertise-addr overrides proxy.advertise-addr when non-empty. --admin-addr overrides the bind of api.addr. \ + --tls-root is added to rust-dataplane.tls-allowed-roots. --log-file is the Rust log path. \ + --drain-grace-seconds overrides proxy.graceful-close-conn-timeout. --health-port and --metrics-addr are CLI-only. \ + Deprecated --log_encoder and --log_level are accepted and ignored.\n\ Environment: TIPROXY_CONFIG, TIPROXY_CONTROL_SOCKET, TIPROXY_CONTROL_UID, TIPROXY_TLS_ROOTS" } @@ -3323,6 +3372,7 @@ mod tests { metrics_addr: None, admin_addr: None, log_file: None, + advertise_addr: None, } ); } @@ -3407,6 +3457,7 @@ mod tests { metrics_addr: None, admin_addr: None, log_file: None, + advertise_addr: None, }; let source = ConfigNamespaceStore::from_toml( b"enable-traffic-replay = false\n", @@ -3508,20 +3559,53 @@ pd-addrs = "routing-pd:2379" } #[test] - fn persistence_rejects_skip_ca_instead_of_silently_downgrading_to_plaintext() { + fn persistence_skip_ca_uses_tls_without_a_ca() { let source = ConfigNamespaceStore::from_toml( b"[security.cluster-tls]\nskip-ca = true\n", None, std::path::Path::new("/tmp"), ) .unwrap_or_else(|error| unreachable!("valid source model: {error}")); + let client = config_persistence_client( + source.current().effective(), + &crate::tls_material::open_tls_roots(&[]), + ) + .unwrap_or_else(|error| unreachable!("skip-ca is a TLS client: {error}")) + .unwrap_or_else(|| unreachable!("default PD address configures persistence")); + assert_eq!( + client.endpoints(), + ["https://127.0.0.1:2379"], + "skip-ca still performs the TLS handshake" + ); + } + + #[test] + fn advertise_addr_and_config_info_follow_the_go_cli() { + let command = parse_options([ + "--config".to_owned(), + "/etc/tiproxy/tiproxy.toml".to_owned(), + "--standalone".to_owned(), + "--advertise-addr".to_owned(), + "192.0.2.10".to_owned(), + "--log_encoder".to_owned(), + "tidb".to_owned(), + "--log-level".to_owned(), + "info".to_owned(), + ]); + let Ok(Command::Run(options)) = command else { + unreachable!("deprecated log flags are ignored") + }; + assert_eq!(options.advertise_addr.as_deref(), Some("192.0.2.10")); + assert!(matches!( + parse_options(["--config-info".to_owned(), "json".to_owned()]), + Ok(Command::ConfigInfo) + )); + let Err(error) = parse_options(["--config-info".to_owned(), "yaml".to_owned()]) else { + unreachable!("only json is supported"); + }; assert!( - config_persistence_client( - source.current().effective(), - &crate::tls_material::open_tls_roots(&[]) - ) - .is_err(), - "skip-ca without a CA must not silently construct a plaintext etcd client" + error.contains("only support json format"), + "unexpected error: {error}" ); } diff --git a/rust/crates/tiproxy-rs/src/tls_material.rs b/rust/crates/tiproxy-rs/src/tls_material.rs index 072600217..7270e01a8 100644 --- a/rust/crates/tiproxy-rs/src/tls_material.rs +++ b/rust/crates/tiproxy-rs/src/tls_material.rs @@ -105,13 +105,40 @@ fn open_dir_capability(canonical_dir: &Path) -> Result { /// # Errors /// /// Returns a bounded, payload-free reason (never the path or the material) when -/// the path is not absolute, resolves outside the roots, has a symlink -/// component, is not a regular file, or exceeds 16 MiB. +/// the path cannot be resolved, escapes a configured root, has a symlink +/// component inside a root, is not a regular file, or exceeds 16 MiB. A +/// relative path is resolved against the process working directory. An empty +/// root list does not add a jail. pub fn read_tls_material(path: &Path, roots: &TlsRoots) -> Result, &'static str> { - let selection = select_root(path, roots)?; + let path = absolute_tls_path(path)?; + if roots.roots.is_empty() { + // Go has no TLS-root jail. An empty allowlist reads the path the + // process working directory resolved, with the same size bound. + return read_unconfined(&path); + } + let selection = select_root(&path, roots)?; traverse_and_read(&selection) } +fn absolute_tls_path(path: &Path) -> Result { + if path.is_absolute() { + Ok(path.to_path_buf()) + } else { + Ok(std::env::current_dir() + .map_err(|_| "tls material path is unavailable")? + .join(path)) + } +} + +fn read_unconfined(path: &Path) -> Result, &'static str> { + let mut file = File::open(path).map_err(|_| "tls material is unavailable")?; + let metadata = file.metadata().map_err(|_| "tls material is unavailable")?; + if !metadata.is_file() { + return Err("tls material must be a regular file"); + } + read_bounded_prefix(&mut file) +} + /// The frozen root plus the root-relative parent components and the /// (uncanonicalized) final basename that a read will traverse. struct Selection<'a> { @@ -241,11 +268,26 @@ mod tests { } #[test] - fn a_relative_path_is_rejected() { + fn a_missing_relative_path_is_unavailable() { let roots = open_tls_roots(&[]); assert!(read_tls_material(std::path::Path::new("ca.pem"), &roots).is_err()); } + #[test] + fn a_relative_path_with_no_roots_reads_from_the_working_directory() { + let name = format!("cptopo-rel-{}.pem", std::process::id()); + let absolute = std::env::current_dir() + .unwrap_or_else(|error| unreachable!("cwd: {error}")) + .join(&name); + let _ = std::fs::remove_file(&absolute); + write(&absolute, b"rel-ca"); + let roots = open_tls_roots(&[]); + let bytes = read_tls_material(std::path::Path::new(&name), &roots) + .unwrap_or_else(|error| unreachable!("relative read: {error}")); + assert_eq!(bytes, b"rel-ca"); + let _ = std::fs::remove_file(&absolute); + } + #[test] fn a_file_outside_the_allowed_roots_is_rejected() { let dir = unique_dir("outside-file"); diff --git a/tests/controlplane/contracts.v1.json b/tests/controlplane/contracts.v1.json index 3f9554e00..46103cde9 100644 --- a/tests/controlplane/contracts.v1.json +++ b/tests/controlplane/contracts.v1.json @@ -121,7 +121,7 @@ "anchors": [ {"path": "pkg/manager/cert/manager_test.go", "contains": "func TestRotate"}, {"path": "pkg/manager/cert/manager_test.go", "contains": "func TestWatchConfig"}, - {"path": "pkg/controlbridge/snapshot.go", "contains": "minimum TLS version must be 1.2 or 1.3"}, + {"path": "pkg/controlbridge/snapshot.go", "contains": "func effectiveMinTLSVersion"}, {"path": "rust/crates/control-config/src/source.rs", "contains": "pub fn refresh_external_material"}, {"path": "rust/crates/tiproxy-rs/src/config_composition.rs", "contains": "pub struct ServingCandidateValidator"}, {"path": "rust/crates/control-config/tests/source.rs", "contains": "fn external_material_refresh_advances_generation_and_retains_last_good_on_rejection"} diff --git a/tests/controlplane/cpadmin/README.md b/tests/controlplane/cpadmin/README.md index 4c6823057..26f28c1c1 100644 --- a/tests/controlplane/cpadmin/README.md +++ b/tests/controlplane/cpadmin/README.md @@ -5,8 +5,8 @@ observations exactly: - **Go**: `pkg/server/api.TestCPAdminCapture` builds the production gin engine through the package's own `createServerWithConfig` helper (real listener, - the unit-test mocks, `enable-traffic-replay = false` as the Rust dataplane - composition requires) and replays `script.json`. Actions in the script toggle + the unit-test mocks, `enable-traffic-replay = false` so both sides answer + the traffic routes with `disabled`) and replays `script.json`. Actions in the script toggle the readiness gate, the namespace manager readiness, the dataplane status reader and `PreClose`, exactly as the Go unit tests do. - **Rust**: `control-admin`'s `cpadmin_replay` example drives the same script diff --git a/tests/controlplane/cpctl/README.md b/tests/controlplane/cpctl/README.md index 852d3e055..d86909466 100644 --- a/tests/controlplane/cpctl/README.md +++ b/tests/controlplane/cpctl/README.md @@ -7,8 +7,8 @@ auto certificates driven with `--insecure`) and twice against the Rust admin listener (`rust/crates/control-admin/examples/cpctl_replay.rs`, same modes), then compares every command's exit code and stdout: `health`, `config get/set`, `namespace list/put/get/import/commit/del` (including missing namespaces) and -the four `traffic` subcommands, which both sides refuse while traffic replay is -disabled. `config get` and the JSON outputs are compared semantically (the TOML +the four `traffic` subcommands, which both sides refuse. The evidence +overlay leaves traffic replay disabled, so both answers use that message. `config get` and the JSON outputs are compared semantically (the TOML renderers differ in layout); everything else must match byte for byte. `@name` arguments refer to the fixture files the script writes on both sides. The CLI's `namespace list` on an empty set fails on both sides (the server