feat: Added separate Database application. - #1230
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a dedicated Watt database background application and routes PostgreSQL operations from the storage app through Platformatic messaging when Watt messaging is available, with tests and acceptance coverage to validate the messaging-based DB path.
Changes:
- Adds Watt runtime configuration for running
storageanddatabaseas separate applications. - Implements a Database Watt worker (pooling, destination resolution, locks, cancellation, limits, metrics) and a storage-side messaging adapter.
- Adds unit tests plus a new Watt-managed acceptance runner and acceptance spec for Database Watt behavior.
Reviewed changes
Copilot reviewed 38 out of 40 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| watt.storage.json | Defines the storage Watt app entrypoint for runtime-managed execution. |
| watt.json | Converts to a Platformatic runtime config and registers storage + database applications. |
| watt.database.json | Defines the database Watt app entrypoint as a background (no server) worker. |
| src/internal/database/watt-connection.ts | Adds storage-side messaging adapter for tenant connections and transactions. |
| src/internal/database/watt-connection.test.ts | Unit tests for messaging adapter behavior (queries, tx lifecycle, errors, cancellation). |
| src/internal/database/pg-connection.ts | Introduces PgTransactionLike / PgTenantConnectionLike interfaces and aligns classes to them. |
| src/internal/database/multitenant-pg.ts | Routes master executor work through Database Watt when messaging is available. |
| src/internal/database/multitenant-pg.test.ts | Tests Watt routing selection for multitenant master queries/transactions. |
| src/internal/database/migrations/migrate.ts | Documents that migrations intentionally remain direct PG (out of Database Watt scope). |
| src/internal/database/client.ts | Routes getPostgresConnection() via Database Watt when messaging is available. |
| src/internal/database/client.test.ts | Tests Watt routing and multitenant forwarded-host validation behavior. |
| src/database/index.ts | Implements Database Watt app handlers and lifecycle (query/tx/locks/cancel/stats/shutdown). |
| src/database/config.ts | Adds Database Watt configuration parsing from environment. |
| src/database/types.ts | Defines Database Watt wire request/response types. |
| src/database/validation.ts | Adds validation helpers for request envelopes, metadata bounds, and payload size. |
| src/database/errors.ts | Defines Database Watt error contract and error serialization to wire responses. |
| src/database/destinations.ts | Implements destination resolution for single-tenant and multitenant/master routing. |
| src/database/pools.ts | Implements pool registry, acquisition limits, eviction, and query execution. |
| src/database/locks.ts | Implements lock registry to pin connections for tx/locked queries and serialize per-lock work. |
| src/database/cancellation.ts | Implements cancellation tracking and PG backend cancel signaling for in-flight work. |
| src/database/result-limits.ts | Enforces row/byte result limits for messaging responses. |
| src/database/ssl.ts | Implements sslmode-derived TLS settings for database connections. |
| src/database/metrics.ts | Registers observable gauges for pool/connection stats via OpenTelemetry. |
| src/database/pg-lib-connection.d.ts | Declares types for pg/lib/connection used by cancellation support. |
| src/database/tests/index.test.ts | Tests Database Watt handlers end-to-end with loopback messaging and mocked pg. |
| src/database/tests/validation.test.ts | Tests request validation bounds and error behavior. |
| src/database/tests/ssl.test.ts | Tests SSL settings behavior for common sslmode scenarios. |
| src/database/tests/result-limits.test.ts | Tests result size/row enforcement behavior. |
| src/database/tests/locks.test.ts | Tests lock registry query/tx behaviors and serialization. |
| src/database/tests/errors.test.ts | Tests error serialization/mapping contract. |
| src/database/tests/destinations.test.ts | Tests single-tenant destination/external pool resolution. |
| src/database/tests/cancellation.test.ts | Tests cancellation registry behavior and PG cancel wiring. |
| src/admin-app.ts | Switches Watt-detection to hasField('applicationId'). |
| src/admin-app.test.ts | Updates admin app tests to use globals update/remove patterns instead of getGlobal mocking. |
| docs/database-watt-postgresql-scope.md | Documents what DB access is routed through Database Watt vs intentionally left direct. |
| acceptance/specs/database-watt.test.ts | Adds acceptance tests validating Database Watt query/tx/cancel/error and multitenant behaviors. |
| acceptance/scripts/run-managed-watt-local.ts | Adds a Watt-managed local acceptance runner that boots both apps and runs acceptance. |
| acceptance/scripts/run-managed-local.ts | Explicitly disables Database Watt acceptance when running the non-Watt managed script. |
| package.json | Adds acceptance:watt script and @platformatic/runtime devDependency. |
| package-lock.json | Locks @platformatic/runtime addition. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| async acquire(destination: DestinationConfig): Promise<PoolClient> { | ||
| const entry = this.getOrCreatePool(destination) | ||
| this.assertCanAcquire(entry) | ||
| this.pendingGlobalAcquisitions++ | ||
|
|
||
| const timeout = createTimeout(this.config.acquireTimeoutMs) | ||
| try { | ||
| return await Promise.race([ | ||
| entry.pool.connect(), | ||
| timeout.promise.then(() => { | ||
| throw new DatabaseWattError('ACQUIRE_TIMEOUT', 'Timed out acquiring database connection') | ||
| }), | ||
| ]) | ||
| } finally { | ||
| timeout.clear() | ||
| this.pendingGlobalAcquisitions-- | ||
| entry.lastUsedAt = Date.now() | ||
| } | ||
| } |
| if (request.readOnly) { | ||
| modes.push('READ ONLY') | ||
| } |
| export async function getWattPostgresConnection( | ||
| options: TenantConnectionOptions | ||
| ): Promise<PgTenantConnection> { | ||
| return new WattPgTenantConnection(options) as unknown as PgTenantConnection | ||
| } |
| async beginTransaction(options?: TransactionOptions): Promise<PgTransaction> { | ||
| const response = await sendDatabaseMessage<{ lockId: string }>('database.beginTransaction', { | ||
| destination: this.destination, | ||
| isolationLevel: options?.isolation, | ||
| operationName: this.operation?.(), | ||
| readOnly: options?.readOnly, | ||
| }) | ||
|
|
||
| return new WattPgTransaction(response.lockId, this.operation) as unknown as PgTransaction | ||
| } |
| export function toErrorResponse(error: unknown, context: ErrorContext = {}): DatabaseErrorResponse { | ||
| if (error instanceof DatabaseWattError) { | ||
| return { | ||
| code: error.code, | ||
| message: error.message, | ||
| requestId: context.requestId, | ||
| operationName: context.operationName, | ||
| destination: context.destination, | ||
| lockId: context.lockId, | ||
| sqlState: error.sqlState, | ||
| stack: error.stack, | ||
| connectionDiscarded: error.connectionDiscarded, | ||
| } |
Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
Coverage Report for CI Build 30076416950Coverage decreased (-0.1%) to 80.268%Details
Uncovered Changes
Coverage RegressionsNo coverage regressions found. Coverage Stats💛 - Coveralls |
# Conflicts: # acceptance/specs/admin.test.ts # src/internal/database/pg-connection.ts
Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
# Conflicts: # Dockerfile # src/admin-app.test.ts # src/admin-app.ts # src/http/plugins/db.ts # src/internal/database/multitenant-pg.ts # src/internal/database/pg-connection.ts # src/storage/database/adapter.ts # src/storage/database/pg.ts # watt.json # watt.storage.json
Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
| this.role = options.user.payload.role || 'anon' | ||
| this.executor = new WattPgExecutor( | ||
| { | ||
| connectionString: options.dbUrl, |
There was a problem hiding this comment.
🟡 Severity: MEDIUM
The full database connection URL—including embedded credentials (username and password)—is placed into the DatabasePoolTarget.connectionString field and transmitted with every database.query, database.acquire, database.beginTransaction, and database.lockedQuery message over the inter-process messaging bus. In multi-tenant deployments each tenant's individual DB URL is serialized per operation. If the Platformatic messaging layer emits debug-level traces or if monitoring captures message payloads, database credentials will be logged for every DB operation.
Helpful? Add 👍 / 👎
💡 Fix Suggestion
Suggestion: Replace the raw connectionString (full database URL with embedded credentials) in DatabasePoolTarget with an opaque pool identifier so that database credentials never travel over the inter-process messaging bus.
The recommended approach is:
-
Add a pool-registration message (e.g.,
database.registerPool) toprotocol.tsthat accepts theconnectionStringonce (during application startup or first use) and returns a stable, opaquepoolId(e.g., a UUID or a hash). The database application side stores the mapping ofpoolId → connectionStringin memory. -
Update
DatabasePoolTargetinprotocol.tsto replaceconnectionString: stringwithpoolId: string. -
Update
WattPgTenantConnection(around line 147–152 inconnection.ts) to call the newdatabase.registerPoolmessage on first use and cache the returnedpoolIdkeyed by the connection URL. Pass thepoolIdinstead of the rawconnectionStringintoDatabasePoolTarget. -
Update
PoolRegistryinpools.tsand the application handlers inapplication.tsto resolve apoolIdback to the actualconnectionStringusing the in-memory registry before creating or reusing a pool.
This ensures credentials are transmitted exactly once (during pool registration, over an already-trusted IPC channel) and all subsequent database.query, database.acquire, and database.beginTransaction messages carry only the opaque poolId, eliminating credential exposure in every operation's message payload.
This PR adds a separate Watt
databaseapplication and routes PostgreSQL operations through Platformatic messaging when Watt messaging is available.Main areas:
Review Order
watt.jsonwatt.database.jsonsrc/database/index.tssrc/database/pools.tssrc/database/locks.tssrc/internal/database/client.tssrc/internal/database/watt-connection.tsacceptance/specs/database-watt.test.tsHigh-Risk Areas
WattPgTenantConnectionandWattPgTransactionare cast toPgTenantConnection/PgTransaction, not actual instances. Review anyinstanceof PgTransactionusage.toPgQueryResult()returns emptycommand,fields, andoid:src/internal/database/watt-connection.ts:372. Any caller relying on those fields may regress.src/database/pools.ts:138.pool.connect():src/database/pools.ts:57. Verify no delayed client leak.beginTransactionvalidation does not appear to validateisolationLevel/readOnly:src/database/validation.ts:5.src/internal/database/watt-connection.ts:339. Confirm this is acceptable.watt.jsonuses schema/runtime3.56.0, while Platformatic dependencies resolve from^3.59.0.Manual Review Focus
Check these behaviors carefully:
Assisted-By: OpenAI:GPT-5.6-Sol <openai/gpt-5.6-sol>