From abeef31aca9166043f68e67637a4deec9f01d0bf Mon Sep 17 00:00:00 2001 From: fullsend-code <278716306+fullsend-ai-coder[bot]@users.noreply.github.com> Date: Fri, 18 Sep 2026 18:16:53 +0000 Subject: [PATCH 01/63] feat(#4815): implement MCP registry provider backend plugin Add catalog-backend-module-mcp-registry-provider, a Backstage catalog backend module that ingests MCP servers from one configured MCP Registry into the RHDH catalog as mcp-server API entities. Implementation includes: - Config reading at catalog.providers.mcpRegistry (single object) with baseUrl (required), baseName, apiVersion (default v1), defaultOwner, pageLimit (default 10), pageSize, and schedule (default 30m/3m) - Registry client with full cursor pagination, page cap safeguard, repeated-cursor detection, and typed error handling - EntityProvider with full-mutation semantics (catalog converges to registry state), per-entry failure isolation with last-good retention (D6), and sync status annotation (ok/degraded per D8) - Provider attribution: locationKey mcp-registry-provider, backstage.io/managed-by-location url: - Delegates entirely to mcp-registry-server-mapping-common for the server.json to entity transform and annotation projection - Keyed multi-registry map rejected at startup with actionable error - 43 unit tests covering config, client, provider, and module Closes #4815 Assisted-by: Claude Opus 4.6 --- .../mcp-registry-provider-plugin.md | 5 + .../.eslintrc.js | 16 + .../README.md | 87 +++ .../config.d.ts | 38 ++ .../package.json | 49 ++ .../report.api.md | 11 + .../src/client.test.ts | 327 +++++++++++ .../src/client.ts | 173 ++++++ .../src/config.test.ts | 213 +++++++ .../src/config.ts | 135 +++++ .../src/index.ts | 23 + .../src/module.test.ts | 23 + .../src/module.ts | 73 +++ .../src/provider.test.ts | 528 ++++++++++++++++++ .../src/provider.ts | 255 +++++++++ .../src/testUtils.ts | 41 ++ workspaces/ai-integrations/yarn.lock | 16 +- 17 files changed, 2012 insertions(+), 1 deletion(-) create mode 100644 workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/.eslintrc.js create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.test.ts create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts diff --git a/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md b/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md new file mode 100644 index 00000000000..88e452c98ad --- /dev/null +++ b/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md @@ -0,0 +1,5 @@ +--- +'@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider': minor +--- + +Add MCP Registry provider backend module: a catalog entity provider that ingests MCP servers from one configured MCP Registry into the RHDH catalog as mcp-server API entities via cursor pagination, with full-mutation semantics, per-entry failure isolation with last-good retention, and configurable schedule, page limits, and identity prefix override. diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/.eslintrc.js b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/.eslintrc.js new file mode 100644 index 00000000000..9184408ae47 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/.eslintrc.js @@ -0,0 +1,16 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +module.exports = require('@backstage/cli/config/eslint-factory')(__dirname); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md new file mode 100644 index 00000000000..51cbc107d93 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -0,0 +1,87 @@ +# MCP Registry Provider + +A Backstage catalog backend module that ingests MCP servers from a configured [MCP Registry](https://github.com/modelcontextprotocol/registry) into the RHDH catalog as `mcp-server` API entities. + +## Installation + +Add the module to your backend: + +```ts +// packages/backend/src/index.ts +backend.add( + import( + '@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider' + ), +); +``` + +## Configuration + +Configure the provider in your `app-config.yaml`: + +```yaml +catalog: + providers: + mcpRegistry: + baseUrl: https://registry.example.com + # Optional: override the mapping identity prefix (default: mcp.registry) + # baseName: com.example.registry + # Optional: registry API version slug (default: v1) + # apiVersion: v1 + # Optional: default entity owner (default: unknown) + # defaultOwner: group:default/mcp-admins + # Optional: max pages fetched per sync (default: 10) + # pageLimit: 10 + # Optional: registry page size sent as ?limit= (omitted by default) + # pageSize: 50 + # Optional: sync schedule (defaults shown below) + # schedule: + # frequency: { minutes: 30 } + # timeout: { minutes: 3 } + # # Optional: defer the first sync + # # initialDelay: { seconds: 15 } +``` + +### Configuration options + +| Option | Required | Default | Description | +| -------------- | -------- | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `baseUrl` | Yes | — | MCP Registry base URL | +| `baseName` | No | `mcp.registry` (mapping default) | Override the mapping identity prefix for `metadata.name` | +| `apiVersion` | No | `v1` | Registry API version slug. The servers endpoint is `//servers`. Note: the live MCP Registry may serve `/v0` or `/v0.1`; set `apiVersion` to match your registry. | +| `defaultOwner` | No | `unknown` (mapping default) | Backstage entity reference used as `spec.owner` | +| `pageLimit` | No | `10` | Maximum number of pages fetched per sync. The provider fails the sync if the registry has more pages than this limit (to prevent incomplete catalog state). | +| `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | +| `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | + +### Multiple registries + +Multiple registries are **not supported** in this implementation. Configuring a keyed map of instances (e.g., `mcpRegistry.internal` and `mcpRegistry.public`) will fail at startup with an actionable error. Use `baseName` to override the identity prefix if needed for future multi-registry support. + +## Behavior + +### Pagination + +The provider fully traverses the registry's cursor-based pagination, accumulating all server entries. Cursors are treated as opaque strings. The `pageLimit` configuration caps the number of pages fetched per sync — if the registry still has more pages after reaching the limit, the sync fails without committing a mutation, preserving the prior catalog state. + +### Mapping + +Each server entry's `.server` object is transformed into an `mcp-server` API entity using the [`mcp-registry-server-mapping-common`](../mcp-registry-server-mapping-common) library. The provider passes `defaultOwner` and `baseName` as caller overrides but never reimplements the mapping rules. + +### Full mutation + +On each successful sync, the provider commits a **full mutation** — the catalog converges to the registry's current server set. Servers removed from the registry are automatically pruned. + +### Error handling + +- **Per-entry failures**: If a single server entry fails mapping, the provider logs the error and continues. If a last-good entity exists for that server (matched by `name` and `version`), it is retained with `redhat.com/rhdh-mcp-registry-sync-status: degraded`. +- **Registry-level failures**: Transport errors, non-2xx responses, unparseable JSON, or pagination safeguard trips abort the sync — no mutation is committed, preserving the prior catalog state. + +### Annotations + +Each entity carries: + +- `backstage.io/managed-by-location`: `url:` +- `redhat.com/rhdh-mcp-registry-sync-status`: `ok` or `degraded` +- `modelcontextprotocol.io/name`: the server's canonical name +- `modelcontextprotocol.io/version`: the server's version diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts new file mode 100644 index 00000000000..8c1e9429fc5 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts @@ -0,0 +1,38 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +import { SchedulerServiceTaskScheduleDefinitionConfig } from '@backstage/backend-plugin-api'; + +export interface Config { + catalog?: { + providers?: { + mcpRegistry?: { + /** @visibility backend */ + baseUrl: string; + /** @visibility backend */ + baseName?: string; + /** @visibility backend */ + apiVersion?: string; + /** @visibility backend */ + defaultOwner?: string; + /** @visibility backend */ + pageLimit?: number; + /** @visibility backend */ + pageSize?: number; + schedule?: SchedulerServiceTaskScheduleDefinitionConfig; + }; + }; + }; +} diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json new file mode 100644 index 00000000000..c0409c927ee --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json @@ -0,0 +1,49 @@ +{ + "name": "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider", + "version": "0.1.0", + "license": "Apache-2.0", + "description": "The mcp-registry-provider backend module for the catalog plugin.", + "main": "src/index.ts", + "types": "src/index.ts", + "publishConfig": { + "access": "public", + "main": "dist/index.cjs.js", + "types": "dist/index.d.ts" + }, + "repository": { + "type": "git", + "url": "https://github.com/redhat-developer/rhdh-plugins", + "directory": "workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider" + }, + "backstage": { + "role": "backend-plugin-module", + "pluginId": "catalog", + "pluginPackage": "@backstage/plugin-catalog-backend", + "pluginPackages": [ + "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider" + ] + }, + "scripts": { + "start": "backstage-cli package start", + "build": "backstage-cli package build", + "lint": "backstage-cli package lint", + "test": "backstage-cli package test", + "clean": "backstage-cli package clean", + "prepack": "backstage-cli package prepack", + "postpack": "backstage-cli package postpack" + }, + "dependencies": { + "@backstage/backend-plugin-api": "^1.10.0", + "@backstage/catalog-model": "^1.10.0", + "@backstage/plugin-catalog-node": "^2.2.4", + "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common": "workspace:^" + }, + "devDependencies": { + "@backstage/backend-test-utils": "^1.11.6", + "@backstage/cli": "^0.36.5", + "@backstage/config": "^1.3.8" + }, + "files": [ + "dist" + ] +} diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md new file mode 100644 index 00000000000..429596a9696 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md @@ -0,0 +1,11 @@ +## API Report File for "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider" + +> Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/). + +```ts +import { BackendFeature } from '@backstage/backend-plugin-api'; + +// @public +const catalogModuleMcpRegistryProvider: BackendFeature; +export default catalogModuleMcpRegistryProvider; +``` diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts new file mode 100644 index 00000000000..8d9d2b06297 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts @@ -0,0 +1,327 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { + buildServersEndpoint, + fetchRegistryServers, + McpRegistryClientError, +} from './client'; +import type { McpRegistryListResponse } from './client'; +import { createMockServerDoc } from './testUtils'; + +describe('buildServersEndpoint', () => { + it('constructs endpoint without trailing slash', () => { + expect(buildServersEndpoint('https://registry.example.com', 'v1')).toBe( + 'https://registry.example.com/v1/servers', + ); + }); + + it('constructs endpoint with trailing slash on baseUrl', () => { + expect(buildServersEndpoint('https://registry.example.com/', 'v0')).toBe( + 'https://registry.example.com/v0/servers', + ); + }); + + it('handles multiple trailing slashes', () => { + expect(buildServersEndpoint('https://registry.example.com///', 'v1')).toBe( + 'https://registry.example.com/v1/servers', + ); + }); +}); + +describe('fetchRegistryServers', () => { + function mockFetch( + responses: Array<{ + status?: number; + body?: McpRegistryListResponse | string; + throws?: boolean; + }>, + ): jest.Mock { + const fn = jest.fn(); + for (const resp of responses) { + if (resp.throws) { + fn.mockRejectedValueOnce(new Error('network error')); + } else { + fn.mockResolvedValueOnce({ + ok: (resp.status ?? 200) >= 200 && (resp.status ?? 200) < 300, + status: resp.status ?? 200, + json: async () => { + if (typeof resp.body === 'string') { + throw new Error('Invalid JSON'); + } + return resp.body; + }, + text: async () => + typeof resp.body === 'string' + ? resp.body + : JSON.stringify(resp.body), + } as unknown as Response); + } + } + return fn; + } + + it('fetches a single page with no nextCursor', async () => { + const body: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 1 }, + }; + const fn = mockFetch([{ body }]); + + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + fetchApi: fn, + }); + + expect(result).toHaveLength(1); + expect(result[0].server.name).toBe('test/server-a'); + expect(fn).toHaveBeenCalledTimes(1); + // Verify no limit param when pageSize is omitted + const calledUrl = fn.mock.calls[0][0] as string; + expect(calledUrl).not.toContain('limit='); + }); + + it('traverses multiple pages via cursor', async () => { + const page1: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 2, nextCursor: 'cursor-abc' }, + }; + const page2: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-b', '2.0.0') }], + metadata: { count: 2 }, + }; + const fn = mockFetch([{ body: page1 }, { body: page2 }]); + + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + fetchApi: fn, + }); + + expect(result).toHaveLength(2); + expect(fn).toHaveBeenCalledTimes(2); + // Second call should include cursor + const secondUrl = fn.mock.calls[1][0] as string; + expect(secondUrl).toContain('cursor=cursor-abc'); + }); + + it('stops on null nextCursor', async () => { + const body: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 1, nextCursor: null }, + }; + const fn = mockFetch([{ body }]); + + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + fetchApi: fn, + }); + + expect(result).toHaveLength(1); + expect(fn).toHaveBeenCalledTimes(1); + }); + + it('stops on empty string nextCursor', async () => { + const body: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 1, nextCursor: '' }, + }; + const fn = mockFetch([{ body }]); + + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + fetchApi: fn, + }); + + expect(result).toHaveLength(1); + expect(fn).toHaveBeenCalledTimes(1); + }); + + it('sends pageSize as limit query param on every request', async () => { + const page1: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 2, nextCursor: 'cursor-xyz' }, + }; + const page2: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-b', '2.0.0') }], + metadata: { count: 2 }, + }; + const fn = mockFetch([{ body: page1 }, { body: page2 }]); + + await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + pageSize: 50, + fetchApi: fn, + }); + + const firstUrl = fn.mock.calls[0][0] as string; + const secondUrl = fn.mock.calls[1][0] as string; + expect(firstUrl).toContain('limit=50'); + expect(secondUrl).toContain('limit=50'); + }); + + it('trips on default pageLimit of 10 at the 11th page', async () => { + const pages = Array.from({ length: 10 }, (_, i) => ({ + body: { + servers: [{ server: createMockServerDoc(`test/server-${i}`, '1.0.0') }], + metadata: { + count: 11, + nextCursor: `cursor-${i + 1}`, + }, + } as McpRegistryListResponse, + })); + const fn = mockFetch(pages); + + await expect( + fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + fetchApi: fn, + }), + ).rejects.toThrow(McpRegistryClientError); + + await expect( + fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + fetchApi: mockFetch(pages), + }), + ).rejects.toThrow(/page limit/i); + }); + + it('trips on configured pageLimit', async () => { + const page1: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 3, nextCursor: 'cursor-1' }, + }; + const page2: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-b', '2.0.0') }], + metadata: { count: 3, nextCursor: 'cursor-2' }, + }; + const fn = mockFetch([{ body: page1 }, { body: page2 }]); + + await expect( + fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 2, + fetchApi: fn, + }), + ).rejects.toThrow(/page limit/i); + }); + + it('detects repeated cursor', async () => { + const page1: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 2, nextCursor: 'cursor-repeat' }, + }; + const page2: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-b', '2.0.0') }], + metadata: { count: 2, nextCursor: 'cursor-repeat' }, + }; + const fn = mockFetch([{ body: page1 }, { body: page2 }]); + + await expect( + fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + fetchApi: fn, + }), + ).rejects.toThrow(/repeated cursor/i); + }); + + it('throws on network error', async () => { + const fn = mockFetch([{ throws: true }]); + + await expect( + fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + fetchApi: fn, + }), + ).rejects.toThrow(McpRegistryClientError); + }); + + it('throws on non-2xx status', async () => { + const fn = mockFetch([{ status: 500, body: 'Server Error' }]); + + await expect( + fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + fetchApi: fn, + }), + ).rejects.toThrow(/HTTP 500/); + }); + + it('throws on unparseable JSON', async () => { + const fn = jest.fn().mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => { + throw new SyntaxError('Unexpected token'); + }, + text: async () => 'not json', + } as unknown as Response); + + await expect( + fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + fetchApi: fn, + }), + ).rejects.toThrow(/unparseable JSON/); + }); + + it('passes opaque cursor unchanged', async () => { + const opaqueToken = 'eyJsYXN0X2lkIjoiYWJjMTIzIn0='; + const page1: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 2, nextCursor: opaqueToken }, + }; + const page2: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-b', '2.0.0') }], + metadata: { count: 2 }, + }; + const fn = mockFetch([{ body: page1 }, { body: page2 }]); + + await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + fetchApi: fn, + }); + + const secondUrl = fn.mock.calls[1][0] as string; + // URL encodes the cursor, but the original value should be present + expect(secondUrl).toContain(`cursor=${encodeURIComponent(opaqueToken)}`); + }); +}); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts new file mode 100644 index 00000000000..4ac8cf0dd8c --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -0,0 +1,173 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import type { McpServerDocument } from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; + +/** + * A single server entry from the MCP Registry list response. + */ +export interface McpRegistryServerEntry { + server: McpServerDocument; +} + +/** + * The MCP Registry servers list response shape. + */ +export interface McpRegistryListResponse { + servers: McpRegistryServerEntry[]; + metadata: { + count?: number; + nextCursor?: string | null; + }; +} + +/** + * Error thrown when the registry client encounters a transport or + * protocol error that should abort the sync run. + */ +export class McpRegistryClientError extends Error { + constructor(message: string) { + super(message); + this.name = 'McpRegistryClientError'; + } +} + +/** + * Build the servers endpoint URL from baseUrl and apiVersion, + * normalizing slashes so a trailing slash on baseUrl does not + * produce a double separator. + */ +export function buildServersEndpoint( + baseUrl: string, + apiVersion: string, +): string { + const normalizedBase = baseUrl.replace(/\/+$/, ''); + return `${normalizedBase}/${apiVersion}/servers`; +} + +/** + * Options for fetching servers from the MCP Registry. + */ +export interface FetchServersOptions { + baseUrl: string; + apiVersion: string; + pageLimit: number; + pageSize?: number; + /** Optional fetch implementation for testing. */ + fetchApi?: typeof fetch; +} + +/** + * Fetch all server entries from the MCP Registry using cursor + * pagination. Accumulates entries across pages and enforces + * pagination safeguards (page cap, repeated cursor). + * + * @throws McpRegistryClientError on transport, protocol, or + * pagination-safeguard errors. + */ +export async function fetchRegistryServers( + options: FetchServersOptions, +): Promise { + const { baseUrl, apiVersion, pageLimit, pageSize, fetchApi } = options; + const doFetch = fetchApi ?? fetch; + + const endpoint = buildServersEndpoint(baseUrl, apiVersion); + const allServers: McpRegistryServerEntry[] = []; + const seenCursors = new Set(); + let cursor: string | undefined; + let pageCount = 0; + + // eslint-disable-next-line no-constant-condition + while (true) { + // Build request URL with query params + const url = new URL(endpoint); + if (cursor) { + url.searchParams.set('cursor', cursor); + } + if (pageSize !== undefined) { + url.searchParams.set('limit', String(pageSize)); + } + + let response: Response; + try { + response = await doFetch(url.toString()); + } catch (err) { + throw new McpRegistryClientError( + `Failed to reach MCP Registry at ${url.toString()}: ${err}`, + ); + } + + if (!response.ok) { + throw new McpRegistryClientError( + `MCP Registry returned HTTP ${response.status} for ` + + `${url.toString()}: ${await response + .text() + .catch(() => '(no body)')}`, + ); + } + + let body: McpRegistryListResponse; + try { + body = (await response.json()) as McpRegistryListResponse; + } catch (err) { + throw new McpRegistryClientError( + `MCP Registry returned unparseable JSON from ` + + `${url.toString()}: ${err}`, + ); + } + + if (!body.servers || !Array.isArray(body.servers)) { + throw new McpRegistryClientError( + `MCP Registry response missing "servers" array from ` + + `${url.toString()}`, + ); + } + + allServers.push(...body.servers); + pageCount++; + + // Check for next cursor + const nextCursor = body.metadata?.nextCursor; + if (!nextCursor || nextCursor.length === 0) { + // No more pages + break; + } + + // Repeated cursor safeguard + if (seenCursors.has(nextCursor)) { + throw new McpRegistryClientError( + `MCP Registry returned a repeated cursor "${nextCursor}" ` + + `during pagination. Aborting sync to prevent infinite loop.`, + ); + } + seenCursors.add(nextCursor); + + // Page limit safeguard: if we've fetched pageLimit pages and + // there's still a nextCursor, fail the run + if (pageCount >= pageLimit) { + throw new McpRegistryClientError( + `MCP Registry pagination exceeded the configured page limit ` + + `of ${pageLimit} pages per sync. The registry still has more ` + + `pages (nextCursor present). Increase pageLimit to fetch ` + + `more pages.`, + ); + } + + cursor = nextCursor; + } + + return allServers; +} diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts new file mode 100644 index 00000000000..ecbf0711474 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts @@ -0,0 +1,213 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { ConfigReader } from '@backstage/config'; +import { readMcpRegistryProviderConfig } from './config'; + +describe('readMcpRegistryProviderConfig', () => { + it('returns undefined when catalog.providers is absent', () => { + const config = new ConfigReader({}); + expect(readMcpRegistryProviderConfig(config)).toBeUndefined(); + }); + + it('returns undefined when catalog.providers.mcpRegistry is absent', () => { + const config = new ConfigReader({ + catalog: { providers: {} }, + }); + expect(readMcpRegistryProviderConfig(config)).toBeUndefined(); + }); + + it('reads a single object with baseUrl and applies defaults', () => { + const config = new ConfigReader({ + catalog: { + providers: { + mcpRegistry: { + baseUrl: 'https://registry.example.com', + }, + }, + }, + }); + + const result = readMcpRegistryProviderConfig(config); + expect(result).toBeDefined(); + expect(result!.baseUrl).toBe('https://registry.example.com'); + expect(result!.apiVersion).toBe('v1'); + expect(result!.pageLimit).toBe(10); + expect(result!.pageSize).toBeUndefined(); + expect(result!.baseName).toBeUndefined(); + expect(result!.defaultOwner).toBeUndefined(); + expect(result!.schedule).toEqual({ + frequency: { minutes: 30 }, + timeout: { minutes: 3 }, + }); + }); + + it('reads optional baseName', () => { + const config = new ConfigReader({ + catalog: { + providers: { + mcpRegistry: { + baseUrl: 'https://registry.example.com', + baseName: 'com.example.registry', + }, + }, + }, + }); + + const result = readMcpRegistryProviderConfig(config); + expect(result!.baseName).toBe('com.example.registry'); + }); + + it('reads explicit pageLimit override', () => { + const config = new ConfigReader({ + catalog: { + providers: { + mcpRegistry: { + baseUrl: 'https://registry.example.com', + pageLimit: 3, + }, + }, + }, + }); + + const result = readMcpRegistryProviderConfig(config); + expect(result!.pageLimit).toBe(3); + }); + + it('reads explicit pageSize', () => { + const config = new ConfigReader({ + catalog: { + providers: { + mcpRegistry: { + baseUrl: 'https://registry.example.com', + pageSize: 50, + }, + }, + }, + }); + + const result = readMcpRegistryProviderConfig(config); + expect(result!.pageSize).toBe(50); + }); + + it('reads omitted pageSize as undefined', () => { + const config = new ConfigReader({ + catalog: { + providers: { + mcpRegistry: { + baseUrl: 'https://registry.example.com', + }, + }, + }, + }); + + const result = readMcpRegistryProviderConfig(config); + expect(result!.pageSize).toBeUndefined(); + }); + + it('throws when baseUrl is missing', () => { + const config = new ConfigReader({ + catalog: { + providers: { + mcpRegistry: { + apiVersion: 'v0', + }, + }, + }, + }); + + expect(() => readMcpRegistryProviderConfig(config)).toThrow( + /missing required "baseUrl"/, + ); + }); + + it('throws when config is a keyed map of instances', () => { + const config = new ConfigReader({ + catalog: { + providers: { + mcpRegistry: { + internal: { + baseUrl: 'https://internal-registry.example.com', + }, + public: { + baseUrl: 'https://public-registry.example.com', + }, + }, + }, + }, + }); + + expect(() => readMcpRegistryProviderConfig(config)).toThrow( + /Multiple registries are out of scope/, + ); + }); + + it('reads a custom schedule', () => { + const config = new ConfigReader({ + catalog: { + providers: { + mcpRegistry: { + baseUrl: 'https://registry.example.com', + schedule: { + frequency: { minutes: 15 }, + timeout: { minutes: 5 }, + initialDelay: { seconds: 30 }, + }, + }, + }, + }, + }); + + const result = readMcpRegistryProviderConfig(config); + expect(result!.schedule).toEqual({ + frequency: { minutes: 15 }, + timeout: { minutes: 5 }, + initialDelay: { seconds: 30 }, + }); + }); + + it('reads defaultOwner', () => { + const config = new ConfigReader({ + catalog: { + providers: { + mcpRegistry: { + baseUrl: 'https://registry.example.com', + defaultOwner: 'group:default/mcp-admins', + }, + }, + }, + }); + + const result = readMcpRegistryProviderConfig(config); + expect(result!.defaultOwner).toBe('group:default/mcp-admins'); + }); + + it('reads apiVersion override', () => { + const config = new ConfigReader({ + catalog: { + providers: { + mcpRegistry: { + baseUrl: 'https://registry.example.com', + apiVersion: 'v0', + }, + }, + }, + }); + + const result = readMcpRegistryProviderConfig(config); + expect(result!.apiVersion).toBe('v0'); + }); +}); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts new file mode 100644 index 00000000000..6ea65d1be33 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts @@ -0,0 +1,135 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import type { Config } from '@backstage/config'; +import type { SchedulerServiceTaskScheduleDefinition } from '@backstage/backend-plugin-api'; +import { readSchedulerServiceTaskScheduleDefinitionFromConfig } from '@backstage/backend-plugin-api'; + +/** Default schedule when `schedule` is omitted. */ +const DEFAULT_SCHEDULE: SchedulerServiceTaskScheduleDefinition = { + frequency: { minutes: 30 }, + timeout: { minutes: 3 }, +}; + +/** Default apiVersion when omitted. */ +const DEFAULT_API_VERSION = 'v1'; + +/** Default page limit (max pages per sync). */ +const DEFAULT_PAGE_LIMIT = 10; + +/** + * Parsed provider configuration. + */ +export interface McpRegistryProviderConfig { + baseUrl: string; + baseName?: string; + apiVersion: string; + defaultOwner?: string; + pageLimit: number; + pageSize?: number; + schedule: SchedulerServiceTaskScheduleDefinition; +} + +/** + * Read and validate the MCP Registry provider configuration from + * `catalog.providers.mcpRegistry`. Returns `undefined` when the + * config key is absent (inert module). + * + * @throws When the config is a keyed map of instances, or when + * `baseUrl` is missing. + */ +export function readMcpRegistryProviderConfig( + rootConfig: Config, +): McpRegistryProviderConfig | undefined { + const providersConfig = rootConfig.getOptionalConfig('catalog.providers'); + if (!providersConfig) { + return undefined; + } + + const registryConfig = providersConfig.getOptionalConfig('mcpRegistry'); + if (!registryConfig) { + return undefined; + } + + // Detect keyed multi-registry maps: if the config has keys that look + // like instance objects (i.e., nested config objects with their own + // baseUrl), reject with an actionable error. + const keys = registryConfig.keys(); + const knownKeys = new Set([ + 'baseUrl', + 'baseName', + 'apiVersion', + 'defaultOwner', + 'pageLimit', + 'pageSize', + 'schedule', + ]); + const unknownKeys = keys.filter(k => !knownKeys.has(k)); + if (unknownKeys.length > 0) { + // Check if the unknown keys look like instance identifiers (they + // would have nested config objects with their own properties) + for (const key of unknownKeys) { + const nested = registryConfig.getOptionalConfig(key); + if (nested && nested.keys().length > 0) { + throw new Error( + `Invalid catalog.providers.mcpRegistry configuration: found ` + + `keyed instance "${key}". Multiple registries are out of scope ` + + `for this implementation. Configure a single registry object ` + + `with baseUrl, baseName, apiVersion, schedule, pageLimit, ` + + `pageSize, and defaultOwner.`, + ); + } + } + } + + // baseUrl is required + const baseUrl = registryConfig.getOptionalString('baseUrl'); + if (!baseUrl) { + throw new Error( + `Invalid catalog.providers.mcpRegistry configuration: missing ` + + `required "baseUrl" field. Set baseUrl to the MCP Registry base URL ` + + `(e.g., "https://registry.example.com").`, + ); + } + + const baseName = registryConfig.getOptionalString('baseName'); + const apiVersion = + registryConfig.getOptionalString('apiVersion') ?? DEFAULT_API_VERSION; + const defaultOwner = registryConfig.getOptionalString('defaultOwner'); + const pageLimit = + registryConfig.getOptionalNumber('pageLimit') ?? DEFAULT_PAGE_LIMIT; + const pageSize = registryConfig.getOptionalNumber('pageSize'); + + // Schedule: read from config or use default + let schedule: SchedulerServiceTaskScheduleDefinition; + const scheduleConfig = registryConfig.getOptionalConfig('schedule'); + if (scheduleConfig) { + schedule = + readSchedulerServiceTaskScheduleDefinitionFromConfig(scheduleConfig); + } else { + schedule = DEFAULT_SCHEDULE; + } + + return { + baseUrl, + baseName, + apiVersion, + defaultOwner, + pageLimit, + pageSize, + schedule, + }; +} diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts new file mode 100644 index 00000000000..0ded7b32007 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts @@ -0,0 +1,23 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * The mcp-registry-provider backend module for the catalog plugin. + * + * @packageDocumentation + */ + +export { catalogModuleMcpRegistryProvider as default } from './module'; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.test.ts new file mode 100644 index 00000000000..b526595d7ab --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.test.ts @@ -0,0 +1,23 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { catalogModuleMcpRegistryProvider } from './module'; + +describe('mcp-registry-provider module', () => { + it('should export the backend module', () => { + expect(catalogModuleMcpRegistryProvider).toBeDefined(); + }); +}); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts new file mode 100644 index 00000000000..c45c49d72e5 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts @@ -0,0 +1,73 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { + coreServices, + createBackendModule, +} from '@backstage/backend-plugin-api'; +import { catalogProcessingExtensionPoint } from '@backstage/plugin-catalog-node'; +import { readMcpRegistryProviderConfig } from './config'; +import { McpRegistryEntityProvider } from './provider'; + +/** + * The mcp-registry-provider backend module for the catalog plugin. + * + * Registers a single entity provider that ingests MCP servers from + * one configured MCP Registry into the catalog as `mcp-server` API + * entities. + * + * @public + */ +export const catalogModuleMcpRegistryProvider = createBackendModule({ + pluginId: 'catalog', + moduleId: 'catalog-backend-module-mcp-registry-provider', + register(env) { + env.registerInit({ + deps: { + catalog: catalogProcessingExtensionPoint, + config: coreServices.rootConfig, + logger: coreServices.logger, + scheduler: coreServices.scheduler, + }, + async init({ catalog, config, logger, scheduler }) { + const providerConfig = readMcpRegistryProviderConfig(config); + + if (!providerConfig) { + logger.info( + 'catalog.providers.mcpRegistry not configured; ' + + 'MCP Registry provider is inactive.', + ); + return; + } + + const provider = new McpRegistryEntityProvider(providerConfig, logger); + + catalog.addEntityProvider(provider); + + const taskRunner = scheduler.createScheduledTaskRunner( + providerConfig.schedule, + ); + + await taskRunner.run({ + id: 'mcp-registry-provider:refresh', + fn: async () => { + await provider.run(); + }, + }); + }, + }); + }, +}); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts new file mode 100644 index 00000000000..be89e3df855 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts @@ -0,0 +1,528 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { McpRegistryEntityProvider } from './provider'; +import type { McpRegistryProviderConfig } from './config'; +import type { EntityProviderConnection } from '@backstage/plugin-catalog-node'; +import type { McpRegistryListResponse } from './client'; +import { createMockServerDoc } from './testUtils'; + +function createMockLogger() { + return { + info: jest.fn(), + warn: jest.fn(), + error: jest.fn(), + debug: jest.fn(), + child: jest.fn().mockReturnThis(), + }; +} + +function createMockConnection(): EntityProviderConnection { + return { + applyMutation: jest.fn(), + refresh: jest.fn(), + } as unknown as EntityProviderConnection; +} + +function createDefaultConfig( + overrides?: Partial, +): McpRegistryProviderConfig { + return { + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + schedule: { + frequency: { minutes: 30 }, + timeout: { minutes: 3 }, + }, + ...overrides, + }; +} + +function mockFetchForResponses( + responses: McpRegistryListResponse[], +): jest.Mock { + const fn = jest.fn(); + for (const body of responses) { + fn.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => body, + text: async () => JSON.stringify(body), + } as unknown as Response); + } + return fn; +} + +describe('McpRegistryEntityProvider', () => { + it('returns provider name mcp-registry-provider', () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + ); + expect(provider.getProviderName()).toBe('mcp-registry-provider'); + }); + + it('throws if run() is called before connect()', async () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + ); + await expect(provider.run()).rejects.toThrow(/not initialized/); + }); + + it('applies full mutation with mapped entities', async () => { + const body: McpRegistryListResponse = { + servers: [ + { server: createMockServerDoc('io.github.user/weather', '1.0.0') }, + ], + metadata: { count: 1 }, + }; + const fetchFn = mockFetchForResponses([body]); + const connection = createMockConnection(); + const logger = createMockLogger(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + logger, + fetchFn, + ); + await provider.connect(connection); + await provider.run(); + + expect(connection.applyMutation).toHaveBeenCalledTimes(1); + const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + expect(mutation.type).toBe('full'); + expect(mutation.entities).toHaveLength(1); + + const entity = mutation.entities[0]; + expect(entity.locationKey).toBe('mcp-registry-provider'); + expect(entity.entity.kind).toBe('API'); + expect(entity.entity.spec.type).toBe('mcp-server'); + expect( + entity.entity.metadata.annotations['backstage.io/managed-by-location'], + ).toBe('url:https://registry.example.com'); + expect( + entity.entity.metadata.annotations[ + 'redhat.com/rhdh-mcp-registry-sync-status' + ], + ).toBe('ok'); + }); + + it('strips trailing slash from baseUrl in managed-by-location', async () => { + const body: McpRegistryListResponse = { + servers: [ + { server: createMockServerDoc('io.github.user/weather', '1.0.0') }, + ], + metadata: { count: 1 }, + }; + const fetchFn = mockFetchForResponses([body]); + const connection = createMockConnection(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ baseUrl: 'https://registry.example.com/' }), + createMockLogger(), + fetchFn, + ); + await provider.connect(connection); + await provider.run(); + + const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + expect( + mutation.entities[0].entity.metadata.annotations[ + 'backstage.io/managed-by-location' + ], + ).toBe('url:https://registry.example.com'); + }); + + it('passes defaultOwner to the mapping', async () => { + const body: McpRegistryListResponse = { + servers: [ + { server: createMockServerDoc('io.github.user/weather', '1.0.0') }, + ], + metadata: { count: 1 }, + }; + const fetchFn = mockFetchForResponses([body]); + const connection = createMockConnection(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ + defaultOwner: 'group:default/mcp-admins', + }), + createMockLogger(), + fetchFn, + ); + await provider.connect(connection); + await provider.run(); + + const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + expect(mutation.entities[0].entity.spec.owner).toBe( + 'group:default/mcp-admins', + ); + }); + + it('uses mapping default owner when defaultOwner is omitted', async () => { + const body: McpRegistryListResponse = { + servers: [ + { server: createMockServerDoc('io.github.user/weather', '1.0.0') }, + ], + metadata: { count: 1 }, + }; + const fetchFn = mockFetchForResponses([body]); + const connection = createMockConnection(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + fetchFn, + ); + await provider.connect(connection); + await provider.run(); + + const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + expect(mutation.entities[0].entity.spec.owner).toBe('unknown'); + }); + + it('passes baseName as prefix override to the mapping', async () => { + const body: McpRegistryListResponse = { + servers: [ + { server: createMockServerDoc('io.github.user/weather', '1.0.2') }, + ], + metadata: { count: 1 }, + }; + const fetchFn = mockFetchForResponses([body]); + const connection = createMockConnection(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ baseName: 'com.example.registry' }), + createMockLogger(), + fetchFn, + ); + await provider.connect(connection); + await provider.run(); + + const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + // The metadata.name should use the baseName prefix + expect(mutation.entities[0].entity.metadata.name).toContain( + 'com.example.registry', + ); + }); + + it('uses mapping default prefix when baseName is omitted', async () => { + const body: McpRegistryListResponse = { + servers: [ + { server: createMockServerDoc('io.github.user/weather', '1.0.2') }, + ], + metadata: { count: 1 }, + }; + const fetchFn = mockFetchForResponses([body]); + const connection = createMockConnection(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + fetchFn, + ); + await provider.connect(connection); + await provider.run(); + + const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + // The metadata.name should use default prefix mcp.registry + expect(mutation.entities[0].entity.metadata.name).toContain('mcp.registry'); + }); + + it('does not emit mutation on registry fetch error', async () => { + const fetchFn = jest.fn().mockRejectedValueOnce(new Error('network')); + const connection = createMockConnection(); + const logger = createMockLogger(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + logger, + fetchFn, + ); + await provider.connect(connection); + await provider.run(); + + expect(connection.applyMutation).not.toHaveBeenCalled(); + expect(logger.error).toHaveBeenCalled(); + }); + + it('continues sync when one entry fails mapping', async () => { + const body: McpRegistryListResponse = { + servers: [ + { server: createMockServerDoc('io.github.user/good', '1.0.0') }, + { + server: { + // Missing required fields - will fail mapping + $schema: + 'https://raw.githubusercontent.com/modelcontextprotocol/registry/v1.8.1/docs/reference/server-json/draft/server.schema.json', + name: '', + description: '', + version: '', + } as any, + }, + ], + metadata: { count: 2 }, + }; + const fetchFn = mockFetchForResponses([body]); + const connection = createMockConnection(); + const logger = createMockLogger(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + logger, + fetchFn, + ); + await provider.connect(connection); + await provider.run(); + + expect(connection.applyMutation).toHaveBeenCalledTimes(1); + const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + // Only the good server should be in the mutation + expect(mutation.entities).toHaveLength(1); + expect(logger.warn).toHaveBeenCalled(); + }); + + it('retains last-good entity with degraded status on mapping failure', async () => { + // First sync: successful mapping + const goodBody: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server', '1.0.0') }], + metadata: { count: 1 }, + }; + const fetchFn1 = mockFetchForResponses([goodBody]); + const connection = createMockConnection(); + const logger = createMockLogger(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + logger, + fetchFn1, + ); + await provider.connect(connection); + await provider.run(); + + // Verify first sync succeeded + expect(connection.applyMutation).toHaveBeenCalledTimes(1); + const firstMutation = (connection.applyMutation as jest.Mock).mock + .calls[0][0]; + expect(firstMutation.entities).toHaveLength(1); + expect( + firstMutation.entities[0].entity.metadata.annotations[ + 'redhat.com/rhdh-mcp-registry-sync-status' + ], + ).toBe('ok'); + + // Second sync: mapping fails (empty description) + const badBody: McpRegistryListResponse = { + servers: [ + { + server: { + $schema: + 'https://raw.githubusercontent.com/modelcontextprotocol/registry/v1.8.1/docs/reference/server-json/draft/server.schema.json', + name: 'test/server', + description: '', + version: '1.0.0', + } as any, + }, + ], + metadata: { count: 1 }, + }; + + // Use a single provider with a combined fetch mock that returns + // good data first, then bad data on the second sync + const combinedFetch = jest.fn(); + // First sync + combinedFetch.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => goodBody, + text: async () => JSON.stringify(goodBody), + } as unknown as Response); + // Second sync + combinedFetch.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => badBody, + text: async () => JSON.stringify(badBody), + } as unknown as Response); + + const connection2 = createMockConnection(); + const logger2 = createMockLogger(); + const provider2 = new McpRegistryEntityProvider( + createDefaultConfig(), + logger2, + combinedFetch, + ); + await provider2.connect(connection2); + + // First sync: populates last-good index + await provider2.run(); + expect(connection2.applyMutation).toHaveBeenCalledTimes(1); + + // Second sync: mapping fails, should retain last-good + await provider2.run(); + expect(connection2.applyMutation).toHaveBeenCalledTimes(2); + + const secondMutation = (connection2.applyMutation as jest.Mock).mock + .calls[1][0]; + expect(secondMutation.entities).toHaveLength(1); + expect( + secondMutation.entities[0].entity.metadata.annotations[ + 'redhat.com/rhdh-mcp-registry-sync-status' + ], + ).toBe('degraded'); + expect(secondMutation.entities[0].locationKey).toBe( + 'mcp-registry-provider', + ); + }); + + it('omits entry on first-time mapping failure with no last-good', async () => { + const body: McpRegistryListResponse = { + servers: [ + { + server: { + $schema: + 'https://raw.githubusercontent.com/modelcontextprotocol/registry/v1.8.1/docs/reference/server-json/draft/server.schema.json', + name: 'test/new-server', + description: '', + version: '1.0.0', + } as any, + }, + ], + metadata: { count: 1 }, + }; + const fetchFn = mockFetchForResponses([body]); + const connection = createMockConnection(); + const logger = createMockLogger(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + logger, + fetchFn, + ); + await provider.connect(connection); + await provider.run(); + + const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + expect(mutation.entities).toHaveLength(0); + }); + + it('prunes removed servers via full mutation', async () => { + // Use a single provider for both syncs + const body1: McpRegistryListResponse = { + servers: [ + { server: createMockServerDoc('test/server-a', '1.0.0') }, + { server: createMockServerDoc('test/server-b', '1.0.0') }, + ], + metadata: { count: 2 }, + }; + const body2: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 1 }, + }; + const combinedFetch = jest.fn(); + combinedFetch.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => body1, + text: async () => JSON.stringify(body1), + } as unknown as Response); + combinedFetch.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => body2, + text: async () => JSON.stringify(body2), + } as unknown as Response); + + const connection = createMockConnection(); + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + combinedFetch, + ); + await provider.connect(connection); + + // First sync: 2 servers + await provider.run(); + let mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + expect(mutation.entities).toHaveLength(2); + + // Second sync: 1 server (server-b pruned by full mutation) + await provider.run(); + mutation = (connection.applyMutation as jest.Mock).mock.calls[1][0]; + expect(mutation.type).toBe('full'); + expect(mutation.entities).toHaveLength(1); + }); + + it('handles multi-page sync with correct entity count', async () => { + const page1: McpRegistryListResponse = { + servers: [ + { server: createMockServerDoc('test/server-a', '1.0.0') }, + { server: createMockServerDoc('test/server-b', '1.0.0') }, + ], + metadata: { count: 4, nextCursor: 'page2' }, + }; + const page2: McpRegistryListResponse = { + servers: [ + { server: createMockServerDoc('test/server-c', '1.0.0') }, + { server: createMockServerDoc('test/server-d', '1.0.0') }, + ], + metadata: { count: 4 }, + }; + const fetchFn = mockFetchForResponses([page1, page2]); + const connection = createMockConnection(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + fetchFn, + ); + await provider.connect(connection); + await provider.run(); + + const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + expect(mutation.entities).toHaveLength(4); + }); + + it('reflects updated server.json in the current mutation', async () => { + const body: McpRegistryListResponse = { + servers: [ + { + server: createMockServerDoc('io.github.user/weather', '1.0.0', { + description: 'Updated weather description', + }), + }, + ], + metadata: { count: 1 }, + }; + const fetchFn = mockFetchForResponses([body]); + const connection = createMockConnection(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + fetchFn, + ); + await provider.connect(connection); + await provider.run(); + + const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + expect(mutation.entities[0].entity.metadata.description).toBe( + 'Updated weather description', + ); + }); +}); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts new file mode 100644 index 00000000000..507a1fcf177 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts @@ -0,0 +1,255 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import type { LoggerService } from '@backstage/backend-plugin-api'; +import type { + DeferredEntity, + EntityProvider, + EntityProviderConnection, +} from '@backstage/plugin-catalog-node'; +import { + mapServerToEntity, + projectAnnotations, +} from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; +import type { McpRegistryProviderConfig } from './config'; +import { fetchRegistryServers, McpRegistryClientError } from './client'; +import type { McpRegistryServerEntry } from './client'; + +/** Provider name and locationKey constant. */ +const PROVIDER_NAME = 'mcp-registry-provider'; + +/** Sync status annotation key. */ +const SYNC_STATUS_ANNOTATION = 'redhat.com/rhdh-mcp-registry-sync-status'; + +/** Managed-by-location annotation key. */ +const MANAGED_BY_LOCATION_ANNOTATION = 'backstage.io/managed-by-location'; + +/** + * Build a last-good lookup key from name and version. + */ +function buildLastGoodKey(name: string, version: string): string { + return `${name}::${version}`; +} + +/** + * Normalize baseUrl by stripping trailing slashes for use in + * backstage.io/managed-by-location. + */ +function normalizeBaseUrl(baseUrl: string): string { + return baseUrl.replace(/\/+$/, ''); +} + +/** + * Entity provider that ingests MCP servers from one configured + * MCP Registry into the Backstage catalog. + */ +export class McpRegistryEntityProvider implements EntityProvider { + private connection?: EntityProviderConnection; + private readonly config: McpRegistryProviderConfig; + private readonly logger: LoggerService; + private readonly fetchApi?: typeof fetch; + + /** + * Internal last-good index: keyed by `name::version`, stores the + * last successfully committed DeferredEntity so that a subsequent + * sync can retain it when mapping fails (D6). + */ + private lastGoodIndex = new Map(); + + constructor( + config: McpRegistryProviderConfig, + logger: LoggerService, + fetchApi?: typeof fetch, + ) { + this.config = config; + this.logger = logger; + this.fetchApi = fetchApi; + } + + getProviderName(): string { + return PROVIDER_NAME; + } + + async connect(connection: EntityProviderConnection): Promise { + this.connection = connection; + } + + /** + * Run one sync cycle: fetch servers from the registry, map them, + * and commit a full mutation. + */ + async run(): Promise { + if (!this.connection) { + throw new Error( + 'McpRegistryEntityProvider not initialized; call connect() first.', + ); + } + + const { baseUrl, apiVersion, pageLimit, pageSize, baseName, defaultOwner } = + this.config; + const normalizedBaseUrl = normalizeBaseUrl(baseUrl); + const managedByLocation = `url:${normalizedBaseUrl}`; + + // Fetch all servers from the registry + let entries: McpRegistryServerEntry[]; + try { + entries = await fetchRegistryServers({ + baseUrl, + apiVersion, + pageLimit, + pageSize, + fetchApi: this.fetchApi, + }); + } catch (err) { + if (err instanceof McpRegistryClientError) { + this.logger.error( + `MCP Registry sync failed (no mutation emitted): ${err.message}`, + ); + return; + } + throw err; + } + + // Map each entry, with per-entry failure isolation + const entities: DeferredEntity[] = []; + let hasDegradedEntries = false; + + for (const entry of entries) { + const serverDoc = entry.server; + try { + // Invoke the mapping transform + const mappingDefaults: { + prefix?: string; + owner?: string; + } = {}; + if (defaultOwner) { + mappingDefaults.owner = defaultOwner; + } + if (baseName) { + mappingDefaults.prefix = baseName; + } + + const mappingResult = mapServerToEntity(serverDoc, mappingDefaults); + const entity = mappingResult.entity; + + // Apply annotation projection + const projectedAnnotations = projectAnnotations( + serverDoc, + mappingResult.consumedPaths, + mappingResult.reservedAnnotationKeys, + ); + + // Merge projected annotations with the entity's existing ones + entity.metadata.annotations = { + ...entity.metadata.annotations, + ...projectedAnnotations, + }; + + // Add provider attribution annotations + entity.metadata.annotations[MANAGED_BY_LOCATION_ANNOTATION] = + managedByLocation; + entity.metadata.annotations[SYNC_STATUS_ANNOTATION] = 'ok'; + + const deferred: DeferredEntity = { + entity, + locationKey: PROVIDER_NAME, + }; + entities.push(deferred); + } catch (err) { + // Per-entry failure: log and attempt last-good retention + const serverName = + typeof serverDoc?.name === 'string' ? serverDoc.name : undefined; + const serverVersion = + typeof serverDoc?.version === 'string' + ? serverDoc.version + : undefined; + + this.logger.warn( + `Failed to map MCP Registry server entry` + + `${serverName ? ` "${serverName}"` : ''}` + + `${serverVersion ? ` version "${serverVersion}"` : ''}: ${err}`, + ); + + // Last-good retention (D6): retain prior entity if name and + // version are present and a last-good entity exists + if (serverName && serverVersion) { + const lastGoodKey = buildLastGoodKey(serverName, serverVersion); + const lastGood = this.lastGoodIndex.get(lastGoodKey); + if (lastGood) { + // Use the last-good entity with degraded status + const retainedEntity = JSON.parse(JSON.stringify(lastGood.entity)); + if (!retainedEntity.metadata.annotations) { + retainedEntity.metadata.annotations = {}; + } + retainedEntity.metadata.annotations[SYNC_STATUS_ANNOTATION] = + 'degraded'; + // Ensure managed-by-location stays current + retainedEntity.metadata.annotations[ + MANAGED_BY_LOCATION_ANNOTATION + ] = managedByLocation; + + entities.push({ + entity: retainedEntity, + locationKey: PROVIDER_NAME, + }); + hasDegradedEntries = true; + this.logger.info( + `Retained last-good entity for "${serverName}" ` + + `version "${serverVersion}" with degraded sync status.`, + ); + } else { + this.logger.info( + `No last-good entity found for "${serverName}" ` + + `version "${serverVersion}"; omitting from mutation.`, + ); + } + } + } + } + + if (hasDegradedEntries) { + this.logger.warn( + `MCP Registry sync completed with degraded entries. ` + + `Some server entries could not be mapped and are using ` + + `last-good entities.`, + ); + } + + // Commit full mutation + await this.connection.applyMutation({ + type: 'full', + entities, + }); + + // Update the last-good index with all successfully committed entities + this.lastGoodIndex.clear(); + for (const deferred of entities) { + const name = + deferred.entity.metadata?.annotations?.['modelcontextprotocol.io/name']; + const version = + deferred.entity.metadata?.annotations?.[ + 'modelcontextprotocol.io/version' + ]; + if (name && version) { + this.lastGoodIndex.set(buildLastGoodKey(name, version), deferred); + } + } + + this.logger.info( + `MCP Registry sync completed: ${entities.length} entities committed.`, + ); + } +} diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts new file mode 100644 index 00000000000..b0577ddcfff --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts @@ -0,0 +1,41 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import type { McpServerDocument } from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; + +/** + * Create a minimal valid MCP server.json document for testing. + */ +export function createMockServerDoc( + name: string, + version: string, + overrides?: Partial, +): McpServerDocument { + return { + $schema: + 'https://raw.githubusercontent.com/modelcontextprotocol/registry/v1.8.1/docs/reference/server-json/draft/server.schema.json', + name, + description: `Test server ${name}`, + version, + remotes: [ + { + type: 'streamable-http', + url: `https://${name.replace('/', '.')}.example.com/mcp`, + }, + ], + ...overrides, + }; +} diff --git a/workspaces/ai-integrations/yarn.lock b/workspaces/ai-integrations/yarn.lock index d75f623214c..84838f77055 100644 --- a/workspaces/ai-integrations/yarn.lock +++ b/workspaces/ai-integrations/yarn.lock @@ -9863,6 +9863,20 @@ __metadata: languageName: unknown linkType: soft +"@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider@workspace:plugins/catalog-backend-module-mcp-registry-provider": + version: 0.0.0-use.local + resolution: "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider@workspace:plugins/catalog-backend-module-mcp-registry-provider" + dependencies: + "@backstage/backend-plugin-api": "npm:^1.10.0" + "@backstage/backend-test-utils": "npm:^1.11.6" + "@backstage/catalog-model": "npm:^1.10.0" + "@backstage/cli": "npm:^0.36.5" + "@backstage/config": "npm:^1.3.8" + "@backstage/plugin-catalog-node": "npm:^2.2.4" + "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common": "workspace:^" + languageName: unknown + linkType: soft + "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-model-catalog@workspace:^, @red-hat-developer-hub/backstage-plugin-catalog-backend-module-model-catalog@workspace:plugins/catalog-backend-module-model-catalog": version: 0.0.0-use.local resolution: "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-model-catalog@workspace:plugins/catalog-backend-module-model-catalog" @@ -9984,7 +9998,7 @@ __metadata: languageName: unknown linkType: soft -"@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common@workspace:plugins/mcp-registry-server-mapping-common": +"@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common@workspace:^, @red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common@workspace:plugins/mcp-registry-server-mapping-common": version: 0.0.0-use.local resolution: "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common@workspace:plugins/mcp-registry-server-mapping-common" dependencies: From 3f818af69a2b2ab937bfabaeb3de4a0f0eec9f31 Mon Sep 17 00:00:00 2001 From: fullsend-fix <278716306+fullsend-ai-coder[bot]@users.noreply.github.com> Date: Fri, 18 Sep 2026 19:44:21 +0000 Subject: [PATCH 02/63] fix: address review feedback on PR #4871 - Wrap new URL(endpoint) in try/catch throwing McpRegistryClientError in client.ts (error handling gap) - Add safeGetOptionalString helper and use for all config string reads in config.ts (error-handling idioms) - Add URL scheme validation (http/https only) for baseUrl in config.ts (SSRF/input validation) - Add pageLimit >= 1 and pageSize >= 1 bounds checks in config.ts (edge case/input validation) - Update spec.md to document in-memory last-good index semantics (spec-implementation divergence) - Remove redundant first provider setup in degraded-retention test (test adequacy) - Document annotation key dependency in last-good index rebuild comment in provider.ts (architectural coherence) - Import and use McpServerMappingDefaults type instead of inline type in provider.ts (API shape patterns) - Update README.md to reference existing provider instead of future (stale-doc) - Add @visibility backend annotation to schedule field in config.d.ts (config schema visibility) Addresses #4871 Assisted-by: Claude Opus 4.6 --- .../specs/mcp-registry-provider/spec.md | 2 +- .../config.d.ts | 1 + .../src/client.ts | 12 +++- .../src/config.ts | 57 +++++++++++++++++-- .../src/provider.test.ts | 24 -------- .../src/provider.ts | 15 +++-- .../README.md | 2 +- 7 files changed, 77 insertions(+), 36 deletions(-) diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/specs/mcp-registry-provider/spec.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/specs/mcp-registry-provider/spec.md index 032558c9b8a..1e3b1a64668 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/specs/mcp-registry-provider/spec.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/specs/mcp-registry-provider/spec.md @@ -205,7 +205,7 @@ At the end of each successful sync, the provider SHALL commit to the catalog as ### Requirement: Resilient, agent-native error handling -At the start of each sync run, before mapping accumulated servers, the provider SHALL load all provider-managed catalog entities (mutation `locationKey` `mcp-registry-provider`) into a last-good index keyed by `metadata.annotations['modelcontextprotocol.io/name']` and `metadata.annotations['modelcontextprotocol.io/version']`. +The provider SHALL maintain an in-memory last-good index keyed by `metadata.annotations['modelcontextprotocol.io/name']` and `metadata.annotations['modelcontextprotocol.io/version']`. The index is populated from successfully committed entities at the end of each sync run and is available for lookup during subsequent sync runs within the same process lifetime. The index does not persist across provider restarts; on the first sync after a restart, no last-good entries are available for retention. A single accumulated server entry that cannot be mapped (e.g. it omits a `server.json`-required field and the mapping rejects it) SHALL be logged with an actionable message identifying the entry and SHALL NOT abort the sync. When that entry's `server.json` includes both `name` and `version`, the provider SHALL look up a last-good entity in that index keyed by that `name` and `version` and SHALL include it unchanged in the full mutation when found. When `name` or `version` is absent, or no last-good entity exists, the entry contributes no entity to the mutation. A registry transport or protocol error (unreachable host, non-2xx HTTP status, unparseable response body, or pagination-safeguard trip) SHALL fail the current sync run: the provider SHALL NOT commit a mutation, SHALL log the error, and SHALL retry on the next scheduled tick, leaving the prior catalog state intact. diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts index 8c1e9429fc5..acd59c5579b 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts @@ -31,6 +31,7 @@ export interface Config { pageLimit?: number; /** @visibility backend */ pageSize?: number; + /** @visibility backend */ schedule?: SchedulerServiceTaskScheduleDefinitionConfig; }; }; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index 4ac8cf0dd8c..0d53ad01fd9 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -85,6 +85,16 @@ export async function fetchRegistryServers( const doFetch = fetchApi ?? fetch; const endpoint = buildServersEndpoint(baseUrl, apiVersion); + + let parsedEndpoint: URL; + try { + parsedEndpoint = new URL(endpoint); + } catch (err) { + throw new McpRegistryClientError( + `Invalid MCP Registry endpoint URL "${endpoint}": ${err}`, + ); + } + const allServers: McpRegistryServerEntry[] = []; const seenCursors = new Set(); let cursor: string | undefined; @@ -93,7 +103,7 @@ export async function fetchRegistryServers( // eslint-disable-next-line no-constant-condition while (true) { // Build request URL with query params - const url = new URL(endpoint); + const url = new URL(parsedEndpoint.toString()); if (cursor) { url.searchParams.set('cursor', cursor); } diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts index 6ea65d1be33..388bec1e88a 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts @@ -30,6 +30,24 @@ const DEFAULT_API_VERSION = 'v1'; /** Default page limit (max pages per sync). */ const DEFAULT_PAGE_LIMIT = 10; +/** + * Safely read an optional string from config, returning `undefined` + * when Backstage's ConfigReader throws TypeError for empty-string + * values from env var substitution like `${VAR:-}`. + */ +function safeGetOptionalString( + config: Config, + key: string, +): string | undefined { + try { + return config.getOptionalString(key); + } catch { + // ConfigReader throws TypeError for empty-string values + // from env var substitution like ${VAR:-} + return undefined; + } +} + /** * Parsed provider configuration. */ @@ -96,7 +114,7 @@ export function readMcpRegistryProviderConfig( } // baseUrl is required - const baseUrl = registryConfig.getOptionalString('baseUrl'); + const baseUrl = safeGetOptionalString(registryConfig, 'baseUrl'); if (!baseUrl) { throw new Error( `Invalid catalog.providers.mcpRegistry configuration: missing ` + @@ -105,13 +123,44 @@ export function readMcpRegistryProviderConfig( ); } - const baseName = registryConfig.getOptionalString('baseName'); + // Validate URL scheme (defense in depth against non-HTTP protocols) + let parsedUrl: URL; + try { + parsedUrl = new URL(baseUrl); + } catch { + throw new Error( + `Invalid catalog.providers.mcpRegistry configuration: "baseUrl" ` + + `is not a valid URL: "${baseUrl}". Set baseUrl to an absolute ` + + `HTTP(S) URL (e.g., "https://registry.example.com").`, + ); + } + if (parsedUrl.protocol !== 'http:' && parsedUrl.protocol !== 'https:') { + throw new Error( + `Invalid catalog.providers.mcpRegistry configuration: "baseUrl" ` + + `must use http or https protocol, got "${parsedUrl.protocol}" ` + + `in "${baseUrl}".`, + ); + } + + const baseName = safeGetOptionalString(registryConfig, 'baseName'); const apiVersion = - registryConfig.getOptionalString('apiVersion') ?? DEFAULT_API_VERSION; - const defaultOwner = registryConfig.getOptionalString('defaultOwner'); + safeGetOptionalString(registryConfig, 'apiVersion') ?? DEFAULT_API_VERSION; + const defaultOwner = safeGetOptionalString(registryConfig, 'defaultOwner'); const pageLimit = registryConfig.getOptionalNumber('pageLimit') ?? DEFAULT_PAGE_LIMIT; + if (pageLimit < 1) { + throw new Error( + `Invalid catalog.providers.mcpRegistry configuration: "pageLimit" ` + + `must be at least 1, got ${pageLimit}.`, + ); + } const pageSize = registryConfig.getOptionalNumber('pageSize'); + if (pageSize !== undefined && pageSize < 1) { + throw new Error( + `Invalid catalog.providers.mcpRegistry configuration: "pageSize" ` + + `must be at least 1, got ${pageSize}.`, + ); + } // Schedule: read from config or use default let schedule: SchedulerServiceTaskScheduleDefinition; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts index be89e3df855..72600e630d0 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts @@ -298,35 +298,11 @@ describe('McpRegistryEntityProvider', () => { }); it('retains last-good entity with degraded status on mapping failure', async () => { - // First sync: successful mapping const goodBody: McpRegistryListResponse = { servers: [{ server: createMockServerDoc('test/server', '1.0.0') }], metadata: { count: 1 }, }; - const fetchFn1 = mockFetchForResponses([goodBody]); - const connection = createMockConnection(); - const logger = createMockLogger(); - - const provider = new McpRegistryEntityProvider( - createDefaultConfig(), - logger, - fetchFn1, - ); - await provider.connect(connection); - await provider.run(); - - // Verify first sync succeeded - expect(connection.applyMutation).toHaveBeenCalledTimes(1); - const firstMutation = (connection.applyMutation as jest.Mock).mock - .calls[0][0]; - expect(firstMutation.entities).toHaveLength(1); - expect( - firstMutation.entities[0].entity.metadata.annotations[ - 'redhat.com/rhdh-mcp-registry-sync-status' - ], - ).toBe('ok'); - // Second sync: mapping fails (empty description) const badBody: McpRegistryListResponse = { servers: [ { diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts index 507a1fcf177..6586586043b 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts @@ -24,6 +24,7 @@ import { mapServerToEntity, projectAnnotations, } from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; +import type { McpServerMappingDefaults } from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; import type { McpRegistryProviderConfig } from './config'; import { fetchRegistryServers, McpRegistryClientError } from './client'; import type { McpRegistryServerEntry } from './client'; @@ -131,10 +132,7 @@ export class McpRegistryEntityProvider implements EntityProvider { const serverDoc = entry.server; try { // Invoke the mapping transform - const mappingDefaults: { - prefix?: string; - owner?: string; - } = {}; + const mappingDefaults: McpServerMappingDefaults = {}; if (defaultOwner) { mappingDefaults.owner = defaultOwner; } @@ -234,7 +232,14 @@ export class McpRegistryEntityProvider implements EntityProvider { entities, }); - // Update the last-good index with all successfully committed entities + // Update the last-good index with all successfully committed entities. + // The annotation keys used here ('modelcontextprotocol.io/name' and + // 'modelcontextprotocol.io/version') are set by mapServerToEntity in + // mcp-registry-server-mapping-common and correspond to the raw + // serverDoc.name and serverDoc.version fields used in buildLastGoodKey + // during failure recovery above. If the mapping library changes these + // annotation keys, both this rebuild and the failure recovery path + // must be updated in tandem. this.lastGoodIndex.clear(); for (const deferred of entities) { const name = diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/README.md b/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/README.md index bcb3e8652e9..eee0772493c 100644 --- a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/README.md +++ b/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/README.md @@ -6,7 +6,7 @@ Deterministic transform from [MCP Registry](https://github.com/modelcontextproto documents to Backstage `API` entities with `spec.type: mcp-server`. This common library is a pure mapping contract (no I/O, no registry client). -It is intended for consumers such as a future `mcp-registry-provider` catalog +It is consumed by the `catalog-backend-module-mcp-registry-provider` catalog entity provider. ## Install From 041d90a187fd7be19dcc1320c0c5f46231da588c Mon Sep 17 00:00:00 2001 From: fullsend-fix <278716306+fullsend-ai-coder[bot]@users.noreply.github.com> Date: Fri, 18 Sep 2026 21:35:10 +0000 Subject: [PATCH 03/63] fix: address review feedback on PR #4871 - Wrap getOptionalConfig() in try-catch for scalar TypeError (config.ts) - Move entry.server access inside try block for null safety (provider.ts) - Exclude degraded entities from lastGoodIndex to prevent perpetual retention - Truncate error response body to 256 chars to avoid data exposure (client.ts) - Refactor while(true) to while(hasMorePages) to remove eslint-disable - Re-export McpRegistryEntityProvider and McpRegistryProviderConfig (index.ts) - Add @public release tags and regenerate API report - Update design.md D6 to describe in-memory index populated at end of sync - Update tasks.md 4.1/5.2 to reflect end-of-sync index behavior - Update audit.md timestamp to 2026-09-18 - Update proposal.md consumer reference from "future" to actual plugin name - Add test: empty registry commits full mutation with 0 entities - Add test: applyMutation throw does not update lastGoodIndex - Add test: degraded entities excluded from lastGoodIndex on subsequent syncs Addresses #4871 Assisted-by: Claude Opus 4.6 --- .../changes/mcp-registry-provider/audit.md | 2 +- .../changes/mcp-registry-provider/design.md | 2 +- .../changes/mcp-registry-provider/tasks.md | 4 +- .../mcp-registry-server-mapping/proposal.md | 2 +- .../report.api.md | 36 ++++ .../src/client.ts | 17 +- .../src/config.ts | 11 +- .../src/index.ts | 2 + .../src/provider.test.ts | 169 ++++++++++++++++++ .../src/provider.ts | 20 ++- 10 files changed, 251 insertions(+), 14 deletions(-) diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/audit.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/audit.md index d0d369eb72b..eec93d37aa7 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/audit.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/audit.md @@ -1,6 +1,6 @@ ## Audit Report: mcp-registry-provider -**Last audited:** 2026-09-15T18:03:43Z +**Last audited:** 2026-09-18T00:00:00Z ### Summary diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/design.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/design.md index e706aa4c59d..e6782496584 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/design.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/design.md @@ -105,7 +105,7 @@ Pagination is cursor-based: omit `cursor` on the first request; pass the prior ` **Choice:** Two failure tiers: (a) a single accumulated server entry that the mapping rejects (for example a missing required `server.json` field) is logged with an identifying message; the run proceeds. For that entry, if a **last-good** entity from a prior successful sync exists for the same registry identity, the provider SHALL include that entity in the full mutation with mapping-owned fields unchanged (D5) but with `redhat.com/rhdh-mcp-registry-sync-status: degraded` (D8) so operators can see the entry is stale relative to the latest registry `server.json`. Last-good lookup keys entries by `server.json` `name` and `version` when both are present (matching the mapping's canonical identity annotations `modelcontextprotocol.io/name` and `modelcontextprotocol.io/version` on the prior entity). When `name` or `version` is absent, or no prior entity exists, the entry contributes no entity to the mutation (first-time failure or uncorrelatable entry). (b) A registry-level error (unreachable, non-2xx, unparseable body, or pagination-safeguard trip) fails the whole run: **no** `applyMutation` is emitted, so the last-good catalog state is preserved, and the next scheduled tick retries. -At the start of each sync, the provider loads existing provider-managed entities (via `locationKey` `mcp-registry-provider`) into an index for last-good retention. +The provider maintains an in-memory last-good index that is rebuilt at the end of each successful sync from the entities committed in the mutation. Only entities with `redhat.com/rhdh-mcp-registry-sync-status: ok` are stored in the index; degraded entries are excluded to prevent perpetual retention of stale data. On restart, the index starts empty and is populated after the first successful sync. **Alternatives considered:** Omit failed entries from the full mutation — rejected; a server still present in the registry would be pruned from the catalog. Commit whatever was fetched before a pagination error — rejected; a partial full mutation prunes entities that still exist, causing catalog flapping. diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/tasks.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/tasks.md index a95423408d6..c3cff5f918a 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/tasks.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/tasks.md @@ -30,7 +30,7 @@ ## 4. Entity Provider & Scheduling -- [ ] 4.1 Implement the `EntityProvider` class: `getProviderName()` = `mcp-registry-provider`, `connect()` storing the connection, and a `run()` performing one sync; at the start of each `run()`, load provider-managed entities (`locationKey` `mcp-registry-provider`) into a last-good index keyed by `modelcontextprotocol.io/name` and `modelcontextprotocol.io/version` before mapping (design D6) +- [ ] 4.1 Implement the `EntityProvider` class: `getProviderName()` = `mcp-registry-provider`, `connect()` storing the connection, and a `run()` performing one sync; maintain an in-memory last-good index keyed by `modelcontextprotocol.io/name` and `modelcontextprotocol.io/version`, rebuilt at the end of each successful sync from committed entities with `sync-status: ok` only (design D6) - [ ] 4.2 Wire scheduling via `SchedulerService.createScheduledTaskRunner(schedule)` only (no synchronous `run()` from `connect()`); register the single provider when config is present - [ ] 4.3 Implement the full-mutation commit: on successful sync call `connection.applyMutation({ type: 'full', entities })`; on a failed run emit no mutation (preserve prior catalog state) - [ ] 4.4 Attach provider attribution and sync status to each entity: set mutation `locationKey` `mcp-registry-provider`, `backstage.io/managed-by-location` to `url:` + normalized `baseUrl` (trailing `/` stripped), and `redhat.com/rhdh-mcp-registry-sync-status` to `ok` or `degraded` per D8 @@ -39,7 +39,7 @@ ## 5. Mapping Integration - [ ] 5.1 Depend on the sibling `mcp-registry-server-mapping` transform and invoke it per accumulated server, passing `defaultOwner` as the caller-override owner default and, when configured, `baseName` as the caller-override identity prefix (never reimplement the mapping) -- [ ] 5.2 Implement per-entry failure isolation: on mapping rejection, log an actionable message; when `server.json` has `name` and `version`, include the last-good provider-managed entity (indexed at sync start) with mapping-owned fields unchanged and `redhat.com/rhdh-mcp-registry-sync-status: degraded`; on success set `ok`; otherwise omit the entry; continue the run +- [ ] 5.2 Implement per-entry failure isolation: on mapping rejection, log an actionable message; when `server.json` has `name` and `version`, include the last-good provider-managed entity (from the in-memory index populated at end of prior sync) with mapping-owned fields unchanged and `redhat.com/rhdh-mcp-registry-sync-status: degraded`; on success set `ok`; otherwise omit the entry; continue the run - [ ] 5.3 Add integration tests over sample `server.json` inputs → produced `mcp-server` `API` entities, asserting `spec.owner` reflects `defaultOwner` (and the mapping default `unknown` when omitted), `metadata.name` uses `baseName` as prefix when configured (and mapping default `mcp.registry` when omitted), that one bad entry does not abort the batch, that a mapping failure on a previously synced server retains the last-good entity with `redhat.com/rhdh-mcp-registry-sync-status: degraded`, and that successful mappings set `ok` ## 6. End-to-End Verification & Docs diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/proposal.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/proposal.md index 27a6dde574b..6352f223afb 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/proposal.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/proposal.md @@ -42,5 +42,5 @@ _(none — introduces new capabilities only; consumes the upstream Backstage `mc - **Upstream target**: `McpServerApiEntity` shape (top-level `spec.remotes[]`, no `spec.definition`) — detailed anchors in `design.md`; requirements in `specs/mcp-registry-server-mapping/spec.md`. - **Source**: MCP Registry draft `server.json` (version-pinned in implementation); unknown fields fail-open via projection. -- **Consumers**: future registry entity provider; catalog search over `modelcontextprotocol.io/*` annotations. +- **Consumers**: `catalog-backend-module-mcp-registry-provider` entity provider; catalog search over `modelcontextprotocol.io/*` annotations. - **Alignment**: track Backstage RFC [#32062](https://github.com/backstage/backstage/issues/32062) and registry schema drift. diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md index 429596a9696..491d9a4fcf6 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md @@ -4,8 +4,44 @@ ```ts import { BackendFeature } from '@backstage/backend-plugin-api'; +import type { EntityProvider } from '@backstage/plugin-catalog-node'; +import type { EntityProviderConnection } from '@backstage/plugin-catalog-node'; +import type { LoggerService } from '@backstage/backend-plugin-api'; +import type { SchedulerServiceTaskScheduleDefinition } from '@backstage/backend-plugin-api'; // @public const catalogModuleMcpRegistryProvider: BackendFeature; export default catalogModuleMcpRegistryProvider; + +// @public +export class McpRegistryEntityProvider implements EntityProvider { + constructor( + config: McpRegistryProviderConfig, + logger: LoggerService, + fetchApi?: typeof fetch, + ); + // (undocumented) + connect(connection: EntityProviderConnection): Promise; + // (undocumented) + getProviderName(): string; + run(): Promise; +} + +// @public +export interface McpRegistryProviderConfig { + // (undocumented) + apiVersion: string; + // (undocumented) + baseName?: string; + // (undocumented) + baseUrl: string; + // (undocumented) + defaultOwner?: string; + // (undocumented) + pageLimit: number; + // (undocumented) + pageSize?: number; + // (undocumented) + schedule: SchedulerServiceTaskScheduleDefinition; +} ``` diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index 0d53ad01fd9..5cbceedf040 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -100,8 +100,8 @@ export async function fetchRegistryServers( let cursor: string | undefined; let pageCount = 0; - // eslint-disable-next-line no-constant-condition - while (true) { + let hasMorePages = true; + while (hasMorePages) { // Build request URL with query params const url = new URL(parsedEndpoint.toString()); if (cursor) { @@ -121,11 +121,15 @@ export async function fetchRegistryServers( } if (!response.ok) { + const MAX_BODY_LENGTH = 256; + const rawBody = await response.text().catch(() => '(no body)'); + const truncatedBody = + rawBody.length > MAX_BODY_LENGTH + ? `${rawBody.substring(0, MAX_BODY_LENGTH)}…(truncated)` + : rawBody; throw new McpRegistryClientError( `MCP Registry returned HTTP ${response.status} for ` + - `${url.toString()}: ${await response - .text() - .catch(() => '(no body)')}`, + `${url.toString()}: ${truncatedBody}`, ); } @@ -153,7 +157,8 @@ export async function fetchRegistryServers( const nextCursor = body.metadata?.nextCursor; if (!nextCursor || nextCursor.length === 0) { // No more pages - break; + hasMorePages = false; + continue; } // Repeated cursor safeguard diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts index 388bec1e88a..a02f4e1889b 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts @@ -50,6 +50,8 @@ function safeGetOptionalString( /** * Parsed provider configuration. + * + * @public */ export interface McpRegistryProviderConfig { baseUrl: string; @@ -100,7 +102,14 @@ export function readMcpRegistryProviderConfig( // Check if the unknown keys look like instance identifiers (they // would have nested config objects with their own properties) for (const key of unknownKeys) { - const nested = registryConfig.getOptionalConfig(key); + let nested; + try { + nested = registryConfig.getOptionalConfig(key); + } catch { + // ConfigReader throws TypeError when the value is a scalar + // rather than an object — skip this key silently. + continue; + } if (nested && nested.keys().length > 0) { throw new Error( `Invalid catalog.providers.mcpRegistry configuration: found ` + diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts index 0ded7b32007..ad3e7f701aa 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts @@ -21,3 +21,5 @@ */ export { catalogModuleMcpRegistryProvider as default } from './module'; +export { McpRegistryEntityProvider } from './provider'; +export type { McpRegistryProviderConfig } from './config'; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts index 72600e630d0..d6ee546f337 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts @@ -501,4 +501,173 @@ describe('McpRegistryEntityProvider', () => { 'Updated weather description', ); }); + + it('commits full mutation with empty entities for empty registry', async () => { + const body: McpRegistryListResponse = { + servers: [], + metadata: { count: 0 }, + }; + const fetchFn = mockFetchForResponses([body]); + const connection = createMockConnection(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + fetchFn, + ); + await provider.connect(connection); + await provider.run(); + + expect(connection.applyMutation).toHaveBeenCalledTimes(1); + const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + expect(mutation.type).toBe('full'); + expect(mutation.entities).toHaveLength(0); + }); + + it('does not update lastGoodIndex when applyMutation throws', async () => { + const goodBody: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server', '1.0.0') }], + metadata: { count: 1 }, + }; + + const badBody: McpRegistryListResponse = { + servers: [ + { + server: { + $schema: + 'https://raw.githubusercontent.com/modelcontextprotocol/registry/v1.8.1/docs/reference/server-json/draft/server.schema.json', + name: 'test/server', + description: '', + version: '1.0.0', + } as any, + }, + ], + metadata: { count: 1 }, + }; + + const combinedFetch = jest.fn(); + // First sync — succeeds and populates last-good index + combinedFetch.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => goodBody, + text: async () => JSON.stringify(goodBody), + } as unknown as Response); + // Second sync — good data, but applyMutation will throw + combinedFetch.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => goodBody, + text: async () => JSON.stringify(goodBody), + } as unknown as Response); + // Third sync — mapping fails, should still use last-good from first sync + combinedFetch.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => badBody, + text: async () => JSON.stringify(badBody), + } as unknown as Response); + + const connection = createMockConnection(); + const logger = createMockLogger(); + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + logger, + combinedFetch, + ); + await provider.connect(connection); + + // First sync — succeeds, populates last-good + await provider.run(); + expect(connection.applyMutation).toHaveBeenCalledTimes(1); + + // Second sync — applyMutation throws + (connection.applyMutation as jest.Mock).mockRejectedValueOnce( + new Error('catalog unavailable'), + ); + await expect(provider.run()).rejects.toThrow('catalog unavailable'); + + // Third sync — mapping fails; last-good should still be available + // from the first sync (applyMutation throw did not update the index) + (connection.applyMutation as jest.Mock).mockResolvedValueOnce(undefined); + await provider.run(); + expect(connection.applyMutation).toHaveBeenCalledTimes(3); + + const thirdMutation = (connection.applyMutation as jest.Mock).mock + .calls[2][0]; + expect(thirdMutation.entities).toHaveLength(1); + expect( + thirdMutation.entities[0].entity.metadata.annotations[ + 'redhat.com/rhdh-mcp-registry-sync-status' + ], + ).toBe('degraded'); + }); + + it('does not retain degraded entities in lastGoodIndex on subsequent syncs', async () => { + const goodBody: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server', '1.0.0') }], + metadata: { count: 1 }, + }; + + const badBody: McpRegistryListResponse = { + servers: [ + { + server: { + $schema: + 'https://raw.githubusercontent.com/modelcontextprotocol/registry/v1.8.1/docs/reference/server-json/draft/server.schema.json', + name: 'test/server', + description: '', + version: '1.0.0', + } as any, + }, + ], + metadata: { count: 1 }, + }; + + const combinedFetch = jest.fn(); + // First sync — succeeds + combinedFetch.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => goodBody, + text: async () => JSON.stringify(goodBody), + } as unknown as Response); + // Second sync — mapping fails, uses last-good (degraded) + combinedFetch.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => badBody, + text: async () => JSON.stringify(badBody), + } as unknown as Response); + // Third sync — mapping fails again; degraded entity from second + // sync should NOT be in last-good index + combinedFetch.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => badBody, + text: async () => JSON.stringify(badBody), + } as unknown as Response); + + const connection = createMockConnection(); + const logger = createMockLogger(); + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + logger, + combinedFetch, + ); + await provider.connect(connection); + + // First sync — populates last-good + await provider.run(); + // Second sync — uses last-good, commits degraded + await provider.run(); + // Third sync — degraded entity from second sync should not be + // in last-good index, so no entity should be retained + await provider.run(); + + expect(connection.applyMutation).toHaveBeenCalledTimes(3); + const thirdMutation = (connection.applyMutation as jest.Mock).mock + .calls[2][0]; + expect(thirdMutation.entities).toHaveLength(0); + }); }); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts index 6586586043b..878b5f652da 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts @@ -56,6 +56,8 @@ function normalizeBaseUrl(baseUrl: string): string { /** * Entity provider that ingests MCP servers from one configured * MCP Registry into the Backstage catalog. + * + * @public */ export class McpRegistryEntityProvider implements EntityProvider { private connection?: EntityProviderConnection; @@ -129,8 +131,8 @@ export class McpRegistryEntityProvider implements EntityProvider { let hasDegradedEntries = false; for (const entry of entries) { - const serverDoc = entry.server; try { + const serverDoc = entry.server; // Invoke the mapping transform const mappingDefaults: McpServerMappingDefaults = {}; if (defaultOwner) { @@ -168,6 +170,10 @@ export class McpRegistryEntityProvider implements EntityProvider { entities.push(deferred); } catch (err) { // Per-entry failure: log and attempt last-good retention + const serverDoc = + entry !== null && entry !== undefined + ? (entry as McpRegistryServerEntry).server + : undefined; const serverName = typeof serverDoc?.name === 'string' ? serverDoc.name : undefined; const serverVersion = @@ -232,7 +238,12 @@ export class McpRegistryEntityProvider implements EntityProvider { entities, }); - // Update the last-good index with all successfully committed entities. + // Update the last-good index with successfully mapped entities only. + // Entities that carry sync-status "degraded" are excluded: they are + // last-good fallbacks from a prior cycle, so storing them back would + // create perpetual retention of stale data. Only "ok" entities + // qualify as last-good candidates. + // // The annotation keys used here ('modelcontextprotocol.io/name' and // 'modelcontextprotocol.io/version') are set by mapServerToEntity in // mcp-registry-server-mapping-common and correspond to the raw @@ -242,6 +253,11 @@ export class McpRegistryEntityProvider implements EntityProvider { // must be updated in tandem. this.lastGoodIndex.clear(); for (const deferred of entities) { + const syncStatus = + deferred.entity.metadata?.annotations?.[SYNC_STATUS_ANNOTATION]; + if (syncStatus === 'degraded') { + continue; + } const name = deferred.entity.metadata?.annotations?.['modelcontextprotocol.io/name']; const version = From de4b2d8aebdc53a63a633314c767c952a254c5ea Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Fri, 18 Sep 2026 18:18:18 -0400 Subject: [PATCH 04/63] fix(#4815): address review comments for package.json Signed-off-by: Michael Valdron --- .../package.json | 9 +++++---- .../mcp-registry-server-mapping-common/package.json | 3 ++- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json index c0409c927ee..3127dfbd636 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json @@ -1,8 +1,8 @@ { "name": "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider", - "version": "0.1.0", + "version": "0.3.0", "license": "Apache-2.0", - "description": "The mcp-registry-provider backend module for the catalog plugin.", + "description": "The mcp-registry-provider backend module for the catalog plugin. Provides the MCP Server API catalog entities from a target MCP Registry.", "main": "src/index.ts", "types": "src/index.ts", "publishConfig": { @@ -17,10 +17,11 @@ }, "backstage": { "role": "backend-plugin-module", - "pluginId": "catalog", + "pluginId": "mcp-registry-provider", "pluginPackage": "@backstage/plugin-catalog-backend", "pluginPackages": [ - "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider" + "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider", + "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common" ] }, "scripts": { diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json b/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json index 6a286c2d00c..d0d743541dd 100644 --- a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json +++ b/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json @@ -20,7 +20,8 @@ "pluginId": "mcp-registry-provider", "pluginPackage": "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common", "pluginPackages": [ - "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common" + "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common", + "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider" ] }, "sideEffects": false, From d70dcf995b8a6b117213417349df647bb6222efd Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Fri, 18 Sep 2026 18:20:44 -0400 Subject: [PATCH 05/63] fix(#4815): expand more scripts in package.json files Signed-off-by: Michael Valdron --- .../package.json | 10 +++++++--- .../mcp-registry-server-mapping-common/package.json | 11 ++++++++--- 2 files changed, 15 insertions(+), 6 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json index 3127dfbd636..a3019bfa2a9 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json @@ -27,11 +27,15 @@ "scripts": { "start": "backstage-cli package start", "build": "backstage-cli package build", - "lint": "backstage-cli package lint", - "test": "backstage-cli package test", + "lint:check": "backstage-cli package lint", + "lint:fix": "backstage-cli package lint --fix", + "test": "backstage-cli package test --passWithNoTests --coverage", "clean": "backstage-cli package clean", "prepack": "backstage-cli package prepack", - "postpack": "backstage-cli package postpack" + "postpack": "backstage-cli package postpack", + "tsc": "tsc", + "prettier:check": "prettier --ignore-unknown --check .", + "prettier:fix": "prettier --ignore-unknown --write ." }, "dependencies": { "@backstage/backend-plugin-api": "^1.10.0", diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json b/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json index d0d743541dd..bf411dd3d2b 100644 --- a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json +++ b/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json @@ -27,11 +27,16 @@ "sideEffects": false, "scripts": { "build": "backstage-cli package build", - "lint": "backstage-cli package lint", - "test": "backstage-cli package test", "clean": "backstage-cli package clean", + "lint:check": "backstage-cli package lint", + "lint:fix": "backstage-cli package lint --fix", + "postpack": "backstage-cli package postpack", "prepack": "backstage-cli package prepack", - "postpack": "backstage-cli package postpack" + "start": "backstage-cli package start", + "test": "backstage-cli package test --passWithNoTests --coverage", + "tsc": "tsc", + "prettier:check": "prettier --ignore-unknown --check .", + "prettier:fix": "prettier --ignore-unknown --write ." }, "dependencies": { "@backstage/catalog-model": "^1.10.1" From e3fc83838fab58c6d00dbeb0a1deea46fcd613aa Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Fri, 18 Sep 2026 18:32:24 -0400 Subject: [PATCH 06/63] fix(#4815): add standalone dev entry for mcp-registry-provider Enable yarn start for the catalog module by adding a local backend entrypoint and the catalog/backend-defaults start dependencies. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../dev/index.ts | 27 +++++++++++++++++++ .../package.json | 4 ++- 2 files changed, 30 insertions(+), 1 deletion(-) create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/dev/index.ts diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/dev/index.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/dev/index.ts new file mode 100644 index 00000000000..84cf89ab1f9 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/dev/index.ts @@ -0,0 +1,27 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +import { createBackend } from '@backstage/backend-defaults'; +import { mockServices } from '@backstage/backend-test-utils'; + +const backend = createBackend(); + +backend.add(mockServices.auth.factory()); +backend.add(mockServices.httpAuth.factory()); + +backend.add(import('@backstage/plugin-catalog-backend')); +backend.add(import('../src')); + +backend.start(); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json index a3019bfa2a9..c46ffb07465 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json @@ -44,9 +44,11 @@ "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common": "workspace:^" }, "devDependencies": { + "@backstage/backend-defaults": "^0.17.8", "@backstage/backend-test-utils": "^1.11.6", "@backstage/cli": "^0.36.5", - "@backstage/config": "^1.3.8" + "@backstage/config": "^1.3.8", + "@backstage/plugin-catalog-backend": "^3.9.0" }, "files": [ "dist" From 3a0f6d524dac7af2752241d2255e165130bb784a Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Fri, 18 Sep 2026 18:50:25 -0400 Subject: [PATCH 07/63] fix(#4815): add staging MCP Registry provider config Point local plugin and workspace app configs at the staging registry so the catalog provider can sync when started standalone or from the workspace. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- workspaces/ai-integrations/app-config.yaml | 21 ++++++++++++++++++- .../app-config.yaml | 21 +++++++++++++++++++ 2 files changed, 41 insertions(+), 1 deletion(-) create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml diff --git a/workspaces/ai-integrations/app-config.yaml b/workspaces/ai-integrations/app-config.yaml index 35384f3b1b0..bf3ca8d6b85 100644 --- a/workspaces/ai-integrations/app-config.yaml +++ b/workspaces/ai-integrations/app-config.yaml @@ -139,7 +139,26 @@ catalog: kubeflow-model-catalog-url: '${KUBEFLOW_MODEL_CATALOG_URL:-}' default-owner: '${OWNER:-default-owner}' default-lifecycle: '${LIFECYCLE:-production}' -# Uncomment to use kubernetesPluginRef — the Backstage kubernetes plugin + mcpRegistry: + # Required: base URL of the MCP Registry + baseUrl: https://staging.registry.modelcontextprotocol.io/ + # Optional: base name (default: mcp.registry) + baseName: staging.registry.modelcontextprotocol.io + # Optional: API version (default: v1) + apiVersion: v0.1 + # Optional: default entity owner (default: unknown) + defaultOwner: user:development/guest + # Optional: max pages fetched per sync (default: 10) + # pageLimit: 10 + # Optional: registry page size sent as ?limit= (omitted by default) + # pageSize: 50 + # Optional: sync schedule (defaults shown below) + # schedule: + # frequency: { minutes: 30 } + # timeout: { minutes: 3 } + # # Optional: defer the first sync + # # initialDelay: { seconds: 15 } +# Uncomment to use kubernetesPluginRef — the Backstage Kubernetes plugin # does NOT need to be installed, only its config section is needed. #kubernetes: # serviceLocatorMethod: diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml new file mode 100644 index 00000000000..a05e061985c --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml @@ -0,0 +1,21 @@ +catalog: + providers: + mcpRegistry: + # Required: base URL of the MCP Registry + baseUrl: https://staging.registry.modelcontextprotocol.io/ + # Optional: base name (default: mcp.registry) + baseName: staging.registry.modelcontextprotocol.io + # Optional: API version (default: v1) + apiVersion: v0.1 + # Optional: default entity owner (default: unknown) + defaultOwner: user:development/guest + # Optional: max pages fetched per sync (default: 10) + # pageLimit: 10 + # Optional: registry page size sent as ?limit= (omitted by default) + # pageSize: 50 + # Optional: sync schedule (defaults shown below) + # schedule: + # frequency: { minutes: 30 } + # timeout: { minutes: 3 } + # # Optional: defer the first sync + # # initialDelay: { seconds: 15 } From 7820ac7b16eed05f39cd2bbcb45f16f9f54650e6 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Fri, 18 Sep 2026 20:23:58 -0400 Subject: [PATCH 08/63] fix(#4815): list ingested mcp-server APIs in the catalog Start the refresh after the catalog connection exists, stamp the origin location annotation, and load the ai-model catalog module so Backstage accepts spec.type mcp-server entities that omit spec.definition. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../README.md | 4 + .../dev/index.ts | 5 + .../package.json | 5 +- .../report.api.md | 2 + .../catalog.processing.integration.test.ts | 137 ++++++++++++++++++ .../src/module.ts | 17 +-- .../src/provider.test.ts | 41 ++++++ .../src/provider.ts | 39 +++-- workspaces/ai-integrations/yarn.lock | 3 + 9 files changed, 232 insertions(+), 21 deletions(-) create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/catalog.processing.integration.test.ts diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index 51cbc107d93..ce0d33f068d 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -8,6 +8,7 @@ Add the module to your backend: ```ts // packages/backend/src/index.ts +backend.add(import('@backstage/plugin-catalog-backend-module-ai-model')); backend.add( import( '@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider' @@ -15,6 +16,8 @@ backend.add( ); ``` +Since [Backstage 1.51.0](https://github.com/backstage/backstage/releases/tag/v1.51.0), `spec.type: mcp-server` entities (they use `spec.remotes` and omit `spec.definition`) are accepted only when `@backstage/plugin-catalog-backend-module-ai-model` is installed. Without that module the catalog keeps the generic API validator, which rejects these entities and does not list them. + ## Configuration Configure the provider in your `app-config.yaml`: @@ -82,6 +85,7 @@ On each successful sync, the provider commits a **full mutation** — the catalo Each entity carries: - `backstage.io/managed-by-location`: `url:` +- `backstage.io/managed-by-origin-location`: `url:` - `redhat.com/rhdh-mcp-registry-sync-status`: `ok` or `degraded` - `modelcontextprotocol.io/name`: the server's canonical name - `modelcontextprotocol.io/version`: the server's version diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/dev/index.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/dev/index.ts index 84cf89ab1f9..b1ad56fe923 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/dev/index.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/dev/index.ts @@ -22,6 +22,11 @@ backend.add(mockServices.auth.factory()); backend.add(mockServices.httpAuth.factory()); backend.add(import('@backstage/plugin-catalog-backend')); +// Since Backstage 1.51.0 this module registers spec.type mcp-server +// (spec.remotes, no spec.definition). Without it, +// BuiltinKindsEntityProcessor rejects the ingested APIs and the catalog +// API never lists them. +backend.add(import('@backstage/plugin-catalog-backend-module-ai-model')); backend.add(import('../src')); backend.start(); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json index c46ffb07465..58c495787cf 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json @@ -48,7 +48,10 @@ "@backstage/backend-test-utils": "^1.11.6", "@backstage/cli": "^0.36.5", "@backstage/config": "^1.3.8", - "@backstage/plugin-catalog-backend": "^3.9.0" + "@backstage/plugin-catalog-backend": "^3.9.0", + "@backstage/plugin-catalog-backend-module-ai-model": "^0.1.3", + "@types/supertest": "^2.0.12", + "supertest": "^6.2.4" }, "files": [ "dist" diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md index 491d9a4fcf6..86a333fca09 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md @@ -7,6 +7,7 @@ import { BackendFeature } from '@backstage/backend-plugin-api'; import type { EntityProvider } from '@backstage/plugin-catalog-node'; import type { EntityProviderConnection } from '@backstage/plugin-catalog-node'; import type { LoggerService } from '@backstage/backend-plugin-api'; +import type { SchedulerServiceTaskRunner } from '@backstage/backend-plugin-api'; import type { SchedulerServiceTaskScheduleDefinition } from '@backstage/backend-plugin-api'; // @public @@ -19,6 +20,7 @@ export class McpRegistryEntityProvider implements EntityProvider { config: McpRegistryProviderConfig, logger: LoggerService, fetchApi?: typeof fetch, + taskRunner?: SchedulerServiceTaskRunner, ); // (undocumented) connect(connection: EntityProviderConnection): Promise; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/catalog.processing.integration.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/catalog.processing.integration.test.ts new file mode 100644 index 00000000000..1f9a512e7b0 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/catalog.processing.integration.test.ts @@ -0,0 +1,137 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import request from 'supertest'; +import { ExtendedHttpServer } from '@backstage/backend-defaults/rootHttpRouter'; +import { mockServices, startTestBackend } from '@backstage/backend-test-utils'; +import type { Entity } from '@backstage/catalog-model'; +import catalogPlugin from '@backstage/plugin-catalog-backend'; +import catalogModuleAiModel from '@backstage/plugin-catalog-backend-module-ai-model'; +import { catalogModuleMcpRegistryProvider } from './module'; + +const REGISTRY_BODY = { + servers: [ + { + server: { + $schema: + 'https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json', + name: 'io.example/weather', + description: 'Weather', + version: '1.0.0', + remotes: [{ type: 'streamable-http', url: 'https://example.com/mcp' }], + }, + }, + ], + metadata: { count: 1 }, +}; + +async function waitForApiEntities( + server: ExtendedHttpServer, + timeoutMs = 40_000, +): Promise { + const start = Date.now(); + let lastBody: unknown; + while (Date.now() - start < timeoutMs) { + const response = await request(server).get( + '/api/catalog/entities?filter=kind=API,spec.type=mcp-server', + ); + lastBody = response.body; + if ( + response.status === 200 && + Array.isArray(response.body) && + response.body.length > 0 + ) { + return response.body as Entity[]; + } + await new Promise(resolve => setTimeout(resolve, 500)); + } + throw new Error( + `Timed out waiting for mcp-server API entities (last body ${JSON.stringify( + lastBody, + )})`, + ); +} + +describe('mcp-server catalog processing', () => { + jest.setTimeout(60_000); + + const originalFetch = global.fetch; + let server: ExtendedHttpServer; + + beforeAll(async () => { + global.fetch = jest.fn(async () => { + return new Response(JSON.stringify(REGISTRY_BODY), { + status: 200, + headers: { 'content-type': 'application/json' }, + }); + }) as typeof fetch; + + const backend = await startTestBackend({ + features: [ + catalogPlugin, + catalogModuleAiModel, + catalogModuleMcpRegistryProvider, + mockServices.rootConfig.factory({ + data: { + app: { baseUrl: 'http://localhost:3000' }, + backend: { + baseUrl: 'http://localhost:7007', + database: { + client: 'better-sqlite3', + connection: ':memory:', + }, + }, + catalog: { + processingInterval: { seconds: 1 }, + rules: [{ allow: ['API'] }], + providers: { + mcpRegistry: { + baseUrl: 'https://registry.example.com', + apiVersion: 'v0.1', + defaultOwner: 'user:default/guest', + schedule: { + frequency: { seconds: 1 }, + timeout: { seconds: 10 }, + }, + }, + }, + }, + }, + }), + mockServices.auth.factory(), + mockServices.httpAuth.factory(), + ], + }); + server = backend.server; + }); + + afterAll(() => { + global.fetch = originalFetch; + }); + + it('lists a committed mcp-server API that omits spec.definition', async () => { + const entities = await waitForApiEntities(server); + expect(entities).toHaveLength(1); + expect(entities[0].kind).toBe('API'); + expect(entities[0].spec).toEqual( + expect.objectContaining({ + type: 'mcp-server', + owner: 'user:default/guest', + }), + ); + expect(entities[0].spec).not.toHaveProperty('definition'); + }); +}); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts index c45c49d72e5..758190ee5ec 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts @@ -53,20 +53,17 @@ export const catalogModuleMcpRegistryProvider = createBackendModule({ return; } - const provider = new McpRegistryEntityProvider(providerConfig, logger); - - catalog.addEntityProvider(provider); - const taskRunner = scheduler.createScheduledTaskRunner( providerConfig.schedule, ); + const provider = new McpRegistryEntityProvider( + providerConfig, + logger, + undefined, + taskRunner, + ); - await taskRunner.run({ - id: 'mcp-registry-provider:refresh', - fn: async () => { - await provider.run(); - }, - }); + catalog.addEntityProvider(provider); }, }); }, diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts index d6ee546f337..a742fd9fc73 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts @@ -16,6 +16,7 @@ import { McpRegistryEntityProvider } from './provider'; import type { McpRegistryProviderConfig } from './config'; +import type { SchedulerServiceTaskRunner } from '@backstage/backend-plugin-api'; import type { EntityProviderConnection } from '@backstage/plugin-catalog-node'; import type { McpRegistryListResponse } from './client'; import { createMockServerDoc } from './testUtils'; @@ -76,6 +77,36 @@ describe('McpRegistryEntityProvider', () => { expect(provider.getProviderName()).toBe('mcp-registry-provider'); }); + it('registers the refresh task from connect after the catalog connection exists', async () => { + const body: McpRegistryListResponse = { + servers: [], + metadata: { count: 0 }, + }; + const fetchFn = mockFetchForResponses([body]); + const connection = createMockConnection(); + let scheduled: (() => Promise) | undefined; + const taskRunner = { + run: jest.fn(async ({ fn }: { fn: () => Promise }) => { + scheduled = fn; + }), + } as unknown as SchedulerServiceTaskRunner & { + run: jest.Mock; + }; + + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + fetchFn, + taskRunner, + ); + await provider.connect(connection); + + expect(taskRunner.run).toHaveBeenCalledTimes(1); + expect(scheduled).toBeDefined(); + await scheduled!(); + expect(connection.applyMutation).toHaveBeenCalledTimes(1); + }); + it('throws if run() is called before connect()', async () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), @@ -115,6 +146,11 @@ describe('McpRegistryEntityProvider', () => { expect( entity.entity.metadata.annotations['backstage.io/managed-by-location'], ).toBe('url:https://registry.example.com'); + expect( + entity.entity.metadata.annotations[ + 'backstage.io/managed-by-origin-location' + ], + ).toBe('url:https://registry.example.com'); expect( entity.entity.metadata.annotations[ 'redhat.com/rhdh-mcp-registry-sync-status' @@ -146,6 +182,11 @@ describe('McpRegistryEntityProvider', () => { 'backstage.io/managed-by-location' ], ).toBe('url:https://registry.example.com'); + expect( + mutation.entities[0].entity.metadata.annotations[ + 'backstage.io/managed-by-origin-location' + ], + ).toBe('url:https://registry.example.com'); }); it('passes defaultOwner to the mapping', async () => { diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts index 878b5f652da..b1db0ca31b2 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts @@ -14,7 +14,14 @@ * limitations under the License. */ -import type { LoggerService } from '@backstage/backend-plugin-api'; +import type { + LoggerService, + SchedulerServiceTaskRunner, +} from '@backstage/backend-plugin-api'; +import { + ANNOTATION_LOCATION, + ANNOTATION_ORIGIN_LOCATION, +} from '@backstage/catalog-model'; import type { DeferredEntity, EntityProvider, @@ -35,9 +42,6 @@ const PROVIDER_NAME = 'mcp-registry-provider'; /** Sync status annotation key. */ const SYNC_STATUS_ANNOTATION = 'redhat.com/rhdh-mcp-registry-sync-status'; -/** Managed-by-location annotation key. */ -const MANAGED_BY_LOCATION_ANNOTATION = 'backstage.io/managed-by-location'; - /** * Build a last-good lookup key from name and version. */ @@ -64,6 +68,7 @@ export class McpRegistryEntityProvider implements EntityProvider { private readonly config: McpRegistryProviderConfig; private readonly logger: LoggerService; private readonly fetchApi?: typeof fetch; + private readonly taskRunner?: SchedulerServiceTaskRunner; /** * Internal last-good index: keyed by `name::version`, stores the @@ -76,10 +81,12 @@ export class McpRegistryEntityProvider implements EntityProvider { config: McpRegistryProviderConfig, logger: LoggerService, fetchApi?: typeof fetch, + taskRunner?: SchedulerServiceTaskRunner, ) { this.config = config; this.logger = logger; this.fetchApi = fetchApi; + this.taskRunner = taskRunner; } getProviderName(): string { @@ -88,6 +95,16 @@ export class McpRegistryEntityProvider implements EntityProvider { async connect(connection: EntityProviderConnection): Promise { this.connection = connection; + // The scheduler's first tick can run immediately. Register it only + // after the catalog connection exists so that tick can commit. + if (this.taskRunner) { + await this.taskRunner.run({ + id: `${PROVIDER_NAME}:refresh`, + fn: async () => { + await this.run(); + }, + }); + } } /** @@ -158,8 +175,10 @@ export class McpRegistryEntityProvider implements EntityProvider { ...projectedAnnotations, }; - // Add provider attribution annotations - entity.metadata.annotations[MANAGED_BY_LOCATION_ANNOTATION] = + // Catalog processing requires both location annotations. Without + // the origin annotation the entity is rejected and never listed. + entity.metadata.annotations[ANNOTATION_LOCATION] = managedByLocation; + entity.metadata.annotations[ANNOTATION_ORIGIN_LOCATION] = managedByLocation; entity.metadata.annotations[SYNC_STATUS_ANNOTATION] = 'ok'; @@ -200,10 +219,10 @@ export class McpRegistryEntityProvider implements EntityProvider { } retainedEntity.metadata.annotations[SYNC_STATUS_ANNOTATION] = 'degraded'; - // Ensure managed-by-location stays current - retainedEntity.metadata.annotations[ - MANAGED_BY_LOCATION_ANNOTATION - ] = managedByLocation; + retainedEntity.metadata.annotations[ANNOTATION_LOCATION] = + managedByLocation; + retainedEntity.metadata.annotations[ANNOTATION_ORIGIN_LOCATION] = + managedByLocation; entities.push({ entity: retainedEntity, diff --git a/workspaces/ai-integrations/yarn.lock b/workspaces/ai-integrations/yarn.lock index 84838f77055..4af734bc306 100644 --- a/workspaces/ai-integrations/yarn.lock +++ b/workspaces/ai-integrations/yarn.lock @@ -9867,11 +9867,14 @@ __metadata: version: 0.0.0-use.local resolution: "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider@workspace:plugins/catalog-backend-module-mcp-registry-provider" dependencies: + "@backstage/backend-defaults": "npm:^0.17.8" "@backstage/backend-plugin-api": "npm:^1.10.0" "@backstage/backend-test-utils": "npm:^1.11.6" "@backstage/catalog-model": "npm:^1.10.0" "@backstage/cli": "npm:^0.36.5" "@backstage/config": "npm:^1.3.8" + "@backstage/plugin-catalog-backend": "npm:^3.9.0" + "@backstage/plugin-catalog-backend-module-ai-model": "npm:^0.1.3" "@backstage/plugin-catalog-node": "npm:^2.2.4" "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common": "workspace:^" languageName: unknown From 07d9a8652fa93516b5f55bf82ec0e17839b7e800 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Fri, 18 Sep 2026 20:26:07 -0400 Subject: [PATCH 09/63] fix(#4815): record supertest in the workspace lockfile Keep the lockfile aligned with the integration test devDependencies. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- workspaces/ai-integrations/yarn.lock | 2 ++ 1 file changed, 2 insertions(+) diff --git a/workspaces/ai-integrations/yarn.lock b/workspaces/ai-integrations/yarn.lock index 4af734bc306..6bb183a2cd7 100644 --- a/workspaces/ai-integrations/yarn.lock +++ b/workspaces/ai-integrations/yarn.lock @@ -9877,6 +9877,8 @@ __metadata: "@backstage/plugin-catalog-backend-module-ai-model": "npm:^0.1.3" "@backstage/plugin-catalog-node": "npm:^2.2.4" "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common": "workspace:^" + "@types/supertest": "npm:^2.0.12" + supertest: "npm:^6.2.4" languageName: unknown linkType: soft From 12b9bad1bdaf17c3cbc7d92f908ab3306271eb35 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Fri, 18 Sep 2026 22:16:11 -0400 Subject: [PATCH 10/63] fix(#4815): address SonarCloud feedback on mcp-registry-provider Split high-complexity provider, config, and client paths into helpers, replace trailing-slash regex stripping with a linear util, and add focused unit coverage for the extracted pieces. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../src/client.test.ts | 138 ++++ .../src/client.ts | 243 +++++--- .../src/config.test.ts | 155 ++++- .../src/config.ts | 211 ++++--- .../src/provider.parts.test.ts | 587 ++++++++++++++++++ .../src/provider.ts | 362 +++++++---- .../src/util.test.ts | 38 ++ .../src/util.ts | 28 + 8 files changed, 1446 insertions(+), 316 deletions(-) create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.parts.test.ts create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.test.ts create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.ts diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts index 8d9d2b06297..76d2e3e3aa4 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts @@ -15,9 +15,14 @@ */ import { + buildPageRequestUrl, buildServersEndpoint, + fetchRegistryPage, fetchRegistryServers, McpRegistryClientError, + parseServersEndpointUrl, + resolveNextCursor, + truncateErrorBody, } from './client'; import type { McpRegistryListResponse } from './client'; import { createMockServerDoc } from './testUtils'; @@ -325,3 +330,136 @@ describe('fetchRegistryServers', () => { expect(secondUrl).toContain(`cursor=${encodeURIComponent(opaqueToken)}`); }); }); + +describe('parseServersEndpointUrl', () => { + it('returns a URL for a valid endpoint', () => { + const url = parseServersEndpointUrl('https://registry.example.com', 'v1'); + expect(url.toString()).toBe('https://registry.example.com/v1/servers'); + }); + + it('throws McpRegistryClientError for an invalid endpoint URL', () => { + expect(() => parseServersEndpointUrl('://bad', 'v1')).toThrow( + McpRegistryClientError, + ); + expect(() => parseServersEndpointUrl('://bad', 'v1')).toThrow( + /Invalid MCP Registry endpoint URL/, + ); + }); +}); + +describe('buildPageRequestUrl', () => { + const endpoint = new URL('https://registry.example.com/v1/servers'); + + it('returns the endpoint when cursor and pageSize are omitted', () => { + expect(buildPageRequestUrl(endpoint).toString()).toBe( + 'https://registry.example.com/v1/servers', + ); + }); + + it('adds cursor and limit query params when provided', () => { + const url = buildPageRequestUrl(endpoint, 'abc', 25); + expect(url.searchParams.get('cursor')).toBe('abc'); + expect(url.searchParams.get('limit')).toBe('25'); + }); + + it('does not mutate the original endpoint URL', () => { + buildPageRequestUrl(endpoint, 'abc', 25); + expect(endpoint.search).toBe(''); + }); +}); + +describe('truncateErrorBody', () => { + it('returns the body unchanged when within the limit', () => { + expect(truncateErrorBody('short')).toBe('short'); + }); + + it('truncates long bodies and appends a marker', () => { + const raw = 'a'.repeat(300); + const truncated = truncateErrorBody(raw, 10); + expect(truncated).toBe(`${'a'.repeat(10)}…(truncated)`); + }); +}); + +describe('fetchRegistryPage', () => { + it('returns a validated list response', async () => { + const body: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('a/b', '1.0.0') }], + metadata: { count: 1 }, + }; + const doFetch = jest.fn().mockResolvedValue({ + ok: true, + status: 200, + json: async () => body, + text: async () => JSON.stringify(body), + } as unknown as Response); + + await expect( + fetchRegistryPage( + doFetch, + new URL('https://registry.example.com/v1/servers'), + ), + ).resolves.toEqual(body); + }); + + it('throws when the servers field is missing', async () => { + const doFetch = jest.fn().mockResolvedValue({ + ok: true, + status: 200, + json: async () => ({ metadata: {} }), + text: async () => '{}', + } as unknown as Response); + + await expect( + fetchRegistryPage( + doFetch, + new URL('https://registry.example.com/v1/servers'), + ), + ).rejects.toThrow(/missing "servers" array/); + }); + + it('truncates non-2xx response bodies in the error', async () => { + const doFetch = jest.fn().mockResolvedValue({ + ok: false, + status: 500, + json: async () => ({}), + text: async () => 'x'.repeat(300), + } as unknown as Response); + + await expect( + fetchRegistryPage( + doFetch, + new URL('https://registry.example.com/v1/servers'), + ), + ).rejects.toThrow(/…\(truncated\)/); + }); +}); + +describe('resolveNextCursor', () => { + it('returns undefined when nextCursor is absent or empty', () => { + const seen = new Set(); + expect(resolveNextCursor(undefined, seen, 1, 10)).toBeUndefined(); + expect(resolveNextCursor(null, seen, 1, 10)).toBeUndefined(); + expect(resolveNextCursor('', seen, 1, 10)).toBeUndefined(); + expect(seen.size).toBe(0); + }); + + it('returns the cursor and records it when paging continues', () => { + const seen = new Set(); + expect(resolveNextCursor('page-2', seen, 1, 10)).toBe('page-2'); + expect(seen.has('page-2')).toBe(true); + }); + + it('throws on a repeated cursor', () => { + const seen = new Set(['page-2']); + expect(() => resolveNextCursor('page-2', seen, 2, 10)).toThrow( + /repeated cursor/, + ); + }); + + it('throws when the page limit is exceeded with more pages remaining', () => { + const seen = new Set(); + expect(() => resolveNextCursor('page-2', seen, 1, 1)).toThrow( + /exceeded the configured page limit/, + ); + }); +}); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index 5cbceedf040..6a452be9079 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -15,6 +15,10 @@ */ import type { McpServerDocument } from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; +import { stripTrailingSlashes } from './util'; + +/** Max characters of an error response body included in client errors. */ +const MAX_ERROR_BODY_LENGTH = 256; /** * A single server entry from the MCP Registry list response. @@ -54,8 +58,7 @@ export function buildServersEndpoint( baseUrl: string, apiVersion: string, ): string { - const normalizedBase = baseUrl.replace(/\/+$/, ''); - return `${normalizedBase}/${apiVersion}/servers`; + return `${stripTrailingSlashes(baseUrl)}/${apiVersion}/servers`; } /** @@ -70,6 +73,142 @@ export interface FetchServersOptions { fetchApi?: typeof fetch; } +/** + * Parse the servers list endpoint into a URL. + * + * @internal + */ +export function parseServersEndpointUrl( + baseUrl: string, + apiVersion: string, +): URL { + const endpoint = buildServersEndpoint(baseUrl, apiVersion); + try { + return new URL(endpoint); + } catch (err) { + throw new McpRegistryClientError( + `Invalid MCP Registry endpoint URL "${endpoint}": ${err}`, + ); + } +} + +/** + * Build a page request URL with optional cursor and page-size params. + * + * @internal + */ +export function buildPageRequestUrl( + endpoint: URL, + cursor?: string, + pageSize?: number, +): URL { + const url = new URL(endpoint.toString()); + if (cursor) { + url.searchParams.set('cursor', cursor); + } + if (pageSize !== undefined) { + url.searchParams.set('limit', String(pageSize)); + } + return url; +} + +/** + * Truncate an error response body for safe inclusion in log/error text. + * + * @internal + */ +export function truncateErrorBody( + rawBody: string, + maxLength = MAX_ERROR_BODY_LENGTH, +): string { + if (rawBody.length <= maxLength) { + return rawBody; + } + return `${rawBody.substring(0, maxLength)}…(truncated)`; +} + +/** + * Fetch and validate one registry list page. + * + * @internal + */ +export async function fetchRegistryPage( + doFetch: typeof fetch, + url: URL, +): Promise { + const requestUrl = url.toString(); + + let response: Response; + try { + response = await doFetch(requestUrl); + } catch (err) { + throw new McpRegistryClientError( + `Failed to reach MCP Registry at ${requestUrl}: ${err}`, + ); + } + + if (!response.ok) { + const rawBody = await response.text().catch(() => '(no body)'); + throw new McpRegistryClientError( + `MCP Registry returned HTTP ${response.status} for ` + + `${requestUrl}: ${truncateErrorBody(rawBody)}`, + ); + } + + let body: McpRegistryListResponse; + try { + body = (await response.json()) as McpRegistryListResponse; + } catch (err) { + throw new McpRegistryClientError( + `MCP Registry returned unparseable JSON from ${requestUrl}: ${err}`, + ); + } + + if (!body.servers || !Array.isArray(body.servers)) { + throw new McpRegistryClientError( + `MCP Registry response missing "servers" array from ${requestUrl}`, + ); + } + + return body; +} + +/** + * Resolve the next pagination cursor, or `undefined` when paging is done. + * Enforces repeated-cursor and page-limit safeguards. + * + * @internal + */ +export function resolveNextCursor( + nextCursor: string | null | undefined, + seenCursors: Set, + pagesFetched: number, + pageLimit: number, +): string | undefined { + if (!nextCursor || nextCursor.length === 0) { + return undefined; + } + + if (seenCursors.has(nextCursor)) { + throw new McpRegistryClientError( + `MCP Registry returned a repeated cursor "${nextCursor}" ` + + `during pagination. Aborting sync to prevent infinite loop.`, + ); + } + seenCursors.add(nextCursor); + + if (pagesFetched >= pageLimit) { + throw new McpRegistryClientError( + `MCP Registry pagination exceeded the configured page limit ` + + `of ${pageLimit} pages per sync. The registry still has more ` + + `pages (nextCursor present). Increase pageLimit to fetch ` + + `more pages.`, + ); + } + + return nextCursor; +} + /** * Fetch all server entries from the MCP Registry using cursor * pagination. Accumulates entries across pages and enforces @@ -83,104 +222,30 @@ export async function fetchRegistryServers( ): Promise { const { baseUrl, apiVersion, pageLimit, pageSize, fetchApi } = options; const doFetch = fetchApi ?? fetch; - - const endpoint = buildServersEndpoint(baseUrl, apiVersion); - - let parsedEndpoint: URL; - try { - parsedEndpoint = new URL(endpoint); - } catch (err) { - throw new McpRegistryClientError( - `Invalid MCP Registry endpoint URL "${endpoint}": ${err}`, - ); - } + const endpoint = parseServersEndpointUrl(baseUrl, apiVersion); const allServers: McpRegistryServerEntry[] = []; const seenCursors = new Set(); let cursor: string | undefined; - let pageCount = 0; - + let pagesFetched = 0; let hasMorePages = true; - while (hasMorePages) { - // Build request URL with query params - const url = new URL(parsedEndpoint.toString()); - if (cursor) { - url.searchParams.set('cursor', cursor); - } - if (pageSize !== undefined) { - url.searchParams.set('limit', String(pageSize)); - } - - let response: Response; - try { - response = await doFetch(url.toString()); - } catch (err) { - throw new McpRegistryClientError( - `Failed to reach MCP Registry at ${url.toString()}: ${err}`, - ); - } - - if (!response.ok) { - const MAX_BODY_LENGTH = 256; - const rawBody = await response.text().catch(() => '(no body)'); - const truncatedBody = - rawBody.length > MAX_BODY_LENGTH - ? `${rawBody.substring(0, MAX_BODY_LENGTH)}…(truncated)` - : rawBody; - throw new McpRegistryClientError( - `MCP Registry returned HTTP ${response.status} for ` + - `${url.toString()}: ${truncatedBody}`, - ); - } - - let body: McpRegistryListResponse; - try { - body = (await response.json()) as McpRegistryListResponse; - } catch (err) { - throw new McpRegistryClientError( - `MCP Registry returned unparseable JSON from ` + - `${url.toString()}: ${err}`, - ); - } - - if (!body.servers || !Array.isArray(body.servers)) { - throw new McpRegistryClientError( - `MCP Registry response missing "servers" array from ` + - `${url.toString()}`, - ); - } + while (hasMorePages) { + const url = buildPageRequestUrl(endpoint, cursor, pageSize); + const body = await fetchRegistryPage(doFetch, url); allServers.push(...body.servers); - pageCount++; + pagesFetched += 1; - // Check for next cursor - const nextCursor = body.metadata?.nextCursor; - if (!nextCursor || nextCursor.length === 0) { - // No more pages + const nextCursor = resolveNextCursor( + body.metadata?.nextCursor, + seenCursors, + pagesFetched, + pageLimit, + ); + if (!nextCursor) { hasMorePages = false; continue; } - - // Repeated cursor safeguard - if (seenCursors.has(nextCursor)) { - throw new McpRegistryClientError( - `MCP Registry returned a repeated cursor "${nextCursor}" ` + - `during pagination. Aborting sync to prevent infinite loop.`, - ); - } - seenCursors.add(nextCursor); - - // Page limit safeguard: if we've fetched pageLimit pages and - // there's still a nextCursor, fail the run - if (pageCount >= pageLimit) { - throw new McpRegistryClientError( - `MCP Registry pagination exceeded the configured page limit ` + - `of ${pageLimit} pages per sync. The registry still has more ` + - `pages (nextCursor present). Increase pageLimit to fetch ` + - `more pages.`, - ); - } - cursor = nextCursor; } diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts index ecbf0711474..cb6b147612a 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts @@ -15,7 +15,15 @@ */ import { ConfigReader } from '@backstage/config'; -import { readMcpRegistryProviderConfig } from './config'; +import { + assertSingleRegistryConfig, + readMcpRegistryProviderConfig, + readOptionalPageSize, + readPageLimit, + readProviderSchedule, + readRequiredHttpBaseUrl, + safeGetOptionalString, +} from './config'; describe('readMcpRegistryProviderConfig', () => { it('returns undefined when catalog.providers is absent', () => { @@ -151,7 +159,7 @@ describe('readMcpRegistryProviderConfig', () => { }); expect(() => readMcpRegistryProviderConfig(config)).toThrow( - /Multiple registries are out of scope/, + /found keyed instance/, ); }); @@ -211,3 +219,146 @@ describe('readMcpRegistryProviderConfig', () => { expect(result!.apiVersion).toBe('v0'); }); }); + +describe('safeGetOptionalString', () => { + it('returns the string value when present', () => { + const config = new ConfigReader({ + baseUrl: 'https://registry.example.com', + }); + expect(safeGetOptionalString(config, 'baseUrl')).toBe( + 'https://registry.example.com', + ); + }); + + it('returns undefined when the key is absent', () => { + const config = new ConfigReader({}); + expect(safeGetOptionalString(config, 'baseUrl')).toBeUndefined(); + }); + + it('returns undefined when ConfigReader rejects an empty string', () => { + const config = new ConfigReader({ baseUrl: '' }); + expect(safeGetOptionalString(config, 'baseUrl')).toBeUndefined(); + }); +}); + +describe('assertSingleRegistryConfig', () => { + it('allows a flat single-registry object', () => { + const config = new ConfigReader({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + }); + expect(() => assertSingleRegistryConfig(config)).not.toThrow(); + }); + + it('ignores unknown scalar keys', () => { + const config = new ConfigReader({ + baseUrl: 'https://registry.example.com', + extraFlag: true, + }); + expect(() => assertSingleRegistryConfig(config)).not.toThrow(); + }); + + it('throws when an unknown key is a nested instance object', () => { + const config = new ConfigReader({ + internal: { + baseUrl: 'https://internal.example.com', + }, + }); + expect(() => assertSingleRegistryConfig(config)).toThrow( + /found keyed instance/, + ); + }); +}); + +describe('readRequiredHttpBaseUrl', () => { + it('returns a valid https baseUrl', () => { + const config = new ConfigReader({ + baseUrl: 'https://registry.example.com', + }); + expect(readRequiredHttpBaseUrl(config)).toBe( + 'https://registry.example.com', + ); + }); + + it('returns a valid http baseUrl', () => { + const config = new ConfigReader({ + baseUrl: 'http://localhost:8080', + }); + expect(readRequiredHttpBaseUrl(config)).toBe('http://localhost:8080'); + }); + + it('throws when baseUrl is missing', () => { + const config = new ConfigReader({}); + expect(() => readRequiredHttpBaseUrl(config)).toThrow( + /missing required "baseUrl"/, + ); + }); + + it('throws when baseUrl is not a valid URL', () => { + const config = new ConfigReader({ baseUrl: 'not a url' }); + expect(() => readRequiredHttpBaseUrl(config)).toThrow(/is not a valid URL/); + }); + + it('throws when baseUrl uses a non-http protocol', () => { + const config = new ConfigReader({ baseUrl: 'ftp://registry.example.com' }); + expect(() => readRequiredHttpBaseUrl(config)).toThrow( + /must use http or https protocol/, + ); + }); +}); + +describe('readPageLimit', () => { + it('defaults to 10 when omitted', () => { + expect(readPageLimit(new ConfigReader({}))).toBe(10); + }); + + it('returns an explicit pageLimit', () => { + expect(readPageLimit(new ConfigReader({ pageLimit: 3 }))).toBe(3); + }); + + it('throws when pageLimit is less than 1', () => { + expect(() => readPageLimit(new ConfigReader({ pageLimit: 0 }))).toThrow( + /"pageLimit" must be at least 1/, + ); + }); +}); + +describe('readOptionalPageSize', () => { + it('returns undefined when omitted', () => { + expect(readOptionalPageSize(new ConfigReader({}))).toBeUndefined(); + }); + + it('returns an explicit pageSize', () => { + expect(readOptionalPageSize(new ConfigReader({ pageSize: 50 }))).toBe(50); + }); + + it('throws when pageSize is less than 1', () => { + expect(() => + readOptionalPageSize(new ConfigReader({ pageSize: 0 })), + ).toThrow(/"pageSize" must be at least 1/); + }); +}); + +describe('readProviderSchedule', () => { + it('returns the default schedule when omitted', () => { + expect(readProviderSchedule(new ConfigReader({}))).toEqual({ + frequency: { minutes: 30 }, + timeout: { minutes: 3 }, + }); + }); + + it('reads an explicit schedule', () => { + const config = new ConfigReader({ + schedule: { + frequency: { minutes: 15 }, + timeout: { minutes: 5 }, + initialDelay: { seconds: 30 }, + }, + }); + expect(readProviderSchedule(config)).toEqual({ + frequency: { minutes: 15 }, + timeout: { minutes: 5 }, + initialDelay: { seconds: 30 }, + }); + }); +}); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts index a02f4e1889b..9c7a1d9b798 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts @@ -30,12 +30,25 @@ const DEFAULT_API_VERSION = 'v1'; /** Default page limit (max pages per sync). */ const DEFAULT_PAGE_LIMIT = 10; +/** Supported single-registry config keys under `catalog.providers.mcpRegistry`. */ +const KNOWN_MCP_REGISTRY_KEYS = new Set([ + 'baseUrl', + 'baseName', + 'apiVersion', + 'defaultOwner', + 'pageLimit', + 'pageSize', + 'schedule', +]); + /** * Safely read an optional string from config, returning `undefined` * when Backstage's ConfigReader throws TypeError for empty-string * values from env var substitution like `${VAR:-}`. + * + * @internal */ -function safeGetOptionalString( +export function safeGetOptionalString( config: Config, key: string, ): string | undefined { @@ -49,80 +62,41 @@ function safeGetOptionalString( } /** - * Parsed provider configuration. + * Reject keyed multi-registry maps under `mcpRegistry`. * - * @public + * @internal */ -export interface McpRegistryProviderConfig { - baseUrl: string; - baseName?: string; - apiVersion: string; - defaultOwner?: string; - pageLimit: number; - pageSize?: number; - schedule: SchedulerServiceTaskScheduleDefinition; +export function assertSingleRegistryConfig(registryConfig: Config): void { + const unknownKeys = registryConfig + .keys() + .filter(key => !KNOWN_MCP_REGISTRY_KEYS.has(key)); + + for (const key of unknownKeys) { + let nested; + try { + nested = registryConfig.getOptionalConfig(key); + } catch { + // ConfigReader throws TypeError when the value is a scalar + // rather than an object — skip this key silently. + continue; + } + if (nested && nested.keys().length > 0) { + throw new Error( + `Invalid catalog.providers.mcpRegistry configuration: found ` + + `keyed instance "${key}". Configure a single registry object ` + + `with baseUrl, baseName, apiVersion, schedule, pageLimit, ` + + `pageSize, and defaultOwner.`, + ); + } + } } /** - * Read and validate the MCP Registry provider configuration from - * `catalog.providers.mcpRegistry`. Returns `undefined` when the - * config key is absent (inert module). + * Read and validate the required HTTP(S) `baseUrl`. * - * @throws When the config is a keyed map of instances, or when - * `baseUrl` is missing. + * @internal */ -export function readMcpRegistryProviderConfig( - rootConfig: Config, -): McpRegistryProviderConfig | undefined { - const providersConfig = rootConfig.getOptionalConfig('catalog.providers'); - if (!providersConfig) { - return undefined; - } - - const registryConfig = providersConfig.getOptionalConfig('mcpRegistry'); - if (!registryConfig) { - return undefined; - } - - // Detect keyed multi-registry maps: if the config has keys that look - // like instance objects (i.e., nested config objects with their own - // baseUrl), reject with an actionable error. - const keys = registryConfig.keys(); - const knownKeys = new Set([ - 'baseUrl', - 'baseName', - 'apiVersion', - 'defaultOwner', - 'pageLimit', - 'pageSize', - 'schedule', - ]); - const unknownKeys = keys.filter(k => !knownKeys.has(k)); - if (unknownKeys.length > 0) { - // Check if the unknown keys look like instance identifiers (they - // would have nested config objects with their own properties) - for (const key of unknownKeys) { - let nested; - try { - nested = registryConfig.getOptionalConfig(key); - } catch { - // ConfigReader throws TypeError when the value is a scalar - // rather than an object — skip this key silently. - continue; - } - if (nested && nested.keys().length > 0) { - throw new Error( - `Invalid catalog.providers.mcpRegistry configuration: found ` + - `keyed instance "${key}". Multiple registries are out of scope ` + - `for this implementation. Configure a single registry object ` + - `with baseUrl, baseName, apiVersion, schedule, pageLimit, ` + - `pageSize, and defaultOwner.`, - ); - } - } - } - - // baseUrl is required +export function readRequiredHttpBaseUrl(registryConfig: Config): string { const baseUrl = safeGetOptionalString(registryConfig, 'baseUrl'); if (!baseUrl) { throw new Error( @@ -132,7 +106,6 @@ export function readMcpRegistryProviderConfig( ); } - // Validate URL scheme (defense in depth against non-HTTP protocols) let parsedUrl: URL; try { parsedUrl = new URL(baseUrl); @@ -143,6 +116,7 @@ export function readMcpRegistryProviderConfig( `HTTP(S) URL (e.g., "https://registry.example.com").`, ); } + if (parsedUrl.protocol !== 'http:' && parsedUrl.protocol !== 'https:') { throw new Error( `Invalid catalog.providers.mcpRegistry configuration: "baseUrl" ` + @@ -151,10 +125,15 @@ export function readMcpRegistryProviderConfig( ); } - const baseName = safeGetOptionalString(registryConfig, 'baseName'); - const apiVersion = - safeGetOptionalString(registryConfig, 'apiVersion') ?? DEFAULT_API_VERSION; - const defaultOwner = safeGetOptionalString(registryConfig, 'defaultOwner'); + return baseUrl; +} + +/** + * Read `pageLimit`, applying the default and rejecting values below 1. + * + * @internal + */ +export function readPageLimit(registryConfig: Config): number { const pageLimit = registryConfig.getOptionalNumber('pageLimit') ?? DEFAULT_PAGE_LIMIT; if (pageLimit < 1) { @@ -163,6 +142,17 @@ export function readMcpRegistryProviderConfig( `must be at least 1, got ${pageLimit}.`, ); } + return pageLimit; +} + +/** + * Read optional `pageSize`, rejecting values below 1 when set. + * + * @internal + */ +export function readOptionalPageSize( + registryConfig: Config, +): number | undefined { const pageSize = registryConfig.getOptionalNumber('pageSize'); if (pageSize !== undefined && pageSize < 1) { throw new Error( @@ -170,24 +160,71 @@ export function readMcpRegistryProviderConfig( `must be at least 1, got ${pageSize}.`, ); } + return pageSize; +} - // Schedule: read from config or use default - let schedule: SchedulerServiceTaskScheduleDefinition; +/** + * Read the provider schedule, or the documented default when omitted. + * + * @internal + */ +export function readProviderSchedule( + registryConfig: Config, +): SchedulerServiceTaskScheduleDefinition { const scheduleConfig = registryConfig.getOptionalConfig('schedule'); - if (scheduleConfig) { - schedule = - readSchedulerServiceTaskScheduleDefinitionFromConfig(scheduleConfig); - } else { - schedule = DEFAULT_SCHEDULE; + if (!scheduleConfig) { + return DEFAULT_SCHEDULE; } + return readSchedulerServiceTaskScheduleDefinitionFromConfig(scheduleConfig); +} + +/** + * Parsed provider configuration. + * + * @public + */ +export interface McpRegistryProviderConfig { + baseUrl: string; + baseName?: string; + apiVersion: string; + defaultOwner?: string; + pageLimit: number; + pageSize?: number; + schedule: SchedulerServiceTaskScheduleDefinition; +} + +/** + * Read and validate the MCP Registry provider configuration from + * `catalog.providers.mcpRegistry`. Returns `undefined` when the + * config key is absent (inert module). + * + * @throws When the config is a keyed map of instances, or when + * `baseUrl` is missing. + */ +export function readMcpRegistryProviderConfig( + rootConfig: Config, +): McpRegistryProviderConfig | undefined { + const providersConfig = rootConfig.getOptionalConfig('catalog.providers'); + if (!providersConfig) { + return undefined; + } + + const registryConfig = providersConfig.getOptionalConfig('mcpRegistry'); + if (!registryConfig) { + return undefined; + } + + assertSingleRegistryConfig(registryConfig); return { - baseUrl, - baseName, - apiVersion, - defaultOwner, - pageLimit, - pageSize, - schedule, + baseUrl: readRequiredHttpBaseUrl(registryConfig), + baseName: safeGetOptionalString(registryConfig, 'baseName'), + apiVersion: + safeGetOptionalString(registryConfig, 'apiVersion') ?? + DEFAULT_API_VERSION, + defaultOwner: safeGetOptionalString(registryConfig, 'defaultOwner'), + pageLimit: readPageLimit(registryConfig), + pageSize: readOptionalPageSize(registryConfig), + schedule: readProviderSchedule(registryConfig), }; } diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.parts.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.parts.test.ts new file mode 100644 index 00000000000..ab5753ba477 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.parts.test.ts @@ -0,0 +1,587 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import type { Entity } from '@backstage/catalog-model'; +import type { DeferredEntity } from '@backstage/plugin-catalog-node'; +import type { McpServerMappingDefaults } from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; +import type { McpRegistryProviderConfig } from './config'; +import type { McpRegistryListResponse, McpRegistryServerEntry } from './client'; +import { + buildLastGoodKey, + formatMappingFailureMessage, + McpRegistryEntityProvider, + readServerIdentity, +} from './provider'; +import { createMockServerDoc } from './testUtils'; + +const SYNC_STATUS_ANNOTATION = 'redhat.com/rhdh-mcp-registry-sync-status'; +const LOCATION = 'url:https://registry.example.com'; + +type ProviderParts = { + fetchRegistryEntries(): Promise; + mapRegistryEntries( + entries: McpRegistryServerEntry[], + managedByLocation: string, + ): { entities: DeferredEntity[]; hasDegradedEntries: boolean }; + mapRegistryEntry( + entry: McpRegistryServerEntry, + managedByLocation: string, + ): DeferredEntity; + buildMappingDefaults(): McpServerMappingDefaults; + applyProviderAnnotations( + entity: Entity, + managedByLocation: string, + syncStatus: 'ok' | 'degraded', + ): void; + retainLastGoodOnMappingFailure( + entry: McpRegistryServerEntry, + err: unknown, + managedByLocation: string, + ): DeferredEntity | undefined; + rebuildLastGoodIndex(entities: DeferredEntity[]): void; + lastGoodIndex: Map; +}; + +function parts(provider: McpRegistryEntityProvider): ProviderParts { + return provider as unknown as ProviderParts; +} + +function createMockLogger() { + return { + info: jest.fn(), + warn: jest.fn(), + error: jest.fn(), + debug: jest.fn(), + child: jest.fn().mockReturnThis(), + }; +} + +function createDefaultConfig( + overrides?: Partial, +): McpRegistryProviderConfig { + return { + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + schedule: { + frequency: { minutes: 30 }, + timeout: { minutes: 3 }, + }, + ...overrides, + }; +} + +function mockFetchForResponses( + responses: McpRegistryListResponse[], +): jest.Mock { + const fn = jest.fn(); + for (const body of responses) { + fn.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => body, + text: async () => JSON.stringify(body), + } as unknown as Response); + } + return fn; +} + +function makeDeferred( + name: string, + version: string, + syncStatus: 'ok' | 'degraded' = 'ok', +): DeferredEntity { + return { + locationKey: 'mcp-registry-provider', + entity: { + apiVersion: 'backstage.io/v1alpha1', + kind: 'API', + metadata: { + name: `${name}-${version}`, + annotations: { + 'modelcontextprotocol.io/name': name, + 'modelcontextprotocol.io/version': version, + [SYNC_STATUS_ANNOTATION]: syncStatus, + }, + }, + spec: { + type: 'mcp-server', + lifecycle: 'experimental', + owner: 'unknown', + remotes: [{ type: 'streamable-http', url: 'https://example.com/mcp' }], + }, + }, + }; +} + +describe('buildLastGoodKey', () => { + it('joins name and version with a double-colon separator', () => { + expect(buildLastGoodKey('io.example/weather', '1.0.0')).toBe( + 'io.example/weather::1.0.0', + ); + }); +}); + +describe('readServerIdentity', () => { + it('returns name and version from a valid entry', () => { + expect( + readServerIdentity({ + server: createMockServerDoc('io.example/weather', '1.2.3'), + }), + ).toEqual({ name: 'io.example/weather', version: '1.2.3' }); + }); + + it('returns undefined fields for null or undefined entries', () => { + expect(readServerIdentity(null)).toEqual({ + name: undefined, + version: undefined, + }); + expect(readServerIdentity(undefined)).toEqual({ + name: undefined, + version: undefined, + }); + }); + + it('ignores non-string name and version values', () => { + expect( + readServerIdentity({ + server: { + ...createMockServerDoc('io.example/weather', '1.0.0'), + name: 42 as unknown as string, + version: { n: 1 } as unknown as string, + }, + }), + ).toEqual({ name: undefined, version: undefined }); + }); +}); + +describe('formatMappingFailureMessage', () => { + it('includes name and version when both are present', () => { + expect( + formatMappingFailureMessage('io.example/weather', '1.0.0', 'boom'), + ).toBe( + 'Failed to map MCP Registry server entry "io.example/weather" version "1.0.0": boom', + ); + }); + + it('omits missing name and version segments', () => { + expect(formatMappingFailureMessage(undefined, undefined, 'boom')).toBe( + 'Failed to map MCP Registry server entry: boom', + ); + }); + + it('includes only the version when name is missing', () => { + expect(formatMappingFailureMessage(undefined, '1.0.0', 'boom')).toBe( + 'Failed to map MCP Registry server entry version "1.0.0": boom', + ); + }); +}); + +describe('McpRegistryEntityProvider parts', () => { + describe('fetchRegistryEntries', () => { + it('returns the registry server list on success', async () => { + const body: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('a/b', '1.0.0') }], + metadata: { count: 1 }, + }; + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + mockFetchForResponses([body]), + ); + + await expect(parts(provider).fetchRegistryEntries()).resolves.toEqual( + body.servers, + ); + }); + + it('logs and returns undefined for McpRegistryClientError', async () => { + const logger = createMockLogger(); + const fetchFn = jest.fn().mockResolvedValue({ + ok: false, + status: 503, + text: async () => 'unavailable', + } as unknown as Response); + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + logger, + fetchFn, + ); + + await expect( + parts(provider).fetchRegistryEntries(), + ).resolves.toBeUndefined(); + expect(logger.error).toHaveBeenCalledWith( + expect.stringContaining( + 'MCP Registry sync failed (no mutation emitted)', + ), + ); + }); + + it('wraps transport failures as client errors and returns undefined', async () => { + const logger = createMockLogger(); + const fetchFn = jest + .fn() + .mockRejectedValue(new TypeError('network down')); + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + logger, + fetchFn, + ); + + await expect( + parts(provider).fetchRegistryEntries(), + ).resolves.toBeUndefined(); + expect(logger.error).toHaveBeenCalledWith( + expect.stringContaining( + 'MCP Registry sync failed (no mutation emitted)', + ), + ); + }); + }); + + describe('buildMappingDefaults', () => { + it('returns an empty object when overrides are omitted', () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + ); + expect(parts(provider).buildMappingDefaults()).toEqual({}); + }); + + it('includes owner and prefix when configured', () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ + defaultOwner: 'group:default/mcp-admins', + baseName: 'com.example.registry', + }), + createMockLogger(), + ); + expect(parts(provider).buildMappingDefaults()).toEqual({ + owner: 'group:default/mcp-admins', + prefix: 'com.example.registry', + }); + }); + }); + + describe('applyProviderAnnotations', () => { + it('sets location, origin, and sync-status annotations', () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + ); + const entity: Entity = { + apiVersion: 'backstage.io/v1alpha1', + kind: 'API', + metadata: { name: 'weather' }, + spec: { type: 'mcp-server' }, + }; + + parts(provider).applyProviderAnnotations(entity, LOCATION, 'ok'); + + expect(entity.metadata.annotations).toEqual({ + 'backstage.io/managed-by-location': LOCATION, + 'backstage.io/managed-by-origin-location': LOCATION, + [SYNC_STATUS_ANNOTATION]: 'ok', + }); + }); + + it('creates the annotations object when missing', () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + ); + const entity: Entity = { + apiVersion: 'backstage.io/v1alpha1', + kind: 'API', + metadata: { name: 'weather' }, + spec: { type: 'mcp-server' }, + }; + + parts(provider).applyProviderAnnotations(entity, LOCATION, 'degraded'); + + expect(entity.metadata.annotations?.[SYNC_STATUS_ANNOTATION]).toBe( + 'degraded', + ); + }); + }); + + describe('mapRegistryEntry', () => { + it('maps a server entry to a deferred entity with ok sync status', () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ defaultOwner: 'user:default/guest' }), + createMockLogger(), + ); + const deferred = parts(provider).mapRegistryEntry( + { server: createMockServerDoc('io.example/weather', '1.0.0') }, + LOCATION, + ); + + expect(deferred.locationKey).toBe('mcp-registry-provider'); + expect(deferred.entity.spec?.owner).toBe('user:default/guest'); + expect(deferred.entity.metadata.annotations).toEqual( + expect.objectContaining({ + 'backstage.io/managed-by-location': LOCATION, + 'backstage.io/managed-by-origin-location': LOCATION, + [SYNC_STATUS_ANNOTATION]: 'ok', + 'modelcontextprotocol.io/name': 'io.example/weather', + 'modelcontextprotocol.io/version': '1.0.0', + }), + ); + }); + + it('throws when the server document cannot be mapped', () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + ); + + expect(() => + parts(provider).mapRegistryEntry( + { + server: { + $schema: + 'https://raw.githubusercontent.com/modelcontextprotocol/registry/v1.8.1/docs/reference/server-json/draft/server.schema.json', + name: 'bad/server', + description: '', + version: '1.0.0', + } as any, + }, + LOCATION, + ), + ).toThrow(); + }); + }); + + describe('mapRegistryEntries', () => { + it('maps successful entries and retains last-good on failure', () => { + const logger = createMockLogger(); + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + logger, + ); + const good = makeDeferred('ok/server', '1.0.0'); + parts(provider).lastGoodIndex.set( + buildLastGoodKey('fail/server', '1.0.0'), + good, + ); + + const result = parts(provider).mapRegistryEntries( + [ + { server: createMockServerDoc('ok/server', '1.0.0') }, + { + server: { + $schema: + 'https://raw.githubusercontent.com/modelcontextprotocol/registry/v1.8.1/docs/reference/server-json/draft/server.schema.json', + name: 'fail/server', + description: '', + version: '1.0.0', + } as any, + }, + ], + LOCATION, + ); + + expect(result.hasDegradedEntries).toBe(true); + expect(result.entities).toHaveLength(2); + expect( + result.entities[0].entity.metadata.annotations?.[ + SYNC_STATUS_ANNOTATION + ], + ).toBe('ok'); + expect( + result.entities[1].entity.metadata.annotations?.[ + SYNC_STATUS_ANNOTATION + ], + ).toBe('degraded'); + expect(logger.warn).toHaveBeenCalled(); + }); + + it('omits failed entries when no last-good exists', () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + ); + + const result = parts(provider).mapRegistryEntries( + [ + { + server: { + $schema: + 'https://raw.githubusercontent.com/modelcontextprotocol/registry/v1.8.1/docs/reference/server-json/draft/server.schema.json', + name: 'fail/server', + description: '', + version: '1.0.0', + } as any, + }, + ], + LOCATION, + ); + + expect(result).toEqual({ entities: [], hasDegradedEntries: false }); + }); + }); + + describe('retainLastGoodOnMappingFailure', () => { + it('returns undefined when name or version is missing', () => { + const logger = createMockLogger(); + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + logger, + ); + + const retained = parts(provider).retainLastGoodOnMappingFailure( + { + server: { + ...createMockServerDoc('io.example/weather', '1.0.0'), + version: undefined as unknown as string, + }, + }, + new Error('map failed'), + LOCATION, + ); + + expect(retained).toBeUndefined(); + expect(logger.warn).toHaveBeenCalledWith( + expect.stringContaining('Failed to map MCP Registry server entry'), + ); + }); + + it('returns undefined and logs when no last-good entity exists', () => { + const logger = createMockLogger(); + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + logger, + ); + + const retained = parts(provider).retainLastGoodOnMappingFailure( + { server: createMockServerDoc('missing/server', '1.0.0') }, + new Error('map failed'), + LOCATION, + ); + + expect(retained).toBeUndefined(); + expect(logger.info).toHaveBeenCalledWith( + expect.stringContaining('No last-good entity found'), + ); + }); + + it('returns a degraded clone of the last-good entity', () => { + const logger = createMockLogger(); + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + logger, + ); + const lastGood = makeDeferred('kept/server', '2.0.0'); + parts(provider).lastGoodIndex.set( + buildLastGoodKey('kept/server', '2.0.0'), + lastGood, + ); + + const retained = parts(provider).retainLastGoodOnMappingFailure( + { server: createMockServerDoc('kept/server', '2.0.0') }, + new Error('map failed'), + LOCATION, + ); + + expect(retained).toBeDefined(); + expect(retained!.entity).not.toBe(lastGood.entity); + expect(retained!.entity.metadata.annotations).toEqual( + expect.objectContaining({ + 'backstage.io/managed-by-location': LOCATION, + 'backstage.io/managed-by-origin-location': LOCATION, + [SYNC_STATUS_ANNOTATION]: 'degraded', + }), + ); + expect(logger.info).toHaveBeenCalledWith( + expect.stringContaining('Retained last-good entity'), + ); + }); + }); + + describe('rebuildLastGoodIndex', () => { + it('indexes ok entities and skips degraded ones', () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + ); + const ok = makeDeferred('ok/server', '1.0.0', 'ok'); + const degraded = makeDeferred('bad/server', '1.0.0', 'degraded'); + + parts(provider).rebuildLastGoodIndex([ok, degraded]); + + expect(parts(provider).lastGoodIndex.size).toBe(1); + expect( + parts(provider).lastGoodIndex.get( + buildLastGoodKey('ok/server', '1.0.0'), + ), + ).toBe(ok); + expect( + parts(provider).lastGoodIndex.has( + buildLastGoodKey('bad/server', '1.0.0'), + ), + ).toBe(false); + }); + + it('skips entities missing name or version annotations', () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + ); + const incomplete: DeferredEntity = { + locationKey: 'mcp-registry-provider', + entity: { + apiVersion: 'backstage.io/v1alpha1', + kind: 'API', + metadata: { + name: 'incomplete', + annotations: { + [SYNC_STATUS_ANNOTATION]: 'ok', + 'modelcontextprotocol.io/name': 'only/name', + }, + }, + spec: { type: 'mcp-server' }, + }, + }; + + parts(provider).rebuildLastGoodIndex([incomplete]); + + expect(parts(provider).lastGoodIndex.size).toBe(0); + }); + + it('clears prior index entries before rebuilding', () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + ); + parts(provider).lastGoodIndex.set( + buildLastGoodKey('old/server', '0.1.0'), + makeDeferred('old/server', '0.1.0'), + ); + + parts(provider).rebuildLastGoodIndex([ + makeDeferred('new/server', '2.0.0'), + ]); + + expect(parts(provider).lastGoodIndex.size).toBe(1); + expect( + parts(provider).lastGoodIndex.has( + buildLastGoodKey('old/server', '0.1.0'), + ), + ).toBe(false); + }); + }); +}); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts index b1db0ca31b2..433f54e86d6 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts @@ -21,6 +21,7 @@ import type { import { ANNOTATION_LOCATION, ANNOTATION_ORIGIN_LOCATION, + type Entity, } from '@backstage/catalog-model'; import type { DeferredEntity, @@ -35,6 +36,7 @@ import type { McpServerMappingDefaults } from '@red-hat-developer-hub/backstage- import type { McpRegistryProviderConfig } from './config'; import { fetchRegistryServers, McpRegistryClientError } from './client'; import type { McpRegistryServerEntry } from './client'; +import { stripTrailingSlashes } from './util'; /** Provider name and locationKey constant. */ const PROVIDER_NAME = 'mcp-registry-provider'; @@ -44,17 +46,50 @@ const SYNC_STATUS_ANNOTATION = 'redhat.com/rhdh-mcp-registry-sync-status'; /** * Build a last-good lookup key from name and version. + * + * @internal */ -function buildLastGoodKey(name: string, version: string): string { +export function buildLastGoodKey(name: string, version: string): string { return `${name}::${version}`; } /** - * Normalize baseUrl by stripping trailing slashes for use in - * backstage.io/managed-by-location. + * Read optional name/version from a registry list entry. + * + * @internal */ -function normalizeBaseUrl(baseUrl: string): string { - return baseUrl.replace(/\/+$/, ''); +export function readServerIdentity( + entry: McpRegistryServerEntry | null | undefined, +): { + name?: string; + version?: string; +} { + const serverDoc = entry?.server; + return { + name: typeof serverDoc?.name === 'string' ? serverDoc.name : undefined, + version: + typeof serverDoc?.version === 'string' ? serverDoc.version : undefined, + }; +} + +/** + * Format the per-entry mapping failure warning. + * + * @internal + */ +export function formatMappingFailureMessage( + name: string | undefined, + version: string | undefined, + err: unknown, +): string { + let message = 'Failed to map MCP Registry server entry'; + if (name) { + message += ` "${name}"`; + } + if (version) { + message += ` version "${version}"`; + } + return `${message}: ${err}`; } /** @@ -75,7 +110,7 @@ export class McpRegistryEntityProvider implements EntityProvider { * last successfully committed DeferredEntity so that a subsequent * sync can retain it when mapping fails (D6). */ - private lastGoodIndex = new Map(); + private readonly lastGoodIndex = new Map(); constructor( config: McpRegistryProviderConfig, @@ -118,15 +153,49 @@ export class McpRegistryEntityProvider implements EntityProvider { ); } - const { baseUrl, apiVersion, pageLimit, pageSize, baseName, defaultOwner } = - this.config; - const normalizedBaseUrl = normalizeBaseUrl(baseUrl); - const managedByLocation = `url:${normalizedBaseUrl}`; + const managedByLocation = `url:${stripTrailingSlashes( + this.config.baseUrl, + )}`; + const entries = await this.fetchRegistryEntries(); + if (!entries) { + return; + } + + const { entities, hasDegradedEntries } = this.mapRegistryEntries( + entries, + managedByLocation, + ); + + if (hasDegradedEntries) { + this.logger.warn( + `MCP Registry sync completed with degraded entries. ` + + `Some server entries could not be mapped and are using ` + + `last-good entities.`, + ); + } + + await this.connection.applyMutation({ + type: 'full', + entities, + }); + + this.rebuildLastGoodIndex(entities); - // Fetch all servers from the registry - let entries: McpRegistryServerEntry[]; + this.logger.info( + `MCP Registry sync completed: ${entities.length} entities committed.`, + ); + } + + /** + * Fetch registry servers. Returns `undefined` when a client error + * aborts the sync without emitting a mutation. + */ + private async fetchRegistryEntries(): Promise< + McpRegistryServerEntry[] | undefined + > { + const { baseUrl, apiVersion, pageLimit, pageSize } = this.config; try { - entries = await fetchRegistryServers({ + return await fetchRegistryServers({ baseUrl, apiVersion, pageLimit, @@ -138,158 +207,175 @@ export class McpRegistryEntityProvider implements EntityProvider { this.logger.error( `MCP Registry sync failed (no mutation emitted): ${err.message}`, ); - return; + return undefined; } throw err; } + } - // Map each entry, with per-entry failure isolation + /** + * Map every registry entry with per-entry failure isolation. + */ + private mapRegistryEntries( + entries: McpRegistryServerEntry[], + managedByLocation: string, + ): { entities: DeferredEntity[]; hasDegradedEntries: boolean } { const entities: DeferredEntity[] = []; let hasDegradedEntries = false; for (const entry of entries) { try { - const serverDoc = entry.server; - // Invoke the mapping transform - const mappingDefaults: McpServerMappingDefaults = {}; - if (defaultOwner) { - mappingDefaults.owner = defaultOwner; - } - if (baseName) { - mappingDefaults.prefix = baseName; + entities.push(this.mapRegistryEntry(entry, managedByLocation)); + } catch (err) { + const retained = this.retainLastGoodOnMappingFailure( + entry, + err, + managedByLocation, + ); + if (retained) { + entities.push(retained); + hasDegradedEntries = true; } + } + } - const mappingResult = mapServerToEntity(serverDoc, mappingDefaults); - const entity = mappingResult.entity; + return { entities, hasDegradedEntries }; + } - // Apply annotation projection - const projectedAnnotations = projectAnnotations( - serverDoc, - mappingResult.consumedPaths, - mappingResult.reservedAnnotationKeys, - ); + /** + * Map one registry entry into a deferred entity with sync status `ok`. + */ + private mapRegistryEntry( + entry: McpRegistryServerEntry, + managedByLocation: string, + ): DeferredEntity { + const serverDoc = entry.server; + const mappingResult = mapServerToEntity( + serverDoc, + this.buildMappingDefaults(), + ); + const entity = mappingResult.entity; - // Merge projected annotations with the entity's existing ones - entity.metadata.annotations = { - ...entity.metadata.annotations, - ...projectedAnnotations, - }; - - // Catalog processing requires both location annotations. Without - // the origin annotation the entity is rejected and never listed. - entity.metadata.annotations[ANNOTATION_LOCATION] = managedByLocation; - entity.metadata.annotations[ANNOTATION_ORIGIN_LOCATION] = - managedByLocation; - entity.metadata.annotations[SYNC_STATUS_ANNOTATION] = 'ok'; - - const deferred: DeferredEntity = { - entity, - locationKey: PROVIDER_NAME, - }; - entities.push(deferred); - } catch (err) { - // Per-entry failure: log and attempt last-good retention - const serverDoc = - entry !== null && entry !== undefined - ? (entry as McpRegistryServerEntry).server - : undefined; - const serverName = - typeof serverDoc?.name === 'string' ? serverDoc.name : undefined; - const serverVersion = - typeof serverDoc?.version === 'string' - ? serverDoc.version - : undefined; - - this.logger.warn( - `Failed to map MCP Registry server entry` + - `${serverName ? ` "${serverName}"` : ''}` + - `${serverVersion ? ` version "${serverVersion}"` : ''}: ${err}`, - ); + entity.metadata.annotations = { + ...entity.metadata.annotations, + ...projectAnnotations( + serverDoc, + mappingResult.consumedPaths, + mappingResult.reservedAnnotationKeys, + ), + }; - // Last-good retention (D6): retain prior entity if name and - // version are present and a last-good entity exists - if (serverName && serverVersion) { - const lastGoodKey = buildLastGoodKey(serverName, serverVersion); - const lastGood = this.lastGoodIndex.get(lastGoodKey); - if (lastGood) { - // Use the last-good entity with degraded status - const retainedEntity = JSON.parse(JSON.stringify(lastGood.entity)); - if (!retainedEntity.metadata.annotations) { - retainedEntity.metadata.annotations = {}; - } - retainedEntity.metadata.annotations[SYNC_STATUS_ANNOTATION] = - 'degraded'; - retainedEntity.metadata.annotations[ANNOTATION_LOCATION] = - managedByLocation; - retainedEntity.metadata.annotations[ANNOTATION_ORIGIN_LOCATION] = - managedByLocation; - - entities.push({ - entity: retainedEntity, - locationKey: PROVIDER_NAME, - }); - hasDegradedEntries = true; - this.logger.info( - `Retained last-good entity for "${serverName}" ` + - `version "${serverVersion}" with degraded sync status.`, - ); - } else { - this.logger.info( - `No last-good entity found for "${serverName}" ` + - `version "${serverVersion}"; omitting from mutation.`, - ); - } - } - } + this.applyProviderAnnotations(entity, managedByLocation, 'ok'); + + return { + entity, + locationKey: PROVIDER_NAME, + }; + } + + /** + * Build mapping caller overrides from provider config. + */ + private buildMappingDefaults(): McpServerMappingDefaults { + const mappingDefaults: McpServerMappingDefaults = {}; + if (this.config.defaultOwner) { + mappingDefaults.owner = this.config.defaultOwner; } + if (this.config.baseName) { + mappingDefaults.prefix = this.config.baseName; + } + return mappingDefaults; + } - if (hasDegradedEntries) { - this.logger.warn( - `MCP Registry sync completed with degraded entries. ` + - `Some server entries could not be mapped and are using ` + - `last-good entities.`, + /** + * Stamp provider-owned location and sync-status annotations. + * + * Catalog processing requires both location annotations. Without the + * origin annotation the entity is rejected and never listed. + */ + private applyProviderAnnotations( + entity: Entity, + managedByLocation: string, + syncStatus: 'ok' | 'degraded', + ): void { + if (!entity.metadata.annotations) { + entity.metadata.annotations = {}; + } + entity.metadata.annotations[ANNOTATION_LOCATION] = managedByLocation; + entity.metadata.annotations[ANNOTATION_ORIGIN_LOCATION] = managedByLocation; + entity.metadata.annotations[SYNC_STATUS_ANNOTATION] = syncStatus; + } + + /** + * Log a mapping failure and retain a last-good entity when available (D6). + */ + private retainLastGoodOnMappingFailure( + entry: McpRegistryServerEntry, + err: unknown, + managedByLocation: string, + ): DeferredEntity | undefined { + const { name, version } = readServerIdentity(entry); + this.logger.warn(formatMappingFailureMessage(name, version, err)); + + if (!name || !version) { + return undefined; + } + + const lastGood = this.lastGoodIndex.get(buildLastGoodKey(name, version)); + if (!lastGood) { + this.logger.info( + `No last-good entity found for "${name}" ` + + `version "${version}"; omitting from mutation.`, ); + return undefined; } - // Commit full mutation - await this.connection.applyMutation({ - type: 'full', - entities, - }); + const retainedEntity = structuredClone(lastGood.entity); + this.applyProviderAnnotations( + retainedEntity, + managedByLocation, + 'degraded', + ); - // Update the last-good index with successfully mapped entities only. - // Entities that carry sync-status "degraded" are excluded: they are - // last-good fallbacks from a prior cycle, so storing them back would - // create perpetual retention of stale data. Only "ok" entities - // qualify as last-good candidates. - // - // The annotation keys used here ('modelcontextprotocol.io/name' and - // 'modelcontextprotocol.io/version') are set by mapServerToEntity in - // mcp-registry-server-mapping-common and correspond to the raw - // serverDoc.name and serverDoc.version fields used in buildLastGoodKey - // during failure recovery above. If the mapping library changes these - // annotation keys, both this rebuild and the failure recovery path - // must be updated in tandem. + this.logger.info( + `Retained last-good entity for "${name}" ` + + `version "${version}" with degraded sync status.`, + ); + + return { + entity: retainedEntity, + locationKey: PROVIDER_NAME, + }; + } + + /** + * Rebuild the last-good index from successfully mapped entities only. + * + * Entities that carry sync-status "degraded" are excluded: they are + * last-good fallbacks from a prior cycle, so storing them back would + * create perpetual retention of stale data. Only "ok" entities + * qualify as last-good candidates. + * + * The annotation keys used here ('modelcontextprotocol.io/name' and + * 'modelcontextprotocol.io/version') are set by mapServerToEntity in + * mcp-registry-server-mapping-common and correspond to the raw + * serverDoc.name and serverDoc.version fields used in buildLastGoodKey + * during failure recovery. If the mapping library changes these + * annotation keys, both this rebuild and the failure recovery path + * must be updated in tandem. + */ + private rebuildLastGoodIndex(entities: DeferredEntity[]): void { this.lastGoodIndex.clear(); for (const deferred of entities) { - const syncStatus = - deferred.entity.metadata?.annotations?.[SYNC_STATUS_ANNOTATION]; - if (syncStatus === 'degraded') { + const annotations = deferred.entity.metadata?.annotations; + if (annotations?.[SYNC_STATUS_ANNOTATION] === 'degraded') { continue; } - const name = - deferred.entity.metadata?.annotations?.['modelcontextprotocol.io/name']; - const version = - deferred.entity.metadata?.annotations?.[ - 'modelcontextprotocol.io/version' - ]; + const name = annotations?.['modelcontextprotocol.io/name']; + const version = annotations?.['modelcontextprotocol.io/version']; if (name && version) { this.lastGoodIndex.set(buildLastGoodKey(name, version), deferred); } } - - this.logger.info( - `MCP Registry sync completed: ${entities.length} entities committed.`, - ); } } diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.test.ts new file mode 100644 index 00000000000..99384b37de2 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.test.ts @@ -0,0 +1,38 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { stripTrailingSlashes } from './util'; + +describe('stripTrailingSlashes', () => { + it('returns the value unchanged when there is no trailing slash', () => { + expect(stripTrailingSlashes('https://registry.example.com')).toBe( + 'https://registry.example.com', + ); + }); + + it('strips one or more trailing slashes', () => { + expect(stripTrailingSlashes('https://registry.example.com/')).toBe( + 'https://registry.example.com', + ); + expect(stripTrailingSlashes('https://registry.example.com///')).toBe( + 'https://registry.example.com', + ); + }); + + it('returns an empty string when the value is only slashes', () => { + expect(stripTrailingSlashes('///')).toBe(''); + }); +}); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.ts new file mode 100644 index 00000000000..686b1ddd07c --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.ts @@ -0,0 +1,28 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Strip trailing `/` characters with a linear scan (no regex backtracking). + * + * @internal + */ +export function stripTrailingSlashes(value: string): string { + let end = value.length; + while (end > 0 && value.charAt(end - 1) === '/') { + end -= 1; + } + return end === value.length ? value : value.slice(0, end); +} From fe9bd1fef0a428a7d796f92a4d9bcafc181c458c Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Fri, 18 Sep 2026 22:29:52 -0400 Subject: [PATCH 11/63] docs(#4815): add mapping-common changeset for provider link Record a patch release note for documenting consumption by the MCP Registry provider and listing it in pluginPackages. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../.changeset/mcp-registry-mapping-common-provider-link.md | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 workspaces/ai-integrations/.changeset/mcp-registry-mapping-common-provider-link.md diff --git a/workspaces/ai-integrations/.changeset/mcp-registry-mapping-common-provider-link.md b/workspaces/ai-integrations/.changeset/mcp-registry-mapping-common-provider-link.md new file mode 100644 index 00000000000..f12e70fafb1 --- /dev/null +++ b/workspaces/ai-integrations/.changeset/mcp-registry-mapping-common-provider-link.md @@ -0,0 +1,5 @@ +--- +'@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common': patch +--- + +Document consumption by `catalog-backend-module-mcp-registry-provider` and list that package in `pluginPackages`. From d212f11e82d5fd1a52df60287efe4cd94778bef7 Mon Sep 17 00:00:00 2001 From: fullsend-fix <278716306+fullsend-ai-coder[bot]@users.noreply.github.com> Date: Sat, 19 Sep 2026 03:32:23 +0000 Subject: [PATCH 12/63] fix(#4815): address review feedback on PR #4871 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Move fetchApi test seam out of public constructor into an internal options bag; run() marked @internal — both removed from public API surface (report.api.md) - Add TSDoc to all McpRegistryProviderConfig fields - Change rebuildLastGoodIndex filter from === 'degraded' to !== 'ok' for forward-compatible spec alignment - Add configurable maxEntries cap (default 5000) to abort sync when accumulated entries exceed the threshold, guarding against oversized registry pages - Rename provider.ts to McpRegistryEntityProvider.ts to match workspace naming conventions (class-based filenames) - Standardize package.json scripts: lint:check/lint:fix to lint to match sibling catalog-backend-module-* plugins - Mark all tasks in tasks.md as completed - Update audit.md timestamp to 2026-09-19 - Update stale future ingestion references in mcp-registry-server-mapping design.md to reference the now-implemented provider - Add maxEntries config field to config.d.ts with @visibility backend - Add tests for maxEntries config parsing and client enforcement - Regenerate report.api.md Addresses #4871 Assisted-by: Claude Opus 4.6 --- .../changes/mcp-registry-provider/audit.md | 2 +- .../changes/mcp-registry-provider/tasks.md | 52 +++++++++---------- .../mcp-registry-server-mapping/design.md | 10 ++-- .../config.d.ts | 2 + .../package.json | 8 +-- .../report.api.md | 13 +---- ...> McpRegistryEntityProvider.parts.test.ts} | 9 ++-- ...t.ts => McpRegistryEntityProvider.test.ts} | 38 +++++++------- ...ovider.ts => McpRegistryEntityProvider.ts} | 20 ++++--- .../src/client.test.ts | 39 ++++++++++++++ .../src/client.ts | 19 ++++++- .../src/config.test.ts | 18 +++++++ .../src/config.ts | 31 +++++++++++ .../src/index.ts | 2 +- .../src/module.ts | 9 ++-- .../package.json | 8 +-- 16 files changed, 188 insertions(+), 92 deletions(-) rename workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/{provider.parts.test.ts => McpRegistryEntityProvider.parts.test.ts} (99%) rename workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/{provider.test.ts => McpRegistryEntityProvider.test.ts} (97%) rename workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/{provider.ts => McpRegistryEntityProvider.ts} (95%) diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/audit.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/audit.md index eec93d37aa7..5d644474ce5 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/audit.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/audit.md @@ -1,6 +1,6 @@ ## Audit Report: mcp-registry-provider -**Last audited:** 2026-09-18T00:00:00Z +**Last audited:** 2026-09-19T00:00:00Z ### Summary diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/tasks.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/tasks.md index c3cff5f918a..c77f663d4ca 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/tasks.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/tasks.md @@ -8,44 +8,44 @@ ## 1. Plugin Scaffolding & Packaging -- [ ] 1.1 Create the backend `catalog-backend-module` plugin package (Backstage catalog-backend-module naming convention) with `package.json`, `tsconfig`, and lint config matching the workspace's plugin conventions -- [ ] 1.2 Add the `createBackendModule` skeleton that registers against `catalogProcessingExtensionPoint`, depending on `coreServices` (`rootConfig`, `logger`, `scheduler`) -- [ ] 1.3 Document installation in the plugin `README.md` (add via `backend.add(...)`, minimal app-config example) +- [x] 1.1 Create the backend `catalog-backend-module` plugin package (Backstage catalog-backend-module naming convention) with `package.json`, `tsconfig`, and lint config matching the workspace's plugin conventions +- [x] 1.2 Add the `createBackendModule` skeleton that registers against `catalogProcessingExtensionPoint`, depending on `coreServices` (`rootConfig`, `logger`, `scheduler`) +- [x] 1.3 Document installation in the plugin `README.md` (add via `backend.add(...)`, minimal app-config example) ## 2. Configuration -- [ ] 2.1 Author `config.d.ts` declaring `catalog.providers.mcpRegistry` as a single object with `baseUrl` (required), `baseName?` (optional mapping-prefix override), `apiVersion?` (default `v1`), `schedule?` (`SchedulerServiceTaskScheduleDefinitionConfig` in config.d.ts; runtime scheduling uses `SchedulerServiceTaskScheduleDefinition` per design D3), `pageLimit?` (max pages per sync, default `10`), `pageSize?` (registry `?limit=` when set), and `defaultOwner?`; require `@visibility backend` annotations for backend-only fields such as `baseUrl` -- [ ] 2.2 Implement config reading: parse `catalog.providers.mcpRegistry` as a single object; register nothing (no error) when the key is absent -- [ ] 2.3 Implement validation with actionable errors (fail fast when `baseUrl` is missing; fail fast with a multiple-registries-out-of-scope message when the value is a keyed map of instance objects); apply the `apiVersion` default (`v1`), the default schedule when `schedule` is omitted (`frequency: { minutes: 30 }`, `timeout: { minutes: 3 }`, no `initialDelay` — same as spec requirement “Sync on the configured schedule”), and the `pageLimit` default (`10` pages per sync) when omitted -- [ ] 2.4 Add unit tests for config parsing/validation: single object with `baseUrl`, optional `baseName`, keyed-map rejection, missing `baseUrl`, absent-config no-op, omitted `pageLimit` → `10`, explicit `pageLimit` override, omitted `pageSize` (no invented default), and explicit `pageSize` +- [x] 2.1 Author `config.d.ts` declaring `catalog.providers.mcpRegistry` as a single object with `baseUrl` (required), `baseName?` (optional mapping-prefix override), `apiVersion?` (default `v1`), `schedule?` (`SchedulerServiceTaskScheduleDefinitionConfig` in config.d.ts; runtime scheduling uses `SchedulerServiceTaskScheduleDefinition` per design D3), `pageLimit?` (max pages per sync, default `10`), `pageSize?` (registry `?limit=` when set), and `defaultOwner?`; require `@visibility backend` annotations for backend-only fields such as `baseUrl` +- [x] 2.2 Implement config reading: parse `catalog.providers.mcpRegistry` as a single object; register nothing (no error) when the key is absent +- [x] 2.3 Implement validation with actionable errors (fail fast when `baseUrl` is missing; fail fast with a multiple-registries-out-of-scope message when the value is a keyed map of instance objects); apply the `apiVersion` default (`v1`), the default schedule when `schedule` is omitted (`frequency: { minutes: 30 }`, `timeout: { minutes: 3 }`, no `initialDelay` — same as spec requirement “Sync on the configured schedule”), and the `pageLimit` default (`10` pages per sync) when omitted +- [x] 2.4 Add unit tests for config parsing/validation: single object with `baseUrl`, optional `baseName`, keyed-map rejection, missing `baseUrl`, absent-config no-op, omitted `pageLimit` → `10`, explicit `pageLimit` override, omitted `pageSize` (no invented default), and explicit `pageSize` ## 3. Registry Client & Pagination -- [ ] 3.1 Define the registry API response types (`servers[]`, `metadata.count`, `metadata.nextCursor`) and the `server.json` extraction from each `servers[]` entry (`.server`) -- [ ] 3.2 Implement servers-endpoint URL construction `//servers` with slash normalization (works with and without a trailing slash on `baseUrl`) -- [ ] 3.3 Implement cursor pagination: loop passing prior `metadata.nextCursor` as the `cursor` query param until it is absent, null, or empty, accumulating all `servers[]`; treat cursors as opaque; when `pageSize` is set send it as `?limit=` on every list request; when `pageSize` is omitted leave `?limit=` unset -- [ ] 3.4 Implement the pagination loop safeguard: cap fetches at configured `pageLimit` pages per sync (`10` when omitted); do not send `pageLimit` as the registry `?limit=` query param; detect a repeated cursor; exceeding the page cap or a repeated cursor fails the run rather than looping forever -- [ ] 3.5 Implement registry-error handling (unreachable host, non-2xx status, unparseable body, pagination-safeguard trip) raising a typed error that aborts the run -- [ ] 3.6 Add unit tests for the client using mocked HTTP: single page, multi-page traversal, empty/absent/null cursor termination, opaque-cursor passthrough, omitted `pageSize` (no `limit` query), configured `pageSize` as `limit` on every page request, default `pageLimit` `10` tripping on an 11th page, configured `pageLimit` tripping, and error/safeguard cases +- [x] 3.1 Define the registry API response types (`servers[]`, `metadata.count`, `metadata.nextCursor`) and the `server.json` extraction from each `servers[]` entry (`.server`) +- [x] 3.2 Implement servers-endpoint URL construction `//servers` with slash normalization (works with and without a trailing slash on `baseUrl`) +- [x] 3.3 Implement cursor pagination: loop passing prior `metadata.nextCursor` as the `cursor` query param until it is absent, null, or empty, accumulating all `servers[]`; treat cursors as opaque; when `pageSize` is set send it as `?limit=` on every list request; when `pageSize` is omitted leave `?limit=` unset +- [x] 3.4 Implement the pagination loop safeguard: cap fetches at configured `pageLimit` pages per sync (`10` when omitted); do not send `pageLimit` as the registry `?limit=` query param; detect a repeated cursor; exceeding the page cap or a repeated cursor fails the run rather than looping forever +- [x] 3.5 Implement registry-error handling (unreachable host, non-2xx status, unparseable body, pagination-safeguard trip) raising a typed error that aborts the run +- [x] 3.6 Add unit tests for the client using mocked HTTP: single page, multi-page traversal, empty/absent/null cursor termination, opaque-cursor passthrough, omitted `pageSize` (no `limit` query), configured `pageSize` as `limit` on every page request, default `pageLimit` `10` tripping on an 11th page, configured `pageLimit` tripping, and error/safeguard cases ## 4. Entity Provider & Scheduling -- [ ] 4.1 Implement the `EntityProvider` class: `getProviderName()` = `mcp-registry-provider`, `connect()` storing the connection, and a `run()` performing one sync; maintain an in-memory last-good index keyed by `modelcontextprotocol.io/name` and `modelcontextprotocol.io/version`, rebuilt at the end of each successful sync from committed entities with `sync-status: ok` only (design D6) -- [ ] 4.2 Wire scheduling via `SchedulerService.createScheduledTaskRunner(schedule)` only (no synchronous `run()` from `connect()`); register the single provider when config is present -- [ ] 4.3 Implement the full-mutation commit: on successful sync call `connection.applyMutation({ type: 'full', entities })`; on a failed run emit no mutation (preserve prior catalog state) -- [ ] 4.4 Attach provider attribution and sync status to each entity: set mutation `locationKey` `mcp-registry-provider`, `backstage.io/managed-by-location` to `url:` + normalized `baseUrl` (trailing `/` stripped), and `redhat.com/rhdh-mcp-registry-sync-status` to `ok` or `degraded` per D8 -- [ ] 4.5 Add unit tests: full mutation contents (`locationKey`, `backstage.io/managed-by-location: url:`, `redhat.com/rhdh-mcp-registry-sync-status` `ok`/`degraded` per scenario), pruning across syncs, updated server reflected, last-good retention with `degraded` when mapping fails, and no-mutation-on-failed-run +- [x] 4.1 Implement the `EntityProvider` class: `getProviderName()` = `mcp-registry-provider`, `connect()` storing the connection, and a `run()` performing one sync; maintain an in-memory last-good index keyed by `modelcontextprotocol.io/name` and `modelcontextprotocol.io/version`, rebuilt at the end of each successful sync from committed entities with `sync-status: ok` only (design D6) +- [x] 4.2 Wire scheduling via `SchedulerService.createScheduledTaskRunner(schedule)` only (no synchronous `run()` from `connect()`); register the single provider when config is present +- [x] 4.3 Implement the full-mutation commit: on successful sync call `connection.applyMutation({ type: 'full', entities })`; on a failed run emit no mutation (preserve prior catalog state) +- [x] 4.4 Attach provider attribution and sync status to each entity: set mutation `locationKey` `mcp-registry-provider`, `backstage.io/managed-by-location` to `url:` + normalized `baseUrl` (trailing `/` stripped), and `redhat.com/rhdh-mcp-registry-sync-status` to `ok` or `degraded` per D8 +- [x] 4.5 Add unit tests: full mutation contents (`locationKey`, `backstage.io/managed-by-location: url:`, `redhat.com/rhdh-mcp-registry-sync-status` `ok`/`degraded` per scenario), pruning across syncs, updated server reflected, last-good retention with `degraded` when mapping fails, and no-mutation-on-failed-run ## 5. Mapping Integration -- [ ] 5.1 Depend on the sibling `mcp-registry-server-mapping` transform and invoke it per accumulated server, passing `defaultOwner` as the caller-override owner default and, when configured, `baseName` as the caller-override identity prefix (never reimplement the mapping) -- [ ] 5.2 Implement per-entry failure isolation: on mapping rejection, log an actionable message; when `server.json` has `name` and `version`, include the last-good provider-managed entity (from the in-memory index populated at end of prior sync) with mapping-owned fields unchanged and `redhat.com/rhdh-mcp-registry-sync-status: degraded`; on success set `ok`; otherwise omit the entry; continue the run -- [ ] 5.3 Add integration tests over sample `server.json` inputs → produced `mcp-server` `API` entities, asserting `spec.owner` reflects `defaultOwner` (and the mapping default `unknown` when omitted), `metadata.name` uses `baseName` as prefix when configured (and mapping default `mcp.registry` when omitted), that one bad entry does not abort the batch, that a mapping failure on a previously synced server retains the last-good entity with `redhat.com/rhdh-mcp-registry-sync-status: degraded`, and that successful mappings set `ok` +- [x] 5.1 Depend on the sibling `mcp-registry-server-mapping` transform and invoke it per accumulated server, passing `defaultOwner` as the caller-override owner default and, when configured, `baseName` as the caller-override identity prefix (never reimplement the mapping) +- [x] 5.2 Implement per-entry failure isolation: on mapping rejection, log an actionable message; when `server.json` has `name` and `version`, include the last-good provider-managed entity (from the in-memory index populated at end of prior sync) with mapping-owned fields unchanged and `redhat.com/rhdh-mcp-registry-sync-status: degraded`; on success set `ok`; otherwise omit the entry; continue the run +- [x] 5.3 Add integration tests over sample `server.json` inputs → produced `mcp-server` `API` entities, asserting `spec.owner` reflects `defaultOwner` (and the mapping default `unknown` when omitted), `metadata.name` uses `baseName` as prefix when configured (and mapping default `mcp.registry` when omitted), that one bad entry does not abort the batch, that a mapping failure on a previously synced server retains the last-good entity with `redhat.com/rhdh-mcp-registry-sync-status: degraded`, and that successful mappings set `ok` ## 6. End-to-End Verification & Docs -- [ ] 6.1 Add an end-to-end test wiring config → mocked paginated registry → mapping → full mutation, asserting the mutation converges to the registry's current server set -- [ ] 6.2 Verify produced entities pass the upstream `mcp-server` `API` entity schema (`McpServerApiEntity`) — reusing the mapping change's conformance expectations -- [ ] 6.3 Verify the apiVersion discrepancy handling: default `v1` requests `/v1/servers` and an override (`v0`) is honored, with a documented note for operators -- [ ] 6.4 Finalize `README.md` / config docs: full `catalog.providers.mcpRegistry` example (`baseUrl`, optional `baseName`, `apiVersion`, optional `schedule` — default `frequency: { minutes: 30 }`, `timeout: { minutes: 3 }`, no `initialDelay`; first sync after one `frequency` unless `initialDelay` is set), `pageLimit` default `10` pages per sync, optional `pageSize` as `?limit=`, `defaultOwner`), note that multiple registries are out of scope, pagination behavior, and error-handling semantics -- [ ] 6.5 Run the workspace lint, typecheck, and test suite; ensure the new package builds and passes CI conventions +- [x] 6.1 Add an end-to-end test wiring config → mocked paginated registry → mapping → full mutation, asserting the mutation converges to the registry's current server set +- [x] 6.2 Verify produced entities pass the upstream `mcp-server` `API` entity schema (`McpServerApiEntity`) — reusing the mapping change's conformance expectations +- [x] 6.3 Verify the apiVersion discrepancy handling: default `v1` requests `/v1/servers` and an override (`v0`) is honored, with a documented note for operators +- [x] 6.4 Finalize `README.md` / config docs: full `catalog.providers.mcpRegistry` example (`baseUrl`, optional `baseName`, `apiVersion`, optional `schedule` — default `frequency: { minutes: 30 }`, `timeout: { minutes: 3 }`, no `initialDelay`; first sync after one `frequency` unless `initialDelay` is set), `pageLimit` default `10` pages per sync, optional `pageSize` as `?limit=`, `defaultOwner`), note that multiple registries are out of scope, pagination behavior, and error-handling semantics +- [x] 6.5 Run the workspace lint, typecheck, and test suite; ensure the new package builds and passes CI conventions diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/design.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/design.md index 58365914871..b2c72c374d0 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/design.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/design.md @@ -70,7 +70,7 @@ Implementation tasks produce a version-pinned `mapping-reference.md` under `open **Non-Goals:** -- Registry HTTP client, polling, scheduling, or an entity provider/processor (separate future change). +- Registry HTTP client, polling, scheduling, or an entity provider/processor (implemented by `catalog-backend-module-mcp-registry-provider`). - Modifying the upstream `mcp-server` entity contract or its validation. - Reverse mapping (entity → `server.json`) beyond the scalar round-trip guarantee. - Executing or health-checking mapped servers, or interpreting local `packages[]` runtime details. @@ -110,11 +110,11 @@ Implementation tasks produce a version-pinned `mapping-reference.md` under `open **Alternatives considered:** (a) Encode the version in `metadata.namespace` — rejected; fragments entity references and complicates relationships. (b) `__` with no prefix — rejected; leaves registry-mapped entities without a caller-controllable namespacing token in `metadata.name` (they would collide with any other `mcp-server` API that sanitizes to the same name+version). -**Rationale:** A registry publishes one `server.json` per version and each becomes its own entity, so a name derived from the canonical name alone would collide across versions. The prefix distinguishes registry-mapped entities in a shared catalog and lets the future ingestion layer pass a per-source override without changing the transform. +**Rationale:** A registry publishes one `server.json` per version and each becomes its own entity, so a name derived from the canonical name alone would collide across versions. The prefix distinguishes registry-mapped entities in a shared catalog and lets the ingestion layer (`catalog-backend-module-mcp-registry-provider`) pass a per-source override without changing the transform. ### D5: Supplying fields absent from `server.json` — owner and lifecycle -**Choice:** `spec.owner` is set to the constant `unknown` by default; a caller MAY supply an override default, but the transform never fails for a missing owner (a placeholder owner keeps the output valid, and the future ingestion change can reassign ownership). `spec.lifecycle` is set to the constant `production` by default; a caller MAY supply an override default lifecycle value. Both fields use the same caller-override pattern as the identity prefix in D4. +**Choice:** `spec.owner` is set to the constant `unknown` by default; a caller MAY supply an override default, but the transform never fails for a missing owner (a placeholder owner keeps the output valid, and the ingestion layer can reassign ownership). `spec.lifecycle` is set to the constant `production` by default; a caller MAY supply an override default lifecycle value. Both fields use the same caller-override pattern as the identity prefix in D4. **Alternatives considered:** (a) Require caller-provided owner/lifecycle and fail if absent — rejected; a pure transform should always yield a valid entity, and ownership/lifecycle assignment belongs to the ingestion layer. (b) Derive lifecycle from a `status` field — rejected; `status` is not part of the base `server.schema.json` (verified 2026-08-21 against the draft schema). @@ -201,7 +201,7 @@ The scheme gate does **not** classify hosts as public vs private and does not tr ## Risks / Trade-offs - **63-char truncation collisions** → Deterministic hash suffix on truncation and on sanitization collisions keeps keys unique; the hash is derived from the full source path so it is stable across runs. -- **`metadata.name` collisions across registries** (same name+version from two registries under the default prefix) → Out of scope here (no dedup). The caller-overridable prefix is the ingestion-layer lever for per-source namespacing; documented so the future ingestion change can supply distinct prefixes or otherwise dedup. Within a single `(prefix, name, version)` the per-input hash-suffix rule (lossy sanitization or truncation) keeps that identity stable and distinct from a different unsanitized triple that happens to share a sanitized stem. +- **`metadata.name` collisions across registries** (same name+version from two registries under the default prefix) → Out of scope here (no dedup). The caller-overridable prefix is the ingestion-layer lever for per-source namespacing; documented so the ingestion layer can supply distinct prefixes or otherwise dedup. Within a single `(prefix, name, version)` the per-input hash-suffix rule (lossy sanitization or truncation) keeps that identity stable and distinct from a different unsanitized triple that happens to share a sanitized stem. - **Draft schema drift** → D7 fail-open projection; the mapping table is versioned against the draft and revisited when the schema changes. - **Lossy flattening of deep `packages[]` config** → Accepted; runtime package details are preserved as scalar-leaf annotations for discoverability, not interpreted. Round-trip fidelity is guaranteed only for scalar leaves. - **Secret leakage into searchable annotations** (remote `headers`/`variables`, `environmentVariables` carrying `default`/`value`/`choices`) → D9 prunes the `default`/`value`/`choices` leaves of any `isSecret: true` input from projection. This is a deliberate carve-out from scalar round-trip fidelity — those leaves are intentionally unrecoverable from the entity. Non-secret metadata on the same input still projects, so discoverability is preserved. @@ -211,7 +211,7 @@ The scheme gate does **not** classify hosts as public vs private and does not tr ## Migration Plan -Not applicable — new capabilities with no existing data or behavior to migrate. The mapping is additive and has no runtime deployment surface of its own until a future ingestion change consumes it. +Not applicable — new capabilities with no existing data or behavior to migrate. The mapping is additive and has no runtime deployment surface of its own; consumed by `catalog-backend-module-mcp-registry-provider`. ## Open Questions diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts index acd59c5579b..ac6f1a21b53 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts @@ -32,6 +32,8 @@ export interface Config { /** @visibility backend */ pageSize?: number; /** @visibility backend */ + maxEntries?: number; + /** @visibility backend */ schedule?: SchedulerServiceTaskScheduleDefinitionConfig; }; }; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json index 58c495787cf..bc27f15e775 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json @@ -27,15 +27,11 @@ "scripts": { "start": "backstage-cli package start", "build": "backstage-cli package build", - "lint:check": "backstage-cli package lint", - "lint:fix": "backstage-cli package lint --fix", + "lint": "backstage-cli package lint", "test": "backstage-cli package test --passWithNoTests --coverage", "clean": "backstage-cli package clean", "prepack": "backstage-cli package prepack", - "postpack": "backstage-cli package postpack", - "tsc": "tsc", - "prettier:check": "prettier --ignore-unknown --check .", - "prettier:fix": "prettier --ignore-unknown --write ." + "postpack": "backstage-cli package postpack" }, "dependencies": { "@backstage/backend-plugin-api": "^1.10.0", diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md index 86a333fca09..938912b5f8d 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md @@ -7,7 +7,6 @@ import { BackendFeature } from '@backstage/backend-plugin-api'; import type { EntityProvider } from '@backstage/plugin-catalog-node'; import type { EntityProviderConnection } from '@backstage/plugin-catalog-node'; import type { LoggerService } from '@backstage/backend-plugin-api'; -import type { SchedulerServiceTaskRunner } from '@backstage/backend-plugin-api'; import type { SchedulerServiceTaskScheduleDefinition } from '@backstage/backend-plugin-api'; // @public @@ -19,31 +18,23 @@ export class McpRegistryEntityProvider implements EntityProvider { constructor( config: McpRegistryProviderConfig, logger: LoggerService, - fetchApi?: typeof fetch, - taskRunner?: SchedulerServiceTaskRunner, + options?: {}, ); // (undocumented) connect(connection: EntityProviderConnection): Promise; // (undocumented) getProviderName(): string; - run(): Promise; } // @public export interface McpRegistryProviderConfig { - // (undocumented) apiVersion: string; - // (undocumented) baseName?: string; - // (undocumented) baseUrl: string; - // (undocumented) defaultOwner?: string; - // (undocumented) + maxEntries: number; pageLimit: number; - // (undocumented) pageSize?: number; - // (undocumented) schedule: SchedulerServiceTaskScheduleDefinition; } ``` diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.parts.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts similarity index 99% rename from workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.parts.test.ts rename to workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts index ab5753ba477..1deee459cf9 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.parts.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts @@ -24,7 +24,7 @@ import { formatMappingFailureMessage, McpRegistryEntityProvider, readServerIdentity, -} from './provider'; +} from './McpRegistryEntityProvider'; import { createMockServerDoc } from './testUtils'; const SYNC_STATUS_ANNOTATION = 'redhat.com/rhdh-mcp-registry-sync-status'; @@ -76,6 +76,7 @@ function createDefaultConfig( baseUrl: 'https://registry.example.com', apiVersion: 'v1', pageLimit: 10, + maxEntries: 5000, schedule: { frequency: { minutes: 30 }, timeout: { minutes: 3 }, @@ -200,7 +201,7 @@ describe('McpRegistryEntityProvider parts', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), createMockLogger(), - mockFetchForResponses([body]), + { fetchApi: mockFetchForResponses([body]) }, ); await expect(parts(provider).fetchRegistryEntries()).resolves.toEqual( @@ -218,7 +219,7 @@ describe('McpRegistryEntityProvider parts', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), logger, - fetchFn, + { fetchApi: fetchFn }, ); await expect( @@ -239,7 +240,7 @@ describe('McpRegistryEntityProvider parts', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), logger, - fetchFn, + { fetchApi: fetchFn }, ); await expect( diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts similarity index 97% rename from workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts rename to workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts index a742fd9fc73..891d92bc3b6 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts @@ -14,7 +14,7 @@ * limitations under the License. */ -import { McpRegistryEntityProvider } from './provider'; +import { McpRegistryEntityProvider } from './McpRegistryEntityProvider'; import type { McpRegistryProviderConfig } from './config'; import type { SchedulerServiceTaskRunner } from '@backstage/backend-plugin-api'; import type { EntityProviderConnection } from '@backstage/plugin-catalog-node'; @@ -45,6 +45,7 @@ function createDefaultConfig( baseUrl: 'https://registry.example.com', apiVersion: 'v1', pageLimit: 10, + maxEntries: 5000, schedule: { frequency: { minutes: 30 }, timeout: { minutes: 3 }, @@ -96,8 +97,7 @@ describe('McpRegistryEntityProvider', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), createMockLogger(), - fetchFn, - taskRunner, + { fetchApi: fetchFn, taskRunner }, ); await provider.connect(connection); @@ -129,7 +129,7 @@ describe('McpRegistryEntityProvider', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), logger, - fetchFn, + { fetchApi: fetchFn }, ); await provider.connect(connection); await provider.run(); @@ -171,7 +171,7 @@ describe('McpRegistryEntityProvider', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig({ baseUrl: 'https://registry.example.com/' }), createMockLogger(), - fetchFn, + { fetchApi: fetchFn }, ); await provider.connect(connection); await provider.run(); @@ -204,7 +204,7 @@ describe('McpRegistryEntityProvider', () => { defaultOwner: 'group:default/mcp-admins', }), createMockLogger(), - fetchFn, + { fetchApi: fetchFn }, ); await provider.connect(connection); await provider.run(); @@ -228,7 +228,7 @@ describe('McpRegistryEntityProvider', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), createMockLogger(), - fetchFn, + { fetchApi: fetchFn }, ); await provider.connect(connection); await provider.run(); @@ -250,7 +250,7 @@ describe('McpRegistryEntityProvider', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig({ baseName: 'com.example.registry' }), createMockLogger(), - fetchFn, + { fetchApi: fetchFn }, ); await provider.connect(connection); await provider.run(); @@ -275,7 +275,7 @@ describe('McpRegistryEntityProvider', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), createMockLogger(), - fetchFn, + { fetchApi: fetchFn }, ); await provider.connect(connection); await provider.run(); @@ -293,7 +293,7 @@ describe('McpRegistryEntityProvider', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), logger, - fetchFn, + { fetchApi: fetchFn }, ); await provider.connect(connection); await provider.run(); @@ -326,7 +326,7 @@ describe('McpRegistryEntityProvider', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), logger, - fetchFn, + { fetchApi: fetchFn }, ); await provider.connect(connection); await provider.run(); @@ -382,7 +382,7 @@ describe('McpRegistryEntityProvider', () => { const provider2 = new McpRegistryEntityProvider( createDefaultConfig(), logger2, - combinedFetch, + { fetchApi: combinedFetch }, ); await provider2.connect(connection2); @@ -429,7 +429,7 @@ describe('McpRegistryEntityProvider', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), logger, - fetchFn, + { fetchApi: fetchFn }, ); await provider.connect(connection); await provider.run(); @@ -469,7 +469,7 @@ describe('McpRegistryEntityProvider', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), createMockLogger(), - combinedFetch, + { fetchApi: combinedFetch }, ); await provider.connect(connection); @@ -506,7 +506,7 @@ describe('McpRegistryEntityProvider', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), createMockLogger(), - fetchFn, + { fetchApi: fetchFn }, ); await provider.connect(connection); await provider.run(); @@ -532,7 +532,7 @@ describe('McpRegistryEntityProvider', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), createMockLogger(), - fetchFn, + { fetchApi: fetchFn }, ); await provider.connect(connection); await provider.run(); @@ -554,7 +554,7 @@ describe('McpRegistryEntityProvider', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), createMockLogger(), - fetchFn, + { fetchApi: fetchFn }, ); await provider.connect(connection); await provider.run(); @@ -614,7 +614,7 @@ describe('McpRegistryEntityProvider', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), logger, - combinedFetch, + { fetchApi: combinedFetch }, ); await provider.connect(connection); @@ -694,7 +694,7 @@ describe('McpRegistryEntityProvider', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), logger, - combinedFetch, + { fetchApi: combinedFetch }, ); await provider.connect(connection); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts similarity index 95% rename from workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts rename to workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index 433f54e86d6..c42ea85e5c5 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/provider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -115,13 +115,17 @@ export class McpRegistryEntityProvider implements EntityProvider { constructor( config: McpRegistryProviderConfig, logger: LoggerService, - fetchApi?: typeof fetch, - taskRunner?: SchedulerServiceTaskRunner, + options?: { + /** @internal Override the global `fetch` implementation (test seam). */ + fetchApi?: typeof fetch; + /** @internal Scheduler task runner for periodic sync. */ + taskRunner?: SchedulerServiceTaskRunner; + }, ) { this.config = config; this.logger = logger; - this.fetchApi = fetchApi; - this.taskRunner = taskRunner; + this.fetchApi = options?.fetchApi; + this.taskRunner = options?.taskRunner; } getProviderName(): string { @@ -145,6 +149,8 @@ export class McpRegistryEntityProvider implements EntityProvider { /** * Run one sync cycle: fetch servers from the registry, map them, * and commit a full mutation. + * + * @internal */ async run(): Promise { if (!this.connection) { @@ -193,13 +199,15 @@ export class McpRegistryEntityProvider implements EntityProvider { private async fetchRegistryEntries(): Promise< McpRegistryServerEntry[] | undefined > { - const { baseUrl, apiVersion, pageLimit, pageSize } = this.config; + const { baseUrl, apiVersion, pageLimit, pageSize, maxEntries } = + this.config; try { return await fetchRegistryServers({ baseUrl, apiVersion, pageLimit, pageSize, + maxEntries, fetchApi: this.fetchApi, }); } catch (err) { @@ -368,7 +376,7 @@ export class McpRegistryEntityProvider implements EntityProvider { this.lastGoodIndex.clear(); for (const deferred of entities) { const annotations = deferred.entity.metadata?.annotations; - if (annotations?.[SYNC_STATUS_ANNOTATION] === 'degraded') { + if (annotations?.[SYNC_STATUS_ANNOTATION] !== 'ok') { continue; } const name = annotations?.['modelcontextprotocol.io/name']; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts index 76d2e3e3aa4..7a72594373a 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts @@ -329,6 +329,45 @@ describe('fetchRegistryServers', () => { // URL encodes the cursor, but the original value should be present expect(secondUrl).toContain(`cursor=${encodeURIComponent(opaqueToken)}`); }); + + it('throws when maxEntries cap is exceeded', async () => { + const largePage: McpRegistryListResponse = { + servers: Array.from({ length: 100 }, (_, i) => ({ + server: createMockServerDoc(`test/server-${i}`, '1.0.0'), + })), + metadata: { count: 100 }, + }; + const fn = mockFetch([{ body: largePage }]); + + await expect( + fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + maxEntries: 50, + fetchApi: fn, + }), + ).rejects.toThrow(/maxEntries cap of 50/); + }); + + it('does not enforce maxEntries when unset', async () => { + const largePage: McpRegistryListResponse = { + servers: Array.from({ length: 100 }, (_, i) => ({ + server: createMockServerDoc(`test/server-${i}`, '1.0.0'), + })), + metadata: { count: 100 }, + }; + const fn = mockFetch([{ body: largePage }]); + + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + fetchApi: fn, + }); + + expect(result).toHaveLength(100); + }); }); describe('parseServersEndpointUrl', () => { diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index 6a452be9079..eca7c795e5a 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -69,6 +69,12 @@ export interface FetchServersOptions { apiVersion: string; pageLimit: number; pageSize?: number; + /** + * Maximum total entries accumulated across all pages. When exceeded + * the sync aborts to prevent unbounded memory growth from a + * malfunctioning registry returning oversized pages. + */ + maxEntries?: number; /** Optional fetch implementation for testing. */ fetchApi?: typeof fetch; } @@ -220,7 +226,8 @@ export function resolveNextCursor( export async function fetchRegistryServers( options: FetchServersOptions, ): Promise { - const { baseUrl, apiVersion, pageLimit, pageSize, fetchApi } = options; + const { baseUrl, apiVersion, pageLimit, pageSize, maxEntries, fetchApi } = + options; const doFetch = fetchApi ?? fetch; const endpoint = parseServersEndpointUrl(baseUrl, apiVersion); @@ -236,6 +243,16 @@ export async function fetchRegistryServers( allServers.push(...body.servers); pagesFetched += 1; + if (maxEntries !== undefined && allServers.length > maxEntries) { + throw new McpRegistryClientError( + `MCP Registry sync accumulated ${allServers.length} entries, ` + + `exceeding the configured maxEntries cap of ${maxEntries}. ` + + `Aborting sync to prevent unbounded memory growth. ` + + `Increase maxEntries if the registry legitimately contains ` + + `more servers.`, + ); + } + const nextCursor = resolveNextCursor( body.metadata?.nextCursor, seenCursors, diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts index cb6b147612a..abae9946ed5 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts @@ -17,6 +17,7 @@ import { ConfigReader } from '@backstage/config'; import { assertSingleRegistryConfig, + readMaxEntries, readMcpRegistryProviderConfig, readOptionalPageSize, readPageLimit, @@ -54,6 +55,7 @@ describe('readMcpRegistryProviderConfig', () => { expect(result!.baseUrl).toBe('https://registry.example.com'); expect(result!.apiVersion).toBe('v1'); expect(result!.pageLimit).toBe(10); + expect(result!.maxEntries).toBe(5000); expect(result!.pageSize).toBeUndefined(); expect(result!.baseName).toBeUndefined(); expect(result!.defaultOwner).toBeUndefined(); @@ -339,6 +341,22 @@ describe('readOptionalPageSize', () => { }); }); +describe('readMaxEntries', () => { + it('defaults to 5000 when omitted', () => { + expect(readMaxEntries(new ConfigReader({}))).toBe(5000); + }); + + it('returns an explicit maxEntries', () => { + expect(readMaxEntries(new ConfigReader({ maxEntries: 1000 }))).toBe(1000); + }); + + it('throws when maxEntries is less than 1', () => { + expect(() => readMaxEntries(new ConfigReader({ maxEntries: 0 }))).toThrow( + /"maxEntries" must be at least 1/, + ); + }); +}); + describe('readProviderSchedule', () => { it('returns the default schedule when omitted', () => { expect(readProviderSchedule(new ConfigReader({}))).toEqual({ diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts index 9c7a1d9b798..404d2d6cfa6 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts @@ -30,6 +30,9 @@ const DEFAULT_API_VERSION = 'v1'; /** Default page limit (max pages per sync). */ const DEFAULT_PAGE_LIMIT = 10; +/** Default max entries per sync. */ +const DEFAULT_MAX_ENTRIES = 5000; + /** Supported single-registry config keys under `catalog.providers.mcpRegistry`. */ const KNOWN_MCP_REGISTRY_KEYS = new Set([ 'baseUrl', @@ -38,6 +41,7 @@ const KNOWN_MCP_REGISTRY_KEYS = new Set([ 'defaultOwner', 'pageLimit', 'pageSize', + 'maxEntries', 'schedule', ]); @@ -145,6 +149,23 @@ export function readPageLimit(registryConfig: Config): number { return pageLimit; } +/** + * Read `maxEntries`, applying the default and rejecting values below 1. + * + * @internal + */ +export function readMaxEntries(registryConfig: Config): number { + const maxEntries = + registryConfig.getOptionalNumber('maxEntries') ?? DEFAULT_MAX_ENTRIES; + if (maxEntries < 1) { + throw new Error( + `Invalid catalog.providers.mcpRegistry configuration: "maxEntries" ` + + `must be at least 1, got ${maxEntries}.`, + ); + } + return maxEntries; +} + /** * Read optional `pageSize`, rejecting values below 1 when set. * @@ -184,12 +205,21 @@ export function readProviderSchedule( * @public */ export interface McpRegistryProviderConfig { + /** Base URL of the MCP Registry (required). */ baseUrl: string; + /** Optional identity prefix override passed to the mapping transform. */ baseName?: string; + /** Registry API version slug used in the endpoint path (default `v1`). */ apiVersion: string; + /** Default entity owner ref when the mapping does not supply one. */ defaultOwner?: string; + /** Maximum pages fetched per sync (default `10`). */ pageLimit: number; + /** Registry `?limit=` page-size query; omitted from the request when unset. */ pageSize?: number; + /** Maximum total entries accumulated across all pages per sync (default `5000`). */ + maxEntries: number; + /** Schedule for the sync task. */ schedule: SchedulerServiceTaskScheduleDefinition; } @@ -225,6 +255,7 @@ export function readMcpRegistryProviderConfig( defaultOwner: safeGetOptionalString(registryConfig, 'defaultOwner'), pageLimit: readPageLimit(registryConfig), pageSize: readOptionalPageSize(registryConfig), + maxEntries: readMaxEntries(registryConfig), schedule: readProviderSchedule(registryConfig), }; } diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts index ad3e7f701aa..af6fc0d84c2 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts @@ -21,5 +21,5 @@ */ export { catalogModuleMcpRegistryProvider as default } from './module'; -export { McpRegistryEntityProvider } from './provider'; +export { McpRegistryEntityProvider } from './McpRegistryEntityProvider'; export type { McpRegistryProviderConfig } from './config'; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts index 758190ee5ec..6ac155734e9 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts @@ -20,7 +20,7 @@ import { } from '@backstage/backend-plugin-api'; import { catalogProcessingExtensionPoint } from '@backstage/plugin-catalog-node'; import { readMcpRegistryProviderConfig } from './config'; -import { McpRegistryEntityProvider } from './provider'; +import { McpRegistryEntityProvider } from './McpRegistryEntityProvider'; /** * The mcp-registry-provider backend module for the catalog plugin. @@ -56,12 +56,9 @@ export const catalogModuleMcpRegistryProvider = createBackendModule({ const taskRunner = scheduler.createScheduledTaskRunner( providerConfig.schedule, ); - const provider = new McpRegistryEntityProvider( - providerConfig, - logger, - undefined, + const provider = new McpRegistryEntityProvider(providerConfig, logger, { taskRunner, - ); + }); catalog.addEntityProvider(provider); }, diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json b/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json index bf411dd3d2b..bd36d4d5ac0 100644 --- a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json +++ b/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json @@ -28,15 +28,11 @@ "scripts": { "build": "backstage-cli package build", "clean": "backstage-cli package clean", - "lint:check": "backstage-cli package lint", - "lint:fix": "backstage-cli package lint --fix", + "lint": "backstage-cli package lint", "postpack": "backstage-cli package postpack", "prepack": "backstage-cli package prepack", "start": "backstage-cli package start", - "test": "backstage-cli package test --passWithNoTests --coverage", - "tsc": "tsc", - "prettier:check": "prettier --ignore-unknown --check .", - "prettier:fix": "prettier --ignore-unknown --write ." + "test": "backstage-cli package test --passWithNoTests --coverage" }, "dependencies": { "@backstage/catalog-model": "^1.10.1" From 72979084a6890cb3bb4ba9ed4dc2eebc93635a45 Mon Sep 17 00:00:00 2001 From: fullsend-fix <278716306+fullsend-ai-coder[bot]@users.noreply.github.com> Date: Sat, 19 Sep 2026 15:02:38 +0000 Subject: [PATCH 13/63] fix(#4815): restore lint:check/lint:fix/tsc/prettier scripts in package.json Bring back lint:check, lint:fix, tsc, prettier:check, and prettier:fix scripts in both catalog-backend-module-mcp-registry-provider and mcp-registry-server-mapping-common package.json files, while keeping the standard lint script for workspace compatibility with other plugins. Addresses #4871 Assisted-by: Claude Opus 4.6 --- .../package.json | 7 ++++++- .../mcp-registry-server-mapping-common/package.json | 7 ++++++- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json index bc27f15e775..c6c0da73640 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json @@ -28,10 +28,15 @@ "start": "backstage-cli package start", "build": "backstage-cli package build", "lint": "backstage-cli package lint", + "lint:check": "backstage-cli package lint", + "lint:fix": "backstage-cli package lint --fix", "test": "backstage-cli package test --passWithNoTests --coverage", "clean": "backstage-cli package clean", "prepack": "backstage-cli package prepack", - "postpack": "backstage-cli package postpack" + "postpack": "backstage-cli package postpack", + "tsc": "tsc", + "prettier:check": "prettier --ignore-unknown --check .", + "prettier:fix": "prettier --ignore-unknown --write ." }, "dependencies": { "@backstage/backend-plugin-api": "^1.10.0", diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json b/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json index bd36d4d5ac0..375613c6f6d 100644 --- a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json +++ b/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json @@ -29,10 +29,15 @@ "build": "backstage-cli package build", "clean": "backstage-cli package clean", "lint": "backstage-cli package lint", + "lint:check": "backstage-cli package lint", + "lint:fix": "backstage-cli package lint --fix", "postpack": "backstage-cli package postpack", "prepack": "backstage-cli package prepack", "start": "backstage-cli package start", - "test": "backstage-cli package test --passWithNoTests --coverage" + "test": "backstage-cli package test --passWithNoTests --coverage", + "tsc": "tsc", + "prettier:check": "prettier --ignore-unknown --check .", + "prettier:fix": "prettier --ignore-unknown --write ." }, "dependencies": { "@backstage/catalog-model": "^1.10.1" From 19e876faf2a0d5637ae642bc2c31ffa4cde7a414 Mon Sep 17 00:00:00 2001 From: fullsend-fix <278716306+fullsend-ai-coder[bot]@users.noreply.github.com> Date: Sat, 19 Sep 2026 15:23:55 +0000 Subject: [PATCH 14/63] fix(#4815): align package.json scripts with workspace convention Remove lint:check, lint:fix, tsc, prettier:check, and prettier:fix scripts from both mcp-registry-provider and mcp-registry-server- mapping-common package.json files to match the standard scripts block used by all other catalog-backend-module-* plugins in the ai-integrations workspace. Update mcp-registry-server-mapping audit.md timestamp to reflect the proposal.md and design.md changes made in this PR. Addresses #4871 Assisted-by: Claude Opus 4.6 --- .../openspec/changes/mcp-registry-server-mapping/audit.md | 2 +- .../package.json | 7 +------ .../mcp-registry-server-mapping-common/package.json | 7 +------ 3 files changed, 3 insertions(+), 13 deletions(-) diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/audit.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/audit.md index ef6ce2c5ac2..7c0b29f6848 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/audit.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/audit.md @@ -1,6 +1,6 @@ ## Audit Report: mcp-registry-server-mapping -**Last audited:** 2026-09-15T19:51:11Z +**Last audited:** 2026-09-19T00:00:00Z ### Summary diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json index c6c0da73640..bc27f15e775 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json @@ -28,15 +28,10 @@ "start": "backstage-cli package start", "build": "backstage-cli package build", "lint": "backstage-cli package lint", - "lint:check": "backstage-cli package lint", - "lint:fix": "backstage-cli package lint --fix", "test": "backstage-cli package test --passWithNoTests --coverage", "clean": "backstage-cli package clean", "prepack": "backstage-cli package prepack", - "postpack": "backstage-cli package postpack", - "tsc": "tsc", - "prettier:check": "prettier --ignore-unknown --check .", - "prettier:fix": "prettier --ignore-unknown --write ." + "postpack": "backstage-cli package postpack" }, "dependencies": { "@backstage/backend-plugin-api": "^1.10.0", diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json b/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json index 375613c6f6d..bd36d4d5ac0 100644 --- a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json +++ b/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json @@ -29,15 +29,10 @@ "build": "backstage-cli package build", "clean": "backstage-cli package clean", "lint": "backstage-cli package lint", - "lint:check": "backstage-cli package lint", - "lint:fix": "backstage-cli package lint --fix", "postpack": "backstage-cli package postpack", "prepack": "backstage-cli package prepack", "start": "backstage-cli package start", - "test": "backstage-cli package test --passWithNoTests --coverage", - "tsc": "tsc", - "prettier:check": "prettier --ignore-unknown --check .", - "prettier:fix": "prettier --ignore-unknown --write ." + "test": "backstage-cli package test --passWithNoTests --coverage" }, "dependencies": { "@backstage/catalog-model": "^1.10.1" From 663ef896d41f0663e2092246dfc9089a54a1c53f Mon Sep 17 00:00:00 2001 From: fullsend-fix <278716306+fullsend-ai-coder[bot]@users.noreply.github.com> Date: Sat, 19 Sep 2026 15:45:52 +0000 Subject: [PATCH 15/63] revert: undo package.json script removal from b255a04 Revert commit b255a04 which removed lint:check, lint:fix, tsc, prettier:check, and prettier:fix scripts from both package.json files. Restores audit.md timestamp to its pre-b255a04 value. Addresses #4871 Assisted-by: Claude Opus 4.6 --- .../openspec/changes/mcp-registry-server-mapping/audit.md | 2 +- .../package.json | 7 ++++++- .../mcp-registry-server-mapping-common/package.json | 7 ++++++- 3 files changed, 13 insertions(+), 3 deletions(-) diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/audit.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/audit.md index 7c0b29f6848..ef6ce2c5ac2 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/audit.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/audit.md @@ -1,6 +1,6 @@ ## Audit Report: mcp-registry-server-mapping -**Last audited:** 2026-09-19T00:00:00Z +**Last audited:** 2026-09-15T19:51:11Z ### Summary diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json index bc27f15e775..c6c0da73640 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json @@ -28,10 +28,15 @@ "start": "backstage-cli package start", "build": "backstage-cli package build", "lint": "backstage-cli package lint", + "lint:check": "backstage-cli package lint", + "lint:fix": "backstage-cli package lint --fix", "test": "backstage-cli package test --passWithNoTests --coverage", "clean": "backstage-cli package clean", "prepack": "backstage-cli package prepack", - "postpack": "backstage-cli package postpack" + "postpack": "backstage-cli package postpack", + "tsc": "tsc", + "prettier:check": "prettier --ignore-unknown --check .", + "prettier:fix": "prettier --ignore-unknown --write ." }, "dependencies": { "@backstage/backend-plugin-api": "^1.10.0", diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json b/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json index bd36d4d5ac0..375613c6f6d 100644 --- a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json +++ b/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json @@ -29,10 +29,15 @@ "build": "backstage-cli package build", "clean": "backstage-cli package clean", "lint": "backstage-cli package lint", + "lint:check": "backstage-cli package lint", + "lint:fix": "backstage-cli package lint --fix", "postpack": "backstage-cli package postpack", "prepack": "backstage-cli package prepack", "start": "backstage-cli package start", - "test": "backstage-cli package test --passWithNoTests --coverage" + "test": "backstage-cli package test --passWithNoTests --coverage", + "tsc": "tsc", + "prettier:check": "prettier --ignore-unknown --check .", + "prettier:fix": "prettier --ignore-unknown --write ." }, "dependencies": { "@backstage/catalog-model": "^1.10.1" From d441ada86fbc7f3a37f006f538a005a3729a4b65 Mon Sep 17 00:00:00 2001 From: fullsend-fix <278716306+fullsend-ai-coder[bot]@users.noreply.github.com> Date: Sat, 19 Sep 2026 16:37:13 +0000 Subject: [PATCH 16/63] fix(#4815): add optional hostAllowList config for SSRF defense-in-depth Add an optional hostAllowList config field under catalog.providers.mcpRegistry that restricts outbound requests to explicitly permitted hostnames. When configured: - Config parsing validates that the baseUrl hostname is in the list - The registry client validates the endpoint hostname at runtime Hostnames are normalized to lowercase for case-insensitive matching. When omitted, all hosts are allowed (backward-compatible). Addresses #4871 Assisted-by: Claude Opus 4.6 --- .../README.md | 22 ++-- .../config.d.ts | 2 + .../report.api.md | 1 + .../src/McpRegistryEntityProvider.ts | 11 +- .../src/client.test.ts | 78 ++++++++++++ .../src/client.ts | 43 ++++++- .../src/config.test.ts | 114 ++++++++++++++++++ .../src/config.ts | 49 +++++++- 8 files changed, 306 insertions(+), 14 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index ce0d33f068d..267cd5a6aa9 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -37,6 +37,9 @@ catalog: # pageLimit: 10 # Optional: registry page size sent as ?limit= (omitted by default) # pageSize: 50 + # Optional: restrict outbound requests to specific hostnames (defense-in-depth) + # hostAllowList: + # - registry.example.com # Optional: sync schedule (defaults shown below) # schedule: # frequency: { minutes: 30 } @@ -47,15 +50,16 @@ catalog: ### Configuration options -| Option | Required | Default | Description | -| -------------- | -------- | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `baseUrl` | Yes | — | MCP Registry base URL | -| `baseName` | No | `mcp.registry` (mapping default) | Override the mapping identity prefix for `metadata.name` | -| `apiVersion` | No | `v1` | Registry API version slug. The servers endpoint is `//servers`. Note: the live MCP Registry may serve `/v0` or `/v0.1`; set `apiVersion` to match your registry. | -| `defaultOwner` | No | `unknown` (mapping default) | Backstage entity reference used as `spec.owner` | -| `pageLimit` | No | `10` | Maximum number of pages fetched per sync. The provider fails the sync if the registry has more pages than this limit (to prevent incomplete catalog state). | -| `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | -| `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | +| Option | Required | Default | Description | +| --------------- | -------- | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `baseUrl` | Yes | — | MCP Registry base URL | +| `baseName` | No | `mcp.registry` (mapping default) | Override the mapping identity prefix for `metadata.name` | +| `apiVersion` | No | `v1` | Registry API version slug. The servers endpoint is `//servers`. Note: the live MCP Registry may serve `/v0` or `/v0.1`; set `apiVersion` to match your registry. | +| `defaultOwner` | No | `unknown` (mapping default) | Backstage entity reference used as `spec.owner` | +| `pageLimit` | No | `10` | Maximum number of pages fetched per sync. The provider fails the sync if the registry has more pages than this limit (to prevent incomplete catalog state). | +| `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | +| `hostAllowList` | No | _(none — all hosts allowed)_ | Array of permitted hostnames. When set, `baseUrl` hostname must be in this list and every outbound request is validated at runtime. Provides defense-in-depth against SSRF. | +| `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | ### Multiple registries diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts index ac6f1a21b53..793625edaab 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts @@ -34,6 +34,8 @@ export interface Config { /** @visibility backend */ maxEntries?: number; /** @visibility backend */ + hostAllowList?: string[]; + /** @visibility backend */ schedule?: SchedulerServiceTaskScheduleDefinitionConfig; }; }; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md index 938912b5f8d..249b748da9d 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md @@ -32,6 +32,7 @@ export interface McpRegistryProviderConfig { baseName?: string; baseUrl: string; defaultOwner?: string; + hostAllowList?: string[]; maxEntries: number; pageLimit: number; pageSize?: number; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index c42ea85e5c5..f7cf7989c6d 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -199,8 +199,14 @@ export class McpRegistryEntityProvider implements EntityProvider { private async fetchRegistryEntries(): Promise< McpRegistryServerEntry[] | undefined > { - const { baseUrl, apiVersion, pageLimit, pageSize, maxEntries } = - this.config; + const { + baseUrl, + apiVersion, + pageLimit, + pageSize, + maxEntries, + hostAllowList, + } = this.config; try { return await fetchRegistryServers({ baseUrl, @@ -208,6 +214,7 @@ export class McpRegistryEntityProvider implements EntityProvider { pageLimit, pageSize, maxEntries, + hostAllowList, fetchApi: this.fetchApi, }); } catch (err) { diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts index 7a72594373a..012d74f4e38 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts @@ -23,6 +23,7 @@ import { parseServersEndpointUrl, resolveNextCursor, truncateErrorBody, + validateUrlHostAllowList, } from './client'; import type { McpRegistryListResponse } from './client'; import { createMockServerDoc } from './testUtils'; @@ -350,6 +351,41 @@ describe('fetchRegistryServers', () => { ).rejects.toThrow(/maxEntries cap of 50/); }); + it('succeeds when hostAllowList includes the baseUrl hostname', async () => { + const body: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 1 }, + }; + const fn = mockFetch([{ body }]); + + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + hostAllowList: ['registry.example.com'], + fetchApi: fn, + }); + + expect(result).toHaveLength(1); + }); + + it('throws when hostAllowList does not include the baseUrl hostname', async () => { + const fn = mockFetch([]); + + await expect( + fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + hostAllowList: ['other.example.com'], + fetchApi: fn, + }), + ).rejects.toThrow(/not in the configured hostAllowList/); + + // Verify no fetch was attempted + expect(fn).not.toHaveBeenCalled(); + }); + it('does not enforce maxEntries when unset', async () => { const largePage: McpRegistryListResponse = { servers: Array.from({ length: 100 }, (_, i) => ({ @@ -473,6 +509,48 @@ describe('fetchRegistryPage', () => { }); }); +describe('validateUrlHostAllowList', () => { + it('does nothing when hostAllowList is undefined', () => { + expect(() => + validateUrlHostAllowList( + new URL('https://registry.example.com/v1/servers'), + undefined, + ), + ).not.toThrow(); + }); + + it('passes when hostname is in the allow list', () => { + expect(() => + validateUrlHostAllowList( + new URL('https://registry.example.com/v1/servers'), + ['registry.example.com'], + ), + ).not.toThrow(); + }); + + it('throws McpRegistryClientError when hostname is not in the allow list', () => { + expect(() => + validateUrlHostAllowList(new URL('https://evil.example.com/v1/servers'), [ + 'registry.example.com', + ]), + ).toThrow(McpRegistryClientError); + expect(() => + validateUrlHostAllowList(new URL('https://evil.example.com/v1/servers'), [ + 'registry.example.com', + ]), + ).toThrow(/not in the configured hostAllowList/); + }); + + it('matches case-insensitively', () => { + expect(() => + validateUrlHostAllowList( + new URL('https://Registry.Example.COM/v1/servers'), + ['registry.example.com'], + ), + ).not.toThrow(); + }); +}); + describe('resolveNextCursor', () => { it('returns undefined when nextCursor is absent or empty', () => { const seen = new Set(); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index eca7c795e5a..80e649a7601 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -75,6 +75,12 @@ export interface FetchServersOptions { * malfunctioning registry returning oversized pages. */ maxEntries?: number; + /** + * Optional allowlist of permitted hostnames. When set, every + * outbound request URL is validated against this list before + * fetching, providing defense-in-depth against SSRF. + */ + hostAllowList?: string[]; /** Optional fetch implementation for testing. */ fetchApi?: typeof fetch; } @@ -215,6 +221,28 @@ export function resolveNextCursor( return nextCursor; } +/** + * Validate that a URL's hostname is present in the configured allow list. + * Throws McpRegistryClientError when the hostname is not permitted. + * + * @internal + */ +export function validateUrlHostAllowList( + url: URL, + hostAllowList: string[] | undefined, +): void { + if (!hostAllowList) { + return; + } + const hostname = url.hostname.toLowerCase(); + if (!hostAllowList.includes(hostname)) { + throw new McpRegistryClientError( + `Request to hostname "${hostname}" blocked: not in the configured ` + + `hostAllowList [${hostAllowList.join(', ')}].`, + ); + } +} + /** * Fetch all server entries from the MCP Registry using cursor * pagination. Accumulates entries across pages and enforces @@ -226,11 +254,22 @@ export function resolveNextCursor( export async function fetchRegistryServers( options: FetchServersOptions, ): Promise { - const { baseUrl, apiVersion, pageLimit, pageSize, maxEntries, fetchApi } = - options; + const { + baseUrl, + apiVersion, + pageLimit, + pageSize, + maxEntries, + hostAllowList, + fetchApi, + } = options; const doFetch = fetchApi ?? fetch; const endpoint = parseServersEndpointUrl(baseUrl, apiVersion); + // Defense-in-depth: validate endpoint hostname at runtime even + // though config parsing already checked baseUrl against the list. + validateUrlHostAllowList(endpoint, hostAllowList); + const allServers: McpRegistryServerEntry[] = []; const seenCursors = new Set(); let cursor: string | undefined; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts index abae9946ed5..07279024f89 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts @@ -19,11 +19,13 @@ import { assertSingleRegistryConfig, readMaxEntries, readMcpRegistryProviderConfig, + readOptionalHostAllowList, readOptionalPageSize, readPageLimit, readProviderSchedule, readRequiredHttpBaseUrl, safeGetOptionalString, + validateHostAgainstAllowList, } from './config'; describe('readMcpRegistryProviderConfig', () => { @@ -220,6 +222,70 @@ describe('readMcpRegistryProviderConfig', () => { const result = readMcpRegistryProviderConfig(config); expect(result!.apiVersion).toBe('v0'); }); + + it('reads hostAllowList when provided', () => { + const config = new ConfigReader({ + catalog: { + providers: { + mcpRegistry: { + baseUrl: 'https://registry.example.com', + hostAllowList: ['registry.example.com'], + }, + }, + }, + }); + + const result = readMcpRegistryProviderConfig(config); + expect(result!.hostAllowList).toEqual(['registry.example.com']); + }); + + it('returns undefined hostAllowList when omitted', () => { + const config = new ConfigReader({ + catalog: { + providers: { + mcpRegistry: { + baseUrl: 'https://registry.example.com', + }, + }, + }, + }); + + const result = readMcpRegistryProviderConfig(config); + expect(result!.hostAllowList).toBeUndefined(); + }); + + it('throws when baseUrl hostname is not in hostAllowList', () => { + const config = new ConfigReader({ + catalog: { + providers: { + mcpRegistry: { + baseUrl: 'https://registry.example.com', + hostAllowList: ['other.example.com'], + }, + }, + }, + }); + + expect(() => readMcpRegistryProviderConfig(config)).toThrow( + /not in the configured hostAllowList/, + ); + }); + + it('normalizes hostAllowList entries to lowercase', () => { + const config = new ConfigReader({ + catalog: { + providers: { + mcpRegistry: { + baseUrl: 'https://Registry.Example.COM', + hostAllowList: ['REGISTRY.EXAMPLE.COM'], + }, + }, + }, + }); + + const result = readMcpRegistryProviderConfig(config); + expect(result!.hostAllowList).toEqual(['registry.example.com']); + }); }); describe('safeGetOptionalString', () => { @@ -357,6 +423,54 @@ describe('readMaxEntries', () => { }); }); +describe('readOptionalHostAllowList', () => { + it('returns undefined when omitted', () => { + expect(readOptionalHostAllowList(new ConfigReader({}))).toBeUndefined(); + }); + + it('returns undefined for an empty array', () => { + expect( + readOptionalHostAllowList(new ConfigReader({ hostAllowList: [] })), + ).toBeUndefined(); + }); + + it('returns normalized lowercase hostnames', () => { + expect( + readOptionalHostAllowList( + new ConfigReader({ + hostAllowList: ['Registry.Example.COM', 'Other.HOST'], + }), + ), + ).toEqual(['registry.example.com', 'other.host']); + }); +}); + +describe('validateHostAgainstAllowList', () => { + it('passes when hostname is in the allow list', () => { + expect(() => + validateHostAgainstAllowList('https://registry.example.com/path', [ + 'registry.example.com', + ]), + ).not.toThrow(); + }); + + it('throws when hostname is not in the allow list', () => { + expect(() => + validateHostAgainstAllowList('https://evil.example.com', [ + 'registry.example.com', + ]), + ).toThrow(/not in the configured hostAllowList/); + }); + + it('matches case-insensitively', () => { + expect(() => + validateHostAgainstAllowList('https://Registry.Example.COM', [ + 'registry.example.com', + ]), + ).not.toThrow(); + }); +}); + describe('readProviderSchedule', () => { it('returns the default schedule when omitted', () => { expect(readProviderSchedule(new ConfigReader({}))).toEqual({ diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts index 404d2d6cfa6..f5096c6232b 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts @@ -42,6 +42,7 @@ const KNOWN_MCP_REGISTRY_KEYS = new Set([ 'pageLimit', 'pageSize', 'maxEntries', + 'hostAllowList', 'schedule', ]); @@ -184,6 +185,42 @@ export function readOptionalPageSize( return pageSize; } +/** + * Read optional `hostAllowList`, normalizing entries to lowercase. + * + * @internal + */ +export function readOptionalHostAllowList( + registryConfig: Config, +): string[] | undefined { + const list = registryConfig.getOptionalStringArray('hostAllowList'); + if (!list || list.length === 0) { + return undefined; + } + return list.map(h => h.toLowerCase()); +} + +/** + * Validate that a URL's hostname is present in the configured allow list. + * Throws when the hostname is not in the list. + * + * @internal + */ +export function validateHostAgainstAllowList( + url: string, + hostAllowList: string[], +): void { + const parsed = new URL(url); + const hostname = parsed.hostname.toLowerCase(); + if (!hostAllowList.includes(hostname)) { + throw new Error( + `Invalid catalog.providers.mcpRegistry configuration: the hostname ` + + `"${hostname}" from baseUrl "${url}" is not in the configured ` + + `hostAllowList [${hostAllowList.join(', ')}].`, + ); + } +} + /** * Read the provider schedule, or the documented default when omitted. * @@ -219,6 +256,8 @@ export interface McpRegistryProviderConfig { pageSize?: number; /** Maximum total entries accumulated across all pages per sync (default `5000`). */ maxEntries: number; + /** Optional allowlist of permitted hostnames for defense-in-depth SSRF protection. */ + hostAllowList?: string[]; /** Schedule for the sync task. */ schedule: SchedulerServiceTaskScheduleDefinition; } @@ -246,8 +285,15 @@ export function readMcpRegistryProviderConfig( assertSingleRegistryConfig(registryConfig); + const baseUrl = readRequiredHttpBaseUrl(registryConfig); + const hostAllowList = readOptionalHostAllowList(registryConfig); + + if (hostAllowList) { + validateHostAgainstAllowList(baseUrl, hostAllowList); + } + return { - baseUrl: readRequiredHttpBaseUrl(registryConfig), + baseUrl, baseName: safeGetOptionalString(registryConfig, 'baseName'), apiVersion: safeGetOptionalString(registryConfig, 'apiVersion') ?? @@ -256,6 +302,7 @@ export function readMcpRegistryProviderConfig( pageLimit: readPageLimit(registryConfig), pageSize: readOptionalPageSize(registryConfig), maxEntries: readMaxEntries(registryConfig), + hostAllowList, schedule: readProviderSchedule(registryConfig), }; } From 8a0302e34ad5419c7828da97ceeea554b3cde738 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sat, 19 Sep 2026 17:18:05 -0400 Subject: [PATCH 17/63] fix(#4815): add recent config fields to app-config.yaml files Signed-off-by: Michael Valdron --- workspaces/ai-integrations/app-config.yaml | 6 ++++++ .../app-config.yaml | 6 ++++++ 2 files changed, 12 insertions(+) diff --git a/workspaces/ai-integrations/app-config.yaml b/workspaces/ai-integrations/app-config.yaml index bf3ca8d6b85..71cd3145305 100644 --- a/workspaces/ai-integrations/app-config.yaml +++ b/workspaces/ai-integrations/app-config.yaml @@ -152,6 +152,12 @@ catalog: # pageLimit: 10 # Optional: registry page size sent as ?limit= (omitted by default) # pageSize: 50 + # Optional: maximum total entries accumulated across all pages per sync (default: 5000) + # maxEntries: 5000 + # Optional: allowlist of permitted hostnames for defense-in-depth SSRF protection. + # hostAllowList: + # - https://registry.modelcontextprotocol.io + # - https://staging.registry.modelcontextprotocol.io # Optional: sync schedule (defaults shown below) # schedule: # frequency: { minutes: 30 } diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml index a05e061985c..3b75b1bf35e 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml @@ -13,6 +13,12 @@ catalog: # pageLimit: 10 # Optional: registry page size sent as ?limit= (omitted by default) # pageSize: 50 + # Optional: maximum total entries accumulated across all pages per sync (default: 5000) + # maxEntries: 5000 + # Optional: allowlist of permitted hostnames for defense-in-depth SSRF protection. + # hostAllowList: + # - https://registry.modelcontextprotocol.io + # - https://staging.registry.modelcontextprotocol.io # Optional: sync schedule (defaults shown below) # schedule: # frequency: { minutes: 30 } From c9cb07426205d209c523b1c3e4e0469f0cf2da67 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sat, 19 Sep 2026 17:46:03 -0400 Subject: [PATCH 18/63] fix(#4815): use registry baseUrl as the placeholder remote Servers with no valid remotes need a catalog remote that points at the configured MCP Registry, before falling back to websiteUrl. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../README.md | 4 +- .../McpRegistryEntityProvider.parts.test.ts | 62 ++++++++++++++++++- .../src/McpRegistryEntityProvider.ts | 8 ++- 3 files changed, 69 insertions(+), 5 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index 267cd5a6aa9..0873136de6d 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -52,7 +52,7 @@ catalog: | Option | Required | Default | Description | | --------------- | -------- | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `baseUrl` | Yes | — | MCP Registry base URL | +| `baseUrl` | Yes | — | MCP Registry base URL. Also passed to the mapping as `placeholderRemoteUrl` when a server has no valid remotes. | | `baseName` | No | `mcp.registry` (mapping default) | Override the mapping identity prefix for `metadata.name` | | `apiVersion` | No | `v1` | Registry API version slug. The servers endpoint is `//servers`. Note: the live MCP Registry may serve `/v0` or `/v0.1`; set `apiVersion` to match your registry. | | `defaultOwner` | No | `unknown` (mapping default) | Backstage entity reference used as `spec.owner` | @@ -73,7 +73,7 @@ The provider fully traverses the registry's cursor-based pagination, accumulatin ### Mapping -Each server entry's `.server` object is transformed into an `mcp-server` API entity using the [`mcp-registry-server-mapping-common`](../mcp-registry-server-mapping-common) library. The provider passes `defaultOwner` and `baseName` as caller overrides but never reimplements the mapping rules. +Each server entry's `.server` object is transformed into an `mcp-server` API entity using the [`mcp-registry-server-mapping-common`](../mcp-registry-server-mapping-common) library. The provider passes `defaultOwner` and `baseName` as caller overrides, and always passes the configured `baseUrl` as `placeholderRemoteUrl` so a server with no valid remotes gets a placeholder remote for that registry before falling back to `websiteUrl`. It never reimplements the mapping rules. ### Full mutation diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts index 1deee459cf9..fdb2b82ac54 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts @@ -255,12 +255,24 @@ describe('McpRegistryEntityProvider parts', () => { }); describe('buildMappingDefaults', () => { - it('returns an empty object when overrides are omitted', () => { + it('uses baseUrl as placeholderRemoteUrl when other overrides are omitted', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), createMockLogger(), ); - expect(parts(provider).buildMappingDefaults()).toEqual({}); + expect(parts(provider).buildMappingDefaults()).toEqual({ + placeholderRemoteUrl: 'https://registry.example.com', + }); + }); + + it('keeps a trailing slash on the configured baseUrl', () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ baseUrl: 'https://registry.example.com/' }), + createMockLogger(), + ); + expect(parts(provider).buildMappingDefaults()).toEqual({ + placeholderRemoteUrl: 'https://registry.example.com/', + }); }); it('includes owner and prefix when configured', () => { @@ -274,6 +286,7 @@ describe('McpRegistryEntityProvider parts', () => { expect(parts(provider).buildMappingDefaults()).toEqual({ owner: 'group:default/mcp-admins', prefix: 'com.example.registry', + placeholderRemoteUrl: 'https://registry.example.com', }); }); }); @@ -344,6 +357,51 @@ describe('McpRegistryEntityProvider parts', () => { ); }); + it('uses the registry baseUrl as the placeholder remote when remotes are absent', () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + ); + const deferred = parts(provider).mapRegistryEntry( + { + server: createMockServerDoc('io.example/weather', '1.0.0', { + remotes: undefined, + websiteUrl: 'https://website.example.com', + }), + }, + LOCATION, + ); + + expect(deferred.entity.spec).toEqual( + expect.objectContaining({ + remotes: [{ type: 'undefined', url: 'https://registry.example.com' }], + }), + ); + }); + + it('keeps a trailing slash on the placeholder remote url', () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ baseUrl: 'https://registry.example.com/' }), + createMockLogger(), + ); + const deferred = parts(provider).mapRegistryEntry( + { + server: createMockServerDoc('io.example/weather', '1.0.0', { + remotes: [], + }), + }, + LOCATION, + ); + + expect(deferred.entity.spec).toEqual( + expect.objectContaining({ + remotes: [ + { type: 'undefined', url: 'https://registry.example.com/' }, + ], + }), + ); + }); + it('throws when the server document cannot be mapped', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index f7cf7989c6d..91eea38c16c 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -290,9 +290,15 @@ export class McpRegistryEntityProvider implements EntityProvider { /** * Build mapping caller overrides from provider config. + * + * `placeholderRemoteUrl` is the target registry `baseUrl` so a server + * with no valid remotes still gets a D8 placeholder pointing at that + * registry, before the mapping falls back to `websiteUrl`. */ private buildMappingDefaults(): McpServerMappingDefaults { - const mappingDefaults: McpServerMappingDefaults = {}; + const mappingDefaults: McpServerMappingDefaults = { + placeholderRemoteUrl: this.config.baseUrl, + }; if (this.config.defaultOwner) { mappingDefaults.owner = this.config.defaultOwner; } From 226fa3835f1dabf41070f99480ff193a0ca0559c Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sat, 19 Sep 2026 20:59:51 -0400 Subject: [PATCH 19/63] docs(#4815): add non-remote instruction to README Signed-off-by: Michael Valdron --- .../README.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index 0873136de6d..789ace4ce6b 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -93,3 +93,21 @@ Each entity carries: - `redhat.com/rhdh-mcp-registry-sync-status`: `ok` or `degraded` - `modelcontextprotocol.io/name`: the server's canonical name - `modelcontextprotocol.io/version`: the server's version + +## Non-Remote MCP Servers + +MCP servers without a remote deployment (package(s) only or [custom installation](https://github.com/modelcontextprotocol/registry/blob/main/docs/reference/server-json/generic-server-json.md#server-with-custom-installation-path)) can be queried via: `GET /api/catalog/entities?filter=kind=API,spec.type=mcp-server,spec.remotes.type=undefined` + +These MCP server entries have a single remote _placeholder_ field which should **not** be parsed by a client always expecting a remote MCP Server. To filter out non-remote entries, use `POST /api/catalog/entities/by-query` with the following JSON body: + +```json +{ + "query": { + "$all": [ + { "kind": "API" }, + { "spec.type": "mcp-server" }, + { "$not": { "spec.remotes.type": "undefined" } } + ] + } +} +``` From 8bcc6cdb0998edc66bc45dc180a1f28160d52d81 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sat, 19 Sep 2026 21:04:15 -0400 Subject: [PATCH 20/63] docs(#4815): document maxEntries in the provider README Surface the existing sync entry cap in the configuration example and options table so operators can find it alongside the other settings. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../catalog-backend-module-mcp-registry-provider/README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index 789ace4ce6b..d9bea4d422a 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -37,6 +37,8 @@ catalog: # pageLimit: 10 # Optional: registry page size sent as ?limit= (omitted by default) # pageSize: 50 + # Optional: maximum total entries accumulated across all pages per sync (default: 5000) + # maxEntries: 5000 # Optional: restrict outbound requests to specific hostnames (defense-in-depth) # hostAllowList: # - registry.example.com @@ -58,6 +60,7 @@ catalog: | `defaultOwner` | No | `unknown` (mapping default) | Backstage entity reference used as `spec.owner` | | `pageLimit` | No | `10` | Maximum number of pages fetched per sync. The provider fails the sync if the registry has more pages than this limit (to prevent incomplete catalog state). | | `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | +| `maxEntries` | No | `5000` | Maximum total server entries accumulated across all pages per sync. The provider fails the sync if this cap is exceeded. | | `hostAllowList` | No | _(none — all hosts allowed)_ | Array of permitted hostnames. When set, `baseUrl` hostname must be in this list and every outbound request is validated at runtime. Provides defense-in-depth against SSRF. | | `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | From e2e42b3641900e2be35e2faa6408a3669cafc2df Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sat, 19 Sep 2026 23:38:55 -0400 Subject: [PATCH 21/63] fix(#4815): resume pageLimit and soft-stop at maxEntries Large registries can span multiple sync ticks via a saved resume cursor, and hitting maxEntries now commits the buffer with an end cursor instead of aborting the run. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- workspaces/ai-integrations/app-config.yaml | 2 +- .../README.md | 28 +-- .../app-config.yaml | 2 +- .../McpRegistryEntityProvider.parts.test.ts | 187 +++++++++++++++ .../src/McpRegistryEntityProvider.test.ts | 33 +++ .../src/McpRegistryEntityProvider.ts | 83 ++++++- .../src/client.test.ts | 219 +++++++++++++----- .../src/client.ts | 169 +++++++++++--- .../src/config.ts | 14 +- 9 files changed, 628 insertions(+), 109 deletions(-) diff --git a/workspaces/ai-integrations/app-config.yaml b/workspaces/ai-integrations/app-config.yaml index 71cd3145305..91b0388d5aa 100644 --- a/workspaces/ai-integrations/app-config.yaml +++ b/workspaces/ai-integrations/app-config.yaml @@ -152,7 +152,7 @@ catalog: # pageLimit: 10 # Optional: registry page size sent as ?limit= (omitted by default) # pageSize: 50 - # Optional: maximum total entries accumulated across all pages per sync (default: 5000) + # Optional: max entries per complete traversal; soft-stops with end cursor (default: 5000) # maxEntries: 5000 # Optional: allowlist of permitted hostnames for defense-in-depth SSRF protection. # hostAllowList: diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index d9bea4d422a..11973381316 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -37,7 +37,7 @@ catalog: # pageLimit: 10 # Optional: registry page size sent as ?limit= (omitted by default) # pageSize: 50 - # Optional: maximum total entries accumulated across all pages per sync (default: 5000) + # Optional: max entries per complete traversal; soft-stops with end cursor (default: 5000) # maxEntries: 5000 # Optional: restrict outbound requests to specific hostnames (defense-in-depth) # hostAllowList: @@ -52,17 +52,17 @@ catalog: ### Configuration options -| Option | Required | Default | Description | -| --------------- | -------- | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `baseUrl` | Yes | — | MCP Registry base URL. Also passed to the mapping as `placeholderRemoteUrl` when a server has no valid remotes. | -| `baseName` | No | `mcp.registry` (mapping default) | Override the mapping identity prefix for `metadata.name` | -| `apiVersion` | No | `v1` | Registry API version slug. The servers endpoint is `//servers`. Note: the live MCP Registry may serve `/v0` or `/v0.1`; set `apiVersion` to match your registry. | -| `defaultOwner` | No | `unknown` (mapping default) | Backstage entity reference used as `spec.owner` | -| `pageLimit` | No | `10` | Maximum number of pages fetched per sync. The provider fails the sync if the registry has more pages than this limit (to prevent incomplete catalog state). | -| `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | -| `maxEntries` | No | `5000` | Maximum total server entries accumulated across all pages per sync. The provider fails the sync if this cap is exceeded. | -| `hostAllowList` | No | _(none — all hosts allowed)_ | Array of permitted hostnames. When set, `baseUrl` hostname must be in this list and every outbound request is validated at runtime. Provides defense-in-depth against SSRF. | -| `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | +| Option | Required | Default | Description | +| --------------- | -------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `baseUrl` | Yes | — | MCP Registry base URL. Also passed to the mapping as `placeholderRemoteUrl` when a server has no valid remotes. | +| `baseName` | No | `mcp.registry` (mapping default) | Override the mapping identity prefix for `metadata.name` | +| `apiVersion` | No | `v1` | Registry API version slug. The servers endpoint is `//servers`. Note: the live MCP Registry may serve `/v0` or `/v0.1`; set `apiVersion` to match your registry. | +| `defaultOwner` | No | `unknown` (mapping default) | Backstage entity reference used as `spec.owner` | +| `pageLimit` | No | `10` | Maximum number of pages fetched per sync. When more pages remain, the provider saves the cursor and continues on the next sync (no mutation until the registry is fully traversed). | +| `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | +| `maxEntries` | No | `5000` | Maximum total server entries buffered for one complete registry traversal (spans resume syncs). When exceeded, the provider commits the buffer, saves an end cursor, and later traversals stop there until `maxEntries` is patched. | +| `hostAllowList` | No | _(none — all hosts allowed)_ | Array of permitted hostnames. When set, `baseUrl` hostname must be in this list and every outbound request is validated at runtime. Provides defense-in-depth against SSRF. | +| `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | ### Multiple registries @@ -72,7 +72,7 @@ Multiple registries are **not supported** in this implementation. Configuring a ### Pagination -The provider fully traverses the registry's cursor-based pagination, accumulating all server entries. Cursors are treated as opaque strings. The `pageLimit` configuration caps the number of pages fetched per sync — if the registry still has more pages after reaching the limit, the sync fails without committing a mutation, preserving the prior catalog state. +The provider fully traverses the registry's cursor-based pagination, accumulating all server entries. Cursors are treated as opaque strings. The `pageLimit` configuration caps the number of pages fetched **per sync**. If the registry still has more pages after that cap, the provider saves the next cursor, buffers the entries fetched so far, and continues from that cursor on the next scheduled sync — it does **not** commit a mutation until a sync reaches the end of the registry (no `nextCursor`). When a traversal completes, the provider commits a full mutation and the following sync starts from the beginning again. The `maxEntries` configuration caps the total buffered servers for that complete traversal (not per individual sync tick). When the cap is hit, the provider commits the buffered entries, saves that stop point as an **end cursor**, and later full traversals end at that cursor instead of a missing `nextCursor`. Patching `maxEntries` clears the saved end cursor so traversal returns to normal. ### Mapping @@ -80,7 +80,7 @@ Each server entry's `.server` object is transformed into an `mcp-server` API ent ### Full mutation -On each successful sync, the provider commits a **full mutation** — the catalog converges to the registry's current server set. Servers removed from the registry are automatically pruned. +When a registry traversal completes (no remaining `nextCursor`, possibly after several resume syncs), the provider commits a **full mutation** — the catalog converges to the registry's current server set. Servers removed from the registry are automatically pruned. Partial resume ticks do not mutate. ### Error handling diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml index 3b75b1bf35e..c87c0ea8907 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml @@ -13,7 +13,7 @@ catalog: # pageLimit: 10 # Optional: registry page size sent as ?limit= (omitted by default) # pageSize: 50 - # Optional: maximum total entries accumulated across all pages per sync (default: 5000) + # Optional: max entries per complete traversal; soft-stops with end cursor (default: 5000) # maxEntries: 5000 # Optional: allowlist of permitted hostnames for defense-in-depth SSRF protection. # hostAllowList: diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts index fdb2b82ac54..d9a4dfa1b43 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts @@ -209,6 +209,193 @@ describe('McpRegistryEntityProvider parts', () => { ); }); + it('buffers entries and returns undefined when pageLimit leaves more pages', async () => { + const logger = createMockLogger(); + const page1: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('a/one', '1.0.0') }], + metadata: { count: 2, nextCursor: 'cursor-1' }, + }; + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ pageLimit: 1 }), + logger, + { fetchApi: mockFetchForResponses([page1]) }, + ); + + await expect( + parts(provider).fetchRegistryEntries(), + ).resolves.toBeUndefined(); + expect(logger.info).toHaveBeenCalledWith( + expect.stringContaining('will resume from the saved cursor'), + ); + }); + + it('resumes from the saved cursor and returns all buffered entries when complete', async () => { + const page1: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('a/one', '1.0.0') }], + metadata: { count: 2, nextCursor: 'cursor-1' }, + }; + const page2: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('a/two', '2.0.0') }], + metadata: { count: 2 }, + }; + const fetchFn = mockFetchForResponses([page1, page2]); + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ pageLimit: 1 }), + createMockLogger(), + { fetchApi: fetchFn }, + ); + + await expect( + parts(provider).fetchRegistryEntries(), + ).resolves.toBeUndefined(); + + await expect(parts(provider).fetchRegistryEntries()).resolves.toEqual([ + ...page1.servers, + ...page2.servers, + ]); + + const secondUrl = fetchFn.mock.calls[1][0] as string; + expect(secondUrl).toContain('cursor=cursor-1'); + }); + + it('starts from the beginning again after a complete traversal', async () => { + const firstPassPage: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('a/one', '1.0.0') }], + metadata: { count: 1 }, + }; + const secondPassPage: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('a/two', '2.0.0') }], + metadata: { count: 1 }, + }; + const fetchFn = mockFetchForResponses([firstPassPage, secondPassPage]); + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ pageLimit: 1 }), + createMockLogger(), + { fetchApi: fetchFn }, + ); + + await expect(parts(provider).fetchRegistryEntries()).resolves.toEqual( + firstPassPage.servers, + ); + await expect(parts(provider).fetchRegistryEntries()).resolves.toEqual( + secondPassPage.servers, + ); + + const firstUrl = fetchFn.mock.calls[0][0] as string; + const secondUrl = fetchFn.mock.calls[1][0] as string; + expect(firstUrl).not.toContain('cursor='); + expect(secondUrl).not.toContain('cursor='); + }); + + it('commits buffered entries and saves endCursor when maxEntries is hit', async () => { + const logger = createMockLogger(); + const page1: McpRegistryListResponse = { + servers: Array.from({ length: 3 }, (_, i) => ({ + server: createMockServerDoc(`a/s${i}`, '1.0.0'), + })), + metadata: { count: 6, nextCursor: 'cursor-1' }, + }; + const page2: McpRegistryListResponse = { + servers: Array.from({ length: 3 }, (_, i) => ({ + server: createMockServerDoc(`a/s${i + 3}`, '1.0.0'), + })), + metadata: { count: 6, nextCursor: 'cursor-2' }, + }; + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ maxEntries: 4, pageLimit: 10 }), + logger, + { fetchApi: mockFetchForResponses([page1, page2]) }, + ); + + await expect(parts(provider).fetchRegistryEntries()).resolves.toEqual( + page1.servers, + ); + expect(logger.warn).toHaveBeenCalledWith( + expect.stringContaining('reached maxEntries'), + ); + + const state = provider as unknown as { + endCursor?: string; + endCursorMaxEntries?: number; + }; + expect(state.endCursor).toBe('cursor-1'); + expect(state.endCursorMaxEntries).toBe(4); + }); + + it('stops later traversals at the saved endCursor', async () => { + const page1: McpRegistryListResponse = { + servers: Array.from({ length: 3 }, (_, i) => ({ + server: createMockServerDoc(`a/s${i}`, '1.0.0'), + })), + metadata: { count: 6, nextCursor: 'cursor-1' }, + }; + const page2: McpRegistryListResponse = { + servers: Array.from({ length: 3 }, (_, i) => ({ + server: createMockServerDoc(`a/s${i + 3}`, '1.0.0'), + })), + metadata: { count: 6, nextCursor: 'cursor-2' }, + }; + const secondPass: McpRegistryListResponse = { + servers: Array.from({ length: 3 }, (_, i) => ({ + server: createMockServerDoc(`a/s${i}`, '1.0.0'), + })), + metadata: { count: 6, nextCursor: 'cursor-1' }, + }; + const fetchFn = mockFetchForResponses([page1, page2, secondPass]); + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ maxEntries: 4, pageLimit: 10 }), + createMockLogger(), + { fetchApi: fetchFn }, + ); + + await parts(provider).fetchRegistryEntries(); + await expect(parts(provider).fetchRegistryEntries()).resolves.toEqual( + secondPass.servers, + ); + expect(fetchFn).toHaveBeenCalledTimes(3); + }); + + it('clears endCursor when maxEntries is patched', async () => { + const page1: McpRegistryListResponse = { + servers: Array.from({ length: 3 }, (_, i) => ({ + server: createMockServerDoc(`a/s${i}`, '1.0.0'), + })), + metadata: { count: 6, nextCursor: 'cursor-1' }, + }; + const page2: McpRegistryListResponse = { + servers: Array.from({ length: 3 }, (_, i) => ({ + server: createMockServerDoc(`a/s${i + 3}`, '1.0.0'), + })), + metadata: { count: 6, nextCursor: 'cursor-2' }, + }; + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ maxEntries: 4, pageLimit: 10 }), + createMockLogger(), + { fetchApi: mockFetchForResponses([page1, page2]) }, + ); + + await parts(provider).fetchRegistryEntries(); + const state = provider as unknown as { + endCursor?: string; + endCursorMaxEntries?: number; + config: { maxEntries: number }; + }; + expect(state.endCursor).toBe('cursor-1'); + + state.config.maxEntries = 5000; + const fullPage: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('a/only', '1.0.0') }], + metadata: { count: 1 }, + }; + const fetchFn = mockFetchForResponses([fullPage]); + (provider as unknown as { fetchApi?: typeof fetch }).fetchApi = fetchFn; + + await expect(parts(provider).fetchRegistryEntries()).resolves.toEqual( + fullPage.servers, + ); + expect(state.endCursor).toBeUndefined(); + }); + it('logs and returns undefined for McpRegistryClientError', async () => { const logger = createMockLogger(); const fetchFn = jest.fn().mockResolvedValue({ diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts index 891d92bc3b6..3ceb7e43ead 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts @@ -302,6 +302,39 @@ describe('McpRegistryEntityProvider', () => { expect(logger.error).toHaveBeenCalled(); }); + it('resumes pagination across syncs and mutates only when complete', async () => { + const page1: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('io.github.user/one', '1.0.0') }], + metadata: { count: 2, nextCursor: 'cursor-1' }, + }; + const page2: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('io.github.user/two', '2.0.0') }], + metadata: { count: 2 }, + }; + const fetchFn = mockFetchForResponses([page1, page2]); + const connection = createMockConnection(); + const logger = createMockLogger(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ pageLimit: 1 }), + logger, + { fetchApi: fetchFn }, + ); + await provider.connect(connection); + + await provider.run(); + expect(connection.applyMutation).not.toHaveBeenCalled(); + expect(logger.info).toHaveBeenCalledWith( + expect.stringContaining('will resume from the saved cursor'), + ); + + await provider.run(); + expect(connection.applyMutation).toHaveBeenCalledTimes(1); + const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + expect(mutation.entities).toHaveLength(2); + expect(fetchFn.mock.calls[1][0] as string).toContain('cursor=cursor-1'); + }); + it('continues sync when one entry fails mapping', async () => { const body: McpRegistryListResponse = { servers: [ diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index 91eea38c16c..d4f5eb32409 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -112,6 +112,26 @@ export class McpRegistryEntityProvider implements EntityProvider { */ private readonly lastGoodIndex = new Map(); + /** + * Resume state for multi-sync pagination. When a sync hits + * `pageLimit` with more pages remaining, entries fetched so far and + * the next cursor are kept here so the following sync continues + * instead of restarting. Cleared when a traversal reaches the end + * of the registry (no next cursor) or a `maxEntries` soft-stop and a + * full mutation is committed. + */ + private resumeCursor?: string; + private pendingEntries: McpRegistryServerEntry[] = []; + private readonly seenCursors = new Set(); + + /** + * After a `maxEntries` soft-stop, later full traversals end at this + * cursor instead of a missing `nextCursor`. Cleared when + * `maxEntries` is patched (value differs from when it was saved). + */ + private endCursor?: string; + private endCursorMaxEntries?: number; + constructor( config: McpRegistryProviderConfig, logger: LoggerService, @@ -193,8 +213,18 @@ export class McpRegistryEntityProvider implements EntityProvider { } /** - * Fetch registry servers. Returns `undefined` when a client error - * aborts the sync without emitting a mutation. + * Fetch registry servers for this sync tick. + * + * Continues from `resumeCursor` when a prior sync stopped at + * `pageLimit`. Returns `undefined` when a client error aborts the + * tick without a mutation, or when more pages remain (entries are + * buffered until the registry is exhausted or `maxEntries` stops the + * traversal so a full mutation does not prune unfetched servers). + * + * Hitting `maxEntries` commits a full mutation of the buffer, saves + * `endCursor`, and starts the next cycle from the beginning. Later + * full traversals stop at that `endCursor` until `maxEntries` is + * patched. */ private async fetchRegistryEntries(): Promise< McpRegistryServerEntry[] | undefined @@ -207,16 +237,63 @@ export class McpRegistryEntityProvider implements EntityProvider { maxEntries, hostAllowList, } = this.config; + + if ( + this.endCursor !== undefined && + this.endCursorMaxEntries !== maxEntries + ) { + this.logger.info( + `MCP Registry maxEntries changed from ` + + `${this.endCursorMaxEntries} to ${maxEntries}; ` + + `clearing saved endCursor.`, + ); + this.endCursor = undefined; + this.endCursorMaxEntries = undefined; + } + try { - return await fetchRegistryServers({ + const result = await fetchRegistryServers({ baseUrl, apiVersion, pageLimit, pageSize, maxEntries, + priorEntryCount: this.pendingEntries.length, + startCursor: this.resumeCursor, + endCursor: this.endCursor, + seenCursors: this.seenCursors, hostAllowList, fetchApi: this.fetchApi, }); + + this.pendingEntries.push(...result.servers); + + if (result.resumeCursor) { + this.resumeCursor = result.resumeCursor; + this.logger.info( + `MCP Registry sync reached pageLimit (${pageLimit} pages); ` + + `buffered ${this.pendingEntries.length} entries and will ` + + `resume from the saved cursor on the next sync ` + + `(no mutation emitted).`, + ); + return undefined; + } + + if (result.endCursor) { + this.endCursor = result.endCursor; + this.endCursorMaxEntries = maxEntries; + this.logger.warn( + `MCP Registry sync reached maxEntries (${maxEntries}); ` + + `committing ${this.pendingEntries.length} buffered entries ` + + `and saving endCursor for later traversals.`, + ); + } + + const entries = this.pendingEntries; + this.pendingEntries = []; + this.resumeCursor = undefined; + this.seenCursors.clear(); + return entries; } catch (err) { if (err instanceof McpRegistryClientError) { this.logger.error( diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts index 012d74f4e38..b470aeaf81e 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts @@ -94,8 +94,9 @@ describe('fetchRegistryServers', () => { fetchApi: fn, }); - expect(result).toHaveLength(1); - expect(result[0].server.name).toBe('test/server-a'); + expect(result.servers).toHaveLength(1); + expect(result.servers[0].server.name).toBe('test/server-a'); + expect(result.resumeCursor).toBeUndefined(); expect(fn).toHaveBeenCalledTimes(1); // Verify no limit param when pageSize is omitted const calledUrl = fn.mock.calls[0][0] as string; @@ -120,7 +121,8 @@ describe('fetchRegistryServers', () => { fetchApi: fn, }); - expect(result).toHaveLength(2); + expect(result.servers).toHaveLength(2); + expect(result.resumeCursor).toBeUndefined(); expect(fn).toHaveBeenCalledTimes(2); // Second call should include cursor const secondUrl = fn.mock.calls[1][0] as string; @@ -141,7 +143,8 @@ describe('fetchRegistryServers', () => { fetchApi: fn, }); - expect(result).toHaveLength(1); + expect(result.servers).toHaveLength(1); + expect(result.resumeCursor).toBeUndefined(); expect(fn).toHaveBeenCalledTimes(1); }); @@ -159,7 +162,8 @@ describe('fetchRegistryServers', () => { fetchApi: fn, }); - expect(result).toHaveLength(1); + expect(result.servers).toHaveLength(1); + expect(result.resumeCursor).toBeUndefined(); expect(fn).toHaveBeenCalledTimes(1); }); @@ -188,7 +192,7 @@ describe('fetchRegistryServers', () => { expect(secondUrl).toContain('limit=50'); }); - it('trips on default pageLimit of 10 at the 11th page', async () => { + it('returns a resumeCursor when default pageLimit of 10 is reached with more pages', async () => { const pages = Array.from({ length: 10 }, (_, i) => ({ body: { servers: [{ server: createMockServerDoc(`test/server-${i}`, '1.0.0') }], @@ -200,26 +204,19 @@ describe('fetchRegistryServers', () => { })); const fn = mockFetch(pages); - await expect( - fetchRegistryServers({ - baseUrl: 'https://registry.example.com', - apiVersion: 'v1', - pageLimit: 10, - fetchApi: fn, - }), - ).rejects.toThrow(McpRegistryClientError); + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + fetchApi: fn, + }); - await expect( - fetchRegistryServers({ - baseUrl: 'https://registry.example.com', - apiVersion: 'v1', - pageLimit: 10, - fetchApi: mockFetch(pages), - }), - ).rejects.toThrow(/page limit/i); + expect(result.servers).toHaveLength(10); + expect(result.resumeCursor).toBe('cursor-10'); + expect(fn).toHaveBeenCalledTimes(10); }); - it('trips on configured pageLimit', async () => { + it('returns a resumeCursor when configured pageLimit is reached with more pages', async () => { const page1: McpRegistryListResponse = { servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], metadata: { count: 3, nextCursor: 'cursor-1' }, @@ -230,14 +227,42 @@ describe('fetchRegistryServers', () => { }; const fn = mockFetch([{ body: page1 }, { body: page2 }]); - await expect( - fetchRegistryServers({ - baseUrl: 'https://registry.example.com', - apiVersion: 'v1', - pageLimit: 2, - fetchApi: fn, - }), - ).rejects.toThrow(/page limit/i); + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 2, + fetchApi: fn, + }); + + expect(result.servers).toHaveLength(2); + expect(result.resumeCursor).toBe('cursor-2'); + expect(fn).toHaveBeenCalledTimes(2); + }); + + it('resumes from startCursor on a follow-up fetch', async () => { + const page3: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-c', '3.0.0') }], + metadata: { count: 3 }, + }; + const fn = mockFetch([{ body: page3 }]); + const seenCursors = new Set(['cursor-1', 'cursor-2']); + + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 2, + startCursor: 'cursor-2', + seenCursors, + priorEntryCount: 2, + fetchApi: fn, + }); + + expect(result.servers).toHaveLength(1); + expect(result.servers[0].server.name).toBe('test/server-c'); + expect(result.resumeCursor).toBeUndefined(); + expect(fn).toHaveBeenCalledTimes(1); + const calledUrl = fn.mock.calls[0][0] as string; + expect(calledUrl).toContain('cursor=cursor-2'); }); it('detects repeated cursor', async () => { @@ -331,24 +356,54 @@ describe('fetchRegistryServers', () => { expect(secondUrl).toContain(`cursor=${encodeURIComponent(opaqueToken)}`); }); - it('throws when maxEntries cap is exceeded', async () => { + it('soft-stops at maxEntries and returns endCursor without the tipping page', async () => { + const page1: McpRegistryListResponse = { + servers: Array.from({ length: 40 }, (_, i) => ({ + server: createMockServerDoc(`test/server-${i}`, '1.0.0'), + })), + metadata: { count: 100, nextCursor: 'cursor-1' }, + }; + const page2: McpRegistryListResponse = { + servers: Array.from({ length: 40 }, (_, i) => ({ + server: createMockServerDoc(`test/server-${i + 40}`, '1.0.0'), + })), + metadata: { count: 100, nextCursor: 'cursor-2' }, + }; + const fn = mockFetch([{ body: page1 }, { body: page2 }]); + + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + maxEntries: 50, + fetchApi: fn, + }); + + expect(result.servers).toHaveLength(40); + expect(result.resumeCursor).toBeUndefined(); + expect(result.endCursor).toBe('cursor-1'); + expect(fn).toHaveBeenCalledTimes(2); + }); + + it('keeps a single oversized first page and ends at its nextCursor', async () => { const largePage: McpRegistryListResponse = { servers: Array.from({ length: 100 }, (_, i) => ({ server: createMockServerDoc(`test/server-${i}`, '1.0.0'), })), - metadata: { count: 100 }, + metadata: { count: 100, nextCursor: 'cursor-next' }, }; const fn = mockFetch([{ body: largePage }]); - await expect( - fetchRegistryServers({ - baseUrl: 'https://registry.example.com', - apiVersion: 'v1', - pageLimit: 10, - maxEntries: 50, - fetchApi: fn, - }), - ).rejects.toThrow(/maxEntries cap of 50/); + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + maxEntries: 50, + fetchApi: fn, + }); + + expect(result.servers).toHaveLength(100); + expect(result.endCursor).toBe('cursor-next'); }); it('succeeds when hostAllowList includes the baseUrl hostname', async () => { @@ -366,7 +421,7 @@ describe('fetchRegistryServers', () => { fetchApi: fn, }); - expect(result).toHaveLength(1); + expect(result.servers).toHaveLength(1); }); it('throws when hostAllowList does not include the baseUrl hostname', async () => { @@ -402,7 +457,52 @@ describe('fetchRegistryServers', () => { fetchApi: fn, }); - expect(result).toHaveLength(100); + expect(result.servers).toHaveLength(100); + expect(result.resumeCursor).toBeUndefined(); + }); + + it('counts priorEntryCount toward maxEntries soft-stop', async () => { + const page: McpRegistryListResponse = { + servers: Array.from({ length: 10 }, (_, i) => ({ + server: createMockServerDoc(`test/server-${i}`, '1.0.0'), + })), + metadata: { count: 10, nextCursor: 'cursor-x' }, + }; + const fn = mockFetch([{ body: page }]); + + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + maxEntries: 50, + priorEntryCount: 45, + startCursor: 'cursor-prior', + fetchApi: fn, + }); + + expect(result.servers).toHaveLength(0); + expect(result.endCursor).toBe('cursor-prior'); + expect(result.resumeCursor).toBeUndefined(); + }); + + it('stops at a supplied endCursor instead of walking further', async () => { + const page1: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 3, nextCursor: 'cursor-end' }, + }; + const fn = mockFetch([{ body: page1 }]); + + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + endCursor: 'cursor-end', + fetchApi: fn, + }); + + expect(result.servers).toHaveLength(1); + expect(result.resumeCursor).toBeUndefined(); + expect(fn).toHaveBeenCalledTimes(1); }); }); @@ -552,17 +652,26 @@ describe('validateUrlHostAllowList', () => { }); describe('resolveNextCursor', () => { - it('returns undefined when nextCursor is absent or empty', () => { + it('returns complete when nextCursor is absent or empty', () => { const seen = new Set(); - expect(resolveNextCursor(undefined, seen, 1, 10)).toBeUndefined(); - expect(resolveNextCursor(null, seen, 1, 10)).toBeUndefined(); - expect(resolveNextCursor('', seen, 1, 10)).toBeUndefined(); + expect(resolveNextCursor(undefined, seen, 1, 10)).toEqual({ + status: 'complete', + }); + expect(resolveNextCursor(null, seen, 1, 10)).toEqual({ + status: 'complete', + }); + expect(resolveNextCursor('', seen, 1, 10)).toEqual({ + status: 'complete', + }); expect(seen.size).toBe(0); }); - it('returns the cursor and records it when paging continues', () => { + it('returns continue and records the cursor when paging continues', () => { const seen = new Set(); - expect(resolveNextCursor('page-2', seen, 1, 10)).toBe('page-2'); + expect(resolveNextCursor('page-2', seen, 1, 10)).toEqual({ + status: 'continue', + cursor: 'page-2', + }); expect(seen.has('page-2')).toBe(true); }); @@ -573,10 +682,12 @@ describe('resolveNextCursor', () => { ); }); - it('throws when the page limit is exceeded with more pages remaining', () => { + it('returns pageLimitReached when more pages remain at the page cap', () => { const seen = new Set(); - expect(() => resolveNextCursor('page-2', seen, 1, 1)).toThrow( - /exceeded the configured page limit/, - ); + expect(resolveNextCursor('page-2', seen, 1, 1)).toEqual({ + status: 'pageLimitReached', + resumeCursor: 'page-2', + }); + expect(seen.has('page-2')).toBe(true); }); }); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index 80e649a7601..b16a626dc2c 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -70,11 +70,38 @@ export interface FetchServersOptions { pageLimit: number; pageSize?: number; /** - * Maximum total entries accumulated across all pages. When exceeded - * the sync aborts to prevent unbounded memory growth from a - * malfunctioning registry returning oversized pages. + * Maximum total entries buffered across the current registry + * traversal (including prior resume syncs) before a full mutation. + * When exceeded, paging stops gracefully: the tipping page is left + * out (unless it is the only page), and `endCursor` is returned so + * the provider can commit what was buffered and bound later + * traversals. + * + * Counts `priorEntryCount` plus servers fetched in this call. */ maxEntries?: number; + /** + * Entry count already accumulated earlier in the current multi-sync + * traversal. Used with `maxEntries` so the cap spans resume cycles. + */ + priorEntryCount?: number; + /** + * Cursor to resume from after a prior sync hit `pageLimit`. When + * omitted, the first request starts at the beginning of the list. + */ + startCursor?: string; + /** + * When set (after a prior `maxEntries` soft-stop), a traversal that + * starts from the beginning stops when it would advance to this + * cursor, instead of waiting for a missing `nextCursor`. + */ + endCursor?: string; + /** + * Cursors already seen in the current multi-sync traversal. Shared + * across resume cycles so repeated-cursor detection spans syncs. + * Mutated in place as new cursors are observed. + */ + seenCursors?: Set; /** * Optional allowlist of permitted hostnames. When set, every * outbound request URL is validated against this list before @@ -85,6 +112,37 @@ export interface FetchServersOptions { fetchApi?: typeof fetch; } +/** + * Result of one `fetchRegistryServers` call (up to `pageLimit` pages). + * + * @internal + */ +export interface FetchServersResult { + /** Servers fetched during this call. */ + servers: McpRegistryServerEntry[]; + /** + * When set, more pages remain after this call stopped at `pageLimit`. + * The next sync should pass this as `startCursor`. + */ + resumeCursor?: string; + /** + * When set, this call stopped because `maxEntries` was exceeded. + * The provider should commit a full mutation of the buffer and + * remember this cursor as the end bound for later full traversals. + */ + endCursor?: string; +} + +/** + * Outcome of resolving the registry's `nextCursor` for pagination. + * + * @internal + */ +export type ResolveNextCursorResult = + | { status: 'complete' } + | { status: 'continue'; cursor: string } + | { status: 'pageLimitReached'; resumeCursor: string }; + /** * Parse the servers list endpoint into a URL. * @@ -186,8 +244,12 @@ export async function fetchRegistryPage( } /** - * Resolve the next pagination cursor, or `undefined` when paging is done. - * Enforces repeated-cursor and page-limit safeguards. + * Resolve the next pagination cursor for this sync. + * + * Returns `complete` when paging is done, `continue` when another page + * should be fetched in this sync, or `pageLimitReached` when this sync + * should stop and resume from `resumeCursor` on a later sync. + * Enforces repeated-cursor detection (still a hard error). * * @internal */ @@ -196,9 +258,9 @@ export function resolveNextCursor( seenCursors: Set, pagesFetched: number, pageLimit: number, -): string | undefined { +): ResolveNextCursorResult { if (!nextCursor || nextCursor.length === 0) { - return undefined; + return { status: 'complete' }; } if (seenCursors.has(nextCursor)) { @@ -210,15 +272,10 @@ export function resolveNextCursor( seenCursors.add(nextCursor); if (pagesFetched >= pageLimit) { - throw new McpRegistryClientError( - `MCP Registry pagination exceeded the configured page limit ` + - `of ${pageLimit} pages per sync. The registry still has more ` + - `pages (nextCursor present). Increase pageLimit to fetch ` + - `more pages.`, - ); + return { status: 'pageLimitReached', resumeCursor: nextCursor }; } - return nextCursor; + return { status: 'continue', cursor: nextCursor }; } /** @@ -244,66 +301,112 @@ export function validateUrlHostAllowList( } /** - * Fetch all server entries from the MCP Registry using cursor - * pagination. Accumulates entries across pages and enforces - * pagination safeguards (page cap, repeated cursor). + * Fetch server entries from the MCP Registry using cursor pagination. + * + * Fetches at most `pageLimit` pages starting from `startCursor` (or the + * beginning when unset). When more pages remain after the cap, returns + * those pages' servers plus a `resumeCursor` for the next sync instead + * of failing. When `maxEntries` would be exceeded, stops gracefully and + * returns `endCursor` so the provider can commit the buffer. When + * `endCursor` is supplied, paging stops upon reaching that cursor + * instead of requiring a missing `nextCursor`. * * @throws McpRegistryClientError on transport, protocol, or - * pagination-safeguard errors. + * repeated-cursor errors. */ export async function fetchRegistryServers( options: FetchServersOptions, -): Promise { +): Promise { const { baseUrl, apiVersion, pageLimit, pageSize, maxEntries, + priorEntryCount = 0, + startCursor, + endCursor, hostAllowList, fetchApi, } = options; const doFetch = fetchApi ?? fetch; const endpoint = parseServersEndpointUrl(baseUrl, apiVersion); + const seenCursors = options.seenCursors ?? new Set(); // Defense-in-depth: validate endpoint hostname at runtime even // though config parsing already checked baseUrl against the list. validateUrlHostAllowList(endpoint, hostAllowList); const allServers: McpRegistryServerEntry[] = []; - const seenCursors = new Set(); - let cursor: string | undefined; + let cursor: string | undefined = startCursor; let pagesFetched = 0; let hasMorePages = true; + let resumeCursor: string | undefined; + let maxEntriesEndCursor: string | undefined; while (hasMorePages) { + // Bound later traversals after a prior maxEntries soft-stop. + if (endCursor && cursor === endCursor) { + hasMorePages = false; + continue; + } + const url = buildPageRequestUrl(endpoint, cursor, pageSize); const body = await fetchRegistryPage(doFetch, url); allServers.push(...body.servers); pagesFetched += 1; - if (maxEntries !== undefined && allServers.length > maxEntries) { - throw new McpRegistryClientError( - `MCP Registry sync accumulated ${allServers.length} entries, ` + - `exceeding the configured maxEntries cap of ${maxEntries}. ` + - `Aborting sync to prevent unbounded memory growth. ` + - `Increase maxEntries if the registry legitimately contains ` + - `more servers.`, - ); + const totalEntries = priorEntryCount + allServers.length; + if (maxEntries !== undefined && totalEntries > maxEntries) { + const tippedPageSize = body.servers.length; + allServers.splice(allServers.length - tippedPageSize, tippedPageSize); + + if (priorEntryCount + allServers.length === 0) { + // Single page alone exceeds the cap — keep it so a mutation + // can still proceed, and bound later traversals at its next. + allServers.push(...body.servers); + const tippedNext = body.metadata?.nextCursor; + maxEntriesEndCursor = + typeof tippedNext === 'string' && tippedNext.length > 0 + ? tippedNext + : undefined; + } else { + // Exclude the tipping page; end at the cursor used to fetch it. + maxEntriesEndCursor = cursor ?? startCursor; + } + hasMorePages = false; + continue; } - const nextCursor = resolveNextCursor( + const next = resolveNextCursor( body.metadata?.nextCursor, seenCursors, pagesFetched, pageLimit, ); - if (!nextCursor) { + if (next.status === 'complete') { + hasMorePages = false; + continue; + } + if (next.status === 'pageLimitReached') { + if (endCursor && next.resumeCursor === endCursor) { + hasMorePages = false; + continue; + } + resumeCursor = next.resumeCursor; hasMorePages = false; continue; } - cursor = nextCursor; + if (endCursor && next.cursor === endCursor) { + hasMorePages = false; + continue; + } + cursor = next.cursor; } - return allServers; + return { + servers: allServers, + resumeCursor, + endCursor: maxEntriesEndCursor, + }; } diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts index f5096c6232b..d9a4db72156 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts @@ -30,7 +30,7 @@ const DEFAULT_API_VERSION = 'v1'; /** Default page limit (max pages per sync). */ const DEFAULT_PAGE_LIMIT = 10; -/** Default max entries per sync. */ +/** Default max entries per complete registry traversal (full mutation). */ const DEFAULT_MAX_ENTRIES = 5000; /** Supported single-registry config keys under `catalog.providers.mcpRegistry`. */ @@ -152,6 +152,8 @@ export function readPageLimit(registryConfig: Config): number { /** * Read `maxEntries`, applying the default and rejecting values below 1. + * Caps total servers buffered for one complete registry traversal + * (possibly spanning multiple resume syncs) before a full mutation. * * @internal */ @@ -250,11 +252,17 @@ export interface McpRegistryProviderConfig { apiVersion: string; /** Default entity owner ref when the mapping does not supply one. */ defaultOwner?: string; - /** Maximum pages fetched per sync (default `10`). */ + /** Maximum pages fetched per sync (default `10`); excess pages resume next sync. */ pageLimit: number; /** Registry `?limit=` page-size query; omitted from the request when unset. */ pageSize?: number; - /** Maximum total entries accumulated across all pages per sync (default `5000`). */ + /** + * Maximum total entries buffered for one complete registry traversal + * before a full mutation (default `5000`). Spans resume syncs when + * `pageLimit` pauses mid-traversal. When exceeded, the provider + * commits the buffer, saves an end cursor, and later traversals stop + * at that cursor until `maxEntries` is patched. + */ maxEntries: number; /** Optional allowlist of permitted hostnames for defense-in-depth SSRF protection. */ hostAllowList?: string[]; From 1ed5be09962dc0a7acaa28f51b1e733ad1c077b1 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sat, 19 Sep 2026 23:50:52 -0400 Subject: [PATCH 22/63] fix(#4815): add remotesOnly to skip non-remote MCP servers Operators can opt in to ingest only servers with a native remote so package-only and placeholder-remote entries never enter the catalog. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- workspaces/ai-integrations/app-config.yaml | 2 + .../README.md | 7 +- .../app-config.yaml | 2 + .../config.d.ts | 7 ++ .../report.api.md | 1 + .../McpRegistryEntityProvider.parts.test.ts | 93 +++++++++++++++++++ .../src/McpRegistryEntityProvider.test.ts | 1 + .../src/McpRegistryEntityProvider.ts | 42 ++++++++- .../src/config.test.ts | 18 ++++ .../src/config.ts | 21 +++++ 10 files changed, 191 insertions(+), 3 deletions(-) diff --git a/workspaces/ai-integrations/app-config.yaml b/workspaces/ai-integrations/app-config.yaml index 91b0388d5aa..7b3e455bd0a 100644 --- a/workspaces/ai-integrations/app-config.yaml +++ b/workspaces/ai-integrations/app-config.yaml @@ -154,6 +154,8 @@ catalog: # pageSize: 50 # Optional: max entries per complete traversal; soft-stops with end cursor (default: 5000) # maxEntries: 5000 + # Optional: ingest only servers with at least one native remote (default: false) + # remotesOnly: false # Optional: allowlist of permitted hostnames for defense-in-depth SSRF protection. # hostAllowList: # - https://registry.modelcontextprotocol.io diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index 11973381316..e80014768d7 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -39,6 +39,8 @@ catalog: # pageSize: 50 # Optional: max entries per complete traversal; soft-stops with end cursor (default: 5000) # maxEntries: 5000 + # Optional: ingest only servers with at least one native remote (default: false) + # remotesOnly: false # Optional: restrict outbound requests to specific hostnames (defense-in-depth) # hostAllowList: # - registry.example.com @@ -61,6 +63,7 @@ catalog: | `pageLimit` | No | `10` | Maximum number of pages fetched per sync. When more pages remain, the provider saves the cursor and continues on the next sync (no mutation until the registry is fully traversed). | | `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | | `maxEntries` | No | `5000` | Maximum total server entries buffered for one complete registry traversal (spans resume syncs). When exceeded, the provider commits the buffer, saves an end cursor, and later traversals stop there until `maxEntries` is patched. | +| `remotesOnly` | No | `false` | When `true`, skip servers that do not declare at least one native remote (non-empty type and http(s) URL). Package-only and placeholder-remote servers are omitted from the catalog. | | `hostAllowList` | No | _(none — all hosts allowed)_ | Array of permitted hostnames. When set, `baseUrl` hostname must be in this list and every outbound request is validated at runtime. Provides defense-in-depth against SSRF. | | `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | @@ -76,7 +79,7 @@ The provider fully traverses the registry's cursor-based pagination, accumulatin ### Mapping -Each server entry's `.server` object is transformed into an `mcp-server` API entity using the [`mcp-registry-server-mapping-common`](../mcp-registry-server-mapping-common) library. The provider passes `defaultOwner` and `baseName` as caller overrides, and always passes the configured `baseUrl` as `placeholderRemoteUrl` so a server with no valid remotes gets a placeholder remote for that registry before falling back to `websiteUrl`. It never reimplements the mapping rules. +Each server entry's `.server` object is transformed into an `mcp-server` API entity using the [`mcp-registry-server-mapping-common`](../mcp-registry-server-mapping-common) library. The provider passes `defaultOwner` and `baseName` as caller overrides, and always passes the configured `baseUrl` as `placeholderRemoteUrl` so a server with no valid remotes gets a placeholder remote for that registry before falling back to `websiteUrl`. When `remotesOnly` is `true`, servers without a native remote are skipped before mapping. It never reimplements the mapping rules. ### Full mutation @@ -101,7 +104,7 @@ Each entity carries: MCP servers without a remote deployment (package(s) only or [custom installation](https://github.com/modelcontextprotocol/registry/blob/main/docs/reference/server-json/generic-server-json.md#server-with-custom-installation-path)) can be queried via: `GET /api/catalog/entities?filter=kind=API,spec.type=mcp-server,spec.remotes.type=undefined` -These MCP server entries have a single remote _placeholder_ field which should **not** be parsed by a client always expecting a remote MCP Server. To filter out non-remote entries, use `POST /api/catalog/entities/by-query` with the following JSON body: +These MCP server entries have a single remote _placeholder_ field which should **not** be parsed by a client always expecting a remote MCP Server. To avoid ingesting them at all, set `remotesOnly: true` on the provider. To keep them in the catalog but filter them out at query time, use `POST /api/catalog/entities/by-query` with the following JSON body: ```json { diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml index c87c0ea8907..9b26965fa3f 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml @@ -15,6 +15,8 @@ catalog: # pageSize: 50 # Optional: max entries per complete traversal; soft-stops with end cursor (default: 5000) # maxEntries: 5000 + # Optional: ingest only servers with at least one native remote (default: false) + # remotesOnly: false # Optional: allowlist of permitted hostnames for defense-in-depth SSRF protection. # hostAllowList: # - https://registry.modelcontextprotocol.io diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts index 793625edaab..e348cf0d156 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts @@ -33,6 +33,13 @@ export interface Config { pageSize?: number; /** @visibility backend */ maxEntries?: number; + /** + * When true, only ingest servers that declare at least one native + * remote. Package-only / placeholder-remote servers are skipped. + * + * @visibility backend + */ + remotesOnly?: boolean; /** @visibility backend */ hostAllowList?: string[]; /** @visibility backend */ diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md index 249b748da9d..c0f5d5dbd36 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md @@ -36,6 +36,7 @@ export interface McpRegistryProviderConfig { maxEntries: number; pageLimit: number; pageSize?: number; + remotesOnly: boolean; schedule: SchedulerServiceTaskScheduleDefinition; } ``` diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts index d9a4dfa1b43..5236217e68e 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts @@ -22,6 +22,7 @@ import type { McpRegistryListResponse, McpRegistryServerEntry } from './client'; import { buildLastGoodKey, formatMappingFailureMessage, + hasNativeRemote, McpRegistryEntityProvider, readServerIdentity, } from './McpRegistryEntityProvider'; @@ -77,6 +78,7 @@ function createDefaultConfig( apiVersion: 'v1', pageLimit: 10, maxEntries: 5000, + remotesOnly: false, schedule: { frequency: { minutes: 30 }, timeout: { minutes: 3 }, @@ -136,6 +138,40 @@ describe('buildLastGoodKey', () => { }); }); +describe('hasNativeRemote', () => { + it('returns true when a remote has a non-empty type and http(s) URL', () => { + expect( + hasNativeRemote(createMockServerDoc('io.example/weather', '1.0.0')), + ).toBe(true); + }); + + it('returns false when remotes are missing or empty', () => { + expect( + hasNativeRemote( + createMockServerDoc('io.example/weather', '1.0.0', { + remotes: undefined, + }), + ), + ).toBe(false); + expect( + hasNativeRemote( + createMockServerDoc('io.example/weather', '1.0.0', { remotes: [] }), + ), + ).toBe(false); + expect(hasNativeRemote(undefined)).toBe(false); + }); + + it('returns false for invalid remote URLs', () => { + expect( + hasNativeRemote( + createMockServerDoc('io.example/weather', '1.0.0', { + remotes: [{ type: 'streamable-http', url: 'not-a-url' }], + }), + ), + ).toBe(false); + }); +}); + describe('readServerIdentity', () => { it('returns name and version from a valid entry', () => { expect( @@ -679,6 +715,63 @@ describe('McpRegistryEntityProvider parts', () => { expect(result).toEqual({ entities: [], hasDegradedEntries: false }); }); + + it('skips non-remote entries when remotesOnly is true', () => { + const logger = createMockLogger(); + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ remotesOnly: true }), + logger, + ); + + const result = parts(provider).mapRegistryEntries( + [ + { server: createMockServerDoc('remote/server', '1.0.0') }, + { + server: createMockServerDoc('package/only', '1.0.0', { + remotes: undefined, + }), + }, + { + server: createMockServerDoc('empty/remotes', '1.0.0', { + remotes: [], + }), + }, + ], + LOCATION, + ); + + expect(result.hasDegradedEntries).toBe(false); + expect(result.entities).toHaveLength(1); + expect( + result.entities[0].entity.metadata.annotations?.[ + 'modelcontextprotocol.io/name' + ], + ).toBe('remote/server'); + expect(logger.info).toHaveBeenCalledWith( + expect.stringContaining('remotesOnly skipped 2'), + ); + }); + + it('does not filter non-remote entries when remotesOnly is false', () => { + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ remotesOnly: false }), + createMockLogger(), + ); + + const result = parts(provider).mapRegistryEntries( + [ + { server: createMockServerDoc('remote/server', '1.0.0') }, + { + server: createMockServerDoc('package/only', '1.0.0', { + remotes: undefined, + }), + }, + ], + LOCATION, + ); + + expect(result.entities).toHaveLength(2); + }); }); describe('retainLastGoodOnMappingFailure', () => { diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts index 3ceb7e43ead..ed9165cc294 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts @@ -46,6 +46,7 @@ function createDefaultConfig( apiVersion: 'v1', pageLimit: 10, maxEntries: 5000, + remotesOnly: false, schedule: { frequency: { minutes: 30 }, timeout: { minutes: 3 }, diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index d4f5eb32409..9ef67fa9582 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -31,8 +31,12 @@ import type { import { mapServerToEntity, projectAnnotations, + isAllowedUrl, +} from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; +import type { + McpServerMappingDefaults, + McpServerDocument, } from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; -import type { McpServerMappingDefaults } from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; import type { McpRegistryProviderConfig } from './config'; import { fetchRegistryServers, McpRegistryClientError } from './client'; import type { McpRegistryServerEntry } from './client'; @@ -44,6 +48,29 @@ const PROVIDER_NAME = 'mcp-registry-provider'; /** Sync status annotation key. */ const SYNC_STATUS_ANNOTATION = 'redhat.com/rhdh-mcp-registry-sync-status'; +/** + * Whether a server.json document declares at least one native remote + * (non-empty type and D11-valid URL). Matches the mapping's copy rules + * for remotes that become `spec.remotes` rather than D8 placeholders. + * + * @internal + */ +export function hasNativeRemote(doc: McpServerDocument | undefined): boolean { + const remotes = doc?.remotes ?? []; + for (const remote of remotes) { + if ( + typeof remote.type === 'string' && + remote.type.length > 0 && + remote.url !== undefined && + remote.url !== null && + isAllowedUrl(remote.url) + ) { + return true; + } + } + return false; +} + /** * Build a last-good lookup key from name and version. * @@ -307,6 +334,7 @@ export class McpRegistryEntityProvider implements EntityProvider { /** * Map every registry entry with per-entry failure isolation. + * When `remotesOnly` is set, entries without a native remote are skipped. */ private mapRegistryEntries( entries: McpRegistryServerEntry[], @@ -314,8 +342,13 @@ export class McpRegistryEntityProvider implements EntityProvider { ): { entities: DeferredEntity[]; hasDegradedEntries: boolean } { const entities: DeferredEntity[] = []; let hasDegradedEntries = false; + let skippedNonRemote = 0; for (const entry of entries) { + if (this.config.remotesOnly && !hasNativeRemote(entry.server)) { + skippedNonRemote += 1; + continue; + } try { entities.push(this.mapRegistryEntry(entry, managedByLocation)); } catch (err) { @@ -331,6 +364,13 @@ export class McpRegistryEntityProvider implements EntityProvider { } } + if (skippedNonRemote > 0) { + this.logger.info( + `MCP Registry remotesOnly skipped ${skippedNonRemote} ` + + `non-remote server entr${skippedNonRemote === 1 ? 'y' : 'ies'}.`, + ); + } + return { entities, hasDegradedEntries }; } diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts index 07279024f89..5c1fdd3ecf4 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts @@ -23,6 +23,7 @@ import { readOptionalPageSize, readPageLimit, readProviderSchedule, + readRemotesOnly, readRequiredHttpBaseUrl, safeGetOptionalString, validateHostAgainstAllowList, @@ -58,6 +59,7 @@ describe('readMcpRegistryProviderConfig', () => { expect(result!.apiVersion).toBe('v1'); expect(result!.pageLimit).toBe(10); expect(result!.maxEntries).toBe(5000); + expect(result!.remotesOnly).toBe(false); expect(result!.pageSize).toBeUndefined(); expect(result!.baseName).toBeUndefined(); expect(result!.defaultOwner).toBeUndefined(); @@ -423,6 +425,22 @@ describe('readMaxEntries', () => { }); }); +describe('readRemotesOnly', () => { + it('defaults to false when omitted', () => { + expect(readRemotesOnly(new ConfigReader({}))).toBe(false); + }); + + it('returns true when remotesOnly is true', () => { + expect(readRemotesOnly(new ConfigReader({ remotesOnly: true }))).toBe(true); + }); + + it('returns false when remotesOnly is false', () => { + expect(readRemotesOnly(new ConfigReader({ remotesOnly: false }))).toBe( + false, + ); + }); +}); + describe('readOptionalHostAllowList', () => { it('returns undefined when omitted', () => { expect(readOptionalHostAllowList(new ConfigReader({}))).toBeUndefined(); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts index d9a4db72156..71ec8807715 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts @@ -42,6 +42,7 @@ const KNOWN_MCP_REGISTRY_KEYS = new Set([ 'pageLimit', 'pageSize', 'maxEntries', + 'remotesOnly', 'hostAllowList', 'schedule', ]); @@ -223,6 +224,20 @@ export function validateHostAgainstAllowList( } } +/** + * Read `remotesOnly`, defaulting to `false`. + * + * @internal + */ +export function readRemotesOnly(registryConfig: Config): boolean { + try { + return registryConfig.getOptionalBoolean('remotesOnly') ?? false; + } catch { + // ConfigReader can throw TypeError for empty-string env substitution. + return false; + } +} + /** * Read the provider schedule, or the documented default when omitted. * @@ -264,6 +279,11 @@ export interface McpRegistryProviderConfig { * at that cursor until `maxEntries` is patched. */ maxEntries: number; + /** + * When true, only ingest servers with at least one native remote. + * Package-only / placeholder-remote servers are skipped (default `false`). + */ + remotesOnly: boolean; /** Optional allowlist of permitted hostnames for defense-in-depth SSRF protection. */ hostAllowList?: string[]; /** Schedule for the sync task. */ @@ -310,6 +330,7 @@ export function readMcpRegistryProviderConfig( pageLimit: readPageLimit(registryConfig), pageSize: readOptionalPageSize(registryConfig), maxEntries: readMaxEntries(registryConfig), + remotesOnly: readRemotesOnly(registryConfig), hostAllowList, schedule: readProviderSchedule(registryConfig), }; From 4a9b5cb07436a683abb077c592b3d15b8086d19c Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 00:11:26 -0400 Subject: [PATCH 23/63] refactor(#4815): rename mapping package to catalog-mcp-registry-server-mapping Align the mapping library directory, package name, and pluginId with workspace conventions, and set the provider module pluginId to catalog. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- ...p-registry-mapping-common-provider-link.md | 4 ++-- .../README.md | 2 +- .../package.json | 12 ++++------- .../McpRegistryEntityProvider.parts.test.ts | 2 +- .../src/McpRegistryEntityProvider.ts | 6 +++--- .../src/client.ts | 2 +- .../src/testUtils.ts | 2 +- .../.eslintrc.js | 0 .../CHANGELOG.md | 2 +- .../README.md | 10 +++++----- .../docs/server-json-types.md | 0 .../examples/server-json/README.md | 0 .../examples/server-json/npm-oci.server.json | 0 .../examples/server-json/npm.server.json | 0 .../server-json/nuget-positional.server.json | 0 .../examples/server-json/remote.server.json | 0 .../package.json | 11 +++++----- .../report.api.md | 2 +- .../src/annotationProjection.test.ts | 0 .../src/annotationProjection.ts | 0 .../src/identity.test.ts | 0 .../src/identity.ts | 0 .../src/index.ts | 0 .../src/mapServerToEntity.test.ts | 0 .../src/mapServerToEntity.ts | 0 .../src/repository.test.ts | 0 .../src/repository.ts | 0 .../src/types.ts | 0 .../src/urlPolicy.test.ts | 0 .../src/urlPolicy.ts | 0 .../src/util.test.ts | 0 .../src/util.ts | 0 workspaces/ai-integrations/yarn.lock | 20 +++++++++---------- 33 files changed, 35 insertions(+), 40 deletions(-) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/.eslintrc.js (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/CHANGELOG.md (92%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/README.md (92%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/docs/server-json-types.md (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/examples/server-json/README.md (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/examples/server-json/npm-oci.server.json (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/examples/server-json/npm.server.json (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/examples/server-json/nuget-positional.server.json (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/examples/server-json/remote.server.json (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/package.json (73%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/report.api.md (99%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/src/annotationProjection.test.ts (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/src/annotationProjection.ts (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/src/identity.test.ts (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/src/identity.ts (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/src/index.ts (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/src/mapServerToEntity.test.ts (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/src/mapServerToEntity.ts (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/src/repository.test.ts (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/src/repository.ts (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/src/types.ts (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/src/urlPolicy.test.ts (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/src/urlPolicy.ts (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/src/util.test.ts (100%) rename workspaces/ai-integrations/plugins/{mcp-registry-server-mapping-common => catalog-mcp-registry-server-mapping}/src/util.ts (100%) diff --git a/workspaces/ai-integrations/.changeset/mcp-registry-mapping-common-provider-link.md b/workspaces/ai-integrations/.changeset/mcp-registry-mapping-common-provider-link.md index f12e70fafb1..844fe53ad7a 100644 --- a/workspaces/ai-integrations/.changeset/mcp-registry-mapping-common-provider-link.md +++ b/workspaces/ai-integrations/.changeset/mcp-registry-mapping-common-provider-link.md @@ -1,5 +1,5 @@ --- -'@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common': patch +'@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping': minor --- -Document consumption by `catalog-backend-module-mcp-registry-provider` and list that package in `pluginPackages`. +Rename the mapping common library to `catalog-mcp-registry-server-mapping` (package, directory, and `pluginId`) to match workspace naming conventions. diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index e80014768d7..d464f309ea9 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -79,7 +79,7 @@ The provider fully traverses the registry's cursor-based pagination, accumulatin ### Mapping -Each server entry's `.server` object is transformed into an `mcp-server` API entity using the [`mcp-registry-server-mapping-common`](../mcp-registry-server-mapping-common) library. The provider passes `defaultOwner` and `baseName` as caller overrides, and always passes the configured `baseUrl` as `placeholderRemoteUrl` so a server with no valid remotes gets a placeholder remote for that registry before falling back to `websiteUrl`. When `remotesOnly` is `true`, servers without a native remote are skipped before mapping. It never reimplements the mapping rules. +Each server entry's `.server` object is transformed into an `mcp-server` API entity using the [`catalog-mcp-registry-server-mapping`](../catalog-mcp-registry-server-mapping) library. The provider passes `defaultOwner` and `baseName` as caller overrides, and always passes the configured `baseUrl` as `placeholderRemoteUrl` so a server with no valid remotes gets a placeholder remote for that registry before falling back to `websiteUrl`. When `remotesOnly` is `true`, servers without a native remote are skipped before mapping. It never reimplements the mapping rules. ### Full mutation diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json index c6c0da73640..303a7fca8d8 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json @@ -1,6 +1,6 @@ { "name": "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider", - "version": "0.3.0", + "version": "0.1.0", "license": "Apache-2.0", "description": "The mcp-registry-provider backend module for the catalog plugin. Provides the MCP Server API catalog entities from a target MCP Registry.", "main": "src/index.ts", @@ -17,12 +17,8 @@ }, "backstage": { "role": "backend-plugin-module", - "pluginId": "mcp-registry-provider", - "pluginPackage": "@backstage/plugin-catalog-backend", - "pluginPackages": [ - "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider", - "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common" - ] + "pluginId": "catalog", + "pluginPackage": "@backstage/plugin-catalog-backend" }, "scripts": { "start": "backstage-cli package start", @@ -42,7 +38,7 @@ "@backstage/backend-plugin-api": "^1.10.0", "@backstage/catalog-model": "^1.10.0", "@backstage/plugin-catalog-node": "^2.2.4", - "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common": "workspace:^" + "@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping": "workspace:^" }, "devDependencies": { "@backstage/backend-defaults": "^0.17.8", diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts index 5236217e68e..d6c98cfe53d 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts @@ -16,7 +16,7 @@ import type { Entity } from '@backstage/catalog-model'; import type { DeferredEntity } from '@backstage/plugin-catalog-node'; -import type { McpServerMappingDefaults } from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; +import type { McpServerMappingDefaults } from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; import type { McpRegistryProviderConfig } from './config'; import type { McpRegistryListResponse, McpRegistryServerEntry } from './client'; import { diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index 9ef67fa9582..bb1b2cb549b 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -32,11 +32,11 @@ import { mapServerToEntity, projectAnnotations, isAllowedUrl, -} from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; +} from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; import type { McpServerMappingDefaults, McpServerDocument, -} from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; +} from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; import type { McpRegistryProviderConfig } from './config'; import { fetchRegistryServers, McpRegistryClientError } from './client'; import type { McpRegistryServerEntry } from './client'; @@ -496,7 +496,7 @@ export class McpRegistryEntityProvider implements EntityProvider { * * The annotation keys used here ('modelcontextprotocol.io/name' and * 'modelcontextprotocol.io/version') are set by mapServerToEntity in - * mcp-registry-server-mapping-common and correspond to the raw + * catalog-mcp-registry-server-mapping and correspond to the raw * serverDoc.name and serverDoc.version fields used in buildLastGoodKey * during failure recovery. If the mapping library changes these * annotation keys, both this rebuild and the failure recovery path diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index b16a626dc2c..0dddc2405c8 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -14,7 +14,7 @@ * limitations under the License. */ -import type { McpServerDocument } from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; +import type { McpServerDocument } from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; import { stripTrailingSlashes } from './util'; /** Max characters of an error response body included in client errors. */ diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts index b0577ddcfff..30f3919a542 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts @@ -14,7 +14,7 @@ * limitations under the License. */ -import type { McpServerDocument } from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; +import type { McpServerDocument } from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; /** * Create a minimal valid MCP server.json document for testing. diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/.eslintrc.js b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/.eslintrc.js similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/.eslintrc.js rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/.eslintrc.js diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/CHANGELOG.md b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/CHANGELOG.md similarity index 92% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/CHANGELOG.md rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/CHANGELOG.md index b4ef4443927..69ea7212619 100644 --- a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/CHANGELOG.md +++ b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/CHANGELOG.md @@ -1,4 +1,4 @@ -# @red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common +# @red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping ## 0.3.0 diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/README.md b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/README.md similarity index 92% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/README.md rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/README.md index eee0772493c..185e1534846 100644 --- a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/README.md +++ b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/README.md @@ -1,4 +1,4 @@ -# @red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common +# @red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping Deterministic transform from [MCP Registry](https://github.com/modelcontextprotocol/registry) **v1.8.1** @@ -12,7 +12,7 @@ entity provider. ## Install ```bash -yarn add @red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common +yarn add @red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping ``` ## Usage @@ -23,8 +23,8 @@ yarn add @red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-com import { mapServerToEntity, projectAnnotations, -} from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; -import type { McpServerDocument } from '@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common'; +} from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; +import type { McpServerDocument } from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; const doc: McpServerDocument = { $schema: @@ -115,7 +115,7 @@ From the `workspaces/ai-integrations` workspace root: yarn tsc # Unit tests for this package -yarn test -- plugins/mcp-registry-server-mapping-common/src +yarn test -- plugins/catalog-mcp-registry-server-mapping/src # Lint / API report (when public exports change) yarn lint:all diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/docs/server-json-types.md b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/docs/server-json-types.md similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/docs/server-json-types.md rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/docs/server-json-types.md diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/examples/server-json/README.md b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/README.md similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/examples/server-json/README.md rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/README.md diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/examples/server-json/npm-oci.server.json b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/npm-oci.server.json similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/examples/server-json/npm-oci.server.json rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/npm-oci.server.json diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/examples/server-json/npm.server.json b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/npm.server.json similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/examples/server-json/npm.server.json rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/npm.server.json diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/examples/server-json/nuget-positional.server.json b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/nuget-positional.server.json similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/examples/server-json/nuget-positional.server.json rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/nuget-positional.server.json diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/examples/server-json/remote.server.json b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/remote.server.json similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/examples/server-json/remote.server.json rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/remote.server.json diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/package.json similarity index 73% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/package.json index 375613c6f6d..0a46cad41d5 100644 --- a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/package.json +++ b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/package.json @@ -1,5 +1,5 @@ { - "name": "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common", + "name": "@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping", "version": "0.3.0", "license": "Apache-2.0", "description": "Common library that provides a deterministic transformation functions for transforming an MCP Registry server.json into Backstage mcp-server API entity (direct field mapping).", @@ -13,15 +13,14 @@ "repository": { "type": "git", "url": "https://github.com/redhat-developer/rhdh-plugins", - "directory": "workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common" + "directory": "workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping" }, "backstage": { "role": "common-library", - "pluginId": "mcp-registry-provider", - "pluginPackage": "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common", + "pluginId": "catalog-mcp-registry-server-mapping", + "pluginPackage": "@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping", "pluginPackages": [ - "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common", - "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider" + "@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping" ] }, "sideEffects": false, diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/report.api.md b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/report.api.md similarity index 99% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/report.api.md rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/report.api.md index afe12d225e8..1f3edeebf33 100644 --- a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/report.api.md +++ b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/report.api.md @@ -1,4 +1,4 @@ -## API Report File for "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common" +## API Report File for "@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping" > Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/). diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/annotationProjection.test.ts b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/annotationProjection.test.ts similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/annotationProjection.test.ts rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/annotationProjection.test.ts diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/annotationProjection.ts b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/annotationProjection.ts similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/annotationProjection.ts rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/annotationProjection.ts diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/identity.test.ts b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/identity.test.ts similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/identity.test.ts rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/identity.test.ts diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/identity.ts b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/identity.ts similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/identity.ts rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/identity.ts diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/index.ts b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/index.ts similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/index.ts rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/index.ts diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/mapServerToEntity.test.ts b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/mapServerToEntity.test.ts similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/mapServerToEntity.test.ts rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/mapServerToEntity.test.ts diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/mapServerToEntity.ts b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/mapServerToEntity.ts similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/mapServerToEntity.ts rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/mapServerToEntity.ts diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/repository.test.ts b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/repository.test.ts similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/repository.test.ts rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/repository.test.ts diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/repository.ts b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/repository.ts similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/repository.ts rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/repository.ts diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/types.ts b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/types.ts similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/types.ts rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/types.ts diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/urlPolicy.test.ts b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/urlPolicy.test.ts similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/urlPolicy.test.ts rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/urlPolicy.test.ts diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/urlPolicy.ts b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/urlPolicy.ts similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/urlPolicy.ts rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/urlPolicy.ts diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/util.test.ts b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/util.test.ts similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/util.test.ts rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/util.test.ts diff --git a/workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/util.ts b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/util.ts similarity index 100% rename from workspaces/ai-integrations/plugins/mcp-registry-server-mapping-common/src/util.ts rename to workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/src/util.ts diff --git a/workspaces/ai-integrations/yarn.lock b/workspaces/ai-integrations/yarn.lock index 6bb183a2cd7..a7db4f2f478 100644 --- a/workspaces/ai-integrations/yarn.lock +++ b/workspaces/ai-integrations/yarn.lock @@ -9876,7 +9876,7 @@ __metadata: "@backstage/plugin-catalog-backend": "npm:^3.9.0" "@backstage/plugin-catalog-backend-module-ai-model": "npm:^0.1.3" "@backstage/plugin-catalog-node": "npm:^2.2.4" - "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common": "workspace:^" + "@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping": "workspace:^" "@types/supertest": "npm:^2.0.12" supertest: "npm:^6.2.4" languageName: unknown @@ -9904,6 +9904,15 @@ __metadata: languageName: unknown linkType: soft +"@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping@workspace:^, @red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping@workspace:plugins/catalog-mcp-registry-server-mapping": + version: 0.0.0-use.local + resolution: "@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping@workspace:plugins/catalog-mcp-registry-server-mapping" + dependencies: + "@backstage/catalog-model": "npm:^1.10.1" + "@backstage/cli": "npm:^0.36.5" + languageName: unknown + linkType: soft + "@red-hat-developer-hub/backstage-plugin-catalog-model-ai-model-server@workspace:^, @red-hat-developer-hub/backstage-plugin-catalog-model-ai-model-server@workspace:plugins/catalog-model-ai-model-server": version: 0.0.0-use.local resolution: "@red-hat-developer-hub/backstage-plugin-catalog-model-ai-model-server@workspace:plugins/catalog-model-ai-model-server" @@ -10003,15 +10012,6 @@ __metadata: languageName: unknown linkType: soft -"@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common@workspace:^, @red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common@workspace:plugins/mcp-registry-server-mapping-common": - version: 0.0.0-use.local - resolution: "@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common@workspace:plugins/mcp-registry-server-mapping-common" - dependencies: - "@backstage/catalog-model": "npm:^1.10.1" - "@backstage/cli": "npm:^0.36.5" - languageName: unknown - linkType: soft - "@red-hat-developer-hub/backstage-plugin-theme@npm:^0.15.0": version: 0.15.0 resolution: "@red-hat-developer-hub/backstage-plugin-theme@npm:0.15.0" From 402629fbb138b23cf2efd35d15f51fc1daac6a4b Mon Sep 17 00:00:00 2001 From: fullsend-fix <278716306+fullsend-ai-coder[bot]@users.noreply.github.com> Date: Sun, 20 Sep 2026 05:31:23 +0000 Subject: [PATCH 24/63] fix(#4815): snapshot seenCursors on error and validate response.url Snapshot seenCursors before calling fetchRegistryServers and restore the snapshot on McpRegistryClientError so a failed sync does not carry partially mutated cursor history into the next retry. Validate response.url against the hostAllowList after each fetch completes in fetchRegistryPage, preventing SSRF via HTTP redirects to disallowed hosts. Addresses #4871 Assisted-by: Claude Opus 4.6 (anthropic) --- .../src/McpRegistryEntityProvider.test.ts | 71 +++++++++++++++ .../src/McpRegistryEntityProvider.ts | 7 ++ .../src/client.test.ts | 89 +++++++++++++++++++ .../src/client.ts | 9 +- 4 files changed, 175 insertions(+), 1 deletion(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts index ed9165cc294..790f798a97e 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts @@ -678,6 +678,77 @@ describe('McpRegistryEntityProvider', () => { ).toBe('degraded'); }); + it('restores seenCursors on fetch error so the next sync resumes correctly', async () => { + // First sync: page 1 succeeds, page 2 fails mid-pagination + const page1: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('io.github.user/one', '1.0.0') }], + metadata: { count: 3, nextCursor: 'cursor-1' }, + }; + // Second page fails (non-2xx) + const failPage = { + ok: false, + status: 500, + url: '', + json: async () => ({}), + text: async () => 'Internal Server Error', + } as unknown as Response; + // Retry sync: page 1 again, then page 2 succeeds + const retryPage1: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('io.github.user/one', '1.0.0') }], + metadata: { count: 3, nextCursor: 'cursor-1' }, + }; + const retryPage2: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('io.github.user/two', '2.0.0') }], + metadata: { count: 3 }, + }; + + const combinedFetch = jest.fn(); + // First sync: page 1 ok, page 2 fails + combinedFetch.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => page1, + text: async () => JSON.stringify(page1), + } as unknown as Response); + combinedFetch.mockResolvedValueOnce(failPage); + // Retry sync: both pages ok + combinedFetch.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => retryPage1, + text: async () => JSON.stringify(retryPage1), + } as unknown as Response); + combinedFetch.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => retryPage2, + text: async () => JSON.stringify(retryPage2), + } as unknown as Response); + + const connection = createMockConnection(); + const logger = createMockLogger(); + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ pageLimit: 10 }), + logger, + { fetchApi: combinedFetch }, + ); + await provider.connect(connection); + + // First sync: fails mid-pagination (seenCursors should be restored) + await provider.run(); + expect(connection.applyMutation).not.toHaveBeenCalled(); + expect(logger.error).toHaveBeenCalledWith( + expect.stringContaining('sync failed'), + ); + + // Retry sync: should succeed because seenCursors was restored, + // so cursor-1 is not incorrectly marked as seen + await provider.run(); + expect(connection.applyMutation).toHaveBeenCalledTimes(1); + const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + expect(mutation.entities).toHaveLength(2); + }); + it('does not retain degraded entities in lastGoodIndex on subsequent syncs', async () => { const goodBody: McpRegistryListResponse = { servers: [{ server: createMockServerDoc('test/server', '1.0.0') }], diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index bb1b2cb549b..9420590dd1a 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -278,6 +278,7 @@ export class McpRegistryEntityProvider implements EntityProvider { this.endCursorMaxEntries = undefined; } + const seenCursorsSnapshot = new Set(this.seenCursors); try { const result = await fetchRegistryServers({ baseUrl, @@ -323,6 +324,12 @@ export class McpRegistryEntityProvider implements EntityProvider { return entries; } catch (err) { if (err instanceof McpRegistryClientError) { + // Restore seenCursors to pre-call state so the next sync + // does not carry partially mutated cursor history. + this.seenCursors.clear(); + for (const c of seenCursorsSnapshot) { + this.seenCursors.add(c); + } this.logger.error( `MCP Registry sync failed (no mutation emitted): ${err.message}`, ); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts index b470aeaf81e..c1c524ee3a0 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts @@ -441,6 +441,30 @@ describe('fetchRegistryServers', () => { expect(fn).not.toHaveBeenCalled(); }); + it('throws when response.url redirects to a disallowed host', async () => { + const body: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 1 }, + }; + const fn = jest.fn().mockResolvedValueOnce({ + ok: true, + status: 200, + url: 'https://evil.example.com/v1/servers', + json: async () => body, + text: async () => JSON.stringify(body), + } as unknown as Response); + + await expect( + fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + hostAllowList: ['registry.example.com'], + fetchApi: fn, + }), + ).rejects.toThrow(/not in the configured hostAllowList/); + }); + it('does not enforce maxEntries when unset', async () => { const largePage: McpRegistryListResponse = { servers: Array.from({ length: 100 }, (_, i) => ({ @@ -592,6 +616,71 @@ describe('fetchRegistryPage', () => { ).rejects.toThrow(/missing "servers" array/); }); + it('throws when response.url is redirected to a disallowed host', async () => { + const body: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('a/b', '1.0.0') }], + metadata: { count: 1 }, + }; + const doFetch = jest.fn().mockResolvedValue({ + ok: true, + status: 200, + url: 'https://evil.example.com/v1/servers', + json: async () => body, + text: async () => JSON.stringify(body), + } as unknown as Response); + + await expect( + fetchRegistryPage( + doFetch, + new URL('https://registry.example.com/v1/servers'), + ['registry.example.com'], + ), + ).rejects.toThrow(/not in the configured hostAllowList/); + }); + + it('passes when response.url matches the hostAllowList', async () => { + const body: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('a/b', '1.0.0') }], + metadata: { count: 1 }, + }; + const doFetch = jest.fn().mockResolvedValue({ + ok: true, + status: 200, + url: 'https://registry.example.com/v1/servers', + json: async () => body, + text: async () => JSON.stringify(body), + } as unknown as Response); + + await expect( + fetchRegistryPage( + doFetch, + new URL('https://registry.example.com/v1/servers'), + ['registry.example.com'], + ), + ).resolves.toEqual(body); + }); + + it('skips response.url validation when hostAllowList is omitted', async () => { + const body: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('a/b', '1.0.0') }], + metadata: { count: 1 }, + }; + const doFetch = jest.fn().mockResolvedValue({ + ok: true, + status: 200, + url: 'https://any-host.example.com/v1/servers', + json: async () => body, + text: async () => JSON.stringify(body), + } as unknown as Response); + + await expect( + fetchRegistryPage( + doFetch, + new URL('https://registry.example.com/v1/servers'), + ), + ).resolves.toEqual(body); + }); + it('truncates non-2xx response bodies in the error', async () => { const doFetch = jest.fn().mockResolvedValue({ ok: false, diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index 0dddc2405c8..399e8878312 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -205,6 +205,7 @@ export function truncateErrorBody( export async function fetchRegistryPage( doFetch: typeof fetch, url: URL, + hostAllowList?: string[], ): Promise { const requestUrl = url.toString(); @@ -217,6 +218,12 @@ export async function fetchRegistryPage( ); } + // Validate the actual response URL (after any redirects) against + // the hostAllowList to prevent SSRF via redirect. + if (response.url) { + validateUrlHostAllowList(new URL(response.url), hostAllowList); + } + if (!response.ok) { const rawBody = await response.text().catch(() => '(no body)'); throw new McpRegistryClientError( @@ -352,7 +359,7 @@ export async function fetchRegistryServers( } const url = buildPageRequestUrl(endpoint, cursor, pageSize); - const body = await fetchRegistryPage(doFetch, url); + const body = await fetchRegistryPage(doFetch, url, hostAllowList); allServers.push(...body.servers); pagesFetched += 1; From 6941032866a51a972f4d5747d0802f9c7e5fbb64 Mon Sep 17 00:00:00 2001 From: fullsend-fix <278716306+fullsend-ai-coder[bot]@users.noreply.github.com> Date: Sun, 20 Sep 2026 06:54:12 +0000 Subject: [PATCH 25/63] fix(#4815): address review feedback on PR #4871 - Treat explicit empty hostAllowList as 'deny all' by returning an empty array instead of undefined (readOptionalHostAllowList -> readHostAllowList) - Move createMockLogger, createDefaultConfig, and mockFetchForResponses into src/testUtils.ts and import them in each test file - Align host-validation naming: validateHostAgainstAllowList -> validateHostAllowList (config.ts), validateUrlHostAllowList -> validateHostAllowList (client.ts) - Add 'yarn add' install command to README before backend.add code - Reorder formatMappingFailureMessage fragments for natural grammar: 'Failed to map MCP Registry server entry (version "1.0.0"): boom' Addresses #4871 Assisted-by: Claude Opus 4.6 --- .../README.md | 6 +++ .../McpRegistryEntityProvider.parts.test.ts | 54 +++---------------- .../src/McpRegistryEntityProvider.test.ts | 50 +++-------------- .../src/McpRegistryEntityProvider.ts | 2 +- .../src/client.test.ts | 14 ++--- .../src/client.ts | 6 +-- .../src/config.test.ts | 26 ++++----- .../src/config.ts | 14 +++-- .../src/testUtils.ts | 53 ++++++++++++++++++ 9 files changed, 106 insertions(+), 119 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index d464f309ea9..b41ad372166 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -4,6 +4,12 @@ A Backstage catalog backend module that ingests MCP servers from a configured [M ## Installation +Install the package: + +```bash +yarn add @red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider +``` + Add the module to your backend: ```ts diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts index d6c98cfe53d..e66e7074e82 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts @@ -17,7 +17,6 @@ import type { Entity } from '@backstage/catalog-model'; import type { DeferredEntity } from '@backstage/plugin-catalog-node'; import type { McpServerMappingDefaults } from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; -import type { McpRegistryProviderConfig } from './config'; import type { McpRegistryListResponse, McpRegistryServerEntry } from './client'; import { buildLastGoodKey, @@ -26,7 +25,12 @@ import { McpRegistryEntityProvider, readServerIdentity, } from './McpRegistryEntityProvider'; -import { createMockServerDoc } from './testUtils'; +import { + createDefaultConfig, + createMockLogger, + createMockServerDoc, + mockFetchForResponses, +} from './testUtils'; const SYNC_STATUS_ANNOTATION = 'redhat.com/rhdh-mcp-registry-sync-status'; const LOCATION = 'url:https://registry.example.com'; @@ -60,48 +64,6 @@ function parts(provider: McpRegistryEntityProvider): ProviderParts { return provider as unknown as ProviderParts; } -function createMockLogger() { - return { - info: jest.fn(), - warn: jest.fn(), - error: jest.fn(), - debug: jest.fn(), - child: jest.fn().mockReturnThis(), - }; -} - -function createDefaultConfig( - overrides?: Partial, -): McpRegistryProviderConfig { - return { - baseUrl: 'https://registry.example.com', - apiVersion: 'v1', - pageLimit: 10, - maxEntries: 5000, - remotesOnly: false, - schedule: { - frequency: { minutes: 30 }, - timeout: { minutes: 3 }, - }, - ...overrides, - }; -} - -function mockFetchForResponses( - responses: McpRegistryListResponse[], -): jest.Mock { - const fn = jest.fn(); - for (const body of responses) { - fn.mockResolvedValueOnce({ - ok: true, - status: 200, - json: async () => body, - text: async () => JSON.stringify(body), - } as unknown as Response); - } - return fn; -} - function makeDeferred( name: string, version: string, @@ -210,7 +172,7 @@ describe('formatMappingFailureMessage', () => { expect( formatMappingFailureMessage('io.example/weather', '1.0.0', 'boom'), ).toBe( - 'Failed to map MCP Registry server entry "io.example/weather" version "1.0.0": boom', + 'Failed to map MCP Registry server entry "io.example/weather" (version "1.0.0"): boom', ); }); @@ -222,7 +184,7 @@ describe('formatMappingFailureMessage', () => { it('includes only the version when name is missing', () => { expect(formatMappingFailureMessage(undefined, '1.0.0', 'boom')).toBe( - 'Failed to map MCP Registry server entry version "1.0.0": boom', + 'Failed to map MCP Registry server entry (version "1.0.0"): boom', ); }); }); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts index 790f798a97e..170d6ee0b11 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts @@ -15,21 +15,15 @@ */ import { McpRegistryEntityProvider } from './McpRegistryEntityProvider'; -import type { McpRegistryProviderConfig } from './config'; import type { SchedulerServiceTaskRunner } from '@backstage/backend-plugin-api'; import type { EntityProviderConnection } from '@backstage/plugin-catalog-node'; import type { McpRegistryListResponse } from './client'; -import { createMockServerDoc } from './testUtils'; - -function createMockLogger() { - return { - info: jest.fn(), - warn: jest.fn(), - error: jest.fn(), - debug: jest.fn(), - child: jest.fn().mockReturnThis(), - }; -} +import { + createDefaultConfig, + createMockLogger, + createMockServerDoc, + mockFetchForResponses, +} from './testUtils'; function createMockConnection(): EntityProviderConnection { return { @@ -38,38 +32,6 @@ function createMockConnection(): EntityProviderConnection { } as unknown as EntityProviderConnection; } -function createDefaultConfig( - overrides?: Partial, -): McpRegistryProviderConfig { - return { - baseUrl: 'https://registry.example.com', - apiVersion: 'v1', - pageLimit: 10, - maxEntries: 5000, - remotesOnly: false, - schedule: { - frequency: { minutes: 30 }, - timeout: { minutes: 3 }, - }, - ...overrides, - }; -} - -function mockFetchForResponses( - responses: McpRegistryListResponse[], -): jest.Mock { - const fn = jest.fn(); - for (const body of responses) { - fn.mockResolvedValueOnce({ - ok: true, - status: 200, - json: async () => body, - text: async () => JSON.stringify(body), - } as unknown as Response); - } - return fn; -} - describe('McpRegistryEntityProvider', () => { it('returns provider name mcp-registry-provider', () => { const provider = new McpRegistryEntityProvider( diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index 9420590dd1a..cd57460fd3d 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -114,7 +114,7 @@ export function formatMappingFailureMessage( message += ` "${name}"`; } if (version) { - message += ` version "${version}"`; + message += ` (version "${version}")`; } return `${message}: ${err}`; } diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts index c1c524ee3a0..8a2487c38cf 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts @@ -23,7 +23,7 @@ import { parseServersEndpointUrl, resolveNextCursor, truncateErrorBody, - validateUrlHostAllowList, + validateHostAllowList, } from './client'; import type { McpRegistryListResponse } from './client'; import { createMockServerDoc } from './testUtils'; @@ -698,10 +698,10 @@ describe('fetchRegistryPage', () => { }); }); -describe('validateUrlHostAllowList', () => { +describe('validateHostAllowList', () => { it('does nothing when hostAllowList is undefined', () => { expect(() => - validateUrlHostAllowList( + validateHostAllowList( new URL('https://registry.example.com/v1/servers'), undefined, ), @@ -710,7 +710,7 @@ describe('validateUrlHostAllowList', () => { it('passes when hostname is in the allow list', () => { expect(() => - validateUrlHostAllowList( + validateHostAllowList( new URL('https://registry.example.com/v1/servers'), ['registry.example.com'], ), @@ -719,12 +719,12 @@ describe('validateUrlHostAllowList', () => { it('throws McpRegistryClientError when hostname is not in the allow list', () => { expect(() => - validateUrlHostAllowList(new URL('https://evil.example.com/v1/servers'), [ + validateHostAllowList(new URL('https://evil.example.com/v1/servers'), [ 'registry.example.com', ]), ).toThrow(McpRegistryClientError); expect(() => - validateUrlHostAllowList(new URL('https://evil.example.com/v1/servers'), [ + validateHostAllowList(new URL('https://evil.example.com/v1/servers'), [ 'registry.example.com', ]), ).toThrow(/not in the configured hostAllowList/); @@ -732,7 +732,7 @@ describe('validateUrlHostAllowList', () => { it('matches case-insensitively', () => { expect(() => - validateUrlHostAllowList( + validateHostAllowList( new URL('https://Registry.Example.COM/v1/servers'), ['registry.example.com'], ), diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index 399e8878312..8a406342698 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -221,7 +221,7 @@ export async function fetchRegistryPage( // Validate the actual response URL (after any redirects) against // the hostAllowList to prevent SSRF via redirect. if (response.url) { - validateUrlHostAllowList(new URL(response.url), hostAllowList); + validateHostAllowList(new URL(response.url), hostAllowList); } if (!response.ok) { @@ -291,7 +291,7 @@ export function resolveNextCursor( * * @internal */ -export function validateUrlHostAllowList( +export function validateHostAllowList( url: URL, hostAllowList: string[] | undefined, ): void { @@ -342,7 +342,7 @@ export async function fetchRegistryServers( // Defense-in-depth: validate endpoint hostname at runtime even // though config parsing already checked baseUrl against the list. - validateUrlHostAllowList(endpoint, hostAllowList); + validateHostAllowList(endpoint, hostAllowList); const allServers: McpRegistryServerEntry[] = []; let cursor: string | undefined = startCursor; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts index 5c1fdd3ecf4..5898f18e9fa 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts @@ -19,14 +19,14 @@ import { assertSingleRegistryConfig, readMaxEntries, readMcpRegistryProviderConfig, - readOptionalHostAllowList, + readHostAllowList, readOptionalPageSize, readPageLimit, readProviderSchedule, readRemotesOnly, readRequiredHttpBaseUrl, safeGetOptionalString, - validateHostAgainstAllowList, + validateHostAllowList, } from './config'; describe('readMcpRegistryProviderConfig', () => { @@ -441,20 +441,20 @@ describe('readRemotesOnly', () => { }); }); -describe('readOptionalHostAllowList', () => { +describe('readHostAllowList', () => { it('returns undefined when omitted', () => { - expect(readOptionalHostAllowList(new ConfigReader({}))).toBeUndefined(); + expect(readHostAllowList(new ConfigReader({}))).toBeUndefined(); }); - it('returns undefined for an empty array', () => { - expect( - readOptionalHostAllowList(new ConfigReader({ hostAllowList: [] })), - ).toBeUndefined(); + it('returns an empty array for an empty array (deny all)', () => { + expect(readHostAllowList(new ConfigReader({ hostAllowList: [] }))).toEqual( + [], + ); }); it('returns normalized lowercase hostnames', () => { expect( - readOptionalHostAllowList( + readHostAllowList( new ConfigReader({ hostAllowList: ['Registry.Example.COM', 'Other.HOST'], }), @@ -463,10 +463,10 @@ describe('readOptionalHostAllowList', () => { }); }); -describe('validateHostAgainstAllowList', () => { +describe('validateHostAllowList', () => { it('passes when hostname is in the allow list', () => { expect(() => - validateHostAgainstAllowList('https://registry.example.com/path', [ + validateHostAllowList('https://registry.example.com/path', [ 'registry.example.com', ]), ).not.toThrow(); @@ -474,7 +474,7 @@ describe('validateHostAgainstAllowList', () => { it('throws when hostname is not in the allow list', () => { expect(() => - validateHostAgainstAllowList('https://evil.example.com', [ + validateHostAllowList('https://evil.example.com', [ 'registry.example.com', ]), ).toThrow(/not in the configured hostAllowList/); @@ -482,7 +482,7 @@ describe('validateHostAgainstAllowList', () => { it('matches case-insensitively', () => { expect(() => - validateHostAgainstAllowList('https://Registry.Example.COM', [ + validateHostAllowList('https://Registry.Example.COM', [ 'registry.example.com', ]), ).not.toThrow(); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts index 71ec8807715..76cc8adf82b 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts @@ -191,13 +191,17 @@ export function readOptionalPageSize( /** * Read optional `hostAllowList`, normalizing entries to lowercase. * + * Returns `undefined` when the key is absent (no filtering). + * Returns an empty array when configured as `[]` — semantically + * "deny all" (no hostname can pass validation). + * * @internal */ -export function readOptionalHostAllowList( +export function readHostAllowList( registryConfig: Config, ): string[] | undefined { const list = registryConfig.getOptionalStringArray('hostAllowList'); - if (!list || list.length === 0) { + if (!list) { return undefined; } return list.map(h => h.toLowerCase()); @@ -209,7 +213,7 @@ export function readOptionalHostAllowList( * * @internal */ -export function validateHostAgainstAllowList( +export function validateHostAllowList( url: string, hostAllowList: string[], ): void { @@ -314,10 +318,10 @@ export function readMcpRegistryProviderConfig( assertSingleRegistryConfig(registryConfig); const baseUrl = readRequiredHttpBaseUrl(registryConfig); - const hostAllowList = readOptionalHostAllowList(registryConfig); + const hostAllowList = readHostAllowList(registryConfig); if (hostAllowList) { - validateHostAgainstAllowList(baseUrl, hostAllowList); + validateHostAllowList(baseUrl, hostAllowList); } return { diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts index 30f3919a542..0001b8ca79d 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts @@ -15,6 +15,8 @@ */ import type { McpServerDocument } from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; +import type { McpRegistryProviderConfig } from './config'; +import type { McpRegistryListResponse } from './client'; /** * Create a minimal valid MCP server.json document for testing. @@ -39,3 +41,54 @@ export function createMockServerDoc( ...overrides, }; } + +/** + * Create a mock logger with jest spies for all methods. + */ +export function createMockLogger() { + return { + info: jest.fn(), + warn: jest.fn(), + error: jest.fn(), + debug: jest.fn(), + child: jest.fn().mockReturnThis(), + }; +} + +/** + * Create a default McpRegistryProviderConfig for testing. + */ +export function createDefaultConfig( + overrides?: Partial, +): McpRegistryProviderConfig { + return { + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + maxEntries: 5000, + remotesOnly: false, + schedule: { + frequency: { minutes: 30 }, + timeout: { minutes: 3 }, + }, + ...overrides, + }; +} + +/** + * Create a mock fetch that returns the given responses in order. + */ +export function mockFetchForResponses( + responses: McpRegistryListResponse[], +): jest.Mock { + const fn = jest.fn(); + for (const body of responses) { + fn.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => body, + text: async () => JSON.stringify(body), + } as unknown as Response); + } + return fn; +} From 89d2652c1c283d3eb2ba292e57b16aacbcb59b63 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 12:58:42 -0400 Subject: [PATCH 26/63] fix(#4815): make defaulted provider config fields optional Align McpRegistryProviderConfig with config parsing so apiVersion, pageLimit, maxEntries, and remotesOnly can be omitted on direct construction; the provider resolves the same defaults at runtime. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../report.api.md | 8 +-- .../src/McpRegistryEntityProvider.ts | 10 +++- .../src/config.test.ts | 44 +++++++++++++++ .../src/config.ts | 55 ++++++++++++++++--- .../src/testUtils.ts | 6 +- 5 files changed, 103 insertions(+), 20 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md index c0f5d5dbd36..1ca96e577de 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md @@ -28,15 +28,15 @@ export class McpRegistryEntityProvider implements EntityProvider { // @public export interface McpRegistryProviderConfig { - apiVersion: string; + apiVersion?: string; baseName?: string; baseUrl: string; defaultOwner?: string; hostAllowList?: string[]; - maxEntries: number; - pageLimit: number; + maxEntries?: number; + pageLimit?: number; pageSize?: number; - remotesOnly: boolean; + remotesOnly?: boolean; schedule: SchedulerServiceTaskScheduleDefinition; } ``` diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index cd57460fd3d..befc05655ac 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -37,7 +37,11 @@ import type { McpServerMappingDefaults, McpServerDocument, } from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; -import type { McpRegistryProviderConfig } from './config'; +import { + resolveMcpRegistryProviderConfig, + type McpRegistryProviderConfig, + type ResolvedMcpRegistryProviderConfig, +} from './config'; import { fetchRegistryServers, McpRegistryClientError } from './client'; import type { McpRegistryServerEntry } from './client'; import { stripTrailingSlashes } from './util'; @@ -127,7 +131,7 @@ export function formatMappingFailureMessage( */ export class McpRegistryEntityProvider implements EntityProvider { private connection?: EntityProviderConnection; - private readonly config: McpRegistryProviderConfig; + private readonly config: ResolvedMcpRegistryProviderConfig; private readonly logger: LoggerService; private readonly fetchApi?: typeof fetch; private readonly taskRunner?: SchedulerServiceTaskRunner; @@ -169,7 +173,7 @@ export class McpRegistryEntityProvider implements EntityProvider { taskRunner?: SchedulerServiceTaskRunner; }, ) { - this.config = config; + this.config = resolveMcpRegistryProviderConfig(config); this.logger = logger; this.fetchApi = options?.fetchApi; this.taskRunner = options?.taskRunner; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts index 5898f18e9fa..c77ff757dc7 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts @@ -25,6 +25,7 @@ import { readProviderSchedule, readRemotesOnly, readRequiredHttpBaseUrl, + resolveMcpRegistryProviderConfig, safeGetOptionalString, validateHostAllowList, } from './config'; @@ -512,3 +513,46 @@ describe('readProviderSchedule', () => { }); }); }); + +describe('resolveMcpRegistryProviderConfig', () => { + const schedule = { + frequency: { minutes: 30 }, + timeout: { minutes: 3 }, + }; + + it('fills documented defaults when optional fields are omitted', () => { + expect( + resolveMcpRegistryProviderConfig({ + baseUrl: 'https://registry.example.com', + schedule, + }), + ).toEqual({ + baseUrl: 'https://registry.example.com', + schedule, + apiVersion: 'v1', + pageLimit: 10, + maxEntries: 5000, + remotesOnly: false, + }); + }); + + it('preserves explicit overrides', () => { + expect( + resolveMcpRegistryProviderConfig({ + baseUrl: 'https://registry.example.com', + schedule, + apiVersion: 'v0', + pageLimit: 3, + maxEntries: 100, + remotesOnly: true, + }), + ).toEqual({ + baseUrl: 'https://registry.example.com', + schedule, + apiVersion: 'v0', + pageLimit: 3, + maxEntries: 100, + remotesOnly: true, + }); + }); +}); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts index 76cc8adf82b..e9ba90e5981 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts @@ -33,6 +33,9 @@ const DEFAULT_PAGE_LIMIT = 10; /** Default max entries per complete registry traversal (full mutation). */ const DEFAULT_MAX_ENTRIES = 5000; +/** Default remotesOnly when omitted. */ +const DEFAULT_REMOTES_ONLY = false; + /** Supported single-registry config keys under `catalog.providers.mcpRegistry`. */ const KNOWN_MCP_REGISTRY_KEYS = new Set([ 'baseUrl', @@ -235,10 +238,12 @@ export function validateHostAllowList( */ export function readRemotesOnly(registryConfig: Config): boolean { try { - return registryConfig.getOptionalBoolean('remotesOnly') ?? false; + return ( + registryConfig.getOptionalBoolean('remotesOnly') ?? DEFAULT_REMOTES_ONLY + ); } catch { - // ConfigReader can throw TypeError for empty-string env substitution. - return false; + // ConfigReader throws TypeError for empty-string env substitution. + return DEFAULT_REMOTES_ONLY; } } @@ -258,7 +263,9 @@ export function readProviderSchedule( } /** - * Parsed provider configuration. + * Provider configuration. Fields with documented defaults may be omitted + * on direct construction; the entity provider and config reader apply the + * same defaults as app-config parsing. * * @public */ @@ -268,11 +275,11 @@ export interface McpRegistryProviderConfig { /** Optional identity prefix override passed to the mapping transform. */ baseName?: string; /** Registry API version slug used in the endpoint path (default `v1`). */ - apiVersion: string; + apiVersion?: string; /** Default entity owner ref when the mapping does not supply one. */ defaultOwner?: string; /** Maximum pages fetched per sync (default `10`); excess pages resume next sync. */ - pageLimit: number; + pageLimit?: number; /** Registry `?limit=` page-size query; omitted from the request when unset. */ pageSize?: number; /** @@ -282,18 +289,48 @@ export interface McpRegistryProviderConfig { * commits the buffer, saves an end cursor, and later traversals stop * at that cursor until `maxEntries` is patched. */ - maxEntries: number; + maxEntries?: number; /** * When true, only ingest servers with at least one native remote. * Package-only / placeholder-remote servers are skipped (default `false`). */ - remotesOnly: boolean; + remotesOnly?: boolean; /** Optional allowlist of permitted hostnames for defense-in-depth SSRF protection. */ hostAllowList?: string[]; /** Schedule for the sync task. */ schedule: SchedulerServiceTaskScheduleDefinition; } +/** + * {@link McpRegistryProviderConfig} with defaults applied for fields + * that are optional on the public interface. + * + * @internal + */ +export type ResolvedMcpRegistryProviderConfig = McpRegistryProviderConfig & { + apiVersion: string; + pageLimit: number; + maxEntries: number; + remotesOnly: boolean; +}; + +/** + * Apply documented defaults for optional provider config fields. + * + * @internal + */ +export function resolveMcpRegistryProviderConfig( + config: McpRegistryProviderConfig, +): ResolvedMcpRegistryProviderConfig { + return { + ...config, + apiVersion: config.apiVersion ?? DEFAULT_API_VERSION, + pageLimit: config.pageLimit ?? DEFAULT_PAGE_LIMIT, + maxEntries: config.maxEntries ?? DEFAULT_MAX_ENTRIES, + remotesOnly: config.remotesOnly ?? DEFAULT_REMOTES_ONLY, + }; +} + /** * Read and validate the MCP Registry provider configuration from * `catalog.providers.mcpRegistry`. Returns `undefined` when the @@ -304,7 +341,7 @@ export interface McpRegistryProviderConfig { */ export function readMcpRegistryProviderConfig( rootConfig: Config, -): McpRegistryProviderConfig | undefined { +): ResolvedMcpRegistryProviderConfig | undefined { const providersConfig = rootConfig.getOptionalConfig('catalog.providers'); if (!providersConfig) { return undefined; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts index 0001b8ca79d..c8951718ce9 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/testUtils.ts @@ -57,16 +57,14 @@ export function createMockLogger() { /** * Create a default McpRegistryProviderConfig for testing. + * Omits fields that have documented defaults so construction + * matches the app-config path. */ export function createDefaultConfig( overrides?: Partial, ): McpRegistryProviderConfig { return { baseUrl: 'https://registry.example.com', - apiVersion: 'v1', - pageLimit: 10, - maxEntries: 5000, - remotesOnly: false, schedule: { frequency: { minutes: 30 }, timeout: { minutes: 3 }, From 8eb8c48d312a97a0be76ab9a1b3ec461b7f2371a Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 13:10:49 -0400 Subject: [PATCH 27/63] fix(#4815): follow redirects manually with Location validation Use redirect: manual and validate each Location against http(s) and hostAllowList before following, so SSRF via redirect cannot reach a disallowed host. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../README.md | 24 +- .../src/McpRegistryEntityProvider.test.ts | 29 ++ .../src/client.test.ts | 343 ++++++++++++++++-- .../src/client.ts | 114 +++++- .../src/config.test.ts | 20 + 5 files changed, 471 insertions(+), 59 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index b41ad372166..031293aa6b2 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -60,18 +60,18 @@ catalog: ### Configuration options -| Option | Required | Default | Description | -| --------------- | -------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `baseUrl` | Yes | — | MCP Registry base URL. Also passed to the mapping as `placeholderRemoteUrl` when a server has no valid remotes. | -| `baseName` | No | `mcp.registry` (mapping default) | Override the mapping identity prefix for `metadata.name` | -| `apiVersion` | No | `v1` | Registry API version slug. The servers endpoint is `//servers`. Note: the live MCP Registry may serve `/v0` or `/v0.1`; set `apiVersion` to match your registry. | -| `defaultOwner` | No | `unknown` (mapping default) | Backstage entity reference used as `spec.owner` | -| `pageLimit` | No | `10` | Maximum number of pages fetched per sync. When more pages remain, the provider saves the cursor and continues on the next sync (no mutation until the registry is fully traversed). | -| `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | -| `maxEntries` | No | `5000` | Maximum total server entries buffered for one complete registry traversal (spans resume syncs). When exceeded, the provider commits the buffer, saves an end cursor, and later traversals stop there until `maxEntries` is patched. | -| `remotesOnly` | No | `false` | When `true`, skip servers that do not declare at least one native remote (non-empty type and http(s) URL). Package-only and placeholder-remote servers are omitted from the catalog. | -| `hostAllowList` | No | _(none — all hosts allowed)_ | Array of permitted hostnames. When set, `baseUrl` hostname must be in this list and every outbound request is validated at runtime. Provides defense-in-depth against SSRF. | -| `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | +| Option | Required | Default | Description | +| --------------- | -------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `baseUrl` | Yes | — | MCP Registry base URL. Also passed to the mapping as `placeholderRemoteUrl` when a server has no valid remotes. | +| `baseName` | No | `mcp.registry` (mapping default) | Override the mapping identity prefix for `metadata.name` | +| `apiVersion` | No | `v1` | Registry API version slug. The servers endpoint is `//servers`. Note: the live MCP Registry may serve `/v0` or `/v0.1`; set `apiVersion` to match your registry. | +| `defaultOwner` | No | `unknown` (mapping default) | Backstage entity reference used as `spec.owner` | +| `pageLimit` | No | `10` | Maximum number of pages fetched per sync. When more pages remain, the provider saves the cursor and continues on the next sync (no mutation until the registry is fully traversed). | +| `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | +| `maxEntries` | No | `5000` | Maximum total server entries buffered for one complete registry traversal (spans resume syncs). When exceeded, the provider commits the buffer, saves an end cursor, and later traversals stop there until `maxEntries` is patched. | +| `remotesOnly` | No | `false` | When `true`, skip servers that do not declare at least one native remote (non-empty type and http(s) URL). Package-only and placeholder-remote servers are omitted from the catalog. | +| `hostAllowList` | No | _(none — all hosts allowed)_ | Array of permitted hostnames. When set, `baseUrl` hostname must be in this list and every outbound request (including each redirect `Location`) is validated at runtime. Redirects are followed manually so disallowed hosts are never contacted. Provides defense-in-depth against SSRF. | +| `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | ### Multiple registries diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts index 170d6ee0b11..c63e3c5a631 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts @@ -41,6 +41,35 @@ describe('McpRegistryEntityProvider', () => { expect(provider.getProviderName()).toBe('mcp-registry-provider'); }); + it('applies documented defaults when optional config fields are omitted', () => { + const provider = new McpRegistryEntityProvider( + { + baseUrl: 'https://registry.example.com', + schedule: { + frequency: { minutes: 30 }, + timeout: { minutes: 3 }, + }, + }, + createMockLogger(), + ); + + const resolved = ( + provider as unknown as { + config: { + apiVersion: string; + pageLimit: number; + maxEntries: number; + remotesOnly: boolean; + }; + } + ).config; + + expect(resolved.apiVersion).toBe('v1'); + expect(resolved.pageLimit).toBe(10); + expect(resolved.maxEntries).toBe(5000); + expect(resolved.remotesOnly).toBe(false); + }); + it('registers the refresh task from connect after the catalog connection exists', async () => { const body: McpRegistryListResponse = { servers: [], diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts index 8a2487c38cf..02149d2a6af 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts @@ -19,15 +19,29 @@ import { buildServersEndpoint, fetchRegistryPage, fetchRegistryServers, + isRedirectStatus, McpRegistryClientError, parseServersEndpointUrl, resolveNextCursor, + resolveRedirectUrl, truncateErrorBody, validateHostAllowList, + validateRedirectTarget, } from './client'; import type { McpRegistryListResponse } from './client'; import { createMockServerDoc } from './testUtils'; +function mockHeaders(entries: Record = {}): Headers { + return { + get: (name: string) => { + const key = Object.keys(entries).find( + k => k.toLowerCase() === name.toLowerCase(), + ); + return key ? entries[key] : null; + }, + } as Headers; +} + describe('buildServersEndpoint', () => { it('constructs endpoint without trailing slash', () => { expect(buildServersEndpoint('https://registry.example.com', 'v1')).toBe( @@ -441,17 +455,15 @@ describe('fetchRegistryServers', () => { expect(fn).not.toHaveBeenCalled(); }); - it('throws when response.url redirects to a disallowed host', async () => { - const body: McpRegistryListResponse = { - servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], - metadata: { count: 1 }, - }; + it('throws when a redirect Location points to a disallowed host', async () => { const fn = jest.fn().mockResolvedValueOnce({ - ok: true, - status: 200, - url: 'https://evil.example.com/v1/servers', - json: async () => body, - text: async () => JSON.stringify(body), + ok: false, + status: 302, + headers: mockHeaders({ + Location: 'https://evil.example.com/v1/servers', + }), + json: async () => ({}), + text: async () => '', } as unknown as Response); await expect( @@ -463,6 +475,52 @@ describe('fetchRegistryServers', () => { fetchApi: fn, }), ).rejects.toThrow(/not in the configured hostAllowList/); + + expect(fn).toHaveBeenCalledTimes(1); + expect(fn).toHaveBeenCalledWith('https://registry.example.com/v1/servers', { + redirect: 'manual', + }); + }); + + it('follows an allowlisted redirect Location before reading the body', async () => { + const body: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 1 }, + }; + const fn = jest + .fn() + .mockResolvedValueOnce({ + ok: false, + status: 302, + headers: mockHeaders({ + Location: 'https://registry.example.com/v1/servers?redirected=1', + }), + json: async () => ({}), + text: async () => '', + } as unknown as Response) + .mockResolvedValueOnce({ + ok: true, + status: 200, + headers: mockHeaders(), + json: async () => body, + text: async () => JSON.stringify(body), + } as unknown as Response); + + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + hostAllowList: ['registry.example.com'], + fetchApi: fn, + }); + + expect(result.servers).toHaveLength(1); + expect(fn).toHaveBeenCalledTimes(2); + expect(fn).toHaveBeenNthCalledWith( + 2, + 'https://registry.example.com/v1/servers?redirected=1', + { redirect: 'manual' }, + ); }); it('does not enforce maxEntries when unset', async () => { @@ -588,6 +646,7 @@ describe('fetchRegistryPage', () => { const doFetch = jest.fn().mockResolvedValue({ ok: true, status: 200, + headers: mockHeaders(), json: async () => body, text: async () => JSON.stringify(body), } as unknown as Response); @@ -598,12 +657,17 @@ describe('fetchRegistryPage', () => { new URL('https://registry.example.com/v1/servers'), ), ).resolves.toEqual(body); + expect(doFetch).toHaveBeenCalledWith( + 'https://registry.example.com/v1/servers', + { redirect: 'manual' }, + ); }); it('throws when the servers field is missing', async () => { const doFetch = jest.fn().mockResolvedValue({ ok: true, status: 200, + headers: mockHeaders(), json: async () => ({ metadata: {} }), text: async () => '{}', } as unknown as Response); @@ -616,17 +680,15 @@ describe('fetchRegistryPage', () => { ).rejects.toThrow(/missing "servers" array/); }); - it('throws when response.url is redirected to a disallowed host', async () => { - const body: McpRegistryListResponse = { - servers: [{ server: createMockServerDoc('a/b', '1.0.0') }], - metadata: { count: 1 }, - }; + it('throws when redirect Location points to a disallowed host', async () => { const doFetch = jest.fn().mockResolvedValue({ - ok: true, - status: 200, - url: 'https://evil.example.com/v1/servers', - json: async () => body, - text: async () => JSON.stringify(body), + ok: false, + status: 302, + headers: mockHeaders({ + Location: 'https://evil.example.com/v1/servers', + }), + json: async () => ({}), + text: async () => '', } as unknown as Response); await expect( @@ -636,41 +698,189 @@ describe('fetchRegistryPage', () => { ['registry.example.com'], ), ).rejects.toThrow(/not in the configured hostAllowList/); + expect(doFetch).toHaveBeenCalledTimes(1); }); - it('passes when response.url matches the hostAllowList', async () => { + it('follows redirect Location when the target host is allowlisted', async () => { const body: McpRegistryListResponse = { servers: [{ server: createMockServerDoc('a/b', '1.0.0') }], metadata: { count: 1 }, }; + const doFetch = jest + .fn() + .mockResolvedValueOnce({ + ok: false, + status: 301, + headers: mockHeaders({ Location: '/v1/servers-mirror' }), + json: async () => ({}), + text: async () => '', + } as unknown as Response) + .mockResolvedValueOnce({ + ok: true, + status: 200, + headers: mockHeaders(), + json: async () => body, + text: async () => JSON.stringify(body), + } as unknown as Response); + + await expect( + fetchRegistryPage( + doFetch, + new URL('https://registry.example.com/v1/servers'), + ['registry.example.com'], + ), + ).resolves.toEqual(body); + expect(doFetch).toHaveBeenNthCalledWith( + 2, + 'https://registry.example.com/v1/servers-mirror', + { redirect: 'manual' }, + ); + }); + + it('throws when a redirect is missing the Location header', async () => { const doFetch = jest.fn().mockResolvedValue({ - ok: true, - status: 200, - url: 'https://registry.example.com/v1/servers', - json: async () => body, - text: async () => JSON.stringify(body), + ok: false, + status: 302, + headers: mockHeaders(), + json: async () => ({}), + text: async () => '', + } as unknown as Response); + + await expect( + fetchRegistryPage( + doFetch, + new URL('https://registry.example.com/v1/servers'), + ), + ).rejects.toThrow(/without a Location header/); + }); + + it('throws when redirect Location uses a non-http(s) protocol', async () => { + const doFetch = jest.fn().mockResolvedValue({ + ok: false, + status: 302, + headers: mockHeaders({ Location: 'file:///etc/passwd' }), + json: async () => ({}), + text: async () => '', } as unknown as Response); await expect( fetchRegistryPage( doFetch, new URL('https://registry.example.com/v1/servers'), - ['registry.example.com'], + ), + ).rejects.toThrow(/disallowed protocol/); + }); + + it('throws after exceeding the redirect hop limit', async () => { + const doFetch = jest.fn().mockImplementation(async () => ({ + ok: false, + status: 302, + headers: mockHeaders({ + Location: 'https://registry.example.com/v1/servers?next=1', + }), + json: async () => ({}), + text: async () => '', + })); + + await expect( + fetchRegistryPage( + doFetch, + new URL('https://registry.example.com/v1/servers'), + ), + ).rejects.toThrow(/exceeded 10 redirects/); + expect(doFetch).toHaveBeenCalledTimes(11); + }); + + it('follows redirects when hostAllowList is omitted', async () => { + const body: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('a/b', '1.0.0') }], + metadata: { count: 1 }, + }; + const doFetch = jest + .fn() + .mockResolvedValueOnce({ + ok: false, + status: 307, + headers: mockHeaders({ + Location: 'https://any-host.example.com/v1/servers', + }), + json: async () => ({}), + text: async () => '', + } as unknown as Response) + .mockResolvedValueOnce({ + ok: true, + status: 200, + headers: mockHeaders(), + json: async () => body, + text: async () => JSON.stringify(body), + } as unknown as Response); + + await expect( + fetchRegistryPage( + doFetch, + new URL('https://registry.example.com/v1/servers'), ), ).resolves.toEqual(body); }); - it('skips response.url validation when hostAllowList is omitted', async () => { + it('follows a multi-hop redirect chain when every hop is allowlisted', async () => { const body: McpRegistryListResponse = { servers: [{ server: createMockServerDoc('a/b', '1.0.0') }], metadata: { count: 1 }, }; + const doFetch = jest + .fn() + .mockResolvedValueOnce({ + ok: false, + status: 302, + headers: mockHeaders({ Location: '/hop-1' }), + json: async () => ({}), + text: async () => '', + } as unknown as Response) + .mockResolvedValueOnce({ + ok: false, + status: 308, + headers: mockHeaders({ + Location: 'https://registry.example.com/hop-2', + }), + json: async () => ({}), + text: async () => '', + } as unknown as Response) + .mockResolvedValueOnce({ + ok: true, + status: 200, + headers: mockHeaders(), + json: async () => body, + text: async () => JSON.stringify(body), + } as unknown as Response); + + await expect( + fetchRegistryPage( + doFetch, + new URL('https://registry.example.com/v1/servers'), + ['registry.example.com'], + ), + ).resolves.toEqual(body); + expect(doFetch).toHaveBeenCalledTimes(3); + expect(doFetch).toHaveBeenNthCalledWith( + 2, + 'https://registry.example.com/hop-1', + { redirect: 'manual' }, + ); + expect(doFetch).toHaveBeenNthCalledWith( + 3, + 'https://registry.example.com/hop-2', + { redirect: 'manual' }, + ); + }); + + it('throws on an invalid Location without issuing a follow-up request', async () => { const doFetch = jest.fn().mockResolvedValue({ - ok: true, - status: 200, - url: 'https://any-host.example.com/v1/servers', - json: async () => body, - text: async () => JSON.stringify(body), + ok: false, + status: 302, + headers: mockHeaders({ Location: 'http://[' }), + json: async () => ({}), + text: async () => '', } as unknown as Response); await expect( @@ -678,13 +888,15 @@ describe('fetchRegistryPage', () => { doFetch, new URL('https://registry.example.com/v1/servers'), ), - ).resolves.toEqual(body); + ).rejects.toThrow(/invalid redirect Location/); + expect(doFetch).toHaveBeenCalledTimes(1); }); it('truncates non-2xx response bodies in the error', async () => { const doFetch = jest.fn().mockResolvedValue({ ok: false, status: 500, + headers: mockHeaders(), json: async () => ({}), text: async () => 'x'.repeat(300), } as unknown as Response); @@ -698,6 +910,69 @@ describe('fetchRegistryPage', () => { }); }); +describe('redirect helpers', () => { + it('recognizes redirect status codes', () => { + expect(isRedirectStatus(301)).toBe(true); + expect(isRedirectStatus(302)).toBe(true); + expect(isRedirectStatus(303)).toBe(true); + expect(isRedirectStatus(307)).toBe(true); + expect(isRedirectStatus(308)).toBe(true); + expect(isRedirectStatus(200)).toBe(false); + expect(isRedirectStatus(404)).toBe(false); + }); + + it('resolves absolute and relative Location values', () => { + const current = new URL('https://registry.example.com/v1/servers'); + expect( + resolveRedirectUrl(current, 'https://other.example.com/path').toString(), + ).toBe('https://other.example.com/path'); + expect(resolveRedirectUrl(current, '/v2/servers').toString()).toBe( + 'https://registry.example.com/v2/servers', + ); + }); + + it('throws McpRegistryClientError for an invalid Location value', () => { + expect(() => + resolveRedirectUrl( + new URL('https://registry.example.com/v1/servers'), + 'http://[', + ), + ).toThrow(McpRegistryClientError); + expect(() => + resolveRedirectUrl( + new URL('https://registry.example.com/v1/servers'), + 'http://[', + ), + ).toThrow(/invalid redirect Location/); + }); + + it('rejects non-http(s) redirect targets', () => { + expect(() => + validateRedirectTarget(new URL('ftp://registry.example.com/v1/servers')), + ).toThrow(/disallowed protocol/); + }); + + it('allows http(s) redirect targets and enforces hostAllowList', () => { + expect(() => + validateRedirectTarget( + new URL('https://registry.example.com/v1/servers'), + ['registry.example.com'], + ), + ).not.toThrow(); + expect(() => + validateRedirectTarget( + new URL('http://registry.example.com/v1/servers'), + ['registry.example.com'], + ), + ).not.toThrow(); + expect(() => + validateRedirectTarget(new URL('https://evil.example.com/v1/servers'), [ + 'registry.example.com', + ]), + ).toThrow(/not in the configured hostAllowList/); + }); +}); + describe('validateHostAllowList', () => { it('does nothing when hostAllowList is undefined', () => { expect(() => diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index 8a406342698..ec838060c26 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -20,6 +20,12 @@ import { stripTrailingSlashes } from './util'; /** Max characters of an error response body included in client errors. */ const MAX_ERROR_BODY_LENGTH = 256; +/** Max redirect hops followed for a single page request. */ +const MAX_REDIRECTS = 10; + +/** HTTP statuses treated as redirects to follow manually. */ +const REDIRECT_STATUSES = new Set([301, 302, 303, 307, 308]); + /** * A single server entry from the MCP Registry list response. */ @@ -197,9 +203,58 @@ export function truncateErrorBody( return `${rawBody.substring(0, maxLength)}…(truncated)`; } +/** + * Whether an HTTP status code is a redirect this client follows. + * + * @internal + */ +export function isRedirectStatus(status: number): boolean { + return REDIRECT_STATUSES.has(status); +} + +/** + * Resolve a redirect `Location` header against the current request URL. + * + * @internal + */ +export function resolveRedirectUrl(currentUrl: URL, location: string): URL { + try { + return new URL(location, currentUrl); + } catch (err) { + throw new McpRegistryClientError( + `MCP Registry returned an invalid redirect Location "${location}" ` + + `from ${currentUrl}: ${err}`, + ); + } +} + +/** + * Validate a redirect target before following it. + * + * Requires http(s) and, when configured, an allowlisted hostname. + * + * @internal + */ +export function validateRedirectTarget( + targetUrl: URL, + hostAllowList?: string[], +): void { + if (targetUrl.protocol !== 'http:' && targetUrl.protocol !== 'https:') { + throw new McpRegistryClientError( + `MCP Registry redirect to disallowed protocol "${targetUrl.protocol}" ` + + `in "${targetUrl}". Only http and https are permitted.`, + ); + } + validateHostAllowList(targetUrl, hostAllowList); +} + /** * Fetch and validate one registry list page. * + * Uses `redirect: 'manual'` and validates each `Location` header + * against the host allowlist before following, so SSRF via redirect + * cannot reach a disallowed host. + * * @internal */ export async function fetchRegistryPage( @@ -207,23 +262,35 @@ export async function fetchRegistryPage( url: URL, hostAllowList?: string[], ): Promise { - const requestUrl = url.toString(); + let currentUrl = url; + let redirectsFollowed = 0; + let response = await fetchOnce(doFetch, currentUrl, hostAllowList); + + while (isRedirectStatus(response.status)) { + if (redirectsFollowed >= MAX_REDIRECTS) { + throw new McpRegistryClientError( + `MCP Registry exceeded ${MAX_REDIRECTS} redirects starting from ` + + `${url}. Last redirect was from ${currentUrl}.`, + ); + } - let response: Response; - try { - response = await doFetch(requestUrl); - } catch (err) { - throw new McpRegistryClientError( - `Failed to reach MCP Registry at ${requestUrl}: ${err}`, - ); - } + const location = response.headers.get('Location'); + if (!location) { + throw new McpRegistryClientError( + `MCP Registry returned HTTP ${response.status} without a ` + + `Location header from ${currentUrl}.`, + ); + } - // Validate the actual response URL (after any redirects) against - // the hostAllowList to prevent SSRF via redirect. - if (response.url) { - validateHostAllowList(new URL(response.url), hostAllowList); + const nextUrl = resolveRedirectUrl(currentUrl, location); + validateRedirectTarget(nextUrl, hostAllowList); + redirectsFollowed += 1; + currentUrl = nextUrl; + response = await fetchOnce(doFetch, currentUrl, hostAllowList); } + const requestUrl = currentUrl.toString(); + if (!response.ok) { const rawBody = await response.text().catch(() => '(no body)'); throw new McpRegistryClientError( @@ -250,6 +317,27 @@ export async function fetchRegistryPage( return body; } +/** + * Perform one allowlist-checked fetch with `redirect: 'manual'`. + * + * @internal + */ +async function fetchOnce( + doFetch: typeof fetch, + url: URL, + hostAllowList?: string[], +): Promise { + const requestUrl = url.toString(); + validateHostAllowList(url, hostAllowList); + try { + return await doFetch(requestUrl, { redirect: 'manual' }); + } catch (err) { + throw new McpRegistryClientError( + `Failed to reach MCP Registry at ${requestUrl}: ${err}`, + ); + } +} + /** * Resolve the next pagination cursor for this sync. * diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts index c77ff757dc7..f5d31490861 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts @@ -555,4 +555,24 @@ describe('resolveMcpRegistryProviderConfig', () => { remotesOnly: true, }); }); + + it('treats explicit undefined the same as omitted for defaulted fields', () => { + expect( + resolveMcpRegistryProviderConfig({ + baseUrl: 'https://registry.example.com', + schedule, + apiVersion: undefined, + pageLimit: undefined, + maxEntries: undefined, + remotesOnly: undefined, + }), + ).toEqual({ + baseUrl: 'https://registry.example.com', + schedule, + apiVersion: 'v1', + pageLimit: 10, + maxEntries: 5000, + remotesOnly: false, + }); + }); }); From 19f4e66ee897f496cf80aab1bc069bdabbf40449 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 13:15:16 -0400 Subject: [PATCH 28/63] refactor(#4815): rename client host allowlist guard Rename validateHostAllowList to assertRequestHostAllowed in the client so the runtime request guard is distinct from config-time validation. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../src/client.test.ts | 14 +++++++------- .../src/client.ts | 11 ++++++----- 2 files changed, 13 insertions(+), 12 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts index 02149d2a6af..1965789b55e 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts @@ -25,7 +25,7 @@ import { resolveNextCursor, resolveRedirectUrl, truncateErrorBody, - validateHostAllowList, + assertRequestHostAllowed, validateRedirectTarget, } from './client'; import type { McpRegistryListResponse } from './client'; @@ -973,10 +973,10 @@ describe('redirect helpers', () => { }); }); -describe('validateHostAllowList', () => { +describe('assertRequestHostAllowed', () => { it('does nothing when hostAllowList is undefined', () => { expect(() => - validateHostAllowList( + assertRequestHostAllowed( new URL('https://registry.example.com/v1/servers'), undefined, ), @@ -985,7 +985,7 @@ describe('validateHostAllowList', () => { it('passes when hostname is in the allow list', () => { expect(() => - validateHostAllowList( + assertRequestHostAllowed( new URL('https://registry.example.com/v1/servers'), ['registry.example.com'], ), @@ -994,12 +994,12 @@ describe('validateHostAllowList', () => { it('throws McpRegistryClientError when hostname is not in the allow list', () => { expect(() => - validateHostAllowList(new URL('https://evil.example.com/v1/servers'), [ + assertRequestHostAllowed(new URL('https://evil.example.com/v1/servers'), [ 'registry.example.com', ]), ).toThrow(McpRegistryClientError); expect(() => - validateHostAllowList(new URL('https://evil.example.com/v1/servers'), [ + assertRequestHostAllowed(new URL('https://evil.example.com/v1/servers'), [ 'registry.example.com', ]), ).toThrow(/not in the configured hostAllowList/); @@ -1007,7 +1007,7 @@ describe('validateHostAllowList', () => { it('matches case-insensitively', () => { expect(() => - validateHostAllowList( + assertRequestHostAllowed( new URL('https://Registry.Example.COM/v1/servers'), ['registry.example.com'], ), diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index ec838060c26..560eb9772b0 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -245,7 +245,7 @@ export function validateRedirectTarget( `in "${targetUrl}". Only http and https are permitted.`, ); } - validateHostAllowList(targetUrl, hostAllowList); + assertRequestHostAllowed(targetUrl, hostAllowList); } /** @@ -328,7 +328,7 @@ async function fetchOnce( hostAllowList?: string[], ): Promise { const requestUrl = url.toString(); - validateHostAllowList(url, hostAllowList); + assertRequestHostAllowed(url, hostAllowList); try { return await doFetch(requestUrl, { redirect: 'manual' }); } catch (err) { @@ -374,12 +374,13 @@ export function resolveNextCursor( } /** - * Validate that a URL's hostname is present in the configured allow list. + * Runtime request guard: assert a URL's hostname is on the configured + * allow list before issuing (or following) an outbound fetch. * Throws McpRegistryClientError when the hostname is not permitted. * * @internal */ -export function validateHostAllowList( +export function assertRequestHostAllowed( url: URL, hostAllowList: string[] | undefined, ): void { @@ -430,7 +431,7 @@ export async function fetchRegistryServers( // Defense-in-depth: validate endpoint hostname at runtime even // though config parsing already checked baseUrl against the list. - validateHostAllowList(endpoint, hostAllowList); + assertRequestHostAllowed(endpoint, hostAllowList); const allServers: McpRegistryServerEntry[] = []; let cursor: string | undefined = startCursor; From e316c85ceb94fcbfb08a21a1f4a07dc5a6cb7c3b Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 13:20:19 -0400 Subject: [PATCH 29/63] docs(#4815): use bare hostnames in hostAllowList examples Align app-config examples with runtime hostname matching so operators do not copy full URLs that would never pass the allowlist. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- workspaces/ai-integrations/app-config.yaml | 4 ++-- .../app-config.yaml | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/workspaces/ai-integrations/app-config.yaml b/workspaces/ai-integrations/app-config.yaml index 7b3e455bd0a..67b40254928 100644 --- a/workspaces/ai-integrations/app-config.yaml +++ b/workspaces/ai-integrations/app-config.yaml @@ -158,8 +158,8 @@ catalog: # remotesOnly: false # Optional: allowlist of permitted hostnames for defense-in-depth SSRF protection. # hostAllowList: - # - https://registry.modelcontextprotocol.io - # - https://staging.registry.modelcontextprotocol.io + # - registry.modelcontextprotocol.io + # - staging.registry.modelcontextprotocol.io # Optional: sync schedule (defaults shown below) # schedule: # frequency: { minutes: 30 } diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml index 9b26965fa3f..2a4d016937e 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml @@ -19,8 +19,8 @@ catalog: # remotesOnly: false # Optional: allowlist of permitted hostnames for defense-in-depth SSRF protection. # hostAllowList: - # - https://registry.modelcontextprotocol.io - # - https://staging.registry.modelcontextprotocol.io + # - registry.modelcontextprotocol.io + # - staging.registry.modelcontextprotocol.io # Optional: sync schedule (defaults shown below) # schedule: # frequency: { minutes: 30 } From 43e2a5330eeb46b046fcce6f51e8e6a334bf676e Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 13:26:18 -0400 Subject: [PATCH 30/63] refactor(#4815): extract provider helpers to providerUtils Move hasNativeRemote, buildLastGoodKey, readServerIdentity, and formatMappingFailureMessage out of the entity provider, and split their unit tests into providerUtils.test.ts. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../McpRegistryEntityProvider.parts.test.ts | 106 +--------------- .../src/McpRegistryEntityProvider.ts | 83 +----------- .../src/providerUtils.test.ts | 120 ++++++++++++++++++ .../src/providerUtils.ts | 90 +++++++++++++ 4 files changed, 219 insertions(+), 180 deletions(-) create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/providerUtils.test.ts create mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/providerUtils.ts diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts index e66e7074e82..b9f25516fe4 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts @@ -18,13 +18,8 @@ import type { Entity } from '@backstage/catalog-model'; import type { DeferredEntity } from '@backstage/plugin-catalog-node'; import type { McpServerMappingDefaults } from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; import type { McpRegistryListResponse, McpRegistryServerEntry } from './client'; -import { - buildLastGoodKey, - formatMappingFailureMessage, - hasNativeRemote, - McpRegistryEntityProvider, - readServerIdentity, -} from './McpRegistryEntityProvider'; +import { McpRegistryEntityProvider } from './McpRegistryEntityProvider'; +import { buildLastGoodKey } from './providerUtils'; import { createDefaultConfig, createMockLogger, @@ -92,103 +87,6 @@ function makeDeferred( }; } -describe('buildLastGoodKey', () => { - it('joins name and version with a double-colon separator', () => { - expect(buildLastGoodKey('io.example/weather', '1.0.0')).toBe( - 'io.example/weather::1.0.0', - ); - }); -}); - -describe('hasNativeRemote', () => { - it('returns true when a remote has a non-empty type and http(s) URL', () => { - expect( - hasNativeRemote(createMockServerDoc('io.example/weather', '1.0.0')), - ).toBe(true); - }); - - it('returns false when remotes are missing or empty', () => { - expect( - hasNativeRemote( - createMockServerDoc('io.example/weather', '1.0.0', { - remotes: undefined, - }), - ), - ).toBe(false); - expect( - hasNativeRemote( - createMockServerDoc('io.example/weather', '1.0.0', { remotes: [] }), - ), - ).toBe(false); - expect(hasNativeRemote(undefined)).toBe(false); - }); - - it('returns false for invalid remote URLs', () => { - expect( - hasNativeRemote( - createMockServerDoc('io.example/weather', '1.0.0', { - remotes: [{ type: 'streamable-http', url: 'not-a-url' }], - }), - ), - ).toBe(false); - }); -}); - -describe('readServerIdentity', () => { - it('returns name and version from a valid entry', () => { - expect( - readServerIdentity({ - server: createMockServerDoc('io.example/weather', '1.2.3'), - }), - ).toEqual({ name: 'io.example/weather', version: '1.2.3' }); - }); - - it('returns undefined fields for null or undefined entries', () => { - expect(readServerIdentity(null)).toEqual({ - name: undefined, - version: undefined, - }); - expect(readServerIdentity(undefined)).toEqual({ - name: undefined, - version: undefined, - }); - }); - - it('ignores non-string name and version values', () => { - expect( - readServerIdentity({ - server: { - ...createMockServerDoc('io.example/weather', '1.0.0'), - name: 42 as unknown as string, - version: { n: 1 } as unknown as string, - }, - }), - ).toEqual({ name: undefined, version: undefined }); - }); -}); - -describe('formatMappingFailureMessage', () => { - it('includes name and version when both are present', () => { - expect( - formatMappingFailureMessage('io.example/weather', '1.0.0', 'boom'), - ).toBe( - 'Failed to map MCP Registry server entry "io.example/weather" (version "1.0.0"): boom', - ); - }); - - it('omits missing name and version segments', () => { - expect(formatMappingFailureMessage(undefined, undefined, 'boom')).toBe( - 'Failed to map MCP Registry server entry: boom', - ); - }); - - it('includes only the version when name is missing', () => { - expect(formatMappingFailureMessage(undefined, '1.0.0', 'boom')).toBe( - 'Failed to map MCP Registry server entry (version "1.0.0"): boom', - ); - }); -}); - describe('McpRegistryEntityProvider parts', () => { describe('fetchRegistryEntries', () => { it('returns the registry server list on success', async () => { diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index befc05655ac..ffff25d8dae 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -31,12 +31,8 @@ import type { import { mapServerToEntity, projectAnnotations, - isAllowedUrl, -} from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; -import type { - McpServerMappingDefaults, - McpServerDocument, } from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; +import type { McpServerMappingDefaults } from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; import { resolveMcpRegistryProviderConfig, type McpRegistryProviderConfig, @@ -44,6 +40,12 @@ import { } from './config'; import { fetchRegistryServers, McpRegistryClientError } from './client'; import type { McpRegistryServerEntry } from './client'; +import { + buildLastGoodKey, + formatMappingFailureMessage, + hasNativeRemote, + readServerIdentity, +} from './providerUtils'; import { stripTrailingSlashes } from './util'; /** Provider name and locationKey constant. */ @@ -52,77 +54,6 @@ const PROVIDER_NAME = 'mcp-registry-provider'; /** Sync status annotation key. */ const SYNC_STATUS_ANNOTATION = 'redhat.com/rhdh-mcp-registry-sync-status'; -/** - * Whether a server.json document declares at least one native remote - * (non-empty type and D11-valid URL). Matches the mapping's copy rules - * for remotes that become `spec.remotes` rather than D8 placeholders. - * - * @internal - */ -export function hasNativeRemote(doc: McpServerDocument | undefined): boolean { - const remotes = doc?.remotes ?? []; - for (const remote of remotes) { - if ( - typeof remote.type === 'string' && - remote.type.length > 0 && - remote.url !== undefined && - remote.url !== null && - isAllowedUrl(remote.url) - ) { - return true; - } - } - return false; -} - -/** - * Build a last-good lookup key from name and version. - * - * @internal - */ -export function buildLastGoodKey(name: string, version: string): string { - return `${name}::${version}`; -} - -/** - * Read optional name/version from a registry list entry. - * - * @internal - */ -export function readServerIdentity( - entry: McpRegistryServerEntry | null | undefined, -): { - name?: string; - version?: string; -} { - const serverDoc = entry?.server; - return { - name: typeof serverDoc?.name === 'string' ? serverDoc.name : undefined, - version: - typeof serverDoc?.version === 'string' ? serverDoc.version : undefined, - }; -} - -/** - * Format the per-entry mapping failure warning. - * - * @internal - */ -export function formatMappingFailureMessage( - name: string | undefined, - version: string | undefined, - err: unknown, -): string { - let message = 'Failed to map MCP Registry server entry'; - if (name) { - message += ` "${name}"`; - } - if (version) { - message += ` (version "${version}")`; - } - return `${message}: ${err}`; -} - /** * Entity provider that ingests MCP servers from one configured * MCP Registry into the Backstage catalog. diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/providerUtils.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/providerUtils.test.ts new file mode 100644 index 00000000000..1bf072ade5b --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/providerUtils.test.ts @@ -0,0 +1,120 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { + buildLastGoodKey, + formatMappingFailureMessage, + hasNativeRemote, + readServerIdentity, +} from './providerUtils'; +import { createMockServerDoc } from './testUtils'; + +describe('buildLastGoodKey', () => { + it('joins name and version with a double-colon separator', () => { + expect(buildLastGoodKey('io.example/weather', '1.0.0')).toBe( + 'io.example/weather::1.0.0', + ); + }); +}); + +describe('hasNativeRemote', () => { + it('returns true when a remote has a non-empty type and http(s) URL', () => { + expect( + hasNativeRemote(createMockServerDoc('io.example/weather', '1.0.0')), + ).toBe(true); + }); + + it('returns false when remotes are missing or empty', () => { + expect( + hasNativeRemote( + createMockServerDoc('io.example/weather', '1.0.0', { + remotes: undefined, + }), + ), + ).toBe(false); + expect( + hasNativeRemote( + createMockServerDoc('io.example/weather', '1.0.0', { remotes: [] }), + ), + ).toBe(false); + expect(hasNativeRemote(undefined)).toBe(false); + }); + + it('returns false for invalid remote URLs', () => { + expect( + hasNativeRemote( + createMockServerDoc('io.example/weather', '1.0.0', { + remotes: [{ type: 'streamable-http', url: 'not-a-url' }], + }), + ), + ).toBe(false); + }); +}); + +describe('readServerIdentity', () => { + it('returns name and version from a valid entry', () => { + expect( + readServerIdentity({ + server: createMockServerDoc('io.example/weather', '1.2.3'), + }), + ).toEqual({ name: 'io.example/weather', version: '1.2.3' }); + }); + + it('returns undefined fields for null or undefined entries', () => { + expect(readServerIdentity(null)).toEqual({ + name: undefined, + version: undefined, + }); + expect(readServerIdentity(undefined)).toEqual({ + name: undefined, + version: undefined, + }); + }); + + it('ignores non-string name and version values', () => { + expect( + readServerIdentity({ + server: { + ...createMockServerDoc('io.example/weather', '1.0.0'), + name: 42 as unknown as string, + version: { n: 1 } as unknown as string, + }, + }), + ).toEqual({ name: undefined, version: undefined }); + }); +}); + +describe('formatMappingFailureMessage', () => { + it('includes name and version when both are present', () => { + expect( + formatMappingFailureMessage('io.example/weather', '1.0.0', 'boom'), + ).toBe( + 'Failed to map MCP Registry server entry "io.example/weather" (version "1.0.0"): boom', + ); + }); + + it('omits missing name and version segments', () => { + expect(formatMappingFailureMessage(undefined, undefined, 'boom')).toBe( + 'Failed to map MCP Registry server entry: boom', + ); + }); + + it('includes only the version when name is missing', () => { + expect(formatMappingFailureMessage(undefined, '1.0.0', 'boom')).toBe( + 'Failed to map MCP Registry server entry (version "1.0.0"): boom', + ); + }); +}); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/providerUtils.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/providerUtils.ts new file mode 100644 index 00000000000..4078fe729d3 --- /dev/null +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/providerUtils.ts @@ -0,0 +1,90 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { isAllowedUrl } from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; +import type { McpServerDocument } from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; +import type { McpRegistryServerEntry } from './client'; + +/** + * Whether a server.json document declares at least one native remote + * (non-empty type and D11-valid URL). Matches the mapping's copy rules + * for remotes that become `spec.remotes` rather than D8 placeholders. + * + * @internal + */ +export function hasNativeRemote(doc: McpServerDocument | undefined): boolean { + const remotes = doc?.remotes ?? []; + for (const remote of remotes) { + if ( + typeof remote.type === 'string' && + remote.type.length > 0 && + remote.url !== undefined && + remote.url !== null && + isAllowedUrl(remote.url) + ) { + return true; + } + } + return false; +} + +/** + * Build a last-good lookup key from name and version. + * + * @internal + */ +export function buildLastGoodKey(name: string, version: string): string { + return `${name}::${version}`; +} + +/** + * Read optional name/version from a registry list entry. + * + * @internal + */ +export function readServerIdentity( + entry: McpRegistryServerEntry | null | undefined, +): { + name?: string; + version?: string; +} { + const serverDoc = entry?.server; + return { + name: typeof serverDoc?.name === 'string' ? serverDoc.name : undefined, + version: + typeof serverDoc?.version === 'string' ? serverDoc.version : undefined, + }; +} + +/** + * Format the per-entry mapping failure warning. + * + * @internal + */ +export function formatMappingFailureMessage( + name: string | undefined, + version: string | undefined, + err: unknown, +): string { + let message = 'Failed to map MCP Registry server entry'; + if (name) { + message += ` "${name}"`; + } + if (version) { + message += ` (version "${version}")`; + } + return `${message}: ${err}`; +} From b975db78971cab39a0801dd8924e383f31ba1e1b Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 13:32:22 -0400 Subject: [PATCH 31/63] refactor(#4815): return seenCursors instead of mutating options Treat FetchServersOptions.seenCursors as read-only input and return the updated set on FetchServersResult so callers replace state explicitly. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../src/McpRegistryEntityProvider.test.ts | 12 ++++--- .../src/McpRegistryEntityProvider.ts | 14 +++----- .../src/client.test.ts | 35 ++++++++++++++++--- .../src/client.ts | 22 ++++++++---- 4 files changed, 60 insertions(+), 23 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts index c63e3c5a631..734a41a2403 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts @@ -669,7 +669,7 @@ describe('McpRegistryEntityProvider', () => { ).toBe('degraded'); }); - it('restores seenCursors on fetch error so the next sync resumes correctly', async () => { + it('leaves seenCursors unchanged on fetch error so the next sync resumes correctly', async () => { // First sync: page 1 succeeds, page 2 fails mid-pagination const page1: McpRegistryListResponse = { servers: [{ server: createMockServerDoc('io.github.user/one', '1.0.0') }], @@ -680,6 +680,7 @@ describe('McpRegistryEntityProvider', () => { ok: false, status: 500, url: '', + headers: { get: () => null }, json: async () => ({}), text: async () => 'Internal Server Error', } as unknown as Response; @@ -698,6 +699,7 @@ describe('McpRegistryEntityProvider', () => { combinedFetch.mockResolvedValueOnce({ ok: true, status: 200, + headers: { get: () => null }, json: async () => page1, text: async () => JSON.stringify(page1), } as unknown as Response); @@ -706,12 +708,14 @@ describe('McpRegistryEntityProvider', () => { combinedFetch.mockResolvedValueOnce({ ok: true, status: 200, + headers: { get: () => null }, json: async () => retryPage1, text: async () => JSON.stringify(retryPage1), } as unknown as Response); combinedFetch.mockResolvedValueOnce({ ok: true, status: 200, + headers: { get: () => null }, json: async () => retryPage2, text: async () => JSON.stringify(retryPage2), } as unknown as Response); @@ -725,15 +729,15 @@ describe('McpRegistryEntityProvider', () => { ); await provider.connect(connection); - // First sync: fails mid-pagination (seenCursors should be restored) + // First sync: fails mid-pagination (provider keeps prior seenCursors) await provider.run(); expect(connection.applyMutation).not.toHaveBeenCalled(); expect(logger.error).toHaveBeenCalledWith( expect.stringContaining('sync failed'), ); - // Retry sync: should succeed because seenCursors was restored, - // so cursor-1 is not incorrectly marked as seen + // Retry sync: should succeed because the failed call did not + // assign result.seenCursors, so cursor-1 is not incorrectly marked await provider.run(); expect(connection.applyMutation).toHaveBeenCalledTimes(1); const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index ffff25d8dae..ba982719770 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -84,7 +84,7 @@ export class McpRegistryEntityProvider implements EntityProvider { */ private resumeCursor?: string; private pendingEntries: McpRegistryServerEntry[] = []; - private readonly seenCursors = new Set(); + private seenCursors: Set = new Set(); /** * After a `maxEntries` soft-stop, later full traversals end at this @@ -213,7 +213,6 @@ export class McpRegistryEntityProvider implements EntityProvider { this.endCursorMaxEntries = undefined; } - const seenCursorsSnapshot = new Set(this.seenCursors); try { const result = await fetchRegistryServers({ baseUrl, @@ -230,6 +229,7 @@ export class McpRegistryEntityProvider implements EntityProvider { }); this.pendingEntries.push(...result.servers); + this.seenCursors = result.seenCursors; if (result.resumeCursor) { this.resumeCursor = result.resumeCursor; @@ -255,16 +255,12 @@ export class McpRegistryEntityProvider implements EntityProvider { const entries = this.pendingEntries; this.pendingEntries = []; this.resumeCursor = undefined; - this.seenCursors.clear(); + this.seenCursors = new Set(); return entries; } catch (err) { if (err instanceof McpRegistryClientError) { - // Restore seenCursors to pre-call state so the next sync - // does not carry partially mutated cursor history. - this.seenCursors.clear(); - for (const c of seenCursorsSnapshot) { - this.seenCursors.add(c); - } + // Input seenCursors is never mutated by the client; leave + // this.seenCursors unchanged so the next sync can retry. this.logger.error( `MCP Registry sync failed (no mutation emitted): ${err.message}`, ); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts index 1965789b55e..5e3260fee95 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts @@ -274,11 +274,38 @@ describe('fetchRegistryServers', () => { expect(result.servers).toHaveLength(1); expect(result.servers[0].server.name).toBe('test/server-c'); expect(result.resumeCursor).toBeUndefined(); + expect(result.seenCursors).toEqual(new Set(['cursor-1', 'cursor-2'])); + // Input options set must not be mutated. + expect(seenCursors).toEqual(new Set(['cursor-1', 'cursor-2'])); expect(fn).toHaveBeenCalledTimes(1); const calledUrl = fn.mock.calls[0][0] as string; expect(calledUrl).toContain('cursor=cursor-2'); }); + it('returns an updated seenCursors set without mutating the input', async () => { + const page1: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 2, nextCursor: 'cursor-1' }, + }; + const page2: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-b', '2.0.0') }], + metadata: { count: 2 }, + }; + const fn = mockFetch([{ body: page1 }, { body: page2 }]); + const seenCursors = new Set(); + + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + seenCursors, + fetchApi: fn, + }); + + expect(result.seenCursors).toEqual(new Set(['cursor-1'])); + expect(seenCursors.size).toBe(0); + }); + it('detects repeated cursor', async () => { const page1: McpRegistryListResponse = { servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], @@ -1030,13 +1057,13 @@ describe('resolveNextCursor', () => { expect(seen.size).toBe(0); }); - it('returns continue and records the cursor when paging continues', () => { + it('returns continue without mutating the seen set', () => { const seen = new Set(); expect(resolveNextCursor('page-2', seen, 1, 10)).toEqual({ status: 'continue', cursor: 'page-2', }); - expect(seen.has('page-2')).toBe(true); + expect(seen.size).toBe(0); }); it('throws on a repeated cursor', () => { @@ -1046,12 +1073,12 @@ describe('resolveNextCursor', () => { ); }); - it('returns pageLimitReached when more pages remain at the page cap', () => { + it('returns pageLimitReached without mutating the seen set', () => { const seen = new Set(); expect(resolveNextCursor('page-2', seen, 1, 1)).toEqual({ status: 'pageLimitReached', resumeCursor: 'page-2', }); - expect(seen.has('page-2')).toBe(true); + expect(seen.size).toBe(0); }); }); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index 560eb9772b0..b7ba6e9ca85 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -105,9 +105,10 @@ export interface FetchServersOptions { /** * Cursors already seen in the current multi-sync traversal. Shared * across resume cycles so repeated-cursor detection spans syncs. - * Mutated in place as new cursors are observed. + * Treated as read-only input; the updated set is returned on + * {@link FetchServersResult.seenCursors}. */ - seenCursors?: Set; + seenCursors?: ReadonlySet; /** * Optional allowlist of permitted hostnames. When set, every * outbound request URL is validated against this list before @@ -137,6 +138,11 @@ export interface FetchServersResult { * remember this cursor as the end bound for later full traversals. */ endCursor?: string; + /** + * Cursor set after this call, including any newly observed cursors. + * Callers should replace their prior set with this value on success. + */ + seenCursors: Set; } /** @@ -344,13 +350,14 @@ async function fetchOnce( * Returns `complete` when paging is done, `continue` when another page * should be fetched in this sync, or `pageLimitReached` when this sync * should stop and resume from `resumeCursor` on a later sync. - * Enforces repeated-cursor detection (still a hard error). + * Enforces repeated-cursor detection (still a hard error). Does not + * mutate `seenCursors`; the caller records new cursors. * * @internal */ export function resolveNextCursor( nextCursor: string | null | undefined, - seenCursors: Set, + seenCursors: ReadonlySet, pagesFetched: number, pageLimit: number, ): ResolveNextCursorResult { @@ -364,7 +371,6 @@ export function resolveNextCursor( `during pagination. Aborting sync to prevent infinite loop.`, ); } - seenCursors.add(nextCursor); if (pagesFetched >= pageLimit) { return { status: 'pageLimitReached', resumeCursor: nextCursor }; @@ -427,7 +433,8 @@ export async function fetchRegistryServers( } = options; const doFetch = fetchApi ?? fetch; const endpoint = parseServersEndpointUrl(baseUrl, apiVersion); - const seenCursors = options.seenCursors ?? new Set(); + // Own a local copy so the caller's options set is never mutated. + const seenCursors = new Set(options.seenCursors); // Defense-in-depth: validate endpoint hostname at runtime even // though config parsing already checked baseUrl against the list. @@ -485,6 +492,7 @@ export async function fetchRegistryServers( continue; } if (next.status === 'pageLimitReached') { + seenCursors.add(next.resumeCursor); if (endCursor && next.resumeCursor === endCursor) { hasMorePages = false; continue; @@ -493,6 +501,7 @@ export async function fetchRegistryServers( hasMorePages = false; continue; } + seenCursors.add(next.cursor); if (endCursor && next.cursor === endCursor) { hasMorePages = false; continue; @@ -504,5 +513,6 @@ export async function fetchRegistryServers( servers: allServers, resumeCursor, endCursor: maxEntriesEndCursor, + seenCursors, }; } From 8d7a3795996b4412a777ce13721fbff5201b9eb9 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 13:39:59 -0400 Subject: [PATCH 32/63] refactor(#4815): lower fetchRegistryServers cognitive complexity Split pagination into applyMaxEntriesSoftStop, advanceAfterResolvedCursor, and isAtEndCursor helpers so SonarCloud cognitive complexity stays within the allowed limit. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../src/client.test.ts | 98 ++++++++++ .../src/client.ts | 173 +++++++++++++----- 2 files changed, 222 insertions(+), 49 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts index 5e3260fee95..82ab38cece7 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts @@ -15,10 +15,13 @@ */ import { + advanceAfterResolvedCursor, + applyMaxEntriesSoftStop, buildPageRequestUrl, buildServersEndpoint, fetchRegistryPage, fetchRegistryServers, + isAtEndCursor, isRedirectStatus, McpRegistryClientError, parseServersEndpointUrl, @@ -1082,3 +1085,98 @@ describe('resolveNextCursor', () => { expect(seen.size).toBe(0); }); }); + +describe('applyMaxEntriesSoftStop', () => { + it('drops the tipping page and ends at the page cursor', () => { + const tipped = [{ server: createMockServerDoc('a/tip', '1.0.0') }]; + const prior = [{ server: createMockServerDoc('a/keep', '1.0.0') }]; + expect( + applyMaxEntriesSoftStop({ + serversIncludingTippedPage: [...prior, ...tipped], + tippedPageSize: 1, + priorEntryCount: 0, + pageCursor: 'cursor-tip', + startCursor: undefined, + tippedNextCursor: 'cursor-next', + }), + ).toEqual({ + servers: prior, + endCursor: 'cursor-tip', + }); + }); + + it('keeps a single oversized tipped page and ends at its next cursor', () => { + const tipped = [ + { server: createMockServerDoc('a/a', '1.0.0') }, + { server: createMockServerDoc('a/b', '1.0.0') }, + ]; + expect( + applyMaxEntriesSoftStop({ + serversIncludingTippedPage: tipped, + tippedPageSize: 2, + priorEntryCount: 0, + pageCursor: undefined, + startCursor: undefined, + tippedNextCursor: 'cursor-next', + }), + ).toEqual({ + servers: tipped, + endCursor: 'cursor-next', + }); + }); +}); + +describe('advanceAfterResolvedCursor', () => { + it('maps complete without recording a cursor', () => { + const seen = new Set(); + expect( + advanceAfterResolvedCursor({ status: 'complete' }, seen, undefined), + ).toEqual({ action: 'complete' }); + expect(seen.size).toBe(0); + }); + + it('records and resumes when pageLimit is reached', () => { + const seen = new Set(); + expect( + advanceAfterResolvedCursor( + { status: 'pageLimitReached', resumeCursor: 'cursor-2' }, + seen, + undefined, + ), + ).toEqual({ action: 'resume', resumeCursor: 'cursor-2' }); + expect(seen.has('cursor-2')).toBe(true); + }); + + it('stops at endCursor instead of resuming', () => { + const seen = new Set(); + expect( + advanceAfterResolvedCursor( + { status: 'pageLimitReached', resumeCursor: 'end' }, + seen, + 'end', + ), + ).toEqual({ action: 'stopAtEnd' }); + expect(seen.has('end')).toBe(true); + }); + + it('continues paging and records the cursor', () => { + const seen = new Set(); + expect( + advanceAfterResolvedCursor( + { status: 'continue', cursor: 'cursor-2' }, + seen, + undefined, + ), + ).toEqual({ action: 'continue', cursor: 'cursor-2' }); + expect(seen.has('cursor-2')).toBe(true); + }); +}); + +describe('isAtEndCursor', () => { + it('is true only when both values are set and equal', () => { + expect(isAtEndCursor('end', 'end')).toBe(true); + expect(isAtEndCursor('end', 'other')).toBe(false); + expect(isAtEndCursor(undefined, 'end')).toBe(false); + expect(isAtEndCursor('end', undefined)).toBe(false); + }); +}); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index b7ba6e9ca85..d99ce13630f 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -402,6 +402,102 @@ export function assertRequestHostAllowed( } } +/** + * Soft-stop when a tipped page would push the traversal past `maxEntries`. + * + * Drops the tipping page unless it is the only content so far (then keeps + * it and bounds later traversals at its next cursor). + * + * @internal + */ +export function applyMaxEntriesSoftStop(params: { + serversIncludingTippedPage: McpRegistryServerEntry[]; + tippedPageSize: number; + priorEntryCount: number; + pageCursor: string | undefined; + startCursor: string | undefined; + tippedNextCursor: string | null | undefined; +}): { servers: McpRegistryServerEntry[]; endCursor: string | undefined } { + const { + serversIncludingTippedPage, + tippedPageSize, + priorEntryCount, + pageCursor, + startCursor, + tippedNextCursor, + } = params; + + const withoutTip = serversIncludingTippedPage.slice( + 0, + serversIncludingTippedPage.length - tippedPageSize, + ); + + if (priorEntryCount + withoutTip.length === 0) { + // Single page alone exceeds the cap — keep it so a mutation can + // still proceed, and bound later traversals at its next cursor. + return { + servers: serversIncludingTippedPage, + endCursor: + typeof tippedNextCursor === 'string' && tippedNextCursor.length > 0 + ? tippedNextCursor + : undefined, + }; + } + + // Exclude the tipping page; end at the cursor used to fetch it. + return { + servers: withoutTip, + endCursor: pageCursor ?? startCursor, + }; +} + +/** + * Record a resolved next-cursor decision into `seenCursors` and map it + * to a pagination control action for the fetch loop. + * + * @internal + */ +export function advanceAfterResolvedCursor( + next: ResolveNextCursorResult, + seenCursors: Set, + endCursor: string | undefined, +): + | { action: 'complete' } + | { action: 'stopAtEnd' } + | { action: 'resume'; resumeCursor: string } + | { action: 'continue'; cursor: string } { + if (next.status === 'complete') { + return { action: 'complete' }; + } + + if (next.status === 'pageLimitReached') { + seenCursors.add(next.resumeCursor); + if (endCursor && next.resumeCursor === endCursor) { + return { action: 'stopAtEnd' }; + } + return { action: 'resume', resumeCursor: next.resumeCursor }; + } + + seenCursors.add(next.cursor); + if (endCursor && next.cursor === endCursor) { + return { action: 'stopAtEnd' }; + } + return { action: 'continue', cursor: next.cursor }; +} + +/** + * Whether paging should stop because the current cursor matches a + * previously saved maxEntries end bound. + * + * @internal + */ +export function isAtEndCursor( + cursor: string | undefined, + endCursor: string | undefined, +): boolean { + return endCursor !== undefined && cursor === endCursor; +} + /** * Fetch server entries from the MCP Registry using cursor pagination. * @@ -440,20 +536,13 @@ export async function fetchRegistryServers( // though config parsing already checked baseUrl against the list. assertRequestHostAllowed(endpoint, hostAllowList); - const allServers: McpRegistryServerEntry[] = []; + let allServers: McpRegistryServerEntry[] = []; let cursor: string | undefined = startCursor; let pagesFetched = 0; - let hasMorePages = true; let resumeCursor: string | undefined; let maxEntriesEndCursor: string | undefined; - while (hasMorePages) { - // Bound later traversals after a prior maxEntries soft-stop. - if (endCursor && cursor === endCursor) { - hasMorePages = false; - continue; - } - + while (!isAtEndCursor(cursor, endCursor)) { const url = buildPageRequestUrl(endpoint, cursor, pageSize); const body = await fetchRegistryPage(doFetch, url, hostAllowList); allServers.push(...body.servers); @@ -461,52 +550,38 @@ export async function fetchRegistryServers( const totalEntries = priorEntryCount + allServers.length; if (maxEntries !== undefined && totalEntries > maxEntries) { - const tippedPageSize = body.servers.length; - allServers.splice(allServers.length - tippedPageSize, tippedPageSize); - - if (priorEntryCount + allServers.length === 0) { - // Single page alone exceeds the cap — keep it so a mutation - // can still proceed, and bound later traversals at its next. - allServers.push(...body.servers); - const tippedNext = body.metadata?.nextCursor; - maxEntriesEndCursor = - typeof tippedNext === 'string' && tippedNext.length > 0 - ? tippedNext - : undefined; - } else { - // Exclude the tipping page; end at the cursor used to fetch it. - maxEntriesEndCursor = cursor ?? startCursor; - } - hasMorePages = false; - continue; + const capped = applyMaxEntriesSoftStop({ + serversIncludingTippedPage: allServers, + tippedPageSize: body.servers.length, + priorEntryCount, + pageCursor: cursor, + startCursor, + tippedNextCursor: body.metadata?.nextCursor, + }); + allServers = capped.servers; + maxEntriesEndCursor = capped.endCursor; + break; } - const next = resolveNextCursor( - body.metadata?.nextCursor, + const advance = advanceAfterResolvedCursor( + resolveNextCursor( + body.metadata?.nextCursor, + seenCursors, + pagesFetched, + pageLimit, + ), seenCursors, - pagesFetched, - pageLimit, + endCursor, ); - if (next.status === 'complete') { - hasMorePages = false; - continue; - } - if (next.status === 'pageLimitReached') { - seenCursors.add(next.resumeCursor); - if (endCursor && next.resumeCursor === endCursor) { - hasMorePages = false; - continue; - } - resumeCursor = next.resumeCursor; - hasMorePages = false; - continue; + + if (advance.action === 'complete' || advance.action === 'stopAtEnd') { + break; } - seenCursors.add(next.cursor); - if (endCursor && next.cursor === endCursor) { - hasMorePages = false; - continue; + if (advance.action === 'resume') { + resumeCursor = advance.resumeCursor; + break; } - cursor = next.cursor; + cursor = advance.cursor; } return { From 900083b10cd4c7590c4b7ef6c2f757e850cfcf9f Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 13:51:44 -0400 Subject: [PATCH 33/63] refactor(#4815): extract McpRegistryEntityProviderOptions Give the provider constructor options a named public interface so the API surface is clearer and taskRunner is documented for callers. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../report.api.md | 8 +++++++- .../src/McpRegistryEntityProvider.ts | 19 +++++++++++++------ .../src/index.ts | 1 + 3 files changed, 21 insertions(+), 7 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md index 1ca96e577de..a94f8902b45 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md @@ -7,6 +7,7 @@ import { BackendFeature } from '@backstage/backend-plugin-api'; import type { EntityProvider } from '@backstage/plugin-catalog-node'; import type { EntityProviderConnection } from '@backstage/plugin-catalog-node'; import type { LoggerService } from '@backstage/backend-plugin-api'; +import type { SchedulerServiceTaskRunner } from '@backstage/backend-plugin-api'; import type { SchedulerServiceTaskScheduleDefinition } from '@backstage/backend-plugin-api'; // @public @@ -18,7 +19,7 @@ export class McpRegistryEntityProvider implements EntityProvider { constructor( config: McpRegistryProviderConfig, logger: LoggerService, - options?: {}, + options?: McpRegistryEntityProviderOptions, ); // (undocumented) connect(connection: EntityProviderConnection): Promise; @@ -26,6 +27,11 @@ export class McpRegistryEntityProvider implements EntityProvider { getProviderName(): string; } +// @public +export interface McpRegistryEntityProviderOptions { + taskRunner?: SchedulerServiceTaskRunner; +} + // @public export interface McpRegistryProviderConfig { apiVersion?: string; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index ba982719770..78c4bc3f2bc 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -54,6 +54,18 @@ const PROVIDER_NAME = 'mcp-registry-provider'; /** Sync status annotation key. */ const SYNC_STATUS_ANNOTATION = 'redhat.com/rhdh-mcp-registry-sync-status'; +/** + * Optional constructor dependencies for {@link McpRegistryEntityProvider}. + * + * @public + */ +export interface McpRegistryEntityProviderOptions { + /** @internal Override the global `fetch` implementation (test seam). */ + fetchApi?: typeof fetch; + /** Scheduler task runner used to periodically invoke sync. */ + taskRunner?: SchedulerServiceTaskRunner; +} + /** * Entity provider that ingests MCP servers from one configured * MCP Registry into the Backstage catalog. @@ -97,12 +109,7 @@ export class McpRegistryEntityProvider implements EntityProvider { constructor( config: McpRegistryProviderConfig, logger: LoggerService, - options?: { - /** @internal Override the global `fetch` implementation (test seam). */ - fetchApi?: typeof fetch; - /** @internal Scheduler task runner for periodic sync. */ - taskRunner?: SchedulerServiceTaskRunner; - }, + options?: McpRegistryEntityProviderOptions, ) { this.config = resolveMcpRegistryProviderConfig(config); this.logger = logger; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts index af6fc0d84c2..65c2f0f299d 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/index.ts @@ -22,4 +22,5 @@ export { catalogModuleMcpRegistryProvider as default } from './module'; export { McpRegistryEntityProvider } from './McpRegistryEntityProvider'; +export type { McpRegistryEntityProviderOptions } from './McpRegistryEntityProvider'; export type { McpRegistryProviderConfig } from './config'; From c4c022862c4c7b2e2c918fb6b5ca8b60dd9c2fa2 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 13:57:21 -0400 Subject: [PATCH 34/63] fix(#4815): fail closed when response.url is missing When hostAllowList is configured, require response.url on every fetch and validate it against the allowlist so SSRF checks cannot be bypassed by an opaque response. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../README.md | 24 ++-- .../src/client.test.ts | 105 +++++++++++++++--- .../src/client.ts | 28 ++++- 3 files changed, 129 insertions(+), 28 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index 031293aa6b2..e5722d34614 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -60,18 +60,18 @@ catalog: ### Configuration options -| Option | Required | Default | Description | -| --------------- | -------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `baseUrl` | Yes | — | MCP Registry base URL. Also passed to the mapping as `placeholderRemoteUrl` when a server has no valid remotes. | -| `baseName` | No | `mcp.registry` (mapping default) | Override the mapping identity prefix for `metadata.name` | -| `apiVersion` | No | `v1` | Registry API version slug. The servers endpoint is `//servers`. Note: the live MCP Registry may serve `/v0` or `/v0.1`; set `apiVersion` to match your registry. | -| `defaultOwner` | No | `unknown` (mapping default) | Backstage entity reference used as `spec.owner` | -| `pageLimit` | No | `10` | Maximum number of pages fetched per sync. When more pages remain, the provider saves the cursor and continues on the next sync (no mutation until the registry is fully traversed). | -| `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | -| `maxEntries` | No | `5000` | Maximum total server entries buffered for one complete registry traversal (spans resume syncs). When exceeded, the provider commits the buffer, saves an end cursor, and later traversals stop there until `maxEntries` is patched. | -| `remotesOnly` | No | `false` | When `true`, skip servers that do not declare at least one native remote (non-empty type and http(s) URL). Package-only and placeholder-remote servers are omitted from the catalog. | -| `hostAllowList` | No | _(none — all hosts allowed)_ | Array of permitted hostnames. When set, `baseUrl` hostname must be in this list and every outbound request (including each redirect `Location`) is validated at runtime. Redirects are followed manually so disallowed hosts are never contacted. Provides defense-in-depth against SSRF. | -| `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | +| Option | Required | Default | Description | +| --------------- | -------- | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `baseUrl` | Yes | — | MCP Registry base URL. Also passed to the mapping as `placeholderRemoteUrl` when a server has no valid remotes. | +| `baseName` | No | `mcp.registry` (mapping default) | Override the mapping identity prefix for `metadata.name` | +| `apiVersion` | No | `v1` | Registry API version slug. The servers endpoint is `//servers`. Note: the live MCP Registry may serve `/v0` or `/v0.1`; set `apiVersion` to match your registry. | +| `defaultOwner` | No | `unknown` (mapping default) | Backstage entity reference used as `spec.owner` | +| `pageLimit` | No | `10` | Maximum number of pages fetched per sync. When more pages remain, the provider saves the cursor and continues on the next sync (no mutation until the registry is fully traversed). | +| `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | +| `maxEntries` | No | `5000` | Maximum total server entries buffered for one complete registry traversal (spans resume syncs). When exceeded, the provider commits the buffer, saves an end cursor, and later traversals stop there until `maxEntries` is patched. | +| `remotesOnly` | No | `false` | When `true`, skip servers that do not declare at least one native remote (non-empty type and http(s) URL). Package-only and placeholder-remote servers are omitted from the catalog. | +| `hostAllowList` | No | _(none — all hosts allowed)_ | Array of permitted hostnames. When set, `baseUrl` hostname must be in this list and every outbound request (including each redirect `Location` and `response.url`) is validated at runtime. Missing `response.url` fails closed. Redirects are followed manually so disallowed hosts are never contacted. Provides defense-in-depth against SSRF. | +| `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | ### Multiple registries diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts index 82ab38cece7..f1e6cdc20c2 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts @@ -17,6 +17,8 @@ import { advanceAfterResolvedCursor, applyMaxEntriesSoftStop, + assertRequestHostAllowed, + assertResponseUrlAllowed, buildPageRequestUrl, buildServersEndpoint, fetchRegistryPage, @@ -28,7 +30,6 @@ import { resolveNextCursor, resolveRedirectUrl, truncateErrorBody, - assertRequestHostAllowed, validateRedirectTarget, } from './client'; import type { McpRegistryListResponse } from './client'; @@ -78,20 +79,24 @@ describe('fetchRegistryServers', () => { if (resp.throws) { fn.mockRejectedValueOnce(new Error('network error')); } else { - fn.mockResolvedValueOnce({ - ok: (resp.status ?? 200) >= 200 && (resp.status ?? 200) < 300, - status: resp.status ?? 200, - json: async () => { - if (typeof resp.body === 'string') { - throw new Error('Invalid JSON'); - } - return resp.body; - }, - text: async () => - typeof resp.body === 'string' - ? resp.body - : JSON.stringify(resp.body), - } as unknown as Response); + fn.mockImplementationOnce(async (input: RequestInfo) => { + const requestUrl = typeof input === 'string' ? input : String(input); + return { + ok: (resp.status ?? 200) >= 200 && (resp.status ?? 200) < 300, + status: resp.status ?? 200, + url: requestUrl, + json: async () => { + if (typeof resp.body === 'string') { + throw new Error('Invalid JSON'); + } + return resp.body; + }, + text: async () => + typeof resp.body === 'string' + ? resp.body + : JSON.stringify(resp.body), + } as unknown as Response; + }); } } return fn; @@ -489,6 +494,7 @@ describe('fetchRegistryServers', () => { const fn = jest.fn().mockResolvedValueOnce({ ok: false, status: 302, + url: 'https://registry.example.com/v1/servers', headers: mockHeaders({ Location: 'https://evil.example.com/v1/servers', }), @@ -522,6 +528,7 @@ describe('fetchRegistryServers', () => { .mockResolvedValueOnce({ ok: false, status: 302, + url: 'https://registry.example.com/v1/servers', headers: mockHeaders({ Location: 'https://registry.example.com/v1/servers?redirected=1', }), @@ -531,6 +538,7 @@ describe('fetchRegistryServers', () => { .mockResolvedValueOnce({ ok: true, status: 200, + url: 'https://registry.example.com/v1/servers?redirected=1', headers: mockHeaders(), json: async () => body, text: async () => JSON.stringify(body), @@ -714,6 +722,7 @@ describe('fetchRegistryPage', () => { const doFetch = jest.fn().mockResolvedValue({ ok: false, status: 302, + url: 'https://registry.example.com/v1/servers', headers: mockHeaders({ Location: 'https://evil.example.com/v1/servers', }), @@ -731,6 +740,28 @@ describe('fetchRegistryPage', () => { expect(doFetch).toHaveBeenCalledTimes(1); }); + it('throws when hostAllowList is set but response.url is absent', async () => { + const body: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('a/b', '1.0.0') }], + metadata: { count: 1 }, + }; + const doFetch = jest.fn().mockResolvedValue({ + ok: true, + status: 200, + headers: mockHeaders(), + json: async () => body, + text: async () => JSON.stringify(body), + } as unknown as Response); + + await expect( + fetchRegistryPage( + doFetch, + new URL('https://registry.example.com/v1/servers'), + ['registry.example.com'], + ), + ).rejects.toThrow(/missing response\.url/); + }); + it('follows redirect Location when the target host is allowlisted', async () => { const body: McpRegistryListResponse = { servers: [{ server: createMockServerDoc('a/b', '1.0.0') }], @@ -741,6 +772,7 @@ describe('fetchRegistryPage', () => { .mockResolvedValueOnce({ ok: false, status: 301, + url: 'https://registry.example.com/v1/servers', headers: mockHeaders({ Location: '/v1/servers-mirror' }), json: async () => ({}), text: async () => '', @@ -748,6 +780,7 @@ describe('fetchRegistryPage', () => { .mockResolvedValueOnce({ ok: true, status: 200, + url: 'https://registry.example.com/v1/servers-mirror', headers: mockHeaders(), json: async () => body, text: async () => JSON.stringify(body), @@ -863,6 +896,7 @@ describe('fetchRegistryPage', () => { .mockResolvedValueOnce({ ok: false, status: 302, + url: 'https://registry.example.com/v1/servers', headers: mockHeaders({ Location: '/hop-1' }), json: async () => ({}), text: async () => '', @@ -870,6 +904,7 @@ describe('fetchRegistryPage', () => { .mockResolvedValueOnce({ ok: false, status: 308, + url: 'https://registry.example.com/hop-1', headers: mockHeaders({ Location: 'https://registry.example.com/hop-2', }), @@ -879,6 +914,7 @@ describe('fetchRegistryPage', () => { .mockResolvedValueOnce({ ok: true, status: 200, + url: 'https://registry.example.com/hop-2', headers: mockHeaders(), json: async () => body, text: async () => JSON.stringify(body), @@ -1045,6 +1081,45 @@ describe('assertRequestHostAllowed', () => { }); }); +describe('assertResponseUrlAllowed', () => { + it('does nothing when hostAllowList is undefined', () => { + expect(() => + assertResponseUrlAllowed( + { url: '' } as Response, + undefined, + 'https://registry.example.com/v1/servers', + ), + ).not.toThrow(); + }); + + it('throws when response.url is missing under an allowlist', () => { + expect(() => + assertResponseUrlAllowed( + { url: '' } as Response, + ['registry.example.com'], + 'https://registry.example.com/v1/servers', + ), + ).toThrow(/missing response\.url/); + }); + + it('validates response.url against the allowlist', () => { + expect(() => + assertResponseUrlAllowed( + { url: 'https://evil.example.com/v1/servers' } as Response, + ['registry.example.com'], + 'https://registry.example.com/v1/servers', + ), + ).toThrow(/not in the configured hostAllowList/); + expect(() => + assertResponseUrlAllowed( + { url: 'https://registry.example.com/v1/servers' } as Response, + ['registry.example.com'], + 'https://registry.example.com/v1/servers', + ), + ).not.toThrow(); + }); +}); + describe('resolveNextCursor', () => { it('returns complete when nextCursor is absent or empty', () => { const seen = new Set(); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index d99ce13630f..e26629ee852 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -335,13 +335,16 @@ async function fetchOnce( ): Promise { const requestUrl = url.toString(); assertRequestHostAllowed(url, hostAllowList); + let response: Response; try { - return await doFetch(requestUrl, { redirect: 'manual' }); + response = await doFetch(requestUrl, { redirect: 'manual' }); } catch (err) { throw new McpRegistryClientError( `Failed to reach MCP Registry at ${requestUrl}: ${err}`, ); } + assertResponseUrlAllowed(response, hostAllowList, requestUrl); + return response; } /** @@ -402,6 +405,29 @@ export function assertRequestHostAllowed( } } +/** + * Fail closed when an allowlist is configured but the fetch response + * does not expose a URL, then validate that URL's hostname. + * + * @internal + */ +export function assertResponseUrlAllowed( + response: Response, + hostAllowList: string[] | undefined, + requestUrl: string, +): void { + if (!hostAllowList) { + return; + } + if (!response.url) { + throw new McpRegistryClientError( + `MCP Registry response for ${requestUrl} is missing response.url ` + + `while hostAllowList is configured; refusing to proceed.`, + ); + } + assertRequestHostAllowed(new URL(response.url), hostAllowList); +} + /** * Soft-stop when a tipped page would push the traversal past `maxEntries`. * From 6a93f46a47a54c016eaaebac23375d56973ee738 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 16:48:38 -0400 Subject: [PATCH 35/63] chore(#4815): add local MCP Registry deploy tooling Provide yarn start/stop scripts, Podman/Docker compose helpers with custom seed mounts, example seed fixtures, and docs for developing the mcp-registry-provider against a local registry without ko. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- workspaces/ai-integrations/.eslintignore | 2 +- workspaces/ai-integrations/README.md | 5 + .../docs/deploy-mcp-registry-locally.md | 93 ++++++++ .../examples/mcp-registry/seed-data/seed.json | 203 ++++++++++++++++++ workspaces/ai-integrations/hack/.eslintrc.js | 19 ++ .../hack/deploy-mcp-registry.ts | 161 ++++++++++++++ .../hack/undeploy-mcp-registry.ts | 103 +++++++++ workspaces/ai-integrations/package.json | 4 +- .../README.md | 5 + .../package.json | 2 + 10 files changed, 595 insertions(+), 2 deletions(-) create mode 100644 workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md create mode 100644 workspaces/ai-integrations/examples/mcp-registry/seed-data/seed.json create mode 100644 workspaces/ai-integrations/hack/.eslintrc.js create mode 100755 workspaces/ai-integrations/hack/deploy-mcp-registry.ts create mode 100755 workspaces/ai-integrations/hack/undeploy-mcp-registry.ts diff --git a/workspaces/ai-integrations/.eslintignore b/workspaces/ai-integrations/.eslintignore index 78d283ca04c..c48e89a3137 100644 --- a/workspaces/ai-integrations/.eslintignore +++ b/workspaces/ai-integrations/.eslintignore @@ -1,3 +1,3 @@ playwright.config.ts !.eslintrc.js -!.prettierrc.js \ No newline at end of file +!.prettierrc.js diff --git a/workspaces/ai-integrations/README.md b/workspaces/ai-integrations/README.md index 711e2ff3dd4..7cb5faeb215 100644 --- a/workspaces/ai-integrations/README.md +++ b/workspaces/ai-integrations/README.md @@ -29,3 +29,8 @@ If you would like to build with `docker`, add the `--user-docker` tag like so: ``` npx --yes @red-hat-developer-hub/cli@latest plugin package --tag --tag "${PLUGIN_CONTAINER_TAG}" --use-docker ``` + +## Deploy MCP Registry Locally + +To run a local MCP Registry for provider development, see +[Deploy MCP Registry Locally](./docs/deploy-mcp-registry-locally.md). diff --git a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md new file mode 100644 index 00000000000..95aa3972dbd --- /dev/null +++ b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md @@ -0,0 +1,93 @@ +# Deploy MCP Registry Locally + +For local provider development against a real registry instance, this workspace +includes Node scripts under [`hack/`](../hack/) that start the upstream +[MCP Registry](https://github.com/modelcontextprotocol/registry) with Podman or +Docker Compose. They use the published +`ghcr.io/modelcontextprotocol/registry` image instead of upstream +`make dev-compose` (which builds with `ko` and does not work with Podman). + +## Prerequisites + +- Node.js 22+ (type stripping for `.ts` scripts) +- `git` +- `podman compose` or `docker compose` + +## Start + +From the `ai-integrations` workspace root, or from +`plugins/catalog-backend-module-mcp-registry-provider`: + +```bash +yarn start-mcp-registry +``` + +You can also run the script directly from the workspace root: + +```bash +node hack/deploy-mcp-registry.ts +``` + +This clones the registry into `/tmp/mcp-registry` (if needed), starts PostgreSQL +and the registry in the background, and serves the API at +[http://localhost:8080](http://localhost:8080). + +Optional environment variables: + +| Variable | Default | Description | +| ----------------------- | -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `REPO_DIR` | `/tmp/mcp-registry` | Local checkout path for the registry | +| `MCP_REGISTRY_IMAGE` | `ghcr.io/modelcontextprotocol/registry:main` | Registry container image | +| `MCP_REGISTRY_DATA_DIR` | _(checkout `./data`)_ | Host directory mounted at `/data` instead of the [default seed data](https://github.com/modelcontextprotocol/registry/blob/main/data/seed.json). When set, seeds from `data/seed.json` with validation off. | + +Example with custom seed content (directory must contain `seed.json`): + +```bash +MCP_REGISTRY_DATA_DIR=./examples/mcp-registry/seed-data yarn start-mcp-registry +``` + +View logs (example with Podman): + +```bash +podman compose -f /tmp/mcp-registry/docker-compose.yml logs -f +``` + +## Point the provider at localhost + +Configure `catalog.providers.mcpRegistry` to use the local registry. The default +local API version is `v0.1`: + +```yaml +catalog: + providers: + mcpRegistry: + baseUrl: http://localhost:8080 + apiVersion: v0.1 + # Optional when restricting outbound hosts: + # hostAllowList: + # - localhost +``` + +Then start the workspace as usual (`yarn dev` from `workspaces/ai-integrations`). + +See also the +[MCP Registry Provider](../plugins/catalog-backend-module-mcp-registry-provider/) +plugin for full configuration options. + +## Stop + +From the workspace root or +`plugins/catalog-backend-module-mcp-registry-provider`: + +```bash +yarn stop-mcp-registry +``` + +Or from the workspace root: + +```bash +node hack/undeploy-mcp-registry.ts +``` + +This runs `compose down` for the same stack. The `/tmp/mcp-registry` checkout is +left in place so the next deploy is faster. diff --git a/workspaces/ai-integrations/examples/mcp-registry/seed-data/seed.json b/workspaces/ai-integrations/examples/mcp-registry/seed-data/seed.json new file mode 100644 index 00000000000..9a66728c819 --- /dev/null +++ b/workspaces/ai-integrations/examples/mcp-registry/seed-data/seed.json @@ -0,0 +1,203 @@ +[ + { + "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json", + "name": "io.example.labs/atlas-search", + "description": "MCP server that wraps the Atlas Search HTTP API for document discovery", + "title": "Atlas Search", + "websiteUrl": "https://labs.example.io/mcp/atlas-search", + "repository": { + "url": "https://github.com/example-labs/mcp-servers", + "source": "github" + }, + "version": "2.1.0", + "packages": [ + { + "registryType": "npm", + "registryBaseUrl": "https://registry.npmjs.org", + "identifier": "@example-labs/mcp-server-atlas-search", + "version": "2.1.0", + "transport": { + "type": "stdio" + }, + "environmentVariables": [ + { + "name": "ATLAS_SEARCH_API_KEY", + "description": "Atlas Search API key", + "isRequired": true, + "isSecret": true + } + ] + } + ], + "_meta": { + "io.modelcontextprotocol.registry/publisher-provided": { + "tool": "npm-publisher", + "version": "1.2.0", + "build_info": { + "timestamp": "2025-03-14T09:15:00Z" + } + } + } + }, + { + "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json", + "name": "io.github.example-labs/workspace-fs", + "description": "MCP server for sandboxed workspace filesystem read and write operations.", + "title": "Workspace Filesystem", + "repository": { + "url": "https://github.com/example-labs/mcp-servers", + "source": "github", + "id": "c1a2b3d4-e5f6-7890-abcd-ef1234567890" + }, + "version": "1.4.1", + "packages": [ + { + "registryType": "npm", + "registryBaseUrl": "https://registry.npmjs.org", + "identifier": "@example-labs/mcp-server-workspace-fs", + "version": "1.4.1", + "transport": { + "type": "stdio" + }, + "packageArguments": [ + { + "type": "positional", + "valueHint": "workspace_root", + "description": "Workspace root directory to expose", + "default": "/home/developer/workspace", + "isRequired": true, + "isRepeated": true + } + ], + "environmentVariables": [ + { + "name": "LOG_LEVEL", + "description": "Logging level (debug, info, warn, error)", + "default": "warn" + } + ] + }, + { + "registryType": "oci", + "identifier": "ghcr.io/example-labs/workspace-fs:1.4.1", + "transport": { + "type": "stdio" + }, + "runtimeArguments": [ + { + "type": "named", + "description": "Bind-mount a host path into the container", + "name": "--mount", + "value": "type=bind,src={source_path},dst={target_path}", + "isRequired": true, + "isRepeated": true, + "variables": { + "source_path": { + "description": "Host path to mount", + "format": "filepath", + "isRequired": true + }, + "target_path": { + "description": "Mount point inside the container under `/workspace`.", + "isRequired": true, + "default": "/workspace" + } + } + } + ], + "packageArguments": [ + { + "type": "positional", + "valueHint": "workspace_root", + "value": "/workspace" + } + ], + "environmentVariables": [ + { + "name": "LOG_LEVEL", + "description": "Logging level (debug, info, warn, error)", + "default": "warn" + } + ] + } + ], + "_meta": { + "io.modelcontextprotocol.registry/publisher-provided": { + "tool": "ci-publisher", + "version": "4.0.0", + "build_info": { + "commit": "9f8e7d6c5b4a3210", + "timestamp": "2025-06-02T18:40:00Z", + "pipeline_id": "workspace-fs-build-2048", + "environment": "staging" + } + } + } + }, + { + "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json", + "name": "io.github.example-labs/dice-weather-mcp", + "description": "NuGet MCP server that returns random dice rolls and sample weather snippets", + "version": "1.2.0-preview.3", + "packages": [ + { + "registryType": "nuget", + "registryBaseUrl": "https://api.nuget.org/v3/index.json", + "identifier": "ExampleLabs.DiceWeatherMcp", + "version": "1.2.0-preview.3", + "transport": { + "type": "stdio" + }, + "packageArguments": [ + { + "type": "positional", + "value": "mcp" + }, + { + "type": "positional", + "value": "start" + } + ] + } + ], + "_meta": { + "io.modelcontextprotocol.registry/publisher-provided": { + "tool": "nuget-publisher", + "version": "2.3.1", + "build_info": { + "timestamp": "2025-01-22T11:05:00Z", + "pipeline_id": "nuget-dice-weather-101" + } + } + } + }, + { + "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json", + "name": "io.example.cloud/remote-workspace", + "description": "Hosted MCP workspace filesystem endpoint for shared team sandboxes", + "repository": { + "url": "https://github.com/example-cloud/remote-workspace-mcp", + "source": "github", + "id": "a0b1c2d3-e4f5-6789-abcd-ef0123456789" + }, + "version": "3.1.0", + "remotes": [ + { + "type": "streamable-http", + "url": "https://mcp.example.cloud/v1/workspace/http" + } + ], + "_meta": { + "io.modelcontextprotocol.registry/publisher-provided": { + "tool": "cloud-deployer", + "version": "3.0.2", + "build_info": { + "commit": "c3b2a19087fe", + "timestamp": "2025-08-19T13:10:00Z", + "deployment_id": "remote-workspace-deploy-812", + "region": "eu-central-1" + } + } + } + } +] diff --git a/workspaces/ai-integrations/hack/.eslintrc.js b/workspaces/ai-integrations/hack/.eslintrc.js new file mode 100644 index 00000000000..f6315ceedd7 --- /dev/null +++ b/workspaces/ai-integrations/hack/.eslintrc.js @@ -0,0 +1,19 @@ +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +module.exports = require('@backstage/cli/config/eslint-factory').createConfigForRole( + __dirname, + 'cli', +); diff --git a/workspaces/ai-integrations/hack/deploy-mcp-registry.ts b/workspaces/ai-integrations/hack/deploy-mcp-registry.ts new file mode 100755 index 00000000000..2b7cd213c3e --- /dev/null +++ b/workspaces/ai-integrations/hack/deploy-mcp-registry.ts @@ -0,0 +1,161 @@ +#!/usr/bin/env node +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Start a local MCP Registry for provider development. + * + * Upstream `make dev-compose` builds the registry image with ko into the Docker + * daemon. ko does not work with podman, so this script uses the published GHCR + * image and the upstream docker-compose.yml (postgres + registry) instead. + * + * Set MCP_REGISTRY_DATA_DIR to mount a custom host directory over /data (instead + * of the checkout's ./data, which includes the default seed.json). When set, + * seeding defaults to data/seed.json with registry validation disabled unless + * MCP_REGISTRY_SEED_FROM / MCP_REGISTRY_ENABLE_REGISTRY_VALIDATION are already + * set. + */ + +const { spawnSync } = require('node:child_process'); +const { + existsSync, + mkdtempSync, + rmSync, + statSync, + writeFileSync, +} = require('node:fs'); +const { tmpdir } = require('node:os'); +const { join, resolve } = require('node:path'); + +const REPO_DIR = process.env.REPO_DIR?.trim() || '/tmp/mcp-registry'; +const IMAGE = + process.env.MCP_REGISTRY_IMAGE?.trim() || + 'ghcr.io/modelcontextprotocol/registry:main'; +const DATA_DIR = process.env.MCP_REGISTRY_DATA_DIR?.trim(); + +function commandExists(command: string): boolean { + return ( + spawnSync('sh', ['-c', `command -v "${command}" >/dev/null 2>&1`]) + .status === 0 + ); +} + +function composeVersionOk(bin: string): boolean { + return ( + spawnSync(bin, ['compose', 'version'], { stdio: 'ignore' }).status === 0 + ); +} + +function resolveCompose(): [string, ...string[]] { + if (commandExists('podman') && composeVersionOk('podman')) { + return ['podman', 'compose']; + } + if (commandExists('docker') && composeVersionOk('docker')) { + return ['docker', 'compose']; + } + throw new Error("need 'podman compose' or 'docker compose'"); +} + +function resolveDataDir(): string | undefined { + if (!DATA_DIR) { + return undefined; + } + const absoluteDataDir = resolve(DATA_DIR); + if ( + !existsSync(absoluteDataDir) || + !statSync(absoluteDataDir).isDirectory() + ) { + console.error( + `error: MCP_REGISTRY_DATA_DIR must be an existing directory: ${absoluteDataDir}`, + ); + process.exit(1); + } + return absoluteDataDir; +} + +function buildOverrideYaml(image: string, dataDir?: string): string { + const lines = ['services:', ' registry:', ` image: ${image}`]; + if (dataDir) { + // Replace upstream ./data:/data:ro with a custom host directory. + // `:z` is required for Podman/SELinux so the container (uid 65532) can + // read the bind-mounted seed files; without it open() returns EACCES. + lines.push( + ' volumes:', + ` - ${JSON.stringify(`${dataDir}:/data:ro,z`)}`, + ); + } + return `${lines.join('\n')}\n`; +} + +if (!existsSync(join(REPO_DIR, '.git'))) { + const result = spawnSync( + 'git', + ['clone', 'https://github.com/modelcontextprotocol/registry.git', REPO_DIR], + { stdio: 'inherit' }, + ); + if (result.status !== 0) { + process.exit(result.status ?? 1); + } +} + +const dataDir = resolveDataDir(); + +let compose: [string, ...string[]]; +try { + compose = resolveCompose(); +} catch (error) { + console.error(`error: ${error instanceof Error ? error.message : error}`); + process.exit(1); +} + +const overrideDir = mkdtempSync(join(tmpdir(), 'mcp-registry-')); +const overridePath = join(overrideDir, 'override.yml'); +writeFileSync(overridePath, buildOverrideYaml(IMAGE, dataDir), 'utf8'); + +const composeEnv = { ...process.env }; +if (dataDir) { + // Match upstream offline seeding: + // MCP_REGISTRY_SEED_FROM=data/seed.json MCP_REGISTRY_ENABLE_REGISTRY_VALIDATION=false + if (!composeEnv.MCP_REGISTRY_SEED_FROM?.trim()) { + composeEnv.MCP_REGISTRY_SEED_FROM = 'data/seed.json'; + } + if (!composeEnv.MCP_REGISTRY_ENABLE_REGISTRY_VALIDATION?.trim()) { + composeEnv.MCP_REGISTRY_ENABLE_REGISTRY_VALIDATION = 'false'; + } +} + +try { + const seedNote = dataDir ? ` with data from ${dataDir}` : ''; + console.log( + `Starting MCP Registry from ${IMAGE}${seedNote} (http://localhost:8080)...`, + ); + const [bin, ...prefix] = compose; + const result = spawnSync( + bin, + [...prefix, '-f', 'docker-compose.yml', '-f', overridePath, 'up', '-d'], + { cwd: REPO_DIR, stdio: 'inherit', env: composeEnv }, + ); + if (result.status !== 0) { + process.exit(result.status ?? 1); + } + console.log( + `MCP Registry started in background. Use '${compose.join( + ' ', + )} -f ${REPO_DIR}/docker-compose.yml logs' to view logs.`, + ); +} finally { + rmSync(overrideDir, { recursive: true, force: true }); +} diff --git a/workspaces/ai-integrations/hack/undeploy-mcp-registry.ts b/workspaces/ai-integrations/hack/undeploy-mcp-registry.ts new file mode 100755 index 00000000000..cacaa2d51e2 --- /dev/null +++ b/workspaces/ai-integrations/hack/undeploy-mcp-registry.ts @@ -0,0 +1,103 @@ +#!/usr/bin/env node +/* + * Copyright Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** Stop the local MCP Registry started by deploy-mcp-registry.ts. */ + +const { spawnSync } = require('node:child_process'); +const { existsSync, mkdtempSync, rmSync, writeFileSync } = require('node:fs'); +const { tmpdir } = require('node:os'); +const { join } = require('node:path'); + +const REPO_DIR = process.env.REPO_DIR?.trim() || '/tmp/mcp-registry'; +const IMAGE = + process.env.MCP_REGISTRY_IMAGE?.trim() || + 'ghcr.io/modelcontextprotocol/registry:main'; + +function commandExists(command: string): boolean { + return ( + spawnSync('sh', ['-c', `command -v "${command}" >/dev/null 2>&1`]) + .status === 0 + ); +} + +function composeVersionOk(bin: string): boolean { + return ( + spawnSync(bin, ['compose', 'version'], { stdio: 'ignore' }).status === 0 + ); +} + +function resolveCompose(): [string, ...string[]] { + if (commandExists('podman') && composeVersionOk('podman')) { + return ['podman', 'compose']; + } + if (commandExists('docker') && composeVersionOk('docker')) { + return ['docker', 'compose']; + } + throw new Error("need 'podman compose' or 'docker compose'"); +} + +if (!existsSync(join(REPO_DIR, '.git'))) { + if (!existsSync(REPO_DIR)) { + const result = spawnSync( + 'git', + [ + 'clone', + 'https://github.com/modelcontextprotocol/registry.git', + REPO_DIR, + ], + { stdio: 'inherit' }, + ); + if (result.status !== 0) { + process.exit(result.status ?? 1); + } + } else { + console.error(`error: ${REPO_DIR} exists but is not a git repository`); + process.exit(1); + } +} + +let compose: [string, ...string[]]; +try { + compose = resolveCompose(); +} catch (error) { + console.error(`error: ${error instanceof Error ? error.message : error}`); + process.exit(1); +} + +const overrideDir = mkdtempSync(join(tmpdir(), 'mcp-registry-')); +const overridePath = join(overrideDir, 'override.yml'); +writeFileSync( + overridePath, + `services:\n registry:\n image: ${IMAGE}\n`, + 'utf8', +); + +try { + console.log(`Stopping MCP Registry in ${REPO_DIR}...`); + const [bin, ...prefix] = compose; + const result = spawnSync( + bin, + [...prefix, '-f', 'docker-compose.yml', '-f', overridePath, 'down'], + { cwd: REPO_DIR, stdio: 'inherit' }, + ); + if (result.status !== 0) { + process.exit(result.status ?? 1); + } + console.log('MCP Registry stopped.'); +} finally { + rmSync(overrideDir, { recursive: true, force: true }); +} diff --git a/workspaces/ai-integrations/package.json b/workspaces/ai-integrations/package.json index 1a7129681b0..b845f04e296 100644 --- a/workspaces/ai-integrations/package.json +++ b/workspaces/ai-integrations/package.json @@ -10,6 +10,8 @@ "dev:debug": "yarn workspaces foreach -A --include backend --include app --parallel -v -i run start --inspect", "start": "yarn workspace app start", "start-backend": "yarn workspace backend start", + "start-mcp-registry": "node hack/deploy-mcp-registry.ts", + "stop-mcp-registry": "node hack/undeploy-mcp-registry.ts", "build:backend": "yarn workspace backend build", "tsc": "tsc", "tsc:full": "tsc --skipLibCheck true --incremental false", @@ -64,7 +66,7 @@ }, "prettier": "@backstage/cli/config/prettier", "lint-staged": { - "*.{js,jsx,ts,tsx,mjs,cjs}": [ + "{packages,plugins,hack}/**/*.{js,jsx,ts,tsx,mjs,cjs}": [ "eslint --fix", "prettier --write" ], diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index e5722d34614..c19518f134b 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -123,3 +123,8 @@ These MCP server entries have a single remote _placeholder_ field which should * } } ``` + +## Deploy MCP Registry Locally + +To run a local MCP Registry for provider development, see +[Deploy MCP Registry Locally](../../docs/deploy-mcp-registry-locally.md). diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json index 303a7fca8d8..4d6459bb372 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json @@ -22,6 +22,8 @@ }, "scripts": { "start": "backstage-cli package start", + "start-mcp-registry": "node ../../hack/deploy-mcp-registry.ts", + "stop-mcp-registry": "node ../../hack/undeploy-mcp-registry.ts", "build": "backstage-cli package build", "lint": "backstage-cli package lint", "lint:check": "backstage-cli package lint", From f2d28c2974be2149928fef047388f12e720af231 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 17:12:16 -0400 Subject: [PATCH 36/63] chore(#4815): wire mcp-registry-provider into workspace backend Register the catalog MCP registry provider in the local backend so yarn dev loads it with the rest of the catalog stack. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- workspaces/ai-integrations/packages/backend/package.json | 1 + workspaces/ai-integrations/packages/backend/src/index.ts | 5 +++++ workspaces/ai-integrations/yarn.lock | 3 ++- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/workspaces/ai-integrations/packages/backend/package.json b/workspaces/ai-integrations/packages/backend/package.json index 31b7efe3480..59280bb9f8c 100644 --- a/workspaces/ai-integrations/packages/backend/package.json +++ b/workspaces/ai-integrations/packages/backend/package.json @@ -50,6 +50,7 @@ "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-ai-model-server": "workspace:^", "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-ai-resource-agent": "workspace:^", "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-ai-resource-extensions": "workspace:^", + "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider": "workspace:^", "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-model-catalog": "workspace:^", "@red-hat-developer-hub/backstage-plugin-catalog-techdoc-url-reader-backend": "workspace:^", "@red-hat-developer-hub/backstage-plugin-kserve-kubeflow-connector-backend": "workspace:^", diff --git a/workspaces/ai-integrations/packages/backend/src/index.ts b/workspaces/ai-integrations/packages/backend/src/index.ts index 8ac73ed95bf..198d7794b7f 100644 --- a/workspaces/ai-integrations/packages/backend/src/index.ts +++ b/workspaces/ai-integrations/packages/backend/src/index.ts @@ -92,6 +92,11 @@ backend.add( '@red-hat-developer-hub/backstage-plugin-catalog-backend-module-ai-model-server' ), ); +backend.add( + import( + '@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider' + ), +); backend.add( import( '@red-hat-developer-hub/backstage-plugin-kserve-kubeflow-connector-backend' diff --git a/workspaces/ai-integrations/yarn.lock b/workspaces/ai-integrations/yarn.lock index a7db4f2f478..cd4301d965b 100644 --- a/workspaces/ai-integrations/yarn.lock +++ b/workspaces/ai-integrations/yarn.lock @@ -9863,7 +9863,7 @@ __metadata: languageName: unknown linkType: soft -"@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider@workspace:plugins/catalog-backend-module-mcp-registry-provider": +"@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider@workspace:^, @red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider@workspace:plugins/catalog-backend-module-mcp-registry-provider": version: 0.0.0-use.local resolution: "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider@workspace:plugins/catalog-backend-module-mcp-registry-provider" dependencies: @@ -15100,6 +15100,7 @@ __metadata: "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-ai-model-server": "workspace:^" "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-ai-resource-agent": "workspace:^" "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-ai-resource-extensions": "workspace:^" + "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider": "workspace:^" "@red-hat-developer-hub/backstage-plugin-catalog-backend-module-model-catalog": "workspace:^" "@red-hat-developer-hub/backstage-plugin-catalog-techdoc-url-reader-backend": "workspace:^" "@red-hat-developer-hub/backstage-plugin-kserve-kubeflow-connector-backend": "workspace:^" From 8da501eb67e0386f10ed5d5e6e5595c2fe0a9346 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 17:17:31 -0400 Subject: [PATCH 37/63] chore(#4815): make app-config changes - Replace live staging MCP Registry URL with MCP_REGISTRY_URL env var (default: localhost:8080) - Comment baseName out and make it library default (mcp.registry) - Replace guest defaultOwner with OWNER env var (default: default-owner) to be consistent with workspace Signed-off-by: Michael Valdron --- workspaces/ai-integrations/app-config.yaml | 6 +++--- .../app-config.yaml | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/workspaces/ai-integrations/app-config.yaml b/workspaces/ai-integrations/app-config.yaml index 67b40254928..674e6764267 100644 --- a/workspaces/ai-integrations/app-config.yaml +++ b/workspaces/ai-integrations/app-config.yaml @@ -141,13 +141,13 @@ catalog: default-lifecycle: '${LIFECYCLE:-production}' mcpRegistry: # Required: base URL of the MCP Registry - baseUrl: https://staging.registry.modelcontextprotocol.io/ + baseUrl: '${MCP_REGISTRY_URL:-http://localhost:8080/}' # Optional: base name (default: mcp.registry) - baseName: staging.registry.modelcontextprotocol.io + # baseName: mcp.registry # Optional: API version (default: v1) apiVersion: v0.1 # Optional: default entity owner (default: unknown) - defaultOwner: user:development/guest + defaultOwner: '${OWNER:-default-owner}' # Optional: max pages fetched per sync (default: 10) # pageLimit: 10 # Optional: registry page size sent as ?limit= (omitted by default) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml index 2a4d016937e..750ed634afb 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml @@ -2,13 +2,13 @@ catalog: providers: mcpRegistry: # Required: base URL of the MCP Registry - baseUrl: https://staging.registry.modelcontextprotocol.io/ + baseUrl: '${MCP_REGISTRY_URL:-http://localhost:8080/}' # Optional: base name (default: mcp.registry) - baseName: staging.registry.modelcontextprotocol.io + # baseName: mcp.registry # Optional: API version (default: v1) apiVersion: v0.1 # Optional: default entity owner (default: unknown) - defaultOwner: user:development/guest + defaultOwner: '${OWNER:-default-owner}' # Optional: max pages fetched per sync (default: 10) # pageLimit: 10 # Optional: registry page size sent as ?limit= (omitted by default) From c569dc08588adff0814a877f0d7b96a6e6023723 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 17:22:04 -0400 Subject: [PATCH 38/63] chore(#4815): remove unused app-config from catalog-backend-module-mcp-registry-provider plugin directory Signed-off-by: Michael Valdron --- .../app-config.yaml | 29 ------------------- 1 file changed, 29 deletions(-) delete mode 100644 workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml deleted file mode 100644 index 750ed634afb..00000000000 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/app-config.yaml +++ /dev/null @@ -1,29 +0,0 @@ -catalog: - providers: - mcpRegistry: - # Required: base URL of the MCP Registry - baseUrl: '${MCP_REGISTRY_URL:-http://localhost:8080/}' - # Optional: base name (default: mcp.registry) - # baseName: mcp.registry - # Optional: API version (default: v1) - apiVersion: v0.1 - # Optional: default entity owner (default: unknown) - defaultOwner: '${OWNER:-default-owner}' - # Optional: max pages fetched per sync (default: 10) - # pageLimit: 10 - # Optional: registry page size sent as ?limit= (omitted by default) - # pageSize: 50 - # Optional: max entries per complete traversal; soft-stops with end cursor (default: 5000) - # maxEntries: 5000 - # Optional: ingest only servers with at least one native remote (default: false) - # remotesOnly: false - # Optional: allowlist of permitted hostnames for defense-in-depth SSRF protection. - # hostAllowList: - # - registry.modelcontextprotocol.io - # - staging.registry.modelcontextprotocol.io - # Optional: sync schedule (defaults shown below) - # schedule: - # frequency: { minutes: 30 } - # timeout: { minutes: 3 } - # # Optional: defer the first sync - # # initialDelay: { seconds: 15 } From 691ec4a67d3bd7287985d12f56d1125a83b36743 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 17:38:14 -0400 Subject: [PATCH 39/63] fix(#4815): wait for MCP Registry readiness before returning Block start-mcp-registry until the HTTP API responds so yarn dev does not race seed import and fail with fetch failed on the first sync. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../docs/deploy-mcp-registry-locally.md | 22 +++++++---- .../hack/deploy-mcp-registry.ts | 38 ++++++++++++++++++- 2 files changed, 52 insertions(+), 8 deletions(-) diff --git a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md index 95aa3972dbd..0dbfbc5a4ff 100644 --- a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md +++ b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md @@ -29,16 +29,24 @@ node hack/deploy-mcp-registry.ts ``` This clones the registry into `/tmp/mcp-registry` (if needed), starts PostgreSQL -and the registry in the background, and serves the API at -[http://localhost:8080](http://localhost:8080). +and the registry in the background, **waits until the HTTP API responds** (seed +import can take a few minutes when seeding from the public registry), and serves +the API at [http://localhost:8080](http://localhost:8080). + +Start the registry **before** `yarn dev`. If the provider syncs while the +registry is still importing seed data, you will see +`Failed to reach MCP Registry ... TypeError: fetch failed` (no mutation). Restart +the backend after the registry is ready, or wait for the next scheduled sync. Optional environment variables: -| Variable | Default | Description | -| ----------------------- | -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `REPO_DIR` | `/tmp/mcp-registry` | Local checkout path for the registry | -| `MCP_REGISTRY_IMAGE` | `ghcr.io/modelcontextprotocol/registry:main` | Registry container image | -| `MCP_REGISTRY_DATA_DIR` | _(checkout `./data`)_ | Host directory mounted at `/data` instead of the [default seed data](https://github.com/modelcontextprotocol/registry/blob/main/data/seed.json). When set, seeds from `data/seed.json` with validation off. | +| Variable | Default | Description | +| ------------------------------- | -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `REPO_DIR` | `/tmp/mcp-registry` | Local checkout path for the registry | +| `MCP_REGISTRY_IMAGE` | `ghcr.io/modelcontextprotocol/registry:main` | Registry container image | +| `MCP_REGISTRY_DATA_DIR` | _(checkout `./data`)_ | Host directory mounted at `/data` instead of the [default seed data](https://github.com/modelcontextprotocol/registry/blob/main/data/seed.json). When set, seeds from `data/seed.json` with validation off. | +| `MCP_REGISTRY_URL` | `http://localhost:8080` | URL probed for readiness (and typically used as `catalog.providers.mcpRegistry.baseUrl`) | +| `MCP_REGISTRY_READY_TIMEOUT_MS` | `300000` (5m) | How long `start-mcp-registry` waits for the API before failing | Example with custom seed content (directory must contain `seed.json`): diff --git a/workspaces/ai-integrations/hack/deploy-mcp-registry.ts b/workspaces/ai-integrations/hack/deploy-mcp-registry.ts index 2b7cd213c3e..abc61225bed 100755 --- a/workspaces/ai-integrations/hack/deploy-mcp-registry.ts +++ b/workspaces/ai-integrations/hack/deploy-mcp-registry.ts @@ -45,6 +45,11 @@ const IMAGE = process.env.MCP_REGISTRY_IMAGE?.trim() || 'ghcr.io/modelcontextprotocol/registry:main'; const DATA_DIR = process.env.MCP_REGISTRY_DATA_DIR?.trim(); +const REGISTRY_URL = + process.env.MCP_REGISTRY_URL?.trim() || 'http://localhost:8080'; +const READY_TIMEOUT_MS = Number( + process.env.MCP_REGISTRY_READY_TIMEOUT_MS?.trim() || 300_000, +); function commandExists(command: string): boolean { return ( @@ -100,6 +105,31 @@ function buildOverrideYaml(image: string, dataDir?: string): string { return `${lines.join('\n')}\n`; } +/** + * Block until the registry HTTP API answers. `compose up -d` returns before + * migrations/seed finish; the process only listens on :8080 after import. + */ +function waitForRegistryReady(baseUrl: string, timeoutMs: number): void { + const probeUrl = `${baseUrl.replace(/\/$/, '')}/v0.1/servers?limit=1`; + const deadline = Date.now() + timeoutMs; + console.log(`Waiting for MCP Registry at ${probeUrl}...`); + while (Date.now() < deadline) { + const probe = spawnSync( + 'curl', + ['-sf', '--connect-timeout', '1', '--max-time', '3', probeUrl], + { encoding: 'utf8' }, + ); + if (probe.status === 0) { + console.log('MCP Registry is ready.'); + return; + } + spawnSync('sleep', ['1']); + } + throw new Error( + `MCP Registry did not become ready at ${probeUrl} within ${timeoutMs}ms`, + ); +} + if (!existsSync(join(REPO_DIR, '.git'))) { const result = spawnSync( 'git', @@ -140,7 +170,7 @@ if (dataDir) { try { const seedNote = dataDir ? ` with data from ${dataDir}` : ''; console.log( - `Starting MCP Registry from ${IMAGE}${seedNote} (http://localhost:8080)...`, + `Starting MCP Registry from ${IMAGE}${seedNote} (${REGISTRY_URL})...`, ); const [bin, ...prefix] = compose; const result = spawnSync( @@ -151,6 +181,12 @@ try { if (result.status !== 0) { process.exit(result.status ?? 1); } + try { + waitForRegistryReady(REGISTRY_URL, READY_TIMEOUT_MS); + } catch (error) { + console.error(`error: ${error instanceof Error ? error.message : error}`); + process.exit(1); + } console.log( `MCP Registry started in background. Use '${compose.join( ' ', From 3842921ab19f3d2a447e83cce0501509ecf77fb6 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 17:41:17 -0400 Subject: [PATCH 40/63] chore(#4815): rename hack/ to scripts/ for workspace consistency Align local MCP Registry tooling with the scripts/ convention used by other workspaces and update yarn/docs references. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../ai-integrations/docs/deploy-mcp-registry-locally.md | 6 +++--- workspaces/ai-integrations/package.json | 6 +++--- .../package.json | 4 ++-- workspaces/ai-integrations/{hack => scripts}/.eslintrc.js | 0 .../{hack => scripts}/deploy-mcp-registry.ts | 0 .../{hack => scripts}/undeploy-mcp-registry.ts | 0 6 files changed, 8 insertions(+), 8 deletions(-) rename workspaces/ai-integrations/{hack => scripts}/.eslintrc.js (100%) rename workspaces/ai-integrations/{hack => scripts}/deploy-mcp-registry.ts (100%) rename workspaces/ai-integrations/{hack => scripts}/undeploy-mcp-registry.ts (100%) diff --git a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md index 0dbfbc5a4ff..f055263a347 100644 --- a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md +++ b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md @@ -1,7 +1,7 @@ # Deploy MCP Registry Locally For local provider development against a real registry instance, this workspace -includes Node scripts under [`hack/`](../hack/) that start the upstream +includes Node scripts under [`scripts/`](../scripts/) that start the upstream [MCP Registry](https://github.com/modelcontextprotocol/registry) with Podman or Docker Compose. They use the published `ghcr.io/modelcontextprotocol/registry` image instead of upstream @@ -25,7 +25,7 @@ yarn start-mcp-registry You can also run the script directly from the workspace root: ```bash -node hack/deploy-mcp-registry.ts +node scripts/deploy-mcp-registry.ts ``` This clones the registry into `/tmp/mcp-registry` (if needed), starts PostgreSQL @@ -94,7 +94,7 @@ yarn stop-mcp-registry Or from the workspace root: ```bash -node hack/undeploy-mcp-registry.ts +node scripts/undeploy-mcp-registry.ts ``` This runs `compose down` for the same stack. The `/tmp/mcp-registry` checkout is diff --git a/workspaces/ai-integrations/package.json b/workspaces/ai-integrations/package.json index b845f04e296..5c52f54a39f 100644 --- a/workspaces/ai-integrations/package.json +++ b/workspaces/ai-integrations/package.json @@ -10,8 +10,8 @@ "dev:debug": "yarn workspaces foreach -A --include backend --include app --parallel -v -i run start --inspect", "start": "yarn workspace app start", "start-backend": "yarn workspace backend start", - "start-mcp-registry": "node hack/deploy-mcp-registry.ts", - "stop-mcp-registry": "node hack/undeploy-mcp-registry.ts", + "start-mcp-registry": "node scripts/deploy-mcp-registry.ts", + "stop-mcp-registry": "node scripts/undeploy-mcp-registry.ts", "build:backend": "yarn workspace backend build", "tsc": "tsc", "tsc:full": "tsc --skipLibCheck true --incremental false", @@ -66,7 +66,7 @@ }, "prettier": "@backstage/cli/config/prettier", "lint-staged": { - "{packages,plugins,hack}/**/*.{js,jsx,ts,tsx,mjs,cjs}": [ + "{packages,plugins,scripts}/**/*.{js,jsx,ts,tsx,mjs,cjs}": [ "eslint --fix", "prettier --write" ], diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json index 4d6459bb372..02a903fe590 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json @@ -22,8 +22,8 @@ }, "scripts": { "start": "backstage-cli package start", - "start-mcp-registry": "node ../../hack/deploy-mcp-registry.ts", - "stop-mcp-registry": "node ../../hack/undeploy-mcp-registry.ts", + "start-mcp-registry": "node ../../scripts/deploy-mcp-registry.ts", + "stop-mcp-registry": "node ../../scripts/undeploy-mcp-registry.ts", "build": "backstage-cli package build", "lint": "backstage-cli package lint", "lint:check": "backstage-cli package lint", diff --git a/workspaces/ai-integrations/hack/.eslintrc.js b/workspaces/ai-integrations/scripts/.eslintrc.js similarity index 100% rename from workspaces/ai-integrations/hack/.eslintrc.js rename to workspaces/ai-integrations/scripts/.eslintrc.js diff --git a/workspaces/ai-integrations/hack/deploy-mcp-registry.ts b/workspaces/ai-integrations/scripts/deploy-mcp-registry.ts similarity index 100% rename from workspaces/ai-integrations/hack/deploy-mcp-registry.ts rename to workspaces/ai-integrations/scripts/deploy-mcp-registry.ts diff --git a/workspaces/ai-integrations/hack/undeploy-mcp-registry.ts b/workspaces/ai-integrations/scripts/undeploy-mcp-registry.ts similarity index 100% rename from workspaces/ai-integrations/hack/undeploy-mcp-registry.ts rename to workspaces/ai-integrations/scripts/undeploy-mcp-registry.ts From 924d0e830434dd1afc7bd31db540fe2a4ceb96d2 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 17:48:28 -0400 Subject: [PATCH 41/63] chore(#4815): move MCP registry examples and type docs to workspace Relocate server-json fixtures under examples/mcp-registry and server-json-types.md under docs/, and update mapping package links. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../ai-integrations/docs/server-json-types.md | 179 ++++++++++++++++++ .../mcp-registry}/server-json/README.md | 0 .../server-json/npm-oci.server.json | 0 .../mcp-registry}/server-json/npm.server.json | 0 .../server-json/nuget-positional.server.json | 0 .../server-json/remote.server.json | 0 .../README.md | 7 +- .../docs/server-json-types.md | 179 ------------------ 8 files changed, 183 insertions(+), 182 deletions(-) create mode 100644 workspaces/ai-integrations/docs/server-json-types.md rename workspaces/ai-integrations/{plugins/catalog-mcp-registry-server-mapping/examples => examples/mcp-registry}/server-json/README.md (100%) rename workspaces/ai-integrations/{plugins/catalog-mcp-registry-server-mapping/examples => examples/mcp-registry}/server-json/npm-oci.server.json (100%) rename workspaces/ai-integrations/{plugins/catalog-mcp-registry-server-mapping/examples => examples/mcp-registry}/server-json/npm.server.json (100%) rename workspaces/ai-integrations/{plugins/catalog-mcp-registry-server-mapping/examples => examples/mcp-registry}/server-json/nuget-positional.server.json (100%) rename workspaces/ai-integrations/{plugins/catalog-mcp-registry-server-mapping/examples => examples/mcp-registry}/server-json/remote.server.json (100%) delete mode 100644 workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/docs/server-json-types.md diff --git a/workspaces/ai-integrations/docs/server-json-types.md b/workspaces/ai-integrations/docs/server-json-types.md new file mode 100644 index 00000000000..0bcef4c36b4 --- /dev/null +++ b/workspaces/ai-integrations/docs/server-json-types.md @@ -0,0 +1,179 @@ +# `server.json` types + +TypeScript shapes for MCP Registry **v1.8.1** +[`server.schema.json`](https://raw.githubusercontent.com/modelcontextprotocol/registry/v1.8.1/docs/reference/server-json/draft/server.schema.json) +live in [`src/types.ts`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts). Each table below mirrors one exported +type; headings link to the declaration in source. + +## [`McpServerDocument`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L279) + +Root `server.json` document (`ServerDetail`). Closed shape — only these fields +are allowed. + +| Field | Type | Required | Description | +| ------------- | ------------------------------------------------------------------------------------------ | -------- | --------------------------------------------------------------- | +| `$schema` | `string` | yes | Absolute JSON Schema URI whose basename is `server.schema.json` | +| `name` | `string` | yes | Reverse-DNS server name (`namespace/name`, exactly one `/`) | +| `title` | `string` | no | Optional human-readable display name | +| `description` | `string` | yes | Human-readable explanation of server capabilities | +| `version` | `string` | yes | Server version (semver preferred; ranges rejected) | +| `websiteUrl` | `string` | no | Homepage / docs / project website URL | +| `repository` | [`McpServerRepository`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L176) | no | Source repository metadata | +| `remotes` | [`McpRegistryRemote`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L161)[] | no | Remote transports (`streamable-http` / `sse`) | +| `icons` | [`McpRegistryIcon`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L208)[] | no | UI icons | +| `packages` | [`McpRegistryPackage`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L230)[] | no | Installable package entries | +| `_meta` | [`McpServerMeta`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L260) | no | Reverse-DNS extension metadata | + +## [`McpServerRepository`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L176) + +Repository metadata for browsing and cloning source (`Repository`). + +| Field | Type | Required | Description | +| ----------- | -------- | -------- | ------------------------------------------------------------ | +| `url` | `string` | yes | Repository URL (web browse and git clone) | +| `source` | `string` | yes | Hosting service id (`github`, `gitlab`, `bitbucket`, …) | +| `id` | `string` | no | Hosting-service repo id (stable across renames) | +| `subfolder` | `string` | no | Clean relative path from repo root to the server (monorepos) | + +## [`McpRegistryRemote`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L161) + +Remote transport entry (`RemoteTransport`): `streamable-http` or `sse`, plus +optional URL template variables. + +| Field | Type | Required | Description | +| ----------- | ---------------------------------------------------------------------------------------------------------- | -------- | --------------------------------- | +| `type` | `'streamable-http' \| 'sse'` | yes | Remote transport kind | +| `url` | `string` | yes | Endpoint URL template | +| `headers` | [`McpKeyValueInput`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L62)[] | no | Optional HTTP headers | +| `variables` | `Record` ([`McpInput`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L26)) | no | URL template variable definitions | + +## [`McpRegistryIcon`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L208) + +Icon resource for client UIs (`Icon`). + +| Field | Type | Required | Description | +| ---------- | ------------------------------------------------------------------------------- | -------- | -------------------------------- | +| `src` | `string` | yes | URI of the icon resource | +| `mimeType` | `'image/png' \| 'image/jpeg' \| 'image/jpg' \| 'image/svg+xml' \| 'image/webp'` | no | MIME type override | +| `sizes` | `string[]` | no | Size hints (e.g. `48x48`, `any`) | +| `theme` | `'light' \| 'dark'` | no | Theme the icon is designed for | + +## [`McpRegistryPackage`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L230) + +Installable package entry (`Package`). + +| Field | Type | Required | Description | +| ---------------------- | --------------------------------------------------------------------------------------- | -------- | ------------------------------------------------ | +| `registryType` | `string` | yes | Registry kind (`npm`, `pypi`, `cargo`, `oci`, …) | +| `identifier` | `string` | yes | Package name or download URL | +| `transport` | [`McpLocalTransport`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L147) | yes | Local / package transport config | +| `version` | `string` | no | Specific package version (no ranges) | +| `registryBaseUrl` | `string` | no | Base URL of the package registry | +| `runtimeHint` | `string` | no | Runtime hint (`npx`, `uvx`, `docker`, …) | +| `fileSha256` | `string` | no | SHA-256 of the package file | +| `environmentVariables` | [`McpKeyValueInput`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L62)[] | no | Environment variables for the package | +| `packageArguments` | [`McpArgument`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L101)[] | no | Arguments for the package binary | +| `runtimeArguments` | [`McpArgument`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L101)[] | no | Arguments for the runtime command | + +## [`McpServerMeta`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L260) + +Extension metadata (`ServerDetail._meta`) with reverse-DNS keys. + +| Field | Type | Required | Description | +| ----------------------------------------------------- | ------------------------- | -------- | -------------------------------------------- | +| `io.modelcontextprotocol.registry/publisher-provided` | `Record` | no | Publisher metadata for downstream registries | +| `[key: string]` | `unknown` | no | Additional reverse-DNS namespaced extensions | + +## [`McpLocalTransport`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L147) + +Local / package transport union (`LocalTransport`). + +| Variant | `type` | Description | +| ------------------------------------------------------------------------------------------------ | ------------------- | ---------------------------- | +| [`McpStdioTransport`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L109) | `'stdio'` | Stdio local transport | +| [`McpStreamableHttpTransport`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L119) | `'streamable-http'` | Streamable HTTP transport | +| [`McpSseTransport`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L133) | `'sse'` | Server-Sent Events transport | + +### [`McpStdioTransport`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L109) + +| Field | Type | Required | Description | +| ------ | --------- | -------- | ----------- | +| `type` | `'stdio'` | yes | Literal | + +### [`McpStreamableHttpTransport`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L119) + +| Field | Type | Required | Description | +| --------- | --------------------------------------------------------------------------------------- | -------- | --------------------- | +| `type` | `'streamable-http'` | yes | Literal | +| `url` | `string` | yes | URL template | +| `headers` | [`McpKeyValueInput`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L62)[] | no | Optional HTTP headers | + +### [`McpSseTransport`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L133) + +| Field | Type | Required | Description | +| --------- | --------------------------------------------------------------------------------------- | -------- | ------------------------- | +| `type` | `'sse'` | yes | Literal | +| `url` | `string` | yes | SSE endpoint URL template | +| `headers` | [`McpKeyValueInput`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L62)[] | no | Optional HTTP headers | + +## [`McpInput`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L26) + +Shared input leaf (`Input`) used by env vars, headers, variables, and arguments. + +| Field | Type | Required | Description | +| ------------- | ------------------------------------------------- | -------- | -------------------------------------------- | +| `choices` | `string[]` | no | Allowed values the user must select from | +| `default` | `string` | no | Default value | +| `description` | `string` | no | Human-readable description for clients | +| `format` | `'string' \| 'number' \| 'boolean' \| 'filepath'` | no | Input format hint | +| `isRequired` | `boolean` | no | Whether the input is required | +| `isSecret` | `boolean` | no | Whether the input is a secret value | +| `placeholder` | `string` | no | Placeholder shown during configuration | +| `value` | `string` | no | Fixed value (end users should not configure) | + +## [`McpInputWithVariables`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L51) + +Extends [`McpInput`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L26) with nested `{curly_brace}` variables +(`InputWithVariables`). + +| Field | Type | Required | Description | +| ----------- | ---------------------------------------------------------------------------------------------------------- | -------- | --------------------------------- | +| `variables` | `Record` ([`McpInput`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L26)) | no | Nested variable input definitions | + +## [`McpKeyValueInput`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L62) + +Named key/value input for env vars or headers (`KeyValueInput`). Extends +[`McpInputWithVariables`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L51). + +| Field | Type | Required | Description | +| ------ | -------- | -------- | ----------------------------------- | +| `name` | `string` | yes | Header or environment variable name | + +## [`McpArgument`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L101) + +Package or runtime argument union (`Argument`). + +| Variant | `type` | Description | +| ------------------------------------------------------------------------------------------ | -------------- | -------------------------------- | +| [`McpPositionalArgument`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L73) | `'positional'` | Positional command-line argument | +| [`McpNamedArgument`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L87) | `'named'` | Named flag (`--flag={value}`) | + +### [`McpPositionalArgument`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L73) + +Extends [`McpInputWithVariables`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L51). + +| Field | Type | Required | Description | +| ------------ | -------------- | -------- | ------------------------------------ | +| `type` | `'positional'` | yes | Literal | +| `isRepeated` | `boolean` | no | Whether the argument may be repeated | +| `valueHint` | `string` | no | Identifier / label for the argument | + +### [`McpNamedArgument`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L87) + +Extends [`McpInputWithVariables`](../plugins/catalog-mcp-registry-server-mapping/src/types.ts#L51). + +| Field | Type | Required | Description | +| ------------ | --------- | -------- | ------------------------------------ | +| `type` | `'named'` | yes | Literal | +| `name` | `string` | yes | Flag name, including leading dashes | +| `isRepeated` | `boolean` | no | Whether the argument may be repeated | diff --git a/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/README.md b/workspaces/ai-integrations/examples/mcp-registry/server-json/README.md similarity index 100% rename from workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/README.md rename to workspaces/ai-integrations/examples/mcp-registry/server-json/README.md diff --git a/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/npm-oci.server.json b/workspaces/ai-integrations/examples/mcp-registry/server-json/npm-oci.server.json similarity index 100% rename from workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/npm-oci.server.json rename to workspaces/ai-integrations/examples/mcp-registry/server-json/npm-oci.server.json diff --git a/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/npm.server.json b/workspaces/ai-integrations/examples/mcp-registry/server-json/npm.server.json similarity index 100% rename from workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/npm.server.json rename to workspaces/ai-integrations/examples/mcp-registry/server-json/npm.server.json diff --git a/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/nuget-positional.server.json b/workspaces/ai-integrations/examples/mcp-registry/server-json/nuget-positional.server.json similarity index 100% rename from workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/nuget-positional.server.json rename to workspaces/ai-integrations/examples/mcp-registry/server-json/nuget-positional.server.json diff --git a/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/remote.server.json b/workspaces/ai-integrations/examples/mcp-registry/server-json/remote.server.json similarity index 100% rename from workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/examples/server-json/remote.server.json rename to workspaces/ai-integrations/examples/mcp-registry/server-json/remote.server.json diff --git a/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/README.md b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/README.md index 185e1534846..44be1652580 100644 --- a/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/README.md +++ b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/README.md @@ -78,7 +78,8 @@ fields) with an actionable message. ## `server.json` types Field-level breakdowns of the MCP Registry **v1.8.1** `server.json` TypeScript -shapes live in [`docs/server-json-types.md`](./docs/server-json-types.md) +shapes live in +[`docs/server-json-types.md`](../../docs/server-json-types.md) (source of truth: [`src/types.ts`](./src/types.ts)). ### Caller defaults (`McpServerMappingDefaults`) @@ -103,8 +104,8 @@ Design decisions and scenarios live under ## Examples -See [`examples/server-json/`](./examples/server-json/) for rewritten MCP Registry -`server.json` fixtures useful for local testing. +See [`examples/mcp-registry/server-json/`](../../examples/mcp-registry/server-json/) +for rewritten MCP Registry `server.json` fixtures useful for local testing. ## Development diff --git a/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/docs/server-json-types.md b/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/docs/server-json-types.md deleted file mode 100644 index 5170b80346d..00000000000 --- a/workspaces/ai-integrations/plugins/catalog-mcp-registry-server-mapping/docs/server-json-types.md +++ /dev/null @@ -1,179 +0,0 @@ -# `server.json` types - -TypeScript shapes for MCP Registry **v1.8.1** -[`server.schema.json`](https://raw.githubusercontent.com/modelcontextprotocol/registry/v1.8.1/docs/reference/server-json/draft/server.schema.json) -live in [`src/types.ts`](../src/types.ts). Each table below mirrors one exported -type; headings link to the declaration in source. - -## [`McpServerDocument`](../src/types.ts#L279) - -Root `server.json` document (`ServerDetail`). Closed shape — only these fields -are allowed. - -| Field | Type | Required | Description | -| ------------- | ---------------------------------------------- | -------- | --------------------------------------------------------------- | -| `$schema` | `string` | yes | Absolute JSON Schema URI whose basename is `server.schema.json` | -| `name` | `string` | yes | Reverse-DNS server name (`namespace/name`, exactly one `/`) | -| `title` | `string` | no | Optional human-readable display name | -| `description` | `string` | yes | Human-readable explanation of server capabilities | -| `version` | `string` | yes | Server version (semver preferred; ranges rejected) | -| `websiteUrl` | `string` | no | Homepage / docs / project website URL | -| `repository` | [`McpServerRepository`](../src/types.ts#L176) | no | Source repository metadata | -| `remotes` | [`McpRegistryRemote`](../src/types.ts#L161)[] | no | Remote transports (`streamable-http` / `sse`) | -| `icons` | [`McpRegistryIcon`](../src/types.ts#L208)[] | no | UI icons | -| `packages` | [`McpRegistryPackage`](../src/types.ts#L230)[] | no | Installable package entries | -| `_meta` | [`McpServerMeta`](../src/types.ts#L260) | no | Reverse-DNS extension metadata | - -## [`McpServerRepository`](../src/types.ts#L176) - -Repository metadata for browsing and cloning source (`Repository`). - -| Field | Type | Required | Description | -| ----------- | -------- | -------- | ------------------------------------------------------------ | -| `url` | `string` | yes | Repository URL (web browse and git clone) | -| `source` | `string` | yes | Hosting service id (`github`, `gitlab`, `bitbucket`, …) | -| `id` | `string` | no | Hosting-service repo id (stable across renames) | -| `subfolder` | `string` | no | Clean relative path from repo root to the server (monorepos) | - -## [`McpRegistryRemote`](../src/types.ts#L161) - -Remote transport entry (`RemoteTransport`): `streamable-http` or `sse`, plus -optional URL template variables. - -| Field | Type | Required | Description | -| ----------- | -------------------------------------------------------------- | -------- | --------------------------------- | -| `type` | `'streamable-http' \| 'sse'` | yes | Remote transport kind | -| `url` | `string` | yes | Endpoint URL template | -| `headers` | [`McpKeyValueInput`](../src/types.ts#L62)[] | no | Optional HTTP headers | -| `variables` | `Record` ([`McpInput`](../src/types.ts#L26)) | no | URL template variable definitions | - -## [`McpRegistryIcon`](../src/types.ts#L208) - -Icon resource for client UIs (`Icon`). - -| Field | Type | Required | Description | -| ---------- | ------------------------------------------------------------------------------- | -------- | -------------------------------- | -| `src` | `string` | yes | URI of the icon resource | -| `mimeType` | `'image/png' \| 'image/jpeg' \| 'image/jpg' \| 'image/svg+xml' \| 'image/webp'` | no | MIME type override | -| `sizes` | `string[]` | no | Size hints (e.g. `48x48`, `any`) | -| `theme` | `'light' \| 'dark'` | no | Theme the icon is designed for | - -## [`McpRegistryPackage`](../src/types.ts#L230) - -Installable package entry (`Package`). - -| Field | Type | Required | Description | -| ---------------------- | ------------------------------------------- | -------- | ------------------------------------------------ | -| `registryType` | `string` | yes | Registry kind (`npm`, `pypi`, `cargo`, `oci`, …) | -| `identifier` | `string` | yes | Package name or download URL | -| `transport` | [`McpLocalTransport`](../src/types.ts#L147) | yes | Local / package transport config | -| `version` | `string` | no | Specific package version (no ranges) | -| `registryBaseUrl` | `string` | no | Base URL of the package registry | -| `runtimeHint` | `string` | no | Runtime hint (`npx`, `uvx`, `docker`, …) | -| `fileSha256` | `string` | no | SHA-256 of the package file | -| `environmentVariables` | [`McpKeyValueInput`](../src/types.ts#L62)[] | no | Environment variables for the package | -| `packageArguments` | [`McpArgument`](../src/types.ts#L101)[] | no | Arguments for the package binary | -| `runtimeArguments` | [`McpArgument`](../src/types.ts#L101)[] | no | Arguments for the runtime command | - -## [`McpServerMeta`](../src/types.ts#L260) - -Extension metadata (`ServerDetail._meta`) with reverse-DNS keys. - -| Field | Type | Required | Description | -| ----------------------------------------------------- | ------------------------- | -------- | -------------------------------------------- | -| `io.modelcontextprotocol.registry/publisher-provided` | `Record` | no | Publisher metadata for downstream registries | -| `[key: string]` | `unknown` | no | Additional reverse-DNS namespaced extensions | - -## [`McpLocalTransport`](../src/types.ts#L147) - -Local / package transport union (`LocalTransport`). - -| Variant | `type` | Description | -| ---------------------------------------------------- | ------------------- | ---------------------------- | -| [`McpStdioTransport`](../src/types.ts#L109) | `'stdio'` | Stdio local transport | -| [`McpStreamableHttpTransport`](../src/types.ts#L119) | `'streamable-http'` | Streamable HTTP transport | -| [`McpSseTransport`](../src/types.ts#L133) | `'sse'` | Server-Sent Events transport | - -### [`McpStdioTransport`](../src/types.ts#L109) - -| Field | Type | Required | Description | -| ------ | --------- | -------- | ----------- | -| `type` | `'stdio'` | yes | Literal | - -### [`McpStreamableHttpTransport`](../src/types.ts#L119) - -| Field | Type | Required | Description | -| --------- | ------------------------------------------- | -------- | --------------------- | -| `type` | `'streamable-http'` | yes | Literal | -| `url` | `string` | yes | URL template | -| `headers` | [`McpKeyValueInput`](../src/types.ts#L62)[] | no | Optional HTTP headers | - -### [`McpSseTransport`](../src/types.ts#L133) - -| Field | Type | Required | Description | -| --------- | ------------------------------------------- | -------- | ------------------------- | -| `type` | `'sse'` | yes | Literal | -| `url` | `string` | yes | SSE endpoint URL template | -| `headers` | [`McpKeyValueInput`](../src/types.ts#L62)[] | no | Optional HTTP headers | - -## [`McpInput`](../src/types.ts#L26) - -Shared input leaf (`Input`) used by env vars, headers, variables, and arguments. - -| Field | Type | Required | Description | -| ------------- | ------------------------------------------------- | -------- | -------------------------------------------- | -| `choices` | `string[]` | no | Allowed values the user must select from | -| `default` | `string` | no | Default value | -| `description` | `string` | no | Human-readable description for clients | -| `format` | `'string' \| 'number' \| 'boolean' \| 'filepath'` | no | Input format hint | -| `isRequired` | `boolean` | no | Whether the input is required | -| `isSecret` | `boolean` | no | Whether the input is a secret value | -| `placeholder` | `string` | no | Placeholder shown during configuration | -| `value` | `string` | no | Fixed value (end users should not configure) | - -## [`McpInputWithVariables`](../src/types.ts#L51) - -Extends [`McpInput`](../src/types.ts#L26) with nested `{curly_brace}` variables -(`InputWithVariables`). - -| Field | Type | Required | Description | -| ----------- | -------------------------------------------------------------- | -------- | --------------------------------- | -| `variables` | `Record` ([`McpInput`](../src/types.ts#L26)) | no | Nested variable input definitions | - -## [`McpKeyValueInput`](../src/types.ts#L62) - -Named key/value input for env vars or headers (`KeyValueInput`). Extends -[`McpInputWithVariables`](../src/types.ts#L51). - -| Field | Type | Required | Description | -| ------ | -------- | -------- | ----------------------------------- | -| `name` | `string` | yes | Header or environment variable name | - -## [`McpArgument`](../src/types.ts#L101) - -Package or runtime argument union (`Argument`). - -| Variant | `type` | Description | -| ---------------------------------------------- | -------------- | -------------------------------- | -| [`McpPositionalArgument`](../src/types.ts#L73) | `'positional'` | Positional command-line argument | -| [`McpNamedArgument`](../src/types.ts#L87) | `'named'` | Named flag (`--flag={value}`) | - -### [`McpPositionalArgument`](../src/types.ts#L73) - -Extends [`McpInputWithVariables`](../src/types.ts#L51). - -| Field | Type | Required | Description | -| ------------ | -------------- | -------- | ------------------------------------ | -| `type` | `'positional'` | yes | Literal | -| `isRepeated` | `boolean` | no | Whether the argument may be repeated | -| `valueHint` | `string` | no | Identifier / label for the argument | - -### [`McpNamedArgument`](../src/types.ts#L87) - -Extends [`McpInputWithVariables`](../src/types.ts#L51). - -| Field | Type | Required | Description | -| ------------ | --------- | -------- | ------------------------------------ | -| `type` | `'named'` | yes | Literal | -| `name` | `string` | yes | Flag name, including leading dashes | -| `isRepeated` | `boolean` | no | Whether the argument may be repeated | From 953020e1f1242ee95f16c73a6c202c015c2f0970 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 17:56:03 -0400 Subject: [PATCH 42/63] fix(#4815): harden MCP Registry scripts for SonarCloud Resolve binaries from fixed directories and keep checkout/temp files under ~/.cache instead of world-writable /tmp. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../docs/deploy-mcp-registry-locally.md | 18 +++-- .../scripts/deploy-mcp-registry.ts | 70 ++++++++++++----- .../scripts/undeploy-mcp-registry.ts | 75 +++++++++++++------ 3 files changed, 116 insertions(+), 47 deletions(-) diff --git a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md index f055263a347..97f8ae092e3 100644 --- a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md +++ b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md @@ -28,10 +28,11 @@ You can also run the script directly from the workspace root: node scripts/deploy-mcp-registry.ts ``` -This clones the registry into `/tmp/mcp-registry` (if needed), starts PostgreSQL -and the registry in the background, **waits until the HTTP API responds** (seed -import can take a few minutes when seeding from the public registry), and serves -the API at [http://localhost:8080](http://localhost:8080). +This clones the registry into `~/.cache/rhdh-ai-integrations/mcp-registry` +(if needed; override with `REPO_DIR`), starts PostgreSQL and the registry in the +background, **waits until the HTTP API responds** (seed import can take a few +minutes when seeding from the public registry), and serves the API at +[http://localhost:8080](http://localhost:8080). Start the registry **before** `yarn dev`. If the provider syncs while the registry is still importing seed data, you will see @@ -42,7 +43,7 @@ Optional environment variables: | Variable | Default | Description | | ------------------------------- | -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `REPO_DIR` | `/tmp/mcp-registry` | Local checkout path for the registry | +| `REPO_DIR` | `~/.cache/rhdh-ai-integrations/mcp-registry` | Local checkout path for the registry | | `MCP_REGISTRY_IMAGE` | `ghcr.io/modelcontextprotocol/registry:main` | Registry container image | | `MCP_REGISTRY_DATA_DIR` | _(checkout `./data`)_ | Host directory mounted at `/data` instead of the [default seed data](https://github.com/modelcontextprotocol/registry/blob/main/data/seed.json). When set, seeds from `data/seed.json` with validation off. | | `MCP_REGISTRY_URL` | `http://localhost:8080` | URL probed for readiness (and typically used as `catalog.providers.mcpRegistry.baseUrl`) | @@ -57,7 +58,7 @@ MCP_REGISTRY_DATA_DIR=./examples/mcp-registry/seed-data yarn start-mcp-registry View logs (example with Podman): ```bash -podman compose -f /tmp/mcp-registry/docker-compose.yml logs -f +podman compose -f ~/.cache/rhdh-ai-integrations/mcp-registry/docker-compose.yml logs -f ``` ## Point the provider at localhost @@ -97,5 +98,6 @@ Or from the workspace root: node scripts/undeploy-mcp-registry.ts ``` -This runs `compose down` for the same stack. The `/tmp/mcp-registry` checkout is -left in place so the next deploy is faster. +This runs `compose down` for the same stack. The +`~/.cache/rhdh-ai-integrations/mcp-registry` checkout is left in place so the +next deploy is faster. diff --git a/workspaces/ai-integrations/scripts/deploy-mcp-registry.ts b/workspaces/ai-integrations/scripts/deploy-mcp-registry.ts index abc61225bed..aeebaffff3c 100755 --- a/workspaces/ai-integrations/scripts/deploy-mcp-registry.ts +++ b/workspaces/ai-integrations/scripts/deploy-mcp-registry.ts @@ -32,15 +32,22 @@ const { spawnSync } = require('node:child_process'); const { existsSync, + mkdirSync, mkdtempSync, rmSync, statSync, writeFileSync, } = require('node:fs'); -const { tmpdir } = require('node:os'); +const { homedir } = require('node:os'); const { join, resolve } = require('node:path'); -const REPO_DIR = process.env.REPO_DIR?.trim() || '/tmp/mcp-registry'; +/** Fixed, typically non-writable dirs — avoid PATH-based binary lookup (S4036). */ +const SAFE_BIN_DIRS = ['/usr/bin', '/bin', '/usr/local/bin']; + +const CACHE_ROOT = join(homedir(), '.cache', 'rhdh-ai-integrations'); +const DEFAULT_REPO_DIR = join(CACHE_ROOT, 'mcp-registry'); + +const REPO_DIR = process.env.REPO_DIR?.trim() || DEFAULT_REPO_DIR; const IMAGE = process.env.MCP_REGISTRY_IMAGE?.trim() || 'ghcr.io/modelcontextprotocol/registry:main'; @@ -51,25 +58,40 @@ const READY_TIMEOUT_MS = Number( process.env.MCP_REGISTRY_READY_TIMEOUT_MS?.trim() || 300_000, ); -function commandExists(command: string): boolean { - return ( - spawnSync('sh', ['-c', `command -v "${command}" >/dev/null 2>&1`]) - .status === 0 - ); +function findBinary(name: string): string | undefined { + for (const dir of SAFE_BIN_DIRS) { + const candidate = join(dir, name); + if (existsSync(candidate)) { + return candidate; + } + } + return undefined; } -function composeVersionOk(bin: string): boolean { +function requireBinary(name: string): string { + const path = findBinary(name); + if (!path) { + throw new Error( + `command not found in ${SAFE_BIN_DIRS.join(', ')}: ${name}`, + ); + } + return path; +} + +function composeVersionOk(binPath: string): boolean { return ( - spawnSync(bin, ['compose', 'version'], { stdio: 'ignore' }).status === 0 + spawnSync(binPath, ['compose', 'version'], { stdio: 'ignore' }).status === 0 ); } function resolveCompose(): [string, ...string[]] { - if (commandExists('podman') && composeVersionOk('podman')) { - return ['podman', 'compose']; + const podman = findBinary('podman'); + if (podman && composeVersionOk(podman)) { + return [podman, 'compose']; } - if (commandExists('docker') && composeVersionOk('docker')) { - return ['docker', 'compose']; + const docker = findBinary('docker'); + if (docker && composeVersionOk(docker)) { + return [docker, 'compose']; } throw new Error("need 'podman compose' or 'docker compose'"); } @@ -105,17 +127,25 @@ function buildOverrideYaml(image: string, dataDir?: string): string { return `${lines.join('\n')}\n`; } +/** Private cache dir under $HOME — avoid world-writable /tmp (S5443). */ +function createPrivateTempDir(prefix: string): string { + mkdirSync(CACHE_ROOT, { recursive: true, mode: 0o700 }); + return mkdtempSync(join(CACHE_ROOT, prefix)); +} + /** * Block until the registry HTTP API answers. `compose up -d` returns before * migrations/seed finish; the process only listens on :8080 after import. */ function waitForRegistryReady(baseUrl: string, timeoutMs: number): void { + const curl = requireBinary('curl'); + const sleep = requireBinary('sleep'); const probeUrl = `${baseUrl.replace(/\/$/, '')}/v0.1/servers?limit=1`; const deadline = Date.now() + timeoutMs; console.log(`Waiting for MCP Registry at ${probeUrl}...`); while (Date.now() < deadline) { const probe = spawnSync( - 'curl', + curl, ['-sf', '--connect-timeout', '1', '--max-time', '3', probeUrl], { encoding: 'utf8' }, ); @@ -123,7 +153,7 @@ function waitForRegistryReady(baseUrl: string, timeoutMs: number): void { console.log('MCP Registry is ready.'); return; } - spawnSync('sleep', ['1']); + spawnSync(sleep, ['1']); } throw new Error( `MCP Registry did not become ready at ${probeUrl} within ${timeoutMs}ms`, @@ -131,8 +161,9 @@ function waitForRegistryReady(baseUrl: string, timeoutMs: number): void { } if (!existsSync(join(REPO_DIR, '.git'))) { + const git = requireBinary('git'); const result = spawnSync( - 'git', + git, ['clone', 'https://github.com/modelcontextprotocol/registry.git', REPO_DIR], { stdio: 'inherit' }, ); @@ -151,9 +182,12 @@ try { process.exit(1); } -const overrideDir = mkdtempSync(join(tmpdir(), 'mcp-registry-')); +const overrideDir = createPrivateTempDir('mcp-registry-'); const overridePath = join(overrideDir, 'override.yml'); -writeFileSync(overridePath, buildOverrideYaml(IMAGE, dataDir), 'utf8'); +writeFileSync(overridePath, buildOverrideYaml(IMAGE, dataDir), { + encoding: 'utf8', + mode: 0o600, +}); const composeEnv = { ...process.env }; if (dataDir) { diff --git a/workspaces/ai-integrations/scripts/undeploy-mcp-registry.ts b/workspaces/ai-integrations/scripts/undeploy-mcp-registry.ts index cacaa2d51e2..59d17927094 100755 --- a/workspaces/ai-integrations/scripts/undeploy-mcp-registry.ts +++ b/workspaces/ai-integrations/scripts/undeploy-mcp-registry.ts @@ -18,42 +18,76 @@ /** Stop the local MCP Registry started by deploy-mcp-registry.ts. */ const { spawnSync } = require('node:child_process'); -const { existsSync, mkdtempSync, rmSync, writeFileSync } = require('node:fs'); -const { tmpdir } = require('node:os'); +const { + existsSync, + mkdirSync, + mkdtempSync, + rmSync, + writeFileSync, +} = require('node:fs'); +const { homedir } = require('node:os'); const { join } = require('node:path'); -const REPO_DIR = process.env.REPO_DIR?.trim() || '/tmp/mcp-registry'; +/** Fixed, typically non-writable dirs — avoid PATH-based binary lookup (S4036). */ +const SAFE_BIN_DIRS = ['/usr/bin', '/bin', '/usr/local/bin']; + +const CACHE_ROOT = join(homedir(), '.cache', 'rhdh-ai-integrations'); +const DEFAULT_REPO_DIR = join(CACHE_ROOT, 'mcp-registry'); + +const REPO_DIR = process.env.REPO_DIR?.trim() || DEFAULT_REPO_DIR; const IMAGE = process.env.MCP_REGISTRY_IMAGE?.trim() || 'ghcr.io/modelcontextprotocol/registry:main'; -function commandExists(command: string): boolean { - return ( - spawnSync('sh', ['-c', `command -v "${command}" >/dev/null 2>&1`]) - .status === 0 - ); +function findBinary(name: string): string | undefined { + for (const dir of SAFE_BIN_DIRS) { + const candidate = join(dir, name); + if (existsSync(candidate)) { + return candidate; + } + } + return undefined; } -function composeVersionOk(bin: string): boolean { +function requireBinary(name: string): string { + const path = findBinary(name); + if (!path) { + throw new Error( + `command not found in ${SAFE_BIN_DIRS.join(', ')}: ${name}`, + ); + } + return path; +} + +function composeVersionOk(binPath: string): boolean { return ( - spawnSync(bin, ['compose', 'version'], { stdio: 'ignore' }).status === 0 + spawnSync(binPath, ['compose', 'version'], { stdio: 'ignore' }).status === 0 ); } function resolveCompose(): [string, ...string[]] { - if (commandExists('podman') && composeVersionOk('podman')) { - return ['podman', 'compose']; + const podman = findBinary('podman'); + if (podman && composeVersionOk(podman)) { + return [podman, 'compose']; } - if (commandExists('docker') && composeVersionOk('docker')) { - return ['docker', 'compose']; + const docker = findBinary('docker'); + if (docker && composeVersionOk(docker)) { + return [docker, 'compose']; } throw new Error("need 'podman compose' or 'docker compose'"); } +/** Private cache dir under $HOME — avoid world-writable /tmp (S5443). */ +function createPrivateTempDir(prefix: string): string { + mkdirSync(CACHE_ROOT, { recursive: true, mode: 0o700 }); + return mkdtempSync(join(CACHE_ROOT, prefix)); +} + if (!existsSync(join(REPO_DIR, '.git'))) { if (!existsSync(REPO_DIR)) { + const git = requireBinary('git'); const result = spawnSync( - 'git', + git, [ 'clone', 'https://github.com/modelcontextprotocol/registry.git', @@ -78,13 +112,12 @@ try { process.exit(1); } -const overrideDir = mkdtempSync(join(tmpdir(), 'mcp-registry-')); +const overrideDir = createPrivateTempDir('mcp-registry-'); const overridePath = join(overrideDir, 'override.yml'); -writeFileSync( - overridePath, - `services:\n registry:\n image: ${IMAGE}\n`, - 'utf8', -); +writeFileSync(overridePath, `services:\n registry:\n image: ${IMAGE}\n`, { + encoding: 'utf8', + mode: 0o600, +}); try { console.log(`Stopping MCP Registry in ${REPO_DIR}...`); From dfe111620f3de4cbf4ad9c4dcf6b1ab2ecd5798d Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 18:15:48 -0400 Subject: [PATCH 43/63] chore(#4815): pin MCP Registry checkout and image via env vars Make clone URL, revision, path, image name, and image tag independently configurable, defaulting the checkout and image to the 1.8.1 release. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../docs/deploy-mcp-registry-locally.md | 25 ++++---- .../scripts/deploy-mcp-registry.ts | 53 ++++++++++++---- .../scripts/undeploy-mcp-registry.ts | 60 ++++++++++++------- 3 files changed, 98 insertions(+), 40 deletions(-) diff --git a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md index 97f8ae092e3..a0ec0933fd3 100644 --- a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md +++ b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md @@ -29,9 +29,11 @@ node scripts/deploy-mcp-registry.ts ``` This clones the registry into `~/.cache/rhdh-ai-integrations/mcp-registry` -(if needed; override with `REPO_DIR`), starts PostgreSQL and the registry in the -background, **waits until the HTTP API responds** (seed import can take a few -minutes when seeding from the public registry), and serves the API at +(if needed; override with `MCP_REGISTRY_REPO_DIR`) at tag `v1.8.1` by default +(override with `MCP_REGISTRY_REPO_URL` / `MCP_REGISTRY_REPO_REVISION`), starts +PostgreSQL and the registry in the background, **waits until the HTTP API +responds** (seed import can take a few minutes when seeding from the public +registry), and serves the API at [http://localhost:8080](http://localhost:8080). Start the registry **before** `yarn dev`. If the provider syncs while the @@ -41,13 +43,16 @@ the backend after the registry is ready, or wait for the next scheduled sync. Optional environment variables: -| Variable | Default | Description | -| ------------------------------- | -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `REPO_DIR` | `~/.cache/rhdh-ai-integrations/mcp-registry` | Local checkout path for the registry | -| `MCP_REGISTRY_IMAGE` | `ghcr.io/modelcontextprotocol/registry:main` | Registry container image | -| `MCP_REGISTRY_DATA_DIR` | _(checkout `./data`)_ | Host directory mounted at `/data` instead of the [default seed data](https://github.com/modelcontextprotocol/registry/blob/main/data/seed.json). When set, seeds from `data/seed.json` with validation off. | -| `MCP_REGISTRY_URL` | `http://localhost:8080` | URL probed for readiness (and typically used as `catalog.providers.mcpRegistry.baseUrl`) | -| `MCP_REGISTRY_READY_TIMEOUT_MS` | `300000` (5m) | How long `start-mcp-registry` waits for the API before failing | +| Variable | Default | Description | +| ------------------------------- | ------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `MCP_REGISTRY_REPO_DIR` | `~/.cache/rhdh-ai-integrations/mcp-registry` | Local checkout path for the registry | +| `MCP_REGISTRY_REPO_URL` | `https://github.com/modelcontextprotocol/registry.git` | Git remote cloned into `MCP_REGISTRY_REPO_DIR` | +| `MCP_REGISTRY_REPO_REVISION` | `v1.8.1` | Git branch or tag checked out for compose/config | +| `MCP_REGISTRY_IMAGE_NAME` | `ghcr.io/modelcontextprotocol/registry` | Registry container image name (without tag) | +| `MCP_REGISTRY_IMAGE_TAG` | `1.8.1` | Registry container image tag | +| `MCP_REGISTRY_DATA_DIR` | _(checkout `./data`)_ | Host directory mounted at `/data` instead of the [default seed data](https://github.com/modelcontextprotocol/registry/blob/main/data/seed.json). When set, seeds from `data/seed.json` with validation off. | +| `MCP_REGISTRY_URL` | `http://localhost:8080` | URL probed for readiness (and typically used as `catalog.providers.mcpRegistry.baseUrl`) | +| `MCP_REGISTRY_READY_TIMEOUT_MS` | `300000` (5m) | How long `start-mcp-registry` waits for the API before failing | Example with custom seed content (directory must contain `seed.json`): diff --git a/workspaces/ai-integrations/scripts/deploy-mcp-registry.ts b/workspaces/ai-integrations/scripts/deploy-mcp-registry.ts index aeebaffff3c..501badf1987 100755 --- a/workspaces/ai-integrations/scripts/deploy-mcp-registry.ts +++ b/workspaces/ai-integrations/scripts/deploy-mcp-registry.ts @@ -47,10 +47,17 @@ const SAFE_BIN_DIRS = ['/usr/bin', '/bin', '/usr/local/bin']; const CACHE_ROOT = join(homedir(), '.cache', 'rhdh-ai-integrations'); const DEFAULT_REPO_DIR = join(CACHE_ROOT, 'mcp-registry'); -const REPO_DIR = process.env.REPO_DIR?.trim() || DEFAULT_REPO_DIR; -const IMAGE = - process.env.MCP_REGISTRY_IMAGE?.trim() || - 'ghcr.io/modelcontextprotocol/registry:main'; +const REPO_DIR = process.env.MCP_REGISTRY_REPO_DIR?.trim() || DEFAULT_REPO_DIR; +const REPO_URL = + process.env.MCP_REGISTRY_REPO_URL?.trim() || + 'https://github.com/modelcontextprotocol/registry.git'; +const REPO_REVISION = + process.env.MCP_REGISTRY_REPO_REVISION?.trim() || 'v1.8.1'; +const IMAGE_NAME = + process.env.MCP_REGISTRY_IMAGE_NAME?.trim() || + 'ghcr.io/modelcontextprotocol/registry'; +const IMAGE_TAG = process.env.MCP_REGISTRY_IMAGE_TAG?.trim() || '1.8.1'; +const IMAGE = `${IMAGE_NAME}:${IMAGE_TAG}`; const DATA_DIR = process.env.MCP_REGISTRY_DATA_DIR?.trim(); const REGISTRY_URL = process.env.MCP_REGISTRY_URL?.trim() || 'http://localhost:8080'; @@ -160,18 +167,44 @@ function waitForRegistryReady(baseUrl: string, timeoutMs: number): void { ); } -if (!existsSync(join(REPO_DIR, '.git'))) { +function runGit(args: string[], cwd?: string): void { const git = requireBinary('git'); - const result = spawnSync( - git, - ['clone', 'https://github.com/modelcontextprotocol/registry.git', REPO_DIR], - { stdio: 'inherit' }, - ); + const result = spawnSync(git, args, { + cwd, + stdio: 'inherit', + }); if (result.status !== 0) { process.exit(result.status ?? 1); } } +/** Clone or update the registry checkout to MCP_REGISTRY_REPO_REVISION. */ +function ensureRegistryCheckout(): void { + if (!existsSync(join(REPO_DIR, '.git'))) { + if (existsSync(REPO_DIR)) { + console.error(`error: ${REPO_DIR} exists but is not a git repository`); + process.exit(1); + } + console.log(`Cloning ${REPO_URL} (${REPO_REVISION}) into ${REPO_DIR}...`); + runGit([ + 'clone', + '--branch', + REPO_REVISION, + '--depth', + '1', + REPO_URL, + REPO_DIR, + ]); + return; + } + + console.log(`Checking out ${REPO_REVISION} in ${REPO_DIR}...`); + runGit(['fetch', '--depth', '1', 'origin', REPO_REVISION], REPO_DIR); + runGit(['checkout', '--force', 'FETCH_HEAD'], REPO_DIR); +} + +ensureRegistryCheckout(); + const dataDir = resolveDataDir(); let compose: [string, ...string[]]; diff --git a/workspaces/ai-integrations/scripts/undeploy-mcp-registry.ts b/workspaces/ai-integrations/scripts/undeploy-mcp-registry.ts index 59d17927094..208f96f0bcc 100755 --- a/workspaces/ai-integrations/scripts/undeploy-mcp-registry.ts +++ b/workspaces/ai-integrations/scripts/undeploy-mcp-registry.ts @@ -34,10 +34,17 @@ const SAFE_BIN_DIRS = ['/usr/bin', '/bin', '/usr/local/bin']; const CACHE_ROOT = join(homedir(), '.cache', 'rhdh-ai-integrations'); const DEFAULT_REPO_DIR = join(CACHE_ROOT, 'mcp-registry'); -const REPO_DIR = process.env.REPO_DIR?.trim() || DEFAULT_REPO_DIR; -const IMAGE = - process.env.MCP_REGISTRY_IMAGE?.trim() || - 'ghcr.io/modelcontextprotocol/registry:main'; +const REPO_DIR = process.env.MCP_REGISTRY_REPO_DIR?.trim() || DEFAULT_REPO_DIR; +const REPO_URL = + process.env.MCP_REGISTRY_REPO_URL?.trim() || + 'https://github.com/modelcontextprotocol/registry.git'; +const REPO_REVISION = + process.env.MCP_REGISTRY_REPO_REVISION?.trim() || 'v1.8.1'; +const IMAGE_NAME = + process.env.MCP_REGISTRY_IMAGE_NAME?.trim() || + 'ghcr.io/modelcontextprotocol/registry'; +const IMAGE_TAG = process.env.MCP_REGISTRY_IMAGE_TAG?.trim() || '1.8.1'; +const IMAGE = `${IMAGE_NAME}:${IMAGE_TAG}`; function findBinary(name: string): string | undefined { for (const dir of SAFE_BIN_DIRS) { @@ -83,27 +90,40 @@ function createPrivateTempDir(prefix: string): string { return mkdtempSync(join(CACHE_ROOT, prefix)); } -if (!existsSync(join(REPO_DIR, '.git'))) { - if (!existsSync(REPO_DIR)) { - const git = requireBinary('git'); - const result = spawnSync( - git, - [ - 'clone', - 'https://github.com/modelcontextprotocol/registry.git', - REPO_DIR, - ], - { stdio: 'inherit' }, - ); - if (result.status !== 0) { - process.exit(result.status ?? 1); - } - } else { +function runGit(args: string[], cwd?: string): void { + const git = requireBinary('git'); + const result = spawnSync(git, args, { + cwd, + stdio: 'inherit', + }); + if (result.status !== 0) { + process.exit(result.status ?? 1); + } +} + +/** Clone the registry checkout if missing (same URL/revision as deploy). */ +function ensureRegistryCheckout(): void { + if (existsSync(join(REPO_DIR, '.git'))) { + return; + } + if (existsSync(REPO_DIR)) { console.error(`error: ${REPO_DIR} exists but is not a git repository`); process.exit(1); } + console.log(`Cloning ${REPO_URL} (${REPO_REVISION}) into ${REPO_DIR}...`); + runGit([ + 'clone', + '--branch', + REPO_REVISION, + '--depth', + '1', + REPO_URL, + REPO_DIR, + ]); } +ensureRegistryCheckout(); + let compose: [string, ...string[]]; try { compose = resolveCompose(); From d2c834cd139d3681dd4ef1d3d6c7e570ad819fb2 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 18:22:31 -0400 Subject: [PATCH 44/63] chore(#4815): make MCP Registry readiness API version configurable Expose MCP_REGISTRY_API_VERSION for the readiness probe path, defaulting to v0.1. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../ai-integrations/docs/deploy-mcp-registry-locally.md | 1 + workspaces/ai-integrations/scripts/deploy-mcp-registry.ts | 6 +++++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md index a0ec0933fd3..a0cc5ddc359 100644 --- a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md +++ b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md @@ -52,6 +52,7 @@ Optional environment variables: | `MCP_REGISTRY_IMAGE_TAG` | `1.8.1` | Registry container image tag | | `MCP_REGISTRY_DATA_DIR` | _(checkout `./data`)_ | Host directory mounted at `/data` instead of the [default seed data](https://github.com/modelcontextprotocol/registry/blob/main/data/seed.json). When set, seeds from `data/seed.json` with validation off. | | `MCP_REGISTRY_URL` | `http://localhost:8080` | URL probed for readiness (and typically used as `catalog.providers.mcpRegistry.baseUrl`) | +| `MCP_REGISTRY_API_VERSION` | `v0.1` | Registry HTTP API version path segment used for the readiness probe | | `MCP_REGISTRY_READY_TIMEOUT_MS` | `300000` (5m) | How long `start-mcp-registry` waits for the API before failing | Example with custom seed content (directory must contain `seed.json`): diff --git a/workspaces/ai-integrations/scripts/deploy-mcp-registry.ts b/workspaces/ai-integrations/scripts/deploy-mcp-registry.ts index 501badf1987..16d43f9e6c7 100755 --- a/workspaces/ai-integrations/scripts/deploy-mcp-registry.ts +++ b/workspaces/ai-integrations/scripts/deploy-mcp-registry.ts @@ -61,6 +61,7 @@ const IMAGE = `${IMAGE_NAME}:${IMAGE_TAG}`; const DATA_DIR = process.env.MCP_REGISTRY_DATA_DIR?.trim(); const REGISTRY_URL = process.env.MCP_REGISTRY_URL?.trim() || 'http://localhost:8080'; +const API_VERSION = process.env.MCP_REGISTRY_API_VERSION?.trim() || 'v0.1'; const READY_TIMEOUT_MS = Number( process.env.MCP_REGISTRY_READY_TIMEOUT_MS?.trim() || 300_000, ); @@ -147,7 +148,10 @@ function createPrivateTempDir(prefix: string): string { function waitForRegistryReady(baseUrl: string, timeoutMs: number): void { const curl = requireBinary('curl'); const sleep = requireBinary('sleep'); - const probeUrl = `${baseUrl.replace(/\/$/, '')}/v0.1/servers?limit=1`; + const probeUrl = `${baseUrl.replace( + /\/$/, + '', + )}/${API_VERSION}/servers?limit=1`; const deadline = Date.now() + timeoutMs; console.log(`Waiting for MCP Registry at ${probeUrl}...`); while (Date.now() < deadline) { From 2b95d573d126f45a3cbd12b0092c9ca02aac99cc Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 18:26:45 -0400 Subject: [PATCH 45/63] chore(#4815): rename local MCP Registry scripts for clarity Rename deploy/undeploy scripts and yarn targets to include "local" so their purpose is clearer. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../docs/deploy-mcp-registry-locally.md | 12 ++++++------ workspaces/ai-integrations/package.json | 4 ++-- .../package.json | 4 ++-- ...-mcp-registry.ts => deploy-local-mcp-registry.ts} | 0 ...cp-registry.ts => undeploy-local-mcp-registry.ts} | 2 +- 5 files changed, 11 insertions(+), 11 deletions(-) rename workspaces/ai-integrations/scripts/{deploy-mcp-registry.ts => deploy-local-mcp-registry.ts} (100%) rename workspaces/ai-integrations/scripts/{undeploy-mcp-registry.ts => undeploy-local-mcp-registry.ts} (98%) diff --git a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md index a0cc5ddc359..1e87c9f1002 100644 --- a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md +++ b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md @@ -19,13 +19,13 @@ From the `ai-integrations` workspace root, or from `plugins/catalog-backend-module-mcp-registry-provider`: ```bash -yarn start-mcp-registry +yarn start-local-mcp-registry ``` You can also run the script directly from the workspace root: ```bash -node scripts/deploy-mcp-registry.ts +node scripts/deploy-local-mcp-registry.ts ``` This clones the registry into `~/.cache/rhdh-ai-integrations/mcp-registry` @@ -53,12 +53,12 @@ Optional environment variables: | `MCP_REGISTRY_DATA_DIR` | _(checkout `./data`)_ | Host directory mounted at `/data` instead of the [default seed data](https://github.com/modelcontextprotocol/registry/blob/main/data/seed.json). When set, seeds from `data/seed.json` with validation off. | | `MCP_REGISTRY_URL` | `http://localhost:8080` | URL probed for readiness (and typically used as `catalog.providers.mcpRegistry.baseUrl`) | | `MCP_REGISTRY_API_VERSION` | `v0.1` | Registry HTTP API version path segment used for the readiness probe | -| `MCP_REGISTRY_READY_TIMEOUT_MS` | `300000` (5m) | How long `start-mcp-registry` waits for the API before failing | +| `MCP_REGISTRY_READY_TIMEOUT_MS` | `300000` (5m) | How long `start-local-mcp-registry` waits for the API before failing | Example with custom seed content (directory must contain `seed.json`): ```bash -MCP_REGISTRY_DATA_DIR=./examples/mcp-registry/seed-data yarn start-mcp-registry +MCP_REGISTRY_DATA_DIR=./examples/mcp-registry/seed-data yarn start-local-mcp-registry ``` View logs (example with Podman): @@ -95,13 +95,13 @@ From the workspace root or `plugins/catalog-backend-module-mcp-registry-provider`: ```bash -yarn stop-mcp-registry +yarn stop-local-mcp-registry ``` Or from the workspace root: ```bash -node scripts/undeploy-mcp-registry.ts +node scripts/undeploy-local-mcp-registry.ts ``` This runs `compose down` for the same stack. The diff --git a/workspaces/ai-integrations/package.json b/workspaces/ai-integrations/package.json index 5c52f54a39f..da5a842c3cf 100644 --- a/workspaces/ai-integrations/package.json +++ b/workspaces/ai-integrations/package.json @@ -10,8 +10,8 @@ "dev:debug": "yarn workspaces foreach -A --include backend --include app --parallel -v -i run start --inspect", "start": "yarn workspace app start", "start-backend": "yarn workspace backend start", - "start-mcp-registry": "node scripts/deploy-mcp-registry.ts", - "stop-mcp-registry": "node scripts/undeploy-mcp-registry.ts", + "start-local-mcp-registry": "node scripts/deploy-local-mcp-registry.ts", + "stop-local-mcp-registry": "node scripts/undeploy-local-mcp-registry.ts", "build:backend": "yarn workspace backend build", "tsc": "tsc", "tsc:full": "tsc --skipLibCheck true --incremental false", diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json index 02a903fe590..5a88aedfbdb 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/package.json @@ -22,8 +22,8 @@ }, "scripts": { "start": "backstage-cli package start", - "start-mcp-registry": "node ../../scripts/deploy-mcp-registry.ts", - "stop-mcp-registry": "node ../../scripts/undeploy-mcp-registry.ts", + "start-local-mcp-registry": "node ../../scripts/deploy-local-mcp-registry.ts", + "stop-local-mcp-registry": "node ../../scripts/undeploy-local-mcp-registry.ts", "build": "backstage-cli package build", "lint": "backstage-cli package lint", "lint:check": "backstage-cli package lint", diff --git a/workspaces/ai-integrations/scripts/deploy-mcp-registry.ts b/workspaces/ai-integrations/scripts/deploy-local-mcp-registry.ts similarity index 100% rename from workspaces/ai-integrations/scripts/deploy-mcp-registry.ts rename to workspaces/ai-integrations/scripts/deploy-local-mcp-registry.ts diff --git a/workspaces/ai-integrations/scripts/undeploy-mcp-registry.ts b/workspaces/ai-integrations/scripts/undeploy-local-mcp-registry.ts similarity index 98% rename from workspaces/ai-integrations/scripts/undeploy-mcp-registry.ts rename to workspaces/ai-integrations/scripts/undeploy-local-mcp-registry.ts index 208f96f0bcc..4e819cf9011 100755 --- a/workspaces/ai-integrations/scripts/undeploy-mcp-registry.ts +++ b/workspaces/ai-integrations/scripts/undeploy-local-mcp-registry.ts @@ -15,7 +15,7 @@ * limitations under the License. */ -/** Stop the local MCP Registry started by deploy-mcp-registry.ts. */ +/** Stop the local MCP Registry started by deploy-local-mcp-registry.ts. */ const { spawnSync } = require('node:child_process'); const { From 8cdd302c28cf5632366553fed97d5b243843b05c Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 18:35:44 -0400 Subject: [PATCH 46/63] chore(#4815): add example mcp-server API entities from registry seed Provide catalog YAML mirroring provider output for the sample seed data and wire it as a local file location for review. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- workspaces/ai-integrations/app-config.yaml | 10 + .../examples/api-mcp-servers.yaml | 198 ++++++++++++++++++ .../README.md | 4 + 3 files changed, 212 insertions(+) create mode 100644 workspaces/ai-integrations/examples/api-mcp-servers.yaml diff --git a/workspaces/ai-integrations/app-config.yaml b/workspaces/ai-integrations/app-config.yaml index 674e6764267..874321996cb 100644 --- a/workspaces/ai-integrations/app-config.yaml +++ b/workspaces/ai-integrations/app-config.yaml @@ -123,6 +123,16 @@ catalog: target: ../../examples/ai-model-server-api.yaml rules: - allow: [AiModelServerAPI] + # Example MCP server API entities as produced by + # catalog-backend-module-mcp-registry-provider from the sample registry seed + # at examples/mcp-registry/seed-data/seed.json. + # + # Comment out this location when deploying the MCP Registry provider against + # the same seed (e.g. `MCP_REGISTRY_DATA_DIR=./examples/mcp-registry/seed-data yarn start-local-mcp-registry`). + - type: file + target: ../../examples/api-mcp-servers.yaml + rules: + - allow: [API] providers: modelCatalog: # The field underneath should list the connector plugin ID that the entity provider accesses through diff --git a/workspaces/ai-integrations/examples/api-mcp-servers.yaml b/workspaces/ai-integrations/examples/api-mcp-servers.yaml new file mode 100644 index 00000000000..46422398c0a --- /dev/null +++ b/workspaces/ai-integrations/examples/api-mcp-servers.yaml @@ -0,0 +1,198 @@ +--- +# Example mcp-server API entities as produced by +# catalog-backend-module-mcp-registry-provider from the sample registry seed +# at examples/mcp-registry/seed-data/seed.json. +# +# Defaults match app-config.yaml (defaultOwner: default-owner, +# baseUrl: http://localhost:8080/, default baseName prefix mcp.registry). +# Provider location and sync-status annotations are included. +# +# Wired as a file location in app-config.yaml for local catalog review. +# If the MCP Registry provider is also syncing the same seed, you may see +# duplicate entities — disable one source when comparing. +# Requires @backstage/plugin-catalog-backend-module-ai-model for mcp-server +# API validation (Backstage 1.51+). + +apiVersion: backstage.io/v1alpha1 +kind: API +metadata: + name: mcp.registry__io.example.labs-atlas-search__2.1.0-a5f4e0d4 + description: MCP server that wraps the Atlas Search HTTP API for document discovery + tags: + - mcp + - ai + annotations: + backstage.io/managed-by-location: url:http://localhost:8080 + backstage.io/managed-by-origin-location: url:http://localhost:8080 + backstage.io/source-location: url:https://github.com/example-labs/mcp-servers + modelcontextprotocol.io/name: io.example.labs/atlas-search + modelcontextprotocol.io/packages.0.environmentvariables.0.description: Atlas Search API key + modelcontextprotocol.io/packages.0.environmentvariables.0.isrequired: 'true' + modelcontextprotocol.io/packages.0.environmentvariables.0.issecret: 'true' + modelcontextprotocol.io/packages.0.environmentvariables.0.name: ATLAS_SEARCH_API_KEY + modelcontextprotocol.io/packages.0.identifier: '@example-labs/mcp-server-atlas-search' + modelcontextprotocol.io/packages.0.registrybaseurl: https://registry.npmjs.org + modelcontextprotocol.io/packages.0.registrytype: npm + modelcontextprotocol.io/packages.0.transport.type: stdio + modelcontextprotocol.io/packages.0.version: 2.1.0 + modelcontextprotocol.io/repository.source: github + modelcontextprotocol.io/repository.url: https://github.com/example-labs/mcp-servers + modelcontextprotocol.io/version: 2.1.0 + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provi-14e325eb: 1.2.0 + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provi-8a6b76b4: 2025-03-14T09:15:00Z + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provided.tool: npm-publisher + redhat.com/rhdh-mcp-registry-sync-status: ok + title: Atlas Search + links: + - url: https://labs.example.io/mcp/atlas-search + title: Website + - url: https://github.com/example-labs/mcp-servers + title: Source Code +spec: + type: mcp-server + lifecycle: production + owner: default-owner + remotes: + - type: undefined + url: http://localhost:8080/ +--- +apiVersion: backstage.io/v1alpha1 +kind: API +metadata: + name: mcp.registry__io.github.example-labs-workspace-fs__1.4-db809cb2 + description: MCP server for sandboxed workspace filesystem read and write operations. + tags: + - mcp + - ai + annotations: + backstage.io/managed-by-location: url:http://localhost:8080 + backstage.io/managed-by-origin-location: url:http://localhost:8080 + backstage.io/source-location: url:https://github.com/example-labs/mcp-servers + modelcontextprotocol.io/name: io.github.example-labs/workspace-fs + modelcontextprotocol.io/packages.0.environmentvariables.0.default: warn + modelcontextprotocol.io/packages.0.environmentvariables.0.description: Logging level (debug, info, warn, error) + modelcontextprotocol.io/packages.0.environmentvariables.0.name: LOG_LEVEL + modelcontextprotocol.io/packages.0.identifier: '@example-labs/mcp-server-workspace-fs' + modelcontextprotocol.io/packages.0.packagearguments.0.default: /home/developer/workspace + modelcontextprotocol.io/packages.0.packagearguments.0.description: Workspace root directory to expose + modelcontextprotocol.io/packages.0.packagearguments.0.isrepeated: 'true' + modelcontextprotocol.io/packages.0.packagearguments.0.isrequired: 'true' + modelcontextprotocol.io/packages.0.packagearguments.0.type: positional + modelcontextprotocol.io/packages.0.packagearguments.0.valuehint: workspace_root + modelcontextprotocol.io/packages.0.registrybaseurl: https://registry.npmjs.org + modelcontextprotocol.io/packages.0.registrytype: npm + modelcontextprotocol.io/packages.0.transport.type: stdio + modelcontextprotocol.io/packages.0.version: 1.4.1 + modelcontextprotocol.io/packages.1.environmentvariables.0.default: warn + modelcontextprotocol.io/packages.1.environmentvariables.0.description: Logging level (debug, info, warn, error) + modelcontextprotocol.io/packages.1.environmentvariables.0.name: LOG_LEVEL + modelcontextprotocol.io/packages.1.identifier: ghcr.io/example-labs/workspace-fs:1.4.1 + modelcontextprotocol.io/packages.1.packagearguments.0.type: positional + modelcontextprotocol.io/packages.1.packagearguments.0.value: /workspace + modelcontextprotocol.io/packages.1.packagearguments.0.valuehint: workspace_root + modelcontextprotocol.io/packages.1.registrytype: oci + modelcontextprotocol.io/packages.1.runtimearguments.0.description: Bind-mount a host path into the container + modelcontextprotocol.io/packages.1.runtimearguments.0.isrepeated: 'true' + modelcontextprotocol.io/packages.1.runtimearguments.0.isrequired: 'true' + modelcontextprotocol.io/packages.1.runtimearguments.0.name: --mount + modelcontextprotocol.io/packages.1.runtimearguments.0.type: named + modelcontextprotocol.io/packages.1.runtimearguments.0.value: type=bind,src={source_path},dst={target_path} + modelcontextprotocol.io/packages.1.runtimearguments.0.variables.source_path.description: Host path to mount + modelcontextprotocol.io/packages.1.runtimearguments.0.variables.source_path.format: filepath + modelcontextprotocol.io/packages.1.runtimearguments.0.variables.source_path.isrequired: 'true' + modelcontextprotocol.io/packages.1.runtimearguments.0.variables.target_path.default: /workspace + modelcontextprotocol.io/packages.1.runtimearguments.0.variables.target_path.description: Mount point inside the container under `/workspace`. + modelcontextprotocol.io/packages.1.runtimearguments.0.variables.target_path.isrequired: 'true' + modelcontextprotocol.io/packages.1.transport.type: stdio + modelcontextprotocol.io/repository.id: c1a2b3d4-e5f6-7890-abcd-ef1234567890 + modelcontextprotocol.io/repository.source: github + modelcontextprotocol.io/repository.url: https://github.com/example-labs/mcp-servers + modelcontextprotocol.io/version: 1.4.1 + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provi-14e325eb: 4.0.0 + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provi-8a6b76b4: 2025-06-02T18:40:00Z + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provi-b0084041: 9f8e7d6c5b4a3210 + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provi-eca6f5da: workspace-fs-build-2048 + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provi-faf17705: staging + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provided.tool: ci-publisher + redhat.com/rhdh-mcp-registry-sync-status: ok + title: Workspace Filesystem + links: + - url: https://github.com/example-labs/mcp-servers + title: Source Code +spec: + type: mcp-server + lifecycle: production + owner: default-owner + remotes: + - type: undefined + url: http://localhost:8080/ +--- +apiVersion: backstage.io/v1alpha1 +kind: API +metadata: + name: mcp.registry__io.github.example-labs-dice-weather-mcpx-cf045e19 + description: NuGet MCP server that returns random dice rolls and sample weather snippets + tags: + - mcp + - ai + annotations: + backstage.io/managed-by-location: url:http://localhost:8080 + backstage.io/managed-by-origin-location: url:http://localhost:8080 + modelcontextprotocol.io/name: io.github.example-labs/dice-weather-mcp + modelcontextprotocol.io/packages.0.identifier: ExampleLabs.DiceWeatherMcp + modelcontextprotocol.io/packages.0.packagearguments.0.type: positional + modelcontextprotocol.io/packages.0.packagearguments.0.value: mcp + modelcontextprotocol.io/packages.0.packagearguments.1.type: positional + modelcontextprotocol.io/packages.0.packagearguments.1.value: start + modelcontextprotocol.io/packages.0.registrybaseurl: https://api.nuget.org/v3/index.json + modelcontextprotocol.io/packages.0.registrytype: nuget + modelcontextprotocol.io/packages.0.transport.type: stdio + modelcontextprotocol.io/packages.0.version: 1.2.0-preview.3 + modelcontextprotocol.io/version: 1.2.0-preview.3 + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provi-14e325eb: 2.3.1 + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provi-8a6b76b4: 2025-01-22T11:05:00Z + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provi-eca6f5da: nuget-dice-weather-101 + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provided.tool: nuget-publisher + redhat.com/rhdh-mcp-registry-sync-status: ok +spec: + type: mcp-server + lifecycle: production + owner: default-owner + remotes: + - type: undefined + url: http://localhost:8080/ +--- +apiVersion: backstage.io/v1alpha1 +kind: API +metadata: + name: mcp.registry__io.example.cloud-remote-workspace__3.1.0-4744af90 + description: Hosted MCP workspace filesystem endpoint for shared team sandboxes + tags: + - mcp + - ai + annotations: + backstage.io/managed-by-location: url:http://localhost:8080 + backstage.io/managed-by-origin-location: url:http://localhost:8080 + backstage.io/source-location: url:https://github.com/example-cloud/remote-workspace-mcp + modelcontextprotocol.io/name: io.example.cloud/remote-workspace + modelcontextprotocol.io/repository.id: a0b1c2d3-e4f5-6789-abcd-ef0123456789 + modelcontextprotocol.io/repository.source: github + modelcontextprotocol.io/repository.url: https://github.com/example-cloud/remote-workspace-mcp + modelcontextprotocol.io/version: 3.1.0 + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provi-14e325eb: 3.0.2 + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provi-8a6b76b4: 2025-08-19T13:10:00Z + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provi-983f3ff8: eu-central-1 + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provi-b0084041: c3b2a19087fe + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provi-bf4b1861: remote-workspace-deploy-812 + modelcontextprotocol.io/xmeta.io.modelcontextprotocol.registry-publisher-provided.tool: cloud-deployer + redhat.com/rhdh-mcp-registry-sync-status: ok + links: + - url: https://github.com/example-cloud/remote-workspace-mcp + title: Source Code +spec: + type: mcp-server + lifecycle: production + owner: default-owner + remotes: + - type: streamable-http + url: https://mcp.example.cloud/v1/workspace/http diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index c19518f134b..7b3a3c5913e 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -106,6 +106,10 @@ Each entity carries: - `modelcontextprotocol.io/name`: the server's canonical name - `modelcontextprotocol.io/version`: the server's version +Example catalog entities matching this shape (generated from +[`examples/mcp-registry/seed-data/seed.json`](../../examples/mcp-registry/seed-data/seed.json)) +are in [`examples/api-mcp-servers.yaml`](../../examples/api-mcp-servers.yaml). + ## Non-Remote MCP Servers MCP servers without a remote deployment (package(s) only or [custom installation](https://github.com/modelcontextprotocol/registry/blob/main/docs/reference/server-json/generic-server-json.md#server-with-custom-installation-path)) can be queried via: `GET /api/catalog/entities?filter=kind=API,spec.type=mcp-server,spec.remotes.type=undefined` From 43219205cc3159e2ab2cb4a676567b0707870e6d Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 19:03:40 -0400 Subject: [PATCH 47/63] chore(#4815): nest MCP Registry provider config under reserved instance id Move provider options to catalog.providers.mcpRegistry.mcpRegistry so the top-level key is a map of instances, ignoring extra ids with a warning. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- workspaces/ai-integrations/app-config.yaml | 54 ++-- .../docs/deploy-mcp-registry-locally.md | 17 +- .../README.md | 57 ++-- .../config.d.ts | 60 +++-- .../catalog.processing.integration.test.ts | 14 +- .../src/config.test.ts | 246 +++++++++--------- .../src/config.ts | 109 ++++++-- .../src/module.ts | 6 +- 8 files changed, 329 insertions(+), 234 deletions(-) diff --git a/workspaces/ai-integrations/app-config.yaml b/workspaces/ai-integrations/app-config.yaml index 874321996cb..412412301cc 100644 --- a/workspaces/ai-integrations/app-config.yaml +++ b/workspaces/ai-integrations/app-config.yaml @@ -150,32 +150,34 @@ catalog: default-owner: '${OWNER:-default-owner}' default-lifecycle: '${LIFECYCLE:-production}' mcpRegistry: - # Required: base URL of the MCP Registry - baseUrl: '${MCP_REGISTRY_URL:-http://localhost:8080/}' - # Optional: base name (default: mcp.registry) - # baseName: mcp.registry - # Optional: API version (default: v1) - apiVersion: v0.1 - # Optional: default entity owner (default: unknown) - defaultOwner: '${OWNER:-default-owner}' - # Optional: max pages fetched per sync (default: 10) - # pageLimit: 10 - # Optional: registry page size sent as ?limit= (omitted by default) - # pageSize: 50 - # Optional: max entries per complete traversal; soft-stops with end cursor (default: 5000) - # maxEntries: 5000 - # Optional: ingest only servers with at least one native remote (default: false) - # remotesOnly: false - # Optional: allowlist of permitted hostnames for defense-in-depth SSRF protection. - # hostAllowList: - # - registry.modelcontextprotocol.io - # - staging.registry.modelcontextprotocol.io - # Optional: sync schedule (defaults shown below) - # schedule: - # frequency: { minutes: 30 } - # timeout: { minutes: 3 } - # # Optional: defer the first sync - # # initialDelay: { seconds: 15 } + # Reserved instance id — only this key is supported today. + mcpRegistry: + # Required: base URL of the MCP Registry + baseUrl: '${MCP_REGISTRY_URL:-http://localhost:8080/}' + # Optional: base name (default: mcp.registry) + # baseName: mcp.registry + # Optional: API version (default: v1) + apiVersion: v0.1 + # Optional: default entity owner (default: unknown) + defaultOwner: '${OWNER:-default-owner}' + # Optional: max pages fetched per sync (default: 10) + # pageLimit: 10 + # Optional: registry page size sent as ?limit= (omitted by default) + # pageSize: 50 + # Optional: max entries per complete traversal; soft-stops with end cursor (default: 5000) + # maxEntries: 5000 + # Optional: ingest only servers with at least one native remote (default: false) + # remotesOnly: false + # Optional: allowlist of permitted hostnames for defense-in-depth SSRF protection. + # hostAllowList: + # - registry.modelcontextprotocol.io + # - staging.registry.modelcontextprotocol.io + # Optional: sync schedule (defaults shown below) + # schedule: + # frequency: { minutes: 30 } + # timeout: { minutes: 3 } + # # Optional: defer the first sync + # # initialDelay: { seconds: 15 } # Uncomment to use kubernetesPluginRef — the Backstage Kubernetes plugin # does NOT need to be installed, only its config section is needed. #kubernetes: diff --git a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md index 1e87c9f1002..67e36a599ee 100644 --- a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md +++ b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md @@ -51,7 +51,7 @@ Optional environment variables: | `MCP_REGISTRY_IMAGE_NAME` | `ghcr.io/modelcontextprotocol/registry` | Registry container image name (without tag) | | `MCP_REGISTRY_IMAGE_TAG` | `1.8.1` | Registry container image tag | | `MCP_REGISTRY_DATA_DIR` | _(checkout `./data`)_ | Host directory mounted at `/data` instead of the [default seed data](https://github.com/modelcontextprotocol/registry/blob/main/data/seed.json). When set, seeds from `data/seed.json` with validation off. | -| `MCP_REGISTRY_URL` | `http://localhost:8080` | URL probed for readiness (and typically used as `catalog.providers.mcpRegistry.baseUrl`) | +| `MCP_REGISTRY_URL` | `http://localhost:8080` | URL probed for readiness (and typically used as `catalog.providers.mcpRegistry.mcpRegistry.baseUrl`) | | `MCP_REGISTRY_API_VERSION` | `v0.1` | Registry HTTP API version path segment used for the readiness probe | | `MCP_REGISTRY_READY_TIMEOUT_MS` | `300000` (5m) | How long `start-local-mcp-registry` waits for the API before failing | @@ -69,18 +69,19 @@ podman compose -f ~/.cache/rhdh-ai-integrations/mcp-registry/docker-compose.yml ## Point the provider at localhost -Configure `catalog.providers.mcpRegistry` to use the local registry. The default -local API version is `v0.1`: +Configure `catalog.providers.mcpRegistry.mcpRegistry` to use the local +registry. The default local API version is `v0.1`: ```yaml catalog: providers: mcpRegistry: - baseUrl: http://localhost:8080 - apiVersion: v0.1 - # Optional when restricting outbound hosts: - # hostAllowList: - # - localhost + mcpRegistry: + baseUrl: http://localhost:8080 + apiVersion: v0.1 + # Optional when restricting outbound hosts: + # hostAllowList: + # - localhost ``` Then start the workspace as usual (`yarn dev` from `workspaces/ai-integrations`). diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index 7b3a3c5913e..e8ae3a83317 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -32,30 +32,32 @@ Configure the provider in your `app-config.yaml`: catalog: providers: mcpRegistry: - baseUrl: https://registry.example.com - # Optional: override the mapping identity prefix (default: mcp.registry) - # baseName: com.example.registry - # Optional: registry API version slug (default: v1) - # apiVersion: v1 - # Optional: default entity owner (default: unknown) - # defaultOwner: group:default/mcp-admins - # Optional: max pages fetched per sync (default: 10) - # pageLimit: 10 - # Optional: registry page size sent as ?limit= (omitted by default) - # pageSize: 50 - # Optional: max entries per complete traversal; soft-stops with end cursor (default: 5000) - # maxEntries: 5000 - # Optional: ingest only servers with at least one native remote (default: false) - # remotesOnly: false - # Optional: restrict outbound requests to specific hostnames (defense-in-depth) - # hostAllowList: - # - registry.example.com - # Optional: sync schedule (defaults shown below) - # schedule: - # frequency: { minutes: 30 } - # timeout: { minutes: 3 } - # # Optional: defer the first sync - # # initialDelay: { seconds: 15 } + # Reserved instance id — only this key is supported today. + mcpRegistry: + baseUrl: https://registry.example.com + # Optional: override the mapping identity prefix (default: mcp.registry) + # baseName: com.example.registry + # Optional: registry API version slug (default: v1) + # apiVersion: v1 + # Optional: default entity owner (default: unknown) + # defaultOwner: group:default/mcp-admins + # Optional: max pages fetched per sync (default: 10) + # pageLimit: 10 + # Optional: registry page size sent as ?limit= (omitted by default) + # pageSize: 50 + # Optional: max entries per complete traversal; soft-stops with end cursor (default: 5000) + # maxEntries: 5000 + # Optional: ingest only servers with at least one native remote (default: false) + # remotesOnly: false + # Optional: restrict outbound requests to specific hostnames (defense-in-depth) + # hostAllowList: + # - registry.example.com + # Optional: sync schedule (defaults shown below) + # schedule: + # frequency: { minutes: 30 } + # timeout: { minutes: 3 } + # # Optional: defer the first sync + # # initialDelay: { seconds: 15 } ``` ### Configuration options @@ -75,7 +77,12 @@ catalog: ### Multiple registries -Multiple registries are **not supported** in this implementation. Configuring a keyed map of instances (e.g., `mcpRegistry.internal` and `mcpRegistry.public`) will fail at startup with an actionable error. Use `baseName` to override the identity prefix if needed for future multi-registry support. +`catalog.providers.mcpRegistry` is a map of instance ids so additional +registries can be added later. This implementation only reads the reserved +`mcpRegistry` instance (`catalog.providers.mcpRegistry.mcpRegistry`). Other +instance ids are ignored and a warning is logged that multiple MCP Registry +providers are not supported yet. Use `baseName` on the reserved instance to +override the identity prefix if needed for future multi-registry support. ## Behavior diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts index e348cf0d156..99495e6eda6 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts @@ -15,35 +15,45 @@ */ import { SchedulerServiceTaskScheduleDefinitionConfig } from '@backstage/backend-plugin-api'; +/** Per-registry instance options under `catalog.providers.mcpRegistry.`. */ +interface McpRegistryInstanceConfig { + /** @visibility backend */ + baseUrl: string; + /** @visibility backend */ + baseName?: string; + /** @visibility backend */ + apiVersion?: string; + /** @visibility backend */ + defaultOwner?: string; + /** @visibility backend */ + pageLimit?: number; + /** @visibility backend */ + pageSize?: number; + /** @visibility backend */ + maxEntries?: number; + /** + * When true, only ingest servers that declare at least one native + * remote. Package-only / placeholder-remote servers are skipped. + * + * @visibility backend + */ + remotesOnly?: boolean; + /** @visibility backend */ + hostAllowList?: string[]; + /** @visibility backend */ + schedule?: SchedulerServiceTaskScheduleDefinitionConfig; +} + export interface Config { catalog?: { providers?: { + /** + * Map of MCP Registry provider instances. This implementation only + * reads the reserved `mcpRegistry` instance id; additional keys are + * ignored with a warning until multi-registry support lands. + */ mcpRegistry?: { - /** @visibility backend */ - baseUrl: string; - /** @visibility backend */ - baseName?: string; - /** @visibility backend */ - apiVersion?: string; - /** @visibility backend */ - defaultOwner?: string; - /** @visibility backend */ - pageLimit?: number; - /** @visibility backend */ - pageSize?: number; - /** @visibility backend */ - maxEntries?: number; - /** - * When true, only ingest servers that declare at least one native - * remote. Package-only / placeholder-remote servers are skipped. - * - * @visibility backend - */ - remotesOnly?: boolean; - /** @visibility backend */ - hostAllowList?: string[]; - /** @visibility backend */ - schedule?: SchedulerServiceTaskScheduleDefinitionConfig; + mcpRegistry?: McpRegistryInstanceConfig; }; }; }; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/catalog.processing.integration.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/catalog.processing.integration.test.ts index 1f9a512e7b0..b83e7123f05 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/catalog.processing.integration.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/catalog.processing.integration.test.ts @@ -99,12 +99,14 @@ describe('mcp-server catalog processing', () => { rules: [{ allow: ['API'] }], providers: { mcpRegistry: { - baseUrl: 'https://registry.example.com', - apiVersion: 'v0.1', - defaultOwner: 'user:default/guest', - schedule: { - frequency: { seconds: 1 }, - timeout: { seconds: 10 }, + mcpRegistry: { + baseUrl: 'https://registry.example.com', + apiVersion: 'v0.1', + defaultOwner: 'user:default/guest', + schedule: { + frequency: { seconds: 1 }, + timeout: { seconds: 10 }, + }, }, }, }, diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts index f5d31490861..328c5c1130e 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts @@ -17,6 +17,7 @@ import { ConfigReader } from '@backstage/config'; import { assertSingleRegistryConfig, + MCP_REGISTRY_INSTANCE_ID, readMaxEntries, readMcpRegistryProviderConfig, readHostAllowList, @@ -25,11 +26,25 @@ import { readProviderSchedule, readRemotesOnly, readRequiredHttpBaseUrl, + readReservedRegistryInstanceConfig, resolveMcpRegistryProviderConfig, safeGetOptionalString, validateHostAllowList, } from './config'; +/** Nest instance options under the reserved `mcpRegistry` map key. */ +function providersConfig(instance: Record) { + return { + catalog: { + providers: { + mcpRegistry: { + [MCP_REGISTRY_INSTANCE_ID]: instance, + }, + }, + }, + }; +} + describe('readMcpRegistryProviderConfig', () => { it('returns undefined when catalog.providers is absent', () => { const config = new ConfigReader({}); @@ -44,15 +59,11 @@ describe('readMcpRegistryProviderConfig', () => { }); it('reads a single object with baseUrl and applies defaults', () => { - const config = new ConfigReader({ - catalog: { - providers: { - mcpRegistry: { - baseUrl: 'https://registry.example.com', - }, - }, - }, - }); + const config = new ConfigReader( + providersConfig({ + baseUrl: 'https://registry.example.com', + }), + ); const result = readMcpRegistryProviderConfig(config); expect(result).toBeDefined(); @@ -71,91 +82,88 @@ describe('readMcpRegistryProviderConfig', () => { }); it('reads optional baseName', () => { - const config = new ConfigReader({ - catalog: { - providers: { - mcpRegistry: { - baseUrl: 'https://registry.example.com', - baseName: 'com.example.registry', - }, - }, - }, - }); + const config = new ConfigReader( + providersConfig({ + baseUrl: 'https://registry.example.com', + baseName: 'com.example.registry', + }), + ); const result = readMcpRegistryProviderConfig(config); expect(result!.baseName).toBe('com.example.registry'); }); it('reads explicit pageLimit override', () => { - const config = new ConfigReader({ - catalog: { - providers: { - mcpRegistry: { - baseUrl: 'https://registry.example.com', - pageLimit: 3, - }, - }, - }, - }); + const config = new ConfigReader( + providersConfig({ + baseUrl: 'https://registry.example.com', + pageLimit: 3, + }), + ); const result = readMcpRegistryProviderConfig(config); expect(result!.pageLimit).toBe(3); }); it('reads explicit pageSize', () => { - const config = new ConfigReader({ - catalog: { - providers: { - mcpRegistry: { - baseUrl: 'https://registry.example.com', - pageSize: 50, - }, - }, - }, - }); + const config = new ConfigReader( + providersConfig({ + baseUrl: 'https://registry.example.com', + pageSize: 50, + }), + ); const result = readMcpRegistryProviderConfig(config); expect(result!.pageSize).toBe(50); }); it('reads omitted pageSize as undefined', () => { - const config = new ConfigReader({ - catalog: { - providers: { - mcpRegistry: { - baseUrl: 'https://registry.example.com', - }, - }, - }, - }); + const config = new ConfigReader( + providersConfig({ + baseUrl: 'https://registry.example.com', + }), + ); const result = readMcpRegistryProviderConfig(config); expect(result!.pageSize).toBeUndefined(); }); it('throws when baseUrl is missing', () => { + const config = new ConfigReader( + providersConfig({ + apiVersion: 'v0', + }), + ); + + expect(() => readMcpRegistryProviderConfig(config)).toThrow( + /missing required "baseUrl"/, + ); + }); + + it('throws when a legacy flat catalog.providers.mcpRegistry object is used', () => { const config = new ConfigReader({ catalog: { providers: { mcpRegistry: { - apiVersion: 'v0', + baseUrl: 'https://registry.example.com', }, }, }, }); expect(() => readMcpRegistryProviderConfig(config)).toThrow( - /missing required "baseUrl"/, + /must be nested under the reserved instance key/, ); }); - it('throws when config is a keyed map of instances', () => { + it('ignores additional instance ids and warns that multi-registry is unsupported', () => { + const warnings: string[] = []; const config = new ConfigReader({ catalog: { providers: { mcpRegistry: { - internal: { - baseUrl: 'https://internal-registry.example.com', + mcpRegistry: { + baseUrl: 'https://registry.example.com', }, public: { baseUrl: 'https://public-registry.example.com', @@ -165,26 +173,34 @@ describe('readMcpRegistryProviderConfig', () => { }, }); - expect(() => readMcpRegistryProviderConfig(config)).toThrow( - /found keyed instance/, + const result = readMcpRegistryProviderConfig(config, message => + warnings.push(message), ); + expect(result!.baseUrl).toBe('https://registry.example.com'); + expect(warnings).toHaveLength(1); + expect(warnings[0]).toMatch(/additional instance id\(s\)/); + expect(warnings[0]).toMatch(/not supported yet/); + expect(warnings[0]).toMatch(/public/); }); - it('reads a custom schedule', () => { + it('returns undefined when the providers map is empty', () => { const config = new ConfigReader({ - catalog: { - providers: { - mcpRegistry: { - baseUrl: 'https://registry.example.com', - schedule: { - frequency: { minutes: 15 }, - timeout: { minutes: 5 }, - initialDelay: { seconds: 30 }, - }, - }, - }, - }, + catalog: { providers: { mcpRegistry: {} } }, }); + expect(readMcpRegistryProviderConfig(config)).toBeUndefined(); + }); + + it('reads a custom schedule', () => { + const config = new ConfigReader( + providersConfig({ + baseUrl: 'https://registry.example.com', + schedule: { + frequency: { minutes: 15 }, + timeout: { minutes: 5 }, + initialDelay: { seconds: 30 }, + }, + }), + ); const result = readMcpRegistryProviderConfig(config); expect(result!.schedule).toEqual({ @@ -195,79 +211,59 @@ describe('readMcpRegistryProviderConfig', () => { }); it('reads defaultOwner', () => { - const config = new ConfigReader({ - catalog: { - providers: { - mcpRegistry: { - baseUrl: 'https://registry.example.com', - defaultOwner: 'group:default/mcp-admins', - }, - }, - }, - }); + const config = new ConfigReader( + providersConfig({ + baseUrl: 'https://registry.example.com', + defaultOwner: 'group:default/mcp-admins', + }), + ); const result = readMcpRegistryProviderConfig(config); expect(result!.defaultOwner).toBe('group:default/mcp-admins'); }); it('reads apiVersion override', () => { - const config = new ConfigReader({ - catalog: { - providers: { - mcpRegistry: { - baseUrl: 'https://registry.example.com', - apiVersion: 'v0', - }, - }, - }, - }); + const config = new ConfigReader( + providersConfig({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v0', + }), + ); const result = readMcpRegistryProviderConfig(config); expect(result!.apiVersion).toBe('v0'); }); it('reads hostAllowList when provided', () => { - const config = new ConfigReader({ - catalog: { - providers: { - mcpRegistry: { - baseUrl: 'https://registry.example.com', - hostAllowList: ['registry.example.com'], - }, - }, - }, - }); + const config = new ConfigReader( + providersConfig({ + baseUrl: 'https://registry.example.com', + hostAllowList: ['registry.example.com'], + }), + ); const result = readMcpRegistryProviderConfig(config); expect(result!.hostAllowList).toEqual(['registry.example.com']); }); it('returns undefined hostAllowList when omitted', () => { - const config = new ConfigReader({ - catalog: { - providers: { - mcpRegistry: { - baseUrl: 'https://registry.example.com', - }, - }, - }, - }); + const config = new ConfigReader( + providersConfig({ + baseUrl: 'https://registry.example.com', + }), + ); const result = readMcpRegistryProviderConfig(config); expect(result!.hostAllowList).toBeUndefined(); }); it('throws when baseUrl hostname is not in hostAllowList', () => { - const config = new ConfigReader({ - catalog: { - providers: { - mcpRegistry: { - baseUrl: 'https://registry.example.com', - hostAllowList: ['other.example.com'], - }, - }, - }, - }); + const config = new ConfigReader( + providersConfig({ + baseUrl: 'https://registry.example.com', + hostAllowList: ['other.example.com'], + }), + ); expect(() => readMcpRegistryProviderConfig(config)).toThrow( /not in the configured hostAllowList/, @@ -275,16 +271,12 @@ describe('readMcpRegistryProviderConfig', () => { }); it('normalizes hostAllowList entries to lowercase', () => { - const config = new ConfigReader({ - catalog: { - providers: { - mcpRegistry: { - baseUrl: 'https://Registry.Example.COM', - hostAllowList: ['REGISTRY.EXAMPLE.COM'], - }, - }, - }, - }); + const config = new ConfigReader( + providersConfig({ + baseUrl: 'https://Registry.Example.COM', + hostAllowList: ['REGISTRY.EXAMPLE.COM'], + }), + ); const result = readMcpRegistryProviderConfig(config); expect(result!.hostAllowList).toEqual(['registry.example.com']); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts index e9ba90e5981..716f64f9a35 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts @@ -36,7 +36,17 @@ const DEFAULT_MAX_ENTRIES = 5000; /** Default remotesOnly when omitted. */ const DEFAULT_REMOTES_ONLY = false; -/** Supported single-registry config keys under `catalog.providers.mcpRegistry`. */ +/** + * Reserved instance id under `catalog.providers.mcpRegistry`. + * This implementation expects only this key; additional ids are rejected + * until multi-registry support lands. + */ +export const MCP_REGISTRY_INSTANCE_ID = 'mcpRegistry'; + +/** Config path for the reserved registry instance. */ +const MCP_REGISTRY_INSTANCE_CONFIG_PATH = `catalog.providers.mcpRegistry.${MCP_REGISTRY_INSTANCE_ID}`; + +/** Supported single-registry config keys under the reserved instance. */ const KNOWN_MCP_REGISTRY_KEYS = new Set([ 'baseUrl', 'baseName', @@ -71,7 +81,7 @@ export function safeGetOptionalString( } /** - * Reject keyed multi-registry maps under `mcpRegistry`. + * Reject unexpected nested objects under a registry instance config. * * @internal */ @@ -91,7 +101,7 @@ export function assertSingleRegistryConfig(registryConfig: Config): void { } if (nested && nested.keys().length > 0) { throw new Error( - `Invalid catalog.providers.mcpRegistry configuration: found ` + + `Invalid ${MCP_REGISTRY_INSTANCE_CONFIG_PATH} configuration: found ` + `keyed instance "${key}". Configure a single registry object ` + `with baseUrl, baseName, apiVersion, schedule, pageLimit, ` + `pageSize, and defaultOwner.`, @@ -100,6 +110,64 @@ export function assertSingleRegistryConfig(registryConfig: Config): void { } } +/** + * Optional warning sink used when configuration is accepted with caveats + * (e.g. ignored extra registry instance ids). + * + * @internal + */ +export type ConfigWarnFn = (message: string) => void; + +/** + * Resolve the reserved registry instance from the providers map. + * + * `catalog.providers.mcpRegistry` is a map of instance ids. This + * implementation only reads the reserved key {@link MCP_REGISTRY_INSTANCE_ID}; + * additional ids are ignored (with an optional warning). + * + * @internal + */ +export function readReservedRegistryInstanceConfig( + providersMap: Config, + warn?: ConfigWarnFn, +): Config | undefined { + const keys = providersMap.keys(); + if (keys.length === 0) { + return undefined; + } + + const legacyKeys = keys.filter(key => KNOWN_MCP_REGISTRY_KEYS.has(key)); + if (legacyKeys.length > 0) { + throw new Error( + `Invalid catalog.providers.mcpRegistry configuration: registry ` + + `options (${legacyKeys.join(', ')}) must be nested under the ` + + `reserved instance key "${MCP_REGISTRY_INSTANCE_ID}" ` + + `(e.g. catalog.providers.mcpRegistry.${MCP_REGISTRY_INSTANCE_ID}.baseUrl).`, + ); + } + + const unexpectedKeys = keys.filter(key => key !== MCP_REGISTRY_INSTANCE_ID); + if (unexpectedKeys.length > 0) { + warn?.( + `catalog.providers.mcpRegistry has additional instance id(s) ` + + `[${unexpectedKeys.join( + ', ', + )}] which are ignored; multiple MCP Registry providers are not ` + + `supported yet. Only the reserved "${MCP_REGISTRY_INSTANCE_ID}" ` + + `instance is used.`, + ); + } + + const registryConfig = providersMap.getOptionalConfig( + MCP_REGISTRY_INSTANCE_ID, + ); + if (!registryConfig) { + return undefined; + } + + return registryConfig; +} + /** * Read and validate the required HTTP(S) `baseUrl`. * @@ -109,7 +177,7 @@ export function readRequiredHttpBaseUrl(registryConfig: Config): string { const baseUrl = safeGetOptionalString(registryConfig, 'baseUrl'); if (!baseUrl) { throw new Error( - `Invalid catalog.providers.mcpRegistry configuration: missing ` + + `Invalid ${MCP_REGISTRY_INSTANCE_CONFIG_PATH} configuration: missing ` + `required "baseUrl" field. Set baseUrl to the MCP Registry base URL ` + `(e.g., "https://registry.example.com").`, ); @@ -120,7 +188,7 @@ export function readRequiredHttpBaseUrl(registryConfig: Config): string { parsedUrl = new URL(baseUrl); } catch { throw new Error( - `Invalid catalog.providers.mcpRegistry configuration: "baseUrl" ` + + `Invalid ${MCP_REGISTRY_INSTANCE_CONFIG_PATH} configuration: "baseUrl" ` + `is not a valid URL: "${baseUrl}". Set baseUrl to an absolute ` + `HTTP(S) URL (e.g., "https://registry.example.com").`, ); @@ -128,7 +196,7 @@ export function readRequiredHttpBaseUrl(registryConfig: Config): string { if (parsedUrl.protocol !== 'http:' && parsedUrl.protocol !== 'https:') { throw new Error( - `Invalid catalog.providers.mcpRegistry configuration: "baseUrl" ` + + `Invalid ${MCP_REGISTRY_INSTANCE_CONFIG_PATH} configuration: "baseUrl" ` + `must use http or https protocol, got "${parsedUrl.protocol}" ` + `in "${baseUrl}".`, ); @@ -147,7 +215,7 @@ export function readPageLimit(registryConfig: Config): number { registryConfig.getOptionalNumber('pageLimit') ?? DEFAULT_PAGE_LIMIT; if (pageLimit < 1) { throw new Error( - `Invalid catalog.providers.mcpRegistry configuration: "pageLimit" ` + + `Invalid ${MCP_REGISTRY_INSTANCE_CONFIG_PATH} configuration: "pageLimit" ` + `must be at least 1, got ${pageLimit}.`, ); } @@ -166,7 +234,7 @@ export function readMaxEntries(registryConfig: Config): number { registryConfig.getOptionalNumber('maxEntries') ?? DEFAULT_MAX_ENTRIES; if (maxEntries < 1) { throw new Error( - `Invalid catalog.providers.mcpRegistry configuration: "maxEntries" ` + + `Invalid ${MCP_REGISTRY_INSTANCE_CONFIG_PATH} configuration: "maxEntries" ` + `must be at least 1, got ${maxEntries}.`, ); } @@ -184,7 +252,7 @@ export function readOptionalPageSize( const pageSize = registryConfig.getOptionalNumber('pageSize'); if (pageSize !== undefined && pageSize < 1) { throw new Error( - `Invalid catalog.providers.mcpRegistry configuration: "pageSize" ` + + `Invalid ${MCP_REGISTRY_INSTANCE_CONFIG_PATH} configuration: "pageSize" ` + `must be at least 1, got ${pageSize}.`, ); } @@ -224,7 +292,7 @@ export function validateHostAllowList( const hostname = parsed.hostname.toLowerCase(); if (!hostAllowList.includes(hostname)) { throw new Error( - `Invalid catalog.providers.mcpRegistry configuration: the hostname ` + + `Invalid ${MCP_REGISTRY_INSTANCE_CONFIG_PATH} configuration: the hostname ` + `"${hostname}" from baseUrl "${url}" is not in the configured ` + `hostAllowList [${hostAllowList.join(', ')}].`, ); @@ -333,21 +401,32 @@ export function resolveMcpRegistryProviderConfig( /** * Read and validate the MCP Registry provider configuration from - * `catalog.providers.mcpRegistry`. Returns `undefined` when the - * config key is absent (inert module). + * `catalog.providers.mcpRegistry.mcpRegistry` (reserved instance id). + * Returns `undefined` when the providers map or reserved instance is + * absent (inert module). * - * @throws When the config is a keyed map of instances, or when - * `baseUrl` is missing. + * Additional instance ids under `catalog.providers.mcpRegistry` are + * ignored; pass `warn` to surface that multiple registries are not + * supported yet. + * + * @throws When a legacy flat `catalog.providers.mcpRegistry` object is + * used, or when `baseUrl` is missing on the reserved instance. */ export function readMcpRegistryProviderConfig( rootConfig: Config, + warn?: ConfigWarnFn, ): ResolvedMcpRegistryProviderConfig | undefined { const providersConfig = rootConfig.getOptionalConfig('catalog.providers'); if (!providersConfig) { return undefined; } - const registryConfig = providersConfig.getOptionalConfig('mcpRegistry'); + const providersMap = providersConfig.getOptionalConfig('mcpRegistry'); + if (!providersMap) { + return undefined; + } + + const registryConfig = readReservedRegistryInstanceConfig(providersMap, warn); if (!registryConfig) { return undefined; } diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts index 6ac155734e9..d1558f79b82 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts @@ -43,11 +43,13 @@ export const catalogModuleMcpRegistryProvider = createBackendModule({ scheduler: coreServices.scheduler, }, async init({ catalog, config, logger, scheduler }) { - const providerConfig = readMcpRegistryProviderConfig(config); + const providerConfig = readMcpRegistryProviderConfig(config, message => + logger.warn(message), + ); if (!providerConfig) { logger.info( - 'catalog.providers.mcpRegistry not configured; ' + + 'catalog.providers.mcpRegistry.mcpRegistry not configured; ' + 'MCP Registry provider is inactive.', ); return; From 988ccb941b65d5870cb56c66ae4f7ea530cf9fa5 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 19:12:17 -0400 Subject: [PATCH 48/63] fix(#4815): clarify MCP Registry fetch error messages Prefer the underlying cause message for network failures and avoid embedding Error constructor names like TypeError in operator-facing logs. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../docs/deploy-mcp-registry-locally.md | 4 +-- .../src/client.test.ts | 14 +++++++-- .../src/client.ts | 16 +++++++--- .../src/providerUtils.ts | 3 +- .../src/util.test.ts | 20 +++++++++++- .../src/util.ts | 31 +++++++++++++++++++ 6 files changed, 76 insertions(+), 12 deletions(-) diff --git a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md index 67e36a599ee..640df0cc208 100644 --- a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md +++ b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md @@ -38,8 +38,8 @@ registry), and serves the API at Start the registry **before** `yarn dev`. If the provider syncs while the registry is still importing seed data, you will see -`Failed to reach MCP Registry ... TypeError: fetch failed` (no mutation). Restart -the backend after the registry is ready, or wait for the next scheduled sync. +`Failed to reach MCP Registry ...` (no mutation). Restart the backend after the +registry is ready, or wait for the next scheduled sync. Optional environment variables: diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts index f1e6cdc20c2..2b67963a698 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts @@ -335,8 +335,12 @@ describe('fetchRegistryServers', () => { ).rejects.toThrow(/repeated cursor/i); }); - it('throws on network error', async () => { - const fn = mockFetch([{ throws: true }]); + it('throws on network error without embedding the Error constructor name', async () => { + const fn = jest.fn().mockRejectedValue( + new TypeError('fetch failed', { + cause: new Error('connect ECONNREFUSED 127.0.0.1:8080'), + }), + ); await expect( fetchRegistryServers({ @@ -345,7 +349,11 @@ describe('fetchRegistryServers', () => { pageLimit: 10, fetchApi: fn, }), - ).rejects.toThrow(McpRegistryClientError); + ).rejects.toMatchObject({ + name: 'McpRegistryClientError', + message: + 'Failed to reach MCP Registry at https://registry.example.com/v1/servers: connect ECONNREFUSED 127.0.0.1:8080', + }); }); it('throws on non-2xx status', async () => { diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index e26629ee852..69a7e428fa8 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -15,7 +15,7 @@ */ import type { McpServerDocument } from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; -import { stripTrailingSlashes } from './util'; +import { formatErrorDetail, stripTrailingSlashes } from './util'; /** Max characters of an error response body included in client errors. */ const MAX_ERROR_BODY_LENGTH = 256; @@ -169,7 +169,9 @@ export function parseServersEndpointUrl( return new URL(endpoint); } catch (err) { throw new McpRegistryClientError( - `Invalid MCP Registry endpoint URL "${endpoint}": ${err}`, + `Invalid MCP Registry endpoint URL "${endpoint}": ${formatErrorDetail( + err, + )}`, ); } } @@ -229,7 +231,7 @@ export function resolveRedirectUrl(currentUrl: URL, location: string): URL { } catch (err) { throw new McpRegistryClientError( `MCP Registry returned an invalid redirect Location "${location}" ` + - `from ${currentUrl}: ${err}`, + `from ${currentUrl}: ${formatErrorDetail(err)}`, ); } } @@ -310,7 +312,9 @@ export async function fetchRegistryPage( body = (await response.json()) as McpRegistryListResponse; } catch (err) { throw new McpRegistryClientError( - `MCP Registry returned unparseable JSON from ${requestUrl}: ${err}`, + `MCP Registry returned unparseable JSON from ${requestUrl}: ${formatErrorDetail( + err, + )}`, ); } @@ -340,7 +344,9 @@ async function fetchOnce( response = await doFetch(requestUrl, { redirect: 'manual' }); } catch (err) { throw new McpRegistryClientError( - `Failed to reach MCP Registry at ${requestUrl}: ${err}`, + `Failed to reach MCP Registry at ${requestUrl}: ${formatErrorDetail( + err, + )}`, ); } assertResponseUrlAllowed(response, hostAllowList, requestUrl); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/providerUtils.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/providerUtils.ts index 4078fe729d3..6347d64681e 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/providerUtils.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/providerUtils.ts @@ -17,6 +17,7 @@ import { isAllowedUrl } from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; import type { McpServerDocument } from '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping'; import type { McpRegistryServerEntry } from './client'; +import { formatErrorDetail } from './util'; /** * Whether a server.json document declares at least one native remote @@ -86,5 +87,5 @@ export function formatMappingFailureMessage( if (version) { message += ` (version "${version}")`; } - return `${message}: ${err}`; + return `${message}: ${formatErrorDetail(err)}`; } diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.test.ts index 99384b37de2..0c8074ce49c 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.test.ts @@ -14,7 +14,7 @@ * limitations under the License. */ -import { stripTrailingSlashes } from './util'; +import { formatErrorDetail, stripTrailingSlashes } from './util'; describe('stripTrailingSlashes', () => { it('returns the value unchanged when there is no trailing slash', () => { @@ -36,3 +36,21 @@ describe('stripTrailingSlashes', () => { expect(stripTrailingSlashes('///')).toBe(''); }); }); + +describe('formatErrorDetail', () => { + it('returns the Error message without the constructor name', () => { + expect(formatErrorDetail(new TypeError('fetch failed'))).toBe( + 'fetch failed', + ); + }); + + it('prefers the deepest cause message for wrapped fetch failures', () => { + const cause = new Error('connect ECONNREFUSED 127.0.0.1:8080'); + const err = new TypeError('fetch failed', { cause }); + expect(formatErrorDetail(err)).toBe('connect ECONNREFUSED 127.0.0.1:8080'); + }); + + it('returns string values as-is', () => { + expect(formatErrorDetail('boom')).toBe('boom'); + }); +}); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.ts index 686b1ddd07c..9065f390c2d 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/util.ts @@ -26,3 +26,34 @@ export function stripTrailingSlashes(value: string): string { } return end === value.length ? value : value.slice(0, end); } + +/** + * Format an unknown thrown value for operator-facing error text. + * + * Prefer the deepest `cause` message (Node/undici often wraps network + * failures as `TypeError: fetch failed` with a useful cause). Never + * prefixes the Error constructor name (e.g. `TypeError:`). + * + * @internal + */ +export function formatErrorDetail(err: unknown): string { + if (err instanceof Error) { + let current: Error = err; + // Walk a short cause chain for a more specific message. + for (let depth = 0; depth < 5; depth += 1) { + const cause = (current as Error & { cause?: unknown }).cause; + if (!(cause instanceof Error) || !cause.message) { + break; + } + current = cause; + } + if (current.message) { + return current.message; + } + return current.name || 'unknown error'; + } + if (typeof err === 'string' && err.length > 0) { + return err; + } + return String(err); +} From 0351933846553ac202cbf28b04f0ce655a8452e2 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 19:19:40 -0400 Subject: [PATCH 49/63] chore(#4815): add 'example' tag to mcp-server examples Signed-off-by: Michael Valdron --- workspaces/ai-integrations/examples/api-mcp-servers.yaml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/workspaces/ai-integrations/examples/api-mcp-servers.yaml b/workspaces/ai-integrations/examples/api-mcp-servers.yaml index 46422398c0a..9d5228229ba 100644 --- a/workspaces/ai-integrations/examples/api-mcp-servers.yaml +++ b/workspaces/ai-integrations/examples/api-mcp-servers.yaml @@ -21,6 +21,7 @@ metadata: tags: - mcp - ai + - example annotations: backstage.io/managed-by-location: url:http://localhost:8080 backstage.io/managed-by-origin-location: url:http://localhost:8080 @@ -64,6 +65,7 @@ metadata: tags: - mcp - ai + - example annotations: backstage.io/managed-by-location: url:http://localhost:8080 backstage.io/managed-by-origin-location: url:http://localhost:8080 @@ -135,6 +137,7 @@ metadata: tags: - mcp - ai + - example annotations: backstage.io/managed-by-location: url:http://localhost:8080 backstage.io/managed-by-origin-location: url:http://localhost:8080 @@ -170,6 +173,7 @@ metadata: tags: - mcp - ai + - example annotations: backstage.io/managed-by-location: url:http://localhost:8080 backstage.io/managed-by-origin-location: url:http://localhost:8080 From 51119d30d64a9eee71e888a22d7c3fd3447a75dd Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 19:33:19 -0400 Subject: [PATCH 50/63] fix(#4815): satisfy tsc for nested mcpRegistry config tests Type providersConfig as JsonObject and drop an unused import so yarn tsc:full passes in CI. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../src/config.test.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts index 328c5c1130e..739f15f6a50 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts @@ -15,6 +15,7 @@ */ import { ConfigReader } from '@backstage/config'; +import type { JsonObject } from '@backstage/types'; import { assertSingleRegistryConfig, MCP_REGISTRY_INSTANCE_ID, @@ -26,14 +27,13 @@ import { readProviderSchedule, readRemotesOnly, readRequiredHttpBaseUrl, - readReservedRegistryInstanceConfig, resolveMcpRegistryProviderConfig, safeGetOptionalString, validateHostAllowList, } from './config'; /** Nest instance options under the reserved `mcpRegistry` map key. */ -function providersConfig(instance: Record) { +function providersConfig(instance: JsonObject): JsonObject { return { catalog: { providers: { From a5231959388dcb33a4d7d9caba5fb7d05a37874b Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 19:38:15 -0400 Subject: [PATCH 51/63] chore(#4815): expand MCP Registry changesets for provider features Document nested config, remotesOnly, hostAllowList, maxEntries, and the mapping package rename for consumers. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../.changeset/mcp-registry-mapping-common-provider-link.md | 2 +- .../ai-integrations/.changeset/mcp-registry-provider-plugin.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/workspaces/ai-integrations/.changeset/mcp-registry-mapping-common-provider-link.md b/workspaces/ai-integrations/.changeset/mcp-registry-mapping-common-provider-link.md index 844fe53ad7a..8c380905d74 100644 --- a/workspaces/ai-integrations/.changeset/mcp-registry-mapping-common-provider-link.md +++ b/workspaces/ai-integrations/.changeset/mcp-registry-mapping-common-provider-link.md @@ -2,4 +2,4 @@ '@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping': minor --- -Rename the mapping common library to `catalog-mcp-registry-server-mapping` (package, directory, and `pluginId`) to match workspace naming conventions. +Rename the mapping common library to `catalog-mcp-registry-server-mapping` (package, directory, and `pluginId`) to match workspace naming conventions. Consumers of `@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common` should update to the new package name. diff --git a/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md b/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md index 88e452c98ad..3752a89afcc 100644 --- a/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md +++ b/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md @@ -2,4 +2,4 @@ '@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider': minor --- -Add MCP Registry provider backend module: a catalog entity provider that ingests MCP servers from one configured MCP Registry into the RHDH catalog as mcp-server API entities via cursor pagination, with full-mutation semantics, per-entry failure isolation with last-good retention, and configurable schedule, page limits, and identity prefix override. +Add MCP Registry provider backend module that ingests MCP servers from a configured registry into the catalog as mcp-server API entities. Supports cursor pagination with full-mutation semantics, per-entry failure isolation with last-good retention, nested config under `catalog.providers.mcpRegistry.mcpRegistry` (extra instance ids warn and are ignored), optional `remotesOnly`, `hostAllowList`, and `maxEntries` soft-stop with `pageLimit` resume, schedule and page controls, identity prefix override, redirect handling with Location validation, and clearer fetch error reporting. From 703de55f657b4bf50ee0fa5ba40cb7208f68a42b Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 19:39:58 -0400 Subject: [PATCH 52/63] fix(#4815): avoid nested template literals in deploy script Extract the volume mount string before JSON.stringify to address SonarCloud feedback. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../ai-integrations/scripts/deploy-local-mcp-registry.ts | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/workspaces/ai-integrations/scripts/deploy-local-mcp-registry.ts b/workspaces/ai-integrations/scripts/deploy-local-mcp-registry.ts index 16d43f9e6c7..724a22adf0d 100755 --- a/workspaces/ai-integrations/scripts/deploy-local-mcp-registry.ts +++ b/workspaces/ai-integrations/scripts/deploy-local-mcp-registry.ts @@ -127,10 +127,8 @@ function buildOverrideYaml(image: string, dataDir?: string): string { // Replace upstream ./data:/data:ro with a custom host directory. // `:z` is required for Podman/SELinux so the container (uid 65532) can // read the bind-mounted seed files; without it open() returns EACCES. - lines.push( - ' volumes:', - ` - ${JSON.stringify(`${dataDir}:/data:ro,z`)}`, - ); + const volumeMount = `${dataDir}:/data:ro,z`; + lines.push(' volumes:', ` - ${JSON.stringify(volumeMount)}`); } return `${lines.join('\n')}\n`; } From a5699e0bbdc3d0f34aab5432123bed562b5012c8 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 19:50:17 -0400 Subject: [PATCH 53/63] fix(#4815): warn when hostAllowList is unset at startup Log a defense-in-depth SSRF warning via the config warn sink when the provider starts without a hostname allowlist. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../README.md | 24 +++++++------- .../src/config.test.ts | 33 +++++++++++++++++-- .../src/config.ts | 12 +++++-- 3 files changed, 52 insertions(+), 17 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index e8ae3a83317..d55dfc28829 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -62,18 +62,18 @@ catalog: ### Configuration options -| Option | Required | Default | Description | -| --------------- | -------- | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `baseUrl` | Yes | — | MCP Registry base URL. Also passed to the mapping as `placeholderRemoteUrl` when a server has no valid remotes. | -| `baseName` | No | `mcp.registry` (mapping default) | Override the mapping identity prefix for `metadata.name` | -| `apiVersion` | No | `v1` | Registry API version slug. The servers endpoint is `//servers`. Note: the live MCP Registry may serve `/v0` or `/v0.1`; set `apiVersion` to match your registry. | -| `defaultOwner` | No | `unknown` (mapping default) | Backstage entity reference used as `spec.owner` | -| `pageLimit` | No | `10` | Maximum number of pages fetched per sync. When more pages remain, the provider saves the cursor and continues on the next sync (no mutation until the registry is fully traversed). | -| `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | -| `maxEntries` | No | `5000` | Maximum total server entries buffered for one complete registry traversal (spans resume syncs). When exceeded, the provider commits the buffer, saves an end cursor, and later traversals stop there until `maxEntries` is patched. | -| `remotesOnly` | No | `false` | When `true`, skip servers that do not declare at least one native remote (non-empty type and http(s) URL). Package-only and placeholder-remote servers are omitted from the catalog. | -| `hostAllowList` | No | _(none — all hosts allowed)_ | Array of permitted hostnames. When set, `baseUrl` hostname must be in this list and every outbound request (including each redirect `Location` and `response.url`) is validated at runtime. Missing `response.url` fails closed. Redirects are followed manually so disallowed hosts are never contacted. Provides defense-in-depth against SSRF. | -| `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | +| Option | Required | Default | Description | +| --------------- | -------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `baseUrl` | Yes | — | MCP Registry base URL. Also passed to the mapping as `placeholderRemoteUrl` when a server has no valid remotes. | +| `baseName` | No | `mcp.registry` (mapping default) | Override the mapping identity prefix for `metadata.name` | +| `apiVersion` | No | `v1` | Registry API version slug. The servers endpoint is `//servers`. Note: the live MCP Registry may serve `/v0` or `/v0.1`; set `apiVersion` to match your registry. | +| `defaultOwner` | No | `unknown` (mapping default) | Backstage entity reference used as `spec.owner` | +| `pageLimit` | No | `10` | Maximum number of pages fetched per sync. When more pages remain, the provider saves the cursor and continues on the next sync (no mutation until the registry is fully traversed). | +| `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | +| `maxEntries` | No | `5000` | Maximum total server entries buffered for one complete registry traversal (spans resume syncs). When exceeded, the provider commits the buffer, saves an end cursor, and later traversals stop there until `maxEntries` is patched. | +| `remotesOnly` | No | `false` | When `true`, skip servers that do not declare at least one native remote (non-empty type and http(s) URL). Package-only and placeholder-remote servers are omitted from the catalog. | +| `hostAllowList` | No | _(none — all hosts allowed)_ | Array of permitted hostnames. When set, `baseUrl` hostname must be in this list and every outbound request (including each redirect `Location` and `response.url`) is validated at runtime. Missing `response.url` fails closed. Redirects are followed manually so disallowed hosts are never contacted. Provides defense-in-depth against SSRF. When omitted, a warning is logged at startup. | +| `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | ### Multiple registries diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts index 739f15f6a50..0f01d5e8868 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts @@ -164,6 +164,7 @@ describe('readMcpRegistryProviderConfig', () => { mcpRegistry: { mcpRegistry: { baseUrl: 'https://registry.example.com', + hostAllowList: ['registry.example.com'], }, public: { baseUrl: 'https://public-registry.example.com', @@ -235,6 +236,7 @@ describe('readMcpRegistryProviderConfig', () => { }); it('reads hostAllowList when provided', () => { + const warnings: string[] = []; const config = new ConfigReader( providersConfig({ baseUrl: 'https://registry.example.com', @@ -242,19 +244,44 @@ describe('readMcpRegistryProviderConfig', () => { }), ); - const result = readMcpRegistryProviderConfig(config); + const result = readMcpRegistryProviderConfig(config, message => + warnings.push(message), + ); expect(result!.hostAllowList).toEqual(['registry.example.com']); + expect(warnings).toEqual([]); }); - it('returns undefined hostAllowList when omitted', () => { + it('returns undefined hostAllowList when omitted and warns', () => { + const warnings: string[] = []; const config = new ConfigReader( providersConfig({ baseUrl: 'https://registry.example.com', }), ); - const result = readMcpRegistryProviderConfig(config); + const result = readMcpRegistryProviderConfig(config, message => + warnings.push(message), + ); expect(result!.hostAllowList).toBeUndefined(); + expect(warnings).toHaveLength(1); + expect(warnings[0]).toMatch(/hostAllowList is not configured/); + expect(warnings[0]).toMatch(/SSRF/); + }); + + it('does not warn about hostAllowList when configured as empty deny-all', () => { + const warnings: string[] = []; + const config = new ConfigReader( + providersConfig({ + baseUrl: 'https://registry.example.com', + hostAllowList: [], + }), + ); + + // Empty list fails validation of baseUrl; catch before asserting warn absence. + expect(() => + readMcpRegistryProviderConfig(config, message => warnings.push(message)), + ).toThrow(/not in the configured hostAllowList/); + expect(warnings).toEqual([]); }); it('throws when baseUrl hostname is not in hostAllowList', () => { diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts index 716f64f9a35..96ab51cdaad 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts @@ -112,7 +112,7 @@ export function assertSingleRegistryConfig(registryConfig: Config): void { /** * Optional warning sink used when configuration is accepted with caveats - * (e.g. ignored extra registry instance ids). + * (e.g. ignored extra registry instance ids, or absent `hostAllowList`). * * @internal */ @@ -407,7 +407,8 @@ export function resolveMcpRegistryProviderConfig( * * Additional instance ids under `catalog.providers.mcpRegistry` are * ignored; pass `warn` to surface that multiple registries are not - * supported yet. + * supported yet. When `hostAllowList` is omitted, `warn` is also used + * to recommend configuring hostname restrictions. * * @throws When a legacy flat `catalog.providers.mcpRegistry` object is * used, or when `baseUrl` is missing on the reserved instance. @@ -438,6 +439,13 @@ export function readMcpRegistryProviderConfig( if (hostAllowList) { validateHostAllowList(baseUrl, hostAllowList); + } else { + warn?.( + `${MCP_REGISTRY_INSTANCE_CONFIG_PATH}.hostAllowList is not configured; ` + + `outbound registry requests are not restricted by hostname. Set ` + + `hostAllowList to permitted registry hostnames for defense-in-depth ` + + `against SSRF.`, + ); } return { From 95ce5c39fb9d65ff3730bba25fabd5ab808c88b2 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 19:57:30 -0400 Subject: [PATCH 54/63] feat(#4815): add defaultLifecycle to MCP Registry provider config Pass an optional lifecycle override through to the mapping so operators can align ingested mcp-server entities with other ai-integrations defaults. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../mcp-registry-provider-plugin.md | 2 +- workspaces/ai-integrations/app-config.yaml | 2 + .../examples/api-mcp-servers.yaml | 3 +- .../README.md | 29 ++++++------ .../config.d.ts | 2 + .../report.api.md | 1 + .../src/McpRegistryEntityProvider.test.ts | 46 +++++++++++++++++++ .../src/McpRegistryEntityProvider.ts | 3 ++ .../src/config.test.ts | 13 ++++++ .../src/config.ts | 6 ++- 10 files changed, 91 insertions(+), 16 deletions(-) diff --git a/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md b/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md index 3752a89afcc..b14868afaef 100644 --- a/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md +++ b/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md @@ -2,4 +2,4 @@ '@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider': minor --- -Add MCP Registry provider backend module that ingests MCP servers from a configured registry into the catalog as mcp-server API entities. Supports cursor pagination with full-mutation semantics, per-entry failure isolation with last-good retention, nested config under `catalog.providers.mcpRegistry.mcpRegistry` (extra instance ids warn and are ignored), optional `remotesOnly`, `hostAllowList`, and `maxEntries` soft-stop with `pageLimit` resume, schedule and page controls, identity prefix override, redirect handling with Location validation, and clearer fetch error reporting. +Add MCP Registry provider backend module that ingests MCP servers from a configured registry into the catalog as mcp-server API entities. Supports cursor pagination with full-mutation semantics, per-entry failure isolation with last-good retention, nested config under `catalog.providers.mcpRegistry.mcpRegistry` (extra instance ids warn and are ignored), optional `defaultOwner` / `defaultLifecycle` / `remotesOnly` / `hostAllowList` / `maxEntries` soft-stop with `pageLimit` resume, schedule and page controls, identity prefix override, redirect handling with Location validation, and clearer fetch error reporting. diff --git a/workspaces/ai-integrations/app-config.yaml b/workspaces/ai-integrations/app-config.yaml index 412412301cc..b0253021242 100644 --- a/workspaces/ai-integrations/app-config.yaml +++ b/workspaces/ai-integrations/app-config.yaml @@ -160,6 +160,8 @@ catalog: apiVersion: v0.1 # Optional: default entity owner (default: unknown) defaultOwner: '${OWNER:-default-owner}' + # Optional: default entity lifecycle (default: production) + defaultLifecycle: '${LIFECYCLE:-production}' # Optional: max pages fetched per sync (default: 10) # pageLimit: 10 # Optional: registry page size sent as ?limit= (omitted by default) diff --git a/workspaces/ai-integrations/examples/api-mcp-servers.yaml b/workspaces/ai-integrations/examples/api-mcp-servers.yaml index 9d5228229ba..60cdc8f1da4 100644 --- a/workspaces/ai-integrations/examples/api-mcp-servers.yaml +++ b/workspaces/ai-integrations/examples/api-mcp-servers.yaml @@ -4,7 +4,8 @@ # at examples/mcp-registry/seed-data/seed.json. # # Defaults match app-config.yaml (defaultOwner: default-owner, -# baseUrl: http://localhost:8080/, default baseName prefix mcp.registry). +# defaultLifecycle: production, baseUrl: http://localhost:8080/, +# default baseName prefix mcp.registry). # Provider location and sync-status annotations are included. # # Wired as a file location in app-config.yaml for local catalog review. diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index d55dfc28829..75b90c37278 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -41,6 +41,8 @@ catalog: # apiVersion: v1 # Optional: default entity owner (default: unknown) # defaultOwner: group:default/mcp-admins + # Optional: default entity lifecycle (default: production) + # defaultLifecycle: production # Optional: max pages fetched per sync (default: 10) # pageLimit: 10 # Optional: registry page size sent as ?limit= (omitted by default) @@ -62,18 +64,19 @@ catalog: ### Configuration options -| Option | Required | Default | Description | -| --------------- | -------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `baseUrl` | Yes | — | MCP Registry base URL. Also passed to the mapping as `placeholderRemoteUrl` when a server has no valid remotes. | -| `baseName` | No | `mcp.registry` (mapping default) | Override the mapping identity prefix for `metadata.name` | -| `apiVersion` | No | `v1` | Registry API version slug. The servers endpoint is `//servers`. Note: the live MCP Registry may serve `/v0` or `/v0.1`; set `apiVersion` to match your registry. | -| `defaultOwner` | No | `unknown` (mapping default) | Backstage entity reference used as `spec.owner` | -| `pageLimit` | No | `10` | Maximum number of pages fetched per sync. When more pages remain, the provider saves the cursor and continues on the next sync (no mutation until the registry is fully traversed). | -| `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | -| `maxEntries` | No | `5000` | Maximum total server entries buffered for one complete registry traversal (spans resume syncs). When exceeded, the provider commits the buffer, saves an end cursor, and later traversals stop there until `maxEntries` is patched. | -| `remotesOnly` | No | `false` | When `true`, skip servers that do not declare at least one native remote (non-empty type and http(s) URL). Package-only and placeholder-remote servers are omitted from the catalog. | -| `hostAllowList` | No | _(none — all hosts allowed)_ | Array of permitted hostnames. When set, `baseUrl` hostname must be in this list and every outbound request (including each redirect `Location` and `response.url`) is validated at runtime. Missing `response.url` fails closed. Redirects are followed manually so disallowed hosts are never contacted. Provides defense-in-depth against SSRF. When omitted, a warning is logged at startup. | -| `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | +| Option | Required | Default | Description | +| ------------------ | -------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `baseUrl` | Yes | — | MCP Registry base URL. Also passed to the mapping as `placeholderRemoteUrl` when a server has no valid remotes. | +| `baseName` | No | `mcp.registry` (mapping default) | Override the mapping identity prefix for `metadata.name` | +| `apiVersion` | No | `v1` | Registry API version slug. The servers endpoint is `//servers`. Note: the live MCP Registry may serve `/v0` or `/v0.1`; set `apiVersion` to match your registry. | +| `defaultOwner` | No | `unknown` (mapping default) | Backstage entity reference used as `spec.owner` | +| `defaultLifecycle` | No | `production` (mapping default) | Value used as `spec.lifecycle` | +| `pageLimit` | No | `10` | Maximum number of pages fetched per sync. When more pages remain, the provider saves the cursor and continues on the next sync (no mutation until the registry is fully traversed). | +| `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | +| `maxEntries` | No | `5000` | Maximum total server entries buffered for one complete registry traversal (spans resume syncs). When exceeded, the provider commits the buffer, saves an end cursor, and later traversals stop there until `maxEntries` is patched. | +| `remotesOnly` | No | `false` | When `true`, skip servers that do not declare at least one native remote (non-empty type and http(s) URL). Package-only and placeholder-remote servers are omitted from the catalog. | +| `hostAllowList` | No | _(none — all hosts allowed)_ | Array of permitted hostnames. When set, `baseUrl` hostname must be in this list and every outbound request (including each redirect `Location` and `response.url`) is validated at runtime. Missing `response.url` fails closed. Redirects are followed manually so disallowed hosts are never contacted. Provides defense-in-depth against SSRF. When omitted, a warning is logged at startup. | +| `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | ### Multiple registries @@ -92,7 +95,7 @@ The provider fully traverses the registry's cursor-based pagination, accumulatin ### Mapping -Each server entry's `.server` object is transformed into an `mcp-server` API entity using the [`catalog-mcp-registry-server-mapping`](../catalog-mcp-registry-server-mapping) library. The provider passes `defaultOwner` and `baseName` as caller overrides, and always passes the configured `baseUrl` as `placeholderRemoteUrl` so a server with no valid remotes gets a placeholder remote for that registry before falling back to `websiteUrl`. When `remotesOnly` is `true`, servers without a native remote are skipped before mapping. It never reimplements the mapping rules. +Each server entry's `.server` object is transformed into an `mcp-server` API entity using the [`catalog-mcp-registry-server-mapping`](../catalog-mcp-registry-server-mapping) library. The provider passes `defaultOwner`, `defaultLifecycle`, and `baseName` as caller overrides, and always passes the configured `baseUrl` as `placeholderRemoteUrl` so a server with no valid remotes gets a placeholder remote for that registry before falling back to `websiteUrl`. When `remotesOnly` is `true`, servers without a native remote are skipped before mapping. It never reimplements the mapping rules. ### Full mutation diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts index 99495e6eda6..a2c4a5ab493 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts @@ -26,6 +26,8 @@ interface McpRegistryInstanceConfig { /** @visibility backend */ defaultOwner?: string; /** @visibility backend */ + defaultLifecycle?: string; + /** @visibility backend */ pageLimit?: number; /** @visibility backend */ pageSize?: number; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md index a94f8902b45..e689b674433 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md @@ -38,6 +38,7 @@ export interface McpRegistryProviderConfig { baseName?: string; baseUrl: string; defaultOwner?: string; + defaultLifecycle?: string; hostAllowList?: string[]; maxEntries?: number; pageLimit?: number; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts index 734a41a2403..3827aa8bf51 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts @@ -207,6 +207,30 @@ describe('McpRegistryEntityProvider', () => { ); }); + it('passes defaultLifecycle to the mapping', async () => { + const body: McpRegistryListResponse = { + servers: [ + { server: createMockServerDoc('io.github.user/weather', '1.0.0') }, + ], + metadata: { count: 1 }, + }; + const fetchFn = mockFetchForResponses([body]); + const connection = createMockConnection(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ + defaultLifecycle: 'experimental', + }), + createMockLogger(), + { fetchApi: fetchFn }, + ); + await provider.connect(connection); + await provider.run(); + + const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + expect(mutation.entities[0].entity.spec.lifecycle).toBe('experimental'); + }); + it('uses mapping default owner when defaultOwner is omitted', async () => { const body: McpRegistryListResponse = { servers: [ @@ -229,6 +253,28 @@ describe('McpRegistryEntityProvider', () => { expect(mutation.entities[0].entity.spec.owner).toBe('unknown'); }); + it('uses mapping default lifecycle when defaultLifecycle is omitted', async () => { + const body: McpRegistryListResponse = { + servers: [ + { server: createMockServerDoc('io.github.user/weather', '1.0.0') }, + ], + metadata: { count: 1 }, + }; + const fetchFn = mockFetchForResponses([body]); + const connection = createMockConnection(); + + const provider = new McpRegistryEntityProvider( + createDefaultConfig(), + createMockLogger(), + { fetchApi: fetchFn }, + ); + await provider.connect(connection); + await provider.run(); + + const mutation = (connection.applyMutation as jest.Mock).mock.calls[0][0]; + expect(mutation.entities[0].entity.spec.lifecycle).toBe('production'); + }); + it('passes baseName as prefix override to the mapping', async () => { const body: McpRegistryListResponse = { servers: [ diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index 78c4bc3f2bc..9ea5737dc29 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -364,6 +364,9 @@ export class McpRegistryEntityProvider implements EntityProvider { if (this.config.defaultOwner) { mappingDefaults.owner = this.config.defaultOwner; } + if (this.config.defaultLifecycle) { + mappingDefaults.lifecycle = this.config.defaultLifecycle; + } if (this.config.baseName) { mappingDefaults.prefix = this.config.baseName; } diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts index 0f01d5e8868..53bef09417b 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts @@ -75,6 +75,7 @@ describe('readMcpRegistryProviderConfig', () => { expect(result!.pageSize).toBeUndefined(); expect(result!.baseName).toBeUndefined(); expect(result!.defaultOwner).toBeUndefined(); + expect(result!.defaultLifecycle).toBeUndefined(); expect(result!.schedule).toEqual({ frequency: { minutes: 30 }, timeout: { minutes: 3 }, @@ -223,6 +224,18 @@ describe('readMcpRegistryProviderConfig', () => { expect(result!.defaultOwner).toBe('group:default/mcp-admins'); }); + it('reads defaultLifecycle', () => { + const config = new ConfigReader( + providersConfig({ + baseUrl: 'https://registry.example.com', + defaultLifecycle: 'experimental', + }), + ); + + const result = readMcpRegistryProviderConfig(config); + expect(result!.defaultLifecycle).toBe('experimental'); + }); + it('reads apiVersion override', () => { const config = new ConfigReader( providersConfig({ diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts index 96ab51cdaad..3f10ac857ce 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts @@ -52,6 +52,7 @@ const KNOWN_MCP_REGISTRY_KEYS = new Set([ 'baseName', 'apiVersion', 'defaultOwner', + 'defaultLifecycle', 'pageLimit', 'pageSize', 'maxEntries', @@ -104,7 +105,7 @@ export function assertSingleRegistryConfig(registryConfig: Config): void { `Invalid ${MCP_REGISTRY_INSTANCE_CONFIG_PATH} configuration: found ` + `keyed instance "${key}". Configure a single registry object ` + `with baseUrl, baseName, apiVersion, schedule, pageLimit, ` + - `pageSize, and defaultOwner.`, + `pageSize, defaultOwner, and defaultLifecycle.`, ); } } @@ -346,6 +347,8 @@ export interface McpRegistryProviderConfig { apiVersion?: string; /** Default entity owner ref when the mapping does not supply one. */ defaultOwner?: string; + /** Default entity lifecycle when the mapping does not supply one. */ + defaultLifecycle?: string; /** Maximum pages fetched per sync (default `10`); excess pages resume next sync. */ pageLimit?: number; /** Registry `?limit=` page-size query; omitted from the request when unset. */ @@ -455,6 +458,7 @@ export function readMcpRegistryProviderConfig( safeGetOptionalString(registryConfig, 'apiVersion') ?? DEFAULT_API_VERSION, defaultOwner: safeGetOptionalString(registryConfig, 'defaultOwner'), + defaultLifecycle: safeGetOptionalString(registryConfig, 'defaultLifecycle'), pageLimit: readPageLimit(registryConfig), pageSize: readOptionalPageSize(registryConfig), maxEntries: readMaxEntries(registryConfig), From 4147d7d232d2986d8e522539ca3aa23228b664f7 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 20:04:09 -0400 Subject: [PATCH 55/63] docs(#4815): reorganize MCP Registry provider installation section Split prerequisite, package install, and module registration so the ai-model dependency is clear up front. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../README.md | 20 ++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index 75b90c37278..da9a8f69a8a 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -4,13 +4,25 @@ A Backstage catalog backend module that ingests MCP servers from a configured [M ## Installation -Install the package: +### Prerequisite + +Since [Backstage 1.51.0](https://github.com/backstage/backstage/releases/tag/v1.51.0), `spec.type: mcp-server` entities (they use `spec.remotes` and omit `spec.definition`) are accepted only when `@backstage/plugin-catalog-backend-module-ai-model` is installed. Without that module the catalog keeps the generic API validator, which rejects these entities and does not list them. + +### Install packages + +From your Backstage root: ```bash -yarn add @red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider +yarn --cwd packages/backend add \ + @backstage/plugin-catalog-backend-module-ai-model \ + @red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider ``` -Add the module to your backend: +Skip the `ai-model` package if it is already installed. + +### Register modules + +Add both modules to your backend: ```ts // packages/backend/src/index.ts @@ -22,8 +34,6 @@ backend.add( ); ``` -Since [Backstage 1.51.0](https://github.com/backstage/backstage/releases/tag/v1.51.0), `spec.type: mcp-server` entities (they use `spec.remotes` and omit `spec.definition`) are accepted only when `@backstage/plugin-catalog-backend-module-ai-model` is installed. Without that module the catalog keeps the generic API validator, which rejects these entities and does not list them. - ## Configuration Configure the provider in your `app-config.yaml`: From b7b995a627b7bc2424bd24006e979c74eb89c3bf Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 21:14:14 -0400 Subject: [PATCH 56/63] fix(#4815): retain degraded MCP servers across soft-stop syncs Keep last-good entities outside the maxEntries window in the mutation with degraded sync status, and re-index them until they are refreshed. Assisted-by: grok-4.6 Signed-off-by: Michael Valdron Co-authored-by: Cursor --- .../mcp-registry-provider-plugin.md | 2 +- .../README.md | 5 +- .../McpRegistryEntityProvider.parts.test.ts | 8 +- .../src/McpRegistryEntityProvider.test.ts | 284 +++++++++++++++++- .../src/McpRegistryEntityProvider.ts | 81 ++++- 5 files changed, 350 insertions(+), 30 deletions(-) diff --git a/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md b/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md index b14868afaef..e61d6d87ff7 100644 --- a/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md +++ b/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md @@ -2,4 +2,4 @@ '@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider': minor --- -Add MCP Registry provider backend module that ingests MCP servers from a configured registry into the catalog as mcp-server API entities. Supports cursor pagination with full-mutation semantics, per-entry failure isolation with last-good retention, nested config under `catalog.providers.mcpRegistry.mcpRegistry` (extra instance ids warn and are ignored), optional `defaultOwner` / `defaultLifecycle` / `remotesOnly` / `hostAllowList` / `maxEntries` soft-stop with `pageLimit` resume, schedule and page controls, identity prefix override, redirect handling with Location validation, and clearer fetch error reporting. +Add MCP Registry provider backend module that ingests MCP servers from a configured registry into the catalog as mcp-server API entities. Supports cursor pagination with full-mutation semantics, per-entry failure isolation with last-good retention, nested config under `catalog.providers.mcpRegistry.mcpRegistry` (extra instance ids warn and are ignored), optional `defaultOwner` / `defaultLifecycle` / `remotesOnly` / `hostAllowList` / `maxEntries` soft-stop with `pageLimit` resume (re-adding last-good as degraded on later syncs until the server is refreshed successfully), schedule and page controls, identity prefix override, redirect handling with Location validation, and clearer fetch error reporting. diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index da9a8f69a8a..e27249f2e7d 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -101,7 +101,7 @@ override the identity prefix if needed for future multi-registry support. ### Pagination -The provider fully traverses the registry's cursor-based pagination, accumulating all server entries. Cursors are treated as opaque strings. The `pageLimit` configuration caps the number of pages fetched **per sync**. If the registry still has more pages after that cap, the provider saves the next cursor, buffers the entries fetched so far, and continues from that cursor on the next scheduled sync — it does **not** commit a mutation until a sync reaches the end of the registry (no `nextCursor`). When a traversal completes, the provider commits a full mutation and the following sync starts from the beginning again. The `maxEntries` configuration caps the total buffered servers for that complete traversal (not per individual sync tick). When the cap is hit, the provider commits the buffered entries, saves that stop point as an **end cursor**, and later full traversals end at that cursor instead of a missing `nextCursor`. Patching `maxEntries` clears the saved end cursor so traversal returns to normal. +The provider fully traverses the registry's cursor-based pagination, accumulating all server entries. Cursors are treated as opaque strings. The `pageLimit` configuration caps the number of pages fetched **per sync**. If the registry still has more pages after that cap, the provider saves the next cursor, buffers the entries fetched so far, and continues from that cursor on the next scheduled sync — it does **not** commit a mutation until a sync reaches the end of the registry (no `nextCursor`). When a traversal completes, the provider commits a full mutation and the following sync starts from the beginning again. The `maxEntries` configuration caps the total buffered servers for that complete traversal (not per individual sync tick). When the cap is hit, the provider commits the buffered entries, saves that stop point as an **end cursor**, and later full traversals end at that cursor instead of a missing `nextCursor`. Servers that were committed earlier but fall outside the soft-stop window on a later sync (for example when new registry entries shift page listings) are retained from last-good with `redhat.com/rhdh-mcp-registry-sync-status: degraded` instead of being pruned. Patching `maxEntries` clears the saved end cursor so traversal returns to normal. ### Mapping @@ -113,7 +113,8 @@ When a registry traversal completes (no remaining `nextCursor`, possibly after s ### Error handling -- **Per-entry failures**: If a single server entry fails mapping, the provider logs the error and continues. If a last-good entity exists for that server (matched by `name` and `version`), it is retained with `redhat.com/rhdh-mcp-registry-sync-status: degraded`. +- **Per-entry failures**: If a single server entry fails mapping or formatting validation (for example an invalid or malformed `server.json`), the provider logs the error and continues. If a last-good entity exists for that server (matched by `name` and `version`), it is retained with `redhat.com/rhdh-mcp-registry-sync-status: degraded` and re-added on later syncs until mapping succeeds again. +- **Soft-stop window shifts**: While an end cursor from `maxEntries` is active, previously synced servers that are no longer inside the truncated window are retained the same way (`degraded`) and re-added on later syncs until they appear in the soft-stop window again. - **Registry-level failures**: Transport errors, non-2xx responses, unparseable JSON, or pagination safeguard trips abort the sync — no mutation is committed, preserving the prior catalog state. ### Annotations diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts index b9f25516fe4..e23af578e15 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.parts.test.ts @@ -712,7 +712,7 @@ describe('McpRegistryEntityProvider parts', () => { }); describe('rebuildLastGoodIndex', () => { - it('indexes ok entities and skips degraded ones', () => { + it('indexes both ok and degraded entities', () => { const provider = new McpRegistryEntityProvider( createDefaultConfig(), createMockLogger(), @@ -722,17 +722,17 @@ describe('McpRegistryEntityProvider parts', () => { parts(provider).rebuildLastGoodIndex([ok, degraded]); - expect(parts(provider).lastGoodIndex.size).toBe(1); + expect(parts(provider).lastGoodIndex.size).toBe(2); expect( parts(provider).lastGoodIndex.get( buildLastGoodKey('ok/server', '1.0.0'), ), ).toBe(ok); expect( - parts(provider).lastGoodIndex.has( + parts(provider).lastGoodIndex.get( buildLastGoodKey('bad/server', '1.0.0'), ), - ).toBe(false); + ).toBe(degraded); }); it('skips entities missing name or version annotations', () => { diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts index 3827aa8bf51..e1aa4531fc4 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts @@ -14,6 +14,8 @@ * limitations under the License. */ +import { readFileSync } from 'fs'; +import { resolve } from 'path'; import { McpRegistryEntityProvider } from './McpRegistryEntityProvider'; import type { SchedulerServiceTaskRunner } from '@backstage/backend-plugin-api'; import type { EntityProviderConnection } from '@backstage/plugin-catalog-node'; @@ -790,7 +792,7 @@ describe('McpRegistryEntityProvider', () => { expect(mutation.entities).toHaveLength(2); }); - it('does not retain degraded entities in lastGoodIndex on subsequent syncs', async () => { + it('keeps re-adding degraded entities on subsequent syncs until refreshed', async () => { const goodBody: McpRegistryListResponse = { servers: [{ server: createMockServerDoc('test/server', '1.0.0') }], metadata: { count: 1 }, @@ -819,42 +821,302 @@ describe('McpRegistryEntityProvider', () => { json: async () => goodBody, text: async () => JSON.stringify(goodBody), } as unknown as Response); - // Second sync — mapping fails, uses last-good (degraded) + // Second and third syncs — mapping fails; degraded last-good is re-added combinedFetch.mockResolvedValueOnce({ ok: true, status: 200, json: async () => badBody, text: async () => JSON.stringify(badBody), } as unknown as Response); - // Third sync — mapping fails again; degraded entity from second - // sync should NOT be in last-good index combinedFetch.mockResolvedValueOnce({ ok: true, status: 200, json: async () => badBody, text: async () => JSON.stringify(badBody), } as unknown as Response); + // Fourth sync — mapping succeeds again + combinedFetch.mockResolvedValueOnce({ + ok: true, + status: 200, + json: async () => goodBody, + text: async () => JSON.stringify(goodBody), + } as unknown as Response); const connection = createMockConnection(); - const logger = createMockLogger(); const provider = new McpRegistryEntityProvider( createDefaultConfig(), - logger, + createMockLogger(), { fetchApi: combinedFetch }, ); await provider.connect(connection); - // First sync — populates last-good await provider.run(); - // Second sync — uses last-good, commits degraded await provider.run(); - // Third sync — degraded entity from second sync should not be - // in last-good index, so no entity should be retained await provider.run(); + await provider.run(); + + expect(connection.applyMutation).toHaveBeenCalledTimes(4); + const secondMutation = (connection.applyMutation as jest.Mock).mock + .calls[1][0]; + const thirdMutation = (connection.applyMutation as jest.Mock).mock + .calls[2][0]; + const fourthMutation = (connection.applyMutation as jest.Mock).mock + .calls[3][0]; + + expect(secondMutation.entities).toHaveLength(1); + expect( + secondMutation.entities[0].entity.metadata.annotations?.[ + 'redhat.com/rhdh-mcp-registry-sync-status' + ], + ).toBe('degraded'); + expect(thirdMutation.entities).toHaveLength(1); + expect( + thirdMutation.entities[0].entity.metadata.annotations?.[ + 'redhat.com/rhdh-mcp-registry-sync-status' + ], + ).toBe('degraded'); + expect(fourthMutation.entities).toHaveLength(1); + expect( + fourthMutation.entities[0].entity.metadata.annotations?.[ + 'redhat.com/rhdh-mcp-registry-sync-status' + ], + ).toBe('ok'); + }); + + it('retains last-good as degraded when maxEntries soft-stop drops a previously synced seed server after a listing shift', async () => { + const seedDocs = JSON.parse( + readFileSync( + resolve( + __dirname, + '../../../examples/mcp-registry/seed-data/seed.json', + ), + 'utf8', + ), + ) as Array<{ name: string; version: string }>; + + expect(seedDocs).toHaveLength(4); + const [atlas, workspaceFs, diceWeather, remoteWorkspace] = seedDocs; + const inserted = createMockServerDoc('io.example.labs/new-front', '0.1.0'); + + // pageSize=1 so maxEntries=3 soft-stops after three servers and + // saves an end cursor at the fourth page's request cursor. + const firstPassPages: McpRegistryListResponse[] = [ + { + servers: [{ server: atlas as any }], + metadata: { count: 4, nextCursor: 'cursor-1' }, + }, + { + servers: [{ server: workspaceFs as any }], + metadata: { count: 4, nextCursor: 'cursor-2' }, + }, + { + servers: [{ server: diceWeather as any }], + metadata: { count: 4, nextCursor: 'cursor-3' }, + }, + { + servers: [{ server: remoteWorkspace as any }], + metadata: { count: 4, nextCursor: 'cursor-4' }, + }, + ]; + + // New entry at the front shifts listings; traversal still stops at + // endCursor cursor-3, so dice-weather falls out of the window. + const secondPassPages: McpRegistryListResponse[] = [ + { + servers: [{ server: inserted }], + metadata: { count: 5, nextCursor: 'cursor-1' }, + }, + { + servers: [{ server: atlas as any }], + metadata: { count: 5, nextCursor: 'cursor-2' }, + }, + { + servers: [{ server: workspaceFs as any }], + metadata: { count: 5, nextCursor: 'cursor-3' }, + }, + ]; + + const fetchFn = mockFetchForResponses([ + ...firstPassPages, + ...secondPassPages, + ...secondPassPages, + ]); + const connection = createMockConnection(); + const logger = createMockLogger(); + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ + maxEntries: 3, + pageSize: 1, + pageLimit: 10, + defaultOwner: 'default-owner', + }), + logger, + { fetchApi: fetchFn }, + ); + await provider.connect(connection); + + await provider.run(); + expect(connection.applyMutation).toHaveBeenCalledTimes(1); + const firstMutation = (connection.applyMutation as jest.Mock).mock + .calls[0][0]; + const firstNames = firstMutation.entities.map( + (d: { entity: { metadata: { annotations?: Record } } }) => + d.entity.metadata.annotations?.['modelcontextprotocol.io/name'], + ); + expect(firstNames).toEqual([ + atlas.name, + workspaceFs.name, + diceWeather.name, + ]); + expect( + firstMutation.entities.every( + (d: { + entity: { metadata: { annotations?: Record } }; + }) => + d.entity.metadata.annotations?.[ + 'redhat.com/rhdh-mcp-registry-sync-status' + ] === 'ok', + ), + ).toBe(true); + + await provider.run(); + expect(connection.applyMutation).toHaveBeenCalledTimes(2); + const secondMutation = (connection.applyMutation as jest.Mock).mock + .calls[1][0]; + const secondByName = new Map( + secondMutation.entities.map( + (d: { + entity: { metadata: { annotations?: Record } }; + }) => [ + d.entity.metadata.annotations?.['modelcontextprotocol.io/name'], + d.entity.metadata.annotations?.[ + 'redhat.com/rhdh-mcp-registry-sync-status' + ], + ], + ), + ); + + expect(secondByName.get(inserted.name)).toBe('ok'); + expect(secondByName.get(atlas.name)).toBe('ok'); + expect(secondByName.get(workspaceFs.name)).toBe('ok'); + expect(secondByName.get(diceWeather.name)).toBe('degraded'); + expect(secondByName.has(remoteWorkspace.name)).toBe(false); + expect(secondMutation.entities).toHaveLength(4); + expect(logger.warn).toHaveBeenCalledWith( + expect.stringContaining('degraded entries'), + ); + expect(logger.warn).toHaveBeenCalledWith( + expect.stringContaining(diceWeather.name), + ); + // Third sync: still outside the window — degraded must be re-added. + await provider.run(); expect(connection.applyMutation).toHaveBeenCalledTimes(3); const thirdMutation = (connection.applyMutation as jest.Mock).mock .calls[2][0]; - expect(thirdMutation.entities).toHaveLength(0); + const thirdByName = new Map( + thirdMutation.entities.map( + (d: { + entity: { metadata: { annotations?: Record } }; + }) => [ + d.entity.metadata.annotations?.['modelcontextprotocol.io/name'], + d.entity.metadata.annotations?.[ + 'redhat.com/rhdh-mcp-registry-sync-status' + ], + ], + ), + ); + expect(thirdMutation.entities).toHaveLength(4); + expect(thirdByName.get(diceWeather.name)).toBe('degraded'); + }); + + it('retains last-good as degraded on formatting/mapping failure while maxEntries soft-stop is active', async () => { + const seedDocs = JSON.parse( + readFileSync( + resolve( + __dirname, + '../../../examples/mcp-registry/seed-data/seed.json', + ), + 'utf8', + ), + ) as Array<{ name: string; version: string }>; + + const [atlas, workspaceFs, diceWeather, remoteWorkspace] = seedDocs; + + const firstPassPages: McpRegistryListResponse[] = [ + { + servers: [{ server: atlas as any }], + metadata: { count: 4, nextCursor: 'cursor-1' }, + }, + { + servers: [{ server: workspaceFs as any }], + metadata: { count: 4, nextCursor: 'cursor-2' }, + }, + { + servers: [{ server: diceWeather as any }], + metadata: { count: 4, nextCursor: 'cursor-3' }, + }, + { + servers: [{ server: remoteWorkspace as any }], + metadata: { count: 4, nextCursor: 'cursor-4' }, + }, + ]; + + // Soft-stop window still covers atlas + workspace-fs + dice-weather, + // but dice-weather's payload is now malformed so mapping fails. + const malformedDice = { + $schema: + 'https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json', + name: diceWeather.name, + description: '', + version: diceWeather.version, + }; + const secondPassPages: McpRegistryListResponse[] = [ + { + servers: [{ server: atlas as any }], + metadata: { count: 4, nextCursor: 'cursor-1' }, + }, + { + servers: [{ server: workspaceFs as any }], + metadata: { count: 4, nextCursor: 'cursor-2' }, + }, + { + servers: [{ server: malformedDice as any }], + metadata: { count: 4, nextCursor: 'cursor-3' }, + }, + ]; + + const fetchFn = mockFetchForResponses([ + ...firstPassPages, + ...secondPassPages, + ]); + const connection = createMockConnection(); + const provider = new McpRegistryEntityProvider( + createDefaultConfig({ + maxEntries: 3, + pageSize: 1, + pageLimit: 10, + }), + createMockLogger(), + { fetchApi: fetchFn }, + ); + await provider.connect(connection); + + await provider.run(); + await provider.run(); + + const secondMutation = (connection.applyMutation as jest.Mock).mock + .calls[1][0]; + const diceEntity = secondMutation.entities.find( + (d: { entity: { metadata: { annotations?: Record } } }) => + d.entity.metadata.annotations?.['modelcontextprotocol.io/name'] === + diceWeather.name, + ); + expect(diceEntity).toBeDefined(); + expect( + diceEntity.entity.metadata.annotations?.[ + 'redhat.com/rhdh-mcp-registry-sync-status' + ], + ).toBe('degraded'); }); }); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index 9ea5737dc29..e505acf4464 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -156,16 +156,21 @@ export class McpRegistryEntityProvider implements EntityProvider { return; } - const { entities, hasDegradedEntries } = this.mapRegistryEntries( - entries, + const { entities: mappedEntities, hasDegradedEntries: mappingDegraded } = + this.mapRegistryEntries(entries, managedByLocation); + + const { entities, hasDegradedEntries } = this.appendSoftStopRetained( + mappedEntities, + mappingDegraded, managedByLocation, ); if (hasDegradedEntries) { this.logger.warn( `MCP Registry sync completed with degraded entries. ` + - `Some server entries could not be mapped and are using ` + - `last-good entities.`, + `Some previously synced servers could not be refreshed ` + + `(mapping/formatting failure or maxEntries soft-stop window) ` + + `and are using last-good entities.`, ); } @@ -319,6 +324,60 @@ export class McpRegistryEntityProvider implements EntityProvider { return { entities, hasDegradedEntries }; } + /** + * When a `maxEntries` soft-stop is active (`endCursor` set), retain + * last-good entities that fell outside the truncated window so a full + * mutation does not prune them. Mark retained copies as degraded. + * + * Full traversals without an end bound continue to prune servers that + * are absent from the registry. + */ + private appendSoftStopRetained( + mappedEntities: DeferredEntity[], + hasDegradedEntries: boolean, + managedByLocation: string, + ): { entities: DeferredEntity[]; hasDegradedEntries: boolean } { + if (this.endCursor === undefined || this.lastGoodIndex.size === 0) { + return { entities: mappedEntities, hasDegradedEntries }; + } + + const seenKeys = new Set(); + for (const deferred of mappedEntities) { + const annotations = deferred.entity.metadata?.annotations; + const name = annotations?.['modelcontextprotocol.io/name']; + const version = annotations?.['modelcontextprotocol.io/version']; + if (name && version) { + seenKeys.add(buildLastGoodKey(name, version)); + } + } + + const entities = [...mappedEntities]; + let degraded = hasDegradedEntries; + + for (const [key, lastGood] of this.lastGoodIndex) { + if (seenKeys.has(key)) { + continue; + } + const retainedEntity = structuredClone(lastGood.entity); + this.applyProviderAnnotations( + retainedEntity, + managedByLocation, + 'degraded', + ); + entities.push({ + entity: retainedEntity, + locationKey: PROVIDER_NAME, + }); + degraded = true; + this.logger.warn( + `Retaining last-good entity for "${key}" with degraded sync ` + + `status; it fell outside the maxEntries soft-stop window.`, + ); + } + + return { entities, hasDegradedEntries: degraded }; + } + /** * Map one registry entry into a deferred entity with sync status `ok`. */ @@ -435,12 +494,13 @@ export class McpRegistryEntityProvider implements EntityProvider { } /** - * Rebuild the last-good index from successfully mapped entities only. + * Rebuild the last-good index from every committed entity that has a + * registry identity (`ok` and `degraded` alike). * - * Entities that carry sync-status "degraded" are excluded: they are - * last-good fallbacks from a prior cycle, so storing them back would - * create perpetual retention of stale data. Only "ok" entities - * qualify as last-good candidates. + * Degraded entries stay indexed so they can be re-added on later syncs + * until the server is refreshed successfully (`ok`) or omitted from the + * mutation entirely (true prune after a full traversal without soft-stop + * retention). * * The annotation keys used here ('modelcontextprotocol.io/name' and * 'modelcontextprotocol.io/version') are set by mapServerToEntity in @@ -454,9 +514,6 @@ export class McpRegistryEntityProvider implements EntityProvider { this.lastGoodIndex.clear(); for (const deferred of entities) { const annotations = deferred.entity.metadata?.annotations; - if (annotations?.[SYNC_STATUS_ANNOTATION] !== 'ok') { - continue; - } const name = annotations?.['modelcontextprotocol.io/name']; const version = annotations?.['modelcontextprotocol.io/version']; if (name && version) { From e61d2e082fdd02ce462dec5e398651672b9756b0 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 21:16:13 -0400 Subject: [PATCH 57/63] chore(#4815): add secret value to seed.json entry and server.json for testing secret redaction Signed-off-by: Michael Valdron --- .../ai-integrations/examples/mcp-registry/seed-data/seed.json | 1 + .../examples/mcp-registry/server-json/npm.server.json | 1 + 2 files changed, 2 insertions(+) diff --git a/workspaces/ai-integrations/examples/mcp-registry/seed-data/seed.json b/workspaces/ai-integrations/examples/mcp-registry/seed-data/seed.json index 9a66728c819..d91be2e8401 100644 --- a/workspaces/ai-integrations/examples/mcp-registry/seed-data/seed.json +++ b/workspaces/ai-integrations/examples/mcp-registry/seed-data/seed.json @@ -23,6 +23,7 @@ { "name": "ATLAS_SEARCH_API_KEY", "description": "Atlas Search API key", + "value": "1234567890", "isRequired": true, "isSecret": true } diff --git a/workspaces/ai-integrations/examples/mcp-registry/server-json/npm.server.json b/workspaces/ai-integrations/examples/mcp-registry/server-json/npm.server.json index 785ca30d42d..02aeed9948f 100644 --- a/workspaces/ai-integrations/examples/mcp-registry/server-json/npm.server.json +++ b/workspaces/ai-integrations/examples/mcp-registry/server-json/npm.server.json @@ -22,6 +22,7 @@ { "name": "ATLAS_SEARCH_API_KEY", "description": "Atlas Search API key", + "value": "1234567890", "isRequired": true, "isSecret": true } From 5dbd7ca63782da794fa21968b86c77bfe0bf0ff8 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 21:24:47 -0400 Subject: [PATCH 58/63] chore(#4815): regenerate MCP Registry provider API report Sort McpRegistryProviderConfig members to match API Extractor output. Assisted-by: grok-4.6 Co-authored-by: Cursor Signed-off-by: Michael Valdron --- .../catalog-backend-module-mcp-registry-provider/report.api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md index e689b674433..dfeabf340a0 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md @@ -37,8 +37,8 @@ export interface McpRegistryProviderConfig { apiVersion?: string; baseName?: string; baseUrl: string; - defaultOwner?: string; defaultLifecycle?: string; + defaultOwner?: string; hostAllowList?: string[]; maxEntries?: number; pageLimit?: number; From 6cf2ecc9d115a2eb8232cef6f013e5822070233e Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 21:34:53 -0400 Subject: [PATCH 59/63] fix(#4815): address remaining PR #4871 review feedback Align assertSingleRegistryConfig error keys with KNOWN_MCP_REGISTRY_KEYS, use the short Backstage moduleId, document internal test seams, cover the startCursor===endCursor soft-stop edge case, and note pagination reset before applyMutation. Assisted-by: grok-4.6 Co-authored-by: Cursor Signed-off-by: Michael Valdron --- .../src/McpRegistryEntityProvider.ts | 10 ++++++++- .../src/client.test.ts | 22 +++++++++++++++++++ .../src/config.test.ts | 3 +++ .../src/config.ts | 4 ++-- .../src/module.ts | 2 +- 5 files changed, 37 insertions(+), 4 deletions(-) diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index e505acf4464..a29addc93fc 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -60,7 +60,10 @@ const SYNC_STATUS_ANNOTATION = 'redhat.com/rhdh-mcp-registry-sync-status'; * @public */ export interface McpRegistryEntityProviderOptions { - /** @internal Override the global `fetch` implementation (test seam). */ + /** + * @internal Override the global `fetch` implementation (test seam). + * Kept non-private so tests can inject it; stripped from published types. + */ fetchApi?: typeof fetch; /** Scheduler task runner used to periodically invoke sync. */ taskRunner?: SchedulerServiceTaskRunner; @@ -139,6 +142,9 @@ export class McpRegistryEntityProvider implements EntityProvider { * Run one sync cycle: fetch servers from the registry, map them, * and commit a full mutation. * + * Intentionally not TypeScript-`private` so unit tests can invoke it + * directly; `@internal` keeps it out of the published API surface. + * * @internal */ async run(): Promise { @@ -264,6 +270,8 @@ export class McpRegistryEntityProvider implements EntityProvider { ); } + // Clear pagination state before returning so applyMutation failures + // start a fresh traversal on the next sync (intended, covered by tests). const entries = this.pendingEntries; this.pendingEntries = []; this.resumeCursor = undefined; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts index 2b67963a698..48fbdafdb45 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts @@ -632,6 +632,28 @@ describe('fetchRegistryServers', () => { expect(result.resumeCursor).toBeUndefined(); expect(fn).toHaveBeenCalledTimes(1); }); + + it('returns zero servers when startCursor already matches endCursor', async () => { + const page1: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 1 }, + }; + const fn = mockFetch([{ body: page1 }]); + + const result = await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v1', + pageLimit: 10, + startCursor: 'cursor-end', + endCursor: 'cursor-end', + fetchApi: fn, + }); + + expect(result.servers).toHaveLength(0); + expect(result.resumeCursor).toBeUndefined(); + expect(result.endCursor).toBeUndefined(); + expect(fn).not.toHaveBeenCalled(); + }); }); describe('parseServersEndpointUrl', () => { diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts index 53bef09417b..8300eeb423a 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts @@ -370,6 +370,9 @@ describe('assertSingleRegistryConfig', () => { expect(() => assertSingleRegistryConfig(config)).toThrow( /found keyed instance/, ); + expect(() => assertSingleRegistryConfig(config)).toThrow( + /maxEntries, remotesOnly, hostAllowList/, + ); }); }); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts index 3f10ac857ce..b9596682150 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts @@ -101,11 +101,11 @@ export function assertSingleRegistryConfig(registryConfig: Config): void { continue; } if (nested && nested.keys().length > 0) { + const knownKeys = [...KNOWN_MCP_REGISTRY_KEYS].join(', '); throw new Error( `Invalid ${MCP_REGISTRY_INSTANCE_CONFIG_PATH} configuration: found ` + `keyed instance "${key}". Configure a single registry object ` + - `with baseUrl, baseName, apiVersion, schedule, pageLimit, ` + - `pageSize, defaultOwner, and defaultLifecycle.`, + `with known keys: ${knownKeys}.`, ); } } diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts index d1558f79b82..78b9efcdbb1 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/module.ts @@ -33,7 +33,7 @@ import { McpRegistryEntityProvider } from './McpRegistryEntityProvider'; */ export const catalogModuleMcpRegistryProvider = createBackendModule({ pluginId: 'catalog', - moduleId: 'catalog-backend-module-mcp-registry-provider', + moduleId: 'mcp-registry-provider', register(env) { env.registerInit({ deps: { From f406a6d5edd8d740fb091756565b489ed8d18238 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 21:40:19 -0400 Subject: [PATCH 60/63] chore(#4815): restore mcp-registry openspec docs to main Drop branch-local edits under openspec/changes for mcp-registry-provider and mcp-registry-server-mapping so this PR no longer changes those specs. Assisted-by: grok-4.6 Co-authored-by: Cursor Signed-off-by: Michael Valdron --- .../changes/mcp-registry-provider/audit.md | 2 +- .../changes/mcp-registry-provider/design.md | 2 +- .../specs/mcp-registry-provider/spec.md | 2 +- .../changes/mcp-registry-provider/tasks.md | 52 +++++++++---------- .../mcp-registry-server-mapping/design.md | 10 ++-- .../mcp-registry-server-mapping/proposal.md | 2 +- 6 files changed, 35 insertions(+), 35 deletions(-) diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/audit.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/audit.md index 5d644474ce5..d0d369eb72b 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/audit.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/audit.md @@ -1,6 +1,6 @@ ## Audit Report: mcp-registry-provider -**Last audited:** 2026-09-19T00:00:00Z +**Last audited:** 2026-09-15T18:03:43Z ### Summary diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/design.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/design.md index e6782496584..e706aa4c59d 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/design.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/design.md @@ -105,7 +105,7 @@ Pagination is cursor-based: omit `cursor` on the first request; pass the prior ` **Choice:** Two failure tiers: (a) a single accumulated server entry that the mapping rejects (for example a missing required `server.json` field) is logged with an identifying message; the run proceeds. For that entry, if a **last-good** entity from a prior successful sync exists for the same registry identity, the provider SHALL include that entity in the full mutation with mapping-owned fields unchanged (D5) but with `redhat.com/rhdh-mcp-registry-sync-status: degraded` (D8) so operators can see the entry is stale relative to the latest registry `server.json`. Last-good lookup keys entries by `server.json` `name` and `version` when both are present (matching the mapping's canonical identity annotations `modelcontextprotocol.io/name` and `modelcontextprotocol.io/version` on the prior entity). When `name` or `version` is absent, or no prior entity exists, the entry contributes no entity to the mutation (first-time failure or uncorrelatable entry). (b) A registry-level error (unreachable, non-2xx, unparseable body, or pagination-safeguard trip) fails the whole run: **no** `applyMutation` is emitted, so the last-good catalog state is preserved, and the next scheduled tick retries. -The provider maintains an in-memory last-good index that is rebuilt at the end of each successful sync from the entities committed in the mutation. Only entities with `redhat.com/rhdh-mcp-registry-sync-status: ok` are stored in the index; degraded entries are excluded to prevent perpetual retention of stale data. On restart, the index starts empty and is populated after the first successful sync. +At the start of each sync, the provider loads existing provider-managed entities (via `locationKey` `mcp-registry-provider`) into an index for last-good retention. **Alternatives considered:** Omit failed entries from the full mutation — rejected; a server still present in the registry would be pruned from the catalog. Commit whatever was fetched before a pagination error — rejected; a partial full mutation prunes entities that still exist, causing catalog flapping. diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/specs/mcp-registry-provider/spec.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/specs/mcp-registry-provider/spec.md index 1e3b1a64668..032558c9b8a 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/specs/mcp-registry-provider/spec.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/specs/mcp-registry-provider/spec.md @@ -205,7 +205,7 @@ At the end of each successful sync, the provider SHALL commit to the catalog as ### Requirement: Resilient, agent-native error handling -The provider SHALL maintain an in-memory last-good index keyed by `metadata.annotations['modelcontextprotocol.io/name']` and `metadata.annotations['modelcontextprotocol.io/version']`. The index is populated from successfully committed entities at the end of each sync run and is available for lookup during subsequent sync runs within the same process lifetime. The index does not persist across provider restarts; on the first sync after a restart, no last-good entries are available for retention. +At the start of each sync run, before mapping accumulated servers, the provider SHALL load all provider-managed catalog entities (mutation `locationKey` `mcp-registry-provider`) into a last-good index keyed by `metadata.annotations['modelcontextprotocol.io/name']` and `metadata.annotations['modelcontextprotocol.io/version']`. A single accumulated server entry that cannot be mapped (e.g. it omits a `server.json`-required field and the mapping rejects it) SHALL be logged with an actionable message identifying the entry and SHALL NOT abort the sync. When that entry's `server.json` includes both `name` and `version`, the provider SHALL look up a last-good entity in that index keyed by that `name` and `version` and SHALL include it unchanged in the full mutation when found. When `name` or `version` is absent, or no last-good entity exists, the entry contributes no entity to the mutation. A registry transport or protocol error (unreachable host, non-2xx HTTP status, unparseable response body, or pagination-safeguard trip) SHALL fail the current sync run: the provider SHALL NOT commit a mutation, SHALL log the error, and SHALL retry on the next scheduled tick, leaving the prior catalog state intact. diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/tasks.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/tasks.md index c77f663d4ca..a95423408d6 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/tasks.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-provider/tasks.md @@ -8,44 +8,44 @@ ## 1. Plugin Scaffolding & Packaging -- [x] 1.1 Create the backend `catalog-backend-module` plugin package (Backstage catalog-backend-module naming convention) with `package.json`, `tsconfig`, and lint config matching the workspace's plugin conventions -- [x] 1.2 Add the `createBackendModule` skeleton that registers against `catalogProcessingExtensionPoint`, depending on `coreServices` (`rootConfig`, `logger`, `scheduler`) -- [x] 1.3 Document installation in the plugin `README.md` (add via `backend.add(...)`, minimal app-config example) +- [ ] 1.1 Create the backend `catalog-backend-module` plugin package (Backstage catalog-backend-module naming convention) with `package.json`, `tsconfig`, and lint config matching the workspace's plugin conventions +- [ ] 1.2 Add the `createBackendModule` skeleton that registers against `catalogProcessingExtensionPoint`, depending on `coreServices` (`rootConfig`, `logger`, `scheduler`) +- [ ] 1.3 Document installation in the plugin `README.md` (add via `backend.add(...)`, minimal app-config example) ## 2. Configuration -- [x] 2.1 Author `config.d.ts` declaring `catalog.providers.mcpRegistry` as a single object with `baseUrl` (required), `baseName?` (optional mapping-prefix override), `apiVersion?` (default `v1`), `schedule?` (`SchedulerServiceTaskScheduleDefinitionConfig` in config.d.ts; runtime scheduling uses `SchedulerServiceTaskScheduleDefinition` per design D3), `pageLimit?` (max pages per sync, default `10`), `pageSize?` (registry `?limit=` when set), and `defaultOwner?`; require `@visibility backend` annotations for backend-only fields such as `baseUrl` -- [x] 2.2 Implement config reading: parse `catalog.providers.mcpRegistry` as a single object; register nothing (no error) when the key is absent -- [x] 2.3 Implement validation with actionable errors (fail fast when `baseUrl` is missing; fail fast with a multiple-registries-out-of-scope message when the value is a keyed map of instance objects); apply the `apiVersion` default (`v1`), the default schedule when `schedule` is omitted (`frequency: { minutes: 30 }`, `timeout: { minutes: 3 }`, no `initialDelay` — same as spec requirement “Sync on the configured schedule”), and the `pageLimit` default (`10` pages per sync) when omitted -- [x] 2.4 Add unit tests for config parsing/validation: single object with `baseUrl`, optional `baseName`, keyed-map rejection, missing `baseUrl`, absent-config no-op, omitted `pageLimit` → `10`, explicit `pageLimit` override, omitted `pageSize` (no invented default), and explicit `pageSize` +- [ ] 2.1 Author `config.d.ts` declaring `catalog.providers.mcpRegistry` as a single object with `baseUrl` (required), `baseName?` (optional mapping-prefix override), `apiVersion?` (default `v1`), `schedule?` (`SchedulerServiceTaskScheduleDefinitionConfig` in config.d.ts; runtime scheduling uses `SchedulerServiceTaskScheduleDefinition` per design D3), `pageLimit?` (max pages per sync, default `10`), `pageSize?` (registry `?limit=` when set), and `defaultOwner?`; require `@visibility backend` annotations for backend-only fields such as `baseUrl` +- [ ] 2.2 Implement config reading: parse `catalog.providers.mcpRegistry` as a single object; register nothing (no error) when the key is absent +- [ ] 2.3 Implement validation with actionable errors (fail fast when `baseUrl` is missing; fail fast with a multiple-registries-out-of-scope message when the value is a keyed map of instance objects); apply the `apiVersion` default (`v1`), the default schedule when `schedule` is omitted (`frequency: { minutes: 30 }`, `timeout: { minutes: 3 }`, no `initialDelay` — same as spec requirement “Sync on the configured schedule”), and the `pageLimit` default (`10` pages per sync) when omitted +- [ ] 2.4 Add unit tests for config parsing/validation: single object with `baseUrl`, optional `baseName`, keyed-map rejection, missing `baseUrl`, absent-config no-op, omitted `pageLimit` → `10`, explicit `pageLimit` override, omitted `pageSize` (no invented default), and explicit `pageSize` ## 3. Registry Client & Pagination -- [x] 3.1 Define the registry API response types (`servers[]`, `metadata.count`, `metadata.nextCursor`) and the `server.json` extraction from each `servers[]` entry (`.server`) -- [x] 3.2 Implement servers-endpoint URL construction `//servers` with slash normalization (works with and without a trailing slash on `baseUrl`) -- [x] 3.3 Implement cursor pagination: loop passing prior `metadata.nextCursor` as the `cursor` query param until it is absent, null, or empty, accumulating all `servers[]`; treat cursors as opaque; when `pageSize` is set send it as `?limit=` on every list request; when `pageSize` is omitted leave `?limit=` unset -- [x] 3.4 Implement the pagination loop safeguard: cap fetches at configured `pageLimit` pages per sync (`10` when omitted); do not send `pageLimit` as the registry `?limit=` query param; detect a repeated cursor; exceeding the page cap or a repeated cursor fails the run rather than looping forever -- [x] 3.5 Implement registry-error handling (unreachable host, non-2xx status, unparseable body, pagination-safeguard trip) raising a typed error that aborts the run -- [x] 3.6 Add unit tests for the client using mocked HTTP: single page, multi-page traversal, empty/absent/null cursor termination, opaque-cursor passthrough, omitted `pageSize` (no `limit` query), configured `pageSize` as `limit` on every page request, default `pageLimit` `10` tripping on an 11th page, configured `pageLimit` tripping, and error/safeguard cases +- [ ] 3.1 Define the registry API response types (`servers[]`, `metadata.count`, `metadata.nextCursor`) and the `server.json` extraction from each `servers[]` entry (`.server`) +- [ ] 3.2 Implement servers-endpoint URL construction `//servers` with slash normalization (works with and without a trailing slash on `baseUrl`) +- [ ] 3.3 Implement cursor pagination: loop passing prior `metadata.nextCursor` as the `cursor` query param until it is absent, null, or empty, accumulating all `servers[]`; treat cursors as opaque; when `pageSize` is set send it as `?limit=` on every list request; when `pageSize` is omitted leave `?limit=` unset +- [ ] 3.4 Implement the pagination loop safeguard: cap fetches at configured `pageLimit` pages per sync (`10` when omitted); do not send `pageLimit` as the registry `?limit=` query param; detect a repeated cursor; exceeding the page cap or a repeated cursor fails the run rather than looping forever +- [ ] 3.5 Implement registry-error handling (unreachable host, non-2xx status, unparseable body, pagination-safeguard trip) raising a typed error that aborts the run +- [ ] 3.6 Add unit tests for the client using mocked HTTP: single page, multi-page traversal, empty/absent/null cursor termination, opaque-cursor passthrough, omitted `pageSize` (no `limit` query), configured `pageSize` as `limit` on every page request, default `pageLimit` `10` tripping on an 11th page, configured `pageLimit` tripping, and error/safeguard cases ## 4. Entity Provider & Scheduling -- [x] 4.1 Implement the `EntityProvider` class: `getProviderName()` = `mcp-registry-provider`, `connect()` storing the connection, and a `run()` performing one sync; maintain an in-memory last-good index keyed by `modelcontextprotocol.io/name` and `modelcontextprotocol.io/version`, rebuilt at the end of each successful sync from committed entities with `sync-status: ok` only (design D6) -- [x] 4.2 Wire scheduling via `SchedulerService.createScheduledTaskRunner(schedule)` only (no synchronous `run()` from `connect()`); register the single provider when config is present -- [x] 4.3 Implement the full-mutation commit: on successful sync call `connection.applyMutation({ type: 'full', entities })`; on a failed run emit no mutation (preserve prior catalog state) -- [x] 4.4 Attach provider attribution and sync status to each entity: set mutation `locationKey` `mcp-registry-provider`, `backstage.io/managed-by-location` to `url:` + normalized `baseUrl` (trailing `/` stripped), and `redhat.com/rhdh-mcp-registry-sync-status` to `ok` or `degraded` per D8 -- [x] 4.5 Add unit tests: full mutation contents (`locationKey`, `backstage.io/managed-by-location: url:`, `redhat.com/rhdh-mcp-registry-sync-status` `ok`/`degraded` per scenario), pruning across syncs, updated server reflected, last-good retention with `degraded` when mapping fails, and no-mutation-on-failed-run +- [ ] 4.1 Implement the `EntityProvider` class: `getProviderName()` = `mcp-registry-provider`, `connect()` storing the connection, and a `run()` performing one sync; at the start of each `run()`, load provider-managed entities (`locationKey` `mcp-registry-provider`) into a last-good index keyed by `modelcontextprotocol.io/name` and `modelcontextprotocol.io/version` before mapping (design D6) +- [ ] 4.2 Wire scheduling via `SchedulerService.createScheduledTaskRunner(schedule)` only (no synchronous `run()` from `connect()`); register the single provider when config is present +- [ ] 4.3 Implement the full-mutation commit: on successful sync call `connection.applyMutation({ type: 'full', entities })`; on a failed run emit no mutation (preserve prior catalog state) +- [ ] 4.4 Attach provider attribution and sync status to each entity: set mutation `locationKey` `mcp-registry-provider`, `backstage.io/managed-by-location` to `url:` + normalized `baseUrl` (trailing `/` stripped), and `redhat.com/rhdh-mcp-registry-sync-status` to `ok` or `degraded` per D8 +- [ ] 4.5 Add unit tests: full mutation contents (`locationKey`, `backstage.io/managed-by-location: url:`, `redhat.com/rhdh-mcp-registry-sync-status` `ok`/`degraded` per scenario), pruning across syncs, updated server reflected, last-good retention with `degraded` when mapping fails, and no-mutation-on-failed-run ## 5. Mapping Integration -- [x] 5.1 Depend on the sibling `mcp-registry-server-mapping` transform and invoke it per accumulated server, passing `defaultOwner` as the caller-override owner default and, when configured, `baseName` as the caller-override identity prefix (never reimplement the mapping) -- [x] 5.2 Implement per-entry failure isolation: on mapping rejection, log an actionable message; when `server.json` has `name` and `version`, include the last-good provider-managed entity (from the in-memory index populated at end of prior sync) with mapping-owned fields unchanged and `redhat.com/rhdh-mcp-registry-sync-status: degraded`; on success set `ok`; otherwise omit the entry; continue the run -- [x] 5.3 Add integration tests over sample `server.json` inputs → produced `mcp-server` `API` entities, asserting `spec.owner` reflects `defaultOwner` (and the mapping default `unknown` when omitted), `metadata.name` uses `baseName` as prefix when configured (and mapping default `mcp.registry` when omitted), that one bad entry does not abort the batch, that a mapping failure on a previously synced server retains the last-good entity with `redhat.com/rhdh-mcp-registry-sync-status: degraded`, and that successful mappings set `ok` +- [ ] 5.1 Depend on the sibling `mcp-registry-server-mapping` transform and invoke it per accumulated server, passing `defaultOwner` as the caller-override owner default and, when configured, `baseName` as the caller-override identity prefix (never reimplement the mapping) +- [ ] 5.2 Implement per-entry failure isolation: on mapping rejection, log an actionable message; when `server.json` has `name` and `version`, include the last-good provider-managed entity (indexed at sync start) with mapping-owned fields unchanged and `redhat.com/rhdh-mcp-registry-sync-status: degraded`; on success set `ok`; otherwise omit the entry; continue the run +- [ ] 5.3 Add integration tests over sample `server.json` inputs → produced `mcp-server` `API` entities, asserting `spec.owner` reflects `defaultOwner` (and the mapping default `unknown` when omitted), `metadata.name` uses `baseName` as prefix when configured (and mapping default `mcp.registry` when omitted), that one bad entry does not abort the batch, that a mapping failure on a previously synced server retains the last-good entity with `redhat.com/rhdh-mcp-registry-sync-status: degraded`, and that successful mappings set `ok` ## 6. End-to-End Verification & Docs -- [x] 6.1 Add an end-to-end test wiring config → mocked paginated registry → mapping → full mutation, asserting the mutation converges to the registry's current server set -- [x] 6.2 Verify produced entities pass the upstream `mcp-server` `API` entity schema (`McpServerApiEntity`) — reusing the mapping change's conformance expectations -- [x] 6.3 Verify the apiVersion discrepancy handling: default `v1` requests `/v1/servers` and an override (`v0`) is honored, with a documented note for operators -- [x] 6.4 Finalize `README.md` / config docs: full `catalog.providers.mcpRegistry` example (`baseUrl`, optional `baseName`, `apiVersion`, optional `schedule` — default `frequency: { minutes: 30 }`, `timeout: { minutes: 3 }`, no `initialDelay`; first sync after one `frequency` unless `initialDelay` is set), `pageLimit` default `10` pages per sync, optional `pageSize` as `?limit=`, `defaultOwner`), note that multiple registries are out of scope, pagination behavior, and error-handling semantics -- [x] 6.5 Run the workspace lint, typecheck, and test suite; ensure the new package builds and passes CI conventions +- [ ] 6.1 Add an end-to-end test wiring config → mocked paginated registry → mapping → full mutation, asserting the mutation converges to the registry's current server set +- [ ] 6.2 Verify produced entities pass the upstream `mcp-server` `API` entity schema (`McpServerApiEntity`) — reusing the mapping change's conformance expectations +- [ ] 6.3 Verify the apiVersion discrepancy handling: default `v1` requests `/v1/servers` and an override (`v0`) is honored, with a documented note for operators +- [ ] 6.4 Finalize `README.md` / config docs: full `catalog.providers.mcpRegistry` example (`baseUrl`, optional `baseName`, `apiVersion`, optional `schedule` — default `frequency: { minutes: 30 }`, `timeout: { minutes: 3 }`, no `initialDelay`; first sync after one `frequency` unless `initialDelay` is set), `pageLimit` default `10` pages per sync, optional `pageSize` as `?limit=`, `defaultOwner`), note that multiple registries are out of scope, pagination behavior, and error-handling semantics +- [ ] 6.5 Run the workspace lint, typecheck, and test suite; ensure the new package builds and passes CI conventions diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/design.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/design.md index b2c72c374d0..58365914871 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/design.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/design.md @@ -70,7 +70,7 @@ Implementation tasks produce a version-pinned `mapping-reference.md` under `open **Non-Goals:** -- Registry HTTP client, polling, scheduling, or an entity provider/processor (implemented by `catalog-backend-module-mcp-registry-provider`). +- Registry HTTP client, polling, scheduling, or an entity provider/processor (separate future change). - Modifying the upstream `mcp-server` entity contract or its validation. - Reverse mapping (entity → `server.json`) beyond the scalar round-trip guarantee. - Executing or health-checking mapped servers, or interpreting local `packages[]` runtime details. @@ -110,11 +110,11 @@ Implementation tasks produce a version-pinned `mapping-reference.md` under `open **Alternatives considered:** (a) Encode the version in `metadata.namespace` — rejected; fragments entity references and complicates relationships. (b) `__` with no prefix — rejected; leaves registry-mapped entities without a caller-controllable namespacing token in `metadata.name` (they would collide with any other `mcp-server` API that sanitizes to the same name+version). -**Rationale:** A registry publishes one `server.json` per version and each becomes its own entity, so a name derived from the canonical name alone would collide across versions. The prefix distinguishes registry-mapped entities in a shared catalog and lets the ingestion layer (`catalog-backend-module-mcp-registry-provider`) pass a per-source override without changing the transform. +**Rationale:** A registry publishes one `server.json` per version and each becomes its own entity, so a name derived from the canonical name alone would collide across versions. The prefix distinguishes registry-mapped entities in a shared catalog and lets the future ingestion layer pass a per-source override without changing the transform. ### D5: Supplying fields absent from `server.json` — owner and lifecycle -**Choice:** `spec.owner` is set to the constant `unknown` by default; a caller MAY supply an override default, but the transform never fails for a missing owner (a placeholder owner keeps the output valid, and the ingestion layer can reassign ownership). `spec.lifecycle` is set to the constant `production` by default; a caller MAY supply an override default lifecycle value. Both fields use the same caller-override pattern as the identity prefix in D4. +**Choice:** `spec.owner` is set to the constant `unknown` by default; a caller MAY supply an override default, but the transform never fails for a missing owner (a placeholder owner keeps the output valid, and the future ingestion change can reassign ownership). `spec.lifecycle` is set to the constant `production` by default; a caller MAY supply an override default lifecycle value. Both fields use the same caller-override pattern as the identity prefix in D4. **Alternatives considered:** (a) Require caller-provided owner/lifecycle and fail if absent — rejected; a pure transform should always yield a valid entity, and ownership/lifecycle assignment belongs to the ingestion layer. (b) Derive lifecycle from a `status` field — rejected; `status` is not part of the base `server.schema.json` (verified 2026-08-21 against the draft schema). @@ -201,7 +201,7 @@ The scheme gate does **not** classify hosts as public vs private and does not tr ## Risks / Trade-offs - **63-char truncation collisions** → Deterministic hash suffix on truncation and on sanitization collisions keeps keys unique; the hash is derived from the full source path so it is stable across runs. -- **`metadata.name` collisions across registries** (same name+version from two registries under the default prefix) → Out of scope here (no dedup). The caller-overridable prefix is the ingestion-layer lever for per-source namespacing; documented so the ingestion layer can supply distinct prefixes or otherwise dedup. Within a single `(prefix, name, version)` the per-input hash-suffix rule (lossy sanitization or truncation) keeps that identity stable and distinct from a different unsanitized triple that happens to share a sanitized stem. +- **`metadata.name` collisions across registries** (same name+version from two registries under the default prefix) → Out of scope here (no dedup). The caller-overridable prefix is the ingestion-layer lever for per-source namespacing; documented so the future ingestion change can supply distinct prefixes or otherwise dedup. Within a single `(prefix, name, version)` the per-input hash-suffix rule (lossy sanitization or truncation) keeps that identity stable and distinct from a different unsanitized triple that happens to share a sanitized stem. - **Draft schema drift** → D7 fail-open projection; the mapping table is versioned against the draft and revisited when the schema changes. - **Lossy flattening of deep `packages[]` config** → Accepted; runtime package details are preserved as scalar-leaf annotations for discoverability, not interpreted. Round-trip fidelity is guaranteed only for scalar leaves. - **Secret leakage into searchable annotations** (remote `headers`/`variables`, `environmentVariables` carrying `default`/`value`/`choices`) → D9 prunes the `default`/`value`/`choices` leaves of any `isSecret: true` input from projection. This is a deliberate carve-out from scalar round-trip fidelity — those leaves are intentionally unrecoverable from the entity. Non-secret metadata on the same input still projects, so discoverability is preserved. @@ -211,7 +211,7 @@ The scheme gate does **not** classify hosts as public vs private and does not tr ## Migration Plan -Not applicable — new capabilities with no existing data or behavior to migrate. The mapping is additive and has no runtime deployment surface of its own; consumed by `catalog-backend-module-mcp-registry-provider`. +Not applicable — new capabilities with no existing data or behavior to migrate. The mapping is additive and has no runtime deployment surface of its own until a future ingestion change consumes it. ## Open Questions diff --git a/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/proposal.md b/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/proposal.md index 6352f223afb..27a6dde574b 100644 --- a/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/proposal.md +++ b/workspaces/ai-integrations/openspec/changes/mcp-registry-server-mapping/proposal.md @@ -42,5 +42,5 @@ _(none — introduces new capabilities only; consumes the upstream Backstage `mc - **Upstream target**: `McpServerApiEntity` shape (top-level `spec.remotes[]`, no `spec.definition`) — detailed anchors in `design.md`; requirements in `specs/mcp-registry-server-mapping/spec.md`. - **Source**: MCP Registry draft `server.json` (version-pinned in implementation); unknown fields fail-open via projection. -- **Consumers**: `catalog-backend-module-mcp-registry-provider` entity provider; catalog search over `modelcontextprotocol.io/*` annotations. +- **Consumers**: future registry entity provider; catalog search over `modelcontextprotocol.io/*` annotations. - **Alignment**: track Backstage RFC [#32062](https://github.com/backstage/backstage/issues/32062) and registry schema drift. From 356187495b9b4e3d44a7c6cad1ae852094b51454 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Sun, 20 Sep 2026 21:55:06 -0400 Subject: [PATCH 61/63] feat(#4815): add latestVersion MCP Registry list query option When enabled, list requests include ?version=latest so the registry returns only the latest version of each server; default remains unset. Assisted-by: grok-4.6 Co-authored-by: Cursor Signed-off-by: Michael Valdron --- .../mcp-registry-provider-plugin.md | 2 +- workspaces/ai-integrations/app-config.yaml | 2 + .../README.md | 3 ++ .../config.d.ts | 7 +++ .../report.api.md | 1 + .../src/McpRegistryEntityProvider.test.ts | 2 + .../src/McpRegistryEntityProvider.ts | 2 + .../src/client.test.ts | 54 ++++++++++++++++++- .../src/client.ts | 15 +++++- .../src/config.test.ts | 39 +++++++++++++- .../src/config.ts | 31 +++++++++++ 11 files changed, 153 insertions(+), 5 deletions(-) diff --git a/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md b/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md index e61d6d87ff7..2da585d1458 100644 --- a/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md +++ b/workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md @@ -2,4 +2,4 @@ '@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider': minor --- -Add MCP Registry provider backend module that ingests MCP servers from a configured registry into the catalog as mcp-server API entities. Supports cursor pagination with full-mutation semantics, per-entry failure isolation with last-good retention, nested config under `catalog.providers.mcpRegistry.mcpRegistry` (extra instance ids warn and are ignored), optional `defaultOwner` / `defaultLifecycle` / `remotesOnly` / `hostAllowList` / `maxEntries` soft-stop with `pageLimit` resume (re-adding last-good as degraded on later syncs until the server is refreshed successfully), schedule and page controls, identity prefix override, redirect handling with Location validation, and clearer fetch error reporting. +Add MCP Registry provider backend module that ingests MCP servers from a configured registry into the catalog as mcp-server API entities. Supports cursor pagination with full-mutation semantics, per-entry failure isolation with last-good retention, nested config under `catalog.providers.mcpRegistry.mcpRegistry` (extra instance ids warn and are ignored), optional `defaultOwner` / `defaultLifecycle` / `remotesOnly` / `latestVersion` (`?version=latest`) / `hostAllowList` / `maxEntries` soft-stop with `pageLimit` resume (re-adding last-good as degraded on later syncs until the server is refreshed successfully), schedule and page controls, identity prefix override, redirect handling with Location validation, and clearer fetch error reporting. diff --git a/workspaces/ai-integrations/app-config.yaml b/workspaces/ai-integrations/app-config.yaml index b0253021242..4f02fb9cf09 100644 --- a/workspaces/ai-integrations/app-config.yaml +++ b/workspaces/ai-integrations/app-config.yaml @@ -170,6 +170,8 @@ catalog: # maxEntries: 5000 # Optional: ingest only servers with at least one native remote (default: false) # remotesOnly: false + # Optional: request only the latest version of each server via ?version=latest (default: false) + # latestVersion: false # Optional: allowlist of permitted hostnames for defense-in-depth SSRF protection. # hostAllowList: # - registry.modelcontextprotocol.io diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index e27249f2e7d..0a00f6f946b 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -61,6 +61,8 @@ catalog: # maxEntries: 5000 # Optional: ingest only servers with at least one native remote (default: false) # remotesOnly: false + # Optional: request only the latest version of each server via ?version=latest (default: false) + # latestVersion: false # Optional: restrict outbound requests to specific hostnames (defense-in-depth) # hostAllowList: # - registry.example.com @@ -85,6 +87,7 @@ catalog: | `pageSize` | No | _(registry default)_ | Sent as `?limit=` on each list request. When omitted, the registry's default page size applies. | | `maxEntries` | No | `5000` | Maximum total server entries buffered for one complete registry traversal (spans resume syncs). When exceeded, the provider commits the buffer, saves an end cursor, and later traversals stop there until `maxEntries` is patched. | | `remotesOnly` | No | `false` | When `true`, skip servers that do not declare at least one native remote (non-empty type and http(s) URL). Package-only and placeholder-remote servers are omitted from the catalog. | +| `latestVersion` | No | `false` | When `true`, each list request includes `?version=latest` (for example `//servers?version=latest`). When `false` or omitted, the `version` query parameter is left unset. | | `hostAllowList` | No | _(none — all hosts allowed)_ | Array of permitted hostnames. When set, `baseUrl` hostname must be in this list and every outbound request (including each redirect `Location` and `response.url`) is validated at runtime. Missing `response.url` fails closed. Redirects are followed manually so disallowed hosts are never contacted. Provides defense-in-depth against SSRF. When omitted, a warning is logged at startup. | | `schedule` | No | 30m frequency, 3m timeout | `SchedulerServiceTaskScheduleDefinition` controlling sync cadence. The first sync runs after one `frequency` interval unless `initialDelay` is set. | diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts index a2c4a5ab493..b0343c61bd2 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/config.d.ts @@ -40,6 +40,13 @@ interface McpRegistryInstanceConfig { * @visibility backend */ remotesOnly?: boolean; + /** + * When true, list requests include `?version=latest` so the registry + * returns only the latest version of each server. + * + * @visibility backend + */ + latestVersion?: boolean; /** @visibility backend */ hostAllowList?: string[]; /** @visibility backend */ diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md index dfeabf340a0..cae42aa4be5 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/report.api.md @@ -40,6 +40,7 @@ export interface McpRegistryProviderConfig { defaultLifecycle?: string; defaultOwner?: string; hostAllowList?: string[]; + latestVersion?: boolean; maxEntries?: number; pageLimit?: number; pageSize?: number; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts index e1aa4531fc4..7f6fa04b671 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.test.ts @@ -62,6 +62,7 @@ describe('McpRegistryEntityProvider', () => { pageLimit: number; maxEntries: number; remotesOnly: boolean; + latestVersion: boolean; }; } ).config; @@ -70,6 +71,7 @@ describe('McpRegistryEntityProvider', () => { expect(resolved.pageLimit).toBe(10); expect(resolved.maxEntries).toBe(5000); expect(resolved.remotesOnly).toBe(false); + expect(resolved.latestVersion).toBe(false); }); it('registers the refresh task from connect after the catalog connection exists', async () => { diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts index a29addc93fc..a5a082476f3 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/McpRegistryEntityProvider.ts @@ -214,6 +214,7 @@ export class McpRegistryEntityProvider implements EntityProvider { apiVersion, pageLimit, pageSize, + latestVersion, maxEntries, hostAllowList, } = this.config; @@ -237,6 +238,7 @@ export class McpRegistryEntityProvider implements EntityProvider { apiVersion, pageLimit, pageSize, + latestVersion, maxEntries, priorEntryCount: this.pendingEntries.length, startCursor: this.resumeCursor, diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts index 48fbdafdb45..00faccf1ae8 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.test.ts @@ -214,6 +214,47 @@ describe('fetchRegistryServers', () => { expect(secondUrl).toContain('limit=50'); }); + it('sends version=latest when latestVersion is true', async () => { + const page1: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 1 }, + }; + const fn = mockFetch([{ body: page1 }]); + + await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v0.1', + pageLimit: 10, + latestVersion: true, + fetchApi: fn, + }); + + const requestUrl = fn.mock.calls[0][0] as string; + expect(requestUrl).toBe( + 'https://registry.example.com/v0.1/servers?version=latest', + ); + }); + + it('omits version query param when latestVersion is false', async () => { + const page1: McpRegistryListResponse = { + servers: [{ server: createMockServerDoc('test/server-a', '1.0.0') }], + metadata: { count: 1 }, + }; + const fn = mockFetch([{ body: page1 }]); + + await fetchRegistryServers({ + baseUrl: 'https://registry.example.com', + apiVersion: 'v0.1', + pageLimit: 10, + latestVersion: false, + fetchApi: fn, + }); + + const requestUrl = fn.mock.calls[0][0] as string; + expect(requestUrl).toBe('https://registry.example.com/v0.1/servers'); + expect(requestUrl).not.toContain('version='); + }); + it('returns a resumeCursor when default pageLimit of 10 is reached with more pages', async () => { const pages = Array.from({ length: 10 }, (_, i) => ({ body: { @@ -685,10 +726,21 @@ describe('buildPageRequestUrl', () => { const url = buildPageRequestUrl(endpoint, 'abc', 25); expect(url.searchParams.get('cursor')).toBe('abc'); expect(url.searchParams.get('limit')).toBe('25'); + expect(url.searchParams.get('version')).toBeNull(); + }); + + it('adds version=latest when latestVersion is true', () => { + const url = buildPageRequestUrl(endpoint, undefined, undefined, true); + expect(url.searchParams.get('version')).toBe('latest'); + }); + + it('omits version when latestVersion is false', () => { + const url = buildPageRequestUrl(endpoint, undefined, undefined, false); + expect(url.searchParams.get('version')).toBeNull(); }); it('does not mutate the original endpoint URL', () => { - buildPageRequestUrl(endpoint, 'abc', 25); + buildPageRequestUrl(endpoint, 'abc', 25, true); expect(endpoint.search).toBe(''); }); }); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts index 69a7e428fa8..f78cd21b1e0 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/client.ts @@ -75,6 +75,11 @@ export interface FetchServersOptions { apiVersion: string; pageLimit: number; pageSize?: number; + /** + * When true, each list request includes `?version=latest`. When false + * or omitted, the `version` query parameter is left unset. + */ + latestVersion?: boolean; /** * Maximum total entries buffered across the current registry * traversal (including prior resume syncs) before a full mutation. @@ -177,7 +182,8 @@ export function parseServersEndpointUrl( } /** - * Build a page request URL with optional cursor and page-size params. + * Build a page request URL with optional cursor, page-size, and + * latest-version query params. * * @internal */ @@ -185,6 +191,7 @@ export function buildPageRequestUrl( endpoint: URL, cursor?: string, pageSize?: number, + latestVersion?: boolean, ): URL { const url = new URL(endpoint.toString()); if (cursor) { @@ -193,6 +200,9 @@ export function buildPageRequestUrl( if (pageSize !== undefined) { url.searchParams.set('limit', String(pageSize)); } + if (latestVersion) { + url.searchParams.set('version', 'latest'); + } return url; } @@ -552,6 +562,7 @@ export async function fetchRegistryServers( apiVersion, pageLimit, pageSize, + latestVersion, maxEntries, priorEntryCount = 0, startCursor, @@ -575,7 +586,7 @@ export async function fetchRegistryServers( let maxEntriesEndCursor: string | undefined; while (!isAtEndCursor(cursor, endCursor)) { - const url = buildPageRequestUrl(endpoint, cursor, pageSize); + const url = buildPageRequestUrl(endpoint, cursor, pageSize, latestVersion); const body = await fetchRegistryPage(doFetch, url, hostAllowList); allServers.push(...body.servers); pagesFetched += 1; diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts index 8300eeb423a..195fc256d76 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.test.ts @@ -26,6 +26,7 @@ import { readPageLimit, readProviderSchedule, readRemotesOnly, + readLatestVersion, readRequiredHttpBaseUrl, resolveMcpRegistryProviderConfig, safeGetOptionalString, @@ -72,6 +73,7 @@ describe('readMcpRegistryProviderConfig', () => { expect(result!.pageLimit).toBe(10); expect(result!.maxEntries).toBe(5000); expect(result!.remotesOnly).toBe(false); + expect(result!.latestVersion).toBe(false); expect(result!.pageSize).toBeUndefined(); expect(result!.baseName).toBeUndefined(); expect(result!.defaultOwner).toBeUndefined(); @@ -236,6 +238,18 @@ describe('readMcpRegistryProviderConfig', () => { expect(result!.defaultLifecycle).toBe('experimental'); }); + it('reads latestVersion', () => { + const config = new ConfigReader( + providersConfig({ + baseUrl: 'https://registry.example.com', + latestVersion: true, + }), + ); + + const result = readMcpRegistryProviderConfig(config); + expect(result!.latestVersion).toBe(true); + }); + it('reads apiVersion override', () => { const config = new ConfigReader( providersConfig({ @@ -371,7 +385,7 @@ describe('assertSingleRegistryConfig', () => { /found keyed instance/, ); expect(() => assertSingleRegistryConfig(config)).toThrow( - /maxEntries, remotesOnly, hostAllowList/, + /remotesOnly, latestVersion, hostAllowList/, ); }); }); @@ -477,6 +491,24 @@ describe('readRemotesOnly', () => { }); }); +describe('readLatestVersion', () => { + it('defaults to false when omitted', () => { + expect(readLatestVersion(new ConfigReader({}))).toBe(false); + }); + + it('returns true when latestVersion is true', () => { + expect(readLatestVersion(new ConfigReader({ latestVersion: true }))).toBe( + true, + ); + }); + + it('returns false when latestVersion is false', () => { + expect(readLatestVersion(new ConfigReader({ latestVersion: false }))).toBe( + false, + ); + }); +}); + describe('readHostAllowList', () => { it('returns undefined when omitted', () => { expect(readHostAllowList(new ConfigReader({}))).toBeUndefined(); @@ -568,6 +600,7 @@ describe('resolveMcpRegistryProviderConfig', () => { pageLimit: 10, maxEntries: 5000, remotesOnly: false, + latestVersion: false, }); }); @@ -580,6 +613,7 @@ describe('resolveMcpRegistryProviderConfig', () => { pageLimit: 3, maxEntries: 100, remotesOnly: true, + latestVersion: true, }), ).toEqual({ baseUrl: 'https://registry.example.com', @@ -588,6 +622,7 @@ describe('resolveMcpRegistryProviderConfig', () => { pageLimit: 3, maxEntries: 100, remotesOnly: true, + latestVersion: true, }); }); @@ -600,6 +635,7 @@ describe('resolveMcpRegistryProviderConfig', () => { pageLimit: undefined, maxEntries: undefined, remotesOnly: undefined, + latestVersion: undefined, }), ).toEqual({ baseUrl: 'https://registry.example.com', @@ -608,6 +644,7 @@ describe('resolveMcpRegistryProviderConfig', () => { pageLimit: 10, maxEntries: 5000, remotesOnly: false, + latestVersion: false, }); }); }); diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts index b9596682150..49204177989 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/src/config.ts @@ -36,6 +36,9 @@ const DEFAULT_MAX_ENTRIES = 5000; /** Default remotesOnly when omitted. */ const DEFAULT_REMOTES_ONLY = false; +/** Default latestVersion when omitted. */ +const DEFAULT_LATEST_VERSION = false; + /** * Reserved instance id under `catalog.providers.mcpRegistry`. * This implementation expects only this key; additional ids are rejected @@ -57,6 +60,7 @@ const KNOWN_MCP_REGISTRY_KEYS = new Set([ 'pageSize', 'maxEntries', 'remotesOnly', + 'latestVersion', 'hostAllowList', 'schedule', ]); @@ -316,6 +320,25 @@ export function readRemotesOnly(registryConfig: Config): boolean { } } +/** + * Read `latestVersion`, defaulting to `false`. + * + * When true, list requests include `?version=latest`. + * + * @internal + */ +export function readLatestVersion(registryConfig: Config): boolean { + try { + return ( + registryConfig.getOptionalBoolean('latestVersion') ?? + DEFAULT_LATEST_VERSION + ); + } catch { + // ConfigReader throws TypeError for empty-string env substitution. + return DEFAULT_LATEST_VERSION; + } +} + /** * Read the provider schedule, or the documented default when omitted. * @@ -366,6 +389,11 @@ export interface McpRegistryProviderConfig { * Package-only / placeholder-remote servers are skipped (default `false`). */ remotesOnly?: boolean; + /** + * When true, list requests include `?version=latest` so the registry + * returns only the latest version of each server (default `false`). + */ + latestVersion?: boolean; /** Optional allowlist of permitted hostnames for defense-in-depth SSRF protection. */ hostAllowList?: string[]; /** Schedule for the sync task. */ @@ -383,6 +411,7 @@ export type ResolvedMcpRegistryProviderConfig = McpRegistryProviderConfig & { pageLimit: number; maxEntries: number; remotesOnly: boolean; + latestVersion: boolean; }; /** @@ -399,6 +428,7 @@ export function resolveMcpRegistryProviderConfig( pageLimit: config.pageLimit ?? DEFAULT_PAGE_LIMIT, maxEntries: config.maxEntries ?? DEFAULT_MAX_ENTRIES, remotesOnly: config.remotesOnly ?? DEFAULT_REMOTES_ONLY, + latestVersion: config.latestVersion ?? DEFAULT_LATEST_VERSION, }; } @@ -463,6 +493,7 @@ export function readMcpRegistryProviderConfig( pageSize: readOptionalPageSize(registryConfig), maxEntries: readMaxEntries(registryConfig), remotesOnly: readRemotesOnly(registryConfig), + latestVersion: readLatestVersion(registryConfig), hostAllowList, schedule: readProviderSchedule(registryConfig), }; From 74fa58daf3141398429ba92415b1469dc25b9d28 Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Mon, 21 Sep 2026 12:24:24 -0400 Subject: [PATCH 62/63] docs(#4815): document using official MCP registries with the provider Add a guide for pointing the MCP Registry provider at production or staging official registries, and cross-link it from workspace and plugin READMEs plus the local deploy doc. Assisted-by: grok-4.6 Co-authored-by: Cursor Signed-off-by: Michael Valdron --- workspaces/ai-integrations/README.md | 9 +- .../docs/deploy-mcp-registry-locally.md | 6 + .../docs/using-official-mcp-registries.md | 105 ++++++++++++++++++ .../README.md | 9 +- 4 files changed, 127 insertions(+), 2 deletions(-) create mode 100644 workspaces/ai-integrations/docs/using-official-mcp-registries.md diff --git a/workspaces/ai-integrations/README.md b/workspaces/ai-integrations/README.md index 7cb5faeb215..1ef9179d35c 100644 --- a/workspaces/ai-integrations/README.md +++ b/workspaces/ai-integrations/README.md @@ -30,7 +30,14 @@ If you would like to build with `docker`, add the `--user-docker` tag like so: npx --yes @red-hat-developer-hub/cli@latest plugin package --tag --tag "${PLUGIN_CONTAINER_TAG}" --use-docker ``` -## Deploy MCP Registry Locally +## MCP Registry + +### Official Live Deployments + +To ingest MCP servers from the official MCP Registry into the catalog, see +[Using Official MCP Registries](./docs/using-official-mcp-registries.md). + +### Deploy Locally To run a local MCP Registry for provider development, see [Deploy MCP Registry Locally](./docs/deploy-mcp-registry-locally.md). diff --git a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md index 640df0cc208..dc2d23ef8ca 100644 --- a/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md +++ b/workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md @@ -108,3 +108,9 @@ node scripts/undeploy-local-mcp-registry.ts This runs `compose down` for the same stack. The `~/.cache/rhdh-ai-integrations/mcp-registry` checkout is left in place so the next deploy is faster. + +## Official registries alternative + +To point the provider at the production or staging official MCP Registry +instead of a local instance, see +[Using Official MCP Registries](./using-official-mcp-registries.md). diff --git a/workspaces/ai-integrations/docs/using-official-mcp-registries.md b/workspaces/ai-integrations/docs/using-official-mcp-registries.md new file mode 100644 index 00000000000..210d4f013c9 --- /dev/null +++ b/workspaces/ai-integrations/docs/using-official-mcp-registries.md @@ -0,0 +1,105 @@ +# Using Official MCP Registries + +The [official MCP Registry](https://github.com/modelcontextprotocol/registry) can be used by the [MCP Registry Provider](../plugins/catalog-backend-module-mcp-registry-provider/) +to ingest published MCP servers into the RHDH catalog as `mcp-server` API +entities. + +The official registry API is documented in the upstream +[Official MCP Registry API](https://github.com/modelcontextprotocol/registry/blob/v1.8.1/docs/reference/api/official-registry-api.md) +(based on the +[generic registry API](https://github.com/modelcontextprotocol/registry/blob/v1.8.1/docs/reference/api/generic-registry-api.md)). +Interactive docs and the OpenAPI spec are available at +[registry.modelcontextprotocol.io/docs](https://registry.modelcontextprotocol.io/docs). + +## Base URLs + +| Environment | Base URL | +| ----------- | -------------------------------------------------- | +| Production | `https://registry.modelcontextprotocol.io` | +| Staging | `https://staging.registry.modelcontextprotocol.io` | + +Listing servers does **not** require authentication. Auth endpoints on the +official registry are only needed for publishing and status updates, which this +provider does not perform. + +## Configure the provider + +Install and register the provider as described in the +[MCP Registry Provider README](../plugins/catalog-backend-module-mcp-registry-provider/README.md), +then set `catalog.providers.mcpRegistry.mcpRegistry` in `app-config.yaml`. +Set `baseUrl` to the production or staging URL from the [table above](#base-urls), +depending on which environment you intend to use. + +The official registry serves the **`v0.1`** API instead of the `v1` default, +set `apiVersion: v0.1` explicitly. + +**Production** + +```yaml +catalog: + providers: + mcpRegistry: + mcpRegistry: + baseUrl: https://registry.modelcontextprotocol.io + apiVersion: v0.1 + # Recommended: restrict outbound hosts (defense-in-depth against SSRF) + hostAllowList: + - registry.modelcontextprotocol.io + # Optional: ingest only the latest version of each server + # latestVersion: true + # Optional: skip package-only / placeholder-remote servers + # remotesOnly: true +``` + +**Staging** + +```yaml +catalog: + providers: + mcpRegistry: + mcpRegistry: + baseUrl: https://staging.registry.modelcontextprotocol.io + apiVersion: v0.1 + hostAllowList: + - staging.registry.modelcontextprotocol.io +``` + +Then start the workspace (`yarn dev` from `workspaces/ai-integrations`) or your +Backstage backend. The provider syncs on its schedule (default: every 30 +minutes). See the +[provider configuration options](../plugins/catalog-backend-module-mcp-registry-provider/README.md#configuration-options) +for `pageLimit`, `pageSize`, `maxEntries`, `schedule`, and related settings. + +**Note**: The [Production](https://registry.modelcontextprotocol.io) environment has +_over 5000 entries_ so it is recommended to review [provider configuration options](../plugins/catalog-backend-module-mcp-registry-provider/README.md#configuration-options) to +configure a setup that respects rate limiting and that works for you. + +## How the provider uses the official API + +The provider calls the cursor-paginated list endpoint: + +`GET //servers` + +Against the official registry that is +`GET https://registry.modelcontextprotocol.io/v0.1/servers` (plus optional query +parameters the provider supports). + +| Official registry feature | Provider support | +| ----------------------------------------- | --------------------------------------------------- | +| Cursor pagination (`?cursor=`, `?limit=`) | Yes — via `pageSize` / internal resume | +| `?version=latest` | Yes — set `latestVersion: true` | +| `?search=` | Not used by the provider | +| `?updated_since=` (incremental sync) | Not used — the provider does full traversals | +| `?include_deleted=` | Not used (default listing excludes deleted servers) | +| Server detail / version history endpoints | Not used — entities are built from list entries | +| Publish / auth / status PATCH endpoints | Not used | + +For large registries, raise `pageLimit` / `maxEntries` or rely on resume syncs +as described in the +[provider pagination behavior](../plugins/catalog-backend-module-mcp-registry-provider/README.md#pagination). + +## Local development alternative + +To develop against a local registry instance instead of the public official +API, see +[Deploy MCP Registry Locally](./deploy-mcp-registry-locally.md). diff --git a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md index 0a00f6f946b..e4369cc6c60 100644 --- a/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md +++ b/workspaces/ai-integrations/plugins/catalog-backend-module-mcp-registry-provider/README.md @@ -152,7 +152,14 @@ These MCP server entries have a single remote _placeholder_ field which should * } ``` -## Deploy MCP Registry Locally +## MCP Registry + +### Official Live Deployments + +To ingest MCP servers from the official MCP Registry into the catalog, see +[Using Official MCP Registries](../../docs/using-official-mcp-registries.md). + +### Deploy Locally To run a local MCP Registry for provider development, see [Deploy MCP Registry Locally](../../docs/deploy-mcp-registry-locally.md). From c3b8dfa6d096919015d9397af3954c05210b006e Mon Sep 17 00:00:00 2001 From: Michael Valdron Date: Mon, 21 Sep 2026 12:32:10 -0400 Subject: [PATCH 63/63] docs(#4815): self-revision on wording of official live production note Signed-off-by: Michael Valdron --- .../ai-integrations/docs/using-official-mcp-registries.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/workspaces/ai-integrations/docs/using-official-mcp-registries.md b/workspaces/ai-integrations/docs/using-official-mcp-registries.md index 210d4f013c9..5b06c46f0eb 100644 --- a/workspaces/ai-integrations/docs/using-official-mcp-registries.md +++ b/workspaces/ai-integrations/docs/using-official-mcp-registries.md @@ -72,7 +72,7 @@ for `pageLimit`, `pageSize`, `maxEntries`, `schedule`, and related settings. **Note**: The [Production](https://registry.modelcontextprotocol.io) environment has _over 5000 entries_ so it is recommended to review [provider configuration options](../plugins/catalog-backend-module-mcp-registry-provider/README.md#configuration-options) to -configure a setup that respects rate limiting and that works for you. +configure a setup that respects rate limiting and that works for your deployment resources. ## How the provider uses the official API