feat(#4815): implement MCP registry provider backend plugin

workspaces/ai-integrations/.changeset/mcp-registry-mapping-common-provider-link.md

@@ -0,0 +1,5 @@
+---
+'@red-hat-developer-hub/backstage-plugin-catalog-mcp-registry-server-mapping': minor
+---
+
+Rename the mapping common library to `catalog-mcp-registry-server-mapping` (package, directory, and `pluginId`) to match workspace naming conventions. Consumers of `@red-hat-developer-hub/backstage-plugin-mcp-registry-server-mapping-common` should update to the new package name.

workspaces/ai-integrations/.changeset/mcp-registry-provider-plugin.md

@@ -0,0 +1,5 @@
+---
+'@red-hat-developer-hub/backstage-plugin-catalog-backend-module-mcp-registry-provider': minor
+---
+
+Add MCP Registry provider backend module that ingests MCP servers from a configured registry into the catalog as mcp-server API entities. Supports cursor pagination with full-mutation semantics, per-entry failure isolation with last-good retention, nested config under `catalog.providers.mcpRegistry.mcpRegistry` (extra instance ids warn and are ignored), optional `defaultOwner` / `defaultLifecycle` / `remotesOnly` / `latestVersion` (`?version=latest`) / `hostAllowList` / `maxEntries` soft-stop with `pageLimit` resume (re-adding last-good as degraded on later syncs until the server is refreshed successfully), schedule and page controls, identity prefix override, redirect handling with Location validation, and clearer fetch error reporting.

workspaces/ai-integrations/.eslintignore

@@ -1,3 +1,3 @@
 playwright.config.ts
 !.eslintrc.js
-!.prettierrc.js
+!.prettierrc.js

workspaces/ai-integrations/README.md

@@ -29,3 +29,15 @@ If you would like to build with `docker`, add the `--user-docker` tag like so:
 ```
 npx --yes @red-hat-developer-hub/cli@latest plugin package --tag --tag "${PLUGIN_CONTAINER_TAG}" --use-docker
 ```
+
+## MCP Registry
+
+### Official Live Deployments
+
+To ingest MCP servers from the official MCP Registry into the catalog, see
+[Using Official MCP Registries](./docs/using-official-mcp-registries.md).
+
+### Deploy Locally
+
+To run a local MCP Registry for provider development, see
+[Deploy MCP Registry Locally](./docs/deploy-mcp-registry-locally.md).

workspaces/ai-integrations/app-config.yaml

@@ -123,6 +123,16 @@ catalog:
       target: ../../examples/ai-model-server-api.yaml
       rules:
         - allow: [AiModelServerAPI]
+    # Example MCP server API entities as produced by
+    # catalog-backend-module-mcp-registry-provider from the sample registry seed
+    # at examples/mcp-registry/seed-data/seed.json.
+    #
+    # Comment out this location when deploying the MCP Registry provider against
+    # the same seed (e.g. `MCP_REGISTRY_DATA_DIR=./examples/mcp-registry/seed-data yarn start-local-mcp-registry`).
+    - type: file
+      target: ../../examples/api-mcp-servers.yaml
+      rules:
+        - allow: [API]
   providers:
     modelCatalog:
       # The field underneath should list the connector plugin ID that the entity provider accesses through
@@ -139,7 +149,40 @@ catalog:
           kubeflow-model-catalog-url: '${KUBEFLOW_MODEL_CATALOG_URL:-}'
           default-owner: '${OWNER:-default-owner}'
           default-lifecycle: '${LIFECYCLE:-production}'
-# Uncomment to use kubernetesPluginRef — the Backstage kubernetes plugin
+    mcpRegistry:
+      # Reserved instance id — only this key is supported today.
+      mcpRegistry:
+        # Required: base URL of the MCP Registry
+        baseUrl: '${MCP_REGISTRY_URL:-http://localhost:8080/}'
+        # Optional: base name (default: mcp.registry)
+        # baseName: mcp.registry
+        # Optional: API version (default: v1)
+        apiVersion: v0.1
+        # Optional: default entity owner (default: unknown)
+        defaultOwner: '${OWNER:-default-owner}'
+        # Optional: default entity lifecycle (default: production)
+        defaultLifecycle: '${LIFECYCLE:-production}'
+        # Optional: max pages fetched per sync (default: 10)
+        # pageLimit: 10
+        # Optional: registry page size sent as ?limit= (omitted by default)
+        # pageSize: 50
+        # Optional: max entries per complete traversal; soft-stops with end cursor (default: 5000)
+        # maxEntries: 5000
+        # Optional: ingest only servers with at least one native remote (default: false)
+        # remotesOnly: false
+        # Optional: request only the latest version of each server via ?version=latest (default: false)
+        # latestVersion: false
+        # Optional: allowlist of permitted hostnames for defense-in-depth SSRF protection.
+        # hostAllowList:
+        #   - registry.modelcontextprotocol.io
+        #   - staging.registry.modelcontextprotocol.io
+        # Optional: sync schedule (defaults shown below)
+        # schedule:
+        #   frequency: { minutes: 30 }
+        #   timeout: { minutes: 3 }
+        #   # Optional: defer the first sync
+        #   # initialDelay: { seconds: 15 }
+# Uncomment to use kubernetesPluginRef — the Backstage Kubernetes plugin
 # does NOT need to be installed, only its config section is needed.
 #kubernetes:
 #   serviceLocatorMethod:

workspaces/ai-integrations/docs/deploy-mcp-registry-locally.md

@@ -0,0 +1,116 @@
+# Deploy MCP Registry Locally
+
+For local provider development against a real registry instance, this workspace
+includes Node scripts under [`scripts/`](../scripts/) that start the upstream
+[MCP Registry](https://github.com/modelcontextprotocol/registry) with Podman or
+Docker Compose. They use the published
+`ghcr.io/modelcontextprotocol/registry` image instead of upstream
+`make dev-compose` (which builds with `ko` and does not work with Podman).
+
+## Prerequisites
+
+- Node.js 22+ (type stripping for `.ts` scripts)
+- `git`
+- `podman compose` or `docker compose`
+
+## Start
+
+From the `ai-integrations` workspace root, or from
+`plugins/catalog-backend-module-mcp-registry-provider`:
+
+```bash
+yarn start-local-mcp-registry
+```
+
+You can also run the script directly from the workspace root:
+
+```bash
+node scripts/deploy-local-mcp-registry.ts
+```
+
+This clones the registry into `~/.cache/rhdh-ai-integrations/mcp-registry`
+(if needed; override with `MCP_REGISTRY_REPO_DIR`) at tag `v1.8.1` by default
+(override with `MCP_REGISTRY_REPO_URL` / `MCP_REGISTRY_REPO_REVISION`), starts
+PostgreSQL and the registry in the background, **waits until the HTTP API
+responds** (seed import can take a few minutes when seeding from the public
+registry), and serves the API at
+[http://localhost:8080](http://localhost:8080).
+
+Start the registry **before** `yarn dev`. If the provider syncs while the
+registry is still importing seed data, you will see
+`Failed to reach MCP Registry ...` (no mutation). Restart the backend after the
+registry is ready, or wait for the next scheduled sync.
+
+Optional environment variables:
+
+| Variable                        | Default                                                | Description                                                                                                                                                                                                 |
+| ------------------------------- | ------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `MCP_REGISTRY_REPO_DIR`         | `~/.cache/rhdh-ai-integrations/mcp-registry`           | Local checkout path for the registry                                                                                                                                                                        |
+| `MCP_REGISTRY_REPO_URL`         | `https://github.com/modelcontextprotocol/registry.git` | Git remote cloned into `MCP_REGISTRY_REPO_DIR`                                                                                                                                                              |
+| `MCP_REGISTRY_REPO_REVISION`    | `v1.8.1`                                               | Git branch or tag checked out for compose/config                                                                                                                                                            |
+| `MCP_REGISTRY_IMAGE_NAME`       | `ghcr.io/modelcontextprotocol/registry`                | Registry container image name (without tag)                                                                                                                                                                 |
+| `MCP_REGISTRY_IMAGE_TAG`        | `1.8.1`                                                | Registry container image tag                                                                                                                                                                                |
+| `MCP_REGISTRY_DATA_DIR`         | _(checkout `./data`)_                                  | Host directory mounted at `/data` instead of the [default seed data](https://github.com/modelcontextprotocol/registry/blob/main/data/seed.json). When set, seeds from `data/seed.json` with validation off. |
+| `MCP_REGISTRY_URL`              | `http://localhost:8080`                                | URL probed for readiness (and typically used as `catalog.providers.mcpRegistry.mcpRegistry.baseUrl`)                                                                                                        |
+| `MCP_REGISTRY_API_VERSION`      | `v0.1`                                                 | Registry HTTP API version path segment used for the readiness probe                                                                                                                                         |
+| `MCP_REGISTRY_READY_TIMEOUT_MS` | `300000` (5m)                                          | How long `start-local-mcp-registry` waits for the API before failing                                                                                                                                        |
+
+Example with custom seed content (directory must contain `seed.json`):
+
+```bash
+MCP_REGISTRY_DATA_DIR=./examples/mcp-registry/seed-data yarn start-local-mcp-registry
+```
+
+View logs (example with Podman):
+
+```bash
+podman compose -f ~/.cache/rhdh-ai-integrations/mcp-registry/docker-compose.yml logs -f
+```
+
+## Point the provider at localhost
+
+Configure `catalog.providers.mcpRegistry.mcpRegistry` to use the local
+registry. The default local API version is `v0.1`:
+
+```yaml
+catalog:
+  providers:
+    mcpRegistry:
+      mcpRegistry:
+        baseUrl: http://localhost:8080
+        apiVersion: v0.1
+        # Optional when restricting outbound hosts:
+        # hostAllowList:
+        #   - localhost
+```
+
+Then start the workspace as usual (`yarn dev` from `workspaces/ai-integrations`).
+
+See also the
+[MCP Registry Provider](../plugins/catalog-backend-module-mcp-registry-provider/)
+plugin for full configuration options.
+
+## Stop
+
+From the workspace root or
+`plugins/catalog-backend-module-mcp-registry-provider`:
+
+```bash
+yarn stop-local-mcp-registry
+```
+
+Or from the workspace root:
+
+```bash
+node scripts/undeploy-local-mcp-registry.ts
+```
+
+This runs `compose down` for the same stack. The
+`~/.cache/rhdh-ai-integrations/mcp-registry` checkout is left in place so the
+next deploy is faster.
+
+## Official registries alternative
+
+To point the provider at the production or staging official MCP Registry
+instead of a local instance, see
+[Using Official MCP Registries](./using-official-mcp-registries.md).