Skip to content

ci: run opt-in integration tests for affected plugins - #685

Open
balkashynov wants to merge 1 commit into
1Password:mainfrom
balkashynov:codex/changed-plugin-integration
Open

balkashynov wants to merge 1 commit into
1Password:mainfrom
balkashynov:codex/changed-plugin-integration

Conversation

@balkashynov

@balkashynov balkashynov commented Oct 14, 2026 •

Copy link
Copy Markdown

Overview

Adds the shared opt-in integration workflow proposed in #680. Pull requests select only integration-test packages affected by their changed files or local Go test dependencies, so unrelated plugin changes avoid Docker pulls. SDK, module, selector and workflow changes select all integration packages; manual dispatch also runs all of them.

The selector compares the PR merge commit with its first parent, includes both sides of renames, and fails visibly if Git or dependency discovery fails. The workflow uses a read-only token, no secrets, bounded jobs and at most two integration packages in parallel. Empty selections skip the integration job.

Reuses the CockroachDB integration tests contributed in #536. They accept SHELL_PLUGINS_INTEGRATION=1 while retaining COCKROACHDB_INTEGRATION=1 compatibility. Ordinary unit tests still skip external dependencies. Please keep this workflow non-required, as requested in #680.

The checks below were actually run; independent maintainer review is welcome.

Type of change

  • Created a new plugin
  • Improved an existing plugin
  • Fixed a bug in an existing plugin
  • Improved contributor utilities or experience

Related Issue(s)

Resolves #680.

How To Test

Fresh local validation on macOS ARM64 with Go 1.24.13:

  • go test -count=1 ./...: 93 packages passed, 816 passing tests/subtests; the dependency-backed test skipped by default.
  • go test -race -count=1 ./cmd/ci/integration-matrix: passed.
  • SHELL_PLUGINS_INTEGRATION=1 go test -count=1 -timeout=10m ./plugins/cockroachdb: passed against the pinned CockroachDB container, including TLS and invalid-credential cases.
  • COCKROACHDB_INTEGRATION=1 go test -count=1 -timeout=10m -run '^TestDatabaseCredentialsIntegration$' ./plugins/cockroachdb: passed with the shared switch unset.
  • make validate, golangci-lint 2.12.2, and actionlint 1.7.12 with ShellCheck 0.11.0: passed.
  • Seven executable-selector smoke cases passed: manual all, unchanged commits, unrelated plugin, affected fixture, shared SDK, selector changes, and a depth-2 clone of a multi-commit PR merge containing a rename.

GitHub-hosted Ubuntu validation of the unchanged source tree passed before the commit-message-only update in fork validation PR #1:

  • Plugin Integration: selection and CockroachDB integration passed using the actual pull_request merge checkout.
  • Test and Validate: full unit tests, schema validation and lint passed.
  • Signed commits: passed. The upstream signed-commit check also passed.

The current signed commit b191f20843e474742d7a7d6b5fccd55b5a7684c2 now also passes upstream Test and Validate, Plugin Integration, and signed-commit checks. Its source tree is identical to the earlier validated commit; only the commit message changed. No real 1Password account is required or used by these tests.

Changelog

Plugin integration tests can run in a separate optional workflow for affected plugins using a shared opt-in convention.

Select affected plugin integration tests and document the shared opt-in convention. Retain the existing CockroachDB opt-in for compatibility.
@balkashynov
balkashynov force-pushed the codex/changed-plugin-integration branch from 35b09b8 to b191f20 Compare October 14, 2026 21:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI] Add an opt-in integration test job that runs only for changed plugins

1 participant