Skip to content

feat(evaluators): put runtime records behind a flag - #288

Open
namrataghadi-galileo wants to merge 1 commit into
mainfrom
feature/SAO-18194-gate-runtime-behind-env-var
Open

namrataghadi-galileo wants to merge 1 commit into
mainfrom
feature/SAO-18194-gate-runtime-behind-env-var

Conversation

@namrataghadi-galileo

@namrataghadi-galileo namrataghadi-galileo commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Gate structured Galileo Luna scorer-invoke data behind GALILEO_FEATURE_FLAG_SCORER_INVOKE_RUNTIME.
  • Only the exact value enabled sends the serialized Galileo record and runtime execution context; an unset or other value preserves the legacy request and its inputs.

Scope

  • User-facing/API changes: None.
  • Internal changes: Read the feature flag within the Galileo evaluator package and gate record generation and execution-context serialization in its Luna client. Existing scorer identity, authentication, timeout, and response handling remain intact.
  • Out of scope: Generic Agent Control code, Orbit calls, and authentication-flow changes.

Risk and Rollout

  • Risk level: low.
  • Rollback plan: Unset the flag or set it to a value other than enabled to restore legacy request behavior.

Testing

  • Added or updated automated tests for missing, disabled, non-exact, and enabled flag behavior.
  • Ran Galileo package tests (make test): 205 passed.
  • Ran Galileo package lint (make lint) and typecheck (make typecheck).
  • Ran make check (individual package test, lint, and typecheck targets were run instead).
  • Manually verified behavior.

Checklist

  • Linked issue/spec (if applicable).
  • Updated docs/examples for user-facing changes.
  • Included any required follow-up tasks.

@codecov

codecov Bot commented Oct 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants