Skip to content

🐛🧑‍💻 - bugfix oversized progress evidence is reported as a retryable provider outage #547

Open

Description

@efraespada

Is there an existing issue for this?

  • I have searched the existing issues.

Which actions are affected?

Commits, Pull Request

Which platforms are affected?

Linux (observed; the policy is platform-independent)

Description

Dogfooding #538 / PR #546 found that a complete, valid large comparison is classified as a temporary provider outage. ProgressDiffRepository preserves its 256,000-byte evidence bound but throws provider.unavailable, with Retryable: Yes. The commit check fails although another identical attempt cannot fit the same deterministic input. Progress is correctly left unchanged and no model review is claimed.

Expected: preserve the bounded/completeness invariant, distinguish oversized evidence from an actual transient provider failure, and provide an actionable operator path. Any future bounded multi-part analysis needs an explicit contract and must not certify completion from truncated or partially reviewed changes. Do not raise the global limit, omit checks, truncate silently or claim this comparison was assessed as a workaround.

Reproducing the issue

  1. Inspect PR [#538] ✨ - feat reuse organization configuration and set up selected repositories in one guided session #546 at head e6622daa33d345de8fba0853f0feef5aa2b1401f against base b6d9910fb7b2d6888c711bacd52b3fc75e7e26dc.
  2. Call GitHub compare with exact SHAs, per_page=1&page=1, as the repository adapter does.
  3. Validate all 148 comparison files and retain generated bundle/catalog metadata without their patches. No non-generated patch is missing.
  4. Serialize the evidence: 658,336 UTF-8 bytes, exceeding MAX_EVIDENCE_BYTES = 256_000 in src/data/repository/progress_diff_repository.ts.
  5. Observe Copilot - Commit run: no progress model request is made; persisted progress is retained, but the workflow fails as a retryable temporary outage.

Useful regression boundaries include the exact byte limit, the 299/300-file completeness boundary, missing patches, generated-metadata handling, SHA freshness, and a real transient provider failure. Error/recovery UX must preserve the distinction between unassessed progress and verified completion.

copilot Version

v3.5.6; reproduced while dogfooding PR #546 (the progress-diff policy is unchanged by that PR).

Relevant Log Output

A complete bounded branch comparison is unavailable. Progress was not assessed and existing state was retained.
Impact: The provider was temporarily unavailable.
Error code: provider.unavailable
Action: Retry when the provider is available.
Retryable: Yes

Additional context and comments

This is separate from the SDD schema failure fixed and published in v3.5.6, and separate from dependency vulnerability remediation. The mandatory check on develop for PR #546 is CI Check; this commit-analysis check is not mandatory. No branch-protection settings, evidence limits, labels or deployment gates were changed to bypass it.

Activity

  1. added
    bugSomething isn't working
    bugfixSomething isn't working
    priority: highCritical issue, needs immediate attention.
    on Oct 17, 2026
  2. added theissue type on Oct 17, 2026
  3. moved this to Todo in vypdevon Oct 17, 2026
  4. vypbot commented on Oct 17, 2026

    @vypbot
    Contributor

    Hi! I’m @vypbot, the Copilot assistant for this repository.

    I can answer questions, explain the codebase, propose implementation and test plans, review issues and pull requests for potential bugs or security problems, and help authorized maintainers apply changes.

    Try /copilot help to see the available commands, or mention @vypbot with your question.

  5. moved this from Todo to In Progress in vypdevon Oct 17, 2026
  6. added
    branchedIndicates that a branch has been created for this issue.
    on Oct 17, 2026
  7. changed the title [-]🐛 - bugfix oversized progress evidence is reported as a retryable provider outage[/-] [+]🐛🧑‍💻 - bugfix oversized progress evidence is reported as a retryable provider outage[/+] on Oct 17, 2026
  8. added
    state:workingWork can proceed on the verified branch or without a branch.
    on Oct 17, 2026
  9. vypbot commented on Oct 17, 2026

    @vypbot
    Contributor

    Implementation plan

    Current status: Ready to start. No action is required from maintainers before implementation.

    1. Define a distinct bounded-evidence failure
      • Add a nonretryable semantic error for complete comparisons that exceed the 256,000-byte evidence limit, separate from transient provider failures and stale SHAs.
      • Keep the existing completeness checks and preserve progress state without requesting model review.
    2. Update the comparison adapter
      • In src/data/repository/progress_diff_repository.ts, raise the new error only when serialized evidence exceeds the byte limit; keep transport failures mapped to provider.unavailable.
      • Retain the 300-file boundary, patch validation, generated-metadata handling, and exact-SHA checks.
    3. Make recovery guidance actionable
      • Map the new error to a clear operator action, such as reducing the change set and rerunning the comparison.
      • Update the error reference and any user-facing presentation so oversized evidence is not described as a temporary outage or assessed progress.
    4. Add focused regression coverage
      • Cover the exact byte limit and one byte over, 299/300 files, missing patches, generated metadata, stale SHAs, and a real transient provider failure.
      • Assert oversized evidence causes no model request and leaves persisted progress unchanged.

    Acceptance: A complete comparison at or below 256,000 serialized UTF-8 bytes remains eligible for review; one above the limit returns a distinct nonretryable error with actionable guidance, makes no model request, and retains existing progress, while transient provider failures remain retryable provider.unavailable errors.

    Need something else? Mention the bot with a question or use /copilot help.

  10. added
    size: XXLIndicates an extremely large task or issue that requires significant time and effort to complete.
    90%Progress: 90%
    and removed on Oct 17, 2026
  11. vypbot commented on Oct 17, 2026

    @vypbot
    Contributor

    Progress: 90% — in progress

    Current status: The comparison includes a distinct nonretryable oversized-evidence error with operator guidance, preserves complete file evidence through bounded multipart assessment, retains transient provider failures as retryable, and adds focused regression tests and documentation. The supplied evidence does not establish that the tests or build passed, and the specification says fresh native acceptance is pending.

    Next: Run the required automated gates and verify native acceptance on the current head, including a successful assessment of the complete larger comparison and terminal oversized-evidence handling with prior progress retained.

  12. vypbot commented on Oct 17, 2026

    @vypbot
    Contributor

    Action required: synchronize the branch

    bugfix/547-bugfix-oversized-progress-evidence-is-reported-as-a-retryable-provider-outage is 44 commits behind its parent branch develop. It also contains 1 commit not present in the parent branch.

    Run /copilot sync-branch in this conversation to merge the parent changes safely. If Git reports conflicts, the configured fixer agent can resolve eligible files before the verification commands run.

    Compare parent and working branch

  13. efraespada commented on Oct 17, 2026

    @efraespada
    MemberAuthor

    Fixed by #546, merged into develop as 1718068 (validated head 9f7f937).

    The native SDD run completes all four bounded source parts, the final assessment and publication with zero Action errors: https://github.com/vypdev/copilot/actions/runs/39079316914. All current-head CI checks pass, including RepoWise and Ubuntu/macOS/Windows package-session validation; 10,268 tests pass and Codecov confirms 100% changed-line coverage. The original bugfix branch is superseded by this merged implementation. Production rollout follows the normal release flow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

90%Progress: 90%branchedIndicates that a branch has been created for this issue.bugSomething isn't workingbugfixSomething isn't workingin-progressStart work on an admitted issue.priority: highCritical issue, needs immediate attention.size: XXLIndicates an extremely large task or issue that requires significant time and effort to complete.state:workingWork can proceed on the verified branch or without a branch.

Type

Projects

  • Status
    In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions