Skip to content

fix: allow npm consumers to install the package - #216

Merged
Abhijeet Prasad (AbhiPrasad) merged 1 commit into
braintrustdata:mainfrom
SiluPanda:fix/npm-consumer-install
Sep 23, 2026
Merged

Abhijeet Prasad (AbhiPrasad) merged 1 commit into
braintrustdata:mainfrom
SiluPanda:fix/npm-consumer-install

Conversation

@SiluPanda

Copy link
Copy Markdown
Contributor

Fixes #212

Summary

  • remove package-manager restrictions from the published engines metadata so npm consumers with engine-strict=true can install autoevals
  • keep repository development pinned to pnpm through the existing packageManager, root-only preinstall check, and lockfile CI gate

npm validates every published engines entry for consumers, so the pnpm/npm/yarn-only metadata made a normal dependency install fail before the package could be used.

Validation

  • fail-first: packed current main, then installed the tarball from an npm consumer with engine-strict=true; npm 11.6.2 failed with EBADENGINE
  • packed this branch and repeated the same consumer install; installation succeeded and import("autoevals") exposed ExactMatch
  • pnpm exec vitest run on the 9 offline JS test files: 66 passed
  • pnpm run build
  • git diff --check

The remaining live-API Vitest files require OpenAI/Braintrust credentials and were not run locally.

Disclosure: this contribution was prepared with OpenAI Codex.

@friederbluemle

Copy link
Copy Markdown

Can someone please review and merge this? I'm seeing this warning in a project that uses npm:

npm warn EBADENGINE Unsupported engine {
npm warn EBADENGINE   package: 'autoevals@0.3.0',
npm warn EBADENGINE   required: {
npm warn EBADENGINE     npm: 'please-use-pnpm',
npm warn EBADENGINE     pnpm: '>=10.27.0',
npm warn EBADENGINE     yarn: 'please-use-pnpm'
npm warn EBADENGINE   },
npm warn EBADENGINE   current: { node: 'v24.19.0', npm: '11.17.0' }
npm warn EBADENGINE }

@SiluPanda

Copy link
Copy Markdown
Contributor Author

Abhijeet Prasad (@AbhiPrasad), could you review this npm-install fix when you have a chance? Another npm consumer has requested review and merge after encountering the same EBADENGINE warning. Thank you!

@AbhiPrasad
Abhijeet Prasad (AbhiPrasad) merged commit 9546b28 into braintrustdata:main Sep 23, 2026
3 of 15 checks passed
@github-actions

github-actions Bot commented Sep 23, 2026

Copy link
Copy Markdown

Braintrust eval report

Autoevals (main-1790175338)

Score Average Improvements Regressions
NumericDiff 78.5% (+1pp) 8 🟢 10 🔴
Time_to_first_token 11.06tok (+1.3tok) 66 🟢 147 🔴
Llm_calls 1.55 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 511.28tok (+8.77tok) 32 🟢 28 🔴
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 473.64tok (+20.87tok) 103 🟢 107 🔴
Completion_reasoning_tokens 354.91tok (+15.42tok) 82 🟢 89 🔴
Completion_accepted_prediction_tokens 0tok (+0tok) - -
Completion_rejected_prediction_tokens 0tok (+0tok) - -
Completion_audio_tokens 0tok (+0tok) - -
Total_tokens 984.91tok (+29.65tok) 106 🟢 105 🔴
Estimated_cost 0$ (+0$) 70 🟢 64 🔴
Duration 11.07s (+1.33s) 67 🟢 148 🔴
Llm_duration 13.9s (+3.5s) 57 🟢 157 🔴

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Package blocks install with npm when using .npmrc file

3 participants