Skip to content

fix(LlamaEmbeddingContext): automatically set the batch size for non-causal embedding models - #691

Open
giladgd wants to merge 2 commits into
masterfrom
gilad/handleNonCasualEmbeddingModels
Open

giladgd wants to merge 2 commits into
masterfrom
gilad/handleNonCasualEmbeddingModels

Conversation

@giladgd

@giladgd giladgd commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Description of change

  • fix(LlamaEmbeddingContext): automatically set the batch size for non-causal embedding models

Resolves #678
Fixes #674

Pull-Request Checklist

  • Code is up-to-date with the master branch
  • npm run format to apply eslint formatting
  • npm run test passes with this change
  • This pull request links relevant issues as Fixes #0000
  • There are new or updated unit tests validating the change
  • Documentation has been updated to reflect this change
  • The new commits and pull request title follow conventions explained in pull request guidelines (PRs that do not follow this convention will not be merged)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: Embeddings from non-causal models silently diverge when the input is longer than the context's batch size

1 participant