SIGN IN SIGN UP

✨ Precise Context Budgeting & Tiktoken Integration (#45)

* feat: Implement precise token counting and context calculation 🎯

This commit introduces significant improvements to how token counts and context windows are handled, leading to more accurate and efficient LLM interactions.

✨ Features
- **Introduce Tiktoken for Accurate Token Counting**: For OpenAI and compatible models, `js-tiktoken` is now used for precise token counting, replacing the previous heuristic methods. This ensures that token limits are respected more accurately.
- **Refine Heuristic Tokenizer**: The heuristic tokenizer has been improved with a more nuanced estimation that considers both character and word counts for a better approximation when precise tokenization is not available.
- **Precise Context Window Calculation**: A new `calculateAvailableContext` function is introduced to determine the exact token budget available for prompts by subtracting output tokens, static prompts, and a safety buffer from the model's maximum input tokens. This prevents exceeding context limits and reduces the likelihood of truncation.
- **Smart Diff Truncation**: The `generateCommitMessage` command now uses the precise context calculation to determine the available space for the diff, ensuring it's truncated only when necessary and to the calculated limit.
- **Inline Completion Context Management**: The `buildInlineCompletionPrompt` function now leverages the new `calculateAvailableContext` for more accurate prefix and suffix trimming, ensuring the model receives the most relevant code snippets within the token limits.

🐛 Fixes
- **Improved Token Count Accuracy**: By integrating `js-tiktoken`, the token counts for various inputs are now significantly more accurate, especially for OpenAI models.
- **Reduced Unnecessary Truncation**: The more precise context calculation minimizes accidental truncation of diffs and code snippets.

🧹 Chores
- **Add `js-tiktoken` Dependency**: The `js-tiktoken` library has been added to `package.json` to support the new tokenizer.
- **Update Version**: The package version has been updated to `1.4.1-dev1`.
- **Test Updates**: Unit tests for token counting and related utilities have been updated to reflect the new, more accurate token estimations.

* updating ci

* chore: version bump for release

* cleaning up temp file
A
amwdrizz committed
babb1c1de57dc8d83da3bccdb40dda2fac4f385c
Parent: d38212c
Committed by GitHub <noreply@github.com> on 2/27/2026, 3:41:49 AM