✨ Precise Context Budgeting & Tiktoken Integration (#45)
* feat: Implement precise token counting and context calculation 🎯 This commit introduces significant improvements to how token counts and context windows are handled, leading to more accurate and efficient LLM interactions. ✨ Features - **Introduce Tiktoken for Accurate Token Counting**: For OpenAI and compatible models, `js-tiktoken` is now used for precise token counting, replacing the previous heuristic methods. This ensures that token limits are respected more accurately. - **Refine Heuristic Tokenizer**: The heuristic tokenizer has been improved with a more nuanced estimation that considers both character and word counts for a better approximation when precise tokenization is not available. - **Precise Context Window Calculation**: A new `calculateAvailableContext` function is introduced to determine the exact token budget available for prompts by subtracting output tokens, static prompts, and a safety buffer from the model's maximum input tokens. This prevents exceeding context limits and reduces the likelihood of truncation. - **Smart Diff Truncation**: The `generateCommitMessage` command now uses the precise context calculation to determine the available space for the diff, ensuring it's truncated only when necessary and to the calculated limit. - **Inline Completion Context Management**: The `buildInlineCompletionPrompt` function now leverages the new `calculateAvailableContext` for more accurate prefix and suffix trimming, ensuring the model receives the most relevant code snippets within the token limits. 🐛 Fixes - **Improved Token Count Accuracy**: By integrating `js-tiktoken`, the token counts for various inputs are now significantly more accurate, especially for OpenAI models. - **Reduced Unnecessary Truncation**: The more precise context calculation minimizes accidental truncation of diffs and code snippets. 🧹 Chores - **Add `js-tiktoken` Dependency**: The `js-tiktoken` library has been added to `package.json` to support the new tokenizer. - **Update Version**: The package version has been updated to `1.4.1-dev1`. - **Test Updates**: Unit tests for token counting and related utilities have been updated to reflect the new, more accurate token estimations. * updating ci * chore: version bump for release * cleaning up temp file
A
amwdrizz committed
babb1c1de57dc8d83da3bccdb40dda2fac4f385c
Parent: d38212c
Committed by GitHub <noreply@github.com>
on 2/27/2026, 3:41:49 AM