SIGN IN SIGN UP

Fix: Missing token count for GPT-4o model. (bumps tiktoken-rs to v0.5.9) (#11893)

Fix: this makes sure we have token counts for the new GPT-4o model.

See: https://github.com/zurawiki/tiktoken-rs/releases/tag/v0.5.9 

Release Notes:

- Fix: Token count was missing for the new GPT-4o model.

(I believe this should go in a 0.136.x release)
T
Toon Willems committed
9969d6c702c64823787e88ccd104a2068dcd6d81
Parent: 8c8c176
Committed by GitHub <noreply@github.com> on 5/16/2024, 11:09:28 AM