SIGN IN SIGN UP
unslothai / unsloth UNCLAIMED

Unsloth Studio is a web UI for training and running open models like Qwen, DeepSeek, gpt-oss and Gemma locally.

0 0 99 Python

fix: honor --ctx-size and other forwarded args from `unsloth studio run` in Studio's context-fit logic (#5815)

* fix: honor --ctx-size and other forwarded args from `unsloth studio run` in Studio's context-fit logic

* refactor: extract resolve_requested_ctx as single source of truth

The test helper was reimplementing the two-line
'ctx_override = parse_ctx_override(...); requested_ctx = ctx_override
if ctx_override is not None else n_ctx' pattern locally, so the test
asserted against its own reimplementation rather than production logic.
Extract the conditional into resolve_requested_ctx and have both the
production caller and the test use it.

* fix(studio): honor pass-through cache type flags in KV VRAM estimate

Studio's KV cache VRAM estimate computed from the first-class
cache_type_kv even when the user passed -ctk/--cache-type-k/-ctv/
--cache-type-v via extras. Those flags reached llama-server fine
(last-wins on the CLI) but the pre-launch estimate kept using the
default f16 bytes-per-element, so GPU placement decisions could be
off when the user lowered cache precision via pass-through.

Adds parse_cache_override + resolve_cache_type_kv in llama_server_args.py
(mirroring parse_ctx_override / resolve_requested_ctx), wires both into
load_model alongside the existing ctx resolution, and adds focused
unit tests for the parser + resolver.

Follow-up to @rolandtannous review on #5815.

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
M
Matt Van Horn committed
15d70a1d7bd2557fa42672a9e5e2672f71a9c18c
Parent: 99e1f67
Committed by GitHub <noreply@github.com> on 5/28/2026, 7:34:35 AM