server: off-by-one in deferred_create_final_checkpoint + shed unused build + AGENTS.md memory budget docs
Three changes from the QA pass: 1. server-context.cpp: deferred_create_final_checkpoint off-by-one. The previous fix in 1585f98b3 sourced prompt_n_tokens from slot.prompt.n_tokens() - slot.n_decoded. That recovers the prompt boundary in most cases but loses a token on the k-th generation: at the moment the deferred flag fires, prompt.tokens has N + (k-1) tokens (the in-loop push_back of the k-th sampled token has not yet happened -- handle_last_sampled_token runs after the sampling loop, not inside it), n_decoded is k, and the right answer is N = (N + k - 1) - k + 1. Without the +1, pos_max covered [0, N-2] instead of [0, N-1] -- the last prompt token's KV entry was stripped from the checkpoint, the next cold start reprocessed that 1 token, and f_keep/f_sim metrics were off by 1. 2. Shed unused tools and the embedded UI from the default CachyLLama server build. CMakeLists.txt and tools/CMakeLists.txt drop the app/ unified binary, examples/, batched-bench, gguf-split, completion, perplexity, parser, tts, cvector-generator, export-lora, fit-params, and results. tools/server/server-http.cpp and tools/server/CMakeLists.txt gate the embedded-frontend assets behind LLAMA_BUILD_UI. LLAMA_BUILD_EXAMPLES/APP/UI now default OFF. Server, imatrix, llama-bench, quantize, tokenize, mtmd, and rpc are still built. Keeps the build small and focused on the server path while letting LLAMA_BUILD_UI=ON still get the web UI. 3. AGENTS.md context checkpoint section: the worst-case memory budget paragraph said the 2 GiB was a cap (small checkpoint counts got capped at 2 GiB); _ckpt_memory_budget() actually returns std::max(2 GiB, n_ctx_checkpoints * 400 MiB), so 2 GiB is the floor -- small counts get the full 2 GiB working set, large counts scale up. Rewording reflects the actual std::max behavior.
F
fewtarius committed
da9a6da82b6a3d43361d915bbac7b4d5d6d022a6
Parent: 1585f98