SIGN IN SIGN UP
unslothai / unsloth UNCLAIMED

Unsloth Studio is a web UI for training and running open models like Qwen, DeepSeek, gpt-oss and Gemma locally.

0 0 98 Python

Studio: source-build arm64 Linux GPU hosts, with a CPU prebuilt fallback (#5924)

* Studio: fall back to source build for arm64 Linux GPU hosts

setup.sh routes a Linux host with any GPU tool to the unslothai/llama.cpp fork,
which publishes only linux-x64 bundles. On an arm64 host with a GPU (GH200,
GB200, DGX Spark) the resolver then selected an x64 CUDA bundle, which cannot run
on aarch64. Routing those hosts to ggml-org instead would install a CPU-only
arm64 build, silently losing the GPU.

Guard resolve_simple_install_release_plans so an arm64 Linux host on the fork
raises PrebuiltFallback before any release is fetched, letting setup.sh do a
source build that actually targets the GPU. x86_64 hosts and arm64 CPU hosts
(which route to ggml-org) are unaffected.

Add tests covering the arm64 fork raise, the x86_64 pass-through, and the arm64
CPU ggml-org path.

* Studio: install ggml-org arm64 CPU prebuilt when the arm64 GPU source build fails

Per review of #5924: arm64 Linux GPU hosts have no CUDA prebuilt anywhere (the
unslothai fork is x64 only, ggml-org ships no Linux CUDA build), so they source
build for the GPU. If that build produces no binary, the host was left without
llama.cpp.

Add a --cpu-fallback flag to install_llama_prebuilt.py that drops the host GPU
attributes so the CPU prebuilt for the host arch is selected (a GPU host cannot
otherwise pick the CPU bundle). setup.sh calls it against ggml-org as a last
resort for arm64 Linux when the source build degraded, installing the
ubuntu-arm64 CPU build instead of leaving the host with no llama.cpp.

Add tests: force_cpu drops GPU attrs before planning, a CPU-forced arm64 host
selects the ggml-org ubuntu-arm64 bundle, and setup.sh wires the fallback.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
D
Daniel Han committed
172d9d1c8e98b136d2fa124bf7f76ece39ee5335
Parent: f695fbd
Committed by GitHub <noreply@github.com> on 6/1/2026, 1:35:43 PM