feat(service): run API mode as a launchd service on macOS (#310) (#321)
## Summary
Replaces the macOS arm of `service_cmd::backend()`, which until now returned `NotSupported` pointing at #310, with a real launchd backend, and adds `/Library/Application Support/all-smi/config.toml` as the macOS system-wide config candidate. The #309 contract is untouched: same `ServiceBackend` methods, same scope semantics, same exit codes, same error variants.
Along the way this fixes a pre-existing bug that made API mode's graceful shutdown unreachable on SIGTERM, which is exactly how `launchctl bootout` and `systemctl stop` end a service.
## What changed
- **`packaging/launchd/com.lablup.all-smi.plist`** (new). The canonical job definition, a self-contained system LaunchDaemon an operator can also copy into `/Library/LaunchDaemons` by hand. Same role `packaging/systemd/all-smi.service` plays on Linux.
- **`src/service_cmd/plist.rs`** (new, 370 lines) + `plist_tests.rs`. Embeds that file with `include_str!` and rewrites `ProgramArguments`, the log paths, and the account keys per scope. Walks key/value pairs rather than lines, because a plist value can be a multi-line `<array>`/`<dict>` and dropping a key has to drop its whole value with it. XML-escapes paths and rejects control characters and non-POSIX account names.
- **`src/service_cmd/launchctl.rs`** (new, 349 lines) + `launchctl_tests.rs`. Layout resolution (plist path, log path, launchd domain, service target), `launchctl` invocation, and parsing of `launchctl print` and `launchctl print-disabled`.
- **`src/service_cmd/launchd.rs`** (new, 319 lines) + `launchd_tests.rs`. Verb policy and the on-disk half: the managed-marker guard, the atomic `0644` plist write, log-directory creation.
- **`src/service_cmd/mod.rs`**. The macOS `backend()` arm now returns `LaunchdBackend`. The user-scope persistence note and the "where settings live" line are now per-platform, because `loginctl enable-linger` and "the environment file" do not exist on macOS.
- **`src/service_cmd/detect.rs`**. The Homebrew refusal gains a macOS-only hint naming `sudo brew services start all-smi`.
- **`src/common/paths.rs`**. `/Library/Application Support/all-smi/config.toml` added as a Tier 2 candidate, ordered after every per-user one.
- **`src/main.rs`**. The API-mode shutdown fix, below.
- **`README.md`**, **`docs/man/all-smi.1`**. macOS subsections appended into the slots #309 left, plus the config-candidate tables.
- **`.github/workflows/ci.yml`**. New gated `launchd-service` job on `macos-14`.
## How launchd differs from systemd, and how the verbs map
launchd has no separate "enabled at boot" state. A plist in `LaunchDaemons`/`LaunchAgents` is bootstrapped automatically at boot or login and `RunAtLoad` starts it from there. So:
- `install` without `--now` writes the plist and stops. That *is* "enabled at boot, not running yet".
- `install --now` boots the job out and back in, because launchd caches a loaded job's definition and bootstrapping over it fails rather than replacing it.
- `stop` boots the job out and leaves the plist, so the service returns at the next boot, matching `systemctl stop`.
- `status` reads plist presence from disk for `installed`, since `launchctl print` only knows about loaded jobs, and reads `launchctl print-disabled` for `enabled`.
- `install` runs `launchctl enable` to clear a persistent disable override, which outlives both the plist and a reboot. `uninstall` deliberately does not `disable`, for the same reason.
## The user-scope trap, measured rather than assumed
The user-scope render drops `UserName`, `GroupName`, and `InitGroups`. This is the launchd analogue of #309's `ProtectKernelModules=` problem, but **the failure mode is the opposite one**, and I want to be precise about it rather than repeat the systemd rationale by analogy.
systemd refuses a user unit it cannot apply: `ProtectKernelModules=` dies at `218/CAPABILITIES` before `ExecStart`. launchd does not refuse. Probe agent bootstrapped into `gui/501` on macOS 26.6 (Darwin 25.6), program printed `id`:
| Plist keys | `launchctl bootstrap` | Effective uid/gid |
|---|---|---|
| none | succeeds | `501` / `20` |
| `UserName root`, `GroupName wheel` | succeeds | `501` / `20` |
| `InitGroups` + `UserName root` | succeeds | `501` / `20` |
They are silently ignored. So they are dropped **not** to avoid a crash, but because keeping them would ship a plist that reads, to anyone auditing what runs privileged on the machine, as a root job when it is not one; and because `launchd.plist(5)` documents them as requiring root without documenting the fallback, so it is unspecified behaviour that is free to become fatal later. Since no bootstrap will ever catch a regression here, a render-time test asserts their absence, and a second test asserts every top-level key in the shipped plist is classified by one of the two lists so adding one without deciding fails in CI.
`HardResourceLimits` is deliberately absent from the template for the mirror-image reason (only root can raise a hard limit, lowering one buys nothing). `SoftResourceLimits` is kept in both scopes, since raising a soft rlimit up to the inherited hard limit needs no privilege.
`--service-user` sets `UserName` and drops `GroupName` rather than mirroring the account name the way the systemd renderer does: macOS has no convention that an account owns an eponymous group, so omitting the key makes launchd use the primary group straight from the password database.
## Bug fix: API mode never took its graceful shutdown path
`run_command` installed a SIGTERM handler calling `std::process::exit(0)`. It won the race against `run_api_mode`'s post-serve cleanup, so **every** SIGTERM dropped the last batch of accumulated Joules and left a stale Unix socket behind. Not an edge case for a service: SIGTERM is how both `launchctl bootout` and `systemctl stop` end one, so it happened on every restart.
`Api` now owns its shutdown the way `Record` already did, including through the `[general].default_mode` redispatch (the recursive call cannot uninstall a handler the outer call already spawned), and the collectors are torn down after `run_api_mode` returns like `view` already does.
Before: SIGTERM produced no `energy WAL: shutdown requested` line. After: it appears, and the process exits in 0.45 s.
## Test plan
Scoped commands only (the orchestrator runs the full suite):
- [x] `cargo fmt --check`
- [x] `cargo clippy --lib --tests -- -D warnings`
- [x] `cargo clippy --bin all-smi -- -D warnings` (run separately on purpose. The library target caught nothing, but the binary target flagged `SERVICE_NAME` as dead: it is reachable only from the systemd backend, which is not compiled in a non-Linux non-test build. Per-target dead-code blindness is real here and the scoped library check does not see it.)
- [x] `cargo test --lib service_cmd::`: 110 passed
- [x] `cargo test --lib common::paths`: 17 passed
- [x] `plutil -lint` on the rendered plist, and `man ./docs/man/all-smi.1` renders clean
Live on an M1 Ultra, macOS 26.6, **user scope only** (see below):
- [x] `status` before install exits 3 and reports "not installed"
- [x] `install --user` writes the plist, creates `~/Library/Logs/all-smi`, and leaves the job unloaded; `status` reports `installed, enabled, stopped` / `not loaded`, exit 3
- [x] `start` bootstraps it; `status` reports running with a pid
- [x] `install --user --now`, `restart`, `stop`, `uninstall`, and a second `install` over our own plist (idempotent)
- [x] `curl localhost:9090/metrics` serves 72 `all_smi_*` lines from the LaunchAgent
- [x] `stop` (i.e. `launchctl bootout`) reaches the final energy-WAL flush, visible in the log
- [x] A hand-written plist is refused by both `install` and `uninstall`, the file is left untouched, and `--force` lifts the refusal
- [x] `launchctl disable` is reported as `"enabled": false` by `status` and cleared by `install`
- [x] System scope without root refuses with "requires root" and writes nothing to `/Library/LaunchDaemons`
- [x] `all-smi config path` lists `/Library/Application Support/all-smi/config.toml` as candidate 3, ordered last
### Apple Silicon metrics under launchd
The LaunchAgent's metric **name set is byte-identical** to a foreground `all-smi api` (`diff` of the sorted metric names: no difference). Values sampled from the agent:
```
all_smi_gpu_utilization 17.19
all_smi_gpu_power_consumption_watts 0.56
all_smi_gpu_temperature_celsius 52
all_smi_cpu_temperature_celsius 65
all_smi_ane_power_watts 0
all_smi_thermal_pressure_info 1
all_smi_cpu_p_cluster_frequency_mhz 3223
all_smi_cpu_e_cluster_frequency_mhz 1978
all_smi_chassis_power_watts 48.12
```
IOReport, the SMC, and `NSProcessInfo.thermalState` all resolve from a launchd context with no TTY and no sudo prompt. Cluster frequencies are non-zero thanks to #317.
Two environment notes worth recording:
1. **`ProcessType Background` costs startup time.** Foreground bind: 0.6 s. Under `taskpolicy -b`: 2.9 s. As a LaunchAgent: 6.3 s. Enumerating the IOReport channel list is the expensive part and background QoS runs it on the E-cores. Kept anyway: a GPU monitor should not steal P-cores from the job it is watching. This is why every readiness wait in CI polls `/metrics` rather than `service status`, which reports running the moment launchd spawns the program.
2. **A binary on an external volume hangs under launchd.** The first attempt pointed the plist at `target/debug/all-smi` on an external volume; the job spawned and blocked indefinitely in `dyld` `open()` on its own binary (TCC gate for launchd-spawned processes on removable volumes). Not a code issue; re-verified from an internal-disk path. Worth knowing if anyone installs from a mounted volume.
## Not verified in this environment
These were explicitly out of scope for this run and their acceptance boxes are left unchecked on the issue.
- **System-domain LaunchDaemon.** No `/Library/LaunchDaemons` write and no `sudo launchctl bootstrap system`. The system-scope code path is therefore exercised only by unit tests plus the live "refuses without root, writes nothing" check.
- **Reboot persistence**, in either the subcommand or the `sudo brew services` form. Not reachable without the system domain.
- **`sudo brew services start all-smi`** and the tap formula end to end. Nothing was pushed to `lablup/homebrew-tap`; see the checklist below.
- **The Homebrew refusal live.** `detect::classify` is unit-tested for all three Homebrew prefixes and a new test asserts the macOS hint text, but no binary was placed under `/opt/homebrew` to trigger it for real.
- **`/Library/Application Support/all-smi/config.toml` being loaded.** Listing is verified live; creating the file needs root. The load path itself is the same `discover_existing_config()` used by the Linux `/etc` candidate, and is covered by unit tests for presence, ordering, and never being the `config init` target.
- **Full `cargo test`.** Only the two scoped filters above were run here.
- **Issue item 4 (daemon context with no GUI session) is only partly reached.** A `gui/$UID` LaunchAgent still belongs to a login session. What is genuinely demonstrated is that the native readers need neither a controlling terminal nor sudo and work at background QoS. Whether they behave identically from a boot-time LaunchDaemon with nobody logged in needs a real headless daemon. **No degradation was found in the part that was reachable**, so no follow-up issue is filed; if the system-domain test later shows one, that is the moment to file it.
## Checklist for a human: the tap `service do` block
**Nothing was pushed to `lablup/homebrew-tap`.** Apply this by hand, the way PR #313 handled the Intel formula stanza. Against `Formula/all-smi.rb` at its current HEAD:
```diff
@@ -27,6 +27,13 @@
man1.install "all-smi.1"
end
+ service do
+ run [opt_bin/"all-smi", "api"]
+ keep_alive true
+ log_path var/"log/all-smi.log"
+ error_log_path var/"log/all-smi.log"
+ process_type :background
+ end
+
test do
output = shell_output("#{bin}/all-smi --version")
assert_match(/all-smi\s+#{version}/, output)
```
Verified locally against a scratch copy, never the tap: `ruby -c` passes, and `brew style` reports exactly the same 4 offenses on the patched file as on the unpatched upstream file (all four are artifacts of linting a bare `.rb` outside a tap directory: Sorbet sigils, frozen-string-literal, class documentation). The service block itself adds none.
Two notes on the block versus the version proposed in the issue:
- `process_type :background` is added so a brew-managed service and a subcommand-installed one get the same CPU scheduling. Without it, which install method you used silently changes the QoS class, which is an unpleasant thing to debug. `Homebrew::Service#process_type` accepts `:background`.
- No hardcoded port or interval, and no `require_root`, so both `brew services start all-smi` and `sudo brew services start all-smi` keep working. The update workflow rewrites only the version/url/sha256 stanzas by name, so this block does not disturb it.
Closes #310 J
Jeongkyu Shin committed
8c822aa4c5683305f300c1becb048bc630fcb6f3
Parent: acb3c94
Committed by GitHub <noreply@github.com>
on 8/5/2026, 11:44:00 AM