Speculative decoding
A built-in MTP head drafts several tokens per step and the full model verifies them. 65% of drafted tokens were accepted in the 4K prefix-cache run.
The local model server built for coding agents.
Qwen3.8-27B on Apple silicon at 75.3 tok/s. Point Claude Code or Codex at localhost. One Swift binary, a menu-bar app, and nothing leaves your Mac.
Download for Mac
$ brew install megasoft1978/strim/strim Live capture on Apple M5 Max, 64 GB, while 2 agent jobs were running. The benchmarks below run one request at a time.
75.3 tok/s
decode at 4K context, 1.50x the next engine
2.81x
four agents on a shared 16K prefix
30.8 GB
peak memory, vs 35.9 to 37.5 GB for rapid-mlx, mlx-vlm and mlx-lm
127.0.0.1binds loopback only; nothing leaves your Mac
Five engines, one Mac. Losses included.
FIG 1 Decode speed at 4K context, 5 engines
Strim0.1.175.3
rapid-mlx0.15.750.1
oMLX0.4.5.dev133.5
mlx-vlm0.7.1032.7
mlx-lm0.32.230.1
tokens per second, higher is better →
Workload: decode at 4K context, Qwen3.8-27B 4-bit, one engine at a time.
Hardware: Apple M5 Max, 64 GB, macOS 27.0.1. Method: medians of 2 passes, 2026-10-28, build ee9f9d8.
Show data table
| Engine | tok/s | Strim vs engine |
|---|---|---|
| Strim | 75.3 | – |
| rapid-mlx (MTP) | 50.1 | 1.50x |
| oMLX (no MTP) | 33.5 | 2.25x |
| mlx-vlm (no MTP) | 32.7 | 2.30x |
| mlx-lm (no MTP) | 30.1 | 2.50x |
Every ratio, wins and losses.
FIG 2 Strim over its closest rival, by workload
4 agents on a shared 16K prefix2.81xStrim 9.77 vs mlx-vlm 3.47 tok/s (closest rival mlx-vlm)
8 agents at once: median first token1.66xStrim 13.3 vs rapid-mlx 22.1 s (closest rival rapid-mlx)
Decode at 4K context1.50xStrim 75.3 vs rapid-mlx 50.1 tok/s (closest rival rapid-mlx)
Decode at 32K context1.24xStrim 52.2 vs rapid-mlx 42.2 tok/s (closest rival rapid-mlx)
8 short requests at once: total throughput1.02xStrim 76.4 vs mlx-vlm 74.7 tok/s (closest rival mlx-vlm)
8 agents at once: total throughput1.00xtieStrim 31.2 vs mlx-vlm 31.2 tok/s (closest rival mlx-vlm)
4 short requests at once: total throughput0.97xslowerStrim 69.2 vs rapid-mlx 71.7 tok/s (closest rival rapid-mlx)
Cold 32K prompt: first token0.96xslowerStrim 37.1 vs mlx-vlm 35.5 s (closest rival mlx-vlm)
Grey bars left of 1x are where Strim is slower. 28 of 30 sessions were clean; claims use only claim-eligible cells. Method · Every cell
Every loss in the data
Not faster: cold 4K/16K/32K prefill + TTFT (2/3/4% behind mlx-vlm); c=4 512-token aggregate (3% behind rapid-mlx); c=8 512-token p95 first token (6% behind mlx-vlm); 4K prompt aggregate: c=4 2%, c=8 4% behind mlx-vlm. All losses: benchmarks.
What is inside
Exact prefix cache
Agents that share a prompt prefix reuse its computed state, so the repeated part is not read again. A 4K prefix hit returned the first token 3.91x sooner.
no network
$ go test ./...
ok stats 0.4s
writes: project folder onlySandboxed agent harness
Bash runs under sandbox-exec with network denied and writes limited to the project worktree.
Your clients, unchanged
OpenAI Chat Completions, Anthropic Messages and OpenAI Responses endpoints, plus an MCP server.
Menu-bar app
Start, stop and pause the server, watch memory and decode speed, and open the full window.
- engine
- server
- harness
- MCP
One Swift binary, no Python
Engine, server, harness and MCP ship as a single native binary. Nothing to set up beyond the model download.
A minute with Strim
- 01
Install
$ brew installmegasoft1978/strim/strim - 02
Get the model
$ strim pullOr let the menu-bar app download it on first launch.
- 03
Point your agent at it
$ export ANTHROPIC_BASE_URL=http://127.0.0.1:8203 $ export OPENAI_BASE_URL=http://127.0.0.1:8203/v1 Start the server from the menu-bar app or with
strim serve.
Works with Claude Code, Codex CLI, OpenAI and Anthropic SDKs, MCP clients.
The app
Live captures of the running app against a real server on Apple M5 Max, 64 GB. Click any screenshot to enlarge.Tap to enlarge.
Install
Apple Silicon, macOS 14 or newer. The app downloads the model on first launch; with the command line, strim pull does it. Tested on Apple M5 Max, 64 GB.
$ brew install megasoft1978/strim/strim $ strim pull# optional: menu-bar app, includes the CLI
$ brew install --cask megasoft1978/strim/strim-app Installs the strim command, then downloads the model.
$ curl -fsSL https://raw.githubusercontent.com/megasoft1978/strim-engine/main/packaging/install.sh | bash $ strim pullNo sudo. Installs to ~/.local/strim and links ~/.local/bin/strim. Checks the SHA-256 before installing.
Download the DMG (arm64)Open the DMG. Drag Strim to Applications. Launch Strim from the menu bar.
Signed with a Developer ID and notarized by Apple: it opens without a Gatekeeper warning.
Questions
Why only one model?
Everything is tuned for one dense model: its attention state, its speculative-decoding head and its prefix cache. That is why these numbers are possible, and why Strim will not run your other models. Others are on the roadmap.
Does it run on a 32 GB Mac or an M1?
Only a 64 GB M5 Max has been tested. For 32 GB the computed estimate is 114K/214K (low) 154K/291K (high) of context (not measured); 16 GB and below are unsupported. See supported scope.
Does speculative decoding change the output?
Draft tokens are verified by the full model, but batching can flip near-tie tokens. Greedy output over 8 fixed prompts and 64 tokens: Strim against itself 8/8 identical, against rapid-mlx 7/8, against mlx-vlm 6/8, against mlx-lm 6/8, against oMLX 7/8. Quality and exactness.
Does anything leave my Mac?
No prompts or files do. The server binds 127.0.0.1 only and requires a bearer token. The one network step is the model download from Hugging Face, with strim pull or on first launch of the app.
How much memory does it need?
Measured peak over the whole benchmark run: 30.8 GB on Apple M5 Max, 64 GB (rapid-mlx, mlx-vlm and mlx-lm peaked at 35.9 to 37.5 GB). For 32 GB the computed estimate is 114K/214K (low) 154K/291K (high) of context, not measured; 16 GB and below are unsupported.
Why use Ollama or LM Studio instead?
Use them if you want many models or a model browser: Strim runs one model, tuned for agent work. The current benchmark does not include them, so no comparison is claimed. See supported scope.
Is the DMG notarized?
Yes. The app and the DMG are signed with a Developer ID and notarized by Apple, so they open without a Gatekeeper warning. Homebrew and curl installs are not quarantined either.