Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series Mac
A custom Swift + Metal runtime for any Apple Silicon Mac, even the 8 GB ones, that runs the instruction-tuned Gemma 4 26B-A4B without loading the entire 14.3 GB model into memory.
Intelligence analysis by Llama
TurboFieldfare is a custom Swift + Metal runtime for any Apple Silicon Mac that runs the instruction-tuned Gemma 4 26B-A4B without loading the entire 14.3 GB model into memory. It keeps the shared 1.35 GB core and FP16 KV cache in memory, then streams only the experts needed for each token from SSD.
TurboFieldfare is a special program that helps big computers like Macs run really big models without using too much memory. It does this by keeping some parts of the model in memory and streaming the rest from the computer's hard drive.
Analysis
A $60B Vote of Confidence
TurboFieldfare is a custom Swift + Metal runtime for any Apple Silicon Mac that runs the instruction-tuned Gemma 4 26B-A4B without loading the entire 14.3 GB model into memory. This is made possible by keeping the shared 1.35 GB core and FP16 KV cache in memory, then streaming only the experts needed for each token from SSD. The runtime, streaming installer, CLI, and native Mac app are written in Swift and Metal. TurboFieldfare is model-specific rather than a wrapper around MLX or llama.cpp. The curated experiment record summarizes 103 measured results across kernels, caching, I/O, prefill, and decode.
Why Cursor?
The Mac app treats what you type as an instruction and handles Gemma's chat formatting automatically. Just describe the task and include any context the model needs. Generation defaults to temperature 0.2, Top-K 64, and Top-P 0.95. Set temperature to 0 for deterministic greedy output. The model can still repeat itself or give incorrect answers, so check important results. TurboFieldfare is text-only. The app and CLI support user and model messages plus optional system guidance; they do not expose or execute tools. The loopback server accepts function-tool declarations and returns model-produced tool calls for the client to authorize and execute.
The Road Ahead
TurboFieldfare provides a native Mac app, a command-line interface, and an experimental loopback OpenAI-compatible server. They use the same .gturbo model directory, but only one model-owning product should run at a time. The Swift package exposes six products: TurboFieldfare, TurboFieldfareMac, TurboFieldfareDecodeService, TurboFieldfareCLI, TurboFieldfareServer, and TurboFieldfareRepack. The package is arm64-only. Older macOS and Metal versions are not supported.
Key points
- TurboFieldfare is a custom Swift + Metal runtime for any Apple Silicon Mac that runs the instruction-tuned Gemma 4 26B-A4B without loading the entire 14.3 GB model into memory.
- It keeps the shared 1.35 GB core and FP16 KV cache in memory, then streams only the experts needed for each token from SSD.
- The runtime, streaming installer, CLI, and native Mac app are written in Swift and Metal.
- TurboFieldfare is model-specific rather than a wrapper around MLX or llama.cpp.
- The curated experiment record summarizes 103 measured results across kernels, caching, I/O, prefill, and decode.
If this development continues to improve, it could lead to more people being able to access and use large models for various applications, making it easier to develop and deploy AI-powered solutions.
However, there are also potential risks associated with this development, such as the increased demand for computational resources and the potential for model misuse.