What It Actually Takes to Run a Frontier Open-Weight Model in Your Own Building

October 15th, 2026

A frontier-class model now fits on a desk

Apple and Meta, at odds over privacy since 2021, are partnering so Meta's new Muse Spark Open model runs on high-end Mac Studio computers. Meta AI chief Alexandr Wang described it to Axios as "mommy and daddy" playing nice again (LinkedIn News). For anyone running AI inside an organization, the part that matters is the second half of the sentence: the model runs on a machine you can order, in your own building, with no cloud in the loop.

Meta calls it the most capable US open-weight model to date. The model card's own scores, 89.0 on GPQA Diamond and 76.4 on Terminal-Bench 2.1, support a frontier-class label, with competitors still ahead on some tests (model card). That turns a question I hear in most engagements with Canadian organizations into something you can price out: can we run a serious model on hardware we own, and what does it cost in money, quality and effort?

This is written for the IT director, data lead or CIO who has been told "just run it on-prem", with the numbers available as of mid-October 2026.

Data residency is the reason to care

Where a model was made and where your data goes are separate questions, and local inference settles the second one. The weights are a file. Run them on a machine in Ottawa and, once the box's outbound traffic is restricted, prompts, documents and outputs never leave the building. Axios makes the same point: run locally, Meta receives no prompts or user data, and Wang told them Meta has neither control over nor visibility into what developers do with it (Axios).

For a Canadian hospital network, law firm or public body, that removes the cross-border transfer and third-party-processing sections of the privacy impact assessment, usually the hardest ones to write. The rest of the assessment still has to be done. But the one question procurement could never verify on its own, where the data physically goes, is now answered by an address.

Two caveats before anyone writes "sovereign AI" in a briefing note. First, open weights still carry a license. The Muse Model License allows commercial use, including fine-tuning, but requires a "Built with Muse Spark Open" notice in your product, comes with an acceptable use policy that counsel will want to read, and requires a separate agreement with Meta if you offer model access as a service via API and gross revenue passes \$20M USD in any 12-month period; internal use is exempt (model card). Second, sovereignty here means control of the data, not independence from suppliers. The chip is Apple's and the weights are Meta's, both American, and the next version arrives when Meta decides it does.

The hardware bill

All 816B parameters have to sit in memory, but only 42B of them work on any given token: Muse Spark Open is a mixture-of-experts model with 256 experts per layer and 8 active per token (Hugging Face). Compute per token is 42B-class; memory has to hold everything. So memory capacity is the gate, memory bandwidth sets the speed, and GPU horsepower comes third.

That is why a Mac Studio is in this story at all. Apple's unified memory gives the CPU and GPU one shared pool, up to 512GB on the M5 Ultra at 1.2TB/s (Apple Canada). A single Mac runs the smaller community quantizations; Meta's own NVFP4 build wants 700GB or more of GPU VRAM, or two Macs linked together (model card, Meta).

Build Memory needed Runs on Source
Unsloth dynamic 1-bit GGUF 180 to 212GB per Unsloth; Meta's developer blog says the smallest Unsloth build needs about 300GB One 256GB Mac Studio if Unsloth's figure holds; otherwise one 512GB Unsloth table, via LinkedIn
2-bit (Unsloth dynamic 2-bit; llama.cpp Q2_K) 250 to 316GB per Unsloth; about 304GB for Q2_K One 512GB Mac Studio. Unsloth lists its smallest 2-bit variant for a 256GB machine, which leaves almost no room for context Unsloth, via LinkedIn, Hugging Face
MLX 3-bit (oQ3e) About 344GB One 512GB Mac Studio Hugging Face
MLX 4-bit About 462GB One 512GB Mac Studio, with little room left for context mlx-community
Meta's NVFP4 build (recommended 1TB) About 1TB across machines, or 700GB+ of GPU VRAM Two 512GB Mac Studios, or four 256GB, linked over Thunderbolt 5 Meta, model card
BF16 (full precision) About 1.5 to 2TB 32 H200 GPUs across four nodes; not a desktop Hugging Face, model card

Meta's recommended 700GB to 1TB sits well above the 462GB that the 4-bit weights take. Meta does not say why; headroom for long context and activations is the likely reason.

Now the bill. Apple Canada lists the M5 Ultra from \$7,799 CAD with 96GB, which fits none of the builds above (Apple Canada). The one configuration with a published price that does fit is Engadget's review unit, 256GB with 2TB of storage, at \$11,299 USD (Engadget). So the cheapest box with a published price is about \$11,300 USD, and it runs this model only if Unsloth's 1-bit figure holds up; Meta's four-box 256GB setup is about \$45,000 USD; and the two-box 512GB setup cannot be priced yet, because none of the pages I checked lists a 512GB price and Apple says that configuration ships in late October.

Speed is the other half. The only absolute figures I have found are on the previous-generation M3 Ultra with 512GB: about 18 tokens per second at 2-bit in llama.cpp, and about 21 tokens per second of generation at 3-bit in MLX, with 225 tokens per second of prompt processing (Hugging Face). Apple's own tests put M5 Ultra prompt processing at up to 4x the M3 Ultra (Apple Canada), but I have not found Muse-specific numbers on the new chip.

For context, 20 tokens per second is a comfortable reading pace for one person. It is not a service.

What you give up

Three things: precision, context and concurrency. None of them shows up in the launch benchmarks, and all three decide whether the box earns its keep.

Precision. Hugging Face's launch post calls the loss at NVFP4 negligible (Hugging Face); I have not seen Meta's own evaluation. The 1-bit and 2-bit builds that fit a single 256GB machine are a different matter, and I have not found a published evaluation of them for this model. In my experience with agentic workloads, quantization loss compounds: a small per-step error rate is fine for a chatbot and ruinous for an agent that makes 30 tool calls before it reports back.

Context. The model supports 1M tokens of context, but context lives in memory too. The KV cache costs about 250KB per token in BF16, or about 266GB for the full 1M window (Hugging Face). On a 512GB Mac Studio with the 462GB 4-bit build, roughly 50GB remains, which is about 200K tokens before KV-cache quantization and before the operating system takes its share. "1M context" on the spec sheet becomes "a large codebase, if you are careful" on the desk.

Concurrency. The published speed figures appear to be single-stream, and I have not found multi-user throughput figures for this model on this hardware. Plan on one or two users per box until you have measured it yourself. This is the constraint that kills most "replace the API" plans, and it has nothing to do with model quality.

"Operate it yourself" is a job description

What Apple and Meta removed is the plumbing, and it is real. MLX now runs the NVFP4 checkpoint and splits every layer across two Mac Studios over Thunderbolt 5 (Meta). Ready-made 3-bit and 4-bit MLX builds are on Hugging Face, and Unsloth Desktop serves the model behind OpenAI- and Anthropic-compatible endpoints, so existing tools can point at it.

What nobody removed is the operating. Your organization now owns a machine that has to be patched, backed up and monitored, with its outbound traffic locked down so the residency claim stays true. Each move to a new quant or model version means re-running the evaluation set, because a quant swap can change behaviour as much as a model swap. Access control, logs, and the afternoon when the box is down and the team's agent with it all land on the same person.

That is a quarter-time role for a small deployment and a full-time one for anything serious, and over three years it can easily cost more than the hardware. It is reasonable to borrow the role for the first 90 days while you decide whether to staff it; it is not reasonable to pretend it does not exist.

Where local beats a cloud API, and where it doesn't

Local wins when the data is sensitive and the volume is steady: a document-review agent working through contracts or patient correspondence all day, or a coding agent on an internal codebase. These run on one box, for one team, with no per-token bill and no transfer to justify.

The API wins when the load is bursty, public-facing or wide. A customer-facing assistant that has to answer 40 people at once belongs on rented capacity; nothing published so far suggests a single Mac Studio can serve that. Most organizations I work with end up hybrid: route by data classification, local for anything marked confidential, API for the rest.

The box pays for itself only if you move real spend onto it, so run the arithmetic before you order:

$$\text{months to break even} = \frac{C}{S_{\text{API}} - S_{\text{local}}}$$

C is the hardware cost. SAPI is the monthly API spend for the workload you would actually move. Slocal is the monthly cost of running the box: power, support, and the operator's time from the section above. As a worked line, \$45,000 of hardware against \$3,000 a month of API spend moved is 15 months before operator time; add a quarter-time operator at \$2,000 a month and it becomes 45 months, past a three-year useful life. When the payback runs past the machine's life, it never pays back on cost and the decision rests on data residency alone. That is a legitimate reason to buy it, but it should be the stated one.

A five-item pilot checklist

If the arithmetic and the data case both hold, run a pilot before a procurement. Five decisions, in order:

  1. Pick one workload with a clear owner. Document review over files that cannot leave the network is the classic first case, because the residency benefit is immediate and the concurrency ceiling does not bite.
  2. Size memory by the context you need, not the model you want. Multiply your longest real prompt in tokens by about 250KB, add the model's footprint from the table above, then add 20 percent.
  3. Evaluate on your own documents, not on launch benchmarks. Build a 50-item test set from real work, score the cloud model you use today, then score the local build at each quant level you are considering.
  4. Name the operator before the box arrives. Patching, updates, access, logs, outbound-traffic rules, and the re-evaluation after every change all belong to one person on paper.
  5. Write the residency claim precisely for the privacy assessment. "Inference runs on hardware at our address, outbound traffic is restricted to a named list, and no prompts, documents or outputs leave the network" is a verifiable sentence. "We use a private AI" is not.

Decide on your data, not on the news cycle

The Apple and Meta story will be forgotten by November; what it marks will not. Local inference of a frontier-class model is now a line item with a price you can bound, a known memory footprint and a known set of trade-offs.

Whether to pay it comes down to three conditions: the data is sensitive enough that residency matters, the volume is steady enough that one box stays busy, and someone will own the machine. When all three hold, the bill above is worth paying. When one of them does not, the cloud API you already use is still the right call, and no headline changes that.