Does anything leave my Mac when I use vMLX?

No. There is no account, no telemetry and no cloud call in the inference path. The only network request the app makes is one you start yourself — downloading a model, or checking for an update.

Which Macs can run vMLX?

Any Apple Silicon Mac on macOS 14 or later, M1 through M5. More unified memory allows bigger models and longer context, but it is not a hard gate — a 16 GB Mac runs small and mid-size models comfortably.

How is vMLX different from LM Studio or Ollama?

Caching and scheduling. Both re-process the conversation from the start on every turn, so a long thread gets progressively slower. vMLX keeps the processed prefix on disk at full precision and resumes from it, and serves several agents from one loaded model rather than one request at a time.

Is the vMLX cache quantised?

No. What is written to disk is full-precision KV. Quantisation applies to live in-memory KV only and never touches stored data, so a restored prefix produces the same answer the original run would have.

Can I use my own models with vMLX?

Yes. Point it at any MLX model, or convert one — a GGUF-to-MLX converter and JANG quantisation are built into the app.

Nothing. It is free and open source under the MIT licence, with no paid tier.

open source · apple silicon · offline

Run the model.
Keep the data.

A local inference engine for Apple Silicon. Paged KV cache, SSD-backed prefix reuse, continuous batching and speculative decoding — the things that make a second turn feel instant instead of making you wait through the whole conversation again. Speaks the OpenAI and Anthropic APIs, so Claude Code and Cursor point at it without changing a line.

↓ download for mac view source

Free · MIT · macOS 14+ · M1 through M5 · no account, no telemetry

vMLX — Console

cache tiers

RAM and SSD, full precision, survives a restart

api dialects

OpenAI and Anthropic, both native

network calls

Nothing in the inference path leaves the Mac

accounts required

No sign-in, no telemetry, no subscription

What's actually in the engine

Most local runners re-read your whole conversation on every turn. vMLX keeps the work it already did, on disk, at full precision — which is where nearly all of the speed comes from.

01 Prefix cache Recognises the part of the conversation it has already processed and starts from there. The longer the thread, the more it saves. ram + ssd

02 Paged KV cache Spills to SSD in fixed-size pages at full precision — not quantised — so a long conversation survives a restart instead of being thrown away. full precision

03 Speculative decoding Drafts several tokens ahead and verifies them in one pass. Accepted drafts are free tokens. draft + verify

04 Continuous batching Several agents share one loaded model without padding waste, so a coding agent and a chat window don't fight over the GPU. one loaded model

05 Built-in converter GGUF to MLX in the app, plus JANG quantisation, so you're not stuck waiting for someone else to publish a conversion. gguf → mlx

06 Unified memory, used properly No copy between host and device and no VRAM ceiling — the model and its cache live in the same pool the GPU already reads from. no vram ceiling

────────────────────────────────────────────────────────────────────────────────────────────────────────

Where the time goes

The claim this engine lives or dies on is simple: a long conversation should not get slower every turn. Turn one pays for the whole prompt. Every turn after it should pay only for what you just added.

time to first token — turn by turn

benchmark pending A published table goes here: context length, prefix hit rate and time to first token for each turn of one conversation, with the host, model and quantisation named beside it. reproducible from the repo · not yet published

Numbers are deliberately absent until they can be published with their conditions attached. A time-to-first-token figure means nothing without the context length it was measured at, and a decode rate means nothing without both the context length and the model — so neither is quoted here as a headline.

What's different here

Three things vMLX does that the common alternatives don't. These are presence-or-absence differences, not tuning differences — and they're the reason the engine exists.

Capability comparison · checked September 2026 against each project's own documentation
Capability	vMLX	LM Studio	Ollama
SSD prefix cacheFull precision, survives a restart	yes	—	—
Anthropic-compatible APINative Messages endpoint, not a shim	yes	—	—
Built-in GGUF → MLX converterPlus JANG quantisation	yes	—	—
OpenAI-compatible APICursor, Continue, Zed, anything else	yes	yes	yes
Ollama-compatible endpointPoint OLLAMA_HOST at it	yes	—	yes
Runs fully offlineNo account, no telemetry	yes	yes	yes

Deliberately not in this table: paged KV cache, continuous batching and speculative decoding. vMLX has all three, but the alternatives ship their own versions of each and a fair comparison would need matched conditions rather than a tick. If something here is out of date, open an issue and it gets corrected.

Inside the app

The engine ships with a full Mac app — sessions, a gateway in front of every loaded model, a model browser, and a live view of what each turn actually cost.

Servers & API — sessions

The session list. Each model server binds its own port and starts and stops on its own.

Servers & API — gateway

One endpoint in front of every loaded model, speaking OpenAI, Anthropic and Ollama.

Models — find & download

Searching Hugging Face and pulling MLX models in the background.

Install it

Download the app, or pull the engine on its own. Both give you the same server on 127.0.0.1:8000.

the apprecommended

# download, drag to Applications, open

→ mlx.studio/download

# model browser, converter, chat UI

# and the server, all in one

the enginepip

$ pip install vmlx

$ vmlx serve mlx-community/Qwen3-8B-4bit

# then point any OpenAI or Anthropic

# client at http://127.0.0.1:8000

Claude Code

Set ANTHROPIC_BASE_URL and go

Cursor

OpenAI-compatible endpoint

Continue

Drop-in provider config

Zed

Custom OpenAI provider

Open WebUI

Points at the same port

Your own script

It's just HTTP and SSE

Questions

Does anything leave my Mac?

No. There is no account, no telemetry and no cloud call in the inference path. The only network request the app ever makes is the one you start yourself — downloading a model, or checking for an update.

Which Macs does it run on?

Any Apple Silicon Mac on macOS 14 or later — M1 through M5. More unified memory means bigger models and longer context; it isn't a hard gate below that. A 16 GB Mac runs small and mid-size models comfortably.

How is this different from LM Studio or Ollama?

Caching and scheduling. Both of those re-process the conversation from the start on every turn, so a long thread gets slower and slower. vMLX keeps the processed prefix on disk at full precision and resumes from it, and it can serve several agents from one loaded model instead of one request at a time.

Is the cache quantised?

No. What gets written to disk is full-precision KV. Quantisation is applied to live in-memory KV only, and it never touches what's stored — so a restored prefix produces the same answer the original run would have.

Can I use my own models?

Yes. Point it at any MLX model, or convert one — the GGUF-to-MLX converter and JANG quantisation are built into the app, so you don't have to wait for someone else to publish a conversion.

What does it cost?

Nothing. It's free and open source under the MIT licence. There is no paid tier and no plan to add one.

Run the model.Keep the data.