The Mac wins by holding a model the discrete card cannot, not by decoding it faster.
A 128GB Mac does not run a 70B model faster than a 3090. It runs one the 3090 cannot hold at all, and that is the whole pitch for MLX-LM.
The thing Apple Silicon removes is the wall between two memory pools. On a discrete machine, the GPU has its own VRAM, and the model’s hot path has to live there while system RAM sits mostly unused for inference. Apple’s chips give the CPU and GPU one physical pool of memory. A 128GB Mac can hand most of that to a model, so a checkpoint that would need two mismatched cards or a CPU spill on a PC becomes one clean allocation. MLX-LM is the package that turns that architecture into a working text endpoint. It loads models from the Hugging Face Hub, quantizes and uploads them, fine-tunes with LoRA or full weights, and runs distributed inference across more than one machine.
Before any of that, the install decision, because MLX ships in two rhythms that do not match. The core MLX framework is at v0.32.2, released August 25, with commits landing daily and more than 28,000 stars behind it. MLX-LM tells a stranger story. Its last tagged release, v0.31.3, shipped April 22, and PyPI still serves that April build. The main branch is 115 commits ahead of the tag as of September 14. That gap is not neglect. It is the opposite. The tag is stale because the project ships to main and rarely stops to cut a release. The practitioner consequence is direct: pip install mlx-lm gives you April, and the architecture support for this month’s models lives on main. Pipeline support for Qwen3-Next, a DeepSeek-V3.2 pipeline path, corrected quantized-tensor handling for GPTQ and nvfp4, and vision-model sanitize fixes all merged in the two weeks before this post. If the model you want is newer than spring, install from git and pin a commit.
Unified memory is a capacity story with an asterisk. The shared pool is not the number printed on the box. macOS needs headroom, and there is a wired-memory ceiling that caps how much the GPU is allowed to pin. When a model is large relative to installed RAM, MLX-LM tries to wire the model and cache to keep it fast, which requires macOS 15 or higher, and it prints a warning when the model is big enough to be slow. The documented fix is a sysctl: raise iogpu.wired_limit_mb above the model’s size in megabytes but below the machine’s total. Set it too low and the model swaps and crawls. Set it too high and you starve the operating system. The knob exists because the pool is shared, not infinite, and getting the allocation right is part of the setup, not an afterthought.
The larger correction is speed. Unified memory removes the PCIe transfer between separate pools. It does not remove the bandwidth limit that governs decode. Token generation is bandwidth-bound, and Apple’s memory bandwidth, while real, sits below a discrete NVIDIA card’s. A 3090 moves memory at close to 940 gigabytes per second. Apple’s unified bandwidth ranges from a few hundred gigabytes per second on the Max tier to roughly 800 on the Ultra. The consequence is the sentence at the top: a model that fits on both a 3090 and a Mac will often decode faster on the 3090. The Mac’s advantage appears only when the model does not fit on the card at all. This is capacity against throughput, not one platform beating another, and naming the axis correctly keeps the choice honest.
MLX also has its own weight format. It does not load GGUF or EXL3. Running a model means using an MLX conversion, pulled from the mlx-community organization or produced locally with mlx_lm.convert -q, which lets you pick the bit width and group size for the quantization. That format lock is the same trade every specialist engine asks: it closes cross-runtime portability to buy native performance on one hardware family. Combined with the version split, it means the fastest way to be disappointed is to pip install mlx-lm, pull a spring-era build, and reach for a model whose architecture only landed on main last week.
The way to use MLX-LM well is to treat the Mac as a capacity instrument and let that decide the workload. The deciding question is never whether this is the fastest decode money can buy. It is whether this model needs to exist in one allocation that a consumer PC would have to split or spill. A large dense model, or a big MoE whose expert pool would otherwise stream from disk on a PC, is exactly the case the shared pool answers. Once the model fits, the second question is whether the bandwidth-bound decode rate clears the usability floor for the task. Batch extraction and asynchronous agent work tolerate a slower token rate. Interactive coding does not, and that is where you measure before committing rather than trusting a chart made on an Ultra when you own a Max.
The distributed path extends the same idea past one machine. Inference with mx.distributed splits a model into pipeline stages across more than one Mac, and the recent work adding pipeline support to the Qwen and DeepSeek families is what makes that practical for current large checkpoints. Two Macs linked over Thunderbolt become a larger pool than either holds alone. This is the month’s opening principle at a different scale: the working set does not have to live in one place. In September’s first post that meant VRAM, RAM, and NVMe on one box. Here it means the combined unified memory of two machines, with the interconnect as the new bottleneck to measure instead of PCIe.
The long-context work that fills a large pool uses the same tools, and they carry the same costs. A rotating fixed-size KV cache caps memory at a quality risk, since a smaller --max-kv-size uses less RAM and forgets more of the context. A smaller prefill step size lowers peak memory while reading a long prompt and slows the read. Prompt caching to a safetensors file avoids recomputing a stable prefix across queries, which pays for a fixed document and does not pay for an agent transcript that keeps changing. These are budget levers, not free speedups, and each one trades a resource you can see for one you have to test.
Fit, speed, and quality stay three separate claims on this platform too. The Mac establishes fit better than anything in its price class, holding weights, cache, context, and macOS in one shared pool. It establishes speed only when the number comes from that exact machine, because bandwidth differs across the M-series and a borrowed Ultra result does not describe your Max. It establishes quality only when a task gate passes after MLX quantization and whatever cache setting you chose to make the context fit. The capacity is genuine. The rest is measured, not assumed.
Unified memory changes what can exist on the machine. It does not change the discipline of proving the thing you loaded is the thing you can use.
If this was useful, forward it to one engineer who needs less noise in their feed.