The useful results came with a failed configuration, a long wait, or a missing reproduction artifact.
A local model recommendation can expire without the model changing. This week’s clearest example is a FreeToken issue where changing one execution flag brought a stalled configuration back to an interactive token rate. The checkpoint stayed put. The hardware stayed put. The placement strategy decided whether either was useful.
That is where the MoE week ends: with evidence more specific than a model-size headline. Four signals deserve attention, from a same-machine regression report to a quantization study that measured completed coding tasks. None of the measurements below are mine, and the distinction between a working configuration and a generally reliable engine matters throughout.
FreeToken issue #436, opened September 10, reports a sharp failure on an RTX PRO 4000 Blackwell with 24GB of VRAM, an EPYC 7302, and PCIe Gen 3 x16. The checkpoint was lovedheart/Qwen3.8-Flash-Next-NVFP4-FP8, with 123GiB of weights. On a source tree incorporating two pull requests, hybrid CPU/GPU execution reported between 0.17 and 1.02 tokens per second and never completed the reporter’s three-prompt run. Switching that same installation to the offload strategy produced 23.07 tokens per second measured across streamed output.
Those numbers are not a clean ratio: one column contains engine-reported rates from incomplete runs, while the other contains an end-to-end decode measurement. The useful observation is the controlled flag change, supported by repeated launches and utilization details. A maintainer’s response reports that hybrid still beat offload on an H100 with matched expert-cache residency and asks for platform diagnostics. The issue remains open at this writing. I would test offload if I hit this symptom, not declare hybrid execution broken everywhere or recommend a blanket downgrade to the reporter’s older, locally patched build.
That distinction also protects Monday’s assessment. The FreeToken paper reports interactive decode and multi-turn agent workloads, including context edits that make state reuse matter. Those remain author-run experiments under their recorded conditions. This week’s issue adds an outside user’s evidence of interactive decode on one newer configuration and failure on another; it does not independently reproduce the paper’s agentic results. A calibration profile can recommend a strategy, but it cannot certify the implementation that will execute it.
The next result pushes model capacity much further and makes the waiting cost unusually visible. ARGODRIVE’s Deltafin fork published a September 8 benchmark package for Kimi K3 on a 128GB M5 Max MacBook Pro, using the internal SSD and three external drives. The fork reports roughly one token per second over a 512-token completion. A separate 512-token prompt test took about 6.3 minutes to reach its first token. Read both numbers before deciding what “runs on a laptop” buys you.
The long-answer decode test used a six-token prompt in raw completion mode, not a chat conversation. Its speculative drafter helped less over 512 generated tokens than over 128, which is why the longer run earns its place. The package’s precision statement also qualifies the headline: expert weights retain their released MXFP4 representation, but the resident trunk is INT8. This is a documented fit and sustained-generation result from a fork developer, not proof of full-precision equivalence, interactive chat, or a completed agent workflow. I like this experiment more after reading its limitations. They tell the next engineer what still needs fixing.
Colibrì’s quieter contribution is a release that makes several existing claims less fragile. Version 1.10.2, released September 6 and still the latest release when checked, fixes a conversion path that could produce a GLM-5.3-Flash container its engine refused. It also corrects a memory guard that counted reclaimable page cache, provoking unnecessary expert eviction, and fixes cancellation during serving. These are the kinds of changes that can alter a local experiment without appearing as a faster token rate in the release title.
The release also documents loading a pruned DeepSeek V4 Flash REAP-150B checkpoint, with confirmation attributed to a second user. Its banner now reports the smaller expert geometry rather than presenting it as the original 284B model. That is useful provenance, not a new performance victory. If you change the expert pool to make a model fit, the result belongs under a different model identity. A cancellation fix and a successful load still leave response quality, long-prompt latency, and repeated tool turns to test.
The strongest quality evidence this week came from a smaller dense model rather than a larger sparse one. Quesma’s Qwen3.8-27B quantization study, which reached Hacker News on September 8, reports that the 17GB Q4_K_M version matched its full-model result on Terminal-Bench 2.1. The study also evaluated graduate-level science questions and instruction following. Four-bit weights held up on those tests; the one-bit variants collapsed on the science benchmark, with longer reasoning sometimes exhausting the budget without returning an answer.
The coding result is more relevant to an agent decision than a plausible paragraph, but its conditions deserve equal space. The author used rented L40S, H100, and H200 GPUs across the experiments, F16 context cache, and a 98K context reservation for the coding benchmark. That does not establish the same complete workload on a 24GB consumer card. More awkwardly, the author says most tested quantization files were subsequently replaced upstream and are no longer available. The observed quality result is useful; exact reproduction is constrained by the missing artifacts. Save the checkpoint revision and file hashes before you spend money evaluating it.
I would use that study to justify testing Q4_K_M against my own tasks before accepting the transfer cost of a much larger model. I would not use it to promise that every four-bit quantization preserves quality, or that quantizing the context cache has the same effect as quantizing weights. The paper-style precision label is only the beginning of the configuration. Reasoning effort, cache precision, and the actual task can still change the outcome after the weights fit.
The evidence now supports narrower claims than a winner’s podium. Deltafin demonstrated storage-backed fit and sustained generation under a documented recipe, with a wait that rules it out as my interactive default. The FreeToken issue provides an outside report of interactive decode, while its multi-turn performance evidence here remains the project’s own. Quesma measured agentic task outcomes through llama.cpp, but on rented hardware rather than a demonstrated consumer workstation recipe. Colibrì’s release improves correctness and operation without establishing a new throughput result. None of those findings supplies the matched comparison that would rank KTransformers and MoE-Infinity against the rest of this week’s engines.
September’s working-set argument gets more demanding from here. The runtime has to put the right bytes near the right compute, then keep the resulting endpoint useful as requests change. Starting tomorrow, the next arc chooses engines for particular machines: Ollama’s convenience path, ExLlamaV3 and TabbyAPI on CUDA, MLX-LM on Apple Silicon, and the limits of splitting work across unequal GPUs. Before choosing an engine for the box, choose the task it is not allowed to fail.
If this was useful, forward it to one engineer who needs less noise in their feed.