Three teams and one academic study spent the week measuring the same 27B model, and the model was never the point.
The model stopped being the variable this week. Three separate teams published quantization work on the same checkpoint, Qwen 3.8 27B, and the interesting differences were never about the weights. They were about how small the model could get while staying useful, how fast it decoded once it fit, and how much power it burned per answer. The runtime arc that closes today asked which engine belongs on which box. This week answered the companion question the arc kept circling: once the model is fixed, what is left to optimize?
ByteShape’s full ShapeLearn release for Qwen 3.8 27B is the one that touches this month’s runtime work directly. The team shipped five GGUF quantizations and plotted each against competing quants on six different GPUs, from an RTX Pro 6000 down to a 5060 Ti. By “frontier” they mean the set of models where no other plotted quant is both faster and more accurate, and all five of their builds land on it. Their default recommendation, an IQ4_XS build at 3.84 bits per weight and 13.1GB, reaches 99.63% of the BF16 model’s aggregate benchmark score by their measurement. A smaller build holds 98.72% at 11.0GB and runs faster. On the RTX 3090, the card this publication benchmarks on, ByteShape reports 49.5 tokens per second for the larger build against 45.8 for the smaller one, a 7.5% throughput cost for a bit under a point of quality.
Those are ByteShape’s own numbers, measured on their own harness, and the quality figures come from their KLD and benchmark setup rather than an independent reproduction. What makes the release matter for a runtime reader is what rides along with the weights. Every GGUF bundles an MTP draft head for speculative decoding, and a separate DFlash2 draft model pushes throughput further at the cost of more memory and no image input. DFlash2 needs llama.cpp b10658 or newer. The quantization is no longer the whole story. The draft model, the KV budget, and the runtime version are all part of what you actually deploy, and a 24GB owner now has to choose between a slightly better score and the context headroom the smaller build leaves behind.
PrismML made the pattern hard to miss. Bonsai 2 27B, released Wednesday, is also built on Qwen 3.8 27B, and it pushes compression to ternary weights: every parameter is minus one, zero, or plus one, with FP16 group scaling. PrismML claims 98.2% aggregate capability retention against the full-precision model, up from 95% in their first Bonsai release two months ago. The model is Apache 2.0, multimodal, and runs on both NVIDIA CUDA and Apple MLX through custom low-bit kernels, which is the cross-platform story the runtime arc spent a week making explicit.
The number that fits September’s argument is the energy one. PrismML reports 0.714 milliwatt-hours per token on an RTX 4090, which they frame as 40% more energy-efficient than an 8B model running in full precision. Take the retention claim as a vendor claim until someone reproduces it. The direction is what counts: a 27B-class model compressed hard enough to matter on a power budget, measured in energy per token rather than gigabytes on disk. That reframing is not marketing. It is the same shift a serious study made independently this week.
The counterweight to all that vendor benchmarking reached the Hacker News front page on Tuesday. Intelligence per Watt, a research paper whose sixth revision landed September 6, proposes exactly the metric the Bonsai energy figure gestures at: task accuracy divided by power draw. The authors evaluated more than twenty local models across eight accelerators against a million real-world queries. Their headline result is that local models at or under 20B active parameters correctly answered 88.7% of those queries, with accuracy varying by domain, and that intelligence per watt improved 5.3 times between 2023 and 2025.
This is the study the month has been circling without naming. Fit was never the real question. The question is how much correct work a power-constrained machine can do, and the answer has been moving fast enough that a laptop-class accelerator is now a serious inference target rather than a demo. The paper is not our measurement, but it is a large cross-hardware study instead of one team grading its own release, and that distinction is the reason to read it before the next vendor chart.
Runtime drift kept its usual pace underneath all of this. ExLlamaV3, the CUDA-resident engine this arc covered four days ago, shipped v1.5.0 on September 13. That is the seventh release since August 31, following a run of point updates roughly every two to three days. TabbyAPI, its OpenAI-compatible server, tracks the same cadence with rolling commits that mostly exist to bump the ExLlamaV3 dependency. If you wrote down a version number when you read Sunday’s post, it is already stale. This is the normal condition of the fast CUDA path, not a defect, and it is the reason the arc kept saying to pin a commit and record it in the benchmark rather than trusting a release tag to hold still.
The most useful signal of the week was not a benchmark. Patrick McCanna’s migration notes, posted this week, describe moving 35kb of frontier-tuned agent prompts onto a self-hosted 27B model running on Ollama and a 128GB AMD Ryzen AI MAX+ 395, with 32GB reserved for the host and the rest given to inference. Privacy was the motive. The finding was mechanical: prompts that ran clean on a frontier API fell apart locally. In his words, Ollama runs out of fuel within three minutes, the agent thrashes on repeated tool calls, and re-reads files it had already read.
The fix was not a bigger model or a faster engine. It was prompt surgery: splitting a monolithic preprompt into single-objective agent definitions the smaller model could hold without losing the thread. That is the same lesson Wednesday’s mixed-GPU post landed from a different direction. A model that fits is not a workflow that works, and the gap between the two is where the real local-inference engineering lives. Nobody’s quantization chart shows you that gap. Only a run against your own tasks does.
The pattern across the week is that Qwen 3.8 27B has quietly become the field’s shared measuring stick. When three teams and an academic study all reach for the same checkpoint, the model is no longer the experiment. The experiment is everything around it: how many bits the weights survive, which draft model rides along, how much power each token costs, and whether the multi-turn workflow holds once the frontier API is gone. That is precisely the ground the next arc covers. Starting tomorrow, September spends every byte on purpose: mixed-precision quantization that targets a hardware envelope, the KV cache you forgot to budget for, speculative decoding as a second model with a job description, and a benchmark protocol built to keep you honest. The model already fits. The open question is what you do with the memory it left behind.
If this was useful, forward it to one engineer who needs less noise in their feed.\