The useful wins came with settings, quality gates, and enough evidence to verify the tradeoff.
The fastest or smallest local AI setting is not a win until the same configuration survives a second question. Model files shrank, decode accelerated, and a tiny model looked fast enough to win. The second question was whether the same configuration still produced useful work at the context length and under the conditions that made the optimization necessary.
That question separated four developments between September 18 and 24. Hugging Face brought packed GGUF weights into Transformers without forcing them back to full precision. Liquid AI published a speculative-decoding result with the drafter, hardware, workloads, and end-to-end limits disclosed. GitHub Next ran 1,200 local decision requests and found that the quickest small model lost too much quality. Compute:Arena shipped the measurement discipline the other three results need.
No universal winner emerged. Better receipts did.
The memory result came from Hugging Face on September 22. Its new Transformers path can load GGUF checkpoints and keep compatible quantized weights packed while ggml-derived Metal kernels operate on them. The published Qwen3.5-4B example shrinks from an 8.42 GB BF16 file to 2.74 GB at Q4_K_M, with Q5_K_M at 3.14 GB and Q6_K at 3.53 GB (Hugging Face announcement). That is a file-size comparison, not a measured peak-memory chart. The packed inference path avoids expanding the whole weight matrix before each decode operation.
The distinction matters because compatibility can erase the savings. Hugging Face says the loader falls back to dequantization when a compatible quantization kernel is unavailable, which uses more memory. The initial packed path is MPS-only and covers Qwen3.5 dense and MoE architectures, including compatible Qwen3.8 checkpoints. Padding and batching still need work. The practical win is one defined Apple Silicon path that can preserve the storage format’s memory advantage inside a familiar Python API.
Quality remains outside that size table. Q4_K_M keeps selected sensitive tensors at higher precision, but Hugging Face still tells users to evaluate the quant on the work they plan to run. That is the right boundary. A 67 percent smaller file can make the model fit; it cannot prove that a coding task, extraction rule, or long-context decision still passes. Fit is the beginning of the test, not its conclusion, and the next optimization spends memory to buy speed.
Liquid AI supplied the speed result on September 24. Its LFM2.5-VL-DSpark release adds a 279.5-million-parameter drafter to the LFM2.5-VL-3B vision-language model, increasing deployed parameter count by 8.9 percent. The drafter proposes blocks of tokens, and the target verifies them while preserving the target distribution under matched sampling. Liquid settled on four draft layers, recommends an inference block size of eight or nine, and measured six MMSpec task categories rather than one friendly caption prompt (Liquid AI release).
The settings make the multiplier useful. On an M5 Max using MLX-VLM, FP16 weights, batch size one, temperature zero, block size eight, and up to 2,048 output tokens, Liquid reports decode improvements from 2.30 to 3.13 times across tasks. End-to-end latency improved 1.56 to 2.62 times. On an M3 Ultra through llama.cpp, decode improved 1.57 to 2.14 times and end-to-end latency improved 1.30 to 1.77 times. The median answer length was 90 tokens, and quantized targets were outside the release’s scope.
Those qualifications are part of the result. Vision encoding and prefill do not become faster because decode speculates. Image tokens make those stages expensive on edge hardware, so a threefold decode gain can produce a much smaller end-to-end improvement. The drafter also consumes memory, even if 8.9 percent looks modest beside the target. DSpark earns a benchmark on supported hardware. It does not earn a promise that every vision workflow becomes three times faster, and the week’s next result showed why latency cannot choose a model alone.
The week’s failed temptation arrived in GitHub Next’s September 18 LocalJev bake-off. The evaluation ran five installed 4-bit models through 1,200 measured requests on an M5 Max with 64 GiB of RAM, using 120 gold-labeled examples across AG News, BoolQ, and SST-5 under short and distracted-input conditions. Gemma 4 E2B posted the lowest short-input median latency at 0.522 seconds. Its macro accuracy was 45.8 percent. Qwen3.6-35B-A3B took 0.889 seconds and reached 76.7 percent (LocalJev evaluation).
The longer condition made the shortcut less attractive. LocalJev inserted 2,048 unrelated background words while retaining and identifying the target. Gemma E2B’s median remained quick at 0.698 seconds, but macro accuracy fell to 40.8 percent and three of 120 requests failed. Qwen’s median rose to 1.499 seconds while macro accuracy held at 69.2 percent with no request failures. This artificial distraction and prefill test, built from a sample too small for a definitive ranking, still rejects “fastest” as the selection rule.
LocalJev supplied the necessary failure case. Choosing the smallest checkpoint is a legitimate latency and memory strategy. It becomes a bad strategy when the service returns well-formed probability vectors for wrong answers. JSON validity, low median time, and successful loading can all survive while semantic quality fails. A longer-input gate does not need to mimic production perfectly to expose a candidate that should not become the default.
These results cannot be ranked against one another. They use different models, hardware, runtimes, workloads, and metrics, and none was independently reproduced here. Their shared value is the reporting contract exposed by their different failure modes.
Compute:Arena’s September 22 launch supplied the missing measurement discipline. Each result identifies the exact model file by SHA-256, namespaces the quantization format, normalizes the chip while preserving the runtime’s original name, and hashes the runtime executable. The harness sweeps prefill from 128 through 16,384 tokens, measures decode over 128 tokens, performs warmups, and records three repetitions. Signed reports retain raw timings, token counts, immutable model revisions, thermal sensor readings, power state, memory pressure, swap, and GPU snapshots (Compute:Arena methodology).
The project also names what the signature cannot prove. A signed report establishes that the artifact did not change after signing; it cannot establish that the machine reported honestly. Its llama.cpp and BaseRT adapters differ in workload order and warmup behavior, so Compute:Arena records that mismatch instead of pretending cross-runtime equivalence. That is the correct posture for a leaderboard. Comparable inputs and disclosed conditions are more valuable than false precision.
A useful optimization report now needs three verdicts. For fit, record actual resident memory or allocation peak instead of inferring it from a filename. Separate prefill, cached time to first token, decode, and end-to-end task time for speed. Test quality on the same task at short input and at the deepest context the workflow will claim. Put model and runtime hashes, raw settings, repetitions, and failures beside the headline number.
The optimization that matters is the one that still wins when fit, speed, and quality disagree.
If this was useful, forward it to one engineer who needs less noise in their feed.