Fit the decision mapping without pretending the score became truth.
A decision can improve without the model learning a new fact or the prompt acquiring another paragraph. Sometimes the useful change is how the application converts existing evidence into an answer. DSPy’s experimental ReAnchor optimizer makes that distinction concrete: keep the instructions and demonstrations, fit the local decision settings, and judge the resulting program against a task metric. That is a narrower and more inspectable intervention than asking another optimizer to rewrite what the model should do.
The release boundary matters here. ReAnchor arrived in DSPy 3.4.0 on September 25, 2026. As checked on October 21, the latest published package is 3.4.1, matching the latest stable GitHub release, published October 12. This review reads the 3.4.1 tagged implementation, not whatever main happens to contain next. It is a source-and-documentation review, not a local benchmark or a claim that a tuned production system has passed evaluation.
The decision-type contract at that tag separates evidence from its decoded consequence. Noul supplies a Boolean decision from a true-probability and a threshold. Score represents an ordered distribution; cuts select a discrete level while leaving its continuous value unchanged. Choice applies local multipliers to option probabilities before selecting a member. Changing one of those settings changes the answer the program uses without necessarily changing the underlying model response. None of that establishes that the evidence is correct.
ReAnchor fits those settings against the metric for the whole program, rather than optimizing an isolated field’s agreement by assumption. The released optimizer entrypoint takes a student program, training examples, and optional validation examples through compile. It returns a copy and leaves the original program unchanged. Its report records before-and-after scores and fitted-field information; validation examples are scored but not used to fit settings. A saved report is useful evidence about that particular optimization run, not permission to deploy it.
Consider a hypothetical support router with separate outputs for urgency, severity, and destination. This is a proposed example, not a system I tested. A threshold can make the urgency branch more conservative. Score cuts can change which severity band receives a case without changing the continuous score. Choice weights can change the selected destination. The product question is whether those combined changes improve the operating objective while preserving mandatory exclusions and a usable nonautomatic path. A router that improves average routing agreement by losing urgent cases has optimized the wrong contract.
The search is bounded rather than a promise to find every possible optimum. In the tagged calibration implementation, candidates come from observed decision evidence. Boolean boundaries are tried between observed probabilities, Score boundaries between mean level indexes, and Choice multipliers around points where the selected option changes. Large candidate sets are thinned. This is a practical search over local decoding settings, not a new training procedure for the upstream model or a guarantee of globally optimal policy.
Its fold check is more consequential than the word calibrate suggests. The released selection logic requires a better training score and an additional check across up to five parts of the training set. Each held-out part is scored using a setting selected on the other parts; the combined held-out result must beat the existing behavior. This discourages a gain confined to a narrow part of the data. It does not certify performance on tomorrow’s traffic or replace a final untouched evaluation.
I would keep that final evaluation outside the tuning conversation. Freeze the labels, predicted event, traffic window, program architecture, backend, and accepted error policy before comparing candidates. Keep ordinary arrivals distinct from deliberately difficult challenge cases. Inspect the changed decisions, not merely the aggregate score: which cases moved, who now owns them, and what happens to the cases the system declines? These are proposed acceptance conditions. A fold result cannot answer a question the metric never asked, and a representative-looking dataset cannot repair an undefined label.
Caching is an execution requirement with limits. The tagged API contract requires it by default because numeric settings stay outside identical requests, allowing cached evidence to be reused. An upstream decision can nevertheless change a downstream input or cause another predictor to run, creating a new request. Promoting an ordinary generative Boolean or Literal output into probability-based execution can also change the request. Budget for the complete program rather than multiplying one cached response by an imagined fixed number of candidates. Turning off the cache check does not make the resulting calls free.
Persist the decision policy with its context. The decision documentation places explicit overrides in predictor fields and preserves them through normal save/load. Omitted settings resolve through the installed version’s defaults. A saved program therefore belongs beside a package pin, compatible signature architecture, evidence snapshot, metric revision, and report. In the proposed router, a deployment review should compare those items before loading a replacement. Identical prompt text is not evidence of identical operating behavior after a dependency or model change.
Do not confuse main with the package you install. PR 10648 merged on October 19 and moves Jev onto the ordinary LM path while removing the older TypeSafe client surface. That is later than the 3.4.1 release reviewed here. Meanwhile, PR 10590, proposing in-memory replay during calibration, and PR 10536, extending ReAnchor and Flex, remain open at this check. Their proposed interfaces and author-reported results are not released capabilities or measurements reproduced for this article.
The usable surface is deliberately narrower. ReAnchor remains experimental, and the tagged reference directs decision outputs toward Predict-based programs rather than RLM decision outputs. Errors in the program or metric stop calibration; malformed generative evidence is not an invitation to count only the successful examples. DSPy’s 3.4.1 compatibility notes also reject null Score descriptions, including affected saved states from 3.4.0. Check the actual environment and saved artifacts before treating a patch upgrade as mechanically irrelevant to this path.
Most importantly, a task-metric gain is not proof of probability calibration. Moving a Noul threshold does not change the underlying true-probability. Moving Score cuts does not repair its distribution. A reweighted Choice can select a different member without making the retained confidence a calibrated probability of the new choice. The released decision-value implementation explicitly separates those meanings. The operational inference is straightforward: assess reliability and accepted-case errors on relevant held-out outcomes, and keep authority outside the score. ReAnchor is worth evaluating when the instruction is adequate but its decision mapping is not. Make the tuned settings earn their place without pretending they changed what the model knows.
Subscribe to Signal Over Noise for practical analysis of AI agents and automation: what to test, which constraints matter, and when a tool is worth using. Bring the next idea to one bounded workflow and check the result before expanding it.
If this was useful, forward it to one engineer who needs less noise in their feed.