A patch that fails review is work performed, but it is not work delivered. Any account of AI coding productivity that loses that distinction will reward the wrong thing. The model can generate another version faster than a maintainer can decide whether the first one belongs in the repository. Counting those versions tells you how busy the workflow was. It leaves the useful result unresolved.
The measurement boundary should sit after acceptance. Start with one task whose completion a person can judge, then retain the effort spent on every attempt to reach it. Rejected changes belong in that record because they consumed attention. They do not become extra completed tasks simply because they arrived as separate patches.
Consider a fictional inventory validator asked to reject a blank product identifier while preserving valid rows and their order. This is a proposed measurement exercise, not a report of a production result. The maintainer freezes that requirement and its expected cases before either a human or an assistant attempts the change. The accepted unit is one bounded behavior change supported by those checks.
An assistant’s first proposal also sorts the inventory. The maintainer rejects it because sorting changes behavior outside the request. A revised proposal removes the sorting but lacks evidence for the blank-input case. It remains under review. A later candidate supplies the required results, and the maintainer accepts that exact revision. The ledger contains several transitions and one accepted task.
That distinction needs an identifier that survives revision. Keep the task identifier stable while candidate identifiers change. Otherwise, each retry can masquerade as a new delivery. Preserve the rejected candidate and its reason beside the replacement. The question at the end is whether the original requirement was satisfied, not how many artifacts accumulated along the way.
Activity has a place in measurement, but it cannot carry the whole conclusion. The authors of the SPACE framework argue that developer productivity spans multiple dimensions and cannot be measured with one metric. That supports treating generated output as one observation among others. It does not validate any particular AI tool or establish savings for this inventory task.
For this narrow exercise, I would keep a measurement sheet with the task, acceptance rule, candidate revision, attempt, disposition, evidence, and reviewer decision. Beside those fields, record preparation, implementation, rework, verification, and human review effort. Leave unobserved values blank and explain the gap. A blank review-time cell means the review effort was not recorded; it does not mean the review took no time.
The sheet needs two clocks. Elapsed time answers when the task became usable, including waiting for review. Active effort answers how much attention people spent on it. Record model execution separately so unattended waiting does not become human labor by arithmetic accident. If a developer reviews another change while a model runs, adding both durations produces a misleading account of their working time.
The boundary also needs a beginning. Start when the assigned task enters the agreed workflow, including the preparation needed to make the request usable. If the assisted route needs a reproduction packet, count its preparation. If the baseline needs the same packet, count it there too. Excluding setup from one route while charging it to the other builds the conclusion into the measurement.
A blank worksheet is useful only if the states have consequences. In this inventory example, the sorting proposal receives a rejected disposition and no accepted-output credit. The candidate without blank-input evidence stays pending. The accepted revision points to the frozen cases, observed results, and maintainer decision. All effort remains attached to the same task, including time spent explaining why the first proposal exceeded scope.
The baseline must face an equivalent job. A hand-written predicate in a familiar repository is a poor comparator for an assisted repair in an unfamiliar importer with missing tests. Match the task shape, existing checks, reviewer standard, and available context as closely as practical. Record meaningful differences before looking at the result. If the baseline is missing, the honest output is a description of the assisted workflow’s cost and outcome, with no savings claim.
Repeating the identical task introduces another complication: the person already knows the solution. Switching tools for the second pass does not erase that knowledge. Use comparable fresh tasks where possible and record order and prior familiarity. A small local comparison can inform the next experiment without proving that the tool caused every difference. Keep the conclusion no broader than the comparison permits.
There is a reasonable objection here. Recording every minute can make a small patch more expensive to evaluate than to write. The answer is a bounded observation window and a short sheet, not a second bureaucracy. Select a few task shapes that matter to the team, agree on the fields in advance, and collect enough detail to find where effort moved. Do not turn a trial worksheet into a permanent ranking of individual developers.
Acceptance itself can move if nobody protects it. A team might accept a narrower result after repeated failures, then compare it with a baseline that met the original requirement. Record that as a scope change and evaluate the new unit separately. The same applies when a reviewer quietly waives a check to close the task. A faster decision under an easier standard is not evidence of faster delivery under the original one.
Later defects deserve a visible place without making measurement endless. Choose a follow-up window before the comparison, and attach reopening or corrective work to the accepted task when it falls inside that window. Report acceptance at the initial cutoff and subsequent rework separately. The initial decision remains historical evidence; it does not promise that the change will never need repair.
The result may be mixed. An assisted route could reach acceptance sooner while consuming more reviewer attention. It could reduce implementation effort while leaving elapsed time unchanged because the review queue dominates. Those are different operational choices. A lead who needs faster response may value the first differently from a maintainer whose review capacity is already exhausted. Keep both observations available for that decision.
Before expanding the workflow, inspect the rejected work and the missing cells as carefully as the accepted count. They show whether the next experiment needs a better task boundary, clearer evidence, or a different allocation of review effort. The useful number starts with the work a maintainer was willing to keep, and carries the cost of getting it there.
Use the SPACE research overview to challenge the metrics in your next bounded workflow comparison.
If this was useful, forward it to one engineer who needs less noise in their feed.


