The quantity validator accepts one and a half units. The test accepts one and a half units too. The assistant has delivered a function, a test suite, and a green result. Every artifact agrees. The inventory rule still says whole units only.
That is the failure worth guarding against when you ask AI to implement and test a change. A test can preserve the implementation’s interpretation of a requirement with perfect consistency. If the interpretation is wrong, adding more assertions around it gives the mistake a more convincing wrapper.
Write the acceptance test outside the patch’s authority. Decide what the program owes its user before inviting an assistant to decide how the program should work. The distinction is about who controls the expected result. Putting a test in another file does not establish it.
Consider a fictional inventory importer reading quantities from a CSV file. Its requirement says that quantity must be a nonnegative integer. That sounds precise until someone asks whether the field containing the character four is valid. Rejecting strings would sound defensible in a function that expects an integer object. It would be a poor default for this importer: Python’s ordinary CSV reader returns fields as strings unless a conversion option is selected. The CSV documentation makes that boundary explicit.
The person responsible for the import therefore needs to choose the representation policy. In this example, the chosen policy is one or more ASCII digits after trimming surrounding whitespace. Leading zeros are allowed. Signs, decimal points, exponent notation, words, and empty values are rejected. The original field must remain available for the error report. These are illustrative product decisions, not universal rules for quantities.
Put those decisions in a small expected-results table. The field “0” should pass, as should “4.” An empty field should fail. “1.5” and “-1” should fail. “04” should pass, “four” should fail, and a four surrounded by spaces should pass. Each row should include a reason tied to the requirement. The numeric string is now an explicitly accepted input, rather than an ambiguity the implementation can quietly resolve.
The reasons are useful because they give a reviewer something to challenge. Why allow leading zeros? Does trimming change only the validation view, or does it also rewrite the stored field? Are non-ASCII digits permitted? A table containing inputs and unexplained booleans is easy to approve without discovering that two people understand the rule differently.
Resolve those questions before treating the table as authoritative. For this demonstration, four additional cases make the boundary clearer: a plus sign followed by four, four written in exponent notation, an Arabic-Indic digit four, and a field containing only spaces. All four are rejected under the chosen policy. Another importer might reasonably accept some of them. The test suite cannot make that choice on its owner’s behalf.
Now freeze the table for the implementation pass. Keep its version with the change request and let the coding assistant read it. Require a separate, visible proposal for any change to an expected result. This is a working agreement about review, not a claim that a file name or checksum prevents editing. A checksum can help detect that the table changed; it cannot establish that the original policy was right.
A local synthetic demonstration shows why that separation matters. The faulty validator converts the trimmed field to a floating-point number and checks whether it is at least zero. It accepts the decimal quantity, the explicit plus sign, exponent notation, and the Arabic-Indic digit. Against the twelve frozen expectations, it passes eight and fails four. That is a result from a deliberately small example, not a measurement of an assistant’s reliability or a report from a production importer.
The corrected validator checks that the entire trimmed field consists of one or more characters from zero through nine. Python’s regular-expression documentation describes fullmatch as matching the whole string. The explicit digit range expresses this example’s ASCII choice. The revised validator passes the same twelve cases. Separate assertions confirm that both versions preserve each raw input field.
The important evidence is the relationship between those runs. The decimal case fails against the faulty implementation and passes against the correction, with the expected outcome unchanged. The ordinary valid inputs continue to pass. Requiring every test to fail before the fix would make no sense; the baseline already handles several cases correctly. What matters is that the suite can detect the defect it is supposed to prevent.
A weak alternative would calculate the expected answer by performing the same float conversion as the validator. That test might look tidy and cover many inputs. It would also rebuild the disputed assumption inside the assertion. Prefer literal outcomes with reasons a domain owner can read. Where an independent reference implementation is appropriate, establish why it is trustworthy before treating agreement with it as evidence.
A second AI review pass can help challenge this packet. Give it the requirement and acceptance table first, and ask which outcomes are ambiguous, which boundary cases are absent, and which assertions merely repeat implementation logic. Show the patch after that review. Changing the order can keep the proposed solution from becoming the reviewer’s starting explanation. It does not guarantee independence, especially when the same model and assumptions are involved.
Treat that review as a source of questions. A proposed new case still needs an expected result chosen for a reason. If the reviewer suggests accepting decimal notation because the numeric value is whole, the owner must decide whether that is compatible with the file contract. Do not quietly amend the table to accommodate the patch and then present the new green result as confirmation of the old requirement.
Frozen expectations are allowed to be wrong. When a legitimate correction emerges, revise the requirement and the table together, record the decision, and rerun both implementations. That history should make clear whether the patch fixed a defect, the product rule changed, or both happened. An acceptance process that cannot correct its own assumptions is brittle in a different direction.
This fixture also has a narrow reach. It checks the quantity decision and preservation of the raw field. It does not establish that CSV rows are routed correctly, errors identify the right line, or the importer handles its largest supported file. Those claims need their own checks. A green quantity test should travel with a description of that boundary.
Before accepting the patch, ask four questions: who chose the outcomes, what disputed assumption does the failing case expose, did the same expectations run against both versions, and what behavior remains outside the fixture? Keep the answers beside the table. You can ask an assistant to write the function and help review the tests. The authority to decide what counts as correct must stay visible throughout the work.
Use Python’s CSV documentation to check the input representation before writing the acceptance table for your next importer.
If this was useful, forward it to one engineer who needs less noise in their feed.