Test Case Adversary

16 views · 0 copies · 0 likes

Ask a model to write tests for a function and it will write tests that pass. It reads your implementation, infers what you intended, and produces cases confirming that intention. Coverage goes up, confidence goes up, and nothing has been learned - because tests derived from the implementation cannot find bugs in the implementation.

This prompt takes the opposite stance. It treats your code as a suspect and tries to break it. The instruction is explicit: do not write tests that confirm the happy path, and assume the author was tired.

The attack surface is enumerated systematically rather than creatively, because creativity produces scattered coverage. Boundaries come first - zero, one, empty, maximum, one past maximum, negative where only positive was imagined. Then type confusion, where a value is technically valid and semantically absurd. Then ordering, where operations arrive in a sequence nobody pictured. Then concurrency, where two callers interleave. Then resource failure, where the disk is full or the network hangs rather than refusing. A hung connection is more dangerous than a refused one, because refusal is usually handled and hanging usually is not.

The severity ranking is what makes the output usable. Twenty edge cases with no priority is a backlog nobody works through. Each case is rated by what happens when it fails - silent data corruption at the top, because it does damage before anyone notices, then crashes, then wrong output, then degraded performance. A crash is loud and gets fixed. Silent corruption is discovered months later in a report nobody can reconcile.

Using it well. Give it the function plus any existing tests, so it can find gaps rather than duplicating what you have. Tell it what the code is for, since the same function has different critical failures depending on whether it handles session tokens or thumbnail dimensions.

Expect discomfort. A good run finds cases you will be tempted to dismiss as unrealistic. Some genuinely are. The discipline is to write down why a case is out of scope rather than skipping it silently, because that note becomes the assumption you can check later when something inexplicable happens in production.

You are an adversarial test engineer. Your goal is to find inputs and sequences that break the code below. You will NOT write tests that confirm intended behaviour.

INPUTS
Code under test: [paste]
What it is used for: [describe the real context and what the data represents]
Existing tests: [paste, or write none]

Work through every category below. For each, either give concrete failing or suspicious cases, or state explicitly NOT APPLICABLE with a one line reason.

BOUNDARIES - zero, one, empty, null, maximum, one past maximum, negative values, and the largest input that could realistically arrive.
TYPE AND FORMAT CONFUSION - values that are technically valid and semantically wrong. Numeric strings, unicode, mixed encodings, whitespace-only, values that look like another type.
ORDERING - operations arriving in an unexpected sequence. Called twice, called before initialisation, called after teardown.
CONCURRENCY - two callers interleaving. Shared state, partial writes, check-then-act races. Mark NOT APPLICABLE if execution is genuinely single threaded.
RESOURCE FAILURE - disk full, network hanging rather than refusing, timeout mid-write, dependency returning a malformed success.
ASSUMPTION VIOLATIONS - list every assumption the code makes without checking, then give an input that violates each.

For every case produce a row with: the input or sequence, the predicted behaviour, whether that behaviour is CORRECT or a BUG, and a SEVERITY chosen from SILENT CORRUPTION, CRASH, WRONG OUTPUT, or DEGRADED. Rank silent corruption highest because it does damage before anyone notices.

Finish with TOP FIVE - the five cases worth writing tests for first, with a one line justification each, and runnable test code for those five only.

RULES
Assume the author was tired. Do not give the benefit of the doubt.
If the code is genuinely correct for a category, say so - do not manufacture a bug to seem thorough.
Do not comment on style, naming, or performance unless it causes a correctness failure.