When you build an automation, you usually start with a dry-run — a rehearsal pass that changes nothing and only reports what it would process. And when that rehearsal ends with "nothing to process," it feels like a success. No errors, after all.
But in that state, nothing has actually been verified. The path where everything goes right, from input to final result, is called the happy path — and you've never once seen your pipeline actually travel it. This post walks through a technique for exactly this situation: seeding a single test document and verifying the automation end-to-end.
A zero-candidate dry-run proves nothing#
When a dry-run ends quietly with zero candidates, it means one of two things: there really was nothing to process, or the candidate-detection logic itself is broken. You can't tell which until you feed it a real candidate.
That's where seeding comes in: deliberately create one input that satisfies the automation's processing conditions, and make it the pipeline's first real target. For example, if your pipeline scans markdown notes and publishes them to an issue tracker, you'd create one test note that meets the publish conditions.
One detail matters here: pin down where the result must appear, right in the test document. Success isn't "an issue got created" — it's "an issue got created in the specified project." That way you also catch bugs where output lands in the wrong place.
Write acceptance criteria as observable artifacts#
Put the acceptance criteria inside the test document, as a checklist. For the note-publishing pipeline above, the criteria might look like this:
- Output created — a new issue appears in the designated project
- Source state updated — the test document's status field changes to "published"
- Metadata recorded — the created issue's ID, URL, and an updated timestamp are written back to the document
- History evidence — a publish commit lands in the remote repository
- Skip on re-run — running the same automation again skips this document with an "already published" reason
Notice what they have in common: every one is an artifact you can see with your own eyes — an issue URL, a status-field diff, a commit in the log. When observable evidence is the standard instead of "it seemed to work," there's no room for interpretation about whether the verification passed.
Why the re-run check matters#
The fifth item is especially important. Automation doesn't run once and stop — it runs repeatedly. If the second run processes the same document again, duplicate outputs start piling up. The property that processing the same input multiple times yields the same result as processing it once is called idempotency — and a proper happy-path verification should include it: "does it quietly skip when run one more time?"
Record what's out of scope, and the cleanup plan#
It's also worth stating in the test document what the verification does NOT cover. For example: processing any other inputs, or the follow-up lifecycle of the created output (closing, assigning, and so on) are out of scope. A narrow scope makes it unambiguous whether the verification is done.
Write down the cleanup plan ahead of time, too — whether the test document and its output get archived or deleted afterward. That keeps test artifacts from drifting around among your real data.
Wrapping up#
End-to-end verification of an automation comes down to this:
- A zero-candidate dry-run isn't a success — it's an unverified state
- Seed a test document that satisfies the conditions and make it the first candidate
- Write acceptance criteria as observable artifacts (issues, diffs, commits)
- Verify the skip on re-run so repeated execution is safe
- Record what's out of scope and the cleanup plan in the document
Next time you build an automation, don't shrug when the dry-run ends quietly. Seed one test candidate and push it all the way through. You have to see the pipeline actually do its job at least once before you can trust it.
Remember that failure is an event, not a person.
— Zig Ziglar
