The automation finished. Can users verify what happened?
From building Notifly to making workflow status, runs, individual cases and failure recovery understandable.
Written by Phan Lam Duy Khoi
An automation should answer four questions: what is the workflow doing, what happened in a run, where did one case go, and what can the user do after an error? A “completed” label answers only part of that.
The question that came up while building Notifly
In the original LinkedIn post, I described running an automation for a contact and seeing a completed status. I still wanted to know which contact ran, which steps they went through, whether an email was sent and where a failure occurred.
I organised the results and history around four levels. This article develops that perspective into design questions. The scenarios and acceptance criteria below are recommendations, not a description of every current Notifly capability or a claim of measured improvement.
Separate workflow status from run status
A workflow can be enabled while one of its runs fails. A paused workflow may also have work already in progress. One shared label makes it hard to tell which object the user is looking at.
Before choosing colours and labels, agree on the object, transitions and meaning of pause: does it stop new entries, in-progress work, or both? These are product rules to settle with engineering and operations.
- Workflow: enabled, paused, or requiring a configuration change?
- Run: queued, processing, ended, or still awaiting confirmation?
- Step: succeeded, failed, skipped, or not reached yet?
Explain the scope and progress of one run
Illustrative example: a run processes 10 contacts, with 7 completed, 2 failed and 1 waiting. These numbers are hypothetical, not Notifly production data. A completed label for the whole group hides the remaining work.
Show start time, last update, scope and how each group is counted. Counts should add up only when they refer to the same unit and non-overlapping groups. If results are incomplete, mark them unknown rather than showing zero. Let users open the relevant failed or waiting cases from the overview.
Trace one contact, not just the totals
Can users find how a contact entered, when they entered, which steps happened and where they stopped? Inconsistent labels or disconnected attempts make people reconstruct the story themselves.
A proposed information model uses run, step and case identifiers with event time and result. OpenTelemetry similarly describes timestamps and trace/span identifiers for technical log context; adopting that tool is not a prerequisite. Show users a readable explanation and a reference they can share with support.
Unknown is different from failed
A timeout can leave an email-provider request unconfirmed. It does not necessarily mean failure.
AWS describes identifying repeated requests to avoid additional effects. Agree on result lookup and retry conditions with engineering. A button lock alone does not establish safety. Labels should reflect confirmed results, not assumptions about delivery.
Make recovery specific
Identify the affected step, confirmed results and permitted next action. Some errors need corrected input; others need waiting or support. They should not all receive the same retry button.
Proposed message: “The email step’s result is not confirmed. Check provider status before retrying.” It preserves uncertainty instead of declaring an unverified failure.
- Does retry cover the group, one contact or one step?
- What happens to completed work?
- Who can retry, and where is permission enforced?
- How are a second click or a reload handled?
Keep logs useful without collecting everything
For each field, ask what decision or investigation it enables, who needs access and whether a reference can replace raw content.
OWASP recommends excluding or protecting access tokens, passwords and sensitive personal data, alongside restricting log access. Define permissions and masking for customer information. An operator-facing history and technical logs may require different detail.
Agree on testable first-version criteria
Start with a run overview, case list and details for the step requiring attention, when the user need and system constraints support that scope. These are proposed criteria, not current Notifly feature promises.
- A failed contact and its affected step can be found from the overview.
- Waiting work prevents an all-successful presentation.
- An unconfirmed external result stays explicit, with risky repeat actions restricted until protection exists.
- An allowed retry explains scope, permissions and effects on completed work.
- Missing permission blocks both sensitive data and actions beyond the interface.
Observe whether users can investigate and act
Candidate signals include time to locate the relevant failure, whether operators identify the next step, support dependency and unintended duplicate external actions. These are proposed measurements, not achieved results.
Give an operator a mixed-result run, ask them to locate one case and explain the next action. If they must guess or ask engineering to interpret the status, the experience needs more work. Try the same exercise with an import, sync, export or order-processing task.