Concept

A tool can do the routine work while you stay in charge of spotting and fixing what goes wrong.

Issue

You can’t do that part well if you lack the facts, the permission or a working way to put things right.

Solutions

Learn what a useful human check needs, then plan a check and recovery.

Say your automated report arrives on time. You open it before a meeting. The figures describe an earlier period. The report ran, but it used an old input.

Now you have several jobs. Spot the mismatch. Work out why. Produce a result you can trust. Knowing where to click “run” doesn’t help with any of them.

The tool can still be doing useful work. The real question is whether your part is doable when something goes wrong.

This module is for tools you already run at work, where you’re the one who has to spot or fix the errors.

What you’ll get from this module

In this module

By the end of this module, you’ll know what makes an automated result easy to check and a fix easy to use. You’ll pick a check that means something, a person who can act and a way to confirm the fixed result. You’ll choose a case from your own work and write out a plan for it. The evidence comes from automation research, decision studies, accounts of real incidents, a driving simulator study and aviation training guidance.

What does useful human intervention require?

A classic analysis: the job that is left.

In 1983, Lisanne Bainbridge looked at the work people are left with when a factory process is automated. Designers can leave people in charge of the tasks they haven’t handed to the machine. That often means watching the process and dealing with anything unusual.

Bainbridge separated two things. One is the skill of running a process by hand. The other is knowing how the process works and what state it is in right now. Someone may know the system in general and still need time to work out what is happening before they take over.

Why it matters: Her point is that people need support to do the work left to them. A person needs facts they can use and time to respond. When a process moves too fast for a person to step in, the protection may need to be built into the system itself.

Read the original analysis of automation’s ironies.

Studies and a debate: the setting matters.

The studies don’t all agree, and the setting makes a difference. In 2014, Linda Onnasch and colleagues combined the findings of 18 experiments. More automation improved routine work. But people did worse when the automation failed, and they knew less about what was going on.

Greg A. Jamieson and Gyrd Skraaning found a different pattern. They studied licensed crews in a realistic nuclear-plant simulator, published online in 2019. With more automation, the crews understood what was happening better, not worse. The expected effects on workload and on how they coped with failures did not appear.

Christopher D. Wickens and colleagues challenged that study in a 2020 reply. They questioned which system was automated and how skill and awareness were measured, among other things. Jamieson and Skraaning answered in 2020 and kept their doubts about applying the trade-off to complex work.

Why it matters: Taken together, these sources give you a reason to look at the real work left to people. Don’t assume that more automation always makes people less able. Routine gains can sit beside hard exceptions. And the cost of a failure is not the same in every setting.

Read the meta-analysis of automation levels. Read the complex-work simulation study. Read the methodological reply. Read the researchers’ response.

Decision studies: useful thinking is not the same as extra effort.

A closer question is what people do when they review a tip from a tool. In 2021, Zana Buçinca, Maja Barbara Malaya and Krzysztof Z. Gajos studied this with a simulated AI. The task was choosing a meal with less carbohydrate.

Their online study recruited 260 adults in the United States through Amazon Mechanical Turk. After exclusions, 199 people were in the main analysis. It compared simple AI explanations with designs that made people stop and think. Some comparisons were within the same people and some between groups.

One result favoured the more demanding designs when the AI was wrong. These designs are called cognitive forcing functions. When people had to name the source of the carbohydrate, estimated overreliance was .48 with cognitive forcing and .64 with simple explanations. That is a gap of 16 points.

Why it matters: That finding is about following a wrong tip on the carbohydrate source. For the full meal choice, following the AI’s wrong tip did not differ significantly between the two designs. Overall task scores across all trials did not differ significantly either. People liked and trusted the demanding designs less, but those gaps were not statistically significant overall.

Read the experiment on deliberate engagement with AI.

Decision studies: part of the reason, or all of it.

A later study helps show why effort by itself is not the goal. In 2025, Sander de Jong and colleagues compared three options: no explanation, a partial one and a full one. The tasks were judging graphs and fixing spelling or grammar.

Partial explanations led to better overall task scores than no explanation. Full explanations scored higher than partial ones, but in the graph study that gap was not statistically significant.

Overall scores and following wrong advice are two different outcomes. The paper reports that people relied less on wrong AI advice with partial explanations than with full ones. They also relied less on correct advice.

Why it matters: The practical lesson is to judge a review by the decisions it helps people make. Making someone work harder is not enough on its own.

Read the experiments comparing partial and full explanations.

Which review gives you a reason?

An official account: messages need a route to action.

Careful thinking also needs facts that reach the right place. The U.S. Securities and Exchange Commission described a monitoring failure in its 2013 account of Knight Capital’s trading incident.

Before markets opened on August 1, 2012, an internal system sent 97 automated emails about a router error. Knight Capital had not set those messages up as system alerts. Nobody acted on them.

The messages were part of a wider failure of rollout and controls. The count is of emails, not of separate faults.

Why it matters: The useful distinction is between producing facts and making action possible. A message can exist with no clear response built around it. Monitoring has to connect the problem to someone who can act.

Read the SEC’s account of Knight Capital’s control failures.

A company’s own incident account: a backup must be usable.

Spotting a problem is only part of the job. GitLab’s account of its outage on January 31, 2017 describes live data removed by accident. Then came trouble finding and using the backups.

Why it matters: Its February 2017 postmortem reported that a software-version mismatch had stopped a backup routine. Error emails were rejected. Regular backup testing had no clear owner. A staging copy made recovery possible, with some data lost.

Read the GitLab database-outage postmortem.

A company’s own recovery account: automation can help.

A separate account by Andreas Brandl, published in February 2019, describes a recovery that worked. GitLab restored a deleted label and its links to issues and merge requests using delayed replication. The company reported doing it in under 90 minutes.

These are the company’s accounts of two different events. The recovery that worked shows why automation can be part of the answer. A state you can restore gives people something real to go back to.

Why it matters: The contrast shows why listing a backup process is not enough. What matters in an exception is whether people can use the way back and check the result.

Read the GitLab account of a successful data recovery.

Explain what makes a way back usable

A simulator study: prepare for the exception.

Training can focus on the exception itself. In 2024, Shannon C. Roberts and colleagues randomly assigned 36 people to three kinds of training in a fixed-base driving simulator.

One group got a structured program of mistakes, mentoring and mastery. One got a user manual. One got placebo training. The structured-program group took back control more effectively when the level-2 driving automation reached its limits.

Why it matters: That result is about taking over in a simulator. It gives a reason to look at targeted training, while keeping the task and the training conditions in view.

Read the simulator study of takeover training.

Aviation guidance: finish the response.

A related principle appears in the Federal Aviation Administration’s 2022 guidance on flightpath management. It advises training and checking a manoeuvre through to a defined, logical endpoint. Stopping at an unrealistic halfway point is not enough.

Why it matters: The link is practical. Starting a response and finishing it are two different things. For an ordinary report, the endpoint is a corrected result that has been checked. The training and guidance come from transport settings, where professional operating procedures apply.

Read the FAA’s flightpath-management guidance.

What this means for you

Start with one tool you already run and the error you’re responsible for handling. Keep the useful routine work it does. Then make the human part of the job clear enough to carry out.

Define what would tell you something is wrong

Name a good result before you choose a check. In the report example, the report must use the period the meeting needs. The check is to compare that required period with the actual date of the input.

This gives you a concrete thing to look for. A finished run tells you the process ran. Checking the input date tells you whether it used the right period.

Find out where you can see the facts that settle the question. If you can’t reach it, write down the missing access, or ask the workflow owner for a view you can use. Choose facts that help you make the decision in front of you.

Keep a meaningful judgment at the review point

Before you accept an automated result that matters, do one of two things. Predict what you expect it to show, or look at a source value that could reveal a problem. Then compare that with the output.

For the report, checking the required period against the dated input gives you a reason to accept or question the result. Copying the report’s headline into a review box would leave that question open.

Keep the check in proportion to the decision. Ask whether it catches a problem that matters and whether the effort is workable. If it adds work without helping you judge the output, change the check instead of adding more clicks.

Connect the discrepancy to an authorized response

Decide who can stop, correct or escalate the problem. For the report, you might be allowed to replace the input and rerun a copy. Someone else may control the live workflow.

Keep those roles clear. Being expected to notice an error doesn’t give you permission to change the system.

Include the time you have to respond. If the result takes effect before anyone can look at it or stop it, ask the owner to close that gap in the design. More vigilance can’t create time or permission you don’t have.

A useful alert tells the right person what looks wrong and where to check it. Make sure the message reaches that person and that they know what they’re allowed to do.

Rehearse through a result you can verify

Use a separate copy of the workflow that you’re allowed to use. Never disrupt a live system on purpose for this exercise. That includes a live report, a trading system or a vehicle. For high-stakes work, follow the professional procedures that apply.

Put one plausible problem into the copy, such as an out-of-date report input. Spot it, explain what caused it, and choose a correction or an escalation. If the workflow really needs a manual fallback, include that fallback in the rehearsal.

Carry on through to the restored result and check it. In the report example, replacing the input is a middle step. The report still has to be generated again from that input. Rerunning the copy and checking its output against the intended source shows whether the fix reached the required result.

If you can’t explain the problem or you don’t have permission to go on, stop and record the gap for the owner. A rehearsal that shows missing access or missing knowledge has found something to repair.

Revisit the plan after a real change to the workflow or after a weakness shows up. Focus on what changed or failed. Don’t add a generic routine that avoids automation altogether.

Could you catch it and put it right?

How about your daily life?

Which automated result do you check before someone else uses it? Think about a recurring report, a scheduled message or a calendar workflow.

Where could an old input or a scheduling clash mean you have to step in? What would you like to understand better about the check or the recovery?

Write down cases that match your real duties. One good idea is enough. Add others if they help you choose.

A situation in my daily lifeWhat I would like to improve
________________________________________
________________________________________

My chosen situation: ____________________

Choose one work tool you already run, where you’re the one who has to spot or fix the errors. You can change your choice. Carry it into the exercise below. Choosing it doesn’t mean you have the permission to change it yet.

Plan a check and recovery

Start with your chosen situation. If you came straight here, choose a work tool you already run, where you’re the one who has to spot or fix the errors.

Begin with a written plan. Any rehearsal must use a separate copy you’re allowed to use. Never disrupt a live system on purpose, including a live report, a trading system or a vehicle. For high-stakes work, follow the professional procedures that apply.

  1. Name the intended result and one plausible exception. Say what the output needs to do and what could make it wrong. For a report, that could be the required period and an out-of-date input.
    Intended result and exception: ____________________
  2. Choose the evidence and judgment. Name the source or system state you can look at. At the review point, predict what you expect, or look at a decisive value, before you accept the output. Write down the problem that would make you act.
    Evidence, judgment and the sign of trouble: ____________________
  3. Identify the response and its owner. Write who can stop, correct or escalate the problem, and whether there is enough time to act. If facts, time or permission are missing, stop the step that depends on it and ask the owner to close the gap.
    Response, person allowed to act, and open needs: ____________________
  4. Rehearse the exception in the authorized copy. Spot it, explain the cause, and correct or escalate it. Include a manual fallback only if this workflow really needs one. If you can’t go on safely, record why and stop.
    What happened in the rehearsal, or what prevented it: ____________________
  5. Check recovery and choose the next action. Restore the intended state and check the output against the right source or rule. Record any open problem, missing permission or missing knowledge for the owner.
    Restored result and its check, or the open gap: ____________________
    Next repair and reason to revisit: ____________________

Ask Jing

Become a Jingius.

Plan a check you can trust

Loading interactive tool…

Your next step

Finish with a check you can explain, a response someone is allowed to carry out, and a checked recovery or a clearly recorded gap. Revisit it after a real change to the workflow or after a weakness shows up.

A note on the evidence

The automation studies give mixed results across settings. A useful result on one decision measure doesn’t show a gain on every measure. The incident accounts show failures and a recovery, and the transport sources are about specialist tasks and training.

Use the lesson to look at your real duties and the support you have. A person’s role in a workflow needs facts, time and permission to match it.

How we prepared this module

This module brings together research and theory, an official enforcement account, company incident reports and transport-training sources. A made-up report problem explains the decisions. The written exercise helps you look at a tool you already run and its way back from an error.

Sources