Concept

Familiar worries about new tools can lead to useful questions. Answer them by looking at the task and how you use the tool.

Issue

A tool can give you a better result and still leave an important question open: what can you still understand or do yourself?

Solutions

Separate what the tool helps you make from what you need to understand or do, then check one way of working.

Say you’ve used an AI tool to prepare an estimate. The result looks useful. Then someone asks you to explain an assumption behind it, and you realize you can’t explain that part yet.

Then you hear two confident opinions. One says tools like this will destroy people’s skills. The other says people always worry about new technology, so there’s nothing to worry about.

Neither opinion answers your real question. You need an estimate that does the job. You also need to understand it well enough to explain the decisions you’re responsible for.

This module is for working adults choosing how to use a tool for a task where both the result and their own understanding or skill matter.

What you’ll get from this module

In this module

By the end of this module, you’ll have a clearer way to judge whether a tool is helping with your work. You’ll see what earlier tools can teach you, and why their history can’t predict your own result. A historian’s study, a museum account, research on calculators, a review of how people use outside aids to think and a classroom AI experiment show the difference. You’ll use what they found to pick a situation from your daily life and work through a short exercise. You’ll finish with a written decision or a specific question to look into.

What earlier tools help us understand

A historian’s study: worry and useful work can happen together.

Ann Blair’s 2017 study looks at how the scholar Conrad Gessner used printing. He worried that there were too many books. At the same time, he supported printing and libraries as ways to keep learning safe. He also used his own books to ask other people to send material for his research.

Gessner could put printing to work and still have concerns about it. He could value what it made possible and still question the material it spread.

Why it matters: His story shows worry and use side by side. It is not a simple change from alarm to acceptance. It describes one scholar’s work. It doesn’t measure how printing changed everyone’s abilities.

Read the historical study of Gessner’s printing practices.

A museum’s history: a tool can give people a real reason to buy it.

A Computer History Museum account describes VisiCalc, the 1979 spreadsheet program from Software Arts. Many customers bought an Apple II computer just to run it.

The spreadsheet gave these customers something they wanted to do with a computer. That reason for buying belongs in the history, next to any worries about the technology.

Why it matters: The account doesn’t tell us that nobody objected to spreadsheets. It does show what a history built only from warnings would miss: people also wanted the useful work a tool made possible.

Read the museum account of VisiCalc.

Research reviews: which skill was tested, and how, both matter.

Why people take up a tool is one question. What happens to their skills is another. In 2003, Aimee J. Ellington combined 54 studies of calculators and learning, from kindergarten through Grade 12. She kept two things apart: using calculators during lessons, and using them during a test.

On tests taken without calculators, students who had been taught with them did slightly better on average at what Ellington called operational skills. These tests mixed calculation with understanding of mathematical ideas, and gave one result for both.

Computational skills meant calculation alone. Here, the result was unclear. The likely range ran from a small disadvantage to a small advantage. That leaves both open. It doesn’t show that the two groups did equally well.

A broad claim about mathematical ability could hide this difference. A small benefit on the mixed tests doesn’t settle what happens to calculation alone. Results also differed from study to study.

An earlier review by Ray Hembree and Donald J. Dessart, published in 1986, combined 79 reports. Its summary said calculators seemed to help average students’ basic pencil-and-paper skills when they were used alongside normal teaching. Grade 4 was the exception. There, steady calculator use seemed to hold back basic skills. Ellington’s later review could neither confirm nor rule out that exception.

Why it matters: Ellington also didn’t have enough evidence on whether skills lasted over time, or carried over into other mathematical work. To make sense of these findings, keep three things together: which skill was measured, how calculators were used in teaching, and whether they were allowed in the test.

Read the calculator-learning synthesis. Read the earlier calculator study’s abstract.

Does one verdict on a tool tell you enough?

A research review: outside help can make you better at a task.

Evan F. Risko and Sam J. Gilbert’s 2016 review looks at cognitive offloading: using actions or outside aids to make a task less demanding on your mind. Notes and reminders are familiar ways to put some of the work outside your head.

The review describes better performance in particular tasks involving memory, arithmetic, perception and spatial reasoning. It brings together research from many areas. It doesn’t estimate one benefit that applies to every tool.

Why it matters: Finishing a task with an aid can be worth it in itself. You may still need to remember, explain or do part of it without that aid. Whether you do depends on what the job needs.

Read the review of cognitive offloading.

Say which part has to come from you

A classroom experiment: better work with help and better learning can come apart.

Hamsa Bastani and colleagues studied two AI tutors in Grade 9–11 mathematics at one high school in Turkey. The experiment ran across four sessions in the 2023–24 school year. Whole classrooms, not single students, were placed in each group.

Students practised with one of two AI tutors, GPT Base or GPT Tutor. GPT Tutor had extra safeguards. A comparison group, called the control group, used course books and notes during practice. All the students then took exams without those resources.

GPT Base raised practice scores while students had it. But compared with the control group, it lowered their later exam scores without help. The drop was 5.4 percentage points. Students did better while the AI help was there, and worse on the later exam without it.

GPT Tutor combined prompts designed to support learning with material supplied by teachers. Its students’ later exam scores were only slightly different from the control group’s. That difference was small enough to be chance, so the study did not show better learning without help. The tutor that was tested also did more than give hints when asked. A request for hints alone would not copy its design.

Why it matters: This was a short mathematics experiment in one school. It makes one difference clear: a stronger result while help is there doesn’t tell you what someone can do once that help is taken away.

Read the author manuscript of the AI tutoring experiment.

These accounts point to useful questions. What result does the tool help produce? Which skill matters? How is the tool used? They don’t show a path that every new tool will follow.

What this means for you

You can take a warning seriously and still look into a tool’s value. Start with the task. What do you want the tool to help with, and what must you still understand or do yourself?

Give the claim something concrete to refer to

When someone says a tool makes people less capable, ask what they mean. Which skill, whose work and which way of using the tool are they talking about? What result would show the problem?

In the made-up estimate example, the useful concern is whether you can explain the assumptions behind the result. That is more specific than asking whether AI makes people less intelligent. It points to something you can examine in your own work.

Also name what you’re comparing. Are you comparing two ways of using the tool? Or this attempt with a similar task you did before? Keep the task and conditions close enough for the difference to mean something. If they differ a lot, note that uncertainty before you draw a conclusion.

A precise warning is a question worth looking into. It is not a conclusion already proved. If the evidence is missing, leave the answer open. Sounding like an old worry doesn’t settle it. Neither does the lack of a quick answer.

Decide what you actually need to retain

The calculator and AI research separate what people do with help from what they can do without it. Use that difference by naming the part of your task that still depends on you.

You might need to explain assumptions, judge whether a result makes sense or do an essential part yourself. You don’t need to turn this into a test of remembering everything the tool stores.

In the estimate example, explaining why an assumption belongs in the calculation may be part of your job. Memorizing every figure may not be. Decide what the job needs before you judge how you did. For this example, explain what you assumed and why it fits the estimate. Repeating the number it produced doesn’t explain your reasoning.

A note, index or reminder can support that work. Use it to solve a specific problem. A complicated system that adds work without helping the task misses the point.

Inspect the result and the required capability separately

If you want to try a different way of using the tool, pick practice material where little is at stake, such as a practice estimate that won’t be used for a real decision. Never remove help you need while driving or during live work that matters.

First check whether the result meets the task’s needs. Then check what you must be able to explain, judge or do without the particular help you’re testing. Keep any other help you need in place.

Feeling confident about the result is a separate observation. It isn’t the same as checking its quality, or showing that you understand it.

For the made-up estimate, check whether the result meets the requirement. Separately, explain the important assumption in your own words before you look back at the tool’s explanation.

If the result is useful but you can’t explain the assumption, you’ve found a gap in this attempt. You haven’t shown that your skill has declined. A hard or unfamiliar task can also lead to a weak first attempt.

Use a fair comparison and note what is still uncertain. If you can’t judge whether the answer is good enough, work out what information or qualified help you need before relying on it.

Change how you use the tool when the outcomes call for it

If the result is useful but you can’t explain a part you need to understand, change how you use the help. You could work through the reasoning yourself before checking with the tool. Then look at the result and your understanding again.

Keep the classroom tutor result in proportion. Its safeguards included material prepared by teachers as well as prompts. Asking any tool for hints is not the same set-up.

Keep a way of working that meets both needs. Change it when the result helps but the understanding you need is weak. Stop using the tool for this task if it doesn’t meet the task’s needs. If either outcome is still unclear, write down the open question rather than treating the trial as a verdict.

You can value the help and still change its role. Gessner’s story shows that worry and use can sit side by side. The classroom comparisons give you a reason to look closely at how the help is given.

How do you judge a tool you rely on?

How about your daily life?

Where do you use help to produce something you also need to understand? You might prepare an estimate, draft an explanation or organize information you’ll need later.

List situations you’d like to look at. One good idea is enough.

A situation in my daily lifeWhat I need to understand or do myself
________________________________________
________________________________________

Which would be useful to try out on practice material where little is at stake?

My chosen situation: ____________________

Take that situation into the exercise. You can change your mind. Choosing it doesn’t mean you’ve found a problem.

Check one way of working

Start with your chosen situation. If you came straight here, choose one task where both a result made with a tool and your own understanding or skill matter.

Use practice material where little is at stake, such as an estimate that won’t be used for a real decision. Never remove help you need while driving or during live work that matters. Keep any other help you need in place.

  1. Name the job and the claim. Describe the result you want and what you must still understand or do yourself. Say whose work you mean and how the tool will be used.
    Task, use, wanted result and required capability: ____________________
  2. Choose what to examine and compare. Decide what kind of evidence you have: the work produced with help, what you can do on your own, or a feeling that you’re good at it. Name a similar task, or another way of working, to compare against.
    Evidence, comparison and unanswered questions: ____________________
  3. Try the practice task and inspect both outcomes. Check the result against what the task requires. Separately, check what you must be able to explain, judge or do without the particular help you’re testing.
    Write the two observations down separately. One weak attempt doesn’t show decline. If you can’t tell whether an answer is good enough, work out what information or qualified help you need before relying on it.
    Assisted result: ____________________
    Required capability and remaining uncertainty: ____________________
  4. Choose what to keep, change or stop. If the result helps but the understanding you need is weak, change how you use the help and check both outcomes again. If there isn’t enough evidence, leave the judgment open and name the next useful question.
    Decision, reason and next check or unresolved question: ____________________

Ask Jing

Become a Jingius.

Check one way of using a tool

Loading interactive tool…

Your next step

You’ve finished when you have a record of both outcomes and a reason for your next choice. Your decision is about this way of doing this task. A familiar story from history can’t make it for you.

A note on the evidence

The historical accounts show particular habits and the reasons people took up a tool. The calculator studies are about learning at school, and their results change with the skill and how it was tested. The AI experiment measures short-term mathematics performance with specific tutor designs.

Use these differences when you judge a claim about your own work. A classroom result can help you see what to look at. It can’t give you the answer for every task at work.

How we prepared this module

This module brings together a historian’s study, a museum account, research on calculators, a review of how people use outside aids to think and an AI tutoring experiment. A made-up estimate shows the difference between producing a result and explaining it. The written exercise applies that difference to one task you choose.

Sources