Concept

A tool can improve your work, and some kinds of help can improve learning. The useful question is what got better.

Issue

A finished answer can look like understanding, even when you haven’t learned how to reach it.

Solutions

Decide what thinking you want help with, then check what the aid enabled.

Say a spreadsheet has worked out several possible outcomes for you. The rows are ready, but you still don’t know which assumption changes the decision. The calculation is done. The question you wanted to answer is still open.

The line to draw is between getting a result with help and understanding the reasoning behind it. You can value the work a tool does while asking what you now understand, can explain or can do. Those benefits are related, but they need different evidence.

This module is for working adults who use outside aids on clear tasks with results they can inspect. It focuses on tasks where you have a clear goal for your reasoning or learning.

What you’ll get from this module

In this module

By the end of this module, you’ll have a clearer way to judge what help from a tool actually gave you. We’ll link classroom trials, a workplace study and experiments on memory and AI advice. You’ll choose an everyday situation, name the work an aid does and check what that help let you work out or learn.

What improves when an aid does part of the work?

Classroom trial: better practice answers can hide weaker learning.

Hamsa Bastani and colleagues’ 2025 study compared two AI tools with a group that had no AI, in one Turkish high school. Nearly 1,000 students took part across four maths sessions. Students in the main sample had been placed in classes at random. Each class then used one of the three study setups.

Both AI tools raised scores during practice with help. But students who used GPT Base, the tool built to give answers, scored worse than the control group on the later exam taken without help. So a finished practice problem gave a misleading picture of what they could solve on their own.

The other tool, GPT Tutor, used material from teachers and hints designed to support learning. The researchers found no statistically significant difference between its exam scores and the control group’s. So the study didn’t show either better or equal exam scores for that group.

Why it matters: The contrast is about these ways of teaching and short-term exams. It shows why the way help is given matters when learning is the goal. Better practice work and better work without help have to be judged separately.

Read the classroom study of AI assistance and mathematics learning.

Classroom trial: designed assistance can improve learning.

A positive result comes from Greg Kestin and colleagues’ 2025 study in Harvard’s introductory physics course. Students had both a specially designed AI tutor and an active-learning class, across two lessons. They were assigned at random, while keeping their usual working groups together.

Test scores straight after the lesson were higher with the tutor. The study compared two whole ways of teaching: tutoring at each student’s own pace outside class, and learning together in class. The carefully designed tutor was part of that comparison.

This finding balances the earlier contrast. Help can support learning, not just help someone finish an answer. But the benefit belongs to the designed setup studied here, at one university over two lessons.

Why it matters: Simply asking a tool for hints is not the same setup. The practical question is whether the help leaves real problems for you to solve, and gives useful feedback as you work through them.

Read the physics study of designed AI tutoring.

What did the help really give you?

Workplace study: faster work and signs of learning.

Erik Brynjolfsson, Danielle Li and Lindsey Raymond’s 2025 study looked at 5,172 customer-support agents as an AI assistant was slowly brought into their workplace. Workers with the assistant solved more problems per hour. The researchers used that slow rollout to compare performance. They didn’t put workers into groups at random, as the classroom trials did.

The researchers also looked at outages, when the assistant’s suggestions weren’t available. Workers who had used the assistant before handled chats faster than they had before they got it. That pattern suggests they may have learned by using it.

The outage analysis measured how long chats took, not whether those chats were solved. Outages weren’t necessarily random, and the estimates weren’t precise. The pattern suggests learning, but doesn’t show it with the same confidence as the result on productivity.

Why it matters: The workplace results add to the chance that people can learn while using help. They also show why the measure matters: faster work with support and what someone can still do without it are different questions.

Read the workplace study of generative AI support.

Memory experiments: knowing you need to remember can make a difference.

Sandra Grinschgl, Frank Papenmeier and Hauke S. Meyerhoff’s 2021 experiments asked students to copy visual patterns. They could look back at a display instead of keeping the whole pattern in their heads. Using an outside aid like this is called offloading. In some conditions, it helped students finish the task more quickly, but they remembered less of it afterward.

In the third experiment, some students could choose how much to rely on the display. Others had to rely on it as much as they could. Some knew a memory test was coming. Among those who had to rely heavily on the display, knowing about the test mostly removed the memory gap. The results were mixed when students could choose how much to rely on it.

The memory test happened in the same session. The study suggests that having a learning goal can matter even when an outside aid is still involved. It doesn’t tell us how often workers use the effort they save to think something else through without being asked.

Why it matters: That difference matters for a task like the spreadsheet example at the start. Finishing the calculation gives you a chance to look at the decision. Whether you take that chance is another question.

Read the experiments on offloading and subsequent memory.

Interface experiment: checking takes more than seeing an explanation.

Zana Buçinca, Maja Barbara Malaya and Krzysztof Z. Gajos’s 2021 experiment used made-up AI advice about swapping ingredients in meals. It compared designs that made people think before seeing the advice with designs that just showed an explanation beside it. The final analysis included 199 adults recruited through an online work platform.

People using the designs that made them think accepted fewer wrong AI suggestions than those shown the simple explanations. They still accepted some wrong advice. The designs that cut this reliance the most got lower ratings, and the benefits differed with how much people liked to think things through.

Why it matters: Seeing an explanation isn’t the same as thinking through whether it is right. This experiment shows why that matters, and also shows that extra effort has costs. Advice that matters to a decision needs looking at. Showing an explanation doesn’t finish that check.

Read the experiment on engagement with AI advice.

Check advice before you take it

What this means for you

Start with what you want the help to do. What will the aid handle, and what should that leave you able to work out? Then choose a check that can show whether you did.

Choose the outcome before judging the aid

If you want a finished task, judge the quality of the result with the aid there. If you want to learn, judge your understanding. A polished answer alone can’t settle both questions.

You might be able to explain why an answer follows, yet still need help to do the task. Explaining the answer shows something about your understanding. It doesn’t show that you can do the task on your own. Describe the result your check actually supports.

What you want to assessWhat to look at
Better assisted workWhether the result made with help meets what the task needs
LearningWhat a suitable test or explanation shows you understand
Unaided capabilityWhat you can do without that help, and when you checked
Combined performanceWhat you and the aid get done together on the set task

The classroom studies make this difference clear. They tested learning beyond the practice answer given with help, and the results depended on how the teaching was set up. Choose help built for learning when understanding is your goal, and keep real work for yourself to solve.

Give the saved work a purpose

Name a question the aid should help you answer. For the made-up spreadsheet, the aid does the sums so you can look at which assumption changes the decision. More rows don’t answer that question on their own.

Work through the comparison or explanation you named. If the result doesn’t help you settle it, write down what is still missing. This keeps the thinking you meant to do in view, instead of letting the amount a tool produces stand in for it.

In the memory experiment, knowing about the test made a difference when students had to rely heavily on the display. That gives you a reason to name your learning goal. You still need to check what you learned.

Use a check that could reveal a problem

Check an answer that matters to your decision against evidence, another option or a requirement that could show it is wrong. A check is useful when its result can change your conclusion. Reading a smooth explanation without making that comparison leaves the key judgment unfinished.

In the made-up spreadsheet, look at the assumption behind a conclusion and compare the outcomes that depend on it. If changing that assumption changes the decision, look closely at the assumption before you accept the conclusion. Explain what evidence supports it. If you can’t confirm it, leave the decision open.

For learning, look at how well you understand the reasoning, rather than counting finished answers. To judge the quality of the output, check whether it meets what the task needs. These checks do different jobs, just as chat length and solved problems did different jobs in the workplace study.

Checking takes effort. The interface experiment suggests that thinking first can cut some mistakes. It also shows that it doesn’t remove them, or help everyone equally. Aim your check at the claim that matters to your decision.

Report the result you actually observed

You may finish with a useful result made with help, a clearer explanation or an open question. Write down which one you have and what supports that description.

In the spreadsheet example, you might explain why an assumption changes the decision and check that explanation. That shows how the calculation helped you reason through the choice. A bigger sheet only shows that more output exists. Keeping those descriptions apart helps you decide what is left to do.

If you claim you can do it without help, say what you did without help and when. A result straight away doesn’t show a lasting skill. You don’t need to turn every useful session with a tool into a claim about getting smarter.

What do you want your tools to leave you with?

How about your daily life?

Where would you like an aid to support a question you still need to think through? You might use a saved reference while comparing options, a spreadsheet while looking at an assumption or learning help while you understand an explanation.

Write down situations with results you can inspect and a goal for your reasoning or learning. One good idea is enough. Add another if it helps you choose.

A situation in my daily lifeWhat I want to resolve or understand
________________________________________
________________________________________

Which situation would be most useful to look at now?

My chosen situation: ____________________

You can change your choice. Carry this situation into the exercise below.

Check what the aid enabled

Start with your chosen situation. If you came straight here, choose a clear everyday task with a result you can inspect. Give it a clear goal for your reasoning or learning.

  1. Name the work and what you want to learn or resolve. Complete this sentence: “This aid handles ___ so I can resolve or understand ___.” Keep the aid’s work and the thinking you mean to do separate.
  2. Do the thinking you named. Work through the comparison, explanation or learning task. When learning is your aim, use help that leaves real problems for you to solve and gives feedback. Write down what you worked out or what is still unclear.
    My comparison, explanation or unresolved question: ____________________
  3. Check the result. Compare important output with evidence, another option or a requirement that could show an error. If you’re judging learning, look at your understanding rather than the finished answer. If information you need is missing, write down that gap instead of treating the check as complete.
    What I checked and what it showed: ____________________
  4. Describe the result accurately. Say whether you judged work done with help, understanding, what you can do without help, or what you and the aid did together. For a result without help, say what help was missing and when you checked. Don’t call a result straight away lasting learning.
    The result I can support, or what remains unresolved: ____________________

Ask Jing

Become a Jingius.

See what the help let you do

Loading interactive tool…

Your next step

You have finished when you can describe what the aid handled and what your check showed about your own thinking. An open gap is a useful result when it shows what still stands in the way of a sound conclusion.

A note on the evidence

These studies are about particular ways of learning, particular jobs and experimental tasks. The classroom results are short-term, and the sign of learning at work comes from outages that weren’t random. The memory and advice experiments explain useful conditions, without showing how often thinking gets better in everyday work.

The important difference is between the outcome you want and the outcome you checked. Useful help can take several forms. Its value needs to be described in terms of the result you can support.

How we prepared this module

This module links classroom, workplace, memory and decision-tool research to a practical exercise. A made-up spreadsheet example shows why finishing a calculation is different from understanding what it means for a decision. The exercise helps you apply that difference to one task.

Sources