Say you’ve worked through a practice calculation with an AI assistant. The answer makes sense. Then you close it, try a similar problem on your own, and realise you don’t know the next step.
That moment is worth looking into. On its own, it doesn’t tell you much. Maybe you knew the method once and forgot it. Maybe you never learned it. Maybe you’re expecting more of yourself than the task needs.
You don’t have to stop using helpful tools to find out. You just need to pick one skill you want to keep, such as explaining a calculation or checking a result, and test it.
This module is for working adults who have already picked a skill to keep and can practise it safely, on low-stakes tasks, spare copies or simulations.
What you’ll get from this module
In this module
By the end of this module, you’ll have a clearer way to answer three questions. What can you do on your own? Has that changed? And is it good enough for what you need? We’ll look at a classroom experiment with AI, research on calculators, quizzes and spaced practice, and experiments on refreshing hands-on skills. Then you’ll pick one situation from your own life and work through a written check. You’ll finish with a record of your attempt and a next practice step, or a clear note of what you still need to find out.
Ask Jing
Become a Jingius.
What happens to learning when tools help?
Classroom experiment: better practice scores can hide weaker learning.
In 2025, Hamsa Bastani and colleagues studied AI help in high-school maths classes in Turkey. Nearly 1,000 students in about 50 classes took part in four 90-minute sessions. Each class was given one of three things: the usual practice materials, a general AI assistant called GPT Base, or a guarded assistant called GPT Tutor. Whole classes were assigned, not individual students.
Students practised with their assigned help, then sat an exam in the same session without it. That let the researchers compare two things: how well students did with help, and how well they did alone afterwards.
Students with GPT Base scored higher during practice. But on the exam without help, they did worse than students with the usual materials. The better practice work gave a false picture. Students got further with help and were then less able to solve similar problems on their own.
Students with GPT Tutor also scored higher during practice. On the exam without help, they did about the same as the students with the usual materials. The researchers found no clear gain or loss. GPT Tutor’s safeguards were a mix of hints, teacher-written solutions and notes on common mistakes. The experiment didn’t test hints on their own.
Why it matters: This matters when learning is part of what you’re after. Help that gets you a better answer and help that leaves you able to solve the next problem are two different results. One point of caution: this experiment was about short-term school learning. It didn’t show adults losing a skill they already had.
Read the classroom experiment on AI assistance and mathematics learning.
Research review: learning with a calculator can still leave you able to work without one.
Aimee J. Ellington’s 2003 review offers a useful counterpoint. She combined 54 studies of school maths teaching and sorted the results by whether students were allowed a calculator during the test.
In 15 of those studies, students who learned with calculators did better on average on calculator-free tests of basic maths skills than students who learned without them. Those skills covered both doing the calculation and understanding the idea behind it.
Here the tool was there while students learned and gone when they were tested. The students still did well. So using a tool doesn’t, by itself, mean a skill is getting weaker. You can learn with a tool and still show you can manage without it.
Why it matters: The result applies to that one group of skills. Ellington couldn’t tell whether it held for every school year. An earlier review had reported that calculators set back fourth-graders’ calculation skills, and her review couldn’t confirm or rule that out. The lesson is to look at how a tool fits into your learning, instead of assuming every use has the same effect.
Read the review of calculators in mathematics teaching.
Research review: trying to remember helps you learn.
If you want to hold on to knowledge, what kind of practice works? In 2021, Chunliang Yang and colleagues reviewed classroom quizzing across 222 studies and 48,478 students.
Overall, quizzing improved results across the kinds of comparisons the studies made. How much it helped depended on things like what the quizzes were compared against and whether students got feedback on their mistakes.
The useful idea here is the difference between seeing an answer and trying to produce it. Pulling the answer from memory before you look is what does the work. Quizzing gives you that chance, and feedback helps you fix what you got wrong.
Why it matters: So for knowledge you want to keep, make your own attempt first, then check it. The evidence comes from classrooms. Applying it to something you want to remember as an adult is a sensible next step, not a proven one.
Read the review of classroom quizzing and learning.
Research review: spread your practice out over time.
Alice Latimier, Hugo Peyre and Franck Ramus asked a related question: does it help to space out your practice? Their 2021 review compared practice spread across separate occasions with practice packed into one go.
Spaced practice led to better memory. The review also compared gaps that grow longer each time with gaps that stay the same, and found no clear winner overall. That doesn’t mean every schedule works equally well.
Why it matters: This adds something to the quizzing lesson. Recalling an answer during one practice session and recalling it again a week later are different tests. Coming back later lets you practise, and see, the second kind.
Read the review of spaced retrieval practice.
Simulation experiments: match the practice to the kind of task.
Remembering facts is only one kind of skill. In 2019, Barbara Frank and Annette Kluge ran three experiments on ways to refresh a hands-on skill, using a simulated task of controlling a wastewater plant.
Beginners learned different kinds of step-by-step tasks. Then, before a later test, groups got one of four things: hands-on practice, a skill test, mental rehearsal, or nothing. Mental rehearsal means going over the steps in your head without doing them.
The researchers measured how well people performed the tasks after a delay. The results depended on the task and on what was measured. No single method came out best every time.
Why it matters: This puts a limit on the memory evidence. If the skill you want to keep is a procedure, your practice needs to cover the actual steps and the result you need to produce. These studies used beginners in a simulation, so think about how closely they match your own task.
Read the experiments comparing skill-refreshing methods.
What this means for you
You can keep using helpful tools and still find out what you can do on your own. Compare your work with and without the one aid you’re looking at. Then ask whether your own work is good enough for what the task needs. These are two different questions, and the sections below take them in turn.
Use a practice worksheet, a spare copy or a simulation. Keep any support you genuinely need, and set aside only the one aid you’re testing. That’s all ‘without help’ means here. You don’t have to strip away every kind of support. And never remove essential support while driving, doing live medical work or anything else where a mistake could hurt someone.
Decide what your check needs to show
Start with the skill you’ve already chosen to keep. Name the one aid you’ll set aside for the check. Decide how good the result has to be and how much time you have. If you’d need a backup, a second way to get the result if the aid fails, name that too.
Together, these tell you what ‘good enough’ looks like. Say your aim is to explain the reasoning behind a practice calculation. Getting the right number from software doesn’t show you can explain it. You’d need to explain the steps and check your explanation against a trusted answer or a clear marking guide.
If your aim is to check a result, then test your checking. Before you look at a suggested answer, write down what you expect the next step to be and how you’d check it. Then use a low-stakes example with a known correct answer and see whether you can spot an error.
Different aims can lead to different conclusions about the same tool. An assistant may make your finished work better while you still need practice explaining or checking it yourself.
Make the comparison fair and concrete
Try the task without the aid before you look at its answer. Note the task, the conditions and how well you expect to do. Afterwards, write down what you got right, any serious mistakes, how long it took and any help you needed. Keep the work so you can compare real attempts later.
Then take a fresh task of similar difficulty and do it with the aid. Record that too. This shows the difference between working with and without the aid. But a difference on its own doesn’t tell you whether your unaided work falls short of what you need.
To look for change over time, compare what you can do without the aid now with what you could do without it before. That earlier work is your starting point. Use tasks of similar difficulty under similar conditions. Tiredness, setting and changes in how you mark the work can all affect the result. And don’t repeat the exact same task. A remembered answer can look like a skill you don’t have.
If you have no earlier unaided work, your first attempt becomes your starting point. It can show where you are now. It can’t show how much you could do before.
Work out what the result means before you act on it
A gap and a decline are different things. A gap means you can’t yet meet the requirement without the aid. A decline means you used to do better without it. To claim a decline, you need comparable earlier work to show it.
| What your record shows | What it means for your next choice |
|---|---|
| Without the aid, you meet the requirement on the tasks you tried. | Record what you showed and under what conditions. Check again later to see what you can still do. |
| Without the aid, you miss the requirement, and you have no earlier work to compare. | Record a current gap. Find the part you struggled with and practise it. |
| Without the aid, repeated comparable attempts fall below your earlier work. | Record a possible decline, with the cause unknown. Check the conditions and try again on another day before calling it a trend. |
| Without the aid, you can’t get the result you need, and your backup fails too. | Record that this requirement currently depends on the aid. Work on the skill or the backup, whichever the requirement calls for. |
The last row is where the difficulty has a real consequence. If you need a particular result and neither your own work nor your backup can produce it, you have a specific problem to fix.
One failure doesn’t prove that a tool caused a change. Stay unsure about the cause while you act on the specific difficulty you’ve found.
Practise the part you need to show
Look at your attempt and find the part you struggled with. If you need to remember something, try to recall the answer or explanation before you look it up. If you need to carry out a procedure, practise the steps on your spare copy or simulation.
Compare your attempt with feedback you trust, fix the error and try a fresh task of similar difficulty. If you don’t have a reliable answer key or a good way to judge the work, sort that out first. Get a qualified person to check where needed before you treat a result as correct.
If you use a tutor or an assistant to help you practise, make your own first attempt and ask for help that leaves a meaningful step for you to do. Check the correction yourself before you try again without help. The classroom experiment suggests the whole set-up matters, not just asking for hints.
Checking an answer or using a backup can also be the skill you practise. If you need to keep going when the aid fails, rehearse your backup in the same low-stakes setting. Name the failure it has to handle. Then ask whether the backup still relies on the very support you’re imagining gone.
If it does, the backup may fail for the same reason as the original aid. Write down what actually worked and what you’re still unsure about. Having a backup is not the same as showing it works when you need it.
Come back after a gap and adjust
Pick a separate date to check again, before the next time you’ll need the skill without help. Use fresh comparable work and the same requirement. If you failed early, or you need the skill often, check sooner.
If your result drops after the gap, add feedback and practice and check again after a shorter gap. If you succeed after the gap on fresh tasks, you can cautiously lengthen the next gap if that’s useful.
The longest gap you’ve succeeded after tells you what you could still do on those tasks. It doesn’t guarantee you’ll manage next time. Use your results to plan the next check and carry on with your ordinary assisted work in the meantime.
If the task itself changes, revisit the requirement. Keep your practice focused on the skill you chose to keep.
How about your daily life?
Where would you like clearer evidence of what you can still do yourself? Perhaps you’ve already chosen to explain a calculation, recall an important idea or complete a familiar sequence without a particular aid.
Or perhaps you want to check a tool’s answer, or rehearse a backup you’d need. Choose a situation you can explore on a practice copy, worksheet or simulation while keeping any support you genuinely need.
Write down one or more possibilities. One good situation is enough.
| A situation in my daily life | The skill I’ve already chosen to keep |
|---|---|
| ____________________ | ____________________ |
| ____________________ | ____________________ |
Which situation would be most useful to explore now?
My chosen situation: ____________________
You can change your mind. Carry your choice into the exercise below. Choosing it doesn’t yet tell you whether there’s a gap.
Check and practise one skill you want to keep
Start with your chosen situation. If you came straight here, pick a skill you already want to keep and a low-stakes task, practice copy or simulation where you can test it.
Keep any support you genuinely need. Set aside only the one aid you’ve named. Never remove essential support while driving, doing live medical work or anything else where a mistake could hurt someone. If you can’t arrange a safe practice setting, stop here and work out what supervision or other setting you need.
- Set the requirement before you test. Name the skill, the aid you’re setting aside and the task conditions. Write down how good the result needs to be, how much time you have and any backup you’d need if the aid fails. Use the real task’s requirements.
Skill, aid, conditions and requirement: ____________________ - Make your first attempt and keep it. Before you look at any suggested answer, write down what you expect to get, what you think the next step is and how you’ll check it. Do the task without the aid. Record what you got right, any serious mistakes, how long it took and what help you needed.
Check it against an answer key, marking guide or qualified person you trust. If you have none, note that the check is missing and leave the result open. If you have no earlier unaided work to compare, treat this attempt as your starting point.
Task version, date, prediction, saved work and checked result: ____________________ - Compare, and test your backup. Do fresh work of similar difficulty with the aid and record the result. Compare it with your unaided attempt and describe the gap between the two.
If you’re testing your ability to spot errors, use a low-stakes example with a known correct answer. If you need to keep going when the aid fails, rehearse your backup in the practice setting.
Ask whether the backup still relies on the support you’re imagining gone. Record the actual quality, time, errors and help, plus anything you couldn’t verify.
Assisted comparison, checking result and any backup result: ____________________ - Choose the reading your evidence supports. If your unaided result meets the requirement, record “kept, on the tasks I tried”. If it misses the requirement, record “gap”.
Record “possible decline, cause unknown” only when repeated comparable unaided attempts fall below your earlier work. Test again on another day before calling it a trend.
If you can’t get the result you need without the aid and your backup fails too, record that the requirement currently depends on the aid. A first failure on its own doesn’t prove you’ve lost a skill or why.
Reading, comparison used and open questions: ____________________ - Practise the hard part, with feedback. Recall the knowledge or carry out the steps before you look for help. Check against feedback you trust, fix the error and try fresh work of similar difficulty.
If you use a tutor, make your own first attempt and leave a meaningful step for yourself. Check its correction yourself. If checking or recovering is the skill you’re after, practise that directly.
Hard part, correction and fresh attempt: ____________________ - Set a later check. Pick a separate date before you next need this skill without help. Use comparable work and conditions. Bring the check forward if you failed early or if you need the skill often.
If your result drops, add practice and feedback and shorten the gap. If you succeed after the gap on fresh tasks, cautiously lengthen it if that’s useful. Revisit your requirement if the task changes.
Next date, task and what would make me check sooner: ____________________
Ask Jing
Become a Jingius.
Your next step
You’ve finished this exercise when you’ve saved your notes, a reading your evidence supports and a date for the next check. If you still lack a safe setting, a way to check answers or a working backup, finish by naming that gap and the next piece of information or help you need.
A note on the evidence
These findings come from different tasks and settings. The classroom AI experiment measured short-term learning in school. The calculator result covers one group of school maths skills. Neither tells you what will happen with every tool or every person.
The memory reviews and the simulation experiments speak to different kinds of practice. Keep that difference in mind when you choose how to practise, and read your own comparisons within the tasks and conditions you actually checked.
How we prepared this module
This module brings together a classroom experiment, research reviews and simulation studies to explain the difference between doing work with help, keeping a skill, and practising it. A made-up practice-calculation moment introduces the idea. The written exercise helps you examine a skill you’ve already chosen to keep.
Sources
- Hamsa Bastani, Osbert Bastani, Alp Sungu, Haosen Ge, Özge Kabakcı and Rei Mariman. (Online 25 June 2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122. Undated author-hosted manuscript; §2, pp.5–8; §3 and Table 1, pp.9–10.
- Aimee J. Ellington. (November 2003). A Meta-Analysis of the Effects of Calculators on Students’ Achievement and Attitude Levels in Precollege Mathematics Classes. Journal for Research in Mathematics Education, 34(5), 433–463. Published-layout article; operational-skills explanation p.442; Table 4 and discussion pp.443–444; grade-specific limitation p.457.
- Chunliang Yang, Liang Luo, Miguel A. Vadillo, Rongjun Yu and David R. Shanks. (April 2021; online 8 March). Testing (quizzing) boosts classroom learning: A systematic and meta-analytic review. Psychological Bulletin, 147(4), 399–435. Author abstract, study count and results paragraphs.
- Alice Latimier, Hugo Peyre and Franck Ramus. (2021; online 7 October 2020). A Meta-Analytic Review of the Benefit of Spacing out Retrieval Practice Episodes on Retention. Educational Psychology Review, 33, 959–987. Publisher abstract, comparisons and results; typography correction recorded 29 June 2021.
- Barbara Frank and Annette Kluge. (2019; online 21 December 2018). Is there one best way to support skill retention? Putting practice, testing and symbolic rehearsal to the test. Zeitschrift für Arbeitswissenschaft, 73, 214–228. Published HTML; §2 “Method”, §3 “Results” and §4 “Discussion”.
