Concept

A familiar alarm can still contain a concern worth examining.

Issue

Calling a warning a panic doesn’t settle what the evidence shows.

Solutions

Learn to separate the warning from its support, then work through one claim.

Say you read a warning about an app you use. The next reply waves it away as another technology panic. Now you have two confident judgments. Neither tells you what actually happened, or to whom.

History can help you spot an argument you’ve heard before. It can’t answer the next question on its own: what does the evidence show this time? One warning may rest on weak evidence. Another may point to a real problem, even if its headline goes too far.

This module is for working adults trying to make sense of broad claims about what technology does to people. It isn’t a way to judge one person’s health problem.

What you’ll get from this module

In this module

By the end of this module, you’ll know how to turn a broad technology warning into a question you can check. You’ll learn what to look for in the evidence behind it, and how to explain what is still unknown. We’ll connect a way of reading history, a review of past media research, a search through old records and two studies to a warning from daily life. You’ll finish by working through one claim yourself.

What history can tell you about a warning

A way of reading history: why the argument can sound familiar.

Amy Orben’s 2020 paper describes what she calls the “Sisyphean cycle of technology panics.” Public worry gets the attention of politicians, and politicians turn to science for answers. Researchers struggle to build on what was learned about earlier technologies. Then attention moves on to something new before that research has done much to shape policy.

Orben is describing how public attention, politics and research can feed into each other. Her account helps explain why a new debate may sound familiar. It doesn’t tell us whether the harm being discussed is real.

Why it matters: That difference matters when a warning drops out of the news. People may have moved on to a new worry while the earlier question is still open. A quieter debate isn’t the same as a settled question.

Read the historical framework on technology panics.

A review of past research: repeated worries don’t always follow the same path.

When a new medium appears, do researchers ask the same questions, in the same order, every time? Jeanne M. Meadowcroft and Daniel G. McDonald tested that idea in a 1983 conference paper.

They looked at 163 studies from more than 88 sources about children and mass media, published between 1911 and 1980. They did not find a fixed order of research topics that repeated with each new medium. Studies about how children used media, and how children differed from each other, turned up all through the period.

So a familiar worry doesn’t always lead through the same set of research questions. That finding comes from the studies they chose and sorted. It doesn’t test every part of Orben’s later account of politics and public attention.

Why it matters: History can help us notice that one worry looks like another. We still need evidence before we treat that likeness as a prediction about a new technology.

Read the conference paper on the history of media research.

Old records: why some evidence against comics could not be trusted.

What would give us a real reason to doubt an old warning? Carol L. Tilley’s 2012 paper went back to the evidence behind Fredric Wertham’s case against comics. Searching his records, she reported that he had altered, exaggerated and made up evidence. That included evidence he said came from his clinical work with young people.

This gives us a reason to distrust the evidence she looked at. The problem isn’t just that an old fear sounds strange today. It’s that the evidence used to support the argument was misrepresented.

Why it matters: That correction still has a limit. Finding serious problems in that evidence doesn’t show that comics could never have had any effect. We can throw out the damaged evidence without treating every question about comics as closed.

Read the abstract of the archival reassessment of Wertham’s evidence.

Randomized trial: one specific concern was backed by an experiment.

Weak evidence behind one warning doesn’t disprove a different concern. In a 2008 trial, Leonard H. Epstein and colleagues assigned 70 families at random to one of two groups. One group joined a program to cut television and computer use. The other was only monitored.

The children were aged 4–7 and met two conditions. Their body mass index, or BMI, was at or above the 75th percentile for their age and sex. BMI compares weight with height. This cutoff picked out children with a higher BMI than others of the same age and sex. They also used television or computers for at least 14 hours a week.

The program aimed to cut that use by 50%. It used controls on the devices, rewards and support from parents. Researchers followed the families for two years. They measured changes in BMI adjusted for age and sex, which puts every child’s BMI on a scale that allows for those differences.

Over the two years, the program brought that measure down a little more than monitoring did. When researchers looked at each check-in on its own, the gap between the groups was statistically significant at six and twelve months. At 24 months, a separate significant gap wasn’t shown.

Why it matters: This is evidence for one program and one outcome in a selected group of children. It matters because it shows that a specific worry about screen use can be backed by an experiment. The result covers the whole program, so it doesn’t pick out an effect of what was on the screen. It also doesn’t set a screen limit for everyone, or tell adults how much to use their devices.

Read the randomized trial of reduced television and computer use.

Does calling it a panic settle it?

A natural experiment: a finding belongs to its original setting.

A social media study gives us another specific finding to think about. In a 2022 paper, Luca Braghieri, Ro’ee Levy and Alexey Makarin looked at Facebook arriving at different U.S. colleges during 2004–2006. They matched the dates it arrived with surveys of students’ mental health.

The researchers compared changes around the different arrival dates. Students weren’t assigned to use Facebook at random. Instead, the study used the fact that access began at different times in different places. To read the result as cause and effect, the researchers had to make assumptions about what would have happened without Facebook. One was that mental health was moving in a similar way across the colleges.

Their preferred estimate was a rise of 0.085 standard deviations in a measure of poor mental health, a single score built from the survey answers. A standard deviation describes how spread out scores are. Here it gives a scale for how much mental health was estimated to get worse. The 0.085 is not the percentage of students harmed.

If the study’s assumptions hold, this is stronger evidence of cause and effect than simply finding that use and symptoms show up together. There is reason to take the result seriously, as long as its setting stays clear. The study was about Facebook arriving on campus. It wasn’t about how all teenagers use smartphones today.

Why it matters: Change the people, the technology or the way it’s used, and you change the question. A finding about those colleges can’t, on its own, answer all of those other questions.

Read the study of Facebook’s college rollout and mental health.

Tell a friend what a study can say

What this means for you

These accounts give different kinds of answers. Some evidence was shown to be unreliable. One program was backed by an experiment. Another study found harm in one setting at one point in history. None of them gives a whole technology one verdict that applies to everyone.

When you come across a warning, focus on the claim it makes and on what its evidence can answer.

Describe the concern before judging it

Name the technology or the way it’s used, the people involved and the outcome being claimed. “This is bad for people” leaves too much open to compare with any study.

Say a headline blames an app for a whole generation’s mental health problems. A study of an earlier platform arriving at colleges answers a narrower question. Compare who was studied, what they had access to or did, what was measured and when.

If those details don’t match the headline, keep what the study does show and leave the broader part open. If you can’t find the source, or can’t tell what it measured, write down that gap. A missing answer is not a reason to be confident, and not a reason to dismiss the worry either.

Separate the way a warning is presented from its evidence

A warning may describe a threat like nothing before it, focus on danger to children, or blame one technology for a complicated social problem. You can notice those features without using them to score how true the warning is.

This works in both directions. Familiar alarm language doesn’t cancel the television trial’s measured result. Finding serious flaws in an old argument doesn’t make the next warning false.

Write down the feature you noticed, then go back to the question of fact. Don’t guess at what the speaker privately wants. You can examine what their argument says without claiming to know why they made it.

Explain what the evidence lets you conclude

Tilley’s research gives a reason to distrust particular evidence against comics. The television and computer trial backs one program and one outcome. The college study supports reading its result as cause and effect, within its time and place and its assumptions.

Those conclusions mean different things. Keep the conditions that matter in your summary, so someone reading it on its own can see what it does and doesn’t show.

For the made-up app headline, the college finding gives a reason to take a specific concern seriously. On its own, it can’t settle the broader claim about a whole generation. You could explain the college finding, then name the open question about the people or the kind of use in the headline.

Leave uncertainty open without ignoring a concrete problem

When the evidence doesn’t settle the claim, say what is still unknown. Don’t turn “unresolved” into either “false” or “definitely harmful.”

That doesn’t mean waiting to deal with a real problem in front of you. A person’s symptoms need proper assessment, and this look at history can’t provide it.

You don’t have to sort a new technology into safe or harmful before you accept that someone is struggling. Keep the broad question about evidence apart from the help a person may need now.

What do you do with a technology warning?

How about your daily life?

Think of a technology warning you’ve come across in a news story, a discussion at work or a conversation with someone you care about. Was the claim specific enough to compare with its evidence?

You might want to understand what a study really measured, why someone called a worry a panic, or which part of a broad claim is still open.

Write down situations you’d like to look into. One good idea is enough.

A claim or discussion in my daily lifeWhat I want to understand
________________________________________
________________________________________

My chosen situation: ____________________

Choose a broad claim about technology, not one person’s health problem. Carry that situation into the exercise below. You can change your choice if the claim turns out to be too vague to check.

Work through one technology claim

Start with your chosen situation. If you came straight here, choose one broad claim about what a technology does to people, from a report or a discussion. This exercise is for making sense of claims, not for judging one person’s symptoms.

  1. Write the claim. Name the technology or the way it’s used, the people involved and the outcome being claimed. Include the time period if it matters. Mark anything the report leaves out.
    Claim and missing details: ____________________
  2. Describe how the warning is presented. Note any familiar alarm language or any appeal to authority. Keep that note apart from your judgment about whether the harm is real. Don’t guess at private motives or work out a panic score.
    Framing I noticed: ____________________
  3. Compare the claim with its evidence. Find out what the source it cites looked at, who it was about, what it compared and what it found. Keep its conditions and its uncertainty attached. If the source is missing or unclear, write down that gap rather than fill it with a guess.
    Finding, conditions or missing source: ____________________
  4. Write only what the evidence supports. Explain whether some evidence was weakened, a narrower concern was backed, or the question is still open. Give your reason. Don’t apply a finding to different people, a different kind of use or a different outcome unless something supports it.
    What I can reasonably conclude: ____________________
  5. Keep the remaining question visible. Write what still needs evidence. If there is a real problem in front of you, keep the right response to it apart from the open broad claim. A person’s symptoms need proper assessment, not a verdict from this exercise.
    Unresolved question or appropriate next response: ____________________

Ask Jing

Become a Jingius.

Check one technology claim

Loading interactive tool…

Your next step

You’re finished when you have a specific claim, a conclusion based on evidence or a gap you’ve named, and the open part clearly written down. You don’t need to declare the technology safe or harmful as a whole.

A note on the evidence

These accounts were chosen, not drawn as a fair sample of technology warnings, so they can’t tell us what share of warnings turned out false. The review of past research looks at research topics. The other studies look at different people, technologies and outcomes. Comparing what each one actually studied is more useful than counting examples on either side.

How we prepared this module

This module connects a way of reading history, a review of past research, findings from old records, a randomized trial and a natural experiment. The made-up headline shows how a broad claim can go further than a study’s question. The written exercise helps you keep the claim, the evidence and what is still uncertain together.

Sources