Psychology/Learn/Reading evidence
Language:EnglishEspañolPortuguês (BR)
Learn
Reading evidence without being fooled
You do not need statistical training to read a psychological claim well. You need six questions and the willingness to stop reading when they are not answered.
Six questions, in order
- Who was studied, and how many?Thirty undergraduates in one country is a starting point, not a finding about people. Ask whether the sample resembles the population the claim is being applied to.
- What was actually measured?A questionnaire about intention is not behaviour. A laboratory task is not a workplace. The gap between the measure and the claim is where most over-reach lives.
- How big was the effect?Not whether it reached significance — how large it was, and whether a difference of that size would matter to anybody in practice.
- Was it reproduced?By an independent group, with a pre-registered plan, at a reasonable sample size. One study is a candidate; several are a finding.
- What is the base rate?A test that is “ninety per cent accurate” for something occurring in one per cent of people produces far more false alarms than hits. This single question dismantles most screening claims.
- Who benefits from the claim?Not an accusation — a routine check. Note the funding, the product, and the certification being sold alongside it.
Four sentences that should make you stop reading
| Sentence | The problem |
|---|---|
| “Studies show…” with no study named. | An unfalsifiable appeal. There is nothing to check. |
| “Ninety-three per cent of communication is non-verbal.” | A misreading of two small 1960s studies on single spoken words in a laboratory. It has never meant what it is used to mean. |
| “This test is used by the FBI / NASA / elite performers.” | Adoption is not validation. Ask for the validation evidence, the population, and the purpose it was validated for. |
| “Neuroscience proves…” | Neuroscience produces evidence, like every other field. “Proves” is a marketing verb. |
A worked check
Suppose you read: “A new five-minute assessment identifies burnout risk with 89 per cent accuracy.” Run the questions. Who was studied — and were they already identified as at risk, which inflates accuracy enormously? What was measured — a self-report questionnaire, or an outcome such as absence or resignation? How big is the effect relative to simply asking someone how they are? Was it reproduced by a group with no commercial interest? What is the base rate in the population where it will be used? Who sells the assessment? A claim that survives all six is worth your attention. Most do not survive the fifth.
The most useful habit
Before reading anybody’s conclusion, read their method and their sample size. If those are not available, you are not reading evidence — you are reading a position.
Educational scope. Everything published on PMA Science Psychology is provided for education, study and professional documentation. Nothing here is a diagnostic instrument, a psychometric test, or a substitute for assessment, treatment, supervision or emergency care by a qualified professional.
If you are at risk right now, contact your local emergency number or a crisis line immediately. Read our boundaries and safety statement.