Do we not read and talk about research findings more often than we critically discuss methods that led to the findings we discuss?
Trained in clinical epidemiology, I often believe we should discuss methods more, especially before we move into processes of changing clinical protocols on treatments and diagnostic methods. EBM!
Nature published a very nice article about how to interpret research claims. The idea was rather to aim the article towards non-scientists but I think their advice is worth to high-light for a medical audience. ( and read the article in full text here)
The 20 tips are...
- Chance cause variation (results can be due to chance)
- No measurement is exact (as we didn't know)
- Bias is rife (it certainly is)
- Bigger is usually better for sample size (yes!)
- Correlation does not imply causation (we all know this, but we tend to forget that)
- Regression to the mean can mislead (it does)
- Extrapolating beyond the data is risky (and set patients at risk)
- Beware the base-rate fallacy (it is hard to diagnose uncommon conditions)
- Controls are important (or rather, they are essential, and it is essential to select controls right)
- Randomization avoids bias (or at least reduces bias)
- Seek replication, not pseudoreplication (research needs to replicated)
- Scientists are human (and therefore im-perfect)
- Significance is significant (but confidence intervals are more important than p-values)
- Separate no effect from non-significance (abscence of evidence is not evidence of abscence)
- Effect size matters (but remember that effects tend to decrease with study size, i.e. the world is not as good as it seems to be in small trials)
- Study relevance limits generalizations (i.e. don't generalize findings among 33-weekers to 23-weekers)
- Feelings influence risk perception (and that's why we tend to be more afraid in a plane than in a car, despite the higher death risk to drive)
- Dependencies change the risks (some factors or events are related, in additive or multiplicative ways)
- Data can be dredged or cherry picked (see #12)
- Extreme measurements may mislead (and usually do not have a single cause)
Recommended Comments