{"slug":"common-statistical-mistakes-research","title":"10 Common Statistical Mistakes in Research (and How to Avoid Them)","excerpt":"From p-hacking to ignoring assumptions, the same statistical errors sink research again and again. Here are ten to watch for in your own analysis.","date":"2026-05-23","author":"DSTechnoverse","image":"https://images.unsplash.com/photo-1434030216411-0b793f4b4173?auto=format&fit=crop&w=1200&q=80","category":"Research Methods","tags":["statistical mistakes","research","methodology","p-value","analysis"],"content":"Reviewers and examiners see the same statistical errors over and over. Most are easy to avoid once you know to look for them. Here are ten of the most common — and the fix for each.\n\n## 1. Choosing the wrong test\n\nRunning a t-test where you need ANOVA, or Pearson where the data is not normal. **Fix:** match the test to your question and data type before you start.\n\n## 2. Ignoring assumptions\n\nTests assume things — normality, equal variances, independence. **Fix:** check the assumptions and switch to non-parametric tests when they fail.\n\n## 3. Confusing correlation with causation\n\nA relationship between two variables does not mean one causes the other. **Fix:** be precise in your language; correlation is not proof of cause.\n\n## 4. p-hacking\n\nRunning many tests until something is \"significant.\" **Fix:** decide your analyses in advance and report everything you ran.\n\n## 5. Misreading the p-value\n\nA p-value is not the probability your hypothesis is true, and p < 0.05 is not \"proof.\" **Fix:** interpret it correctly and report effect sizes too.\n\n## 6. Ignoring effect size\n\nA result can be statistically significant but trivially small. **Fix:** always report effect sizes (Cohen's d, r, odds ratios) alongside p-values.\n\n## 7. Too small a sample\n\nUnderpowered studies miss real effects. **Fix:** calculate sample size before collecting data.\n\n## 8. Not handling missing data honestly\n\nSilently dropping cases can bias results. **Fix:** report how much data was missing and how you handled it.\n\n## 9. Over-interpreting the results\n\nClaiming more than the data supports, or generalising beyond your sample. **Fix:** keep conclusions within the limits of your design.\n\n## 10. Poor reporting\n\nTables without context, results that do not match the numbers. **Fix:** report clearly — test used, statistic, degrees of freedom, p-value, and effect size.\n\n## The common thread\n\nAlmost every mistake here comes from **rushing the thinking**, not the maths. Decide your methods in advance, respect the assumptions, and report honestly. Do that and your research will survive scrutiny that sinks weaker work.\n\n## How each mistake is detected and repaired\n\nNaming the ten errors is the easy part. What makes them fixable is knowing what each looks like in real output and what to do once you find it.\n\n| Mistake | How to detect it | The repair |\n|---|---|---|\n| Wrong test for the design | Paired data analysed with an independent test; 3 groups with multiple t-tests | Re-select from outcome type, group count and independence |\n| Assumptions never checked | No Levene, no Q-Q plot, no VIF anywhere in the output | Run diagnostics; switch to a robust or non-parametric method |\n| Correlation read as causation | The discussion uses \"leads to\" for observational data | Rewrite as association; state confounders explicitly |\n| p-hacking | More analyses run than reported; subgroups appearing only in results | Pre-register; disclose every test performed |\n| p-value misread | \"There is a 3% chance the null is true\" | Restate as evidence against the null under assumption |\n| Effect size absent | A results table of p-values with no magnitudes | Add d, eta squared or odds ratios with CIs |\n| Underpowered sample | No power calculation; small n with a null result | Report the detectable effect size; frame as inconclusive |\n| Missing data mishandled | Sample size varies between tables with no explanation | Diagnose the mechanism; use multiple imputation |\n| Over-interpretation | Conclusions broader than the sampling frame | Bound the claim to the population sampled |\n| Poor reporting | Missing df, missing n, p = .000 | Apply a reporting checklist before submission |\n\n## The three that cost the most\n\n**Ignoring clustering.** A survey that sampled 40 villages and then 25 households in each is not 1,000 independent observations. Analysed as though it were, every standard error is too small and every significance test overstated — sometimes dramatically. Nothing in the dataset flags this; you have to know how the sample was drawn. The repair is a mixed-effects model, clustered standard errors, or survey-weighted procedures. This is the most consequential error in Indian household survey work and the one least often caught in review.\n\n**Stepwise variable selection.** Letting software choose predictors by p-value produces different models from near-identical datasets, inflates R squared, and yields p-values that no longer mean what they claim. Specify your model from theory and prior evidence, and report the model you specified.\n\n**HARKing — hypothesising after results are known.** Running twenty comparisons, finding two significant, and presenting those two as the study's hypotheses. This is not a grey area, and it is the mechanism behind a large share of findings that fail to replicate. If a finding was unplanned, label it exploratory. That single clause preserves the credibility of everything else in the paper.\n\n## The mistakes specific to survey research\n\nBeyond the general ten, field survey work has its own recurring failures.\n\n| Mistake | Consequence |\n|---|---|\n| Analysing a single Likert item with ANOVA | Ordinal data treated as interval; use Kruskal-Wallis |\n| Summing scale items without checking reliability | The composite may measure two unrelated things |\n| Not reverse-scoring negative items | Alpha collapses; the finding inverts |\n| Ignoring survey weights | Estimates biased toward over-sampled groups |\n| Treating \"don't know\" as a neutral midpoint | Non-opinion contaminates the centre of the distribution |\n| Comparing baseline and endline on different samples | Attrition mistaken for change |\n| Chi-square with expected counts below 5 | The test is invalid; use Fisher's exact or merge categories |\n| Reporting percentages from 15 interviews | Implies precision purposive sampling cannot support |\n\nThe attrition point deserves emphasis for anyone doing [baseline and endline evaluation](/blog/baseline-endline-impact-evaluation). If poorer households are likelier to migrate between rounds, the endline sample is systematically better off than the baseline sample — and the programme will appear to have raised incomes even if it did nothing at all. Always compare baseline characteristics of those who remained against those lost, and report the comparison.\n\n## A pre-submission audit\n\nFifteen minutes with this list catches most of what reviewers return work for.\n\n| Check | Confirmed? |\n|---|---|\n| Every test justified by the design, not by habit | |\n| Assumption checks reported, not just performed | |\n| Every p-value accompanied by an effect size | |\n| Confidence intervals present for the main findings | |\n| Exact p-values, no p = .000 | |\n| Degrees of freedom and n reported for every test | |\n| Missing data mechanism and handling described | |\n| Every analysis run is disclosed, including the failures | |\n| Exploratory analyses labelled as exploratory | |\n| Causal language used only where the design supports it | |\n| Clustering, weighting or repeated measures accounted for | |\n| Conclusions bounded by the population actually sampled | |\n\nThe eighth row is the one people resist. Disclosing that you ran eleven comparisons and two were significant feels like weakening the paper. It is the opposite: an undisclosed multiplicity problem discovered by a reviewer is far more damaging than a disclosed one, and the disclosure often costs nothing more than a sentence in the methods.\n\n## Why these persist\n\nAlmost none of this is caused by ignorance of the correct method. Three structural pressures do most of the damage: publication systems that reward positive findings, deadlines that make a proper power calculation feel like a luxury, and software that runs any procedure you ask for without comment.\n\nSPSS will not object to a t-test on clustered data. R will not warn you that your predictors were chosen by stepwise selection. The reasoning has to happen before the software is opened, which is the argument for settling the analysis plan while designing the study — covered in [the five steps of hypothesis testing](/blog/hypothesis-testing-steps), [how to choose the right statistical test](/blog/choosing-the-right-statistical-test) and [how to calculate sample size](/blog/sample-size-calculation-guide).\n\nFor reporting standards, [how to present statistical results](/blog/presenting-statistical-results). Our [statistical and research data analysis service](/services/statistical-research-data-analysis) applies the audit above as standard. The [EQUATOR Network](https://www.equator-network.org/) hosts the reporting guidelines for every major study design and is the best single resource for checking your write-up against field expectations.\n\n## Frequently Asked Questions\n\n### What is the most common statistical mistake in research?\n\nChoosing a test that does not match the study design — most often analysing paired data as independent, running several t-tests where ANOVA is required, or treating clustered survey data as a simple random sample. All three inflate significance.\n\n### What is p-hacking and how do I avoid it?\n\nRunning many analyses, subgroups or model specifications and reporting only those that reached significance. Avoid it by fixing your primary outcome and analysis plan before collecting data, correcting for multiple comparisons, and disclosing every test you ran.\n\n### Why is correlation not causation?\n\nBecause an unmeasured third factor may drive both variables, or the causal direction may run the other way. Regression controls only for variables you actually measured. Causal claims need a design — randomisation, instrumental variables, or a credible difference-in-differences setup.\n\n### Do I have to report effect sizes?\n\nYes, and most journals now require it. A p-value indicates whether an effect is detectable; the effect size indicates whether it matters. Without it, a reader cannot distinguish a trivial difference in a huge sample from an important one.\n\n### What happens if I ignore clustering in survey data?\n\nYour standard errors will be too small and your p-values too optimistic, sometimes by a large margin. A survey of 25 households in each of 40 villages carries far less independent information than 1,000 randomly selected households.\n\n### How do I know if my sample was too small?\n\nCompute the effect size your design could detect at 80% power and compare it against effects considered meaningful in your field. If your study could only have detected an implausibly large effect, a null result is uninformative rather than reassuring — and should be reported that way.\n\n\n---\n\n**Need help turning your data into defensible results?** DSTechnoverse offers statistical and research data analysis services in SPSS, R, and Python — survey analysis, thesis and dissertation support, and business statistics. [Get in touch](/contact) to discuss your dataset.","related":[{"slug":"choosing-the-right-statistical-test","title":"How to Choose the Right Statistical Test for Your Data"},{"slug":"how-to-conduct-socio-economic-survey","title":"How to Conduct a Socio-Economic Survey Step by Step"},{"slug":"reducing-survey-bias-field","title":"How to Reduce Bias in Field Surveys"},{"slug":"presenting-statistical-results","title":"How to Present Statistical Results in a Report or Paper"}]}