Biostatistics — Statistical Tests

• Fundamental Probability Principle: All statistical tests operate under the null hypothesis (H₀) framework, assuming no true difference exists between compared groups. When H₀ is true, if we repeated our study 100 times with proper randomization, approximately 5 times we would see our observed result purely by chance (at p=0.05 significance level). This reflects the inherent variability in sampling from populations with natural biological and measurement variation.

• Type I and Type II Error Architecture: Type I error (α, false positive) occurs when we reject a true null hypothesis, concluding a treatment works when it doesn't—the basis for the p=0.05 threshold. Type II error (β, false negative) occurs when we fail to reject a false null hypothesis, missing a true treatment effect. Statistical power (1-β) represents the probability of detecting a true effect when it exists; most studies aim for 80-90% power. These errors operate inversely: lowering the significance threshold (p<0.01) reduces Type I error but increases Type II error unless sample size increases.

• Sampling Distribution and Central Limit Theorem: Any sample statistic (mean, proportion, difference in means) has its own distribution. The Central Limit Theorem states that as sample size increases, the sampling distribution of the mean approaches normality regardless of the underlying population distribution. This principle justifies using parametric tests (t-tests, ANOVA) even with moderately non-normal data when n>30. Standard error (SE) quantifies the variability of the sample estimate: SE = SD/√n, explaining why larger samples produce narrower confidence intervals and greater precision.

• Effect Size and Clinical Significance: A statistically significant result (p<0.05) does not guarantee clinical importance. Effect size (Cohen's d, odds ratio, relative risk) measures the magnitude of difference independent of sample size. A large trial might show a statistically significant but clinically trivial difference (e.g., 2-pound weight difference), while a small trial with a clinically meaningful difference might miss statistical significance. Understanding this distinction prevents inappropriate clinical implementation of statistically significant but clinically irrelevant findings.

• Data Type (Categorical vs Continuous): Categorical data (nominal: disease present/absent; ordinal: severity scores 1-5) require different tests than continuous data (interval: temperature, cholesterol; ratio: age, weight with meaningful zero). Attempting parametric tests on categorical data violates mathematical assumptions and produces invalid p-values. This is the most frequent source of test selection errors in published research.

• Distribution of Data (Normal vs Non-normal): Parametric tests (t-test, ANOVA, linear regression) assume normality of the underlying distribution, particularly important with small sample sizes (n<30). Non-parametric tests (Mann-Whitney U, Kruskal-Wallis, Spearman) make no distributional assumptions and work with ranks rather than actual values, making them appropriate for skewed data, ordinal data, or when normality cannot be confirmed via Shapiro-Wilk test or Q-Q plots.

• Sample Size and Study Design: Large sample sizes (n>1000) approach parametric test validity even with non-normal data; small samples (n<20) require strict normality. Matched/paired designs (before-after, case-control matched pairs) require paired tests. Independent samples require unpaired tests. Longitudinal studies with repeated measures require specialized tests (repeated-measures ANOVA, mixed models) accounting for within-subject correlation.

• Number of Groups Being Compared: Comparing two groups (treatment vs control) uses two-sample tests, while comparing three or more groups requires ANOVA or Kruskal-Wallis to control for multiple comparisons problem—with many tests, Type I error accumulates. Post-hoc corrections (Bonferroni, Tukey) adjust significance thresholds to maintain overall alpha at 0.05.

• Relationship Between Variables: Correlation assumes both variables are continuous and tests their linear association. Regression assesses whether one continuous variable predicts another while controlling for confounders. Contingency tables assess association between two categorical variables. Failure to match test to the relationship being examined invalidates conclusions.

• Comparing Continuous Outcomes Between Two Independent Groups: When evaluating whether a new antihypertensive lowers blood pressure more than standard therapy, we collect continuous BP measurements in two separate groups. The independent samples t-test is appropriate if data are normally distributed; if skewed (as often occurs with biomarkers), Mann-Whitney U test is required. These tests generate a p-value indicating the probability that observed BP differences occurred randomly.

• Comparing Continuous Outcomes in Paired/Matched Settings: A study measuring cholesterol before and after statin therapy in the same patients requires paired t-test (or Wilcoxon signed-rank test if non-normal), which accounts for individual variation by analyzing differences within subjects rather than between-subject variability, increasing power to detect effects.

• Comparing Three or More Groups: Multi-arm trials comparing placebo, low-dose, and high-dose interventions require ANOVA (parametric) or Kruskal-Wallis (non-parametric) to test whether any group differs from others. Using multiple t-tests (placebo vs low-dose, placebo vs high-dose, low-dose vs high-dose) inflates Type I error; ANOVA controls overall alpha at 0.05.

• Assessing Association Between Categorical Variables: A case-control study examining smoking status (yes/no) and lung cancer (yes/no) uses chi-square test to determine whether the association is statistically significant, generating a contingency table with observed vs expected frequencies under independence.

• Measuring Linear Association Between Continuous Variables: Assessing whether hemoglobin A1c predicts mortality in diabetics uses Pearson correlation (parametric, assumes normality) or Spearman correlation (non-parametric, ranks data). Results generate correlation coefficient r (range -1 to +1) and p-value testing whether r differs significantly from zero.

• Predicting One Variable from Multiple Others: Linear regression (continuous outcome) or logistic regression (binary outcome) determines which clinical variables independently predict outcomes while adjusting for confounders. Logistic regression generates odds ratios with 95% CI for each predictor, answering: "Does this variable increase disease odds, and by how much?"

• Step 1 - Classify the Outcome Variable: Determine whether primary outcome is continuous (mean value), categorical (proportion/percentage), time-to-event (survival analysis), or ordinal (ranked categories). This single determination eliminates ~70% of incorrect test choices. Continuous outcomes suggest t-test/ANOVA/correlation/regression; categorical outcomes suggest chi-square/Fisher's exact/logistic regression.

• Step 2 - Assess Data Distribution (Normality Testing): For continuous data, perform Shapiro-Wilk test (n<50) or Kolmogorov-Smirnov test (n>50) to evaluate normality. Visually inspect Q-Q plots (if points align with diagonal line, data are normally distributed) or histograms (bell-shaped = normal). With n>30, moderate non-normality is acceptable for parametric tests due to Central Limit Theorem. With n<30, non-normality mandates non-parametric testing.

• Step 3 - Count Groups and Determine Independence: Two independent groups = t-test or Mann-Whitney U. Two paired/matched groups = paired t-test or Wilcoxon signed-rank. Three+ independent groups = ANOVA or Kruskal-Wallis. Three+ paired groups = repeated-measures ANOVA or Friedmanchev test. Failure to recognize pairing (e.g., using unpaired t-test on paired data) reduces statistical power and produces invalid p-values.

• Step 4 - Assess Variance Homogeneity (Parametric Tests): Parametric tests assume equal variances between groups. Levene's test evaluates this assumption; if p<0.05, variances are unequal, requiring Welch's t-test correction. With roughly equal sample sizes and similar variances, standard t-test is robust.

• Decision Tree Summary:

• Interpretation of Key Statistical Outputs:
• p-value <0.05: Reject null hypothesis; observed difference unlikely due to chance alone; result is statistically significant
• p-value >0.05: Fail to reject null hypothesis; insufficient evidence of true difference; absence of evidence is NOT evidence of absence (may represent Type II error)
• 95% Confidence Interval: If we repeated the study 100 times, 95 times the true population parameter would fall within this range. CIs that exclude zero (for differences) or one (for ratios/odds ratios) indicate statistical significance at p<0.05
• Cohen's d (effect size): d=0.2 (small), d=0.5 (medium), d=0.8 (large); clinically meaningful even if not statistically significant
• Correlation coefficient (r): r=±0.1-0.3 (weak), r=±0.3-0.7 (moderate), r=±0.7-0.9 (strong), r=±0.9-1.0 (very strong)

• First-Line Test Selection Protocol: (1) Identify primary outcome and classify as continuous/categorical/time-to-event. (2) Determine number of groups and independence structure. (3) For continuous data, test normality; for categorical data, ensure expected cell frequencies ≥5 for chi-square. (4) Apply parametric tests (t-test, ANOVA, Pearson correlation, linear regression) if assumptions met, non-parametric alternatives if violated. (5) Report p-value, test statistic (t, F, χ², r, β), effect size, and 95% CI.

• Parametric Tests for Continuous Data:
• Independent samples t-test: Compares means of two unrelated groups. Assumes normality and equal variances. Test statistic: t = (mean₁ - mean₂)/(SE of difference). Degrees of freedom = n₁ + n₂ - 2. Example: New drug vs placebo on LDL cholesterol.
• Welch's t-test: Modification allowing unequal variances; preferred when Levene's test p<0.05 or sample sizes vastly different.
• One-way ANOVA (parametric): Compares means across 3+ independent groups. Partitions total variance into between-group and within-group components. F-statistic = (between-group variance)/(within-group variance). Assumes normality and equal variances. Example: Comparing A1c in three different diabetes medications.
• Repeated-measures ANOVA: Analyzes multiple measurements on same subjects over time, controlling for individual baseline differences. Accounts for within-subject correlation. Example: Blood pressure measured at baseline, 4 weeks, 8 weeks, 12 weeks on antihypertensive.

• Non-Parametric Tests for Continuous or Ordinal Data:
• Mann-Whitney U test: Non-parametric alternative to independent t-test. Ranks all data points, then compares rank sums between groups. Appropriate for non-normal data, small samples, or ordinal data. Example: Comparing pain scores (0-10 scale) between treatment groups.
• Wilcoxon signed-rank test: Non-parametric alternative to paired t-test. Ranks absolute differences, comparing positive vs negative rank sums. Example: Comparing before-after symptom scores.
• Kruskal-Wallis test: Non-parametric alternative to ANOVA. Ranks all data across groups, comparing median ranks. Example: Comparing disease severity scores across three treatment arms.

• Tests for Categorical Data:
• Chi-square test (χ²): Compares observed frequencies in contingency table to expected frequencies under independence. χ² = Σ[(observed-expected)²/expected]. Requires expected frequencies ≥5 in each cell. Degrees of freedom = (rows-1)(columns-1). Example: Association between smoking status and lung cancer in 2×2 table.
• Fisher's exact test: Used when chi-square assumptions violated (expected frequency <5). Calculates exact p-value for 2×2 tables. Preferred for small sample sizes. Example: Rare adverse event (n=5) vs expected (n=20) in small trial.
• McNemar's test: Chi-square variant for paired categorical data. Accounts for agreement within pairs. Example: Diagnostic test A vs B performed on same patients.

• Tests for Association Between Continuous Variables:
• Pearson correlation: Tests linear association between two continuous normally-distributed variables. Produces correlation coefficient r (-1 to +1) and p-value testing r≠0. Example: Correlation between BMI and blood pressure.
• Spearman rank correlation: Non-parametric alternative. Ranks both variables, then calculates correlation on ranks. Appropriate for non-normal data, ordinal data, or curvilinear relationships. Example: Correlation between income level (ordinal) and health outcomes.

• Regression for Prediction/Confounding Adjustment:
• Linear regression (simple): Predicts continuous outcome from single continuous predictor. Equation: Y = β₀ + β₁X + ε. Example: Predicting LDL from statin dose.
• Multiple linear regression: Predicts continuous outcome from multiple predictors while adjusting for confounders. Example: Does age independently predict mortality after adjusting for comorbidities?
• Logistic regression: Predicts binary outcome (yes/no, disease/no disease) from continuous or categorical predictors. Generates odds ratios (OR) with 95% CI for each predictor. OR>1 indicates increased odds; OR<1 indicates decreased odds. Example: Predicting myocardial infarction risk from age, smoking, cholesterol, diabetes.

• Multiple Comparisons Correction: When performing multiple statistical tests on same dataset, probability of Type I error accumulates. Bonferroni correction divides significance threshold by number of comparisons (p=0.05/3=0.0167 if 3 comparisons). Tukey's HSD or Scheffe's test used post-hoc after significant ANOVA for pairwise

Test-selection reflexes (the single most tested skill)

• Two means, two independent groups → independent t-test; three or more means → ANOVA. The classic distractor is running repeated t-tests across three arms, which inflates Type I error — ANOVA exists precisely to hold overall α at 0.05.
• "Same patients measured twice" (before/after, right eye vs left eye, matched pairs) → paired t-test, not the independent version. Missing the pairing throws away the within-subject variance reduction and loses power.
• Two categorical variables / percentages in a contingency table → chi-square. If any expected cell count is small (conventionally <5), switch to Fisher's exact test; for paired categorical data (same patients, two diagnostic tests) use McNemar's test.
• Mnemonic: T for two means, ANOVA for three or more means ("ANOVA = ANalysis Of VAriance," but it compares means), Chi-square for categorical/counted data.

Interpretation traps

• A significant ANOVA tells you only that at least one group differs — the next best step is a post-hoc pairwise test with correction (Tukey, Bonferroni), never eyeballing the group means.
• Statistical ≠ clinical significance. Huge samples make trivial differences "significant"; the American Statistical Association's 2016 statement on p-values explicitly warns against treating p<0.05 as proof of importance. Report effect size and 95% CI.
• A 95% CI that crosses 1 (ratios: OR, RR, HR) or 0 (differences in means) is non-significant at α=0.05 — this equivalence is tested constantly.
• p>0.05 does not prove no effect; an underpowered study (β too high) yields a false negative. Power rises with larger n, larger effect size, and lower variance.
• Non-normal or ordinal data (pain scores, Likert scales, skewed biomarkers) → the rank-based twin: Mann-Whitney U (unpaired), Wilcoxon signed-rank (paired), Kruskal-Wallis (3+ groups).
• Correlation coefficient r measures linear association only — r near zero does not exclude a strong curvilinear relationship, and correlation never establishes causation.