Calculate P-values for Z, T, Chi-square, and F tests. Determine statistical significance with power analysis and effect size calculation.
A P-value is the probability of obtaining results as extreme as observed, assuming the null hypothesis (H₀) is true. If P=0.03, there's a 3% chance of seeing such extreme results if H₀ is true. When P < α (usually 0.05), we reject H₀. Smaller P-values provide stronger evidence against H₀, but don't indicate effect size or practical importance. P=0.001 and P=0.049 are both significant but differ in strength of evidence.
Z-test is used when population standard deviation (σ) is known and sample size is large (n≥30): Z = (x̄ - μ₀) / (σ / √n). T-test is used when σ is unknown and we use sample SD (s), or when n<30: T = (x̄ - μ₀) / (s / √n) with df=n-1. T-distribution has heavier tails than normal distribution.
Chi-square (χ²) test is used for independence testing or goodness-of-fit with categorical variables: χ² = Σ(Observed - Expected)² / Expected. Used for independence tests (e.g., gender vs product preference), homogeneity tests (comparing proportions across groups), and goodness-of-fit tests (whether observed distribution matches theoretical distribution).
F-test compares variances between two groups or multiple group means in ANOVA: F = s₁² / s₂² (larger variance / smaller variance). ANOVA uses F-test to compare three or more group means simultaneously, avoiding inflated Type I error from multiple t-tests. Larger F-values indicate between-group variance exceeds within-group variance.
Statistical Power is the probability of correctly detecting an effect when it exists: Power = 1 - β (Type II error). Target is usually 0.8 (80%) or higher. Effect Size measures practical significance. Cohen's d = (Mean₁ - Mean₂) / Pooled SD. d=0.2 is small, d=0.5 is medium, d=0.8 is large. Significant P-value with small effect size may lack practical importance (with very large samples).
Common misconceptions: ① "P=0.05 means 5% chance H₀ is true" is wrong. P-value is "probability of data given H₀", not "probability of H₀ given data". ② "P<0.05 means large effect" is wrong. P-value indicates significance, not effect size. ③ Don't treat 0.05 as absolute cutoff. Proper use: Report P-values with confidence intervals, effect sizes, and practical significance. Use pre-registration and avoid P-hacking.