Statistics
Choosing and interpreting statistical tests, regressions and significance.
Download all 71
- Analyse A/B test results
Analyses A/B test results with a sample-ratio-mismatch check, effect sizes, confidence intervals and guardrail metrics, ending in a ship, iterate or stop call. Use when an experiment ends.
- Analyse Likert-scale data
Analyses Likert-scale survey data with distribution summaries, diverging bar charts and tests suited to ordinal items or multi-item scales. Use before reporting agreement scores.
- Analyse data from a very small sample
Analyses data already collected from a very small sample with exact tests, honest intervals and optional Bayesian estimates, and says plainly what such data can and cannot show.
- Compute a bootstrap or permutation interval
Computes bootstrap confidence intervals or permutation tests for medians, ratios, percentiles or custom metrics, choosing the resampling unit and interval method, with runnable code.
- Build a baseline characteristics table
Builds the statistical content of a Table 1 comparing groups, picking the right summary for each variable and using standardised differences rather than p-values to show balance.
- Build a composite index
Builds a weighted composite index from several indicators, covering direction, normalisation, weighting, aggregation and rank sensitivity checks. Use for scorecards and health scores.
- Build a control chart
Builds a statistical process control chart with the right chart type, control limits, special-cause rules and an interpretation that separates signal from noise. Use to monitor a process over time.
- Build survey weights
Builds survey weights by post-stratification or raking to population targets, trims extremes and reports design effect and effective sample size. Use when a sample does not match its population.
- Calculate and convert effect sizes
Calculates effect sizes (Cohen's d, Hedges' g, eta squared, r, odds ratio) from raw data or reported statistics, converts between them, adds confidence intervals and explains the practical size.
- Calculate inter-rater reliability
Calculates and interprets inter-rater reliability, choosing kappa, weighted kappa, Krippendorff's alpha or the right ICC form for the design. Use when checking that coders or judges agree.
- Build a measurement uncertainty budget
Builds a measurement uncertainty budget for a lab or calibration measurement with sources, distributions, sensitivity coefficients, combined and expanded uncertainty, showing all the maths for review.
- Calculate process capability
Calculates Cp, Cpk, Pp and Ppk against specification limits after checking stability and normality, estimates the defect rate, and says what to fix, centring or spread.
- Calculate sample size
Computes the sample size or statistical power for an experiment or survey, shows the formula and assumptions, and gives a sensitivity table. Use before launching an A/B test, study or survey.
- Check an analysis for pitfalls
Reviews an analysis for statistical pitfalls such as Simpson's paradox, p-hacking, survivorship, base rates and causal over-claims before it is shared. Use as a pre-publication review.
- Check if a metric change is just noise
Judges whether this period's rise or drop in a KPI is beyond normal variation, using its history, count-based intervals or control limits, and gives a plain verdict with what to check next.
- Check whether non-normal data is a problem
Decides whether non-normal data actually matters for the planned analysis, using plots on the right quantity, and picks a fix (do nothing, transform, robust errors, bootstrap or nonparametric).
- Check the reliability of a scale
Computes Cronbach's alpha or McDonald's omega, item-total correlations and alpha-if-item-deleted for collected scale data, and says what to report and what alpha cannot tell you.
- Choose a statistical test
Picks the right statistical test for a research question and data shape, explains its assumptions and how to check them, and gives code to run it. Use before testing a difference or relationship.
- Code categorical variables for a regression
Sets up dummy, effect or ordered coding for categorical predictors in a regression, picks reference levels, handles rare levels, and explains how to read the resulting coefficients.
- Compare two distributions beyond the mean
Compares two distributions on spread, shape and tails with quantile differences, shift plots and Kolmogorov-Smirnov or Anderson-Darling tests. Use when means hide what changed.
- Compute a survey margin of error
Computes margins of error and confidence intervals for survey results, including subgroups and gaps between answers, and states what they do not cover. Use before reporting poll numbers.
- Correct for multiple comparisons
Decides whether and how to adjust p-values for many tests (Holm, Bonferroni, Benjamini-Hochberg, gatekeeping), applies it and states what can still be claimed. Use with many outcomes or segments.
- Decide what to do with outliers
Decides whether extreme values in an analysis are errors or real, sets a rule you can defend, and reports results with and without them using robust methods where they fit.
- Design a conjoint study
Designs a choice-based conjoint study with attributes, levels, design, sample size and an analysis plan for preference shares and willingness to pay. Use before pricing decisions.
- Design a factorial experiment
Designs a full or fractional factorial experiment for a process or product, choosing factors, levels, runs, aliasing, randomisation and the analysis, within a stated run budget.
- Estimate a causal effect from observational data
Estimates a causal effect from observational data with a fitting design (difference-in-differences, matching, regression discontinuity), assumptions and robustness checks. Use when no experiment ran.
- Estimate price elasticity
Estimates price elasticity of demand from price and volume history or a price test, with the method, confounders, a confidence range and how to use it in pricing. Use before changing prices.
- Evaluate a classifier or score with ROC
Evaluates a risk score, screening test or model against known outcomes with ROC and AUC with intervals, precision-recall, a confusion matrix and calibration, and judges if it is fit for use.
- Evaluate programme outcomes
Evaluates a programme's outcomes with a pre-post or comparison-group design, effect sizes, attrition checks and honest limitations, and drafts funder-ready wording. Use when reporting impact.
- Explain and convert odds ratios and relative risk
Converts between odds ratios, relative risk, absolute risk difference and number needed to treat, using a baseline risk, and explains what a reported risk figure means in real numbers.
- Explain a statistics concept
Explains a statistics concept such as a p-value, confidence interval or power, with intuition, a worked example, a simulation and common misreadings. Use to finally get it.
- Explain a test result with base rates
Explains what a positive or negative test result means using base rates, sensitivity and specificity, worked through with natural frequencies. Use for medical, screening, fraud or quality tests.
- Find change points in a time series
Finds when a time series shifted to a new level, slope or variability using change-point methods, then checks each shift against known events before anyone reads cause into it.
- Fit a probability distribution to data
Fits candidate distributions such as normal, lognormal, gamma or Weibull to observed data, compares the fits, and picks one for simulation, percentiles or risk estimates.
- Interpret an interaction effect
Interprets an interaction or moderation term with simple slopes, effects at meaningful values, a plot and plain wording, also on the probability scale. Use when an effect depends on a third variable.
- Interpret regression output
Explains regression output in plain language (coefficients, intervals, p-values, fit) and what it does and does not let you conclude. Use when you have a model summary and need to explain it.
- Interpret statistical test output
Explains every number in pasted output from a t-test, ANOVA, chi-square, correlation or non-parametric test in R, SPSS, Excel, Python or Stata, checks it is consistent, and says what it shows.
- Make a Fermi estimate
Makes a Fermi estimate by decomposing a quantity, stating assumptions with ranges, cross-checking from another angle and naming the data that would tighten it. Use for sizing when no data exists.
- Model count and rate data
Models counts or rates (complaints, defects, incidents) with Poisson or negative binomial regression, exposure offsets, overdispersion and zero-inflation checks, and rate ratios in plain words.
- Plan acceptance sampling for incoming goods or batches
Plans acceptance sampling for incoming goods or production batches with sample size, acceptance number, producer and consumer risks from the operating characteristic curve, and a results record.
- Plan how to handle missing data
Diagnoses why values are missing, weighs MCAR, MAR and MNAR, and picks complete-case analysis, multiple imputation or weighting with a sensitivity check. Use before analysing incomplete data.
- Plan a sequential test
Plans an experiment that can be checked before it ends, with group sequential boundaries or always-valid tests, a look schedule, stop rules and the sample cost. Use when stakeholders want to peek.
- Run an analysis in SPSS
Gives the SPSS menu path and pasteable syntax for a described analysis, then explains each output table, which rows to read, and what to report in APA-style text.
- Adjust a group comparison for baseline with ANCOVA
Compares groups on a follow-up score adjusted for the baseline score with ANCOVA, checks slopes and assumptions, and reports adjusted means and the adjusted difference with an interval.
- Run an ANOVA with post-hoc comparisons
Compares three or more groups with one-way or two-way ANOVA, Welch ANOVA or Kruskal-Wallis, then the right post-hoc test, with assumption checks, effect sizes and a write-up.
- Run a Bayesian A/B test analysis
Analyses an A/B test the Bayesian way, with priors, posteriors, probability to beat control, expected loss and a decision rule, explained for non-statisticians. Use as a product or growth analyst.
- Run a Bland-Altman method comparison
Compares two measurement methods with a Bland-Altman analysis, giving bias, limits of agreement with confidence intervals, proportional bias checks and a verdict against a clinical or practical limit.
- Test association in a contingency table
Tests whether two categorical variables are related with chi-square, Fisher's exact or McNemar, checks expected counts, and reports Cramér's V, risk differences and residuals.
- Run and report a correlation analysis
Runs Pearson, Spearman or partial correlations with confidence intervals, checks scatterplots for outliers, nonlinearity and subgroups, and words the result without causal claims.
- Run a factor analysis
Plans and interprets an exploratory or confirmatory factor analysis of survey items, with assumption checks, factor retention, fit indices and reliability. Use when validating a scale.
- Run a gauge R&R study
Plans and analyses a gauge repeatability and reproducibility study, splitting measurement variation into repeatability, reproducibility and part variation, with a verdict on the gauge.
- Run a mediation analysis
Plans and interprets a mediation analysis splitting an effect into direct and indirect paths, with bootstrap intervals, causal assumptions and sensitivity checks. Use when asking how an effect works.
- Run a Monte Carlo simulation
Builds a Monte Carlo simulation for a decision or forecast with justified input distributions, correlations and code or spreadsheet steps. Use when a single-number estimate hides the risk.
- Run a multilevel model
Plans and interprets a multilevel or mixed-effects model for nested or repeated data, with centring, random-effects choices, code, diagnostics and a reporting template. Use when rows are clustered.
- Compare paired or before-after measurements
Analyses before-after or matched measurements with a paired t-test or Wilcoxon signed-rank test, checks the pairing and reports the average change with a confidence interval.
- Run a panel data regression
Plans and interprets fixed-effects or random-effects regressions on panel data (units observed over time) with two-way effects, clustered standard errors and a defensible choice between models.
- Run a principal component analysis
Runs PCA to reduce many numeric variables to a few components, choosing how many to keep, reading loadings and scores, and saying when PCA is the wrong tool for the question.
- Run a regression analysis
Builds a regression analysis for a question, covering model choice, variables, diagnostics, interpretation and limits, with runnable code in Python, R or Excel. Use as an analyst or student.
- Run a repeated-measures or mixed ANOVA
Runs a repeated-measures or mixed (between-within) ANOVA with sphericity checks and corrections, follow-up comparisons and effect sizes, and says when a mixed model is the better choice.
- Run statistical tests in Excel
Runs t-tests, ANOVA, chi-square, correlation and regression in Excel with the Analysis ToolPak or worksheet functions, and reads the output correctly, including its traps.
- Run a structural equation model
Specifies and interprets a structural equation model with a measurement part and structural paths, judging fit indices honestly and limiting model modifications to ones theory supports.
- Run a survival (time-to-event) analysis
Runs a time-to-event analysis (Kaplan-Meier, Cox) for churn, failure or time-to-hire, handling censoring correctly, with code and a plain reading. Use when the question is how long until.
- Compare two independent groups
Compares two independent groups with a Welch t-test or Mann-Whitney test, giving the difference, a confidence interval, an effect size and a plain-language result. Use for any A versus B question.
- Consulting statistician
Consulting statistician who asks how the data were produced before analysing them, chooses methods that fit the question, checks assumptions and refuses to over-claim. Use for any data analysis.
- Test whether two groups are practically the same
Tests whether two groups or methods are practically equivalent or non-inferior with TOST and a pre-set margin, instead of reading a non-significant p-value as proof of no difference.
- Test whether a time series has a trend
Tests whether a series is really rising or falling, using regression or Mann-Kendall with Sen's slope, while handling autocorrelation, seasonality and short windows honestly.
- Work out a probability
Works out the probability behind a real-life question, such as at least one success, shared birthdays, lottery odds or repeated tries, step by step with the assumptions stated and a simulation check.
- Write a Python statistical analysis
Writes a reproducible Python analysis with statsmodels and scipy for a described dataset and question, with data checks, assumption checks and effect sizes. Use when others must re-run it.
- Write an R analysis script
Writes a reproducible R (tidyverse) analysis script for a described dataset and question, with import, checks, analysis, plots and saved outputs. Use when you need an analysis others can re-run.
- Write a Stata do-file
Writes a reproducible Stata do-file for a described dataset and question, with logging, data checks, the right model and standard errors, and tables exported for the paper or report.
- Write a statistical analysis plan
Writes a statistical analysis plan before data collection with estimands, models, multiplicity, missing data and sensitivity analyses. Use for trials and pre-registrations.