Tukey HSD post hoc decision guide showing ANOVA, Tukey-Kramer, Games-Howell, and pairwise confidence intervals.

Tukey HSD Calculator & Post Hoc Test Guide

A Tukey HSD calculator compares every pair of group means while accounting for the fact that many comparisons are being made. It is most useful when all pairwise comparisons matter and a common within-group variance is a reasonable model assumption.

The calculation is only one part of the decision. Equal and unequal sample sizes require slightly different Tukey formulas, unequal variances can point to Games-Howell instead, and a non-significant comparison does not prove that two populations are equal. This guide moves from method selection to calculation, interpretation, troubleshooting, and reporting so the result answers the research question rather than simply producing a p-value.

Tukey HSD at a Glance: What It Tells You

Tukey’s Honestly Significant Difference test is an all-pairs multiple-comparison procedure. If an analysis contains groups A, B, C, and D, Tukey evaluates A-B, A-C, A-D, B-C, B-D, and C-D as one comparison family rather than treating six separate tests as unrelated.

For groups, the number of unique pairwise comparisons is:

Five groups therefore create 10 comparisons; ten groups create 45. The describes Tukey’s method as applying simultaneously to the complete set of pairwise mean differences and using the studentized range distribution for that purpose.

The traditional equal-sample-size decision rule is:

Modern software usually gives more informative output: the estimated mean difference, a Tukey-adjusted p-value, and a simultaneous confidence interval. A confidence interval that excludes zero corresponds to rejecting the no-difference hypothesis for that pair at the stated familywise level.

Tukey HSD Calculator

A useful calculator should answer two questions: Which pairs are distinguishable under the Tukey procedure, and how large could those differences plausibly be? It should also make the method transparent enough that a reader can tell whether standard Tukey HSD or the unequal-sample-size Tukey-Kramer calculation was used.

Calculator input guide. For the accompanying web calculator, enter MSE, residual degrees of freedom, alpha, and one line per group containing a label, mean, and sample size. Equal sample sizes use the standard Tukey HSD margin; unequal sample sizes automatically use the Tukey-Kramer pair-specific margin. The calculator reports the mean difference, simultaneous confidence interval, and significance decision. Exact Tukey-adjusted p-values should be obtained from a full statistical implementation such as R, SciPy, Statsmodels, SPSS, SAS, or Minitab.

For an ANOVA-summary calculator, the essential inputs are the number of groups, each group mean, Mean Square Error (MSE, also called MS Error or MS Within), residual or error degrees of freedom, each group sample size, and the chosen familywise significance level .

If every group has the same , the calculator can use one common HSD margin. If the group sizes differ, each pair needs its own Tukey-Kramer margin because its standard error depends on the two sample sizes involved. states this explicitly: for unequal sample sizes, each interval width is calculated separately depending on sample size.

A strong result table should show the mean difference, simultaneous confidence interval, significance decision, and the exact method used. If adjusted p-values are provided by the implementation, they should be labeled as already adjusted for the Tukey comparison family.

When Should You Use Tukey HSD?

Use Tukey HSD when every group needs to be compared with every other group and the pooled common-variance model is defensible. Typical examples include comparing all teaching methods, all fertilizer treatments, all manufacturing settings, or all experimental conditions.

The most important question comes before the calculation: Do all pairs actually belong to the scientific question? If each treatment only needs to be compared with a single control, Dunnett’s procedure is designed for that smaller comparison family. If only a few hypotheses were specified before seeing the data, planned contrasts may match the study better than automatically testing every possible pair. If broad arbitrary contrasts are the target, Scheffe’s method addresses a wider family than Tukey.

Animated decision flow for selecting Tukey HSD, Tukey-Kramer, Games-Howell, or a more targeted comparison method.

Do you need a significant ANOVA before Tukey HSD?

A common teaching workflow is to fit a one-way ANOVA, inspect the omnibus F-test, and use post hoc comparisons to explain a significant overall result. That is a coherent workflow, but it should not be turned into the absolute claim that Tukey’s procedure becomes mathematically invalid whenever the omnibus F-test is not significant.

Tukey controls multiplicity within its own all-pairs family. notes that most multiple-comparison procedures can find significant contrasts when the overall F-test is nonsignificant and that adding a preliminary F-test can reduce their power. The practical approach is to follow the prespecified analysis plan and research question. In a conventional exploratory ANOVA workflow, examining the omnibus result first remains sensible; it is simply not a universal mathematical gatekeeper for every Tukey analysis.

Can Tukey HSD be used with two groups?

With two groups there is only one comparison, so the main reason for an all-pairs multiple-comparison procedure disappears. A suitable two-sample test is usually clearer.

Assumptions of the Tukey HSD Test

Tukey inherits the error structure of the ANOVA model on which it is based. The key assumptions are independent observations, approximately normal model errors within groups, and a common finite within-group variance for the standard pooled-variance procedure. lists these assumptions and distinguishes Tukey HSD, Tukey-Kramer, and Games-Howell according to sample-size and variance conditions.

Design question	Why it matters	Practical response
Are observations independent?	Correlation changes the sampling variance.	Use a model that reflects pairing, clustering, or repeated measurements.
Are model errors reasonably compatible with the ANOVA model?	The reference distribution assumes normal errors.	Inspect residuals, especially with small samples or extreme outliers.
Is a common within-group variance credible?	Tukey pools the error variance across groups.	Consider Games-Howell if equal variance is not credible.
Are group sizes equal?	Equal gives one HSD margin; unequal changes pairwise precision.	Use Tukey-Kramer when differs but the common-variance model remains appropriate.
Do all pairwise comparisons matter?	Tukey allocates error control to the complete all-pairs family.	Use Dunnett, planned contrasts, or another method when the hypothesis family is narrower.

What if Levene’s test is significant?

Do not reduce the choice to a mechanical rule such as “Levene p < .05 means Games-Howell” without looking at the design. The underlying issue is whether pooling the group variances into one MSE is scientifically and statistically credible.

Variance heterogeneity becomes particularly problematic when group sizes also differ. Current warns that Tukey HSD is not robust to heteroscedasticity, especially when group sizes vary. If the equal-variance model is not credible and all-pairs comparisons are still required, Games-Howell is usually the more appropriate starting point.

Repeated measures need a different model

A simple independent-groups Tukey calculator should not be applied directly to repeated observations from the same participants. The dependence among repeated measurements must first be represented in an appropriate repeated-measures or mixed model. Pairwise comparisons can then be constructed from that model with a suitable multiplicity adjustment.

How Tukey HSD Works

The statistical problem is multiplicity. If five means are present, there are 10 possible pairs. Running 10 ordinary t-tests and treating each as if it were the only test gives the analysis many opportunities to produce a false positive somewhere in the family.

Tukey solves the all-pairs version of that problem using the studentized range distribution. NIST defines the studentized range as a range divided by an independent estimate of standard deviation and uses its distribution to construct simultaneous pairwise limits.

The threshold changes for sensible reasons. More within-group variability increases uncertainty and widens the required difference. Larger samples make group means more precise and narrow the margin. Increasing the number of groups changes the studentized-range critical value because more pairwise opportunities are being considered.

This is why a five-unit mean difference can be compelling in one experiment and unremarkable in another: significance is not a property of the difference alone.

Tukey HSD Formula Explained

For equal group sizes, the familiar critical difference is:

Here, is the studentized-range critical value, is the number of groups, is the residual degrees of freedom, is the pooled within-group error variance from the ANOVA model, and is the common sample size.

A pair is declared significantly different when:

The corresponding simultaneous confidence interval is:

For unequal sample sizes, the Tukey-Kramer interval changes to:

This pair-specific formula is documented in . It is more accurate than teaching Tukey-Kramer simply as “replace with the harmonic mean,” because the general pairwise calculation explicitly depends on and .

A printed q table is still useful for learning or checking hand calculations, but modern software calculates studentized-range quantities directly. R, SciPy, Statsmodels, SPSS, SAS, and Minitab remove the need to interpolate a table by hand.

How to Perform a Tukey Post Hoc Test Step by Step

The highest-impact step is choosing the correct comparison framework before doing arithmetic. First fit the appropriate ANOVA model and identify its residual error term. Then inspect whether independence, residual behavior, and the variance structure are compatible with the model. Next define the comparison family: all pairs, treatments versus a control, or a smaller set of planned hypotheses.

If all pairs matter and a common variance is credible, use standard Tukey HSD for equal group sizes or Tukey-Kramer for unequal group sizes. Compute each mean difference and its simultaneous interval, or obtain the corresponding adjusted p-value from statistical software. Finally, interpret direction, magnitude, uncertainty, and practical importance together.

The calculation should not be allowed to hide a design problem. Getting to another decimal place is far less important than recognizing that a repeated-measures design, strong heteroscedasticity, or a control-only hypothesis family requires a different analysis.

How to Interpret Tukey HSD Results

Suppose an output reports A-C = 11.2, adjusted , and a 95% simultaneous confidence interval of [3.85, 18.55]. A is estimated to be 11.2 units higher than C. The interval excludes zero, so the pair is statistically distinguishable at the stated familywise level. The interval also communicates uncertainty: values from roughly 3.9 to 18.6 units remain plausible under the model and procedure.

Animated Tukey confidence intervals showing why intervals that exclude zero indicate detected pairwise differences.

Adjusted p-values

A Tukey adjusted p-value already accounts for the all-pairs comparison family. It is normally compared directly with the chosen . Applying an additional Bonferroni correction to the same Tukey-adjusted p-values would usually adjust the same multiplicity problem twice.

Confidence intervals containing zero

A simultaneous Tukey confidence interval that contains zero means the procedure did not detect a difference for that pair at the chosen familywise level. It does not show that the two population means are identical.

Statistical significance is not practical significance

A small difference can become statistically detectable in a precise, high-powered study. A large estimated difference can remain uncertain in a noisy or small study. For decisions, report the raw mean difference and its interval, and add an appropriate effect-size measure when the discipline expects one. The p-value answers a detectability question; it does not by itself answer whether the difference is important enough to change practice.

Compact Letter Displays: What Do a, b, and ab Mean?

Compact Letter Displays (CLDs) compress many pairwise decisions into a set of letters. If A is labeled a, C is labeled b, and B is labeled ab, A and C share no letter and are significantly different under the procedure. B shares a letter with both A and C, so neither B-A nor B-C was declared significant.

The critical wording is “not declared significantly different.” A shared letter does not establish statistical equivalence.

The explicitly warns that CLDs are prone to this misinterpretation: means sharing a grouping symbol were not shown to be different, but they were not shown to be the same either. If the scientific question is whether differences are small enough to count as equivalent, use an equivalence-testing framework with a prespecified meaningful margin.

For publication figures, simultaneous confidence-interval plots often communicate direction, magnitude, and uncertainty more faithfully than letters alone.

Worked Tukey HSD Examples

The following examples are synthetic and are included to verify the mechanics rather than represent findings from a real study.

Teaching methods: a complete balanced example

Suppose three teaching methods have means A = 85.3, B = 81.7, and C = 74.1, with in every group. The ANOVA model gives and .

At , the studentized-range critical value is approximately 3.506. The standard-error term is:

so the HSD margin is approximately:

Comparison	Mean difference	Tukey-adjusted p	95% simultaneous CI	Interpretation
A vs B	3.60	.4554	[-3.75, 10.95]	Difference not detected
A vs C	11.20	.0022	[3.85, 18.55]	Significant
B vs C	7.60	.0417	[0.25, 14.95]	Significant

The useful conclusion is not simply “C is different.” A-C has a clear positive interval, while B-C only just excludes zero. The confidence intervals reveal that the certainty around those two significant findings is not the same.

Four dosage groups: neighboring means need not differ

Suppose mean reductions are Placebo = 2.1, Low = 5.8, Medium = 9.4, and High = 13.2, with , , and . The HSD margin is approximately 5.86.

Comparison	Absolute difference	Tukey-adjusted p	Result
Placebo vs Low	3.7	.3301	Not significant
Placebo vs Medium	7.3	.0103	Significant
Placebo vs High	11.1	< .001	Significant
Low vs Medium	3.6	.3535	Not significant
Low vs High	7.4	.0092	Significant
Medium vs High	3.8	.3076	Not significant

This pattern shows why a monotonic sequence of means does not imply that every neighboring step is statistically distinguishable.

Five fertilizer treatments: check every pair

Suppose , , , and means F1 = 28.4, F2 = 34.1, F3 = 41.7, F4 = 38.9, and F5 = 29.8. The HSD margin is approximately 6.02.

The significant comparisons are F1-F3, F1-F4, F2-F3, F3-F5, and F4-F5. F2-F3 is easy to miss in a shortened example because its 7.6-unit difference is less visually dramatic than several others, but it still exceeds the Tukey threshold. This is one reason a complete pairwise table is safer than describing only “key comparisons.”

Tukey-Kramer for Unequal Sample Sizes

Tukey-Kramer keeps the common-variance all-pairs framework but allows group sizes to differ. Instead of one common HSD margin, each pair receives a margin based on its two sample sizes.

Animated Tukey-Kramer diagram showing pair-specific confidence margins for unequal sample sizes.

Consider A = 85.3, B = 81.7, and C = 74.1 with , , and , while and . The approximate 95% Tukey-Kramer margins are 7.80 for A-B, 7.51 for A-C, and 7.04 for B-C. The pairs do not have equal precision because the sample sizes differ.

This distinction matters in real software. Current states that its TUKEY option becomes Tukey-Kramer when group sizes differ. Current makes the same distinction automatically when equal_var=True and the samples have unequal sizes.

Unequal sample sizes alone are not the most serious warning sign. The more difficult situation is unequal sample sizes combined with unequal variances, because the pooled MSE no longer represents all groups well.

Tukey HSD vs Games-Howell, Dunnett, Bonferroni, and Scheffe

There is no universally best post hoc test. The choice should follow the comparison family and variance structure rather than habit.

Research situation	Better starting point	Why
Every pair matters; common variance is credible	Tukey HSD / Tukey-Kramer	Designed specifically for an all-pairs family
Every pair matters; variances are unequal	Games-Howell	Uses pair-specific variance estimates and degrees of freedom
Each treatment is compared only with a control	Dunnett	Does not spend multiplicity control on irrelevant treatment-treatment pairs
A few hypotheses were prespecified	Planned contrasts with suitable adjustment	Matches the actual scientific questions
Broad arbitrary contrasts are required	Scheffe	Protects a wider family of contrasts
A flexible correction is needed for selected tests	Holm or Bonferroni	General-purpose multiplicity control

Tukey HSD vs Games-Howell

Tukey assumes a common within-group variance. Games-Howell relaxes that assumption and uses pair-specific variance estimates with Satterthwaite-style degrees of freedom. exposes this directly: use_var="equal" returns Tukey-HSD results, while use_var="unequal" returns Games-Howell results.

Tukey HSD vs Dunnett

Tukey is appropriate when A-B, A-C, B-C, and every other pair matter. Dunnett is more targeted when each treatment only needs to be compared with one control. describes Tukey as an all-pairs method and Dunnett as a treatment-versus-control procedure.

Tukey HSD vs Bonferroni or Holm

Bonferroni and Holm are general multiplicity adjustments. They can be appropriate for a deliberately selected family of comparisons. When the scientific question is specifically every pair of means, Tukey uses the structure of the all-pairs problem rather than treating the tests as an arbitrary collection.

Tukey HSD vs Scheffe

Scheffe protects all linear contrasts, a broader inferential family than pairwise differences alone. That flexibility can be useful, but it is more protection than necessary when the only target is all pairwise means.

How to Run Tukey HSD in R, Python, SPSS, SAS, Minitab, and Excel

Modern statistical software should usually be preferred over manual q-table interpolation because it uses full numerical precision and can return confidence intervals and adjusted p-values.

R

For a fitted one-way ANOVA model, R’s standard function is:

fit <- aov(score ~ group, data = dat) TukeyHSD(fit)

The current describes confidence intervals based on the studentized range and an adjustment for mildly unbalanced designs.

Python with SciPy

Current SciPy provides:

from scipy.stats import tukey_hsd result = tukey_hsd(group_a, group_b, group_c)

With equal_var=True, SciPy uses Tukey HSD for equal sample sizes and Tukey-Kramer for unequal sample sizes. With equal_var=False, it uses Games-Howell. Confidence intervals are available from the result object. ()

Python with Statsmodels

from statsmodels.stats.multicomp import pairwise_tukeyhsd result = pairwise_tukeyhsd(values, groups, use_var="equal")

In Statsmodels 0.15, use_var="unequal" switches to Games-Howell. This is a meaningful update because older Python tutorials may describe a separate third-party workflow for unequal variances. ()

SPSS

In SPSS Statistics, the one-way ANOVA Post Hoc dialog includes Tukey among procedures that assume equal variances and Games-Howell among procedures that do not. IBM also lists Dunnett for comparisons against a control. ()

SAS

In PROC GLM, the TUKEY option performs the studentized-range test for all main-effect means; unequal group sizes invoke Tukey-Kramer. The Welch option produces a separate variance-robust omnibus test and does not make the remaining pooled-variance post hoc procedures variance-robust. ()

Minitab

Minitab’s one-way ANOVA comparison workflow reports simultaneous intervals and grouping information. A pairwise simultaneous interval that excludes zero indicates a detected difference at the stated familywise level.

Excel

Excel’s standard Analysis ToolPak can produce one-way ANOVA output, but it does not provide a native Tukey post hoc command. A verified worksheet implementation, statistical add-in, external calculator, or package such as R or Python is needed for the Tukey stage.

Why manual and software results sometimes disagree

Small discrepancies usually come from q-table interpolation, rounding means or MSE before calculation, using the equal- formula on unequal groups, selecting the wrong residual degrees of freedom, or comparing against software that uses the full fitted-model error structure. Recalculate from unrounded values before assuming one implementation is wrong.

How to Report Tukey HSD Results

A good report says what differed, in which direction, by how much, and with what uncertainty. Report the relevant model context, mean difference, Tukey-adjusted p-value, and simultaneous confidence interval when available.

A concise reporting example is:

A one-way ANOVA indicated differences among teaching methods. Tukey-adjusted pairwise comparisons showed that Method A scored 11.2 points higher than Method C, adjusted p = .0022, 95% simultaneous CI [3.85, 18.55]. Method B also scored 7.6 points higher than Method C, adjusted p = .0417, 95% simultaneous CI [0.25, 14.95]. The comparison between Methods A and B did not detect a statistically significant difference, adjusted p = .4554.

Avoid changing the last sentence to “A and B were statistically identical.” Difference testing and equivalence testing answer different questions.

When an effect-size measure is appropriate to the field, report it alongside the raw-unit difference rather than using it as a replacement. For visualization, a pairwise simultaneous CI plot is generally more informative than stars alone and less likely than CLDs to imply equivalence.

Common Problems After Running Tukey HSD

ANOVA is significant but no Tukey pair is significant

This can occur. The omnibus F-test and simultaneous pairwise comparisons test different hypothesis structures. An overall pattern can be strong enough to reject equality of all means even when no individual Tukey comparison clears the more demanding all-pairs threshold. Treat this as a statistical result to interpret, not as a software contradiction.

A Tukey pair is significant but the omnibus ANOVA is not

This can also occur, which is why the “significant ANOVA is always mandatory” rule is too categorical. SAS documentation explicitly notes that most multiple-comparison methods can identify significant contrasts when the omnibus test is nonsignificant. The right response is to check the analysis plan, comparison family, assumptions, and whether the analysis was selected before or after looking at the data.

The calculator result does not match software

First check whether the groups have unequal . Then check q precision, MSE, residual degrees of freedom, alpha, sign conventions, and rounding. If the software fitted a more complex model than a one-way independent-groups ANOVA, its pairwise comparisons may use a covariance structure that a simple HSD calculator cannot reproduce.

The confidence interval is close to zero

Do not convert a borderline interval into a binary story. A lower limit of 0.02 and a lower limit of -0.02 represent almost the same numerical uncertainty even though they fall on opposite sides of a significance threshold. Report the interval and effect magnitude so readers can see that uncertainty.

Frequently Asked Questions

What is the Tukey HSD test?

Tukey HSD is a multiple-comparison procedure for testing every pair of group means while controlling error across the all-pairs comparison family.

What does HSD stand for?

HSD means Honestly Significant Difference, the traditional name for the critical mean difference used in Tukey’s procedure.

When should I use Tukey HSD?

Use it when every pairwise mean comparison is relevant and the common within-group variance assumption is reasonable. With unequal sample sizes but a credible common variance, use the Tukey-Kramer form.

What is the difference between Tukey HSD and Games-Howell?

Tukey pools a common within-group variance. Games-Howell allows variances to differ and uses pair-specific variance estimates and degrees of freedom, making it better suited to heteroscedastic all-pairs comparisons.

Can Tukey HSD handle unequal sample sizes?

Yes, through Tukey-Kramer. The standard error is calculated separately for each pair using both sample sizes rather than one common .

What does a Tukey adjusted p-value mean?

It is a p-value calibrated for the complete Tukey pairwise family. Compare it directly with the chosen familywise alpha unless the analysis plan defines an additional multiplicity layer.

What does it mean when a Tukey confidence interval includes zero?

The procedure did not detect a difference for that pair at the stated simultaneous confidence level. This is not evidence that the two population means are equivalent.

Is Tukey HSD appropriate after a two-way ANOVA?

Tukey adjustments can be applied to pairwise comparisons derived from factorial models, but the target must be defined first. Main effects, simple effects, estimated marginal means, and comparisons of interaction cells are different questions. A simple one-way calculator should not be used as a substitute for the fitted factorial model.

Is Tukey HSD the same as Tukey-Kramer?

They are part of the same all-pairs framework. Standard Tukey HSD refers to the balanced equal-sample-size formulation; Tukey-Kramer modifies the pairwise standard error for unequal group sizes.

Quick Summary: Choosing and Interpreting Tukey HSD

The best use of a Tukey HSD calculator is not simply to obtain a significant/not-significant label. First decide whether all pairwise comparisons are the relevant scientific family and whether a pooled common-variance model is credible. Use standard Tukey HSD for balanced all-pairs comparisons, Tukey-Kramer when sample sizes differ but common variance remains appropriate, and Games-Howell when the equal-variance assumption is not credible.

Then interpret the mean differences, simultaneous confidence intervals, and Tukey-adjusted p-values together. If only control comparisons or predefined hypotheses matter, a more targeted procedure can answer the research question with less unnecessary testing. Most importantly, never turn “not significantly different” into “proven equal.”