What Is a t-Test?

A t-test is an inferential statistical method used to test a hypothesis about a population mean or a difference between means. It uses the t-distribution, which accounts for the extra uncertainty created when the population standard deviation is unknown and must be estimated from sample data.

Depending on the study design, a t-test can compare one sample with a fixed value, compare two unrelated groups, or compare two linked measurements. The output normally includes a t-statistic, degrees of freedom and a p-value. A complete analysis should also report the estimated effect, a confidence interval and an effect-size measure.

GENERAL FORMULA: t = observed effect ÷ estimated standard error. The numerator is the mean or mean difference of interest, and the denominator reflects its sampling uncertainty.

When Should You Use a t-Test?

• The outcome is numerical, and its mean is scientifically meaningful.
• The research question concerns one population mean or a difference between two population means.
• The observations match the independence or pairing structure required by the selected test.
• Extreme outliers, data-entry errors and serious measurement problems have been investigated.
• The procedure is selected from the study design, not from whichever method produces the smallest p-value.

Typical applications include comparing mean blood pressure between treatment groups, mean test scores under two teaching methods, average production time under two processes, or reaction time before and after an intervention.

SCOPE: A t-test is designed for means. It is not automatically suitable for proportions, counts, categories, ranks, survival outcomes, clustered observations or repeated measurements collected at several time points.

How to Choose the Correct t-Test

The first decision is whether the observations are independent or linked. This determines the standard error and cannot be corrected later by changing a software option.

Research design	Appropriate procedure	Typical question
One numerical sample vs a fixed value	One-sample t-test	Is the population mean different from 100?
Two unrelated groups	Welch or pooled independent t-test	Do the two population means differ?
Same participants measured twice	Paired-samples t-test	Is the average change different from zero?
Matched participants or linked pairs	Paired-samples t-test	Is the mean within-pair difference zero?

BEST-PRACTICE DEFAULT: For two independent groups, Welch’s t-test is often the safer default when equal population variances cannot be clearly justified.

The Main Types of t-Tests

1. One-sample t-test

A one-sample t-test compares the sample mean with a specified reference value. For example, a manufacturer may claim that average battery life is 10 hours. The test evaluates whether the population mean differs from 10 hours.

Component	Expression
Null hypothesis	H₀: μ = μ₀
Two-sided alternative	H₁: μ ≠ μ₀
Test statistic	t = (x̄ − μ₀) / (s / √n)
Degrees of freedom	df = n − 1

2. Independent-samples t-test

An independent-samples t-test compares the means of two unrelated groups, such as a treatment group and a control group. No observation in one group should be naturally paired with an observation in the other group.

Pooled-variance version

The traditional Student independent t-test assumes equal population variances and combines the group variance estimates into a pooled estimate. This assumption should be supported by the design and subject-matter context, not accepted only because a preliminary variance test is non-significant.

3. Welch’s t-test

Welch’s t-test also compares two independent means, but it does not assume equal population variances. It adjusts both the standard error and degrees of freedom and is particularly useful when group sizes or variances differ.

• Group variances are noticeably different.
• Sample sizes are unequal.
• The equal-variance assumption cannot be defended.
• One group is both smaller and more variable than the other.

4. Paired-samples t-test

A paired-samples t-test compares linked observations, such as measurements from the same participants before and after an intervention. The analysis converts each pair into a difference score and then tests whether the mean difference is zero.

CRITICAL ASSUMPTION: For a paired t-test, the normality condition concerns the distribution of the difference scores, not the two raw measurement sets considered separately.

Independent design	Paired design
Different people or units in each group	The same people or linked units provide both values
Between-group variability enters the standard error	Within-pair differences are analysed
Use Welch or pooled independent t-test	Use paired-samples t-test
Example: Drug A group vs Drug B group	Example: blood pressure before vs after treatment

One-Tailed vs Two-Tailed Tests

A two-tailed test evaluates differences in either direction. A one-tailed test evaluates a direction that must be specified before the data are examined.

Direction	Alternative hypothesis	Use when
Two-tailed	H₁: μ₁ − μ₂ ≠ 0	Both increases and decreases would matter.
Upper-tailed	H₁: μ₁ − μ₂ > 0	Only an increase is relevant and this was prespecified.
Lower-tailed	H₁: μ₁ − μ₂ < 0	Only a decrease is relevant and this was prespecified.

DO NOT REVERSE-ENGINEER THE HYPOTHESIS: Choosing a one-tailed test after seeing the direction of the result invalidates the intended error rate. In exploratory work, a two-tailed test is usually more defensible.

How a t-Test Works

1. Estimate the relevant sample mean or mean difference.
2. Estimate its standard error.
3. Divide the estimate by the standard error to obtain the t-statistic.
4. Use the appropriate t-distribution and degrees of freedom to calculate a p-value and confidence interval.
5. Interpret the estimate, uncertainty and practical importance together.

The sign of the t-statistic indicates direction. Its absolute value shows how many estimated standard errors the observed effect lies from the null value. A t-value of 0 means the estimate equals the null value; t = 2 means it is two standard errors above it; and t = −3 means it is three standard errors below it.

Degrees of Freedom, p-Values and Confidence Intervals

Degrees of freedom

Degrees of freedom describe how much independent information is available for estimating uncertainty. They determine the shape of the t-distribution used for inference.

Procedure	Degrees of freedom
One-sample t-test	n − 1
Paired t-test	n − 1, where n is the number of complete pairs
Pooled independent t-test	n₁ + n₂ − 2
Welch’s t-test	Approximate value based on both variances and sample sizes

What the p-value means

The p-value is the probability, under the null hypothesis and the rest of the statistical model, of obtaining a result at least as incompatible with the null hypothesis as the result observed. It is not the probability that the null hypothesis is true and it is not a measure of practical importance.

INTERPRET CAREFULLY: A p-value below .05 does not prove the alternative hypothesis. A p-value above .05 does not prove equality or the absence of an effect.

Confidence intervals and effect sizes

A confidence interval gives a range of population-effect values that are reasonably compatible with the data and model. It communicates direction, magnitude and precision more clearly than a significance label alone.

An effect size describes the magnitude of a difference. Cohen’s d is commonly used for two-group mean comparisons, while Hedges’ g applies a small-sample correction. Conventional labels such as small, medium and large should not replace interpretation in the context of the field, measurement scale, costs and benefits.

Assumptions and Robustness Checks

Assumption or check	What it means
Numerical outcome	The dependent variable is quantitative and a mean is meaningful.
Correct dependence structure	Independent groups require independent observations; paired designs require linked observations within pairs.
Credible sampling or assignment	Random sampling supports generalisation; random assignment supports causal treatment comparisons.
No unaddressed extreme outliers	Means and standard deviations are sensitive to extreme observations, especially in small samples.
Reasonable distributional behaviour	Normality concerns the relevant population distribution or paired differences.
Equal variances only when pooling	The equal-variance assumption applies to the pooled independent test, not Welch’s test.

How to assess normality responsibly

• Inspect histograms, box plots and Q–Q plots.
• Consider sample size, group balance, skewness and outliers.
• Use knowledge of the measurement process and data-generating mechanism.
• Do not treat a Shapiro–Wilk result as an automatic pass/fail switch.
• For a paired test, inspect the paired differences.

WHY NORMALITY TESTS CAN MISLEAD: With large samples, a formal test may detect minor harmless deviations. With small samples, it may lack power to detect important departures. Visual and substantive checks remain essential.

Outlier protocol

1. Check whether the value is a data-entry or measurement error.
2. Review the observation in scientific context.
3. Apply exclusion rules defined before examining outcomes whenever possible.
4. Report all exclusions transparently.
5. Use robust or resampling methods as a sensitivity analysis when appropriate.

Worked Welch’s t-Test Example

Suppose a researcher compares examination scores from two independent teaching methods.

Group	Sample size	Mean	Standard deviation
Method A	30	82	8
Method B	28	76	9

1. Estimated mean difference: 82 − 76 = 6 points.
2. Standard error: √(8²/30 + 9²/28) ≈ 2.242.
3. t-statistic: 6 / 2.242 ≈ 2.68.
4. Welch–Satterthwaite degrees of freedom: approximately 54.11.
5. Two-sided p-value: approximately .010.
6. 95% confidence interval: approximately [1.51, 10.49].
7. Descriptive Cohen’s d using the pooled standard deviation: approximately 0.71.

Mean difference	t(df)	p-value	95% CI
6.00	2.68 (54.11)	.010	[1.51, 10.49]

INTERPRETATION: Method A scored six points higher on average. Under the Welch model, the data were inconsistent with a zero population mean difference at the .05 level. The interval indicates that differences from about 1.51 to 10.49 points remain compatible with the data and model.

Publication-ready result

Students taught with Method A scored higher (M = 82.00, SD = 8.00) than students taught with Method B (M = 76.00, SD = 9.00). A two-sided Welch’s independent-samples t-test estimated a mean difference of 6.00 points, 95% CI [1.51, 10.49], t(54.11) = 2.68, p = .010, Cohen’s d = 0.71.

How to Report a t-Test

• State the test type and whether it was one-sided or two-sided.
• Report sample sizes, means and standard deviations.
• Report the estimated mean or mean difference.
• Give the t-statistic and degrees of freedom.
• Report the exact p-value whenever possible.
• Include a confidence interval for the effect.
• Include an effect-size estimate and explain how it was calculated.
• Describe relevant assumption checks, exclusions and sensitivity analyses.

AVOID BINARY REPORTING: Do not report only “significant” or “not significant.” Readers need the estimate, uncertainty and design information to judge practical meaning.

Reporting templates

Paired test: Scores increased from before the intervention (M = __, SD = __) to after the intervention (M = __, SD = __). The mean paired change was __, 95% CI [__, __], t(df) = __, p = __, paired effect size = __.

One-sample test: The sample mean (M = __, SD = __) differed from the reference value of __ by __ units, 95% CI [__, __], t(df) = __, p = __, effect size = __.

R, Python and Excel Examples

R

# Welch independent-samples t-test result <- t.test(score ~ group, data = my_data, alternative = "two.sided", conf.level = 0.95) print(result) # Paired t-test t.test(my_data$after, my_data$before, paired = TRUE) # One-sample t-test t.test(my_data$score, mu = 100)

R DEFAULT: For two independent groups, R’s t.test() uses Welch’s procedure unless var.equal = TRUE is specified.

Python with SciPy

from scipy import stats result = stats.ttest_ind(group_a, group_b, equal_var=False, alternative="two-sided") print(result.statistic, result.pvalue) paired = stats.ttest_rel(after, before, alternative="two-sided") one_sample = stats.ttest_1samp(scores, popmean=100, alternative="two-sided")

Microsoft Excel

General syntax: =T.TEST(array1, array2, tails, type)

Type argument	Procedure
1	Paired t-test
2	Two-sample equal-variance t-test
3	Two-sample unequal-variance test

=T.TEST(A2:A31, B2:B29, 2, 3)

EXCEL LIMITATION: T.TEST returns a p-value, not a complete research report. Calculate or obtain the mean difference, confidence interval, t-statistic, degrees of freedom and effect size separately.

t-Test vs Other Statistical Tests

Comparison	Key distinction
t-test vs z-test	A z-test uses a known population standard deviation. A t-test estimates uncertainty from the sample. Sample size 30 is not a universal switch.
t-test vs ANOVA	A t-test handles one or two means. ANOVA or regression handles three or more groups or more complex designs.
t-test vs Mann–Whitney U	A t-test directly concerns means. Mann–Whitney is rank-based and does not automatically test a difference in medians.
Paired t-test vs Wilcoxon signed-rank	The paired t-test targets mean differences. Wilcoxon uses ranks and has different assumptions and interpretation.
t-test vs permutation or bootstrap	Resampling can support robustness or unusual designs, but it must preserve pairing, clustering and repeated-measure structures.

Power and Sample Size

Statistical power is the probability that a testing procedure rejects the null hypothesis when a specified nonzero effect is present. Power generally increases with larger samples, larger true effects and lower measurement noise. It decreases with small samples, small effects, high variability and inefficient group allocation.

• Specify a scientifically meaningful effect.
• Use a realistic variance estimate.
• State the planned test, sidedness and significance level.
• Choose the desired statistical power.
• Allow for attrition or missing data.
• For paired studies, account for the expected within-person correlation.

Common Mistakes

Mistake	Better practice
Choosing the procedure from the smallest p-value	Select the test from the design and analysis plan.
Treating repeated observations as independent	Use paired, repeated-measures, clustered or mixed-effects methods.
Assuming equal variances without justification	Prefer Welch’s test when pooling is not defensible.
Using normality tests as pass/fail gates	Combine plots, sample size, skewness, outliers and subject knowledge.
Reporting only the p-value	Report the estimate, confidence interval and effect size.
Interpreting non-significance as equality	Use equivalence testing when the goal is to show an effect is sufficiently small.
Deleting outliers to obtain significance	Use transparent, defensible exclusion rules.
Making causal claims from observational data	A t-test alone cannot remove confounding.

Frequently Asked Questions

What is a t-test in simple terms?

It evaluates whether an observed mean or difference between means is large relative to its estimated sampling uncertainty.

What are the main types of t-tests?

The broad designs are one-sample, independent-samples and paired-samples tests. Independent tests include pooled and Welch versions.

What is Welch’s t-test?

It compares two independent means without assuming equal population variances and adjusts the standard error and degrees of freedom.

Does a t-test require normally distributed raw data?

The relevant condition depends on the design. For a paired test, it concerns the paired differences. Moderate departures may be acceptable in larger samples without severe outliers.

Can a t-test compare more than two groups?

A standard independent t-test compares two groups. Use ANOVA, regression or another joint model for three or more groups.

Does p < .05 prove the alternative hypothesis?

No. It means the result crossed the selected decision threshold under the null model and assumptions.

Does p > .05 prove there is no difference?

No. It indicates insufficient evidence to reject the specified null hypothesis. The confidence interval shows which effects remain plausible.

What should be reported with a t-test?

Report descriptive statistics, the estimated effect, confidence interval, t-statistic, degrees of freedom, exact p-value, effect size and relevant assumption information.

Conclusion

The t-test remains one of the most useful procedures for analysing means, but reliable use requires more than comparing a p-value with .05. Start by identifying whether observations are independent or paired, select the correct procedure before examining results, and investigate data quality, outliers and the assumptions relevant to that procedure.

For two independent groups, Welch’s t-test is often a sensible choice when equal population variances cannot be confidently justified. For repeated or matched observations, analyse within-pair differences with a paired t-test. Interpret the p-value alongside the estimated difference, confidence interval, effect size, power and research design.

FINAL TAKEAWAY: Choose the test from the design, report the effect with uncertainty, and interpret statistical evidence in context.

References and Further Reading

1. NIST/SEMATECH e-Handbook of Statistical Methods: two-sample t-test.
2. R Documentation: Student’s t-Test (t.test).
3. Penn State STAT 200: paired means.
4. American Statistical Association: statement on statistical significance and p-values.
5. SciPy documentation: scipy.stats.ttest_ind, ttest_rel and ttest_1samp.