What Estimating the SE CostsStatLens

Instructor notes — what an estimated standard error costs the interval, for a mean and for a proportion

← Open the demo · print this page for the lectern.

The one-sentence version

A confidence interval is x̄ ± multiplier × SE. The honest margin needs σ, which you do not have. Substituting s keeps the multiplier honest-looking and makes the interval the wrong length — too short, on average — and t* is the number chosen to put the long-run capture rate back where the label says.

Running it ~10 min

Open on recipe 1 (?me=1, the default). Point at the axis: it is in true standard errors, so nothing on it depends on the sample. The shaded band is the middle 95% of sample means — the ones a correctly-sized interval reaches μ from. Drag x̄ in and out of the band and the interval flips with it, which is last lesson’s picture (Why ±2 SE Works).
Ask what is wrong with recipe 1 before you click anything. The answer is not that it is inaccurate — it is exactly right. It is that σ is not a number you have. That distinction is worth a minute.
Click recipe 2. Nothing changes except the letter in the numerator — same 1.96 — and the interval visibly shortens, because this sample’s s came in at 0.90 σ. At the default position x̄ = 1.90 SE, it now misses. Say the uncomfortable part out loud: this x̄ is inside the middle 95%. It is a perfectly typical sample. And the interval missed anyway.
Click recipe 3. t* = 2.179 instead of 1.960, the interval lengthens back to 1.961 SE, and μ is caught again.
Now break it. Drag This sample’s s up past 1.0. Recipe 2 is now too long, and recipe 3 is longer still. This is the step most demos skip, and it is the one that stops students believing something false.

The misconception this demo can create if you stop at step 4. “t* corrects for s being too small.” It does not. t* is one fixed number for a given n and level; it cannot know which way your s went. In a sample where s came in low, the t interval is still too short. What t* fixes is the rate, over all samples — which is why the tally, not the single picture, is the proof.

The tally is the argument ~5 min

Press Draw 100 three or four times. All three recipes are scored on the same samples, so the comparison is clean. The right-hand column is exact arithmetic rather than the tally, and the simulation walks onto it:

Margin of error	Long-run capture at 95%, n = 13	n = 5	n = 40
z*σ/√n	95.0%	95.0%	95.0%
z*s/√n	92.6%	87.8%	94.3%
t*s/√n	95.0%	95.0%	95.0%

Set n = 5 if you want the gap to be unmissable: a “95%” interval that captures 88% of the time. Set n = 40 and the two usable recipes nearly agree — which is the honest reason nobody worries about this in a large sample, and why the distinction is a small-sample one.

Discussion question. s is very nearly unbiased — at n = 13 its average is about 0.98 σ, two per cent low. But t* is eleven per cent bigger than z*. Where do the other nine points come from?

The answer: not from bias, from variability. The losses and the gains are not symmetric. A sample whose s comes in low gets a short interval — and a short interval is exactly what you cannot afford. A sample whose s comes in high gets a long one, but it was going to capture μ anyway, so the extra length buys nothing back. Dividing by a random quantity gives (x̄−μ)/(s/√n) heavier tails than a normal, and heavier tails need a wider multiplier. That is the t distribution, and that is the whole of it.

The proportion mode ~8 min

The Parameter: Mean / Proportion toggle changes which substitution is on trial. For a mean you do not have σ and you put s in its place. For a proportion you do not have p — and the stand-in is p̂, the very number the interval is built around.

The structural difference, worth pointing at. Mean mode has two knobs: where x̄ landed, and what s came out as. They are separate numbers from the same sample. Proportion mode has one, because p̂ has to do both jobs — it is the estimate and the thing inside the standard error. Drag it and watch the interval's length change as it moves, which never happens in mean mode.

Which way it goes is worth predicting before you drag. Since p̂(1−p̂) peaks at 0.5, a p̂ that lands nearer 0.5 than p is inflates the estimated SE and makes the interval too long; one that lands further out shrinks it and makes the interval too short — and short is the one you cannot afford.

Open on the defaults (p = 0.20, n = 50). np = 10 exactly, so the success-failure condition is met. Ask what coverage they expect. The answer in the table is 93.8%, and that is exact arithmetic, not a simulation wobble.
Draw 100, three or four times. Both recipes are scored on the same samples. The tally walks onto the exact column.
Slide p down to 0.10 and n to 40. Now np = 4, the condition fails, and coverage is 91.4%. At p = 0.05, n = 20 it is 64%.
Ask for the fix. In mean mode a bigger multiplier did it. Here there is no third row — and the absence is the point.

The sentence the two modes earn together. Estimating the standard error costs you coverage. For a mean, a bigger multiplier buys it back exactly, at every μ and σ — that is t*. For a proportion, no multiplier can, because how far short it falls depends on p, which is the thing you do not know. That is why this course simulates.

The long-run column is exact, and that matters here. p̂ can only be k/n, so the column sums the binomial over the n+1 values it can take rather than integrating a normal. The two disagree by two or three points, in both directions — and at one particular (p, n) the comparison can even come out backwards. Don't teach from a single cell; slide p and teach the pattern.

Expert mode only, and deliberately so. Turn on Show: Detailed and two shaded strips appear, decomposing the shortfall: ordinary samples whose interval misses anyway, and unusual ones that reach p regardless because their estimated SE came out large. It is a genuinely nice picture and it is above this course — it answers "where exactly did the lost coverage go", which no intro student is asking. Leave it off unless you have a reason.

Where it fits

The mean mode goes immediately before the first one-sample t interval, and immediately after Why ±2 SE Works — that page establishes the capture argument with the multiplier taken for granted, and this one asks where the multiplier comes from. It pairs with Confidence Interval Coverage, which measures the same rate for intervals built on real data, and with t Distribution Calculator for looking t* up directly.

Links worth having ready

?n=13&s=0.9&xbar=1.9&me=1 — the opening position (this is also the default).
?n=5&me=2 — the small-sample version, where z*s/√n runs at 87.8%.
?n=40&me=2 — the large-sample version, where it is 94.3% and nobody cares.
?s=1.25&me=3 — the sample where s came in high and t* makes an already-long interval longer.

Credit

Both modes are Todd Will's, taught in Mathematica and asked for here via Jeff — the mean in October 2026, the proportion on 2026-10-12, with the brief "showing the effect of replacing p with p̂, a parallel to showing the effect of replacing σ with s". Putting them behind one toggle is what makes that parallel something you can see rather than something a caption claims.

His version fixes s = 0.9 σ and defines t* = z* ÷ 0.9 so the third recipe lands exactly on the first. ?n=13&s=0.9 reproduces that almost to the decimal, because t* at 12 df really is 1.112 × z*. The s slider and the tally are the additions, for the reason in the red box above.

← Back to the demo