Instructor Guide: Confidence Intervals by Repeated Sampling

Companion notes for the StatLens conceptual demo — coverage drawn on the sampling distribution, and what “95% confident” is a claim about

Launch in Presentation Mode

Overview ~12 min

The Sampling Distribution Lab's three tiers — population, one sample, the distribution of the statistic — with one addition: every sample gets a confidence interval, and every dot is coloured by whether that interval caught μ. Blue caught it; orange missed.

The interval is the one students actually compute, x̄ ± t*s/√n, using each sample's own s. So the widths differ dot to dot and the coverage lands near the stated level rather than exactly on it — which is the honest version and the more interesting one.

How this differs from CI Coverage. CI Coverage draws the intervals as a stack of bars and counts them. This draws the same count on the sampling distribution, so you can see which samples produce the misses: they are mostly the ones far from μ. Run them in either order; they answer the same question with different pictures.

Running it

  1. Press +1 a few times. Each draw puts a sample in the middle tier with its interval drawn under the axis, and drops one coloured dot into the distribution. Ask after a miss: what was unusual about that sample?
  2. Press +100. About 95 blue, about 5 orange. Ask students to describe where the orange dots are before you say anything.
  3. Hover along the distribution. Each dot unpacks into its sample and its interval. Near the middle they all reach μ; out in the tails they stop short.
  4. Switch to 99%. The samples do not change — only the intervals. More of them reach μ, and every interval is wider. That trade is the whole content of “choose a confidence level”.
The misconception this page exists for. “There is a 95% chance μ is in my interval.” Your interval either contains μ or it does not — there is no chance left in it once the sample is drawn. The 95% is the success rate of the recipe across the samples you might have drawn, which is exactly what the field of dots shows.

The move that makes it land ~4 min

Press Hide μ.

Every dot goes grey, μ leaves all three charts, and the count is replaced by a sentence saying nothing can be counted. This is not the page being coy: with μ hidden there is genuinely no way to tell which intervals worked — not for the page, and not for a statistician with real data. That is the situation every student is in every time they compute an interval.

Ask, with μ hidden: “Point at a dot. Did that interval catch μ?” Let the silence do the work. Then: “What would you need to know to answer?”

Then press Reveal μ. The colours snap back. The sentence worth saying out loud: your dot was one colour or the other the whole time. Hiding μ never changed which — it only took away your ability to see it. That is why the claim has to be about the method.

For a lecture, open the page with ?parameter=hidden&samples=100. The class meets a grey field of dots and an unanswerable question, and you reveal μ at the moment it pays off. ?seed=anything reproduces the identical run.

Common questions

“Why isn't it exactly 95?”

Two reasons, both worth naming. It is a random sample of samples, so the count wobbles. And the interval uses s in place of σ — the t distribution corrects for that on average, not in every case. At small n the coverage can sit a little under the advertised level.

“Why did that dot near the middle miss?”

Because its s came out unusually small, so its interval was unusually narrow. This is the detail the coloured-dot picture reveals that a stack of bars hides: capture is not purely a matter of being close to μ. Hover the dot and compare its s with the population σ.

“Does a bigger sample make my interval right?”

No — it makes it narrower, not more certain to be correct. Raise n and the dots pull in toward μ and the intervals shrink, but the proportion that capture stays about the same. The level controls the hit rate; n controls the precision.

Where it fits

After sampling distributions, before or beside bootstrap intervals. The sequence that works: Sampling Distributions (what varies) → here (what a method built on that variation achieves) → Why the Percentile CI Works (why resampling one sample may stand in for sampling the population at all).

StatLens · Confidence Intervals by Repeated Sampling · conceptual/population-ci/