Sampling with/without replacement

Tags:

If samples are taken without replacement(i.e., sample taken is not put into the bin again) from population, then the sample follows hypergeometric distribution. In this case, each sample is not independent from each other. For example, if I take 1 out of the bin containing 1, 1, 2, 2, 3, then there’s no choice but to pick one from 1, 2, 2, 3 if I want one more sample.

On the other hand, if samples are taken with replacement, then it follows binomial distribution. And in this case, samples are independent from each other. If I pick 1 out of 1, 1, 2, 2, 3 and then put 1 back to the bin for picking up another sample, then I can choose one from 1, 1, 2, 2, 3 once again.

What’s interesting, however, is that hypergeometric distribution can be approximated by binomial distribution if the population size is very large compared to the sample size (which usually is true). Note, once again, that I’m saying that population size matters and not the sample size.

Read, for example, Sampling With and Without Replacement and the Hypergeometric and Binomial Distributions by Jonathan D. Cryer,  http://www.ma.utexas.edu/users/parker/sampling/repl.htm and http://www.ma.utexas.edu/users/parker/sampling/woreplshort.htm.