Binomial distribution

((StatsPsy}}


 * See binomial (disambiguation) for a list of other topics using that name.

In probability theory and statistics, the binomial distribution is the discrete probability distribution of the number of successes in a sequence of n independent yes/no experiments, each of which yields success with probability p. Such a success/failure experiment is also called a Bernoulli experiment or Bernoulli trial. In fact, when n = 1, then the binomial distribution is the Bernoulli distribution. The binomial distribution is the basis for the popular binomial test of statistical significance.

Occurrence
A typical example is the following: assume 5% of the population is HIV-positive. You pick 500 people randomly. How likely is it that you get 30 or more HIV-positives? The number of HIV-positives you pick is a random variable X which follows a binomial distribution with n = 500 and p = 0.05 (when picking the people with replacement). We are interested in the probability Pr[X &ge; 30].

Probability mass function
In general, if the random variable X follows the binomial distribution with parameters n and p, we write X ~ B(n, p). The probability of getting exactly k successes is given by the probability mass function:


 * $$f(k;n,p)={n\choose k}p^k(1-p)^{n-k}\,$$

for $$k=0,1,2,\dots,n$$ and where


 * $${n\choose k}=\frac{n!}{k!(n-k)!}$$

is the binomial coefficient "n choose k" (also denoted C(n, k) or nCk), whence the name of the distribution. The formula can be understood as follows: we want k successes (pk) and n &minus; k failures ((1 &minus; p)n &minus; k). However, the k successes can occur anywhere among the n trials, and there are C(n, k) different ways of distributing k successes in a sequence of n trials.

Distribution function
The cumulative distribution function can be expressed in terms of the regularized incomplete beta function, as follows:


 * $$ F(k;n,p) = I_{1-p}(n-k, k+1) \!$$.

For $$k \leq n\,p$$, upper bounds for the lower tail of the distribution function can be derived. In particular, Hoeffding's inequality yields the bound


 * $$ F(k;n,p) \leq \exp\left(-2 \frac{(n\,p-k)^2}{n}\right), \!$$

and Chernoff's inequality can be used to derive the bound


 * $$ F(k;n,p) \leq \exp\left(-\frac{1}{2\,p} \frac{(n\,p-k)^2}{n}\right). \!$$

Mean, standard deviation, and mode
If X ~ B(n, p) (that is, X is a binomially distributed random variate), then the expected value of X is


 * $$E[X]=np\,$$

and the variance is


 * $$\mbox{var}(X)=np(1-p).\,$$

This fact is easily proven as follows. Suppose first that we have exactly one Bernoulli trial. We have two possible outcomes, 1 and 0, with the first having probability p and the second having probability 1 &minus; p; the mean for this trial is given by &mu; = p. Using the usual formula for standard deviation, we have


 * $$\sigma = \sqrt{\sum_{k=1}^n (x_k - \mu)^2 f(x_k)} = \sqrt{(1 - p)^2p + (-p)^2(1 - p)} = \sqrt{p(1-p)} \Rightarrow \sigma^2 = p(1 - p).$$

Now suppose that we want the variance for n such trials (i.e. for the general binomial distribution). Since the trials are independent, we may add the variances for each trial, giving


 * $$\sigma^2_n = \sum_{k=1}^n \sigma^2 = np(1 - p). \quad \Box$$

The most likely value or mode of X is given by the largest integer less than or equal to (n + 1)p; if m = (n + 1)p is itself an integer, then m &minus; 1 and m are both modes.

Is it a binomial distribution? A mnemonic

 * Bi = Are there TWO possible outcomes? (i.e., yes or no, win or lose)
 * Nom = Is there a fixed NUMBER of observations or items of interest?
 * I = Is each observation INDEPENDENT?
 * Al = Is the probability for ALL outcomes equal?

(However, the letters nom are actually derived from a Latin word that means "name", not "number".)

Relations to other distributions

 * If X ~ B(n, p) and Y ~ B(m, p) are independent binomial variables, then X + Y is again a binomial variable; its distribution is


 * $$X+Y \sim B(n+m, p).\,$$


 * Two other important distributions arise as approximations of binomial distributions:




 * If n is large enough and the skew of the distribution is not too great, then an excellent approximation(provided a suitable continuity correction is used) to B(n, p) is given by the normal distribution


 * $$ N(np, np(1-p)).\,$$


 * Specifically, if both np and n(1 &minus; p) are greater than 5 (the specific number varies from source to source, and is arbitrary; some sources give 10), then the normal approximation is suitable. Another commonly used "rule" holds that the above normal approximation is appropriate only if


 * $$\mu \pm 3 \sigma = np \pm 3 \sqrt{np(1-p)} \in [0,n]$$.


 * This approximation is a huge time-saver; historically, it was the first use of the normal distribution, introduced in Abraham de Moivre's book The Doctrine of Chances in 1733. Nowadays, it can be seen as a consequence of the central limit theorem since B(n, p) is a sum of n independent, identically distributed 0-1 indicator variables. Warning: this approximation gives inaccurate results unless a continuity correction is used. Note: that the picture gives the normal and binomial probability density functions (PDF) and not the cumulative distribution functions.


 * For example, suppose you randomly sample n people out of a large population and ask them whether they agree with a certain statement. The proportion of people who agree will of course depend on the sample. If you sampled groups of n people repeatedly and truly randomly, the proportions would follow an approximate normal distribution with mean equal to the true proportion p of agreement in the population and with standard deviation &sigma; = (p(1 &minus; p)/n)1/2. Large sample sizes n are good because the standard deviation gets smaller, which allows a more precise estimate of the unknown parameter p.


 * If n is large and p is small, so that np is of moderate size, then the Poisson distribution with parameter &lambda; = np is a good approximation to B(n, p).

The formula for Bézier curves was inspired by the binomial distribution.

Limits of binomial distributions

 * As n approaches &infin; and p approaches 0 while np remains fixed at &lambda; > 0 or at least np approaches &lambda; > 0, then the Binomial(n, p) distribution approaches the Poisson distribution with expected value &lambda;.


 * As n approaches &infin; while p remains fixed, the distribution of


 * $${X-np \over \sqrt{np(1-p)\ }}$$


 * approaches the normal distribution with expected value 0 and variance 1.