Ellsberg paradox

The Ellsberg paradox is a paradox in decision theory and experimental economics in which people's choices violate the expected utility hypothesis. It is generally taken to be evidence for ambiguity aversion. The paradox was popularized by Daniel Ellsberg, although a version of it was noted considerably earlier by John Maynard Keynes (Keynes 1921, pp.75-76, p.315, ft.2).

The paradox
Suppose you have an urn containing 30 red balls and 60 other balls that are either black or yellow. You don't know how many black or yellow balls there are, but that the total number of black balls plus the total number of yellow balls equals 60. The balls are well mixed so that each individual ball is as likely to be drawn as any other. You are now given a choice between two gambles:

Also you are given the choice between these two gambles (about a different draw from the same urn):

Since the prizes are exactly the same, it follows that you will prefer Gamble A to Gamble B if, and only if, you believe that drawing a red ball is more likely than drawing a black ball (according to expected utility theory). Also, there would be no clear preference between the choices if you thought that a red ball was as likely as a black ball. Similarly it follows that you will prefer Gamble C to Gamble D if, and only if, you believe that drawing a red or yellow ball is more likely than drawing a black or yellow ball. If drawing a red ball is more likely than drawing a black ball, then drawing a red or yellow ball is also more likely than drawing a black or yellow ball. So, supposing you prefer Gamble A to Gamble B, it follows that you will also prefer Gamble C to Gamble D. And, supposing instead that you prefer Gamble D to Gamble C, it follows that you will also prefer Gamble B to Gamble A.

When surveyed, however, most people strictly prefer Gamble A to Gamble B and Gamble D to Gamble C. Therefore, some assumptions of the expected utility theory are violated.

Mathematical demonstration
Mathematically, your estimated probabilities of each color ball can be represented as: R, Y, and B. If you strictly prefer Gamble A to Gamble B, by utility theory, it is presumed this preference is reflected by the expected utilities of the two gambles: specifically, it must be the case that

$$R \cdot U(\$100) + (1-R) \cdot U(\$0) > B\cdot U(\$100) + (1-B) \cdot U(\$0) $$

where $$U(\cdot)$$ is your utility function. If $$U(\$100) > U(\$0)$$ (you strictly prefer $100 to nothing), this simplifies to:

$$R > B \; $$

If you also strictly prefer Gamble D to Gamble C, the following inequality is similarly obtained:

$$B\cdot U(\$100) + Y\cdot U(\$100) + R \cdot U(\$0) > R \cdot U(\$100) + Y\cdot U(\$100) + B \cdot U(\$0) $$

This simplifies to:

$$B > R \;$$

This contradiction indicates that your preferences are inconsistent with expected-utility theory.

Generality of the paradox
Note that the result holds regardless of your utility function. Indeed, the amount of the payoff is likewise irrelevant. Whichever gamble you choose, the prize for winning it is the same, and the cost of losing it is the same (no cost), so ultimately, there are only two outcomes: you receive a specific amount of money, or you receive nothing. Therefore it is sufficient to assume that you prefer receiving some money to receiving nothing (and in fact, this assumption is not necessary -- in the mathematical treatment above, it was assumed U($100) > U($0), but a contradiction can still be obtained for U($100) < U($0) and for U($100) = U($0).

In addition, the result holds regardless of your risk aversion. All the gambles involve risk. By choosing Gamble D, you have a 1 in 3 chance of receiving nothing, and by choosing Gamble A, you have a 2 in 3 chance of receiving nothing. If Gamble A was less risky than Gamble B, it would follow that Gamble C was less risky than Gamble D (and vice versa), so, risk is not averted in this way.

However, because the exact chances of winning are known for Gambles A and D, and not known for Gambles B and C, this can be taken as evidence for some sort of ambiguity aversion which cannot be accounted for in expected utility theory. It has been demonstrated that this phenomenon occurs only when the choice set permits comparison of the ambiguous proposition with a less vague proposition (but not when ambiguous propositions are evaluated in isolation; See Fox and Tversky, 1995).

Possible Explanations
There have been various attempts to provide decision-theoretic explanations of Ellsberg's observation. Since the probabilistic information available to the decision-maker is incomplete, these attempts sometimes focus on quantifying the non-probabilistic ambiguity which the decision-maker faces. That is, these alternative approaches sometimes suppose that the agent formulates a subjective (though not necessarily Bayesian) probability for possible outcomes.

One such attempt is based on info-gap decision theory. The agent is told precise probabilities of some outcomes, though the practical meaning of the probability numbers is not entirely clear. For instance, in the gambles discussed above, the probability of a red ball is 30/90, which is a precise number. Nonetheless, the agent may not distinguish, intuitively, between this and, say, 30/91. No probability information whatsoever is provided regarding other outcomes, so the agent has very unclear subjective impressions of these probabilities.

In light of the ambiguity in the probabilities of the outcomes, the agent is unable to evaluate a precise expected utility. Consequently, a choice based on maximizing the expected utility is also impossible. The info-gap approach supposes that the agent implicitly formulates info-gap models for the subjectively uncertain probabilities. The agent then tries to satisfice the expected utility and to maximize the robustness against uncertainty in the imprecise probabilities. This robust-satisficing approach can be developed explicitly to show that the  choices of decision-makers should display precisely the preference reversal which Ellsberg observed (Ben-Haim, 2006, section 11.1).