Probability

The chance of at least one is one minus the chance of none.

Before this Lesson

A lab runs eight independent tests, and each has a 10% chance of failing. What's the chance that at least one fails? Adding gets you 80%, which sounds fine until eleven tests give you 110%. No chance can top 100%, so adding was wrong all along. The real answer is about 57%, and two rules get you there.

Counting what can happen

When every outcome is equally likely, probability is counting: the outcomes you care about, divided by all the outcomes there are. Two dice can land 6×6=366 \times 6 = 36 ways, all equally likely, as long as a 1 then a 6 and a 6 then a 1 count as different.

Second die First die 1 1 2 2 3 3 4 4 5 5 6 6 2 3 4 5 6 7 3 4 5 6 7 8 4 5 6 7 8 9 5 6 7 8 9 10 6 7 8 9 10 11 7 8 9 10 11 12
The 36 outcomes of two dice, with the sum in each cell

Six cells hold a 7, so the chance of a sum of 7 is 6/36≈0.1676/36 \approx 0.167. As the Handbook's Laws of Probability put it, every probability lies from 0, impossible, to 1, certain. And an event either happens or it doesn't, so those two chances add to 1:

P(not A)=1−P(A)P(\text{not } A) = 1 - P(A)

The Handbook doesn't print this complement rule, but reach for it whenever the event you want covers most of the possibilities. A sum of at least 3 takes in 35 cells. Only the corner cell is left out, so the chance is 1−1/36≈0.9721 - 1/36 \approx 0.972.

Checkpoint

Two dice are rolled. What is the probability, as a decimal, that their sum is less than 11?

"Or" adds, minus the overlap

A foundry inspects 100 castings. Forty are cracked, 20 are out of tolerance, and 15 are both. How many have a defect? Not 60. The 15 with both problems got counted twice, so take them out once: 40+20−15=4540 + 20 - 15 = 45.

100 castings Cracked: 40 Out of tolerance: 20 25 15 5 No defect: 55
100 castings sorted by two defects

Divide by 100, and you have Property 2 in the Handbook's Laws of Probability:

P(A+B)=P(A)+P(B)−P(A,B)P(A + B) = P(A) + P(B) - P(A, B)
Reading the Handbook's notation

The plus sign and comma aren't arithmetic. P(A+B)P(A + B) is the chance of A or B or both, and P(A,B)P(A, B) is the chance of both.

Events that can't happen together, like a sum of 2 and a sum of 7 on one roll, are mutually exclusive. With P(A,B)=0P(A, B) = 0, plain adding is correct.

"And" multiplies, if the events are independent

Two events are independent when one happening doesn't change the chance of the other. If pump A starts 90% of the time, and pump B starts 80% of the time no matter what A does, both start on 80% of that 90%, a fraction of a fraction: 0.72.

P(A,B)=P(A) P(B)(independent events only)P(A, B) = P(A)\,P(B) \qquad \text{(independent events only)}

That condition matters. The castings fail it: (0.40)(0.20)=0.08(0.40)(0.20) = 0.08, but the actual overlap is 0.15.

Now back to the eight tests. "At least one fails" can happen in a pile of ways. "None fails" happens one way: all eight pass. So take the complement:

P(at least one)=1−P(none)P(\text{at least one}) = 1 - P(\text{none})

Each test passes with probability 0.9, so P(none)=0.98=0.430P(\text{none}) = 0.9^8 = 0.430, and at least one fails with probability 0.570. Adding failed because failures overlap: a run with three failures got counted three times.

Worked Example Two independent sensors
1 of 3

Two independent sensors watch for seepage at a dam. One detects it with probability 0.95, the other with probability 0.70. Find the probability that both detect it, and that at least one does.

"Given" shrinks the world

A factory's two machines made these parts last month:

Last month's 1,000 parts
MachineDefectiveGoodTotal
Machine 112588600
Machine 220380400
Total329681,000

Suppose you know a part came from Machine 2. Everything outside that row is gone, and your world has shrunk to 400 parts, 20 of them defective. That's a conditional probability, with a bar that reads "given":

P(defective∣Machine 2)=20400=0.05P(\text{defective} \mid \text{Machine 2}) = \frac{20}{400} = 0.05

Flip the question. You know a part is defective, so now your world is the Defective column:

P(Machine 2∣defective)=2032=0.625P(\text{Machine 2} \mid \text{defective}) = \frac{20}{32} = 0.625

It's the same 20 on top, over two different worlds. P(A∣B)P(A \mid B) and P(B∣A)P(B \mid A) are different questions, so always ask which world the "given" leaves you in.

The Handbook's Property 3, the Law of Compound or Joint Probability, says this in symbols:

P(A,B)=P(A) P(B∣A)=P(B) P(A∣B)P(A, B) = P(A)\,P(B \mid A) = P(B)\,P(A \mid B)

The chance of both is the chance of A, times the chance of B in A's world. If A doesn't change B's chance, P(B∣A)P(B \mid A) is just P(B)P(B), and you're back to the multiplication rule, which the Handbook doesn't print on its own.

Bayes' theorem is that table in disguise

The exam often hides the table behind percentages. Machine 1 makes 60% of the parts, and 2% of its parts are defective. Machine 2 makes 40%, and 5% of its parts are defective. A part is defective. What's the chance it came from Machine 2? Follow each path a part can take, multiplying along the way as Property 3 says.

Path probability 0.60 0.40 0.02 0.98 0.05 0.95 Part Machine 1 Machine 2 Defective Good Defective Good 0.012 0.588 0.020 0.380
Every path a part can take through the factory

"Given defective" keeps only the two paths that end in Defective. Machine 2 holds 0.020 of their combined 0.032, so the chance is 0.625, the same as the table. That's all of the Handbook's Bayes' Theorem:

P(Bj∣A)=P(Bj) P(A∣Bj)∑i=1nP(A∣Bi) P(Bi)P(B_j \mid A) = \frac{P(B_j)\,P(A \mid B_j)}{\sum_{i=1}^{n} P(A \mid B_i)\,P(B_i)}

The BBs are the machines, which must cover every possibility with no overlap, and AA is "defective." The top is the one path you're asking about, and the bottom is every path that reaches AA.

Checkpoint
1 of 2
Results for 400 concrete cylinders
PlantFailedPassedTotal
Plant A10240250
Plant B15135150
Total25375400

A cylinder chosen at random has failed. What is the probability that it came from Plant B?

Mutually exclusive isn't independent

These two terms sound like cousins but are closer to opposites, and the Handbook defines neither. Mutually exclusive events can't happen together. Independent events tell you nothing about each other. But if A and B are mutually exclusive and A happens, you know for certain that B didn't, and that's a lot to learn.

P(A) = 0.3 and P(B) = 0.4 in both squares Mutually exclusive A B P(A, B) = 0 Independent A B A and B P(A, B) = 0.12
Two pairs of events, with area standing for probability
On the right, B fills 40% of the square and also 40% of A's strip.

To tell them apart, check P(A,B)P(A, B): 0 means mutually exclusive, and P(A) P(B)P(A)\,P(B) means independent. When the exam asks which probability statements are valid, three more checks help: every probability lies from 0 to 1, P(A,B)P(A, B) can't exceed either event, and P(A+B)P(A + B) can't fall short of either.

Checkpoint
1 of 2

For two events, P(A)=0.5P(A) = 0.5, P(B)=0.4P(B) = 0.4, and P(A,B)=0.2P(A, B) = 0.2. Which describes them?

A.
Mutually exclusive, but not independent
B.
Both mutually exclusive and independent
C.
Independent, but not mutually exclusive
D.
Neither mutually exclusive nor independent

Exactly x out of n: the binomial

Now repeat one yes-or-no trial nn times, independently, with the same probability each time. The count of how many times it happens follows the binomial distribution.

Four tests each fail with probability 0.1. What's the chance that exactly two fail? Fail, fail, pass, pass has probability (0.1)(0.1)(0.9)(0.9)=0.0081(0.1)(0.1)(0.9)(0.9) = 0.0081. But that's only one ordering.

Test 1 Test 2 Test 3 Test 4 Probability F F P P 0.1 × 0.1 × 0.9 × 0.9 F P F P 0.1 × 0.9 × 0.1 × 0.9 F P P F 0.1 × 0.9 × 0.9 × 0.1 P F F P 0.9 × 0.1 × 0.1 × 0.9 P F P F 0.9 × 0.1 × 0.9 × 0.1 P P F F 0.9 × 0.9 × 0.1 × 0.1 F = fails (0.1) P = passes (0.9)
All six ways two of four tests can fail

Every row holds the same factors, shuffled, so every row has the same probability. The rows are mutually exclusive, so they add: 6×0.0081=0.04866 \times 0.0081 = 0.0486.

That's the whole binomial: the number of orderings, times the probability of one. The Handbook's Binomial Distribution section writes it with q=1−pq = 1 - p:

Pn(x)=C(n,x) pxqn−x=n!x! (n−x)! pxqn−xP_n(x) = C(n, x)\,p^x q^{n-x} = \frac{n!}{x!\,(n-x)!}\,p^x q^{n-x}

C(n,x)C(n, x), the number of combinations, counts the ways to choose which xx of the nn trials are the ones being counted, and your calculator's combinations function computes it. One catch: the Handbook calls pp the probability of "success," but success is only a label for whatever xx counts. When xx counts failed tests, pp is the chance of failing.

Worked Example Two failures in eight tests
1 of 3

Each of eight independent tests passes with probability 0.9. Find the probability that exactly two fail.

At most and at least

A range of counts is a sum of exact counts. Here's every possibility for the eight tests:

0 0.1 0.2 0.3 0.4 0.5 0 1 2 3 4 5 6 7 8 0.430 0.383 0.149 0.033 0.005 At most 1: At least 2: 0.813 0.187 Number of failures, x P8(x)
The number of failures in eight tests, split between 1 and 2

A quantity set by chance, like this count, is a random variable, written XX, and xx is one value it can take. The bars add to 1, and the dashed line puts each one on exactly one side, so:

P(X≥k)=1−P(X≤k−1)P(X \ge k) = 1 - P(X \le k - 1)

That k−1k - 1 is where people slip: "at least 2" leaves out 0 and 1, not 2. "More than 2" means at least 3, and "at least one" is k=1k = 1, one minus none again.

The Handbook's Cumulative Binomial Probabilities table adds the bars for you, listing P(X≤x)P(X \le x). Here's a piece of it.

Part of the cumulative binomial table, P(X ≤ x)
xn = 8, p = 0.10n = 10, p = 0.10n = 10, p = 0.20
00.4300.3490.107
10.8130.7360.376
20.9620.9300.678
30.9950.9870.879

Adapted from the FE Reference Handbook, Cumulative Binomial Probabilities

For eight tests, P(X≥2)=1−0.813=0.187P(X \ge 2) = 1 - 0.813 = 0.187. The real table runs nn and xx down its rows and pp across its top, and prints 0.813 as 8.13E-01. It stops at n=10n = 10, apart from 15 and 20, and has no column for 0.05. Off the table, use the formula.

Checkpoint
1 of 3

Each of 12 independent welds fails with probability 0.05. What is the probability that at least one fails?

The mean and spread of a binomial count

Run 100 tests that each fail 10% of the time, and you'd expect about 10 failures. That's the mean, npnp. The Handbook's Binomial Distribution section gives the variance, and its Probability and Density Functions: Means and Variances table lists both:

μ=npσ2=npq=np(1−p)\mu = np \qquad \sigma^2 = npq = np(1 - p)

If pp is 0 or 1, nothing is uncertain, and the variance comes out 0. The standard deviation σ\sigma is the square root of the variance. For those 100 tests, μ=10\mu = 10, σ2=9\sigma^2 = 9, and σ=3\sigma = 3.

Checkpoint

Each of 100 independent pile tests passes with probability 0.8. For the number of failed tests, what are the mean, variance, and standard deviation?

A.
Mean 20, variance 4, standard deviation 2
B.
Mean 80, variance 16, standard deviation 4
C.
Mean 20, variance 20, standard deviation 4.47
D.
Mean 20, variance 16, standard deviation 4

Expected value is a weighted average

A crew's equipment breaks down some number of times each day:

Breakdowns per day
x0123
P(X = x)0.500.300.150.05

Picture 100 days: about 50 with none, 30 with one, 15 with two, and 5 with three. That's 75 breakdowns, or 0.75 a day. No day ever has 0.75 breakdowns, so "expected" is a misleading word. It means the long-run average.

Dividing by 100 turns the day counts back into probabilities, so it's each value times its probability, added up. The Handbook's Expected Values section writes it with f(xk)=P(X=xk)f(x_k) = P(X = x_k):

μ=E[X]=∑k=1nxk f(xk)\mu = E[X] = \sum_{k=1}^{n} x_k\, f(x_k)

Stand each probability on a plank as a weight at its xx, and the mean is where the plank balances.

0.50 0 0.30 1 0.15 2 0.05 3 μ = 0.75 x (breakdowns per day)
Breakdown probabilities balanced on a plank

Variance is also a weighted average, of the squared distance from the mean:

σ2=V[X]=∑k=1n(xk−μ)2f(xk)\sigma^2 = V[X] = \sum_{k=1}^{n} (x_k - \mu)^2 f(x_k)

When XX can take any value in a range, a density f(x)f(x) replaces the table, and probability is area under it, the idea behind The Normal Distribution. A thin slice f(x) dxf(x)\,dx plays the part of a probability, so the sum becomes the Handbook's integral:

μ=E[X]=∫−∞∞x f(x) dx\mu = E[X] = \int_{-\infty}^{\infty} x\, f(x)\,dx

Integrate only where the density isn't zero. For f(x)=x/2f(x) = x/2 from 0 to 2:

μ=∫02x⋅x2 dx=[x36]02=43\mu = \int_0^2 x \cdot \frac{x}{2}\,dx = \left[\frac{x^3}{6}\right]_0^2 = \frac{4}{3}
f(x) 0.5 1 0 1 2 f(x) = x/2 Area = 1 μ = 4/3 x
The density f(x) = x/2 balanced at its centroid
Worked Example Trucks at a gate
1 of 3
Trucks arriving at a gate in five minutes
x012
P(X = x)0.20.50.3

Find the mean, variance, and standard deviation of the count.

Recap

  • For equally likely outcomes, probability is favorable over total, and the chance of at least one is one minus the chance of none.
  • “Or” adds, minus any overlap. “And” multiplies only for independent events, and mutually exclusive events aren't independent.
  • P(A∣B)P(A \mid B) shrinks the world to B. Bayes' theorem is one path over all the paths that reach A.
  • The binomial is the number of orderings times the probability of one, with pp belonging to whatever xx counts. Its mean is npnp, its variance is npqnpq, and at least kk is 1−P(X≤k−1)1 - P(X \le k - 1).
  • An expected value is a probability-weighted average, the point where the probabilities balance.