Describing Data
Two numbers summarize most data sets: where the values cluster and how far they spread.
You specified 4,000 psi concrete, and two suppliers each send you five cylinder breaks. Both sets average exactly 4,000 psi, so by the average it's a tie. But Supplier A's breaks all land within 150 psi of that number, and Supplier B's miss it by as much as 700. You know which one you'd rather pour a column with. This Lesson gives you the numbers that tell them apart.
The mean is a balance point
The mean is the average you already know: add the values and divide by how many there are. The Handbook's "Dispersion, Mean, Median, and Mode Values" section gives it as
Here ("x-bar") is the mean of a sample of values, and means "add them all up." The Handbook capitalizes the symbols, as . A whole population of size has its mean written .
Here's a better picture than "add and divide." Put a one-pound weight at each value along a stiff, weightless ruler, and the mean is where the ruler balances. To see why, find each value's deviation, its signed distance from the mean. Supplier B's breaks of 3,400, 3,600, 4,000, 4,300, and 4,700 psi have deviations of −600, −400, 0, +300, and +700 psi. Those below the mean total −1,000, and those above total +1,000. Deviations from the mean cancel exactly, for any data set. Each one is that weight's turning effect about the balance point, so the ruler sits level.
Spread: the typical distance from the mean
You want the typical size of a deviation, and averaging them gives zero every time. So get rid of the signs by squaring:
Divide that total by the count, more or less, and you have the variance. Its units are square psi, which mean nothing to anyone, so take the square root to get back to psi. That's the standard deviation: square, average, root.
The "more or less" is the catch. The Handbook prints one version for a whole population and one for a sample, and and are their square roots:
A whole population gives you the true mean to measure from. A sample usually stands in for something bigger: those five cylinders stand in for all the concrete in the pour, and you don't know its true mean, so you measure from instead. But is a flattering choice, because it's the one point that makes these five squared deviations as small as possible. Measure from 4,100 psi instead of 4,000, and the total grows to 1,150,000 psi². So deviations from a sample's own mean run slightly small, and dividing by instead of corrects for it.
Supplier A works out to psi. There's what the average couldn't show you: the same mean, with over five times the scatter at Supplier B. Any data set deserves both questions: where do the values center, and how far do they spread?
Whole population or sample?
So before you reach for a formula, ask one question: is this data set everything you care about, or a sample of something bigger? "The entire population," "all," or "every" point to . "A random sample" or "test results" point to , and so does silence, since measurements are nearly always a sample of something bigger.
Treated as an entire population, Supplier B's breaks give psi instead of 524. The form always comes out smaller, and the gap fades as the data set grows.
Your calculator won't choose for you. One-variable statistics reports both standard deviations, usually labeled for the form and for the form. Reading the right line is the whole job, and for a variance, square that line.
When you only get the totals
Sometimes a lab report gives you only , , and . The mean is immediate: . For the spread, the sum of squared deviations can be rewritten using only those totals:
Read the numerator as a correction. measures every value from zero, so most of it just reflects the data sitting far from zero. The term is exactly that part. Subtract it, and the spread around is what's left. Supplier B has and , so
That's the same total as before. Those two big numbers nearly cancel, so keep every digit until after you subtract. The Handbook doesn't print this as a variance, so a question that needs it should state it, as the next one does.
The middle value, the commonest value, and the width
The median is the middle value once the data are sorted. The Handbook's rule: with odd, it's item , and with even, it's the average of items and . Sort first, because the middle of an unsorted list is just whatever got written there. The mode is the value that occurs most often, and the range is the largest value minus the smallest.
Data often arrive rolled up in a frequency table, and that's where people stumble. Here are twenty inspected precast panels:
| Defects per panel | Panels | Cumulative count |
|---|---|---|
| 0 | 7 | 7 |
| 1 | 8 | 15 |
| 2 | 3 | 18 |
| 3 | 1 | 19 |
| 4 | 1 | 20 |
The listed values run from 0 to 4, and the middle of that list is 2. That's the trap, because the median is the middle observation, not the middle row. A frequency table is a sorted list written compactly, so unroll it.
With , the middle observations are the 10th and 11th. The cumulative counts, a running total of the Panels column, say the first 7 are 0s and 8 through 15 are 1s, so both are 1, and the median is 1. The Handbook says nothing about frequency tables, so the cumulative count is yours to bring.
Spread relative to size
A standard deviation of 300 psi is ordinary on 4,000 psi concrete and alarming on a flexural test that averages 600 psi. The Handbook's sample coefficient of variation compares spread with size:
The units cancel, so you can compare entirely different tests. The Handbook's statistics chapter prints as a plain ratio, while its environmental section multiplies by 100 to make a percent, so give the form the question asks for. Our two suppliers come in at 2.3% and 13%.
When the plain mean is the wrong average
A plain mean gives every value one equal vote. The Handbook prints three alternatives for when that's wrong.
Weighted mean
Say 4 ft of soil at 110 lb/ft³ lies over 6 ft at 120 lb/ft³. The average isn't 115, because there's more of the heavier soil. So each value gets votes according to how much of it there is:
A frequency table is the same idea with the counts as the weights, though the Handbook leaves you to notice that.
Geometric mean
Some quantities multiply instead of add. If traffic grows by a factor of 1.10 one year and 1.50 the next, the total factor is 1.65, and the single factor that gives the same total when applied twice is , not the plain average of 1.30. The Handbook's sample geometric mean is the th root of the product:
Root-mean-square
Read the name backward for the recipe: square each value, take the mean, then take the root. That's the Handbook's sample root-mean-square value:
Squaring gives big values extra say, which suits anything whose effect grows with the square of its size, like the energy in a water wave. In fact, is the RMS of the deviations.
As a check, for positive values that aren't all equal, the geometric mean falls below the plain mean, and the RMS falls above it.
What moves when the data change
An outlier
Take 2, 3, 4, 6, and 10, with a mean of 5 and a median of 4, and change the 10 to 30. The mean jumps to 9. The median stays at 4, because it depends only on order: the top value is still on top, however far up it went. A mode, if the data have one, doesn't move either, unless the outlier lands on a repeated value. Spread reacts hardest of all, with the range going from 8 to 28 and from 3.16 to 11.8.
A shift or a scale
Add a constant to every value, such as a new datum, and the data set slides along the line without changing shape. Multiply every value by a constant, such as a unit conversion, and it stretches. Anything that says where the data are follows both. Anything that measures width ignores the slide and stretches with the stretch, and variance, built from squared distances, stretches by the square of the factor.
| Measure | Add to all | Multiply all by |
|---|---|---|
| Mean, median, mode | Add | Multiply by |
| Range, standard deviation | No change | Multiply by |
| Variance | No change | Multiply by |
Skew
Weekly rainfall has lots of ordinary weeks and the occasional huge storm, so its distribution has a long tail on the high side. That's right-skewed. The mode sits at the peak. The median splits the area in half, and the tail puts extra area on the right, so the median lands right of the peak. The mean is the balance point, and a long tail has a lot of leverage, so it gets pulled farther still.
A left-skewed distribution is the mirror image. Either way, reading from the tail inward gives mean, median, mode, which is conveniently alphabetical. The Handbook prints no rules for skew, shifts, or scaling, so reason these out from the pictures.