Tag Archives: Survey Research

A LITTLE LESSON ABOUT EXTRAPOLATING

The extrapolation in the accompanying cartoon (from Randall Monroe’s website, xkcd.com) is rediculous. No one would ever do that kind of silly data-based projection into the future. In many areas of our daily lives, however, people make unjustified predictions based on existing, accurate data. Consider these 2 examples, one dealing with the stock market and the other concerning survey research.

How do people tend to invest their money in the stock market? From controlled experiments as well as from observational studies, the findings are the same. When the stock market has been doing well, most people are “bullish” and want to invest more. In contrast, when there’s been a recent drop in stock values, the typical investor gets “bearish” and wants to sell. The term extrapolation bias has been coined to describe cases like these wherein people think that the future will be a continuation of the past. You, too, possess this bias if you make short-term predictions that fail to consider (1) the variability of data points used to form a “trend line” and (2) the possibility that a trend line can change its direction and, for example, begin to angle down even though it has been angling up.

If you receive a mailed or online survey, do you fill it out and send it in? If you do, you help to increase the survey’s response rate: the percentage of contacted people who complete and return the survey. In a recent research study (http://gradworks.umi.com/33/91/3391754.html), the response rate was only 8.41%. Despite receiving completed surveys from just 241 of the 2,865 people initially contacted, the researcher extrapolated the study’s findings to all of the individuals to whom the survey was sent. This is an unjustified thing to do because “nonrespondents” may well be different from those from whom data are collected. Most likely, we all are guilty of this kind of extrapolation-beyond-the-data. We hear opinions expressed by trusted friends, relatives, co-workers, neighbors, bloggers, or TV analysts, and then we presume that others have the same thoughts. ’Tis a risky thing to do!

Leave a comment

Filed under Jokes & Humor, Mini-Lessons

WHAT SEEMS TO BE TRUE OFTEN ISN’T

Many People Get Fooled

The Motley Fool provides advice on money management and investing. However, its recommendations can and should be used by people in other fields. For example, the following 20-word tip, from the “Fool’s School,” should be memorized by everyone who encounters statistically-based claims or findings in politics, medicine, psychology, education, and all other arenas of our lives:

“Never blindly accept what you read. Think critically about not just words, but numbers. They’re not always what they seem.”

Here are 5 examples illustrating how numbers in statistics often do NOT mean what they seem to indicate:

Example A

If the 14 players on a basketball team have a median height of 6 feet 6 inches, it might seem that 7 of those athletes must be shorter than 6’6” whereas 7 must be taller than that. Wrong!

Example B

If the data on 2 variables produce a correlation of +.50, it might seem that the strength of the measured relationship is exactly midway between being ultra weak and ultra strong. Not so!

Example C

If a carefully conducted scientific survey indicates that Candidate X currently has the support of 57% of likely voters with a margin of error of plus or minus 3 percentage points, it might seem that a duplicate survey conducted on the same day in the same way would show Candidate X’s support to be somewhere between 54% and 60%. Bad thought!

Example D

If a null hypothesis is tested and the data analysis indicates that p = .02, it might seem that there’s only a 2% chance that the null hypothesis is true. Nope!

Example E

If, in a multiple regression study, the correlation between a particular independent variable and the dependent variable is r = 0.00, it might seem that this independent variable is totally useless as a predictor. Not necessarily!

The Motley Fool’s admonition, shown above in italics, contains 20 words. If you can’t commit to memory the entirety of this important warning, here’s a condensed version of it:

“Numbers. They’re not always what they seem.”

Leave a comment

Filed under Mini-Lessons, Misconceptions