How many survey subjects is enough?
How many people is it necessary to reach when trying to determine what a county of roughly 21,000 people is thinking?
Turns out it is actually a small percentage, although the accuracy of the outcome can vary due to a variety of different factors.
This question arose as a citizen-led pool advisory committee recently gathered the necessary funding to conduct a scientifically valid survey regarding the replacement of the Prineville swimming pool. The survey, administered by Fairbank, Maslin, Metz & Associates (FM3), a California-based firm, is intended to not only reveal how many members of the community want to replace the pool, but how much more taxpayers are will to pay for it.
To fund the survey, the committee had to persuade the Crook County Parks and Recreation District, the Crook County Court and the Prineville City Council to contribute $5,000 apiece. As local leaders considered the expense, some questioned whether the suggested sample size of 300 people, about 1.4 percent of the population, was sufficient.
In response, other local officials confirmed that it indeed was enough people, due to the random nature in which the survey would be conducted as well as other factors. And those responses also appear to match other expert evaluations.
For example, according to data provided by SurveyMonkey, which facilitates many online surveys, a sample size of only 88 people is needed to reflect the views of 1,000 people, within a 10 percent margin of error. To achieve a 5 percent margin of error, that number climbs 278 people. However, little changes when sampling for 10,000 people. Only 96 are needed for a 10 percent margin of error and 370 for a 5 percent error margin. To determine the views of 100,000 people, again 96 is a sufficient sample size for a 10 percent margin of error and 383 people for a 5 percent margin.
While margin of error is an important consideration, it is not the only factor related to survey accuracy.
"A confidence level is the likelihood that the sample you picked mattered in the results you got," SurveyMonkey explains on its website. "The calculation is usually done in the following way: If you picked 30 more samples randomly from your population, how often would the results you got in your one sample significantly differ from those other 30 samples?"
A 95 percent confidence level would therefore mean that you would get the same results 95 percent of the time.
Like margin of error, the sample size for certain confidence levels doesn't drastically change as the population surveyed increases. According to SurveyMonkey, to achieve a 95 percent confidence level for a population of 1,000 people, a 278-person sample size is sufficient. For 10,000 people, it takes a sample size of 370 to achieve the same confidence level. That sample size only increases to 383 for a population of 100,000.
While margin of error and confidence level dictate sample size, the ability to accurately predict the margin of error and confidence level hinges on conducting a truly random survey.
According to Pew Research Center, which does polling nationwide for a variety of topics, there are two broad ways to draw a sample for a survey: probability sampling and non-probability sampling.
"Most samples used at Pew Research Center are probability (also called random) samples, so-called because nearly every person in the population of interest has a known, and non-zero chance of being selected for the sample," the organization explains on its website. "By contrast, non-probability samples, by definition, are drawn in such a way that it is impossible to assign a probability of selection to the members of the population.
Pew Research Center conducts its surveys using a variety of methods to contact people, but the most common options are by telephone and by email. The organization noted that the growing use of cellphone use instead of landlines has caused a shift in how survey data is collected.
"Currently, nearly half of Americans have only a cellphone. Because many people can no longer be reached by landline telephone, the representativeness of telephone surveys based only on a random sample of households with landline telephone service has come under increased scrutiny," the organization states. "Many pollsters and survey methodologists, including those at Pew Research Center, are studying how cellphones impact telephone surveying."
Pool advisory committee chair Wayne Looney has stated that FM3 is planning to first email Crook County residents to randomly reach the 300-person sample size, before ultimately moving to phone calls.
Once the results of the upcoming survey are known, Looney said the committee will act according to what the data determines is the will of the Crook County community.