New Web-Based Opinion Polls Fuel Validity Debate

A trio of new Web-based user polls have again highlighted the validity of using online phenomenon polls as a tool for market research. The three surveys, all of which involved e-commerce, were conducted by Opinion Research Corporation International and sponsored by priceline.com.

Questions asked by critics about such Net-based surveys often center on the apparent inability of researchers to obtain a balanced and representative sample. Other problems, for example, involve the validity of drawing industry-wide conclusions involving the 60 million plus Internet users from samples often numbering less than 1,000.

Janet Westergaard, president of Esearch.com, an Internet data collection company, said, "Every method of polling introduces biases, and researchers now accept the fact that no form of data collection provides a truly balanced sample."

Despite that, Westergaard sees respect for the Web as a vehicle for such research as having changed in the past year. "We've been doing Web polling for nearly four years," she said, "and the attitudes towards the accuracy and relevance of this method have taken a 180-degree turn for the better in the last 12 months."

The three new surveys apparently highlight the growing use of e-commerce as a viable form of shopping, and the way "word-of-mouth" can contribute to its popularity.

The three surveys contained sample sizes numbering from just over 400 to 1,000, and claim to show that an average US adult tells 8.6 other people about their favorite movie and 6.1 other people about their favorite restaurant. The average Internet shopper, however, tells 12 others about his or her Internet shopping experiences, the study said.

Of the results, James Ware, vice president of research for The Concours Group, said: "Normally a handful of referrals, like those with movies and restaurants, would be considered exceptional. These statistics demonstrate just how big a force e-commerce is becoming."

While some analysts consider Internet surveys as only meaningful in certain circumstances - in those areas involving an entirely wired community, for example - others consider them the future of market research.

Even supporters of Web-based studies have reservations, however. Manfred Kuechler, a Hunter College sociology professor and an observer of European and US polling methodologies, said, "Given the ever-increasing problems with getting truly representative samples in conventional ways - face-to-face or telephone interviews - the future clearly belongs to Internet polling."

"A number of serious statistical problems still need to be resolved," Kuechler said. "Right now, most Internet polls are just garbage - though often quite entertaining."

But the same argument about sample size representing the entire Internet community can evidently be levied against applying standard small sample sizes to the general population as well.

Westergaard said that a sample size of 1,000 is a traditional sampling number. "We're following the traditional paradigm of what you would consider a representative sample generally," she said. "So researchers will take a telephone sample of 1,000 people and apply their findings to the rest of the population as well."

"There are standards in the research industry that say that if you're trying to apply a statistic to the general population, a sample size of 500 gives you a margin of error of, I think it is, 4.4 percent, while a sample size of 1,000 gives you a margin of error of 3.5 percent," Westergaard also said.

She concluded, "The research community is applying the same principles to the Internet. So it goes along exactly with traditional research....where they make generalizations about the general population, which is bigger then the Internet population, based on the small sample sizes."