The Pros and Cons of Online Surveys


More and more businesses are relying on online surveys to learn what their customers want, need, and value. Well-constructed online surveys can be effective market research tools, but they do have some disadvantages. Learning more about the pros and cons of online surveys can help you make better use of these powerful tools.

What’s So Great About Online Surveys?
Many things. For one, they can be very fast. A survey posted on a popular website can generate thousands of responses within a few hours. People who respond to email invitations to take a survey usually reply in a day or so. Contrast this with the painfully slow process of conducting a survey by snail mail, or the labor-intensive process of conducting a phone survey.

You can improve response time by keeping online surveys short, with “powerful” questions and smooth transitions. Powerful questions are ones that provide valuable knowledge. In constructing a survey, take a hard look at each question. Ask yourself if it will yield information that you will be able to use somehow. If it won’t, leave it out. In designing surveys, remember that shorter is usually better.

It’s also a good idea to try questions out on a small sample before using them with a broad population. That way you can see which questions provide useful information.

Another important advantage of online surveys is that they are relatively inexpensive. Once the survey has been constructed, it costs very little to administer it. You don’t need to pay for postage or telemarketing services, and you don’t need to hire someone to tabulate the results. Survey software makes collecting and analyzing responses easy, accurate, and inexpensive. Online surveys can be do-it-yourself projects. There is usually no need to hire expensive consultants to conduct them.

Online surveys give you greater flexibility than surveys conducted on paper or over the phone. You can use photos, video, sound, and just about any interactive medium you can think of. Sophisticated survey programs let respondents easily skip questions that don’t apply to them. For example, if a respondent checks “male,” he will automatically skip questions about pantyhose.

Online surveys are also flexible in the sense that people can complete them on their own schedule. In that way, they are less disruptive than phone surveys or in-person surveys in a mall.

Survey experts believe that people are more willing to answer questions about sensitive topics, such as sex or drug use, when replying to a computer rather than a person. Answers to such questions are also more likely to be honest.

Last, but certainly not least, responses to an online survey can be easily tabulated, sorted, analyzed, and converted to graphic representations.

The Down Side
Most of the disadvantages of online surveys are really no different from the disadvantages of surveys in general. For one thing, there is always a question about the sample. Is it large enough and does it accurately reflect the population you are interested in? Screening questions at the beginning of the survey can help ensure that you are asking the right people.

Then there is the question of honesty. While many experts believe that people are more honest when responding to a computer than to a person, there will always be exceptions. For example, landlords might under-report the number of tenants in an apartment to skirt residency laws. If people are being paid to complete a survey, they might just fill in answers randomly to finish as quickly as possible. (A carefully constructed survey can check for such “non-reliable respondents.”)

There is also sometimes a difference between what people say and what they do. For example, people might tell you that they floss their teeth religiously once a day because they know that’s what they’re supposed to do, even if they don’t.

Finally, you might find it easier to compose a survey using a word processing program rather than survey software. That is why it is important to choose user-friendly software with people behind it who are willing to help you construct truly efficient surveys.

On balance, the advantages of online surveys far outweigh the disadvantages. They are an efficient, economical way to get to know your customers.



9 Tips to Writing Good Survey Questions

Today, surveys still carry much importance in gathering metrics and understanding people’s opinions. In business, customers provide valuable feedback on products and services. In education, children’s measures of intelligence are gathered through “survey tests”. In Economics, unemployment rates are calculated by the Bureau of Labor Statistics via phone, mail, and online surveys. The point is, asking questions is important!

It is essential to any market researcher to write questions that are understandable and can provide accurate and consistent results. What is a good question? A good survey question produces answers that are reliable and valid measures of something we want to describe. Just the style of writing a question can alter its meaning to the participant. Here are a few ways to write questions that can make a survey more effective.

  1. Keep the language very simple. A survey question is not a test of your vocabulary skills. The use of language that is incomprehensible to the participant will only distort the findings. So avoid framing a question like “What is the frequency of your culinary visits to the nearest non gourmet eatery in a week”. Keep it simple with “How many times per week do you go to the nearest fast food joint”.
  2. Ensure Answers are Consistent. Answers should be structured so that there is only one way to answer. For example, the open-ended question “What type of shampoo do you use?” can be answered with a brand, a shampoo type, or none-I don’t use shampoo. Also ensure that all possible choices are included, if you suspect there could be other choices, include an “Other” category. Mineful’s online survey software allows you to include the “Other” category and adds a space for the respondent to elaborate.
  3. Avoid using negatively worded questions. Double negatives are a no-no. Instead of asking “Should the government not provide free medical support to pregnant women?” rephrase the questions as “Should the government provide free medical support to pregnant women?”.
  4. Use close ended questions as much as possible. Any marketing analyst knows that analyzing close ended questions provides more opportunites for advanced analysis. Yes, open-ended questions allow for text analysis, but survey reseach methodology goes far and beyond when analyzing close ended question. As an example, avoid using a question like “What kind of food do you usually order in a restaurant?”. Instead change it to “Which of the following do you usually order when you visit a restaurant?”.
  5. Avoid leading questions. Adding a personal opinion to a question when it is not necessary can bias the answers of the participants. For example, the question “Has the new, modern and state of the art audio system installed in the auditorium improved the voice quality” is sure to lead the participant to answer “yes”. This can be avoided by just asking “Has the voice quality in the auditorium improved”.
  6. Do not assume participants know. There are some terms that not everyone would be familiar with, or have sufficient knowledge about. For example while asking” Are you in favor of Proposition 13” it has been assumed that everyone knows what it means. Who is your sampling frame (the total population you are interested in describing)? Do they understand what these technical terms mean? Is the language adequate for them?
  7. Minimize choices. Survey questions like ranking items by importance should not have more than 5-6 options. As the number of items increases the reliability of the answer falls. If a choice is likely to obtain less than 5 responses, it is recommended to merge it with another category or inside an “Other” option.
  8. Pre-test your questionnaire. Once the survey is complete, before sending it to the target audience, send it as a test to a small group of people you know to check if the responses obtained are consistent. They will also be able to tell you if any question was confusing or should be altered to make it easier to answer.
  9. Ensure responses are valid. There are several ways to determine if the responses you obtained are reliable and trustworthy.
    • Evaluate the strength of anticipated relationships - if you expected 50% and got 80% that should ring a big alarm.
    • Compare to similar questions - Surveys sometimes include similar questions at the begining and at the end to ensure that the respondent is consistent with their answers. Similarly, one can compare responses of the same person at different points in time.
    • Construct test questions that lead respondents to only one answer - For example, “This question is very long…Answer the third choice for validation purposes”. This helps validate that the respondent is not speed responding the survey in order to obtain their incentive.
    • Reinterview in person or by phone – Only for highly speculative responses that carry significant weight. For example, at BLS, they would reinterview companies like Boeing and GE before releasing the unemployment rates since their employee numbers carry significant weight for the national unemployment rate.


Online surveys have transformed the field of market research. Retailers in particular are discovering how the latest internet survey software can provide valuable insights into how customers make decisions.

Going Beyond Sales Data
Traditionally retailers have relied mainly on sales data to determine their customers’ preferences. While this approach certainly has merit, it leaves many questions unanswered. Would customers prefer different brands if you offered them? Would they visit your stores more often if their shopping experience was more pleasant? Would they come more often if stores were open longer hours? These are the kinds of questions that online surveys can address.

A simple “Where would you go to buy…?” survey can give you a good idea of how customers think of your stores. In this type of survey you give customers four or five options — your store and three or four of your competitors. Then you ask a series of questions. If you operated a department store, you might ask:

  • Where would you go to buy underwear?
  • Where would you go to buy makeup?
  • Where would you go to buy jewelry?

The answers will give you a good idea of how you match up with your competitors.

An online survey can also tell you what products or brands your customers would like to find in your stores. For example, if you operated a clothing store for fashion-conscious women, your survey might show customers a list of designers and ask them to choose the ones they like the most.

Surveys sometimes ask about customer service, but unfortunately the questions they ask are often too general. For example, “How would you rate our customer service?” The answers to questions like that are practically useless. Instead you might ask customers to rate your sales staff in terms of:

  • Knowledge of products
  • Courtesy
  • Helpfulness
  • Appearance

If you operate a bricks and mortar store, you can ask customers to rate your establishment in terms of cleanliness and ease of parking. If you’re an online merchant, you can ask which features of your website your customers would like to see changed.

Online surveys can also help you fine tune your advertising to address specific groups of customers. For example, you might ask what newspapers customers subscribe to or what kinds of TV programs they like to watch.

Finding people to take part in surveys is usually not difficult. One of the most effective methods is to create a customer loyalty program of some sort. For example, you might tell customers that you will send them early notices about special sales if they will give you their email address. This approach allows you to do some marketing and market research at the same time.

Surveys Deliver for Safeway.com

Safeway.com is an online store that allows customers to do their grocery shopping without leaving their homes. Groceries are delivered right to their door. The only direct contact the store has with its customers is through the delivery person, who may or may not be a good source of information about customers.

Safeway felt that it didn’t understand its online customers as well as it should, so it hired a consultant to develop an online survey. The survey focused mainly on customer service, asking customers to rate seven key “touch points.”

The survey also helped Safeway refine its website to make it easier for customers to navigate. And it asked some open-ended questions to give customers a chance to say exactly what was on their minds.

In a business like Safeway.com, which has email addresses for all of its customers, creating a mailing list for a survey was easy. And survey software made it easy to analyze the results.

Like many retailers, Safeway.com found online surveys a convenient way to learn how it could do a better job of meeting its customers’ needs.



Characteristics of Internet Users

Anyone who wants to do online surveys or market research online should be curious about who actually uses the Internet. Although usage patterns are constantly changing, it is still possible to create a fairly clear profile of Internet users.

Who Uses the Internet?
In 2008, Asia accounted for the largest number of Internet users, followed by Europe and then North America. (http://www.internetworldstats.com/). As a percentage of the population, North America leads the world, with almost 75% of the population having access to the Internet and the basic skills needed to use it.

In the U.S., women account for slightly more than half of Internet users (51%). In terms of ethnic background, the numbers are generally in line with the population as a whole. Approximately 74% of users are white, 11% are Hispanic, 9% are African-American, and the remaining 6% belong to other racial or ethnic groups.

The percentage of the population that uses the Internet on a regular basis stays fairly constant until about age 55. Then it begins to decline steadily, from about 67% for the 50-54 age group to about 16% for people over 75. (www.clickz.com/3446641)

More than half of Internet users (54%) live in suburbs, 30% live in cities, and 16% live in rural areas.

What People Are Doing Online?
So what are all these people doing on the Internet? The most frequent activity is still email, although it is being challenged for the top spot by online searches. (www.pewinternet.org/Data-Tools). Other frequent uses include searching for a map or driving directions, looking for information on a hobby or interest, checking the weather, and getting news. Online commerce (shopping, banking, paying bills) continues to grow in popularity, but it is not one of the most frequent activities.

How people use the Internet depends to some extent on who they are. Men are more likely than women to get news, buy travel services, check sports scores, and participate in online auctions. Women are more likely than men to get health information, use support-group Web sites, and get information about spiritual and religious topics. Young Internet users (ages 18-29) are more likely than others to do research for school, use instant messaging, listen to music, use dating sites, and share files. (www.infoplease.com/ipa/A0931238.html)

New Trends in Access and Content
One of the most significant trends in Internet use involves how people are accessing the Internet. In December 2007, about six months after release of Apple’s iPhone, almost 40% of Internet users said that they were accessing the Internet with mobile devices at least some of the time. (www.clickz.com/3633197) This trend is likely to continue as mobile devices become more affordable.

Online video is also having a powerful impact on how people (especially younger people) use the Internet. The explosive growth of YouTube is only part of the picture. TV networks and news organizations are turning increasingly to online video as a new source of income. In 2007, people in the 18-24 age group spent nearly as much time watching online video as they did watching programs on their DVRs. (http://online.wsj.com/article/SB123535779391045623.html) From an advertiser’s standpoint, online video may be a better investment because users can’t fast-forward through ads.

Social networking sites such as Facebook are also growing exponentially. It seems safe to predict that the popularity of such sites will continue, but any predictions about the Internet are risky. By the time you read this, Internet users might have moved on to something no one could have expected.



| Back to top