Acting on Analytics: The Official Mineful Blog

Why Surveys Fail

As I have mentioned before, the biggest frustration for survey consultants is organizations not using survey results. Although this is understandable under certain circumstances, most of the time it is a simple case of organizations who didn’t realize the seriousness of the problems that they were facing and just simply give up.

But what about those organizations that really try but simply don’t know how to proceed? Obviously, the survey consultant has let them down.

All too often, these organizations build action plans based on issues receiving low survey scores. Makes sense, right? Maybe not. After all, a low survey score, whether based on the response scale score itself or on a benchmark comparison, is a bad thing, right? It indicates a problem, right? Maybe not.

The point here is that all survey questions are not created equal. That is, all survey questions do not have the same impact on organizational performance. Now, I am not talking about a bad question, that is, one that should not have been asked in the first place. I am talking about a good question that provides valuable information but is not as strongly related to performance as are other questions. The information from these questions may help the organization understand the overall work environment better, but they should not set the prime focus.

If a low survey score does not necessarily translate into a problem of immediate concern how do we determine what is a problem of immediate concern? A statistical technique called Driver Analysis provides the solution. Driver Analysis will determine the one or more survey items that should be the immediate focus. Here’s what comes as a surprise. The survey item identified by this analysis may or may not be a low scoring item. That’s right, the organization’s primary focus may be on an area that they are already doing OK or even well on. Remember, we want to identify the item that provides the most leverage on organizational performance. Low scores are good to look at but they may not tell us what we really need to know.

About the author: Ray Seghers is an organizational improvement consultant specializing in employee engagement and customer loyalty surveys as well as employee commitment research.
Blog URL: http://rayseghers.wordpress.com
Business URL: http://www.segherssurveyconsulting.com



Marketing campaigns typically involve three stages. First an advertiser or ad agency hires a marketing research professional to collect data about the target market: demographics, purchasing habits, wants and needs. Then the professional consultant analyzes the data, attempting to create a customer profile that can form the basis for a marketing plan. Finally, the advertiser or ad agency creates and implements a marketing plan using the research it has commissioned.

At least that’s the way things used to work. Today, digital tools available from companies like Mineful enable advertisers and ad agency to do their own research, in effect cutting out the middle man between themselves and the marketplace. The potential for cost savings with this new approach is obvious. Data collection and analysis can be expensive, and if marketers can handle these two stages of the process themselves they can save a considerable amount of money. But there are also other advantages of taking control of your market research.

Research on Your Own Terms

Although it can be worthwhile sometimes to get an outsider’s perspective on your market research efforts, hiring a consultant to collect and analyze market data has some definite drawbacks. After all, no one knows your business better than you.

Conducting your own research gives you complete control of the process. You can decide which potential customers to survey, how and when to contact them, and exactly what to ask. If the results of a survey don’t provide the kind of information you need, you can easily change the survey parameters and try again. Keeping the cost of surveys down also allows you to reach more people or conduct surveys more frequently.

Taking charge of the analysis stage of the research process gives you more control over how you use the data you collect. For example, if you decide to change your store hours or product offerings, you can use the results of your last survey to see how customers might respond to these changes. Since you have complete control of your data, you don’t need to go back to consultants when you want run a new analysis.

Powerful, Easy-to-Use Tools
One of the main reasons why more and more companies are taking over their own market research is that the digital tools available for this purpose have become increasingly easy to use. In the past you would probably have to hire a statistician to help you analyze survey data. But now you can conduct your own sophisticated analysis using tools that require no specialized knowledge. For example, Mineful’s software lets you use a tool called conjoint analysis to see how responses to various questions are related. This sort of analysis can help you understand such things as how customers make trade-offs between price and desirable product features.

Survey software also makes it easy to display the results of your analysis using widely understood formats such as pie charts and bar charts. Dashboards, a relatively feature, summarize data in a format that can be understood with just a glance. Dashboards make it easy to create different types of displays for different levels or functions in your organization.

With tools like these, it’s no wonder why a growing number businesses are taking control of their own market research.



It has been a difficult decade for the hospitality business. The awful events of September 11, 2001, led to a sharp decline in leisure travel. Just when business was getting back to normal, the economy turned sour, forcing both business and leisure travelers to cut back on their plans.

In an effort to cut costs to cope with the decline in business, some hotels have reduced staff and eliminated some amenities for guests. But at what point do such cutbacks threaten guest satisfaction and lead to even further erosion in business? How can hotels boost customer loyalty by focusing on improving things that guests really care about? Those are the crucial questions that hotels must consider as they try to control costs while at the same time keeping their guests satisfied.

Knowing what to cut
In February 2009, research company TNS conducted a survey of 2,500 adults to determine how cuts in hotel services might affect their choice of hotels (http://www.quirks.com/articles/2009/20090505.aspx). Specifically, the survey asked about five types of reductions in services:

  • Reduced entertainment, such as free in-room access to premium movie channels
  • Reduced to-door services, such as in-room checkout and delivery of newspapers.
  • Reduced personal assistance, such as help with luggage.
  • Reduced free amenities, such as hand lotion or mouthwash.
  • Reduced hours for service for restaurants, hotel stores, or business centers.

The survey found that guests cared least about door-to-door services and personal assistance. In fact, most respondents said that cutting these items would have no effect at all on their choice of hotels. Since both of these services are staff-intensive, reductions in these areas could lead to significant savings without threatening hotel loyalty.

The survey also found that the biggest area of risk would be cutting free amenities. Over a third of respondents said that cutting amenities would affect their choice of hotels.

The bigger picture
This survey provides valuable information for hotels that are thinking of cutting back on services, but hotel satisfaction surveys can also serve a broader purpose. A well-designed survey can help a hotel boost guest satisfaction by identifying areas where services or amenities need to be improved. This link (http://hotel_satisfaction.nisurvey.com) will take you to a good hotel survey example.

This survey asks about a broad range of things, from ease of check-in to cleanliness of rooms. Many of these things can be improved without significant cost because they involve nothing more than a change in staff behavior. In fact, hotel market research has revealed that “employee performance” is the most important factor guests consider in differentiating one hotel from another (http://www.quirks.com/articles/2007/20071005.aspx). It also has a major impact on customer loyalty and repeat business.

Employee performance includes such things as:

  • Efficiency
  • Friendliness
  • Hospitality
  • Courtesy
  • Promptness
  • Responsiveness

The bad news is that recent hotel satisfaction surveys show a significant decline in these qualities. According to Hospitalitynet.org, “Guests did not feel as welcome, and staff friendliness scores were down. Guests also felt less pampered and less entertained during their stay.” The good news is that hotels can turn this situation around with a greater commitment to customer satisfaction and a relatively small investment in staff training.



Measuring Product Involvement

What is product involvement? Marketers have come up with a variety of definitions for this concept, but basically it comes down to this: product involvement is a measure of a customer’s interest in a particular type of product and commitment to a particular brand.

Let’s say you love motorcycles, especially Harleys. You wouldn’t even consider buying a different brand. Obviously you have a high product involvement. (Someone once said that Harley-Davidson is the only brand that inspires so much product involvement that millions of people are willing to have the company logo tattooed on their bodies.)

In general, the more expensive the product, the greater degree of product involvement. People will spend a lot of time researching different makes of cars. They may even consider the car they choose a part of their self-image. Compare this sort of involvement with the attitude most people have toward something less expensive, for example, paper towels. Most people don’t care very much about paper towels, and they have little brand loyalty. They might not even know the name of the brand they usually buy.

Still, there are some inexpensive products that inspire a high degree of product involvement — for example, shampoo. Some women are very concerned about the appearance of their hair. They may have a strong commitment to a certain brand of shampoo because they feel it makes their hair particularly soft and shiny. (Men, on the other hand, typically don’t have much product involvement with shampoo.)

Whether a consumer is buying shampoo or a motor cycle, there is a strong relationship between brand loyalty and how important the product is to the customer. The more a consumer cares about a particular type of product, the more loyal the consumer is likely to be to a particular brand.

Measuring Product Involvement
A simple product questionnaire can be an effective way of measuring product involvement. One question might ask how important a particular type of product is to the respondent on a scale of 1 to 10. A second question might ask how loyal the respondent feels to a particular brand. If most people feel that a product is very important but they feel little brand loyalty, this situation represents a real opportunity for a marketer. With the right message, a marketer can take advantage of this situation to build brand loyalty.

On the other hand, if a survey reveals that most people feel a particular product is not very important, it would probably be a waste of money to try build brand loyalty. Most people don’t feel strongly enough about the product to care which brand they buy.

It’s important to remember that product involvement is different from product evaluation. Respondents might rate a brand highly on an evaluation, but if the product is not important to them they will probably not have much brand loyalty.

Predicting Product Involvement with New Products
New product surveys can help marketers predict what sort of involvement a new product might create. The key here is to ask questions about similar products that a respondent might already be familiar with. For example, suppose you were going to introduce a new line of organic snack foods. Your survey might ask how important people consider snack foods and how important they consider organic foods in general. These questions will help you forecast the potential level of interest in your product, and its potential for building brand loyalty.



In a severe recession like the one we are now experiencing, consumers change their behavior. They look for cheaper alternatives to the products they usually buy. They weigh major purchases more carefully and spend more time comparison shopping. They may even decide that things they valued in the past, such as keeping up with the latest fashions, aren’t really that important any more.

These changes in consumer behavior make market research more necessary than ever. With customers cutting back on their spending and being more selective, companies can’t afford to make mistakes in their marketing efforts. Yet the recession has led most major marketers to cut their research budgets by 10 to 20 percent. To deal with these reductions, marketing managers need to do what consumers are doing: take a good hard look at how they are spending their money and focus on things that provide the greatest return on their investment.

Going Online
Many marketing consultants are suggesting a practical way to deal with tighter research budgets: shift more resources to online surveys. Telephone surveys and in-person surveys can be expensive and slow. For cost reasons, they are usually limited to fairly small samples. Each additional respondent adds to the cost of the survey.

On the other hand, online surveys can reach a large number of customers for a reasonable amount of money. Each additional respondent adds almost nothing to the cost of the survey. And online surveys are fast and efficient. Customer feedback can be analyzed easily to gain a better understanding of how the recession is affecting buying decisions. For example, conjoint analysis can reveal the tradeoff customers are making between price and desirable features. Online survey providers such as Mineful can help marketers make the most of this type of analysis.

Focusing on Core Customers
Tighter research budgets mean that marketers need to focus more on their core customers, the ones who are most loyal and most profitable. Maintaining strong relationships with these customers is the key to minimizing the pain of the economic slowdown.

Some companies may be tempted to take these customers for granted and focus their market research almost exclusively on attracting new business. But that would be a mistake. Even the most loyal customers are feeling the pinch of the recession, and marketers need to understand how to keep these people satisfied.

Looking for New Opportunities
The recession has presented serious challenges for most businesses, but it has also created some interesting opportunities. Marketers have always looked for ways to convince customers to rethink their purchasing habits and decisions, and the recession is forcing customers to do just that.

Customers seem to be more willing than ever to try new brands and new products. They are studying ads more closely and taking a harder look at what they are getting for their money. Companies that make an effort to understand what customers are looking for in this tough economy will do a better job of attracting new business. These new customer relationships will be even more valuable when the economy turns around.



Marketers would love to be able to read the minds of consumers. They would love to know what precise combination of price, features, benefits, wants, and needs triggers the decision to buy a particular product. There are many ways to approach this mysterious subject. One that lends itself nicely to online surveys is what is often called a “purchase intent survey.” Here’s how it works.

A survey might ask a question using a five point scale such as:
Which is most true about this product?

__ I definitely will buy this product
__ I probably will buy this product
__ I might buy this product
__ I probably will not buy this product
__ I definitely will not buy this product

You can use information form past online surveys to determine just how these statements translate into actual purchases. For example, you might learn form past consumer behavior that 80% of the people who say they will definitely buy a product actually do. Or you might find that 30% of the people who say they will probably NOT buy a product actually do. (Yes, it’s true that a significant number of people who say they will probably not buy something end up buying it anyway.)

As part of your purchase intent survey you will ask a few questions that will help you build a profile of the respondents. For example, you might ask which newspapers they read or which radio stations they listen to. Now it will be fairly easy for you to target your advertising to those people who “definitely will buy” your product. Since 80% of these people will actually become purchasers, you will earn a fantastic return on your advertising dollars!

You will also gain more valuable data on the relationship between “purchase intent” and actual purchases. The more data you collect, the more accurately you can predict the potential revenues that will be generated by your marketing campaign.

Predicting Impulse Purchases
Conducting online surveys of purchase intent can yield some useful (and profitable!) information, but not every purchase results from a deliberate decision. For example, you go to the drug store to buy cold medicine but you see a candy bar next to the cash register that looks too delicious to pass up. Or you go shopping for a how-to book online but you see an ad for a new mystery by one of your favorite writers and you decide you have to have it. You had no intention to buy the candy or the mystery, but you did anyway.

Online survey software can also be used to predict this sort of unpredictable behavior. You might ask people about the last time they bought something on impulse, with no previous intent. You also could ask what they were planning to buy when they made their impulse purchase. You might find, for example, that a surprising number of people buy candy when they go shopping for cold medicine, possibly as a way to cheer themselves up. This sort of information can help you make decisions about product placement, whether you sell things in a store or online.

You will probably never be able to find the exact combination of factors that will determine purchase intent, but well-designed surveys can give you valuable insights into how customers make up their minds.

Image from: NeuroscienceMarketing.com



Tight budgets are forcing many organizations to take a close look at the meetings and conferences they sponsor. While these events can be highly productive and worthwhile, they can also be expensive. Online surveys can help you plan and evaluate meetings and conferences so that you can make the most of the time and resources you put into these events.

Before, During, and After
If you have enough lead time, you can do a pre-meeting survey to help plan the event. For example, you might ask people what kinds of speakers they would like to hear, or what kinds of demonstrations they would like to see. If you are planning a trade show, you could ask what kinds of products and exhibitors people might be interested in. A pre-meeting survey can also give you an idea of how much time to devote to different topics.
If you have the resources available, you could conduct an online survey during the event itself by setting up kiosks in the exhibit area. The main value of this approach is that it gives you an immediate idea of how people are responding to the event, while the experience is still fresh in their minds. Completing a survey also gives people a break between attending presentations or visiting exhibitors.
Most surveys are conducted after an event. In general, the sooner you do the survey the better. People are more likely respond to a survey immediately after an event, and their responses are more likely to provide useful information. The farther away you get from an event, the more responses will be affected by the filter of memory.
The main purpose of conducting a survey after an event is to do a better job of planning similar events in the future. A survey can tell you which speakers people liked or disliked, which exhibitors they found most relevant, and which meals or social occasions they enjoyed the most. A survey can also give you useful information about the event itself — for example, what people thought about the venue, the registration process, and the schedule.

What to Ask
Unless you already have a good idea of who is attending your event, you will probably want to ask a few questions to create a profile of attendees. For example, you might ask, “Which industry does your company operate in?” or “What was your main reason for attending this conference?”

You will also probably want to ask about the event itself. For example:
• How would you rate the conference facility?
• How would you rate the ease of registration?
• How would you rate the conference publications?
As we have already seen, it is common to ask about the speakers, the program, and the exhibitors. This information is less important as an evaluation of the present conference than as a tool for planning other conferences in the future.
As with any survey, there is no point in asking a question about something that you would never consider changing. For example, if you are committed to holding future meetings at the same site, there is no reason to ask people to suggest a different one.
By the same token, you should try to make people feel that you are actually listening to their responses to your survey. For example, if the great majority of people found one of the speakers boring and uninformative, it would probably be a mistake to invite that person to speak at your next conference.
If you ask the right questions and you pay attention to the responses, surveys can help you plan events that almost everyone — including you — will consider worthwhile.


Whether you’re selling hot dogs or homeowners insurance, keeping your customers satisfied is essential to the survival and success of your business. Fortunately, the latest online survey software simplifies the task of collecting and analyzing data on customer satisfaction. These are powerful tools, but to use them most effectively you need to be careful not to define customer satisfaction too narrowly. Let’s take a broad look at this topic and then consider some specific aspects of customer satisfaction that you should probably be measuring.

Beyond Smiley Faces
Broadly defined, customer satisfaction is a measurement of how well your product or service meets your customers’ expectations. If you are not meeting your customers’ expectations, they may start to look elsewhere for their hot dogs or homeowners insurance. If you are meeting their expectations fully, they are likely to stick with you, unless they think one of your competitors is offering something better. If you exceed their expectations, not only will they stick with you but they might even recommend you to their friends.

A customer satisfaction survey usually includes a general question like, “Overall, how satisfied are you with Henry’s Hot Dogs?” That’s not a bad question, but the information it provides is not all that useful. What does it mean if a customer gives your product four smiley faces rather than five? How will you use that information?

To get more valuable information out of a customer satisfaction survey, you need to be more specific. People have expectations about all sorts of things. An effective survey about homeowners insurance might ask customers how satisfied they are with your product in terms of:
• Price
• Ease of purchase
• Breadth of coverage
• Helpfulness of customer service personnel
• Explanation of policy terms
You might also ask customers which of those items is most important to them in deciding whose policy to buy. Then you can use a technique called multivariate analysis to determine which areas of customer satisfaction offer the greatest opportunities for improvement. For example, suppose that most customers thought “breadth of coverage” was “very important” but they were only “somewhat satisfied” with the coverage your policies offered. This would obviously be an area that you would want to address. (This is a simplified example. Multivariate analysis can actually produce much more sophisticated measurements of how different aspects of customer service are related.)

Measuring the Intangibles
It’s important to ask customers how they feel about the attributes or benefits of your products, but you can’t stop there. Purchasing decisions involve more than just an objective evaluation of competing products. These decisions are also affected by brand loyalty and other intangibles that can be difficult to measure.

One way to evaluate brand loyalty is to simply ask how long a customer has been using your products. You might also ask how often a customer chooses products from your competitors. A less direct approach is to ask customers to rate your products in terms of overall quality. This type of question will not give you detailed information, but it will tell you how customers feel about your products in general. You can also get a sense of this by asking two crucial questions:
• Do you think you will purchase our product the next time you need hot dogs?
• Would you recommend our products to friends?
Some marketers believe that the second question is essential in any customer satisfaction survey.

Keeping It Current
Markets are constantly changing. New competitors and new products may enter the field, and you may need to adapt to changes in the needs and expectations of customers. To keep up with these developments, you need to review your customer satisfaction surveys frequently to make sure you are asking the right questions.



Advertising effectiveness surveys help a business make the most of its advertising budget.

Advertising decisions have become extremely complicated. Many people — especially young people — spend more time online than watching television. But are online ads as effective as TV ads? And if a business decides to use TV ads, which of the hundreds of cable channels should it focus on? And what about the print media and outdoor advertising? What place do they have in a well-planned marketing campaign?

The answers to all these questions are different for different products and for different segments of the market. Marketers need reliable data to make informed decisions about their advertising budgets. Fortunately, online surveys provide an economical way to gather and analyze this information.

An advertising effectiveness survey can determine how consumers react to an ad, what they remember, how they felt afterwords, how the ad can be improved, and if the ad’s intention was served. A sequence of surveys can measure brand awareness before, during, and after a campaign. As with other types of online surveys, the results can be analyzed by age, gender, income, and other variables to ensure that ads are reaching the correct demographic group.

Some businesses rely on sales performance to gauge the effectiveness of their advertising. They assume that if they are doing better than their competitors, their ads must be working. This approach ignores the fact that sales are determined by a multitude of factors, including price, quality, and competition. A business may be doing well, but more effective ads could make its performance even better.



Savvy business owners know that marketing should begin long before their products hit the stores. Marketing should begin with the product itself, and with an in-depth understanding customer’s needs and preferences.

Powerful new questionnaire software gives businesses a revealing glimpse into the minds of potential customers. Market research surveys can help a business rethink product design, or maybe just fine tune it a bit. In simple terms, these surveys can help a business give people what they want.

New product surveys try to get at the perceived needs of customers. They ask questions like:

  • “How much cargo space do you need in your truck?”
  • “How many watts does your backup power supply need to generate?”
  • “How often do your drive on icy roads?”

Questions like these can help a business decide if its product line meets the needs of potential customers, or if there are gaps in the line that need to be filled. Online survey software allows a business to sort responses by age, gender, location, income and many other variables. This enables a business to target products to the needs of specific market segments.

New product surveys also try to determine the benefits that are most important to customers. Take vitamins as an example. Are customers mainly interested in vitamins that will help them feel better or boost their memory or improve their heart health? Questionnaire software uses sophisticated ranking questions to help isolate the benefits that people care about most.

New product surveys also help a business learn what features are important to customers. If customers show little interest in a particular feature, a business may decide to offer it as an option rather than a standard part of its products. On the other hand, if a feature shows up on a survey as being very popular, a business might decide to include it in all of its products, and emphasize it in its marketing campaign.



| Back to top