Church of the Customer Blog
« Remodeling customer surveys | Main | Are you hyperconnected? »
May 14, 2008
Remodeling customer surveys
The other day, a big company sent me an email, asking for feedback about my recent purchase experience.
That's the good news. Even though sites like SurveyMonkey have made it ridiculously inexpensive (and easy) to gather and calculate quantitative customer feedback, I'm rarely asked for feedback. When a request arrives, I always consider it.
The bad news is that this particular company's web-based survey was too long. Some of its questions backed me into a corner. And it didn't ask me the most important question of all: would I recommend them.
I almost gave up after the second question: "How long has it been since you last used our service? Less than 2-3 months, Less than a year, 1-2 years, 3-4 years, More than 5 years." (I continued, knowing it might make for a good blog post.) I couldn't remember the last time I used them, but "I don't know" wasn't an option for that question, nor any others. Memory isn't factory sealed. Besides, what bearing does it have on my recent experience?
I trudged through some 15 pointless questions, dying to give up after each one. The things we do for blogging.
"Would you use us again?" the survey finally asked.
I don't know, I thought. Yes or no were the only answer options.
Well now. The product was good, but the service was pretty bad. If I found a viable alternative, I'd switch. So I answered no. It wasn't the correct answer but backed into a corner, why say yes? The answer is not always binary.
The survey concluded by asking me my age, income, gender, etc. Questions that help ensure survey drop-off rates.
What the survey never asked: If I'd recommend the company, and what I'd say if I did (or how I would recommend against them). This national company, known for its transportation solutions, squandered a free opportunity to understand word of mouth.
Which leads us to what a good survey does to gather valuable customer feedback:
- Its first question is: "Based on your recent experience with us, would you recommend us to your friends, family, colleagues, etc.?" Yes, no, or I don't know are the possible answers. (You could use the Net Promotoer methodology here, too.)
- Based on the answer to question 1, the survey then asks, "Tell us more about the reasons for your previous answer." Then I could select from a pre-determined list of reasons for my answer, or blank boxes for me to write my own.
- It would ask me how I would describe the company and/or my experience to friends and colleagues. Again, a list of possible answers could be presented along with a blank field for my own description.
- Finally, it would ask me how the company could improve. I could rank the importance of specific items or provide my own idea which, who knows, could be the dumbest idea ever or somewhat innovative. Process improvement is a never-ending marathon.
That's it. A short and easy survey based on recommendability. The data are actionable for operations, marketing and human resources, which could tie results to team reviews or if done right, to a key metric any employee can appreciate: compensation.
Other blogs that reference Remodeling customer surveys:
To me there are 2 main reasons to run a survey:
1) gather numerical customer data that can support a particular claim or internal business metric.
2) actually learn about the ways that the product is meeting your customer's needs or how it isn't.
The first type of survey is often configured to deliver a desired result and are not usually open to gathering useful customer information.
The second type is far more valuable since it should be attempting to stimulate a passionate response from the person taking the survey. It should probe known product weaknesses and seek insight into how customers currently solve problems despite your products failings. The results may be depressing to a marketing team, but to a product management team they will be gold. Since a pain point that customers are passionate about is a value proposition if it can be resolved.
So I prefer customer 'dis-satisfaction' surveys to get innovative insights into customer problems.
PS: the font size in to comment box is microscopic and hard to read. I will be happier if you fix it. 8-)
The local gas / electric company called with some survey questions and I volunteered for the 6 minute question (why so long?) session as I wanted to help them out.
What followed was mainly pointless questions like "what percent of appliances in your house are Energy Star compliant?" How am I supposed to know that? I think 100% but who knows. And the questions had mind numbing detail like "how many DVD players do you have, but not including Tivo" Why do they care? I was prepared to give them my shoe size next. And how many computers do I have? Why do you want to rob the place?
Since they know who am I they know how much gas / electricity I use and probably the size of my house from public records. All they need to ask is if I have single or double pane windows, how many people live here, and how many large appliances I have and if they are gas or electric. Simple and straight to the point. They'll get a lot more responses (I was about to hang up) and they can go through 5 times more people per hour.
Oh and will I get the results of this survey? Probably not. What reward am I getting out of this? This "survey" was bogus and won't see the light of day.
Great post, Ben. I am in the preliminary planning stages of developing a survey for my next book, and this is a great guide for what to think about. I had already decided to keep it short and sweet, but you raised excellent points about making sure everyone has an answer to click that pertains to them!
Thanks!
Great article! I love the part about being actionable. Too often companies collect a bunch of info and discover that they can't really to anything with it.
I've added some of your information to a marketing wiki. If anybody wants to add their own advice...or correct mine...please feel free to edit.
http://whamwiki.com/index.php?title=Conducting_Customer_Satisfaction_Surveys
I think the way a company writes its surveys says a lot about the company.
For example, a customer driven company really wants to know about how they can improve their service. The bureaucratic and product driven company merely wants to look like they care about you the customer and is looking for statistics for statistics' sake.
The best questions are open ended. But computers can't compile this kind of information. So they use yes and no answers even if they really don't address service issues.
Forbid that the company might actually have to get a human being involved in reading the answers and writing up a subjective summary of the comments! That costs too much and takes too much effort just to find out how the customer really feels.
I'm a huge fan of asking the Ultimate Question. It makes so much sense to shorten the survey and only ask the few questions that matter. Too bad most companies still throw these massive questionaires at people, collecting data that is useless if it's ever actually looked at.
There is hope though. I've noticed a good amount of companies now ask that key, would you recommend, question. Just recently Apple just starting using NPS, so hooray for them.
Thanks for continuing to bring this up. Great post!
This reminded me of this survey I received in the mail from Nissan about the 350Z I bought back in 2005 and haven't owned since 2006 (not a family vehicle). I open this survey and it reminded me of some standardized test I took back in Elementary school. It was 8 pages long and they even included a $1 bill too. At least I could pick up a Clif Bar on the way to the gym ;)
Ben,
Thanks for a good post. I have written numerous times for Gartner on Best Practices for creating surveys, what to get out of them, etc. Obviously I don't have the space, nor would I try, to add them all here... but I wanted to leave you the top 9 best practices for building surveys -- which build on what you said here:
1. Know what you want to get - one single objective
2. Determine success factors and goals
3. Adhere to the "keep it short and simple (KISS)" principle
4. Ensure timely and consistent gathering across channels
5. Read the answers and and ACT on them
6. Repeat it, surveys are only valid in historical trend lines
7. Close the loop, let them know what you did with the feedback
8. Try not to use edit-boxes, else you have to read them ALL
9. Always use a 4-degree scale, remove the fence-sitting
@nick: The text size in the comment box is pretty small. We'll get it bumped up.
@Blogreader: Sounds like too many departments had input on that gas company survey which is, unfortunately, too common.
@Michele: We do what we can here at CotC.
@EdD: Some pointless survey questions are purely based the input of a muckety-muck who says, "Ya know, a really interesting data point would be..."
@Ted: Good observation about companies and their surveys. And the ones that don't survey are ones that *really* don't want to hear what customers have to say.
@Mike: Didn't know Apple was using NPS. Good for them.
@BJ: No offense to those involved, but the surveys from JD Power are bad. In fact, the JD Power surveys may be the worst of all.
@Esteban: Love your list. I might take issue with item 8 only because it doesn't allow for people to identify new landmines or new opportunities. Item 9 is good -- no fence-sitting. My biggest issue with NPS is the 10-point scale. That's a huge spread. Thanks for sharing the list.
Surveys have always bothered me as well, as they never seem to ask you the pertinant questions about what you really want to say. Maybe there should be surveys conducted for survey companies, about how much they suck, and why big business can never get it right.
Like your other commenters I agree with you in the general premise that companies often ask the wrong questions in their surveys. More commonly they make them so God-awfully long that the survey's length actually filters out a good majority of people who might otherwise be willing to help the company with their input.
From my experience leading the TurboTax Inner Circle (a group of customers who give us feedback that we use to improve our product and marketing efforts at TurboTax) I've learned that you can even shorten the number of questions you've got on your list by 1... most commonly people answer question #4 (how the company could improve) the same way they answered question #2 (Tell us more about the reasons for your previous answer).
The great thing about working with the Inner Circle is they have my personal email address, most of them, and they keep me honest. If they think I'm asking the wrong question, they don't hesitate to tell me the "right" questions to ask. When I get those emails it often casts a different light on the whole reason we were asking questions in the first place. They keep us from having an "outside-in" view of TurboTax - just one more reason to have a relationship with our most passionate customers. Why every other major corporation doesn't do this I'll never know!
Oops, I meant "inside-out" in my last comment. (Not "outside-in")
This is EXACTLY what I needed right now! THANK YOU THANK YOU THANK YOU! Your blog is more valuable than college! Then again I dropped out so what do I know!
Thanks again!
DJ
For small businesses, using the *would you recommend us to others* question has paid off big in my experience. Searching for word of mouth tactics for a client, I asked the question during a brief call with one of his raving fan customers. The customer said she would definitely recommend him and when I asked her how she would to do that, she came up with a brilliant WOM idea. She invited her friends--like her, badly in need of retirement planning help--to her home for an informal session with my client. Instant success. No PowerPoint. No marketing collateral. The informal gatherings grew from there.
I now always ask a recommender: How would you like to spread the word?
You are right on the money with your comments. We host customer feedback surveys for businesses based upon the work of "The Ultimate Question". We assure customers that they can complete the survey in sixty seconds or less. Our clients get more feedback today than they ever have using the long surveys. Better yet is complaining customers are more likely to take a minute to send their feedback giving our clients an opportunity to turn the situation around.
Using Net Promoter we score the results for our clients at a location level. They now have the ability to compare their location to locations within their chain or to locations in similar businesses. It has proved to be a very effective management tool.
The other idea would be to only use a survey when you need it. I am tired of my Lexus dealership sending me a survey and calling me after every service. The same with the surveys that are sent out after a sales presentation where the sales rep preps you how to answer so he will look good. What looks great up in the pulpit sometimes looks awfully foolish out in the pews.
I just had dinner at TGI Friday's, and was offered a free appetizer on my next visit for calling an 800 line from my cell phone before I left the restaurant, and punching in answers to a few survey questions. The survey was a bit too long, but they've got the right idea.
Love this post - thank you. I do surveys for my small business clients in order to help them better understand their existing clients, understand how they can improve, and to get great testimonials. The simpler, the better.
I have stopped taking surveys. To date, no survey has ever come through the mail or to my inbox with a place for me to really give my answers. I don't mind the questions...even inane ones (one does enjoy a good laugh, sometimes) but what burns me up is when I have 5 choices and NONE of them are relevant to me. Because I cannot really answer the question, it makes all other answers invalid. I have stopped mid-survey so many times, because there is no where to explain why their answers don't fit my situation... that I am done with surveys.
Unless they're from people I know, about subjects that matter. Like their business... if I get hung up on one of those surveys, I can email my contact and explain why I did not finish the survey.
I have long since stopped giving out any data about me, my business, or my family. If I get to a point in the survey where they ask my age, income, etc. I, like many others, quit. In this day and age, why companies don't know this and also why they do not ask the referral question is baffling to me. It seems it is their loss - in many different ways!
Another resource of good referrals, while we are on the subject, is Client Advisory Boards. Referral rates from a client advisory board is typically many times higher than from a formal referral program. I work for CABHQ and we help clients in the creation and facilitation of Customer Advisory Boards. Our clients are wowed by the increased number of referrals from their board members, and that is just one of the many advantages of having a strategic relationship with your clients.
I think you and some of the people responding do not know much about market research. Yes, there are a lot of poorly written questionnaires out there - boring, too long, poorly designed. But often there are reasons why certain questions are asked or response choices offered that are not apparent to the respondent. For example, asking about the type and number of electronic devices in your home could be used to segment the audience into groups of heavy and light technology users. Analyzing the survey results by these two groups can provide important marketing, communications and product development information to the manufacturer. I might care when you last purchased my product because product reliability has changed over time and I need to put your other responses into context.
Sometimes response choices are specifically designed to make the respondent commit to an opinion. Yes this can be frustrating, but again when used judiciously this can provide valuable information in the analysis.
I could go on, but basically my point is that the respondent cannot know the objective of the research, how the results are going to be used and why the research was commissioned in the first place.If the interviewer says the survey will take 5 minutes and in reality it takes 15 minutes, that is wrong. If the response choices are ambiguous (How old are you? 18-25; 25-30; 30 and over. What if I'm 17 or if I'm 25, which box do I check?) That is wrong. But just because you don't like the response options doesn't mean that the questionnaire is bad.
@Marjorie: The problem with most marketing research meant to capture data about a customer's experience vs. more in-depth customer segmentation is unnecessary granularity.
"Reliability over time," as you mention, is a good example. A previous purchase may or may not necessarily be an indicator of reliability. It could be price sensitivity, discounts, competing product availability... hmmm... color choices -- pretty soon you have a typically long and meaningless survey. You could fill up a telephone book with cross-tabbed variables.
Id' much rather understand reliability as a subset of why someone recommends/doesn't recommend the brand/service.
As for response options, the point that I and others are making is that they are often not applicable or irrelevant. For a survey designer to not include a "don't know/not applicable" selection, or a text box for their own answer is, I fear, trying to game the results or simply befuddled.
@GL: I recently received a survey from Yamaha asking about my Vino ownership.
I'm a somewhat qualified survey responder -- I even blogged about the affection I have for my Vino.
But I took one look at the length of the survey beast and said forget it.
Excellent article. In my years in marketing I've always found this to be true, keep it simple! Otherwise people ignore you.
Nathan
http://www.hiyaya.com
Thank you, good post. I already have a page on my sute for suggestions, but after reading this post I've put as my e-mail signature the Ultimate Question, pointing to that page for an anonimous answer. I'll see if it works.
I have noticed that also the feedback requests from Google are terribly long.
And the text size in your comment box is still terribly small.
Hi
This is bondjames
I think this site is providing information about cash surveys. I think this site is to be lot of helpful to who wants to do survey they require to see this.
==================
bondjames
Cash Surveys
Yes, surveymonkey is an excellent tool!!
I use it always!!
I think we should search for some other survey tool which is complately free.
I am sure there must be some other tools in the market.
