March 5, 2019

B2B Marketing and the Art of the Survey

Surveys can provide a wealth of data on any sort of topic. But there is an art to crafting the perfect survey, one that invites honest and valuable feedback without steering the respondents, and doesn’t damage the responses by overwhelming the survey takers.

Leela Srinivasan, CMO of SurveyMonkey, took some time after recording her recent RTU episode to discuss survey creation with Drew. Well-constructed surveys are crucial for informing creative, effective marketing efforts. To learn more about artfully crafting questions, audience selection, getting people to take your survey, making sense of the information, and more, check out this brief, bonus episode!

Connect With Leela Srinivasan:

Resources & People Mentioned

Connect with Drew

Full Transcript: Drew Neisser in conversation with Leela Srinivasan

Drew Neisser: We’re doing a bonus episode with Leela Srinivasan, who is the CMO of SurveyMonkey. If you’ve just found this on renegade.com, you’ll want to listen to Part 1 first but what we’re going to do now is a quick tutorial on surveys. You’re a B2B organization. There are lots of ways that you can field surveys. Is there a B2B brand that you’re working with right now that you think is using your platform in an interesting way? That might be a good place to inspire folks.

Leela Srinivasan: Oh goodness, is there a B2B brand? I think I am currently most inspired by what Box is doing in the enterprise. Box is on the track to try and provide what they call mind-blowing experiences to their customers. What Box realized is that in order to deliver these mind-blowing experiences they need to be listening to their customers through that customer journey, gathering feedback through surveys, and making sure that feedback is part of the central nervous system of Box, as Box is on the front lines dealing with those customers helping them solve problems, helping them navigate their products, and so forth.

I think Box is on a tremendous job of embracing our enterprise SurveyMonkey system and leveraging our integration with systems of record that they use, most notably Salesforce, to make sure that as they are gathering that feedback it is flowing straight into Salesforce and enabling their customer success and sales and marketing and so forth to really have a full understanding of the customer.

We talk about customer journey so much and it can be really hard to form a single opinion of what any one customer is going through, but I think the way they use SurveyMonkey Enterprise in conjunction with integrations to make that happen is really a great example for any organization to follow.

Drew Neisser: I think a lot of people would think about voice of the customer research as an aggregate thing and not as an individual thing that a salesperson or a customer rep could actually use to help inform their service. That’s really interesting.

Leela Srinivasan: Yeah. And in fact, Drew, if I may, you can go even earlier in the customer lifecycle. Here at SurveyMonkey, we leverage our integration with Marketo so that when a potential next customer completes a form to ask for a demo, they receive an email and embedded into which is a short survey so we can understand more about them before our BDR or SDR picks up the phone and has a conversation with them. All of that information actually flows into Marketo, so we’re creating ways to accelerate the discussion on an individual basis by knowing more about the individual prospect.

Drew Neisser: We’ll get really basic here. First, let’s assume we made the decision and we’re gonna do some research. What’s the biggest mistake that organizations typically make when they say, “We need to do some research on this.”

Leela Srinivasan: This actually goes back to my consulting days. Before I got into marketing, I was in management consulting and one of the things we used to do in consulting was do what we would call blanking out the deck. In other words, you have to start with the end in mind. Know the specific questions that you are trying to understand or are trying to get to the bottom of.

If you can visualize what the output would look like, literally what the slides might look like that you’re presenting in the next board meeting or at your next executive team meeting, then that can help to inform the question set. That’s the place I always start—what are we really trying to understand and what would we hope to learn? What’s the hypothesis that would underpin this particular survey?

Drew Neisser: I love that and that’s often the way we use it for our clients where we’re helping to use the surveys to get some content. But often, you bring confirmation bias to this and you design the survey to get the answers that you want. That’s not hard. One way to do it is to design it to get the answers that you want. Another way to do it is to misdesign it because you don’t know how to ask the right questions. How do you help? Confirmation bias is such a natural thing. It is the beginning of any curiosity, “Well, I think I know what the answer is, and if I ask these questions this way, I can pretty much confirm.”

Leela Srinivasan: That’s a good question. You have to try and be as objective as possible and really map out the mutually exclusive, collectively exhaustive reasons for something being the way it is. I think this gets at, you know, if you’re asking the question of “Why X?” then the options that you offer up become really important to make sure you’re trying to cover as much of as many of the bases as possible without overwhelming the respondent with a laundry list that is three and a half screens long.

I think one very practical thing you can do before you launch any survey is run it by a few people in the organization because, invariably, if you design this survey in a silo and have a really strong idea of what you’re going after, it’s much likelier that you will miss something if you don’t open up to feedback from your colleagues in helping to strengthen the storyline and strengthen the understanding of the underlying issue.

Drew Neisser: Yeah, I think that’s such an important point. A lot of times when we pre-test surveys, we’re just testing to make sure they work.

Leela Srinivasan: Right. We’re focused on the logic. And that’s fine, you want to check that too. You want to make sure that if you’re forcing somebody to give another answer, that they actually tell you what the other is or that they can’t choose two answers that are contradictory. That’s essential, but you also have to just take a step back and look at the specifics, look at the real guts of the content, and get those other opinions to flag any blind spots.

Drew Neisser: One of the big issues with surveys and fielding the surveys, particularly if you’re trying to create say, information that you could use in PR. You want to come up with an idea that the brand can own, like you guys could have a curiosity index where you measure “How curious is America versus London?” You would have this global curiosity index.

The way you design that research, obviously, is important, but who you ask is also important. Because there is some, very little, but there is some online bias. The offer that you put out there could create a bias too. There’s a value exchange here, and some people would be fine with a $5 gift card, but other people might need $200 dollars to get their time. That was like five questions in one but, from an audience standpoint, when you’re just building who you talk to, what is some of the guidance that you provide?

Leela Srinivasan: All right. In five words or less, right?

Drew Neisser: Exactly.

Leela Srinivasan: All those questions are very valid ones. I guess the first thing I would say is don’t just ask your friends. I think there is a tendency to sort of be quote-unquote “lazy” and pursue the audiences from whom you can get quick answers. That’s fine, you’ll get a read, but it will often be a biased read unless you open up to a broader audience.

Your point on collectors even, the ways in which you collect the feedback or the mechanisms that you use, is a good one too. For example, we enable surveys to be served up through Facebook Messenger for instance—that will reach a certain audience that other mechanisms might not. We also offer something called SurveyMonkey Anywhere, which allows feedback at trade shows or on the factory floor or somewhere where there isn’t Wi-Fi connectivity.

Definitely thinking about the different populations that you’re able to tap into—some of those, you’ll be able to get to yourselves, others you can turn to third-party services. We happen to offer a panel called SurveyMonkey Audience, which is just so fast to deploy to, and we can help you to diversify your audience as well there. So, definitely be thinking about the diversity of the audience and, again, thinking ahead to the output that you want to share with your key audiences, are there specific populations? Do you want to segment based on age or gender or ethnicity or a particular segment that matters to you? Make sure that you are asking enough of those people to hit your quota or your targets.

Drew Neisser: I’m so glad you brought up a SurveyMonkey Anywhere because I was with a group of CMOs recently down in DC and one of the questions came up about event experiences and how expensive it was to do them. One of the suggestions that I made to them was, well, while you have these folks there and you’re giving all this value, give them a three-minute survey on an iPad, and then suddenly you have research that you can use to extend the event.

You could at minimum measure the experience that they just had, but ideally, you can be gathering information that can continue the story. It’s sort of like what you did with the launch of “Power the Curious” because you got folks to fill out these surveys and then that became content which you could then extend out later on.

Leela Srinivasan: On the topic of events, the last couple of speaking engagements I’ve done, I’ve embedded a QR code in my slides where, if you have an iPhone, you can just flip open the camera and it will automatically take you to a survey link. You can get that three-question survey filled out on the spot by the audience and have your phone open and actually see the results as they come in and be able to real-time share insights back with the audience. It just adds another layer of interaction to your presentation.

Drew Neisser: Here we are all thinking that QR codes are dead, but what’s funny is, I was at an event the other day and someone just popped LinkedIn’s QR code and suddenly everybody in the room started using it and it became the fastest way of exchanging information. I was like, “Oh my gosh, this is like what Snapchat was doing before Snapchat died which I thought might save the QR codes, but that’s not going to happen. Did people do it? QR codes in and of itself are one of those things that just seems to have done great in Japan and then Americans just have not adopted it.

Leela Srinivasan: Yeah totally. I had the same degree of skepticism around QR codes in general before we actually launched this. I think the difference is that if you have an iPhone with our QR code, you don’t need to download a reader or have any other sort of technology. It is literally as simple as pointing your camera at the screen as if you’re about to take a picture of it and that URL will just pop open in your browser.

I think it’s become so seamless and so easy that we definitely are seeing a lot of uptake in that type of situation. Now, I will also say that when I was speaking in Mexico last year, I tried the same thing, and iPhone doesn’t have quite the same penetration down south of the border as it does north, so I didn’t get the same volume of responses. To your point around do people actually do this or not, if you’re relying on them to have a QR code reader on their phone that they have to go fiddle around with, maybe it’s not quite as much of a slam dunk. But, thanks Apple, I guess.

Drew Neisser: Yes, exactly. It’s not going to happen. One of the other keys there is you mentioned only having like three questions. In this study that we did recently for my book, we promised that it was a four-minute survey or less and in fact, the average was 3 minutes and 51 seconds. There were many other questions that I wanted to ask, but I said I’d rather hit my response number so that I had statistically significant data as opposed to getting a few more pieces of information. I know there’s a tradeoff—there must be a curve—but we got 100% of the people who started the survey completed the survey, which is a good sign that you’re not too long. What is the threshold, or is there one, for length?

Leela Srinivasan: It’s a great question and there is definitely a curve and a tradeoff that you make. I think what you described in terms of really disciplining yourself to ask fewer questions in the hope of getting higher quality information is definitely a great way to go. One thing you might want to think about within even that construct is, what types of questions are you answering?

What we do see is we see a lot of people—and I suppose this is the downfall of curiosity—you could be endlessly curious about what your customers are thinking or what your prospects are your employees are thinking, but in the contract of the survey, if you have too many open-ended questions where you’re expecting people to actually generate that content or that idea, then that’s where you start to see your response rates really go down. Another important consideration is, what is your blend of multiple-choice, easy for somebody to quickly answer versus the questions where you’re making them think a bit harder? That will impact your response rate as well.

Drew Neisser: Exactly. I did a couple of true/false questions right away just make it easy. “Oh, that’s gonna be easy! Oh, wait. Not quite that easy.” But yeah, that makes a lot of sense.

Now you were starting to get at this issue which I talked a little bit about which is, the questions that you ask matter and how you asked the questions, but there’s also this other big massive issue which is that there’s no shortage of data but there’s a definitely a dearth of insights. You can ask this and there’s not enough time to analyze all this stuff and this is where, I would say, that there is some science, some research professionals—it’s not anthropology, but they’re people who understand the meaning of questions and linguistics and these are people who get paid to create tests. They actually know what it means to ask a question. How do organizations, and we’re talking say mid-sized B2B businesses, make sure that they’re actually finding insights out of all this data?

Leela Srinivasan: It’s a good question and one that, at SurveyMonkey, we’re actually investing quite heavily in our analyze dashboards, which are where your results are aggregated in a way that you can hopefully easily understand the output and share it with broader sections of your organization. Back to the question design and understanding what you’re trying to solve for at the very outset, I think that can help. Then as we continue to go forward, we’ll keep investing in the analyze portion to try and make it easier and easier for people to really digest and know what to do with the results.

Drew Neisser: This is where you’re getting into AI in terms of both using it to help form the questions as well as how to interpret it?

Leela Srinivasan: Yeah. There’s the of the AI or ML portion on the front end of where we help you to optimize your question set, so the particular questions that you’re asking as well as the set of potential answers. Through our 19+ years of learning, continuously learning from all of the surveys flowing through our system, we have pretty good insight as you can imagine on the format of questions that will work and the options that you should offer up to respond.

We’re doing that guiding on the front end. We’re also scoring surveys through SurveyMonkey Genius to make sure that you give yourself the best possible chance of quality answers and quality responses. Then on the back end, as the results start to flow in, those are surfaced in what we call our analyze dashboards, and that is the opportunity for you to better understand the information that’s flowing in. As we continue to make that information shareable with your broader organization, that will help you to socialize the findings and have the really important discussion of “What do we do with this? What is next?”

Drew Neisser: Where I was going is, and it’s interesting because you went into package and distribute, where I was going is—it takes certain types of minds to connect disparate dots. It’s unlikely that a computer would have been able to write the line “Power the Curious.” That’s not even structurally, one might argue, not quite a sentence, not quite words. There’s something goofy going on that allows you to think about those words differently and which is why they work so well. There’s something off about it that makes you go, “Wait, I’m curious about that.” A computer can’t do that, so the research might have found this notion of curiosity as something, but I guess what I’m trying to get to is the human part of interpretation. It’s the need to have individuals who can not just crunch the numbers but also think about the data imaginatively.

Leela Srinivasan: I think that’s a really prized skillset, and fortunately, I don’t think that need is going anywhere. I think that people who can bring that eye for the insight to the table are already in high demand and I think will remain in high demand as we go forward.

The one thing that we are doing to help accelerate the understanding of the results is to help organizations put them in the context of other data that they might have at their disposal. We see organizations pushing survey results into other systems of record, we see them pushing SurveyMonkey data into Tableau and pairing it with other data to better understand the correlations and how specific aspects of respondents might tie in with the results.

This is why I think data science, analytics, insights—I mean these are all such hot areas within marketing and well beyond marketing. I suppose the good news is that as we continue to evolve the tools and the technology to deliver more AI, to deliver more machine learning, to help answer those questions, we still at this point need that human interjection to really fully interpret. I think this is where the open-ended questions of really hearing that voice of customer layered in with some of the numerical results really kind of kick your understanding up a notch.

Drew Neisser: We’re going to wrap this up but I have to ask a question— could you guys come up with something other than NPS, please? You’re on the brink—it’s funny, you guys are the facilitator for all this research but you haven’t developed a proprietary thing and yet you probably could. I was with a CMO who said, “Our NPS is 92.” I went, “Okay, that’s ridiculous. That’s useless and absolutely stupid information.” What do they do? “Oh,  we’re perfect.” What! I don’t know. I’m very familiar with it. I’ve seen it used and dumb information is the problem.

Leela Srinivasan: Well, I hear you, Drew. Two things. Number one—I say this full disclosure—I worked at Bain & Company and NPS was developed at Bain as IP. It was developed as the ultimate loyalty metric, the ultimate indicator of whether or not a customer is likely to stay or go. I think it still has a ton of utility in that area and what is important is not the question of the score itself, but the why and undoing that root cause analysis to understand the drivers of that over time. That’s the first thing. The second is, I will take your challenge back to my head of research and I will tell him that Drew challenged us to come up with something, and maybe it’s a discussion that we just might have.

Drew Neisser: There you go. Love that. Let’s wrap this up. I think it’s really helpful. Two dos and a don’t—you’re a B2B marketer, you’re thinking about all the research that you’re doing—two dos and a don’t.

Leela Srinivasan: All right. Do listen incessantly to your customers. You cannot have enough customer feedback. Do think about how you can package that in a way that will benefit future generations of customers. Don’t just ask your friends.

Drew Neisser: There you go. Perfect. The last thing I want to add, and this is one of the things that we’ve done a lot—think about the research not just as a dead-end that you get some information and maybe you make a change here. If you do your research right, so often there’s PR opportunity that can come out of it. One study equals multiple pieces of content as well as potential organizational change or product change or service change and all this good stuff. All right. Thank you, Leela, for Part 2 of a really really interesting episode. I wish you all the best of luck.

Leela Srinivasan: Thank you so much, Drew. It’s been a pleasure.

Quotes from Leela Srinivasan

In order to deliver those mind blowing experiences, they needed to be listening to their customers throughout the customer journey, gathering feedback through surveys, and making sure that the feedback was part of their central nervous.