Navigating the Politics of Concept Testing
There is nothing in the world of market research so deceptively complicated and political as concept testing. This can be especially frustrating because a concept test should be one of the most straight-forward and elegantly simple pieces of research a market researcher or insights manager has to field.
I have described concept testing before, but again, in short, in a concept test you put a new (or existing) idea on a piece of paper and ask consumers, “Would you buy this?” In a perfect world, the consumers would be able to tell you “yes” or “no”, you’d record the answer, report it back to the organization, and make the right business call on the product idea in question. Sadly, the world is not perfect. Even less so in concept testing. If you’ve ever felt the pain of the political swirl that surrounds concept testing, or you hope to avoid it, this post is for you.
When Concept Testing, Who are My Stakeholders?
In concept testing, there are typically three main stakeholders that a market research manager could need to align to get the test to field smoothly:
- An Advertising Agency: Typically, they wrote the verbiage of the concept and probably provided the product art as well. They feel pressure to deliver for the marketer who is paying them to produce amazing advertising and product copy that sells.
- A Research & Development Manager: They invented the product or the technology in the product. They usually feel pressure to see their product brought from concept to reality. Often, they have a lot of emotional investment in the success of their brain-child.
- A Marketer: Quite simply, it’s their job to bring this thing to market. They feel pressure from their boss, R&D, and their bank account to make that happen.
In your organization, there may be other people, or other job functions who have a vested interest in seeing this new product move from idea to reality. Whoever those people are, you need to make sure you align them to the following:
- The exact version(s) of the concepts being tested.
- Exactly how many will be tested (if cost-justified).
- What methods/suppliers you will use to evaluate and measure the concept.
- How you will gauge “success” or “failure” in the test.
- What will happen next in each possible “success” or “failure” scenario.
If you are not in alignment on those critical elements you are in for a lot of pain.
Aligning On What Exactly is Going Into Concept Testing:
Never, ever, ever, allow the test stimuli to be a “moving target.” The entire group of stakeholders must be 100% aligned to what will be tested. Marketers and their agency partners, God love them, are constantly tweaking their concepts ALL THE TIME. In my experience, this word-niggling has very little bearing on the outcome of the concept testing itself. What it does have bearing on are the politics around the concept.
For the advertising agency, the concept is “precious.” It’s their work, their creative, and their contract that they are putting in your hands to test. Allowing the concept to be a dynamic document before, during, and after testing can nullify the results of the test. If scores are bad, it creates the out of “Well, the concept has moved in a new direction since then.” If scores are good, it still may be hard to get the organization to stick to the “old version” of the concept because the new one must be “that much better.”
If you are going to spend the time, money, and effort to do concept testing for your organization, make sure whatever goes into testing is THE concept. Set a drop-dead date for revisions, and get alignment across all stake-holders that the concept will not change until the concept testing is over — and only let it change then if results are not satisfactory.
Aligning on the Exact Method of Concept Testing:
One of the great paradoxes of being a market researcher is how quickly you can go from being someone’s best friend and ally to their goat. Concept testing is a fast way to either end of that spectrum. If the results are good, few people care how you got them, so long as they are good. If the results are bad, your competence, the credibility of the supplier, and the short-comings of the methodology can all go under a microscope.
Therefore, if the results of the concept testing are poor, you may find yourself having to rebut one of the following arguments:
- “Every test we do with that supplier comes back low.”
- “I don’t think that this methodology is the right one to measure X, Y, Z nuance of our product or industry.”
- “Are you sure you analyzed this data correctly? How many of these concept test have you done in your career?”
If you have taken the time to align the organization to the “how” of testing, and have everyone holding hands in agreement, these questions shouldn’t come-up. If you have pre-concept-testing agreement on the supplier and method, the answer to these questions should be, “We all aligned to this before the test, nothing has changed since then. The scores are what they are. We need to proceed with the actions those scores indicate.”
Aligning On What to Do Next After Concept Testing:
As I said above, if the results of the test go smoothly, things are usually good. The worst case scenario of having all good scores is deciding among a pile of great concepts which one is “best.” Do plan for how you will decide a “best” of the “best” among good concepts. Don’t “good problems” like this for granted. They can still be problems.
In contrast, if the results of the test are poor, you are almost certain to have some tough conversations. If they go poorly without pre-alignment, those conversations can be much tougher. To avoid this, make sure you are pre-aligned on the success criteria AND the exact organizational go-do’s in each and every possible scenario. Having a grid that outlines in (tedious) detail every possible result can save hours of debate afterwards. First, it makes you think-through every possible outcome so you already have a plan. Second, when you recommend that plan, it won’t come as a surprise to any of your stakeholders.
Concept testing would be pretty straight-forward if it were all about methodology. However, business is also a game of human relationships. As Aristotle described us, humans are “political animals.” If you plan ahead for the political swirl that concept testing brings with it, your life will be much easier.
If you have additional thoughts, examples, or tactics to share to help manage the politics around concept testing, we’d love your input. Feel free to add your ideas in the comments below.