Acceleration

Problem interviews are probably the most important and, at the same time, most overlooked and neglected tool to understand about your customers’s needs and problems.

First and foremost, problem interviews are exclusively about customer’s problem(s) and they must not include any reference or question about the solution you are developing or planning to develop. For this reason, you might have noticed that the “Problem Interview” section of the C2E Validation Report does not include an input box for your solution hypothesis, but only for customer segments and problems.

As with all other lean experiments, the focus is on learning and not selling. Additionally, for problem interviews you are not even going to gather any learning about your (future) solution, but only on everything that concerns with the current customer pains and how/where they look for solutions, if they currently make use of alternatives, and if their problem is strong enough to justify paying for a solution.

Unfortunately, asking direct questions to your customers will give you useless information.

Keep in mind an important fact: there is a huge difference between what customers say and what they do. For several psychological bias effects, people tend to try to appear smart or modern/cool when answering questions or they try not to pose themselves strongly against the interviewer point of view (or at least what they think the interviewer point of view is). As lean startup practitioners usually say, customers “don’t want to call your baby ugly” (the baby is your startup/product).

But you really need to know, at this point, if your “baby” is ugly to the eyes of your customers. Otherwise, you’ll end up understanding this the hard way, by loosing time and money on products no one wants to buy.

Therefore, to avoid bias effects from interviewees, you need to ask open ended questions and, first of all, let your customers do the talking. The more you talk, the more they will figure out what you want to hear and they will simply say that to you to avoid the risk of being “wrong” or disappointing you.

Open ended questions are those that do not imply a yes or no answer, and that do not lead the answer in any direction. Let’s look at an example.

The first question suggests that there is something wrong with current mobile payment systems, and the interviewee might think that he/she would appear not smart enough if they claimed to be satisfied instead. There is a very high risk of collecting misleading answers, with corresponding wrong decision making for the business. Sometimes this negative effect is referred under the name “Observer bias“.

The example above also leads to another serious factor to take into account: the so-called “Confirmation Bias“. It is a natural tendency of the entrepreneur conducting the interview to try to collect proof supporting his/her own assumptions, rather than listening and learning instead.

For this reason, the “Problem Interview” section of C2E Validation Report asks you to fill in some of the strategies and specific key questions that you made use of to minimize the negative phenomena described above. This information are crucial to demonstrate the quality of your validation experiment, so please add as much as possible (you also have the option to extend the page with additional space if needed).

Solution interviews are apparently similar to problem interviews and in fact most people tend to execute the two together. This practice, however, reduces the amount of learning you are able to achieve, especially regarding the validation of the customer problem.

The Validation Report has separate sections for the two kinds of interview because it is paramount to conduct separate experiments for them. It is perfectly reasonable to conduct both experiments with identical assumptions and identical hypothesis, but they should be planned and executed separately.

Aim of solution interviews is to test the validity of your solution’s hypothesis and assumptions. Again, the focus is learning, not selling. The distinctive trait of solution interviews is to actually put your solution (your product or service) in front of a potential customer. It might look like a sales pitch, but it’s not.

The “Solution Interview” page of the Validation report template expects you to fill it in with data regarding the measured collection of “currency”. This term identifies any kind of objective indication of interest coming from your customers. If you’re not familiar with the concept of currency, let’s look at an example.

As with problem interviews, there is a big difference between what customers say and do. Read about the famous paradox of the “yellow walkman”, a typical example of customers’ interest expressed about a proposed product (the yellow walkman itself) and a conflicting behaviour arising immediately after (preferring the traditional black walkman as the free gift for participants). Due to this paradox between customer’s words and actions, you should make sure that your proposal is able to trigger some form of tangible commitment from customers.

With regard to how you will present your solution to customers, the Validation Report includes some possible options, leaving you with the possibility to add and describe your own personal method. There’s no requirement to have a fully functional product/prototype to conduct solution interview experiments. You could easily make solution validation tests with some of the cheapest and quickest alternatives included in the Validation Report template, such as mockups, paper sketches or even basic marketing collaterals, describing your product’s features and benefits.

Problem and solution interviews are exceptional tools to learn about customers’ behavior and expectations. However they have an implicit flaw: they are inevitably based on limited numbers, therefore they are not able to provide you with statistically relevant data.

Online surveys are instead perfect to reach wider audiences and be able, for instance, to determine quantitative data about frequency and dimensions of certain crucial business paramenters, such as potential dimension of a customer segment, percentage of individuals with a segment that have certain traits that make them unreachable by your value proposition, and so forth.

The typical survey is delivered online, and is composed of a set of questions requesting the customer input in the form of yes/no, multiple answers, priority lists, side by side comparisons, matrices, etc. The typical online survey should contain no more that 10-12 questions, to reduce the effort required on the customer’s side and maximize the number of respondents.

Due to the fact that online surveys collection is an unsupervised process, surveys are not the right tool to begin your validation with. In fact, they are not able to promote significant learning, especially when you are still validating your critical assumptions.

For this reason, the Validation Report includes the “Customer Survey” page after the problem and solution interviews. The rationale is that you should first validate the core set of problem, customer segment and solution hypothesis and then, with that knowledge in mind, design an online survey to collect further quantitative details on the underlying, or related, assumptions.

Here’s a few great online services that you could use to design and deliver your customer surveys and analyze results:

Detailed instructions on how to fill in the “Video Experience Prototype” page of the Validation Report will be available soon. Sorry for the inconvenience.

In the meanwhile, you could take a look at a notable example of an MVP realized in the form of a video describing a future product. This is from the initial Dropbox campaign, back in 2008.