How to Validate Your Product Idea with an Initial Quick Experiment

Notes on a blogpost on by Grace Ng, co-founder of Javelin, software for testing the viability of new ideas.

Step 1 Design an effective experiment

Break down your idea into testable elements. An effective experiment must clearly define:

  1. Hypothesis
  2. Riskiest assumption
  3. Method
  4. Minimum criterion for success

Lean Startup Machine created a tool for this called Experiment Board which is definitely worth checking out.

Step 2 Construct a hypothesis

  1. Define your customer—identify which customer segment experiences the most pain and start with them. (Separate your customers into segments with specific differentiating characteristics, such as lifestyle and environmental factors—being very specific here allows you to pivot if your product doesn’t resonate with the initial set of customers.)
  2. Define the problem—create a phrase that defines the problem from the CUSTOMER’S point of view, not your company’s. Keep it short and specific; if it’s too broad it’s difficult to test and to define actionable results.
  3. Form a hypothesis—brainstorm some differing customers and problems. Then combine one customer and one problem (“I believe customer x has a problem achieving goal y”) to form your testable hypothesis.

Step 3 Identify your riskiest assumption

  1. Start by brainstorming a few core assumptions that are central to the viability of your hypothesis or business. Think of it as a behavior, mentality or action that needs to be true to validate your hypothesis.
  2. Get suggestions from others on the team to ensure you haven’t overlooked any basic assumptions.
  3. By testing the riskiest assumption first, your experiment cycles is sped up: if the riskiest assumption is invalidated. you are most likely on the wrong path with your product.

Step 4 Choose a method

  1. Method: Exploration. Conduct qualitative interviews to verify and deepen your understanding of the problem. You may quickly realize the opportunity isn’t as large as you thought or that there’s a bigger problem to solve instead.
  2. Method: Pitch. Sell the concept to customers before building the product by building a simple website that requires users to submit their email/pay a small deposit etc to measure your customer’s determination to solve the problem.
  3. Method: Concierge. Personally deliver the service to customers to test how satisfied they are with your solution, whether the value proposition is strong enough, what is actually useful, will they return/recommend to a friend etc.

Step 5 Set a minimum criterion for success

  1. Decide what constitutes success (the weakest outcome you will accept to continue pursuing the idea), and what constitutes failure (usually factors like budget, opportunity cost, size of market, level of demand etc).
  2. The criterion is usually expressed as a fraction: “I expect x number of people out of y number of people to exhibit behavior z.”
  3. Determine how much revenue is required in order to continue development.
  4. Testing as few as 10 people may be enough at this point. Once you have validated your hypothesis, you can scale up the experiments to test with larger samples or other segments.

Step 6 Run the experiment

  1. Make sure the entire team agrees with the experiment–use the Experience Board here if you have one. You and everyone must be accountable to the results.

Step 7 Analyze the results, decide on next steps

  1. Document the results and learnings (use the Experience Board if you have one). Did your experiment meet its criterion for success? If so, you can move forward to the next risk test.
  2. If not, formulate a new hypthesis based on your learnings. Then run this test again with the new hypothesis.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s