Experiments Workshop for Recruit

1. Learn-Measure-Build
Craft Useful Experiments By
Running Backwards
Jason Fraser & Janice Fraser
February 2014

2. LUXR.CO JUNE 2012

3. LUXR.CO JUNE 2012
TWEET!
#LeanStartup
@clevergirl
@luxrco

4. © 2013 LUXr Incorporated, www.luxr.co
June & Pete’s
Wedding Cake

5. © 2013 LUXr Incorporated, www.luxr.co
http://saptstrength.com/
Pete June

6. © 2013 LUXr Incorporated, www.luxr.co
April
http://saptstrength.com/
May
June
July
August

7. © 2013 LUXr Incorporated, www.luxr.co
sedonaweddingcakes.com

8. © 2013 LUXr Incorporated, www.luxr.co
freebestpictures.blogspot.com

9. © 2013 LUXr Incorporated, www.luxr.co
thekitchn.com

10. © 2013 LUXr Incorporated, www.luxr.co
Zoe Clark, The Cake Parlour

11. © 2013 LUXr Incorporated, www.luxr.co
sedonaweddingcakes.com
Zoe Clark, The Cake Parlour

12. © 2013 LUXr Incorporated, www.luxr.co
LEAN
Startup

13. © LUXR.CO APRIL 2013
LEAN
Startup

14. © LUXR.CO APRIL 2013
Build
measure
learn!

15. © LUXR.CO APRIL 2013

16. © LUXR.CO 2013
Lean Startup is NOT
Cheap Startup
Fast Startup
Shortcut Startup

17. © LUXR.CO APRIL 2013
An approach for building companies that are
creating new products and services in
situations of extreme uncertainty.
The approach advocates creating small
products that test the entrepreneurʼs
assumptions, and using customer feedback to
evolve the product, thereby reducing waste.
Lean Startup is...

18. © LUXR.CO APRIL 2013
1. List your assumptions.
2. Understand your customers.
3. Experiment efficiently.
4. Adjust direction based on evidence.

19. © LUXR.CO APRIL 2013
1. List your assumptions.
2. Understand your customers.
3. Experiment efficiently.
4. Adjust direction based on evidence.

20. © LUXR.CO APRIL 2013
Go backward to go forward.
Learn
Measure
Build

21. © LUXR.CO 2013
Plot the difference
THINK
release
MAKE
release
MAKE
release
MAKE
RISK=UNVALIDATEDEFFORT
TIME

22. © LUXR.CO 2013
Lots of little wigglesRISK=UNVALIDATEDEFFORT
TIME

23. © LUXR.CO 2013
RISK=UNVALIDATEDEFFORT
TIME
Each wiggle is a learning cycle.
MAKE
releaseBUILD
LEARN
BUILD
MEASURE

24. LUXR.CO JUNE 2012
There’s a pattern to
startup work:
ideation
decision
making
ideation etc.

25. LUXR.CO JUNE 2012
In foolproof
experiments, the
results should be self-
evident.

26. LUXR.CO JUNE 2012
1. Choose the problem to tackle. Clarify what’s
uncertain, what you need to learn.
2. Understand the possible answers.
3. Brainstorm “what you could do to prove it”.
4. Choose the most broadly differentiated set of
indicators.
5. Run the experiment.
6. Write down the results...on a sheet of paper.
7. Project the results into your future.
7 steps to foolproof experiments

27. LUXR.CO JUNE 2012
The Workshop

28. LUXR.CO JUNE 2012
Product
What is the product weʼre working on?
Today’s Challenge

29. LUXR.CO JUNE 2012
A Real Product
What is the product weʼre working on?
Weʼre going to make a Business Canvas.
Today’s Challenge

30. LUXR.CO JUNE 2012
ENGINE OF
GROWTH
VALUE STREAM &
MARKET STAKEHOLDERS DISTRIBUTION
COSTREVENUE
PROBLEM
CUSTOMER SOLUTION
VALUE PROP
1-Viral
2-Sticky
3-Paid
1. Choose the problem to tackle. Clarify what’s
uncertain, what you need to learn.

31. LUXR.CO JUNE 2012
{Activity}
Identify assumptions
10 assumptions, 1 per sticky

32. LUXR.CO JUNE 2012
{Activity}
Divide into 2 piles
Will kill the company
in the next 6 months
if weʼre wrong
Everything
else

33. LUXR.CO JUNE 2012
{Activity}
Review & de-dupe

34. LUXR.CO JUNE 2012
{Activity}
Stack rank the urgents

35. LUXR.CO JUNE 2012
{Activity}
Pick the top one

36. LUXR.CO JUNE 2012
✓ Choose the problem to tackle. Clarify what’s
uncertain, what you need to learn.
2. Understand the possible answers.
Your Top Risk
People don’t
like my
product.
Possible
Answers
Don’t like it.
Do like it.
“Meh.”

37. LUXR.CO JUNE 2012
✓ Choose the problem to tackle. Clarify what’s
uncertain, what you need to learn.
✓ Understand the possible answers.
3. Brainstorm “what you could do to prove it”.
7 steps to foolproof experiments

38. LUXR.CO JUNE 2012
Experiment Framework
For Example:
We believe people like [customer type] have a need for
(or problem doing) [need/action/behavior].
The smallest thing we can do to prove that need is
[experiment].
We will know we have succeeded when [quantitative/
measurable outcome] or [qualitative/observable outcome].
Every experiment has three parts:
1. Hypothesis that is provable/disprovable
2. The experiment itself; the thing you build
3. An indicator of result

39. LUXR.CO JUNE 2012
{Activity}
Design an experiment
to learn if this is true.
we believe that ________________
_________________________________
_________________________________
_________________________________
_________________________________
_________________________________
_________________________________
__________________________
Briefly describe it.
State how you will know if the
hypothesis is valid or invalid.
This can be quantitative
evidence or qualitative.
How much time/money/effort
will it take?

40. LUXR.CO JUNE 2012
{Activity}
Discuss, then pick the
experiment to run.
we believe that ________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
____________ in 2 days.
Decide
we believe that ________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
____________ in 2 weeks.
we believe that ________________
_________________________________
________________________________
________________________________
________________________________
_______________________________
_______________________________
____________ in 2 months.
✔

41. LUXR.CO JUNE 2012
✓ Choose the problem to tackle. Clarify what’s
uncertain, what you need to learn.
✓ Understand the possible answers.
✓ Brainstorm “what you could do to prove it”.
4. Choose the most broadly differentiated set of
indicators.
7 steps to foolproof experiments
TODAY:
Choose an experiment you
can do from this room. Not a survey.

42. LUXR.CO JUNE 2012
Experiments that were designed
assumption experiment
measurable/observable
outcome
busy people need a service
to help them get
menial stuff done
fb ad, neighborhood targeting
presents an offer, landing page.
Phone number or tweet. No out
of pocket cost.
10% ctr on ad, 10% ctr
on landing page. 2 days
busy people need a service
to help them get
menial stuff done
find busy people (in offices). Set up a
“shop”. Sit in their reception and see if
we can get jobs. Prevail upon friend
who runs 300-person company
5% of employees will make
a hire. 1 repeat booking
2 wks
busy people need a service
to help them get
menial stuff done
ad in 10 different offices,
diff types of companies
2 weeks.

43. LUXR.CO JUNE 2012
Yusuke
Tell your MVP Story!

44. LUXR.CO JUNE 2012
1. Choose the problem to tackle. Clarify what’s
uncertain, what you need to learn.
2. Understand the possible answers.
3. Brainstorm “what you could do to prove it”.
4. Choose the most broadly differentiated set of
indicators.
5. Run the experiment.
6. Write down the results...on a sheet of paper.
7. Project the results into your future.
7 steps to foolproof experiments

45. LUXR.CO JUNE 2012
What did you learn?
Top 5 things
{Debrief}

46. LUXR.CO JUNE 2012
Design the next
experiment:
2 days, 2 weeks, 2
months
{Next Steps}

47. LUXR.CO JUNE 2012
What’s the right size for
the experiment?
{Choose}

48. LUXR.CO JUNE 2012
Techniques Checklist
• Brainstorm quietly
• Use Sticky Notes for ideas
• Self-Edit
• Go to the wall for team sharing
• Stack-ranking
• Roman Voting
• Right-Sizing the work

49. LUXR.CO JUNE 2012
FISHBOWL