Ash Maurya, Spark59, Innovation Accounting in Practice @ The Lean Startup Conference 12/3/12

1. INNOVATION
ACCOUNTING
A Spark59 Case-Study
ASH MAURYA
@ashmaurya
ashmaurya.com
spark59.com

2. In God we trust,
all others bring data.
-W. Edwards Deming

3. Startups don’t starve,
they drown.
-Shawn Carolan, Menlo Ventures

4. What you wanted

5. WTF
What you wanted The reality

6. 01 Establish a standard measure of progress.

7. Financial accounting Innovation Accounting

8. Dave McClure’s Pirate Metrics
How do users find you? ACQUISITION
Do users have a great first experience? ACTIVATION
Do users come back? RETENTION
How do you make money? REVENUE
Do users tell others? REFERRAL

9. 02 Measure everything as a cohort.

10. You cannot step in the
same river twice.
-Heraclitus

11. Signups

12. Signups
Activation

13. Signups
Activation
Paid Conversion

14. Signups
Activation
Paid Conversion

15. Signups
Activation
Paid Conversion

16. Signups
Activation
Paid Conversion

18. 03 Focus on a single key metric.

19. 2%

20. 2%

22. 04 Build a continuous feedback loop with
customers for rapid hypothesis generation.

26. 1. Too busy.
2. Need more information.
3. Just kicking tires.

27. 1. Too busy.
2. Need more information.
3. Just kicking tires.

29. Results?

30. No significant impact.

31. Except...

35. REVISED
03 Focus on a single key metric,
but always monitor the full customer lifecycle.

36. A/B Tests
7-Day Course (A/B Test)
300 Participants have run this experiment. Time Range: July 9, 2012 - July 13, 2012
Control Hypothesis A ?
651 participants (50%) 652 participants (50%)
activated 35% 34%
retention 31% 45%
referred 19% 20%
paid 2% 4%
More data is needed to determine a winner.
Hypothesis A is currently winning with a 44% improvement and a 80% chance to beat the baseline.

37. Lifecycle Messaging
Day 1 Welcome + Week 1 Challenge
Day 3 Educational Content #1
Day 5 Educational Content #2

38. Lifecycle Messaging
Day 1 Welcome + Week 1 Challenge
Day 3 Educational Content #1
Day 5 Educational Content #2
What happened #1 Help content Offer help
Day 9 Check-in + Week 2 Challenge
Day 14 Educational Content #3
Day 20 Educational Content #4
What happened #2 Help content Offer help
Check-in + Week 4 challenge
Day 24 + soft sell
Day 29 Offer Bonus
Extend trial Ask for feedback Offer incentive
...

39. 05 Metrics alone are not enough.

40. Standard Measure of
Progress
COHORT
SPEED
METRICS Rapid Hypotheses
Formulation
SPLIT LIFECYCLE
LEARNING
TESTING FOCUS
MESSAGING
Experiments

41. For an extended version of this talk, visit:
http://bit.ly/InnovationAccounting

42. THANKS!
For an extended version of this talk, visit:
http://bit.ly/InnovationAccounting
Lean Canvas
Ash Maurya | Author, Running Lean | Founder, Spark59 | @ashmaurya
Life is too short to build something nobody wants