This document outlines a scientific, hypothesis-driven approach to product management. It discusses focusing on root causes, eliminating bias, and providing real user value. The process involves making observations, forming a hypothesis, running experiments to test the hypothesis, refining the hypothesis based on results, and validating findings with metrics. Key steps are defining the problem, building testable hypotheses, analyzing relevant data, refining over time, and measuring success. Forming good hypotheses requires identifying clear questions and variables. The goal is to objectively solve real problems through replicable, user-centric experiments.

1. S§
Validation and Hypothesis based
Product Management

2. Focus
• Root cause for problems.
• Ditches ideas that irrelevant.
Why should you care?
Feasible
• Solves the real problem.
• No wasted efforts.
Objectivity
• Eliminate personal bias for
what causes what.
Replicable
• Results can be used as facts.
• Results are durable.
User-Centric
• Providing real value to users.

3. The Approach: Step by Step Process (Scientific Method)
1- Observations
2- Hypothesis
3- Experiment
4- Refine
5- Validate
Questions to answer. Observing a trend or a problem.
What is a hypothesis? How to build one?
& Good vs Bad hypothesis.
How to test hypothesis? What data to look at?
Refine hypothesis
How to measure the effect? Success metrics!
FYI: Scientific Method was founded by Ibn Al Haytham around 250 years ago.

4. Observations
Observing a trend or a problem.

5. Sources for trends or problems
Customer Service
SalesOnline Discussions
Product Vision

6. Sources for trends or problems

7. Impact Based Prioritization

8. Prioritization Methods
Relative Weighting RICE Analysis
Based on business metric Based on usage frequency
Perfect for new features Perfect for optimizing existing features
Considers development effort Considers development effort
Reach, Impact, Confidence, Effort

9. Relative Weighting
Prioritization Criteria Evaluation Factors
Increase Sales; maximize ROI
Increase customer trust
Establish a competitive
advantage
Improve productivity
Cost of development
Value Score

10. Prioritized List
Issue Type and ID ROI ICS ECA IP
Total
Value
Value
Percent
Estimate
Cost
Percent
Priority
Add FB sign up
Feature 10
Cash on Delivery
Bug 3
Feature 7
1 6 1 3
3 1 6 1
6 6 1 1
1 8 1 1
6 6 6 3
11
11
14
11
21
10
20
20
40
55
Add Gmail sign up 3 6 6 6 21 15
Pin Address on map 6 6 3 6 21 35
Bug 5 3 3 3 3 12 25
Feature 8 1 3 6 1 11 40
8%
8%
11%
8%
16%
16%
16%
9%
8%
4%
8%
8%
15%
21%
6%
13%
10%
15%
2
1
1.37
0.54
0.76
2.66
1.23
0.9
0.54

11. RICE Analysis

12. Prioritized List

13. 1- Observations
2- Hypothesis
3- Experiment
4- Refine
5- Validate
The Approach: Step by Step Process
Questions to answer. Observing a trend or a problem.
What is a hypothesis? How to build one?
& Good vs Bad hypothesis.
How to test hypothesis? What data to look at?
Refine hypothesis
How to measure the effect? Success metrics!

14. Hypothesis
What is a hypothesis? How to build it?
& Good vs Bad one.

15. What is a hypothesis?
"If _____[I do this] _____, then _____[this]_____ will happen."
If I added gmail login, then number of registered users will increase.
If I include Cash on Delivery payment method during checkout, then number of purchases will increase.
If I enable customers to pin their location on a map, then number of purchases will increase.

16. Hypotheses Tips
Before you make a hypothesis, you have to clearly identify the question
you are interested in studying.
The question comes first
A hypothesis is a
statement, not a question
Your hypothesis is not the scientific question in your project. The
hypothesis is an educated, testable prediction about what will happen.
Make it clear A good hypothesis is written in clear and simple language.
Keep the variables in mind
A good hypothesis defines the variables in easy-to-measure terms, like
who the participants are, what changes during the testing.
Make sure your
hypothesis is “testable”
Don't bite off more than
you can chew!
To prove or disprove your hypothesis, you need to be able to do an
experiment and take measurements to see how two things are related.
Make sure your hypothesis is a specific statement relating to a single
experiment.

17. Good vs bad hypothesis
Good Hypothesis Bad Hypothesis
Testable
Simple
Written as a statement
Establishes the
participants & variables
Predicts effect
Not testable
Not simply explained
Written as a questions
Doesn’t identify
participants & variables
Cannot use to predict
effect

18. Good vs bad hypothesis
Good Hypothesis Bad Hypothesis
If I added gmail login, then number of
registered users will increase.
If I include Cash on Delivery payment
method during checkout, then number of
purchases will increase.
If I enable customers to pin their location
on a map, then number of purchases will
increase.
Would adding more login options increase
number of registered users?
Cash on delivery payment is requested by
users
We have 10,000 calls from users to track
their orders

19. The point is to prove or disprove a hypothesis
Disproving a hypothesis matters as much as proving it. Both leads to form better
conclusions about the problem at hand!

20. Sources for trends or problems

21. 1- Observations
2- Hypothesis
3- Experiment
4- Refine
5- Validate
The Approach: Step by Step Process
Questions to answer. Observing a trend or a problem.
What is a hypothesis? How to build one?
& Good vs Bad hypothesis.
How to test hypothesis? What data to look at?
Refine hypothesis
How to measure the effect? Success metrics!

22. Experiment
How to test hypothesis? What data to look at?

23. Creating a mathematical model
What questions do you want to
answer?
What data do you want to look
for?
Where the data you want is
available?
How to use the data available?
How to verify the data?
Will adding Gmail login improve sign up?
How many customers have signed up
using Gmail email?
Customers data base
Find % of customers signed up with Gmail.
Gmail customers / all customers = % of gmail customers.
Consider removing dummy created accounts

24. Creating a mathematical model
What questions do you want to
answer?
What data do you want to look
for?
Where the data you want is
available?
How to use the data available?
How to verify the data?
Will adding Gmail login improve sign up?
How many customers have signed up
using Gmail email?
Customers data base
Find % of customers signed up with Gmail.
Gmail customers / all customers = % of
gmail customers.
Consider removing dummy created
accounts
150K
150K / 1.3M = 12%

25. 1- Observations
2- Hypothesis
3- Experiment
4- Refine
5- Validate
The Approach: Step by Step Process
Questions to answer. Observing a trend or a problem.
What is a hypothesis? How to build one?
& Good vs Bad hypothesis.
How to test hypothesis? What data to look at?
Refine hypothesis
How to measure the effect? Success metrics!

26. Refine
Refine mathematical model, removing bias!

27. Will adding Gmail login improve sign up?
How many customers have signed up
using Gmail email?
Find % of customers signed up with Gmail.
Gmail customers / all customers = % of
gmail customers.
Consider removing dummy created
accounts
150K
150K / 1.3M = 12%
What questions do you want to
answer?
What data do you want to look
for?
Where the data you want is
available?
How to use the data available?
How to verify the data?
Creating a mathematical model
Customers data base

28. We live in a dynamic world, so always consider things
in a timely manner!

29. Creating a mathematical model
What questions do you want to
answer?
What data do you want to look
for?
Where the data you want is
available?
How to use the data available?
How to verify the data?
Will adding Gmail login improve sign up?
How many customers have signed up
using Gmail email in the past 6 months?
Customers data base
Find % of customers signed up with Gmail
in the past 6 months.
Gmail sign ups / all sign ups in the past 6
months = % of gmail customers.
Consider removing dummy created
accounts
120K
120K / 600K = 25%

30. 1- Observations
2- Hypothesis
3- Experiment
4- Refine
5- Validate
The Approach: Step by Step Process
Questions to answer. Observing a trend or a problem.
What is a hypothesis? How to build one?
& Good vs Bad hypothesis.
How to test hypothesis? What data to look at?
Refine hypothesis based on test.
How to measure the effect? Success metrics!

31. Validate
How to measure the effect? Success metrics!

32. What is a success metric?
Metrics are a lens into your product’s
health and performance.

33. Success Metric

34. Measure the uplift

35. 1- Observations
2- Hypothesis
3- Experiment
4- Refine
5- Validate
The Approach: Step by Step Process
Questions to answer. Observing a trend or a problem.
What is a hypothesis? How to build one?
& Good vs Bad hypothesis.
How to test hypothesis? What data to look at?
Refine hypothesis.
How to measure the effect? Success metrics!

36. Thank you
Abdallah Al-Khalidi
Linkedin: abzkhaldi | Twitter: @abzkhaldi
Email: abdallah.khalidi@gmail.com