SlideShare ist ein Scribd-Unternehmen logo
1 von 20
Downloaden Sie, um offline zu lesen
Customer Behaviour Modelling
Insights from customer data
Redbubble
Anuj Luthra
Product Development
● Millions of users
● Lots of ideas
● Lots of unquantified & unvalidated assumptions
● What are the Biggest problems
● What should we pursue first = best opportunity
● We want to build the right thing
Existing Techniques
● User interviews and surveys
○ Interpretation of wants and needs is tricky
■ Not dependable
○ Expensive & time consuming
● Analytic tools (GoogleAnalytics, Flurry) provide high
level views
○ difficult to gauge effect of each variable on its own :
Lots of factors at play, how much did a singular thing
affect the outcome
What is lacking
● Ability to get insights from real user actions/visits
● Make it Quick and Cheap to support/reject assumptions
● Confidence, like probabilities, external factors and stuff :
-)
http://explosm.net/comics/2964/
What we do
● Statistical modelling of customer data and infer
● Quantification of relative impact of the user behaviours
and visit attributes
“Lets put some science in data analysis”
● Give a starting point
● Define the goal for measuring success
● Keeps you focussed and honest
● Hunches are powerful - use domain knowledge
Strongest Hypotheses
Identify hypotheses
○ HypothesisA: “Users jumping along & looking at
multiple search result pages are having a bad
experience”
○ HypothesisB: “Users navigating to a listing from
search results are having a good experience”
○ HypothesisC: “Users typing in keywords in search
box multiple times are not having a good
experience”
Measurable User Journeys
● Identify particular user journeys in a visit
○ hypothesisA: SPPPSPSP
○ hypothesisB: SLL
● Journeys don’t need to be exclusive - they are not!
● Lots of log parsing, mapreduce
● Usually the process varies for each business
Data Preparation
● Start with a small sample size
● Focus more on quality
● Look out for anomalies & outliers
● Remove correlated variables - noise
Data Visualization
● Visualize your data
○ Simple Histogram will tell you a lot of things
○ Scatter plots are good for identifying outliers
Regression analysis
● Statistical process for estimating the relationships
among variables
● Choice of method largely depends of the form of data
and variable types
● Linear regression is your go-to method for initial pokes
● Poisson or logit model are also very useful tools for
most ecommerce related datasets
Example (Using R)
Independent Variables Estimate Std. Error z value Pr(>z) Significance
clickThroughToListings 0.34065 0.12654 2.692 0.00710
**
pagingAroundSearchResults -0.28925 0.08688 -3.329 0.00087
***
usingSearchBoxTooMuch 0.12038 0.12608 0.955 0.33967
glm(
formula = addToCart ~ clickThroughToListings +
pagingAroundSearchResults +
usingSearchBoxTooMuch,
family = "binomial",
data = summary.df
)
Independent Variables Estimate Std. Error z value Pr(>z) Significance
clickThroughToListings 0.34065 0.12654 2.692 0.00710
**
pagingAroundSearchResults -0.28925 0.08688 -3.329 0.00087
***
usingSearchBoxTooMuch 0.12038 0.12608 0.955 0.33967
How to interpret signal
Direction
How to interpret signal
Independent Variables Estimate Std. Error z value Pr(>z) Significance
clickThroughToListings 0.34065 0.12654 2.692 0.00710
**
pagingAroundSearchResults -0.28925 0.08688 -3.329 0.00087
***
usingSearchBoxTooMuch 0.12038 0.12608 0.955 0.33967
Significance
Concrete Direction
● Now we know which user segments present a real
opportunity to make improvements
● How big is the customer segment = problem size
● Knowing problem size helps in prioritizing
Summary
Methodology:
1. Gather Strongest Hypotheses
2. Construct Measurable User Journeys
3. Choose & Apply statistical methods
4. Support/reject hypothesis
5. Repeat-Refine
Toolkit
● BigQuery: Query parsing, mapreduce
● R: Data visualization, cleaning, augmentation, statistical
methods
● Ruby: Scripting
● Coffee: ‘Cos
Found it interesting?
Come and talk to us

Weitere ähnliche Inhalte

Kürzlich hochgeladen

Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...Amil baba
 
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfsdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfJulia Kaye
 
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
Engineering Mechanics  Chapter 5  Equilibrium of a Rigid BodyEngineering Mechanics  Chapter 5  Equilibrium of a Rigid Body
Engineering Mechanics Chapter 5 Equilibrium of a Rigid BodyAhmadHajasad2
 
The relationship between iot and communication technology
The relationship between iot and communication technologyThe relationship between iot and communication technology
The relationship between iot and communication technologyabdulkadirmukarram03
 
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS Bahzad5
 
Nodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptxNodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptxwendy cai
 
Landsman converter for power factor improvement
Landsman converter for power factor improvementLandsman converter for power factor improvement
Landsman converter for power factor improvementVijayMuni2
 
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdfSummer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdfNaveenVerma126
 
Modelling Guide for Timber Structures - FPInnovations
Modelling Guide for Timber Structures - FPInnovationsModelling Guide for Timber Structures - FPInnovations
Modelling Guide for Timber Structures - FPInnovationsYusuf Yıldız
 
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...Amil baba
 
solar wireless electric vechicle charging system
solar wireless electric vechicle charging systemsolar wireless electric vechicle charging system
solar wireless electric vechicle charging systemgokuldongala
 
Design of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptxDesign of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptxYogeshKumarKJMIT
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical SensorTanvir Moin
 
Clutches and brkesSelect any 3 position random motion out of real world and d...
Clutches and brkesSelect any 3 position random motion out of real world and d...Clutches and brkesSelect any 3 position random motion out of real world and d...
Clutches and brkesSelect any 3 position random motion out of real world and d...sahb78428
 
Power System electrical and electronics .pptx
Power System electrical and electronics .pptxPower System electrical and electronics .pptx
Power System electrical and electronics .pptxMUKULKUMAR210
 
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxUNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxrealme6igamerr
 
nvidia AI-gtc 2024 partial slide deck.pptx
nvidia AI-gtc 2024 partial slide deck.pptxnvidia AI-gtc 2024 partial slide deck.pptx
nvidia AI-gtc 2024 partial slide deck.pptxjasonsedano2
 
Guardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSecGuardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSecTrupti Shiralkar, CISSP
 

Kürzlich hochgeladen (20)

計劃趕得上變化
計劃趕得上變化計劃趕得上變化
計劃趕得上變化
 
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
 
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfsdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
 
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
Engineering Mechanics  Chapter 5  Equilibrium of a Rigid BodyEngineering Mechanics  Chapter 5  Equilibrium of a Rigid Body
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
 
The relationship between iot and communication technology
The relationship between iot and communication technologyThe relationship between iot and communication technology
The relationship between iot and communication technology
 
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
 
Nodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptxNodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptx
 
Landsman converter for power factor improvement
Landsman converter for power factor improvementLandsman converter for power factor improvement
Landsman converter for power factor improvement
 
Lecture 4 .pdf
Lecture 4                              .pdfLecture 4                              .pdf
Lecture 4 .pdf
 
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdfSummer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
 
Modelling Guide for Timber Structures - FPInnovations
Modelling Guide for Timber Structures - FPInnovationsModelling Guide for Timber Structures - FPInnovations
Modelling Guide for Timber Structures - FPInnovations
 
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
 
solar wireless electric vechicle charging system
solar wireless electric vechicle charging systemsolar wireless electric vechicle charging system
solar wireless electric vechicle charging system
 
Design of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptxDesign of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptx
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical Sensor
 
Clutches and brkesSelect any 3 position random motion out of real world and d...
Clutches and brkesSelect any 3 position random motion out of real world and d...Clutches and brkesSelect any 3 position random motion out of real world and d...
Clutches and brkesSelect any 3 position random motion out of real world and d...
 
Power System electrical and electronics .pptx
Power System electrical and electronics .pptxPower System electrical and electronics .pptx
Power System electrical and electronics .pptx
 
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxUNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
 
nvidia AI-gtc 2024 partial slide deck.pptx
nvidia AI-gtc 2024 partial slide deck.pptxnvidia AI-gtc 2024 partial slide deck.pptx
nvidia AI-gtc 2024 partial slide deck.pptx
 
Guardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSecGuardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSec
 

Customer behaviour modelling - tech presentation

  • 3. Product Development ● Millions of users ● Lots of ideas ● Lots of unquantified & unvalidated assumptions ● What are the Biggest problems ● What should we pursue first = best opportunity ● We want to build the right thing
  • 4. Existing Techniques ● User interviews and surveys ○ Interpretation of wants and needs is tricky ■ Not dependable ○ Expensive & time consuming ● Analytic tools (GoogleAnalytics, Flurry) provide high level views ○ difficult to gauge effect of each variable on its own : Lots of factors at play, how much did a singular thing affect the outcome
  • 5. What is lacking ● Ability to get insights from real user actions/visits ● Make it Quick and Cheap to support/reject assumptions ● Confidence, like probabilities, external factors and stuff : -)
  • 7. What we do ● Statistical modelling of customer data and infer ● Quantification of relative impact of the user behaviours and visit attributes “Lets put some science in data analysis”
  • 8. ● Give a starting point ● Define the goal for measuring success ● Keeps you focussed and honest ● Hunches are powerful - use domain knowledge Strongest Hypotheses
  • 9. Identify hypotheses ○ HypothesisA: “Users jumping along & looking at multiple search result pages are having a bad experience” ○ HypothesisB: “Users navigating to a listing from search results are having a good experience” ○ HypothesisC: “Users typing in keywords in search box multiple times are not having a good experience”
  • 10. Measurable User Journeys ● Identify particular user journeys in a visit ○ hypothesisA: SPPPSPSP ○ hypothesisB: SLL ● Journeys don’t need to be exclusive - they are not! ● Lots of log parsing, mapreduce ● Usually the process varies for each business
  • 11. Data Preparation ● Start with a small sample size ● Focus more on quality ● Look out for anomalies & outliers ● Remove correlated variables - noise
  • 12. Data Visualization ● Visualize your data ○ Simple Histogram will tell you a lot of things ○ Scatter plots are good for identifying outliers
  • 13. Regression analysis ● Statistical process for estimating the relationships among variables ● Choice of method largely depends of the form of data and variable types ● Linear regression is your go-to method for initial pokes ● Poisson or logit model are also very useful tools for most ecommerce related datasets
  • 14. Example (Using R) Independent Variables Estimate Std. Error z value Pr(>z) Significance clickThroughToListings 0.34065 0.12654 2.692 0.00710 ** pagingAroundSearchResults -0.28925 0.08688 -3.329 0.00087 *** usingSearchBoxTooMuch 0.12038 0.12608 0.955 0.33967 glm( formula = addToCart ~ clickThroughToListings + pagingAroundSearchResults + usingSearchBoxTooMuch, family = "binomial", data = summary.df )
  • 15. Independent Variables Estimate Std. Error z value Pr(>z) Significance clickThroughToListings 0.34065 0.12654 2.692 0.00710 ** pagingAroundSearchResults -0.28925 0.08688 -3.329 0.00087 *** usingSearchBoxTooMuch 0.12038 0.12608 0.955 0.33967 How to interpret signal Direction
  • 16. How to interpret signal Independent Variables Estimate Std. Error z value Pr(>z) Significance clickThroughToListings 0.34065 0.12654 2.692 0.00710 ** pagingAroundSearchResults -0.28925 0.08688 -3.329 0.00087 *** usingSearchBoxTooMuch 0.12038 0.12608 0.955 0.33967 Significance
  • 17. Concrete Direction ● Now we know which user segments present a real opportunity to make improvements ● How big is the customer segment = problem size ● Knowing problem size helps in prioritizing
  • 18. Summary Methodology: 1. Gather Strongest Hypotheses 2. Construct Measurable User Journeys 3. Choose & Apply statistical methods 4. Support/reject hypothesis 5. Repeat-Refine
  • 19. Toolkit ● BigQuery: Query parsing, mapreduce ● R: Data visualization, cleaning, augmentation, statistical methods ● Ruby: Scripting ● Coffee: ‘Cos
  • 20. Found it interesting? Come and talk to us