Presented at the Habit Summit, this talk covers how to measure how engaged your users are.

1. Measuring For Engagement
1
John Egan
Lead Growth Engineer @ Pinterest

2. 2
New Users
MAUs
Dormant Users

3. 3
New Users
MAUs
Dormant Users

4. 4
New Users
Dormant Users
Core
Casual
Marginal

5. 5
New Users
MAUs
Dormant Users

6. 6
• Retail shopping rewards app
• Led Growth Engineering team
• Grew from 1MM to 8MM users in
3 years
• Acquired for $200MM
• Visual discovery & bookmarking
• Eng lead for R&R team
• Increased mobile WAUs by 20%
• Increased mobile signups by
10%

7. Activation
7

8. Retained MAUs
• Percentage of new signups that are still using the app a month later
• Figure out what a good retention number is for your industry
8

9. 1d7s
• Percentage of new signups that use the app 1 or more times in the week
following signup
• Quicker to run experiments with
• At Pinterest this is highly correlated with user’s long-term retention
9

10. Cohort Heatmap
10Source: Placeholder Numbers

11. Cohort Heatmap
11Source: Placeholder Numbers

12. Cohort Heatmap
12Source: Placeholder Numbers

13. Engagement
13

14. One Key Engagement Metric
Medium: Total Time Spent Reading (TTR)
Twitter: MAUs with 7+ visits a month (7d28s)
Uber: Weekly Trips
Facebook: WAUs with 6+ visits a week (6L7s)
Pinterest: Weekly Active Repiners or Clickers (WARCs)
14

15. Finding Your Key Engagement Metric
1) What are the actions a user has to take to get value from your product?
2) What is the frequency someone need to use your product
15

16. User States
Core: Active multiple times a week
Casual: Active once or twice a week
Marginal: Active a couple times a month
New: Joined in the past 28 days
Dormant: Not active for 28 days
Resurrected: Was dormant, but became active
again in the past 28 days
16

17. Badging
Holdout
Experiment
2:50 PM 100%
17

18. Measuring Effect on Engagement
18

19. *AU Ratios
19

20. 20
DAU MAU DAU/MAU ratio
Knighthood 30,014 389,583 7.70%
RockYou Pets 50,482 582,720 8.66%
Tetris Friends 100,404 841,223 11.94%
Superpoke Pets 299,505 2,296,175 13.04%
Mobsters 539,012 3,431,063 15.71%
Vampire Wars 687,067 3,794,869 18.11%
Country Story 1,647,763 8,097,356 20.35%
Fish World 2,084,358 8,581,587 24.29%
Pet Society 5,046,347 20,480,397 24.64%
Mafia Wars 6,473,504 25,431,622 25.45%
Restaurant City
5,033,468 17,250,419 29.18%
Cafe World 7,409,711 22,062,364 33.59%
Courtesy Social Times & Nabeel Hyatt

21. Retention &
Resurrection
21

22. Measuring Engagement for Emails/Push
• Positive measures of quality
- Must lift key engagement metric
- Minimum required CTR rates
• Negative measures of quality
- App deletions
- Spam reports
- Unsubscribes
22

23. Experiment Segmentation
23

24. All Users
24

25. Marginal Users (~1 month)
25

26. Core Users (multiple times a week)
26

27. Wrap Up
• 1d7s to experiment on activation
• Cohort heatmaps to track new user retention curve
• Define your key engagement metric
• Use user states to understand how engaged users are
• Make sure emails/notifications for true engagement
• Use segmentation for experiments & metrics in general
Confidential 27

28. John Egan
Growth Blog: jwegan.com
Email: jwegan@gmail.com
Twitter: @jwegan_com
Pinterest: pinterest.com/jwegan
28