The document summarizes the process taken by Duolingo to improve their Stories feature based on user research. It describes how Duolingo conducted interviews and usability testing with both active and lapsed users to identify key pain points. This revealed that Stories content did not connect well to courses, was difficult to use on phones, did not match users' ability levels, and lacked features for practicing or reviewing words. Duolingo prioritized addressing these issues through product and content changes, which led to improved Stories usage over time based on subsequent usage data. The main lessons highlighted are to invest in understanding users, focus on the most impactful insights, prioritize changes realistically, and ensure AB tests are grounded in strong product sense.

1. October 7, 2019
Lessons From A Cellist
(Or, what product people can learn from user research)

2. Conor Walsh (me!)
Senior PM @ Duolingo
The Problem
Our Process
Learnings
Agenda

3. The Problem

4. Stories Usage: Jan-Aug ‘18

5. Duolingo Stories

6. Bigger than DAUs
Challenges
- Declining usage
- Flat or declining D1, D7, D14 retention rates
- Legacy of past experiments
- Lack of direction
- Resource constraints
- Testing culture
- Management concerns and pressure

7. Our Process

8. The Plan of Attack
User interviews
Usability testing
Identify key pain points
Prioritize and execute

9. - Active + Churned
- All Courses
- Open-ended interviews
- Validate
User Interviews

10. Let your users teach you!

11. “Duolingo is like practicing scales;
Stories is like sight reading”
Insight:
- Amy Domingues, Stories User

12. Pain Point Impact
Stories content doesn’t connect with a course High
Hard to use Stories on your phone High
Stories don’t match ability High
Users can’t practice the words they’ve learned Medium - High
Users don’t realize there are hints Medium
Users don’t realize how long a story is Medium
Stories not engaging Medium
Hard to revisit or review a story Medium
Users can’t participate in a story Medium

13. So many changes!
● Stories on mobile
● New onboarding
● Content gating
● Listen-only mode
● Review mode
● Course-aligned stories
● A1 and A2 stories
Product Changes Content Changes

15. Stories Usage: Aug ‘18 - Sep ‘19

16. Stories Usage: Jan-Aug ‘18

17. Stories Usage: Jan ‘18 - Sep ‘19

18. Stories Usage: Jan ‘18 - Sep ‘19

19. Learnings

20. The TL;DR
Invest in understanding your users
Understand your ah-ha moment
Prioritize ruthlessly and realistically
AB tests are only as good as your product sense

21. Questions?