Weitere ähnliche Inhalte Ähnlich wie [mobiconf 2014] Shazam mobile apps - Data Driven Project Management (20) Kürzlich hochgeladen (20) [mobiconf 2014] Shazam mobile apps - Data Driven Project Management2. © 2014 Shazam
Entertainment
About Shazam
music recognition service and apps
founded in 1999
launched in the UK in 2002
one of the most popular mobile apps
500M+ downloads
3. © 2014 Shazam
Entertainment
Shazam technology - then
1. call 2580
2. play the music
3. wait for SMS with results
(phone you most likely used in 2002)
4. © 2014 Shazam
Entertainment
Shazam technology - now
since 2008 - iPhone and Android apps
since 2011 - integrated with Spotify
since 2014 - integrated with iOS8’s Siri
music streaming, TV ads, live TV,
and more
5. Shazam is now a part of popular culture...
© 2014 Shazam
Entertainment
8. Agenda
organisation
embrace the change
spiral development
why and what do we measure
estimation (and causation bias)
handling dependencies
11. © 2014 Shazam
Entertainment
Technical Project Management
the scientific method
hypothesis experiment
analyse
data
draw
conclusions
improve continuously
(Kaizen)
13. © 2014 Shazam
Entertainment
We use Kanban
visualise workflow
limit Work In Progress
(WIP)
14. © 2014 Shazam
Entertainment
We use Kanban
visualise workflow
limit Work In Progress
manage the flow
15. © 2014 Shazam
Entertainment
We use Kanban
visualise workflow
limit Work In Progress
manage the flow
define explicit policies
16. © 2014 Shazam
Entertainment
We use Kanban
visualise workflow
limit Work In Progress
manage the flow
define explicit policies
improve collaboratively
17. © 2014 Shazam
Entertainment
Why Kanban
Our work is complex, requirements and circumstances change.
(fixed scope iterations won’t work)
We can be flexible with release cadence.
(we release when we’re ready)
Visibility and metrics.
(value stream mapped on the board)
18. © 2014 Shazam
Entertainment
This talk is not about theory
why limit work in progress?
why focus on finishing, not starting?
why slack is good for efficiency, ‘busyness’ is not?
why early feedback loops are necessary?
effectiveness vs efficiency?
20. © 2014 Shazam
Entertainment
What do we care about
100+
Users. We now have millions monthly active users.
21. © 2014 Shazam
Entertainment
Types of work
new functionalities commercial deals
deadlines
(quality still matters)
defined scope
(flexible interpretation)
no deadlines
(quality most important)
flexible scope
(UX most important)
22. © 2014 Shazam
Entertainment
Types of work
new functionalities commercial deals
requirements change rapidly
in both cases
23. © 2014 Shazam
Entertainment
Hack time (3rd type of work)
some time can be used to work on any project
Shazam for Mac was
one of them
27. © 2014 Shazam
Entertainment
Real Options model
● options have value
(you’ll have more information/make better decision later)
● options expire
(ability to execute option ceases to exist at some point)
● never commit early unless you know why
(sometimes no decision is the best decision)
28. © 2014 Shazam
Entertainment
Real Options in practice
● no long queues
● designs and specs refined as we go
● A/B testing
● shorter release cycles
● irreversible commitments
29. © 2014 Shazam
Entertainment
Real Options - example
early commitment
backing out of early commitment
backlog too long
late commitment
31. © 2014 Shazam
Entertainment
Ceremonies
ceremonies limited to a minimum
meetings often ad hoc, here and now
● morning standups
● retrospectives
● visual reviews
● bug triage meetings
● detailed planning
● estimation
32. © 2014 Shazam
Entertainment
Iterative/spiral development
1. update requirements
2. implement
3. visually review new
functionalities
(usually work in progress)
4. collect feedback
5. repeat until signed off
33. © 2014 Shazam
Entertainment
Visual reviews
in the breakout area for everyone to join
34. © 2014 Shazam
Entertainment
Visual reviews
in the breakout Terminal 6 area for everyone to join
36. © 2014 Shazam
Entertainment
T6 - taking the metaphor further
flight board
shows status of epics
all teams included
we track more than this...
38. © 2014 Shazam
Entertainment
Data gathering
"In God we trust; all others must bring data."
Edward Deming
"What's measured improves."
Peter F. Drucker
39. © 2014 Shazam
Entertainment
Data gathering
information radiators across
the office
42. © 2014 Shazam
Entertainment
Estimation scale we use
1
TFB - too f* big
NFC - no f* clue
[give me a minute]
cards in the photo by
Lunar Logic (@Lunar_Logic)
43. FAQ 1
As a Product Manager, when can I expect this Story
to be completed?
44. © 2014 Shazam
Entertainment
Disney/Cycle Times
time for a Story to get
implemented and tested
(or time from entering queue until
finishing the ride in Disneyland)
having lots of samples we
can use statistics
(and talk about probabilities)
45. © 2014 Shazam
Entertainment
Disney/Cycle Times
parallel
streams of
work
Cycle Times
t
work done
start of test
start of development
46. © 2014 Shazam
Entertainment
Disney/Cycle Times - statistics
histogram of Disney
Times (Cycle Times) density function
50th, 85th percentile
(50/85% items get finished within this time)
47. © 2014 Shazam
Entertainment
When a Story will get done?
1. You can get this story within X days with 85%
probability.
2. Probability of you getting it within a week is Y%.
48. © 2014 Shazam
Entertainment
Scientific method in action
we track median (50th percentile) Cycle Times:
tracking metrics in time allows us to use scientific method
49. © 2014 Shazam
Entertainment
Hypothesis
limiting Work In Progress will shorten Cycle Times
according to Little’s Law:
avg Cycle Time =
avg Work In Progress
avg Throughput
50. © 2014 Shazam
Entertainment
Correlation
Work In Progress limit for
Development lowered
correlated results
next day
51. Does eating ice cream make you more
© 2014 Shazam
Entertainment
prone to drowning?
52. © 2014 Shazam
Entertainment
Causation?
we had a glitch in the system, but over the next
month results matched our hypothesis, and we
could replicate the experiment
53. FAQ 2
Ok, so when can we get all these 10 stories done?
54. © 2014 Shazam
Entertainment
Modelling parallel work
parallel work is difficult to model with cycle times
it’s easier with Takt Times
Takt Times
parallel
streams of
work
Cycle Times
t
55. © 2014 Shazam
Entertainment
Distribution of average Takt Times
create set of average Takt Time samples based on similar project
average
resample and
Σmany times
(>1000)
average Takt distribution of average Takt Times
Times
Takt Times
56. Predicting delivery time for N items
© 2014 Shazam
Entertainment
average TT
* N
repeat many
times (>1000)
average TT
average TT
average TT
average expected delivery
distribution of expected delivery times
Takt Times
times
85% probability to deliver within this time
57. © 2014 Shazam
Entertainment
When will all the Stories get done?
1. You can get these Stories within X days with 85%
probability.
2. Probability of you getting them within a week is Y%.
58. FAQ 3
Why did we end up with large number of stories to
test at once?
Do some types of items take longer to develop than others?
Are there times when teams are more productive?
Did we split Stories properly or we missed splitting some big
ones?
59. © 2014 Shazam
Entertainment
Patterns and improvement
Being aware of patterns enables us to act on them:
● introduce better policies
(e.g. all UI assets for a Story done before dev commences)
● change development cycle
(e.g. release more often to prevent batching)
● treat some types of work differently
(e.g. allow extra dev time)
66. © 2014 Shazam
Entertainment
Blockers/dependencies
● visualise
● prioritise
(what is blocked, for how long)
● act
● collect statistics and review them
(have any recent changes to the process change amount
of blocked items/how long they’re blocked for?)
67. © 2014 Shazam
Entertainment
What should I do
first? - answers
Team A: finally doing well
Team B: keeps getting
blocked
4 blockers including 1 old
Team C: doing well
Team D: 1 new blocker
68. Cumulative Density
Function
(80% blocked items
unblocked in ~3 days)
© 2014 Shazam
Entertainment
Blockers statistics
trend of increasing
median time to
unblock
69. Summary
● use Kanban for flexibility, continuous improvement
● embrace change, use Real Options rules
● daily visual reviews allow fast feedback cycle
● make decisions based on data
● make estimates based on statistical analysis
● handle dependencies and blockers
71. © 2014 Shazam
Entertainment
References
● Real Options: commitment-thebook.com, Olav Maassen, Chris Matts
● Real Options: http://www.infoq.com/articles/real-options-enhance-agility
● Real Options: http://decision-coach.com/lean-and-real-options/
● “High level planning using Monte Carlo simulation”, Dimitar Bakardzhiev
● “Thinking, fast and slow”, Daniel Kahneman
● http://en.wikipedia.org/wiki/Takt_time