This document discusses using metrics to understand and improve processes. It introduces cycle time and throughput as key metrics, and shows how visualizing data through charts like a cycle time scatter plot can help analyze relationships and tell stories. Probability is discussed as a way to set expectations and manage uncertainty. Leading indicators like cumulative flow diagrams are presented as a way to predict and improve processes by understanding work in progress and flow.

1. Using Metrics
as a Map
@catswetel
cat.swetel@praxisflow.com
7 Oct 2016 12:20-12:50pm in Pentland West
#lascot16

2. “Cat Swetel is one of
the few A.I.s to have
passed both the Turing
and Bechdel tests.”
-- Will Evans
“Cat Swetel is one of the
most effective and
capable producers of
carbon dioxide in a North
America. Possibly the
world.”
-- Jeff Kosciejew
Who is Cat?
“Cat Swetel does
fine at some stuff.”
-- Steve

3. AGENDA
Some awesome
lagging indicators
Using different views to tell
different stories or different
versions of the same story
Leading indicators????
Relationships
@CATSWETEL

4. Popular
Agile
metrics
pic: @codinghorror
read the replies: bit.ly/10xfingers

5. “Everything’s made up
and the points don’t
matter.”
-Drew Carey
Assumptions:
● Our customers care about time
○ Calendar days
○ Global markets and teams
● Our customers care about value
● We have data (per unit of value)
○ Start date
○ End date
○ Date delivered
● The problem is usually (almost
always) the system, not the
people

6. “Everything’s made up
and the points don’t
matter.”
-Drew Carey
What’s the value of
metrics?
● Tell stories, build
shared context
● Make decisions in
context

7. Cycle Time Scatter Plot
@CATSWETEL

8. Cycle Time
Units of time per unit of value
e.g.
this story took 5 days
this ticket took 8 hours
this feature took 2 weeks
End Date - Start Date = Cycle Time
Where does it begin and end?
@CATSWETEL

9. Cycle Time Scatter Plot
@CATSWETEL

10. “Never triangle-face!
I hate triangle face!
It scares me!”
--Danny McBride in Your Highness

11. @CATSWETEL
Run chart?
Control chart?
Cycle Time Scatter Plot

12. Cycle time? Is there any old way?
Probably at project level or maybe in ops
there will be SLAs. This is a way under
utilized metric. IMO
Frequency
Cycle Time Distribution
@CATSWETEL

13. Weibull Curve
You can still buy Weebles!
amzn.to/1WCxdze
@CATSWETEL

14. Weibull Curve
Great example from Eli
Goldratt’s Critical Chain:
How long does it take you
to get to work?
@CATSWETEL

15. 85% Probability
Cycle Time Scatter Plot
with probability
@CATSWETEL

16. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
WTF?!
@CATSWETEL
Cycle Time Scatter Plot

17. Cycle time? Is there any old way?
Probably at project level or maybe in ops
there will be SLAs. This is a way under
utilized metric. IMO
Frequency
Cycle Time Distribution
@CATSWETEL

18. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
WTF?!
@CATSWETEL
Cycle Time Scatter Plot

19. Cycle time? Is there any old way?
Probably at project level or maybe in ops
there will be SLAs. This is a way under
utilized metric. IMO
Frequency
Cycle Time Distribution
@CATSWETEL

20. But what if it doesn’t look like that? What if
it looks like a camel?
1 2 3 4 5 6 7 8 9 10 11
Multimodal
@CATSWETEL

21. 1 2 3 4 5 6 7 8 9 10 11
Multimodal
Why?
● Different work item types
● Different work flows
● External dependency
● …etc
@CATSWETEL

22. 1 2 3 4 5 6 7 8 9 10 11
Multimodal
Beware the blob.
@CATSWETEL

23. @CATSWETEL
Cycle Time Scatter Plot
85% Probability

24. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
Cycle Time Scatter Plot
probability??
95%
probability
50%
probability
@CATSWETEL

25. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
Cycle Time Scatter Plot
with throughput, max and min
@CATSWETEL

26. Throughput
Units of value per unit of time
e.g.
stories per sprint
tickets per day
features per month
@CATSWETEL

27. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
WTF?!
@CATSWETEL
Cycle Time Scatter Plot

28. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
WTF?!
@CATSWETEL
Cycle Time Scatter Plot
????????????

29. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
9 9
11
6
4
6 4
10
13
Cycle Time Scatter Plot
throughput
@CATSWETEL

30. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
9 9
11
6
4
6 4
10
13
Cycle Time Scatter Plot
throughput
@CATSWETEL

31. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
9 9
11
6
4
6 4
10
13
Cycle Time Scatter Plot
throughput + probability
95%
probability
50%
probability
@CATSWETEL

32. Cycle Time Scatter Plot
95%
probability
50%
probability
30
16
Good for managing customer expectations
Good for CCPM
@CATSWETEL

33. Cycle Time Scatter Plot
95%
probability
50%
probability
30
16
@CATSWETEL
“What’s the likelihood we’ll hit this date?”
“How quickly can you get this to me?”

34. Cycle Time Scatter Plot
95%
probability
50%
probability
30
16
What is the cost of more certainty?
@CATSWETEL

35. Foreshadowing....
What is the cost of more certainty?

36. LAGGING vs LEADING
INDICATORS
@CATSWETEL

37. Lagging Indicators
@CATSWETEL

38. Predictive metrics!
Is there such a thing common in agile teams today?
Ummmm not really
DONE
REVIEW
DOING
TO DO
TUE WED THU FRI MON TUE WED
@CATSWETEL
Leading Indicator: Cumulative Flow Diagram

39. @CATSWETEL
Leading Indicator:
Cumulative Flow Diagram

40. @CATSWETEL
Leading Indicator:
Cumulative Flow Diagram
?

41. Little’s Law
average # of items in
a system
average
arrival rate
average time spent
in the system= *
average time spent
in the system =
average # of items in
a system /
average
throughput
@CATSWETEL

42. Little’s Law
average # of items in
a system
average
arrival rate
average time spent
in the system= *
average time spent
in the system =
average # of items in
a system /
average
throughput
@CATSWETEL

43. Little’s Law
average # of items in
a system
average
arrival rate
average time spent
in the system= *
average time spent
in the system =
average # of items in
a system /
average
throughput
@CATSWETEL
“Every time you violate an assumption of Little’s Law
your process becomes less predictable.”
--Dan Vacanti in Actionable Agile Metrics

44. Data is all about
stories and
relationships
@CATSWETEL

45. The (useful) Agile metrics you might not
know:
Cycle time, throughput, WIP
Know the bounds (time and space)
Different views change and often enrich
the story
The value of metrics is in the
relationships

46. Using Metrics
as a Map
@catswetel
cat.swetel@praxisflow.com