Queues introduce delays and economic costs in product development. Common queues include features waiting to be designed/developed/tested/released and bugs waiting for triage/fixing/verification. Large queues increase cycle times, variability, and require management. They also delay feedback and hurt morale. The presenter advocates reducing queues by using swarming, generalizing work across teams rather than specializing, obliterating unnecessary queues, and reducing testing batch sizes from months to weeks or even days through continuous delivery. Case studies show how these techniques helped the presenter's company ship features more quickly while maintaining quality.
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
An Invisible Money Drain: How Queues Waste Resources and Delay Value
1. Queues: An Invisible
Money Drain
Phil Sarin
VP Engineering, GameChanger Media
@philsarin
http://216ways.net
2. “It ain’t what you don’t know that gets you into
trouble. It’s what you know for sure that just ain’t so.
— Mark Twain
3. Things I once believed
You shouldn’t release often because releases have high overhead.
You need to track all of your bugs.
Slack is bad. Your team should be busy.
Specialists will always be more productive than generalists.
7. “Queues are the root
cause of the majority of
economic waste in product
development.”
— Donald G. Reinertsen
8. Examples of queues
Feature Queues
Waiting to get designed
Waiting to get developed
Waiting for testing
Waiting to get released
Waiting to get validated
Bug Queues
Waiting for triage
Waiting to get fixed
Waiting to get verified
Waiting for release
9. What’s wrong with big queues?
Big queues increase cycle time and delay cost
Big queues increase variability
Big queues require management
Big queues delay feedback
Big queues hurt morale
17. Why small batches are (usually) better
Realize benefits sooner
Faster feedback
Higher motivation
Less schedule slippage
18. Old School: Releases are expensive so let’s batch up work for
release.
New School: Let’s drive down the cost of releases so that we can
release more often.
19. Queue takeaways
Queues introduce delay.
Queues have economic and psychological costs.
High capacity utilization drives up queue size.
Small batches can reduce the cost of queues.
20. Part 2: How we apply this
theory at GameChanger
Flickr: Jeff Turner
30. Multiple queues,
1 server each
Single queue,
Single server
Single queue,
pool of servers
Sustained huge queues
are least likely
31. Pressures to specialize
A specialty differentiates your business
“I don’t want to touch that code!”
Initial builder owns it forever.
Early adopters become de-facto specialists.
Some people want specialist careers.
37. 2 months development
2 weeks
testing
2 months development
2 weeks
testing
4 months bug-driven development
38. 2 weeks
dev
test
2 weeks
dev
test
2 weeks
dev
test
2 weeks
dev
test
2 weeks
dev
test
Can we reduce this batch size further?
Our first batch size reduction
48. Things I once believed
You shouldn’t release often because releases have high overhead.
You need to track all of your bugs.
Slack is bad. Your team should be busy.
Specialists will always be more productive than generalists.
50. Takeaways
Queues are real and expensive
You should know where your queues are
There are a bunch of ways to reduce the cost of queues
51. Further Reading
The Principles of Product Development Flow, Donald G. Reinertsen
“Software Inventory” by Joel Spolsky: http://www.joelonsoftware.com/
items/2012/07/09.html
“How we fixed more bugs by deleting our bug DB” and “Building
around generalists” on my blog (216ways.net)