2. A Little Bit Of History
Requirements
Functional Spec
Design
Implementation
Test
Deploy
23 July 2010 http://twitter.com/jdrumgoole 2
3. Why Waterfall Sucks
• Can’t tell good software by observation
• What we see changes what we want
• Users are rotten at articulating their desires
– Obsessed with How rather than What
• Domain knowledge car crash
• Requirements change over time
• Responsiveness
23 July 2010 http://twitter.com/jdrumgoole 3
4. Waterfall Works…
• When specs are detailed and unchanging
• E.g X400, TCP/IP stack, SMTP etc.
• Requirement to deliver all at once e.g CREST
• Minimal technical innovation required
• UI Free Deployments
• Project team uses appropriate process/tools
23 July 2010 http://twitter.com/jdrumgoole 4
5. Performance Potential
Mentor
Move
Support
Manage
23 July 2010 http://twitter.com/jdrumgoole 5
6. Software Sector
• Highly Motivated individuals
• Don’t need more process
• Need more mentoring
• .. So PRINCE2/Six Sigma etc. Won’t help
• Some things need lots of process
– Pharma, Big Oil, Agriculture
– Software development ain’t one of them
23 July 2010 http://twitter.com/jdrumgoole 6
7. Conway’s Law
Organisations which design systems … are
constrained to produce designs which are
copies of the communications structures of
these organisations
23 July 2010 http://twitter.com/jdrumgoole 7
8. Software Development
• Takes 15 minutes to get in the zone
• Interrupts reset the 15 minute timer
• Takes 6 months to build product value
• Takes 18 months to build market value
• Great developers don’t know how they do it
• 10000 hours of practice (Outliers)
23 July 2010 http://twitter.com/jdrumgoole 8
9. Agile Manifesto
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
http://agilemanifesto.org/
23 July 2010 http://twitter.com/jdrumgoole 9
10. What does this mean?
• Avoiding Software Engineering
• Mostly we don’t know what we are doing
• … so do less of it until we do know
• Lets look at some specific processes
Extreme Programming & SCRUM
23 July 2010 http://twitter.com/jdrumgoole 10
11. XP & SCRUM
• Loose Definitions
–XP is what the programmers do
–SCRUM is what the management do
23 July 2010 http://twitter.com/jdrumgoole 11
12. Extreme Programming
• On site Customer
• User Stories
• Simplest possible Design
• Pair Programming
• Short Iterations
• Test Driven Development
• Refactoring
• Continuous Integration
• Incremental Releases
• Visibility
23 July 2010 http://twitter.com/jdrumgoole 12
13. XP: On Site Customer
• Customer available to team
• Detailed domain knowledge
• Understanding of ROI
• Explains priorities from a business perspective
• Balances effort /reward
• Builds a relationship with team
23 July 2010 http://twitter.com/jdrumgoole 13
14. XP : User Stories
• Simple stories that detail end-user functions
• Written by the user
• Fit on a single card
• Focus on simple incremental improvements
• Prune the backlog regularily
• Priorities driven by customer demands
23 July 2010 http://twitter.com/jdrumgoole 14
15. XP: Simplest Possible Design
• Over Architecture
• You are probably building the wrong thing
• Linear list/flat file preferred to DB table
• Validate that the feature is desired
• Optimise performance as a refactoring
• Sometimes slow is good enough
23 July 2010 http://twitter.com/jdrumgoole 15
16. XP : Pair Programming
• Most difficult XP practice to adopt
• Not a master/slave, mentor/acolyte Role
• A sharing by peers
• Oversight at the coal face
• The minute you leave the code your
knowledge decays exponentially
• Moment of creation
• Eliminates “ownership issues”
23 July 2010 http://twitter.com/jdrumgoole 16
17. Short Iterations
• Don’t go dark
• Validate your assumptions
• Get customer feedback early and often
• Don’t do work on non-essential stuff
• Discover changes in priorities
• Making a wrong turn at the start
23 July 2010 http://twitter.com/jdrumgoole 17
18. Test Driven Development
• Write the tests first
• Tests fail initially
• Tests start to pass as code gets written
• Refactoring breaks tests and then passes
• Unit Tests (class level)
• Acceptances Tests (end user level)
• Use automation (xUnit, Hudson, Ant etc.)
• If it doesn’t have a test it doesn’t exist
• Design for automated test
23 July 2010 http://twitter.com/jdrumgoole 18
19. Refactoring
• Driven by user stories
• Performance demands a new sort
• Flat files cannot be restructured for new data
• Increase users from 10 to 1000
• Environment changes (Windows to Linux)
• Software Rot reduction
23 July 2010 http://twitter.com/jdrumgoole 19
20. Continuous Integration
• Single Code Repository
• Automate the Build
• Make The Build Self Testing
• Everyone commits to mainline everyday
• Every commit builds the system
• Keep the build fast
• Test in a production clone
• Make it easy to get a production copy
23 July 2010 http://twitter.com/jdrumgoole 20
21. Incremental Releases
• Get the smallest feature working
• Use tests to validate working system
• Make sure each release modifies a small part
of the system
• Understand all changes
• Focus on end-to-end functionality
23 July 2010 http://twitter.com/jdrumgoole 21
22. Visibility
• Broken tests detected and fixed
• Broken builds detected and fixed
• Clear stats on:
– No of builds
– No of tests
– No of deployments
– Development is Data Centric
23 July 2010 http://twitter.com/jdrumgoole 22
23. XP : Velocity
• An understanding of how many stories we can
complete
• Measured like the weather
• Feedback loop leads to consistent stories
23 July 2010 http://twitter.com/jdrumgoole 23
24. XP : Example Velocity
350
Example Velocity for SaaS Project
300
250
Tickets Closed
200
150
Total
100
50
0
Sprints (Two Weeks)
23 July 2010 http://twitter.com/jdrumgoole 24
25. XP: 80/20 Rule
120
Example Velocity for SaaS Project
Example Velocity for SaaS Project
100
80
Tickets Closed
60
KP
DR
40
20
0
Sprints (Two Weeks)
Sprints (Two Weeks)
23 July 2010 http://twitter.com/jdrumgoole 25
26. Velocity Errors
• Need to focus on consistent sprint tasks
• Tasks must be > 4 hrs
• Tasks must be < 16 hrs
• Break up or coalesce
• Product Backlog too coarse grained
• 80/20 rule goes for performance
23 July 2010 http://twitter.com/jdrumgoole 26
27. Exercise
• Write two User Stories for your software
23 July 2010 http://twitter.com/jdrumgoole 27
28. SCRUM
Scrum is an agile process that allows us to
focus on delivering the highest business
value in the shortest time
23 July 2010 http://twitter.com/jdrumgoole 28
29. General George Patton
A good plan, violently executed
now, is better than a perfect
plan next week
23 July 2010 http://twitter.com/jdrumgoole 29
30. Its Been Around a While
• Jeff Sutherland
– Initial scrums at Easel Corp in 1993
– IDX and 500+ people doing Scrum
– Scrum presented at OOPLSA 96
• Ken Schwaber
– Scrum presented at OOPSLA 96
– Author of three books on Scrum
• Mike Beedle
– Scrum patterns in PLOPD4
• Ken Schwaber and Mike Cohn
– Co-founded Scrum Alliance in 2002
23 July 2010 http://twitter.com/jdrumgoole 30
31. SCRUM Users
• Microsoft • Intuit
• Yahoo • Nielsen Media
• Google • First American Real Estate
• Electronic Arts • BMC Software
• Philips • John Deere
• Siemens • Lexis Nexis
• Nokia • Sabre
• IBM • Salesforce.com
• Capital One • Time Warner
• BBC • Oce
23 July 2010 http://twitter.com/jdrumgoole 31
32. Key Facets
• Self-organizing teams
• Product progresses in a series of two- to
four-week “sprints”
• Requirements are captured as items in a list
of “product backlog”
• No specific engineering practices prescribed
• Uses generative rules to create an agile
environment for delivering projects
• One of the “agile processes”
23 July 2010 http://twitter.com/jdrumgoole 32
34. The Sprint – Coin of the Realm
• A release is broken down into sprints
• Sprints are of fixed identical duration
– Helps manage velocity
– Helps keep a rhythm
• Team decides sprint duration
• No changes during a sprint
23 July 2010 http://twitter.com/jdrumgoole 34
36. Roles: Product Owner
• Define the features of the product
• Decide on release date and content
• Be responsible for the profitability of the
product (ROI)
• Prioritize features according to market value
• Adjust features and priority every
iteration, as needed
• Accept or reject work results
23 July 2010 http://twitter.com/jdrumgoole 36
37. Roles : Scrum Master
• Represents management to the project
• Responsible for enacting Scrum values and
practices
• Removes impediments
• Ensure that the team is fully functional and
productive
• Enable close cooperation across all roles and
functions
• Shield the team from external interferences
23 July 2010 http://twitter.com/jdrumgoole 37
38. Roles: The Team
• Typically 5-9 people
• Cross-functional:
– Programmers, testers, user experience
designers, etc.
• Members should be full-time
– May be exceptions (e.g., database administrator)
• Teams are self-organizing
– Ideally, no titles but rarely a possibility
• Membership should change only between
sprints
23 July 2010 http://twitter.com/jdrumgoole 38
39. Process : Sprint Planning
• The Product Backlog
23 July 2010 http://twitter.com/jdrumgoole 39
40. Processes : Sprint Planning
• Sprint Prioritization
– Analyse and evaluate sprint backlog
– Takes account of changing business priorities
– Define Sprint Goal
– Feed into Sprint Plan
23 July 2010 http://twitter.com/jdrumgoole 40
41. Processes : Sprint Plan
• Decide how to achieve sprint goal
• Create sprint backlog from product backlog
– A list of product backlog tasks aligned with goal
• Time estimates for sprint backlog
• Sprint backlog is the Sprint Plan
23 July 2010 http://twitter.com/jdrumgoole 41
42. Example : Product Backlog
• Product Backlog : Invoicing System
Feature Development
Effort (Days)
Allow login using facebook credentials 5
Generate PDF copies of invoices 7
Graph credit risk 4
Send duplicates of all invoices 3
Annotate invoice 3
Customise invoice look and feel 5
Support multi-currency 10
Support time sheets for work 10
Store list of clients and allow editing 4
23 July 2010 http://twitter.com/jdrumgoole 42
43. Exercise
• Prioritise Two Features
• Write sprint backlog with estimates
• Present to group
23 July 2010 http://twitter.com/jdrumgoole 43
44. Process : The Daily Scrum
• Parameters
– Daily
– 15-minutes
– Stand-up
• Not for problem solving
– Whole world is invited
– Only team members, ScrumMaster, product
owner, can talk
• Helps avoid other unnecessary meetings
23 July 2010 http://twitter.com/jdrumgoole 44
45. Process : Daily Scrum
• Three Questions for each Team Member
– What did you do yesterday?
– What will you do today?
– Is there anything in your way?
• Not status updates for Scrum Master
• Commitments to your peers
• Keeps SCRUM moving forward
23 July 2010 http://twitter.com/jdrumgoole 45
46. Processes : Sprint Review
• Team presents Sprint Goal
• Ideally a Demo
• Informal
– 2-hour prep time rule
– No slides
• Whole team participates
• Invite the world
23 July 2010 http://twitter.com/jdrumgoole 46
47. Process : Sprint Retrospective
• Periodically take a look at what is and is not
working
• Typically 15–30 minutes
• Done after every sprint
• Whole team participates
– ScrumMaster
– Product owner
– Team
– Possibly customers and others
23 July 2010 http://twitter.com/jdrumgoole 47
49. Artefacts : Product Backlog
• The engine that drives scrum
• All desired work (from users perspective)
• Articulated as value delivered to customer
• Prioritised by Product Owner
• Should have rough development estimates
• Reprioritised at the start of each sprint
23 July 2010 http://twitter.com/jdrumgoole 49
50. Artefacts : Product Backlog
• Avoids “infrastructure”
• Avoids “architecture astronauts”
• Focuses all efforts on business value
• Sprint Backlog articulates technical issues
• Anyone can add to the backlog
• Product Owner sets priority
• Starvation is healthy
23 July 2010 http://twitter.com/jdrumgoole 50
51. Artefacts : Sprint Backlog
• Sprint Goal
– Centralising paradigm of the overall sprint
– Put an umbrella over a number of backlog items
– Can be one backlog item
• Example
– Support multiple users per account
– Integrate Google Docs
– Support Sage Accounting
23 July 2010 http://twitter.com/jdrumgoole 51
52. Artefacts : Sprint Backlog
• Individuals sign up for work of their own choosing
– Work is never assigned, collective ownership
• Estimated work remaining is updated daily
• Any team member can add, delete or change the
sprint backlog
• Work for the sprint emerges
• If work is unclear, define a sprint backlog item with a
larger amount of time and break it down later
• Update work remaining as more becomes known
23 July 2010 http://twitter.com/jdrumgoole 52
53. Artefacts : Sprint Backlog
• Sprint Tasks
– Each task 8-16 hours
– Should be achievable by any team member
– Task time limits avoids “skunk works”
23 July 2010 http://twitter.com/jdrumgoole 53
54. Artefacts : Burn Down Chart
23 July 2010 http://twitter.com/jdrumgoole 54
55. Agile Deployment
• Use what works for you
• Phase 1
– Test Driven Development
– Continuous Integration
• Phase 2
– Continuous Integration
– Shorter release cycles
23 July 2010 http://twitter.com/jdrumgoole 55
56. What can go wrong Internally?
• No onsite customer
• Sprints too long
• TDD Lip service (all tests pass, nothing works)
• Techno Bigotry
– The solution to everything is a NoSQL Db
• Over architecting solutions
23 July 2010 http://twitter.com/jdrumgoole 56
57. What can go wrong Externally?
• No management buy in
• Expectations that 12 month long waterfall
schedules will hit their dates
• Expectations of Quarterly “move the market”
releases
• No domain expertise
• The blame game
23 July 2010 http://twitter.com/jdrumgoole 57
58. Tools Overview
• Source Code Control
• Ticket/Story Tracking
• Burn Down Charts
• Sprints
• Continuous Integration
• Unit Testing
• Acceptance Testing
23 July 2010 http://twitter.com/jdrumgoole 58
59. Source Code Control
• Subversion
– Centralised
– Mature
– Good tool support
– Lots of hosting providers
– TortoiseSVN for Windows Clients
• GitHub
– Distributed
– Younger Lineage
– Better for Branching
– A few hosting providers
23 July 2010 http://twitter.com/jdrumgoole 59
60. Tickets: Trac
• Trac
– Open Source
– Simple wiki and ticketing support
– Integration with Mylyn
– Lots of hosting providers
– Very simple clean interface
– Can be deployed on-premise
– Nice integration with subversion
23 July 2010 http://twitter.com/jdrumgoole 60
61. Tickets : Assembla
• Nice Ticket interface
• Mylyn integration for Eclipse
• Client interface with good scrum support
• Good wiki integration
• Good Multi-project support
23 July 2010 http://twitter.com/jdrumgoole 61
62. Tickets: JIRA
• JIRA
– Hosted or on-premise
– Great ticketing system
– Lots of features and integrations
– Comes with full development suite
– $125 a month for 5 developers
23 July 2010 http://twitter.com/jdrumgoole 62
63. Burn Down : Charts
• Assembla
• Wush
• Pretty easy to knock out in Excel
– If they come for free, use them
– Don’t pay for them
– Consider tracking Unit tests and Tickets Closed
23 July 2010 http://twitter.com/jdrumgoole 63
64. Sprints
• Wush
• Assembla
• JIRA
– All support sprint structures
– Progress indicators
– Correlation with source code control system
• Backlog management is hard in Wush
• Multi-projects hard in Wush
23 July 2010 http://twitter.com/jdrumgoole 64
65. Continuous Integration
• Hudson
– Apache, easy to setup, works for all scripted builds
– Great UI
• Cruise Control
– Older than hudson, but does .NET stuff
• Continuum
– Apache project, primarily Java builds
• Bamboo
– Slick but need to pay for it
23 July 2010 http://twitter.com/jdrumgoole 65
66. Unit Testing
• Various xUnit Frameworks
• Use one
• Good integration with eclipse for JUnit
• Every language has a framework
• http://en.wikipedia.org/wiki/List_of_unit_testing_fra
meworks (66 listed)
23 July 2010 http://twitter.com/jdrumgoole 66
67. Acceptance Testing
• Selenium
– Plugin and IDE
– Stores web requests and results
– Reports success/failure
• FIT
– Create “fixtures” in Word/Excel
– Test using FIT library
23 July 2010 http://twitter.com/jdrumgoole 67
68. Summary
• Get the individual activities working
• Get the group activities working
• Embraced rather than imposed
• Automation really helps
• Deliver value before delivering a sea change
• Start Small
23 July 2010 http://twitter.com/jdrumgoole 68
69. Deming
In God We Trust.
All others bring data.
23 July 2010 http://twitter.com/jdrumgoole 69