Presentation from Scrum Gathering Global Event in Atlanta 2012. Presenting a case study of large agile projects in publicv sector and discussing good agile practices.

1. Large public sector projects
– what determines failure or success?
Remi Hansen, PROMIS AS
25.09.2014 • © PROMIS AS 1

2. Benefits from attending
• Learn what factors are important for succeeding with large government Scrum
projects and what pitfalls to look out for.
25.09.2014 • © PROMIS AS 2

3. The Presenter – Remi Hansen
• B.Sc. In SW Engineering, M. Sc. in Industrial Economics
• More than 20 years experience from
the IT Consulting business
• A few years as a system developer
• Primarily project manager and advisor / Business Consultant
on strategic projects in both private and public sector
• Some years as line manager – incl. Head of the Project Management Community in
one of Scandinavia’s largest IT consultancy groups, with more than 150 PMs with
extensive use of Scrum in their projects
• Certified Project Management Professional (PMP), PRINCE2 Practitioner,
IT Project Professional (ITPP), CSPO, ISTQB SW Testing Foundation and ITIL.
• Experience with agile projects started in 2002.
• Currently Senior PM in PROMIS, a leading provider of agile project
management services in Norway (www.promis.no)
25.09.2014 • © PROMIS AS 3

4. Background
• The Nordic model – mixed market economy, welfare states
• “It is distinguished from other welfare states with similar goals by its emphasis
on maximizing labor force participation, promoting gender equality, egalitarian
and extensive benefit levels, large magnitude of redistribution, and liberal use
of expansionary fiscal policy.” (Wikipedia)
• Public sector > 45 % of GNP
• Huge public sector spending  large and complex IT systems
• Our fair share of bad projects
• Agile (Scrum) has become the standard approach for public sector projects
• Same motivation as in private sector – a fixed and stable requirements specification
will not give the desired outcome
25.09.2014 • © PROMIS AS 4

5. Agenda
 Introduction 10 min
• Brief presentation of the three projects: 10 min
• Six main recommendations: 60 min
• Lessons learned summary: 10 min
• Q&A
25.09.2014 • © PROMIS AS 5

6. The case study projects
25.09.2014 • © PROMIS AS 6

7. Case study
• Lessons learned from 3 federal
government projects
• Over 1,000,000 hours of effort
• Not a scientific study
• 1st and 2nd hand information used
• Factual information is from public sources,
the rest is partly my personal opinion and
partly from sources listed
25.09.2014 • © PROMIS AS 7
Photo (Flickr):
Okko Pyykkö

8. Norwegian Public Service Pension Fund (SPK)
PERFORM project
• SPK
• Administers pension entitlements of $ 59 billion on behalf of the Norwegian state
for almost 1 million former and existing employees in the public sector
• $ 3,2 billion in pension payments (2009)
• Purpose / benefit of the project
• The pension reform
• Outdated technology platform
• New benefit type introduced
• Status: Just completed – very successful, seen as best practice in agile
implementation
• Size:
• 13 teams, 2+1 main suppliers, 175 people
• 300 epics  2500 user stories
• 12 releases over 4 years
• $ 175 Million
25.09.2014 • © PROMIS AS 8

9. Norwegian Public Service Pension Fund
PERFORM project
• Interesting feature
• Huge project – needed to start development before legislation was passed
• Contract and sourcing setup
• Multisourced
• T & M on Solution Description phase – the customer retained responsibility for
analysis and conceptual design
• Suppliers responsible for construction of their deliveries
• Each delivery (release) negotiated with a separate target price – i.e. target price on
user story level
• Target price 50 / 50 (no cap)
25.09.2014 • © PROMIS AS 9

10. Statnett
LARM Project
Statnett
• State Enterprise responsible for all high
voltage electricity transmission
and distribution in Norway
• Owns, operates and regulates
the national main grid
• Statnett co-ordinates supply and demand, and owns the main Norwegian power grid.
25.09.2014 • © PROMIS AS 10
Photo: Statnett.no

11. Statnett
LARM Project
LARM
• Community-critical system for ensuring the responsiveness of
the national power grid
• Status:
• Ongoing (midway)
• Challenged (overruns and conflicting interests)
• Size:
• 2 mid-sized Scrum teams initially – gradually increased staffing
• $ 20 million (supplier cost only), 3 year project
• Interesting feature
• Very specialized domain – few users, but community critical operation
• Contract and sourcing setup
• Target price with customer friendly profile (very close to fixed price contract)
• All activities and phases included in target price
• Price commitment on total scope Photo: Statnett.no
25.09.2014 • © PROMIS AS 11

12. The Norwegian Public Roads Administration
AUTOSYS project
The NPRA
• Responsible for the planning, construction and operation of the national and
county road networks, vehicle inspection and requirements, driver training and
licensing
Autosys
• Replacing 30 year old mainframe system
• More self-service, fewer manual processes
• 20 000 users, 100 million transactions / year
8 million vehicles and 3,5 million driver licenses
• 5 releases in 3 years (overall 5 year project)
• SOA platform
• Status: Ongoing, challenged
• Size:
• 65 on supplier side, 35 on customer side
• 4 Scrum teams
• $ 100-150 million project budget ($70 M supplier cost)
25.09.2014 • © PROMIS AS 12
Photo: Vegvesen.no

13. The Norwegian Public Roads Administration
AUTOSYS project
• Interesting feature
• Previous attempt to run the project terminated – generally seen as a failure outside
the NPRA (called a “scandal” by the press)
• High visibility to the public
• Contract and sourcing setup
• Target price with customer friendly profile (close to fixed price contract)
• All activities and phases included in target price
• Price commitment on total scope
25.09.2014 • © PROMIS AS 13

14. Case study projects summarized
• Large to very large projects in public sector
• Very different domains and parts of the state administration
• All Scrum based projects
• One successfully completed, two ongoing challenged
• All rely on IT suppliers governed by contractual obligations
• All target price contracts (PS2000 based), but with very different customer /
supplier balance
25.09.2014 • © PROMIS AS 14

15. Agenda
 Introduction 10 min
 Brief presentation of the three projects: 10 min
• Six main recommendations: 60 min
• Lessons learned summary: 10 min
• Q&A
25.09.2014 • © PROMIS AS 15

16. Main recommendation areas
Six main recommendations:
1. Enterprise scale Scrum implementation
2. Soft skills and cultural aspects
3. Business value and product backlog issues
4. Integration of test and development tasks in sprint teams
5. Decision making and team interaction
6. Specific public sector challenges and contractual implications
25.09.2014 • © PROMIS AS 16

17. Area 1: Enterprise scale Scrum
implementation
25.09.2014 • © PROMIS AS 17

18. Findings
Enterprise scale Scrum implementation
What is needed to run a large Scrum project?
Wanting to be agile, but also maintaining predictability and control – different levels
of agility seen
Different approaches to scaling the projects’ Scrum implementation regarding
a) Organization, roles and responsibilities
b) Processes
c) Planning and controlling
d) Testing and approval
25.09.2014 • © PROMIS AS 18

19. Findings
Enterprise scale Scrum implementation
Contracts
• Add complexity – A large supporting organization is generally not need with two
teams, but a large contract requires administrative overhead
• Customers (also in private sector) want supplier commitment to cost /
productivity / buy-in
• A large project under a contract is different from just adding some extra Scrum
teams to your in-house Scrum development
Revisited later!
25.09.2014 • © PROMIS AS 19

20. Recommendations
a) Organization, roles and responsibilities
• Everybody cannot overview every aspect of the project when they get large –
division of responsibility needed
More people  narrower field of responsibility and more interfaces to other
roles  bigger chance of miscommunication
• More roles added  needs clearly defined responsibilities
25.09.2014 • © PROMIS AS 20

21. Scrum Team
Scrum master
Subject Matter
Expert (Cust.)
Test specialist
25.09.2014 • © PROMIS AS 21
(Cust.)
Technical
Architect
Developer / tester
Developer / tester Developer / tester
Developer / tester Developer / tester
Scrum team

22. Example organization
Joint
Steering Committee
Customer’s
Programme
Director
Supplier
Programme
Manager
Customer
Deliverables and
testing
Sub-project
Development
Solution
Architect
25.09.2014 • © PROMIS AS 22
Scrum
team 1
Scrum
team 2
Scrum
team 3
Scrum
team 4
Supplier
Project Owner
Test
Manager
Technical
Architect
Usability
experts
Config. Mgr.
Tech. support
Test specialist
Technical tests
Test specialist
Functional tests
Training & impl.
Co-ordinator
Oracle Enterprise
Architect
Business Process
Expert
Supplier Internal SC Customer
Internal SC
Customer
Implementation
project
Maintenance
coordinator
(from Delivery 1)

23. Recommendations
a) Organization, roles and responsibilities
Some relevant roles
• Project management – large financial impact, …
• Product team – instead of single Product Owner (covered later)
• Architecture / technical environments team
25.09.2014 • © PROMIS AS 23

24. Recommendations
b) Processes
• More defined processes are necessary in large projects
• More roles and people, more fine grained division of work
• Not the level of documentation that counts,
it’s the level of common understanding!
25.09.2014 • © PROMIS AS 24
Photo (Flickr):
IvanWalsh.com
Photo (Flickr):
Samuel Mann

25. Recommendations
c) Planning and controlling
• A release master plan (solution roadmap) – not just a product backlog
• Control gates: Book keeping for each sprint (earned value calculations etc) –
covered later
25.09.2014 • © PROMIS AS 25

26. Recommendations
c) Planning and controlling – Release Master Plan
• The Release Master Plan is a tool for planning handover and implementation of
new functionality
• A release demands capacity in the organization – the must be prepared
• The Release Master Plan is a tool for the project team and forms the foundation for
all sprints together to form a predictable and acceptable delivery
• The Release Master Plan identifies the number of releases, their size and
timing
• Orchestrates deliveries and pinpoints internal and external dependencies
• Shows milestones and phases toward go-live
• Basis for budgeting
• Guiding principle: Realization of business value!
25.09.2014 • © PROMIS AS 26

27. Recommendations
c) Planning and controlling – Release Master Plan
Content:
• Functional target
• What epics (functional and technical) are included?
• High level priorities
• Team capacities
• The number and duration of sprints
• Planned dates for sprint demos, control gates, start and finish for acceptance
test, go-live
25.09.2014 • © PROMIS AS 27

28. Recommendations
d) Testing and approval
• Control gates following each sprint
• Dedicated test phase (addressed later)
25.09.2014 • © PROMIS AS 28

29. Recommendations
d) Testing and approval – Control gate at sprint completion
• To check if the user stories meet Definition of done, the project performs a
«control gate»
• Administrative «tail» - aftermath of one sprint spills over into the next sprint –
Should be kept as short as possible to avoid discontinuity
• With some training, this may be performed in 2-3 working days following the
sprint demo
• During these days, it is ‘all customer representatives to the pumps’
• Functional verification by Product Owner and his / her team
• Checking code quality
• Checking architectural guidelines
• Checking test documentation
• Checking other documentation
• Contractual milestone
25.09.2014 • © PROMIS AS 29

30. The Anatomy of the Sprint
Sprint demo Sprint X
Sprint planning Sprint X+1 starts with decomposition
Sprint backlog for Sprint X+1 ready
Control gate:
1. Evaluation of Sprint X
2. Approval of sprint plan for Sprint X+1
3. Updating the risk matrix
4. Change control
5. Repatriation of agreed outstanding issues to the
product backlog
25.09.2014 • © PROMIS AS 30

31. Construction phase: The control gates
Blue line: Sprint team
Red line: Product Owner
Yellow line: Verification, Control gate and Definition of Done
25.09.2014 • © PROMIS AS 31
Construction phase
Solution description
phase

32. Planning and controlling considerations in CG
 What was the productivity of the last sprint?
 How was the product quality?
 How is the risk picture evolving?
 What can we improve – emphasis on learning and continous improvements
25.09.2014 • © PROMIS AS 32

33. Recommendations
d) Testing and approval – Commercial reconciliation
• Payments are sometimes linked to progress in development
• Our clear recommendation is to avoid commercial conditions linked to the
Control Gates – commercial reconciliation should wait until Acceptance testing
is completed
25.09.2014 • © PROMIS AS 33
Photo (Flickr):
Images_of_Money

34. Enterprise scale Scrum implementation
Key Learning Points
 Size and contractual obligations drives the need for a
supporting project organization
 Keep organization and processes on a practical level
to maintain agility
 Use a Release Master Plan for planning and
communication
 Execute a Control Gate process after each sprint for
control (progress / velocity, productivity, risk, etc.)
25.09.2014 • © PROMIS AS 34
Photo (Flickr):
Horia Varlan

35. Area 2: Soft skills and cultural aspects
25.09.2014 • © PROMIS AS 35

36. Findings
Soft skills and cultural aspects
• The projects differed significantly in their ability to build a common project
culture
• Contract parties not bonding together is very counterproductive – more focus
on handovers than the actual work to be done
• Project initiation takes time and should be given sufficient time
• Depending on both customer and supplier – T&M pricing is a good idea in this
phase, to ensure enough time and effort is spent on initiation
• Different cultures collided – washed away the basis for mutual understanding
and trust impeding effective interaction
• Customers adapting bad practice based on lack of experience
– Knowing when to seek external help is very useful
• Lacking the courage to take decisions severely hampers progress
25.09.2014 • © PROMIS AS 36

37. A clash of organizational cultures
Stereotypical public sector officer
behavior
Anticipated contribution in an agile
system development project
Work according to regulations Design future work processes based on
business value
Working with individual cases See different types of processes in
relation to simplify and standardize
Concrete work patterns "as is" – few
thoughts about future processes
Conceptual modeling of future processes –
limited knowledge of existing processes
Exact rules and all exceptions Good enough and main processes first
Collective / Agency Individual / user
Long-term production perspective Project delivery now
Don’t forget: What we expect from customer representatives
may be very different from what they do on a daily basis
– different jobs attract different people!
25.09.2014 • © PROMIS AS 37

38. Recommendations
Project culture and soft skills
• Invest sufficient time in creating a common project culture - focused on creating
business value
• Customer representatives in the Scrum teams is a good start!
• Take into account that cultural mismatches affect productivity
25.09.2014 • © PROMIS AS 38

39. Recommendations
Avoiding bad practice
• Contracts emphasis the supplier vs. customer obligations, less on the project
as one group
 Avoid contractual thinking from team members on all levels – confine that to
the management level
• Appoint managers who are not afraid to take decisions
• Make sure to get external help to promote better practice – a good way to
resolve conflicts
• Use advisors to assist in resolution, not for gathering ammunition for a possible
blame game
25.09.2014 • © PROMIS AS 39

40. Soft skills and cultural aspects
Key Learning Points
 Organizational culture aspects may influence your
project’s ability to succeed – act accordingly!
 Scrum is relying on effective decision making – create
a culture to promote that
25.09.2014 • © PROMIS AS 40
Photo (Flickr):
Horia Varlan

41. Area 3: Business value and product backlog
issues
25.09.2014 • © PROMIS AS 41

42. Findings
Business value and product backlog issues
• Poor quality of the product backlog is devastating for the project
• Too large user stories  impossible to remove anything from the scope
(every user story contains something critical)
• Poorly defined user stories  many discussions  late completion of user story 
limited time for testing
• Ambiguities  low productivity and many assumptions (risk)
• Decision making: real (and lasting!) prioritization is difficult for inexperienced
P.O.s
• A lot of accumulated needs in the organization
• A tendency to behave like kids in a candy store, if there are no mechanisms to
control scope
• Responsibilities for different aspects of the product backlog is a source of
misunderstanding and controversy
25.09.2014 • © PROMIS AS 42

43. Findings
The vicious circle of poor backlog quality
 Poor quality of the sprint backlog (capacity and/or expertise)
 Abundant clarifications needed - takes more time from the P.O. and Scrum
teams
 Late clarifications  late completion of user stories
 Poor tests and low productivity (developers losing focus, multitasking unclear
tasks)
 More effort needed from both the customer and the supplier into the next sprint
to sort out whether user stories are “done”
 Prolonged Control Gate period
 Reduced opportunity for P.O. team to contribute to the planning and launching
of the next sprint (busy doing backward-facing activities) – introducing
uncertainty in the sprint plan
 Lack of commitment from the teams – not possible to deliver the sprint contract
anyway, because of the poor sprint backlog quality
25.09.2014 • © PROMIS AS 43

44. Findings
Business value and product backlog issues
• The P.O. team did not work effectively – the Chief P.O. had to take all
decisions
• The customer did not meet their obligations regarding Business Analysts and
Test Specialist – partly having the wrong profiles
• The resulting lack of clarifications and scoping (acceptance criteria) gave extra
load on the P.O.
25.09.2014 • © PROMIS AS 44

45. Recommendations
Good product backlog practice
• The customer must control the scope!
• A well organized P.O. Team supporting the Chief P.O. is needed
• Individual P.O.s must co-ordinate across domains – creating one coherent
backlog
• Appoint a Business Analyst to each sprint team = P.O.’s delegates to the
teams will relieve the load on the P.O.
– Preferably Customer Business Rep., but in a long project even a hired
Business Analysts can add value
25.09.2014 • © PROMIS AS 45

46. Recommendations
Good product backlog practice
• Continuously work with grooming the backlog – invest enough capacity to
clearly define User stories, and give priority
• Setting priorities takes a lot more effort than one should expect – working
towards the line organization and avoiding rematches
• Put in place a method for assessing business value – to avoid somebody’s pet
requirements sneak past prioritization
• View the User story estimate as the budget for the task – don’t estimate the
task again with high level of freedom and creativity
• Challenge the P.O. to find a simpler solution if the scope doesn’t match the estimate
25.09.2014 • © PROMIS AS 46

47. Product owner team
Development team 1 Development team 2 Development team 3
25.09.2014 • © PROMIS AS 47
Product owner team
Product owner

48. Business value and product backlog issues
Key Learning Points
• The quality of the backlog cannot be stressed enough –
the project success completely relies on it!
 Ensure enough effort is spent grooming the backlog
• Use business value as basis for prioritization
• Set up a P.O. team as described and appoint a
Business Analyst to each scrum team
25.09.2014 • © PROMIS AS 48
Photo (Flickr):
Horia Varlan

49. Area 4: Integration of test and development
tasks in the sprint teams
25.09.2014 • © PROMIS AS 49

50. Findings
Integration of test and development tasks
• Different system test approaches taken – integration of test into development
seems to give the best result
• The customer did not participate with test specialist in the Scrum teams as
dictated by contract
• Absence of testing capacity in the teams was compensated with extra effort
from the test team
• Role confusion (became part of the development team and not a gatekeeper to dev.)
• Capacity problems in the Test team  not well prepared for system test
• Poor testing in the sprints  the customer had to test more in Control Gate
(after sprint demo)
• Delayed CG and sprint plan for next sprint
• More book keeping and duplication of tests at the expense of other important work
• Aftermath stealing attention from planning
• Test data issues
25.09.2014 • © PROMIS AS 50

51. System Hardening sprints
• 2-3 sprints after the development sprints to perform system test and complete
outstanding work (typically documentation and clean-up) – done in parallel with
solution description for the next delivery
• Is that a good idea?
 There’s always some activities not completed. May give better preparation for and
smoother execution of Acceptance test
 Real integration test will not be done along the way  becomes a waterfall
approach, where testing is postponed (long time from programming to testing)
 Risk mitigation is delayed
 Very demanding for the project to do both system testing and solution design in
parallel
 Probably not a good idea – include system testing in the sprints instead to
minimize time from programming to testing
25.09.2014 • © PROMIS AS 51

52. Recommendations
Integration of test and development tasks
• The Test Manager should state quality requirements to development teams –
i.e. Definition of Done related to testing
• The Test Manager should have the authority to label user stories «Done» - i.e.
accepting what user stories will be demonstrated at sprint completion
• The Test Manager must have the motivation to withhold user stories not ready
for demo
• Allocate time in the coming sprint to do system tests of previous sprints delivery
– rather than waiting and doing all system testing in the end (hardening phase)
25.09.2014 • © PROMIS AS 52

53. Recommendations
Integration of test and development tasks
• There should be very limited need for the customer to perform separate tests –
rather verifying the test activities undertaken by the supplier
• In traditional waterfall environments «acceptance» is a very important decision
– such customer’s need reassurance that approving a user story does not
mean they have accepted the quality of the system (just that this part of the job
is done and in accordance with expectations) – i.e. commitment from the
customer on the content
25.09.2014 • © PROMIS AS 53

54. Recommendations
Integration of test and development tasks
• Integrate test into Scrum teams
• Better flow and continuity
• Close cooperation between business, developers and testers
• Appoint one in each Scrum team as Team Test Responsible
• No formal responsibilty for the product quality (still team responsibilty)
• Assignment to keep a close look at the testing activities and remind the others to do
their testing
• Planning, design, test conditions, development
and test performed collectively by the team
• Testers are no longer quality police
– instead valued members of the team
• Unit tests, integration tests, functional system
test and system integration tests
in construction sprints
25.09.2014 • © PROMIS AS 54

55. Test in agile development
• Tests are run continuously through the period
• Acceptance criteria written in parallel with
description of needs and solution design
• Tests written in advance of programing and
run in the sprints for completed user stories
• What can be approved, should be approved
progressively (in Control Gates)
25.09.2014 • © PROMIS AS 55

56. Integration of test and development tasks in the sprint teams
Key Learning Points
 Maximize the testing done as part of the development
sprints
 Appoint a test responsible in each sprint team
 Empower the Test Manager to set process
requirements
 Avoid pointless replication of tests on the Customer
side
25.09.2014 • © PROMIS AS 56
Photo (Flickr):
Horia Varlan

57. Area 5: Decision making and team interaction
25.09.2014 • © PROMIS AS 57

58. Findings
Decision making and team interaction
• It takes courage to manage an agile project – No collective decision process to
hide behind
• E.g. as Product Owner you put your head on the block – will public sector employees
feel they can gain benefits making that worthwhile?
• Demanding to say “we don’t need that documented”
• Blame game mindset introduce waste and is self-perpetuating – fight it!
• The relentless will to learn and improve is not evident in every project – those
who learn fast seems to succeed
25.09.2014 • © PROMIS AS 58

59. Findings
Decision making and team interaction
• With larger projects the teams need guidelines – don’t leave it to the teams to
sort out how the project should work
• Delegate the task, but make a central decision
• All projects used self-organizing teams – with success
• Able to manage their own work and maintain good motivation
• Retrospectives worked well in all projects
• The teams used a wide range of techniques in the retrospectives – the process
worked well
• Important to summarize recommendations and actions across teams (incl. the
customer)
• With slippage and scope challenges the teams started to take shortcuts
• Measurement primarily on velocity – pressure to take on more new functionality
• Started building technical debt and accumulated risk
• Stopped taking responsibility for cross-team issues
25.09.2014 • © PROMIS AS 59

60. Recommendations
Decision making and team interaction
• Defer decisions
• Let those best qualified take the
decision – create a culture for
delegation and trust
• Use “mini demo” to reduce risk
• Introduce friendly competition between
the teams
25.09.2014 • © PROMIS AS 60
Photo (Flickr):
Budzlife

61. Recommendations
Communication boosters
• Separate daily Scrums are not sufficient
for coordination
• Scrum of Scrums
• Meta Scrum
• «Town hall meetings»
• Communities of practice
• Use visualization actively
to communicate concepts, plans,
and progress – the master release plan
is a good starting point
25.09.2014 • © PROMIS AS 61
Photo (Flickr):
jardenberg

62. Decision making and team interaction
Key Learning Points
 Key stakeholders must be courageous - Strive for a
culture based on trust
 Stress the importance of retrospectives and pursue
improvements
 Use communication boosters to ensure everybody is
on the same page
25.09.2014 • © PROMIS AS 62
Photo (Flickr):
Horia Varlan

63. Area 6: Specific public sector challenges and
contractual implications
25.09.2014 • © PROMIS AS 63

64. Findings
Specific public sector challenges and contractual implications
• Public sector is engineered for cost control – «must» have a contract with
supplier obligations (a T&M project is difficult to fund in public sector budget
processes)
• Little personal benefit from taking risk and accountability
• No culture for accountability for project benefits – without financial
measurements the planned benefits may be too vague
• Two of the projects seem to be systematically underestimated – this
undermines the target price mechanism
• Sticking to unrealistic estimates is counterproductive – the supplier will not deliver as
good as it can and the customer will probably not get a very good system
• Two of the projects included the initiation phase in the target price, with
penalties on “initiation complete” milestone. The supplier thus defined itself as
finished on time, even if the delivery was not complete. That was paid dearly in
later stages.
25.09.2014 • © PROMIS AS 64

65. Findings
Specific public sector challenges and contractual implications
• In PERFORM the customer chose a balanced contract and took responsibility
for scope control
• Contract assignment per supplier per delivery at the end of each Solution
Description phase
• The challenged projects were more focused on cost control than having a
balanced contract
• Close to fixed price (cap on risk sharing)
• Including initiation and solution description in the fixed price
• Commitment on price for the complete scope, without any means of adjustment to
actual productivity
• This seemed to also reduce the customer’s focus on scope control – defining
that as supplier responsibility
• When overruns appeared (effectively making the price model = fixed price) we
saw a “culture of blame”
25.09.2014 • © PROMIS AS 65

66. Contracting price models:
• Time&Material:
• The Customer pays for all worked hours, disregarding
productivity and quality – all risk is on the Customer
• Fixed Price:
• The Customer pays the same amount, disregarding the hours
needed to produce the scope with agreed quality – all risk is
on the Supplier
• Target Price:
• Customer and Supplier share benefits or losses in an agreed
ratio – if this ratio is 50%, the risk is evenly distributed
between the parties
25.09.2014 • © PROMIS AS 66

67. The hourly rate curves –
comparing different price formats
25.09.2014 • © PROMIS AS 67

68. What price model is beneficiary in agile projects?
Target price is best suited because:
• Agile is a close and mutual co-operation
between customer and supplier
• Target price with 50/50 sharing of
incentives and sanctions ensure the
customer and supplier works towards the
same goal
• Target price is suitable when requirements
are not very detailed
• Target price is suitable for projects with risk
• Target price can be based on a less
detailed basis than traditional fixed price
25.09.2014 • © PROMIS AS 68
Price model Average overrun
Time & Material based
(4 projects)
55%
Fixed price (5 projects) 33%
Target price (7 projects) 10%
Others (2 projects) 13%
Total (18 projects) 27%
• Fixed price assumes
requirements are described in
detail and that uncertainty is
low.
• T & M is suitable when scope is
not well defined and uncertainty
is high.

69. Why Target Price Contracts with Risk Sharing?
• The Supplier is invited to commit to estimates with a higher degree of
uncertainty than in traditional fixed price contracts
• This is paramount in agile system development projects, where one of the main
principles is to avoid waste
• In particular, effort can be reduced on initial specification, detailed design and
planning
• The Customer may produce tender documents with high level requirements
• The Suppliers produce solution descriptions, estimated with a significant
degree of uncertainty
• In this way, both parties reduce work load in the initial bidding
phases of the project
25.09.2014 • © PROMIS AS 69

70. Recommendations
Specific public sector challenges and contractual implications
A balanced contract is necessary to ensure commitment and quality
• Implies 50 / 50 risk sharing with no cap
• Not agile to commit to up-front estimates for several years based on very
limited knowledge – risk for the supplier and ultimately the customer
• Include mechanisms to avoid commitment to estimates over a very long time
• Negotiating target price per delivery (release)
• Other mechanisms to adjust to actual velocity and productivity
• Consider T&M for the initiation phase, to ensure no corners are cut
25.09.2014 • © PROMIS AS 70

71. PS2000 – a Target Price Contracting standard
• A Norwegian contracting standard in the IT industry
• A result of a research program during the years 1997 – 2000
• By 2012, used in a number of large system development projects in the
Norwegian public sector
• Also used in private sector and internationally
• Referred to in Mary and Tom Poppendieck:
’Implementing Lean Software Development:
From Concept to Cash’ Addison-Wesley 2007
25.09.2014 • © PROMIS AS 71

72. PS2000 Standard contract - English version
Requirement
Analysis
Progression
CG1
Iterative
construction phase
Solution
Description
http://www.dataforeningen.no/it-contract-standards.146223.no.html
25.09.2014 • © PROMIS AS 72
Acceptance an-d
Completion phase
Detailed planning
Analysis and design
Testing Development
CGn
C
CG2
PMS 0
Contract Signing
PMS 1
Approved Solution
Description
PMS 2
Delivery ready
for Acceptance
PMS 3
Accepted
Delivery

73. In summary - Advantages with PS2000 Agile
• Corresponding to the principles in Agile in a better way than alternative
contracting models
• Not unlike agreements to deliver competence and capacity (as in
Time&Material contracts), but framing in the scope, time, cost and risk
• Suitable for repeating releases in a frame agreement for new development or
application maintenance
• An incentive for the supplier to deliver more functionality within agreed time
limits, but with satisfactory quality
• The main success factor is that the parties agree on the process manging the
Product Backlog, Sprints and Control Gates, with the corresponding roles
• Change control becomes lean – a signed Product Backlog after each sprint
documents the agreed changes
25.09.2014 • © PROMIS AS 73

74. Specific public sector challenges and contractual implications
Key Learning Points
 Agile projects can be run under a contract, given that
the contract
 Is balanced
 Is engineered for agile execution
 Lets scope be defined as late as possible
(preferably contracting each release as a separate target
price delivery)
 PS2000 Agile is such a contract standard
25.09.2014 • © PROMIS AS 74
Photo (Flickr):
Horia Varlan

75. Agenda
 Introduction 10 min
 Brief presentation of the three projects: 10 min
 Six main recommendations: 60 min
• Lessons learned summary: 10 min
• Q&A
25.09.2014 • © PROMIS AS 75

76. Summary
25.09.2014 • © PROMIS AS 76

77. Summarizing some major issues
• Impression that all the projects would be agile – only one of them were
• Contractual constraints (~ fixed price)
• Cultural challenges – still a fixed price mindset on the customer side
• Customer not participating as actively as anticipated – both staffing and decision
making
 Scrum does not necessarily make a project agile!
• Truly agile execution is possible within a contract constraint, with the right
contract
• Naive to expect Scrum / agile to solve every problem
• Even the most successful project had challenges in the first stage – but a
willingness to learn & adapt brought the project on the path of good practice
25.09.2014 • © PROMIS AS 77

78. Some smart moves in the PERFORM project
paving the way for success
• Gained experience with Scrum before starting the project
• Understood the value of a balanced contract
• Initiation phase on T&M price model
• Had own Scrum teams in parallel with
supplier teams
• Invested in good procedures and tools for migration
and configuration control
• Test integrated part of development
• Hired external help for agile project management
coupled with active top management involvement
• The customer took overall responsibility for all processes – the main suppliers
had responsibility for construction of their parts for every release
• A continuous focus on improvement
 Truly agile execution!
25.09.2014 • © PROMIS AS 78
Photo (Flickr):
apdk

79. 8 defining features of agile projects
1. Business value is the most important criteria for quality and direction
2. Continuous prioritization of functionality based on cost / benefit
3. Close communication between the business side and developers
4. Short iterations to delivery of executable code
5. Frequent releases to production (integrated and tested)
6. Binding decisions taken as late as possible (‘Rolling Wave Planning’)
7. Evaluation, learning and improvement along the way
8. Autonomy: Self-organizing cross-functional teams
25.09.2014 • © PROMIS AS 79

80. Defining features of agile projects
Feature Degree of
divergence in case
25.09.2014 • © PROMIS AS 80
study
Covariance
between good agile
practice and
success
Business value focus High
Continuous prioritization High
Communication with business Medium-High
Short iteration – executable code Low
Frequent releases Low
Deferring decisions High
Learning Medium-High
Autonomy Low

81. Other reflections
• Using agile methodology does not replace good craftsmanship in software
engineering
• Emphasis on architecture
• A good process to establish the product backlog
• Best practice in conceptual design / modelling and code standards
• Configuration control and build (CI)
• Estimation
• Test strategy and plans
• Agile projects facing non-agile environments (e.g. interfacing systems with
decided release plans for the coming year) is high risk – must be solved
through governance
25.09.2014 • © PROMIS AS 81

82. Public sector specifics?
• Wide variety of organizations, management and culture in public sector – not
possible to see the whole sector as one homogeneous group
• The recommendations given here applies equally to private sector projects
25.09.2014 • © PROMIS AS 82

83. References
• IT Project Professional certification curriculum - http://smidigeprosjekter.no/itpp
• Upcoming book – working title ‘Agile Contracting’
• PS2000 Standard Contract
• The Norwegian Computer Society offer a complete set of standard contracts for
software development and maintenance and management.
• The PS2000 Standard Contract (standard version) is based on iterative
development. An agile version exists with alternative set of annexes, expanded and
adjusted according to agile methodology. The terminology is mainly based on Scrum
• http://www.dataforeningen.no/it-contract-standards.146223.no.html
• Complete english version - PS2000 Agile (PS2000 Agile, Maintenance, Framework,
Service Operations) ~ 1000 USD
• PRINCE2
25.09.2014 • © PROMIS AS 83

84. Questions or comments?
25.09.2014 • © PROMIS AS 84
Photo (Flickr):
Horia Varlan

85. Thank you for the attention!
rh@promis.no
25.09.2014 • © PROMIS AS 85