From Continuous Integration to Continuous Delivery and DevOps
•Download as PPTX, PDF•
7 likes•2,862 views
An overview of Continuous Delivery from a business and a technical point of view. Includes an overview of: - business value proposition of CD - prerequisites and tips for CD implementation - CD implementation was stories and strategies - CD technical practices
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to From Continuous Integration to Continuous Delivery and DevOps
Similar to From Continuous Integration to Continuous Delivery and DevOps (20)
More from Luca Minudel
More from Luca Minudel (20)
Recently uploaded
Recently uploaded (20)
From Continuous Integration to Continuous Delivery and DevOps
- 1. FROM C.I. TO CD & DEVOPS CONTINUOUS DELIVERY OVERVIEW Luca Minudel
- 2. LUCA MINUDEL DID CONTINUOUS DELIVERY AT: WORKS @ SESSION’S LINKS TWITTER @SMHARTERLTD
- 3. CD’S BUSINESS VALUE PROPOSITION Business side of CD 3
- 4. SIEMENS S62 Used in Italy from 1962 to 1980, 18 years
- 6. 6
- 7. RAI WHY CONTINUOUS DELIVERY? 1) REDUCE RISK Build the right thing, build it right – Avoid expensive mistakes
- 8. UNRELEASED CHANGES = RISK 1 1 2 1 2 3 4 1 2 3 Value Release Time
- 9. 1 2 3 4 1 1 2 1 2 3 RELEASED CHANGES = VALUE RELEASED TO 1. ANYONE, OR 2. EARLY ADOPTERS, 3. REQUESTORS OF THE FEATURE, 4. INTERNAL USERS
- 10. RAI WHY CONTINUOUS DELIVERY? 1) OVERCOME THE ‘90% DONE’ SYNDROME Done and done
- 11. 1
- 12. Validated learning over working software (over comprehensive documentation) Kent Beck
- 13. RAI WHY CONTINUOUS DELIVERY? 2) ALIGNMENT BETWEEN IT - BUSINESS - MARKET Play ball together
- 15. FAST FREQUENT COMUNICATION Sharing the same context
- 16. RAI WHY CONTINUOUS DELIVERY? 3) INNOVATE FASTER One step ahead
- 17. RAI WHY CONTINUOUS DELIVERY? 1)+3) FASTER AND SAFER TOGETHER Have your cake and eat it too
- 19. IMPLEMENTING CD: PREREQUISITES & QUICK SELF-ASSESSMENT IT side of CD
- 20. PREREQUISITES TO CD Iterative software development & Continuous Integration
- 21. from Adaptive Leadership by Jim Highsmith
- 22. ITERATIVE DEVELOPMENT Use with care.
- 25. CONTINUOUS INTEGRATION Is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily, leading to multiple integrations per day. Martin Fowler
- 26. CONTINUOUS INTEGRATION Is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily, leading to multiple integrations per day. Each integration is verified by an automated build including tests to detect integration errors as quickly as possible. Martin Fowler
- 27. C.I. IMPLIES => TRUNK BASED DEVELOPMENT All development is done on the mainline (also known as the head or trunk) of the source-code repository. All the developers commit the code to the mainline at least once per day. Developers commit only potentially releasable code using practices like latent-code patterns, feature toggles, and branch by abstraction. Every new release is built from the mainline.
- 28. C.I. IMPLIES => TRUNK BASED DEVELOPMENT Merge Merge Trunk NO FEATURE BRANCHING
- 29. C.I. IMPLIES => TRUNK BASED DEVELOPMENT Trunk 2 2 3 4 2 3 1 1 1 1 4 5 2 3 1 2 1 3 2 1 1 4 3 2
- 30. C.I. IMPLIES => TRUNK BASED DEVELOPMENT Merge Merge Trunk 2 2 3 4 2 3 1 1 1 1 4 5 2 3 1 2 1 3 2 1 1 4 3 2 1 4 3 2 1 4 5 2 3
- 31. C.I. IMPLIES => TRUNK BASED DEVELOPMENT Trunk 1 2 3 4 5 1 1 3 42 4 52 1 3 2 4 3
- 34. Continuous Delivery is a software development discipline where you build software in such a way that the software can be released to production at any time - Martin Fowler.
- 35. Continuous Delivery aims to reduce the cost, time, and risk of delivering incremental changes to users - Jez Humble.
- 36. ARE YOU DOING CONTINUOUS DELIVERY?
- 37. THE CD WORKING GROUP AT THOUGHTWORKS SAYS You are doing CD when: ①your software is deployable throughout its lifecycle ②your team prioritizes keeping the software deployable over working on new features
- 38. THE CD WORKING GROUP AT THOUGHTWORKS SAYS You are doing CD when: ①your software is deployable throughout its lifecycle ②your team prioritizes keeping the software deployable over working on new features ③anybody can get fast, automated feedback on the production readiness of their systems whenever somebody makes a change to them
- 39. THE CD WORKING GROUP AT THOUGHTWORKS SAYS You are doing CD when: ①your software is deployable throughout its lifecycle ②your team prioritizes keeping the software deployable over working on new features ③anybody can get fast, automated feedback on the production readiness of their systems whenever somebody makes a change to them ④you can perform push-button deployments of any version of the software to any environment on demand.
- 40. CD WAR STORIES Case studies with numbers
- 41. SAP 65K employees, 30K database tables BEFORE: 2010 • Business frustrated • 6-12 months from idea to production • Prod. Error => S**t-storm & late nights • Monthly production releases AFTER: 2012 • Business happy • 1 week from idea to production • Prod. Error => less then 1min rollback • Two production releases per week, 8x increase!!!
- 42. HP LASERJET FIRMWARE TEAM Between 400-800 developers, 10+MLOC BEFORE: 2008 • Only 5% of time available for creating/supporting new features • A code branch for each LaserJet model, manual integration & testing • Max two releases per year AFTER: 2011 • 40% of time available creating or supporting new features, 8x increase!!! • One main branch for all products, automated integration & testing • 10-15 candidate releases per day
- 44. IMPLEMENTING CD: STRATEGIES, TIPS, COMMON MISTAKES How to
- 45. Skills & Practices Automation & Tools Architecture & Design GREEN FIELD – 3/6 MONTHS • BUILD AUTOMATION • DEPLOY AUTOMATION • REMEDIATION AUT. • TEST AUTOMATION • INFRASTRUCTURE • MONITORING • CONFIGURABILITY FOR MULTIPLE ENVIRONMENTS • TESTABILITY • HOT DEPLOYABILITY • REMEDIABILITY • TRUNK BASED DEV.
- 46. Automation & Tools Architecture & Design BROWN FIELD - INITIAL 3/6 MONTHS - LOW HANGING FRUIT • BUILD AUTOMATION • DEPLOY AUTOMATION • BASIC REMEDIATION AUTOMATION • INFRASTRUCTURE • CONFIGURABILITY FOR MULTIPLE ENVIRONMENTS
- 47. BROWN FIELD – 1/2 YEARS BASED ON SIZE – TO THE FINISH LINE Skills & Practices Automation & Tools Architecture & Design • FULL REMEDIATION AUTOMATION • TEST AUTOMATION • MONITORING • TESTABILITY • HOT DEPLOYABILITY • REMEDIABILITY • TRUNK BASED DEV. Automation & Tools Architecture & Design BROWN FIELD - INITIAL 3/6 MONTHS - LOW HANGING FRUIT • BUILD AUTOMATION • DEPLOY AUTOMATION • BASIC REMEDIATION AUTOMATION • INFRASTRUCTURE • CONFIGURABILITY FOR MULTIPLE ENVIRONMENTS
- 48. ON HIRING DEVOPS Hey HR, creating a separate role for DevOps misses the point
- 49. DEVOPS DEFINITION A term coined by Patrick Debois To encourage people to think about software development and software support in a holistic way, as opposed to two separate activities.
- 52. DEVOPS DEFINITION Development OperationsDevOps DEVOPS ANTI-PATTERN
- 53. SELECTING CD TOOLS Individuals and interactions over processes and tools
- 54. TOOLS FOR CONTINUOUS DELIVERY ■ Package management: RPM, WiX, Wise, … ■ Infrastructure management: Puppet, Chef, … ■ CD server: Go , … ■ Dependencies management: … ■ Binaries repository: … ■ … ■ …
- 55. If there is any rule to selecting tools to support software delivery and support, it is to assume that: any tools chosen may need to be changed in the future - Kief Morris.
- 56. WWW.GO.CD OPEN SOURCE - FREE DOWNLOAD
- 57. ON CD & AGILE Agile Manifesto
- 58. AGILE MANIFESTO Principle #1 Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
- 59. ON CD & LEAN? Lean principles
- 60. LEAN SOFTWARE DEVELOPMENT Principle #5 Deliver as fast as possible.
- 61. TECHNICAL PRACTICES OVERVIEW To get it right
- 62. MAIN CD PRINCIPLES & PRACTICES How to?
- 63. Create a repeatable, reliable way to release software
- 64. Everybody is responsible for the delivery process
- 66. AUTOMATE ALMOST EVERYTHING The build Database changes Deployment to test/staging/production environments Tests Remediation plans Monitoring Infrastructure as code
- 67. THE DEPLOYMENT PIPELINE IS: 1) A Model of your process for getting software from version control into the hands of your users. 2) An Implementation that automates each stage of the process your software goes through after every change, from check-in to release – and it may also contain few manual stages such as approvals. 3) A Visualisation in real-time of the status of software code-base after every change, for all stages from check-in to release.
- 69. THE DEPLOYMENT PIPELINE BACKLOG PLANNING MEETING CODE & COMMIT BUILD UNIT TESTS INTEGRATION, ACCEPTANCE, … TESTS MANUAL APPROVAL CODE CHANGE READY TO GO LIVE ANATOMY OF A BASIC PIPELINE
- 70. AUTOMATE TESTS
- 72. AUTOMATE INFRASTRUCTURE If someone threw a server out of the window, how long would it take to recreate it?
- 73. CONCLUSIONS To get it right
- 74. LET’S FINISH FROM THE START: WHY CONTINUOUS DELIVERY?
- 75. We cannot always decide when/where change will come. But we can decide where each change will take us… …as long as we move faster than the change
- 76. WHAT IS CONTINUOUS DELIVERY? AGAIN Do me a sketch!
- 80. THANK YOU ! WAR STORIES, BOOKS, & SLIDES WILL BE ON TWITTER @SmHarterLTD LOOKING FOR A - CD ASSESSMENT - CD TRAINING EMAIL ME AT: LUCA.MINUDEL @ SMHARTER.COM
Editor's Notes
- Speech notes -------------------------------- Used for 20 years The only major upgrade I know of was a new color. industry trend of shortening products’ life cycles, moving from years and months to weeks and days. This phone was in use for 20 years in Italy. From the 60’. The only major upgrade I know of was a new color. My smartphone is 18 months old, I receive apps updates almost every day, I already updated firmware twice, and 2 new models has been presented since I bought it. The Industry is constantly shortening products’ life cycles. CD enables IT to lead this trend, release cycles are moving from years and months to weeks and days or even hours. Extra notes: Industry trend of shortening products’ life cycles, more and more teams and organizations release-cycles are moving from years and months to weeks and days.
- Speech notes -------------------------------- Main question here is: is CD beneficial to your business? How? I’ll share answer I found in my experience. I encourage you to find your own answers for your specific context.
- Speech notes -------------------------------- A vintage test card of Italy's public broadcasting company:RAI From the name, RAI, the initials of the 3 main benefits I observed:
- Speech notes -------------------------------- - from risky painful releases to predictable, safe and regular events => a release become a non-event - marketing events decoupled by tech releases, without risks of failure CD turns risky painful releases to production into predictable, safe and regular events or non events. from a technical point of view, they can still be marketing events but without risks and anxiety. Thanks to early and frequent releases, it is possible to see if the organization is building the right thing in the right way
- - More Features implemented over time - No value created until delivery For all this time we don’t know if features are what user wanted, if they solve user’s problem, or if there is a tech problem we keep going without knowing it, until we release
- Speech notes -------------------------------- - Every feature is released early and it create value - Problems are detected early, when it is easier and cheaper to fix them. - The consequences of a mistake are limited by the small scope and size of each incremental release For Anyone => preferred option Early adopters Requestor of the feature Internal users
- Speech notes -------------------------------- Remaining 10% get almost never completed problems discovered only late during integration or hardening release to production followed by weeks of expensive bug-fixing and firefighting With CD instead real progress becomes - Visible - Tangible Have you ever be accountable for the outcome of a project? If so you probably experienced the ‘90% done’ syndrome, when the gant chart reach the 90% progress and then the remaining 10% get almost never completed. It usually happens when deadlines loom, all certainties about the real progress of a project disappear. Some other times significant problems are discovered only late into the integration or hardening phase. §Other times it is the release to production that becomes a lengthy and painful experience, followed by weeks of bug-fixing and firefighting. With CD instead the real progress of the release becomes Visible Tangible in every moment
- Speech notes -------------------------------- For Anyone => preferred option Early adopters Requestor of the feature Internal users
- Speech notes -------------------------------- Many companies I visited in EU and US told me about misalignment they experience between IT and the business. Sometime IT and the business were blaming each other.
- Speech notes -------------------------------- Sometime even between different department inside IT there’s misalignment. Look at this picture
- Speech notes -------------------------------- - reduces the distance between business – customers/market => a checkpoint at every release, many chances to steer - Support mutual understanding business – IT => software dev become tangible when released => closer collaboration around each “feature” implemented CD spurs closer communication, mutual understanding, synchronization and cooperation between business and IT, and between all roles and departments involved in the project. CD also reduces the distance between the business and the customers. Furthermore, CD closes the gap between market, business, and IT.
- Speech notes -------------------------------- Life before Google maps, explorers like Cock or Magellano or Colombo Nowadays exploring new markets, new technologies, new domains and applications increase organization’s responsiveness: it reduces the time required to react to new circumstances. Before Google maps people could get lost and large parts of this planet were unknowns. There were explorers like Cock or Magellano or Colombo that planned the expeditions to cross the edge of the known and discover new places. Nowadays we are still - Exploring new markets - Experiment new technologies - Discovering new domains and applications CD support this exploration, make it easier and faster CD can increase organization’s responsiveness: it reduces the time required to react to new circumstances.
- Speech notes -------------------------------- When I joined Ferrari F1 racing team to help advancing the adoption of Lean and Agile practices, we have been asked to go faster and safer. You don’t want a bug to compromise the result of a race or the championship. It seems a paradox. Actually CD helped here.
- Speech notes -------------------------------- CD resolve the tension between ITops and Prod/SW Dev When IT Ops wins, When Prod Dev wins, Instead collaboration that increasing the throughput + stabilizing production systems, increasing innovation + predictability This is another apparent paradox. On the right: IT operations aims to keep the production systems up and running in a stable way. On the left: Product development aims to release new, live features to stay ahead of the competition. These two goals have traditionally been seen as conflicting, like in a zero-sum game. When Prod Dev wins, devs throws releases over the wall and IT Ops release them and pray and sometime start with firefighting. When IT Ops wins, organization implement governance standards as SOX, PCI DSS, ITIL, COBIL in a very bureaucratic way that slow down new releases and development CD instead has the paradoxical effect of increasing the throughput of new features and stabilizing production systems, increasing innovation and predictability together at the same time. Extra notes: Pressure to change ○ pressure to innovate (i.e. stay ahead of competitors); ○ technical novelty and variety in the technologies used for the project; ○ uncertainty (e.g. from the domain, the market, the users, the partners, the suppliers) Need for stability ○ interruption or disruption of services provided cause significant loss of money; ○ services provided are life-critical CD also helps with the compliance to standards such as SOX, PCI DSS, ITIL, and COBIT. And in many cases, problems with external policies are due to implementations of an interpretation of a policy, while there are other, lightweight ways to achieve the policy goals that actually produce better outcomes.
- Let’s look at implementing cd, starting with prerequisites, and checking how much cd are we already doing
- Speech notes -------------------------------- - Let’s start with prerequisites to CD! After a public workshop a CTO told me he recognized many of the problems I described during the day and that solutions proposed made lot of sense to him. He also told that the IT dep was not ready to adopt those advanced practices so he asked where they could start to prepare for CD?
- Speech notes -------------------------------- 2 prereqs Each build on top of the other, Iterative dev and CI Jim Highsmith, one of the 2 authors of the Agile Manifesto that work in TW (the other is Fowler) suggest this progression: Start with iterative development Add CI Add CD Extra notes:
- Speech notes -------------------------------- At every iteration a new feature is created, an existing feature is improved/evolved, ready to be released experimented first in the 1930 in the 1990 its use became common. In 2000 US Department of Defense states a clear preference for Iterative development. Nowadays Waterfall and its evolutions and extensions such as V-Model are considered obsolete. And just-do-it approaches inadequate for teams bigger then 2
- Speech notes -------------------------------- - incremental vs evolutionary (better) - After evolutionary and iterations shorter than 1 week => flow-pull systems such as Kanban
- - After CI : Deploy & Test automation - That Includes architecture improvements to support automatic configuration for multiple environments (test, staging, prod) Design to make code testable Architecture to Enable hot deploys Automate remediation plans
- Mentioned 1st time 1991 (by Grady Booch) 1st article on CI in 2000, focus on devs integrating, not on tool Here definition from MF
- Build and automated tests run and return a report in 5-15mins
- Speech notes -------------------------------- Can skip
- Merge in mainline are so frequent that branches are rare exceptions. Features not finished are hidden from the front-end (with feature toggles), and back-end code is tested for not breaking existing functionalities (latent code, branch by abstraction).
- When devs works on different branches
- Each branch accumulates overtime lot of potentially conflicting or interfering changes
- - Only when devs start the merge war, conflicts and interferences are spot, - resolving merge conflicts is painful and time-consuming, when not impossible
- With frequent merges on the mainline, conflicts are less likely, and very easy to fix with a quick chat Interferences are spot immediately what they are cheaper to fix
- Speech notes -------------------------------- After the Whys we can talk about the whats. What is CD than ?
- Speech notes -------------------------------- CD has 3 fundamental ingredients. People/social ingredient is the most underestimated and misunderstood ingredient.
- - Def by Martin Fowler - Can you release in production at any time during the sprint/iteration ?
- Def by Jez Humble, the guy how wrote the book. notice this business point of view
- Speech notes -------------------------------- - Let’s do a quick self-assessment (a proper assessment take between 1 and 3, but this can give us a general idea of there your team/org stands)
- Speech notes -------------------------------- 1 – from the first to the last iteration of your release, anytime during any iteration 2 – Let’s make an example, I’m pushing a feature that needs a new db table: db change script and update to the remediation plan
- Speech notes -------------------------------- - typically in 5 to 15 minutes you get a report with the results - When you do manual QA and it takes days you cannot get this fast feedback
- Speech notes -------------------------------- - Anyone can do it (QA, Bus stakeholder, PO/PM), it’s completely automatic, no need for manual steps - even with db changes, configuration changes, etc.
- 1-2 weeks for QA => Less than 1 day for QA
- they improved the productivity of developers by 2-3x, Like in the consumer printer market, it was incredibly competitive, where new and innovative offerings were showing up nearly every month. Their problem was that the firmware group was having tremendous difficulty keeping up with the demand for new, innovative features,
- - Simplification reduced planning effort from 20% to 5%, one of the biggest saving Build cycle time: 1 week -> 3 hours (10-15 builds per day)
- Green field product release
- - Things easy to achieve by a small group of people, and everyone benefit from them after they are done - Basic remediation plan = only for backward compatible changes
- - Changes that involve anyone (i.e. keep existing tests working when changing a feature, create remediation plan also for backward incompatible changes)
- Speech notes -------------------------------- Do companies need to hire specifically for DevOps? Extra notes:
- Speech notes -------------------------------- DevOps doesn't require particular roles; in fact, many believe that creating a separate role for DevOps misses the point that developers and operations people should share ownership of the software, end to end. DevOps extends collaboration to include infrastructure, release management, support, security, and other roles in IT operations. Extra notes:
- Speech notes -------------------------------- This is the traditional divide between devs and IT Ops Different skillset, language, bosses
- Speech notes -------------------------------- Solution: A IT Ops joining dev team since the beginning of a new request to add requirements related to deployment automation and support A dev joining IT Ops team to help automate the release and to help with support They rotate, build a common language and mutual understanding and so collaboration
- Speech notes -------------------------------- This misses the point Moves devs and IT Ops even more far away
- Speech notes -------------------------------- Companies often ask for advises on picking the right tool
- Speech notes -------------------------------- This is an example of different types of tools and some tools I’ll give you soon a reference with a complete list of tool Extra notes:
- Speech notes -------------------------------- Kief Morris, author of Infrastructure as code book
- Speech notes -------------------------------- A successful product will grow – maybe tool will be changed Good separation of responsibility between tool and script/automation you create A successful product will grow over time and so will the related code-base. The implementation of CD will thus have to evolve too. In order to support this growth easily, a CD tool should implement the pipeline concept as a native first-class concept. It should be cross-platform compatible to support a large variety of technologies. And it should be able to distribute the build tasks across many computers, as in a build-grid, to support increased workload. It should support scripting to enable refactoring and support VCS. And finally it should follow a clear separation of concerns to enable a proper integration of different tools and to allow easy replacement of tools over time when needed. Extra notes:
- Speech notes -------------------------------- Companies that are transitioning to Agile and Lean ask what’s the relation with CD and Agile and Lean, are they compatible? They support each other
- Can skip
- Speech notes -------------------------------- Let’s have an overview of the main principles and practices
- Speech notes -------------------------------- All remaining principles derive from this main principle. The same for bug fixes, new features, new products, or emergencies This main principle incorporates the whole idea of CD: create a repeatable, reliable way to release software.
- Speech notes -------------------------------- Everybody Devs, IT Ops, Marketing, etc. is responsible for making sure the software constantly and increasingly generates business value. Have the sw fulfill the real needs of the users keeping the software functioning well in production Extra notes: All the people involved throughout the value stream of the product, from concept to cash, should share this common goal and be evaluated as a team against this common goal. This is essential to creating an environment where communication and collaboration patterns required to succeed are possible and likely to happen.
- Speech notes -------------------------------- The key idea is to automate the recovery after a bad release into production. When a new version of an application is released into production but starts to malfunction or users report a showstopper bug, different techniques can help to remedy the problem. Challenge All the cases where there are 2 moving parts Data bases double challenge because the size of the data Decouple the deploy of the moving parts for example with Rollbacks and forward-compatible interim versions. Extra notes:
- Can skip
- Speech notes -------------------------------- Test/stage envs equal to prod Safely replace prod servers/infra Easy support for many nodes Automate the creation of test/prod environments Networking (DNS, etc) Web/mail servers Etc Infrastructure code. The goal of infrastructure automation and infrastructure as code is to have environments that are always in a well-known state and to avoid unnecessary variations that can invalidate the result of the tests or be a source of bugs in production that are very hard to reproduce. And it become quick and easy to add or replace a server and to create a test environment that is as similar as possible to the prod one. Extra notes: Infrastructure automation is intended to automatically create from scratch every test and production environment, from the operating system to the installation and configuration of the required software and services (networking, DNS, Web server, email server, etc.), and to automatically apply all the configurations expected for the environment.
- Speech notes -------------------------------- Extra notes: