SlideShare ist ein Scribd-Unternehmen logo
1 von 53
Downloaden Sie, um offline zu lesen
Vladimir Bakhov
AT-Consulting
vbakhov@at-consulting.ru
+7 (905) 7165446

Svetlana Panfilova
AT-Consulting
spanfilova@at-consulting.ru
+7 (903) 1696490

Google group for this presentation is vobaks
Source codes are available at Google Code: Project OraCI.(
http://code.google.com/p/oraci/ )

P.S. Since I mostly use the Google Translate service, I’d be really happy if someone
could verify the English translation.




                                                                                       1
Q: What is Continuous Integration?
A: System to make developer’s life much more comfortable!

Continuous integration aims to improve the quality of software, and to reduce the
time taken to deliver it, by replacing the traditional practice of applying quality
control after completing all development. CI comes in compliance with agile dogmas
about early, automated and continuous quality assurance.




                                                                                      2
Let’s look at what we had before CI




                                      3
4
5
6
7
8
9
10
11
12
Disadvantages
initial setup time required
well-developed test-suite required to achieve automated testing advantages
large-scale refactoring can be troublesome due to continuously changing code base
hardware costs for build machines can be significant
Many teams using CI report that the advantages of CI well outweigh the
disadvantages. The effect of finding and fixing integration bugs early in the
development process saves both time and money over the lifespan of a project

http://en.wikipedia.org/wiki/Continuous_integration




                                                                                    13
Application Value is in source code and configuration files
Database      Not only the source code is valuable, but the current data

P.S. By application I mean non-database project artifacts – programs by Java, .NET
etc.




                                                                                     14
Application
Deployment is done by replacing old binary files for new ones.


Database
While deploying new database release, not only source code must be replaced, but
also scripts for structural changes and data migrations must be run. To keep integrity
constrains all deployment scripts must be run in strict order.




                                                                                         15
Application
Easy to use (generally integrated in IDE) Version Control repository with well
documented development methodology.

Database
Fewer teams use version control for large database development. No general rules
for e.g. branching or multi-user development.




                                                                                   16
Application Roll back to previous version is generally easy.

Database
Roll back to previous version needs costly development of manual script, which is
dependable on current version of database and its data.
Sometimes it’s easy. In some cases roll back is almost impossible.
(e.g. bank can afford only one downtime window for 8-hours release deployment
with hard data migrations. Since users start working and populating new data
structures you simple cannot drop their changes)




                                                                                    17
Application Plenty of out of box continuous integration solutions. Some of them
may automate your deployment process within several mouse clicks.

Database There is no general support for database development.
Most database development teams use manual or semi-automated process to
produce deployment scripts.




                                                                                  18
19
This is the most important slide.
Database CI is relatively complex development tool.
Good news is You can learn many wonderful things in life.
Does not matter how huge and complicated you database is – continuous integration
is beautiful technology you can master.




                                                                                    20
21
22
You must have test automation. Otherwise CI has no much sense. We use UTPLSql
for oracle unit testing. You also can use Toad Code Tester, SQL Unit with SQL
Developer or self-coded tool. Test automation for DB is topic in it’s own. Later we
created highly flexible tool with Web GUI for large ETL processes test automation.
For now it supports ETL systems with Informatica, Oracle Warehouse builder and SAS
Data Integration Studio.




                                                                                      23
CI software is needed for automated deployment scripts building, their deployment
and testing with test automation systems. It also integrates nice reporting tools
about build health.




                                                                                    24
Version Control is the blood for Database CI.
I will demonstrate on Subversion as our favorite VCS. Likely there is no restriction to
use other VCS of your choice.




                                                                                          25
Well documented VSC project structures won’t seamlessly fit database development
projects.
We will demonstrate required adaptation to enjoy all CI benefits.




                                                                                   26
All actions are scripted (No changes with GUI, e.g. adding column or comment by
SQL Navigator GUI).
All database objects should be scripted as sql: packages, procedures, views, triggers
etc. Database structure changes and data migrations should be as sql files: table
creations, alter operations, insert operations etc. Scripts – general project artifact.
  Scripts should be checked into Version Control System
  Scripts have owners, and could be assigned and maintained
  Scripts are testable
  Contrary to manual actions, scripts are safe to deploy&test multiple times
  Scripts can be deployed on multiple databases and environments
  Scripts can be linked with Bug Tracking system (like Jira). You can see what objects
were changed within particular task… by whom and when.




                                                                                          27
If Live Demo is not applicable then the basic concept will appear at slides 33-36.




                                                                                     28
History of all changes is in the TRUNK.
TRUNK – single database master-copy




                                          29
Convenient for developers – code objects, structure and data change scripts are
not spread by different release folders. You don’t need to remember where are
they now or look for them. To start development you always go to TRUNC, open
needed source file (where files are structured by schema and db object type).
Then you do the coding, compiling and testing as usual, just simple press “save”
before exit.
When you start practicing branched development the system will automatically
control already deployed objects or structure or data change scripts. Thus
nothing will be lost or deployed twice.




                                                                                   30
31
Day.
Iterative development. Commiting of project artifacts to version control system

Night.
1) Prodlike FlashBack() or Cold Backup(). Backup of all changes. Return of Prodlike
   environment to state fully identical to current production environment
2) Automated generation of deployment script as a difference of SVN «Prod» and
   «Trunk» folders. Files in this difference must be run in strictly predefined order.
   The order is important part of this methodology. This difference goes to «Patch»
   folder
3) Automated deployment to ProdLike environment
4) Run of automated testing
5) Deployment script goes to release repository.
6) Report about build health (succeeded deployment and tests) is published at the
   CI web dashboard
7) Involved members get e-mails about build health or founded bugs




                                                                                         32
Get reference Prodlike DB first.
           Export/import production parts by impdp utility
           From production backup
Clean it. Reduce complexity and number of db objects to dramatically increase
deployment time and database size (removing all but one partition per table and
so on)
Save new light dump to SVN «dump» folder. The ProdLike DB is ready for intensive
use within development and CI processes.




                                                                                   33
«Prod» folder keeps the current production code. This folder is used for comparing
in deployment script generation process.
Structure of «trunk» folder is similar to «prod». The development take place in
«trunk».
At the point zero «trunk» and «prod» folders are exact copies.

To save production DDL code to SVN we use UTL_FILE. Each object has its own file.
  Types, Type Bodies. Creating and droping types in correct order. Drop cascade (by
dba_dependencies), then created according to their dependencies. Grants for them.
All types are in one file thus redeployed all together if needed. Dependable types is
actually the hardest part.
   Triggers. View triggers and materialized view triggers must be removed and be
part of view and mat.view source code.
  Functions, Procedures, Packages. Also their grants.
  Views, also their triggers and grants.
  Materialized Views. also their triggers, indexes and grants.
  Package Bodies.
  …




                                                                                        34
By executing svn «diff» command we are getting difference of «prod» and «trunk»
in changed files
«export» command over «diff» results gets us something very close to
deployment script. Regularly we export to «patch» folder.
«patch» folder structure is identical to «prod» and «trunk». It now contains only
changed files and not deployed «before» and «after» objects.




                                                                                    35
ANT: SQLPLUS calls all files of «patch» folder tagged to «release x.x…» folder in very
strict order, so that any our work is deployable.

There are few ways to actually deploy release according to your situation


  Sequential deployment of all sql files in «release x.x..» folder. Files are sorted by
  subfolder and names. In this case deployment administrator must use svn, call ant
  for deployment and set deployment params (like sid, login, pass) with properties
  file (build.properties).
  Concatination of all sorted sql-files to one big “release.sql”. Deployment
  administrator now can just run it with sqlplus. No need for extra actions with ant
  or svn. It’s best suited for outsourced development.


Features of deployment with ant

  To deploy sql-files by sqlplus we use incanto pluging for ant. It gives us portability
  to different environments.
  Some files may be removed from release by their mask. It can be done by using
  attribute excludes of concat and fileset tasks. You can exclude autotesting




                                                                                           36
packages, which are good only for testing and CI environments.




                                                                 36
Connect you ant deployment algorithm with svn and any CI software.
Connect you autotests to CI software(we use maven-utplsql-plugin for you TeemCity
CI software and UTPLsql autotests).
Enjoy you automation!




                                                                                    37
38
39
40
Actually You don’t have to do all the tests. Your autotesting system can get all
changed objects and track their dependencies. Then it can test only by applicable
tests. In oracle you do this with dba_dependencies, with ETL software you have to
track dependencies by their XML or otherwise.




                                                                                    41
Optimize your autotests
        Use lightweight synthetic data
        Automate generation of synthetic data
Make build and test fast. Invest in infrastructure
Use CI software capabilities to run parallel builds on multiple machines (Build
Agents)




                                                                                  42
43
Developer can build his own light version of database from dump files and
deployment scripts from version control system. He can do it in any needed
environment for branched development and testing.




                                                                             44
Do continuous integration.
Try to switch from nightly builds to more frequent build (e.g. every 2 hours). Then to
build&test by every commit to source control.
Try to separate your quick and slow test. Slow for weekends, quick for now. Track
dependencies of changed objects and tests.




                                                                                         45
46
47
48
http://en.wikipedia.org/wiki/Continuous_integration




                                                      49
50
51
52

Weitere ähnliche Inhalte

Was ist angesagt?

DBmaestro's State of the Database Continuous Delivery Survey- Findings Revealed
DBmaestro's State of the Database Continuous Delivery Survey- Findings RevealedDBmaestro's State of the Database Continuous Delivery Survey- Findings Revealed
DBmaestro's State of the Database Continuous Delivery Survey- Findings RevealedDBmaestro - Database DevOps
 
Why retail companies can't afford database downtime
Why retail companies can't afford database downtimeWhy retail companies can't afford database downtime
Why retail companies can't afford database downtimeDBmaestro - Database DevOps
 
Continuous Delivery & the Database- The Final Frontier
Continuous Delivery & the Database- The Final FrontierContinuous Delivery & the Database- The Final Frontier
Continuous Delivery & the Database- The Final FrontierDBmaestro - Database DevOps
 
Continuous Delivery & the Database - the Final Frontier
Continuous Delivery & the Database - the Final FrontierContinuous Delivery & the Database - the Final Frontier
Continuous Delivery & the Database - the Final FrontierXebiaLabs
 
Delphix and DBmaestro
Delphix and DBmaestroDelphix and DBmaestro
Delphix and DBmaestroKyle Hailey
 
Automate Database Deployment - SQL In The City Workshop
Automate Database Deployment - SQL In The City WorkshopAutomate Database Deployment - SQL In The City Workshop
Automate Database Deployment - SQL In The City WorkshopRed Gate Software
 
The Future of Change Management and DevOps for Dummies
The Future of Change Management and DevOps for DummiesThe Future of Change Management and DevOps for Dummies
The Future of Change Management and DevOps for DummiesDBmaestro - Database DevOps
 
DevOps Patterns Distilled: Implementing The Needed Practices In Practical Steps
DevOps Patterns Distilled: Implementing The Needed Practices In Practical StepsDevOps Patterns Distilled: Implementing The Needed Practices In Practical Steps
DevOps Patterns Distilled: Implementing The Needed Practices In Practical StepsCA Technologies
 
Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...
Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...
Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...Simplilearn
 
CI/CD Best Practices for Your DevOps Journey
CI/CD Best  Practices for Your DevOps JourneyCI/CD Best  Practices for Your DevOps Journey
CI/CD Best Practices for Your DevOps JourneyDevOps.com
 
Continuous Delivery Distilled
Continuous Delivery DistilledContinuous Delivery Distilled
Continuous Delivery DistilledMatt Callanan
 
Challenges and best practices of database continuous delivery
Challenges and best practices of database continuous deliveryChallenges and best practices of database continuous delivery
Challenges and best practices of database continuous deliveryDBmaestro - Database DevOps
 
Unified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOpsUnified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOpsXebiaLabs
 
Iltam database version control
Iltam database version controlIltam database version control
Iltam database version controluridbmaestro
 
Understanding DevOps in simpler way with Continuous Delivery
Understanding DevOps in simpler way with Continuous DeliveryUnderstanding DevOps in simpler way with Continuous Delivery
Understanding DevOps in simpler way with Continuous DeliverySwapnil Jain
 

Was ist angesagt? (20)

DBmaestro's State of the Database Continuous Delivery Survey- Findings Revealed
DBmaestro's State of the Database Continuous Delivery Survey- Findings RevealedDBmaestro's State of the Database Continuous Delivery Survey- Findings Revealed
DBmaestro's State of the Database Continuous Delivery Survey- Findings Revealed
 
Why retail companies can't afford database downtime
Why retail companies can't afford database downtimeWhy retail companies can't afford database downtime
Why retail companies can't afford database downtime
 
Continuous Delivery & the Database- The Final Frontier
Continuous Delivery & the Database- The Final FrontierContinuous Delivery & the Database- The Final Frontier
Continuous Delivery & the Database- The Final Frontier
 
Continuous Delivery & the Database - the Final Frontier
Continuous Delivery & the Database - the Final FrontierContinuous Delivery & the Database - the Final Frontier
Continuous Delivery & the Database - the Final Frontier
 
Delphix and DBmaestro
Delphix and DBmaestroDelphix and DBmaestro
Delphix and DBmaestro
 
Automate Database Deployment - SQL In The City Workshop
Automate Database Deployment - SQL In The City WorkshopAutomate Database Deployment - SQL In The City Workshop
Automate Database Deployment - SQL In The City Workshop
 
The Future of Change Management and DevOps for Dummies
The Future of Change Management and DevOps for DummiesThe Future of Change Management and DevOps for Dummies
The Future of Change Management and DevOps for Dummies
 
Unlocking the Potential of Database Automation
Unlocking the Potential of Database AutomationUnlocking the Potential of Database Automation
Unlocking the Potential of Database Automation
 
DevOps Patterns Distilled: Implementing The Needed Practices In Practical Steps
DevOps Patterns Distilled: Implementing The Needed Practices In Practical StepsDevOps Patterns Distilled: Implementing The Needed Practices In Practical Steps
DevOps Patterns Distilled: Implementing The Needed Practices In Practical Steps
 
DevOps introduction
DevOps introductionDevOps introduction
DevOps introduction
 
Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...
Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...
Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...
 
CI/CD Best Practices for Your DevOps Journey
CI/CD Best  Practices for Your DevOps JourneyCI/CD Best  Practices for Your DevOps Journey
CI/CD Best Practices for Your DevOps Journey
 
Continuous Delivery Distilled
Continuous Delivery DistilledContinuous Delivery Distilled
Continuous Delivery Distilled
 
Challenges and best practices of database continuous delivery
Challenges and best practices of database continuous deliveryChallenges and best practices of database continuous delivery
Challenges and best practices of database continuous delivery
 
Unified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOpsUnified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOps
 
"DevOps > CI+CD "
"DevOps > CI+CD ""DevOps > CI+CD "
"DevOps > CI+CD "
 
Iltam database version control
Iltam database version controlIltam database version control
Iltam database version control
 
Top DevOps tools
Top DevOps toolsTop DevOps tools
Top DevOps tools
 
A True Story of Why QA Loves DevOps
A True Story of Why QA Loves DevOpsA True Story of Why QA Loves DevOps
A True Story of Why QA Loves DevOps
 
Understanding DevOps in simpler way with Continuous Delivery
Understanding DevOps in simpler way with Continuous DeliveryUnderstanding DevOps in simpler way with Continuous Delivery
Understanding DevOps in simpler way with Continuous Delivery
 

Andere mochten auch

Continuous Integration - Oracle Database Objects
Continuous Integration - Oracle Database ObjectsContinuous Integration - Oracle Database Objects
Continuous Integration - Oracle Database ObjectsPrabhu Ramasamy
 
Using puppet to leverage DevOps in Large Enterprise Oracle Environments
Using puppet to leverage DevOps in Large Enterprise Oracle Environments Using puppet to leverage DevOps in Large Enterprise Oracle Environments
Using puppet to leverage DevOps in Large Enterprise Oracle Environments Bert Hajee
 
Edition Based Redefinition - Continuous Database Application Evolution with O...
Edition Based Redefinition - Continuous Database Application Evolution with O...Edition Based Redefinition - Continuous Database Application Evolution with O...
Edition Based Redefinition - Continuous Database Application Evolution with O...Lucas Jellema
 
Why source control your Oracle Database?
Why source control your Oracle Database?Why source control your Oracle Database?
Why source control your Oracle Database?Red Gate Software
 
Creep Coursework Presentation
Creep Coursework PresentationCreep Coursework Presentation
Creep Coursework Presentationkess1a
 
портфолио голубович
портфолио голубовичпортфолио голубович
портфолио голубовичgolubovicholga
 
Psicopedagoga rj.com.br Cadastro
Psicopedagoga rj.com.br   CadastroPsicopedagoga rj.com.br   Cadastro
Psicopedagoga rj.com.br CadastroPsicopedagogaRJ
 
Copia de resumen qué son los mapas conceptuales.doc%0 a
Copia de resumen qué son los mapas conceptuales.doc%0 aCopia de resumen qué son los mapas conceptuales.doc%0 a
Copia de resumen qué son los mapas conceptuales.doc%0 anoeliavillar
 
Pre Production (Planning)
Pre Production (Planning)Pre Production (Planning)
Pre Production (Planning)Rahul Karavadra
 
Context Based Learning for GIS: an Interdisciplinary Approach
Context Based Learning for GIS: an Interdisciplinary ApproachContext Based Learning for GIS: an Interdisciplinary Approach
Context Based Learning for GIS: an Interdisciplinary ApproachPatrick Rickles
 
Nature and animal conservation by art
Nature and animal conservation by artNature and animal conservation by art
Nature and animal conservation by artART Raviteja akarapu
 
Twenty is Plenty
Twenty is PlentyTwenty is Plenty
Twenty is PlentyBob Ward
 
SQL Server R Services: What Every SQL Professional Should Know
SQL Server R Services: What Every SQL Professional Should KnowSQL Server R Services: What Every SQL Professional Should Know
SQL Server R Services: What Every SQL Professional Should KnowBob Ward
 
Qué fue el holocausto judío
Qué fue el holocausto judío Qué fue el holocausto judío
Qué fue el holocausto judío Yamila Valdez
 
SQL Server It Just Runs Faster
SQL Server It Just Runs FasterSQL Server It Just Runs Faster
SQL Server It Just Runs FasterBob Ward
 

Andere mochten auch (20)

Continuous Integration - Oracle Database Objects
Continuous Integration - Oracle Database ObjectsContinuous Integration - Oracle Database Objects
Continuous Integration - Oracle Database Objects
 
DevOps in your Oracle Stack
DevOps in your Oracle StackDevOps in your Oracle Stack
DevOps in your Oracle Stack
 
Using puppet to leverage DevOps in Large Enterprise Oracle Environments
Using puppet to leverage DevOps in Large Enterprise Oracle Environments Using puppet to leverage DevOps in Large Enterprise Oracle Environments
Using puppet to leverage DevOps in Large Enterprise Oracle Environments
 
Edition Based Redefinition - Continuous Database Application Evolution with O...
Edition Based Redefinition - Continuous Database Application Evolution with O...Edition Based Redefinition - Continuous Database Application Evolution with O...
Edition Based Redefinition - Continuous Database Application Evolution with O...
 
Why source control your Oracle Database?
Why source control your Oracle Database?Why source control your Oracle Database?
Why source control your Oracle Database?
 
Creep Coursework Presentation
Creep Coursework PresentationCreep Coursework Presentation
Creep Coursework Presentation
 
портфолио голубович
портфолио голубовичпортфолио голубович
портфолио голубович
 
Psicopedagoga rj.com.br Cadastro
Psicopedagoga rj.com.br   CadastroPsicopedagoga rj.com.br   Cadastro
Psicopedagoga rj.com.br Cadastro
 
Copia de resumen qué son los mapas conceptuales.doc%0 a
Copia de resumen qué son los mapas conceptuales.doc%0 aCopia de resumen qué son los mapas conceptuales.doc%0 a
Copia de resumen qué son los mapas conceptuales.doc%0 a
 
Pre Production (Planning)
Pre Production (Planning)Pre Production (Planning)
Pre Production (Planning)
 
Context Based Learning for GIS: an Interdisciplinary Approach
Context Based Learning for GIS: an Interdisciplinary ApproachContext Based Learning for GIS: an Interdisciplinary Approach
Context Based Learning for GIS: an Interdisciplinary Approach
 
CopyofAResume
CopyofAResumeCopyofAResume
CopyofAResume
 
Portfolio Draft
Portfolio DraftPortfolio Draft
Portfolio Draft
 
Nature and animal conservation by art
Nature and animal conservation by artNature and animal conservation by art
Nature and animal conservation by art
 
Twenty is Plenty
Twenty is PlentyTwenty is Plenty
Twenty is Plenty
 
ckitterman resume
ckitterman resumeckitterman resume
ckitterman resume
 
SQL Server R Services: What Every SQL Professional Should Know
SQL Server R Services: What Every SQL Professional Should KnowSQL Server R Services: What Every SQL Professional Should Know
SQL Server R Services: What Every SQL Professional Should Know
 
Qué fue el holocausto judío
Qué fue el holocausto judío Qué fue el holocausto judío
Qué fue el holocausto judío
 
Accounting concepts
Accounting conceptsAccounting concepts
Accounting concepts
 
SQL Server It Just Runs Faster
SQL Server It Just Runs FasterSQL Server It Just Runs Faster
SQL Server It Just Runs Faster
 

Ähnlich wie Continuous Integration for Oracle Database Development

Devops phase-1
Devops phase-1Devops phase-1
Devops phase-1G R VISHAL
 
Kelly potvin nosurprises_odtug_oow12
Kelly potvin nosurprises_odtug_oow12Kelly potvin nosurprises_odtug_oow12
Kelly potvin nosurprises_odtug_oow12Enkitec
 
Continuous Delivery: Fly the Friendly CI in Pivotal Cloud Foundry with Concourse
Continuous Delivery: Fly the Friendly CI in Pivotal Cloud Foundry with ConcourseContinuous Delivery: Fly the Friendly CI in Pivotal Cloud Foundry with Concourse
Continuous Delivery: Fly the Friendly CI in Pivotal Cloud Foundry with ConcourseVMware Tanzu
 
Achieving Full Stack DevOps at Colonial Life
Achieving Full Stack DevOps at Colonial Life Achieving Full Stack DevOps at Colonial Life
Achieving Full Stack DevOps at Colonial Life DevOps.com
 
Relational Database CI/CD
Relational Database CI/CDRelational Database CI/CD
Relational Database CI/CDJasmin Fluri
 
Using Containers to More Effectively Manage DevOps Continuous Integration
Using Containers to More Effectively Manage DevOps Continuous IntegrationUsing Containers to More Effectively Manage DevOps Continuous Integration
Using Containers to More Effectively Manage DevOps Continuous IntegrationCognizant
 
The world of Docker and Kubernetes
The world of Docker and Kubernetes The world of Docker and Kubernetes
The world of Docker and Kubernetes vty
 
Containers: DevOp Enablers of Technical Solutions
Containers: DevOp Enablers of Technical SolutionsContainers: DevOp Enablers of Technical Solutions
Containers: DevOp Enablers of Technical SolutionsJules Pierre-Louis
 
Continuous Database Delivery - 7/12/2018
Continuous Database Delivery - 7/12/2018Continuous Database Delivery - 7/12/2018
Continuous Database Delivery - 7/12/2018David P. Moore
 
DevOps to DevSecOps Journey..
DevOps to DevSecOps Journey..DevOps to DevSecOps Journey..
DevOps to DevSecOps Journey..Siddharth Joshi
 
Continuous Integration & Continuous Delivery
Continuous Integration & Continuous DeliveryContinuous Integration & Continuous Delivery
Continuous Integration & Continuous DeliveryDatabricks
 
Slide DevSecOps Microservices
Slide DevSecOps Microservices Slide DevSecOps Microservices
Slide DevSecOps Microservices Hendri Karisma
 
CNCF Introduction - Feb 2018
CNCF Introduction - Feb 2018CNCF Introduction - Feb 2018
CNCF Introduction - Feb 2018Krishna-Kumar
 
Presentation 1 open source tools in continuous integration environment v1.0
Presentation 1   open source tools in continuous integration environment v1.0Presentation 1   open source tools in continuous integration environment v1.0
Presentation 1 open source tools in continuous integration environment v1.0Jasmine Conseil
 
SDLC & DevOps Transformation with Agile
SDLC & DevOps Transformation with AgileSDLC & DevOps Transformation with Agile
SDLC & DevOps Transformation with AgileAbdel Moneim Emad
 
Intro to DevOps 4 undergraduates
Intro to DevOps 4 undergraduates Intro to DevOps 4 undergraduates
Intro to DevOps 4 undergraduates Liran Levy
 
Agile Network India | Continuous Integration & Continuous Deployment & Automa...
Agile Network India | Continuous Integration & Continuous Deployment & Automa...Agile Network India | Continuous Integration & Continuous Deployment & Automa...
Agile Network India | Continuous Integration & Continuous Deployment & Automa...AgileNetwork
 
hcp-as-continuous-integration-build-artifact-storage-system
hcp-as-continuous-integration-build-artifact-storage-systemhcp-as-continuous-integration-build-artifact-storage-system
hcp-as-continuous-integration-build-artifact-storage-systemIngrid Fernandez, PhD
 

Ähnlich wie Continuous Integration for Oracle Database Development (20)

Devops phase-1
Devops phase-1Devops phase-1
Devops phase-1
 
Kelly potvin nosurprises_odtug_oow12
Kelly potvin nosurprises_odtug_oow12Kelly potvin nosurprises_odtug_oow12
Kelly potvin nosurprises_odtug_oow12
 
Continuous Delivery: Fly the Friendly CI in Pivotal Cloud Foundry with Concourse
Continuous Delivery: Fly the Friendly CI in Pivotal Cloud Foundry with ConcourseContinuous Delivery: Fly the Friendly CI in Pivotal Cloud Foundry with Concourse
Continuous Delivery: Fly the Friendly CI in Pivotal Cloud Foundry with Concourse
 
Achieving Full Stack DevOps at Colonial Life
Achieving Full Stack DevOps at Colonial Life Achieving Full Stack DevOps at Colonial Life
Achieving Full Stack DevOps at Colonial Life
 
Relational Database CI/CD
Relational Database CI/CDRelational Database CI/CD
Relational Database CI/CD
 
Using Containers to More Effectively Manage DevOps Continuous Integration
Using Containers to More Effectively Manage DevOps Continuous IntegrationUsing Containers to More Effectively Manage DevOps Continuous Integration
Using Containers to More Effectively Manage DevOps Continuous Integration
 
The world of Docker and Kubernetes
The world of Docker and Kubernetes The world of Docker and Kubernetes
The world of Docker and Kubernetes
 
Containers: DevOp Enablers of Technical Solutions
Containers: DevOp Enablers of Technical SolutionsContainers: DevOp Enablers of Technical Solutions
Containers: DevOp Enablers of Technical Solutions
 
Continuous Database Delivery - 7/12/2018
Continuous Database Delivery - 7/12/2018Continuous Database Delivery - 7/12/2018
Continuous Database Delivery - 7/12/2018
 
Tour of Azure DevOps
Tour of Azure DevOpsTour of Azure DevOps
Tour of Azure DevOps
 
CICD with Jenkins
CICD with JenkinsCICD with Jenkins
CICD with Jenkins
 
DevOps to DevSecOps Journey..
DevOps to DevSecOps Journey..DevOps to DevSecOps Journey..
DevOps to DevSecOps Journey..
 
Continuous Integration & Continuous Delivery
Continuous Integration & Continuous DeliveryContinuous Integration & Continuous Delivery
Continuous Integration & Continuous Delivery
 
Slide DevSecOps Microservices
Slide DevSecOps Microservices Slide DevSecOps Microservices
Slide DevSecOps Microservices
 
CNCF Introduction - Feb 2018
CNCF Introduction - Feb 2018CNCF Introduction - Feb 2018
CNCF Introduction - Feb 2018
 
Presentation 1 open source tools in continuous integration environment v1.0
Presentation 1   open source tools in continuous integration environment v1.0Presentation 1   open source tools in continuous integration environment v1.0
Presentation 1 open source tools in continuous integration environment v1.0
 
SDLC & DevOps Transformation with Agile
SDLC & DevOps Transformation with AgileSDLC & DevOps Transformation with Agile
SDLC & DevOps Transformation with Agile
 
Intro to DevOps 4 undergraduates
Intro to DevOps 4 undergraduates Intro to DevOps 4 undergraduates
Intro to DevOps 4 undergraduates
 
Agile Network India | Continuous Integration & Continuous Deployment & Automa...
Agile Network India | Continuous Integration & Continuous Deployment & Automa...Agile Network India | Continuous Integration & Continuous Deployment & Automa...
Agile Network India | Continuous Integration & Continuous Deployment & Automa...
 
hcp-as-continuous-integration-build-artifact-storage-system
hcp-as-continuous-integration-build-artifact-storage-systemhcp-as-continuous-integration-build-artifact-storage-system
hcp-as-continuous-integration-build-artifact-storage-system
 

Kürzlich hochgeladen

Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 

Kürzlich hochgeladen (20)

Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 

Continuous Integration for Oracle Database Development

  • 1. Vladimir Bakhov AT-Consulting vbakhov@at-consulting.ru +7 (905) 7165446 Svetlana Panfilova AT-Consulting spanfilova@at-consulting.ru +7 (903) 1696490 Google group for this presentation is vobaks Source codes are available at Google Code: Project OraCI.( http://code.google.com/p/oraci/ ) P.S. Since I mostly use the Google Translate service, I’d be really happy if someone could verify the English translation. 1
  • 2. Q: What is Continuous Integration? A: System to make developer’s life much more comfortable! Continuous integration aims to improve the quality of software, and to reduce the time taken to deliver it, by replacing the traditional practice of applying quality control after completing all development. CI comes in compliance with agile dogmas about early, automated and continuous quality assurance. 2
  • 3. Let’s look at what we had before CI 3
  • 4. 4
  • 5. 5
  • 6. 6
  • 7. 7
  • 8. 8
  • 9. 9
  • 10. 10
  • 11. 11
  • 12. 12
  • 13. Disadvantages initial setup time required well-developed test-suite required to achieve automated testing advantages large-scale refactoring can be troublesome due to continuously changing code base hardware costs for build machines can be significant Many teams using CI report that the advantages of CI well outweigh the disadvantages. The effect of finding and fixing integration bugs early in the development process saves both time and money over the lifespan of a project http://en.wikipedia.org/wiki/Continuous_integration 13
  • 14. Application Value is in source code and configuration files Database Not only the source code is valuable, but the current data P.S. By application I mean non-database project artifacts – programs by Java, .NET etc. 14
  • 15. Application Deployment is done by replacing old binary files for new ones. Database While deploying new database release, not only source code must be replaced, but also scripts for structural changes and data migrations must be run. To keep integrity constrains all deployment scripts must be run in strict order. 15
  • 16. Application Easy to use (generally integrated in IDE) Version Control repository with well documented development methodology. Database Fewer teams use version control for large database development. No general rules for e.g. branching or multi-user development. 16
  • 17. Application Roll back to previous version is generally easy. Database Roll back to previous version needs costly development of manual script, which is dependable on current version of database and its data. Sometimes it’s easy. In some cases roll back is almost impossible. (e.g. bank can afford only one downtime window for 8-hours release deployment with hard data migrations. Since users start working and populating new data structures you simple cannot drop their changes) 17
  • 18. Application Plenty of out of box continuous integration solutions. Some of them may automate your deployment process within several mouse clicks. Database There is no general support for database development. Most database development teams use manual or semi-automated process to produce deployment scripts. 18
  • 19. 19
  • 20. This is the most important slide. Database CI is relatively complex development tool. Good news is You can learn many wonderful things in life. Does not matter how huge and complicated you database is – continuous integration is beautiful technology you can master. 20
  • 21. 21
  • 22. 22
  • 23. You must have test automation. Otherwise CI has no much sense. We use UTPLSql for oracle unit testing. You also can use Toad Code Tester, SQL Unit with SQL Developer or self-coded tool. Test automation for DB is topic in it’s own. Later we created highly flexible tool with Web GUI for large ETL processes test automation. For now it supports ETL systems with Informatica, Oracle Warehouse builder and SAS Data Integration Studio. 23
  • 24. CI software is needed for automated deployment scripts building, their deployment and testing with test automation systems. It also integrates nice reporting tools about build health. 24
  • 25. Version Control is the blood for Database CI. I will demonstrate on Subversion as our favorite VCS. Likely there is no restriction to use other VCS of your choice. 25
  • 26. Well documented VSC project structures won’t seamlessly fit database development projects. We will demonstrate required adaptation to enjoy all CI benefits. 26
  • 27. All actions are scripted (No changes with GUI, e.g. adding column or comment by SQL Navigator GUI). All database objects should be scripted as sql: packages, procedures, views, triggers etc. Database structure changes and data migrations should be as sql files: table creations, alter operations, insert operations etc. Scripts – general project artifact. Scripts should be checked into Version Control System Scripts have owners, and could be assigned and maintained Scripts are testable Contrary to manual actions, scripts are safe to deploy&test multiple times Scripts can be deployed on multiple databases and environments Scripts can be linked with Bug Tracking system (like Jira). You can see what objects were changed within particular task… by whom and when. 27
  • 28. If Live Demo is not applicable then the basic concept will appear at slides 33-36. 28
  • 29. History of all changes is in the TRUNK. TRUNK – single database master-copy 29
  • 30. Convenient for developers – code objects, structure and data change scripts are not spread by different release folders. You don’t need to remember where are they now or look for them. To start development you always go to TRUNC, open needed source file (where files are structured by schema and db object type). Then you do the coding, compiling and testing as usual, just simple press “save” before exit. When you start practicing branched development the system will automatically control already deployed objects or structure or data change scripts. Thus nothing will be lost or deployed twice. 30
  • 31. 31
  • 32. Day. Iterative development. Commiting of project artifacts to version control system Night. 1) Prodlike FlashBack() or Cold Backup(). Backup of all changes. Return of Prodlike environment to state fully identical to current production environment 2) Automated generation of deployment script as a difference of SVN «Prod» and «Trunk» folders. Files in this difference must be run in strictly predefined order. The order is important part of this methodology. This difference goes to «Patch» folder 3) Automated deployment to ProdLike environment 4) Run of automated testing 5) Deployment script goes to release repository. 6) Report about build health (succeeded deployment and tests) is published at the CI web dashboard 7) Involved members get e-mails about build health or founded bugs 32
  • 33. Get reference Prodlike DB first. Export/import production parts by impdp utility From production backup Clean it. Reduce complexity and number of db objects to dramatically increase deployment time and database size (removing all but one partition per table and so on) Save new light dump to SVN «dump» folder. The ProdLike DB is ready for intensive use within development and CI processes. 33
  • 34. «Prod» folder keeps the current production code. This folder is used for comparing in deployment script generation process. Structure of «trunk» folder is similar to «prod». The development take place in «trunk». At the point zero «trunk» and «prod» folders are exact copies. To save production DDL code to SVN we use UTL_FILE. Each object has its own file. Types, Type Bodies. Creating and droping types in correct order. Drop cascade (by dba_dependencies), then created according to their dependencies. Grants for them. All types are in one file thus redeployed all together if needed. Dependable types is actually the hardest part. Triggers. View triggers and materialized view triggers must be removed and be part of view and mat.view source code. Functions, Procedures, Packages. Also their grants. Views, also their triggers and grants. Materialized Views. also their triggers, indexes and grants. Package Bodies. … 34
  • 35. By executing svn «diff» command we are getting difference of «prod» and «trunk» in changed files «export» command over «diff» results gets us something very close to deployment script. Regularly we export to «patch» folder. «patch» folder structure is identical to «prod» and «trunk». It now contains only changed files and not deployed «before» and «after» objects. 35
  • 36. ANT: SQLPLUS calls all files of «patch» folder tagged to «release x.x…» folder in very strict order, so that any our work is deployable. There are few ways to actually deploy release according to your situation Sequential deployment of all sql files in «release x.x..» folder. Files are sorted by subfolder and names. In this case deployment administrator must use svn, call ant for deployment and set deployment params (like sid, login, pass) with properties file (build.properties). Concatination of all sorted sql-files to one big “release.sql”. Deployment administrator now can just run it with sqlplus. No need for extra actions with ant or svn. It’s best suited for outsourced development. Features of deployment with ant To deploy sql-files by sqlplus we use incanto pluging for ant. It gives us portability to different environments. Some files may be removed from release by their mask. It can be done by using attribute excludes of concat and fileset tasks. You can exclude autotesting 36
  • 37. packages, which are good only for testing and CI environments. 36
  • 38. Connect you ant deployment algorithm with svn and any CI software. Connect you autotests to CI software(we use maven-utplsql-plugin for you TeemCity CI software and UTPLsql autotests). Enjoy you automation! 37
  • 39. 38
  • 40. 39
  • 41. 40
  • 42. Actually You don’t have to do all the tests. Your autotesting system can get all changed objects and track their dependencies. Then it can test only by applicable tests. In oracle you do this with dba_dependencies, with ETL software you have to track dependencies by their XML or otherwise. 41
  • 43. Optimize your autotests Use lightweight synthetic data Automate generation of synthetic data Make build and test fast. Invest in infrastructure Use CI software capabilities to run parallel builds on multiple machines (Build Agents) 42
  • 44. 43
  • 45. Developer can build his own light version of database from dump files and deployment scripts from version control system. He can do it in any needed environment for branched development and testing. 44
  • 46. Do continuous integration. Try to switch from nightly builds to more frequent build (e.g. every 2 hours). Then to build&test by every commit to source control. Try to separate your quick and slow test. Slow for weekends, quick for now. Track dependencies of changed objects and tests. 45
  • 47. 46
  • 48. 47
  • 49. 48
  • 51. 50
  • 52. 51
  • 53. 52