SlideShare ist ein Scribd-Unternehmen logo
1 von 21
Downloaden Sie, um offline zu lesen
IBM Global Services – Testing Competency
IBM Confidential © Copyright IBM Corporation 200501/23/2005
Strategy, Approach & Planning
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
2
2
IBM Confidential
Agenda
 Introduction
 Effective Automation
 Fundamental Approach
 Real Facts & Think Factor
 Considerations for Automation
 Automation Start-up Strategies to Avoid
 Guidelines & Approach
 Automation Methodologies
 Overview on Automation Tools
 References
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
3
3
IBM Confidential
Introduction
 GUI automation is always viewed as the SILVER BULLET, which will save money,
time and resources
 Test automation raises our hopes, yet often frustrates & disappoints us
 Promises to deliver us from a tough situation, yet can create as many problems as
it solves
"Embarking on a successful automation test project is similar to embarking
on a successful software development project“
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
4
4
IBM Confidential
Effective Automation
 Reduced utilization of manual resources
 More efficient use of scarce hardware resources
 Reduced test cycle time
 Reduced cost of testing, after initial cost of implementation
 Improved reliability of the testing
 Repeatable tests, thus overcoming boredom-induced missed bugs
 Improved test coverage, since testing may be done more frequently
 Development of programming skills
 May allow the use of less skilled staff to do the actual testing
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
5
5
IBM Confidential
Fundamental Approach
 Understand the need / requirement for automation
 Point out pitfalls to avoid
 Promote an evolving implementation
 Begin simple and increase in complexity as the team's skills increase
"Learning from others is the most effective way to grow your automation expertise"
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
6
6
IBM Confidential
Real Facts & Think factors
Real Fact
 Automation is initially expensive and it may not do a better job than manual testing
in some situations
Think Factors
1. How much will automating and running a test cost, versus running it manually?
2. Automated test has a limited lifetime in which it must recoup the additional cost.
Will this test be reusable long enough? What factors will prevent it from being
reusable?
3. The automated test will find bugs it was designed to test for and in addition it will
find bugs that are not directly related to the test. What is the likelihood that this test
will find additional bugs?
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
7
7
IBM Confidential
Real Facts & Think factors
Real Fact
 Automation is initially expensive and it may not do a better job than manual testing
in some situations
Think Factors
1. How much will automating and running a test cost, versus running it manually?
2. Automated test has a limited lifetime in which it must recoup the additional cost.
Will this test be reusable long enough? What factors will prevent it from being
reusable?
3. The automated test will find bugs it was designed to test for and in addition it will
find bugs that are not directly related to the test. What is the likelihood that this test
will find additional bugs?
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
8
8
IBM Confidential
Considerations for Automation
 Test cases those are long running, repetitive, and non-subjective
 Projects / Products that have a working, relatively stable GUI
 Projects / Products that will span multiple releases.
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
9
9
IBM Confidential
Guidelines & Approach
Record / Playback initially using the Testing tool’s wizard or toolbar will be easier but
hard to maintain because they represent not only WHAT needs to be tested, but
HOW it is to be tested
Guideline: 1
Once familiarity in scripting is achieved, continue using the scripting language of an
automation tool to provide your own implementation logic (framework) that is separate
from your test case scripts
 Requires more investment upfront, but the code will be less expensive to maintain
over the life span of multiple product releases
Example:
Imagine having recorded test case scripts for a banking application that initially
allowed the user to simply click "OK" to commit a sensitive transaction, then the
application developers changed the navigational logic requiring the user to type "yes"
to confirm these transactions. Every recorded test case that has hard coded "OK"
clicks relating to the sensitive transaction commits would require maintenance.
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
10
10
IBM Confidential
Guidelines & Approach (Contd..)
Automation code without any Discipline - Tendency to jump in and start
developing new test scripts without collaborating with other team's members and
without establishing a plan, structure, and/or standards is natural. However, in the
long term this undisciplined approach is not productive. Any test case scripts and
frameworks that are produced in this manner tend to lack such characteristics as:
maintainability, reusability, readability, flexibility, reliability, effectiveness, and efficiency
Guideline: 2
Establish complete Automation plan, Structure, and Standards
“Quality automation effort should follow a software development process”
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
11
11
IBM Confidential
Guidelines & Approach (Contd..)
Guideline: 3
“Separating the What from the How”
An approach to separate what needs to be tested from how it is to be tested. To
accomplish this separation, guideline would suggest to have the test script writers
write the test scripts to identify what needs to be tested and automation developers
write a framework to handle the implementation (i.e., the navigation logic, window
processing logic, and verification logic) of your tests.
Framework - "a supporting or enclosing structure”
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
12
12
IBM Confidential
Guideline: 4
Use the 6 phases for implementing the best course of Automated Testing action
- Evaluation
- Planning
- Design
- Implementation
- Deployment
- Review
Iteration Cycle – Plan for future changes
Guidelines & Approach (Contd..)
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
13
13
IBM Confidential
Guidelines & Approach (Contd..)
Guideline: 5
A. Define Automation Goals & Prioritize
At every point in the specification process, you can check the "testability" of the requirement or
specification by ensuring it is S.M.A.R.T.
S - Specific
M - Measurable
A - Attainable or Achievable
R - Realistic
T - Timely
B. Define Automation Strategy
- GUI - Roundtrip / Segment, Define objects, Synchronization
- Types of Testing
- Strategy Considerations
1. Partial automation
2. Time cost
3. Testing delay
4. Inertia
5. Risk of missed bugs
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
14
14
IBM Confidential
Guidelines & Approach (Contd..)
Guideline: 6
Define Users and Skill Level Required / Desired
- Test Script Writers
- Automation Developers
Guideline: 7
Resource Planning / Source Code Management
- Hardware
- Software
- People
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
15
15
IBM Confidential
Guidelines & Approach (Contd..)
Guideline: 8
Define Design Goals & Prioritize
- Maintainability
- Structured
- Well Documented
- Reviewed
- Framework Specific Goals
- Modularity
- Reusability
- Usability
- Scalability
- Reusability
- Portability
- Robustness
- Reliability
- Efficiency
- Flexibility
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
16
16
IBM Confidential
Guidelines & Approach (Contd..)
Guideline: 9
Determine the Automation Interface Architecture
- Data Driven Interface
- Programming Interface
Guideline: 10
Determine the Automation Framework Architecture
- Error Recovery
- Support of Custom Controls
- Product Testability Issues
- NLS – Localization
Guideline: 11
Define Standards – Naming & Code Conventions
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
17
17
IBM Confidential
Guidelines & Approach (Contd..)
Components of Automated Test System
Test Script Writers
Automation Developers
Test Scripts
“WHAT”
Interface
Framework
“HOW”
Documents
 Framework Usage Notes
 Guidelines
 Naming Conventions
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
18
18
IBM Confidential
Automation Methodologies
 Functional Decomposition Method
This methodology is to reduce all the test cases to their most fundamental
tasks, and write User-Defined Functions, Business Function Scripts, and
"Sub-routine" or "Utility" Scripts which perform these tasks independently of
one another
 Key-Word Driven or Test Plan Driven Method
This method uses the actual Test Case document developed by the tester
using a spreadsheet containing special "Key-Words". This method preserves
most of the advantages of the "Functional Decomposition" method, while
eliminating most of the disadvantages. In this method, the entire process is
data-driven, including functionality.
The Keyword controls the entire processing
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
19
19
IBM Confidential
Overview on Automation Tools
 Most of the Automated Tools in the market remain as an integrated suite for Test
Documentations, Scripting, Executions & Defect Tracking
 Automation tool does exist for specific testing needs such as
Performance Testing
Code Coverage
Memory Leak Detection
 Automation Tools available in the market which are more on focus today includes
 Rational Tools – Robot, Test Manager, ClearQuest
 Mercury Interactive – WinRunner, LoadRunner, QTP, Astra Load Test
 Segue – Silk Performer
IBM Global ServicesIBM Global Services – Testing Competency
© Copyright IBM Corporation 2005
20
20
IBM Confidential
References
Web Sites for Additional Resources
1. http://www.io.com/~wazmo/qa/ Software Testing Hotlist, maintained by Bret
Pettichord, has links to many of the articles and papers mentioned in this document
2. http://www.softwareqatest.com/index.html Software QA and Testing Resource
Center, maintained by Rick Hower, links
3. http://qaforums.com/ QA Forums, maintained by individual moderators, links to
forums on Quality Assurance, Tools, Consulting & Employment
Bibliography
1. Dustin, Elfriede, Rashka, Jeff, and Paul, John (1999) Automated Software Testing:
Introduction, Management and Performance. Addison-Wesley Longman, Inc. Reading,
Massachusetts
2. Fewster, Mark and Graham, Dorothy (1999) Software Test Automation: Effective
use of test execution tools. Addison-Wesley: ACM Press, New York
3. Kaner, Cem, Falk, Jack, and Nguyen, Hung Quoc (1999) Testing Computer
Software. Wiley Computer Publishing, New York
IBM Global Services – Testing Competency
IBM Confidential © Copyright IBM Corporation 200501/23/2005

Weitere ähnliche Inhalte

Was ist angesagt?

Test Automation Best Practices (with SOA test approach)
Test Automation Best Practices (with SOA test approach)Test Automation Best Practices (with SOA test approach)
Test Automation Best Practices (with SOA test approach)Leonard Fingerman
 
Introduction to Automation Testing
Introduction to Automation TestingIntroduction to Automation Testing
Introduction to Automation TestingArchana Krushnan
 
Test Automation
Test AutomationTest Automation
Test Automationrockoder
 
Test Automation Framework Design | www.idexcel.com
Test Automation Framework Design | www.idexcel.comTest Automation Framework Design | www.idexcel.com
Test Automation Framework Design | www.idexcel.comIdexcel Technologies
 
Test Automation Strategy
Test Automation StrategyTest Automation Strategy
Test Automation StrategyMartin Ruddy
 
Role Of Qa And Testing In Agile 1225221397167302 8
Role Of Qa And Testing In Agile 1225221397167302 8Role Of Qa And Testing In Agile 1225221397167302 8
Role Of Qa And Testing In Agile 1225221397167302 8a34sharm
 
Test Automation Framework Designs
Test Automation Framework DesignsTest Automation Framework Designs
Test Automation Framework DesignsSauce Labs
 
Final Automation Testing
Final Automation TestingFinal Automation Testing
Final Automation Testingpriya_trivedi
 
Automated Testing vs Manual Testing
Automated Testing vs Manual TestingAutomated Testing vs Manual Testing
Automated Testing vs Manual TestingDirecti Group
 
automation testing benefits
automation testing benefitsautomation testing benefits
automation testing benefitsnazeer pasha
 
Performance testing presentation
Performance testing presentationPerformance testing presentation
Performance testing presentationBelatrix Software
 
Test automation methodologies
Test automation methodologiesTest automation methodologies
Test automation methodologiesMesut Günes
 
6 Ways to Measure the ROI of Automated Testing
6 Ways to Measure the ROI of Automated Testing6 Ways to Measure the ROI of Automated Testing
6 Ways to Measure the ROI of Automated TestingSmartBear
 

Was ist angesagt? (20)

Test Automation
Test AutomationTest Automation
Test Automation
 
Test Automation Best Practices (with SOA test approach)
Test Automation Best Practices (with SOA test approach)Test Automation Best Practices (with SOA test approach)
Test Automation Best Practices (with SOA test approach)
 
Introduction to Automation Testing
Introduction to Automation TestingIntroduction to Automation Testing
Introduction to Automation Testing
 
Automation Testing
Automation TestingAutomation Testing
Automation Testing
 
Test Automation
Test AutomationTest Automation
Test Automation
 
Test Automation Framework Design | www.idexcel.com
Test Automation Framework Design | www.idexcel.comTest Automation Framework Design | www.idexcel.com
Test Automation Framework Design | www.idexcel.com
 
Automation With A Tool Demo
Automation With A Tool DemoAutomation With A Tool Demo
Automation With A Tool Demo
 
Test Automation Strategy
Test Automation StrategyTest Automation Strategy
Test Automation Strategy
 
Role Of Qa And Testing In Agile 1225221397167302 8
Role Of Qa And Testing In Agile 1225221397167302 8Role Of Qa And Testing In Agile 1225221397167302 8
Role Of Qa And Testing In Agile 1225221397167302 8
 
Introduction to Software Test Automation
Introduction to Software Test AutomationIntroduction to Software Test Automation
Introduction to Software Test Automation
 
Test Automation Framework Designs
Test Automation Framework DesignsTest Automation Framework Designs
Test Automation Framework Designs
 
Final Automation Testing
Final Automation TestingFinal Automation Testing
Final Automation Testing
 
Automated Testing vs Manual Testing
Automated Testing vs Manual TestingAutomated Testing vs Manual Testing
Automated Testing vs Manual Testing
 
automation testing benefits
automation testing benefitsautomation testing benefits
automation testing benefits
 
Automation testing
Automation testingAutomation testing
Automation testing
 
QA Best Practices in Agile World_new
QA Best Practices in Agile World_newQA Best Practices in Agile World_new
QA Best Practices in Agile World_new
 
Automation Testing
Automation TestingAutomation Testing
Automation Testing
 
Performance testing presentation
Performance testing presentationPerformance testing presentation
Performance testing presentation
 
Test automation methodologies
Test automation methodologiesTest automation methodologies
Test automation methodologies
 
6 Ways to Measure the ROI of Automated Testing
6 Ways to Measure the ROI of Automated Testing6 Ways to Measure the ROI of Automated Testing
6 Ways to Measure the ROI of Automated Testing
 

Ähnlich wie Automation testing strategy, approach & planning

4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM
4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM
4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQMIBM Rational
 
Build And Test Automation - Shortening the Feedback Loop
Build And Test Automation - Shortening the Feedback LoopBuild And Test Automation - Shortening the Feedback Loop
Build And Test Automation - Shortening the Feedback LoopRally Software
 
RDz for DevOps Webcast Series: Implementing Continuous Integration with RDz
RDz for DevOps Webcast Series: Implementing Continuous Integration with RDzRDz for DevOps Webcast Series: Implementing Continuous Integration with RDz
RDz for DevOps Webcast Series: Implementing Continuous Integration with RDzSusan Yoskin
 
Test Automation: Investment Today Pays Back Tomorrow
Test Automation: Investment Today Pays Back TomorrowTest Automation: Investment Today Pays Back Tomorrow
Test Automation: Investment Today Pays Back TomorrowTechWell
 
Webinar Presentation: Best Practices in QA Testing - Leveraging Open Source T...
Webinar Presentation: Best Practices in QA Testing - Leveraging Open Source T...Webinar Presentation: Best Practices in QA Testing - Leveraging Open Source T...
Webinar Presentation: Best Practices in QA Testing - Leveraging Open Source T...Emtec Inc.
 
Keyword Driven Automation
Keyword Driven AutomationKeyword Driven Automation
Keyword Driven AutomationPankaj Goel
 
Atagg2015 - Agile Testing by Leveraging Cloud
Atagg2015 - Agile Testing by Leveraging CloudAtagg2015 - Agile Testing by Leveraging Cloud
Atagg2015 - Agile Testing by Leveraging CloudAgile Testing Alliance
 
Agile Development in Aerospace and Defense
Agile Development in Aerospace and DefenseAgile Development in Aerospace and Defense
Agile Development in Aerospace and DefenseJim Nickel
 
IBM DevOps Enabling continuous integration & delivery
IBM DevOps Enabling continuous integration & deliveryIBM DevOps Enabling continuous integration & delivery
IBM DevOps Enabling continuous integration & deliveryRoberto Pozzi
 
Continuous Engineering with IBM Rational RELM
Continuous Engineering with IBM Rational RELMContinuous Engineering with IBM Rational RELM
Continuous Engineering with IBM Rational RELMgjuljo
 
Coml Psg Automation Approach
Coml Psg Automation ApproachComl Psg Automation Approach
Coml Psg Automation Approachroopavani
 
DevOps in the Hybrid Cloud
DevOps in the Hybrid CloudDevOps in the Hybrid Cloud
DevOps in the Hybrid CloudRichard Irving
 
6.13.2013 2013 - Software, System, & IT Architecture - Good Design is Good Bu...
6.13.2013 2013 - Software, System, & IT Architecture - Good Design is Good Bu...6.13.2013 2013 - Software, System, & IT Architecture - Good Design is Good Bu...
6.13.2013 2013 - Software, System, & IT Architecture - Good Design is Good Bu...IBM Rational
 
Universal test solutions customer testimonial 10192013-v2.2
Universal test solutions customer testimonial 10192013-v2.2Universal test solutions customer testimonial 10192013-v2.2
Universal test solutions customer testimonial 10192013-v2.2Universal Technology Solutions
 
A Complete Guide to Functional Testing
A Complete Guide to Functional TestingA Complete Guide to Functional Testing
A Complete Guide to Functional TestingAbhay Kumar
 
Accelerate Software Delivery with DevOps
Accelerate Software Delivery with DevOpsAccelerate Software Delivery with DevOps
Accelerate Software Delivery with DevOpsAlan Kan
 
Automation Best Practices.pptx
Automation Best Practices.pptxAutomation Best Practices.pptx
Automation Best Practices.pptxpavelpopov43
 

Ähnlich wie Automation testing strategy, approach & planning (20)

4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM
4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM
4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM
 
Build And Test Automation - Shortening the Feedback Loop
Build And Test Automation - Shortening the Feedback LoopBuild And Test Automation - Shortening the Feedback Loop
Build And Test Automation - Shortening the Feedback Loop
 
RDz for DevOps Webcast Series: Implementing Continuous Integration with RDz
RDz for DevOps Webcast Series: Implementing Continuous Integration with RDzRDz for DevOps Webcast Series: Implementing Continuous Integration with RDz
RDz for DevOps Webcast Series: Implementing Continuous Integration with RDz
 
Test Automation: Investment Today Pays Back Tomorrow
Test Automation: Investment Today Pays Back TomorrowTest Automation: Investment Today Pays Back Tomorrow
Test Automation: Investment Today Pays Back Tomorrow
 
Webinar Presentation: Best Practices in QA Testing - Leveraging Open Source T...
Webinar Presentation: Best Practices in QA Testing - Leveraging Open Source T...Webinar Presentation: Best Practices in QA Testing - Leveraging Open Source T...
Webinar Presentation: Best Practices in QA Testing - Leveraging Open Source T...
 
Qtp - Introduction values
Qtp - Introduction valuesQtp - Introduction values
Qtp - Introduction values
 
Keyword Driven Automation
Keyword Driven AutomationKeyword Driven Automation
Keyword Driven Automation
 
Atagg2015 - Agile Testing by Leveraging Cloud
Atagg2015 - Agile Testing by Leveraging CloudAtagg2015 - Agile Testing by Leveraging Cloud
Atagg2015 - Agile Testing by Leveraging Cloud
 
Agile Development in Aerospace and Defense
Agile Development in Aerospace and DefenseAgile Development in Aerospace and Defense
Agile Development in Aerospace and Defense
 
Future of QA
Future of QAFuture of QA
Future of QA
 
Futureofqa
FutureofqaFutureofqa
Futureofqa
 
IBM DevOps Enabling continuous integration & delivery
IBM DevOps Enabling continuous integration & deliveryIBM DevOps Enabling continuous integration & delivery
IBM DevOps Enabling continuous integration & delivery
 
Continuous Engineering with IBM Rational RELM
Continuous Engineering with IBM Rational RELMContinuous Engineering with IBM Rational RELM
Continuous Engineering with IBM Rational RELM
 
Coml Psg Automation Approach
Coml Psg Automation ApproachComl Psg Automation Approach
Coml Psg Automation Approach
 
DevOps in the Hybrid Cloud
DevOps in the Hybrid CloudDevOps in the Hybrid Cloud
DevOps in the Hybrid Cloud
 
6.13.2013 2013 - Software, System, & IT Architecture - Good Design is Good Bu...
6.13.2013 2013 - Software, System, & IT Architecture - Good Design is Good Bu...6.13.2013 2013 - Software, System, & IT Architecture - Good Design is Good Bu...
6.13.2013 2013 - Software, System, & IT Architecture - Good Design is Good Bu...
 
Universal test solutions customer testimonial 10192013-v2.2
Universal test solutions customer testimonial 10192013-v2.2Universal test solutions customer testimonial 10192013-v2.2
Universal test solutions customer testimonial 10192013-v2.2
 
A Complete Guide to Functional Testing
A Complete Guide to Functional TestingA Complete Guide to Functional Testing
A Complete Guide to Functional Testing
 
Accelerate Software Delivery with DevOps
Accelerate Software Delivery with DevOpsAccelerate Software Delivery with DevOps
Accelerate Software Delivery with DevOps
 
Automation Best Practices.pptx
Automation Best Practices.pptxAutomation Best Practices.pptx
Automation Best Practices.pptx
 

Mehr von SivaprasanthRentala1975 (20)

Testing using load runner performance testing
Testing using load runner  performance testingTesting using load runner  performance testing
Testing using load runner performance testing
 
Why contract savings are missing
Why contract savings are missingWhy contract savings are missing
Why contract savings are missing
 
Voyager scm
Voyager scmVoyager scm
Voyager scm
 
Volumes
VolumesVolumes
Volumes
 
The new mainframe
The new mainframeThe new mainframe
The new mainframe
 
Test execution may_04_2006
Test execution may_04_2006Test execution may_04_2006
Test execution may_04_2006
 
Telecom testing
Telecom testingTelecom testing
Telecom testing
 
Six sigma
Six sigmaSix sigma
Six sigma
 
Sdlc models
Sdlc modelsSdlc models
Sdlc models
 
Sdlc
SdlcSdlc
Sdlc
 
Scm
ScmScm
Scm
 
Pt presentation1
Pt presentation1Pt presentation1
Pt presentation1
 
Pp employee learnnig initiative-iss-220-g tle 2007
Pp employee learnnig initiative-iss-220-g tle 2007Pp employee learnnig initiative-iss-220-g tle 2007
Pp employee learnnig initiative-iss-220-g tle 2007
 
Performance testing and rpt
Performance testing and rptPerformance testing and rpt
Performance testing and rpt
 
Patents
PatentsPatents
Patents
 
Xpediter kanbay
Xpediter kanbayXpediter kanbay
Xpediter kanbay
 
Copy of good quotes.ppt
Copy of good quotes.pptCopy of good quotes.ppt
Copy of good quotes.ppt
 
Contract savings new
Contract savings newContract savings new
Contract savings new
 
Contract savings schema
Contract savings schemaContract savings schema
Contract savings schema
 
Contract savings may 10th 2004 by gm euro
Contract savings may 10th 2004 by gm euroContract savings may 10th 2004 by gm euro
Contract savings may 10th 2004 by gm euro
 

Kürzlich hochgeladen

Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.raviapr7
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRATanmoy Mishra
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...raviapr7
 
5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...CaraSkikne1
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxDr. Asif Anas
 
Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphNetziValdelomar1
 
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptxPISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptxEduSkills OECD
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapitolTechU
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17Celine George
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesCeline George
 
How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17Celine George
 
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptxSandy Millin
 
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxiammrhaywood
 
In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxAditiChauhan701637
 
Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.EnglishCEIPdeSigeiro
 
The basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxThe basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxheathfieldcps1
 
The Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsThe Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsEugene Lysak
 
How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17Celine George
 
M-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxM-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxDr. Santhosh Kumar. N
 

Kürzlich hochgeladen (20)

Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...
 
5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptx
 
Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a Paragraph
 
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptxPISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptx
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 Sales
 
How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17
 
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
 
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
 
In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptx
 
Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.
 
Finals of Kant get Marx 2.0 : a general politics quiz
Finals of Kant get Marx 2.0 : a general politics quizFinals of Kant get Marx 2.0 : a general politics quiz
Finals of Kant get Marx 2.0 : a general politics quiz
 
The basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxThe basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptx
 
The Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsThe Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George Wells
 
How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17
 
M-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxM-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptx
 

Automation testing strategy, approach & planning

  • 1. IBM Global Services – Testing Competency IBM Confidential © Copyright IBM Corporation 200501/23/2005 Strategy, Approach & Planning
  • 2. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 2 2 IBM Confidential Agenda  Introduction  Effective Automation  Fundamental Approach  Real Facts & Think Factor  Considerations for Automation  Automation Start-up Strategies to Avoid  Guidelines & Approach  Automation Methodologies  Overview on Automation Tools  References
  • 3. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 3 3 IBM Confidential Introduction  GUI automation is always viewed as the SILVER BULLET, which will save money, time and resources  Test automation raises our hopes, yet often frustrates & disappoints us  Promises to deliver us from a tough situation, yet can create as many problems as it solves "Embarking on a successful automation test project is similar to embarking on a successful software development project“
  • 4. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 4 4 IBM Confidential Effective Automation  Reduced utilization of manual resources  More efficient use of scarce hardware resources  Reduced test cycle time  Reduced cost of testing, after initial cost of implementation  Improved reliability of the testing  Repeatable tests, thus overcoming boredom-induced missed bugs  Improved test coverage, since testing may be done more frequently  Development of programming skills  May allow the use of less skilled staff to do the actual testing
  • 5. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 5 5 IBM Confidential Fundamental Approach  Understand the need / requirement for automation  Point out pitfalls to avoid  Promote an evolving implementation  Begin simple and increase in complexity as the team's skills increase "Learning from others is the most effective way to grow your automation expertise"
  • 6. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 6 6 IBM Confidential Real Facts & Think factors Real Fact  Automation is initially expensive and it may not do a better job than manual testing in some situations Think Factors 1. How much will automating and running a test cost, versus running it manually? 2. Automated test has a limited lifetime in which it must recoup the additional cost. Will this test be reusable long enough? What factors will prevent it from being reusable? 3. The automated test will find bugs it was designed to test for and in addition it will find bugs that are not directly related to the test. What is the likelihood that this test will find additional bugs?
  • 7. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 7 7 IBM Confidential Real Facts & Think factors Real Fact  Automation is initially expensive and it may not do a better job than manual testing in some situations Think Factors 1. How much will automating and running a test cost, versus running it manually? 2. Automated test has a limited lifetime in which it must recoup the additional cost. Will this test be reusable long enough? What factors will prevent it from being reusable? 3. The automated test will find bugs it was designed to test for and in addition it will find bugs that are not directly related to the test. What is the likelihood that this test will find additional bugs?
  • 8. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 8 8 IBM Confidential Considerations for Automation  Test cases those are long running, repetitive, and non-subjective  Projects / Products that have a working, relatively stable GUI  Projects / Products that will span multiple releases.
  • 9. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 9 9 IBM Confidential Guidelines & Approach Record / Playback initially using the Testing tool’s wizard or toolbar will be easier but hard to maintain because they represent not only WHAT needs to be tested, but HOW it is to be tested Guideline: 1 Once familiarity in scripting is achieved, continue using the scripting language of an automation tool to provide your own implementation logic (framework) that is separate from your test case scripts  Requires more investment upfront, but the code will be less expensive to maintain over the life span of multiple product releases Example: Imagine having recorded test case scripts for a banking application that initially allowed the user to simply click "OK" to commit a sensitive transaction, then the application developers changed the navigational logic requiring the user to type "yes" to confirm these transactions. Every recorded test case that has hard coded "OK" clicks relating to the sensitive transaction commits would require maintenance.
  • 10. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 10 10 IBM Confidential Guidelines & Approach (Contd..) Automation code without any Discipline - Tendency to jump in and start developing new test scripts without collaborating with other team's members and without establishing a plan, structure, and/or standards is natural. However, in the long term this undisciplined approach is not productive. Any test case scripts and frameworks that are produced in this manner tend to lack such characteristics as: maintainability, reusability, readability, flexibility, reliability, effectiveness, and efficiency Guideline: 2 Establish complete Automation plan, Structure, and Standards “Quality automation effort should follow a software development process”
  • 11. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 11 11 IBM Confidential Guidelines & Approach (Contd..) Guideline: 3 “Separating the What from the How” An approach to separate what needs to be tested from how it is to be tested. To accomplish this separation, guideline would suggest to have the test script writers write the test scripts to identify what needs to be tested and automation developers write a framework to handle the implementation (i.e., the navigation logic, window processing logic, and verification logic) of your tests. Framework - "a supporting or enclosing structure”
  • 12. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 12 12 IBM Confidential Guideline: 4 Use the 6 phases for implementing the best course of Automated Testing action - Evaluation - Planning - Design - Implementation - Deployment - Review Iteration Cycle – Plan for future changes Guidelines & Approach (Contd..)
  • 13. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 13 13 IBM Confidential Guidelines & Approach (Contd..) Guideline: 5 A. Define Automation Goals & Prioritize At every point in the specification process, you can check the "testability" of the requirement or specification by ensuring it is S.M.A.R.T. S - Specific M - Measurable A - Attainable or Achievable R - Realistic T - Timely B. Define Automation Strategy - GUI - Roundtrip / Segment, Define objects, Synchronization - Types of Testing - Strategy Considerations 1. Partial automation 2. Time cost 3. Testing delay 4. Inertia 5. Risk of missed bugs
  • 14. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 14 14 IBM Confidential Guidelines & Approach (Contd..) Guideline: 6 Define Users and Skill Level Required / Desired - Test Script Writers - Automation Developers Guideline: 7 Resource Planning / Source Code Management - Hardware - Software - People
  • 15. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 15 15 IBM Confidential Guidelines & Approach (Contd..) Guideline: 8 Define Design Goals & Prioritize - Maintainability - Structured - Well Documented - Reviewed - Framework Specific Goals - Modularity - Reusability - Usability - Scalability - Reusability - Portability - Robustness - Reliability - Efficiency - Flexibility
  • 16. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 16 16 IBM Confidential Guidelines & Approach (Contd..) Guideline: 9 Determine the Automation Interface Architecture - Data Driven Interface - Programming Interface Guideline: 10 Determine the Automation Framework Architecture - Error Recovery - Support of Custom Controls - Product Testability Issues - NLS – Localization Guideline: 11 Define Standards – Naming & Code Conventions
  • 17. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 17 17 IBM Confidential Guidelines & Approach (Contd..) Components of Automated Test System Test Script Writers Automation Developers Test Scripts “WHAT” Interface Framework “HOW” Documents  Framework Usage Notes  Guidelines  Naming Conventions
  • 18. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 18 18 IBM Confidential Automation Methodologies  Functional Decomposition Method This methodology is to reduce all the test cases to their most fundamental tasks, and write User-Defined Functions, Business Function Scripts, and "Sub-routine" or "Utility" Scripts which perform these tasks independently of one another  Key-Word Driven or Test Plan Driven Method This method uses the actual Test Case document developed by the tester using a spreadsheet containing special "Key-Words". This method preserves most of the advantages of the "Functional Decomposition" method, while eliminating most of the disadvantages. In this method, the entire process is data-driven, including functionality. The Keyword controls the entire processing
  • 19. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 19 19 IBM Confidential Overview on Automation Tools  Most of the Automated Tools in the market remain as an integrated suite for Test Documentations, Scripting, Executions & Defect Tracking  Automation tool does exist for specific testing needs such as Performance Testing Code Coverage Memory Leak Detection  Automation Tools available in the market which are more on focus today includes  Rational Tools – Robot, Test Manager, ClearQuest  Mercury Interactive – WinRunner, LoadRunner, QTP, Astra Load Test  Segue – Silk Performer
  • 20. IBM Global ServicesIBM Global Services – Testing Competency © Copyright IBM Corporation 2005 20 20 IBM Confidential References Web Sites for Additional Resources 1. http://www.io.com/~wazmo/qa/ Software Testing Hotlist, maintained by Bret Pettichord, has links to many of the articles and papers mentioned in this document 2. http://www.softwareqatest.com/index.html Software QA and Testing Resource Center, maintained by Rick Hower, links 3. http://qaforums.com/ QA Forums, maintained by individual moderators, links to forums on Quality Assurance, Tools, Consulting & Employment Bibliography 1. Dustin, Elfriede, Rashka, Jeff, and Paul, John (1999) Automated Software Testing: Introduction, Management and Performance. Addison-Wesley Longman, Inc. Reading, Massachusetts 2. Fewster, Mark and Graham, Dorothy (1999) Software Test Automation: Effective use of test execution tools. Addison-Wesley: ACM Press, New York 3. Kaner, Cem, Falk, Jack, and Nguyen, Hung Quoc (1999) Testing Computer Software. Wiley Computer Publishing, New York
  • 21. IBM Global Services – Testing Competency IBM Confidential © Copyright IBM Corporation 200501/23/2005