Software testing basic concepts

© Copyright 2006 FPT Software 1© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2
Software TestingSoftware Testing
Basic ConceptsBasic Concepts
Instructor: Dang Tu LinhInstructor: Dang Tu Linh

AgendaAgenda
 Duration: 1.5 Hours
 Purpose: Understand basic Test concepts
 Audience: All Members
 Content:
1. Introduction
2. Defect concepts
3. Test process
4. Test plan & test design
5. Testing strategy

IntroductionIntroduction
Objectives:
To arrive at a common understanding of the scope of testing activity
To familiarize the audience with the terminology used in the testing work
Understand the required test skills of developers

Typical Project Blues…Typical Project Blues…

Testing definitionTesting definition
Testing is the process of exercising or evaluating a system or
system component by manual or automated means to verify
that it satisfies specified requirements - (IEEE 83a)
Process:
 Sequence of steps performed for a given purpose. (IEEE)
Software Process
 A set of activities, methods, practices, and transformations
that people use to develop and maintain software and
associated products.(SEI-CMM)

Testing ObjectivesTesting Objectives
Primary: Execute a program with the intent of finding errors to
 Determine whether system meets specifications
 Determine whether system meets user’s needs
Secondary
 Continuously improve the testing process

MisunderstandingsMisunderstandings
about testingabout testing
Testing is debugging
Testing is not the job of a programmer
If programmers were more careful testing would be
unnecessary
Testing never ends
Testing activities start only after the coding is complete
Testing is not a creative task

Theory & RealityTheory & Reality
Theory
 Need sufficient time
 Baselined & freezed requirements
 Automation testing
 Have trained testers: test process, business flow,…
Reality
 Time Pressures to keep delivery date
 Continuously updated requirements
 Don’t know when to stop: Update & update Requirement
continuously (CRs)  Tester updates TC and re-tests
 Manual testing
 Lack of dedicated software test environment: share, steal
 Inexperienced testers without appropriate training

Defect conceptsDefect concepts
Objectives:
Understand the definition of defect
 Understand the cost of a bug to software development
 Understand the common types of defects and their
perceived severity
What is defect?What is defect?
A defect is any error found by testing and reviewing
activities (All errors found by internal reviewer, external
reviewer and customer).

Where defects come from?Where defects come from?
Products
 SRS, DD, Source code, Test cases, etc.
Quality Control
 Review, Test, Audit, Inspection
Processes
 Requirements, Design, Coding, Test, etc.
Other sources:
 Change requests, misunderstanding, carelessness, planning,
etc.

Cost of defectsCost of defects

Defect still exists in the product, change status to ERROR
FSoft Defect lifecycleFSoft Defect lifecycle
Defect classification:
 Open defects
 Closed defects
 Leakage
Log defects Analyze defects Fix defects Re-test
Step 1 Step 2 Step 3 Step 4
What: To log defect into DMS with
clear information: project, module,
stage, QC activity, severity, type,
due date, tester, created date ...
Who: Tester
Defect’s status: ERROR
What: -To analyze the causes of the defect, find-
out solution and assign to person who is
responsible for correcting it.
- To accept the defect if it’s acceptable (by
concession of authority or customer).
Who: Project Leader
Defect’s status: ASSIGNED / ACCEPTED
What: To correct the defect and
perform unit testing to make sure
the defect has been removed.
Who: Developer
Defect’s status:
CORRECTED
What: To retest the function(s)
containing the defect and make sure
that the defect has been corrected
completely
Who: Tester
Defect’s status: CLOSED

Defect status:
• Error
• Assigned
• Corrected
• Cancelled
• Accepted: by PM/PTL/Customer
• Closed: by tester/PM/PTL
Leakage Defects:
• Acceptance test
• After Release test
• After Release review
Fsoft Defect classificationFsoft Defect classification

Defect type
• Functionality
o Requirement misunderstanding
o Feature missing
o Coding logic
o Business logic
• User Interface
• Performance
• Design issue
• Coding standard
Severity: How bad the problem is
(some time disagreement btw. Dev & Tester)
 Fatal: the system is broken
 Serious: can not work around
 Medium: impact the function
 Cosmetic: not impact the function

Defect origin
 Requirement
 Design
 Coding
 Test
 CM
 Document control
Priority
 Immediately
 High priority
 Normal priority
 Low priority
Work product
 Software module
 Software Package
 ADD
 DDD
 Test Cases (UT/IT/ST)
Quality control activity
 Code review
 Unit test
 Integration test
 System test
 Document review
 Inspection
 Audit

Test ProcessTest Process
Objectives:
 Understand the steps followed for a test process
 Understand test planning process
 FSoft Test Process

Tester CompetencyTester Competency
Soft skills (non-technical attributes)
 Discipline and Perseverance
• Testing is repetitive and requests a lot of manual effort
• Have the ability to withstand pressures and workload
• Say ‘no’ to managers when quality is insufficient
 Reading skills: study many docs of specs, design
 Communication & Interpreter skills
• Both Verbal and Written communication
• Diplomatic skill
• Be able to communicate with technical and non-technical
people, engineers, managers, customers
 Negative thinking: foresee things that can go wrong, evaluate risks
 Attitude: ‘test to break’ attitude
 Time management and Effort prioritization

Tester CompetencyTester Competency
Technical skills
 Familiarity with software development architectures,
processes
 Familiarity with testing methodologies
 Very clearly articulate specs, designs, business rules,
inspection reports, configurations, code changes, TP,
TC, bug reports, user manuals,…
 Know how and where to look for bugs
 Know how to report the defects effectively for quickly
reproduce

The Test ProcessThe Test Process
Data Analysis
- Analyze
root
Causes of
defects
- Identify
actions
Test Execution
Perform
testing
Bug
Reporting
- Bug Fixing
- TC Updating
Bug Tracking
(Re-test)
Test
Report
Test Preparation
SRS study
Create
Test
Design
Review
&
Approval
- Develop test
cases, test
scripts
- Prepare test
data
Test
Planning
Create
Test Plan
(by
Test leader)
Definition Solution Construction Transition Termination
Initiation Project life cycle
Test process

Input/Output for testingInput/Output for testing
Input:
• Customer requirements and Acceptance criteria
• Change requests
• Software Requirement Specification (SRS)
• Design documents (ADD, DDD)
• Programs (Modules)
Output:
• Test documents: Test plan, Test cases and procedures,
Test script, Test data
• Defect list
• Test report (Pass/Fail of each TC, Summary report)

FSoft Test resourcesFSoft Test resources
Guideline: Test process & Test guideline
Templates for test documents
 Test Plan
 Test case specification
 Test report
 Defect analysis report
FSOFT tools
 Defect tracking tool: DMS
 Test Effort tracking tool: Timesheet
 Test schedule: FSoft Insight
 Test automation tools
– Rational Robot (Functional & Performance test)
– OpenSTA (Open source)
– Witir (Open source)
FATA: Test Council

Test strategyTest strategy
Objectives
 Understand Unit Testing concept
 Understand Integration Testing concept
 Understand System Testing concept
 Understand Acceptance Testing concept
 Overview of Test types

V-ModelV-Model

Testing stagesTesting stages
There are 4 stages of Testing:
 Unit Test
 Integration Test
 System Test
 Acceptance Test

Unit TestUnit Test
 Purpose: to verify that the component/module
functions properly
 Check:
– internal data structures
– Logic
– boundary conditions for input/output data
 Normally: White box oriented
 Doer: Development team
 Metrics: Test coverage:
– Statement
– Branch
– Path
– Condition

Integration TestIntegration Test
 Purpose:
• To ensure that code is implemented and designed
properly
• to take unit tested modules and build a program
structure that has been dictated by design
 IT only after UT
 Combining the individual components to uncover
errors associated with interfacing
 Normally: Black box oriented
 Doer: independent Test team

System TestSystem Test
 Purpose: To ensure that the system does what the
requirement specifies
 Types
– Function Testing
– Performance Testing
– Installation Testing
 Doer: independent Test team

Acceptance TestAcceptance Test
 Customer’s way to verify that what was wanted is
what is built
 Uncovers more than requirements discrepancies
 Allows the customers to determine what they really
want, whether specified in the document or not.
 New problems may arise
 Customers may decide that the problem as changed
and a different solution is needed
 Doer: Customer/independent Test team

Testing typesTesting types
Functional test
 Function testing
 User Interface testing
 Data and database integrity testing
 Business cycle testing
User Interface: Layout
Performance test
 Performance profiling
 Load testing
 Stress testing
 Volume testing
Access control & Security test
Regression test

Test techniques/methodsTest techniques/methods
White Box Testing
 It guarantees that all independent paths within a module
have been exercised at least once
 Exercise all logical decisions on their true an false sides
 Execute all loops their boundaries and within their
operational bounds
 Exercise internal data structures to assure their validity
Black Box Testing
 Focuses on functional requirement of the software
 Derive sets of input conditions that will fully exercise all
functional requirements for a program
 Not an alternative to white box

Test Plan, Test DesignTest Plan, Test Design
& Test Cases& Test Cases
Objectives
 Understand the principles of test plan & test design
 Understand the concepts for test case design
 Understand the structure of test cases

Test Planning: ObjectivesTest Planning: Objectives
Test planning is to:
 Define Test strategy:
– What to test: list of requirements to test
– How to test (Test techniques)
– Test types
– Test tool, ….
 Define the responsibilities
 Decide test metrics & manage through metrics
 “When to Stop Testing?”
– Not good Unit Test
– Low quality of code,…
– Meet requested Test coverage
– All Test cases are tested and passed

Test PlanTest Plan
 What to do
 Who does it: Test team, Development team
 How to do it: Techniques
 When to do it

Test PlanTest Plan
Test Plan to define:
 Scope of test: stages and types of test
 Risks may affect testing: quantify the loss of the risk
 Training needs for testers
 Requirements to test: what will be tested
 Testing strategy: how will the test be performed
 Completion criteria (Pass/Fail)
 Criteria to stop testing
 Tools to be used for testing
 Testing resource and environment (Responsibilities of testers)
 Test Milestone: schedule of testing activities
 Test deliverables: outputs of testing

When to Stop Testing?When to Stop Testing?
Not good UT: Defects of UT do not meet Defect plan
Low quality of code: too many defects (over the number
identified in planning phase/kick-off meeting)
Does not meet requested Test coverage (by
customer/PM).
It must be identified at planning phase.

Test design structureTest design structure
Specifications
 Basic flow
 Alternative flow
Test Design
 All test cases should be traceable to requirements
 Design framework and rules for test cases (large/average
items)
 Plan number of test cases for normal, abnormal & boundary
data
 Assign testers for module/function/large items

Test DesignTest Design
Structure/Framework for Test cases (TC):
• Modules or functions
• Large items
• GUI
• Functions
• Performance
Number of TC:
 Total
 For each module
 For each type: GUI/Function
 For normal/abnormal/boundary cases

Test CaseTest Case
Test environment
Test condition
A test case includes:
• Action (sample: click on Add button, on a link, input birth
day…)
• Input data (valid/invalid/boundary data)
• Expected result (message, …)
 Good test case:
 Structure of TC is clear and reasonable.
 Follow requirement closely.
 Cover all of cases that can occur.
 Description and test conditions is brief and easy to
understand
 Procedure and expected result is a step by step process

Testing MeasurementTesting Measurement
Objectives
 Understand the need for metrics and measurement in
Testing
 Understand the minimal metrics required for the
measurement of the testing activity

Examples of testing measurement?Examples of testing measurement?
 Product: We’ve tested 80% of the lines of code
 Plan: We’ve run 80% of the test cases
 Results: We’ve discovered 593 defects
 Effort: We’ve worked 80 hours a week on this for 4 months.
We’ve run 7,243 tests
 Quality of Testing: Beta testers have found 30 defects that we
missed. Our regression tests seems ineffective
 History across projects: At this milestone on previous projects,
we had fewer than 12% of the defects found still open. We
should be at that percentage on this product too

How do we measure Testing inHow do we measure Testing in FSOFTFSOFT??
Measurements:
 Defects in DMS
 Test effort in Timesheet
 Defect rate in Fsoft Insight: Weighted defects/ project size (in UCP)
 Test coverage: number of executed test cases/ Total number of test
cases
 Test successful coverage: number of test cases executed
successfully/total number of test cases
 Defect removal efficiency
Metrics to assess tester performance:
 Test effectiveness
• Weighted defects/ Test execution effort
 Leakage
• Weighted defects found after release/ project size

Q & AQ & A

Software testing basic concepts

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (16)

Similar to Software testing basic concepts

Similar to Software testing basic concepts (20)

Recently uploaded

Recently uploaded (20)

Software testing basic concepts

Editor's Notes