Presentation from Coverity at the Merge World Tour in India - Coverity P4 Conference

2. Coverity Development Testing
Accelerating Risk Mitigation through Continuous
Integration and Development Testing
Raj Mathur
Country Manager
Coverity India

3. “Software is Eating the World”
Software
- Marc Andreessen
Health
Financial
Communications
SCM / Logistics
Enterprise
Mobile
81% of business leaders believe
technology is a fundamental element
of their business model
Over 60 million tablets and 175 million
smart phones will be in the workplace
by the end of 2012
By 2016, open source software will be
included in mission-critical
applications within 99%
of Global 2000 enterprises
Automotive

5. Our Value
Development Testing is transforming
software development by:
Reducing operational costs
Accelerating development and time to market
Protecting brands from catastrophic failure

6. Why All the Risk?
Software Complexity and Speed Have
Outpaced Legacy Testing Methods
Development
Testing
Software Complexity
Time to Market
Testing Methods
Security Testing
Functional Testing
Performance Testing
Manual Testing

7. Fewer defects escape dev
Design Development
QA +
Security Audit
Deployment
Our Mission and Passion:
Moving Quality, Security and Testing to the Left
5x cost 10x cost 30x cost

8. Transformation Maturity Model
8
Development Testing Adoption
Integration
into
SDLC
Level 1
Automatic Defect
Detection
Detection of
critical quality and
security defects as
part of SW build
process.
No new defects
introduced.
Level 2
Identification of
Residual Risk
Level 3
Developer
Workflow
Optimization
Integration into
the existing SDLC
using a common
workflow for all
defects and test
effectiveness
issues.
Level 4
Code Governance
Establish and
enforce consistent
source code
quality and
security policies.
Establish source
code acceptance
criteria.
Level 5
Enterprise Code
Assurance
All legacy defects
eliminated, build
fails if new defects
are introduced.
All critical code
and code impacted
by change is
tested.
High
High
Identification of
areas of risk
caused by
insufficient
automated testing.
Ensure critical
code is prioritized
and tested.

9. How Static Analysis Works
9
Explains the
location and
root cause of
defects
Manage and
share triage of
defects across
teams
Mimicks the
behavior of
dozens of
compilers
Integrates with
existing build
systems
Statically tests
all execution
paths
Finds defects
and
inconsistent
coding patterns
Analyze
Build
Present &
Manage

10. Meaningful, real results
Focus on finding real defects, not style violations or superficial issues.
Over 12 years of experience analyzing open source and commercial code.
Industry-leading low false positive rate
False positive rates typically below 15%.
False positives waste time, hinder adoption, and reduce trust in the results.
Broadest Checker Library + Deepest Algorithms
Optimal balance of breadth, depth, and scalability to large code bases.
High Quality Results
10 Confidential: For Coverity and Partner use only. Copyright Coverity, Inc., 2011

11. We Find Critical Defects
• Tomcat Webserver 5.5.17
• Open source server for web applications
• Among several hundred defects, we found a “reverse lock
bug” that can lead to deadlock of the entire server
• Very rare event - Very hard to find with traditional testing

12. Unit Testing Effectiveness
High
Risk
Code
High
Risk
Code

13. Risk Mitigation
13
In my critical code, each
component whose behavior was
modified (directly or indirectly) in
the last release must be 100%
tested (excluding error-handling)
Organization
Defines a Test
Policy
Test Advisor
Evaluates Test
Policy
Developers Get
Actionable
Work Items
Existing Coverity Static
Analysis Engine
+
New Tools that we Built
Consistent UI

14. Risk Mitigation Architecture
14 For Coverity and Partner use only. Copyright Coverity, Inc., 2012
Test Advice
Actionable work items to address risk
due to inadequate testing
Test Policy Evaluation
• Critical code analysis
• Change impact analysis
• Test execution analysis
Test Monitoring
Code Ownership
and Change History
Static Code
Analysis
Customized
Test Policy

15. Automate testing within the inner loop of
development
01001011
0101101011001
01101011000011
010100101101
01011001
Writes code
01
Creates unit test
Analyzes code
Fixes critical issues
Interprocedural quality
and security defects
New tests required
because of change
impact
Source Control
Management System
Centralized build
is generated
Assigned back to
appropriate developer
New issues are found
Prioritized defects
Prioritized tests

16. Integration into development workflow
IDE | Defect tracking | SCM | Build/CI | ALM
Analysis
Accuracy
Proven false positive rate
of less than 10%
on codebases over 1M
lines of code
Remediation
Guidance
Show path to defect and
fix guidance in context of
developer’s code
patent-pending security
remediation engine
Performance
and Scale
Proven scale on
codebases up to 100M
Analysis runs in minutes
to hours vs. days to weeks
The industry’s first developer-friendly
software testing platform
“Coverity enables developers to produce secure code and gives developers a more positive
attitude about addressing security, while ultimately leads to fixing defects.”
-Gerold Hubner, Chief Product Security Officer at SAP

17. The Workflow
17
• Security Audits
• Product Release
Management
(For illustration only – other workflow integrations possible)
QA
Nightly/Continuous
Build
• Desktop Analysis
• Review defects
• Prioritize actions
• Make fixes
• Track progress
Code Check In
Static Analysis
Results
• Functional Testing
• Performance Testing
• Stress Testing
• Integration Testing
Development
Product
Release &
Management
Static Analysis
Results

18. Issue Responsibility Is Critical
18

19. Ingredients for Success
19
Code
Build
Test
Nightly Build
Continuous
Integration
High-Fidelity
Code
Compilation
High-
Performance
Analysis
Low False
Positive Rate
Detecting
Critical
Defects
Easy Defect
Navigation and
Comprehension
Comprehensive
Triage and
Remediation
Management
Visibility and
Governance
Team
Collaboration

20. Ingredients for Success
20
Code
Build
Test
Nightly Build
Continuous
Integration
High-Fidelity
Code
Compilation
High-
Performance
Analysis
Low False
Positive Rate
Detecting
Critical
Defects
Easy Defect
Navigation and
Comprehension
Comprehensive
Triage and
Remediation
Management
Visibility and
Governance
Team
Collaboration
Developer
Adoption
Workflow
Integration
Management
Oversight

21. Governance with Metrics
Automated high-fidelity
analysis on daily basis
21
Fast and educated triage
of results to categorize
and prioritize issues
Accurate
Data
Precise actions based
on comprehensive
data analysis
Trusted
Data

22. Organizational ‘Heat Map’
Confidential: For Coverity and Partner use only. Copyright Coverity, Inc., 2013
22

23. Gain executive level visibility into risk
Across teams, projects and components

24. Common usage scenarios
24
For Coverity and Partner use only. Copyright Coverity, Inc., 2013
Increase development testing adoption and ROI
Metrics to track adoption
• Daily unique users
• Monthly unique users
• Issues introduced
• Issues resolved
• And many others …

25. Common usage scenarios
25
For Coverity and Partner use only. Copyright Coverity, Inc., 2013
Improve time to market
Early visibility into issues
• Outstanding issue count
• Resolved issue count
• Outstanding issue by impact
• Defect density
• And many others …

26. Common usage scenarios
26
For Coverity and Partner use only. Copyright Coverity, Inc., 2013
Mitigate risk
Establish a stage gate with
risk metrics
• Defect density
• Outstanding issues by
impact
• Test policy violations
• And many others …

27. Coverity Development Testing Platform
Security
Advisor
Test
Advisor
Analysis Packs
Coverity SAVE™
Static Analysis Verification Engine
SDLC
Integrations
Policy Manager
Quality
Advisor
Architecture
Analysis
Dynamic
Analysis
Java
FindBugs™
Analysis
Analysis
Integration
Toolkit
Coverity Connect
Test
Execution
Third Party
Metrics
Build/
Continuous
Integration
HP ALM
IDE
Code
Coverage
Defect
Tracking
SCM
Confidential: For Coverity and Partner use only. Copyright 2012 Coverity, Inc.

28. Analyze Accurately detect issues difficult to
find through traditional testing
Remediate Quickly and efficiently manage issues
to resolution
Govern Enforce a consistent standard for
quality, security and testing
Three Step Process to Development Testing
28 For Coverity and Partner use only. Copyright Coverity, Inc., 2012

29. Analysis Foundation: Coverity SAVE®
Static Analysis Verification Engine
Award-winning analysis engine with patented techniques based on a
decade of R&D and analysis of over 5 billion lines of proprietary and
open source code
29
Interprocedural
Data Flow
Boolean
Satisfiability
Global
Data Flow
Change Impact
Analysis
Accurate Compilation
False Positive Intelligence
White Box Fuzzer
Enterprise
Framework
Analyzer
Proprietary Code | Open Source Code
Statistical
Analysis
Language Idioms
And Design
Patterns

30. Issues
Manage defects and untested code violations
in a single interface and with a robust
repository
Developers
Workflow
Coverity Connect: Collaborative Issue Mgmt.
Connecting…
Collaborate across distributed teams with
and enterprise framework
Manage issues within your standard SDLC
30 For Coverity and Partner use only. Copyright Coverity, Inc., 2012

31. Remediate Critical Quality Defects
Leveraging a Robust Issue Management Repository
Prioritize and
filter based
on impact
Identify
the exact
path to
the defect
Automatically
assign defects
to owners
Automatically
identify every
occurrence of a
defect across
branches
CWE compatible
mapping and
knowledge base
31 For Coverity and Partner use only. Copyright Coverity, Inc., 2012

32. Analyze and Remediate Defects From Within the
Eclipse or Visual Studio IDE
32 For Coverity and Partner use only. Copyright Coverity, Inc., 2012

33. 33
Customers
• Over 1,100 customers (5.0B
LOC under mgmt)
• 18 of top 20 sw/hw firms.,
• 10 of 10 A&D Firms
• 8 of 10 Telecom
• 4 of 5 Security Firms
• 4 of 5 Exchanges
• ‘Gate’ (mandate) across supply
chain for many of the products
that you use today!
• 300 open source projects
Financial
• Fastest growing company in the
sector
• Record revenue growth 3+ years
in a row
• Almost three times the market
share of the nearest vendor -
VDC
• Backing from Benchmark Capital
and Foundation Capital
Company
• Founded in 2003 at Stanford
University Research Lab
• DHS Standard - Open Source Scan
(14B LOC)
• #1 in software quality analysis -
IDC (2012)
• #1 in Development Testing
(transformation) – Voke 2012
• 300 employees, 11 offices, 10
countries
Coverity: Leader in Development Testing
Technical Leadership
• Andy Chou, CTO &
Founder
• Dr. Dawson Engler, Prof
Stanford University, Grace
Murray Hopper Award
(2009)
• Dr. Andreas Kuehlmann,
Prof Cal, Past President of
EDA Council of IEEE
• Over a dozen patents
• CODiE Award finalist 2013
best security solution
• CODiE Award winner 2012
best software development
solution

34. Over 1,100 of the World’s Leading Brands use Coverity
34

35. Thank You
India Office Address :
Coverity (India) Pvt Ltd.,
Level 14, Concorde Towers
UB City, #1, Vittal Mallya Road
Bangalore – 560 001
Tel: +91 80 6759 0494
Mob: +91 98801 66186
eMail : rmathur@coverity.com