Presentation by Maurizio Mancini of Exempio at Agile Tour Ottawa in November 2017 on how to Scale Quality in Software by Building it in.

1. Scaling Quality by Building it in
MAURIZIO MANCINI • EXEMPIO • MAURIZIO.MANCINI@EXEMPIO.COM

2. • A leader in the quality and process industries with
a sixth sense for Agile, quality, and business
process.
• I have been refining my Agile approach for more
than 10 years.
• My passion is to deliver quality software and to
see how Agile can help teams deliver quality
software!
Maurizio Mancini
Agile 2014 – Agile: One Size does not fit all!
Walmart Labs California 2014 – Quality and Process
Atlassian Summit 2014 – From Incremental & Iterative to Agile: What
is the right process for your team?
Quest 2015 – Building a QA Team that Matters
Cutter Summit 2015 – Agile Testing – What’s that all about?
Atlassian Summit 2015 – How to Build in Quality from Day 1 using
Lean QA and Agile Testing
Agile Tour Montreal 2016 – Le guide de réparation de l’équipe Agile :
La recette secrète !
Agile Tour Ottawa 2016 – How to Reboot Your Agile Team!
Global Scrum Gathering San Diego 2017 – How to Reboot Your Agile
Team!
Agile 2017 – How to Reboot Your Agile Team!

3. • Most Common Pitfalls for Scaling Quality
• Building Blocks for a Quality Mindset
• How Agile roles help in Building in Quality
• Cautionary Tales and Guidelines for Test Automation
• Recommendations for Test Automation

4. Version One Survey
One of the top five reasons for adopting Agile is to “enhance software quality”
Source: https://explore.versionone.com/state-of-agile/versionone-11th-annual-state-of-agile-report-2

5. I can’t do that because…

6. 3 Most Common
Pitfalls for
Scaling Quality
4

7. Most common quality pitfalls for Scaling Quality
Agile Team will do it all

8. Think Again!
Most common quality pitfalls for Scaling Quality
Agile Team will do it all

9. Most common quality pitfalls for Scaling Quality
Agile Teams don’t need to think about Integration Testing until much later…

10. Think Again!
Most common quality pitfalls for Scaling Quality
Agile Teams don’t need to think about Integration Testing until much later…

11. Test Automation will do it all!
Most common quality pitfalls for Scaling Quality

12. Think Again!
Most common quality pitfalls for Scaling Quality
Test Automation will do it all!

13. Most common quality pitfalls for Scaling Quality
We will reuse tests for multiple purposes
GLUE
STITCH
4

14. While in theory it is possible
Think Again!
In reality, it is very difficult
We will reuse tests for multiple purposes 4Most common quality pitfalls for Scaling Quality

15. Think Again!
We will reuse tests for multiple purposes
Most common quality pitfalls for Scaling Quality
4

16. 3 Building Blocks
for a
Quality Mindset

17. What level of risk is the organization willing to accept?
1

20. Establish a Corporate Definition of Quality
Levels Sample Systems
Quality
Approach Company Target
Level of
Risk
Perfect
Medical – Life Dependant
NASA
QA No
Excellent
High Volume Systems like
Software Fulfillment Systems
(i.e. APP Store)
QA
Good
CRM Systems
Mobile Non-Transactional
APPS
Websites
QA
Break and Fix
QC High
Risk
Example - 3 Levels of Quality

21. Why is it acceptable to write
code without thinking of how to
test it?
2

23. “Quality is something everyone
wants, as long as it doesn’t cost
anything.”

24. NIKE Inc (2001): Problems with their supply-chain
management system which resulted in a $100 million loss.
AT&T Wireless (2004): Customer Relations Management (CRM)
upgrade lead to $100 million loss in revenue.
Knight Capital Trading (2012): Software glitch cost the firm
$440 million in 30 minutes using a flawed software algorithm.
Nest Thermostat (2016): Software glitch leaves users with cold
houses in the middle of winter.
Citigroup (2016): Software bug costs Citigroup $7 million.

25. Why is it acceptable to write
code without thinking of how to
test it?

26. No Line of Code is written without thinking of how it will be tested

27. 3

28. Everyone

29. The Team
PO
ScrumMaster
Each Role Thinks “How Do I Test This?”
Do we have everything
we need
to test the Story?
DEVOPS
Integration Tests
Environment Tests
Deployment Tests
Unit Testing
Automated Tests
Exploratory/Manual Tests
Integration Tests
Acceptance
Criteria

30. A Test First mind set will
Build in Quality!

31. Agile Coach
The Team
Product Management
DEVOPS

32. Agile Coach

33. Agile Coach establishes a Test First Mind Set

34. Corporate Definition of Done Guideline

35. Definition of Done with Quality Goals

36. Agile Coach
helping establish a
Test First Mind Set

37. The Team

38. Break down that Dev QA wall

39. Best way I have found
to start breaking
down the wall …

40. One Story at a Time

41. I am a developer… I don’t test

42. It is not a one person team

43. It is not easy to change stripes

44. But through coaching and
Everyone having a
Test First Mind Set
Setting common
quality goals
And

45. Product Management

46. Use ATDD/
BDD
Building in Quality starts with
an Agile Product Management
organization
APM means just enough
product definition

47. Why use ATDD/BDD?
• Focuses Product Owners, Developers and QA
• Everyone speaking a common language
• Help’s the team progress from thinking
about what feature they are working on to
“How the feature is going to be tested”
• Can be used for Test Automation

48. Example
Feature: Amazon Shopping
Scenario: Amazon Login
#Given When Then And But
Given the url is opened
And I hover Your Account
When I click Sign In
Then I enter an email
Then I enter a password
And I click Sign In
Then I should see the welcome page

49. Which method should you use?
Focused on
developer coding
the test
Focused on using
English like
Syntax
PO should still be
comfortable
working with this
type of syntax
Code still required
but is one step
removed
ATDD/BDD
TDD
The Team
PO & The Team

50. The goal of these methods …
ATDD/BDD
TDD

51. Think Quality First!

52. DEVOPS

53. Quality at the Agile Team level is Essential… but
At some point it has to
all come together

54. Most software applications interact with other APPS
Integrate and Test Often
Continuous Integration and Testing

55. System Integration Testing (SIT) is performed to ensure that all related
systems exchange data seamlessly, verifying a system’s ability to
operate as expected with other systems within the same environment.
Integrate and Test Often
• Use tests purposefully designed with this goal in mind.
• They are high level tests, focused on the flow of data.
During the sprint - Using a Scrum of Scrums
like process, teams should Integrate and test
often

56. CI helps the integration process but depending on the application, your approach to System
Integration Testing (SIT) will vary.
Continuous Integration (CI) and Testing
Setup an automated pipeline for SIT
If it is too hard, use a combination of automated and manual SIT testing

57. People in the team with DEVOPS skills will help establish a CI process
DEVOPS

58. Example of CI/CD Pipelines
Source: https://docs.gocd.org/current/

59. Test Automation
Cautionary Tales and
Guidelines

60. Test Automation
Record and Playback – Be careful of the huge promises
It cannot be used effectively to scale anything
“Scriptless Tools”

61. Test Automation can get
Out of Control

62. Test Automation is NOT Automatic

63. Writing Code
to Test Code

64. Remember the Paths….

65. Building Blocks of a Quality Mindset
Test Automation A.I.

66. It is an integral part of every story and is a team responsibility.
Test Automation is NOT only a QA Effort

67. Are we done yet?

68. Lifecycle of an Automated Test
Create
Stabilize
Execute &
Maintain
Deprecate

69. 3 Recommendations
for
Test Automation

70. Test
Automation is
done in the
Team and
Owned by the
Team

71. Automate what makes
sense
Full Coverage may sound
great but at what cost?
Is it realistic?

72. Automation is NOT a replacement for Exploratory Testing

73. Key Takeaways

74. I can’t do that because…
What is the alternative?

75. Popular Software Engineering Fallacy
We don’t have the time to do
it right….
But we have the time to do
it again… and again…
and again…

76. Scaling Quality starts by Building it in

77. $$$
Cheaper to
Build in Quality
than to
Test it in
$$$$$$$$$

78. Don’t Compromise
or Give Up on
Quality
Deliver Quality

79. Thank You!
MAURIZIO MANCINI • EXEMPIO • MAURIZIO.MANCINI@EXEMPIO.COM