Performance and Feature comparison of different frameworks of GoLang v/s Non-GoLang Testing Frameworks.
This presentation have been presented in GopherConIndia-16
2. OUTLINE
BDD Testing Frameworks
Ginkgo + Gomega
Comparison with Cucumber
Test Execution
Performance
Test Scenario
Performance Statistics
Unit Testing Framework
‘testing’ + ‘testify’
Comparison with JUnit
REST API Testing
Frameworks
Frisby
Comparison with RESTAssured
3. UNIT TESTING – “testing + testify”
What is Unit Testing?
Unit testing is a software development process in which the smallest testable parts
of an application, called units, are individually and independently scrutinized for
proper operation.
Unit Testing covered by GoLang’s ‘testing’ package
‘testing’ package’s few features
Benchmarking
Code Coverage
Parallel Execution
‘testify’ extension complements ‘testing’ package by providing
support for
Assertion
Mocking
4. “testing + testify” v/s “JUnit”
JUnit is famous unit testing framework for Java
Improvement Areas for GoLang’s ‘testing’ package:
Test Fixtures
Parameterized Test
Data Provider
5. BDD TESTING – “Ginkgo + Gomega”
What is BDD?
Behavior-driven development (BDD) is a software development methodology in which an
application is specified and designed by describing how its behavior should appear to an
outside observer.
GoLang’s BDD Testing Frameworks:
Ginkgo
GoConvey
GoDog
GoSpec
GoSpecify
Ginkgo + Gomega:
Ginkgo is BDD style GoLang testing framework
Gomega is matcher/assertion library
6. “Ginkgo + Gomega” v/s “Cucumber”
Features of Ginkgo
Test Fixtures
Parallel Specs
Asynchronous Tests
Table Driven Tests
‘Cucumber’ is well-known BDD Testing Framework, having bindings
for Ruby, Java, JavaScript etc.
Improvement Areas for ‘Ginkgo’:
Feature Files
Integration with Selenium for Web Testing
Performance
7. REST API TESTING – “Frisby”
What is REST API Testing?
API testing is in many respects like testing software at the user-interface level,
only instead of testing by means of standard user inputs and outputs, you use
software to send calls to the API, get output, and validate it.
GoLang’s REST API Testing Frameworks:
Frisby
RestIt
Frisby
It’s a REST API Testing framework, inspired by Frisby.js
Simple to Implement Tests
8. “Frisby” v/s “RESTAssured”
Features of Frisby
Covers all basic HTTP methods, like Get, Post, Put, Patch, Delete
Provides Assertions
JSON & XML Parsers
‘RESTAssured’ is well-known REST API Testing Framework for
Java
Improvement Areas for ‘Frisby’:
BDD style
Continuous Integration
Data Provider / Table Driven Tests
9. TEST SCENARIO
Very Basic Test Scenario, which will define type of ‘Triangle’
based on dimension of sides of triangle
Side3 = ‘z’
x == y == z
“Equilateral”
x == y || y == z || x == z
“Isosceles”
x != y != z
“Scalene”
10. PERFORMANCE STATISTICS
UNIT Testing BDD Testing REST API Testing
Go (Testing +
Testify)
JUnit Ginkgo Cucumber Frisby RESTAssured
Run_1# 0.008* 0.031 1.49 0.511 0.056 2.18
Run_2 0.009 0.026 1.36 0.531 0.056 2.14
Run_3 0.011 0.026 1.38 0.521 0.057 2.11
Run_4 0.009 0.028 1.38 0.532 0.056 2.04
Run_5 0.009 0.026 1.36 0.534 0.056 2.18
Run_6 0.009 0.028 1.39 0.532 0.057 2.07
Run_7 0.009 0.026 1.37 0.552 0.054 2.07
Run_8 0.009 0.026 1.39 0.507 0.053 2.07
Run_9 0.009 0.026 1.35 0.536 0.055 2.09
Run_10 0.009 0.026 1.39 0.518 0.056 2.1
Average
Execution Time
0.0091 0.0269 1.386 0.5274 0.0556 2.105
* All Times are in Seconds
# Each Run consisting execution of 100 TestCases per
framework