Introduction of Testing¶
Here, we provide a brief introduction to backend testing for Majority. Backend QA introduction slides
Automated tests¶
The developers are responsible for writing and maintaining the automated tests. ITPs runs in PR pipelines.Regression test Runs in Release pipelines against stage environment. Smoke test runs hourly against stage environment
Unit Tests¶
The base level of testing is the unit tests. A unit test is a test of the smallest piece of code that can be logically isolated in a system. Often times this is a C# method. Any dependencies are not included or tested.
Currently we are very light on Unit Tests, and if they exists are mixed together with Integration Tests.
Integration Tests (also called ITP tests)¶
An integration test at Majority tests that the area works as a whole, but does not test its dependencies (other areas or third parties) which are mocked. Note that the database is seen as part of the area and not mocked.
Integration tests should:
- Test that the methods in the Service layer works
- Test that the methods in the API layer works
- Test that relevant events are handled correctly
- Test that the correct events are published
Regression Tests (End-to-end API tests)¶
Regressions tests at Majority test the outer API of an Area as seen from outside of the Majority backend. They are end-to-end API tests covering multiple APIs and workflows. Basically what the Apps, Web or other (backend) external system would see. 3rd parties are not mocked unless necessary, instead often a 3rd party testing environment is used. Because of the large number of dependencies, including to 3rd parties they can sometimes be "flaky".
To help with writing and maintaining regression tests there is a special service, Internal Testing API, that is used to provide helpers to create test state and validate test outcomes not possible or not easy through the actual API. The Internal Testing API is also used by App teams test.
Regression tests should:
- Test all the API endpoints used by Apps, Sales Tool and Web. (Hydra API is not tested by regression tests)
- Test the "happy path" & a selection of other test scenarios.
Smoke Regressions¶
Smoke Regressions are a selection of regression tests that tests the most important parts of the backend. They are run on a schedule to monitor the whole state of the staging environment.
Postman Toolkit¶
When performing ad-hoc or exploratory testing, you often need to quickly inspect API responses, adjust payloads on the fly, or work with multiple endpoints—including internal testing APIs. Postman is an excellent tool to help streamline these tasks.
We've set up a few test collections for you to reuse in the Postman 'Backend API Testing' workspace. You can log in as either a Majority user or a web user and easily call internal testing API from there.
Exploratory Testing¶
We primarily conduct exploratory testing early in the development cycle to identify bugs and provide rapid feedback to the product team. Our approach includes both large-scale exploratory testing and the recently introduced ensemble exploratory testing. Following these sessions, developers can prioritize critical scenarios for automated end-to-end regression tests.
QA Forum¶
Our biweekly QA forum creates a space for collaborative learning and improvement in our testing practices. We'll be exploring topics like different testing approaches, analyzing real-world scenarios, discussing automation wins, introducing helpful tools, integrating testing earlier in development, and ensuring comprehensive quality through non-functional testing. Your insights and experiences are invaluable – we encourage everyone to participate!