Skip to content

Regression test guidelines

Our regression tests are designed as End-to-End API tests, simulating real usage scenarios within the app. These tests aim to ensure that the APIs function correctly as they would in actual frontend use cases.

Key Features of Our Regression Tests

  1. Comprehensive Coverage:

    • Each test typically involves multiple API endpoints to complete a single scenario.
    • The goal is to ensure that all API endpoints are covered across the tests.

  2. High Efficiency in Test Automation and Maintenance:

    • The tests are structured to be highly efficient, both in terms of automation and ongoing maintenance.
    • This efficiency helps in quickly identifying bugs early in the development process.

  3. Integration with CI/CD Pipeline:

    • The tests are designed to run quickly and integrate seamlessly within the CI/CD pipeline.
    • This integration ensures that any issues are caught promptly during the continuous integration and deployment process.

  4. Scenario-Based Testing:

    • The tests aim to cover all API endpoints and workflows with a minimal number of scenarios.
    • By focusing on essential scenarios, we achieve comprehensive coverage with fewer tests.

  5. Real Usage Simulation:

    • The tests simulate the real usage of APIs as they would be utilized by the frontend of the application.
    • This approach ensures that the APIs are tested in the same way they are used in production.

Repositories

  • Area-Specific Regression Tests:
    • Lives in the area-specific repository.
    • Project names are RegressionTest and RegressionTest.Contract
  • Internal testing API:
    • Lives in be-internaltesting repository.
  • Smoke Tests:
    • Lives in be-smoketests repository.

Actions in regression test contract

  • Any API calling
  • Small Aggregates (Actions within an Area)
    • Activate Physical Card
    • Sign in with SMS
  • Larger Aggregates (Shared Across Areas)
    • CreateUser
    • DoRemittance

Request builder in regression test contract

  • Default test request body

Services access and DB connections from internal testing API

  • Service access Example:
    • GetUserInfoByMsisdn
  • DB connection example:
    • GetEmailVerificationSecret

Recommendations for Regression Test Creation

  • Reusable Test Steps:
    • Avoid Direct API Calls: Test steps should not directly call API endpoints. Instead, they should invoke methods defined in the regression test contract. This approach enhances maintainability and simplifies exploratory testing.
  • Clear Comments:
    • Write clear and concise comments for each test step.
    • Purpose: This makes the scenario easy to understand for anyone reviewing or maintaining the tests.
  • Correct Assertions:
    • Include assertions to validate:
      • API Responses: Ensure that the API responses are as expected.
      • Database State: Validate the SQL database for correct data insertion, update, or deletion.
      • Balance in Firebase: Check the balance in Firebase, if applicable.
    • Negative Case Handling: Include assertions for exceptions or error responses in negative test cases to ensure robustness.
  • Use runsettings:
    • Use a .runsettings file to make the regression test use <ASPNETCORE_ENVIRONMENT>Staging</ASPNETCORE_ENVIRONMENT> so that when running it uses the services deployed in Staging by default.

Reference