Migrate integration tests to use a containerised environment

[mridang]

Isolate Integration Tests with TestContainers

This initiative aims to refactor our integration tests to run against a Docker Compose setup using TestContainers instead of a production instance. This will address the issue of failing public contribution PR tests due to missing credentials and ensure consistent testing against a specific version of our dependencies.

Problem

Currently, our integration tests run against a production instance, leading to several challenges:

• Failing Public PR Tests: Public contribution pull requests fail because the necessary production instance tokens are not exposed to external contributors.
• Dependency on Cloud Instance: Tests are dependent on the availability and specific state of a cloud instance, which can lead to inconsistencies.
• Lack of Version Control: We cannot guarantee testing against a specific version of our dependencies (e.g., Zitadel) when relying on a cloud instance.

Impact

Migrating to a TestContainers-based Docker Compose setup will result in:

• Enabled Public Contributions: Public PRs will be able to run integration tests successfully, fostering community engagement.
• Consistent Testing Environment: Tests will run against a controlled, versioned environment, improving reliability.
• Reduced External Dependencies: Less reliance on external cloud services for CI/CD testing.
• Improved Test Reliability: Eliminates flakiness associated with external service dependencies.

Solution / Tasks

• Containerize Dependencies: Create Docker images or use existing ones for dependencies like Zitadel that are needed for integration tests.
• Implement TestContainers Setup: Configure integration tests to spin up the necessary Docker Compose services using TestContainers before test execution.
• Remove GitHub Environment Credentials: Eliminate sensitive production instance tokens from GitHub Actions environments.
• Update CI/CD Workflows: Adjust GitHub Actions to run integration tests against the TestContainers setup.
• Document Local Setup: Provide clear instructions for developers to run integration tests locally using Docker and TestContainers.
• Testing: Thoroughly test the new integration test setup in both local development and CI/CD pipelines to ensure correctness and stability.

Expected Outcomes

• All integration tests run successfully in public contribution PRs.
• Integration tests are executed against a consistent, versioned Dockerized environment.
• No sensitive production credentials are required for running integration tests in CI.
• Improved local development experience for running integration tests.
• Enhanced confidence in test results due to a controlled testing environment.