Amazon Nova Act for QA Automation

Nova Act QA demonstrates how to use Amazon Nova Act to automate quality assurance (QA) and end-to-end testing in a web browser. Using the Amazon Nova Act SDK, this project leverages pytest for parallel test execution with custom HTML report generation via the pytest-html-nova-act plugin.

Repository Structure

sample-nova-act-qa/
├── pyproject.toml  # Project dependencies and pytest config
├── .env.sample     # Sample .env file to copy
└── src/
    └── nova_act_qa/
        ├── config/  # Configuration management
        ├── tests/   # Test suite directory
        │   ├── conftest.py  # Pytest fixtures
        └── utils/   # Utility functions and types

Pytest Plugins

The project uses several pytest plugins to enhance testing capabilities:

1. pytest-html:

   • Generates HTML test reports
   • Includes test results, durations, and failure details

2. pytest-html-nova-act:

   • Integrates the Nova Act SDK screenshots and log output with the pytest-html report

3. pytest-xdist:

   • Enables parallel test execution
   • Distributes tests across multiple CPU cores

These plugins are automatically installed with the project dependencies. No additional configuration is required.

Nova Act Test Methods

This project extends the base Nova Act SDK NovaAct class with enhanced testing capabilities:

1. test()

   • Flexible assertion with schema validation
   • Supports exact matching and contains operators
   • Handles primitive types and complex JSON structures

2. test_bool()

   • Simplified boolean assertions
   • Default expectation of True

3. test_str()

   • String-specific assertions
   • Supports exact and partial matching

Example usage:

nova.test_bool("Am I on the landing page?")
nova.test_str("Text input validation message", "Please enter a valid email address")
nova.test(
    "Product price tiers",
    [
        {"name": "Bronze", "price": 0.99},
        {"name": "Silver", "price": 4.99},
        {"name": "Gold", "price": 9.99}
    ],
    {
        "type": "array",
        "items": {
            "type": "object",
            "properties": {
                "name": {"type": "string"},
                "price": {"type": "number"}
            }
        }
    })

See src/nova_act_qa/utils/nova_act.py for complete reference

Project Usage

Prerequisites

• Python 3.11 or higher
• pip 23.0 or higher
• Operating system:
  • macOS (Sierra or later)
  • Ubuntu (22.04 LTS or later)
  • Windows:
    • Windows 10 or later
    • Windows Subsystem for Linux 2 (WSL2)
• Nova Act API key
  • Visit Nova Act home page to generate your API key

Installation

# Clone the repository
git clone <repository-url>
cd nova-act-qa

# Install dependencies using pip
pip install .

Environment setup

1. Create a .env file based on the provided template:

cp .env.sample .env

2. Edit your new .env file to include your NOVA_ACT_API_KEY
   • Visit Nova Act home page to generate your API key

Quick start

1. Run the test suite:

pytest

2. View test HTML report:

open reports/report.html

Test Output

Nova Act SDK logs

The project creates a .nova_act directory structure for each test in the project root. These directories include the Nova Act SDK HTML log output and browser user data directories:

.nova_act/
├── logs/                 # Test execution logs
│   └── {module}/{test}/  # Logs organized by module and test name
└── user-data-dir/        # Browser user data directories
    └── {module}/{test}/  # Separate profile per test

See the Nova Act SDK documentation for more details on logging.

Test report

An HTML report is automatically generated in the reports directory at the project root for each test run as configured in pyproject.toml. The report includes:

• Test results and durations
• Failure details and stack traces
• Nova Act SDK screenshots and logs

See the pytest-html documentation for more details.

Test Configuration

The pyproject.toml file has a [tool.pytest.ini_options] section which defines the pytest configuration for the tests. The default settings include:

• -n auto: Automatically detects CPU cores and runs tests in parallel
• --dist loadgroup: Distributes tests in the same group on the same worker to enable sequential runs
• --html=reports/report.html: Defines the location to write the pytest HTML report
• --self-contained-html: Creates a standalone HTML report file with embedded assets
• --add-nova-act-report: Integrates Nova Act screenshots and logs into the HTML report

This configuration can be extended to customize your tests as needed.

Test Grouping for Sequential Execution

When using pytest-xdist for parallel execution, tests that need to share data or run sequentially can be grouped using the @pytest.mark.xdist_group decorator. This feature requires the --dist loadgroup option (configured by default) to ensure tests in the same group run on the same worker process in order.

For example:

@pytest.mark.xdist_group("test_group")
def test_one():
    pass

@pytest.mark.xdist_group("test_group")
def test_two():
    pass

This ensures that dependent tests run sequentially while maintaining parallel execution for independent tests. See test_share_data.py for an example of this. See the pytest-xdist docs for more information.

Sharing Data Between Tests

Pytest provides a built-in cache that allows tests to share data within the same test run. This is useful for storing data that one test extracts and another test consumes.

def test_store_data(pytestconfig):
    pytestconfig.cache.set("key", "value")

def test_retrieve_data(pytestconfig):
    data = pytestconfig.cache.get("key", "default")

Note: When tests need to share data within the same test run, you must use the @pytest.mark.xdist_group decorator from the previous section to ensure they run on the same worker process. See test_share_data.py for a complete example of data sharing between dependent tests using both the cache and xdist grouping.

Test Automation

This project contains a GitHub Actions workflow template to automate running pytest and report generation when pull requests are made to the main branch.

The workflow file can be found in the root of the project: ~/.github/workflows/nova-act-qa.yml.

Prerequisites

The workflow requires configuring a GitHub Secret to store your Nova Act API Key:

1. Go to your Repository Settings
2. Navigate to Secrets and variables > Actions
3. Create a new Repository secret named NOVA_ACT_API_KEY
4. Input your API key value

More details can be found in the GitHub Documentation.

Workflow details

Trigger

The workflow is triggered on pull requests targeting the main branch

Jobs and steps

The workflow consists of a single job named test with multiple steps:

1. Checkout repo

   • Uses actions/checkout@v4 to fetch the repository code

2. Install Python 3.11

   • Uses actions/setup-python@v5 to set up Python 3.11
   • Configures pip caching using pyproject.toml

3. Install Python dependencies

   • Upgrades pip
   • Installs the project dependencies
   • Installs Nova Act package (version 1.x)

4. Playwright Setup

   • Gets installed Playwright version for caching
   • Caches Playwright using actions/cache@v4
   • Installs Playwright OS dependencies and Chromium if cache miss
   • Installs only OS dependencies if cache hit

5. Generate test identifier

   • Creates a unique identifier for the test run using timestamp and run ID

6. Run tests

   • Sets up environment variables for the Nova Act SDK
   • Runs pytest with HTML report generation
   • Spread parallel tests out equally across available CPUs

7. Upload report

   • Uses actions/upload-artifact@v4 to upload test results as GitHub artifacts
   • The report can be downloaded by navigating to this step in the workflow run summary

8. Check test results

   • Workflow fails if tests do not pass

Next Steps

Extend this repo and write your own tests in src/noca_act_qa/tests!

Additional Resources

• Amazon Nova Act
• Amazon Nova Act SDK
• pytest-html-nova-act