Skip to content

Migrate CI to RecordedTests by default #1390

@scbedd

Description

@scbedd

This issue covers the CI cutover/build definition updates that finalize the CI changes that are related to #1208 .

Where we're at

Here's the situation where we're at once #1208 is complete:

  • Any tests that can be recorded are. If a tool is changed, the recorded tests are run as a step in the public build.
  • There are exceptions due to non-http usage or not being factored properly for recordings.
    • Speech (websockets use for synthesizers)
    • PostGres (tcp database connection)
    • ConfidentialLedger (presence of a built-in internal identity client creating a new SSL cert for a connection. need to make this available externally somehow and utilize the testproxy adminclient to set the certificate before invoking the tests.)
  • The mcp - pullrequest - live build still triggers and runs relevant livetests for the change (including a default set of 3)

Where we want to go

I had a chat with @hallipr and I agree that we should go in the following direction.

  • Eliminate the public build entirely. Leave mcp - pullrequest - live triggered only for internal collaborators.
    • External contribution which is some tiny number of our PRs will need to be manually triggered. We should do something about copilot prs here though, as that is an avenue we want to continue to enhance.
  • Unit test invocation remains unchanged in the non-live portion of mcp - pullrequest - live (the Build stage)
  • For the packages that are changed, run recorded tests if they exist.
    • For packages that have no recorded tests, like PostGres and Speech, run them as a live portion of pullrequest - live.
    • This is a logical change from existing, as right now we run ALL livetests in the live portion of mcp - pullrequest - live regardless of presence of recorded tests.

This should get us quick builds where possible, while also ensuring we don't accidentally check in anything broken.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    Untriaged

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions