Skip to content

Conversation

@yasserfaraazkhan
Copy link
Contributor

@yasserfaraazkhan yasserfaraazkhan commented Sep 22, 2025

This pull request introduces significant improvements to the Electron end-to-end (E2E) testing infrastructure, focusing on increased reliability, better cross-platform support (especially for macOS-15), and improved test stability. The changes span CI workflow enhancements, E2E environment robustness, dependency updates, and targeted test reliability fixes.

CI Workflow Improvements:

  • Refactored .github/workflows/e2e-functional.yml to consolidate and generalize status updates for all platforms, using actions/github-script to dynamically update commit statuses for each OS, and removed redundant jobs for individual platforms. [1] [2]
  • Enhanced macOS preparation in .github/workflows/e2e-functional-template.yml by adding explicit cleanup of Electron system folders, ensuring Electron.app is properly configured and signed on macOS-15, and improving build and test steps for reliability.
  • Set NODE_ENV: "test" in workflow environments for consistency across runs.

E2E Environment Robustness:

  • Improved cleanup routines in e2e/modules/environment.js to remove Electron single-instance lock files on macOS, preventing launch failures, and switched to more robust directory removal methods. [1] [2] [3]
  • Enhanced Electron launch options with additional flags for stability and security, increased timeouts for slow macOS-15 initialization, and improved window readiness polling to ensure tests only proceed when the app is ready.

Dependency Updates:

  • Updated several dependencies in e2e/package.json, including @aws-sdk, axios, chai, fast-xml-parser, mochawesome, nan, node-abi, playwright, and mochawesome-report-generator, to address security, compatibility, and feature improvements.
  • Adjusted E2E test scripts to include additional Electron flags for better compatibility in CI environments.

Test Stability and Reliability:

  • Added retries and waiting logic in deeplink.test.js and focus.test.js to handle asynchronous server map population and slow initialization, especially on Windows and macOS, reducing flaky test failures. [1] [2] [3]

Other Notable Changes:

  • Removed the pull_request trigger from the E2E workflow, making it only runnable via manual dispatch.
  • Improved environment variable handling and resource paths for tests, ensuring consistency and reliability.

These changes collectively make the E2E testing pipeline more robust, especially on macOS-15, and reduce sources of test flakiness across platforms.

NONE

@yasserfaraazkhan yasserfaraazkhan added the E2E/Run Run Desktop E2E Tests label Sep 22, 2025
@yasserfaraazkhan yasserfaraazkhan added E2E/Run Run Desktop E2E Tests and removed E2E/Run Run Desktop E2E Tests labels Sep 22, 2025
@yasserfaraazkhan yasserfaraazkhan changed the title Add workflow_call to e2e yml E2E: Skip the notification prompt when NODE_ENV= test Sep 22, 2025
name: Electron Playwright Tests

on:
pull_request:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when we add label it was creating multiple jobs in delivery platform. I've remove this trigger type

image

Copy link
Contributor

@saturninoabril saturninoabril left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// For certain permission types, we need to confirm with the user
if (authorizablePermissionTypes.includes(permission) || isExternalFullscreen) {
// In test mode, do not show dialog, always allow
if (process.env.NODE_ENV === 'test') {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@enzowritescode Are we okay with this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@enzowritescode double checking with you on this code

@yasserfaraazkhan yasserfaraazkhan removed the E2E/Run Run Desktop E2E Tests label Nov 14, 2025
@yasserfaraazkhan yasserfaraazkhan removed the E2E/Run Run Desktop E2E Tests label Dec 15, 2025
@devinbinnie
Copy link
Member

@yasserfaraazkhan Where are we on the changes here?

@yasserfaraazkhan
Copy link
Contributor Author

@yasserfaraazkhan Where are we on the changes here?

@devinbinnie I've run the tests. Can you check the reports?
I see just a 1 crash in macos and windows. Rest seems to be some failure. I'll spend time to run them and check locally as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants