Skip to content

Replace Karma with Vitest for JavaScript testing#2197

Merged
milmazz merged 3 commits intomainfrom
milton/js
Feb 9, 2026
Merged

Replace Karma with Vitest for JavaScript testing#2197
milmazz merged 3 commits intomainfrom
milton/js

Conversation

@milmazz
Copy link
Member

@milmazz milmazz commented Feb 9, 2026

This PR mainly tackles the following issues:

  • Replace Karma, which is deprecated, with a modern testing framework such as Vitest. Vitest provides native ESM support, jsdom for DOM testing, and eliminates the need for Chrome and xvfb-run in CI.
  • While I was working on this, I updated other JS dependencies to fix some vulnerabilities

Karma -> Vitest

  • Remove karma, karma-*, and mocha dependencies
  • Add vitest and jsdom
  • Translate Chai BDD assertions to Vitest matchers
  • Rename Mocha before to Vitest beforeAll
  • Simplify CI workflow by removing xvfb/Chrome setup

While we don't have a huge suite of tests for JS, we went from 17s to 2s when we run npm run --prefix assets test via CI.

Package updates

When I ran:

$ npm install --prefix assets                                                                                                                                                                                                                                                                                                              (base)
npm WARN deprecated glob@8.1.0: Glob versions prior to v9 are no longer supported
npm WARN deprecated text-encoding@0.6.4: no longer maintained
npm WARN deprecated samsam@1.3.0: This package has been deprecated in favour of @sinonjs/samsam
npm WARN deprecated formatio@1.2.0: This package is unmaintained. Use @sinonjs/formatio instead

added 117 packages, removed 1545 packages, changed 204 packages, and audited 366 packages in 4s

98 packages are looking for funding
  run `npm fund` for details

7 vulnerabilities (2 low, 5 high)

After running npm audit fix --prefix assets I got:

$ npm audit fix --prefix assets                                                                                                                                                                                                                                                                                                                (base)

added 2 packages, removed 2 packages, changed 9 packages, and audited 366 packages in 847ms

98 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

After installing the initial packages you get the following warning:

```
npm install --prefix assets                                                                                                                                                                                                                                                                                                              (base)
npm WARN deprecated glob@8.1.0: Glob versions prior to v9 are no longer supported
npm WARN deprecated text-encoding@0.6.4: no longer maintained
npm WARN deprecated samsam@1.3.0: This package has been deprecated in favour of @sinonjs/samsam
npm WARN deprecated formatio@1.2.0: This package is unmaintained. Use @sinonjs/formatio instead

added 117 packages, removed 1545 packages, changed 204 packages, and audited 366 packages in 4s

98 packages are looking for funding
  run `npm fund` for details

7 vulnerabilities (2 low, 5 high)
```

After running `npm audit fix --prefix assets` I got:

```
npm audit fix --prefix assets                                                                                                                                                                                                                                                                                                                (base)

added 2 packages, removed 2 packages, changed 9 packages, and audited 366 packages in 847ms

98 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
```
@github-actions
Copy link

github-actions bot commented Feb 9, 2026

### `test:coverage`

```bash
npm run --prefix assets test:coverage
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new script will allow developers to track coverage in case they want to verify or improve our current testing suite.

This is the current report in case you're curious:

npm run --prefix assets test:coverage                                                                                                                  (base)

> test:coverage
> vitest run --coverage


 RUN  v3.2.4 /Users/milmazz/Dev/elixir-lang/ex_doc/assets
      Coverage enabled with v8

 ✓ test/helpers.spec.js (1 test) 2ms
 ✓ test/autocomplete/suggestions.spec.js (16 tests) 6ms
 ✓ test/tooltips/hints.spec.js (2 tests) 7ms

 Test Files  3 passed (3)
      Tests  19 passed (19)
   Start at  23:45:56
   Duration  563ms (transform 39ms, setup 0ms, collect 67ms, tests 15ms, environment 628ms, prepare 127ms)

 % Coverage report from v8
----------------------------|---------|----------|---------|---------|----------------------------------------------------------------------
File                        | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------------------------|---------|----------|---------|---------|----------------------------------------------------------------------
All files                   |   17.22 |    68.46 |   37.14 |   17.22 |
 js                         |     7.3 |    21.73 |    12.5 |     7.3 |
  constants.js              |       0 |        0 |       0 |       0 | 1-7
  content.js                |       0 |        0 |       0 |       0 | 1-24
  copy-button.js            |       0 |        0 |       0 |       0 | 1-45
  copy-markdown.js          |       0 |        0 |       0 |       0 | 1-64
  globals.js                |   81.81 |    33.33 |      50 |   81.81 | 14-15
  helpers.js                |   53.24 |      100 |   21.42 |   53.24 | 46-47,58-128,137-139,154-157,166-171,192-202,210-211,219-221,232-242
  keyboard-shortcuts.js     |       0 |        0 |       0 |       0 | 1-126
  makeup.js                 |       0 |        0 |       0 |       0 | 1-28
  modal.js                  |       0 |        0 |       0 |       0 | 1-98
  quick-switch.js           |       0 |        0 |       0 |       0 | 1-239
  search-bar.js             |       0 |        0 |       0 |       0 | 1-335
  search-page.js            |       0 |        0 |       0 |       0 | 1-302
  settings-store.js         |       0 |        0 |       0 |       0 | 1-122
  settings.js               |       0 |        0 |       0 |       0 | 1-105
  swup.js                   |       0 |        0 |       0 |       0 | 1-40
  tabsets.js                |       0 |        0 |       0 |       0 | 1-147
  theme.js                  |       0 |        0 |       0 |       0 | 1-34
  toast.js                  |       0 |        0 |       0 |       0 | 1-25
 js/autocomplete            |   74.11 |    93.24 |   95.23 |   74.11 |
  autocomplete-list.js      |       0 |        0 |       0 |       0 | 1-108
  suggestions.js            |    98.9 |    94.52 |     100 |    98.9 | 154,183-184
 js/entry                   |       0 |        0 |       0 |       0 |
  epub.js                   |       0 |        0 |       0 |       0 | 1-6
  html.js                   |       0 |        0 |       0 |       0 | 1-19
  inline_html.js            |       0 |        0 |       0 |       0 | 1-34
 js/handlebars              |       0 |        0 |       0 |       0 |
  helpers.js                |       0 |        0 |       0 |       0 | 1-17
 js/sidebar                 |       0 |        0 |       0 |       0 |
  constants.js              |       0 |        0 |       0 |       0 | 1-7
  sidebar-drawer.js         |       0 |        0 |       0 |       0 | 1-128
  sidebar-list.js           |       0 |        0 |       0 |       0 | 1-263
  sidebar-version-select.js |       0 |        0 |       0 |       0 | 1-126
 js/tooltips                |   31.23 |    33.33 |   22.22 |   31.23 |
  hint-page.js              |       0 |        0 |       0 |       0 | 1-35
  hints.js                  |   66.24 |       50 |   28.57 |   66.24 | 51-63,73-80,88-91,99-133,139-142
  tooltips.js               |       0 |        0 |       0 |       0 | 1-177
----------------------------|---------|----------|---------|---------|----------------------------------------------------------------------

While the console report is a bit verbose, the tested modules have good coverage:

  • js/autocomplete/suggestions.js — 98.9% statements
  • js/globals.js — 81.8% statements
  • js/tooltips/hints.js — 66.2% statements
  • js/helpers.js — 53.2% statements

Overall is 17.2% across all 39 JS files, reflecting that only 3 modules are currently tested.

The command also provides an alternative view that you can browse by opening: assets/coverage/index.html

Image

@milmazz milmazz marked this pull request as ready for review February 9, 2026 06:01
@milmazz milmazz requested a review from josevalim February 9, 2026 06:04
Copy link
Member

@josevalim josevalim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beautiful!!!!

@milmazz milmazz merged commit d2587ce into main Feb 9, 2026
12 checks passed
@milmazz milmazz deleted the milton/js branch February 9, 2026 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants