Samuel Attard
8287520b9b
* test: assert with vitest's matchers instead of chai
Every assertion moves from chai's chainable language to vitest's own
matchers, so an assertion is always a call and never a bare property read:
- `to.equal` → `toBe`, `to.deep.equal`/`eql` → `toEqual`, `to.be.true` →
`toBe(true)`, `to.have.property` → `toHaveProperty`, `to.include` →
`toContain`/`toMatchObject`/`toContainEqual`, `to.be.a('string')` →
`toBeTypeOf('string')`, `instanceOf` → `toBeInstanceOf`, `lengthOf` →
`toHaveLength`, `above`/`least`/`below`/`most` → `toBeGreaterThan` and
friends, `match` → `toMatch`, `throw` → `toThrow`, `oneOf` → `toContain`
with the operands swapped, `closeTo(e, d)` → `Math.abs(x - e) <= d`,
`members`/`keys` → `arrayContaining`/`Object.keys`. A chain that kept
going after `.property()` or `.a()` becomes one matcher where vitest has
one (`toHaveProperty(key, expect.any(String))`) and separate assertions
otherwise. chai-as-promised becomes `.resolves`/`.rejects`, a failed
manual check is `assert.fail()` rather than `new chai.AssertionError`.
- vitest reports promise assertions that are not awaited; the app badge,
webview zoom and `import('electron')` specs had some that never ran and
now do (the service worker badge ones waited for a count the worker had
already cleared, so those just check the worker's answer; the zoom fixture
issued loadURL() from dom-ready, which aborts, and now waits for the load).
- Code that runs outside the spec module graph (itremote bodies in a
renderer, the utility-process net harness, two crash-case apps) is
CommonJS and vitest refuses `require('vitest')`, so it loads the same
module through spec/lib/vitest-cjs.cjs.
- A lint rule (script/lint-plugins/no-chai-assertions.mjs) rejects chai
chains on `expect()`, and no-unused-expressions is on for the specs, so
neither form of property-style assertion can come back.
- chai, chai-as-promised, dirty-chai and their @types leave
spec/package.json; the refresh-page fixture served chai.js only as "a
large script" and serves the jquery fixture instead.
* test: return promises instead of taking mocha's `done` callback
vitest has no `done` parameter. The tests and hooks that used one now
return a promise instead; where the callback was only ever called as
`done()` it is the promise's `resolve`, and where it was also handed to an
error-first API or called as `done(error)` it keeps mocha's meaning
(reject on a truthy argument, resolve otherwise) through a local `done`.
* test: move the spec runner's own semantics out of the mocha shim
Three things the shim did are not mocha compatibility but how the Electron
specs need to run, whichever API they are written against: every hook and
test starts from a fresh macrotask so that Chromium's message loop turns
between them, `defer()`-ed cleanup runs before the enclosing suites' own
afterEach hooks, and spec/disabled-tests.json can switch a test off by its
title. Those now live in a small custom vitest runner
(spec/vitest/runner.ts) that applies to every test, and the shim only
translates mocha's API.
* test: write the specs against vitest's API instead of mocha's
Mechanical conversion of every spec file and helper:
- `describe`/`it`/`beforeEach`/... are imported from 'vitest' rather than
taken from the runner's globals; `before`/`after` become
`beforeAll`/`afterAll`.
- `this.timeout(n)` becomes the `{ timeout: n }` option of the test or
suite (a hook's second argument; hooks inside a suite that had a timeout
get it explicitly, since vitest's suite option only reaches the tests), `this.retries(n)` `{ retry: n }`, and
`this.skip()` the test context's `ctx.skip()`. The two `before` hooks that
skipped a whole suite now compute the condition up front
(`describe.runIf`) or per test. `function () {}` callbacks that only
existed for `this` are arrows again.
- `ifit`/`ifdescribe` map to `it.skip`/`describe.skip`, mocha's chained
`it(...).timeout(n)` and `this.bail()` are gone, and the two helpers that
read `this.test` use the context's task state.
* test: remove the mocha compatibility layer
Nothing imports mocha's API any more, so spec/vitest/mocha-compat.ts and the
setup file that installed its globals go, together with @types/mocha and the
Mocha type overloads in spec/ambient.d.ts. Timeouts are vitest's own again
(30 s per test and hook, ELECTRON_SPEC_TIMEOUT raises it for the sanitizer
jobs, replacing MOCHA_TIMEOUT), the junit path variable becomes
ELECTRON_SPEC_JUNIT_FILE, and `closeAllWindows` loses the parameter that a
hook's context argument would otherwise land in.
4.9 KiB
Testing
We aim to keep the code coverage of Electron high. We ask that all pull request not only pass all existing tests, but ideally also add new tests to cover changed code and new scenarios. Ensuring that we capture as many code paths and use cases of Electron as possible ensures that we all ship apps with fewer bugs.
This repository comes with linting rules for both JavaScript and C++ – as well as unit and integration tests. To learn more about Electron's coding style, please see the coding-style document.
Linting
To ensure that your changes are in compliance with the Electron coding
style, run npm run lint, which will run a variety of linting checks
against your changes depending on which areas of the code they touch.
Many of these checks are included as precommit hooks, so it's likely your error would be caught at commit time.
Unit Tests
If you are not using build-tools,
ensure that the name you have configured for your
local build of Electron is one of Testing, Release, Default, or
you have set process.env.ELECTRON_OUT_DIR. Without these set, Electron will fail
to perform some pre-testing steps.
To run all unit tests, run npm run test. The unit tests are an Electron
app (surprise!) that can be found in the spec folder. Note that it has
its own package.json and that its dependencies are therefore not defined
in the top-level package.json.
The suite is driven by vitest: the vitest CLI runs
under Node.js and starts several instances of the spec app, each of which
runs one spec file at a time in Electron's main process. Suites that need the
machine to themselves (window focus, the clipboard, the screen, global
shortcuts) are declared with describe(title, { tags: ['serial'] }, fn) and
run one at a time after everything else has finished. Set
ELECTRON_SPEC_WORKERS to change how many Electron processes run at once.
To run only some spec files, pass them with --files:
npm run test -- --files spec/api-app.spec.ts. To run only tests whose
name matches a pattern, run npm run test -- -g=PATTERN, replacing
PATTERN with a regex. As an example: If you want to run only IPC tests, you
would run npm run test -- -g ipc.
Specs import describe, it, expect and the hooks from vitest and assert
with its matchers (expect(x).toBe(y), toEqual, toHaveProperty, ...);
chai-style chains (expect(x).to.equal(y)) are rejected by the linter because
their property form can silently assert nothing. A few Electron specific
behaviours (a macrotask between tests, defer() cleanup,
spec/disabled-tests.json) live in spec/vitest/runner.ts.
Node.js Smoke Tests
If you've made changes that might affect the way Node.js is embedded into Electron, we have a test runner that runs all of the tests from Node.js, using Electron's custom fork of Node.js.
To run all of the Node.js tests:
$ node script/node-spec-runner.js
To run a single Node.js test:
$ node script/node-spec-runner.js parallel/test-crypto-keygen
where the argument passed to the runner is the path to the test in the Node.js source tree.
Testing on Windows 10 devices
Extra steps to run the unit test:
-
Visual Studio 2019 must be installed.
-
Node headers have to be compiled for your configuration.
ninja -C out\Testing electron:node_headers -
The electron.lib has to be copied as node.lib.
cd out\Testing mkdir gen\node_headers\Release copy electron.lib gen\node_headers\Release\node.lib
Missing fonts
Some Windows 10 devices do not ship with the Meiryo font installed, which may cause a font fallback test to fail. To install Meiryo:
- Push the Windows key and search for Manage optional features.
- Click Add a feature.
- Select Japanese Supplemental Fonts and click Install.
Pixel measurements
Some tests which rely on precise pixel measurements may not work correctly on devices with Hi-DPI screen settings due to floating point precision errors. To run these tests correctly, make sure the device is set to 100% scaling.
To configure display scaling:
- Push the Windows key and search for Display settings.
- Under Scale and layout, make sure that the device is set to 100%.
Multi-Monitor Tests
Some Electron APIs require testing across multiple displays, such as screen detection, window positioning, and display-related events. For contributors working on these features, the virtualDisplay native addon enables you to create and position virtual displays programmatically, making it possible to test multi-monitor scenarios without any physical hardware.
For detailed information on using virtual displays in your tests, see Multi-Monitor Testing.
Platform support: macOS only