mirror of
https://github.com/nodejs/node.git
synced 2026-10-28 10:15:07 +08:00
test_runner: extend tag filter with boolean expression DSL
Upgrades the experimental tag filter introduced by stage 1 to accept a boolean expression instead of a literal tag name. Grammar: `and`/`&&`, `or`/`||`, `not`/`!`, parentheses for grouping, and `*` wildcards inside identifiers. Standard precedence (`not > and > or`); binary operators are left-associative. Word forms require whitespace separation; punctuation forms do not. Untagged tests evaluate `false` for any include expression and `true` for `not X`, so excluding tags does not accidentally remove untagged tests. The flag and `testTagFilters` option are still repeatable; multiple expressions still AND together. Malformed expressions fail fast at the parent process at startup. Tag value validation tightens to reject whitespace, operator characters (`& | ! ( ) *`), and the reserved words `and`/`or`/`not` in any casing - a breaking change relative to the stage 1 ship, acceptable at Stability 1.0 (Early development). Signed-off-by: atlowChemi <chemi@atlow.co.il> PR-URL: https://github.com/nodejs/node/pull/63054 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
This commit is contained in:
1 parent
1d9d49c1be
commit
a674c5d1a3
11 files changed
+928
-105
No files matched your search
+14
-8
@@ -1499,7 +1499,7 @@ Enable module mocking in the test runner.
|
||||
|
||||
This feature requires `--allow-worker` if used with the [Permission Model][].
|
||||
|
||||
### `--experimental-test-tag-filter=<tag>`
|
||||
### `--experimental-test-tag-filter='<expr>'`
|
||||
|
||||
<!-- YAML
|
||||
added:
|
||||
@@ -1509,14 +1509,20 @@ added:
|
||||
|
||||
> Stability: 1.0 - Early development
|
||||
|
||||
Run only tests whose tag set contains `<tag>`. Tests declare tags via the
|
||||
`tags` option on `test()`, `it()`, `suite()`, or `describe()`; tags
|
||||
inherit from suites to nested tests by union. Filtering is
|
||||
case-insensitive.
|
||||
Run only tests that match the provided boolean tag-filter expression. Tests
|
||||
declare tags via the `tags` option on `test()`, `it()`, `suite()`, or
|
||||
`describe()`. Tags inherit from suites to nested tests by union.
|
||||
|
||||
The flag may be specified more than once; tests must contain **every**
|
||||
filter value to run. See [Test tags][] for details on declaring and
|
||||
inheriting tags.
|
||||
The expression supports boolean operators (`and`/`&&`, `or`/`||`,
|
||||
`not`/`!`), parentheses for grouping, and `*` wildcards inside identifiers.
|
||||
Standard precedence applies: `not` binds tighter than `and`, which binds
|
||||
tighter than `or`. See [Test tags][] for the full grammar and behavior.
|
||||
|
||||
The flag may be specified more than once; multiple expressions are combined
|
||||
with AND, so a test must satisfy every expression to run.
|
||||
|
||||
A malformed expression causes the test runner to exit with a non-zero status
|
||||
before running any tests.
|
||||
|
||||
### `--experimental-vfs`
|
||||
|
||||
|
||||
+76
-23
@@ -491,8 +491,8 @@ added:
|
||||
|
||||
Tags annotate tests and suites with arbitrary string labels. The
|
||||
[`--experimental-test-tag-filter`][] CLI flag (or the `testTagFilters`
|
||||
option on [`run()`][]) selects tests whose tag set contains every
|
||||
provided filter value.
|
||||
option on [`run()`][]) selects tests by a boolean expression over those
|
||||
labels.
|
||||
|
||||
Tags are an alternative to encoding metadata into test names. They are
|
||||
useful for cross-cutting axes such as subsystem, speed bucket, flakiness,
|
||||
@@ -525,37 +525,89 @@ describe('database', { tags: ['db'] }, () => {
|
||||
});
|
||||
```
|
||||
|
||||
Tag values must be non-empty strings. Tags are matched case-insensitively;
|
||||
the canonical form is lowercase. Duplicates within a single `tags` array
|
||||
are collapsed on the lowercased form, preserving the first-seen
|
||||
declaration order.
|
||||
Tag values must be non-empty strings that contain no whitespace, no
|
||||
operator characters (`& | ! ( ) *`), and are not the reserved words
|
||||
`'and'`, `'or'`, or `'not'` in any casing. Tags are matched
|
||||
case-insensitively; the canonical form is lowercase. Duplicates within a
|
||||
single `tags` array are collapsed on the lowercased form, preserving the
|
||||
first-seen declaration order.
|
||||
|
||||
Hooks (`before`, `after`, `beforeEach`, `afterEach`) do not declare their
|
||||
own tags. They run as part of their owning suite, which carries the
|
||||
suite's tags.
|
||||
|
||||
### Filtering by tag
|
||||
### Filtering syntax
|
||||
|
||||
Each [`--experimental-test-tag-filter`][] value is a literal tag name. A
|
||||
test runs only when its tag set contains that name. The flag may be
|
||||
specified more than once; tests must match **every** filter to run. The
|
||||
same applies to the `testTagFilters` array on [`run()`][]. Filters are
|
||||
case-insensitive and AND'd with [`--test-name-pattern`][],
|
||||
[`--test-skip-pattern`][], and `.only` filtering.
|
||||
The filter expression supports:
|
||||
|
||||
Untagged tests are excluded under any non-empty filter, since the filter
|
||||
requires the tag to be present.
|
||||
* Identifiers—any non-whitespace, non-operator characters. A literal
|
||||
identifier matches a tag of the same value (case-insensitive).
|
||||
* `*` wildcards inside an identifier match any sequence of characters.
|
||||
A bare `*` matches any tagged test.
|
||||
* Boolean operators with two equivalent forms:
|
||||
* `and` / `&&`
|
||||
* `or` / `||`
|
||||
* `not` / `!`
|
||||
* Parentheses for grouping.
|
||||
|
||||
### Reading tags from inside a test
|
||||
The word forms (`and`, `or`, `not`) require whitespace separation; the
|
||||
punctuation forms do not.
|
||||
|
||||
#### Operator precedence
|
||||
|
||||
The expression is evaluated with the standard precedence
|
||||
`not > and > or`. Binary operators are left-associative.
|
||||
|
||||
| Expression | Equivalent grouping |
|
||||
| -------------- | ------------------- |
|
||||
| `a or b and c` | `a or (b and c)` |
|
||||
| `not a and b` | `(not a) and b` |
|
||||
|
||||
Use parentheses to override:
|
||||
|
||||
| Expression | Selects |
|
||||
| ------------------------------ | ------------------------------------------ |
|
||||
| `(unit or smoke) and not slow` | unit-or-smoke tests that are not also slow |
|
||||
| `db && !flaky` | db tests that are not flaky |
|
||||
| `*` | every tagged test |
|
||||
|
||||
#### Untagged tests
|
||||
|
||||
Untagged tests behave as if they have an empty tag set. As a result:
|
||||
|
||||
| Filter expression | Untagged test | Why |
|
||||
| ------------------------ | ------------- | ------------------------------------------------ |
|
||||
| `db` | excluded | Positive match against an empty tag set is false |
|
||||
| `*` | excluded | The bare wildcard requires at least one tag |
|
||||
| `db or unit` | excluded | Both branches are false against an empty tag set |
|
||||
| `not flaky` | included | Negation against an empty tag set is true |
|
||||
| `not flaky and not slow` | included | Both negations are true against an empty tag set |
|
||||
| `db or not flaky` | included | The negated branch is true |
|
||||
|
||||
For example, `--experimental-test-tag-filter='not flaky'` runs every test
|
||||
that is not tagged `flaky`, including all untagged tests.
|
||||
|
||||
#### Composing multiple filters
|
||||
|
||||
[`--experimental-test-tag-filter`][] may be specified more than once on the
|
||||
command line. Multiple expressions compose by AND—a test must satisfy
|
||||
every expression to run. The same applies to passing an array to
|
||||
`testTagFilters` on [`run()`][]. The tag filter is also AND'd with
|
||||
[`--test-name-pattern`][], [`--test-skip-pattern`][], and `.only`
|
||||
filtering.
|
||||
|
||||
#### Reading tags from inside a test
|
||||
|
||||
The [`TestContext`][] object exposes the test's tags as a frozen array
|
||||
through [`context.tags`][], so tests can branch on their own metadata.
|
||||
|
||||
### Errors
|
||||
#### Errors
|
||||
|
||||
A tag value that violates the validation rules above throws
|
||||
`ERR_INVALID_ARG_VALUE` at the registration site, before any test runs.
|
||||
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`.
|
||||
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`. A malformed
|
||||
filter expression on the CLI causes the test runner to exit with a
|
||||
non-zero status before running any test files.
|
||||
|
||||
## Extraneous asynchronous activity
|
||||
|
||||
@@ -830,7 +882,7 @@ test runner functionality:
|
||||
|
||||
* `--test` - Prevented to avoid recursive test execution
|
||||
* `--experimental-test-coverage` - Managed by the test runner
|
||||
* `--experimental-test-tag-filter` - Filter values are validated by the parent
|
||||
* `--experimental-test-tag-filter` - Filter expressions are validated by the parent
|
||||
process and re-emitted to child processes
|
||||
* `--watch` - Watch mode is handled at the parent level
|
||||
* `--experimental-default-config-file` - Config file loading is handled by the parent
|
||||
@@ -1746,10 +1798,11 @@ changes:
|
||||
For each test that is executed, any corresponding test hooks, such as
|
||||
`beforeEach()`, are also run.
|
||||
**Default:** `undefined`.
|
||||
* `testTagFilters` {string|string\[]} A tag name, or an array of tag names,
|
||||
used to filter tests by their declared tags. Tests must contain every
|
||||
listed tag to run. Equivalent to passing [`--experimental-test-tag-filter`][]
|
||||
on the command line. See [Test tags][]. **Default:** `undefined`.
|
||||
* `testTagFilters` {string|string\[]} A boolean expression, or an array of
|
||||
boolean expressions, used to filter tests by their declared tags.
|
||||
Multiple expressions compose by AND. Equivalent to passing
|
||||
[`--experimental-test-tag-filter`][] on the command line. See
|
||||
[Test tags][]. **Default:** `undefined`.
|
||||
* `timeout` {number} A number of milliseconds the test execution will
|
||||
fail after.
|
||||
If unspecified, subtests inherit this value from their parent.
|
||||
|
||||
+12
-8
@@ -814,14 +814,18 @@ collecting code coverage from tests for more details.
|
||||
Enable module mocking in the test runner.
|
||||
This feature requires \fB--allow-worker\fR if used with the Permission Model.
|
||||
.
|
||||
.It Fl -experimental-test-tag-filter Ns = Ns Ar <tag>
|
||||
Run only tests whose tag set contains \fB<tag>\fR. Tests declare tags via the
|
||||
\fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or \fBdescribe()\fR; tags
|
||||
inherit from suites to nested tests by union. Filtering is
|
||||
case-insensitive.
|
||||
The flag may be specified more than once; tests must contain \fBevery\fR
|
||||
filter value to run. See Test tags for details on declaring and
|
||||
inheriting tags.
|
||||
.It Fl -experimental-test-tag-filter Ns = Ns Ar '<expr>'
|
||||
Run only tests that match the provided boolean tag-filter expression. Tests
|
||||
declare tags via the \fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or
|
||||
\fBdescribe()\fR. Tags inherit from suites to nested tests by union.
|
||||
The expression supports boolean operators (\fBand\fR/\fB&&\fR, \fBor\fR/\fB||\fR,
|
||||
\fBnot\fR/\fB!\fR), parentheses for grouping, and \fB*\fR wildcards inside identifiers.
|
||||
Standard precedence applies: \fBnot\fR binds tighter than \fBand\fR, which binds
|
||||
tighter than \fBor\fR. See Test tags for the full grammar and behavior.
|
||||
The flag may be specified more than once; multiple expressions are combined
|
||||
with AND, so a test must satisfy every expression to run.
|
||||
A malformed expression causes the test runner to exit with a non-zero status
|
||||
before running any tests.
|
||||
.
|
||||
.It Fl -experimental-vfs
|
||||
Enable the experimental \fBnode:vfs\fR module.
|
||||
|
||||
@@ -12,6 +12,7 @@ const {
|
||||
ArrayPrototypePush,
|
||||
ArrayPrototypePushApply,
|
||||
ArrayPrototypeShift,
|
||||
ArrayPrototypeSlice,
|
||||
ArrayPrototypeSome,
|
||||
ArrayPrototypeSort,
|
||||
MathMax,
|
||||
@@ -93,7 +94,7 @@ const {
|
||||
parseCommandLine,
|
||||
} = require('internal/test_runner/utils');
|
||||
const {
|
||||
validateAndCanonicalizeTagFilter,
|
||||
parseTagFilterExpression,
|
||||
} = require('internal/test_runner/tag_filter');
|
||||
const { Glob } = require('internal/fs/glob');
|
||||
const { once } = require('events');
|
||||
@@ -182,7 +183,7 @@ function getRunArgs(path, { forceExit,
|
||||
inspectPort,
|
||||
testNamePatterns,
|
||||
testSkipPatterns,
|
||||
testTagFilters,
|
||||
testTagFilterExpressions,
|
||||
only,
|
||||
hasFiles,
|
||||
testFiles,
|
||||
@@ -224,8 +225,8 @@ function getRunArgs(path, { forceExit,
|
||||
if (testSkipPatterns != null) {
|
||||
ArrayPrototypeForEach(testSkipPatterns, (pattern) => ArrayPrototypePush(runArgs, `--test-skip-pattern=${pattern}`));
|
||||
}
|
||||
if (testTagFilters != null) {
|
||||
ArrayPrototypeForEach(testTagFilters, (value) => ArrayPrototypePush(runArgs, `--experimental-test-tag-filter=${value}`));
|
||||
if (testTagFilterExpressions != null) {
|
||||
ArrayPrototypeForEach(testTagFilterExpressions, (expr) => ArrayPrototypePush(runArgs, `--experimental-test-tag-filter=${expr}`));
|
||||
}
|
||||
if (only === true) {
|
||||
ArrayPrototypePush(runArgs, '--test-only');
|
||||
@@ -872,19 +873,37 @@ function run(options = kEmptyObject) {
|
||||
});
|
||||
}
|
||||
|
||||
// The public contract of testTagFilters is `string | string[]`. The
|
||||
// parseCommandLine bootstrap path piggybacks the already-parsed AST array
|
||||
// on the same field, identifiable by the sibling testTagFilterExpressions
|
||||
// field which only that path sets. When that marker is present and the
|
||||
// first element isn't a string, treat the array as ASTs and skip the
|
||||
// public validation loop. Otherwise validate every element as a string,
|
||||
// so any non-string input throws ERR_INVALID_ARG_TYPE with the offending
|
||||
// index regardless of position.
|
||||
let testTagFilterExpressions = null;
|
||||
if (testTagFilters != null) {
|
||||
if (!ArrayIsArray(testTagFilters)) {
|
||||
testTagFilters = [testTagFilters];
|
||||
}
|
||||
if (testTagFilters.length === 0) {
|
||||
testTagFilters = null;
|
||||
} else if (options.testTagFilterExpressions != null &&
|
||||
typeof testTagFilters[0] !== 'string') {
|
||||
// Internal bootstrap: trust the AST array as already-parsed.
|
||||
} else {
|
||||
emitExperimentalWarning('Test tags');
|
||||
testTagFilters = ArrayPrototypeMap(testTagFilters, (value, i) => (
|
||||
validateAndCanonicalizeTagFilter(value, `options.testTagFilters[${i}]`)
|
||||
));
|
||||
testTagFilterExpressions = ArrayPrototypeSlice(testTagFilters);
|
||||
testTagFilters = ArrayPrototypeMap(testTagFilters, (value, i) => {
|
||||
const name = `options.testTagFilters[${i}]`;
|
||||
if (typeof value !== 'string') {
|
||||
throw new ERR_INVALID_ARG_TYPE(name, 'string', value);
|
||||
}
|
||||
return parseTagFilterExpression(value, name);
|
||||
});
|
||||
}
|
||||
}
|
||||
testTagFilterExpressions ??= options.testTagFilterExpressions;
|
||||
|
||||
validateOneOf(isolation, 'options.isolation', ['process', 'none']);
|
||||
validateBoolean(coverage, 'options.coverage');
|
||||
@@ -986,7 +1005,7 @@ function run(options = kEmptyObject) {
|
||||
inspectPort,
|
||||
testNamePatterns,
|
||||
testSkipPatterns,
|
||||
testTagFilters,
|
||||
testTagFilterExpressions,
|
||||
hasFiles: files != null,
|
||||
globPatterns,
|
||||
only,
|
||||
|
||||
@@ -2,8 +2,14 @@
|
||||
const {
|
||||
ArrayPrototypePush,
|
||||
ObjectFreeze,
|
||||
RegExp,
|
||||
RegExpPrototypeExec,
|
||||
SafeSet,
|
||||
StringPrototypeCharCodeAt,
|
||||
StringPrototypeSlice,
|
||||
StringPrototypeSplit,
|
||||
StringPrototypeToLowerCase,
|
||||
StringPrototypeTrim,
|
||||
} = primordials;
|
||||
|
||||
const {
|
||||
@@ -19,14 +25,68 @@ const {
|
||||
validateString,
|
||||
} = require('internal/validators');
|
||||
|
||||
const kCharAmpersand = 0x26;
|
||||
const kCharBackslash = 0x5C;
|
||||
const kCharCarriageReturn = 0x0D;
|
||||
const kCharCaret = 0x5E;
|
||||
const kCharDollar = 0x24;
|
||||
const kCharDot = 0x2E;
|
||||
const kCharExclaim = 0x21;
|
||||
const kCharFormFeed = 0x0C;
|
||||
const kCharLBrace = 0x7B;
|
||||
const kCharLBracket = 0x5B;
|
||||
const kCharLineFeed = 0x0A;
|
||||
const kCharLParen = 0x28;
|
||||
const kCharPipe = 0x7C;
|
||||
const kCharPlus = 0x2B;
|
||||
const kCharQuestion = 0x3F;
|
||||
const kCharRBrace = 0x7D;
|
||||
const kCharRBracket = 0x5D;
|
||||
const kCharRParen = 0x29;
|
||||
const kCharSpace = 0x20;
|
||||
const kCharStar = 0x2A;
|
||||
const kCharTab = 0x09;
|
||||
const kCharVerticalTab = 0x0B;
|
||||
|
||||
// Single `&` and `|` are allowed here: standalone they have no operator
|
||||
// meaning, only the doubled forms `&&` and `||` do.
|
||||
const kTagForbiddenChars = new SafeSet([
|
||||
kCharTab, kCharLineFeed, kCharVerticalTab, kCharFormFeed, kCharCarriageReturn, kCharSpace,
|
||||
kCharAmpersand, kCharExclaim, kCharLParen, kCharPipe, kCharRParen, kCharStar,
|
||||
]);
|
||||
|
||||
const kReservedWords = new SafeSet(['and', 'or', 'not']);
|
||||
|
||||
/**
|
||||
* Frozen, shared empty array used as the canonical "no tags" value for
|
||||
* tests that don't declare their own and have no tagged ancestors.
|
||||
* Sharing one frozen instance avoids per-test allocation.
|
||||
* @type {readonly string[]}
|
||||
*/
|
||||
const kEmptyTagArray = ObjectFreeze([]);
|
||||
|
||||
function isWhitespace(ch) {
|
||||
return ch === kCharTab || ch === kCharLineFeed || ch === kCharVerticalTab ||
|
||||
ch === kCharFormFeed || ch === kCharCarriageReturn || ch === kCharSpace;
|
||||
}
|
||||
|
||||
// `*` is intentionally not a break char: it's allowed inside an identifier
|
||||
// and becomes a wildcard.
|
||||
function isIdentBreak(ch) {
|
||||
return isWhitespace(ch) ||
|
||||
ch === kCharAmpersand || ch === kCharPipe ||
|
||||
ch === kCharExclaim || ch === kCharLParen || ch === kCharRParen;
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate and canonicalize a `tags` option value supplied to `test()`,
|
||||
* `it()`, `suite()`, or `describe()`.
|
||||
*
|
||||
* Each tag must be a non-empty string. Tags are canonicalized to lowercase
|
||||
* and deduplicated, preserving first-seen declaration order.
|
||||
* Each tag must be a non-empty string that contains no whitespace, no
|
||||
* operator characters (`& | ! ( ) *`), and is not the reserved word
|
||||
* `'and'`, `'or'`, or `'not'` in any casing. Tags are canonicalized to
|
||||
* lowercase and deduplicated on the lowercased form, preserving the
|
||||
* first-seen declaration order.
|
||||
*
|
||||
* Calling this with a non-empty array also emits the one-shot
|
||||
* `ExperimentalWarning` for the test-tags feature.
|
||||
@@ -37,7 +97,8 @@ const kEmptyTagArray = ObjectFreeze([]);
|
||||
* Returns an empty array when `tags` is `[]`.
|
||||
* @throws {ERR_INVALID_ARG_TYPE} If `tags` is not an array, or any element
|
||||
* is not a string.
|
||||
* @throws {ERR_INVALID_ARG_VALUE} If any element is empty.
|
||||
* @throws {ERR_INVALID_ARG_VALUE} If any element is empty, contains a
|
||||
* forbidden character, or is a reserved word in any casing.
|
||||
*/
|
||||
function validateAndCanonicalizeTagValues(tags, optName) {
|
||||
validateArray(tags, optName);
|
||||
@@ -55,7 +116,34 @@ function validateAndCanonicalizeTagValues(tags, optName) {
|
||||
const tag = tags[i];
|
||||
const elemName = `${optName}[${i}]`;
|
||||
|
||||
const lower = validateAndCanonicalizeTagFilter(tag, elemName);
|
||||
validateString(tag, elemName);
|
||||
|
||||
if (tag.length === 0) {
|
||||
throw new ERR_INVALID_ARG_VALUE(elemName, tag, 'must not be empty');
|
||||
}
|
||||
|
||||
for (let j = 0; j < tag.length; j++) {
|
||||
const ch = StringPrototypeCharCodeAt(tag, j);
|
||||
if (kTagForbiddenChars.has(ch)) {
|
||||
throw new ERR_INVALID_ARG_VALUE(
|
||||
elemName,
|
||||
tag,
|
||||
'contains a forbidden character. Tags must not contain whitespace, ' +
|
||||
'operator characters (&, |, !, (, ), *), or be the reserved words ' +
|
||||
'\'and\', \'or\', or \'not\'',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const lower = StringPrototypeToLowerCase(tag);
|
||||
|
||||
if (kReservedWords.has(lower)) {
|
||||
throw new ERR_INVALID_ARG_VALUE(
|
||||
elemName,
|
||||
tag,
|
||||
'must not be the reserved word \'and\', \'or\', or \'not\'',
|
||||
);
|
||||
}
|
||||
|
||||
if (!seen.has(lower)) {
|
||||
seen.add(lower);
|
||||
@@ -66,43 +154,341 @@ function validateAndCanonicalizeTagValues(tags, optName) {
|
||||
return ObjectFreeze(result);
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate and canonicalize a single tag-filter string supplied via
|
||||
* `--experimental-test-tag-filter` or `testTagFilters`.
|
||||
*
|
||||
* Filters are literal tag names: a test passes the filter when its tag set
|
||||
* contains the lowercased filter value. Multi-occurrence composes by AND
|
||||
* at the call site (see `evaluateTagFilters`).
|
||||
* @param {unknown} value - The supplied filter string.
|
||||
* @param {string} name - Argument name embedded in thrown error messages.
|
||||
* @returns {string} The lowercased filter value.
|
||||
* @throws {ERR_INVALID_ARG_TYPE} If `value` is not a string.
|
||||
* @throws {ERR_INVALID_ARG_VALUE} If `value` is empty.
|
||||
*/
|
||||
function validateAndCanonicalizeTagFilter(value, name) {
|
||||
validateString(value, name);
|
||||
// ---------------------------------------------------------------------------
|
||||
// Lexer
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
if (value.length === 0) {
|
||||
throw new ERR_INVALID_ARG_VALUE(name, value, 'must not be empty');
|
||||
// Token types
|
||||
const T_AND = 'AND';
|
||||
const T_OR = 'OR';
|
||||
const T_NOT = 'NOT';
|
||||
const T_LPAREN = 'LPAREN';
|
||||
const T_RPAREN = 'RPAREN';
|
||||
const T_IDENT = 'IDENT';
|
||||
const T_EOF = 'EOF';
|
||||
|
||||
function lex(expr, name) {
|
||||
const tokens = [];
|
||||
let i = 0;
|
||||
const len = expr.length;
|
||||
|
||||
while (i < len) {
|
||||
const ch = StringPrototypeCharCodeAt(expr, i);
|
||||
|
||||
if (isWhitespace(ch)) {
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ch === kCharAmpersand) {
|
||||
if (i + 1 < len && StringPrototypeCharCodeAt(expr, i + 1) === kCharAmpersand) {
|
||||
ArrayPrototypePush(tokens, { __proto__: null, type: T_AND, pos: i });
|
||||
i += 2;
|
||||
} else {
|
||||
throw new ERR_INVALID_ARG_VALUE(
|
||||
name, expr,
|
||||
`unexpected '&' at position ${i}; use '&&' or 'and'`,
|
||||
);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ch === kCharPipe) {
|
||||
if (i + 1 < len && StringPrototypeCharCodeAt(expr, i + 1) === kCharPipe) {
|
||||
ArrayPrototypePush(tokens, { __proto__: null, type: T_OR, pos: i });
|
||||
i += 2;
|
||||
} else {
|
||||
throw new ERR_INVALID_ARG_VALUE(
|
||||
name, expr,
|
||||
`unexpected '|' at position ${i}; use '||' or 'or'`,
|
||||
);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ch === kCharExclaim) {
|
||||
ArrayPrototypePush(tokens, { __proto__: null, type: T_NOT, pos: i });
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ch === kCharLParen) {
|
||||
ArrayPrototypePush(tokens, { __proto__: null, type: T_LPAREN, pos: i });
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ch === kCharRParen) {
|
||||
ArrayPrototypePush(tokens, { __proto__: null, type: T_RPAREN, pos: i });
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
const start = i;
|
||||
while (i < len && !isIdentBreak(StringPrototypeCharCodeAt(expr, i))) {
|
||||
i++;
|
||||
}
|
||||
const ident = StringPrototypeSlice(expr, start, i);
|
||||
const lower = StringPrototypeToLowerCase(ident);
|
||||
|
||||
if (lower === 'and') {
|
||||
ArrayPrototypePush(tokens, { __proto__: null, type: T_AND, pos: start });
|
||||
} else if (lower === 'or') {
|
||||
ArrayPrototypePush(tokens, { __proto__: null, type: T_OR, pos: start });
|
||||
} else if (lower === 'not') {
|
||||
ArrayPrototypePush(tokens, { __proto__: null, type: T_NOT, pos: start });
|
||||
} else {
|
||||
ArrayPrototypePush(tokens, { __proto__: null, type: T_IDENT, value: ident, pos: start });
|
||||
}
|
||||
}
|
||||
|
||||
return StringPrototypeToLowerCase(value);
|
||||
ArrayPrototypePush(tokens, { __proto__: null, type: T_EOF, pos: len });
|
||||
return tokens;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Regex escaping for wildcard tags
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// `*` is the only metacharacter intentionally not escaped: the caller
|
||||
// handles it as a wildcard.
|
||||
function escapeForRegExp(str) {
|
||||
let result = '';
|
||||
for (let i = 0; i < str.length; i++) {
|
||||
const ch = StringPrototypeCharCodeAt(str, i);
|
||||
if (
|
||||
ch === kCharLBracket || ch === kCharRBracket ||
|
||||
ch === kCharLParen || ch === kCharRParen ||
|
||||
ch === kCharLBrace || ch === kCharRBrace ||
|
||||
ch === kCharCaret || ch === kCharDollar ||
|
||||
ch === kCharDot || ch === kCharPlus ||
|
||||
ch === kCharQuestion || ch === kCharBackslash ||
|
||||
ch === kCharPipe
|
||||
) {
|
||||
result += '\\';
|
||||
}
|
||||
result += StringPrototypeSlice(str, i, i + 1);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function buildWildcardRegex(ident) {
|
||||
// Lowercase the source first and drop the /i flag, so wildcard matching
|
||||
// uses the same casing semantics as literal matching (which lowercases the
|
||||
// tag value at validation time). Using /i would diverge for non-ASCII
|
||||
// characters whose String.prototype.toLowerCase() output differs from
|
||||
// RegExp /i case folding (e.g. Turkish I-with-dot U+0130).
|
||||
const parts = StringPrototypeSplit(StringPrototypeToLowerCase(ident), '*');
|
||||
let pattern = '^';
|
||||
for (let i = 0; i < parts.length; i++) {
|
||||
if (i > 0) {
|
||||
pattern += '.*';
|
||||
}
|
||||
pattern += escapeForRegExp(parts[i]);
|
||||
}
|
||||
pattern += '$';
|
||||
return new RegExp(pattern);
|
||||
}
|
||||
|
||||
function makeTagNode(ident) {
|
||||
for (let i = 0; i < ident.length; i++) {
|
||||
if (StringPrototypeCharCodeAt(ident, i) === kCharStar) {
|
||||
return { __proto__: null, type: 'wildcard', re: buildWildcardRegex(ident) };
|
||||
}
|
||||
}
|
||||
return { __proto__: null, type: 'tag', value: StringPrototypeToLowerCase(ident) };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Parser - recursive descent
|
||||
// Precedence: not > and > or (standard Boolean)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Parse a tag filter expression string into an AST suitable for
|
||||
* `evaluateTagFilters`.
|
||||
*
|
||||
* Supported syntax:
|
||||
* - Identifiers: any non-whitespace, non-operator characters; case-
|
||||
* insensitive when matched. A `*` inside an identifier becomes a
|
||||
* wildcard that matches any sequence of characters; a bare `*`
|
||||
* matches any non-empty tag.
|
||||
* - Boolean operators: `and`/`&&`, `or`/`||`, `not`/`!`, with the
|
||||
* standard precedence `not > and > or` and left-associative binary
|
||||
* operators. Word forms must be whitespace-separated; punctuation
|
||||
* forms do not.
|
||||
* - Parentheses for grouping.
|
||||
* @param {string} expr - The expression to parse.
|
||||
* @param {string} name - Option name to embed in thrown error messages
|
||||
* (typically `'--experimental-test-tag-filter[<i>]'` or
|
||||
* `'options.testTagFilters[<i>]'`).
|
||||
* @returns {object} An opaque AST node. The shape is internal; pass it
|
||||
* only to `evaluateTagFilters`.
|
||||
* @throws {ERR_INVALID_ARG_TYPE} If `expr` is not a string.
|
||||
* @throws {ERR_INVALID_ARG_VALUE} If `expr` is empty, contains only
|
||||
* whitespace, or has a syntax error (unbalanced parentheses, dangling
|
||||
* operator, adjacent identifiers, etc.). The error message includes
|
||||
* the offending position.
|
||||
*/
|
||||
function parseTagFilterExpression(expr, name) {
|
||||
validateString(expr, name);
|
||||
|
||||
if (StringPrototypeTrim(expr).length === 0) {
|
||||
throw new ERR_INVALID_ARG_VALUE(name, expr, 'must not be empty');
|
||||
}
|
||||
|
||||
const tokens = lex(expr, name);
|
||||
let pos = 0;
|
||||
|
||||
function peek() {
|
||||
return tokens[pos];
|
||||
}
|
||||
|
||||
function consume() {
|
||||
return tokens[pos++];
|
||||
}
|
||||
|
||||
function parseExpr() {
|
||||
return parseOr();
|
||||
}
|
||||
|
||||
function parseOr() {
|
||||
let left = parseAnd();
|
||||
while (peek().type === T_OR) {
|
||||
consume();
|
||||
const right = parseAnd();
|
||||
left = { __proto__: null, type: 'or', left, right };
|
||||
}
|
||||
return left;
|
||||
}
|
||||
|
||||
function parseAnd() {
|
||||
let left = parseNot();
|
||||
while (peek().type === T_AND) {
|
||||
consume();
|
||||
const right = parseNot();
|
||||
left = { __proto__: null, type: 'and', left, right };
|
||||
}
|
||||
return left;
|
||||
}
|
||||
|
||||
function parseNot() {
|
||||
if (peek().type === T_NOT) {
|
||||
const { pos: notPos } = consume();
|
||||
if (peek().type === T_EOF) {
|
||||
throw new ERR_INVALID_ARG_VALUE(
|
||||
name, expr,
|
||||
`expected operand after 'not' at position ${notPos}`,
|
||||
);
|
||||
}
|
||||
const operand = parseNot();
|
||||
return { __proto__: null, type: 'not', operand };
|
||||
}
|
||||
|
||||
if (peek().type === T_LPAREN) {
|
||||
const { pos: lPos } = consume();
|
||||
if (peek().type === T_EOF) {
|
||||
throw new ERR_INVALID_ARG_VALUE(
|
||||
name, expr,
|
||||
`unmatched '(' at position ${lPos}`,
|
||||
);
|
||||
}
|
||||
const inner = parseExpr();
|
||||
if (peek().type !== T_RPAREN) {
|
||||
throw new ERR_INVALID_ARG_VALUE(
|
||||
name, expr,
|
||||
`expected ')' at position ${peek().pos}`,
|
||||
);
|
||||
}
|
||||
consume();
|
||||
return inner;
|
||||
}
|
||||
|
||||
if (peek().type === T_IDENT) {
|
||||
const { value } = consume();
|
||||
return makeTagNode(value);
|
||||
}
|
||||
|
||||
const tok = peek();
|
||||
throw new ERR_INVALID_ARG_VALUE(
|
||||
name, expr,
|
||||
`unexpected token '${tok.type}' at position ${tok.pos}`,
|
||||
);
|
||||
}
|
||||
|
||||
const ast = parseExpr();
|
||||
|
||||
if (peek().type !== T_EOF) {
|
||||
const tok = peek();
|
||||
throw new ERR_INVALID_ARG_VALUE(
|
||||
name, expr,
|
||||
`unexpected token at position ${tok.pos}`,
|
||||
);
|
||||
}
|
||||
|
||||
return ast;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Evaluator
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Evaluate a tag filter AST against a set of test tags.
|
||||
*
|
||||
* Untagged tests have an empty SafeSet. The semantics are:
|
||||
* - An include expression (tag/wildcard/and/or) evaluates false for empty set.
|
||||
* - 'not X' evaluates true when X is false (so untagged tests pass 'not X').
|
||||
* @param {object} ast - AST node from parseTagFilterExpression.
|
||||
* @param {SafeSet} tags - Lowercased canonical tags for the test.
|
||||
* @returns {boolean}
|
||||
*/
|
||||
function evaluateTagFilter(ast, tags) {
|
||||
const { type } = ast;
|
||||
|
||||
if (type === 'or') {
|
||||
return evaluateTagFilter(ast.left, tags) || evaluateTagFilter(ast.right, tags);
|
||||
}
|
||||
|
||||
if (type === 'and') {
|
||||
return evaluateTagFilter(ast.left, tags) && evaluateTagFilter(ast.right, tags);
|
||||
}
|
||||
|
||||
if (type === 'not') {
|
||||
return !evaluateTagFilter(ast.operand, tags);
|
||||
}
|
||||
|
||||
if (type === 'tag') {
|
||||
return tags.has(ast.value);
|
||||
}
|
||||
|
||||
// type === 'wildcard'
|
||||
let matched = false;
|
||||
tags.forEach((tag) => {
|
||||
if (!matched && RegExpPrototypeExec(ast.re, tag) !== null) {
|
||||
matched = true;
|
||||
}
|
||||
});
|
||||
return matched;
|
||||
}
|
||||
|
||||
/**
|
||||
* Evaluate a list of tag filters against a test's tag set.
|
||||
* Evaluate one or more tag filter ASTs against a test's tag set.
|
||||
*
|
||||
* Multiple filters compose by AND: a test passes only when every filter is
|
||||
* present in its tag set. An empty `filters` array always passes. Untagged
|
||||
* tests have an empty tag set and so are excluded under any non-empty
|
||||
* filter.
|
||||
* @param {string[]} filters - Lowercased canonical filter values.
|
||||
* @param {SafeSet<string>} tags - The test's lowercased canonical tag set.
|
||||
* Multiple ASTs compose by AND: a test passes only when every AST
|
||||
* evaluates true for it. An empty `asts` array always passes.
|
||||
*
|
||||
* Untagged tests (an empty `tags` set) fail any include expression
|
||||
* (`tag`, `wildcard`, `and`, `or`) and pass `not X`, so excluding tags
|
||||
* does not accidentally remove untagged tests.
|
||||
* @param {object[]} asts - ASTs returned by `parseTagFilterExpression`.
|
||||
* @param {SafeSet<string>} tags - The test's lowercased canonical tag
|
||||
* set (use `Set` in non-internal contexts; both work).
|
||||
* @returns {boolean} `true` if the test should run.
|
||||
*/
|
||||
function evaluateTagFilters(filters, tags) {
|
||||
for (let i = 0; i < filters.length; i++) {
|
||||
if (!tags.has(filters[i])) {
|
||||
function evaluateTagFilters(asts, tags) {
|
||||
for (let i = 0; i < asts.length; i++) {
|
||||
if (!evaluateTagFilter(asts[i], tags)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -112,6 +498,6 @@ function evaluateTagFilters(filters, tags) {
|
||||
module.exports = {
|
||||
evaluateTagFilters,
|
||||
kEmptyTagArray,
|
||||
validateAndCanonicalizeTagFilter,
|
||||
parseTagFilterExpression,
|
||||
validateAndCanonicalizeTagValues,
|
||||
};
|
||||
@@ -30,7 +30,7 @@ const {
|
||||
} = primordials;
|
||||
|
||||
const {
|
||||
validateAndCanonicalizeTagFilter,
|
||||
parseTagFilterExpression,
|
||||
} = require('internal/test_runner/tag_filter');
|
||||
|
||||
const { AsyncResource } = require('async_hooks');
|
||||
@@ -273,6 +273,7 @@ function parseCommandLine() {
|
||||
let testNamePatterns = mapPatternFlagToRegExArray('--test-name-pattern');
|
||||
let testSkipPatterns = mapPatternFlagToRegExArray('--test-skip-pattern');
|
||||
let testTagFilters = null;
|
||||
let testTagFilterExpressions = null;
|
||||
|
||||
if (isChildProcessV8) {
|
||||
kBuiltinReporters.set('v8-serializer', 'internal/test_runner/reporter/v8-serializer');
|
||||
@@ -308,13 +309,14 @@ function parseCommandLine() {
|
||||
const tagFilterFlag = getOptionValue('--experimental-test-tag-filter');
|
||||
if (tagFilterFlag?.length > 0) {
|
||||
emitExperimentalWarning('Test tags');
|
||||
// File wrappers are exempt from tag filtering, so holding the filters
|
||||
// in the parent is safe under any isolation mode; under
|
||||
// isolation='process' the canonical values are re-emitted to the
|
||||
// child processes, which apply the filter themselves.
|
||||
// File wrappers are exempt from tag filtering, so holding the parsed
|
||||
// ASTs in the parent is safe under any isolation mode; under
|
||||
// isolation='process' the source expressions are re-emitted to the
|
||||
// child processes, which re-parse and apply the filter themselves.
|
||||
testTagFilterExpressions = tagFilterFlag;
|
||||
testTagFilters = ArrayPrototypeMap(
|
||||
tagFilterFlag,
|
||||
(value, i) => validateAndCanonicalizeTagFilter(value, `--experimental-test-tag-filter[${i}]`),
|
||||
(expr, i) => parseTagFilterExpression(expr, `--experimental-test-tag-filter[${i}]`),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -359,9 +361,10 @@ function parseCommandLine() {
|
||||
const tagFilterFlag = getOptionValue('--experimental-test-tag-filter');
|
||||
if (tagFilterFlag?.length > 0) {
|
||||
emitExperimentalWarning('Test tags');
|
||||
testTagFilterExpressions = tagFilterFlag;
|
||||
testTagFilters = ArrayPrototypeMap(
|
||||
tagFilterFlag,
|
||||
(value, i) => validateAndCanonicalizeTagFilter(value, `--experimental-test-tag-filter[${i}]`),
|
||||
(expr, i) => parseTagFilterExpression(expr, `--experimental-test-tag-filter[${i}]`),
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -426,6 +429,7 @@ function parseCommandLine() {
|
||||
sourceMaps,
|
||||
testNamePatterns,
|
||||
testSkipPatterns,
|
||||
testTagFilterExpressions,
|
||||
testTagFilters,
|
||||
timeout,
|
||||
updateSnapshots,
|
||||
|
||||
@@ -62,32 +62,55 @@ test('untagged parent runs only the matching tagged child', () => {
|
||||
assert.doesNotMatch(stdout, /plain sibling/);
|
||||
});
|
||||
|
||||
test('repeated --experimental-test-tag-filter ANDs together', () => {
|
||||
// db AND integration: only "db plus integration" satisfies both.
|
||||
const { status, stdout } = run([
|
||||
'--experimental-test-tag-filter=db',
|
||||
'--experimental-test-tag-filter=integration',
|
||||
]);
|
||||
test('not X also includes untagged tests', () => {
|
||||
const { status, stdout } = run(['--experimental-test-tag-filter=not flaky']);
|
||||
assert.strictEqual(status, 0);
|
||||
pass(stdout, 1);
|
||||
assert.match(stdout, /ok \d+ - db plus integration/);
|
||||
assert.doesNotMatch(stdout, /^ok \d+ - db only/m);
|
||||
assert.doesNotMatch(stdout, /^ok \d+ - db flaky/m);
|
||||
pass(stdout, 8);
|
||||
assert.match(stdout, /ok \d+ - untagged/);
|
||||
assert.doesNotMatch(stdout, /db flaky/);
|
||||
assert.doesNotMatch(stdout, /^ok \d+ - only flaky/m);
|
||||
});
|
||||
|
||||
test('untagged tests are excluded under any positive filter', () => {
|
||||
const { status, stdout } = run(['--experimental-test-tag-filter=db']);
|
||||
test('repeated --experimental-test-tag-filter ANDs together', () => {
|
||||
const { status, stdout } = run([
|
||||
'--experimental-test-tag-filter=db',
|
||||
'--experimental-test-tag-filter=not flaky',
|
||||
]);
|
||||
assert.strictEqual(status, 0);
|
||||
pass(stdout, 2);
|
||||
assert.match(stdout, /ok \d+ - db only/);
|
||||
assert.match(stdout, /ok \d+ - db plus integration/);
|
||||
assert.doesNotMatch(stdout, /db flaky/);
|
||||
});
|
||||
|
||||
test('bare * matches any tagged test, excludes untagged', () => {
|
||||
const { status, stdout } = run(['--experimental-test-tag-filter=*']);
|
||||
assert.strictEqual(status, 0);
|
||||
pass(stdout, 8);
|
||||
assert.doesNotMatch(stdout, /^ok \d+ - untagged/m);
|
||||
});
|
||||
|
||||
test('wildcard prefix: db* matches db, db:postgres, etc.', () => {
|
||||
const { status, stdout } = run(['--experimental-test-tag-filter=db*']);
|
||||
assert.strictEqual(status, 0);
|
||||
pass(stdout, 4);
|
||||
assert.match(stdout, /db wildcard match/);
|
||||
});
|
||||
|
||||
test('boolean composition: unit && !slow', () => {
|
||||
const { status, stdout } = run(['--experimental-test-tag-filter=unit && !slow']);
|
||||
assert.strictEqual(status, 0);
|
||||
pass(stdout, 1);
|
||||
assert.match(stdout, /ok \d+ - unit only/);
|
||||
});
|
||||
|
||||
test('isolation=none produces identical filtered set', () => {
|
||||
const proc = run(['--experimental-test-tag-filter=db']);
|
||||
const none = run(['--experimental-test-tag-filter=db'], ['--test-isolation=none']);
|
||||
const proc = run(['--experimental-test-tag-filter=unit && !slow']);
|
||||
const none = run(['--experimental-test-tag-filter=unit && !slow'], ['--test-isolation=none']);
|
||||
assert.strictEqual(proc.status, 0);
|
||||
assert.strictEqual(none.status, 0);
|
||||
pass(proc.stdout, 3);
|
||||
pass(none.stdout, 3);
|
||||
pass(proc.stdout, 1);
|
||||
pass(none.stdout, 1);
|
||||
});
|
||||
|
||||
test('combined with --test-skip-pattern (pure AND)', () => {
|
||||
@@ -101,8 +124,44 @@ test('combined with --test-skip-pattern (pure AND)', () => {
|
||||
assert.doesNotMatch(stdout, /db flaky/);
|
||||
});
|
||||
|
||||
test('malformed expression: unbalanced parenthesis errors at startup', () => {
|
||||
const { status, stderr, stdout } = run(['--experimental-test-tag-filter=(a']);
|
||||
assert.notStrictEqual(status, 0);
|
||||
// Match the full sentinel error line so noise elsewhere in stderr can't
|
||||
// satisfy the assertion via interleaved substrings.
|
||||
assert.match(
|
||||
stderr,
|
||||
/^TypeError \[ERR_INVALID_ARG_VALUE\]: The argument '--experimental-test-tag-filter\[0\]' expected '\)' at position \d+/m,
|
||||
);
|
||||
// No file should have been spawned.
|
||||
assert.doesNotMatch(stdout, /^ok 1 -/m);
|
||||
});
|
||||
|
||||
test('malformed expression: dangling operator errors at startup', () => {
|
||||
const { status, stderr } = run(['--experimental-test-tag-filter=a and']);
|
||||
assert.notStrictEqual(status, 0);
|
||||
assert.match(
|
||||
stderr,
|
||||
/^TypeError \[ERR_INVALID_ARG_VALUE\]: The argument '--experimental-test-tag-filter\[0\]' unexpected token 'EOF' at position \d+/m,
|
||||
);
|
||||
});
|
||||
|
||||
test('empty --experimental-test-tag-filter= is rejected by argv parser', () => {
|
||||
const { status, stderr } = run(['--experimental-test-tag-filter=']);
|
||||
assert.notStrictEqual(status, 0);
|
||||
assert.match(stderr, /requires an argument/);
|
||||
});
|
||||
|
||||
test('reserved word as tag value errors at registration', () => {
|
||||
const child = spawnSync(process.execPath, [
|
||||
'-e',
|
||||
"require('node:test').test('x', { tags: ['and'] }, () => {});",
|
||||
]);
|
||||
assert.notStrictEqual(child.status, 0);
|
||||
// Match the full sentinel error line so noise elsewhere in stderr can't
|
||||
// satisfy the assertion via interleaved substrings.
|
||||
assert.match(
|
||||
child.stderr.toString(),
|
||||
/^TypeError \[ERR_INVALID_ARG_VALUE\]: The property 'options\.tags\[0\]' must not be the reserved word/m,
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,257 @@
|
||||
// Flags: --expose-internals
|
||||
|
||||
import '../common/index.mjs';
|
||||
import assert from 'node:assert';
|
||||
import { test } from 'node:test';
|
||||
import tagFilter from 'internal/test_runner/tag_filter';
|
||||
|
||||
const { evaluateTagFilters, parseTagFilterExpression } = tagFilter;
|
||||
|
||||
function tagSet(...tags) {
|
||||
return new Set(tags);
|
||||
}
|
||||
|
||||
function matches(expr, ...tags) {
|
||||
return evaluateTagFilters([parseTagFilterExpression(expr, 'expr')], tagSet(...tags));
|
||||
}
|
||||
|
||||
test('single identifier', () => {
|
||||
assert.strictEqual(matches('db', 'db'), true);
|
||||
assert.strictEqual(matches('db', 'other'), false);
|
||||
assert.strictEqual(matches('db'), false);
|
||||
});
|
||||
|
||||
test('case-insensitive match', () => {
|
||||
// The evaluator expects an already-lowercased tag set, so 'DB' in the set
|
||||
// must not match 'db' in the filter.
|
||||
assert.strictEqual(matches('DB', 'db'), true);
|
||||
assert.strictEqual(matches('db', 'DB'), false);
|
||||
});
|
||||
|
||||
test('and / && have higher precedence than or / ||', () => {
|
||||
for (const expr of ['a or b and c', 'a || b && c']) {
|
||||
assert.strictEqual(matches(expr, 'a'), true);
|
||||
assert.strictEqual(matches(expr, 'b'), false);
|
||||
assert.strictEqual(matches(expr, 'c'), false);
|
||||
assert.strictEqual(matches(expr, 'a', 'b'), true);
|
||||
assert.strictEqual(matches(expr, 'a', 'c'), true);
|
||||
assert.strictEqual(matches(expr, 'b', 'c'), true);
|
||||
assert.strictEqual(matches(expr, 'a', 'b', 'c'), true);
|
||||
}
|
||||
});
|
||||
|
||||
test('not has higher precedence than and', () => {
|
||||
for (const expr of ['not a and b', '!a && b']) {
|
||||
assert.strictEqual(matches(expr, 'b'), true);
|
||||
assert.strictEqual(matches(expr, 'a', 'b'), false);
|
||||
}
|
||||
});
|
||||
|
||||
test('not has higher precedence than or', () => {
|
||||
for (const expr of ['not a or b', '!a || b']) {
|
||||
assert.strictEqual(matches(expr), true);
|
||||
assert.strictEqual(matches(expr, 'a'), false);
|
||||
assert.strictEqual(matches(expr, 'b'), true);
|
||||
assert.strictEqual(matches(expr, 'a', 'b'), true);
|
||||
}
|
||||
});
|
||||
|
||||
test('parentheses override precedence', () => {
|
||||
for (const expr of ['(a or b) and c', '(a || b) && c']) {
|
||||
assert.strictEqual(matches(expr, 'a'), false);
|
||||
assert.strictEqual(matches(expr, 'c'), false);
|
||||
assert.strictEqual(matches(expr, 'a', 'c'), true);
|
||||
assert.strictEqual(matches(expr, 'b', 'c'), true);
|
||||
}
|
||||
});
|
||||
|
||||
test('double negation', () => {
|
||||
for (const expr of ['not not a', '!!a']) {
|
||||
assert.strictEqual(matches(expr, 'a'), true);
|
||||
assert.strictEqual(matches(expr), false);
|
||||
}
|
||||
});
|
||||
|
||||
test('mixing word and punctuation forms in one expression', () => {
|
||||
for (const expr of ['unit && not flaky', 'unit and !flaky']) {
|
||||
assert.strictEqual(matches(expr, 'unit'), true);
|
||||
assert.strictEqual(matches(expr, 'unit', 'flaky'), false);
|
||||
}
|
||||
});
|
||||
|
||||
test('mixed forms inside parentheses, word and punct equivalence', () => {
|
||||
// Each pair below is the same expression in word form and punctuation form.
|
||||
for (const expr of ['a and (b || not c)', 'a && (b or not c)']) {
|
||||
assert.strictEqual(matches(expr, 'a'), true);
|
||||
assert.strictEqual(matches(expr, 'a', 'b'), true);
|
||||
assert.strictEqual(matches(expr, 'a', 'c'), false);
|
||||
assert.strictEqual(matches(expr, 'a', 'b', 'c'), true);
|
||||
assert.strictEqual(matches(expr, 'b'), false);
|
||||
}
|
||||
|
||||
for (const expr of ['(a || b) && !c', '(a or b) and not c']) {
|
||||
assert.strictEqual(matches(expr, 'a'), true);
|
||||
assert.strictEqual(matches(expr, 'b'), true);
|
||||
assert.strictEqual(matches(expr, 'a', 'c'), false);
|
||||
assert.strictEqual(matches(expr, 'c'), false);
|
||||
}
|
||||
|
||||
for (const expr of ['not (a or b) and c', '!(a || b) && c']) {
|
||||
assert.strictEqual(matches(expr, 'c'), true);
|
||||
assert.strictEqual(matches(expr, 'a', 'c'), false);
|
||||
assert.strictEqual(matches(expr, 'b', 'c'), false);
|
||||
}
|
||||
|
||||
for (const expr of ['!(a && b) || c', 'not (a and b) or c']) {
|
||||
assert.strictEqual(matches(expr, 'a'), true);
|
||||
assert.strictEqual(matches(expr, 'a', 'b'), false);
|
||||
assert.strictEqual(matches(expr, 'a', 'b', 'c'), true);
|
||||
}
|
||||
});
|
||||
|
||||
test('nested parentheses and wildcards mixed with word forms', () => {
|
||||
for (const expr of [
|
||||
'(db* or unit) and not (slow || flaky)',
|
||||
'(db* || unit) && !(slow or flaky)',
|
||||
]) {
|
||||
assert.strictEqual(matches(expr, 'db'), true);
|
||||
assert.strictEqual(matches(expr, 'db:postgres'), true);
|
||||
assert.strictEqual(matches(expr, 'unit'), true);
|
||||
assert.strictEqual(matches(expr, 'unit', 'slow'), false);
|
||||
assert.strictEqual(matches(expr, 'db', 'flaky'), false);
|
||||
assert.strictEqual(matches(expr, 'mongo'), false);
|
||||
}
|
||||
});
|
||||
|
||||
test("'notslow' is one identifier, 'not slow' is two tokens", () => {
|
||||
assert.strictEqual(matches('notslow', 'notslow'), true);
|
||||
assert.strictEqual(matches('notslow', 'slow'), false);
|
||||
assert.strictEqual(matches('not slow'), true);
|
||||
assert.strictEqual(matches('not slow', 'slow'), false);
|
||||
});
|
||||
|
||||
test('punctuation does not need whitespace separation', () => {
|
||||
assert.strictEqual(matches('!a', 'b'), true);
|
||||
assert.strictEqual(matches('!a', 'a'), false);
|
||||
assert.strictEqual(matches('a&&b', 'a', 'b'), true);
|
||||
assert.strictEqual(matches('a||b', 'a'), true);
|
||||
});
|
||||
|
||||
test('wildcard expands across allowed alphabet', () => {
|
||||
assert.strictEqual(matches('db*', 'db'), true);
|
||||
assert.strictEqual(matches('db*', 'db1'), true);
|
||||
assert.strictEqual(matches('db*', 'db:postgres'), true);
|
||||
assert.strictEqual(matches('db*', 'mongo'), false);
|
||||
});
|
||||
|
||||
test('bare * matches any tagged test', () => {
|
||||
assert.strictEqual(matches('*', 'anything'), true);
|
||||
assert.strictEqual(matches('*', 'a', 'b'), true);
|
||||
assert.strictEqual(matches('*'), false);
|
||||
});
|
||||
|
||||
test('wildcard in middle and at start', () => {
|
||||
assert.strictEqual(matches('*db', 'mydb'), true);
|
||||
assert.strictEqual(matches('*db', 'db'), true);
|
||||
assert.strictEqual(matches('*db', 'dbmy'), false);
|
||||
assert.strictEqual(matches('a*z', 'abz'), true);
|
||||
assert.strictEqual(matches('a*z', 'az'), true);
|
||||
assert.strictEqual(matches('a*z', 'abc'), false);
|
||||
});
|
||||
|
||||
test('regex metacharacters in tags are escaped', () => {
|
||||
// The literal `.` in the filter must match a literal dot only, not any
|
||||
// arbitrary character (which would be the unescaped regex meaning).
|
||||
assert.strictEqual(matches('a.b*', 'a.b'), true);
|
||||
assert.strictEqual(matches('a.b*', 'axb'), false);
|
||||
});
|
||||
|
||||
test('wildcard case-folding matches literal-tag case-folding for non-ASCII', () => {
|
||||
// Authoring a tag 'İ' (U+0130) goes through String.prototype.toLowerCase
|
||||
// and stores 'i̇' (two code points). Wildcard matching must use the same
|
||||
// fold so authors see consistent behavior across literal and wildcard
|
||||
// filters.
|
||||
const folded = 'İ'.toLowerCase();
|
||||
assert.strictEqual(matches('İ', folded), true);
|
||||
assert.strictEqual(matches('İ*', folded), true);
|
||||
assert.strictEqual(matches('İ*', `${folded}foo`), true);
|
||||
assert.strictEqual(matches('*İ', `foo${folded}`), true);
|
||||
});
|
||||
|
||||
test('untagged test fails any include expression', () => {
|
||||
for (const expr of ['a', 'a or b', 'a || b', 'a and b', 'a && b', '*']) {
|
||||
assert.strictEqual(matches(expr), false);
|
||||
}
|
||||
});
|
||||
|
||||
test("'not X' against an untagged test is true", () => {
|
||||
for (const expr of ['not a', '!a']) {
|
||||
assert.strictEqual(matches(expr), true);
|
||||
}
|
||||
});
|
||||
|
||||
test('multiple filters AND together', () => {
|
||||
const f1 = parseTagFilterExpression('a', 'expr');
|
||||
const f2 = parseTagFilterExpression('not b', 'expr');
|
||||
assert.strictEqual(evaluateTagFilters([f1, f2], tagSet('a')), true);
|
||||
assert.strictEqual(evaluateTagFilters([f1, f2], tagSet('a', 'b')), false);
|
||||
assert.strictEqual(evaluateTagFilters([f1, f2], tagSet('b')), false);
|
||||
assert.strictEqual(evaluateTagFilters([f1, f2], tagSet()), false);
|
||||
});
|
||||
|
||||
test('empty filter list always passes', () => {
|
||||
assert.strictEqual(evaluateTagFilters([], tagSet()), true);
|
||||
assert.strictEqual(evaluateTagFilters([], tagSet('a')), true);
|
||||
});
|
||||
|
||||
// --- Malformed expressions -------------------------------------------------
|
||||
|
||||
function expectInvalid(expr) {
|
||||
assert.throws(
|
||||
() => parseTagFilterExpression(expr, 'expr'),
|
||||
{ code: 'ERR_INVALID_ARG_VALUE' },
|
||||
`expected '${expr}' to be invalid`,
|
||||
);
|
||||
}
|
||||
|
||||
test('rejects empty / whitespace-only expressions', () => {
|
||||
for (const expr of ['', ' ', '\t\n']) {
|
||||
expectInvalid(expr);
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects unbalanced parentheses', () => {
|
||||
for (const expr of ['(', '((', '(a', 'a)', '((a)', '(a))']) {
|
||||
expectInvalid(expr);
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects dangling operators', () => {
|
||||
for (const expr of ['not', 'a and', 'a or', '&& a', '|| a', 'a &&', 'a ||']) {
|
||||
expectInvalid(expr);
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects two adjacent identifiers', () => {
|
||||
expectInvalid('foo bar');
|
||||
});
|
||||
|
||||
test('rejects single & and single |', () => {
|
||||
for (const expr of ['a & b', 'a | b']) {
|
||||
expectInvalid(expr);
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects empty parentheses', () => {
|
||||
expectInvalid('()');
|
||||
});
|
||||
|
||||
test('rejects non-string expressions', () => {
|
||||
for (const expr of [42, null, undefined, true, [], {}, Symbol('s')]) {
|
||||
assert.throws(
|
||||
() => parseTagFilterExpression(expr, 'expr'),
|
||||
{ code: 'ERR_INVALID_ARG_TYPE' },
|
||||
`expected ${String(expr)} to be rejected`,
|
||||
);
|
||||
}
|
||||
});
|
||||
@@ -101,4 +101,12 @@ describe('tag-bearing event payloads', { concurrency: false }, () => {
|
||||
// eslint-disable-next-line no-unused-vars
|
||||
for await (const _ of stream);
|
||||
});
|
||||
|
||||
it('process isolation forwards --experimental-test-tag-filter to children', async () => {
|
||||
const stream = run({ files: [fixture], testTagFilters: ['not nonexistent'] });
|
||||
stream.on('test:fail', common.mustNotCall());
|
||||
stream.on('test:pass', common.mustCall(13));
|
||||
// eslint-disable-next-line no-unused-vars
|
||||
for await (const _ of stream);
|
||||
});
|
||||
});
|
||||
@@ -50,7 +50,7 @@ test('--experimental-test-tag-filter fires the warning', () => {
|
||||
const { stderr } = runChild([
|
||||
'--test',
|
||||
'--test-reporter=tap',
|
||||
'--experimental-test-tag-filter=db',
|
||||
'--experimental-test-tag-filter=*',
|
||||
'--test-isolation=none',
|
||||
fixture,
|
||||
]);
|
||||
|
||||
@@ -37,6 +37,26 @@ test('empty-string tag throws ERR_INVALID_ARG_VALUE', () => {
|
||||
);
|
||||
});
|
||||
|
||||
test('forbidden characters throw ERR_INVALID_ARG_VALUE', () => {
|
||||
for (const bad of ['a b', 'a\tb', 'a\nb', 'a&b', 'a|b', 'a!b', 'a(b', 'a)b', 'a*b', '*', '&&', '||']) {
|
||||
assert.throws(
|
||||
() => test('x', { tags: [bad] }, () => {}),
|
||||
{ code: 'ERR_INVALID_ARG_VALUE' },
|
||||
`expected throw for tag=${JSON.stringify(bad)}`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('reserved words throw ERR_INVALID_ARG_VALUE in any casing', () => {
|
||||
for (const word of ['and', 'AND', 'And', 'or', 'OR', 'Or', 'not', 'NOT', 'Not']) {
|
||||
assert.throws(
|
||||
() => test('x', { tags: [word] }, () => {}),
|
||||
{ code: 'ERR_INVALID_ARG_VALUE' },
|
||||
`expected throw for reserved word ${word}`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('Unicode and most punctuation are allowed', () => {
|
||||
// None of these should throw.
|
||||
test('unicode-1', { tags: ['café'] }, () => {});
|
||||
@@ -80,7 +100,7 @@ test('suite() and describe() validate tags identically', () => {
|
||||
{ code: 'ERR_INVALID_ARG_TYPE' },
|
||||
);
|
||||
assert.throws(
|
||||
() => describe('s', { tags: [''] }, () => {}),
|
||||
() => describe('s', { tags: ['and'] }, () => {}),
|
||||
{ code: 'ERR_INVALID_ARG_VALUE' },
|
||||
);
|
||||
});
|
||||
@@ -92,6 +112,13 @@ test('it() validates tags identically to test()', () => {
|
||||
);
|
||||
});
|
||||
|
||||
test('it() validates tags identically to test()', () => {
|
||||
assert.throws(
|
||||
() => it('i', { tags: ['a b'] }, () => {}),
|
||||
{ code: 'ERR_INVALID_ARG_VALUE' },
|
||||
);
|
||||
});
|
||||
|
||||
test('run() rejects non-string testTagFilters values', () => {
|
||||
for (const bad of [[42], [{}], [null], [undefined], [true], [Symbol('s')]]) {
|
||||
assert.throws(
|
||||
|
||||
Reference in new issue
Block a user