test_runner: extend tag filter with boolean expression DSL

Upgrades the experimental tag filter introduced by stage 1 to accept a
boolean expression instead of a literal tag name.

Grammar: `and`/`&&`, `or`/`||`, `not`/`!`, parentheses for grouping,
and `*` wildcards inside identifiers. Standard precedence
(`not > and > or`); binary operators are left-associative. Word forms
require whitespace separation; punctuation forms do not. Untagged
tests evaluate `false` for any include expression and `true` for
`not X`, so excluding tags does not accidentally remove untagged
tests.

The flag and `testTagFilters` option are still repeatable; multiple
expressions still AND together. Malformed expressions fail fast at
the parent process at startup.

Tag value validation tightens to reject whitespace, operator
characters (`& | ! ( ) *`), and the reserved words `and`/`or`/`not`
in any casing - a breaking change relative to the stage 1 ship,
acceptable at Stability 1.0 (Early development).

Signed-off-by: atlowChemi <chemi@atlow.co.il>
PR-URL: https://github.com/nodejs/node/pull/63054
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
This commit is contained in:
Chemi Atlow authored and GitHub committed 2026-08-13 13:26:26 +00:00
1 parent 1d9d49c1be
commit a674c5d1a3
11 files changed
+928 -105

No files matched your search

+14 -8
View File
@@ -1499,7 +1499,7 @@ Enable module mocking in the test runner.
This feature requires `--allow-worker` if used with the [Permission Model][].
### `--experimental-test-tag-filter=<tag>`
### `--experimental-test-tag-filter='<expr>'`
<!-- YAML
added:
@@ -1509,14 +1509,20 @@ added:
> Stability: 1.0 - Early development
Run only tests whose tag set contains `<tag>`. Tests declare tags via the
`tags` option on `test()`, `it()`, `suite()`, or `describe()`; tags
inherit from suites to nested tests by union. Filtering is
case-insensitive.
Run only tests that match the provided boolean tag-filter expression. Tests
declare tags via the `tags` option on `test()`, `it()`, `suite()`, or
`describe()`. Tags inherit from suites to nested tests by union.
The flag may be specified more than once; tests must contain **every**
filter value to run. See [Test tags][] for details on declaring and
inheriting tags.
The expression supports boolean operators (`and`/`&&`, `or`/`||`,
`not`/`!`), parentheses for grouping, and `*` wildcards inside identifiers.
Standard precedence applies: `not` binds tighter than `and`, which binds
tighter than `or`. See [Test tags][] for the full grammar and behavior.
The flag may be specified more than once; multiple expressions are combined
with AND, so a test must satisfy every expression to run.
A malformed expression causes the test runner to exit with a non-zero status
before running any tests.
### `--experimental-vfs`
+76 -23
View File
@@ -491,8 +491,8 @@ added:
Tags annotate tests and suites with arbitrary string labels. The
[`--experimental-test-tag-filter`][] CLI flag (or the `testTagFilters`
option on [`run()`][]) selects tests whose tag set contains every
provided filter value.
option on [`run()`][]) selects tests by a boolean expression over those
labels.
Tags are an alternative to encoding metadata into test names. They are
useful for cross-cutting axes such as subsystem, speed bucket, flakiness,
@@ -525,37 +525,89 @@ describe('database', { tags: ['db'] }, () => {
});
```
Tag values must be non-empty strings. Tags are matched case-insensitively;
the canonical form is lowercase. Duplicates within a single `tags` array
are collapsed on the lowercased form, preserving the first-seen
declaration order.
Tag values must be non-empty strings that contain no whitespace, no
operator characters (`& | ! ( ) *`), and are not the reserved words
`'and'`, `'or'`, or `'not'` in any casing. Tags are matched
case-insensitively; the canonical form is lowercase. Duplicates within a
single `tags` array are collapsed on the lowercased form, preserving the
first-seen declaration order.
Hooks (`before`, `after`, `beforeEach`, `afterEach`) do not declare their
own tags. They run as part of their owning suite, which carries the
suite's tags.
### Filtering by tag
### Filtering syntax
Each [`--experimental-test-tag-filter`][] value is a literal tag name. A
test runs only when its tag set contains that name. The flag may be
specified more than once; tests must match **every** filter to run. The
same applies to the `testTagFilters` array on [`run()`][]. Filters are
case-insensitive and AND'd with [`--test-name-pattern`][],
[`--test-skip-pattern`][], and `.only` filtering.
The filter expression supports:
Untagged tests are excluded under any non-empty filter, since the filter
requires the tag to be present.
* Identifiers—any non-whitespace, non-operator characters. A literal
identifier matches a tag of the same value (case-insensitive).
* `*` wildcards inside an identifier match any sequence of characters.
A bare `*` matches any tagged test.
* Boolean operators with two equivalent forms:
* `and` / `&&`
* `or` / `||`
* `not` / `!`
* Parentheses for grouping.
### Reading tags from inside a test
The word forms (`and`, `or`, `not`) require whitespace separation; the
punctuation forms do not.
#### Operator precedence
The expression is evaluated with the standard precedence
`not > and > or`. Binary operators are left-associative.
| Expression | Equivalent grouping |
| -------------- | ------------------- |
| `a or b and c` | `a or (b and c)` |
| `not a and b` | `(not a) and b` |
Use parentheses to override:
| Expression | Selects |
| ------------------------------ | ------------------------------------------ |
| `(unit or smoke) and not slow` | unit-or-smoke tests that are not also slow |
| `db && !flaky` | db tests that are not flaky |
| `*` | every tagged test |
#### Untagged tests
Untagged tests behave as if they have an empty tag set. As a result:
| Filter expression | Untagged test | Why |
| ------------------------ | ------------- | ------------------------------------------------ |
| `db` | excluded | Positive match against an empty tag set is false |
| `*` | excluded | The bare wildcard requires at least one tag |
| `db or unit` | excluded | Both branches are false against an empty tag set |
| `not flaky` | included | Negation against an empty tag set is true |
| `not flaky and not slow` | included | Both negations are true against an empty tag set |
| `db or not flaky` | included | The negated branch is true |
For example, `--experimental-test-tag-filter='not flaky'` runs every test
that is not tagged `flaky`, including all untagged tests.
#### Composing multiple filters
[`--experimental-test-tag-filter`][] may be specified more than once on the
command line. Multiple expressions compose by AND—a test must satisfy
every expression to run. The same applies to passing an array to
`testTagFilters` on [`run()`][]. The tag filter is also AND'd with
[`--test-name-pattern`][], [`--test-skip-pattern`][], and `.only`
filtering.
#### Reading tags from inside a test
The [`TestContext`][] object exposes the test's tags as a frozen array
through [`context.tags`][], so tests can branch on their own metadata.
### Errors
#### Errors
A tag value that violates the validation rules above throws
`ERR_INVALID_ARG_VALUE` at the registration site, before any test runs.
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`.
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`. A malformed
filter expression on the CLI causes the test runner to exit with a
non-zero status before running any test files.
## Extraneous asynchronous activity
@@ -830,7 +882,7 @@ test runner functionality:
* `--test` - Prevented to avoid recursive test execution
* `--experimental-test-coverage` - Managed by the test runner
* `--experimental-test-tag-filter` - Filter values are validated by the parent
* `--experimental-test-tag-filter` - Filter expressions are validated by the parent
process and re-emitted to child processes
* `--watch` - Watch mode is handled at the parent level
* `--experimental-default-config-file` - Config file loading is handled by the parent
@@ -1746,10 +1798,11 @@ changes:
For each test that is executed, any corresponding test hooks, such as
`beforeEach()`, are also run.
**Default:** `undefined`.
* `testTagFilters` {string|string\[]} A tag name, or an array of tag names,
used to filter tests by their declared tags. Tests must contain every
listed tag to run. Equivalent to passing [`--experimental-test-tag-filter`][]
on the command line. See [Test tags][]. **Default:** `undefined`.
* `testTagFilters` {string|string\[]} A boolean expression, or an array of
boolean expressions, used to filter tests by their declared tags.
Multiple expressions compose by AND. Equivalent to passing
[`--experimental-test-tag-filter`][] on the command line. See
[Test tags][]. **Default:** `undefined`.
* `timeout` {number} A number of milliseconds the test execution will
fail after.
If unspecified, subtests inherit this value from their parent.
+12 -8
View File
@@ -814,14 +814,18 @@ collecting code coverage from tests for more details.
Enable module mocking in the test runner.
This feature requires \fB--allow-worker\fR if used with the Permission Model.
.
.It Fl -experimental-test-tag-filter Ns = Ns Ar <tag>
Run only tests whose tag set contains \fB<tag>\fR. Tests declare tags via the
\fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or \fBdescribe()\fR; tags
inherit from suites to nested tests by union. Filtering is
case-insensitive.
The flag may be specified more than once; tests must contain \fBevery\fR
filter value to run. See Test tags for details on declaring and
inheriting tags.
.It Fl -experimental-test-tag-filter Ns = Ns Ar '<expr>'
Run only tests that match the provided boolean tag-filter expression. Tests
declare tags via the \fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or
\fBdescribe()\fR. Tags inherit from suites to nested tests by union.
The expression supports boolean operators (\fBand\fR/\fB&&\fR, \fBor\fR/\fB||\fR,
\fBnot\fR/\fB!\fR), parentheses for grouping, and \fB*\fR wildcards inside identifiers.
Standard precedence applies: \fBnot\fR binds tighter than \fBand\fR, which binds
tighter than \fBor\fR. See Test tags for the full grammar and behavior.
The flag may be specified more than once; multiple expressions are combined
with AND, so a test must satisfy every expression to run.
A malformed expression causes the test runner to exit with a non-zero status
before running any tests.
.
.It Fl -experimental-vfs
Enable the experimental \fBnode:vfs\fR module.
+27 -8
View File
@@ -12,6 +12,7 @@ const {
ArrayPrototypePush,
ArrayPrototypePushApply,
ArrayPrototypeShift,
ArrayPrototypeSlice,
ArrayPrototypeSome,
ArrayPrototypeSort,
MathMax,
@@ -93,7 +94,7 @@ const {
parseCommandLine,
} = require('internal/test_runner/utils');
const {
validateAndCanonicalizeTagFilter,
parseTagFilterExpression,
} = require('internal/test_runner/tag_filter');
const { Glob } = require('internal/fs/glob');
const { once } = require('events');
@@ -182,7 +183,7 @@ function getRunArgs(path, { forceExit,
inspectPort,
testNamePatterns,
testSkipPatterns,
testTagFilters,
testTagFilterExpressions,
only,
hasFiles,
testFiles,
@@ -224,8 +225,8 @@ function getRunArgs(path, { forceExit,
if (testSkipPatterns != null) {
ArrayPrototypeForEach(testSkipPatterns, (pattern) => ArrayPrototypePush(runArgs, `--test-skip-pattern=${pattern}`));
}
if (testTagFilters != null) {
ArrayPrototypeForEach(testTagFilters, (value) => ArrayPrototypePush(runArgs, `--experimental-test-tag-filter=${value}`));
if (testTagFilterExpressions != null) {
ArrayPrototypeForEach(testTagFilterExpressions, (expr) => ArrayPrototypePush(runArgs, `--experimental-test-tag-filter=${expr}`));
}
if (only === true) {
ArrayPrototypePush(runArgs, '--test-only');
@@ -872,19 +873,37 @@ function run(options = kEmptyObject) {
});
}
// The public contract of testTagFilters is `string | string[]`. The
// parseCommandLine bootstrap path piggybacks the already-parsed AST array
// on the same field, identifiable by the sibling testTagFilterExpressions
// field which only that path sets. When that marker is present and the
// first element isn't a string, treat the array as ASTs and skip the
// public validation loop. Otherwise validate every element as a string,
// so any non-string input throws ERR_INVALID_ARG_TYPE with the offending
// index regardless of position.
let testTagFilterExpressions = null;
if (testTagFilters != null) {
if (!ArrayIsArray(testTagFilters)) {
testTagFilters = [testTagFilters];
}
if (testTagFilters.length === 0) {
testTagFilters = null;
} else if (options.testTagFilterExpressions != null &&
typeof testTagFilters[0] !== 'string') {
// Internal bootstrap: trust the AST array as already-parsed.
} else {
emitExperimentalWarning('Test tags');
testTagFilters = ArrayPrototypeMap(testTagFilters, (value, i) => (
validateAndCanonicalizeTagFilter(value, `options.testTagFilters[${i}]`)
));
testTagFilterExpressions = ArrayPrototypeSlice(testTagFilters);
testTagFilters = ArrayPrototypeMap(testTagFilters, (value, i) => {
const name = `options.testTagFilters[${i}]`;
if (typeof value !== 'string') {
throw new ERR_INVALID_ARG_TYPE(name, 'string', value);
}
return parseTagFilterExpression(value, name);
});
}
}
testTagFilterExpressions ??= options.testTagFilterExpressions;
validateOneOf(isolation, 'options.isolation', ['process', 'none']);
validateBoolean(coverage, 'options.coverage');
@@ -986,7 +1005,7 @@ function run(options = kEmptyObject) {
inspectPort,
testNamePatterns,
testSkipPatterns,
testTagFilters,
testTagFilterExpressions,
hasFiles: files != null,
globPatterns,
only,
+419 -33
View File
@@ -2,8 +2,14 @@
const {
ArrayPrototypePush,
ObjectFreeze,
RegExp,
RegExpPrototypeExec,
SafeSet,
StringPrototypeCharCodeAt,
StringPrototypeSlice,
StringPrototypeSplit,
StringPrototypeToLowerCase,
StringPrototypeTrim,
} = primordials;
const {
@@ -19,14 +25,68 @@ const {
validateString,
} = require('internal/validators');
const kCharAmpersand = 0x26;
const kCharBackslash = 0x5C;
const kCharCarriageReturn = 0x0D;
const kCharCaret = 0x5E;
const kCharDollar = 0x24;
const kCharDot = 0x2E;
const kCharExclaim = 0x21;
const kCharFormFeed = 0x0C;
const kCharLBrace = 0x7B;
const kCharLBracket = 0x5B;
const kCharLineFeed = 0x0A;
const kCharLParen = 0x28;
const kCharPipe = 0x7C;
const kCharPlus = 0x2B;
const kCharQuestion = 0x3F;
const kCharRBrace = 0x7D;
const kCharRBracket = 0x5D;
const kCharRParen = 0x29;
const kCharSpace = 0x20;
const kCharStar = 0x2A;
const kCharTab = 0x09;
const kCharVerticalTab = 0x0B;
// Single `&` and `|` are allowed here: standalone they have no operator
// meaning, only the doubled forms `&&` and `||` do.
const kTagForbiddenChars = new SafeSet([
kCharTab, kCharLineFeed, kCharVerticalTab, kCharFormFeed, kCharCarriageReturn, kCharSpace,
kCharAmpersand, kCharExclaim, kCharLParen, kCharPipe, kCharRParen, kCharStar,
]);
const kReservedWords = new SafeSet(['and', 'or', 'not']);
/**
* Frozen, shared empty array used as the canonical "no tags" value for
* tests that don't declare their own and have no tagged ancestors.
* Sharing one frozen instance avoids per-test allocation.
* @type {readonly string[]}
*/
const kEmptyTagArray = ObjectFreeze([]);
function isWhitespace(ch) {
return ch === kCharTab || ch === kCharLineFeed || ch === kCharVerticalTab ||
ch === kCharFormFeed || ch === kCharCarriageReturn || ch === kCharSpace;
}
// `*` is intentionally not a break char: it's allowed inside an identifier
// and becomes a wildcard.
function isIdentBreak(ch) {
return isWhitespace(ch) ||
ch === kCharAmpersand || ch === kCharPipe ||
ch === kCharExclaim || ch === kCharLParen || ch === kCharRParen;
}
/**
* Validate and canonicalize a `tags` option value supplied to `test()`,
* `it()`, `suite()`, or `describe()`.
*
* Each tag must be a non-empty string. Tags are canonicalized to lowercase
* and deduplicated, preserving first-seen declaration order.
* Each tag must be a non-empty string that contains no whitespace, no
* operator characters (`& | ! ( ) *`), and is not the reserved word
* `'and'`, `'or'`, or `'not'` in any casing. Tags are canonicalized to
* lowercase and deduplicated on the lowercased form, preserving the
* first-seen declaration order.
*
* Calling this with a non-empty array also emits the one-shot
* `ExperimentalWarning` for the test-tags feature.
@@ -37,7 +97,8 @@ const kEmptyTagArray = ObjectFreeze([]);
* Returns an empty array when `tags` is `[]`.
* @throws {ERR_INVALID_ARG_TYPE} If `tags` is not an array, or any element
* is not a string.
* @throws {ERR_INVALID_ARG_VALUE} If any element is empty.
* @throws {ERR_INVALID_ARG_VALUE} If any element is empty, contains a
* forbidden character, or is a reserved word in any casing.
*/
function validateAndCanonicalizeTagValues(tags, optName) {
validateArray(tags, optName);
@@ -55,7 +116,34 @@ function validateAndCanonicalizeTagValues(tags, optName) {
const tag = tags[i];
const elemName = `${optName}[${i}]`;
const lower = validateAndCanonicalizeTagFilter(tag, elemName);
validateString(tag, elemName);
if (tag.length === 0) {
throw new ERR_INVALID_ARG_VALUE(elemName, tag, 'must not be empty');
}
for (let j = 0; j < tag.length; j++) {
const ch = StringPrototypeCharCodeAt(tag, j);
if (kTagForbiddenChars.has(ch)) {
throw new ERR_INVALID_ARG_VALUE(
elemName,
tag,
'contains a forbidden character. Tags must not contain whitespace, ' +
'operator characters (&, |, !, (, ), *), or be the reserved words ' +
'\'and\', \'or\', or \'not\'',
);
}
}
const lower = StringPrototypeToLowerCase(tag);
if (kReservedWords.has(lower)) {
throw new ERR_INVALID_ARG_VALUE(
elemName,
tag,
'must not be the reserved word \'and\', \'or\', or \'not\'',
);
}
if (!seen.has(lower)) {
seen.add(lower);
@@ -66,43 +154,341 @@ function validateAndCanonicalizeTagValues(tags, optName) {
return ObjectFreeze(result);
}
/**
* Validate and canonicalize a single tag-filter string supplied via
* `--experimental-test-tag-filter` or `testTagFilters`.
*
* Filters are literal tag names: a test passes the filter when its tag set
* contains the lowercased filter value. Multi-occurrence composes by AND
* at the call site (see `evaluateTagFilters`).
* @param {unknown} value - The supplied filter string.
* @param {string} name - Argument name embedded in thrown error messages.
* @returns {string} The lowercased filter value.
* @throws {ERR_INVALID_ARG_TYPE} If `value` is not a string.
* @throws {ERR_INVALID_ARG_VALUE} If `value` is empty.
*/
function validateAndCanonicalizeTagFilter(value, name) {
validateString(value, name);
// ---------------------------------------------------------------------------
// Lexer
// ---------------------------------------------------------------------------
if (value.length === 0) {
throw new ERR_INVALID_ARG_VALUE(name, value, 'must not be empty');
// Token types
const T_AND = 'AND';
const T_OR = 'OR';
const T_NOT = 'NOT';
const T_LPAREN = 'LPAREN';
const T_RPAREN = 'RPAREN';
const T_IDENT = 'IDENT';
const T_EOF = 'EOF';
function lex(expr, name) {
const tokens = [];
let i = 0;
const len = expr.length;
while (i < len) {
const ch = StringPrototypeCharCodeAt(expr, i);
if (isWhitespace(ch)) {
i++;
continue;
}
if (ch === kCharAmpersand) {
if (i + 1 < len && StringPrototypeCharCodeAt(expr, i + 1) === kCharAmpersand) {
ArrayPrototypePush(tokens, { __proto__: null, type: T_AND, pos: i });
i += 2;
} else {
throw new ERR_INVALID_ARG_VALUE(
name, expr,
`unexpected '&' at position ${i}; use '&&' or 'and'`,
);
}
continue;
}
if (ch === kCharPipe) {
if (i + 1 < len && StringPrototypeCharCodeAt(expr, i + 1) === kCharPipe) {
ArrayPrototypePush(tokens, { __proto__: null, type: T_OR, pos: i });
i += 2;
} else {
throw new ERR_INVALID_ARG_VALUE(
name, expr,
`unexpected '|' at position ${i}; use '||' or 'or'`,
);
}
continue;
}
if (ch === kCharExclaim) {
ArrayPrototypePush(tokens, { __proto__: null, type: T_NOT, pos: i });
i++;
continue;
}
if (ch === kCharLParen) {
ArrayPrototypePush(tokens, { __proto__: null, type: T_LPAREN, pos: i });
i++;
continue;
}
if (ch === kCharRParen) {
ArrayPrototypePush(tokens, { __proto__: null, type: T_RPAREN, pos: i });
i++;
continue;
}
const start = i;
while (i < len && !isIdentBreak(StringPrototypeCharCodeAt(expr, i))) {
i++;
}
const ident = StringPrototypeSlice(expr, start, i);
const lower = StringPrototypeToLowerCase(ident);
if (lower === 'and') {
ArrayPrototypePush(tokens, { __proto__: null, type: T_AND, pos: start });
} else if (lower === 'or') {
ArrayPrototypePush(tokens, { __proto__: null, type: T_OR, pos: start });
} else if (lower === 'not') {
ArrayPrototypePush(tokens, { __proto__: null, type: T_NOT, pos: start });
} else {
ArrayPrototypePush(tokens, { __proto__: null, type: T_IDENT, value: ident, pos: start });
}
}
return StringPrototypeToLowerCase(value);
ArrayPrototypePush(tokens, { __proto__: null, type: T_EOF, pos: len });
return tokens;
}
// ---------------------------------------------------------------------------
// Regex escaping for wildcard tags
// ---------------------------------------------------------------------------
// `*` is the only metacharacter intentionally not escaped: the caller
// handles it as a wildcard.
function escapeForRegExp(str) {
let result = '';
for (let i = 0; i < str.length; i++) {
const ch = StringPrototypeCharCodeAt(str, i);
if (
ch === kCharLBracket || ch === kCharRBracket ||
ch === kCharLParen || ch === kCharRParen ||
ch === kCharLBrace || ch === kCharRBrace ||
ch === kCharCaret || ch === kCharDollar ||
ch === kCharDot || ch === kCharPlus ||
ch === kCharQuestion || ch === kCharBackslash ||
ch === kCharPipe
) {
result += '\\';
}
result += StringPrototypeSlice(str, i, i + 1);
}
return result;
}
function buildWildcardRegex(ident) {
// Lowercase the source first and drop the /i flag, so wildcard matching
// uses the same casing semantics as literal matching (which lowercases the
// tag value at validation time). Using /i would diverge for non-ASCII
// characters whose String.prototype.toLowerCase() output differs from
// RegExp /i case folding (e.g. Turkish I-with-dot U+0130).
const parts = StringPrototypeSplit(StringPrototypeToLowerCase(ident), '*');
let pattern = '^';
for (let i = 0; i < parts.length; i++) {
if (i > 0) {
pattern += '.*';
}
pattern += escapeForRegExp(parts[i]);
}
pattern += '$';
return new RegExp(pattern);
}
function makeTagNode(ident) {
for (let i = 0; i < ident.length; i++) {
if (StringPrototypeCharCodeAt(ident, i) === kCharStar) {
return { __proto__: null, type: 'wildcard', re: buildWildcardRegex(ident) };
}
}
return { __proto__: null, type: 'tag', value: StringPrototypeToLowerCase(ident) };
}
// ---------------------------------------------------------------------------
// Parser - recursive descent
// Precedence: not > and > or (standard Boolean)
// ---------------------------------------------------------------------------
/**
* Parse a tag filter expression string into an AST suitable for
* `evaluateTagFilters`.
*
* Supported syntax:
* - Identifiers: any non-whitespace, non-operator characters; case-
* insensitive when matched. A `*` inside an identifier becomes a
* wildcard that matches any sequence of characters; a bare `*`
* matches any non-empty tag.
* - Boolean operators: `and`/`&&`, `or`/`||`, `not`/`!`, with the
* standard precedence `not > and > or` and left-associative binary
* operators. Word forms must be whitespace-separated; punctuation
* forms do not.
* - Parentheses for grouping.
* @param {string} expr - The expression to parse.
* @param {string} name - Option name to embed in thrown error messages
* (typically `'--experimental-test-tag-filter[<i>]'` or
* `'options.testTagFilters[<i>]'`).
* @returns {object} An opaque AST node. The shape is internal; pass it
* only to `evaluateTagFilters`.
* @throws {ERR_INVALID_ARG_TYPE} If `expr` is not a string.
* @throws {ERR_INVALID_ARG_VALUE} If `expr` is empty, contains only
* whitespace, or has a syntax error (unbalanced parentheses, dangling
* operator, adjacent identifiers, etc.). The error message includes
* the offending position.
*/
function parseTagFilterExpression(expr, name) {
validateString(expr, name);
if (StringPrototypeTrim(expr).length === 0) {
throw new ERR_INVALID_ARG_VALUE(name, expr, 'must not be empty');
}
const tokens = lex(expr, name);
let pos = 0;
function peek() {
return tokens[pos];
}
function consume() {
return tokens[pos++];
}
function parseExpr() {
return parseOr();
}
function parseOr() {
let left = parseAnd();
while (peek().type === T_OR) {
consume();
const right = parseAnd();
left = { __proto__: null, type: 'or', left, right };
}
return left;
}
function parseAnd() {
let left = parseNot();
while (peek().type === T_AND) {
consume();
const right = parseNot();
left = { __proto__: null, type: 'and', left, right };
}
return left;
}
function parseNot() {
if (peek().type === T_NOT) {
const { pos: notPos } = consume();
if (peek().type === T_EOF) {
throw new ERR_INVALID_ARG_VALUE(
name, expr,
`expected operand after 'not' at position ${notPos}`,
);
}
const operand = parseNot();
return { __proto__: null, type: 'not', operand };
}
if (peek().type === T_LPAREN) {
const { pos: lPos } = consume();
if (peek().type === T_EOF) {
throw new ERR_INVALID_ARG_VALUE(
name, expr,
`unmatched '(' at position ${lPos}`,
);
}
const inner = parseExpr();
if (peek().type !== T_RPAREN) {
throw new ERR_INVALID_ARG_VALUE(
name, expr,
`expected ')' at position ${peek().pos}`,
);
}
consume();
return inner;
}
if (peek().type === T_IDENT) {
const { value } = consume();
return makeTagNode(value);
}
const tok = peek();
throw new ERR_INVALID_ARG_VALUE(
name, expr,
`unexpected token '${tok.type}' at position ${tok.pos}`,
);
}
const ast = parseExpr();
if (peek().type !== T_EOF) {
const tok = peek();
throw new ERR_INVALID_ARG_VALUE(
name, expr,
`unexpected token at position ${tok.pos}`,
);
}
return ast;
}
// ---------------------------------------------------------------------------
// Evaluator
// ---------------------------------------------------------------------------
/**
* Evaluate a tag filter AST against a set of test tags.
*
* Untagged tests have an empty SafeSet. The semantics are:
* - An include expression (tag/wildcard/and/or) evaluates false for empty set.
* - 'not X' evaluates true when X is false (so untagged tests pass 'not X').
* @param {object} ast - AST node from parseTagFilterExpression.
* @param {SafeSet} tags - Lowercased canonical tags for the test.
* @returns {boolean}
*/
function evaluateTagFilter(ast, tags) {
const { type } = ast;
if (type === 'or') {
return evaluateTagFilter(ast.left, tags) || evaluateTagFilter(ast.right, tags);
}
if (type === 'and') {
return evaluateTagFilter(ast.left, tags) && evaluateTagFilter(ast.right, tags);
}
if (type === 'not') {
return !evaluateTagFilter(ast.operand, tags);
}
if (type === 'tag') {
return tags.has(ast.value);
}
// type === 'wildcard'
let matched = false;
tags.forEach((tag) => {
if (!matched && RegExpPrototypeExec(ast.re, tag) !== null) {
matched = true;
}
});
return matched;
}
/**
* Evaluate a list of tag filters against a test's tag set.
* Evaluate one or more tag filter ASTs against a test's tag set.
*
* Multiple filters compose by AND: a test passes only when every filter is
* present in its tag set. An empty `filters` array always passes. Untagged
* tests have an empty tag set and so are excluded under any non-empty
* filter.
* @param {string[]} filters - Lowercased canonical filter values.
* @param {SafeSet<string>} tags - The test's lowercased canonical tag set.
* Multiple ASTs compose by AND: a test passes only when every AST
* evaluates true for it. An empty `asts` array always passes.
*
* Untagged tests (an empty `tags` set) fail any include expression
* (`tag`, `wildcard`, `and`, `or`) and pass `not X`, so excluding tags
* does not accidentally remove untagged tests.
* @param {object[]} asts - ASTs returned by `parseTagFilterExpression`.
* @param {SafeSet<string>} tags - The test's lowercased canonical tag
* set (use `Set` in non-internal contexts; both work).
* @returns {boolean} `true` if the test should run.
*/
function evaluateTagFilters(filters, tags) {
for (let i = 0; i < filters.length; i++) {
if (!tags.has(filters[i])) {
function evaluateTagFilters(asts, tags) {
for (let i = 0; i < asts.length; i++) {
if (!evaluateTagFilter(asts[i], tags)) {
return false;
}
}
@@ -112,6 +498,6 @@ function evaluateTagFilters(filters, tags) {
module.exports = {
evaluateTagFilters,
kEmptyTagArray,
validateAndCanonicalizeTagFilter,
parseTagFilterExpression,
validateAndCanonicalizeTagValues,
};
+11 -7
View File
@@ -30,7 +30,7 @@ const {
} = primordials;
const {
validateAndCanonicalizeTagFilter,
parseTagFilterExpression,
} = require('internal/test_runner/tag_filter');
const { AsyncResource } = require('async_hooks');
@@ -273,6 +273,7 @@ function parseCommandLine() {
let testNamePatterns = mapPatternFlagToRegExArray('--test-name-pattern');
let testSkipPatterns = mapPatternFlagToRegExArray('--test-skip-pattern');
let testTagFilters = null;
let testTagFilterExpressions = null;
if (isChildProcessV8) {
kBuiltinReporters.set('v8-serializer', 'internal/test_runner/reporter/v8-serializer');
@@ -308,13 +309,14 @@ function parseCommandLine() {
const tagFilterFlag = getOptionValue('--experimental-test-tag-filter');
if (tagFilterFlag?.length > 0) {
emitExperimentalWarning('Test tags');
// File wrappers are exempt from tag filtering, so holding the filters
// in the parent is safe under any isolation mode; under
// isolation='process' the canonical values are re-emitted to the
// child processes, which apply the filter themselves.
// File wrappers are exempt from tag filtering, so holding the parsed
// ASTs in the parent is safe under any isolation mode; under
// isolation='process' the source expressions are re-emitted to the
// child processes, which re-parse and apply the filter themselves.
testTagFilterExpressions = tagFilterFlag;
testTagFilters = ArrayPrototypeMap(
tagFilterFlag,
(value, i) => validateAndCanonicalizeTagFilter(value, `--experimental-test-tag-filter[${i}]`),
(expr, i) => parseTagFilterExpression(expr, `--experimental-test-tag-filter[${i}]`),
);
}
@@ -359,9 +361,10 @@ function parseCommandLine() {
const tagFilterFlag = getOptionValue('--experimental-test-tag-filter');
if (tagFilterFlag?.length > 0) {
emitExperimentalWarning('Test tags');
testTagFilterExpressions = tagFilterFlag;
testTagFilters = ArrayPrototypeMap(
tagFilterFlag,
(value, i) => validateAndCanonicalizeTagFilter(value, `--experimental-test-tag-filter[${i}]`),
(expr, i) => parseTagFilterExpression(expr, `--experimental-test-tag-filter[${i}]`),
);
}
}
@@ -426,6 +429,7 @@ function parseCommandLine() {
sourceMaps,
testNamePatterns,
testSkipPatterns,
testTagFilterExpressions,
testTagFilters,
timeout,
updateSnapshots,
+75 -16
View File
@@ -62,32 +62,55 @@ test('untagged parent runs only the matching tagged child', () => {
assert.doesNotMatch(stdout, /plain sibling/);
});
test('repeated --experimental-test-tag-filter ANDs together', () => {
// db AND integration: only "db plus integration" satisfies both.
const { status, stdout } = run([
'--experimental-test-tag-filter=db',
'--experimental-test-tag-filter=integration',
]);
test('not X also includes untagged tests', () => {
const { status, stdout } = run(['--experimental-test-tag-filter=not flaky']);
assert.strictEqual(status, 0);
pass(stdout, 1);
assert.match(stdout, /ok \d+ - db plus integration/);
assert.doesNotMatch(stdout, /^ok \d+ - db only/m);
assert.doesNotMatch(stdout, /^ok \d+ - db flaky/m);
pass(stdout, 8);
assert.match(stdout, /ok \d+ - untagged/);
assert.doesNotMatch(stdout, /db flaky/);
assert.doesNotMatch(stdout, /^ok \d+ - only flaky/m);
});
test('untagged tests are excluded under any positive filter', () => {
const { status, stdout } = run(['--experimental-test-tag-filter=db']);
test('repeated --experimental-test-tag-filter ANDs together', () => {
const { status, stdout } = run([
'--experimental-test-tag-filter=db',
'--experimental-test-tag-filter=not flaky',
]);
assert.strictEqual(status, 0);
pass(stdout, 2);
assert.match(stdout, /ok \d+ - db only/);
assert.match(stdout, /ok \d+ - db plus integration/);
assert.doesNotMatch(stdout, /db flaky/);
});
test('bare * matches any tagged test, excludes untagged', () => {
const { status, stdout } = run(['--experimental-test-tag-filter=*']);
assert.strictEqual(status, 0);
pass(stdout, 8);
assert.doesNotMatch(stdout, /^ok \d+ - untagged/m);
});
test('wildcard prefix: db* matches db, db:postgres, etc.', () => {
const { status, stdout } = run(['--experimental-test-tag-filter=db*']);
assert.strictEqual(status, 0);
pass(stdout, 4);
assert.match(stdout, /db wildcard match/);
});
test('boolean composition: unit && !slow', () => {
const { status, stdout } = run(['--experimental-test-tag-filter=unit && !slow']);
assert.strictEqual(status, 0);
pass(stdout, 1);
assert.match(stdout, /ok \d+ - unit only/);
});
test('isolation=none produces identical filtered set', () => {
const proc = run(['--experimental-test-tag-filter=db']);
const none = run(['--experimental-test-tag-filter=db'], ['--test-isolation=none']);
const proc = run(['--experimental-test-tag-filter=unit && !slow']);
const none = run(['--experimental-test-tag-filter=unit && !slow'], ['--test-isolation=none']);
assert.strictEqual(proc.status, 0);
assert.strictEqual(none.status, 0);
pass(proc.stdout, 3);
pass(none.stdout, 3);
pass(proc.stdout, 1);
pass(none.stdout, 1);
});
test('combined with --test-skip-pattern (pure AND)', () => {
@@ -101,8 +124,44 @@ test('combined with --test-skip-pattern (pure AND)', () => {
assert.doesNotMatch(stdout, /db flaky/);
});
test('malformed expression: unbalanced parenthesis errors at startup', () => {
const { status, stderr, stdout } = run(['--experimental-test-tag-filter=(a']);
assert.notStrictEqual(status, 0);
// Match the full sentinel error line so noise elsewhere in stderr can't
// satisfy the assertion via interleaved substrings.
assert.match(
stderr,
/^TypeError \[ERR_INVALID_ARG_VALUE\]: The argument '--experimental-test-tag-filter\[0\]' expected '\)' at position \d+/m,
);
// No file should have been spawned.
assert.doesNotMatch(stdout, /^ok 1 -/m);
});
test('malformed expression: dangling operator errors at startup', () => {
const { status, stderr } = run(['--experimental-test-tag-filter=a and']);
assert.notStrictEqual(status, 0);
assert.match(
stderr,
/^TypeError \[ERR_INVALID_ARG_VALUE\]: The argument '--experimental-test-tag-filter\[0\]' unexpected token 'EOF' at position \d+/m,
);
});
test('empty --experimental-test-tag-filter= is rejected by argv parser', () => {
const { status, stderr } = run(['--experimental-test-tag-filter=']);
assert.notStrictEqual(status, 0);
assert.match(stderr, /requires an argument/);
});
test('reserved word as tag value errors at registration', () => {
const child = spawnSync(process.execPath, [
'-e',
"require('node:test').test('x', { tags: ['and'] }, () => {});",
]);
assert.notStrictEqual(child.status, 0);
// Match the full sentinel error line so noise elsewhere in stderr can't
// satisfy the assertion via interleaved substrings.
assert.match(
child.stderr.toString(),
/^TypeError \[ERR_INVALID_ARG_VALUE\]: The property 'options\.tags\[0\]' must not be the reserved word/m,
);
});
@@ -0,0 +1,257 @@
// Flags: --expose-internals
import '../common/index.mjs';
import assert from 'node:assert';
import { test } from 'node:test';
import tagFilter from 'internal/test_runner/tag_filter';
const { evaluateTagFilters, parseTagFilterExpression } = tagFilter;
function tagSet(...tags) {
return new Set(tags);
}
function matches(expr, ...tags) {
return evaluateTagFilters([parseTagFilterExpression(expr, 'expr')], tagSet(...tags));
}
test('single identifier', () => {
assert.strictEqual(matches('db', 'db'), true);
assert.strictEqual(matches('db', 'other'), false);
assert.strictEqual(matches('db'), false);
});
test('case-insensitive match', () => {
// The evaluator expects an already-lowercased tag set, so 'DB' in the set
// must not match 'db' in the filter.
assert.strictEqual(matches('DB', 'db'), true);
assert.strictEqual(matches('db', 'DB'), false);
});
test('and / && have higher precedence than or / ||', () => {
for (const expr of ['a or b and c', 'a || b && c']) {
assert.strictEqual(matches(expr, 'a'), true);
assert.strictEqual(matches(expr, 'b'), false);
assert.strictEqual(matches(expr, 'c'), false);
assert.strictEqual(matches(expr, 'a', 'b'), true);
assert.strictEqual(matches(expr, 'a', 'c'), true);
assert.strictEqual(matches(expr, 'b', 'c'), true);
assert.strictEqual(matches(expr, 'a', 'b', 'c'), true);
}
});
test('not has higher precedence than and', () => {
for (const expr of ['not a and b', '!a && b']) {
assert.strictEqual(matches(expr, 'b'), true);
assert.strictEqual(matches(expr, 'a', 'b'), false);
}
});
test('not has higher precedence than or', () => {
for (const expr of ['not a or b', '!a || b']) {
assert.strictEqual(matches(expr), true);
assert.strictEqual(matches(expr, 'a'), false);
assert.strictEqual(matches(expr, 'b'), true);
assert.strictEqual(matches(expr, 'a', 'b'), true);
}
});
test('parentheses override precedence', () => {
for (const expr of ['(a or b) and c', '(a || b) && c']) {
assert.strictEqual(matches(expr, 'a'), false);
assert.strictEqual(matches(expr, 'c'), false);
assert.strictEqual(matches(expr, 'a', 'c'), true);
assert.strictEqual(matches(expr, 'b', 'c'), true);
}
});
test('double negation', () => {
for (const expr of ['not not a', '!!a']) {
assert.strictEqual(matches(expr, 'a'), true);
assert.strictEqual(matches(expr), false);
}
});
test('mixing word and punctuation forms in one expression', () => {
for (const expr of ['unit && not flaky', 'unit and !flaky']) {
assert.strictEqual(matches(expr, 'unit'), true);
assert.strictEqual(matches(expr, 'unit', 'flaky'), false);
}
});
test('mixed forms inside parentheses, word and punct equivalence', () => {
// Each pair below is the same expression in word form and punctuation form.
for (const expr of ['a and (b || not c)', 'a && (b or not c)']) {
assert.strictEqual(matches(expr, 'a'), true);
assert.strictEqual(matches(expr, 'a', 'b'), true);
assert.strictEqual(matches(expr, 'a', 'c'), false);
assert.strictEqual(matches(expr, 'a', 'b', 'c'), true);
assert.strictEqual(matches(expr, 'b'), false);
}
for (const expr of ['(a || b) && !c', '(a or b) and not c']) {
assert.strictEqual(matches(expr, 'a'), true);
assert.strictEqual(matches(expr, 'b'), true);
assert.strictEqual(matches(expr, 'a', 'c'), false);
assert.strictEqual(matches(expr, 'c'), false);
}
for (const expr of ['not (a or b) and c', '!(a || b) && c']) {
assert.strictEqual(matches(expr, 'c'), true);
assert.strictEqual(matches(expr, 'a', 'c'), false);
assert.strictEqual(matches(expr, 'b', 'c'), false);
}
for (const expr of ['!(a && b) || c', 'not (a and b) or c']) {
assert.strictEqual(matches(expr, 'a'), true);
assert.strictEqual(matches(expr, 'a', 'b'), false);
assert.strictEqual(matches(expr, 'a', 'b', 'c'), true);
}
});
test('nested parentheses and wildcards mixed with word forms', () => {
for (const expr of [
'(db* or unit) and not (slow || flaky)',
'(db* || unit) && !(slow or flaky)',
]) {
assert.strictEqual(matches(expr, 'db'), true);
assert.strictEqual(matches(expr, 'db:postgres'), true);
assert.strictEqual(matches(expr, 'unit'), true);
assert.strictEqual(matches(expr, 'unit', 'slow'), false);
assert.strictEqual(matches(expr, 'db', 'flaky'), false);
assert.strictEqual(matches(expr, 'mongo'), false);
}
});
test("'notslow' is one identifier, 'not slow' is two tokens", () => {
assert.strictEqual(matches('notslow', 'notslow'), true);
assert.strictEqual(matches('notslow', 'slow'), false);
assert.strictEqual(matches('not slow'), true);
assert.strictEqual(matches('not slow', 'slow'), false);
});
test('punctuation does not need whitespace separation', () => {
assert.strictEqual(matches('!a', 'b'), true);
assert.strictEqual(matches('!a', 'a'), false);
assert.strictEqual(matches('a&&b', 'a', 'b'), true);
assert.strictEqual(matches('a||b', 'a'), true);
});
test('wildcard expands across allowed alphabet', () => {
assert.strictEqual(matches('db*', 'db'), true);
assert.strictEqual(matches('db*', 'db1'), true);
assert.strictEqual(matches('db*', 'db:postgres'), true);
assert.strictEqual(matches('db*', 'mongo'), false);
});
test('bare * matches any tagged test', () => {
assert.strictEqual(matches('*', 'anything'), true);
assert.strictEqual(matches('*', 'a', 'b'), true);
assert.strictEqual(matches('*'), false);
});
test('wildcard in middle and at start', () => {
assert.strictEqual(matches('*db', 'mydb'), true);
assert.strictEqual(matches('*db', 'db'), true);
assert.strictEqual(matches('*db', 'dbmy'), false);
assert.strictEqual(matches('a*z', 'abz'), true);
assert.strictEqual(matches('a*z', 'az'), true);
assert.strictEqual(matches('a*z', 'abc'), false);
});
test('regex metacharacters in tags are escaped', () => {
// The literal `.` in the filter must match a literal dot only, not any
// arbitrary character (which would be the unescaped regex meaning).
assert.strictEqual(matches('a.b*', 'a.b'), true);
assert.strictEqual(matches('a.b*', 'axb'), false);
});
test('wildcard case-folding matches literal-tag case-folding for non-ASCII', () => {
// Authoring a tag 'İ' (U+0130) goes through String.prototype.toLowerCase
// and stores 'i̇' (two code points). Wildcard matching must use the same
// fold so authors see consistent behavior across literal and wildcard
// filters.
const folded = 'İ'.toLowerCase();
assert.strictEqual(matches('İ', folded), true);
assert.strictEqual(matches('İ*', folded), true);
assert.strictEqual(matches('İ*', `${folded}foo`), true);
assert.strictEqual(matches('*İ', `foo${folded}`), true);
});
test('untagged test fails any include expression', () => {
for (const expr of ['a', 'a or b', 'a || b', 'a and b', 'a && b', '*']) {
assert.strictEqual(matches(expr), false);
}
});
test("'not X' against an untagged test is true", () => {
for (const expr of ['not a', '!a']) {
assert.strictEqual(matches(expr), true);
}
});
test('multiple filters AND together', () => {
const f1 = parseTagFilterExpression('a', 'expr');
const f2 = parseTagFilterExpression('not b', 'expr');
assert.strictEqual(evaluateTagFilters([f1, f2], tagSet('a')), true);
assert.strictEqual(evaluateTagFilters([f1, f2], tagSet('a', 'b')), false);
assert.strictEqual(evaluateTagFilters([f1, f2], tagSet('b')), false);
assert.strictEqual(evaluateTagFilters([f1, f2], tagSet()), false);
});
test('empty filter list always passes', () => {
assert.strictEqual(evaluateTagFilters([], tagSet()), true);
assert.strictEqual(evaluateTagFilters([], tagSet('a')), true);
});
// --- Malformed expressions -------------------------------------------------
function expectInvalid(expr) {
assert.throws(
() => parseTagFilterExpression(expr, 'expr'),
{ code: 'ERR_INVALID_ARG_VALUE' },
`expected '${expr}' to be invalid`,
);
}
test('rejects empty / whitespace-only expressions', () => {
for (const expr of ['', ' ', '\t\n']) {
expectInvalid(expr);
}
});
test('rejects unbalanced parentheses', () => {
for (const expr of ['(', '((', '(a', 'a)', '((a)', '(a))']) {
expectInvalid(expr);
}
});
test('rejects dangling operators', () => {
for (const expr of ['not', 'a and', 'a or', '&& a', '|| a', 'a &&', 'a ||']) {
expectInvalid(expr);
}
});
test('rejects two adjacent identifiers', () => {
expectInvalid('foo bar');
});
test('rejects single & and single |', () => {
for (const expr of ['a & b', 'a | b']) {
expectInvalid(expr);
}
});
test('rejects empty parentheses', () => {
expectInvalid('()');
});
test('rejects non-string expressions', () => {
for (const expr of [42, null, undefined, true, [], {}, Symbol('s')]) {
assert.throws(
() => parseTagFilterExpression(expr, 'expr'),
{ code: 'ERR_INVALID_ARG_TYPE' },
`expected ${String(expr)} to be rejected`,
);
}
});
@@ -101,4 +101,12 @@ describe('tag-bearing event payloads', { concurrency: false }, () => {
// eslint-disable-next-line no-unused-vars
for await (const _ of stream);
});
it('process isolation forwards --experimental-test-tag-filter to children', async () => {
const stream = run({ files: [fixture], testTagFilters: ['not nonexistent'] });
stream.on('test:fail', common.mustNotCall());
stream.on('test:pass', common.mustCall(13));
// eslint-disable-next-line no-unused-vars
for await (const _ of stream);
});
});
@@ -50,7 +50,7 @@ test('--experimental-test-tag-filter fires the warning', () => {
const { stderr } = runChild([
'--test',
'--test-reporter=tap',
'--experimental-test-tag-filter=db',
'--experimental-test-tag-filter=*',
'--test-isolation=none',
fixture,
]);
+28 -1
View File
@@ -37,6 +37,26 @@ test('empty-string tag throws ERR_INVALID_ARG_VALUE', () => {
);
});
test('forbidden characters throw ERR_INVALID_ARG_VALUE', () => {
for (const bad of ['a b', 'a\tb', 'a\nb', 'a&b', 'a|b', 'a!b', 'a(b', 'a)b', 'a*b', '*', '&&', '||']) {
assert.throws(
() => test('x', { tags: [bad] }, () => {}),
{ code: 'ERR_INVALID_ARG_VALUE' },
`expected throw for tag=${JSON.stringify(bad)}`,
);
}
});
test('reserved words throw ERR_INVALID_ARG_VALUE in any casing', () => {
for (const word of ['and', 'AND', 'And', 'or', 'OR', 'Or', 'not', 'NOT', 'Not']) {
assert.throws(
() => test('x', { tags: [word] }, () => {}),
{ code: 'ERR_INVALID_ARG_VALUE' },
`expected throw for reserved word ${word}`,
);
}
});
test('Unicode and most punctuation are allowed', () => {
// None of these should throw.
test('unicode-1', { tags: ['café'] }, () => {});
@@ -80,7 +100,7 @@ test('suite() and describe() validate tags identically', () => {
{ code: 'ERR_INVALID_ARG_TYPE' },
);
assert.throws(
() => describe('s', { tags: [''] }, () => {}),
() => describe('s', { tags: ['and'] }, () => {}),
{ code: 'ERR_INVALID_ARG_VALUE' },
);
});
@@ -92,6 +112,13 @@ test('it() validates tags identically to test()', () => {
);
});
test('it() validates tags identically to test()', () => {
assert.throws(
() => it('i', { tags: ['a b'] }, () => {}),
{ code: 'ERR_INVALID_ARG_VALUE' },
);
});
test('run() rejects non-string testTagFilters values', () => {
for (const bad of [[42], [{}], [null], [undefined], [true], [Symbol('s')]]) {
assert.throws(