--vfs-mount mounted a source without running it, and shared one ordered
list of sources with --vfs-load, so neither option could say which entry
it had contributed: the entry point was recovered from the position of
--vfs-load among the mounts, in a list NODE_OPTIONS could prepend to.
Nothing needs more than one mount from the command line: a program that
wants more can mount them itself through node:vfs, where it also gets
the instance.
Remove --vfs-mount, leaving --vfs-load with the single source it mounts
and runs, and reserve layer 0 for that source, numbering the file
systems a program mounts itself from 1. The source is then at the same
mount point in every thread, whatever else that thread mounts -
including a thread where a --require preload mounted a file system of
its own first - so a path into it stays valid in a worker.
A worker still does not run that entry point: it inherits the source but
not the decision to load from it. A worker created with its own execArgv
inherits neither, so the documentation now says that such a worker must
be given --experimental-vfs and --vfs-load again to run a script from
the mount, and that --experimental-vfs is also what makes node:vfs
available to the worker's own code.
ERR_VFS_INVALID_TARGET now names --vfs-load as the source's origin, and
the startup test moves to test-vfs-load.js, with the cases that covered
mounting without loading removed and cases for the reserved mount point
added.
Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
PR-URL: https://github.com/nodejs/node/pull/66162
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Requiring `node:trace_events` and calling `createTracing()` aborted
the process, and `getEnabledCategories()` dereferenced a null pointer,
when Node.js runs on an embedder's own platform
(`kNoInitializeNodeV8Platform`): no `tracing::Agent` exists then, and
`lib/trace_events.js` only checked the compile-time `hasTracing` flag
and `ownsProcessState` before handing the null agent to the binding.
Have the binding report whether an agent exists and throw the existing
`ERR_TRACE_EVENTS_UNAVAILABLE` when it does not, as for a
`--without-v8-platform` build.
Refs: https://github.com/nodejs/node/pull/19803
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65954
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Mounting a virtual file system requires calling vfs.mount() from inside
the program, so a program cannot itself be served from one: something
already running has to mount the VFS first.
Add two startup flags. --vfs-mount=<source> mounts a directory or an
archive as a virtual file system, and may be repeated.
--vfs-load=<source> mounts that source the same way and additionally
runs the entry point and all subsequent require()/import resolution
against it rather than against the real file system, so an application
can be run straight out of a directory or a ZIP archive:
node --experimental-vfs --vfs-load=my-app.zip
Both options append to one list, so mounts happen in the order written:
node --experimental-vfs --vfs-mount=a --vfs-load=b --vfs-mount=c
mounts a, b and c in that order and runs b. Mounting the same source
twice mounts it twice, at two mount points, and the entry point comes
from the mount --vfs-load contributed rather than the earlier one.
The provider backing a source is chosen from the source itself rather
than from its name: a directory is served by RealFSProvider and a file
whose bytes are a ZIP archive by ZipProvider, so an archive can carry
any extension. vfs.registerProvider() registers a provider for formats
there is no built-in for; selection is deferred until -r and --import
preloads have run, so a preloaded module can register one before its
source is claimed.
The entry point comes from the mount the way `node <directory>` takes
one, from package.json "main" or index.js. Nothing is consumed as an
entry point argument, so every positional reaches the program unchanged
from argv[2] on, and argv[1] reports the named source rather than the
reserved mount point, which is an opaque implementation detail.
--vfs-load may be given at most once, and is not permitted in
NODE_OPTIONS: which entry point runs is the command line's decision, and
NODE_OPTIONS is parsed first, so an environment variable could otherwise
redirect any invocation. --vfs-mount is permitted there. Workers inherit
the same mounts in the same order but not the loading: a worker mounts
what the parent mounted and runs its own entry point.
Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
PR-URL: https://github.com/nodejs/node/pull/65748
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Often closely related to util.debounce use cases, this adds an
implementation of throttle(fn) .. returns a function that limits
the number of concurrent invocations of fn.
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
PR-URL: https://github.com/nodejs/node/pull/65899
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
This adds support for transferring net.BoundSocket instances to other
threads via the worker_threads postMessage() transfer list, and for
sending them to child processes as the sendHandle argument of
subprocess.send(), following on from the BoundSocket introduction.
A BoundSocket reserves a port synchronously at construction time.
Making it transferable means a port can be reserved on one thread or
process and the bound (but not yet listening or connected) TCP handle
handed off to another to listen or connect on, without racing on the
bind.
For threads, BoundSocket implements kTransfer/kTransferList/
kDeserialize, moving the underlying TCP handle with the same mechanism
used for net.Socket and net.Server transfer. For child processes, the
handleConversion entry reuses the same transfer protocol on the
sending side and the same _TransferredBoundSocket deserialization path
on the receiving side; the underlying transport is that of cluster's
shared-handle scheduling: SCM_RIGHTS on Unix and WSADuplicateSocket on
Windows, both of which carry bind state.
In both cases the source instance is left in the adopted state:
address(), fd() and close() throw ERR_SOCKET_HANDLE_ADOPTED. Transfer
requires an un-adopted, open TCP handle, otherwise
ERR_WORKER_HANDLE_NOT_TRANSFERABLE is thrown; pipe (path) binds are
not transferable and throw ERR_INVALID_HANDLE_TYPE when sent over IPC.
On the receiving side the local address is re-derived from the handle
rather than trusted from serialized state.
Signed-off-by: Guy Bedford <guybedford@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/64725
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Add ZIP archive support to the node:zlib module through three classes
and a set of helpers:
- ZipEntry: a single archive member, with buffered reads (content()),
bounded-memory streaming reads (contentIterator()), and
create()/createStream() for building members.
- ZipFile: random access to an archive backed by a file descriptor,
reading members lazily without retaining their content and writing
new members in place; opened with open()/openSync().
- ZipBuffer: a zero-copy, in-memory view over an archive already held
in a Buffer.
createZipArchive() serializes a sequence of entries into an archive
byte stream, and setMaxZipContentSize() bounds the default in-memory
decompression size. Every operation has both an asynchronous and a
synchronous form.
Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
PR-URL: https://github.com/nodejs/node/pull/64339
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Aviv Keller <me@aviv.sh>
This brings back several improvements that were reverted by mistake when
landing https://redirect.github.com/nodejs/node/pull/64154
- Update the documentation about how the removal of side effects of
source collection
- Add non-enumerable `requireStack` and `topLevelAwaitLocations`
properties to `ERR_REQUIRE_ASYNC_MODULE`, the latter is only
populated when --experimental-print-required-tla is enabled
- Add "Required module: <url>" to the error message to identify the
required ESM entry point regardless of whether the flag is enabled
- Fix TLA caret column from 0-based to 1-based
- Store module source via a private symbol instead of a public property
- Use `hasAsyncGraph` (post-instantiation) in `throwIfAsyncGraph`
instead of walking the graph before instantiation
- Merge the require stack checking into the
`common.expectRequiredTLAError` helper.
- Removed tests that are made redundant by the snapshot tests
Signed-off-by: Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/64260
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Port the domain module from createHook (async_hooks) to
AsyncLocalStorage using the AsyncContextFrame-based implementation.
Key changes:
- Use AsyncLocalStorage for domain context propagation instead of
async_hooks.createHook()
- Lazy initialization that triggers AsyncContextFrame prototype swap
on first domain use
- Use enterWith instead of ALS.run() so domain context is NOT
automatically restored on exception - this matches the original
domain.run() behavior where exit() only runs on success
- Add ERR_ASYNC_RESOURCE_DOMAIN_REMOVED error for AsyncResource.domain
- Update DEP0097 to End-of-Life status
- Remove tests that relied on the removed MakeCallback domain property
The domain module now uses the AsyncContextFrame version of
AsyncLocalStorage directly for proper context propagation across
async boundaries.
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/61095
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Map BoringSSL's native renegotiation failure to
ERR_TLS_RENEGOTIATION_UNSUPPORTED when TLSSocket#renegotiate() is
called. This avoids exposing an implementation-specific OpenSSL error
when the TLS backend does not support caller-initiated renegotiation.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/63161
Reviewed-By: Tim Perry <pimterry@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
WebAssembly is disabled when Node.js is run with --jitless. The
internal TypeScript stripper relies on WebAssembly, and previously
failed obscurely with a ReferenceError in this mode.
This commit adds an explicit check for WebAssembly support when
loading the TypeScript parser. If WebAssembly is unavailable,
it now throws a descriptive ERR_WEBASSEMBLY_NOT_SUPPORTED error.
Fixes: https://github.com/nodejs/node/issues/61353
PR-URL: https://github.com/nodejs/node/pull/61382
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
This has been released to all active LTS release lines and many
popular packages have been shipping ESM-only distributions based
on the existence of this feature on active LTS. The current documented
behaviors are unlikely to change in a breaking manner within a major
release - as everything with the module loaders, there can always be
subtle changes to undocumented surface that can break edge cases that
assume undocumented behaviors, due to Hyrum's law; but the stability
of this feature is now not different than any other stable parts of
the module loader. It's time to mark it as stable.
PR-URL: https://github.com/nodejs/node/pull/60959
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Jordan Harband <ljharb@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
ESM resolution and loading is now always synchronous from a
non-loader-hook thread. If no asynchrnous loader hooks are
registered, the resolution/loading is entirely synchronous.
If asynchronous loader hooks are registered, these would be
synchronous on the non-loader-hook thread, and asynchronous
on the loader hook thread.
This avoids several races caused by async/sync loading sharing
the same cache. In particular, asynchronous loader hooks
now works with `require(esm)` - previously it tends to break
due to races.
In addition, when an asynchronous loader hook
returns a promise that never settles, the main thread no longer
silently exits with exit code 13, leaving the code below
any module loading calls silently ignored without being executed.
Instead, it now throws ERR_ASYNC_LOADER_REQUEST_NEVER_SETTLED
which can be caught and handled by the main thread. If the module
request comes from `import()`, the never-settling promise is
now relayed to the result returned by `import()`.
Drive-by: when annotating the error about importing undetectable
named exports from CommonJS, it now no longer reload the source
code of the CommonJS module, and instead reuses format information
cached when the module was loaded for linking.
PR-URL: https://github.com/nodejs/node/pull/60380
Fixes: https://github.com/nodejs/node/issues/59666
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Filip Skokan