On a mounted VFS, `readlink()`, `realpath()` and `mkdtemp()` only
returned a Buffer for `{ encoding: 'buffer' }`. The string form
`'buffer'`, any other encoding, and a Buffer `mkdtemp()` prefix were
ignored. The `VirtualFileSystem` `readlink` and `realpath` methods
ignored the encoding with `MemoryProvider` and passed it on to
`RealFSProvider`, which threw `ERR_INVALID_ARG_TYPE` for `'buffer'`,
while for other encodings `realpath()` failed and `readlink()`
returned the host path of absolute targets.
Get these paths from the provider as strings and apply the encoding
to the result, normalizing the options with `getOptions()` and
converting with `encodeRealpathResult()` as `fs` does. The latter
moves to `internal/fs/utils` so the VFS can use it.
Signed-off-by: Christian Aurich Zanettini Martins <christian.aurichzm@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/66352
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
`isInt32()` accepts -0 because `-0 === (-0 | 0)`, but V8 does not
represent -0 as an Int32 value, so `Value::IsInt32()` rejects it. The
utf8 fast paths of `readFileSync()` and `writeFileSync()` hand the value
straight to the binding, which then took it for a path and aborted on
the null check.
Coerce -0 to 0 before the call, matching `getValidatedFd()` and the rest
of fs, where -0 is a valid way to name file descriptor 0.
Signed-off-by: Christian Aurich <christian.aurichzm@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65888
Fixes: https://github.com/nodejs/node/issues/65886
Reviewed-By: Xuguang Mei <meixuguang@gmail.com>
openAsBlob() and openAsBlobSync() read the MIME type as
options.type || '' before validating it, causing falsy non-string
values to be replaced with the default before reaching validateString().
As a result, { type: 0 }, { type: false }, { type: null }, and
{ type: NaN } are accepted as an empty type, while { type: 1 }
throws ERR_INVALID_ARG_TYPE.
Default the option only when it is undefined so all other values are
validated against the documented string type.
Signed-off-by: Christian Aurich Zanettini Martins <christian.aurichzm@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/66125
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Several `node:fs` entry points behave differently for a mounted path
than for a real one, because of how the call reaches the VFS hooks.
Make them behave as they do for a real path:
* Add the `watchFile`, `unwatchFile` and `promisesWatch` handlers,
backed by the provider's stat watcher and async watcher; those calls
threw a TypeError before. Have `watch` refuse a path that does not
exist with ENOENT instead of handing back a watcher that polls
forever and keeps the process alive.
* Convert timestamps and validate arguments before the hook runs in
`utimes`, `lutimes` and `readdir` (sync, callback and promise forms),
so a mounted path gets the same ERR_INVALID_ARG_* errors and the same
seconds-since-epoch numbers as a real one.
* Pass the mode and times through to the `fchmod` and `futimes` hooks
and route them to the handle's entry, so descriptor operations take
effect like their path forms instead of being no-ops; the memory
handle validates the way a FileHandle would since one calls it
directly.
* Treat a `mkdtemp` prefix as text rather than a path when it ends in a
separator, so the directory is created inside the intended parent.
* Map the first directory a recursive `mkdir` created back under the
mount point instead of returning the provider-relative path.
* Make disposing an already closed virtual `Dir` a no-op, as on the
native `Dir`, instead of rejecting with ERR_DIR_CLOSED.
test-vfs-fs-hook-gaps adds a test per gap, stating the real-fs outcome
as the expectation. The existing file handle test asserted that
`chmod()` and `utimes()` without arguments were no-ops; they now
validate and apply, so it exercises that instead.
Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
PR-URL: https://github.com/nodejs/node/pull/65852
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
--permission-audit is documented to perform permission checks without
denying access, publishing each violation through the diagnostics
channel while execution continues. Three code paths still enforced:
* fs.lstat() and fs.symlink() are checked in the JavaScript layer,
which threw ERR_ACCESS_DENIED without consulting audit mode. The
isAuditMode() helper introduced for ffi was never wired into fs.
* Addon loading was disabled upfront in Environment setup, so
process.dlopen() failed with ERR_DLOPEN_DISABLED before reaching
the audit-aware permission check, and no audit event was published.
Skip the throw in the JavaScript checks when audit mode is on, keeping
the permission.has() call first so the diagnostics channel event is
still published. Leave native addons enabled in audit mode while still
applying the kAddon scope, so DLOpen() publishes the denial through
the channel and continues to the regular loading path. This also
covers the worker path, which inherits kNoNativeAddons from the same
option.
Each fs API carries its own copy of the check in its sync, callback
and promise flavour, so the regression test exercises all six call
sites, asserting the --permission enforce behaviour as a control.
Fixes: https://github.com/nodejs/node/issues/65419
Refs: https://github.com/nodejs/node/pull/63047
Assisted-by: claude:opus-5
Signed-off-by: Ji Hoon Kang <ivory.ma9ic@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65659
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65489
Refs: https://github.com/nodejs/node/pull/65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
readdir({ recursive: true }) asked the binding for names only and then
called internalModuleStat() on every entry to find the directories to
descend into; with withFileTypes it built the Dirents and still stat()ed
every entry that was not already a directory. Both variants also ran
path.join() and path.relative() per entry to build the relative result.
Ask the binding for file types in all cases, descend into directories
directly, and only stat() symbolic links and entries of unknown type
(which is what could point to a directory). The relative name is the
parent's prefix plus the entry name. Results, their order and the
symlink-following behavior are unchanged for fs.readdirSync, fs.readdir
and fs.promises.readdir.
The known_issues test for Buffer paths (#58892) called back without
checking the error; the error now reaches the callback instead of being
thrown from the completion handler, so the test asserts success to keep
expressing the issue.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65487
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Contexts that are not deserialized from the built-in snapshot -- worker
threads, and the main context of embedders that create their own
isolate or of `node --no-node-snapshot` -- compile (with the code cache
at best) every builtin the bootstrap touches, so each eagerly required
builtin is startup time (~0.15-0.4 ms apiece). A number of them are only
required eagerly so that they end up in the snapshot, or for features
the bootstrap path never uses.
Load lazily what those paths do not need:
- is_main_thread.js: preload util, url, the ESM loader (translators,
resolver, module_job/map, source maps, node:module, vm modules, mime,
data_url, the TypeScript stripper), internal/blob and
internal/dns/utils only while building a snapshot; they load on first
use otherwise.
- fs: internal/blob (+ internal/encoding and its tables) is only used
by fs.openAsBlob().
- internal/url: internal/data_url (+ internal/mime) is only used by the
Buffer-returning file URL helpers.
- internal/process/execution, the CommonJS loader, esm/translators and
esm/load: the TypeScript stripper and data: URL helpers are only
needed for TypeScript sources / data: URLs.
- pre_execution: internal/dns/utils (+ internal/net) is only needed up
front to validate an explicit --dns-result-order or to register the
resolver's snapshot serializer; the default order becomes the
variable's initializer.
- internal/worker: event_loop_utilization and error_serdes are only
needed once a sub-worker's ELU is read or it reports an error.
- worker_threads: `locks` is defined lazily, like util's lazy exports.
Main-thread startup with the snapshot is unchanged (the same modules
are preloaded into it; the bootstrap-modules test lists are adjusted).
A bare worker compiles 95 -> 83 builtins (cold start -5%); without the
snapshot an empty CommonJS entry point compiles 76 -> 59 builtins and an
empty ES module entry point 76 -> 69.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65329
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Aviv Keller <me@aviv.sh>
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
While walking a path, `realpathSync` skips the components it already
knows are real, and in that branch it reads the shared stat buffer to
decide whether the walk has reached a pipe or a socket. That buffer
holds the result of the last stat made anywhere in the process rather
than the last one made by the walk, so an unrelated stat of a FIFO ends
the walk early and the path comes back with its symlinks unresolved. The
unresolved path is then written to the cache, so every later resolution
repeats it.
The walk only takes that branch once the ancestors are established as
real, which is the state the module loader's cache is in. The test goes
through `require()` to reach it, where the stale read costs a second
copy of a module reached through a symlink.
Signed-off-by: Hendrik Liebau <mail@hendrik-liebau.de>
PR-URL: https://github.com/nodejs/node/pull/65113
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Add mount/unmount lifecycle on `VirtualFileSystem`, a handler registry
that fs.js and fs/promises.js consult via `vfsState.handlers`, and a
router that maps absolute paths to the VFS that owns them. When a VFS
is mounted, the public `fs.*` and `fs/promises` APIs (including
streams, `fs.watch`, and `opendir`) dispatch to the provider for paths
under the mount point, and fall through to the real filesystem
otherwise. Includes per-method dispatch tests, error-path coverage,
multi-mount routing tests, and router unit tests.
Ref: https://github.com/nodejs/node/pull/63115
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/63537
Refs: https://github.com/nodejs/node/pull/63115
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
As a first step to porting portions of the pino structured
logger into the runtime, this commit ports the SonicBoom
module to the fs module as Utf8Stream.
This is a faithful port of the SonicBoom module with some
modern updates, such as converting to a Class and using
Symbol.dispose. The bulk of the implementation is unchanged
from the original.
PR-URL: https://github.com/nodejs/node/pull/58897
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Christian Aurich Zanettini Martins