Necessarily semver-major.
When `--permission` is on, every env var not matched by
`--allow-env` is removed at startup. It takes names,
prefix patterns (`PREFIX_*`), or `*`, repeatable or
comma-sep'd.
There are a range of env vars that Node.js itself uses,
and a default range that are generally known to be safe
in common usage. These are never scrubbed. These include
things like `NODE_OPTIONS`, `NODE_EXTRA_CA_CERTS`, `PATH`,
`HOME`, etc. `NODE_ENV` is not in the defaults and must
be allowed explicitly.
Proxy vars (`HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY`) are
also not in the defaults since they can carry credentials.
When `--use-env-proxy` or `NODE_USE_ENV_PROXY` is set and
any of them were removed, a single warning naming them is
emitted.
Env vars can be dropped at runtime after reading using
`permission.drop()`. This is a stronger protection than
using `process.env.FOO = undefined` because it will
scrub the env var also from the environment block.
On Linux, the removed entries are overwritten in the
initial environment block. fs reads of any other
process's /proc/<pid>/environ, ancestors included, are
denied regardless of `--allow-fs-read`. A process's own
is readable only with `--allow-env=*`. Symlinks are
resolved before the check so paths like
/dev/fd/../../<ppid>/environ are caught. The check only
canonicalizes paths that statfs() reports are on procfs.
On Windows, removal also clears the C runtime's copy
of the environ using _wputenv_s.
Reading a removed name returns undefined, warns once per
name, and publishes to a diagnostics channel.
Env file keys are allowed. If the user had reason to pass
in an env file the assumption is they meant to allow them.
File-source config (node.config.json and NODE_OPTIONS
from a .env file) can only narrow the allow list.
Embedders must call ScrubProcessEnvironment() themselves
on startup. This is left up to the embedder to determine
the exact timing but needs to be called before startup
actually happens.
Child processes are started with `--allow-env=*`. Those
either receive the explicit env they were started with
or only the env they inherit from the parent. Since the
parent process is scrubbed, and the child cannot read
any other process's /proc/<pid>/environ, it should never
see more than the parent can.
Main part of the impl was done by hand. Docs, tests,
verification pass, and cleanup nits were automated.
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
PR-URL: https://github.com/nodejs/node/pull/66132
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Requiring `node:trace_events` and calling `createTracing()` aborted
the process, and `getEnabledCategories()` dereferenced a null pointer,
when Node.js runs on an embedder's own platform
(`kNoInitializeNodeV8Platform`): no `tracing::Agent` exists then, and
`lib/trace_events.js` only checked the compile-time `hasTracing` flag
and `ownsProcessState` before handing the null agent to the binding.
Have the binding report whether an agent exists and throw the existing
`ERR_TRACE_EVENTS_UNAVAILABLE` when it does not, as for a
`--without-v8-platform` build.
Refs: https://github.com/nodejs/node/pull/19803
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65954
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
`NewIsolate()` creates every isolate from the snapshot blob the first
isolate in the process used, because V8 shares the read-only heap
between isolates, and did so by keeping a pointer to the first
`CreateParams`. When that blob came from an `EmbedderSnapshotData` the
embedder had since released, e.g. a second
`CommonEnvironmentSetup::CreateFromSnapshot()` after the first setup and
its snapshot were destroyed, V8 deserialized freed memory.
Record the first blob and external references under a mutex instead of
copying the caller's `CreateParams`, and make `~SnapshotData()` leave
that one blob allocated, since its owner can go away before the last
isolate is created. Nothing is copied and `node` itself is unaffected.
embedtest grows an `--embedder-run-twice` switch so the sequence can be
tested.
Refs: https://github.com/nodejs/node/pull/45885
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65779
Refs: https://github.com/nodejs/node/pull/32984
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Environments created from the built-in snapshot get the builtins' code
cache from that snapshot. An embedder that bootstraps an Environment
from scratch (its own isolate and context, no EmbedderSnapshotData) has
no way to provide one: every builtin the bootstrap touches is compiled
from source in every such process, and each of them then serializes a
fresh cache (SaveCodeCache) that only a later worker thread would ever
consume.
Add node::EmbedderBuiltinCodeCache for that case. Its entries pair a
builtin id with a v8::ScriptCompiler::CachedData; Generate(context)
compiles every builtin in a context of the right kind of isolate and
returns them for a build step to embed, and
SetBuiltinCodeCache(isolate_data, cache) attaches them to an
IsolateData, next to where a snapshot's code cache lives, so that every
Environment created from it afterwards starts with them. The setter
runs CachedData::CompatibilityCheck() against the isolate first and
returns the result, leaving the IsolateData untouched for a cache made
with another V8 version, flag set or read-only snapshot. Entries from a
snapshot still merge with it (RefreshCodeCache() now merges instead of
assuming a single call). CreateEnvironment() plus LoadEnvironment() of
an empty script goes from about 38 ms to 13 ms with a cache attached.
ProcessInitializationFlags::kNoHarvestBuiltinCodeCache stops
serializing caches for builtins compiled without one, for embedders
that supply their own or never create workers. The default is
unchanged because worker threads copy the harvested cache.
A cctest generates a cache, attaches it, checks that a new
Environment's bootstrap compiles from it and that a corrupted cache is
refused; embedtest gains --no-harvest-builtin-code-cache for a test
that a worker does or does not find a harvested cache depending on the
flag.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65352
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
`NewIsolate()` deferred `SetIsolateErrorHandlers()` when snapshot data
was passed, and `CreateEnvironment()` later installed the handlers with
default `IsolateSettings` after deserializing the main context. An
embedder's `fatal_error_callback`, `oom_error_callback`,
`should_abort_on_uncaught_exception_callback` and
`prepare_stack_trace_callback` were therefore dropped whenever a
snapshot was used, and the per-isolate message listener was added even
if `MESSAGE_LISTENER_WITH_ERROR_LEVEL` had been cleared. The only way
to keep custom handlers was to call `SetIsolateUpForNode()` again after
`CreateEnvironment()`.
Install all handlers in `NewIsolate()` regardless of snapshot data, as
its documentation already describes, and stop touching isolate
handlers in `CreateEnvironment()`. The deferral dates from the initial
isolate snapshot work; every handler already copes with a missing
`Environment`, since without a snapshot they are installed before any
context exists, and workers have been calling `SetIsolateUpForNode()`
right after a snapshot `NewIsolate()` anyway.
Refs: https://github.com/nodejs/node/pull/27321
Refs: https://github.com/nodejs/node/pull/45888
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65407
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
CommonEnvironmentSetup::CreateForSnapshotting() always created the
SnapshotCreator without a startup blob, so V8 set up its heap from
scratch. That is impossible when the V8 that Node.js is linked against
only carries the deserializer (external startup data, as Chromium builds
it), and it puts the resulting snapshot on a different read-only heap
lineage than isolates the embedder creates from its own blob. Embedders
in that situation cannot use the snapshot support at all today.
Add SnapshotConfig::base_blob, an optional caller-owned v8::StartupData
that is handed to the SnapshotCreator so that V8 deserializes its heap
from that blob and Node.js adds its isolate data and contexts on top,
the way Blink's context snapshot is built. node_mksnapshot accepts
--v8-snapshot-blob=<file> for hosts that build Node.js with such a V8;
Node.js's own build never passes it and is unchanged. Consuming the
result needs no changes.
embedtest can create a plain V8 startup blob and build the embedder
snapshot on top of one, and a test round-trips argv through a snapshot
built that way.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65374
Reviewed-By: James M Snell <jasnell@gmail.com>
Split the tests so that it's easier to isolate and debug issues,
and we can run the test cases in parallel to speed up. Also
add more comments about what they are testing.
PR-URL: https://github.com/nodejs/node/pull/61571
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
As V8 is moving towards built-in CppHeap creation, change the
management so that the automatic CppHeap creation on Node.js's end
is also enforced at Isolate creation time.
1. If embedder uses NewIsolate(), either they use
IsolateSettings::cpp_heap to specify a CppHeap that will be owned
by V8, or if it's not configured, Node.js will create a CppHeap
that will be owned by V8.
2. If the embedder uses SetIsolateUpForNode(),
IsolateSettings::cpp_heap will be ignored (as V8 has deprecated
attaching CppHeap post-isolate-creation). The embedders need to
ensure that the v8::Isolate has a CppHeap attached while it's
still used by Node.js, preferably using v8::CreateParams.
See https://issues.chromium.org/issues/42203693 for details. In
future version of V8, this CppHeap will be created by V8 if not
provided, and we can remove our own "if no CppHeap provided,
create one" code in NewIsolate().
PR-URL: https://github.com/nodejs/node/pull/58070
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
- Add support for --build-snapshot-config which allows passing
snapshot configurations via a JSON configuration file.
- Add support for node::SnapshotConfig in the embedder API
The initial configurable options are:
- "builder" (SnapshotConfig::builder_script_path): path to the
builder script.
- "withoutCodeCache" (SnapshotFlags::kWithoutCodeCache): disable
code cache generation.
PR-URL: https://github.com/nodejs/node/pull/50453
Refs: https://github.com/nodejs/node/issues/42566
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
There were a few bugs in the original test that went unnoticed
because with the bug the test did not actually get run anymore.
This patch fixes the argument computation by accounting filtering
of the arguments, and uses spawnSyncAndExit{WithoutError} in
the test to enable better logging when the test fails.
PR-URL: https://github.com/nodejs/node/pull/49506
Fixes: https://github.com/nodejs/node/issues/49501
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
- Run the embedder entry point directly through
runEmbedderEntryPoint(), instead of going through another
JS -> C++ trip through the function returned by
getEmbedderEntryFunction()
- For --build-snapshot, read the snapshot script code directly in C++
and pass it to SnapshotBuilder::Generate(), this makes the entry point
more explicit instead of hiding it in JS land, and also makes it
possible to invoke SnapshotBuilder::Generate() internally to create
a custom snapshot.
- Previously we used process.execPath for the embedder to create
__filename and __dirname in the snapshot builder script while using
process.argv[1] for --build-snapshot (where it's always set) which
results in inconsistencies. We now require the embedder to also set
args[1] when creating the Environment if they intend to run snapshot
scripts with a context that contains __filename and __dirname, which
would be derived from args[1]. If they prefer not to include
build-time paths in the snapshot, we now provide
node::GetAnonymousMainPath() as an alternative.
PR-URL: https://github.com/nodejs/node/pull/48242
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Since SEA is very similar in principle to embedding functionality,
it makes sense to share code paths where possible. This commit does
so and addresses a `TODO` while doing so.
It also adds a utility to directly run CJS code to the embedder
startup callback, which comes in handy for this purpose.
Finally, this commit is breaking because it aligns the behavior
of `require()`ing internal modules; previously, embedders could
use the `require` function that they received to do so.
(If this is not considered breaking because accessing internals
is not covered by the API, then this would need ABI compatibility
patches for becoming fully non-breaking.)
PR-URL: https://github.com/nodejs/node/pull/46825
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Add experimental support for loading snapshots in the embedder API
by adding a public opaque wrapper for our `SnapshotData` struct and
allowing embedders to pass it to the relevant setup functions.
Where applicable, use these helpers to deduplicate existing code
in Node.js’s startup path.
This has shown a 40 % startup performance increase for a real-world
application, even with the somewhat limited current support for
built-in modules.
The documentation includes a note about no guarantees for API or
ABI stability for this feature while it is experimental.
PR-URL: https://github.com/nodejs/node/pull/45888
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
So far, process initialization has been a bit all over the place
in Node.js. `InitializeNodeWithArgs()` is our main public API
for this, but inclusion of items in it vs. `InitializeOncePerProcess()`
and `PlatformInit()` has been random at best. Likewise,
some pieces of initialization have been guarded by
`NODE_SHARED_MODE`, but also fairly randomly and without
any meaningful connection to shared library usage.
This leaves embedders in a position to cherry-pick some of
the initialization code into their own code to make their
application behave like typical Node.js applications to the
degree to which they desire it.
Electron takes an alternative route and makes direct use of
`InitializeOncePerProcess()` already while it is a private
API, with a `TODO` to add it to the public API in Node.js.
This commit addresses that `TODO`, and `TODO`s around the
`NODE_SHARED_MODE` usage. Specifically:
- `InitializeOncePerProcess()` and `TearDownOncePerProcess()`
are added to the public API.
- The `flags` option of these functions are merged with the
`flags` option for `InitializeNodeWithArgs()`, since they
essentially share the same semantics.
- The return value of the function is made an abstract class,
rather than a struct, for easier API/ABI stability.
- Initialization code from `main()` is brought into these
functions (since that makes sense in general).
- Add a `TODO` for turning `InitializeNodeWithArgs()` into
a small wrapper around `InitializeOncePerProcess()` and
eventually removing it (at least one major release cycle
each, presumably).
- Remove `NODE_SHARED_MODE` guards and replace them with
runtime options.
PR-URL: https://github.com/nodejs/node/pull/44121
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Provide helpers for a) spinning the event loop and
b) setting up and tearing down the objects involved in a single
Node.js instance, as they would typically be used.
The former helper is also usable inside Node.js itself,
for both Worker and main threads.
PR-URL: https://github.com/nodejs/node/pull/35597
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
libuv 1.39.0 will begin requiring uv_setup_args() to be called
before attempting to access the process title. This commit adds
uv_setup_args() calls that were missing in order for the test
suite to pass (and updates the documentation).
PR-URL: https://github.com/nodejs/node/pull/34751
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
James M Snell