Nodejs/node
1
0
Fork 0
mirror of https://github.com/nodejs/node.git synced 2024-11-21 10:59:27 +00:00
Code Issues Actions 25 Packages Projects Releases Wiki Activity

vm: support using the default loader to handle dynamic import()

Browse Source 
This patch adds support for using
`vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER` as
`importModuleDynamically` in all APIs that take the option
except `vm.SourceTextModule`. This allows users to have a shortcut
to support dynamic import() in the compiled code without missing
the compilation cache if they don't need customization of the
loading process. We emit an experimental warning when the
`import()` is actually handled by the default loader through
this option instead of requiring `--experimental-vm-modules`.

In addition this refactors the documentation for
`importModuleDynamically` and adds a dedicated section for it
with examples.

`vm.SourceTextModule` is not supported in this patch because
it needs additional refactoring to handle `initializeImportMeta`,
which can be done in a follow-up.

PR-URL: https://github.com/nodejs/node/pull/51244
Fixes: https://github.com/nodejs/node/issues/51154
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
This commit is contained in:
Joyee Cheung 2024-02-01 12:45:42 +01:00 committed by GitHub
parent dd4767f99f
commit ad0bcb9c02
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
15 changed files with 593 additions and 192 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

412
doc/api/vm.md
View File

@ -58,6 +58,11 @@ executed in specific contexts.
<!-- YAML
added: v0.3.1
changes:
- version:
- REPLACEME
pr-url: https://github.com/nodejs/node/pull/51244
description: Added support for
`vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER`.
- version:
- v17.0.0
- v16.12.0
@ -94,21 +99,13 @@ changes:
depending on whether code cache data is produced successfully.
This option is **deprecated** in favor of `script.createCachedData()`.
**Default:** `false`.
* `importModuleDynamically` {Function} Called during evaluation of this module
when `import()` is called. If this option is not specified, calls to
`import()` will reject with [`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING`][].
This option is part of the experimental modules API. We do not recommend
using it in a production environment. If `--experimental-vm-modules` isn't
set, this callback will be ignored and calls to `import()` will reject with
[`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING_FLAG`][].
* `specifier` {string} specifier passed to `import()`
* `script` {vm.Script}
* `importAttributes` {Object} The `"with"` value passed to the
[`optionsExpression`][] optional parameter, or an empty object if no value
was provided.
* Returns: {Module Namespace Object|vm.Module} Returning a `vm.Module` is
recommended in order to take advantage of error tracking, and to avoid
issues with namespaces that contain `then` function exports.
* `importModuleDynamically`
{Function|vm.constants.USE\_MAIN\_CONTEXT\_DEFAULT\_LOADER}
Used to specify how the modules should be loaded during the evaluation
of this script when `import()` is called. This option is part of the
experimental modules API. We do not recommend using it in a production
environment. For detailed information, see
[Support of dynamic `import()` in compilation APIs][].
If `options` is a string, then it specifies the filename.
@ -769,20 +766,12 @@ changes:
to initialize the `import.meta`.
* `meta` {import.meta}
* `module` {vm.SourceTextModule}
* `importModuleDynamically` {Function} Called during evaluation of this module
when `import()` is called. If this option is not specified, calls to
`import()` will reject with [`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING`][].
If `--experimental-vm-modules` isn't set, this callback will be ignored
and calls to `import()` will reject with
[`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING_FLAG`][].
* `specifier` {string} specifier passed to `import()`
* `module` {vm.Module}
* `importAttributes` {Object} The `"with"` value passed to the
[`optionsExpression`][] optional parameter, or an empty object if no value
was provided.
* Returns: {Module Namespace Object|vm.Module} Returning a `vm.Module` is
recommended in order to take advantage of error tracking, and to avoid
issues with namespaces that contain `then` function exports.
* `importModuleDynamically` {Function} Used to specify the
how the modules should be loaded during the evaluation of this module
when `import()` is called. This option is part of the experimental
modules API. We do not recommend using it in a production environment.
For detailed information, see
[Support of dynamic `import()` in compilation APIs][].
Creates a new `SourceTextModule` instance.
@ -981,6 +970,11 @@ const vm = require('node:vm');
<!-- YAML
added: v10.10.0
changes:
- version:
- REPLACEME
pr-url: https://github.com/nodejs/node/pull/51244
description: Added support for
`vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER`.
- version:
- v19.6.0
- v18.15.0
@ -1029,32 +1023,54 @@ changes:
* `contextExtensions` {Object\[]} An array containing a collection of context
extensions (objects wrapping the current scope) to be applied while
compiling. **Default:** `[]`.
* `importModuleDynamically` {Function} Called during evaluation of this module
when `import()` is called. If this option is not specified, calls to
`import()` will reject with [`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING`][].
This option is part of the experimental modules API, and should not be
considered stable. If `--experimental-vm-modules` isn't
set, this callback will be ignored and calls to `import()` will reject with
[`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING_FLAG`][].
* `specifier` {string} specifier passed to `import()`
* `function` {Function}
* `importAttributes` {Object} The `"with"` value passed to the
[`optionsExpression`][] optional parameter, or an empty object if no value
was provided.
* Returns: {Module Namespace Object|vm.Module} Returning a `vm.Module` is
recommended in order to take advantage of error tracking, and to avoid
issues with namespaces that contain `then` function exports.
* `importModuleDynamically`
{Function|vm.constants.USE\_MAIN\_CONTEXT\_DEFAULT\_LOADER}
Used to specify the how the modules should be loaded during the evaluation of
this function when `import()` is called. This option is part of the
experimental modules API. We do not recommend using it in a production
environment. For detailed information, see
[Support of dynamic `import()` in compilation APIs][].
* Returns: {Function}
Compiles the given code into the provided context (if no context is
supplied, the current context is used), and returns it wrapped inside a
function with the given `params`.
## `vm.constants`
<!-- YAML
added: REPLACEME
-->
* {Object}
Returns an object containing commonly used constants for VM operations.
### `vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER`
<!-- YAML
added: REPLACEME
-->
> Stability: 1.1 - Active development
A constant that can be used as the `importModuleDynamically` option to
`vm.Script` and `vm.compileFunction()` so that Node.js uses the default
ESM loader from the main context to load the requested module.
For detailed information, see
[Support of dynamic `import()` in compilation APIs][].
## `vm.createContext([contextObject[, options]])`
<!-- YAML
added: v0.3.1
changes:
- version:
- REPLACEME
pr-url: https://github.com/nodejs/node/pull/51244
description: Added support for
`vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER`.
- version:
- v21.2.0
- v20.11.0
@ -1092,21 +1108,13 @@ changes:
scheduled through `Promise`s and `async function`s) will be run immediately
after a script has run through [`script.runInContext()`][].
They are included in the `timeout` and `breakOnSigint` scopes in that case.
* `importModuleDynamically` {Function} Called when `import()` is called in
this context without a referrer script or module. If this option is not
specified, calls to `import()` will reject with
[`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING`][]. If
`--experimental-vm-modules` isn't set, this callback will be ignored and
calls to `import()` will reject with
[`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING_FLAG`][].
* `specifier` {string} specifier passed to `import()`
* `contextObject` {Object} contextified object
* `importAttributes` {Object} The `"with"` value passed to the
[`optionsExpression`][] optional parameter, or an empty object if no value
was provided.
* Returns: {Module Namespace Object|vm.Module} Returning a `vm.Module` is
recommended in order to take advantage of error tracking, and to avoid
issues with namespaces that contain `then` function exports.
* `importModuleDynamically`
{Function|vm.constants.USE\_MAIN\_CONTEXT\_DEFAULT\_LOADER}
Used to specify the how the modules should be loaded when `import()` is
called in this context without a referrer script or module. This option is
part of the experimental modules API. We do not recommend using it in a
production environment. For detailed information, see
[Support of dynamic `import()` in compilation APIs][].
* Returns: {Object} contextified object.
If given a `contextObject`, the `vm.createContext()` method will [prepare
@ -1240,6 +1248,11 @@ vm.measureMemory({ mode: 'detailed', execution: 'eager' })
<!-- YAML
added: v0.3.1
changes:
- version:
- REPLACEME
pr-url: https://github.com/nodejs/node/pull/51244
description: Added support for
`vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER`.
- version:
- v17.0.0
- v16.12.0
@ -1275,22 +1288,13 @@ changes:
* `cachedData` {Buffer|TypedArray|DataView} Provides an optional `Buffer` or
`TypedArray`, or `DataView` with V8's code cache data for the supplied
source.
* `importModuleDynamically` {Function} Called during evaluation of this module
when `import()` is called. If this option is not specified, calls to
`import()` will reject with [`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING`][].
This option is part of the experimental modules API. We do not recommend
using it in a production environment. If `--experimental-vm-modules` isn't
set, this callback will be ignored and calls to `import()` will reject with
[`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING_FLAG`][].
* `specifier` {string} specifier passed to `import()`
* `script` {vm.Script}
* `importAttributes` {Object} The `"with"` value passed to the
[`optionsExpression`][] optional parameter, or an empty object if no value
was provided.
* Returns: {Module Namespace Object|vm.Module} Returning a `vm.Module` is
recommended in order to take advantage of error tracking, and to avoid
issues with namespaces that contain `then` function exports.
* Returns: {any} the result of the very last statement executed in the script.
* `importModuleDynamically`
{Function|vm.constants.USE\_MAIN\_CONTEXT\_DEFAULT\_LOADER}
Used to specify the how the modules should be loaded during the evaluation
of this script when `import()` is called. This option is part of the
experimental modules API. We do not recommend using it in a production
environment. For detailed information, see
[Support of dynamic `import()` in compilation APIs][].
The `vm.runInContext()` method compiles `code`, runs it within the context of
the `contextifiedObject`, then returns the result. Running code does not have
@ -1320,6 +1324,11 @@ console.log(contextObject);
<!-- YAML
added: v0.3.1
changes:
- version:
- REPLACEME
pr-url: https://github.com/nodejs/node/pull/51244
description: Added support for
`vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER`.
- version:
- v17.0.0
- v16.12.0
@ -1376,21 +1385,13 @@ changes:
* `cachedData` {Buffer|TypedArray|DataView} Provides an optional `Buffer` or
`TypedArray`, or `DataView` with V8's code cache data for the supplied
source.
* `importModuleDynamically` {Function} Called during evaluation of this module
when `import()` is called. If this option is not specified, calls to
`import()` will reject with [`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING`][].
This option is part of the experimental modules API. We do not recommend
using it in a production environment. If `--experimental-vm-modules` isn't
set, this callback will be ignored and calls to `import()` will reject with
[`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING_FLAG`][].
* `specifier` {string} specifier passed to `import()`
* `script` {vm.Script}
* `importAttributes` {Object} The `"with"` value passed to the
[`optionsExpression`][] optional parameter, or an empty object if no value
was provided.
* Returns: {Module Namespace Object|vm.Module} Returning a `vm.Module` is
recommended in order to take advantage of error tracking, and to avoid
issues with namespaces that contain `then` function exports.
* `importModuleDynamically`
{Function|vm.constants.USE\_MAIN\_CONTEXT\_DEFAULT\_LOADER}
Used to specify the how the modules should be loaded during the evaluation
of this script when `import()` is called. This option is part of the
experimental modules API. We do not recommend using it in a production
environment. For detailed information, see
[Support of dynamic `import()` in compilation APIs][].
* `microtaskMode` {string} If set to `afterEvaluate`, microtasks (tasks
scheduled through `Promise`s and `async function`s) will be run immediately
after the script has run. They are included in the `timeout` and
@ -1425,6 +1426,11 @@ console.log(contextObject);
<!-- YAML
added: v0.3.1
changes:
- version:
- REPLACEME
pr-url: https://github.com/nodejs/node/pull/51244
description: Added support for
`vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER`.
- version:
- v17.0.0
- v16.12.0
@ -1458,21 +1464,13 @@ changes:
* `cachedData` {Buffer|TypedArray|DataView} Provides an optional `Buffer` or
`TypedArray`, or `DataView` with V8's code cache data for the supplied
source.
* `importModuleDynamically` {Function} Called during evaluation of this module
when `import()` is called. If this option is not specified, calls to
`import()` will reject with [`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING`][].
This option is part of the experimental modules API. We do not recommend
using it in a production environment. If `--experimental-vm-modules` isn't
set, this callback will be ignored and calls to `import()` will reject with
[`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING_FLAG`][].
* `specifier` {string} specifier passed to `import()`
* `script` {vm.Script}
* `importAttributes` {Object} The `"with"` value passed to the
[`optionsExpression`][] optional parameter, or an empty object if no value
was provided.
* Returns: {Module Namespace Object|vm.Module} Returning a `vm.Module` is
recommended in order to take advantage of error tracking, and to avoid
issues with namespaces that contain `then` function exports.
* `importModuleDynamically`
{Function|vm.constants.USE\_MAIN\_CONTEXT\_DEFAULT\_LOADER}
Used to specify the how the modules should be loaded during the evaluation
of this script when `import()` is called. This option is part of the
experimental modules API. We do not recommend using it in a production
environment. For detailed information, see
[Support of dynamic `import()` in compilation APIs][].
* Returns: {any} the result of the very last statement executed in the script.
`vm.runInThisContext()` compiles `code`, runs it within the context of the
@ -1618,6 +1616,209 @@ inside a `vm.Context`, functions passed to them will be added to global queues,
which are shared by all contexts. Therefore, callbacks passed to those functions
are not controllable through the timeout either.
## Support of dynamic `import()` in compilation APIs
The following APIs support an `importModuleDynamically` option to enable dynamic
`import()` in code compiled by the vm module.
* `new vm.Script`
* `vm.compileFunction()`
* `new vm.SourceTextModule`
* `vm.runInThisContext()`
* `vm.runInContext()`
* `vm.runInNewContext()`
* `vm.createContext()`
This option is still part of the experimental modules API. We do not recommend
using it in a production environment.
### When the `importModuleDynamically` option is not specified or undefined
If this option is not specified, or if it's `undefined`, code containing
`import()` can still be compiled by the vm APIs, but when the compiled code is
executed and it actually calls `import()`, the result will reject with
[`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING`][].
### When `importModuleDynamically` is `vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER`
This option is currently not supported for `vm.SourceTextModule`.
With this option, when an `import()` is initiated in the compiled code, Node.js
would use the default ESM loader from the main context to load the requested
module and return it to the code being executed.
This gives access to Node.js built-in modules such as `fs` or `http`
to the code being compiled. If the code is executed in a different context,
be aware that the objects created by modules loaded from the main context
are still from the main context and not `instanceof` built-in classes in the
new context.
```cjs
const { Script, constants } = require('node:vm');
const script = new Script(
'import("node:fs").then(({readFile}) => readFile instanceof Function)',
{ importModuleDynamically: constants.USE_MAIN_CONTEXT_DEFAULT_LOADER });
// false: URL loaded from the main context is not an instance of the Function
// class in the new context.
script.runInNewContext().then(console.log);
```
```mjs
import { Script, constants } from 'node:vm';
const script = new Script(
'import("node:fs").then(({readFile}) => readFile instanceof Function)',
{ importModuleDynamically: constants.USE_MAIN_CONTEXT_DEFAULT_LOADER });
// false: URL loaded from the main context is not an instance of the Function
// class in the new context.
script.runInNewContext().then(console.log);
```
This option also allows the script or function to load user modules:
```mjs
import { Script, constants } from 'node:vm';
import { resolve } from 'node:path';
import { writeFileSync } from 'node:fs';
// Write test.js and test.txt to the directory where the current script
// being run is located.
writeFileSync(resolve(import.meta.dirname, 'test.mjs'),
'export const filename = "./test.json";');
writeFileSync(resolve(import.meta.dirname, 'test.json'),
'{"hello": "world"}');
// Compile a script that loads test.mjs and then test.json
// as if the script is placed in the same directory.
const script = new Script(
`(async function() {
const { filename } = await import('./test.mjs');
return import(filename, { with: { type: 'json' } })
})();`,
{
filename: resolve(import.meta.dirname, 'test-with-default.js'),
importModuleDynamically: constants.USE_MAIN_CONTEXT_DEFAULT_LOADER,
});
// { default: { hello: 'world' } }
script.runInThisContext().then(console.log);
```
```cjs
const { Script, constants } = require('node:vm');
const { resolve } = require('node:path');
const { writeFileSync } = require('node:fs');
// Write test.js and test.txt to the directory where the current script
// being run is located.
writeFileSync(resolve(__dirname, 'test.mjs'),
'export const filename = "./test.json";');
writeFileSync(resolve(__dirname, 'test.json'),
'{"hello": "world"}');
// Compile a script that loads test.mjs and then test.json
// as if the script is placed in the same directory.
const script = new Script(
`(async function() {
const { filename } = await import('./test.mjs');
return import(filename, { with: { type: 'json' } })
})();`,
{
filename: resolve(__dirname, 'test-with-default.js'),
importModuleDynamically: constants.USE_MAIN_CONTEXT_DEFAULT_LOADER,
});
// { default: { hello: 'world' } }
script.runInThisContext().then(console.log);
```
There are a few caveats with loading user modules using the default loader
from the main context:
1. The module being resolved would be relative to the `filename` option passed
to `vm.Script` or `vm.compileFunction()`. The resolution can work with a
`filename` that's either an absolute path or a URL string. If `filename` is
a string that's neither an absolute path or a URL, or if it's undefined,
the resolution will be relative to the current working directory
of the process. In the case of `vm.createContext()`, the resolution is always
relative to the current working directory since this option is only used when
there isn't a referrer script or module.
2. For any given `filename` that resolves to a specific path, once the process
manages to load a particular module from that path, the result may be cached,
and subsequent load of the same module from the same path would return the
same thing. If the `filename` is a URL string, the cache would not be hit
if it has different search parameters. For `filename`s that are not URL
strings, there is currently no way to bypass the caching behavior.
### When `importModuleDynamically` is a function
When `importModuleDynamically` is a function, it will be invoked when `import()`
is called in the compiled code for users to customize how the requested module
should be compiled and evaluated. Currently, the Node.js instance must be
launched with the `--experimental-vm-modules` flag for this option to work. If
the flag isn't set, this callback will be ignored. If the code evaluated
actually calls to `import()`, the result will reject with
[`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING_FLAG`][].
The callback `importModuleDynamically(specifier, referrer, importAttributes)`
has the following signature:
* `specifier` {string} specifier passed to `import()`
* `referrer` {vm.Script|Function|vm.SourceTextModule|Object}
The referrer is the compiled `vm.Script` for `new vm.Script`,
`vm.runInThisContext`, `vm.runInContext` and `vm.runInNewContext`. It's the
compiled `Function` for `vm.compileFunction`, the compiled
`vm.SourceTextModule` for `new vm.SourceTextModule`, and the context `Object`
for `vm.createContext()`.
* `importAttributes` {Object} The `"with"` value passed to the
[`optionsExpression`][] optional parameter, or an empty object if no value was
provided.
* Returns: {Module Namespace Object|vm.Module} Returning a `vm.Module` is
recommended in order to take advantage of error tracking, and to avoid issues
with namespaces that contain `then` function exports.
```mjs
// This script must be run with --experimental-vm-modules.
import { Script, SyntheticModule } from 'node:vm';
const script = new Script('import("foo.json", { with: { type: "json" } })', {
async importModuleDynamically(specifier, referrer, importAttributes) {
console.log(specifier); // 'foo.json'
console.log(referrer); // The compiled script
console.log(importAttributes); // { type: 'json' }
const m = new SyntheticModule(['bar'], () => { });
await m.link(() => { });
m.setExport('bar', { hello: 'world' });
return m;
},
});
const result = await script.runInThisContext();
console.log(result); // { bar: { hello: 'world' } }
```
```cjs
// This script must be run with --experimental-vm-modules.
const { Script, SyntheticModule } = require('node:vm');
(async function main() {
const script = new Script('import("foo.json", { with: { type: "json" } })', {
async importModuleDynamically(specifier, referrer, importAttributes) {
console.log(specifier); // 'foo.json'
console.log(referrer); // The compiled script
console.log(importAttributes); // { type: 'json' }
const m = new SyntheticModule(['bar'], () => { });
await m.link(() => { });
m.setExport('bar', { hello: 'world' });
return m;
},
});
const result = await script.runInThisContext();
console.log(result); // { bar: { hello: 'world' } }
})();
```
[Cyclic Module Record]: https://tc39.es/ecma262/#sec-cyclic-module-records
[ECMAScript Module Loader]: esm.md#modules-ecmascript-modules
[Evaluate() concrete method]: https://tc39.es/ecma262/#sec-moduleevaluation
@ -1626,6 +1827,7 @@ are not controllable through the timeout either.
[Link() concrete method]: https://tc39.es/ecma262/#sec-moduledeclarationlinking
[Module Record]: https://262.ecma-international.org/14.0/#sec-abstract-module-records
[Source Text Module Record]: https://tc39.es/ecma262/#sec-source-text-module-records
[Support of dynamic `import()` in compilation APIs]: #support-of-dynamic-import-in-compilation-apis
[Synthetic Module Record]: https://heycam.github.io/webidl/#synthetic-module-records
[V8 Embedder's Guide]: https://v8.dev/docs/embed#contexts
[`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING_FLAG`]: errors.md#err_vm_dynamic_import_callback_missing_flag

1
lib/internal/bootstrap/switches/is_main_thread.js
View File

@ -294,7 +294,6 @@ require('url'); // eslint-disable-line no-restricted-modules
internalBinding('module_wrap');
require('internal/modules/cjs/loader');
require('internal/modules/esm/utils');
require('internal/vm/module');
// Needed to refresh the time origin.
require('internal/perf/utils');

18
lib/internal/modules/cjs/loader.js
View File

@ -52,7 +52,6 @@ const {
SafeMap,
SafeWeakMap,
String,
Symbol,
StringPrototypeCharAt,
StringPrototypeCharCodeAt,
StringPrototypeEndsWith,
@ -114,7 +113,6 @@ const {
initializeCjsConditions,
loadBuiltinModule,
makeRequireFunction,
normalizeReferrerURL,
stripBOM,
toRealPath,
} = require('internal/modules/helpers');
@ -125,12 +123,10 @@ const policy = getLazy(
);
const shouldReportRequiredModules = getLazy(() => process.env.WATCH_REPORT_DEPENDENCIES);
const getCascadedLoader = getLazy(
() => require('internal/process/esm_loader').esmLoader,
);
const permission = require('internal/process/permission');
const {
vm_dynamic_import_default_internal,
} = internalBinding('symbols');
// Whether any user-provided CJS modules had been loaded (executed).
// Used for internal assertions.
let hasLoadedAnyUserCJSModule = false;
@ -1257,12 +1253,8 @@ let hasPausedEntry = false;
* @param {object} codeCache The SEA code cache
*/
function wrapSafe(filename, content, cjsModuleInstance, codeCache) {
const hostDefinedOptionId = Symbol(`cjs:${filename}`);
async function importModuleDynamically(specifier, _, importAttributes) {
const cascadedLoader = getCascadedLoader();
return cascadedLoader.import(specifier, normalizeReferrerURL(filename),
importAttributes);
}
const hostDefinedOptionId = vm_dynamic_import_default_internal;
const importModuleDynamically = vm_dynamic_import_default_internal;
if (patched) {
const wrapped = Module.wrap(content);
const script = makeContextifyScript(

14
lib/internal/modules/esm/translators.js
View File

@ -15,7 +15,6 @@ const {
StringPrototypeReplaceAll,
StringPrototypeSlice,
StringPrototypeStartsWith,
Symbol,
SyntaxErrorPrototype,
globalThis: { WebAssembly },
} = primordials;
@ -59,7 +58,9 @@ const { ModuleWrap } = moduleWrap;
const asyncESM = require('internal/process/esm_loader');
const { emitWarningSync } = require('internal/process/warning');
const { internalCompileFunction } = require('internal/vm');
const {
vm_dynamic_import_default_internal,
} = internalBinding('symbols');
// Lazy-loading to avoid circular dependencies.
let getSourceSync;
/**
@ -206,9 +207,8 @@ function enrichCJSError(err, content, filename) {
*/
function loadCJSModule(module, source, url, filename) {
let compiledWrapper;
async function importModuleDynamically(specifier, _, importAttributes) {
return asyncESM.esmLoader.import(specifier, url, importAttributes);
}
const hostDefinedOptionId = vm_dynamic_import_default_internal;
const importModuleDynamically = vm_dynamic_import_default_internal;
try {
compiledWrapper = internalCompileFunction(
source, // code,
@ -226,8 +226,8 @@ function loadCJSModule(module, source, url, filename) {
'__filename',
'__dirname',
],
Symbol(`cjs:${filename}`), // hostDefinedOptionsId
importModuleDynamically, // importModuleDynamically
hostDefinedOptionId, // hostDefinedOptionsId
importModuleDynamically, // importModuleDynamically
).function;
} catch (err) {
enrichCJSError(err, source, filename);

71
lib/internal/modules/esm/utils.js
View File

@ -4,7 +4,6 @@ const {
ArrayIsArray,
SafeSet,
SafeWeakMap,
Symbol,
ObjectFreeze,
} = primordials;
@ -14,8 +13,10 @@ const {
},
} = internalBinding('util');
const {
default_host_defined_options,
vm_dynamic_import_default_internal,
vm_dynamic_import_main_context_default,
vm_dynamic_import_missing_flag,
vm_dynamic_import_no_callback,
} = internalBinding('symbols');
const {
@ -28,12 +29,19 @@ const {
loadPreloadModules,
initializeFrozenIntrinsics,
} = require('internal/process/pre_execution');
const { getCWDURL } = require('internal/util');
const {
emitExperimentalWarning,
getCWDURL,
getLazy,
} = require('internal/util');
const {
setImportModuleDynamicallyCallback,
setInitializeImportMetaObjectCallback,
} = internalBinding('module_wrap');
const assert = require('internal/assert');
const {
normalizeReferrerURL,
} = require('internal/modules/helpers');
let defaultConditions;
/**
@ -145,8 +153,10 @@ const moduleRegistries = new SafeWeakMap();
*/
function registerModule(referrer, registry) {
const idSymbol = referrer[host_defined_option_symbol];
if (idSymbol === default_host_defined_options ||
idSymbol === vm_dynamic_import_missing_flag) {
if (idSymbol === vm_dynamic_import_no_callback ||
idSymbol === vm_dynamic_import_missing_flag ||
idSymbol === vm_dynamic_import_main_context_default ||
idSymbol === vm_dynamic_import_default_internal) {
// The referrer is compiled without custom callbacks, so there is
// no registry to hold on to. We'll throw
// ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING when a callback is
@ -158,26 +168,6 @@ function registerModule(referrer, registry) {
moduleRegistries.set(idSymbol, registry);
}
/**
* Registers the ModuleRegistry for dynamic import() calls with a realm
* as the referrer. Similar to {@link registerModule}, but this function
* generates a new id symbol instead of using the one from the referrer
* object.
* @param {globalThis} globalThis The globalThis object of the realm.
* @param {ModuleRegistry} registry
*/
function registerRealm(globalThis, registry) {
let idSymbol = globalThis[host_defined_option_symbol];
// If the per-realm host-defined options is already registered, do nothing.
if (idSymbol) {
return;
}
// Otherwise, register the per-realm host-defined options.
idSymbol = Symbol('Realm globalThis');
globalThis[host_defined_option_symbol] = idSymbol;
moduleRegistries.set(idSymbol, registry);
}
/**
* Defines the `import.meta` object for a given module.
* @param {symbol} symbol - Reference to the module.
@ -191,16 +181,44 @@ function initializeImportMetaObject(symbol, meta) {
}
}
}
const getCascadedLoader = getLazy(
() => require('internal/process/esm_loader').esmLoader,
);
/**
* Proxy the dynamic import to the default loader.
* @param {string} specifier - The module specifier string.
* @param {Record<string, string>} attributes - The import attributes object.
* @param {string|null|undefined} referrerName - name of the referrer.
* @returns {Promise<import('internal/modules/esm/loader.js').ModuleExports>} - The imported module object.
*/
function defaultImportModuleDynamically(specifier, attributes, referrerName) {
const parentURL = normalizeReferrerURL(referrerName);
return getCascadedLoader().import(specifier, parentURL, attributes);
}
/**
* Asynchronously imports a module dynamically using a callback function. The native callback.
* @param {symbol} referrerSymbol - Referrer symbol of the registered script, function, module, or contextified object.
* @param {string} specifier - The module specifier string.
* @param {Record<string, string>} attributes - The import attributes object.
* @param {string|null|undefined} referrerName - name of the referrer.
* @returns {Promise<import('internal/modules/esm/loader.js').ModuleExports>} - The imported module object.
* @throws {ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING} - If the callback function is missing.
*/
async function importModuleDynamicallyCallback(referrerSymbol, specifier, attributes) {
async function importModuleDynamicallyCallback(referrerSymbol, specifier, attributes, referrerName) {
// For user-provided vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER, emit the warning
// and fall back to the default loader.
if (referrerSymbol === vm_dynamic_import_main_context_default) {
emitExperimentalWarning('vm.USE_MAIN_CONTEXT_DEFAULT_LOADER');
return defaultImportModuleDynamically(specifier, attributes, referrerName);
}
// For script compiled internally that should use the default loader to handle dynamic
// import, proxy the request to the default loader without the warning.
if (referrerSymbol === vm_dynamic_import_default_internal) {
return defaultImportModuleDynamically(specifier, attributes, referrerName);
}
if (moduleRegistries.has(referrerSymbol)) {
const { importModuleDynamically, callbackReferrer } = moduleRegistries.get(referrerSymbol);
if (importModuleDynamically !== undefined) {
@ -273,7 +291,6 @@ async function initializeHooks() {
module.exports = {
registerModule,
registerRealm,
initializeESM,
initializeHooks,
getDefaultConditions,

35
lib/internal/modules/helpers.js
View File

@ -23,16 +23,19 @@ const { validateString } = require('internal/validators');
const fs = require('fs'); // Import all of `fs` so that it can be monkey-patched.
const internalFS = require('internal/fs/utils');
const path = require('path');
const { pathToFileURL, fileURLToPath, URL } = require('internal/url');
const { pathToFileURL, fileURLToPath } = require('internal/url');
const assert = require('internal/assert');
const { getOptionValue } = require('internal/options');
const { setOwnProperty } = require('internal/util');
const { inspect } = require('internal/util/inspect');
const {
privateSymbols: {
require_private_symbol,
},
} = internalBinding('util');
const { canParse: URLCanParse } = internalBinding('url');
let debug = require('internal/util/debuglog').debuglog('module', (fn) => {
debug = fn;
@ -288,14 +291,32 @@ function addBuiltinLibsToObject(object, dummyModuleName) {
}
/**
* If a referrer is an URL instance or absolute path, convert it into an URL string.
* @param {string | URL} referrer
* Normalize the referrer name as a URL.
* If it's a string containing an absolute path or a URL it's normalized as
* a URL string.
* Otherwise it's returned as undefined.
* @param {string | null | undefined} referrerName
* @returns {string | undefined}
*/
function normalizeReferrerURL(referrer) {
if (typeof referrer === 'string' && path.isAbsolute(referrer)) {
return pathToFileURL(referrer).href;
function normalizeReferrerURL(referrerName) {
if (referrerName === null || referrerName === undefined) {
return undefined;
}
return new URL(referrer).href;
if (typeof referrerName === 'string') {
if (path.isAbsolute(referrerName)) {
return pathToFileURL(referrerName).href;
}
if (StringPrototypeStartsWith(referrerName, 'file://') ||
URLCanParse(referrerName)) {
return referrerName;
}
return undefined;
}
assert.fail('Unreachable code reached by ' + inspect(referrerName));
}
module.exports = {

26
lib/internal/process/pre_execution.js
View File

@ -66,7 +66,6 @@ function prepareWorkerThreadExecution() {
}
function prepareShadowRealmExecution() {
const { registerRealm } = require('internal/modules/esm/utils');
// Patch the process object with legacy properties and normalizations.
// Do not expand argv1 as it is not available in ShadowRealm.
patchProcessObject(false);
@ -74,15 +73,24 @@ function prepareShadowRealmExecution() {
// Disable custom loaders in ShadowRealm.
setupUserModules(true);
registerRealm(globalThis, {
__proto__: null,
importModuleDynamically: (specifier, _referrer, attributes) => {
// The handler for `ShadowRealm.prototype.importValue`.
const { esmLoader } = require('internal/process/esm_loader');
// `parentURL` is not set in the case of a ShadowRealm top-level import.
return esmLoader.import(specifier, undefined, attributes);
const {
privateSymbols: {
host_defined_option_symbol,
},
});
} = internalBinding('util');
const {
vm_dynamic_import_default_internal,
} = internalBinding('symbols');
// For ShadowRealm.prototype.importValue(), the referrer name is
// always null, so the native ImportModuleDynamically() callback would
// always fallback to look up the host-defined option from the
// global object using host_defined_option_symbol. Using
// vm_dynamic_import_default_internal as the host-defined option
// instructs the JS-land importModuleDynamicallyCallback() to
// proxy the request to defaultImportModuleDynamically().
globalThis[host_defined_option_symbol] =
vm_dynamic_import_default_internal;
}
function prepareExecution(options) {

8
lib/internal/source_map/source_map_cache.js
View File

@ -107,12 +107,10 @@ function extractSourceMapURLMagicComment(content) {
function maybeCacheSourceMap(filename, content, cjsModuleInstance, isGeneratedSource, sourceURL, sourceMapURL) {
const sourceMapsEnabled = getSourceMapsEnabled();
if (!(process.env.NODE_V8_COVERAGE || sourceMapsEnabled)) return;
try {
const { normalizeReferrerURL } = require('internal/modules/helpers');
filename = normalizeReferrerURL(filename);
} catch (err) {
const { normalizeReferrerURL } = require('internal/modules/helpers');
filename = normalizeReferrerURL(filename);
if (filename === undefined) {
// This is most likely an invalid filename in sourceURL of [eval]-wrapper.
debug(err);
return;
}

27
lib/internal/vm.js
View File

@ -14,7 +14,9 @@ const {
runInContext,
} = ContextifyScript.prototype;
const {
default_host_defined_options,
vm_dynamic_import_default_internal,
vm_dynamic_import_main_context_default,
vm_dynamic_import_no_callback,
vm_dynamic_import_missing_flag,
} = internalBinding('symbols');
const {
@ -27,7 +29,6 @@ const {
getOptionValue,
} = require('internal/options');
function isContext(object) {
validateObject(object, 'object', kValidateObjectAllowArray);
@ -35,6 +36,11 @@ function isContext(object) {
}
function getHostDefinedOptionId(importModuleDynamically, hint) {
if (importModuleDynamically === vm_dynamic_import_main_context_default ||
importModuleDynamically === vm_dynamic_import_default_internal) {
return importModuleDynamically;
}
if (importModuleDynamically !== undefined) {
// Check that it's either undefined or a function before we pass
// it into the native constructor.
@ -45,7 +51,7 @@ function getHostDefinedOptionId(importModuleDynamically, hint) {
// We need a default host defined options that are the same for all
// scripts not needing custom module callbacks so that the isolate
// compilation cache can be hit.
return default_host_defined_options;
return vm_dynamic_import_no_callback;
}
// We should've thrown here immediately when we introduced
// --experimental-vm-modules and importModuleDynamically, but since
@ -61,6 +67,13 @@ function getHostDefinedOptionId(importModuleDynamically, hint) {
}
function registerImportModuleDynamically(referrer, importModuleDynamically) {
// If it's undefined or certain known symbol, there's no customization so
// no need to register anything.
if (importModuleDynamically === undefined ||
importModuleDynamically === vm_dynamic_import_main_context_default ||
importModuleDynamically === vm_dynamic_import_default_internal) {
return;
}
const { importModuleDynamicallyWrap } = require('internal/vm/module');
const { registerModule } = require('internal/modules/esm/utils');
registerModule(referrer, {
@ -99,9 +112,7 @@ function internalCompileFunction(
result.function.cachedDataRejected = result.cachedDataRejected;
}
if (importModuleDynamically !== undefined) {
registerImportModuleDynamically(result.function, importModuleDynamically);
}
registerImportModuleDynamically(result.function, importModuleDynamically);
return result;
}
@ -132,9 +143,7 @@ function makeContextifyScript(code,
throw e; /* node-do-not-add-exception-line */
}
if (importModuleDynamically !== undefined) {
registerImportModuleDynamically(script, importModuleDynamically);
}
registerImportModuleDynamically(script, importModuleDynamically);
return script;
}

20
lib/vm.js
View File

@ -23,6 +23,7 @@
const {
ArrayPrototypeForEach,
ObjectFreeze,
Symbol,
PromiseReject,
ReflectApply,
@ -61,6 +62,9 @@ const {
isContext,
registerImportModuleDynamically,
} = require('internal/vm');
const {
vm_dynamic_import_main_context_default,
} = internalBinding('symbols');
const kParsingContext = Symbol('script parsing context');
class Script extends ContextifyScript {
@ -108,9 +112,7 @@ class Script extends ContextifyScript {
throw e; /* node-do-not-add-exception-line */
}
if (importModuleDynamically !== undefined) {
registerImportModuleDynamically(this, importModuleDynamically);
}
registerImportModuleDynamically(this, importModuleDynamically);
}
runInThisContext(options) {
@ -245,9 +247,7 @@ function createContext(contextObject = {}, options = kEmptyObject) {
makeContext(contextObject, name, origin, strings, wasm, microtaskQueue, hostDefinedOptionId);
// Register the context scope callback after the context was initialized.
if (importModuleDynamically !== undefined) {
registerImportModuleDynamically(contextObject, importModuleDynamically);
}
registerImportModuleDynamically(contextObject, importModuleDynamically);
return contextObject;
}
@ -378,6 +378,13 @@ function measureMemory(options = kEmptyObject) {
return result;
}
const vmConstants = {
__proto__: null,
USE_MAIN_CONTEXT_DEFAULT_LOADER: vm_dynamic_import_main_context_default,
};
ObjectFreeze(vmConstants);
module.exports = {
Script,
createContext,
@ -388,6 +395,7 @@ module.exports = {
isContext,
compileFunction,
measureMemory,
constants: vmConstants,
};
// The vm module is patched to include vm.Module, vm.SourceTextModule

7
src/env_properties.h
View File

@ -34,7 +34,6 @@
// Symbols are per-isolate primitives but Environment proxies them
// for the sake of convenience.
#define PER_ISOLATE_SYMBOL_PROPERTIES(V) \
V(default_host_defined_options, "default_host_defined_options") \
V(fs_use_promises_symbol, "fs_use_promises_symbol") \
V(async_id_symbol, "async_id_symbol") \
V(handle_onclose_symbol, "handle_onclose") \
@ -48,7 +47,11 @@
V(onpskexchange_symbol, "onpskexchange") \
V(resource_symbol, "resource_symbol") \
V(trigger_async_id_symbol, "trigger_async_id_symbol") \
V(vm_dynamic_import_missing_flag, "vm_dynamic_import_missing_flag")
V(vm_dynamic_import_default_internal, "vm_dynamic_import_default_internal") \
V(vm_dynamic_import_main_context_default, \
"vm_dynamic_import_main_context_default") \
V(vm_dynamic_import_missing_flag, "vm_dynamic_import_missing_flag") \
V(vm_dynamic_import_no_callback, "vm_dynamic_import_no_callback")
// Strings are per-isolate primitives but Environment proxies them
// for the sake of convenience. Strings should be ASCII-only.

1
src/module_wrap.cc
View File

@ -601,6 +601,7 @@ static MaybeLocal<Promise> ImportModuleDynamically(
id,
Local<Value>(specifier),
attributes,
resource_name,
};
Local<Value> result;

142
test/es-module/test-vm-main-context-default-loader.js Normal file
View File

@ -0,0 +1,142 @@
'use strict';
const common = require('../common');
// Can't process.chdir() in worker.
common.skipIfWorker();
const tmpdir = require('../common/tmpdir');
const fixtures = require('../common/fixtures');
const url = require('url');
const fs = require('fs');
const {
compileFunction,
Script,
createContext,
constants: { USE_MAIN_CONTEXT_DEFAULT_LOADER },
} = require('vm');
const assert = require('assert');
common.expectWarning('ExperimentalWarning',
'vm.USE_MAIN_CONTEXT_DEFAULT_LOADER is an experimental feature and might change at any time');
assert(
!process.execArgv.includes('--experimental-vm-modules'),
'This test must be run without --experimental-vm-modules');
assert.strictEqual(typeof USE_MAIN_CONTEXT_DEFAULT_LOADER, 'symbol');
async function testNotFoundErrors(options) {
// Import user modules.
const script = new Script('import("./message.mjs")', options);
// Use try-catch for better async stack traces in the logs.
await assert.rejects(script.runInThisContext(), { code: 'ERR_MODULE_NOT_FOUND' });
const imported = compileFunction('return import("./message.mjs")', [], options)();
// Use try-catch for better async stack traces in the logs.
await assert.rejects(imported, { code: 'ERR_MODULE_NOT_FOUND' });
}
async function testLoader(options) {
{
// Import built-in modules
const script = new Script('import("fs")', options);
let result = await script.runInThisContext();
assert.strictEqual(result.constants.F_OK, fs.constants.F_OK);
const imported = compileFunction('return import("fs")', [], options)();
result = await imported;
assert.strictEqual(result.constants.F_OK, fs.constants.F_OK);
}
const moduleUrl = fixtures.fileURL('es-modules', 'message.mjs');
fs.copyFileSync(moduleUrl, tmpdir.resolve('message.mjs'));
{
const namespace = await import(moduleUrl);
const script = new Script('import("./message.mjs")', options);
const result = await script.runInThisContext();
assert.deepStrictEqual(result, namespace);
}
{
const namespace = await import(moduleUrl);
const imported = compileFunction('return import("./message.mjs")', [], options)();
const result = await imported;
assert.deepStrictEqual(result, namespace);
}
}
async function main() {
{
// Importing with absolute path as filename.
tmpdir.refresh();
const filename = tmpdir.resolve('index.js');
const options = {
filename,
importModuleDynamically: USE_MAIN_CONTEXT_DEFAULT_LOADER
};
await testNotFoundErrors(options);
await testLoader(options);
}
{
// Importing with file:// URL as filename.
tmpdir.refresh();
// We use a search parameter to bypass caching.
const filename = url.pathToFileURL(tmpdir.resolve('index.js')).href + '?t=1';
const options = {
filename,
importModuleDynamically: USE_MAIN_CONTEXT_DEFAULT_LOADER
};
await testNotFoundErrors(options);
await testLoader(options);
}
{
// For undefined or non-path/URL filenames, import() should resolve to the cwd.
tmpdir.refresh();
process.chdir(tmpdir.path);
const undefinedOptions = {
importModuleDynamically: USE_MAIN_CONTEXT_DEFAULT_LOADER
};
const nonPathOptions = {
filename: 'non-path',
importModuleDynamically: USE_MAIN_CONTEXT_DEFAULT_LOADER
};
// Run the error tests first to avoid caching.
await testNotFoundErrors(undefinedOptions);
await testNotFoundErrors(nonPathOptions);
// createContext() with null referrer also resolves to cwd.
{
const options = {
importModuleDynamically: USE_MAIN_CONTEXT_DEFAULT_LOADER,
};
const ctx = createContext({}, options);
const s = new Script('Promise.resolve("import(\'./message.mjs\')").then(eval)', {
importModuleDynamically: common.mustNotCall(),
});
await assert.rejects(s.runInContext(ctx), { code: 'ERR_MODULE_NOT_FOUND' });
}
await testLoader(undefinedOptions);
await testLoader(nonPathOptions);
{
const options = {
importModuleDynamically: USE_MAIN_CONTEXT_DEFAULT_LOADER,
};
const ctx = createContext({}, options);
const moduleUrl = fixtures.fileURL('es-modules', 'message.mjs');
const namespace = await import(moduleUrl.href);
const script = new Script('Promise.resolve("import(\'./message.mjs\')").then(eval)', {
importModuleDynamically: common.mustNotCall(),
});
const result = await script.runInContext(ctx);
assert.deepStrictEqual(result, namespace);
}
}
}
main().catch(common.mustNotCall());

1
test/parallel/test-bootstrap-modules.js
View File

@ -107,7 +107,6 @@ expected.atRunTime = new Set([
'NativeModule internal/net',
'NativeModule internal/dns/utils',
'NativeModule internal/process/pre_execution',
'NativeModule internal/vm/module',
'NativeModule internal/modules/esm/utils',
]);

2
tools/doc/type-parser.mjs
View File

@ -227,6 +227,8 @@ const customTypesMap = {
'vm.Module': 'vm.html#class-vmmodule',
'vm.Script': 'vm.html#class-vmscript',
'vm.SourceTextModule': 'vm.html#class-vmsourcetextmodule',
'vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER':
'vm.html#vmconstantsuse_main_context_default_loader',
'MessagePort': 'worker_threads.html#class-messageport',
'Worker': 'worker_threads.html#class-worker',