Commit Graph

155 Commits

Author SHA1 Message Date
David Sherret
8be2bbf074
feat: Wasm module support (#26668)
Support for Wasm modules.

Note this implements the standard where the default export is the
instance (not the module). The module will come later with source phase
imports.

```ts
import { add } from "./math.wasm";

console.log(add(1, 2));
```
2024-11-19 18:59:23 -05:00
Leo Kettmeir
628816448e
refactor: update deno_doc, use prismjs, remove internal reference html generation logic (#26885)
Some checks are pending
ci / pre-build (push) Waiting to run
ci / ${{ matrix.job }} ${{ matrix.profile }} ${{ matrix.os }}-${{ matrix.arch }} (aarch64, test, linux, debug, ubicloud-standard-16-arm) (push) Blocked by required conditions
ci / ${{ matrix.job }} ${{ matrix.profile }} ${{ matrix.os }}-${{ matrix.arch }} (aarch64, test, linux, release, ubicloud-standard-16-arm, true) (push) Blocked by required conditions
ci / ${{ matrix.job }} ${{ matrix.profile }} ${{ matrix.os }}-${{ matrix.arch }} (aarch64, test, macos, debug, ${{ github.repository == 'denoland/deno' && startsWith(github.ref, 'refs/tags/') && 'self-hosted' || 'macos-14' }}) (push) Blocked by required conditions
ci / ${{ matrix.job }} ${{ matrix.profile }} ${{ matrix.os }}-${{ matrix.arch }} (aarch64, test, macos, release, ${{ (!contains(github.event.pull_request.labels.*.name, 'ci-full') && (github.event_name == 'pull_request')) && 'ubuntu-24.04' || github.reposit… (push) Blocked by required conditions
ci / ${{ matrix.job }} ${{ matrix.profile }} ${{ matrix.os }}-${{ matrix.arch }} (x86_64, bench, linux, release, ${{ (!contains(github.event.pull_request.labels.*.name, 'ci-full') && (github.event_name == 'pull_request' && !contains(github.event.pull_reques… (push) Blocked by required conditions
ci / ${{ matrix.job }} ${{ matrix.profile }} ${{ matrix.os }}-${{ matrix.arch }} (x86_64, lint, linux, debug, ubuntu-24.04) (push) Blocked by required conditions
ci / ${{ matrix.job }} ${{ matrix.profile }} ${{ matrix.os }}-${{ matrix.arch }} (x86_64, lint, macos, debug, macos-13) (push) Blocked by required conditions
ci / ${{ matrix.job }} ${{ matrix.profile }} ${{ matrix.os }}-${{ matrix.arch }} (x86_64, lint, windows, debug, windows-2022) (push) Blocked by required conditions
ci / ${{ matrix.job }} ${{ matrix.profile }} ${{ matrix.os }}-${{ matrix.arch }} (x86_64, test, linux, debug, ubuntu-24.04, true) (push) Blocked by required conditions
ci / ${{ matrix.job }} ${{ matrix.profile }} ${{ matrix.os }}-${{ matrix.arch }} (x86_64, test, linux, release, ${{ github.repository == 'denoland/deno' && 'ubuntu-24.04-xl' || 'ubuntu-24.04' }}, true, ${{ !startsWith(github.ref, 'refs/tags/') }}) (push) Blocked by required conditions
ci / ${{ matrix.job }} ${{ matrix.profile }} ${{ matrix.os }}-${{ matrix.arch }} (x86_64, test, macos, debug, macos-13) (push) Blocked by required conditions
ci / ${{ matrix.job }} ${{ matrix.profile }} ${{ matrix.os }}-${{ matrix.arch }} (x86_64, test, macos, release, ${{ (!contains(github.event.pull_request.labels.*.name, 'ci-full') && (github.event_name == 'pull_request')) && 'ubuntu-24.04' || 'macos-13' }}, … (push) Blocked by required conditions
ci / ${{ matrix.job }} ${{ matrix.profile }} ${{ matrix.os }}-${{ matrix.arch }} (x86_64, test, windows, debug, windows-2022) (push) Blocked by required conditions
ci / ${{ matrix.job }} ${{ matrix.profile }} ${{ matrix.os }}-${{ matrix.arch }} (x86_64, test, windows, release, ${{ (!contains(github.event.pull_request.labels.*.name, 'ci-full') && (github.event_name == 'pull_request')) && 'ubuntu-24.04' || github.reposi… (push) Blocked by required conditions
ci / publish canary (push) Blocked by required conditions
2024-11-19 08:56:04 -08:00
David Sherret
186b52731c
fix(node): handle resolving ".//<something>" in npm packages (#26920)
The issue was this package had an import like: `".//index.js"` and we
resolved that as specified, but node normalizes it to `"./index.js"` so
we have to copy node.
2024-11-19 09:57:12 -05:00
David Sherret
617350e79c
refactor(resolver): move more resolution code into deno_resolver (#26873)
Follow-up to cjs refactor.

This moves most of the resolution code into the deno_resolver crate.
Still pending is the npm resolution code.
2024-11-14 15:24:25 -05:00
David Sherret
f091d1ad69
feat(node): stabilize detecting if CJS via "type": "commonjs" in a package.json (#26439)
This will respect `"type": "commonjs"` in a package.json to determine if
`.js`/`.jsx`/`.ts`/.tsx` files are CJS or ESM. If the file is found to
be ESM it will be loaded as ESM though.
2024-11-13 15:10:09 +00:00
David Sherret
826e42a5b5
fix: improved support for cjs and cts modules (#26558)
* cts support
* better cjs/cts type checking
* deno compile cjs/cts support
* More efficient detect cjs (going towards stabilization)
* Determination of whether .js, .ts, .jsx, or .tsx is cjs or esm is only
done after loading
* Support `import x = require(...);`

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-11-01 12:27:00 -04:00
David Sherret
f0f476e584
perf: pass transpiled module to deno_core as known string (#26555) 2024-10-26 13:41:09 -04:00
David Sherret
1a0cb5b531
feat(unstable): --unstable-detect-cjs for respecting explicit "type": "commonjs" (#26149)
When using the `--unstable-detect-cjs` flag or adding `"unstable":
["detect-cjs"]` to a deno.json, it will make a JS file CJS if the
closest package.json contains `"type": "commonjs"` and the file is not
an ESM module (no TLA, no `import.meta`, no `import`/`export`).
2024-10-14 20:48:39 -04:00
David Sherret
183130ff31
refactor: cleanup for creating worker structs (#25933) 2024-09-29 20:07:50 -04:00
Bartek Iwańczuk
5504acea67
feat: add --allow-import flag (#25469)
This replaces `--allow-net` for import permissions and makes the
security sandbox stricter by also checking permissions for statically
analyzable imports.

By default, this has a value of
`--allow-import=deno.land:443,jsr.io:443,esm.sh:443,raw.githubusercontent.com:443,gist.githubusercontent.com:443`,
but that can be overridden by providing a different set of hosts.

Additionally, when no value is provided, import permissions are inferred
from the CLI arguments so the following works because
`fresh.deno.dev:443` will be added to the list of allowed imports:

```ts
deno run -A -r https://fresh.deno.dev
```

---------

Co-authored-by: David Sherret <dsherret@gmail.com>
2024-09-26 01:50:54 +00:00
Leo Kettmeir
a1d0a427e8
feat: default to TS for file extension and support ext flag in more scenarios (#25472)
Closes #11220
Currently does lint, fmt, and repl
2024-09-18 21:15:13 +02:00
David Sherret
62e952559f
refactor(permissions): split up Descriptor into Allow, Deny, and Query (#25508)
This makes the permission system more versatile.
2024-09-16 21:39:37 +01:00
Asher Gomez
5e0b2aa473
chore: remove prevent_v8_code_cache from CliModuleLoaderInner (#25566) 2024-09-11 19:06:11 +00:00
David Sherret
c6d1b0a1cc
fix(byonm): resolve npm deps of jsr deps (#25399)
This allows using npm deps of jsr deps without having to add them to the
root package.json.

Works by taking the package requirement and scanning the
`node_modules/.deno` directory for the best matching package, so it
relies on deno's node_modules structure.

Additionally to make the transition from package.json to deno.json
easier, Deno now:

1. Installs npm deps in a deno.json at the same time as installing npm
deps from a package.json.
2. Uses the alias in the import map for `node_modules/<alias>` for
better package.json compatiblity.
2024-09-04 14:00:44 +00:00
Nathan Whitaker
48da3c17ea
fix(add): Handle packages without root exports (#25102)
Fixes #24607.

This PR makes the logic that caches top level dependencies (things
present in import map) smarter, so we handle JSR dependencies without
root exports.
2024-08-21 15:23:32 -07:00
Bartek Iwańczuk
b5051e25c2
feat: Deprecate "import assertions" with a warning (#24743)
This commit deprecates "import assertions" proposal that has been
replaced with "import attributes".

Any time an import assertion is encountered a warning will be printed
to the terminal. This warning will be printed for both local and 
remote files (ie. user code and dependencies).

Import assertions support will be removed in Deno 2.
2024-08-19 22:36:35 +02:00
snek
f57745fe21
feat: upgrade V8 to 12.8 (#24693)
- upgrade to v8 12.8
  - optimizes DataView bigint methods
  - fixes global interceptors
  - includes CPED methods for ALS
- fix global resolution
- makes global resolution consistent using host_defined_options.
originally a separate patch but due to the global interceptor bug it
needs to be included in this pr for all tests to pass.
2024-07-31 23:22:34 +00:00
David Sherret
50fa4d7ef5
fix: actually add missing node:readline/promises module (#24772)
Closes #24768
2024-07-28 15:41:10 -04:00
David Sherret
3bf147fe28
refactor: decouple node resolution from deno_core (#24724) 2024-07-25 19:08:14 -04:00
David Sherret
52ababc4bf
fix(node): better detection for when to surface node resolution errors (#24653) 2024-07-24 00:22:24 +00:00
Bartek Iwańczuk
445e05a39d
Reland "fix: CFunctionInfo and CTypeInfo leaks (#24634)" (#24692)
Reverted in
95847f4e94.
2024-07-24 02:11:38 +02:00
David Sherret
9114a2df69
fix(upgrade): do not error if config in cwd invalid (#24689)
```
> deno upgrade
error: Unsupported lockfile version 'invalid'. Try upgrading Deno or recreating the lockfile.
V:\scratch
> V:\deno\target\debug\deno upgrade
Looking up latest version
Local deno version 1.45.3 is the most recent release
```

Closes #24517
Closes #20729
2024-07-23 19:00:48 -04:00
Bartek Iwańczuk
95847f4e94
Revert "fix: CFunctionInfo and CTypeInfo leaks (#24634)" (#24665)
This reverts commit 6c5905dbc3.

Reverting because of errors like this in CI:
```
#
# Fatal error in , line 0
# Check failed: FastApiCallNode::ArityForArgc(c_arg_count, slow_arg_count) == value_input_count.
#
#
#
#FailureMessage Object: 0x7fd5b3ffd9d0
==== C stack trace ===============================

    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x1111883) [0x55b8d2107883]
    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x1110b6b) [0x55b8d2106b6b]
    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x110c6f9) [0x55b8d21026f9]
    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x20413cd) [0x55b8d30373cd]
    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x2034222) [0x55b8d302a222]
    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x202ff44) [0x55b8d3025f44]
    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x1e0b253) [0x55b8d2e01253]
    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x1dfcd96) [0x55b8d2df2d96]
    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x1df92aa) [0x55b8d2def2aa]
    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x1df8b71) [0x55b8d2deeb71]
    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x11ce49b) [0x55b8d21c449b]
    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x121fb23) [0x55b8d2215b23]
    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x1221359) [0x55b8d2217359]
    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x1112f3b) [0x55b8d2108f3b]
    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x111792b) [0x55b8d210d92b]
    /home/runner/work/deno/deno/target/release/deps/url_ops-e4068c650cedb9fa(+0x110df0f) [0x55b8d2103f0f]
    /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7fd5cb294ac3]
    /lib/x86_64-linux-gnu/libc.so.6(+0x126850) [0x7fd5cb326850]
```
2024-07-22 14:58:58 +02:00
Divy Srivastava
6c5905dbc3
fix: CFunctionInfo and CTypeInfo leaks (#24634)
Trying out the deno_core patch

Ref https://github.com/denoland/deno_core/pull/832

Closes https://github.com/denoland/deno/issues/24575

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-07-21 03:05:35 +02:00
David Sherret
839caf6faf
refactor: use concrete error types for node resolution (#24470)
This will help clean up some of the code in the CLI because we'll be
able to tell how the resolution failed (not part of this PR).
2024-07-09 12:15:03 -04:00
David Sherret
147411e64b
feat: npm workspace and better Deno workspace support (#24334)
Adds much better support for the unstable Deno workspaces as well as
support for npm workspaces. npm workspaces is still lacking in that we
only install packages into the root node_modules folder. We'll make it
smarter over time in order for it to figure out when to add node_modules
folders within packages.

This includes a breaking change in config file resolution where we stop
searching for config files on the first found package.json unless it's
in a workspace. For the previous behaviour, the root deno.json needs to
be updated to be a workspace by adding `"workspace":
["./path-to-pkg-json-folder-goes-here"]`. See details in
https://github.com/denoland/deno_config/pull/66

Closes #24340
Closes #24159
Closes #24161
Closes #22020
Closes #18546
Closes #16106
Closes #24160
2024-07-04 00:54:33 +00:00
Nathan Whitaker
c13b6d1413
feat(cli): Add --frozen flag to error out if lockfile is out of date (#24355)
Closes #18296.

Adds a `--frozen` (alias `--frozen-lockfile`) flag that errors out if
the lockfile is out of date. This is useful for running in CI (where an
out of date lockfile is usually a mistake) or to prevent accidental
changes in dependencies.

![Screenshot 2024-06-26 at 7 11
13 PM](https://github.com/denoland/deno/assets/17734409/538404b8-b422-4f05-89e8-4c9b1c248576)
2024-07-02 15:00:16 -07:00
Nathan Whitaker
bc8a0e6e68
refactor(cli): Create wrapper around deno_lockfile::Lockfile (#24366)
As suggested in
https://github.com/denoland/deno/pull/24355#discussion_r1657875422.

I wasn't able to hide the mutex stuff as much as I'd like (ended up just
adding an escape hatch `inner()` method that locks the inner mutex),
because you can't return references to the inner fields through a mutex.

This is mostly motivated by the frozen lockfile changes
2024-06-28 17:18:21 -07:00
Nathan Whitaker
a7531361ef
fix(cli): Explicitly cache NPM packages during deno install (#24190)
Fixes a regression introduced in
https://github.com/denoland/deno/pull/24170, where we wouldn't actually
set up the node modules dir on `deno install` if there was an up to date
deno lockfile present.

Previously we were relying on the fact that resolving pending module
resolution called `cache_packages` (which sets up the node modules dir).
When pending resolutions were removed, and the `resolve_pending`
function with it, we also removed the `cache_packages` call needed to
set up node modules.
2024-06-12 16:06:41 -07:00
David Sherret
4bc96c5d2a
fix(npm): resolve dynamic npm imports individually (#24170)
* https://github.com/denoland/deno_npm/pull/57
* https://github.com/denoland/deno_graph/pull/498

Closes https://github.com/denoland/deno/issues/17802
2024-06-11 08:55:12 -04:00
David Sherret
386d5c8310
refactor: remove PermissionsContainer in deno_runtime (#24119)
Also removes permissions being passed in for node resolution. It was
completely useless because we only checked it for reading package.json
files, but Deno reading package.json files for resolution is perfectly
fine.

My guess is this is also a perf improvement because Deno is doing less
work.
2024-06-06 23:37:53 -04:00
David Sherret
a013b9113d
fix: support importing statically unanalyzable npm specifiers (#24107)
Closes https://github.com/denoland/deno/issues/20479
Closes https://github.com/denoland/deno/issues/18744
2024-06-05 15:18:06 -04:00
David Sherret
7ed90a20d0
fix: better handling of npm resolution occurring on workers (#24094)
Closes https://github.com/denoland/deno/issues/24063
2024-06-05 17:04:16 +02:00
David Sherret
814ac9a75d
perf: avoid building module graph if dynamic specifier already in graph (#24035) 2024-05-29 14:59:35 -04:00
David Sherret
94f040ac28
fix: bump cache sqlite dbs to v2 for WAL journal mode change (#24030)
In https://github.com/denoland/deno/pull/23955 we changed the sqlite db
journal mode to WAL. This causes issues when someone is running an old
version of Deno using TRUNCATE and a new version because the two fight
against each other.
2024-05-29 18:38:18 +00:00
David Sherret
a8923534ed
chore: set lockfile as having no content changes after write (#24023)
Slight perf regression when updating deno_lockfile in
https://github.com/denoland/deno/pull/23979
2024-05-28 23:40:40 -04:00
David Sherret
448fe67b7a
feat(vendor): support modifying remote files in vendor folder without checksum errors (#23979)
Includes:

* https://github.com/denoland/deno_graph/pull/486
* https://github.com/denoland/deno_graph/pull/488
* https://github.com/denoland/deno_lockfile/pull/25
* https://github.com/denoland/deno_lockfile/pull/22
* https://github.com/denoland/deno_graph/pull/483
* https://github.com/denoland/deno_graph/pull/470
2024-05-28 14:58:43 -04:00
David Sherret
b21004b1d1
fix: use hash of in-memory bytes only for code cache (#23966)
* https://github.com/denoland/deno_core/pull/752
* https://github.com/denoland/deno_core/pull/753

Did benchmarking on this and it's slightly faster (couple ms) or equal
to in performance as main.

Closes #23904
2024-05-24 10:15:46 -04:00
David Sherret
54eb930e8c
perf: resolver - skip cwd lookup if able (#23851)
The cwd lookup was taking 2% of a flamegraph I was looking at.
2024-05-21 10:38:06 -04:00
David Sherret
a2dbcf9e0a
perf: analyze cjs exports and emit typescript in parallel (#23856) 2024-05-18 11:42:03 -04:00
Divy Srivastava
88983fb3eb
fix(node): seperate worker module cache (#23634)
Construct a new module graph container for workers instead of sharing it
with the main worker.

Fixes #17248
Fixes #23461

---------

Co-authored-by: David Sherret <dsherret@gmail.com>
2024-05-16 07:09:35 +00:00
Bartek Iwańczuk
4e23a5b1fc
FUTURE: deno install changes (#23498)
This PR implements the changes we plan to make to `deno install` in deno
2.0.
- `deno install` without arguments caches dependencies from
`package.json` / `deno.json` and sets up the `node_modules` folder
- `deno install <pkg>` adds the package to the config file (either
`package.json` or `deno.json`), i.e. it aliases `deno add`
- `deno add` can also add deps to `package.json` (this is gated behind
`DENO_FUTURE` due to uncertainty around handling projects with both
`deno.json` and `package.json`)
- `deno install -g <bin>` installs a package as a globally available
binary (the same as `deno install <bin>` in 1.0)

---------

Co-authored-by: Nathan Whitaker <nathan@deno.com>
2024-05-08 12:34:46 -07:00
Igor Zinkovsky
b3d7df5535
perf: v8 code cache (#23081)
This PR enables V8 code cache for ES modules and for `require` scripts
through `op_eval_context`. Code cache artifacts are transparently stored
and fetched using sqlite db and are passed to V8. `--no-code-cache` can
be used to disable.

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-04-17 07:19:55 -07:00
Bartek Iwańczuk
2166aa8fb6
chore: upgrade deno_core to 0.272.0 (#23022) 2024-03-21 13:57:32 -07:00
David Sherret
43d066cb70
fix: stop type checking during runtime (#22854)
In addition to the reasons for this outlined by @nayeemrmn in #14877
(which I think are reasons alone to not do this), this simplifies things
a lot because then we don't need to implement the following:

1. Need to handle a JSR module dynamically importing a module within it.
2. Need to handle importing an export of a JSR dep then another export
dynamically loaded later.

Additionally, people should be running `deno check dynamic_import.ts`
instead of relying on this behaviour.

Landing this as a fix because it's blocking people in some scenarios and
the current behaviour is broken (I didn't even have to change any tests
to remove this, which is bad).

Closes #22852
Closes #14877
Closes #22580
2024-03-13 16:38:01 -04:00
David Sherret
594d8397ad
fix(publish): properly display graph validation errors (#22775)
The graph validation errors were displaying cryptically during publish.
This fixes that.
2024-03-07 17:30:30 +01:00
David Sherret
3eaf174bfc
fix(node): improve cjs tracking (#22673)
We were missing saying that a file is CJS when some Deno code imported
from the node_modules directory at runtime.
2024-03-06 00:23:51 +00:00
David Sherret
918c5e648f
fix(jsr): do not allow importing a non-JSR url via unanalyzable dynamic import from JSR (#22623)
A security feature of JSR is that it is self contained other than npm
dependencies. At publish time, the registry rejects packages that write
code like this:

```ts
const data = await import("https://example.com/evil.js");
```

However, this can be trivially bypassed by writing code that the
registry cannot statically analyze for. This PR prevents Deno from
loading dynamic imports that do this.
2024-02-28 16:30:45 -05:00
David Sherret
9166d8a4e9
feat(publish): type check on publish (#22506)
Supersedes #22501 and also fixes that issue.
2024-02-21 08:35:25 -05:00
David Sherret
f90889e5ee
perf(jsr): fast check cache and lazy fast check graph (#22485) 2024-02-20 21:29:57 +00:00