Commit Graph

411 Commits

Author SHA1 Message Date
Leo Kettmeir
cf49599359
feat: permission stack traces in ops (#26938)
This commit improves permission prompts by adding an option
to print a full trace of where the permissions is being requested.

Due to big performance hint of stack trace collection, this is only
enabled when `DENO_TRACE_PERMISSIONS` env var is present.

Closes https://github.com/denoland/deno/issues/20756

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-11-20 21:24:04 +00:00
Luca Casonato
594a99817c
feat(runtime): remove public OTEL trace API (#26854)
This PR removes the public Deno.tracing.Span API.
We are not confident we can ship an API that is
better than the `@opentelemetry/api` API, because
V8 CPED does not support us using `using` to
manage span context. If this changes, we can
revisit this decision. For now, users wanting
custom spans can instrument their code using
the `@opentelemetry/api` API and `@deno/otel`.

This PR also speeds up the OTEL trace generation
by a 30% by using Uint8Array instead of
strings for the trace ID and span ID.
2024-11-18 23:55:22 +00:00
Bartek Iwańczuk
c9baf3849f
perf: use available system memory for v8 isolate memory limit (#26868)
Instead of using the default 1.4Gb limit (which was meant for browser
tabs) configure V8 to
set the heap limit to the amount of memory available in the system.

Closes https://github.com/denoland/deno/issues/23424
Closes https://github.com/denoland/deno/issues/26435
Closes https://github.com/denoland/deno/issues/21226
2024-11-15 10:33:03 +01:00
snek
4e899d48cf
fix: otel resiliency (#26857)
Improving the breadth of collected data, and ensuring that the collected
data is more likely to be successfully reported.

- Use `log` crate in more places
- Hook up `log` crate to otel
- Switch to process-wide otel processors
- Handle places that use `process::exit`

Also adds a more robust testing framework, with a deterministic tracing
setting.

Refs: https://github.com/denoland/deno/issues/26852
2024-11-14 12:16:28 +00:00
snek
aa546189be
feat: OpenTelemetry Tracing API and Exporting (#26710)
Initial import of OTEL code supporting tracing. Metrics soon to come.
Implements APIs for https://jsr.io/@deno/otel so that code using
OpenTelemetry.js just works tm.

There is still a lot of work to do with configuration and adding
built-in tracing to core APIs, which will come in followup PRs.

---------

Co-authored-by: Luca Casonato <hello@lcas.dev>
2024-11-13 10:38:46 +00:00
David Sherret
119910f339
fix(permissions): say to use --allow-run instead of --allow-all (#26842)
For https://github.com/denoland/deno/issues/26839
2024-11-12 17:14:19 -05:00
Leo Kettmeir
fe9f0ee593
refactor(runtime/permissions): use concrete error types (#26464) 2024-11-04 09:17:21 -08:00
Divy Srivastava
be969cb532
fix: share inotify fd across watchers (#26200)
Fixes https://github.com/denoland/deno/issues/26104
Fixes https://github.com/denoland/deno/issues/26071
Fixes https://github.com/denoland/deno/issues/17757
2024-10-23 09:22:58 +05:30
Leo Kettmeir
f26c8bcf31
refactor(runtime/ops): use concrete error types (#26409) 2024-10-22 01:41:08 -07:00
Leo Kettmeir
473e3069de
chore: update nix crate (#26422)
Dedupes nix dependency, since `rustyline` depends on a newer version
that what we currently use
2024-10-19 21:59:39 +00:00
Leo Kettmeir
2c3900370a
refactor(ext/http): use concrete error types (#26377) 2024-10-18 15:57:12 -07:00
Leo Kettmeir
d047cab14b
refactor(ext/websocket): use concrete error type (#26226) 2024-10-18 19:30:46 +00:00
Leo Kettmeir
eca83fc9b4
refactor(ext/web): use concrete error types (#26185) 2024-10-17 19:05:38 +00:00
Patrick Uftring
e61e2e4b5c
refactor: use macros for signal table (#26214) 2024-10-17 17:05:57 +05:30
Leo Kettmeir
cb385d9e4a
refactor(ext/webstorage): use concrete error types (#26173) 2024-10-14 13:53:17 -07:00
David Sherret
c8f692057b
refactor: bury descriptor parsing in PermissionsContainer (#25936)
Closes https://github.com/denoland/deno/issues/25634
2024-09-30 09:19:24 -04:00
David Sherret
fc739dc5eb
refactor: use deno_path_util (#25918) 2024-09-28 07:55:01 -04:00
Nathan Whitaker
fbddd5a2eb
fix(node): Pass NPM_PROCESS_STATE to subprocesses via temp file instead of env var (#25896)
Fixes https://github.com/denoland/deno/issues/25401. Fixes
https://github.com/denoland/deno/issues/25841. Fixes
https://github.com/denoland/deno/issues/25891.
2024-09-27 12:35:37 -07:00
Luca Casonato
3134abefa4
BREAKING(ext/net): improved error code accuracy (#25383)
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-09-27 14:07:20 +00:00
Bartek Iwańczuk
aaf2bf4bfb
chore: upgrade deno_core (#25674)
No functional changes, just removes dead code.
2024-09-17 01:13:34 +00:00
Nathan Whitaker
e81c67ba22
feat(permissions): Deno.mainModule doesn't require permissions (#25667)
Closes https://github.com/denoland/deno/issues/7315.
2024-09-16 13:43:52 -07:00
David Sherret
62e952559f
refactor(permissions): split up Descriptor into Allow, Deny, and Query (#25508)
This makes the permission system more versatile.
2024-09-16 21:39:37 +01:00
Nathan Whitaker
18b89d948d
fix(ext/node): Implement detached option in child_process (#25218)
Fixes https://github.com/denoland/deno/issues/25193.
2024-09-12 19:24:58 +00:00
Asher Gomez
8476bbff9a
feat: stabilize Deno.createHttpClient() (#25569)
Closes #25518
2024-09-12 10:46:48 +10:00
Luca Casonato
7bfcb4dd10
feat(cli): use NotCapable error for permission errors (#25431)
Closes #7394

---------

Co-authored-by: snek <snek@deno.com>
2024-09-10 11:12:24 -07:00
Bartek Iwańczuk
064a73f7a0
BREAKING: Remove --unstable flag (#25522)
This commit effectively removes the --unstable flag.

It's still being parsed, but it only prints a warning that a granular
flag should be used instead and doesn't actually enable any
unstable feature.

Closes https://github.com/denoland/deno/issues/25485
Closes https://github.com/denoland/deno/issues/23237
2024-09-09 23:44:29 +02:00
Divy Srivastava
ea8bf0945a
fix(ext/node): report freemem() on Linux in bytes (#25511) 2024-09-09 17:03:17 +05:30
Bartek Iwańczuk
5dedb49ac4
refactor(permissions): remove FromStr implementations, add ::parse methods (#25473)
The `.parse()` calls in permission code are only making it more
confusing, verbosity
is encouraged and welcome in this code even at the cost of not being
concise.

Left a couple TODOs to not use `AnyError`.
2024-09-06 11:28:53 +02:00
David Sherret
dd208a6df0
fix(regression): do not expose resolved path in Deno.Command permission denied error (#25434)
Regression from https://github.com/denoland/deno/pull/25370
2024-09-04 22:57:49 +00:00
David Sherret
74fc66da11
fix: lock down allow-run permissions more (#25370)
`--allow-run` even with an allow list has essentially been
`--allow-all`... this locks it down more.

1. Resolves allow list for `--allow-run=` on startup to an absolute
path, then uses these paths when evaluating if a command can execute.
Also, adds these paths to `--deny-write`
1. Resolves the environment (cwd and env vars) before evaluating
permissions and before executing a command. Then uses this environment
to evaluate the permissions and then evaluate the command.
2024-09-04 14:51:24 +02:00
Luca Casonato
5cf97f539b
BREAKING(permissions): remove --allow-hrtime (#25367)
Remove `--allow-hrtime` and `--deny-hrtime`. We are doing this because
it is already possible to get access to high resolution timers through
workers and SharedArrayBuffer.

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-09-03 11:24:25 +02:00
Bartek Iwańczuk
71e4ac774b
BREAKING(unstable): drop support for Deno.run.{clearEnv,gid,uid} (#25371)
These are unstable options and the APIs is now deprecated. To limit
amount of unstable flags we elected to have these APIs removed.
2024-09-03 11:07:02 +02:00
David Sherret
c6793f52b9
fix(permissions): disallow any LD_ or DYLD_ prefixed env var without full --allow-run permissions (#25271)
Follow up to https://github.com/denoland/deno/pull/25221

I looked into what the list was and it was quite extensive, so I think
as suggested in
https://github.com/denoland/deno/issues/11964#issuecomment-2314585135 we
should disallow this for any `LD_` prefixed env var.
2024-08-28 21:11:37 -04:00
David Sherret
1a6fd38f2f
fix(permissions): disallow launching subprocess with LD_PRELOAD env var without full run permissions (#25221)
Ref https://github.com/denoland/deno/pull/25215

Closes https://github.com/denoland/deno/issues/11964
2024-08-27 22:03:09 -04:00
HasanAlrimawi
672ce3041a
fix: removed unstable-htttp from deno help (#25216)
Closes #25210 .

Removed --unstable-http from being displayed on deno run --help=unstable

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-08-27 13:45:27 +02:00
snek
c0aa68a1e2
feat: upgrade deno_core (#25042)
- Update ffi turbocall to use revised fast call api
- Remove `v8_version` function calls
- `*mut OwnedIsolate` is no longer stored in OpCtx gotham store
2024-08-19 14:51:16 +00:00
Bartek Iwańczuk
2bb013f9ba
refactor: version module exports a single const struct (#25014)
This commit rewrites the internal `version` module that exported
various information about the current executable. Instead of exporting
several consts, we are now exporting a single const structure that 
contains all the necessary information.

This is the first step towards cleaning up how we use this information
and should allow us to use SUI to be able to patch this information
in already produced binary making it easier to cut new releases.

---------

Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
2024-08-15 23:47:16 +02:00
Nathan Whitaker
8749d651fb
fix(node): Create additional pipes for child processes (#25016)
Linux/macos only currently.

Part of https://github.com/denoland/deno/issues/23524 (fixes it on
platforms other than windows).
Part of #16899  (fixes it on platforms other than windows).

After this PR, playwright is functional on mac/linux.
2024-08-15 09:38:46 -07:00
Roy Ivy III
a9cdfdc98e
fix(runtime/windows): fix calculation of console size (#23873) 2024-08-12 17:36:37 -04:00
Marvin Hagemeister
9d6da1036d
fix: rename watch event missing (#24893)
This PR ensures that we forward a `rename` event in our file watcher.
The rust lib we use combines that with the `modify` event.

This fixes a compatibility issue with Node too, which sends the `rename`
event as well.

Fixes https://github.com/denoland/deno/issues/24880
2024-08-07 18:15:57 +02:00
snek
71ca61e189
Revert "feat: async context" (#24856)
Reverts denoland/deno#24402

deno_web can't depend on code in runtime
2024-08-02 18:16:59 +00:00
snek
3a1a1cc030
feat: async context (#24402)
We are switching to ContinuationPreservedEmbedderData. This allows
adding async context tracking to the various async operations that deno
provides.

Fixes: https://github.com/denoland/deno/issues/7010
Fixes: https://github.com/denoland/deno/issues/22886
Fixes: https://github.com/denoland/deno/issues/24368
2024-08-02 08:14:35 -07:00
Nathan Whitaker
cd59fc53a5
fix(node): Rework node:child_process IPC (#24763)
Fixes https://github.com/denoland/deno/issues/24756. Fixes
https://github.com/denoland/deno/issues/24796.

This also gets vitest working when using
[`--pool=forks`](https://vitest.dev/guide/improving-performance#pool)
(which is the default as of vitest 2.0). Ref
https://github.com/denoland/deno/issues/23882.

---

This PR resolves a handful of issues with child_process IPC. In
particular:

- We didn't support sending typed array views over IPC
- Opening an IPC channel resulted in the event loop never exiting
- Sending a `null` over IPC would terminate the channel
- There was some UB in the read implementation (transmuting an `&[u8]`
to `&mut [u8]`)
- The `send` method wasn't returning anything, so there was no way to
signal backpressure (this also resulted in the benchmark
`child_process_ipc.mjs` being misleading, as it tried to respect
backpressure. That gave node much worse results at larger message sizes,
and gave us much worse results at smaller message sizes).
- We weren't setting up the `channel` property on the `process` global
(or on the `ChildProcess` object), and also didn't have a way to
ref/unref the channel
- Calling `kill` multiple times (or disconnecting the channel, then
calling kill) would throw an error
- Node couldn't spawn a deno subprocess and communicate with it over IPC
2024-07-30 16:13:24 -07:00
Satya Rohith
8c2f1f5a55
chore: upgrade to rust 1.80 (#24778) 2024-07-29 12:58:04 -04:00
Nathan Whitaker
29934d558c
fix(node): Run node compat tests listed in the ignore field (and fix the ones that fail) (#24631)
The intent is that those tests will be executed, but our check that the
files are up to date won't overwrite the contents of the tests. This is
useful when a test needs some manual edits to work.

It turns out we weren't actually running them.

---

This ended up turning into a couple of small bug fixes to get the tests
passing:

- We weren't canonicalizing the exec path properly (it sometimes still
had `..` or `.` in it)
- We weren't accepting strings in `process.exit`

There was one failure I couldn't figure out quickly, so I disabled the
test for now, and filed a follow up issue: #24694
2024-07-24 03:12:08 +00:00
Marvin Hagemeister
ee2e693340
fix(node): support tty.hasColors() and tty.getColorDepth() (#24619)
This PR adds support for
[`tty.WriteStream.prototype.hasColors()`](https://nodejs.org/api/tty.html#writestreamhascolorscount-env)
and
[`tty.WriteStream.prototype.getColorDepth()`](https://nodejs.org/api/tty.html#writestreamgetcolordepthenv).

I couldn't find any usage on GitHub which passes parameters to it.
Therefore I've skipped adding support for the `env` parameter to keep
our snapshot size small.

Based on https://github.com/denoland/deno_terminal/pull/3

Fixes https://github.com/denoland/deno/issues/24616
2024-07-19 12:39:05 +02:00
Bartek Iwańczuk
7b33623b1d
Reland "refactor(fetch): reimplement fetch with hyper instead of reqwest" (#24593)
Originally landed in
f6fd6619e7.
Reverted in https://github.com/denoland/deno/pull/24574.

This reland contains a fix that sends "Accept: */*" header for calls made
from "FileFetcher". Absence of this header made downloading source code
from JSR broken. This is tested by ensuring this header is present in the
test server that servers JSR packages.

---------

Co-authored-by: Sean McArthur <sean@seanmonstar.com>
2024-07-18 01:37:31 +02:00
Ryan Dahl
e0cfc9da39
Revert "refactor(fetch): reimplement fetch with hyper instead of reqwest (#24237)" (#24574)
This reverts commit f6fd6619e7.

I'm seeing a difference between canary and 1.45.2. In
`deno-docs/reference_gen` I can't download dax when running `deno task
types`

```
~/src/deno-docs/reference_gen# deno upgrade --canary
Looking up latest canary version
Found latest version f6fd6619e7
Downloading https://dl.deno.land/canary/f6fd6619e708a515831f707438368d81b0c9aa56/deno-aarch64-apple-darwin.zip
Deno is upgrading to version f6fd6619e7
Archive:  /var/folders/9v/kys6gqns6kl8nksyn4l1f9v40000gn/T/.tmpb5lDnq/deno.zip
  inflating: deno
Upgraded successfully

~/src/deno-docs/reference_gen# deno -v
deno 1.45.2+f6fd661

~/src/deno-docs/reference_gen# rm -rf /Users/ry/Library/Caches/deno

~/src/deno-docs/reference_gen# deno task types
Task types deno task types:deno && deno task types:node
Task types:deno deno run --allow-read --allow-write --allow-run --allow-env --allow-sys deno-docs.ts
error: JSR package manifest for '@david/dax' failed to load. expected value at line 1 column 1
    at file:///Users/ry/src/deno-docs/reference_gen/deno-docs.ts:2:15

~/src/deno-docs/reference_gen# deno upgrade --version 1.45.2
Downloading https://github.com/denoland/deno/releases/download/v1.45.2/deno-aarch64-apple-darwin.zip
Deno is upgrading to version 1.45.2
Archive:  /var/folders/9v/kys6gqns6kl8nksyn4l1f9v40000gn/T/.tmp3R7uhF/deno.zip
  inflating: deno
Upgraded successfully

~/src/deno-docs/reference_gen# rm -rf /Users/ry/Library/Caches/deno

~/src/deno-docs/reference_gen# deno task types
Task types deno task types:deno && deno task types:node
Task types:deno deno run --allow-read --allow-write --allow-run --allow-env --allow-sys deno-docs.ts
Task types:node deno run --allow-read --allow-write=. --allow-env --allow-sys node-docs.ts
```
2024-07-13 21:08:23 +00:00
Sean McArthur
f6fd6619e7
refactor(fetch): reimplement fetch with hyper instead of reqwest (#24237)
This commit re-implements `ext/fetch` and all dependent crates
using `hyper` and `hyper-util`, instead of `reqwest`.

The reasoning is that we want to have greater control and access
to low level `hyper` APIs when implementing `fetch` API as well
as `node:http` module.

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-07-13 00:51:37 +02:00
Luca Casonato
74ac29bae6
fix(permissions): handle ipv6 addresses correctly (#24397)
Also don't panic on invalid domain names and addresses.

Extracted with cleanups up from #24080

Co-authored-by: Yazan AbdAl-Rahman <yazan.abdalrahman@exalt.ps>
2024-07-05 23:45:06 +01:00