Fixes https://github.com/denoland/deno/issues/26177
The significant delay was caused by Nagel's algorithm + delayed ACKs in
Linux kernels. Here's the [kernel
patch](https://lwn.net/Articles/502585/) which added 40ms
`tcp_default_delack_min`
```
$ deno run -A pg-bench.mjs # main
Tue Oct 15 2024 12:27:22 GMT+0530 (India Standard Time): 42ms
$ target/release/deno run -A pg-bench.mjs # this patch
Tue Oct 15 2024 12:28:02 GMT+0530 (India Standard Time): 1ms
```
```js
import { Buffer } from "node:buffer";
import pg from 'pg'
const { Client } = pg
const client = new Client({
connectionString: 'postgresql://postgres:postgres@127.0.0.1:5432/postgres'
})
await client.connect()
async function fetch() {
const startPerf = performance.now();
const res = await client.query(`select
$1::int as int,
$2 as string,
$3::timestamp with time zone as timestamp,
$4 as null,
$5::bool as boolean,
$6::bytea as bytea,
$7::jsonb as json
`, [
1337,
'wat',
new Date().toISOString(),
null,
false,
Buffer.from('awesome'),
JSON.stringify([{ some: 'json' }, { array: 'object' }])
])
console.log(`${new Date()}: ${Math.round(performance.now() - startPerf)}ms`)
}
for(;;) await fetch();
```
Fixes#22995. Fixes#23000.
There were a handful of bugs here causing the hang (each with a
corresponding minimized test):
- We were canceling recv futures when `receiveMessageOnPort` was called,
but this caused the "receive loop" in the message port to exit. This was
due to the fact that `CancelHandle`s are never reset (i.e., once you
`cancel` a `CancelHandle`, it remains cancelled). That meant that after
`receieveMessageOnPort` was called, the subsequent calls to
`op_message_port_recv_message` would throw `Interrupted` exceptions, and
we would exit the loop.
The cancellation, however, isn't actually necessary.
`op_message_port_recv_message` only borrows the underlying port for long
enough to poll the receiver, so the borrow there could never overlap
with `op_message_port_recv_message_sync`.
- Calling `MessagePort.unref()` caused the "receive loop" in the message
port to exit. This was because we were setting
`messageEventListenerCount` to 0 on unref. Not only does that break the
counter when multiple `MessagePort`s are present in the same thread, but
we also exited the "receive loop" whenever the listener count was 0. I
assume this was to prevent the recv promise from keeping the event loop
open.
Instead of this, I chose to just unref the recv promise as needed to
control the event loop.
- The last bug causing the hang (which was a doozy to debug) ended up
being an unfortunate interaction between how we implement our
messageport "receive loop" and a pattern found in `npm:piscina` (which
angular uses). The gist of it is that piscina uses an atomic wait loop
along with `receiveMessageOnPort` in its worker threads, and as the
worker is getting started, the following incredibly convoluted series of
events occurs:
1. Parent sends a MessagePort `p` to worker
2. Parent sends a message `m` to the port `p`
3. Parent notifies the worker with `Atomics.notify` that a new message
is available
4. Worker receives message, adds "message" listener to port `p`
5. Adding the listener triggers `MessagePort.start()` on `p`
6. Receive loop in MessagePort.start receives the message `m`, but then
hits an await point and yields (before dispatching the "message" event)
7. Worker continues execution, starts the atomic wait loop, and
immediately receives the existing notification from the parent that a
message is available
8. Worker attempts to receive the new message `m` with
`receiveMessageOnPort`, but this returns `undefined` because the receive
loop already took the message in 6
9. Atomic wait loop continues to next iteration, waiting for the next
message with `Atomic.wait`
10. `Atomic.wait` blocks the worker thread, which prevents the receive
loop from continuing and dispatching the "message" event for the
received message
11. The parent waits for the worker to respond to the first message, and
waits
12. The thread can't make any more progress, and the whole process hangs
The fix I've chosen here (which I don't particularly love, but it works)
is to just delay the `MessagePort.start` call until the end of the event
loop turn, so that the atomic wait loop receives the message first. This
prevents the hang.
---
Those were the main issues causing the hang. There ended up being a few
other small bugs as well, namely `exit` being emitted multiple times,
and not patching up the message port when it's received by
`receiveMessageOnPort`.
Testing once again if the crates are being properly released.
---------
Co-authored-by: bartlomieju <bartlomieju@users.noreply.github.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Test run before Deno 2.0 release to make sure that the publishing
process passes correctly.
---------
Co-authored-by: bartlomieju <bartlomieju@users.noreply.github.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Aligns the error messages in the ext folder to be in-line with the Deno
style guide.
https://github.com/denoland/deno/issues/25269
<!--
Before submitting a PR, please read
https://docs.deno.com/runtime/manual/references/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix#7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
Add an implementation of cpu_info() for OpenBSD, that returns a
correctly-sized array. Since Rust's libc bindings for OpenBSD do not
contain all symbols necessary for a full implementation and it is not
planned to add them, this solution at least avoids problems with code
that relies on cpu_info() purely for the size of the returned array to
derive the number of available CPUs.
This addresses https://github.com/denoland/deno/issues/25621
Fixes https://github.com/denoland/deno/issues/23508
`width` and `height` are required to configure the wgpu surface because
Deno is headless and depends on user to create a window. The options
were non-standard extension of `GPUCanvasConfiguration#configure`.
This PR adds a required options parameter with the `width` and `height`
options to `Deno.UnsafeWindowSurface` constructor.
```typescript
// Old, non-standard extension of GPUCanvasConfiguration
const surface = new Deno.UnsafeWindowSurface("x11", displayHandle, windowHandle);
const context = surface.getContext();
context.configure({ width: 600, height: 800, /* ... */ });
```
```typescript
// New
const surface = new Deno.UnsafeWindowSurface({
system: "x11",
windowHandle,
displayHandle,
width: 600,
height: 800,
});
const context = surface.getContext();
context.configure({ /* ... */ });
```
This PR optimizes the case when `performance.measure()` needs to find
the startMark by name. It is a simple change on `findMostRecent` fn to
avoiding copying and reversing the complete entries list.
Adds minor missing tests for:
- `clearMarks()`, general
- `clearMeasures()`, general
- `measure()`, case when the startMarks name exists more than once
### Benchmarks
#### main
```
CPU | AMD Ryzen 7 PRO 6850U with Radeon Graphics
Runtime | Deno 2.0.0-rc.4 (x86_64-unknown-linux-gnu)
benchmark time/iter (avg) iter/s (min … max) p75 p99 p995
---------------------- ----------------------------- --------------------- --------------------------
worst case measure() 2.1 ms 486.9 ( 1.7 ms … 2.4 ms) 2.2 ms 2.4 ms 2.4 ms
```
#### this PR
```
CPU | AMD Ryzen 7 PRO 6850U with Radeon Graphics
Runtime | Deno 2.0.0-rc.4 (x86_64-unknown-linux-gnu)
benchmark time/iter (avg) iter/s (min … max) p75 p99 p995
---------------------- ----------------------------- --------------------- --------------------------
worst case measure() 966.3 µs 1,035 (876.9 µs … 1.1 ms) 1.0 ms 1.1 ms 1.1 ms
```
```ts
Deno.bench("worst case measure()", (b) => {
performance.mark('start');
for (let i = 0; i < 1e5; i += 1) {
performance.mark(crypto.randomUUID());
}
b.start();
performance.measure('total', 'start');
b.end();
performance.clearMarks();
performance.clearMeasures();
});
```
Fixes rsbuild running in deno.
You can look at the test to see what was failing, the gist is that we
were trying to statically analyze the re-exports of a CJS script, and if
we couldn't find the source for the re-exported file we would fail.
Instead, we should just treat these as if they were too dynamic to
analyze, and let it fail (or succeed) at runtime. This aligns with
node's behavior.
Aligns the error messages in the ext/http and a few messages in the
ext/fetch folder to be in-line with the Deno style guide.
This change-set also removes some unnecessary checks in the 00_serve.ts.
These options were recently removed, so it doesn't make sense to check
for them anymore.
https://github.com/denoland/deno/issues/25269
This fixes the fast path for `readableStreamCollectIntoUint8Array` to
only trigger if the readable stream has not yet been disturbed -
because otherwise we may not be able to close it if the
read errors.
This commit fixes the error format when the cause is assigned
separately, ensuring that the cause is only printed once instead of
twice.
The fix addresses issue
[#21651](https://github.com/denoland/deno/issues/21651).
Contributing toward #24236
- Swapped `Object.assign` for `ObjectAssign` primordial.
- Removed referencing TODO comment.
Please disregard if no longer desired.
Apparently `path/posix` and `path/win32` have circular exports. I do not
know why.
Additionally there's a deprecated function `_makeLong` which is just
`toNamespacedPath`
Closes#20613.
Reimplements the serialization on top of the v8 APIs instead of
deno_core. Implements `v8.Serializer`, `v8.DefaultSerializer`,
`v8.Deserializer`, and `v8.DefaultSerializer`.
implement require(esm) using `op_import_sync` from deno_core.
possible future changes:
- cts and mts
- replace Deno.core.evalContext to optimize esm syntax detection
Fixes: https://github.com/denoland/deno/issues/25487
<!--
Before submitting a PR, please read
https://docs.deno.com/runtime/manual/references/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix#7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
Mark `op_require_break_on_next_statement` as reentrant and properly
release borrow on the `OpState`. This fixes `BorrowMut` assertions when
running with inspector + op metrics.
Fixes https://github.com/denoland/deno/issues/25515
A workaround for the issue #25480
`Deno.Listener` can't be closed synchronously after `accept()` is
called. This PR delays the `accept` call 2 ticks (The listener callback
is called 1 tick later. So the 1 tick delay is not enough), and makes
`net.Server` capable of being closed synchronously.
This unblocks `npm:detect-port` and `npm:portfinder`
closes#18301closes#25175
Closes https://github.com/denoland/deno/issues/25321
Ended up being a larger refactoring, since we're now juggling
(potentially) two config files in the same `add`, instead of choosing
one. I don't love the shape of the code, but I think it's good enough
Some smaller side improvements:
- `deno remove` supports `jsonc`
- `deno install --dev` will be a really simple change
- if `deno remove` removes the last import/dependency in the
`imports`/`dependencies`/`devDependencies` field, it removes the field
instead of leaving an empty object
This commit adds:
- `addAbortListener` in `node:events`
- `aborted` in `node:util`
- `execPath` and `execvArgs` named export from `node:process`
- `getDefaultHighWaterMark` from `node:stream`
The `execPath` is very hacky - because module namespaces can not have
real getters, `execPath` is an object with a `toString()` method that on
call returns the actual `execPath`, and replaces the `execPath` binding
with the string. This is done so that we don't require the `execPath`
permission on startup.
This commit changes when to cause the hostname substition of `0.0.0.0` ->
`localhost`.
Currently we substitute `localhost` to the hostname on windows before
calling `options.onListen`, which prevents the users to do more advanced
thing using hostname string like
https://github.com/denoland/std/issues/5558. This PR changes it not to
substitute it when the user provide `onListen` callback.
closes#24776
unblocks https://github.com/denoland/std/issues/5558
This change fixes the handling of upgraded socket from `node:http` module.
In `op_node_http_fetch_response_upgrade`, we create DuplexStream paired
with `hyper::upgrade::Upgraded`. When the connection is closed from the
server, the read result from `Upgraded` becomes 0. However because we
don't close the paired DuplexStream at that point, the Socket object in
JS side keeps alive even after the server closed. That caused the issue
#20179
This change fixes it by closing the paired DuplexStream when the
`Upgraded` stream returns 0 read result.
closes#20179
This allows using npm deps of jsr deps without having to add them to the
root package.json.
Works by taking the package requirement and scanning the
`node_modules/.deno` directory for the best matching package, so it
relies on deno's node_modules structure.
Additionally to make the transition from package.json to deno.json
easier, Deno now:
1. Installs npm deps in a deno.json at the same time as installing npm
deps from a package.json.
2. Uses the alias in the import map for `node_modules/<alias>` for
better package.json compatiblity.
Remove `--allow-hrtime` and `--deny-hrtime`. We are doing this because
it is already possible to get access to high resolution timers through
workers and SharedArrayBuffer.
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
`deno bundle` now produces:
```
error: ⚠️ `deno bundle` was removed in Deno 2.
See the Deno 1.x to 2.x Migration Guide for migration instructions: https://docs.deno.com/runtime/manual/advanced/migrate_deprecations
```
`deno bundle --help` now produces:
```
⚠️ `deno bundle` was removed in Deno 2.
See the Deno 1.x to 2.x Migration Guide for migration instructions: https://docs.deno.com/runtime/manual/advanced/migrate_deprecations
Usage: deno bundle [OPTIONS]
Options:
-q, --quiet Suppress diagnostic output
--unstable Enable all unstable features and APIs. Instead of using this flag, consider enabling individual unstable features
To view the list of individual unstable feature flags, run this command again with --help=unstable
```
