Commit Graph

8 Commits

Author SHA1 Message Date
Yagiz Nizipli
cde6dccb65
tools: refactor js2c.cc to use c++20
PR-URL: https://github.com/nodejs/node/pull/54849
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
2024-09-19 19:21:21 +00:00
Michaël Zasso
7e6d92c485
tools: update ESLint to v9 and use flat config
Closes: https://github.com/nodejs/node/issues/52567
PR-URL: https://github.com/nodejs/node/pull/52780
Fixes: https://github.com/nodejs/node/issues/52567
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2024-05-23 19:45:18 +00:00
Michael Dawson
c43a944231 tools: fix error reported by coverity in js2c.cc
Coverity reported using the wrong argument types
to printf

Signed-off-by: Michael Dawson <midawson@redhat.com>
PR-URL: https://github.com/nodejs/node/pull/52142
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2024-03-25 09:52:53 -04:00
Cheng Zhao
d8586db699
tools: fix missing [[fallthrough]] in js2c
The latest clang warns about this.

PR-URL: https://github.com/nodejs/node/pull/51845
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2024-02-25 08:10:00 +00:00
Joyee Cheung
72df124e38
build: encode non-ASCII Latin1 characters as one byte in JS2C
Previously we had two encodings for JS files:

1. If a file contains only ASCII characters, encode it as a one-byte
  string (interpreted as uint8_t array during loading).
2. If a file contains any characters with code point above 127,
  encode it as a two-byte string (interpreted as uint16_t array
  during loading).

This was done because V8 only supports Latin-1 and UTF16 encoding
as underlying representation for strings. To store the JS code
as external strings to save encoding cost and memory overhead
we need to follow the representations supported by V8.
Notice that there is a gap in the Latin1 range (128-255) that we
encoded as two-byte, which was an undocumented TODO for a long
time. That was fine previously because then files that contained
code points beyond the 0-127 range contained code points >255.
Now we have undici which contains code points in the range 0-255
(minus a replaceable code point >255). So this patch adds handling
for the 128-255 range to reduce the size overhead caused by encoding
them as two-byte. This could reduce the size of the binary by
~500KB and helps future files with this kind of code points.

Drive-by: replace `’` with `'` in undici.js to make it a Latin-1
only string. That could be removed if undici updates itself to
replace this character in the comment.

PR-URL: https://github.com/nodejs/node/pull/51605
Reviewed-By: Daniel Lemire <daniel@lemire.me>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
2024-02-17 17:09:24 +00:00
Cheng Zhao
def014dff3
tools: allow passing absolute path of config.gypi in js2c
PR-URL: https://github.com/nodejs/node/pull/49162
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
2023-09-07 09:20:38 +00:00
Keyhan Vakil
5c1233dfbc
tools: speedup compilation of js2c output
Incremental compilation of Node.js is slow. Currently on a powerful
Linux machine, it takes about 9 seconds and 830 MB of memory to compile
`gen/node_javascript.cc` with g++. This is the longest step when
recompiling a small change to a Javascript file.

`gen/node_javascript.cc` contains a lot of large binary literals of our
Javascript source code. It is well-known that embedding large binary
literals as C/C++ arrays is slow. One workaround is to include the data
as string literals instead. This is particularly nice for the Javascript
included via js2c, which look better as string literals anyway.

Add a build flag `NODE_JS2C_USE_STRING_LITERALS` to js2c. When this flag
is set, we emit string literals instead of array literals, i.e.:

```c++
// old: static const uint8_t X[] = { ... };
static const uint8_t *X = R"JS2C1b732aee(...)JS2C1b732aee";

// old: static const uint16_t Y[] = { ... };
static const uint16_t *Y = uR"JS2C1b732aee(...)JS2C1b732aee";
```

This requires some modest refactoring in order to deal with the flag
being on or off, but the new code itself is actually shorter.

I only enabled the new flag on Linux/macOS, since those are systems that
I have available for testing. On my Linux system with gcc, it speeds up
compilation by 5.5s (9.0s -> 3.5s). On my Mac system with clang, it
speeds up compilation by 2.2s (3.7s -> 1.5s). (I don't think this flag
will work with MSVC, but it'd probably speed up clang on windows.)

The long-term goal here is probably to allow this to occur incrementally
per Javascript file & in parallel, to avoid recompiling all of
`gen/node_javascript.cc`. Unfortunately the necessary gyp incantations
seem impossible (or at least, far beyond me). Anyway, a 60% speedup is a
nice enough win.

Refs: https://github.com/nodejs/node/issues/47984
PR-URL: https://github.com/nodejs/node/pull/48160
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2023-06-24 15:52:29 +00:00
Joyee Cheung
7fba2c223d
tools: port js2c.py to C++
This makes it easier to use third-party dependencies in this tool
(e.g. adding compression using algorithms not available in Python).
It is also much faster - locally js2c.py takes ~1.5s to generate the
output whereas this version takes ~0.1s - and consumes less memory
(~110MB v.s. 66MB).

This also modifies the js2c.py a bit to simplify the output, making
it easier to compare with one generated by the C++ version. Locally
the output from the two are identical. We'll remove js2c.py in a
subsequent commit when the C++ version is used by default.

PR-URL: https://github.com/nodejs/node/pull/46997
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2023-05-24 13:32:58 +02:00