Fix the Python test harness so that it no longer treats the `# skipped`
part of the summary at the end of the built-in test runner output as
marking the test as skipped.
PR-URL: https://github.com/nodejs/node/pull/53545
Fixes: https://github.com/nodejs/node/issues/50346
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
The output of `node -p process.config.variables.asan` includes
a newline character so it's never exactly "1", which means
asan is always "off" for the status files. This fixes the
detection by stripping whitespaces from the output.
PR-URL: https://github.com/nodejs/node/pull/52766
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Always use `process.config.variables.asan`.
This removes the need for a special ASAN env var.
PR-URL: https://github.com/nodejs/node/pull/52430
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
PR-URL: https://github.com/nodejs/node/pull/50363
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
PR-URL: https://github.com/nodejs/node/pull/47422
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
PR-URL: https://github.com/nodejs/node/pull/44490
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Zeyu "Alex" Yang <himself65@outlook.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Christian Clauss <cclauss@me.com>
PR-URL: https://github.com/nodejs/node/pull/47175
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Christian Clauss <cclauss@me.com>
We run all WPT from one subset in the same process using workers.
As the number of the tests grow, it can take longer to run some of the
subsets, but it's still overall faster than running them in different
processes. This patch increases the timeout
for WPT to prevent the test from failing because it takes longer
to run (even though it would still complete at some point).
PR-URL: https://github.com/nodejs/node/pull/44574
Refs: https://github.com/nodejs/reliability/issues/371
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
PR #43954 introduced the use of f-strings to `test.py` making the script
only work with python versions >= 3.6. Remove that use to make it
compatible again with older python versions.
PR-URL: https://github.com/nodejs/node/pull/44407
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Refs: https://github.com/nodejs/node/issues/42379
A number of tests have been failing internmittently
on the Rasberry PIs with timeouts since the update
to OpenSSL 3.0. Give them more time.
Signed-off-by: Michael Dawson <mdawson@devrus.com>
PR-URL: https://github.com/nodejs/node/pull/42380
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/40945
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
The win10-COMPILED_BY-vs2019 hosts in CI are very slow and timing out on
certain tests in pummel and (previously) benchmark. Increase timeout
from 4x to 6x.
PR-URL: https://github.com/nodejs/node/pull/40684
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Voltrex <mohammadkeyvanzade94@gmail.com>
test-benchmark-buffer is consistently timing out on a single Windows
host in CI. Rather than try to figure out if we need to scale the
timeout up for a certain memory limit or chip speed or something else,
let's increase the timeout for benchmark tests in general.
PR-URL: https://github.com/nodejs/node/pull/40684
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Voltrex <mohammadkeyvanzade94@gmail.com>
`output.diagnostic` is a list that is appended to on SmartOS when
retrying a test due to `ECONNREFUSED`. The test runner checks if
`output.diagnostic` is truthy and, if so, assigns its value to
`self.traceback`. However `self.traceback` is supposed to be a string,
and `_printDiagnostic()` in the `TapProgressIndicator` attempts to call
`splitlines()` on it, which fails if it is a list with:
AttributeError: 'list' object has no attribute 'splitlines'
PR-URL: https://github.com/nodejs/node/pull/38289
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Christian Clauss <cclauss@me.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
It's possible to annotate failures in Actions by printing
"::error file={},line={},col={}::{message}". This methos is preferrable
over using a problem matcher because problem matchers only allow
single-line messages, whereas ::error allows multi-line messages.
PR-URL: https://github.com/nodejs/node/pull/34590
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
This fixes the following crash on Windows for me. I don't know
why this I only started to see this now, but anyway, the new timeout
value is still longer than a week and a half.
File "tools/test.py", line 1725, in <module>
sys.exit(Main())
File "tools/test.py", line 1701, in Main
if RunTestCases(cases_to_run, options.progress, \
options.j, options.flaky_tests):
File "tools/test.py", line 923, in RunTestCases
return progress.Run(tasks)
File "tools/test.py", line 145, in Run
thread.join(timeout=10000000)
File "C:\Users\anna\AppData\Local\Programs\Python\Python38-32\ \
lib\threading.py", line 1015, in join
self._wait_for_tstate_lock(timeout=max(timeout, 0))
File "C:\Users\anna\AppData\Local\Programs\Python\Python38-32\ \
lib\threading.py", line 1027, in _wait_for_tstate_lock
elif lock.acquire(block, timeout):
OverflowError: timeout value is too large
PR-URL: https://github.com/nodejs/node/pull/32868
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Bartosz Sosnowski <bartosz@janeasystems.com>
Add an embedder cctest that also covers a multi-Environment situation,
including worker_threads-style inspector support.
Co-authored-by: Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/30467
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
A logfile must be opened as a text file and encoding is specified.
Then it can accept a string. Current behaviour we get,
Message: 'ok 2834 sequential/test-worker-prof'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
File "/usr/lib64/python3.7/logging/__init__.py", line 1037, in emit
stream.write(msg + self.terminator)
TypeError: a bytes-like object is required, not 'str'
Call stack:
File "tools/test.py", line 1734, in <module>
sys.exit(Main())
File "tools/test.py", line 1710, in Main
if RunTestCases(cases_to_run, options.progress, options.j, options.flaky_tests):
File "tools/test.py", line 933, in RunTestCases
return progress.Run(tasks)
File "tools/test.py", line 141, in Run
self.RunSingle(False, 0)
File "tools/test.py", line 202, in RunSingle
self.HasRun(output)
File "tools/test.py", line 365, in HasRun
logger.info(' ---')
PR-URL: https://github.com/nodejs/node/pull/31156
Reviewed-By: Christian Clauss <cclauss@me.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>
Explicitly open files with utf8 encoding, otherwise the system could use
another encoding such as latin1 by default.
PR-URL: https://github.com/nodejs/node/pull/30023
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Christian Clauss <cclauss@me.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Tests can leave processes running blocking the tmpdir. This does not
yet prevent tests from doing that, but prevents failures on
subsequent tests.
PR-URL: https://github.com/nodejs/node/pull/28858
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Some pty tests persistently hung on the AIX CI buildbots. Fix that by
adding a helper script that properly sets up the pty before spawning
the script under test.
On investigation I discovered that the test runner hung when it tried
to close the slave pty's file descriptor, probably due to a bug in
AIX's pty implementation. I could reproduce it with a short C program.
The test runner also leaked file descriptors to the child process.
I couldn't convince python's `subprocess.Popen()` to do what I wanted
it to do so I opted to move the logic to a helper script that can do
fork/setsid/etc. without having to worry about stomping on state in
tools/test.py.
In the process I also uncovered some bugs in the pty module of the
python distro that ships with macOS 10.14, leading me to reimplement
a sizable chunk of the functionality of that module.
And last but not least, of course there are differences between ptys
on different platforms and the helper script has to paper over that.
Of course.
Really, this commit took me longer to put together than I care to admit.
Caveat emptor: this commit takes the hacky ^D feeding to the slave out
of tools/test.py and puts it in the *.in input files. You can also feed
other control characters to tests, like ^C or ^Z, simply by inserting
them into the corresponding input file. I think that's nice.
Fixes: https://github.com/nodejs/build/issues/1820
Fixes: https://github.com/nodejs/node/issues/28489
PR-URL: https://github.com/nodejs/node/pull/28600
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
This fixes the necessary plumbing to make the --shell argument
have its intended effect.
PR-URL: https://github.com/nodejs/node/pull/26449
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
This way we can specify a custom path for the test folder, e.g.
when building addons separately from the source tree.
PR-URL: https://github.com/nodejs/node/pull/26093
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Instead of using process.config.variables.v8_enable_inspector
to detect whether inspector is enabled in the build.
PR-URL: https://github.com/nodejs/node/pull/25819
Refs: https://github.com/nodejs/node/issues/25343
Reviewed-By: Eugene Ostroukhov <eostroukhov@google.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Enable marking of coverage tests so that we can
allow some tests to fail without blocking the generation
of coverage data. This will later allow us to
fail the coverage job if other kinds of errors occur and
to capture which tests we believe are not running properly
with coverage enabled.
PR-URL: https://github.com/nodejs/node/pull/25671
Reviewed-By: Ben Coe <bencoe@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Having an experimental feature behind a flag makes change
if we are expecting significant breaking changes to its API.
Since the Worker API has been essentially stable since
its initial introduction, and no noticeable doubt about
possibly not keeping the feature around has been voiced,
removing the flag and thereby reducing the barrier to experimentation,
and consequently receiving feedback on the implementation,
seems like a good idea.
PR-URL: https://github.com/nodejs/node/pull/25361
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Yuta Hiroto <hello@hiroppy.me>
Reviewed-By: Shingo Inoue <leko.noor@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Masashi Hirano <shisama07@gmail.com>
Reviewed-By: Weijia Wang <starkwang@126.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Some tests which create files and check file permissions assume the
umask is compatible with 022, and break when set to something like 007.
Explicitly set umask to 022
PR-URL: https://github.com/nodejs/node/pull/25213
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
