Nodejs/node
1
0
Fork 0
mirror of https://github.com/nodejs/node.git synced 2024-11-21 10:59:27 +00:00
Code Issues Actions 25 Packages Projects Releases Wiki Activity
ee46d2297c
node/lib/internal/encoding.js

613 lines
17 KiB
JavaScript
Raw Normal View History

util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
'use strict';
// An implementation of the WHATWG Encoding Standard
// https://encoding.spec.whatwg.org
lib: flatten access to primordials Store all primordials as properties of the primordials object. Static functions are prefixed by the constructor's name and prototype methods are prefixed by the constructor's name followed by "Prototype". For example: primordials.Object.keys becomes primordials.ObjectKeys. PR-URL: https://github.com/nodejs/node/pull/30610 Refs: https://github.com/nodejs/node/issues/29766 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2019-11-22 17:04:46 +00:00
const {
util: add fast path for utf8 encoding Co-authored-by: Anna Henningsen <anna@addaleax.net> PR-URL: https://github.com/nodejs/node/pull/45412 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
2022-11-12 21:31:44 +00:00
Boolean,
lib: flatten access to primordials Store all primordials as properties of the primordials object. Static functions are prefixed by the constructor's name and prototype methods are prefixed by the constructor's name followed by "Prototype". For example: primordials.Object.keys becomes primordials.ObjectKeys. PR-URL: https://github.com/nodejs/node/pull/30610 Refs: https://github.com/nodejs/node/issues/29766 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2019-11-22 17:04:46 +00:00
ObjectDefineProperties,
ObjectGetOwnPropertyDescriptors,
lib: use null-prototype objects for property descriptors Refs: https://github.com/nodejs/node/pull/42921 PR-URL: https://github.com/nodejs/node/pull/43270 Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2022-06-03 08:23:58 +00:00
ObjectSetPrototypeOf,
ObjectValues,
lib: refactor to use more primordials in internal/encoding.js PR-URL: https://github.com/nodejs/node/pull/36480 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Pooja D P <Pooja.D.P@ibm.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2020-12-11 14:34:02 +00:00
SafeMap,
StringPrototypeSlice,
lib: replace Symbol global by the primordials Symbol PR-URL: https://github.com/nodejs/node/pull/30737 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2019-11-30 15:55:29 +00:00
Symbol,
lib: update Symbol.toStringTag by SymbolToStringTag primordial PR-URL: https://github.com/nodejs/node/pull/30908 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>
2019-12-11 20:08:38 +00:00
SymbolToStringTag,
lib: use remaining typed arrays from primordials PR-URL: https://github.com/nodejs/node/pull/35499 Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Ujjwal Sharma <ryzokuken@disroot.org> Reviewed-By: Shingo Inoue <leko.noor@gmail.com>
2020-09-28 14:56:54 +00:00
Uint8Array,
lib: flatten access to primordials Store all primordials as properties of the primordials object. Static functions are prefixed by the constructor's name and prototype methods are prefixed by the constructor's name followed by "Prototype". For example: primordials.Object.keys becomes primordials.ObjectKeys. PR-URL: https://github.com/nodejs/node/pull/30610 Refs: https://github.com/nodejs/node/issues/29766 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2019-11-22 17:04:46 +00:00
} = primordials;
lib: enforce the use of Object from primordials PR-URL: https://github.com/nodejs/node/pull/27146 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Gus Caplan <me@gus.host>
2019-04-09 07:55:53 +00:00
lib: port remaining errors to new system PR-URL: https://github.com/nodejs/node/pull/19137 Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2018-03-04 21:16:24 +00:00
const {
ERR_ENCODING_NOT_SUPPORTED,
ERR_INVALID_ARG_TYPE,
ERR_INVALID_THIS,
lib: add trailing commas to more internal files PR-URL: https://github.com/nodejs/node/pull/46811 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Debadree Chatterjee <debadree333@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2023-02-26 10:34:02 +00:00
ERR_NO_ICU,
lib: port remaining errors to new system PR-URL: https://github.com/nodejs/node/pull/19137 Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2018-03-04 21:16:24 +00:00
} = require('internal/errors').codes;
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
const kHandle = Symbol('handle');
const kFlags = Symbol('flags');
const kEncoding = Symbol('encoding');
const kDecoder = Symbol('decoder');
const kEncoder = Symbol('encoder');
util: add fast path for text-decoder fatal flag PR-URL: https://github.com/nodejs/node/pull/45803 Reviewed-By: Robert Nagy <ronagy@icloud.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Michael Dawson <midawson@redhat.com>
2022-12-15 23:11:07 +00:00
const kFatal = Symbol('kFatal');
util: add fast path for utf8 encoding Co-authored-by: Anna Henningsen <anna@addaleax.net> PR-URL: https://github.com/nodejs/node/pull/45412 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
2022-11-12 21:31:44 +00:00
const kUTF8FastPath = Symbol('kUTF8FastPath');
const kIgnoreBOM = Symbol('kIgnoreBOM');
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
const {
getConstructorOf,
lib: move kEnumerableProperty to internal/util PR-URL: https://github.com/nodejs/node/pull/41877 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-02-11 17:30:47 +00:00
customInspectSymbol: inspect,
lib: use `kEmptyObject` in various places PR-URL: https://github.com/nodejs/node/pull/43159 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-05-21 09:59:38 +00:00
kEmptyObject,
lib: move kEnumerableProperty to internal/util PR-URL: https://github.com/nodejs/node/pull/41877 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-02-11 17:30:47 +00:00
kEnumerableProperty,
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
} = require('internal/util');
const {
util: text decoding allows SharedArrayBuffer PR-URL: https://github.com/nodejs/node/pull/32203 Fixes: https://github.com/nodejs/node/issues/32199 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Gus Caplan <me@gus.host> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2020-03-11 16:01:14 +00:00
isAnyArrayBuffer,
util: add encodeInto to TextEncoder Add function encodeInto to TextEncoder, and add MessageChannel to the encodeInto.any.js test. Fixes: https://github.com/nodejs/node/issues/28851 Fixes: https://github.com/nodejs/node/issues/26904 Refs: https://github.com/nodejs/node/pull/28862 Co-authored-by: AtticusYang <yyongtai@163.com> PR-URL: https://github.com/nodejs/node/pull/29524 Reviewed-By: David Carlier <devnexen@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
2019-09-10 23:13:01 +00:00
isArrayBufferView,
lib: add trailing commas to more internal files PR-URL: https://github.com/nodejs/node/pull/46811 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Debadree Chatterjee <debadree333@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2023-02-26 10:34:02 +00:00
isUint8Array,
lib: expose all type checks from the internal types module Combine all type checks on the internal types module and do not use the types binding anywhere else anymore. This makes sure all of those checks exist when required. PR-URL: https://github.com/nodejs/node/pull/25149 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Denys Otrishko <shishugi@gmail.com>
2018-12-20 16:35:40 +00:00
} = require('internal/util/types');
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
lib: use `validateObject` Used the validateObject() validator to keep consistency instead of a custom validator which was only used to validate objects. Refs: #39595 PR-URL: https://github.com/nodejs/node/pull/39605 Refs: https://github.com/nodejs/node/pull/39595 Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Darshan Sen <raisinten@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
2021-07-31 10:50:19 +00:00
const {
validateString,
validateObject,
stream: implement `min` option for `ReadableStreamBYOBReader.read` PR-URL: https://github.com/nodejs/node/pull/50888 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
2024-01-04 11:37:44 +00:00
kValidateObjectAllowObjectsAndNull,
lib: use `validateObject` Used the validateObject() validator to keep consistency instead of a custom validator which was only used to validate objects. Refs: #39595 PR-URL: https://github.com/nodejs/node/pull/39605 Refs: https://github.com/nodejs/node/pull/39595 Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Darshan Sen <raisinten@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
2021-07-31 10:50:19 +00:00
} = require('internal/validators');
src: use AliasedUint32Array for encodeInto results Getting the buffer from a TypedArray created from the JS land incurs a copy. For encodeInto() results we can just use an AliasedArray and let the binding always own the store. PR-URL: https://github.com/nodejs/node/pull/46658 Reviewed-By: Darshan Sen <raisinten@gmail.com>
2023-02-11 18:25:00 +00:00
const binding = internalBinding('encoding_binding');
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
const {
util: add encodeInto to TextEncoder Add function encodeInto to TextEncoder, and add MessageChannel to the encodeInto.any.js test. Fixes: https://github.com/nodejs/node/issues/28851 Fixes: https://github.com/nodejs/node/issues/26904 Refs: https://github.com/nodejs/node/pull/28862 Co-authored-by: AtticusYang <yyongtai@163.com> PR-URL: https://github.com/nodejs/node/pull/29524 Reviewed-By: David Carlier <devnexen@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
2019-09-10 23:13:01 +00:00
encodeInto,
src: make AliasedBuffers in the binding data weak The binding data holds references to the AliasedBuffers directly from their wrappers which already ensures that the AliasedBuffers won't be accessed when the wrappers are GC'ed. So we can just make the global references to the AliasedBuffers weak. This way we can simply deserialize the typed arrays when deserialize the binding data and avoid the extra Object::Set() calls. It also eliminates the caveat in the JS land where aliased buffers must be dynamically read from the binding. PR-URL: https://github.com/nodejs/node/pull/47354 Refs: https://github.com/nodejs/node/issues/47353 Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
2023-04-20 03:28:35 +00:00
encodeIntoResults,
util: add fast path for utf8 encoding Co-authored-by: Anna Henningsen <anna@addaleax.net> PR-URL: https://github.com/nodejs/node/pull/45412 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
2022-11-12 21:31:44 +00:00
encodeUtf8String,
decodeUTF8,
src: use AliasedUint32Array for encodeInto results Getting the buffer from a TypedArray created from the JS land incurs a copy. For encodeInto() results we can just use an AliasedArray and let the binding always own the store. PR-URL: https://github.com/nodejs/node/pull/46658 Reviewed-By: Darshan Sen <raisinten@gmail.com>
2023-02-11 18:25:00 +00:00
} = binding;
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
lib: remove unnecessary lazy loading in `internal/encoding` PR-URL: https://github.com/nodejs/node/pull/45810 Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2022-12-12 15:17:53 +00:00
const { Buffer } = require('buffer');
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
lib: extract validation functions PR-URL: https://github.com/nodejs/node/pull/18421 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2018-01-28 18:25:56 +00:00
function validateEncoder(obj) {
if (obj == null || obj[kEncoder] !== true)
lib: port remaining errors to new system PR-URL: https://github.com/nodejs/node/pull/19137 Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2018-03-04 21:16:24 +00:00
throw new ERR_INVALID_THIS('TextEncoder');
lib: extract validation functions PR-URL: https://github.com/nodejs/node/pull/18421 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2018-01-28 18:25:56 +00:00
}
function validateDecoder(obj) {
if (obj == null || obj[kDecoder] !== true)
lib: port remaining errors to new system PR-URL: https://github.com/nodejs/node/pull/19137 Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2018-03-04 21:16:24 +00:00
throw new ERR_INVALID_THIS('TextDecoder');
lib: extract validation functions PR-URL: https://github.com/nodejs/node/pull/18421 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2018-01-28 18:25:56 +00:00
}
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
const CONVERTER_FLAGS_FLUSH = 0x1;
const CONVERTER_FLAGS_FATAL = 0x2;
const CONVERTER_FLAGS_IGNORE_BOM = 0x4;
const empty = new Uint8Array(0);
lib: refactor to use more primordials in internal/encoding.js PR-URL: https://github.com/nodejs/node/pull/36480 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Pooja D P <Pooja.D.P@ibm.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2020-12-11 14:34:02 +00:00
const encodings = new SafeMap([
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
['unicode-1-1-utf-8', 'utf-8'],
lib: update encoding sets in `WHATWG API` PR-URL: https://github.com/nodejs/node/pull/49610 Refs: https://encoding.spec.whatwg.org/#names-and-labels Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2023-09-23 12:20:39 +00:00
['unicode11utf8', 'utf-8'],
['unicode20utf8', 'utf-8'],
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
['utf8', 'utf-8'],
['utf-8', 'utf-8'],
lib: update encoding sets in `WHATWG API` PR-URL: https://github.com/nodejs/node/pull/49610 Refs: https://encoding.spec.whatwg.org/#names-and-labels Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2023-09-23 12:20:39 +00:00
['x-unicode20utf8', 'utf-8'],
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
['866', 'ibm866'],
['cp866', 'ibm866'],
['csibm866', 'ibm866'],
['ibm866', 'ibm866'],
['csisolatin2', 'iso-8859-2'],
['iso-8859-2', 'iso-8859-2'],
['iso-ir-101', 'iso-8859-2'],
['iso8859-2', 'iso-8859-2'],
['iso88592', 'iso-8859-2'],
['iso_8859-2', 'iso-8859-2'],
['iso_8859-2:1987', 'iso-8859-2'],
['l2', 'iso-8859-2'],
['latin2', 'iso-8859-2'],
['csisolatin3', 'iso-8859-3'],
['iso-8859-3', 'iso-8859-3'],
['iso-ir-109', 'iso-8859-3'],
['iso8859-3', 'iso-8859-3'],
['iso88593', 'iso-8859-3'],
['iso_8859-3', 'iso-8859-3'],
['iso_8859-3:1988', 'iso-8859-3'],
['l3', 'iso-8859-3'],
['latin3', 'iso-8859-3'],
['csisolatin4', 'iso-8859-4'],
['iso-8859-4', 'iso-8859-4'],
['iso-ir-110', 'iso-8859-4'],
['iso8859-4', 'iso-8859-4'],
['iso88594', 'iso-8859-4'],
['iso_8859-4', 'iso-8859-4'],
['iso_8859-4:1988', 'iso-8859-4'],
['l4', 'iso-8859-4'],
['latin4', 'iso-8859-4'],
['csisolatincyrillic', 'iso-8859-5'],
['cyrillic', 'iso-8859-5'],
['iso-8859-5', 'iso-8859-5'],
['iso-ir-144', 'iso-8859-5'],
['iso8859-5', 'iso-8859-5'],
['iso88595', 'iso-8859-5'],
['iso_8859-5', 'iso-8859-5'],
['iso_8859-5:1988', 'iso-8859-5'],
['arabic', 'iso-8859-6'],
['asmo-708', 'iso-8859-6'],
['csiso88596e', 'iso-8859-6'],
['csiso88596i', 'iso-8859-6'],
['csisolatinarabic', 'iso-8859-6'],
['ecma-114', 'iso-8859-6'],
['iso-8859-6', 'iso-8859-6'],
['iso-8859-6-e', 'iso-8859-6'],
['iso-8859-6-i', 'iso-8859-6'],
['iso-ir-127', 'iso-8859-6'],
['iso8859-6', 'iso-8859-6'],
['iso88596', 'iso-8859-6'],
['iso_8859-6', 'iso-8859-6'],
['iso_8859-6:1987', 'iso-8859-6'],
['csisolatingreek', 'iso-8859-7'],
['ecma-118', 'iso-8859-7'],
['elot_928', 'iso-8859-7'],
['greek', 'iso-8859-7'],
['greek8', 'iso-8859-7'],
['iso-8859-7', 'iso-8859-7'],
['iso-ir-126', 'iso-8859-7'],
['iso8859-7', 'iso-8859-7'],
['iso88597', 'iso-8859-7'],
['iso_8859-7', 'iso-8859-7'],
['iso_8859-7:1987', 'iso-8859-7'],
['sun_eu_greek', 'iso-8859-7'],
['csiso88598e', 'iso-8859-8'],
['csisolatinhebrew', 'iso-8859-8'],
['hebrew', 'iso-8859-8'],
['iso-8859-8', 'iso-8859-8'],
['iso-8859-8-e', 'iso-8859-8'],
['iso-ir-138', 'iso-8859-8'],
['iso8859-8', 'iso-8859-8'],
['iso88598', 'iso-8859-8'],
['iso_8859-8', 'iso-8859-8'],
['iso_8859-8:1988', 'iso-8859-8'],
['visual', 'iso-8859-8'],
['csiso88598i', 'iso-8859-8-i'],
['iso-8859-8-i', 'iso-8859-8-i'],
['logical', 'iso-8859-8-i'],
['csisolatin6', 'iso-8859-10'],
['iso-8859-10', 'iso-8859-10'],
['iso-ir-157', 'iso-8859-10'],
['iso8859-10', 'iso-8859-10'],
['iso885910', 'iso-8859-10'],
['l6', 'iso-8859-10'],
['latin6', 'iso-8859-10'],
['iso-8859-13', 'iso-8859-13'],
['iso8859-13', 'iso-8859-13'],
['iso885913', 'iso-8859-13'],
['iso-8859-14', 'iso-8859-14'],
['iso8859-14', 'iso-8859-14'],
['iso885914', 'iso-8859-14'],
['csisolatin9', 'iso-8859-15'],
['iso-8859-15', 'iso-8859-15'],
['iso8859-15', 'iso-8859-15'],
['iso885915', 'iso-8859-15'],
['iso_8859-15', 'iso-8859-15'],
['l9', 'iso-8859-15'],
lib: update encoding sets in `WHATWG API` PR-URL: https://github.com/nodejs/node/pull/49610 Refs: https://encoding.spec.whatwg.org/#names-and-labels Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2023-09-23 12:20:39 +00:00
['iso-8859-16', 'iso-8859-16'],
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
['cskoi8r', 'koi8-r'],
['koi', 'koi8-r'],
['koi8', 'koi8-r'],
['koi8-r', 'koi8-r'],
['koi8_r', 'koi8-r'],
['koi8-ru', 'koi8-u'],
['koi8-u', 'koi8-u'],
['csmacintosh', 'macintosh'],
['mac', 'macintosh'],
['macintosh', 'macintosh'],
['x-mac-roman', 'macintosh'],
['dos-874', 'windows-874'],
['iso-8859-11', 'windows-874'],
['iso8859-11', 'windows-874'],
['iso885911', 'windows-874'],
['tis-620', 'windows-874'],
['windows-874', 'windows-874'],
['cp1250', 'windows-1250'],
['windows-1250', 'windows-1250'],
['x-cp1250', 'windows-1250'],
['cp1251', 'windows-1251'],
['windows-1251', 'windows-1251'],
['x-cp1251', 'windows-1251'],
['ansi_x3.4-1968', 'windows-1252'],
['ascii', 'windows-1252'],
['cp1252', 'windows-1252'],
['cp819', 'windows-1252'],
['csisolatin1', 'windows-1252'],
['ibm819', 'windows-1252'],
['iso-8859-1', 'windows-1252'],
['iso-ir-100', 'windows-1252'],
['iso8859-1', 'windows-1252'],
['iso88591', 'windows-1252'],
['iso_8859-1', 'windows-1252'],
['iso_8859-1:1987', 'windows-1252'],
['l1', 'windows-1252'],
['latin1', 'windows-1252'],
['us-ascii', 'windows-1252'],
['windows-1252', 'windows-1252'],
['x-cp1252', 'windows-1252'],
['cp1253', 'windows-1253'],
['windows-1253', 'windows-1253'],
['x-cp1253', 'windows-1253'],
['cp1254', 'windows-1254'],
['csisolatin5', 'windows-1254'],
['iso-8859-9', 'windows-1254'],
['iso-ir-148', 'windows-1254'],
['iso8859-9', 'windows-1254'],
['iso88599', 'windows-1254'],
['iso_8859-9', 'windows-1254'],
['iso_8859-9:1989', 'windows-1254'],
['l5', 'windows-1254'],
['latin5', 'windows-1254'],
['windows-1254', 'windows-1254'],
['x-cp1254', 'windows-1254'],
['cp1255', 'windows-1255'],
['windows-1255', 'windows-1255'],
['x-cp1255', 'windows-1255'],
['cp1256', 'windows-1256'],
['windows-1256', 'windows-1256'],
['x-cp1256', 'windows-1256'],
['cp1257', 'windows-1257'],
['windows-1257', 'windows-1257'],
['x-cp1257', 'windows-1257'],
['cp1258', 'windows-1258'],
['windows-1258', 'windows-1258'],
['x-cp1258', 'windows-1258'],
['x-mac-cyrillic', 'x-mac-cyrillic'],
['x-mac-ukrainian', 'x-mac-cyrillic'],
['chinese', 'gbk'],
['csgb2312', 'gbk'],
['csiso58gb231280', 'gbk'],
['gb2312', 'gbk'],
['gb_2312', 'gbk'],
['gb_2312-80', 'gbk'],
['gbk', 'gbk'],
['iso-ir-58', 'gbk'],
['x-gbk', 'gbk'],
['gb18030', 'gb18030'],
['big5', 'big5'],
['big5-hkscs', 'big5'],
['cn-big5', 'big5'],
['csbig5', 'big5'],
['x-x-big5', 'big5'],
['cseucpkdfmtjapanese', 'euc-jp'],
['euc-jp', 'euc-jp'],
['x-euc-jp', 'euc-jp'],
['csiso2022jp', 'iso-2022-jp'],
['iso-2022-jp', 'iso-2022-jp'],
['csshiftjis', 'shift_jis'],
['ms932', 'shift_jis'],
['ms_kanji', 'shift_jis'],
['shift-jis', 'shift_jis'],
['shift_jis', 'shift_jis'],
['sjis', 'shift_jis'],
['windows-31j', 'shift_jis'],
['x-sjis', 'shift_jis'],
['cseuckr', 'euc-kr'],
['csksc56011987', 'euc-kr'],
['euc-kr', 'euc-kr'],
['iso-ir-149', 'euc-kr'],
['korean', 'euc-kr'],
['ks_c_5601-1987', 'euc-kr'],
['ks_c_5601-1989', 'euc-kr'],
['ksc5601', 'euc-kr'],
['ksc_5601', 'euc-kr'],
['windows-949', 'euc-kr'],
lib: update encoding sets in `WHATWG API` PR-URL: https://github.com/nodejs/node/pull/49610 Refs: https://encoding.spec.whatwg.org/#names-and-labels Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2023-09-23 12:20:39 +00:00
['csiso2022kr', 'replacement'],
['hz-gb-2312', 'replacement'],
['iso-2022-cn', 'replacement'],
['iso-2022-cn-ext', 'replacement'],
['iso-2022-kr', 'replacement'],
['replacement', 'replacement'],
['unicodefffe', 'utf-16be'],
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
['utf-16be', 'utf-16be'],
lib: update encoding sets in `WHATWG API` PR-URL: https://github.com/nodejs/node/pull/49610 Refs: https://encoding.spec.whatwg.org/#names-and-labels Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2023-09-23 12:20:39 +00:00
['csunicode', 'utf-16le'],
['iso-10646-ucs-2', 'utf-16le'],
['ucs-2', 'utf-16le'],
['unicode', 'utf-16le'],
['unicodefeff', 'utf-16le'],
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
['utf-16le', 'utf-16le'],
doc,lib: prepare for stricter multi-line array linting We're about to turn on a requirement for dangling commas. PR-URL: https://github.com/nodejs/node/pull/37088 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2021-02-06 13:36:58 +00:00
['utf-16', 'utf-16le'],
lib: update encoding sets in `WHATWG API` PR-URL: https://github.com/nodejs/node/pull/49610 Refs: https://encoding.spec.whatwg.org/#names-and-labels Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2023-09-23 12:20:39 +00:00
['x-user-defined', 'x-user-defined'],
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
]);
// Unfortunately, String.prototype.trim also removes non-ascii whitespace,
// so we have to do this manually
function trimAsciiWhitespace(label) {
lib: replace var with let/const PR-URL: https://github.com/nodejs/node/pull/30390 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2019-11-12 15:08:20 +00:00
let s = 0;
let e = label.length;
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
while (s < e && (
label[s] === '\u0009' ||
label[s] === '\u000a' ||
label[s] === '\u000c' ||
label[s] === '\u000d' ||
label[s] === '\u0020')) {
s++;
}
while (e > s && (
label[e - 1] === '\u0009' ||
label[e - 1] === '\u000a' ||
label[e - 1] === '\u000c' ||
label[e - 1] === '\u000d' ||
label[e - 1] === '\u0020')) {
e--;
}
lib: refactor to use more primordials in internal/encoding.js PR-URL: https://github.com/nodejs/node/pull/36480 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Pooja D P <Pooja.D.P@ibm.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2020-12-11 14:34:02 +00:00
return StringPrototypeSlice(label, s, e);
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
}
function getEncodingFromLabel(label) {
const enc = encodings.get(label);
if (enc !== undefined) return enc;
return encodings.get(trimAsciiWhitespace(label.toLowerCase()));
}
class TextEncoder {
constructor() {
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
this[kEncoder] = true;
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
}
get encoding() {
lib: extract validation functions PR-URL: https://github.com/nodejs/node/pull/18421 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2018-01-28 18:25:56 +00:00
validateEncoder(this);
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
return 'utf-8';
}
encode(input = '') {
lib: extract validation functions PR-URL: https://github.com/nodejs/node/pull/18421 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2018-01-28 18:25:56 +00:00
validateEncoder(this);
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
return encodeUtf8String(`${input}`);
}
util: add encodeInto to TextEncoder Add function encodeInto to TextEncoder, and add MessageChannel to the encodeInto.any.js test. Fixes: https://github.com/nodejs/node/issues/28851 Fixes: https://github.com/nodejs/node/issues/26904 Refs: https://github.com/nodejs/node/pull/28862 Co-authored-by: AtticusYang <yyongtai@163.com> PR-URL: https://github.com/nodejs/node/pull/29524 Reviewed-By: David Carlier <devnexen@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
2019-09-10 23:13:01 +00:00
encodeInto(src, dest) {
validateEncoder(this);
validateString(src, 'src');
if (!dest || !isUint8Array(dest))
throw new ERR_INVALID_ARG_TYPE('dest', 'Uint8Array', dest);
src: use AliasedUint32Array for encodeInto results Getting the buffer from a TypedArray created from the JS land incurs a copy. For encodeInto() results we can just use an AliasedArray and let the binding always own the store. PR-URL: https://github.com/nodejs/node/pull/46658 Reviewed-By: Darshan Sen <raisinten@gmail.com>
2023-02-11 18:25:00 +00:00
encodeInto(src, dest);
// We need to read from the binding here since the buffer gets refreshed
// from the snapshot.
src: make AliasedBuffers in the binding data weak The binding data holds references to the AliasedBuffers directly from their wrappers which already ensures that the AliasedBuffers won't be accessed when the wrappers are GC'ed. So we can just make the global references to the AliasedBuffers weak. This way we can simply deserialize the typed arrays when deserialize the binding data and avoid the extra Object::Set() calls. It also eliminates the caveat in the JS land where aliased buffers must be dynamically read from the binding. PR-URL: https://github.com/nodejs/node/pull/47354 Refs: https://github.com/nodejs/node/issues/47353 Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
2023-04-20 03:28:35 +00:00
const { 0: read, 1: written } = encodeIntoResults;
src: use AliasedUint32Array for encodeInto results Getting the buffer from a TypedArray created from the JS land incurs a copy. For encodeInto() results we can just use an AliasedArray and let the binding always own the store. PR-URL: https://github.com/nodejs/node/pull/46658 Reviewed-By: Darshan Sen <raisinten@gmail.com>
2023-02-11 18:25:00 +00:00
return { read, written };
util: add encodeInto to TextEncoder Add function encodeInto to TextEncoder, and add MessageChannel to the encodeInto.any.js test. Fixes: https://github.com/nodejs/node/issues/28851 Fixes: https://github.com/nodejs/node/issues/26904 Refs: https://github.com/nodejs/node/pull/28862 Co-authored-by: AtticusYang <yyongtai@163.com> PR-URL: https://github.com/nodejs/node/pull/29524 Reviewed-By: David Carlier <devnexen@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
2019-09-10 23:13:01 +00:00
}
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
[inspect](depth, opts) {
lib: extract validation functions PR-URL: https://github.com/nodejs/node/pull/18421 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2018-01-28 18:25:56 +00:00
validateEncoder(this);
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
if (typeof depth === 'number' && depth < 0)
lib: don't use `util.inspect()` internals This makes sure the internal `stylize` function is not used to render anything and instead just uses the regular inspect function in case of reaching the maximum depth level. PR-URL: https://github.com/nodejs/node/pull/24971 Refs: https://github.com/nodejs/node/issues/24765 Reviewed-By: Gus Caplan <me@gus.host> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com>
2018-12-11 21:42:34 +00:00
return this;
benchmark,lib: change var to const Refs: https://github.com/nodejs/node/pull/26679 PR-URL: https://github.com/nodejs/node/pull/26915 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Refael Ackermann <refack@gmail.com>
2019-03-26 04:21:27 +00:00
const ctor = getConstructorOf(this);
tools: add `prefer-proto` rule fixup: add support for `Object.create(null)` fixup: extend to any 1-argument Object.create call fixup: add tests PR-URL: https://github.com/nodejs/node/pull/46083 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com> Reviewed-By: Darshan Sen <raisinten@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-01-10 05:38:36 +00:00
const obj = { __proto__: {
lib: add trailing commas to more internal files PR-URL: https://github.com/nodejs/node/pull/46811 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Debadree Chatterjee <debadree333@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2023-02-26 10:34:02 +00:00
constructor: ctor === null ? TextEncoder : ctor,
tools: add `prefer-proto` rule fixup: add support for `Object.create(null)` fixup: extend to any 1-argument Object.create call fixup: add tests PR-URL: https://github.com/nodejs/node/pull/46083 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com> Reviewed-By: Darshan Sen <raisinten@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-01-10 05:38:36 +00:00
} };
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
obj.encoding = this.encoding;
// Lazy to avoid circular dependency
lib: remove usage of require('util') Remove the usage of `require('util').inspect`. PR-URL: https://github.com/nodejs/node/pull/26779 Refs: https://github.com/nodejs/node/issues/26546 Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Yongsheng Zhang <zyszys98@gmail.com>
2019-03-19 14:04:38 +00:00
return require('internal/util/inspect').inspect(obj, opts);
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
}
}
lib: flatten access to primordials Store all primordials as properties of the primordials object. Static functions are prefixed by the constructor's name and prototype methods are prefixed by the constructor's name followed by "Prototype". For example: primordials.Object.keys becomes primordials.ObjectKeys. PR-URL: https://github.com/nodejs/node/pull/30610 Refs: https://github.com/nodejs/node/issues/29766 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2019-11-22 17:04:46 +00:00
ObjectDefineProperties(
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
TextEncoder.prototype, {
lib: move kEnumerableProperty to internal/util PR-URL: https://github.com/nodejs/node/pull/41877 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-02-11 17:30:47 +00:00
'encode': kEnumerableProperty,
'encodeInto': kEnumerableProperty,
'encoding': kEnumerableProperty,
lib: use null-prototype objects for property descriptors Refs: https://github.com/nodejs/node/pull/42921 PR-URL: https://github.com/nodejs/node/pull/43270 Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2022-06-03 08:23:58 +00:00
[SymbolToStringTag]: { __proto__: null, configurable: true, value: 'TextEncoder' },
test,doc,lib: adjust object literal newlines for lint rule Before enabling object-curly-newline for our ESLint rules, adjust files to comply with it. Refs: https://eslint.org/docs/rules/object-curly-newline PR-URL: https://github.com/nodejs/node/pull/37040 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2021-01-23 19:46:29 +00:00
});
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
src: remove function hasTextDecoder in encoding.js also... return TextDecoder directly from factories PR-URL: https://github.com/nodejs/node/pull/23625 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
2018-10-12 18:32:55 +00:00
const TextDecoder =
src,lib: make process.binding('config') internal PR-URL: https://github.com/nodejs/node/pull/23400 Refs: https://github.com/nodejs/node/issues/22160 Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2018-10-08 23:10:07 +00:00
internalBinding('config').hasIntl ?
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
makeTextDecoderICU() :
makeTextDecoderJS();
function makeTextDecoderICU() {
const {
decode: _decode,
getConverter,
icu: make process.binding('icu') internal PR-URL: https://github.com/nodejs/node/pull/23234 Refs: https://github.com/nodejs/node/issues/22160 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Gus Caplan <me@gus.host>
2018-10-03 02:30:07 +00:00
} = internalBinding('icu');
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
class TextDecoder {
lib: use `kEmptyObject` in various places PR-URL: https://github.com/nodejs/node/pull/43159 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-05-21 09:59:38 +00:00
constructor(encoding = 'utf-8', options = kEmptyObject) {
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
encoding = `${encoding}`;
lib: reduce overhead of validateObject PR-URL: https://github.com/nodejs/node/pull/49928 Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-10-02 12:56:39 +00:00
validateObject(options, 'options', kValidateObjectAllowObjectsAndNull);
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
const enc = getEncodingFromLabel(encoding);
if (enc === undefined)
lib: port remaining errors to new system PR-URL: https://github.com/nodejs/node/pull/19137 Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2018-03-04 21:16:24 +00:00
throw new ERR_ENCODING_NOT_SUPPORTED(encoding);
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
lib: replace var with let/const PR-URL: https://github.com/nodejs/node/pull/30390 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2019-11-12 15:08:20 +00:00
let flags = 0;
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
if (options !== null) {
flags |= options.fatal ? CONVERTER_FLAGS_FATAL : 0;
flags |= options.ignoreBOM ? CONVERTER_FLAGS_IGNORE_BOM : 0;
}
this[kDecoder] = true;
this[kFlags] = flags;
this[kEncoding] = enc;
util: add fast path for utf8 encoding Co-authored-by: Anna Henningsen <anna@addaleax.net> PR-URL: https://github.com/nodejs/node/pull/45412 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
2022-11-12 21:31:44 +00:00
this[kIgnoreBOM] = Boolean(options?.ignoreBOM);
util: add fast path for text-decoder fatal flag PR-URL: https://github.com/nodejs/node/pull/45803 Reviewed-By: Robert Nagy <ronagy@icloud.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Michael Dawson <midawson@redhat.com>
2022-12-15 23:11:07 +00:00
this[kFatal] = Boolean(options?.fatal);
// Only support fast path for UTF-8.
this[kUTF8FastPath] = enc === 'utf-8';
util: add fast path for utf8 encoding Co-authored-by: Anna Henningsen <anna@addaleax.net> PR-URL: https://github.com/nodejs/node/pull/45412 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
2022-11-12 21:31:44 +00:00
this[kHandle] = undefined;
util: add fast path for text-decoder fatal flag PR-URL: https://github.com/nodejs/node/pull/45803 Reviewed-By: Robert Nagy <ronagy@icloud.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Michael Dawson <midawson@redhat.com>
2022-12-15 23:11:07 +00:00
if (!this[kUTF8FastPath]) {
util: add fast path for utf8 encoding Co-authored-by: Anna Henningsen <anna@addaleax.net> PR-URL: https://github.com/nodejs/node/pull/45412 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
2022-11-12 21:31:44 +00:00
this.#prepareConverter();
}
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
}
util: add fast path for utf8 encoding Co-authored-by: Anna Henningsen <anna@addaleax.net> PR-URL: https://github.com/nodejs/node/pull/45412 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
2022-11-12 21:31:44 +00:00
#prepareConverter() {
if (this[kHandle] !== undefined) return;
const handle = getConverter(this[kEncoding], this[kFlags]);
if (handle === undefined)
throw new ERR_ENCODING_NOT_SUPPORTED(this[kEncoding]);
this[kHandle] = handle;
}
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
lib: use `kEmptyObject` in various places PR-URL: https://github.com/nodejs/node/pull/43159 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-05-21 09:59:38 +00:00
decode(input = empty, options = kEmptyObject) {
lib: extract validation functions PR-URL: https://github.com/nodejs/node/pull/18421 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2018-01-28 18:25:56 +00:00
validateDecoder(this);
util: add fast path for utf8 encoding Co-authored-by: Anna Henningsen <anna@addaleax.net> PR-URL: https://github.com/nodejs/node/pull/45412 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
2022-11-12 21:31:44 +00:00
this[kUTF8FastPath] &&= !(options?.stream);
if (this[kUTF8FastPath]) {
util: add fast path for text-decoder fatal flag PR-URL: https://github.com/nodejs/node/pull/45803 Reviewed-By: Robert Nagy <ronagy@icloud.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Michael Dawson <midawson@redhat.com>
2022-12-15 23:11:07 +00:00
return decodeUTF8(input, this[kIgnoreBOM], this[kFatal]);
util: add fast path for utf8 encoding Co-authored-by: Anna Henningsen <anna@addaleax.net> PR-URL: https://github.com/nodejs/node/pull/45412 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
2022-11-12 21:31:44 +00:00
}
this.#prepareConverter();
lib: reduce overhead of validateObject PR-URL: https://github.com/nodejs/node/pull/49928 Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-10-02 12:56:39 +00:00
validateObject(options, 'options', kValidateObjectAllowObjectsAndNull);
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
lib: replace var with let/const PR-URL: https://github.com/nodejs/node/pull/30390 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2019-11-12 15:08:20 +00:00
let flags = 0;
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
if (options !== null)
flags |= options.stream ? 0 : CONVERTER_FLAGS_FLUSH;
util: improve text decoder performance PR-URL: https://github.com/nodejs/node/pull/45388 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>
2022-11-09 14:18:42 +00:00
return _decode(this[kHandle], input, flags, this.encoding);
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
}
}
src: remove function hasTextDecoder in encoding.js also... return TextDecoder directly from factories PR-URL: https://github.com/nodejs/node/pull/23625 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
2018-10-12 18:32:55 +00:00
return TextDecoder;
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
}
function makeTextDecoderJS() {
lib: replace var with let/const PR-URL: https://github.com/nodejs/node/pull/30390 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2019-11-12 15:08:20 +00:00
let StringDecoder;
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
function lazyStringDecoder() {
if (StringDecoder === undefined)
({ StringDecoder } = require('string_decoder'));
return StringDecoder;
}
const kBOMSeen = Symbol('BOM seen');
function hasConverter(encoding) {
return encoding === 'utf-8' || encoding === 'utf-16le';
}
class TextDecoder {
lib: use `kEmptyObject` in various places PR-URL: https://github.com/nodejs/node/pull/43159 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-05-21 09:59:38 +00:00
constructor(encoding = 'utf-8', options = kEmptyObject) {
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
encoding = `${encoding}`;
lib: reduce overhead of validateObject PR-URL: https://github.com/nodejs/node/pull/49928 Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-10-02 12:56:39 +00:00
validateObject(options, 'options', kValidateObjectAllowObjectsAndNull);
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
const enc = getEncodingFromLabel(encoding);
if (enc === undefined || !hasConverter(enc))
lib: port remaining errors to new system PR-URL: https://github.com/nodejs/node/pull/19137 Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2018-03-04 21:16:24 +00:00
throw new ERR_ENCODING_NOT_SUPPORTED(encoding);
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
lib: replace var with let/const PR-URL: https://github.com/nodejs/node/pull/30390 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2019-11-12 15:08:20 +00:00
let flags = 0;
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
if (options !== null) {
if (options.fatal) {
lib: port remaining errors to new system PR-URL: https://github.com/nodejs/node/pull/19137 Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2018-03-04 21:16:24 +00:00
throw new ERR_NO_ICU('"fatal" option');
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
}
flags |= options.ignoreBOM ? CONVERTER_FLAGS_IGNORE_BOM : 0;
}
this[kDecoder] = true;
// StringDecoder will normalize WHATWG encoding to Node.js encoding.
this[kHandle] = new (lazyStringDecoder())(enc);
this[kFlags] = flags;
this[kEncoding] = enc;
this[kBOMSeen] = false;
}
lib: use `kEmptyObject` in various places PR-URL: https://github.com/nodejs/node/pull/43159 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-05-21 09:59:38 +00:00
decode(input = empty, options = kEmptyObject) {
lib: extract validation functions PR-URL: https://github.com/nodejs/node/pull/18421 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2018-01-28 18:25:56 +00:00
validateDecoder(this);
util: text decoding allows SharedArrayBuffer PR-URL: https://github.com/nodejs/node/pull/32203 Fixes: https://github.com/nodejs/node/issues/32199 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Gus Caplan <me@gus.host> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2020-03-11 16:01:14 +00:00
if (isAnyArrayBuffer(input)) {
lib: fix TypeError when converting a detached buffer source PR-URL: https://github.com/nodejs/node/pull/44020 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-11-01 23:59:26 +00:00
try {
lib: remove unnecessary lazy loading in `internal/encoding` PR-URL: https://github.com/nodejs/node/pull/45810 Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2022-12-12 15:17:53 +00:00
input = Buffer.from(input);
lib: fix TypeError when converting a detached buffer source PR-URL: https://github.com/nodejs/node/pull/44020 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-11-01 23:59:26 +00:00
} catch {
input = empty;
}
lib: faster type checks for some types PR-URL: https://github.com/nodejs/node/pull/15663 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Benedikt Meurer <benedikt.meurer@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
2017-09-28 07:16:41 +00:00
} else if (isArrayBufferView(input)) {
lib: fix TypeError when converting a detached buffer source PR-URL: https://github.com/nodejs/node/pull/44020 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-11-01 23:59:26 +00:00
try {
lib: remove unnecessary lazy loading in `internal/encoding` PR-URL: https://github.com/nodejs/node/pull/45810 Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2022-12-12 15:17:53 +00:00
input = Buffer.from(input.buffer, input.byteOffset,
input.byteLength);
lib: fix TypeError when converting a detached buffer source PR-URL: https://github.com/nodejs/node/pull/44020 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-11-01 23:59:26 +00:00
} catch {
input = empty;
}
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
} else {
lib: port remaining errors to new system PR-URL: https://github.com/nodejs/node/pull/19137 Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2018-03-04 21:16:24 +00:00
throw new ERR_INVALID_ARG_TYPE('input',
lib: always show ERR_INVALID_ARG_TYPE received part This makes a effort to make sure all of these errors will actually also show the received input. On top of that it refactors a few tests for better maintainability. It will also change the returned type to always be a simple typeof instead of special handling null. PR-URL: https://github.com/nodejs/node/pull/19445 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>
2018-03-19 12:33:46 +00:00
['ArrayBuffer', 'ArrayBufferView'],
input);
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
}
lib: reduce overhead of validateObject PR-URL: https://github.com/nodejs/node/pull/49928 Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-10-02 12:56:39 +00:00
validateObject(options, 'options', kValidateObjectAllowObjectsAndNull);
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
if (this[kFlags] & CONVERTER_FLAGS_FLUSH) {
this[kBOMSeen] = false;
}
if (options !== null && options.stream) {
this[kFlags] &= ~CONVERTER_FLAGS_FLUSH;
} else {
this[kFlags] |= CONVERTER_FLAGS_FLUSH;
}
encoding: make TextDecoder handle BOM correctly Do not accept the BOM if it comes from a different encoding, and only discard the BOM after it has actually been read (including when it is spread over multiple chunks in streaming mode). Fixes: https://github.com/nodejs/node/issues/25315 PR-URL: https://github.com/nodejs/node/pull/30132 Reviewed-By: Gus Caplan <me@gus.host>
2019-10-26 14:27:51 +00:00
let result = this[kFlags] & CONVERTER_FLAGS_FLUSH ?
this[kHandle].end(input) :
this[kHandle].write(input);
if (result.length > 0 &&
!this[kBOMSeen] &&
!(this[kFlags] & CONVERTER_FLAGS_IGNORE_BOM)) {
// If the very first result in the stream is a BOM, and we are not
// explicitly told to ignore it, then we discard it.
if (result[0] === '\ufeff') {
lib: refactor to use more primordials in internal/encoding.js PR-URL: https://github.com/nodejs/node/pull/36480 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Pooja D P <Pooja.D.P@ibm.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2020-12-11 14:34:02 +00:00
result = StringPrototypeSlice(result, 1);
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
}
this[kBOMSeen] = true;
}
encoding: make TextDecoder handle BOM correctly Do not accept the BOM if it comes from a different encoding, and only discard the BOM after it has actually been read (including when it is spread over multiple chunks in streaming mode). Fixes: https://github.com/nodejs/node/issues/25315 PR-URL: https://github.com/nodejs/node/pull/30132 Reviewed-By: Gus Caplan <me@gus.host>
2019-10-26 14:27:51 +00:00
return result;
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
}
}
src: remove function hasTextDecoder in encoding.js also... return TextDecoder directly from factories PR-URL: https://github.com/nodejs/node/pull/23625 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
2018-10-12 18:32:55 +00:00
return TextDecoder;
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
}
// Mix in some shared properties.
lib: use null-prototype objects for property descriptors Refs: https://github.com/nodejs/node/pull/42921 PR-URL: https://github.com/nodejs/node/pull/43270 Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2022-06-03 08:23:58 +00:00
const sharedProperties = ObjectGetOwnPropertyDescriptors({
get encoding() {
validateDecoder(this);
return this[kEncoding];
},
get fatal() {
validateDecoder(this);
return (this[kFlags] & CONVERTER_FLAGS_FATAL) === CONVERTER_FLAGS_FATAL;
},
get ignoreBOM() {
validateDecoder(this);
return (this[kFlags] & CONVERTER_FLAGS_IGNORE_BOM) ===
util: fix inspection of class instance prototypes To achieve this, some internal custom inspect functions had to be changed. They relied upon the former behavior. Signed-off-by: Ruben Bridgewater <ruben@bridgewater.de> PR-URL: https://github.com/nodejs/node/pull/33449 Fixes: https://github.com/nodejs/node/issues/33419 Reviewed-By: Anto Aravinth <anto.aravinth.cse@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2020-05-17 17:07:04 +00:00
CONVERTER_FLAGS_IGNORE_BOM;
lib: use null-prototype objects for property descriptors Refs: https://github.com/nodejs/node/pull/42921 PR-URL: https://github.com/nodejs/node/pull/43270 Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2022-06-03 08:23:58 +00:00
},
util: fix inspection of class instance prototypes To achieve this, some internal custom inspect functions had to be changed. They relied upon the former behavior. Signed-off-by: Ruben Bridgewater <ruben@bridgewater.de> PR-URL: https://github.com/nodejs/node/pull/33449 Fixes: https://github.com/nodejs/node/issues/33419 Reviewed-By: Anto Aravinth <anto.aravinth.cse@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2020-05-17 17:07:04 +00:00
lib: use null-prototype objects for property descriptors Refs: https://github.com/nodejs/node/pull/42921 PR-URL: https://github.com/nodejs/node/pull/43270 Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2022-06-03 08:23:58 +00:00
[inspect](depth, opts) {
validateDecoder(this);
if (typeof depth === 'number' && depth < 0)
return this;
const constructor = getConstructorOf(this) || TextDecoder;
tools: add `prefer-proto` rule fixup: add support for `Object.create(null)` fixup: extend to any 1-argument Object.create call fixup: add tests PR-URL: https://github.com/nodejs/node/pull/46083 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com> Reviewed-By: Darshan Sen <raisinten@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-01-10 05:38:36 +00:00
const obj = { __proto__: { constructor } };
lib: use null-prototype objects for property descriptors Refs: https://github.com/nodejs/node/pull/42921 PR-URL: https://github.com/nodejs/node/pull/43270 Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2022-06-03 08:23:58 +00:00
obj.encoding = this.encoding;
obj.fatal = this.fatal;
obj.ignoreBOM = this.ignoreBOM;
if (opts.showHidden) {
obj[kFlags] = this[kFlags];
obj[kHandle] = this[kHandle];
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
}
lib: use null-prototype objects for property descriptors Refs: https://github.com/nodejs/node/pull/42921 PR-URL: https://github.com/nodejs/node/pull/43270 Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2022-06-03 08:23:58 +00:00
// Lazy to avoid circular dependency
const { inspect } = require('internal/util/inspect');
return `${constructor.name} ${inspect(obj)}`;
lib: add trailing commas to more internal files PR-URL: https://github.com/nodejs/node/pull/46811 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Debadree Chatterjee <debadree333@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2023-02-26 10:34:02 +00:00
},
lib: use null-prototype objects for property descriptors Refs: https://github.com/nodejs/node/pull/42921 PR-URL: https://github.com/nodejs/node/pull/43270 Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2022-06-03 08:23:58 +00:00
});
const propertiesValues = ObjectValues(sharedProperties);
for (let i = 0; i < propertiesValues.length; i++) {
// We want to use null-prototype objects to not rely on globally mutable
// %Object.prototype%.
ObjectSetPrototypeOf(propertiesValues[i], null);
}
sharedProperties[inspect].enumerable = false;
util: fix inspection of class instance prototypes To achieve this, some internal custom inspect functions had to be changed. They relied upon the former behavior. Signed-off-by: Ruben Bridgewater <ruben@bridgewater.de> PR-URL: https://github.com/nodejs/node/pull/33449 Fixes: https://github.com/nodejs/node/issues/33419 Reviewed-By: Anto Aravinth <anto.aravinth.cse@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2020-05-17 17:07:04 +00:00
ObjectDefineProperties(TextDecoder.prototype, {
lib: move kEnumerableProperty to internal/util PR-URL: https://github.com/nodejs/node/pull/41877 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-02-11 17:30:47 +00:00
decode: kEnumerableProperty,
lib: use null-prototype objects for property descriptors Refs: https://github.com/nodejs/node/pull/42921 PR-URL: https://github.com/nodejs/node/pull/43270 Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2022-06-03 08:23:58 +00:00
...sharedProperties,
util: fix inspection of class instance prototypes To achieve this, some internal custom inspect functions had to be changed. They relied upon the former behavior. Signed-off-by: Ruben Bridgewater <ruben@bridgewater.de> PR-URL: https://github.com/nodejs/node/pull/33449 Fixes: https://github.com/nodejs/node/issues/33419 Reviewed-By: Anto Aravinth <anto.aravinth.cse@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2020-05-17 17:07:04 +00:00
[SymbolToStringTag]: {
lib: use null-prototype objects for property descriptors Refs: https://github.com/nodejs/node/pull/42921 PR-URL: https://github.com/nodejs/node/pull/43270 Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com> Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2022-06-03 08:23:58 +00:00
__proto__: null,
util: fix inspection of class instance prototypes To achieve this, some internal custom inspect functions had to be changed. They relied upon the former behavior. Signed-off-by: Ruben Bridgewater <ruben@bridgewater.de> PR-URL: https://github.com/nodejs/node/pull/33449 Fixes: https://github.com/nodejs/node/issues/33419 Reviewed-By: Anto Aravinth <anto.aravinth.cse@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2020-05-17 17:07:04 +00:00
configurable: true,
lib: add trailing commas to more internal files PR-URL: https://github.com/nodejs/node/pull/46811 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Debadree Chatterjee <debadree333@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2023-02-26 10:34:02 +00:00
value: 'TextDecoder',
},
util: fix inspection of class instance prototypes To achieve this, some internal custom inspect functions had to be changed. They relied upon the former behavior. Signed-off-by: Ruben Bridgewater <ruben@bridgewater.de> PR-URL: https://github.com/nodejs/node/pull/33449 Fixes: https://github.com/nodejs/node/issues/33419 Reviewed-By: Anto Aravinth <anto.aravinth.cse@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2020-05-17 17:07:04 +00:00
});
encoding: rudimentary TextDecoder support w/o ICU Also split up the tests. PR-URL: https://github.com/nodejs/node/pull/14489 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2017-07-30 10:09:13 +00:00
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
module.exports = {
getEncodingFromLabel,
TextDecoder,
lib: add trailing commas to more internal files PR-URL: https://github.com/nodejs/node/pull/46811 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Debadree Chatterjee <debadree333@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2023-02-26 10:34:02 +00:00
TextEncoder,
util: implement WHATWG Encoding Standard API Provide an (initially experimental) implementation of the WHATWG Encoding Standard API (`TextDecoder` and `TextEncoder`). The is the same API implemented on the browser side. By default, with small-icu, only the UTF-8, UTF-16le and UTF-16be decoders are supported. With full-icu enabled, every encoding other than iso-8859-16 is supported. This provides a basic test, but does not include the full web platform tests. Note: many of the web platform tests for this would fail by default because we ship with small-icu by default. A process warning will be emitted on first use to indicate that the API is still experimental. No runtime flag is required to use the feature. Refs: https://encoding.spec.whatwg.org/ PR-URL: https://github.com/nodejs/node/pull/13644 Reviewed-By: Timothy Gu <timothygu99@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2017-06-12 15:25:53 +00:00
};
Reference in New Issue Copy Permalink