Mirrors/servo
1
0
Fork 0
mirror of https://github.com/servo/servo.git synced 2025-08-14 18:05:36 +01:00
Code Issues Projects Releases Packages Wiki Activity

Script: Implement TextDecoderStream (#38112)

Browse source 
This PR implements the `TextDecoderStream`. Other than introducing the
necessary mod and webidl files corresponding to `TextDecoderStream`,
this PR also involves some changes in `TextDecoder` and
`TrasnformStream`:

- The common part that can be shared between `TextDecoder` and
`TextDecoderStream` are extracted into a separate type
`script::dom::textdecodercommon::TextDecoderCommon`. This type could
probably use a different name because there is an interface called
`TextDecoderCommon` in the spec
(https://encoding.spec.whatwg.org/#textdecodercommon) which just gets
included in `TextDecoder` and `TextDecoderStream`.
- The three algorithms in `TransformStream` (`cancel`, `flush`, and
`transform`) all have become `enum` that has a `Js` variant for a JS
function object and a `Native` variant for a rust trait object. Whether
the cancel algorithm needs this enum type is debatable as I did not find
any interface in the spec that explicitly sets the cancel algorithm.

Testing: Existing WPT tests `tests/wpt/tests/encoding/stream` should be
sufficient
Fixes: #37723

---------

Signed-off-by: minghuaw <michael.wu1107@gmail.com>
Signed-off-by: minghuaw <wuminghua7@huawei.com>
Signed-off-by: Minghua Wu <michael.wu1107@gmail.com>
This commit is contained in:
minghuaw 2025-07-29 12:18:15 +08:00 • committed by GitHub
parent 25822920cf
commit 554b2da1ad
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
25 changed files with 797 additions and 752 deletions
Download patch file Download diff file Expand all files Collapse all files

54
tests/wpt/meta/encoding/idlharness.any.js.ini vendored
View file

@ -5,33 +5,6 @@
expected: ERROR
[idlharness.any.html]
[TextDecoderStream interface: existence and properties of interface object]
expected: FAIL
[TextDecoderStream interface object length]
expected: FAIL
[TextDecoderStream interface object name]
expected: FAIL
[TextDecoderStream interface: existence and properties of interface prototype object]
expected: FAIL
[TextDecoderStream interface: existence and properties of interface prototype object's "constructor" property]
expected: FAIL
[TextDecoderStream interface: existence and properties of interface prototype object's @@unscopables property]
expected: FAIL
[TextDecoderStream interface: attribute encoding]
expected: FAIL
[TextDecoderStream interface: attribute fatal]
expected: FAIL
[TextDecoderStream interface: attribute ignoreBOM]
expected: FAIL
[TextEncoderStream interface: existence and properties of interface object]
expected: FAIL
@ -55,33 +28,6 @@
[idlharness.any.worker.html]
[TextDecoderStream interface: existence and properties of interface object]
expected: FAIL
[TextDecoderStream interface object length]
expected: FAIL
[TextDecoderStream interface object name]
expected: FAIL
[TextDecoderStream interface: existence and properties of interface prototype object]
expected: FAIL
[TextDecoderStream interface: existence and properties of interface prototype object's "constructor" property]
expected: FAIL
[TextDecoderStream interface: existence and properties of interface prototype object's @@unscopables property]
expected: FAIL
[TextDecoderStream interface: attribute encoding]
expected: FAIL
[TextDecoderStream interface: attribute fatal]
expected: FAIL
[TextDecoderStream interface: attribute ignoreBOM]
expected: FAIL
[TextEncoderStream interface: existence and properties of interface object]
expected: FAIL

8
tests/wpt/meta/encoding/legacy-mb-schinese/gb18030/gb18030-decoder.any.js.ini vendored
View file

@ -1,8 +0,0 @@
[gb18030-decoder.any.worker.html]
[gb18030 decoder: two bytes 0xFE 0xFF]
expected: FAIL
[gb18030-decoder.any.html]
[gb18030 decoder: two bytes 0xFE 0xFF]
expected: FAIL

12
tests/wpt/meta/encoding/streams/backpressure.any.js.ini vendored
View file

@ -1,10 +1,4 @@
[backpressure.any.worker.html]
[write() should not complete until read relieves backpressure for TextDecoderStream]
expected: FAIL
[additional writes should wait for backpressure to be relieved for class TextDecoderStream]
expected: FAIL
[write() should not complete until read relieves backpressure for TextEncoderStream]
expected: FAIL
@ -19,12 +13,6 @@
expected: ERROR
[backpressure.any.html]
[write() should not complete until read relieves backpressure for TextDecoderStream]
expected: FAIL
[additional writes should wait for backpressure to be relieved for class TextDecoderStream]
expected: FAIL
[write() should not complete until read relieves backpressure for TextEncoderStream]
expected: FAIL

168
tests/wpt/meta/encoding/streams/decode-attributes.any.js.ini vendored
View file

@ -2,179 +2,11 @@
expected: ERROR
[decode-attributes.any.html]
[encoding attribute should have correct value for 'unicode-1-1-utf-8']
expected: FAIL
[encoding attribute should have correct value for 'iso-8859-2']
expected: FAIL
[encoding attribute should have correct value for 'ascii']
expected: FAIL
[encoding attribute should have correct value for 'utf-16']
expected: FAIL
[setting fatal to 'false' should set the attribute to false]
expected: FAIL
[setting ignoreBOM to 'false' should set the attribute to false]
expected: FAIL
[setting fatal to '0' should set the attribute to false]
expected: FAIL
[setting ignoreBOM to '0' should set the attribute to false]
expected: FAIL
[setting fatal to '' should set the attribute to false]
expected: FAIL
[setting ignoreBOM to '' should set the attribute to false]
expected: FAIL
[setting fatal to 'undefined' should set the attribute to false]
expected: FAIL
[setting ignoreBOM to 'undefined' should set the attribute to false]
expected: FAIL
[setting fatal to 'null' should set the attribute to false]
expected: FAIL
[setting ignoreBOM to 'null' should set the attribute to false]
expected: FAIL
[setting fatal to 'true' should set the attribute to true]
expected: FAIL
[setting ignoreBOM to 'true' should set the attribute to true]
expected: FAIL
[setting fatal to '1' should set the attribute to true]
expected: FAIL
[setting ignoreBOM to '1' should set the attribute to true]
expected: FAIL
[setting fatal to '[object Object\]' should set the attribute to true]
expected: FAIL
[setting ignoreBOM to '[object Object\]' should set the attribute to true]
expected: FAIL
[setting fatal to '' should set the attribute to true]
expected: FAIL
[setting ignoreBOM to '' should set the attribute to true]
expected: FAIL
[setting fatal to 'yes' should set the attribute to true]
expected: FAIL
[setting ignoreBOM to 'yes' should set the attribute to true]
expected: FAIL
[constructing with an invalid encoding should throw]
expected: FAIL
[constructing with a non-stringifiable encoding should throw]
expected: FAIL
[a throwing fatal member should cause the constructor to throw]
expected: FAIL
[a throwing ignoreBOM member should cause the constructor to throw]
expected: FAIL
[decode-attributes.any.serviceworker.html]
expected: ERROR
[decode-attributes.any.worker.html]
[encoding attribute should have correct value for 'unicode-1-1-utf-8']
expected: FAIL
[encoding attribute should have correct value for 'iso-8859-2']
expected: FAIL
[encoding attribute should have correct value for 'ascii']
expected: FAIL
[encoding attribute should have correct value for 'utf-16']
expected: FAIL
[setting fatal to 'false' should set the attribute to false]
expected: FAIL
[setting ignoreBOM to 'false' should set the attribute to false]
expected: FAIL
[setting fatal to '0' should set the attribute to false]
expected: FAIL
[setting ignoreBOM to '0' should set the attribute to false]
expected: FAIL
[setting fatal to '' should set the attribute to false]
expected: FAIL
[setting ignoreBOM to '' should set the attribute to false]
expected: FAIL
[setting fatal to 'undefined' should set the attribute to false]
expected: FAIL
[setting ignoreBOM to 'undefined' should set the attribute to false]
expected: FAIL
[setting fatal to 'null' should set the attribute to false]
expected: FAIL
[setting ignoreBOM to 'null' should set the attribute to false]
expected: FAIL
[setting fatal to 'true' should set the attribute to true]
expected: FAIL
[setting ignoreBOM to 'true' should set the attribute to true]
expected: FAIL
[setting fatal to '1' should set the attribute to true]
expected: FAIL
[setting ignoreBOM to '1' should set the attribute to true]
expected: FAIL
[setting fatal to '[object Object\]' should set the attribute to true]
expected: FAIL
[setting ignoreBOM to '[object Object\]' should set the attribute to true]
expected: FAIL
[setting fatal to '' should set the attribute to true]
expected: FAIL
[setting ignoreBOM to '' should set the attribute to true]
expected: FAIL
[setting fatal to 'yes' should set the attribute to true]
expected: FAIL
[setting ignoreBOM to 'yes' should set the attribute to true]
expected: FAIL
[constructing with an invalid encoding should throw]
expected: FAIL
[constructing with a non-stringifiable encoding should throw]
expected: FAIL
[a throwing fatal member should cause the constructor to throw]
expected: FAIL
[a throwing ignoreBOM member should cause the constructor to throw]
expected: FAIL
[decode-attributes.any.shadowrealm.html]
expected: TIMEOUT

30
tests/wpt/meta/encoding/streams/decode-bad-chunks.any.js.ini vendored
View file

@ -1,36 +1,6 @@
[decode-bad-chunks.any.html]
[chunk of type undefined should error the stream]
expected: FAIL
[chunk of type null should error the stream]
expected: FAIL
[chunk of type numeric should error the stream]
expected: FAIL
[chunk of type object, not BufferSource should error the stream]
expected: FAIL
[chunk of type array should error the stream]
expected: FAIL
[decode-bad-chunks.any.worker.html]
[chunk of type undefined should error the stream]
expected: FAIL
[chunk of type null should error the stream]
expected: FAIL
[chunk of type numeric should error the stream]
expected: FAIL
[chunk of type object, not BufferSource should error the stream]
expected: FAIL
[chunk of type array should error the stream]
expected: FAIL
[decode-bad-chunks.any.sharedworker.html]
expected: ERROR

71
tests/wpt/meta/encoding/streams/decode-ignore-bom.any.js.ini vendored
View file

@ -2,79 +2,8 @@
expected: ERROR
[decode-ignore-bom.any.html]
[ignoreBOM should work for encoding utf-8, split at character 0]
expected: FAIL
[ignoreBOM should work for encoding utf-8, split at character 1]
expected: FAIL
[ignoreBOM should work for encoding utf-8, split at character 2]
expected: FAIL
[ignoreBOM should work for encoding utf-8, split at character 3]
expected: FAIL
[ignoreBOM should work for encoding utf-16le, split at character 0]
expected: FAIL
[ignoreBOM should work for encoding utf-16le, split at character 1]
expected: FAIL
[ignoreBOM should work for encoding utf-16le, split at character 2]
expected: FAIL
[ignoreBOM should work for encoding utf-16le, split at character 3]
expected: FAIL
[ignoreBOM should work for encoding utf-16be, split at character 0]
expected: FAIL
[ignoreBOM should work for encoding utf-16be, split at character 1]
expected: FAIL
[ignoreBOM should work for encoding utf-16be, split at character 2]
expected: FAIL
[ignoreBOM should work for encoding utf-16be, split at character 3]
expected: FAIL
[decode-ignore-bom.any.serviceworker.html]
expected: ERROR
[decode-ignore-bom.any.worker.html]
[ignoreBOM should work for encoding utf-8, split at character 0]
expected: FAIL
[ignoreBOM should work for encoding utf-8, split at character 1]
expected: FAIL
[ignoreBOM should work for encoding utf-8, split at character 2]
expected: FAIL
[ignoreBOM should work for encoding utf-8, split at character 3]
expected: FAIL
[ignoreBOM should work for encoding utf-16le, split at character 0]
expected: FAIL
[ignoreBOM should work for encoding utf-16le, split at character 1]
expected: FAIL
[ignoreBOM should work for encoding utf-16le, split at character 2]
expected: FAIL
[ignoreBOM should work for encoding utf-16le, split at character 3]
expected: FAIL
[ignoreBOM should work for encoding utf-16be, split at character 0]
expected: FAIL
[ignoreBOM should work for encoding utf-16be, split at character 1]
expected: FAIL
[ignoreBOM should work for encoding utf-16be, split at character 2]
expected: FAIL
[ignoreBOM should work for encoding utf-16be, split at character 3]
expected: FAIL

12
tests/wpt/meta/encoding/streams/decode-incomplete-input.any.js.ini vendored
View file

@ -1,21 +1,9 @@
[decode-incomplete-input.any.html]
[incomplete input with error mode "replacement" should end with a replacement character]
expected: FAIL
[incomplete input with error mode "fatal" should error the stream]
expected: FAIL
[decode-incomplete-input.any.sharedworker.html]
expected: ERROR
[decode-incomplete-input.any.worker.html]
[incomplete input with error mode "replacement" should end with a replacement character]
expected: FAIL
[incomplete input with error mode "fatal" should error the stream]
expected: FAIL
[decode-incomplete-input.any.serviceworker.html]
expected: ERROR

78
tests/wpt/meta/encoding/streams/decode-non-utf8.any.js.ini vendored
View file

@ -2,86 +2,8 @@
expected: ERROR
[decode-non-utf8.any.worker.html]
[TextDecoderStream should be able to reject invalid sequences in Shift_JIS]
expected: FAIL
[TextDecoderStream should be able to decode invalid sequences in Shift_JIS]
expected: FAIL
[TextDecoderStream should be able to reject invalid sequences in UTF-16LE]
expected: FAIL
[TextDecoderStream should be able to reject invalid sequences in UTF-16BE]
expected: FAIL
[TextDecoderStream should be able to decode invalid sequences in ISO-2022-JP]
expected: FAIL
[TextDecoderStream should be able to decode ISO-2022-JP]
expected: FAIL
[TextDecoderStream should be able to reject invalid sequences in ISO-2022-JP]
expected: FAIL
[TextDecoderStream should be able to decode Shift_JIS]
expected: FAIL
[TextDecoderStream should be able to decode invalid sequences in UTF-16LE]
expected: FAIL
[TextDecoderStream should be able to decode UTF-16BE]
expected: FAIL
[TextDecoderStream should be able to decode invalid sequences in UTF-16BE]
expected: FAIL
[TextDecoderStream should be able to decode UTF-16LE]
expected: FAIL
[TextDecoderStream should be able to decode ISO-8859-14]
expected: FAIL
[decode-non-utf8.any.html]
[TextDecoderStream should be able to reject invalid sequences in Shift_JIS]
expected: FAIL
[TextDecoderStream should be able to decode invalid sequences in Shift_JIS]
expected: FAIL
[TextDecoderStream should be able to reject invalid sequences in UTF-16LE]
expected: FAIL
[TextDecoderStream should be able to reject invalid sequences in UTF-16BE]
expected: FAIL
[TextDecoderStream should be able to decode invalid sequences in ISO-2022-JP]
expected: FAIL
[TextDecoderStream should be able to decode ISO-2022-JP]
expected: FAIL
[TextDecoderStream should be able to reject invalid sequences in ISO-2022-JP]
expected: FAIL
[TextDecoderStream should be able to decode Shift_JIS]
expected: FAIL
[TextDecoderStream should be able to decode invalid sequences in UTF-16LE]
expected: FAIL
[TextDecoderStream should be able to decode UTF-16BE]
expected: FAIL
[TextDecoderStream should be able to decode invalid sequences in UTF-16BE]
expected: FAIL
[TextDecoderStream should be able to decode UTF-16LE]
expected: FAIL
[TextDecoderStream should be able to decode ISO-8859-14]
expected: FAIL
[decode-non-utf8.any.serviceworker.html]
expected: ERROR

65
tests/wpt/meta/encoding/streams/decode-split-character.any.js.ini vendored
View file

@ -5,70 +5,5 @@
expected: ERROR
[decode-split-character.any.worker.html]
[a code point split between chunks should not be emitted until all bytes are available; split point = 2]
expected: FAIL
[a code point split between chunks should not be emitted until all bytes are available; split point = 3]
expected: FAIL
[a code point split between chunks should not be emitted until all bytes are available; split point = 4]
expected: FAIL
[a code point split between chunks should not be emitted until all bytes are available; split point = 5]
expected: FAIL
[a code point should be emitted as soon as all bytes are available]
expected: FAIL
[an empty chunk inside a code point split between chunks should not change the output; split point = 1]
expected: FAIL
[an empty chunk inside a code point split between chunks should not change the output; split point = 2]
expected: FAIL
[an empty chunk inside a code point split between chunks should not change the output; split point = 3]
expected: FAIL
[an empty chunk inside a code point split between chunks should not change the output; split point = 4]
expected: FAIL
[an empty chunk inside a code point split between chunks should not change the output; split point = 5]
expected: FAIL
[an empty chunk inside a code point split between chunks should not change the output; split point = 6]
expected: FAIL
[decode-split-character.any.html]
[a code point split between chunks should not be emitted until all bytes are available; split point = 2]
expected: FAIL
[a code point split between chunks should not be emitted until all bytes are available; split point = 3]
expected: FAIL
[a code point split between chunks should not be emitted until all bytes are available; split point = 4]
expected: FAIL
[a code point split between chunks should not be emitted until all bytes are available; split point = 5]
expected: FAIL
[a code point should be emitted as soon as all bytes are available]
expected: FAIL
[an empty chunk inside a code point split between chunks should not change the output; split point = 1]
expected: FAIL
[an empty chunk inside a code point split between chunks should not change the output; split point = 2]
expected: FAIL
[an empty chunk inside a code point split between chunks should not change the output; split point = 3]
expected: FAIL
[an empty chunk inside a code point split between chunks should not change the output; split point = 4]
expected: FAIL
[an empty chunk inside a code point split between chunks should not change the output; split point = 5]
expected: FAIL
[an empty chunk inside a code point split between chunks should not change the output; split point = 6]
expected: FAIL

29
tests/wpt/meta/encoding/streams/decode-utf8.any.js.ini vendored
View file

@ -6,35 +6,6 @@
[decode-utf8.any.worker.html]
expected: ERROR
[decoding one UTF-8 chunk should give one output string - ArrayBuffer]
expected: FAIL
[decoding an empty chunk should give no output chunks - ArrayBuffer]
expected: FAIL
[an initial empty chunk should be ignored - ArrayBuffer]
expected: FAIL
[a trailing empty chunk should be ignored - ArrayBuffer]
expected: FAIL
[UTF-8 EOF handling - ArrayBuffer]
expected: FAIL
[decode-utf8.any.html]
expected: ERROR
[decoding one UTF-8 chunk should give one output string - ArrayBuffer]
expected: FAIL
[decoding an empty chunk should give no output chunks - ArrayBuffer]
expected: FAIL
[an initial empty chunk should be ignored - ArrayBuffer]
expected: FAIL
[a trailing empty chunk should be ignored - ArrayBuffer]
expected: FAIL
[UTF-8 EOF handling - ArrayBuffer]
expected: FAIL

6
tests/wpt/meta/encoding/streams/invalid-realm.window.js.ini vendored
View file

@ -1,12 +1,6 @@
[invalid-realm.window.html]
[TextDecoderStream: write in detached realm should succeed]
expected: FAIL
[TextEncoderStream: write in detached realm should succeed]
expected: FAIL
[TextEncoderStream: close in detached realm should succeed]
expected: FAIL
[TextDecoderStream: close in detached realm should succeed]
expected: FAIL

6
tests/wpt/meta/encoding/streams/readable-writable-properties.any.js.ini vendored
View file

@ -2,9 +2,6 @@
[TextEncoderStream readable and writable properties must pass brand checks]
expected: FAIL
[TextDecoderStream readable and writable properties must pass brand checks]
expected: FAIL
[readable-writable-properties.any.sharedworker.html]
expected: ERROR
@ -16,9 +13,6 @@
[TextEncoderStream readable and writable properties must pass brand checks]
expected: FAIL
[TextDecoderStream readable and writable properties must pass brand checks]
expected: FAIL
[readable-writable-properties.any.shadowrealm.html]
expected: TIMEOUT

21
tests/wpt/meta/encoding/streams/realms.window.js.ini vendored
View file

@ -13,24 +13,3 @@
[TypeError for unconvertable chunk should come from constructor realm of TextEncoderStream]
expected: FAIL
[a TextDecoderStream object should be associated with the realm the constructor came from]
expected: FAIL
[TextDecoderStream's readable and writable attributes should come from the same realm as the constructor definition]
expected: FAIL
[the result object when read is called after write should come from the same realm as the constructor of TextDecoderStream]
expected: FAIL
[the result object when write is called with a pending read should come from the same realm as the constructor of TextDecoderStream]
expected: FAIL
[TypeError for chunk with the wrong type should come from constructor realm of TextDecoderStream]
expected: FAIL
[TypeError for invalid chunk should come from constructor realm of TextDecoderStream]
expected: FAIL
[TypeError for incomplete input should come from constructor realm of TextDecoderStream]
expected: FAIL

4
tests/wpt/mozilla/meta/MANIFEST.json vendored
View file

@ -13725,14 +13725,14 @@
]
],
"interfaces.https.html": [
"641c5ba19d389390b7b51da7644f011b0c42f33a",
"efb780c382456b1fda514b33fe5b317c447fa09e",
[
null,
{}
]
],
"interfaces.worker.js": [
"463bfc25211014203a5094baae4b4e2d890bf9aa",
"a02605ff8e63db701c77d734037fcb33d76418d0",
[
"mozilla/interfaces.worker.html",
{}

1
tests/wpt/mozilla/tests/mozilla/interfaces.https.html vendored
View file

@ -326,6 +326,7 @@ test_interfaces([
"TextTrackCueList",
"TextTrackList",
"TextDecoder",
"TextDecoderStream",
"TextEncoder",
"TimeRanges",
"Touch",

1
tests/wpt/mozilla/tests/mozilla/interfaces.worker.js vendored
View file

@ -110,6 +110,7 @@ test_interfaces([
"SecurityPolicyViolationEvent",
"ServiceWorkerContainer",
"TextDecoder",
"TextDecoderStream",
"TextEncoder",
"TrustedHTML",
"TrustedScript",