Mirrors/servo
1
0
Fork 0
mirror of https://github.com/servo/servo.git synced 2025-09-14 17:08:22 +01:00
Code Issues Projects Releases Packages Wiki Activity

Update web-platform-tests to revision 9ca57e052ba1b19fa3dd46c6aa656e8d529469a8

Browse source
This commit is contained in:
WPT Sync Bot 2020-09-12 08:20:19 +00:00
parent 68cb8f3d59
commit 75d6484415
1377 changed files with 31062 additions and 16983 deletions
Download patch file Download diff file Expand all files Collapse all files

175
tests/wpt/web-platform-tests/docs/writing-tests/idlharness.md
View file

@ -1,113 +1,86 @@
# idlharness.js API
# IDL Tests (idlharness.js)
## Introduction ##
`idlharness.js` automatically generates browser tests for WebIDL interfaces, using
the testharness.js framework. To use, first include the following:
`idlharness.js` generates tests for Web IDL fragments, using the
[JavaScript Tests (`testharness.js`)](testharness.md) infrastructure. You typically want to use
`.any.js` or `.window.js` for this to avoid having to write unnessary boilerplate.
```html
<script src=/resources/testharness.js></script>
<script src=/resources/testharnessreport.js></script>
<script src=/resources/WebIDLParser.js></script>
<script src=/resources/idlharness.js></script>
```
## Adding IDL fragments
Then you'll need some type of IDLs. Here's some script that can be run on a
spec written in HTML, which will grab all the elements with `class="idl"`,
concatenate them, and replace the body so you can copy-paste:
Web IDL is automatically scraped from specifications and added to the `/interfaces/` directory. See
the [README](https://github.com/web-platform-tests/wpt/blob/master/interfaces/README.md) there for
details.
## Testing IDL fragments
For example, the Fetch API's IDL is tested in
[`/fetch/api/idlharness.any.js`](https://github.com/web-platform-tests/wpt/blob/master/fetch/api/idlharness.any.js):
```js
var s = "";
[].forEach.call(document.getElementsByClassName("idl"), function(idl) {
//https://www.w3.org/Bugs/Public/show_bug.cgi?id=14914
if (!idl.classList.contains("extract"))
{
if (s !== "")
{
s += "\n\n";
// META: global=window,worker
// META: script=/resources/WebIDLParser.js
// META: script=/resources/idlharness.js
// META: timeout=long
idl_test(
['fetch'],
['referrer-policy', 'html', 'dom'],
idl_array => {
idl_array.add_objects({
Headers: ["new Headers()"],
Request: ["new Request('about:blank')"],
Response: ["new Response()"],
});
if (self.GLOBAL.isWindow()) {
idl_array.add_objects({ Window: ['window'] });
} else if (self.GLOBAL.isWorker()) {
idl_array.add_objects({ WorkerGlobalScope: ['self'] });
}
s += idl.textContent.replace(/ +$/mg, "");
}
});
document.body.innerHTML = '<pre></pre>';
document.body.firstChild.textContent = s;
);
```
Note how it includes `/resources/WebIDLParser.js` and `/resources/idlharness.js` in addition to
`testharness.js` and `testharnessreport.js` (automatically included due to usage of `.any.js`).
These are needed to make the `idl_test` function work.
Once you have that, put it in your script somehow. The easiest way is to
embed it literally in an HTML file with `<script type=text/plain>` or similar,
so that you don't have to do any escaping. Another possibility is to put it
in a separate .idl file that's fetched via XHR or similar. Sample usage:
The `idl_test` function takes three arguments:
```js
var idl_array = new IdlArray();
idl_array.add_untested_idls("interface Node { readonly attribute DOMString nodeName; };");
idl_array.add_idls("interface Document : Node { readonly attribute DOMString URL; };");
idl_array.add_objects({Document: ["document"]});
idl_array.test();
```
* _srcs_: a list of specifications whose IDL you want to test. The names here need to match the filenames (excluding the extension) in `/interfaces/`.
* _deps_: a list of specifications the IDL listed in _srcs_ depends upon. Be careful to list them in the order that the dependencies are revealed.
* _setup_func_: a function or async function that takes care of creating the various objects that you want to test.
This tests that `window.Document` exists and meets all the requirements of
WebIDL. It also tests that window.document (the result of evaluating the
string "document") has URL and nodeName properties that behave as they
should, and otherwise meets WebIDL's requirements for an object whose
primary interface is Document. It does not test that window.Node exists,
which is what you want if the Node interface is already tested in some other
specification's suite and your specification only extends or refers to it.
Of course, each IDL string can define many different things, and calls to
add_objects() can register many different objects for different interfaces:
this is a very simple example.
## Methods of `IdlArray` ##
## Public methods of IdlArray ##
IdlArray objects can be obtained with `new IdlArray()`. Anything not
documented in this section should be considered an implementation detail,
and outside callers should not use it.
### `add_idls(idl_string)`
Parses `idl_string` (throwing on parse error) and adds the results to the
IdlArray. All the definitions will be tested when you run test(). If
some of the definitions refer to other definitions, those must be present
too. For instance, if `idl_string` says that `Document` inherits from `Node`,
the `Node` interface must also have been provided in some call to `add_idls()`
or `add_untested_idls()`.
### `add_untested_idls(idl_string)`
Like `add_idls()`, but the definitions will not be tested. If an untested
interface is added and then extended with a tested partial interface, the
members of the partial interface will still be tested. Also, all the
members will still be tested for objects added with `add_objects()`, because
you probably want to test that (for instance) window.document has all the
properties from `Node`, not just `Document`, even if the `Node` interface itself
is tested in a different test suite.
`IdlArray` objects can be obtained through the _setup_func_ argument of `idl_test`. Anything not
documented in this section should be considered an implementation detail, and outside callers should
not use it.
### `add_objects(dict)`
`dict` should be an object whose keys are the names of interfaces or
exceptions, and whose values are arrays of strings. When an interface or
exception is tested, every string registered for it with `add_objects()`
will be evaluated, and tests will be run on the result to verify that it
correctly implements that interface or exception. This is the only way to
test anything about `[NoInterfaceObject]` interfaces, and there are many
tests that can't be run on any interface without an object to fiddle with.
The interface has to be the *primary* interface of all the objects
provided. For example, don't pass `{Node: ["document"]}`, but rather
`{Document: ["document"]}`. Assuming the `Document` interface was declared to
inherit from `Node`, this will automatically test that document implements
the `Node` interface too.
_dict_ should be an object whose keys are the names of interfaces or exceptions, and whose values
are arrays of strings. When an interface or exception is tested, every string registered for it
with `add_objects()` will be evaluated, and tests will be run on the result to verify that it
correctly implements that interface or exception. This is the only way to test anything about
`[NoInterfaceObject]` interfaces, and there are many tests that can't be run on any interface
without an object to fiddle with.
Warning: methods will be called on any provided objects, in a manner that
WebIDL requires be safe. For instance, if a method has mandatory
arguments, the test suite will try calling it with too few arguments to
see if it throws an exception. If an implementation incorrectly runs the
function instead of throwing, this might have side effects, possibly even
preventing the test suite from running correctly.
The interface has to be the *primary* interface of all the objects provided. For example, don't
pass `{Node: ["document"]}`, but rather `{Document: ["document"]}`. Assuming the `Document`
interface was declared to inherit from `Node`, this will automatically test that document implements
the `Node` interface too.
Warning: methods will be called on any provided objects, in a manner that WebIDL requires be safe.
For instance, if a method has mandatory arguments, the test suite will try calling it with too few
arguments to see if it throws an exception. If an implementation incorrectly runs the function
instead of throwing, this might have side effects, possibly even preventing the test suite from
running correctly.
### `prevent_multiple_testing(name)`
This is a niche method for use in case you're testing many objects that
implement the same interfaces, and don't want to retest the same
interfaces every single time. For instance, HTML defines many interfaces
that all inherit from `HTMLElement`, so the HTML test suite has something
like
This is a niche method for use in case you're testing many objects that implement the same
interfaces, and don't want to retest the same interfaces every single time. For instance, HTML
defines many interfaces that all inherit from `HTMLElement`, so the HTML test suite has something
like
```js
.add_objects({
@ -118,17 +91,11 @@ and outside callers should not use it.
})
```
and so on for dozens of element types. This would mean that it would
retest that each and every one of those elements implements `HTMLElement`,
`Element`, and `Node`, which would be thousands of basically redundant tests.
The test suite therefore calls `prevent_multiple_testing("HTMLElement")`.
This means that once one object has been tested to implement `HTMLElement`
and its ancestors, no other object will be. Thus in the example code
above, the harness would test that `document.documentElement` correctly
implements `HTMLHtmlElement`, `HTMLElement`, `Element`, and `Node`; but
`document.head` would only be tested for `HTMLHeadElement`, and so on for
further objects.
### `test()`
Run all tests. This should be called after you've called all other
methods to add IDLs and objects.
and so on for dozens of element types. This would mean that it would retest that each and every one
of those elements implements `HTMLElement`, `Element`, and `Node`, which would be thousands of
basically redundant tests. The test suite therefore calls `prevent_multiple_testing("HTMLElement")`.
This means that once one object has been tested to implement `HTMLElement` and its ancestors, no
other object will be. Thus in the example code above, the harness would test that
`document.documentElement` correctly implements `HTMLHtmlElement`, `HTMLElement`, `Element`, and
`Node`; but `document.head` would only be tested for `HTMLHeadElement`, and so on for further
objects.

8
tests/wpt/web-platform-tests/docs/writing-tests/server-pipes.md
View file

@ -91,6 +91,14 @@ whereas:
Will cause example.html to be returned with both text/html and
text/plain content-type headers.
If the comma (`,`) or closing parenthesis (`)`) characters appear in the header
value, those characters must be escaped with a backslash (`\`):
example?pipe=header(Expires,Thu\,%2014%20Aug%201986%2018:00:00%20GMT)
(Note that the programming environment from which the request is issued may
require that the backslash character itself be escaped.)
### `slice`
Used to send only part of a response body. Takes the start and,

35
tests/wpt/web-platform-tests/docs/writing-tests/testharness-api.md
View file

@ -185,7 +185,34 @@ assertions don't need to be wrapped in `step` or `step_func`
calls. However when mixing event handlers and `promise_test`, the
event handler callback functions *do* need to be wrapped since an
exception in these functions does not cause the promise chain to
reject.
reject. The best way to simplify tests and avoid confusion is to **limit the
code in Promise "executor" functions to only track asynchronous operations**;
place fallible assertion code in subsequent reaction handlers.
For example, instead of
```js
promise_test(t => {
return new Promise(resolve => {
window.addEventListener("DOMContentLoaded", t.step_func(event => {
assert_true(event.bubbles, "bubbles should be true");
resolve();
}));
});
}, "DOMContentLoaded");
```
Try,
```js
promise_test(() => {
return new Promise(resolve => {
window.addEventListener("DOMContentLoaded", resolve);
}).then(event => {
assert_true(event.bubbles, "bubbles should be true");
});
}, "DOMContentLoaded");
```
Unlike Asynchronous Tests, Promise Tests don't start running until after the
previous Promise Test finishes. [Under rare
@ -407,14 +434,14 @@ These functions are preferred over `step_timeout` as they end when a condition o
`step_wait(cond, description, timeout=3000, interval=100)` is useful inside `promise_test`, e.g.:
```js
promise_test(t => {
promise_test(async t => {
// …
await t.step_wait(() => frame.contentDocument === null, "Frame navigated to a cross-origin document");
// …
}, "");
```
`step_wait_func(cond, func, description, timeout=3000, interval=100)` & `step_wait_func(cond, func, description, timeout=3000, interval=100)` are useful inside `async_test`:
`step_wait_func(cond, func, description, timeout=3000, interval=100)` and `step_wait_func_done(cond, func, description, timeout=3000, interval=100)` are useful inside `async_test`:
```js
async_test(t => {
@ -839,6 +866,8 @@ asserts that `expected` is an Array, and `actual` is equal to one of the
members i.e. `expected.indexOf(actual) != -1`
### `assert_object_equals(actual, expected, description)`
**DEPRECATED**: see [issue #2033](https://github.com/web-platform-tests/wpt/issues/2033).
asserts that `actual` is an object and not null and that all enumerable
properties on `actual` are own properties on `expected` with the same values,
recursing if the value is an object and not null.

227
tests/wpt/web-platform-tests/docs/writing-tests/testharness.md
View file

@ -1,114 +1,94 @@
# JavaScript Tests (testharness.js)
```eval_rst
.. toctree::
:maxdepth: 1
idlharness
testharness-api
testdriver-extension-tutorial
testdriver
```
testharness.js tests are the correct type of test to write in any
JavaScript tests are the correct type of test to write in any
situation where you are not specifically interested in the rendering
of a page, and where human interaction isn't required; these tests are
written in JavaScript using a framework called `testharness.js`. It is
documented in two sections:
written in JavaScript using a framework called `testharness.js`.
A high-level overview is provided below and more information can be found here:
* [testharness.js Documentation](testharness-api.md) — An introduction
to the library and a detailed API reference.
to the library and a detailed API reference. [The tutorial on writing a
testharness.js test](testharness-tutorial) provides a concise guide to writing
a test — a good place to start for newcomers to the project.
* [testdriver.js Automation](testdriver.md) — Automating end user actions, such as moving or
clicking a mouse. See also the
[testdriver.js extension tutorial](testdriver-extension-tutorial.md) for adding new commands.
* [idlharness.js Documentation](idlharness.md) — A library for testing
IDL interfaces using `testharness.js`.
See [server features](server-features.md) for advanced testing features that are commonly used
with testharness.js. See also the [general guidelines](general-guidelines.md) for all test types.
with JavaScript tests. See also the [general guidelines](general-guidelines.md) for all test types.
This page describes testharness.js exhaustively; [the tutorial on writing a
testharness.js test](testharness-tutorial) provides a concise guide to writing
a test--a good place to start for newcomers to the project.
## Window tests
## Variants
### Without HTML boilerplate (`.window.js`)
A test file can have multiple variants by including `meta` elements,
for example:
Create a JavaScript file whose filename ends in `.window.js` to have the necessary HTML boilerplate
generated for you at `.window.html`. I.e., for `test.window.js` the server will ensure
`test.window.html` is available.
```html
<meta name="variant" content="">
<meta name="variant" content="?wss">
In this JavaScript file you can place one or more tests, as follows:
```js
test(() => {
// Place assertions and logic here
assert_equals(document.characterSet, "UTF-8");
}, "Ensure HTML boilerplate uses UTF-8"); // This is the title of the test
```
The test can then do different things based on the URL.
If you only need to test a [single thing](testharness-api.md#single-page-tests), you could also use:
```js
// META: title=Ensure HTML boilerplate uses UTF-8
setup({ single_test: true });
assert_equals(document.characterSet, "UTF-8");
done();
```
There are two utility scripts in that work well together with variants,
`/common/subset-tests.js` and `/common/subset-tests-by-key.js`, where
a test that would otherwise have too many tests to be useful can be
split up in ranges of subtests. For example:
See [asynchronous (`async_test()`)](testharness-api.md#asynchronous-tests) and
[promise tests (`promise_test()`)](testharness-api.md#promise-tests) for more involved setups.
### With HTML boilerplate
You need to be a bit more explicit and include the `testharness.js` framework directly as well as an
additional file used by implementations:
```html
<!doctype html>
<title>Testing variants</title>
<meta name="variant" content="?1-1000">
<meta name="variant" content="?1001-2000">
<meta name="variant" content="?2001-last">
<script src="/resources/testharness.js">
<script src="/resources/testharnessreport.js">
<script src="/common/subset-tests.js">
<script>
const tests = [
{ fn: t => { ... }, name: "..." },
... lots of tests ...
];
for (const test of tests) {
subsetTest(async_test, test.fn, test.name);
}
</script>
<meta charset=utf-8>
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<body>
<script>
test(() => {
assert_equals(document.characterSet, "UTF-8");
}, "Ensure UTF-8 declaration is observed");
</script>
```
With `subsetTestByKey`, the key is given as the first argument, and the
query string can include or exclude a key (will be matched as a regular
expression).
Here too you could avoid the wrapper `test()` function:
```html
<!doctype html>
<title>Testing variants by key</title>
<meta name="variant" content="?include=Foo">
<meta name="variant" content="?include=Bar">
<meta name="variant" content="?exclude=(Foo|Bar)">
<script src="/resources/testharness.js">
<script src="/resources/testharnessreport.js">
<script src="/common/subset-tests-by-key.js">
<script>
subsetTestByKey("Foo", async_test, () => { ... }, "Testing foo");
...
</script>
<meta charset=utf-8>
<title>Ensure UTF-8 declaration is observed</title>
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<body>
<script>
setup({ single_test: true });
assert_equals(document.characterSet, "UTF-8");
done();
</script>
```
## Auto-generated test boilerplate
In this case the test title is taken from the `title` element.
While most JavaScript tests require a certain amount of HTML
boilerplate to include the test library, etc., tests which are
expressible purely in script (e.g. tests for workers) can have all the
needed HTML and script boilerplate auto-generated.
## Dedicated worker tests (`.worker.js`)
### Standalone window tests
Tests that only require a script file running in window scope can use
standalone window tests. In this case the test is a javascript file
with the extension `.window.js`. This is sourced from a generated
document which sources `testharness.js`, `testharnessreport.js` and
the test script. For a source script with the name
`example.window.js`, the corresponding test resource will be
`example.window.html`.
### Standalone workers tests
Tests that only require assertions in a dedicated worker scope can use
standalone workers tests. In this case, the test is a JavaScript file
with extension `.worker.js` that imports `testharness.js`. The test can
then use all the usual APIs, and can be run from the path to the
JavaScript file with the `.js` removed.
Create a JavaScript file that imports `testharness.js` and whose filename ends in `.worker.js` to
have the necessary HTML boilerplate generated for you at `.worker.html`.
For example, one could write a test for the `FileReaderSync` API by
creating a `FileAPI/FileReaderSync.worker.js` as follows:
@ -125,7 +105,10 @@ done();
This test could then be run from `FileAPI/FileReaderSync.worker.html`.
### Multi-global tests
(Removing the need for `importScripts()` and `done()` is tracked in
[issue #11529](https://github.com/web-platform-tests/wpt/issues/11529).)
## Tests for other or multiple globals (`.any.js`)
Tests for features that exist in multiple global scopes can be written in a way
that they are automatically run in several scopes. In this case, the test is a
@ -178,16 +161,18 @@ be made available by the framework:
self.GLOBAL.isWindow()
self.GLOBAL.isWorker()
Although [the global `done` function must be explicitly invoked for most
Although [the global `done()` function must be explicitly invoked for most
dedicated worker tests and shared worker
tests](testharness-api.html#determining-when-all-tests-are-complete), it is
automatically invoked for tests defined using the "multi-global" pattern.
### Specifying a test title in auto-generated boilerplate tests
## Other features of `.window.js`, `.worker.js` and `.any.js`
### Specifying a test title
Use `// META: title=This is the title of the test` at the beginning of the resource.
### Including other JavaScript resources in auto-generated boilerplate tests
### Including other JavaScript files
Use `// META: script=link/to/resource.js` at the beginning of the resource. For example,
@ -198,11 +183,11 @@ Use `// META: script=link/to/resource.js` at the beginning of the resource. For
can be used to include both the global and a local `utils.js` in a test.
### Specifying a timeout of long in auto-generated boilerplate tests
### Specifying a timeout of long
Use `// META: timeout=long` at the beginning of the resource.
### Specifying test [variants](#variants) in auto-generated boilerplate tests
### Specifying test [variants](#variants)
Use `// META: variant=url-suffix` at the beginning of the resource. For example,
@ -210,3 +195,71 @@ Use `// META: variant=url-suffix` at the beginning of the resource. For example,
// META: variant=
// META: variant=?wss
```
## Variants
A test file can have multiple variants by including `meta` elements,
for example:
```html
<meta name="variant" content="">
<meta name="variant" content="?wss">
```
Test runners will execute the test for each variant specified, appending the corresponding content
attribute value to the URL of the test as they do so.
`/common/subset-tests.js` and `/common/subset-tests-by-key.js` are two utility scripts that work
well together with variants, allowing a test to be split up into subtests in cases when there are
otherwise too many tests to complete inside the timeout. For example:
```html
<!doctype html>
<title>Testing variants</title>
<meta name="variant" content="?1-1000">
<meta name="variant" content="?1001-2000">
<meta name="variant" content="?2001-last">
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="/common/subset-tests.js">
<script>
const tests = [
{ fn: t => { ... }, name: "..." },
... lots of tests ...
];
for (const test of tests) {
subsetTest(async_test, test.fn, test.name);
}
</script>
```
With `subsetTestByKey`, the key is given as the first argument, and the
query string can include or exclude a key (which will be matched as a regular
expression).
```html
<!doctype html>
<title>Testing variants by key</title>
<meta name="variant" content="?include=Foo">
<meta name="variant" content="?include=Bar">
<meta name="variant" content="?exclude=(Foo|Bar)">
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="/common/subset-tests-by-key.js"></script>
<script>
subsetTestByKey("Foo", async_test, () => { ... }, "Testing foo");
...
</script>
```
## Table of Contents
```eval_rst
.. toctree::
:maxdepth: 1
testharness-api
testdriver
testdriver-extension-tutorial
idlharness
```