Update web-platform-tests to revision 81962ac8802223d038b188b6f9cb88a0a9c5beee

tests/wpt/web-platform-tests/docs/_running-tests/chrome_android.md

@@ -19,7 +19,6 @@ Until we find a better way, we need to root the Android device and update the
 127.0.0.1   www2.web-platform.test
 127.0.0.1   xn--n8j6ds53lwwkrqhv28a.web-platform.test
 127.0.0.1   xn--lve-6lad.web-platform.test
-0.0.0.0     nonexistent-origin.web-platform.test
 ```
 
 ## CA certificate

tests/wpt/web-platform-tests/docs/_writing-tests/server-features.md

@@ -14,15 +14,18 @@ generation. This is supported through the
 
 ### Tests Involving Multiple Origins
 
-In the test environment, five subdomains are available: `www`, `www1`,
-`www2`, `天気の良い日`, and `élève`; there is also
-`nonexistent-origin` which is guaranteed not to resolve. In addition,
-the HTTP server listens on two ports, and the WebSockets server on
-one. These subdomains and ports must be used for cross-origin
-tests. Tests must not hardcode the hostname of the server that they
-expect to be running on or the port numbers, as these are not
-guaranteed by the test environment. Instead they can get this
-information in one of two ways:
+Our test servers are guaranteed to be accessible through two domains
+and five subdomains under each. The 'main' domain is unnamed; the
+other is called 'alt'. These subdomains are: `www`, `www1`, `www2`,
+`天気の良い日`, and `élève`; there is also `nonexistent` which is
+guaranteed not to resolve. In addition, the HTTP server listens on two
+ports, and the WebSockets server on one. These subdomains and ports
+must be used for cross-origin tests.
+
+Tests must not hardcode the hostname of the server that they expect to
+be running on or the port numbers, as these are not guaranteed by the
+test environment. Instead they can get this information in one of two
+ways:
 
 * From script, using the `location` API.
 
@@ -33,15 +36,19 @@ form `{name}.sub.{ext}` e.g. `example-test.sub.html` or be referenced
 through a URL containing `pipe=sub` in the query string
 e.g. `example-test.html?pipe=sub`. The substitution syntax uses `{%
 raw %}{{ }}{% endraw %}` to delimit items for substitution. For
-example to substitute in the host name on which the tests are running,
-one would write: `{% raw %}{{host}}{% endraw %}`.
+example to substitute in the main host name, one would write:
+`{% raw %}{{host}}{% endraw %}`.
 
+To get full domains, including subdomains, there is the `hosts`
+dictionary, where the first dimension is the name of the domain, and
+the second the subdomain. For example, `{% raw %}{{hosts[][www]}}{%
+endraw %}` would give the `www` subdomain under the main (unnamed)
+domain, and `{% raw %}{{hosts[alt][élève]}}{% endraw %}` would give
+the `élève` subdomain under the alt domain.
 
-As well as the host, one can get full domains, including subdomains
-using the `domains` dictionary. For example, `{% raw
-%}{{domains[www]}}{% endraw %}` or `{% raw %}{{domains[élève]}}{%
-endraw %}` would be replaced by the full qualified domain name of the
-respective subdomains.
+For mostly historic reasons, the subdomains of the main domain are
+also available under the `domains` dictionary; this is identical to
+`hosts[]`.
 
 Ports are also available on a per-protocol basis. For example, `{% raw
 %}{{ports[ws][0]}}{% endraw %}` is replaced with the first (and only)

tests/wpt/web-platform-tests/docs/_writing-tests/testdriver-tutorial.md

@@ -0,0 +1,317 @@
+# Adding new commands to testdriver.js
+
+## Assumptions
+We assume the following in this writeup:
+ - You know what web-platform-tests is and you have a working checkout and can run tests
+ - You know what WebDriver or Selenium is
+ - Familiarity with JavaScript and Python
+
+## Introduction!
+
+Let's implement window resizing. We can do this via the [Set Window Rect](https://w3c.github.io/webdriver/webdriver-spec.html#dfn-set-window-rect) command in WebDriver.
+
+First, we need to think of what the API will look like a little. We will be using Selenium and Marionette for this, so we can look and see that they take in x, y coordinates, width and height integers.
+
+The first part of this will be browser agnostic, but later we will need to implement a specific layer for each browser (here we will do Firefox and Chrome).
+
+## Code!
+
+### [resources/testdriver.js](resources/testdriver.js)
+
+This is the main entry point the tests get. Here we need to add a function to the `test_driver` object that will call the `test_driver_internal` object.
+
+```javascript
+window.test_driver = {
+
+    // other commands...
+
+    /**
+    * Triggers browser window to be resized and relocated
+    *
+    * This matches the behaviour of the {@link
+    * https://w3c.github.io/webdriver/webdriver-spec.html#dfn-set-window-rect|WebDriver
+    * Set Window Rect command}.
+    *
+    * @param {Integer} x - The x coordinate of the top left of the window
+    * @param {Integer} y - The x coordinate of the top left of the window
+    * @param {Integer} width - The width of the window
+    * @param {Integer} height - The width of the window
+    * @returns {Promise} fulfilled after window rect is set occurs, or rejected in
+    *                    the cases the WebDriver command errors
+    */
+    set_window_rect: function(x, y, width, height) {
+        return window.test_driver_internal.set_element_rect(x, y, width, height);
+    }
+```
+
+In the same file, lets add to the internal object. ( do we need to do this?) (make sure to do this if the internal call has different arguments than the external call, especially if it calls multiple internal calls)
+
+```javascript
+window.test_driver_internal = {
+
+    // other commands...
+
+    /**
+     * Triggers browser window to be resized and relocated
+     *
+     * This matches the behaviour of the {@link
+     * https://w3c.github.io/webdriver/webdriver-spec.html#dfn-set-window-rect|WebDriver
+     * Set Window Rect command}.
+     *
+     * @param {Integer} x - The x coordinate of the top left of the window
+     * @param {Integer} y - The x coordinate of the top left of the window
+     * @param {Integer} width - The width of the window
+     * @param {Integer} height - The width of the window
+     * @returns {Promise} fulfilled after window rect is set occurs, or rejected in
+     *                    the cases the WebDriver command errors
+     */
+    set_window_rect: function(x, y, width, height) {
+        return Promise.reject(new Error("unimplemented"))
+    }
+```
+We will leave this unimplemented and override it in another file. Lets do that now!
+
+### [wptrunner/wptrunner/testdriver-extra.js](tools/wptrunner/wptrunner/testdriver-extra.js)
+
+This will be the default function called when invoking the test driver commands (sometimes it is overridden by testdriver-vendor.js, but this is outside the scope of this writeup).
+
+```javascript
+window.test_driver_internal.set_element_rect = function(x, y, width, height) {
+    const pending_promise = new Promise(function(resolve, reject) {
+        pending_resolve = resolve;
+        pending_reject = reject;
+    });
+    window.opener.postMessage(
+        {"type": "action", "action": "set_window_rect", "x": x, "y": y, "width": width, "height": height}, "*");
+    return pending_promise;
+};
+```
+The main thing here is the `postMessage` argument. The first argument is an object with properties
+ - `type`: this always has to be the string `"action"`
+ - `action`: the name of the testdriver command this defines (in this case, `set_window_rect`)
+ - any other things you want to pass to the next point of execution (in this case, the x, y coordinates and the width and height)
+
+<!-- The pending promise needs to be there as it is resolved when the window recieves a completion message from the executor. -->
+The pending promise is out of scope of this function and is resolved when the window recieves a completion message from the executor.
+This happens here in the same file:
+
+```javascript
+    let pending_resolve = null;
+    let pending_reject = null;
+    window.addEventListener("message", function(event) {
+        const data = event.data;
+
+        if (typeof data !== "object" && data !== null) {
+            return;
+        }
+
+        if (data.type !== "testdriver-complete") {
+            return;
+        }
+
+        if (data.status === "success") {
+            pending_resolve();
+        } else {
+            pending_reject();
+        }
+    });
+```
+
+One limitation this introduces is that only one testdriver call can be made at one time since the `pending_resolve` and `pending_reject` variables are in an outer scope.
+
+Next, this is passed to the executor and protocol in wptrunner. Time to switch to Python!
+
+[tools/wptrunner/wptrunner/executors/protocol.py](tools/wptrunner/wptrunner/executors/protocol.py)
+
+```python
+class SetWindowRectProtocolPart(ProtocolPart):
+    """Protocol part for resizing and changing location of window"""
+    __metaclass__ = ABCMeta
+
+    name = "set_window_rect"
+
+    @abstractmethod
+    def set_window_rect(self, x, y, width, height):
+        """Change the window rect
+
+        :param x: The x coordinate of the top left of the window.
+        :param y: The y coordinate of the top left of the window.
+        :param width: The width of the window.
+        :param height: The height of the window."""
+        pass
+```
+
+Next we change the base executor.
+
+[tools/wptrunner/wptrunner/executors/base.py](tools/wptrunner/wptrunner/executors/base.py)
+
+```python
+class CallbackHandler(object):
+    """Handle callbacks from testdriver-using tests.
+
+    The default implementation here makes sense for things that are roughly like
+    WebDriver. Things that are more different to WebDriver may need to create a
+    fully custom implementation."""
+
+    def __init__(self, logger, protocol, test_window):
+        self.protocol = protocol
+        self.test_window = test_window
+        self.logger = logger
+        self.callbacks = {
+            "action": self.process_action,
+            "complete": self.process_complete
+        }
+
+        self.actions = {
+            "click": ClickAction(self.logger, self.protocol),
+            "send_keys": SendKeysAction(self.logger, self.protocol),
+            {other actions},
+            "set_window_rect": SetWindowRectAction(self.logger, self.protocol) # add this!
+        }
+```
+
+```python
+class SetWindowRectAction(object):
+    def __init__(self, logger, protocol):
+        self.logger = logger
+        self.protocol = protocol
+
+    def __call__(self, payload):
+        x, y, width, height = payload["x"], payload["y"], payload["width"], payload["height"]
+        self.logger.debug("Setting window rect to be: x=%s, y=%s, width=%s, height=%s"
+                          .format(x, y, width, height))
+        self.protocol.set_window_rect.set_window_rect(x, y, width, height)
+```
+
+Don't forget to write docs in ```testdriver.md```.
+Now we write the browser specific implementations.
+
+### Chrome
+
+We will use [executorselenium](tools/wptrunner/wptrunner/executors/executorselenium.py) and use the Selenium API (in the future there are plans to use the WebDriver API directly).
+
+There isn't too much work to do here, we just need to define a subclass of the protocol part we defined earlier.
+
+```python
+class SeleniumSetWindowRectProtocolPart(SetWindowRectProtocolPart):
+    def setup(self):
+        self.webdriver = self.parent.webdriver
+
+    def set_window_rect(self, x, y, width, height):
+        return self.webdriver.set_window_rect(x, y, width, height)
+```
+
+Make sure to import the protocol part too!
+
+```python
+from .protocol import (BaseProtocolPart,
+                       TestharnessProtocolPart,
+                       Protocol,
+                       SelectorProtocolPart,
+                       ClickProtocolPart,
+                       SendKeysProtocolPart,
+                       {... other protocol parts}
+                       SetWindowRectProtocolPart, # add this!
+                       TestDriverProtocolPart)
+```
+
+Here we have the setup method which just redefines the webdriver object at this level. The important part is the `set_window_rect` function (and it's important it is named that since we called it that earlier). This will be call the Selenium API for [set window rect](http://selenium-python.readthedocs.io/api.html#selenium.webdriver.remote.webdriver.WebDriver.set_window_rect) (`self.webdriver` is a Selenium WebDriver instance here).
+
+Finally, we just need to tell the SeleniumProtocol to implement this part.
+
+```python
+class SeleniumProtocol(Protocol):
+    implements = [SeleniumBaseProtocolPart,
+                  SeleniumTestharnessProtocolPart,
+                  SeleniumSelectorProtocolPart,
+                  SeleniumClickProtocolPart,
+                  SeleniumSendKeysProtocolPart,
+                  {... other protocol parts}
+                  SeleniumSetWindowRectProtocolPart,
+                  SeleniumTestDriverProtocolPart]
+```
+
+
+### Firefox
+We use the [set window rect](http://marionette-client.readthedocs.io/en/master/reference.html#marionette_driver.marionette.Marionette.set_window_rect) Marionette command.
+
+We will use [executormarionette](tools/wptrunner/wptrunner/executors/executormarionette.py) and use the Marionette Python API.
+
+We have little actual work to do here! We just need to define a subclass of the protocol part we defined earlier.
+
+```python
+class MarionetteSetWindowRectProtocolPart(SetWindowRectProtocolPart):
+    def setup(self):
+        self.marionette = self.parent.marionette
+
+    def set_window_rect(self, x, y, width, height):
+        return self.marionette.set_window_rect(x, y, width, height)
+```
+
+Make sure to import the protocol part too!
+
+```python
+from .protocol import (BaseProtocolPart,
+                       TestharnessProtocolPart,
+                       Protocol,
+                       SelectorProtocolPart,
+                       ClickProtocolPart,
+                       SendKeysProtocolPart,
+                       {... other protocol parts}
+                       SetWindowRectProtocolPart, # add this!
+                       TestDriverProtocolPart)
+```
+
+Here we have the setup method which just redefines the webdriver object at this level. The important part is the `set_window_rect` function (and it's important it is named that since we called it that earlier). This will be call the Marionette API for [set window rect](http://marionette-client.readthedocs.io/en/master/reference.html#marionette_driver.marionette.Marionette.set_window_rect) (`self.marionette` is a marionette instance here).
+
+Finally, we just need to tell the SeleniumProtocol to implement this part.
+
+```python
+class MarionetteProtocol(Protocol):
+    implements = [MarionetteBaseProtocolPart,
+                  MarionetteTestharnessProtocolPart,
+                  MarionettePrefsProtocolPart,
+                  MarionetteStorageProtocolPart,
+                  MarionetteSelectorProtocolPart,
+                  MarionetteClickProtocolPart,
+                  MarionetteSendKeysProtocolPart,
+                  {... other protocol parts}
+                  MarionetteSetWindowRectProtocolPart # add this
+                  MarionetteTestDriverProtocolPart]
+```
+
+### Other Browsers
+
+Other browsers may also use executorselenium (such as safari), or a completely new executor (such as servo). For these, you must change the executor in the same way as we did with chrome and firefox.
+
+### Write an infra test
+
+Make sure to add a test to `infrastructure/testdriver` :)
+
+Here is some template code!
+
+```html
+<!DOCTYPE html>
+<meta charset="utf-8">
+<title>TestDriver set window rect method</title>
+<script src="/resources/testharness.js"></script>
+<script src="/resources/testharnessreport.js"></script>
+<script src="/resources/testdriver.js"></script>
+<script src="/resources/testdriver-vendor.js"></script>
+
+<script>
+promise_test(async t => {
+  await test_driver.set_window_rect(100, 100, 100, 100);
+  // do something
+}
+</script>
+```
+### What about testdriver-vendor.js?
+
+The file [testdriver-vendor.js](resources/testdriver-vendor.js) is the equivalent to testdriver-extra.js above, except is
+run instead of testdriver-extra.js in browser specific test environments. For example, in [Chromium LayoutTests](https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/?q=LayoutTests&sq=package:chromium&dr).
+
+### What if I need to return a value from my testdriver API?
+
+We currently don't have this capability, but it is coming soon and will be documented. The bug is [here](https://github.com/w3c/web-platform-tests/issues/10716)
+

tests/wpt/web-platform-tests/docs/_writing-tests/testharness-api.md

@@ -668,6 +668,17 @@ or a [`ServiceWorker`](https://slightlyoff.github.io/ServiceWorker/spec/service_
 Once called, the containing document fetches all the tests from the worker and
 behaves as if those tests were running in the containing document itself.
 
+`fetch_tests_from_worker` returns a promise that resolves once all the remote
+tests have completed. This is useful if you're importing tests from multiple
+workers and want to ensure they run in series:
+
+```js
+(async function() {
+  await fetch_tests_from_worker(new Worker("worker-1.js"));
+  await fetch_tests_from_worker(new Worker("worker-2.js"));
+})();
+```
+
 ## List of Assertions ##
 
 ### `assert_true(actual, description)`

tests/wpt/web-platform-tests/docs/_writing-tests/testharness.md

@@ -19,6 +19,44 @@ documented in two sections:
 See [server features][] for advanced testing features that are commonly used
 with testharness.js. See also the [general guidelines][] for all test types.
 
+## Variants
+
+A test file can have multiple variants by including `meta` elements,
+for example:
+
+```
+<meta name="variant" content="">
+<meta name="variant" content="?wss">
+```
+
+The test can then do different things based on the URL.
+
+There is a utility script in `/common/subset-tests.js` that works
+well together with variants, where a test that would otherwise have
+too many tests to be useful can be split up in ranges of subtests.
+For example:
+
+```
+<!doctype html>
+<title>Testing variants</title>
+<meta name="variant" content="?1-1000">
+<meta name="variant" content="?1001-2000">
+<meta name="variant" content="?2001-last">
+<script src="/resources/testharness.js">
+<script src="/resources/testharnessreport.js">
+<script src="/common/subset-tests.js">
+<script>
+ const tests = [
+                 { fn: t => { ... }, name: "..." },
+                 ... lots of tests ...
+               ];
+ for (const test of tests) {
+   subsetTest(async_test, test.fn, test.name);
+ }
+</script>
+```
+
+
 ## Auto-generated test boilerplate
 
 While most JavaScript tests require a certain amount of HTML
@@ -121,7 +159,7 @@ can be used to include both the global and a local `utils.js` in a test.
 
 Use `// META: timeout=long` at the beginning of the resource.
 
-### Specifying test variants in auto-generated boilerplate tests
+### Specifying test [variants](#variants) in auto-generated boilerplate tests
 
 Use `// META: variant=url-suffix` at the beginning of the resource. For example,
 

tests/wpt/web-platform-tests/docs/introduction.md

@@ -114,10 +114,10 @@ For example, on most UNIX-like systems, you can setup the hosts file with:
 ./wpt make-hosts-file | sudo tee -a /etc/hosts
 ```
 
-And on Windows (note this requires an Administrator privileged shell):
+And on Windows (this must be run in a PowerShell session with Administrator privileges):
 
 ```bash
-python wpt make-hosts-file >> %SystemRoot%\System32\drivers\etc\hosts
+python wpt make-hosts-file | Out-File %SystemRoot%\System32\drivers\etc\hosts -Encoding ascii -Append
 ```
 
 If you are behind a proxy, you also need to make sure the domains above are
@@ -130,20 +130,17 @@ The test environment can then be started using
 This will start HTTP servers on two ports and a websockets server on
 one port. By default the web servers start on ports 8000 and 8443 and the other
 ports are randomly-chosen free ports. Tests must be loaded from the
-*first* HTTP server in the output. To change the ports, copy the
-`config.default.json` file to `config.json` and edit the new file,
-replacing the part that reads:
+*first* HTTP server in the output. To change the ports,
+create a `config.json` file in the wpt root directory, and add
+port definitions of your choice e.g.:
 
 ```
-"http": [8000, "auto"],
-"https":[8443]
-```
-
-to some ports of your choice e.g.
-
-```
-"http": [1234, "auto"],
-"https":[5678]
+{
+  "ports": {
+    "http": [1234, "auto"],
+    "https":[5678]
+  }
+}
 ```
 
 After your `hosts` file is configured, the servers will be locally accessible at: