mirror of
https://github.com/servo/servo.git
synced 2025-07-20 22:03:42 +01:00
180 lines
6.4 KiB
Markdown
180 lines
6.4 KiB
Markdown
---
|
|
layout: page
|
|
title: Introduction
|
|
---
|
|
|
|
web-platform-tests is a W3C-coordinated effort to build a
|
|
cross-browser testsuite for the majority of
|
|
the [web platform][web-platform]; it excludes only ECMAScript (whose
|
|
testsuite lives in [test262][test262]) and WebGL (whose testsuite
|
|
lives in [WebGL][WebGL]).
|
|
|
|
## Help!
|
|
|
|
If you get stuck or want clarification about anything, feel free to
|
|
ask on either the [mailing list][public-test-infra] or [IRC][]
|
|
([webclient][web irc], join channel `#testing`); IRC is generally
|
|
busiest during the European working day but frequently has people on
|
|
it at all times and should probably be the general first port of call
|
|
for any help.
|
|
|
|
|
|
## Testsuite Design
|
|
|
|
The vast majority of the testsuite is formed of HTML pages, which can
|
|
be loaded in a browser and either programmatically provide a result or
|
|
provide a set of steps to run the test and obtain the result.
|
|
|
|
The tests are, in general, short, cross-platform, and self-contained,
|
|
and should be easy to run in any browser.
|
|
|
|
|
|
## Test Layout
|
|
|
|
Each top level directory in the repository corresponds to tests for a
|
|
single specification, with the exception of `css/` which contains
|
|
testsuites for CSS WG specifications. For W3C specs, these directories
|
|
are typically named after the shortname of the spec (i.e. the name
|
|
used for snapshot publications under `/TR/`); for WHATWG specs, they
|
|
are typically named after the subdomain of the spec (i.e. trimming
|
|
`.spec.whatwg.org` from the URL); for other specs, something deemed
|
|
sensible is used. In any case, there are occasional exceptions for
|
|
historic reasons.
|
|
|
|
Within the specification-specific directory there are two common ways
|
|
of laying out tests: the first is a flat structure which is sometimes
|
|
adopted for very short specifications; the alternative is a nested
|
|
structure with each subdirectory corresponding to the id of a heading
|
|
in the specification. The latter provides some implicit metadata about
|
|
the part of a specification being tested according to its location in
|
|
the filesystem, and is preferred for larger specifications.
|
|
|
|
|
|
## Test Types
|
|
|
|
The testsuite has a few types of tests, outlined below:
|
|
|
|
* [testharness.js][] tests, which are run
|
|
through a JS harness and report their result back with JS.
|
|
|
|
* [Reftests][], which render two (or more) web
|
|
pages and combine them with equality assertions about their
|
|
rendering (e.g., `A.html` and `B.html` must render identically), run
|
|
either by the user switching between tabs/windows and trying to
|
|
observe differences or through automated scripts.
|
|
|
|
* [Visual tests][visual] which display a page where the
|
|
result is determined either by a human looking at it or by comparing
|
|
it with a saved screenshot for that user agent on that platform.
|
|
|
|
* [Manual tests][manual], which rely on a human to run
|
|
them and determine their result.
|
|
|
|
* WebDriver tests, which are used for testing the WebDriver protocol
|
|
itself.
|
|
|
|
|
|
## GitHub
|
|
|
|
[GitHub](https://github.com/w3c/web-platform-tests/) is used both for [issue tracking](https://github.com/w3c/web-platform-tests/issues) and [test submissions](https://github.com/w3c/web-platform-tests/pulls); we
|
|
provide [a limited introduction][github-intro] to both git and
|
|
GitHub.
|
|
|
|
Pull Requests are automatically labeled based on the directory the
|
|
files they change are in; there are also comments added automatically
|
|
to notify a number of people: this list of people comes from OWNERS
|
|
files in those same directories and their parents (i.e., they work
|
|
recursively: `a/OWNERS` will get notified for `a/foo.html` and
|
|
`a/b/bar.html`).
|
|
|
|
If you want to be notified about changes to tests in a directory, feel
|
|
free to add yourself to the OWNERS file: there's no requirement to own
|
|
anything as a result!
|
|
|
|
|
|
## Local Setup
|
|
|
|
The tests are designed to be run from your local computer. The test
|
|
environment requires [Python 2.7+](http://www.python.org/downloads) (but not Python 3.x).
|
|
|
|
On Windows, be sure to add the Python directory (`c:\python2x`, by default) to
|
|
your `%Path%` [Environment Variable](http://www.computerhope.com/issues/ch000549.htm),
|
|
and read the [Windows Notes](#windows-notes) section below.
|
|
|
|
To get the tests running, you need to set up the test domains in your
|
|
[`hosts` file](http://en.wikipedia.org/wiki/Hosts_%28file%29%23Location_in_the_file_system). The
|
|
following entries are required:
|
|
|
|
```
|
|
127.0.0.1 web-platform.test
|
|
127.0.0.1 www.web-platform.test
|
|
127.0.0.1 www1.web-platform.test
|
|
127.0.0.1 www2.web-platform.test
|
|
127.0.0.1 xn--n8j6ds53lwwkrqhv28a.web-platform.test
|
|
127.0.0.1 xn--lve-6lad.web-platform.test
|
|
0.0.0.0 nonexistent-origin.web-platform.test
|
|
```
|
|
|
|
If you are behind a proxy, you also need to make sure the domains above are
|
|
excluded from your proxy lookups.
|
|
|
|
The test environment can then be started using
|
|
|
|
./wpt serve
|
|
|
|
This will start HTTP servers on two ports and a websockets server on
|
|
one port. By default one web server starts on port 8000 and the other
|
|
ports are randomly-chosen free ports. Tests must be loaded from the
|
|
*first* HTTP server in the output. To change the ports, copy the
|
|
`config.default.json` file to `config.json` and edit the new file,
|
|
replacing the part that reads:
|
|
|
|
```
|
|
"http": [8000, "auto"]
|
|
```
|
|
|
|
to some port of your choice e.g.
|
|
|
|
```
|
|
"http": [1234, "auto"]
|
|
```
|
|
|
|
## Running tests automatically
|
|
|
|
The `wpt run` command provides a frontend for running tests automatically
|
|
in various browsers. The general syntax is:
|
|
|
|
```
|
|
wpt run [options] <product> [test paths]
|
|
```
|
|
|
|
e.g. to run `dom/historical.html` in Firefox, the required command is:
|
|
|
|
```
|
|
wpt run firefox dom/historical.html
|
|
```
|
|
|
|
### Windows Notes
|
|
|
|
Generally Windows Subsystem for Linux will provide the smoothest user
|
|
experience for running web-platform-tests on Windows.
|
|
|
|
The standard Windows shell requires that all `wpt` commands are prefixed
|
|
by the Python binary i.e. assuming `python` is on your path the server is
|
|
started using:
|
|
|
|
`python wpt serve`
|
|
|
|
|
|
[web-platform]: https://platform.html5.org
|
|
[test262]: https://github.com/tc39/test262
|
|
[webgl]: https://github.com/KhronosGroup/WebGL
|
|
[public-test-infra]: https://lists.w3.org/Archives/Public/public-test-infra/
|
|
[IRC]: irc://irc.w3.org:6667/testing
|
|
[web irc]: http://irc.w3.org
|
|
|
|
[reftests]: {{ site.baseurl }}{% link _writing-tests/reftests.md %}
|
|
[testharness.js]: {{ site.baseurl }}{% link _writing-tests/testharness.md %}
|
|
[visual]: {{ site.baseurl }}{% link _writing-tests/visual.md %}
|
|
[manual]: {{ site.baseurl }}{% link _writing-tests/manual.md %}
|
|
[github-intro]: {{ site.baseurl }}{% link _appendix/github-intro.md %}
|