mirror of
https://github.com/servo/servo.git
synced 2025-08-14 01:45:33 +01:00
Update web-platform-tests to revision d7afcb8708eac08a614d161d5622a48172daf7e3
This commit is contained in:
WPT Sync Bot
Josh Matthews
parent
6f8bb4dd40
commit
edff458e23
791 changed files with 17647 additions and 10322 deletions
7
tests/wpt/web-platform-tests/docs/.gitignore
vendored
7
tests/wpt/web-platform-tests/docs/.gitignore
vendored
|
|
@ -1,6 +1 @@
|
|||
_site
|
||||
.sass-cache
|
||||
.jekyll-metadata
|
||||
.bundle/
|
||||
Gemfile.lock
|
||||
vendor/bundle/
|
||||
_build/
|
||||
|
|
|
|||
|
|
@ -1 +0,0 @@
|
|||
web-platform-tests.org
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
source 'https://rubygems.org'
|
||||
|
||||
require 'json'
|
||||
require 'open-uri'
|
||||
versions = JSON.parse(open('https://pages.github.com/versions.json').read)
|
||||
|
||||
gem 'github-pages', versions['github-pages']
|
||||
19
tests/wpt/web-platform-tests/docs/Makefile
Normal file
19
tests/wpt/web-platform-tests/docs/Makefile
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
# Minimal makefile for Sphinx documentation
|
||||
#
|
||||
|
||||
# You can set these variables from the command line.
|
||||
SPHINXOPTS =
|
||||
SPHINXBUILD = sphinx-build
|
||||
SOURCEDIR = .
|
||||
BUILDDIR = _build
|
||||
|
||||
# Put it first so that "make" without argument is like "make help".
|
||||
help:
|
||||
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
.PHONY: help Makefile
|
||||
|
||||
# Catch-all target: route all unknown targets to Sphinx using the new
|
||||
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
|
||||
%: Makefile
|
||||
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
24
tests/wpt/web-platform-tests/docs/README.md
Normal file
24
tests/wpt/web-platform-tests/docs/README.md
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
# WPT documentation
|
||||
|
||||
The documentation for the web-platform-tests project is built using [the Sphinx
|
||||
documentation generator](http://www.sphinx-doc.org). [The GitHub Actions
|
||||
service](https://github.com/features/actions) is configured to automatically
|
||||
update the public website each time changes are merged to the repository.
|
||||
|
||||
## Local Development
|
||||
|
||||
If you would like to build the site locally, follow these instructions.
|
||||
|
||||
1. Install the system dependencies. The free and open source software tools
|
||||
[Python](https://www.python.org/) and [Git](https://git-scm.com/) are
|
||||
required. Each website has instructions for downloading and installing on a
|
||||
variety of systems.
|
||||
2. Download the source code. Clone this repository using the `git clone`
|
||||
command.
|
||||
3. Install the Python dependencies. Run the following command in a terminal
|
||||
from the "docs" directory of the WPT repository:
|
||||
|
||||
pip install -r requirements.txt
|
||||
|
||||
4. Build the documentation. Windows users should execute the `make.bat` batch
|
||||
file. GNU/Linux and macOS users should use the `make` command.
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
---
|
||||
layout: page
|
||||
title: Appendices
|
||||
order: -1
|
||||
---
|
||||
|
||||
{% assign appendix = site.appendix | sort: "order" %}
|
||||
{% for page in appendix %}{% if page.title and page.order != -1 %}
|
||||
* [{{ page.title }}]({{ page.url | relative_url }}) {{ ""
|
||||
}}{% endif %}{% endfor %}
|
||||
|
|
@ -1,48 +0,0 @@
|
|||
# Welcome to Jekyll!
|
||||
#
|
||||
# This config file is meant for settings that affect your whole blog, values
|
||||
# which you are expected to set up once and rarely edit after that. If you find
|
||||
# yourself editing this file very often, consider using Jekyll's data files
|
||||
# feature for the data you need to update frequently.
|
||||
#
|
||||
# For technical reasons, this file is *NOT* reloaded automatically when you use
|
||||
# 'bundle exec jekyll serve'. If you change this file, please restart the server process.
|
||||
|
||||
# Site settings
|
||||
# These are used to personalize your new site. If you look in the HTML files,
|
||||
# you will see them accessed via {{ site.title }}, {{ site.email }}, and so on.
|
||||
# You can create any custom variable you would like, and they will be accessible
|
||||
# in the templates via {{ site.myvariable }}.
|
||||
title: web-platform-tests documentation
|
||||
baseurl: "" # the subpath of your site, e.g. /blog
|
||||
url: "" # the base hostname & protocol for your site, e.g. http://example.com
|
||||
|
||||
collections:
|
||||
admin:
|
||||
output: true
|
||||
appendix:
|
||||
output: true
|
||||
reviewing-tests:
|
||||
output: true
|
||||
running-tests:
|
||||
output: true
|
||||
writing-tests:
|
||||
output: true
|
||||
|
||||
# Build settings
|
||||
markdown: kramdown
|
||||
theme: minima
|
||||
gems:
|
||||
- jekyll-optional-front-matter
|
||||
- jekyll-readme-index
|
||||
- jekyll-default-layout
|
||||
- jekyll-titles-from-headings
|
||||
exclude:
|
||||
- Gemfile
|
||||
- Gemfile.lock
|
||||
- vendor/bundle/
|
||||
- .bundle/
|
||||
- OWNERS
|
||||
- META.yml
|
||||
sass:
|
||||
sass_dir: assets/
|
||||
|
|
@ -20,7 +20,7 @@
|
|||
<li><a class="page-link" href="{{ site.baseurl }}{% link introduction.md %}">
|
||||
Introduction
|
||||
</a>
|
||||
<li><a class="page-link" href="{{ site.baseurl }}{% link _writing-tests/index.md %}">
|
||||
<li><a class="page-link" href="{{ site.baseurl }}{% link writing-tests/index.md %}">
|
||||
Writing Tests
|
||||
</a>
|
||||
<ul>
|
||||
|
|
@ -35,10 +35,10 @@
|
|||
{% endif %}
|
||||
{% endfor %}
|
||||
</ul>
|
||||
<li><a class="page-link" href="{{ site.baseurl }}{% link _running-tests/index.md %}">
|
||||
<li><a class="page-link" href="{{ site.baseurl }}{% link running-tests/index.md %}">
|
||||
Running Tests
|
||||
</a>
|
||||
<li><a class="page-link" href="{{ site.baseurl }}{% link _reviewing-tests/index.md %}">
|
||||
<li><a class="page-link" href="{{ site.baseurl }}{% link reviewing-tests/index.md %}">
|
||||
Reviewing Tests
|
||||
</a>
|
||||
<ul>
|
||||
|
|
@ -53,7 +53,7 @@
|
|||
{% endif %}
|
||||
{% endfor %}
|
||||
</ul>
|
||||
<li><a class="page-link" href="{{ site.baseurl }}{% link _appendix/index.md %}">
|
||||
<li><a class="page-link" href="{{ site.baseurl }}{% link appendix/index.md %}">
|
||||
Appendices
|
||||
</a>
|
||||
<ul>
|
||||
|
|
|
|||
|
|
@ -1,18 +0,0 @@
|
|||
---
|
||||
layout: page
|
||||
title: Running Tests
|
||||
---
|
||||
|
||||
The simplest way to run the tests is via the public website. More detail on
|
||||
that approach is available in [Running tests from the Web][from-web].
|
||||
|
||||
Contributors who are interested in modifying and creating tests should refer to
|
||||
[Running Tests from the Local System][from-local-system].
|
||||
|
||||
Advanced use cases may call for a customized method of executing the tests.
|
||||
Guidelines for writing a custom "runner" are available at [Writing Your Own
|
||||
Runner][custom-runner].
|
||||
|
||||
[custom-runner]: {{ site.baseurl }}{% link _running-tests/custom-runner.md %}
|
||||
[from-web]: {{ site.baseurl }}{% link _running-tests/from-web.md %}
|
||||
[from-local-system]: {{ site.baseurl }}{% link _running-tests/from-local-system.md %}
|
||||
|
|
@ -1,73 +0,0 @@
|
|||
---
|
||||
layout: page
|
||||
title: Writing Tests
|
||||
order: -1
|
||||
---
|
||||
|
||||
If you haven't already, it's strongly recommended to read
|
||||
the [introduction][] first, as it introduces the various test types.
|
||||
|
||||
There's also a load of [general guidelines][] that apply to all tests.
|
||||
|
||||
## Test Type
|
||||
|
||||
Tests in this project use a few different approaches to verify expected
|
||||
behavior. The tests can be classified based on the way they express
|
||||
expectations:
|
||||
|
||||
* Rendering tests should be used to verify that the browser graphically
|
||||
displays pages as expected. See the [rendering test guidelines][rendering]
|
||||
for tips on how to write great rendering tests. There are a few different
|
||||
ways to write rendering tests:
|
||||
|
||||
* [Reftests][] should be used to test rendering and layout. They consist of
|
||||
two or more pages with assertions as to whether they render identically or
|
||||
not.
|
||||
|
||||
* [Visual tests][visual] should be used for checking rendering where there is
|
||||
a large number of conforming renderings such that reftests are impractical.
|
||||
They consist of a page that renders to final state at which point a
|
||||
screenshot can be taken and compared to an expected rendering for that user
|
||||
agent on that platform.
|
||||
|
||||
* [testharness.js][] tests should be used (where possible!) for testing
|
||||
everything else. They are built with the testharness.js unit testing
|
||||
framework, and consist of assertions written in JavaScript.
|
||||
|
||||
* [wdspec][] tests are written in Python using
|
||||
[pytest](https://docs.pytest.org/en/latest/) and test [the WebDriver browser
|
||||
automation protocol](https://w3c.github.io/webdriver/)
|
||||
|
||||
* [Manual tests][manual] are used as a last resort for anything that can't be
|
||||
tested using any of the above. They consist of a page that needs manual
|
||||
interaction or verification of the final result.
|
||||
|
||||
In general, there is a strong preference towards reftests and testharness.js
|
||||
tests types (as they can be easily run without human interaction), so they
|
||||
should be used in preference to the others even if it results in a
|
||||
somewhat cumbersome test; there is a far weaker preference between the
|
||||
two test types, and it is at times advisable to use testharness.js tests
|
||||
for things which would typically be tested using reftests but for
|
||||
which it would be overly cumbersome.
|
||||
|
||||
See [file names][] for test types and features determined by the file names,
|
||||
and [server features][] for advanced testing features.
|
||||
|
||||
## Submitting Tests
|
||||
|
||||
Once you've written tests, please submit them using
|
||||
the [typical GitHub Pull Request workflow][submission-process]; please
|
||||
make sure you run the [`lint` script][lint-tool] before opening a pull request!
|
||||
|
||||
[introduction]: {{ site.baseurl }}{% link introduction.md %}
|
||||
[file names]: {{ site.baseurl }}{% link _writing-tests/file-names.md %}
|
||||
[general guidelines]: {{ site.baseurl }}{% link _writing-tests/general-guidelines.md %}
|
||||
[reftests]: {{ site.baseurl }}{% link _writing-tests/reftests.md %}
|
||||
[rendering]: {{ site.baseurl }}{% link _writing-tests/rendering.md %}
|
||||
[server features]: {{ site.baseurl }}{% link _writing-tests/server-features.md %}
|
||||
[testharness.js]: {{ site.baseurl }}{% link _writing-tests/testharness.md %}
|
||||
[visual]: {{ site.baseurl }}{% link _writing-tests/visual.md %}
|
||||
[manual]: {{ site.baseurl }}{% link _writing-tests/manual.md %}
|
||||
[submission-process]: {{ site.baseurl }}{% link _writing-tests/submission-process.md %}
|
||||
[lint-tool]: {{ site.baseurl }}{% link _writing-tests/lint-tool.md %}
|
||||
[wdspec]: {{ site.baseurl }}{% link _writing-tests/wdspec.md %}
|
||||
|
|
@ -1,8 +1,4 @@
|
|||
---
|
||||
layout: page
|
||||
title: Project Administration
|
||||
order: -1
|
||||
---
|
||||
# Project Administration
|
||||
|
||||
This section documents all the information necessary to administer the
|
||||
infrastructure which makes the project possible.
|
||||
|
|
@ -11,10 +7,15 @@ infrastructure which makes the project possible.
|
|||
|
||||
Some aspects of the infrastructure are only accessible to administrators.
|
||||
|
||||
Project | Secret | Owners
|
||||
--------------------------|--------------------------|-------------
|
||||
[results-collection] | root SSH keys | boaz@bocoup.com, mike@bocoup.com, rick@bocoup.com
|
||||
[results-collection] | Password for app secrets | boaz@bocoup.com, mike@bocoup.com, rick@bocoup.com
|
||||
```eval_rst
|
||||
========================= ========================= =========================
|
||||
Project Secret Owners
|
||||
========================= ========================= =========================
|
||||
[results-collection] root SSH keys boaz@bocoup.com, mike@bocoup.com, rick@bocoup.com
|
||||
[results-collection] Password for app secrets boaz@bocoup.com, mike@bocoup.com, rick@bocoup.com
|
||||
========================= ========================= =========================
|
||||
|
||||
```
|
||||
|
||||
SSL certificates for all HTTPS-enabled domains are retrieved via [Let's
|
||||
Encrypt](https://letsencrypt.org/), so that data does not represent an
|
||||
|
|
@ -1,8 +1,5 @@
|
|||
---
|
||||
layout: page
|
||||
title: Introduction to GitHub
|
||||
order: 1
|
||||
---
|
||||
# Introduction to GitHub
|
||||
|
||||
All the basics that you need to know are documented on this page, but for the
|
||||
full GitHub documentation, visit [help.github.com][help].
|
||||
|
||||
|
|
@ -46,12 +43,12 @@ If you are a first-time GitHub user, read on for more details of the workflow.
|
|||
## Fork the test repository
|
||||
|
||||
Now that you have Git set up, you will need to fork the test repository. This
|
||||
will enable you to [submit][submit] your tests using a pull request (more on this
|
||||
[below][submit]).
|
||||
will enable you to [submit](#submit) your tests using a pull request (more on this
|
||||
[below](#submit)).
|
||||
|
||||
1. In the browser, go to [web-platform-tests on GitHub][main-repo].
|
||||
|
||||
2. Click the ![fork][forkbtn] button in the upper right.
|
||||
2. Click the  button in the upper right.
|
||||
|
||||
3. The fork will take several seconds, then you will be redirected to your
|
||||
GitHub page for this forked repository.
|
||||
|
|
@ -107,9 +104,9 @@ repository.
|
|||
|
||||
## Configure your environment
|
||||
|
||||
If all you intend to do is to load [manual tests][manual-tests] or [ref tests][ref-tests] from your local file system,
|
||||
If all you intend to do is to load [manual tests](../writing-tests/manual) or [ref tests](../writing-tests/reftests) from your local file system,
|
||||
the above setup should be sufficient.
|
||||
But many tests (and in particular, all [testharness.js tests][testharness-tests]) require a local web server.
|
||||
But many tests (and in particular, all [testharness.js tests](../writing-tests/testharness)) require a local web server.
|
||||
|
||||
See [Local Setup][local-setup] for more information.
|
||||
|
||||
|
|
@ -120,7 +117,7 @@ Now that you have everything locally, create a branch for your tests.
|
|||
_Note: If you have already been through these steps and created a branch
|
||||
and now want to create another branch, you should always do so from the
|
||||
master branch. To do this follow the steps from the beginning of the [previous
|
||||
section][remote-upstream]. If you don't start with a clean master
|
||||
section](#configure-remote-upstream). If you don't start with a clean master
|
||||
branch you will end up with a big nested mess._
|
||||
|
||||
At the command line:
|
||||
|
|
@ -136,7 +133,7 @@ For Example:_
|
|||
$ git checkout -b flexbox-flex-direction-prop
|
||||
|
||||
You're ready to start writing tests! Come back to this page you're ready to
|
||||
[commit][commit] them or [submit][submit] them for review.
|
||||
[commit](#commit) them or [submit](#submit) them for review.
|
||||
|
||||
|
||||
## Commit
|
||||
|
|
@ -196,7 +193,7 @@ of the repository is located.
|
|||
|
||||
3. Fix any mistake it reports and [commit](#commit) again.
|
||||
|
||||
For more details, see the [documentation about the lint tool][lint-tool]
|
||||
For more details, see the [documentation about the lint tool](../writing-tests/lint-tool).
|
||||
|
||||
## Submit
|
||||
|
||||
|
|
@ -224,7 +221,7 @@ them back up to the server:
|
|||
GitHub UI. Below is one method and others can be found on
|
||||
[GitHub.com][github-createpr]
|
||||
|
||||
1. Click the ![new pull request][pullrequestbtn] button.
|
||||
1. Click the  button.
|
||||
|
||||
2. On the left, you should see the base repo is the
|
||||
web-platform-tests/wpt. On the right, you should see your fork of that
|
||||
|
|
@ -234,7 +231,7 @@ GitHub UI. Below is one method and others can be found on
|
|||
make sure your fork and your `topic`
|
||||
branch is selected on the right side.
|
||||
|
||||
3. Select the ![create pull request][createpr] button at the top.
|
||||
3. Select the  button at the top.
|
||||
|
||||
4. Scroll down and review the diff
|
||||
|
||||
|
|
@ -246,11 +243,11 @@ GitHub UI. Below is one method and others can be found on
|
|||
6. If you'd like to add more detailed comments, use the comment field
|
||||
below.
|
||||
|
||||
7. Click ![the create pull request button][createpr]
|
||||
7. Click 
|
||||
|
||||
|
||||
4. Wait for feedback on your pull request and once your pull request is
|
||||
accepted, delete your branch (see '[When Pull Request is Accepted][cleanup]').
|
||||
accepted, delete your branch (see '[When Pull Request is Accepted](#cleanup)').
|
||||
|
||||
That's it! Your pull request will go into a queue and will be reviewed soon.
|
||||
|
||||
|
|
@ -289,7 +286,7 @@ on the test but you should delete your branch. This can easily be done in
|
|||
the GitHub UI by navigating to the pull requests and clicking the
|
||||
'Delete Branch' button.
|
||||
|
||||
![pull request accepted delete branch][praccepteddelete]
|
||||
|
||||
|
||||
Alternatively, you can delete the branch on the command line.
|
||||
|
||||
|
|
@ -307,7 +304,7 @@ The following workflow is recommended:
|
|||
6. After the PR has been accepted, delete the branch. (Every new PR should
|
||||
come from a new branch.)
|
||||
7. Synchronize your fork with the W3C repository by fetching your upstream and
|
||||
merging it. (See '[Configure Remote / Upstream][remote-upstream]')
|
||||
merging it. (See '[Configure Remote / Upstream](#configure-remote-upstream)')
|
||||
|
||||
You need to be able to set up remote upstream, etc. Please refer to [Pro Git
|
||||
Book][git-book] and enjoy reading.
|
||||
|
|
@ -332,17 +329,17 @@ but the reviewer still wants to make some simple tweaks to the tests before merg
|
|||
it is possible to do so via the Github web UI.
|
||||
|
||||
1. Open the pull request. E.g. https://github.com/web-platform-tests/wpt/pull/1234
|
||||
2. Go to the ![Files changed][files-changed] view (e.g. https://github.com/web-platform-tests/wpt/pull/1234/files)
|
||||
3. Locate the files you wish to change, and click the ![pencil][pencil-icon] icon in the upper right corner
|
||||
2. Go to the  view (e.g. https://github.com/web-platform-tests/wpt/pull/1234/files)
|
||||
3. Locate the files you wish to change, and click the  icon in the upper right corner
|
||||
4. Make the desired change
|
||||
5. Write a commit message (including a good title) at the bottom
|
||||
6. Make sure the ![Commit directly to the [name-of-the-PR-branch] branch.][commit-directly] radio button is selected.
|
||||
6. Make sure the ![Commit directly to the [name-of-the-PR-branch] branch.](../assets/commit-directly.png) radio button is selected.
|
||||
|
||||
_Note: If the PR predates the introduction of this feature by Github,
|
||||
or if the author of the PR has disabled write-access by reviewers to the PR branch,
|
||||
this may not be available,
|
||||
and your only option would be to commit to a new branch, creating a new PR._
|
||||
7. Click the ![Commit Changes][commitbtn] button.
|
||||
7. Click the  button.
|
||||
|
||||
|
||||
### The Normal Way
|
||||
|
|
@ -357,7 +354,7 @@ If you don't have one, go [fork](#fork), [clone](#clone), and [configure](#confi
|
|||
4. Checkout that branch: `git checkout <name-of-the-PR-branch>`
|
||||
|
||||
_The relevant `<author-id>`, `<repo-name>`, and `<name-of-the-PR-branch>` can be found by looking for this sentence in on the Github page of the PR:
|
||||
![Add more commits by pushing to the name-of-the-PR-branch branch on author-id/repo-name.][more-commits]_
|
||||
_
|
||||
|
||||
If all you meant to do was reviewing files locally, you're all set.
|
||||
If you wish to make changes to the PR branch:
|
||||
|
|
@ -390,14 +387,7 @@ If you do not expect work with more PRs from the same author,
|
|||
you may also discard your connection to their repo:
|
||||
`git remote remove <author-id>`
|
||||
|
||||
[local-setup]: {{ site.baseurl }}/introduction.html#local-setup
|
||||
[manual-tests]: {{ site.baseurl }}/writing-tests/manual.html
|
||||
[ref-tests]: {{ site.baseurl }}/writing-tests/reftests.html
|
||||
[testharness-tests]: {{ site.baseurl }}/writing-tests/testharness.html
|
||||
[branch]: #branch
|
||||
[commit]: #commit
|
||||
[clone]: #clone
|
||||
[forkbtn]: {{ site.baseurl }}{% link /assets/forkbtn.png %}
|
||||
[local-setup]: ../introduction#local-setup
|
||||
[git]: https://git-scm.com/downloads
|
||||
[git-book]: https://git-scm.com/book
|
||||
[github]: https://github.com/
|
||||
|
|
@ -407,16 +397,4 @@ you may also discard your connection to their repo:
|
|||
[help]: https://help.github.com/
|
||||
[main-repo]: https://github.com/web-platform-tests/wpt
|
||||
[password-caching]: https://help.github.com/articles/caching-your-github-password-in-git
|
||||
[pullrequestbtn]: {{ site.baseurl }}{% link /assets/pullrequestbtn.png %}
|
||||
[createpr]: {{ site.baseurl }}{% link /assets/createpr.png %}
|
||||
[praccepteddelete]: {{ site.baseurl }}{% link /assets/praccepteddelete.png %}
|
||||
[submit]: #submit
|
||||
[remote-upstream]: #configure-remote-upstream
|
||||
[cleanup]: #cleanup
|
||||
[pencil-icon]: {{ site.baseurl }}{% link /assets/pencil-icon.png %}
|
||||
[commitbtn]: {{ site.baseurl }}{% link /assets/commitbtn.png %}
|
||||
[commit-directly]: {{ site.baseurl }}{% link /assets/commit-directly.png %}
|
||||
[files-changed]: {{ site.baseurl }}{% link /assets/files-changed.png %}
|
||||
[more-commits]: {{ site.baseurl }}{% link /assets/more-commits.png %}
|
||||
[github flow]: https://guides.github.com/introduction/flow/
|
||||
[lint-tool]: {{ site.baseurl }}/writing-tests/lint-tool.html
|
||||
10
tests/wpt/web-platform-tests/docs/appendix/index.md
Normal file
10
tests/wpt/web-platform-tests/docs/appendix/index.md
Normal file
|
|
@ -0,0 +1,10 @@
|
|||
# Appendices
|
||||
|
||||
```eval_rst
|
||||
.. toctree ::
|
||||
:maxdepth: 1
|
||||
|
||||
github-intro
|
||||
reverting
|
||||
test-templates
|
||||
```
|
||||
|
|
@ -1,8 +1,5 @@
|
|||
---
|
||||
layout: page
|
||||
title: Reverting Changes
|
||||
order: 3
|
||||
---
|
||||
# Reverting Changes
|
||||
|
||||
Testing is imperfect and from time to time changes are merged into master which
|
||||
break things for users of web-platform-tests. Such breakage can include:
|
||||
|
||||
|
|
@ -1,8 +1,5 @@
|
|||
---
|
||||
layout: page
|
||||
title: Test Templates
|
||||
order: 2
|
||||
---
|
||||
# Test Templates
|
||||
|
||||
This page contains templates for creating tests. The template syntax
|
||||
is compatible with several popular editors including TextMate, Sublime
|
||||
Text, and emacs' YASnippet mode.
|
||||
|
|
@ -1,172 +0,0 @@
|
|||
---
|
||||
# Only the main Sass file needs front matter (the dashes are enough)
|
||||
---
|
||||
@charset "utf-8";
|
||||
|
||||
// Namespaces
|
||||
@namespace svg "http://www.w3.org/2000/svg";
|
||||
|
||||
// Our variables
|
||||
$base-font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
|
||||
$base-font-size: 16px;
|
||||
$base-font-weight: 400;
|
||||
$small-font-size: $base-font-size * 0.875;
|
||||
$base-line-height: 1.5;
|
||||
|
||||
$spacing-unit: 30px;
|
||||
|
||||
$text-color: #111;
|
||||
$background-color: #fdfdfd;
|
||||
$brand-color: #2a7ae2;
|
||||
|
||||
$grey-color: #828282;
|
||||
$grey-color-light: lighten($grey-color, 40%);
|
||||
$grey-color-dark: darken($grey-color, 25%);
|
||||
|
||||
// Width of the content area
|
||||
$content-width: 800px;
|
||||
|
||||
$on-palm: 600px;
|
||||
$on-laptop: 800px;
|
||||
|
||||
// Minima also includes a mixin for defining media queries.
|
||||
// Use media queries like this:
|
||||
// @include media-query($on-palm) {
|
||||
// .wrapper {
|
||||
// padding-right: $spacing-unit / 2;
|
||||
// padding-left: $spacing-unit / 2;
|
||||
// }
|
||||
// }
|
||||
|
||||
// Import partials from the `minima` theme.
|
||||
@import "minima";
|
||||
|
||||
// Our custom styles
|
||||
|
||||
// Home page blocks
|
||||
nav.home {
|
||||
ul {
|
||||
list-style: none;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
li {
|
||||
display: block;
|
||||
width: 50%;
|
||||
float: left;
|
||||
box-sizing: border-box;
|
||||
margin-bottom: $spacing-unit / 2;
|
||||
}
|
||||
|
||||
li:nth-child(2n+1) {
|
||||
clear: both;
|
||||
padding-right: $spacing-unit / 2;
|
||||
}
|
||||
|
||||
li:nth-child(2n) {
|
||||
padding-left: $spacing-unit / 2;
|
||||
}
|
||||
|
||||
li h2 {
|
||||
font-size: 20px;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
li p {
|
||||
font-size: $small-font-size;
|
||||
}
|
||||
|
||||
div {
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
svg {
|
||||
color: black;
|
||||
width: auto;
|
||||
height: $spacing-unit * 2;
|
||||
}
|
||||
|
||||
svg|*[fill] {
|
||||
fill: currentColor;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
// Site nav (class name is to avoid conflict with minima defined .site-nav)
|
||||
.wpt-site-nav {
|
||||
float: right;
|
||||
line-height: normal;
|
||||
position: absolute;
|
||||
top: 9px;
|
||||
right: $spacing-unit / 2;
|
||||
background-color: $background-color;
|
||||
border: 1px solid $grey-color-light;
|
||||
border-radius: 5px;
|
||||
text-align: right;
|
||||
max-height: calc(100vh - 38px);
|
||||
overflow: auto;
|
||||
|
||||
ul {
|
||||
list-style: none;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
ul ul {
|
||||
padding-right: $spacing-unit / 4;
|
||||
border-right: ($spacing-unit / 4 * 3) solid $grey-color-light;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.page-link {
|
||||
color: $text-color;
|
||||
line-height: $base-line-height;
|
||||
display: block;
|
||||
padding: 5px 10px;
|
||||
margin-left: 20px;
|
||||
}
|
||||
|
||||
.menu-icon {
|
||||
display: block;
|
||||
float: right;
|
||||
width: 36px;
|
||||
height: 26px;
|
||||
line-height: 0;
|
||||
padding-top: 10px;
|
||||
text-align: center;
|
||||
|
||||
> svg path {
|
||||
fill: $grey-color-dark;
|
||||
}
|
||||
}
|
||||
|
||||
.trigger {
|
||||
clear: both;
|
||||
display: none;
|
||||
padding-bottom: 5px;
|
||||
}
|
||||
|
||||
&:hover .trigger,
|
||||
&:focus .trigger {
|
||||
display: block;
|
||||
}
|
||||
|
||||
/* duplicated from above to avoid breaking UAs that don't support it */
|
||||
&:focus-within .trigger {
|
||||
display: block;
|
||||
}
|
||||
|
||||
.trigger[aria-hidden="true"] {
|
||||
display: none !important;
|
||||
}
|
||||
|
||||
.trigger[aria-hidden="false"] {
|
||||
display: block !important;
|
||||
}
|
||||
}
|
||||
|
||||
@import "gh-fork-ribbon";
|
||||
|
||||
.github-fork-ribbon:before {
|
||||
background-color: $grey-color-dark;
|
||||
}
|
||||
|
|
@ -1,20 +0,0 @@
|
|||
(function() {
|
||||
var site_nav = document.querySelector(".wpt-site-nav");
|
||||
var trigger = document.querySelector(".wpt-site-nav .trigger");
|
||||
|
||||
var show = function(e) {
|
||||
trigger.setAttribute("aria-hidden", "false");
|
||||
};
|
||||
|
||||
var hide_if_relatedTarget_elsewhere = function(e) {
|
||||
if (!site_nav.contains(e.relatedTarget)) {
|
||||
trigger.setAttribute("aria-hidden", "true");
|
||||
}
|
||||
};
|
||||
|
||||
site_nav.addEventListener("focus", show, false);
|
||||
site_nav.addEventListener("blur", hide_if_relatedTarget_elsewhere, true);
|
||||
|
||||
site_nav.addEventListener("mouseenter", show, false);
|
||||
site_nav.addEventListener("mouseleave", hide_if_relatedTarget_elsewhere, false);
|
||||
})();
|
||||
186
tests/wpt/web-platform-tests/docs/conf.py
Normal file
186
tests/wpt/web-platform-tests/docs/conf.py
Normal file
|
|
@ -0,0 +1,186 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
#
|
||||
# Configuration file for the Sphinx documentation builder.
|
||||
#
|
||||
# This file does only contain a selection of the most common options. For a
|
||||
# full list see the documentation:
|
||||
# http://www.sphinx-doc.org/en/master/config
|
||||
|
||||
# -- Path setup --------------------------------------------------------------
|
||||
|
||||
# If extensions (or modules to document with autodoc) are in another directory,
|
||||
# add these directories to sys.path here. If the directory is relative to the
|
||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||
#
|
||||
#import os
|
||||
#import sys
|
||||
#sys.path.insert(0, os.path.abspath('.'))
|
||||
|
||||
# -- Project information -----------------------------------------------------
|
||||
|
||||
project = u'web-platform-tests'
|
||||
copyright = u'2019, wpt contributors'
|
||||
author = u'wpt contributors'
|
||||
|
||||
# The short X.Y version
|
||||
version = u''
|
||||
# The full version, including alpha/beta/rc tags
|
||||
release = u''
|
||||
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
|
||||
# If your documentation needs a minimal Sphinx version, state it here.
|
||||
#
|
||||
# needs_sphinx = '1.0'
|
||||
|
||||
# Add any Sphinx extension module names here, as strings. They can be
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
|
||||
# ones.
|
||||
extensions = [
|
||||
'recommonmark'
|
||||
]
|
||||
|
||||
# Add any paths that contain templates here, relative to this directory.
|
||||
templates_path = ['_templates']
|
||||
|
||||
# The suffix(es) of source filenames.
|
||||
# You can specify multiple suffix as a list of string:
|
||||
#
|
||||
# source_suffix = ['.rst', '.md']
|
||||
source_suffix = '.rst'
|
||||
|
||||
# The master toctree document.
|
||||
master_doc = 'index'
|
||||
|
||||
# The language for content autogenerated by Sphinx. Refer to documentation
|
||||
# for a list of supported languages.
|
||||
#
|
||||
# This is also used if you do content translation via gettext catalogs.
|
||||
# Usually you set "language" from the command line for these cases.
|
||||
language = None
|
||||
|
||||
# List of patterns, relative to source directory, that match files and
|
||||
# directories to ignore when looking for source files.
|
||||
# This pattern also affects html_static_path and html_extra_path.
|
||||
exclude_patterns = [
|
||||
'_build',
|
||||
'Thumbs.db',
|
||||
'.DS_Store',
|
||||
'tools/.tox',
|
||||
'tools/third_party',
|
||||
'_venv'
|
||||
]
|
||||
|
||||
# Enable inline reStructured Text within Markdown-formatted files
|
||||
# https://recommonmark.readthedocs.io/en/latest/auto_structify.html
|
||||
from recommonmark.transform import AutoStructify
|
||||
def setup(app):
|
||||
app.add_transform(AutoStructify)
|
||||
|
||||
# The name of the Pygments (syntax highlighting) style to use.
|
||||
pygments_style = None
|
||||
|
||||
|
||||
# -- Options for HTML output -------------------------------------------------
|
||||
|
||||
# The theme to use for HTML and HTML Help pages. See the documentation for
|
||||
# a list of builtin themes.
|
||||
#
|
||||
html_theme = 'alabaster'
|
||||
|
||||
# Theme options are theme-specific and customize the look and feel of a theme
|
||||
# further. For a list of options available for each theme, see the
|
||||
# documentation.
|
||||
#
|
||||
# html_theme_options = {}
|
||||
|
||||
# Add any paths that contain custom static files (such as style sheets) here,
|
||||
# relative to this directory. They are copied after the builtin static files,
|
||||
# so a file named "default.css" will overwrite the builtin "default.css".
|
||||
html_static_path = ['assets']
|
||||
|
||||
# Custom sidebar templates, must be a dictionary that maps document names
|
||||
# to template names.
|
||||
#
|
||||
# The default sidebars (for documents that don't match any pattern) are
|
||||
# defined by theme itself. Builtin themes are using these templates by
|
||||
# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
|
||||
# 'searchbox.html']``.
|
||||
#
|
||||
# html_sidebars = {}
|
||||
|
||||
|
||||
# -- Options for HTMLHelp output ---------------------------------------------
|
||||
|
||||
# Output file base name for HTML help builder.
|
||||
htmlhelp_basename = 'web-platform-testsdoc'
|
||||
|
||||
|
||||
# -- Options for LaTeX output ------------------------------------------------
|
||||
|
||||
latex_elements = {
|
||||
# The paper size ('letterpaper' or 'a4paper').
|
||||
#
|
||||
# 'papersize': 'letterpaper',
|
||||
|
||||
# The font size ('10pt', '11pt' or '12pt').
|
||||
#
|
||||
# 'pointsize': '10pt',
|
||||
|
||||
# Additional stuff for the LaTeX preamble.
|
||||
#
|
||||
# 'preamble': '',
|
||||
|
||||
# Latex figure (float) alignment
|
||||
#
|
||||
# 'figure_align': 'htbp',
|
||||
}
|
||||
|
||||
# Grouping the document tree into LaTeX files. List of tuples
|
||||
# (source start file, target name, title,
|
||||
# author, documentclass [howto, manual, or own class]).
|
||||
latex_documents = [
|
||||
(master_doc, 'web-platform-tests.tex', u'web-platform-tests Documentation',
|
||||
u'wpt contributors', 'manual'),
|
||||
]
|
||||
|
||||
|
||||
# -- Options for manual page output ------------------------------------------
|
||||
|
||||
# One entry per manual page. List of tuples
|
||||
# (source start file, name, description, authors, manual section).
|
||||
man_pages = [
|
||||
(master_doc, 'web-platform-tests', u'web-platform-tests Documentation',
|
||||
[author], 1)
|
||||
]
|
||||
|
||||
|
||||
# -- Options for Texinfo output ----------------------------------------------
|
||||
|
||||
# Grouping the document tree into Texinfo files. List of tuples
|
||||
# (source start file, target name, title, author,
|
||||
# dir menu entry, description, category)
|
||||
texinfo_documents = [
|
||||
(master_doc, 'web-platform-tests', u'web-platform-tests Documentation',
|
||||
author, 'web-platform-tests', 'One line description of project.',
|
||||
'Miscellaneous'),
|
||||
]
|
||||
|
||||
|
||||
# -- Options for Epub output -------------------------------------------------
|
||||
|
||||
# Bibliographic Dublin Core info.
|
||||
epub_title = project
|
||||
|
||||
# The unique identifier of the text. This can be a ISBN number
|
||||
# or the project homepage.
|
||||
#
|
||||
# epub_identifier = ''
|
||||
|
||||
# A unique identification for the text.
|
||||
#
|
||||
# epub_uid = ''
|
||||
|
||||
# A list of files that should not be packed into the epub file.
|
||||
epub_exclude_files = ['search.html']
|
||||
|
|
@ -1,76 +0,0 @@
|
|||
---
|
||||
layout: page
|
||||
---
|
||||
|
||||
<nav class="home">
|
||||
<ul>
|
||||
<li>
|
||||
<a href="{{ site.baseurl }}{% link introduction.md %}">
|
||||
<div>
|
||||
{% include svg/octicons/tools.svg %}
|
||||
</div>
|
||||
<h2>
|
||||
Introduction
|
||||
</h2>
|
||||
</a>
|
||||
<p>You should probably start here! Contains a high-level
|
||||
overview of the testsuite and how to get set up locally.</p>
|
||||
</li>
|
||||
<li>
|
||||
<a href="{{ site.baseurl }}{% link _writing-tests/index.md %}">
|
||||
<div>
|
||||
{% include svg/octicons/pencil.svg %}
|
||||
</div>
|
||||
<h2>
|
||||
Writing Tests
|
||||
</h2>
|
||||
</a>
|
||||
<p>Guidelines and advice for those writing tests.</p>
|
||||
</li>
|
||||
<li>
|
||||
<a href="{{ site.baseurl }}{% link _running-tests/index.md %}">
|
||||
<div>
|
||||
{% include svg/triangle-right.svg %}
|
||||
</div>
|
||||
<h2>
|
||||
Running Tests
|
||||
</h2>
|
||||
</a>
|
||||
<p>How to run tests, either those you just wrote or those that
|
||||
already exist.</p>
|
||||
</li>
|
||||
<li>
|
||||
<a href="{{ site.baseurl }}{% link _reviewing-tests/index.md %}">
|
||||
<div>
|
||||
{% include svg/octicons/git-pull-request.svg %}
|
||||
</div>
|
||||
<h2>
|
||||
Reviewing Tests
|
||||
</h2>
|
||||
</a>
|
||||
<p>Some hints and tips for those reviewing tests.</p>
|
||||
</li>
|
||||
<li>
|
||||
<a href="{{ site.baseurl }}{% link _admin/index.md %}">
|
||||
<div>
|
||||
{% include svg/octicons/key.svg %}
|
||||
</div>
|
||||
<h2>
|
||||
Administration
|
||||
</h2>
|
||||
</a>
|
||||
<p>Instruction for project administrators.</p>
|
||||
</li>
|
||||
<li>
|
||||
<a href="{{ site.baseurl }}{% link _appendix/index.md %}">
|
||||
<div>
|
||||
{% include svg/octicons/package.svg %}
|
||||
</div>
|
||||
<h2>
|
||||
Appendices
|
||||
</h2>
|
||||
</a>
|
||||
<p>A load of other useful resources.</p>
|
||||
</li>
|
||||
</ul>
|
||||
</nav>
|
||||
12
tests/wpt/web-platform-tests/docs/index.rst
Normal file
12
tests/wpt/web-platform-tests/docs/index.rst
Normal file
|
|
@ -0,0 +1,12 @@
|
|||
web-platform-tests documentation
|
||||
================================
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
introduction
|
||||
running-tests/index
|
||||
writing-tests/index
|
||||
reviewing-tests/index
|
||||
admin/index
|
||||
appendix/index
|
||||
|
|
@ -1,7 +1,4 @@
|
|||
---
|
||||
layout: page
|
||||
title: Introduction
|
||||
---
|
||||
# Introduction
|
||||
|
||||
web-platform-tests is a W3C-coordinated effort to build a
|
||||
cross-browser testsuite for the majority of
|
||||
|
|
@ -108,10 +105,10 @@ free to add yourself to the META.yml file!
|
|||
[IRC]: irc://irc.w3.org:6667/testing
|
||||
[web irc]: http://irc.w3.org
|
||||
|
||||
[reftests]: {{ site.baseurl }}{% link _writing-tests/reftests.md %}
|
||||
[testharness.js]: {{ site.baseurl }}{% link _writing-tests/testharness.md %}
|
||||
[visual]: {{ site.baseurl }}{% link _writing-tests/visual.md %}
|
||||
[manual]: {{ site.baseurl }}{% link _writing-tests/manual.md %}
|
||||
[github-intro]: {{ site.baseurl }}{% link _appendix/github-intro.md %}
|
||||
[running-from-local-system]: {{ site.baseurl}}{% link _running-tests/from-local-system.md %}
|
||||
[wdspec]: {{ site.baseurl }}{% link _writing-tests/wdspec.md %}
|
||||
[reftests]: writing-tests/reftests
|
||||
[testharness.js]: writing-tests/testharness
|
||||
[visual]: writing-tests/visual
|
||||
[manual]: writing-tests/manual
|
||||
[github-intro]: appendix/github-intro
|
||||
[running-from-local-system]: running-tests/from-local-system
|
||||
[wdspec]: writing-tests/wdspec
|
||||
|
|
|
|||
35
tests/wpt/web-platform-tests/docs/make.bat
Normal file
35
tests/wpt/web-platform-tests/docs/make.bat
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
@ECHO OFF
|
||||
|
||||
pushd %~dp0
|
||||
|
||||
REM Command file for Sphinx documentation
|
||||
|
||||
if "%SPHINXBUILD%" == "" (
|
||||
set SPHINXBUILD=sphinx-build
|
||||
)
|
||||
set SOURCEDIR=.
|
||||
set BUILDDIR=_build
|
||||
|
||||
if "%1" == "" goto help
|
||||
|
||||
%SPHINXBUILD% >NUL 2>NUL
|
||||
if errorlevel 9009 (
|
||||
echo.
|
||||
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
|
||||
echo.installed, then set the SPHINXBUILD environment variable to point
|
||||
echo.to the full path of the 'sphinx-build' executable. Alternatively you
|
||||
echo.may add the Sphinx directory to PATH.
|
||||
echo.
|
||||
echo.If you don't have Sphinx installed, grab it from
|
||||
echo.http://sphinx-doc.org/
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
|
||||
goto end
|
||||
|
||||
:help
|
||||
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
|
||||
|
||||
:end
|
||||
popd
|
||||
2
tests/wpt/web-platform-tests/docs/requirements.txt
Normal file
2
tests/wpt/web-platform-tests/docs/requirements.txt
Normal file
|
|
@ -0,0 +1,2 @@
|
|||
recommonmark==0.5.0
|
||||
Sphinx==1.8.5
|
||||
|
|
@ -1,126 +1,102 @@
|
|||
---
|
||||
layout: page
|
||||
title: Review Checklist
|
||||
order: 2
|
||||
---
|
||||
# Review Checklist
|
||||
|
||||
The following checklist is provided as a guideline to assist in reviewing
|
||||
tests; in case of any contradiction with requirements stated elsewhere in the
|
||||
documentation it should be ignored
|
||||
(please [file a bug](https://github.com/web-platform-tests/wpt/issues/new)!).
|
||||
|
||||
As noted on the [reviewing tests][review index] page, nits need not block PRs
|
||||
As noted on the [reviewing tests](index) page, nits need not block PRs
|
||||
from landing.
|
||||
|
||||
|
||||
## All tests
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The CI jobs on the pull request have passed.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
It is obvious what the test is trying to test.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The test passes when it's supposed to pass.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The test fails when it's supposed to fail.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The test is testing what it thinks it's testing.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The spec backs up the expected behavior in the test.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
The test is automated as either [reftest][reftest] or
|
||||
a [script test][scripttest] unless there's a very good reason for it not to be.
|
||||
<label><input type="checkbox">
|
||||
The test is automated as either [reftest](../writing-tests/reftests) or
|
||||
a [script test](../writing-tests/testharness) unless there's a very good reason for it not to be.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The test does not use external resources.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The test does not use proprietary features (vendor-prefixed or otherwise).
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The test does not contain commented-out code.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The test is placed in the relevant directory.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The test has a reasonable and concise filename.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
If the test needs code running on the server side, the server code must be
|
||||
written in Python, and the Python code must not do anything potentially unsafe.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
If the test needs to be run in some non-standard configuration or needs user
|
||||
interaction, it is a manual test.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
**Nit**: The title is descriptive but not too wordy.
|
||||
</label>
|
||||
|
||||
|
||||
## Reftests Only
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The reference file is accurate and will render pixel-perfect
|
||||
identically to the test on all platforms.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The reference file uses a different technique that won't fail in
|
||||
the same way as the test.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The test and reference render within a 800x600 viewport, only displaying
|
||||
scrollbars if their presence is being tested.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
**Nit**: The test has a self-describing statement.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
**Nit**: The self-describing statement is accurate, precise, simple, and
|
||||
self-explanatory. Someone with no technical knowledge should be able to say
|
||||
whether the test passed or failed within a few seconds, and not need to spend
|
||||
|
|
@ -130,67 +106,52 @@ several minutes thinking or asking questions.
|
|||
|
||||
## Script Tests Only
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The number of tests in each file and the test names are consistent
|
||||
across runs and browsers. It is best to avoid the pattern where there is
|
||||
a test that asserts that the feature is supported and bails out without
|
||||
running the rest of the tests in the file if it isn't.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The test avoids patterns that make it less likely to be stable.
|
||||
In particular, tests should avoid setting internal timeouts, since the
|
||||
time taken to run it may vary on different devices; events should be used
|
||||
instead (if at all possible).
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The test uses the most specific asserts possible (e.g. doesn't use
|
||||
`assert_true` for everything).
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The test uses `idlharness.js` if it is testing basic IDL-defined behavior.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
**Nit**: Tests in a single file are separated by one empty line.
|
||||
</label>
|
||||
|
||||
|
||||
## Visual Tests Only
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The test has a self-describing statement.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The self-describing statement is accurate, precise, simple, and
|
||||
self-explanatory. Someone with no technical knowledge should be able to say
|
||||
whether the test passed or failed within a few seconds, and not need to spend
|
||||
several minutes thinking or asking questions.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The test renders within a 800x600 viewport, only displaying scrollbars if their
|
||||
presence is being tested.
|
||||
</label>
|
||||
|
||||
<label>
|
||||
<input type="checkbox">
|
||||
<label><input type="checkbox">
|
||||
The test renders to a fixed, static page with no animation.
|
||||
</label>
|
||||
|
||||
|
||||
[review index]: {{ site.baseurl }}{% link _reviewing-tests/index.md %}
|
||||
[general guidelines]: {{ site.baseurl }}{% link _writing-tests/general-guidelines.md %}
|
||||
[reftest]: {{ site.baseurl }}{% link _writing-tests/reftests.md %}
|
||||
[scripttest]: {{ site.baseurl }}{% link _writing-tests/testharness.md %}
|
||||
|
|
@ -1,8 +1,4 @@
|
|||
---
|
||||
layout: page
|
||||
title: Email Filters
|
||||
order: 3
|
||||
---
|
||||
# Email Filters
|
||||
|
||||
See the [GitHub support page](https://help.github.com/articles/about-email-notifications/)
|
||||
for how to filter certain types of email notifications. However, the most
|
||||
|
|
@ -1,18 +1,21 @@
|
|||
---
|
||||
layout: page
|
||||
title: Reviewing Tests
|
||||
order: -1
|
||||
---
|
||||
# Reviewing Tests
|
||||
|
||||
In order to encourage a high level of quality in the W3C test
|
||||
suites, test contributions must be reviewed by a peer.
|
||||
|
||||
```eval_rst
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
checklist
|
||||
email
|
||||
```
|
||||
|
||||
## Test Review Policy
|
||||
|
||||
The reviewer can be anyone (other than the original test author) that
|
||||
has the required experience with both the spec under test and with
|
||||
the [general test guidelines][general guidelines].
|
||||
the [general test guidelines](../writing-tests/general-guidelines).
|
||||
|
||||
The review must happen in public, but there is no requirement for it
|
||||
to happen in any specific location. In particular if a vendor is
|
||||
|
|
@ -32,7 +35,4 @@ from vendors it is frequently the case that the author has moved on to
|
|||
working on other things as tests frequently only get pushed upstream
|
||||
once the code lands in their implementation.
|
||||
|
||||
To assist with test reviews, a [review checklist][] is available.
|
||||
|
||||
[general guidelines]: {{ site.baseurl }}{% link _writing-tests/general-guidelines.md %}
|
||||
[review checklist]: {{ site.baseurl }}{% link _reviewing-tests/checklist.md %}
|
||||
To assist with test reviews, a [review checklist](checklist) is available.
|
||||
|
|
@ -1,7 +1,5 @@
|
|||
---
|
||||
layout: page
|
||||
title: Chrome
|
||||
---
|
||||
# Chrome
|
||||
|
||||
When running Chrome, there are some useful command line arguments.
|
||||
|
||||
You can inform `wpt` of the release channel of Chrome using `--channel`.
|
||||
|
|
@ -1,7 +1,5 @@
|
|||
---
|
||||
layout: page
|
||||
title: Chrome for Android
|
||||
---
|
||||
# Chrome for Android
|
||||
|
||||
To run WPT on Chrome on an Android device, some additional set up is required.
|
||||
|
||||
First of all, as usual Android development, we need to have `adb` and be able to
|
||||
|
|
@ -1,7 +1,4 @@
|
|||
---
|
||||
layout: page
|
||||
title: Writing Your Own Runner
|
||||
---
|
||||
# Writing Your Own Runner
|
||||
|
||||
Most test runners have two stages: finding all tests, followed by
|
||||
executing them (or a subset thereof).
|
||||
|
|
@ -1,7 +1,4 @@
|
|||
---
|
||||
layout: page
|
||||
title: Running Tests from the Local System
|
||||
---
|
||||
# Running Tests from the Local System
|
||||
|
||||
The tests are designed to be run from your local computer.
|
||||
|
||||
|
|
@ -75,7 +72,7 @@ http://web-platform.test:8000/<br>
|
|||
https://web-platform.test:8443/ *
|
||||
|
||||
This server has all the capabilities of the publicly-deployed version--see
|
||||
[Running the Tests from the Web][from-web].
|
||||
[Running the Tests from the Web](from-web).
|
||||
|
||||
\**See [Trusting Root CA](https://github.com/web-platform-tests/wpt/blob/master/README.md#trusting-root-ca)*
|
||||
|
||||
|
|
@ -102,13 +99,10 @@ customising the test run:
|
|||
|
||||
Additional browser-specific documentation:
|
||||
|
||||
* [Chrome][chrome]
|
||||
```eval_rst
|
||||
.. toctree::
|
||||
|
||||
* [Chrome for Android][chrome android]
|
||||
|
||||
* [Safari][safari]
|
||||
|
||||
[from-web]: {{ site.baseurl }}{% link _running-tests/from-web.md %}
|
||||
[chrome]: {{ site.baseurl }}{% link _running-tests/chrome.md %}
|
||||
[chrome android]: {{ site.baseurl }}{% link _running-tests/chrome_android.md %}
|
||||
[safari]: {{ site.baseurl }}{% link _running-tests/safari.md %}
|
||||
chrome
|
||||
chrome_android
|
||||
safari
|
||||
```
|
||||
|
|
@ -1,7 +1,4 @@
|
|||
---
|
||||
layout: page
|
||||
title: Running Tests from the Web
|
||||
---
|
||||
# Running Tests from the Web
|
||||
|
||||
Tests that have been merged on GitHub are mirrored at [http://w3c-test.org/][w3c-test].
|
||||
[On properly-configured systems](from-local-system), local files may also be
|
||||
|
|
@ -23,4 +20,3 @@ Because it runs entirely in-browser, this runner cannot deal with
|
|||
edge-cases like tests that cause the browser to crash or hang.
|
||||
|
||||
[w3c-test]: http://w3c-test.org
|
||||
[from-local-system]: {{ site.baseurl }}{% link _running-tests/from-local-system.md %}
|
||||
19
tests/wpt/web-platform-tests/docs/running-tests/index.md
Normal file
19
tests/wpt/web-platform-tests/docs/running-tests/index.md
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
# Running Tests
|
||||
|
||||
```eval_rst
|
||||
.. toctree::
|
||||
|
||||
from-web
|
||||
from-local-system
|
||||
custom-runner
|
||||
```
|
||||
|
||||
The simplest way to run the tests is via the public website. More detail on
|
||||
that approach is available in [Running tests from the Web](from-web).
|
||||
|
||||
Contributors who are interested in modifying and creating tests should refer to
|
||||
[Running Tests from the Local System](from-local-system).
|
||||
|
||||
Advanced use cases may call for a customized method of executing the tests.
|
||||
Guidelines for writing a custom "runner" are available at [Writing Your Own
|
||||
Runner](custom-runner).
|
||||
|
|
@ -1,7 +1,5 @@
|
|||
---
|
||||
layout: page
|
||||
title: Safari
|
||||
---
|
||||
# Safari
|
||||
|
||||
To run Safari on macOS, some manual setup is required. Some steps are different
|
||||
for Safari and Safari Technology Preview.
|
||||
|
||||
|
|
@ -1,8 +1,4 @@
|
|||
---
|
||||
layout: page
|
||||
title: The Ahem Font
|
||||
order: 11
|
||||
---
|
||||
# The Ahem Font
|
||||
|
||||
A font called [Ahem][ahem-readme] has been developed which consists of
|
||||
some very well defined glyphs of precise sizes and shapes; it is
|
||||
|
|
@ -1,8 +1,4 @@
|
|||
---
|
||||
layout: page
|
||||
title: Test Assumptions
|
||||
order: 13
|
||||
---
|
||||
# Test Assumptions
|
||||
|
||||
The tests make a number of assumptions of the user agent, and new
|
||||
tests can freely rely on these assumptions being true:
|
||||
|
|
@ -1,8 +1,5 @@
|
|||
---
|
||||
layout: page
|
||||
title: CSS Metadata
|
||||
order: 14
|
||||
---
|
||||
# CSS Metadata
|
||||
|
||||
CSS tests have some additional requirements for metadata.
|
||||
|
||||
### Specification Links
|
||||
|
|
@ -1,8 +1,5 @@
|
|||
---
|
||||
layout: page
|
||||
title: CSS User Stylesheets
|
||||
order: 15
|
||||
---
|
||||
# CSS User Stylesheets
|
||||
|
||||
Some test may require special user style sheets to be applied in order
|
||||
for the case to be verified. In order for proper indications and
|
||||
prerequisite to be displayed every user style sheet should contain the
|
||||
|
|
@ -1,8 +1,4 @@
|
|||
---
|
||||
layout: page
|
||||
title: File Name Flags
|
||||
order: 2
|
||||
---
|
||||
# File Name Flags
|
||||
|
||||
The test filename is significant in determining the type of test it
|
||||
contains, and enabling specific optional features. This page documents
|
||||
|
|
@ -52,7 +48,7 @@ themselves precede any test type flag, but are otherwise unordered.
|
|||
|
||||
`.any`
|
||||
: (js files only) Indicates that the file generates tests in which it
|
||||
is [run in multiple scopes][multi-global-tests].
|
||||
is [run in multiple scopes](testharness).
|
||||
|
||||
`.optional`
|
||||
: Indicates that a test makes assertions about optional behavior in a
|
||||
|
|
@ -69,5 +65,4 @@ themselves precede any test type flag, but are otherwise unordered.
|
|||
|
||||
|
||||
[server-side substitution]: https://wptserve.readthedocs.io/en/latest/pipes.html#sub
|
||||
[multi-global-tests]: {{ site.baseurl }}{% link _writing-tests/testharness.md %}#multi-global-tests
|
||||
[RFC 2119]: https://tools.ietf.org/html/rfc2119
|
||||
|
|
@ -1,8 +1,4 @@
|
|||
---
|
||||
layout: page
|
||||
title: General Test Guidelines
|
||||
order: 2
|
||||
---
|
||||
# General Test Guidelines
|
||||
|
||||
### File Paths and Names
|
||||
|
||||
|
|
@ -131,7 +127,7 @@ typically configured with access to external resources disabled, so
|
|||
tests that try to access them will fail. Where tests want to use
|
||||
multiple hosts this is possible through a known set of subdomains and
|
||||
the
|
||||
[text substitution features of wptserve]({{ site.baseurl }}{% link _writing-tests/server-features.md %}#tests-involving-multiple-origins).
|
||||
[text substitution features of wptserve]({{ site.baseurl }}{% link writing-tests/server-features.md %}#tests-involving-multiple-origins).
|
||||
|
||||
|
||||
### Be Self-Describing
|
||||
|
|
@ -181,10 +177,10 @@ for CSS have some additional requirements for:
|
|||
* [User style sheets][css-user-styles].
|
||||
|
||||
|
||||
[server features]: {{ site.baseurl }}{% link _writing-tests/server-features.md %}
|
||||
[assumptions]: {{ site.baseurl }}{% link _writing-tests/assumptions.md %}
|
||||
[ahem]: {{ site.baseurl }}{% link _writing-tests/ahem.md %}
|
||||
[lint-tool]: {{ site.baseurl }}{% link _writing-tests/lint-tool.md %}
|
||||
[css-metadata]: {{ site.baseurl }}{% link _writing-tests/css-metadata.md %}
|
||||
[css-user-styles]: {{ site.baseurl }}{% link _writing-tests/css-user-styles.md %}
|
||||
[file-name-flags]: {{ site.baseurl }}{% link _writing-tests/file-names.md %}
|
||||
[server features]: server-features
|
||||
[assumptions]: assumptions
|
||||
[ahem]: ahem
|
||||
[lint-tool]: lint-tool
|
||||
[css-metadata]: css-metadata
|
||||
[css-user-styles]: css-user-styles
|
||||
[file-name-flags]: file-names
|
||||
|
|
@ -1,8 +1,4 @@
|
|||
---
|
||||
layout: page
|
||||
title: Writing H2 Tests
|
||||
order: 2
|
||||
---
|
||||
# Writing H2 Tests
|
||||
|
||||
> <b>Important:</b> The HTTP/2.0 server requires you to have Python 2.7.10+
|
||||
and OpenSSL 1.0.2+. This is because HTTP/2.0 is negotiated using the
|
||||
|
|
@ -1,8 +1,4 @@
|
|||
---
|
||||
layout: page
|
||||
title: idlharness.js API
|
||||
order: 6
|
||||
---
|
||||
# idlharness.js API
|
||||
|
||||
## Introduction ##
|
||||
|
||||
78
tests/wpt/web-platform-tests/docs/writing-tests/index.md
Normal file
78
tests/wpt/web-platform-tests/docs/writing-tests/index.md
Normal file
|
|
@ -0,0 +1,78 @@
|
|||
# Writing Tests
|
||||
|
||||
If you haven't already, it's strongly recommended to read
|
||||
the [introduction](../introduction) first, as it introduces the various test types.
|
||||
|
||||
There's also a load of [general guidelines](general-guidelines) that apply to all tests.
|
||||
|
||||
```eval_rst
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
general-guidelines
|
||||
ahem
|
||||
assumptions
|
||||
css-metadata
|
||||
css-user-styles
|
||||
file-names
|
||||
h2tests
|
||||
lint-tool
|
||||
manual
|
||||
reftests
|
||||
rendering
|
||||
server-features
|
||||
submission-process
|
||||
testharness
|
||||
visual
|
||||
wdspec
|
||||
```
|
||||
|
||||
## Test Type
|
||||
|
||||
Tests in this project use a few different approaches to verify expected
|
||||
behavior. The tests can be classified based on the way they express
|
||||
expectations:
|
||||
|
||||
* Rendering tests should be used to verify that the browser graphically
|
||||
displays pages as expected. See the [rendering test guidelines][rendering]
|
||||
for tips on how to write great rendering tests. There are a few different
|
||||
ways to write rendering tests:
|
||||
|
||||
* [Reftests](reftests) should be used to test rendering and layout. They
|
||||
consist of two or more pages with assertions as to whether they render
|
||||
identically or not.
|
||||
|
||||
* [Visual tests](visual) should be used for checking rendering where there is
|
||||
a large number of conforming renderings such that reftests are impractical.
|
||||
They consist of a page that renders to final state at which point a
|
||||
screenshot can be taken and compared to an expected rendering for that user
|
||||
agent on that platform.
|
||||
|
||||
* [testharness.js](testharness) tests should be used (where possible!) for
|
||||
testing everything else. They are built with the testharness.js unit testing
|
||||
framework, and consist of assertions written in JavaScript.
|
||||
|
||||
* [wdspec](wdspec) tests are written in Python using
|
||||
[pytest](https://docs.pytest.org/en/latest/) and test [the WebDriver browser
|
||||
automation protocol](https://w3c.github.io/webdriver/)
|
||||
|
||||
* [Manual tests](manual) are used as a last resort for anything that can't be
|
||||
tested using any of the above. They consist of a page that needs manual
|
||||
interaction or verification of the final result.
|
||||
|
||||
In general, there is a strong preference towards reftests and testharness.js
|
||||
tests types (as they can be easily run without human interaction), so they
|
||||
should be used in preference to the others even if it results in a
|
||||
somewhat cumbersome test; there is a far weaker preference between the
|
||||
two test types, and it is at times advisable to use testharness.js tests
|
||||
for things which would typically be tested using reftests but for
|
||||
which it would be overly cumbersome.
|
||||
|
||||
See [file names](file-names) for test types and features determined by the file names,
|
||||
and [server features](server-features) for advanced testing features.
|
||||
|
||||
## Submitting Tests
|
||||
|
||||
Once you've written tests, please submit them using
|
||||
the [typical GitHub Pull Request workflow](submission-process); please
|
||||
make sure you run the [`lint` script](lint-tool) before opening a pull request!
|
||||
|
|
@ -1,8 +1,5 @@
|
|||
---
|
||||
layout: page
|
||||
title: Lint Tool
|
||||
order: 9
|
||||
---
|
||||
# Lint Tool
|
||||
|
||||
We have a lint tool for catching common mistakes in test files. You can run
|
||||
it manually by running the `wpt lint` command from the root of your local
|
||||
web-platform-tests working directory like this:
|
||||
|
|
@ -1,8 +1,5 @@
|
|||
---
|
||||
layout: page
|
||||
title: Manual Tests
|
||||
order: 8
|
||||
---
|
||||
# Manual Tests
|
||||
|
||||
Some testing scenarios are intrinsically difficult to automate and
|
||||
require a human to run the test and check the pass condition.
|
||||
|
||||
|
|
@ -42,7 +39,7 @@ after the main filename but before the extension).
|
|||
|
||||
Manual tests must be
|
||||
fully
|
||||
[self-describing]({{ site.baseurl }}{% link _writing-tests/general-guidelines.md %}#be-self-describing).
|
||||
[self-describing](general-guidelines).
|
||||
It is particularly important for these tests that it is easy to
|
||||
determine the result from the information provided in the page to the
|
||||
tester, because a tester may have hundreds of tests to get through and
|
||||
|
|
@ -1,8 +1,4 @@
|
|||
---
|
||||
layout: page
|
||||
title: Reftests
|
||||
order: 3
|
||||
---
|
||||
# Reftests
|
||||
|
||||
Reftests are one of the primary tools for testing things relating to
|
||||
rendering; they are made up of the test and one or more other pages
|
||||
|
|
@ -225,6 +221,5 @@ except that the code behind it is different.
|
|||
<p>WAS</p>
|
||||
```
|
||||
|
||||
|
||||
[general guidelines]: {{ site.baseurl }}{% link _writing-tests/general-guidelines.md %}
|
||||
[rendering]: {{ site.baseurl }}{% link _writing-tests/rendering.md %}
|
||||
[general guidelines]: general-guidelines
|
||||
[rendering]: rendering
|
||||
|
|
@ -1,11 +1,7 @@
|
|||
---
|
||||
layout: page
|
||||
title: Rendering Test Guidelines
|
||||
order: 10
|
||||
---
|
||||
# Rendering Test Guidelines
|
||||
|
||||
There are a number of techniques typically used when writing rendering tests;
|
||||
these are especially using for [visual][] tests which need to be manually
|
||||
these are especially using for [visual](visual) tests which need to be manually
|
||||
judged and following common patterns makes it easier to correctly tell if a
|
||||
given test passed or not.
|
||||
|
||||
|
|
@ -88,8 +84,4 @@ Obviously, none of this can be taken as absolute rules, as testing of
|
|||
color-related functionality will necessitate using some of these
|
||||
colors!
|
||||
|
||||
|
||||
[ahem]: {{ site.baseurl }}{% link _writing-tests/ahem.md %}
|
||||
[visual]: {{ site.baseurl }}{% link _writing-tests/visual.md %}
|
||||
|
||||
[ref-filled-green-100px-square]: https://github.com/w3c/csswg-test/blob/master/reference/ref-filled-green-100px-square.xht
|
||||
|
|
@ -1,15 +1,11 @@
|
|||
---
|
||||
layout: page
|
||||
title: Server Features
|
||||
order: 12
|
||||
---
|
||||
# Server Features
|
||||
|
||||
## Advanced Testing Features
|
||||
|
||||
Certain test scenarios require more than just static HTML
|
||||
generation. This is supported through the
|
||||
[wptserve](http://wptserve.readthedocs.io) server, and controlled by
|
||||
[file name flags][file names]. Several scenarios in particular are common:
|
||||
[file name flags](file-names). Several scenarios in particular are common:
|
||||
|
||||
|
||||
### Tests Involving Multiple Origins
|
||||
|
|
@ -100,11 +96,8 @@ The server now has a prototype HTTP/2.0 server which gives you access to
|
|||
some of the HTTP/2.0 specific functionality. Currently, the server is off
|
||||
by default and needs to be run using `./wpt serve --h2` in order to enable it.
|
||||
The HTTP/2.0 server supports handlers that work per-frame; these, along with the
|
||||
API are documented in [Writing H2 Tests][h2tests]
|
||||
API are documented in [Writing H2 Tests](h2tests).
|
||||
|
||||
> <b>Important:</b> The HTTP/2.0 server requires you to have Python 2.7.10+
|
||||
and OpenSSL 1.0.2+. This is because HTTP/2.0 is negotiated using the
|
||||
[TLS ALPN](https://tools.ietf.org/html/rfc7301) extension, which is only supported in [OpenSSL 1.0.2](https://www.openssl.org/news/openssl-1.0.2-notes.html) and up.
|
||||
|
||||
[file names]: {{ site.baseurl }}{% link _writing-tests/file-names.md %}
|
||||
[h2tests]: {{ site.baseurl }}{% link _writing-tests/h2tests.md %}
|
||||
|
|
@ -1,8 +1,5 @@
|
|||
---
|
||||
layout: page
|
||||
title: Submitting Tests
|
||||
order: 16
|
||||
---
|
||||
# Submitting Tests
|
||||
|
||||
Test submission is via the typical [GitHub workflow][github flow]:
|
||||
|
||||
* Fork the [GitHub repository][repo] (and make sure you're still relatively in
|
||||
|
|
@ -34,7 +31,7 @@ on, e.g. `git checkout -b topic-name`
|
|||
request will be automatically merged.
|
||||
|
||||
For detailed guidelines on setup and each of these steps, please refer to the
|
||||
[Github Test Submission][github-intro] documentation.
|
||||
[Github Test Submission](../appendix/github-intro) documentation.
|
||||
|
||||
Hop on to the [mailing list][public-test-infra] or [IRC][]
|
||||
([webclient][web irc], join channel `#testing`) if you have an issue. There is
|
||||
|
|
@ -42,7 +39,6 @@ no need to announce your review request, as soon as you make a Pull Request
|
|||
GitHub will inform interested parties.
|
||||
|
||||
[repo]: https://github.com/web-platform-tests/wpt/
|
||||
[github-intro]: {{ site.baseurl }}{% link _appendix/github-intro.md %}
|
||||
[github flow]: https://guides.github.com/introduction/flow/
|
||||
[public-test-infra]: https://lists.w3.org/Archives/Public/public-test-infra/
|
||||
[IRC]: irc://irc.w3.org:6667/testing
|
||||
|
|
@ -1,10 +1,6 @@
|
|||
---
|
||||
layout: page
|
||||
title: testdriver.js Tutorial
|
||||
order: 8.6
|
||||
---
|
||||
# testdriver.js Tutorial
|
||||
|
||||
# Adding new commands to testdriver.js
|
||||
Adding new commands to testdriver.js
|
||||
|
||||
## Assumptions
|
||||
We assume the following in this writeup:
|
||||
|
|
@ -22,7 +18,7 @@ The first part of this will be browser agnostic, but later we will need to imple
|
|||
|
||||
## Code!
|
||||
|
||||
### [resources/testdriver.js](resources/testdriver.js)
|
||||
### [resources/testdriver.js](https://github.com/web-platform-tests/wpt/blob/master/resources/testdriver.js)
|
||||
|
||||
This is the main entry point the tests get. Here we need to add a function to the `test_driver` object that will call the `test_driver_internal` object.
|
||||
|
||||
|
|
@ -77,7 +73,7 @@ window.test_driver_internal = {
|
|||
```
|
||||
We will leave this unimplemented and override it in another file. Lets do that now!
|
||||
|
||||
### [wptrunner/wptrunner/testdriver-extra.js](tools/wptrunner/wptrunner/testdriver-extra.js)
|
||||
### [wptrunner/wptrunner/testdriver-extra.js](https://github.com/web-platform-tests/wpt/blob/master/tools/wptrunner/wptrunner/testdriver-extra.js)
|
||||
|
||||
This will be the default function called when invoking the test driver commands (sometimes it is overridden by testdriver-vendor.js, but this is outside the scope of this writeup).
|
||||
|
||||
|
|
@ -127,7 +123,7 @@ One limitation this introduces is that only one testdriver call can be made at o
|
|||
|
||||
Next, this is passed to the executor and protocol in wptrunner. Time to switch to Python!
|
||||
|
||||
[tools/wptrunner/wptrunner/executors/protocol.py](tools/wptrunner/wptrunner/executors/protocol.py)
|
||||
[tools/wptrunner/wptrunner/executors/protocol.py](https://github.com/web-platform-tests/wpt/blob/master/tools/wptrunner/wptrunner/executors/protocol.py)
|
||||
|
||||
```python
|
||||
class SetWindowRectProtocolPart(ProtocolPart):
|
||||
|
|
@ -149,7 +145,7 @@ class SetWindowRectProtocolPart(ProtocolPart):
|
|||
|
||||
Next we change the base executor.
|
||||
|
||||
[tools/wptrunner/wptrunner/executors/base.py](tools/wptrunner/wptrunner/executors/base.py)
|
||||
[tools/wptrunner/wptrunner/executors/base.py](https://github.com/web-platform-tests/wpt/blob/master/tools/wptrunner/wptrunner/executors/base.py)
|
||||
|
||||
```python
|
||||
class CallbackHandler(object):
|
||||
|
|
@ -194,7 +190,7 @@ Now we write the browser specific implementations.
|
|||
|
||||
### Chrome
|
||||
|
||||
We will use [executorselenium](tools/wptrunner/wptrunner/executors/executorselenium.py) and use the Selenium API (in the future there are plans to use the WebDriver API directly).
|
||||
We will use [executorselenium](https://github.com/web-platform-tests/wpt/blob/master/tools/wptrunner/wptrunner/executors/executorselenium.py) and use the Selenium API (in the future there are plans to use the WebDriver API directly).
|
||||
|
||||
There isn't too much work to do here, we just need to define a subclass of the protocol part we defined earlier.
|
||||
|
||||
|
|
@ -241,7 +237,7 @@ class SeleniumProtocol(Protocol):
|
|||
### Firefox
|
||||
We use the [set window rect](https://firefox-source-docs.mozilla.org/python/marionette_driver.html#marionette_driver.marionette.Marionette.set_window_rect) Marionette command.
|
||||
|
||||
We will use [executormarionette](tools/wptrunner/wptrunner/executors/executormarionette.py) and use the Marionette Python API.
|
||||
We will use [executormarionette](https://github.com/web-platform-tests/wpt/blob/master/tools/wptrunner/wptrunner/executors/executormarionette.py) and use the Marionette Python API.
|
||||
|
||||
We have little actual work to do here! We just need to define a subclass of the protocol part we defined earlier.
|
||||
|
||||
|
|
@ -314,7 +310,7 @@ promise_test(async t => {
|
|||
```
|
||||
### What about testdriver-vendor.js?
|
||||
|
||||
The file [testdriver-vendor.js](resources/testdriver-vendor.js) is the equivalent to testdriver-extra.js above, except it is
|
||||
The file [testdriver-vendor.js](https://github.com/web-platform-tests/wpt/blob/master/resources/testdriver-vendor.js) is the equivalent to testdriver-extra.js above, except it is
|
||||
run instead of testdriver-extra.js in browser specific test environments. For example, in [Chromium LayoutTests](https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/?q=LayoutTests&sq=package:chromium&dr).
|
||||
|
||||
### What if I need to return a value from my testdriver API?
|
||||
|
|
@ -1,15 +1,11 @@
|
|||
---
|
||||
layout: page
|
||||
title: testdriver.js Automation
|
||||
order: 8.5
|
||||
---
|
||||
# testdriver.js Automation
|
||||
|
||||
testdriver.js provides a means to automate tests that cannot be
|
||||
written purely using web platform APIs. Outside of automation
|
||||
contexts, it allows human operators to provide expected input
|
||||
manually (for operations which may be described in simple terms).
|
||||
|
||||
It is currently supported only for [testharness.js][testharness]
|
||||
It is currently supported only for [testharness.js](testharness)
|
||||
tests.
|
||||
|
||||
## API
|
||||
|
|
@ -126,4 +122,3 @@ To send special keys, one must send the respective key's codepoint. Since this u
|
|||
For example, to send the tab key you would send "\uE004".
|
||||
|
||||
[activation]: https://html.spec.whatwg.org/multipage/interaction.html#activation
|
||||
[testharness]: {{ site.baseurl }}{% link _writing-tests/testharness.md %}
|
||||
|
|
@ -1,8 +1,4 @@
|
|||
---
|
||||
layout: page
|
||||
title: testharness.js API
|
||||
order: 5
|
||||
---
|
||||
# testharness.js API
|
||||
|
||||
## Introduction ##
|
||||
|
||||
|
|
@ -1,8 +1,14 @@
|
|||
---
|
||||
layout: page
|
||||
title: testharness.js Tests
|
||||
order: 4
|
||||
---
|
||||
# testharness.js Tests
|
||||
|
||||
```eval_rst
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
idlharness
|
||||
testharness-api
|
||||
testdriver-tutorial
|
||||
testdriver
|
||||
```
|
||||
|
||||
testharness.js tests are the correct type of test to write in any
|
||||
situation where you are not specifically interested in the rendering
|
||||
|
|
@ -10,14 +16,14 @@ of a page, and where human interaction isn't required; these tests are
|
|||
written in JavaScript using a framework called `testharness.js`. It is
|
||||
documented in two sections:
|
||||
|
||||
* [testharness.js Documentation][testharness-api] — An introduction
|
||||
* [testharness.js Documentation](testharness-api) — An introduction
|
||||
to the library and a detailed API reference.
|
||||
|
||||
* [idlharness.js Documentation][idlharness] — A library for testing
|
||||
* [idlharness.js Documentation](idlharness) — A library for testing
|
||||
IDL interfaces using `testharness.js`.
|
||||
|
||||
See [server features][] for advanced testing features that are commonly used
|
||||
with testharness.js. See also the [general guidelines][] for all test types.
|
||||
See [server features](server-features) for advanced testing features that are commonly used
|
||||
with testharness.js. See also the [general guidelines](general-guidelines) for all test types.
|
||||
|
||||
## Variants
|
||||
|
||||
|
|
@ -199,9 +205,3 @@ Use `// META: variant=url-suffix` at the beginning of the resource. For example,
|
|||
// META: variant=
|
||||
// META: variant=?wss
|
||||
```
|
||||
|
||||
|
||||
[general guidelines]: {{ site.baseurl }}{% link _writing-tests/general-guidelines.md %}
|
||||
[testharness-api]: {{ site.baseurl }}{% link _writing-tests/testharness-api.md %}
|
||||
[idlharness]: {{ site.baseurl }}{% link _writing-tests/idlharness.md %}
|
||||
[server features]: {{ site.baseurl }}{% link _writing-tests/server-features.md %}
|
||||
|
|
@ -1,11 +1,7 @@
|
|||
---
|
||||
layout: page
|
||||
title: Visual Tests
|
||||
order: 7
|
||||
---
|
||||
# Visual Tests
|
||||
|
||||
Visual tests are typically used when testing rendering of things that
|
||||
cannot be tested with [reftests][].
|
||||
cannot be tested with [reftests](reftests).
|
||||
|
||||
Their main advantage of over manual tests is they can be verified using
|
||||
browser-specific and platform-specific screenshots; note, however, that many
|
||||
|
|
@ -18,15 +14,10 @@ Visuals tests are test files which have `-visual` at the end of their
|
|||
filename, before the extension. There is nothing needed in them to
|
||||
make them work.
|
||||
|
||||
They should follow the [general test guidelines][general guidelines],
|
||||
They should follow the [general test guidelines](general guidelines),
|
||||
especially noting the requirement to be self-describing (i.e., they
|
||||
must give a clear pass condition in their rendering).
|
||||
|
||||
Similarly, they should consider the [rendering test guidelines][rendering],
|
||||
Similarly, they should consider the [rendering test guidelines](rendering),
|
||||
especially those about color, to ensure those running the test don't
|
||||
incorrectly judge its result.
|
||||
|
||||
|
||||
[general guidelines]: {{ site.baseurl }}{% link _writing-tests/general-guidelines.md %}
|
||||
[reftests]: {{ site.baseurl }}{% link _writing-tests/reftests.md %}
|
||||
[rendering]: {{ site.baseurl }}{% link _writing-tests/rendering.md %}
|
||||
|
|
@ -1,8 +1,4 @@
|
|||
---
|
||||
layout: page
|
||||
title: wdspec tests
|
||||
order: -1
|
||||
---
|
||||
# wdspec tests
|
||||
|
||||
The term "wdspec" describes a type of test in WPT which verifies some aspect of
|
||||
[the WebDriver protocol](https://w3c.github.io/webdriver/). These tests are
|
||||
|
|
@ -15,9 +11,9 @@ command under test. For example, tests for [the Close Window
|
|||
command](https://w3c.github.io/webdriver/#close-window) are located in then
|
||||
`close_window` directory.
|
||||
|
||||
Similar to [testharness.js][] tests, wdspec tests contain within them any
|
||||
number of "sub-tests." Sub-tests are defined as Python functions whose name
|
||||
begins with `test_`, e.g. `test_stale_element`.
|
||||
Similar to [testharness.js](testharness) tests, wdspec tests contain within
|
||||
them any number of "sub-tests." Sub-tests are defined as Python functions whose
|
||||
name begins with `test_`, e.g. `test_stale_element`.
|
||||
|
||||
## The `webdriver` client library
|
||||
|
||||
|
|
@ -71,5 +67,3 @@ a WebDriver session) are defined in terms of Pytest "fixtures" and must be
|
|||
loaded accordingly. For more detail on how to define and use test fixtures,
|
||||
please refer to [the pytest project's documentation on the
|
||||
topic](https://docs.pytest.org/en/latest/fixture.html).
|
||||
|
||||
[testharness.js]: {{ site.baseurl }}{% link _writing-tests/testharness.md %}
|
||||
Loading…
Add table
Add a link
Reference in a new issue