mirror of
https://github.com/servo/servo.git
synced 2025-06-06 16:45:39 +00:00
103 lines
4.2 KiB
Markdown
103 lines
4.2 KiB
Markdown
This folder contains the web platform tests and the code required to integrate
|
|
them with Servo.
|
|
|
|
Contents
|
|
========
|
|
|
|
In particular, this folder contains:
|
|
|
|
* `config.ini`: some configuration for the wpt libraries;
|
|
* `include.ini`: the subset of tests we currently run;
|
|
* `run.py` and `run.sh`: glue code to run the tests in Servo;
|
|
* `metadata`: expected failures for the tests we run;
|
|
* `web-platform-tests`: submodule pointer to the actual tests.
|
|
|
|
Running the tests
|
|
=================
|
|
|
|
The simplest way to run the tests in Servo is `make check-wpt` in the build
|
|
directory. This will run the subset of JavaScript tests defined in
|
|
`include.ini` and log the output to stdout.
|
|
|
|
Options can be passed through to `run.sh` by using the `WPTARGS` environment
|
|
variable. Some useful options are:
|
|
|
|
* `--include`: specifies which test(s) to run.
|
|
For example, `--include=/dom` runs all the DOM tests, `--include=/dom/errors`
|
|
runs all the DOM error tests and
|
|
`--include=/dom/errors/DOMException-constants.html` runs one specific test.
|
|
(Note that this overrides `include.ini` completely.)
|
|
* `--processes`: specifies the number of parallel processes to use (default 1).
|
|
When this argument is passed, the runner will spawn multiple instances of
|
|
Servo simultaneously to run multiple tests in parallel for more efficiency
|
|
(especially on multi-core processors).
|
|
|
|
Running the tests without make
|
|
------------------------------
|
|
|
|
When avoiding `make` for some reason, one can run `run.py` directly. However,
|
|
this requires in the first place that the virtualenv has been set up (which can
|
|
be done by running `make check-wpt` in advance). Then run from the build
|
|
directory:
|
|
|
|
source _virtualenv/bin/activate
|
|
python $srcdir/src/test/wpt/run.py --config srcdir/src/test/wpt/config.ini
|
|
|
|
with any other desired arguments.
|
|
|
|
Running the tests manually
|
|
--------------------------
|
|
|
|
It can be useful to run a test without the interference of the test runner, for
|
|
example when using a debugger such as `gdb`. In that case, start the server by
|
|
first adding the following to the system's hosts file:
|
|
|
|
127.0.0.1 www.web-platform.test
|
|
127.0.0.1 www1.web-platform.test
|
|
127.0.0.1 www2.web-platform.test
|
|
127.0.0.1 web-platform.test
|
|
127.0.0.1 xn--n8j6ds53lwwkrqhv28a.web-platform.test
|
|
127.0.0.1 xn--lve-6lad.web-platform.test
|
|
|
|
and then running `python serve.py` from `src/tests/wpt/web-platform-tests`.
|
|
Then navigate Servo to `http://web-platform.test:8000/path/to/test`.
|
|
|
|
Updating test expectations
|
|
==========================
|
|
|
|
When fixing a bug that causes the result of a test to change, the expected
|
|
results for that test need to be changed. This can be done manually, by editing
|
|
the `.ini` file under the `metadata` folder that corresponds to the test. In
|
|
this case, remove the references to tests whose expectation is now `PASS`, and
|
|
remove `.ini` files that no longer contain any expectations.
|
|
|
|
When a larger number of changes is required, this process can be automated.
|
|
This first requires saving the raw, unformatted log from a test run, for
|
|
example by running `WPTARGS=--log-raw /tmp/servo.log make check-wpt`. Once the
|
|
log is saved, run from the build directory:
|
|
|
|
source _virtualenv/bin/activate
|
|
_virtualenv/bin/wptupdate \
|
|
--ignore-existing \
|
|
--config $srcdir/src/test/wpt/config.ini \
|
|
/tmp/servo.log
|
|
|
|
This will update the `.ini` files under the `metadata` folder; commit those
|
|
changes along with the code changes that require them.
|
|
|
|
Updating the upstream tests
|
|
===========================
|
|
|
|
In order to update the upstream tests, fetch the latest commits on the `master`
|
|
branch of the upstream `git@github.com:w3c/web-platform-tests.git` repository.
|
|
and create a new branch `servo_[current date]` with those commits. Then
|
|
cherry-pick our changes to the `resources` submodule; those should be the
|
|
latest commits on the branch currently used by Servo. If the `resources`
|
|
submodule has been updated upstream, this will also require cherry-picking the
|
|
changes there. Finally, push the `servo_[current date]` to our fork at
|
|
`git@github.com:servo/web-platform-tests.git` and create a pull request to the
|
|
Servo repository with:
|
|
* the submodule pointer update;
|
|
* an update to `MANIFEST.json` generated by running `tools/scripts/manifest.py`
|
|
in web-platform-tests;
|
|
* the updated test expectations.
|