Update the README files to reflect the changes to the update procedure.

This commit is contained in:
James Graham 2015-04-01 13:27:27 +01:00
parent d27cda44ce
commit 6b8aaf3c57
2 changed files with 115 additions and 47 deletions

41
python/README.md Normal file
View file

@ -0,0 +1,41 @@
This directory contains various Python modules used to support servo
development.
= mach =
The command dispatch framework used to wrap the build system and test
harnesses.
= mozdebug =
mozbase module containing information about various debuggers.
This can be updated by copying the latest version from
https://hg.mozilla.org/mozilla-central/file/tip/testing/mozbase/mozdebug
= mozinfo =
Mozbase module for extracting information about the host hardware /
os.
This can be updated by copying the latest version from
hg.mozilla.org/mozilla-central/file/tip/testing/mozbase/mozinfo
= mozlog =
A mozbase logging module required for wptrunner output and command
line arguments.
This can be updated by copying the latest version from
hg.mozilla.org/mozilla-central/file/tip/testing/mozbase/mozlog
= servo =
servo-specific python code e.g. implementations of mach commands. This
is the canonical repository for this code.
== toml ==
Python module for reading toml files.
This can be updated from https://github.com/uiri/toml

View file

@ -1,46 +1,54 @@
This folder contains the web platform tests and the code required to integrate This folder contains the web platform tests, CSS WG tests, and the
them with Servo. code required to integrate them with Servo.
Contents Contents
======== ========
In particular, this folder contains: In particular, this folder contains:
* `config.ini`: some configuration for the wpt libraries; * `config.ini`: some configuration for the web-platform-tests.
* `include.ini`: the subset of tests we currently run; * `include.ini`: the subset of web-platform-tests we currently run.
* `run.py` and `run.sh`: glue code to run the tests in Servo; * `config_css.ini`: some configuration for the CSSWG tests.
* `metadata`: expected failures for the tests we run; * `include_css.ini`: the subset of the CSSWG tests we currently run.
* `web-platform-tests`: submodule pointer to the actual tests. * `run_wpt.py` glue code to run the web-platform-tests in Servo.
* `run_css.py` glue code to run the CSSWG tests in Servo.
* `run.py` common code used by `run_wpt.py` and `run_css.py`.
* `web-platform-tests`: copy of the web-platform-tests.
* `metadata`: expected failures for the web-platform-tests we run.
* `css-tests` copy of the built CSSWG tests.
* `metadata-css` expected failures for the CSSWG tests we run.
Running the tests Running the tests
================= =================
The simplest way to run the tests in Servo is `./mach test-wpt` in the root The simplest way to run the web-platform-tests in Servo is `./mach
directory. This will run the subset of JavaScript tests defined in test-wpt` in the root directory. This will run the subset of
`include.ini` and log the output to stdout. JavaScript tests defined in `include.ini` and log the output to
stdout.
Some of the arguments of `./mach test-wpt` include: Similarly the CSSWG tests can be run using `./mach test-css`.
* `--include`: specifies which test(s) to run. A subset of tests may be run by providing positional arguments to the
For example, `--include=/dom` runs all the DOM tests, `--include=/dom/errors` mach command, either as filesystem paths or as test urls e.g.
runs all the DOM error tests and
`--include=/dom/errors/DOMException-constants.html` runs one specific test. Paths should be relative to the `web-platform-tests` submodule. ./mach test-wpt tests/wpt/web-platform-tests/dom/historical.html
(Note that this overrides `include.ini` completely.)
* `--processes`: specifies the number of parallel processes to use (default 1). to run the dom/historical.html test, or
When this argument is passed, the runner will spawn multiple instances of
Servo simultaneously to run multiple tests in parallel for more efficiency ./mach test-wpt dom
(especially on multi-core processors).
to run all the DOM tests.
There are also a large number of command line options accepted by the
test harness; these are documented by running with `--help`.
Running the tests without mach Running the tests without mach
------------------------------ ------------------------------
When avoiding `mach` for some reason, one can run `run.py` directly. However, When avoiding `mach` for some reason, one can run either `run_wpt.py`
this requires in the first place that the virtualenv has been set up. To set up ir `run_css.py` directly. However, this requires that all the
the virtualenv and run the tests run the following from the root directory: dependencies for `wptrunner` are avaliable in the current python
environment.
bash tests/wpt/run.sh
You can substitute `bash` for another shell which supports `source`.
Running the tests manually Running the tests manually
-------------------------- --------------------------
@ -73,28 +81,47 @@ This first requires saving the raw, unformatted log from a test run, for
example by running `./mach test-wpt --log-raw /tmp/servo.log`. Once the example by running `./mach test-wpt --log-raw /tmp/servo.log`. Once the
log is saved, run from the root directory: log is saved, run from the root directory:
source tests/wpt/_virtualenv/bin/activate ./mach update-wpt /tmp/servo.log
tests/wpt/_virtualenv/bin/wptupdate \
--ignore-existing \
--config tests/wpt/config.ini \
/tmp/servo.log
This will update the `.ini` files under the `metadata` folder; commit those For CSSWG tests a similar prcedure works, with `./mach test-css` and
changes along with the code changes that require them. `./mach update-css`.
Editing tests
=============
web-platform-tests may be edited in-place and the changes committed to
the servo tree. These changes will be upstreamed when the tests are
next synced.
For CSS tests this kind of in-place update is not possible because the
tests have a build step before they are pulled into the servo
repository. Therefore corrections must be submitted directly to the
source repository.
Updating the upstream tests Updating the upstream tests
=========================== ===========================
In order to update the upstream tests, fetch the latest commits on the `master` In order to update the tests from upstream use the same mach update
branch of the upstream `git@github.com:w3c/web-platform-tests.git` repository. commands. e.g. to update the web-platform-tests:
and create a new branch `servo_[current date]` with those commits. Then
cherry-pick our changes to the `resources` submodule; those should be the ./mach update-wpt --sync
latest commits on the branch currently used by Servo. If the `resources` ./mach test-wpt --log-raw=update.log
submodule has been updated upstream, this will also require cherry-picking the ./mach update-wpt update.log
changes there. Finally, push the `servo_[current date]` to our fork at
`git@github.com:servo/web-platform-tests.git` and create a pull request to the This should create two commits in your servo repository with the
Servo repository with: updated tests and updated metadata. The same process works for the
* the submodule pointer update; CSSWG tests, using the `update-css` and `test-css` mach commands.
* an update to `MANIFEST.json` generated by running `tools/scripts/manifest.py`
in web-platform-tests; Updating the test harness
* the updated test expectations. =========================
The easiest way to update the test harness is using git:
cd tests/wpt/harness
git init .
git remote add origin https://github.com/w3c/wptrunner
git fetch origin
git checkout master origin/master
cd ../../..
At this point you should commit the updated files in the *servo* git repository.