mirror of
https://github.com/servo/servo.git
synced 2025-08-23 14:25:33 +01:00
Update web-platform-tests and CSS tests.
- Update CSS tests to revision e05bfd5e30ed662c2f8a353577003f8eed230180. - Update web-platform-tests to revision a052787dd5c069a340031011196b73affbd68cd9.
This commit is contained in:
parent
fb4f421c8b
commit
296fa2512b
21852 changed files with 2080936 additions and 892894 deletions
219
tests/wpt/css-tests/tools/pytest/doc/en/parametrize.rst
Normal file
219
tests/wpt/css-tests/tools/pytest/doc/en/parametrize.rst
Normal file
|
@ -0,0 +1,219 @@
|
|||
|
||||
.. _`test generators`:
|
||||
.. _`parametrizing-tests`:
|
||||
.. _`parametrized test functions`:
|
||||
.. _`parametrize`:
|
||||
|
||||
.. _`parametrize-basics`:
|
||||
|
||||
Parametrizing fixtures and test functions
|
||||
==========================================================================
|
||||
|
||||
pytest supports test parametrization in several well-integrated ways:
|
||||
|
||||
- :py:func:`pytest.fixture` allows to define :ref:`parametrization
|
||||
at the level of fixture functions <fixture-parametrize>`.
|
||||
|
||||
* `@pytest.mark.parametrize`_ allows to define parametrization at the
|
||||
function or class level, provides multiple argument/fixture sets
|
||||
for a particular test function or class.
|
||||
|
||||
* `pytest_generate_tests`_ enables implementing your own custom
|
||||
dynamic parametrization scheme or extensions.
|
||||
|
||||
.. _parametrizemark:
|
||||
.. _`@pytest.mark.parametrize`:
|
||||
|
||||
|
||||
``@pytest.mark.parametrize``: parametrizing test functions
|
||||
---------------------------------------------------------------------
|
||||
|
||||
.. regendoc: wipe
|
||||
|
||||
.. versionadded:: 2.2
|
||||
.. versionchanged:: 2.4
|
||||
Several improvements.
|
||||
|
||||
The builtin ``pytest.mark.parametrize`` decorator enables
|
||||
parametrization of arguments for a test function. Here is a typical example
|
||||
of a test function that implements checking that a certain input leads
|
||||
to an expected output::
|
||||
|
||||
# content of test_expectation.py
|
||||
import pytest
|
||||
@pytest.mark.parametrize("test_input,expected", [
|
||||
("3+5", 8),
|
||||
("2+4", 6),
|
||||
("6*9", 42),
|
||||
])
|
||||
def test_eval(test_input, expected):
|
||||
assert eval(test_input) == expected
|
||||
|
||||
Here, the ``@parametrize`` decorator defines three different ``(test_input,expected)``
|
||||
tuples so that the ``test_eval`` function will run three times using
|
||||
them in turn::
|
||||
|
||||
$ py.test
|
||||
======= test session starts ========
|
||||
platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
|
||||
rootdir: $REGENDOC_TMPDIR, inifile:
|
||||
collected 3 items
|
||||
|
||||
test_expectation.py ..F
|
||||
|
||||
======= FAILURES ========
|
||||
_______ test_eval[6*9-42] ________
|
||||
|
||||
test_input = '6*9', expected = 42
|
||||
|
||||
@pytest.mark.parametrize("test_input,expected", [
|
||||
("3+5", 8),
|
||||
("2+4", 6),
|
||||
("6*9", 42),
|
||||
])
|
||||
def test_eval(test_input, expected):
|
||||
> assert eval(test_input) == expected
|
||||
E assert 54 == 42
|
||||
E + where 54 = eval('6*9')
|
||||
|
||||
test_expectation.py:8: AssertionError
|
||||
======= 1 failed, 2 passed in 0.12 seconds ========
|
||||
|
||||
As designed in this example, only one pair of input/output values fails
|
||||
the simple test function. And as usual with test function arguments,
|
||||
you can see the ``input`` and ``output`` values in the traceback.
|
||||
|
||||
Note that you could also use the parametrize marker on a class or a module
|
||||
(see :ref:`mark`) which would invoke several functions with the argument sets.
|
||||
|
||||
It is also possible to mark individual test instances within parametrize,
|
||||
for example with the builtin ``mark.xfail``::
|
||||
|
||||
# content of test_expectation.py
|
||||
import pytest
|
||||
@pytest.mark.parametrize("test_input,expected", [
|
||||
("3+5", 8),
|
||||
("2+4", 6),
|
||||
pytest.mark.xfail(("6*9", 42)),
|
||||
])
|
||||
def test_eval(test_input, expected):
|
||||
assert eval(test_input) == expected
|
||||
|
||||
Let's run this::
|
||||
|
||||
$ py.test
|
||||
======= test session starts ========
|
||||
platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
|
||||
rootdir: $REGENDOC_TMPDIR, inifile:
|
||||
collected 3 items
|
||||
|
||||
test_expectation.py ..x
|
||||
|
||||
======= 2 passed, 1 xfailed in 0.12 seconds ========
|
||||
|
||||
The one parameter set which caused a failure previously now
|
||||
shows up as an "xfailed (expected to fail)" test.
|
||||
|
||||
To get all combinations of multiple parametrized arguments you can stack
|
||||
``parametrize`` decorators::
|
||||
|
||||
import pytest
|
||||
@pytest.mark.parametrize("x", [0, 1])
|
||||
@pytest.mark.parametrize("y", [2, 3])
|
||||
def test_foo(x, y):
|
||||
pass
|
||||
|
||||
This will run the test with the arguments set to x=0/y=2, x=0/y=3, x=1/y=2 and
|
||||
x=1/y=3.
|
||||
|
||||
.. note::
|
||||
|
||||
In versions prior to 2.4 one needed to specify the argument
|
||||
names as a tuple. This remains valid but the simpler ``"name1,name2,..."``
|
||||
comma-separated-string syntax is now advertised first because
|
||||
it's easier to write and produces less line noise.
|
||||
|
||||
.. _`pytest_generate_tests`:
|
||||
|
||||
Basic ``pytest_generate_tests`` example
|
||||
---------------------------------------------
|
||||
|
||||
Sometimes you may want to implement your own parametrization scheme
|
||||
or implement some dynamism for determining the parameters or scope
|
||||
of a fixture. For this, you can use the ``pytest_generate_tests`` hook
|
||||
which is called when collecting a test function. Through the passed in
|
||||
``metafunc`` object you can inspect the requesting test context and, most
|
||||
importantly, you can call ``metafunc.parametrize()`` to cause
|
||||
parametrization.
|
||||
|
||||
For example, let's say we want to run a test taking string inputs which
|
||||
we want to set via a new ``pytest`` command line option. Let's first write
|
||||
a simple test accepting a ``stringinput`` fixture function argument::
|
||||
|
||||
# content of test_strings.py
|
||||
|
||||
def test_valid_string(stringinput):
|
||||
assert stringinput.isalpha()
|
||||
|
||||
Now we add a ``conftest.py`` file containing the addition of a
|
||||
command line option and the parametrization of our test function::
|
||||
|
||||
# content of conftest.py
|
||||
|
||||
def pytest_addoption(parser):
|
||||
parser.addoption("--stringinput", action="append", default=[],
|
||||
help="list of stringinputs to pass to test functions")
|
||||
|
||||
def pytest_generate_tests(metafunc):
|
||||
if 'stringinput' in metafunc.fixturenames:
|
||||
metafunc.parametrize("stringinput",
|
||||
metafunc.config.option.stringinput)
|
||||
|
||||
If we now pass two stringinput values, our test will run twice::
|
||||
|
||||
$ py.test -q --stringinput="hello" --stringinput="world" test_strings.py
|
||||
..
|
||||
2 passed in 0.12 seconds
|
||||
|
||||
Let's also run with a stringinput that will lead to a failing test::
|
||||
|
||||
$ py.test -q --stringinput="!" test_strings.py
|
||||
F
|
||||
======= FAILURES ========
|
||||
_______ test_valid_string[!] ________
|
||||
|
||||
stringinput = '!'
|
||||
|
||||
def test_valid_string(stringinput):
|
||||
> assert stringinput.isalpha()
|
||||
E assert <built-in method isalpha of str object at 0xdeadbeef>()
|
||||
E + where <built-in method isalpha of str object at 0xdeadbeef> = '!'.isalpha
|
||||
|
||||
test_strings.py:3: AssertionError
|
||||
1 failed in 0.12 seconds
|
||||
|
||||
As expected our test function fails.
|
||||
|
||||
If you don't specify a stringinput it will be skipped because
|
||||
``metafunc.parametrize()`` will be called with an empty parameter
|
||||
list::
|
||||
|
||||
$ py.test -q -rs test_strings.py
|
||||
s
|
||||
======= short test summary info ========
|
||||
SKIP [1] $PYTHON_PREFIX/lib/python3.4/site-packages/_pytest/python.py:1419: got empty parameter set, function test_valid_string at $REGENDOC_TMPDIR/test_strings.py:1
|
||||
1 skipped in 0.12 seconds
|
||||
|
||||
For further examples, you might want to look at :ref:`more
|
||||
parametrization examples <paramexamples>`.
|
||||
|
||||
.. _`metafunc object`:
|
||||
|
||||
The **metafunc** object
|
||||
-------------------------------------------
|
||||
|
||||
.. currentmodule:: _pytest.python
|
||||
.. autoclass:: Metafunc
|
||||
|
||||
.. automethod:: Metafunc.parametrize
|
||||
.. automethod:: Metafunc.addcall(funcargs=None,id=_notexists,param=_notexists)
|
Loading…
Add table
Add a link
Reference in a new issue