In #7348 `os.walk` was replaced with `os.listdir`. The latter is not
recursive, which results in only the root directory files getting linted
The changes to `ignored_files` are needed because calling `os.walk(".")`
results in `./` getting prefixed before each path
Add tidy check for unused reftest html files
Refs: https://github.com/servo/servo/issues/7078
Sample output:
```
$ time ./mach test-tidy
tests/ref/background_image_a.html not used or commented out in basic.list
tests/ref/background_image_ref.html not used or commented out in basic.list
tests/ref/canvas_linear_gradient_a.html not used or commented out in basic.list
tests/ref/canvas_linear_gradient_ref.html not used or commented out in basic.list
tests/ref/canvas_radial_gradient_a.html not used or commented out in basic.list
tests/ref/canvas_radial_gradient_ref.html not used or commented out in basic.list
tests/ref/inline_border_a.html not used or commented out in basic.list
tests/ref/inline_border_b.html not used or commented out in basic.list
tests/ref/inline_text_align_a.html not used or commented out in basic.list
tests/ref/inline_text_align_b.html not used or commented out in basic.list
tests/ref/link_style_dynamic_addition.html not used or commented out in basic.list
tests/ref/link_style_dynamic_addition_ref.html not used or commented out in basic.list
tests/ref/overflow_position_abs_inside_normal_a.html not used or commented out in basic.list
tests/ref/overflow_position_abs_inside_normal_b.html not used or commented out in basic.list
tests/ref/overflow_position_abs_simple_a.html not used or commented out in basic.list
tests/ref/overflow_position_abs_simple_b.html not used or commented out in basic.list
tests/ref/position_fixed_a.html not used or commented out in basic.list
tests/ref/position_fixed_b.html not used or commented out in basic.list
tests/ref/position_fixed_simple_a.html not used or commented out in basic.list
tests/ref/position_fixed_simple_b.html not used or commented out in basic.list
tests/ref/position_fixed_static_y_a.html not used or commented out in basic.list
tests/ref/position_fixed_static_y_b.html not used or commented out in basic.list
tests/ref/style_is_in_doc.html not used or commented out in basic.list
tests/ref/style_is_in_doc_ref.html not used or commented out in basic.list
tests/ref/table_specified_width_a.html not used or commented out in basic.list
tests/ref/table_specified_width_ref.html not used or commented out in basic.list
tests/ref/text_decoration_propagation_a.html not used or commented out in basic.list
tests/ref/text_decoration_propagation_b.html not used or commented out in basic.list
tests/ref/text_shadow_multiple_shadows_a.html not used or commented out in basic.list
tests/ref/text_shadow_multiple_shadows_ref.html not used or commented out in basic.list
tests/ref/viewport_percentage_vmin_vmax_b.html not used or commented out in basic.list
tests/ref/viewport_percentage_vw_vh_b.html not used or commented out in basic.list
tests/ref/white_space_intrinsic_sizes_a.html not used or commented out in basic.list
tests/ref/white_space_intrinsic_sizes_ref.html not used or commented out in basic.list
tests/ref/fonts/takao-p-gothic/COPYING.html not used or commented out in basic.list
tests/ref/iframe/multiple_external_child.html not used or commented out in basic.list
...
```
I thought it might be helpful to say which files aren't used in `basic.list`.
`./mach test-tidy` is a second or two slower on my laptop.
Not sure if this counts as a warning or an error that should return 1.
Not sure whether unused file output should go before or after the line specific errors.
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/7118)
<!-- Reviewable:end -->
Prior to this commit:
* Our Python dependency story was a bit of a mess. We had complete
Python packages (wheels and directories) living in-tree, despite
not having any changes from upstream. This is particularly bad because
`setup.py` never gets run on these packages which could (sometimes
silently) unintended breakage.
* Python virtual environments (virtualenv) were only utilized for
testing web-platform tests
After this commit:
* A single virtualenv (`python/_virtualenv`) is activated upon *every*
call to mach
* A requirements file (`python/requirements.txt`) is added to describe
the dependencies needed by Python modules in `python/`. The child
commit immediately following this will remove all the dependencies
no longer needed in-tree (for the sake of keeping this commit
readable).
Relevant to https://github.com/servo/servo/issues/861
Fixes https://github.com/servo/servo/issues/6999
If someone were to write:
def hello() :
print "hello world"
flake8 would warn:
stdin:54:45: E203 whitespace before ':'
Normally there are only three colons in a flake8 error message, but this
one has four, which causes issue with this line:
_, line_num, _, message = error.split(":")
...causing this error:
ValueError: too many values to unpack
This commit updates the `str.split` call to utilize the `maxsplit`
parameter to prevent this error from occurring.
Currently, there are a few linting functions that only run on certain
filetypes (determined by the file extension). Prior to this commit, the
special cases were handled in a parent function with a conditional. This
commit changes the system so each linting function gets passed a
filename so the function can determine whether it should run or not
based on the file extension.
I also refactored flake8 linting slightly. From what I've read so far of
the code, flake8 itself will only print the results directly to stdout
(though the linter would report the quantity of errors detected).
Prior to this commit, we would let flake8 print directly to stdout and
just determine if there were >0 errors reported. This commit (sort of
hackily) temporarily captures stdout when we call flake8 so we can do
what we want with the output, allowing us to `yield` the line number
and message like we do with the other linting functions.
In my opinion, both of these changes isolate specific behaviors/checks
into their respective linting functions instead of having them handled
at a more global level.
In addition to the changes above:
* The whitespace linter now runs on WebIDL and TOML files
* The license header linter now runs on WebIDL files
If there's an error in the command-line arguments for `mach build`, we should
print it before starting a (potentially) long bootstrap process, not after.
