In #7348 `os.walk` was replaced with `os.listdir`. The latter is not
recursive, which results in only the root directory files getting linted
The changes to `ignored_files` are needed because calling `os.walk(".")`
results in `./` getting prefixed before each path
Add tidy check for unused reftest html files
Refs: https://github.com/servo/servo/issues/7078
Sample output:
```
$ time ./mach test-tidy
tests/ref/background_image_a.html not used or commented out in basic.list
tests/ref/background_image_ref.html not used or commented out in basic.list
tests/ref/canvas_linear_gradient_a.html not used or commented out in basic.list
tests/ref/canvas_linear_gradient_ref.html not used or commented out in basic.list
tests/ref/canvas_radial_gradient_a.html not used or commented out in basic.list
tests/ref/canvas_radial_gradient_ref.html not used or commented out in basic.list
tests/ref/inline_border_a.html not used or commented out in basic.list
tests/ref/inline_border_b.html not used or commented out in basic.list
tests/ref/inline_text_align_a.html not used or commented out in basic.list
tests/ref/inline_text_align_b.html not used or commented out in basic.list
tests/ref/link_style_dynamic_addition.html not used or commented out in basic.list
tests/ref/link_style_dynamic_addition_ref.html not used or commented out in basic.list
tests/ref/overflow_position_abs_inside_normal_a.html not used or commented out in basic.list
tests/ref/overflow_position_abs_inside_normal_b.html not used or commented out in basic.list
tests/ref/overflow_position_abs_simple_a.html not used or commented out in basic.list
tests/ref/overflow_position_abs_simple_b.html not used or commented out in basic.list
tests/ref/position_fixed_a.html not used or commented out in basic.list
tests/ref/position_fixed_b.html not used or commented out in basic.list
tests/ref/position_fixed_simple_a.html not used or commented out in basic.list
tests/ref/position_fixed_simple_b.html not used or commented out in basic.list
tests/ref/position_fixed_static_y_a.html not used or commented out in basic.list
tests/ref/position_fixed_static_y_b.html not used or commented out in basic.list
tests/ref/style_is_in_doc.html not used or commented out in basic.list
tests/ref/style_is_in_doc_ref.html not used or commented out in basic.list
tests/ref/table_specified_width_a.html not used or commented out in basic.list
tests/ref/table_specified_width_ref.html not used or commented out in basic.list
tests/ref/text_decoration_propagation_a.html not used or commented out in basic.list
tests/ref/text_decoration_propagation_b.html not used or commented out in basic.list
tests/ref/text_shadow_multiple_shadows_a.html not used or commented out in basic.list
tests/ref/text_shadow_multiple_shadows_ref.html not used or commented out in basic.list
tests/ref/viewport_percentage_vmin_vmax_b.html not used or commented out in basic.list
tests/ref/viewport_percentage_vw_vh_b.html not used or commented out in basic.list
tests/ref/white_space_intrinsic_sizes_a.html not used or commented out in basic.list
tests/ref/white_space_intrinsic_sizes_ref.html not used or commented out in basic.list
tests/ref/fonts/takao-p-gothic/COPYING.html not used or commented out in basic.list
tests/ref/iframe/multiple_external_child.html not used or commented out in basic.list
...
```
I thought it might be helpful to say which files aren't used in `basic.list`.
`./mach test-tidy` is a second or two slower on my laptop.
Not sure if this counts as a warning or an error that should return 1.
Not sure whether unused file output should go before or after the line specific errors.
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/7118)
<!-- Reviewable:end -->
Prior to this commit:
* Our Python dependency story was a bit of a mess. We had complete
Python packages (wheels and directories) living in-tree, despite
not having any changes from upstream. This is particularly bad because
`setup.py` never gets run on these packages which could (sometimes
silently) unintended breakage.
* Python virtual environments (virtualenv) were only utilized for
testing web-platform tests
After this commit:
* A single virtualenv (`python/_virtualenv`) is activated upon *every*
call to mach
* A requirements file (`python/requirements.txt`) is added to describe
the dependencies needed by Python modules in `python/`. The child
commit immediately following this will remove all the dependencies
no longer needed in-tree (for the sake of keeping this commit
readable).
Relevant to https://github.com/servo/servo/issues/861
Fixes https://github.com/servo/servo/issues/6999
If someone were to write:
def hello() :
print "hello world"
flake8 would warn:
stdin:54:45: E203 whitespace before ':'
Normally there are only three colons in a flake8 error message, but this
one has four, which causes issue with this line:
_, line_num, _, message = error.split(":")
...causing this error:
ValueError: too many values to unpack
This commit updates the `str.split` call to utilize the `maxsplit`
parameter to prevent this error from occurring.
Currently, there are a few linting functions that only run on certain
filetypes (determined by the file extension). Prior to this commit, the
special cases were handled in a parent function with a conditional. This
commit changes the system so each linting function gets passed a
filename so the function can determine whether it should run or not
based on the file extension.
I also refactored flake8 linting slightly. From what I've read so far of
the code, flake8 itself will only print the results directly to stdout
(though the linter would report the quantity of errors detected).
Prior to this commit, we would let flake8 print directly to stdout and
just determine if there were >0 errors reported. This commit (sort of
hackily) temporarily captures stdout when we call flake8 so we can do
what we want with the output, allowing us to `yield` the line number
and message like we do with the other linting functions.
In my opinion, both of these changes isolate specific behaviors/checks
into their respective linting functions instead of having them handled
at a more global level.
In addition to the changes above:
* The whitespace linter now runs on WebIDL and TOML files
* The license header linter now runs on WebIDL files
Recently, I found myself reading through the Python codegen scripts that
live in 'components/script/dom/bindings/*' and noticed that there were
many tidy violations: unnecessary semicolons, weird spacing, unused
variables, lack of license headers, etc. Considering these files are now
living in our tree and mostly maintained directly by contributors of
Servo (as opposed to being from upstream), I feel these files should not
be excluded from our normal tidy process. This commit removes the
blacklist on these files and fixes all tidy violations.
I added these subdirectories to the blacklist because they appear to be
maintained upstream somewhere else:
* "components/script/dom/bindings/codegen/parser/*",
* "components/script/dom/bindings/codegen/ply/*",
Also, I added a '# noqa' comment which tells us to ignore the
flake8 errors for that line. I chose to ignore this (instead of fixing
it) to make the work for this commit simpler for me.
