Commit graph

302 commits

Author SHA1 Message Date
bors-servo
8b6b2e97c8 Auto merge of #19323 - asajeffrey:test-perf-run-nightly, r=edunham
Run test-perf on linux-nightly.

<!-- Please describe your changes on the following line: -->

Run test-perf on linux-nightly.

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix #12988
- [X] These changes do not require tests because this is test infrastructure

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/19323)
<!-- Reviewable:end -->
2017-11-27 16:12:14 -06:00
bors-servo
218edeb883 Auto merge of #19336 - servo:jdm-patch-6, r=emilio
Instruct how to deal with outdated manifests.

<!-- Reviewable:start -->
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/19336)
<!-- Reviewable:end -->
2017-11-22 10:36:04 -06:00
Josh Matthews
b45e4fecff
Address tidy checks. 2017-11-22 10:48:51 -05:00
Simon Sapin
b1ce298d4b Replace compiletest suite by doc-tests with compile_fail
compiletest-rs use internal rustc APIs and is broken in today’s Nightly.
rustdoc however is maintained with rustc and so much less fragile.
2017-11-22 16:25:17 +01:00
Simon Sapin
d96f0ff6a7 Use rustdoc --document-private-item instead of deprecated flags
… and use $RUSTDOCFLAGS instead of $RUSTDOC with a wrapper script
2017-11-22 15:32:40 +01:00
Josh Matthews
b501c8fa03
Instruct how to deal with outdated manifests. 2017-11-22 08:16:54 -05:00
Josh Matthews
da46fe4812
Unbreak docs build. 2017-11-22 07:47:04 -05:00
Josh Matthews
5a36c481f0
Use same build environment during mutation testing. 2017-11-21 13:47:54 -05:00
Alan Jeffrey
1a59ea9733 Run test-perf on linux-nightly. 2017-11-21 09:45:34 -06:00
Alan Jeffrey
be5eaa4447 Submit test-perf CSV files to S3. 2017-11-21 09:27:30 -06:00
Alan Jeffrey
a57ba2a317 Include OS and architecture in test-perf data. 2017-11-20 14:12:21 -06:00
bors-servo
98c9e5f1ac Auto merge of #19245 - asajeffrey:test-perf-update-manifest, r=jdm
Update tp5n manifest for test-perf

<!-- Please describe your changes on the following line: -->

Updated test-perf tp5n manifest.

This PR provides a `test.manifest` file, that contains a subset of the tp5n tests that don't timeout (at least on my machine).

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes do not require tests because this is test infrastructure

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/19245)
<!-- Reviewable:end -->
2017-11-20 11:09:38 -06:00
bors-servo
b5a205d92e Auto merge of #19257 - asajeffrey:test-perf-include-date-in-csv, r=jdm
Include a YYYYMMDD date field in the raw test-perf CSV to make Google Data Studio happy

<!-- Please describe your changes on the following line: -->

AFAICT Google Data Studio needs a YYYMMDD date string in the raw data to be able to provide time-indexed graphs. `TODATE` is not converting unix timestamps to YYYYMMDD.

https://www.en.advertisercommunity.com/t5/Data-Studio/Can-Data-Studio-Date-Dimension-use-epoch-in-seconds-from/td-p/839649

This PR is a workaround, and includes a date field in the raw CSV.

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes do not require tests because this is test infrastructure

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/19257)
<!-- Reviewable:end -->
2017-11-16 16:46:26 -06:00
Alan Jeffrey
ed36323e0b Include a YYYYMMDD date field in the raw test-perf CSV to make Google Data Studio happy. 2017-11-16 16:36:10 -06:00
bors-servo
50f11e3584 Auto merge of #18984 - dsandeephegde:master, r=jdm
Initial steps of Mutation testing

<!-- Please describe your changes on the following line: -->
- Added one strategy of mutation which is replacing occurrences && to ||.
- Added test mapping framework for running mutation tests corresponding to a mutant.
- Added one test_mapping.json to map source file in a folder to WPT test.
- Added README mentioning about Mutation testing.
- Added CI script to invoke mutation test.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes #18529 (github issue number if applicable).
- [x] These changes do not require tests because it is a python script to run mutation test and does not change any behavior.

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/18984)
<!-- Reviewable:end -->
2017-11-16 11:51:01 -06:00
Alan Jeffrey
b923559d55 Update tp5n manifest for test-perf. 2017-11-16 09:33:43 -06:00
bors-servo
ee0294380b Auto merge of #19233 - asajeffrey:test-perf-csv, r=jdm
Save test-perf data in csv

<!-- Please describe your changes on the following line: -->

Most data analytics and visualization tools (e.g. Google Data Studio and Amazon Quick Sight) seem to want .csv input, not .json. This PR makes .csv the default data format.

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes do not require tests because it's test infrastructure

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/19233)
<!-- Reviewable:end -->
2017-11-16 00:55:30 -06:00
bors-servo
e6b05fa204 Auto merge of #19206 - asajeffrey:test-perf-add-differ-summary, r=aneeshusa
Add total time to test_differ in etc/ci/performance

<!-- Please describe your changes on the following line: -->

Add the total time to test_differ in etc/ci/performance.

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes do not require tests because this is test infrastructure

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/19206)
<!-- Reviewable:end -->
2017-11-15 18:34:12 -06:00
Alan Jeffrey
45d5d53e55 Added title to example test perf document. 2017-11-15 12:24:11 -06:00
Alan Jeffrey
a1dfce4d3c Made .csv the default file format for perf tests. 2017-11-15 12:24:10 -06:00
Alan Jeffrey
d817b224f5 Support saving test data to .csv as well as .json. 2017-11-15 12:24:09 -06:00
Alan Jeffrey
201f931fa1 Add total time to test_differ in etc/ci/performance. 2017-11-15 11:54:45 -06:00
Alan Jeffrey
b845d44aa2 Add 'Error loading page' case for validating correctness of test-perf data. 2017-11-15 11:42:21 -06:00
Anthony Ramine
23d7783e02 Make script depend on gecko-media
It's not used yet though.
2017-11-14 17:33:25 +01:00
Simon Sapin
3264453284 Stop using "alternate" rustc builds.
With https://github.com/rust-lang/rust/pull/45810,
normal Nightly now has LLVM assertions disabled.
2017-11-14 09:08:11 +01:00
bors-servo
5f33d35cde Auto merge of #19207 - asajeffrey:test-perf-add-base-url-option, r=edunham
Add --base option to test-perf.

<!-- Please describe your changes on the following line: -->

Add a `--base <URL>` option to test-perf, which is handy for two reasons: a) it reduces randomness in tests by allowing tests to be file URLs, and b) it doesn't require running the test suite as root (the tp5n manifest hardwires tests served from port 80 on localhost).

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes do not require tests because this is test infrastructure

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/19207)
<!-- Reviewable:end -->
2017-11-13 19:51:32 -06:00
Alan Jeffrey
109df3328a Add --base option to test-perf. 2017-11-13 18:58:06 -06:00
Alan Jeffrey
45a63a8960 Update URL for tp5n archive. 2017-11-13 18:04:51 -06:00
Sandeep Hegde
2f900a0427 Added PS1 variable before activating virtualenv 2017-11-13 08:05:34 -05:00
Sandeep Hegde
ed47f89e79 Changed invocation of muatation test in CI to bash script to use virtualenv 2017-11-13 01:01:09 -05:00
Sandeep Hegde
cc6c2eea6e Added mutation test summary and made it exit with relevant exit code 2017-11-12 23:43:04 -05:00
Josh Matthews
86e3f0deef Use failure reporting API on CI. 2017-11-07 14:01:24 -05:00
Josh Matthews
c0f85579c5 Allow specifying alternative tracker endpoints. 2017-11-07 13:40:07 -05:00
Sandeep Hegde
58ab11cf9b Fixed few tidy errors 2017-11-06 13:00:13 -05:00
Sandeep Hegde
af605a6e09 Added Mutation Test to CI 2017-11-06 13:00:12 -05:00
Josh Matthews
00940911c5 Divide up CI jobs more evenly. 2017-11-02 09:22:01 -04:00
Simon Sapin
4c36ba1f78 Run 'test-stylo' on Windows CI
Exercise the previous commit’s fix.

This used to fail to link:
https://github.com/rust-lang/rust/pull/44603#issuecomment-338807312
2017-10-24 16:50:14 +02:00
Josh Matthews
d923fb9d30 Remove duplicate CEF builds 2017-10-16 19:58:49 -04:00
Simon Sapin
115d859551 Build with --features default-except-unstable on CI. 2017-10-16 20:19:20 +02:00
Josh Matthews
398a15e6e0 Make update_manifest.sh more verbose. 2017-10-09 06:21:48 -04:00
Josh Matthews
a7815aa9bd
Use homebrew zlib on CI builders. 2017-08-19 09:35:42 +02:00
Nikhil Shagrithaya
04c15888ce Run a subset of tests using async html tokenizer 2017-07-29 19:32:36 +05:30
Gecko Backout
f022936ebb Backed out changeset a417b9d7712d for vendoring bustage. r=backout on a CLOSED TREE
Backs out https://github.com/servo/servo/pull/17565
2017-07-28 22:06:00 +00:00
Nikhil Shagrithaya
0750fdc370 Run a subset of tests using async html tokenizer 2017-07-28 21:16:38 +05:30
E. Dunham
e868c4c9b7 Run chaos_monkey_test on the non-gating builder 2017-07-14 10:02:09 -07:00
Nazım Can Altınova
6da69f687b
Add an index.html page for geckolib docs 2017-06-12 03:27:50 +03:00
Josh Matthews
73496ffea8 Copy all geckolib dependency docs for upload. 2017-06-08 15:03:47 -04:00
bors-servo
1aa3eec66b Auto merge of #17115 - aneeshusa:temporarily-disable-macbrew-updater, r=emilio
Disable macbrew updater

<!-- Please describe your changes on the following line: -->

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [ ] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #__ (github issue number if applicable).

<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because _____

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/17115)
<!-- Reviewable:end -->
2017-05-31 17:00:15 -07:00
Aneesh Agrawal
32c973d118 Temporarily disable macbrew updater 2017-05-31 17:54:35 -04:00
Imanol Fernandez
04fb628d20 Make armv7-linux-androideabi default target on Android 2017-05-31 22:31:26 +02:00