Commit graph

50 commits

Author SHA1 Message Date
Alan Jeffrey
be5eaa4447 Submit test-perf CSV files to S3. 2017-11-21 09:27:30 -06:00
Alan Jeffrey
a57ba2a317 Include OS and architecture in test-perf data. 2017-11-20 14:12:21 -06:00
bors-servo
98c9e5f1ac Auto merge of #19245 - asajeffrey:test-perf-update-manifest, r=jdm
Update tp5n manifest for test-perf

<!-- Please describe your changes on the following line: -->

Updated test-perf tp5n manifest.

This PR provides a `test.manifest` file, that contains a subset of the tp5n tests that don't timeout (at least on my machine).

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes do not require tests because this is test infrastructure

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/19245)
<!-- Reviewable:end -->
2017-11-20 11:09:38 -06:00
Alan Jeffrey
ed36323e0b Include a YYYYMMDD date field in the raw test-perf CSV to make Google Data Studio happy. 2017-11-16 16:36:10 -06:00
Alan Jeffrey
b923559d55 Update tp5n manifest for test-perf. 2017-11-16 09:33:43 -06:00
bors-servo
ee0294380b Auto merge of #19233 - asajeffrey:test-perf-csv, r=jdm
Save test-perf data in csv

<!-- Please describe your changes on the following line: -->

Most data analytics and visualization tools (e.g. Google Data Studio and Amazon Quick Sight) seem to want .csv input, not .json. This PR makes .csv the default data format.

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes do not require tests because it's test infrastructure

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/19233)
<!-- Reviewable:end -->
2017-11-16 00:55:30 -06:00
bors-servo
e6b05fa204 Auto merge of #19206 - asajeffrey:test-perf-add-differ-summary, r=aneeshusa
Add total time to test_differ in etc/ci/performance

<!-- Please describe your changes on the following line: -->

Add the total time to test_differ in etc/ci/performance.

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes do not require tests because this is test infrastructure

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/19206)
<!-- Reviewable:end -->
2017-11-15 18:34:12 -06:00
Alan Jeffrey
45d5d53e55 Added title to example test perf document. 2017-11-15 12:24:11 -06:00
Alan Jeffrey
a1dfce4d3c Made .csv the default file format for perf tests. 2017-11-15 12:24:10 -06:00
Alan Jeffrey
d817b224f5 Support saving test data to .csv as well as .json. 2017-11-15 12:24:09 -06:00
Alan Jeffrey
201f931fa1 Add total time to test_differ in etc/ci/performance. 2017-11-15 11:54:45 -06:00
Alan Jeffrey
b845d44aa2 Add 'Error loading page' case for validating correctness of test-perf data. 2017-11-15 11:42:21 -06:00
bors-servo
5f33d35cde Auto merge of #19207 - asajeffrey:test-perf-add-base-url-option, r=edunham
Add --base option to test-perf.

<!-- Please describe your changes on the following line: -->

Add a `--base <URL>` option to test-perf, which is handy for two reasons: a) it reduces randomness in tests by allowing tests to be file URLs, and b) it doesn't require running the test suite as root (the tp5n manifest hardwires tests served from port 80 on localhost).

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes do not require tests because this is test infrastructure

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/19207)
<!-- Reviewable:end -->
2017-11-13 19:51:32 -06:00
Alan Jeffrey
109df3328a Add --base option to test-perf. 2017-11-13 18:58:06 -06:00
Alan Jeffrey
45a63a8960 Update URL for tp5n archive. 2017-11-13 18:04:51 -06:00
Shing Lyu
aaa0d46135 Remove link to servo from treeherder job detail to reduce noise 2017-03-07 10:20:47 +08:00
Shing Lyu
7aa3350d45 Added async performance test 2017-02-08 10:46:00 +08:00
Shing Lyu
d8501badc0 Read firefox path from environment variable for performance test 2017-01-12 11:39:24 +08:00
Shing Lyu
ddd03229e0 Add an option to submit test-perf result to perfherder 2016-12-01 10:23:27 +08:00
Shing Lyu
ecfabb6e9f Use resource manager in gecko_driver and nits 2016-11-16 17:53:55 +08:00
Shing Lyu
d9a7221b0c Remove a level of unnessary function call 2016-11-16 17:16:03 +08:00
Shing Lyu
9f0b685268 Space nit 2016-11-16 17:09:08 +08:00
Shing Lyu
d8b862e4d7 Various easy nits 2016-11-16 17:06:14 +08:00
Shing Lyu
6110017569 Use selenium to drive gecko test 2016-11-14 11:43:23 +08:00
Shing Lyu
2407fe614b Log and readme improvements 2016-11-07 15:20:37 +08:00
Shing Lyu
9aaa088624 Use headless rendering for performance test 2016-11-07 15:20:36 +08:00
Shing Lyu
d941c5b916 Cached tp5 zip downloading and unzipping 2016-11-07 15:20:34 +08:00
bors-servo
58e6d7bd58 Auto merge of #13248 - asajeffrey:etc-ci-performance-gitignore, r=shinglyu
Added etc/ci/performance downloads to .gitignore.

<!-- Please describe your changes on the following line: -->

Added the files downloaded for `etc/ci/performance` to `.gitignore`, which gets rid of lot of spurious reporting from `git status`.

r? @shinglyu

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes do not require tests because it's about git

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/13248)
<!-- Reviewable:end -->
2016-09-26 06:03:55 -05:00
CHANG FU CHIAO
fd204e0365 Create ./mach test-perf command
in testing_commands.py, I linked the new command to test_perf.sh
in test_perf.sh, it would prepare tp5n.zip and virtual environment
then run test_all.sh
2016-09-25 14:49:51 +08:00
Alan Jeffrey
ac59f56f98 Added etc/ci/performance downloads to .gitignore. 2016-09-22 09:59:44 -05:00
bors-servo
9e349544f0 Auto merge of #12946 - asajeffrey:etc-ci-performance-run-locally, r=shinglyu
Etc ci performance run locally

<!-- Please describe your changes on the following line: -->

Changes to `etc/ci/performance` to get it to run locally on my machine at least!

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes do not require tests because this is performance-testing code

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/12946)
<!-- Reviewable:end -->
2016-09-06 04:07:05 -05:00
Alan Jeffrey
105b11ffcd Made local perf testing the default. 2016-09-05 08:43:03 -05:00
CHANG FU CHIAO
1dcc8e69fe FIX TreeherderClient deprecated param
According to warning from TreeherderClient(),
the parameters `host` and `protocol` are deprecated.
Then use `server_url` instead.
2016-08-27 13:43:43 +08:00
Alan Jeffrey
bc07d99391 Making test-tidy happy. 2016-08-20 12:06:51 -05:00
Alan Jeffrey
646d9c8988 Disabled xunlei.com perf testcase. 2016-08-20 12:01:00 -05:00
Alan Jeffrey
9ebd156889 Updated format string for test_differ.py. 2016-08-20 12:00:20 -05:00
Alan Jeffrey
0bd05e50c8 Ooops, checked in a test version. 2016-08-19 16:29:34 -05:00
Alan Jeffrey
fb289812cf Added test differ. 2016-08-19 16:17:13 -05:00
Alan Jeffrey
425b44f78e Made test-tidy happy. 2016-08-19 14:08:50 -05:00
Alan Jeffrey
f81e32e334 Added --local option to test_all.sh. 2016-08-19 14:00:58 -05:00
Alan Jeffrey
2857dbabbe Include document title in report. 2016-08-19 13:51:12 -05:00
Alan Jeffrey
08d15e9466 Fixed formatting of timeout argument. 2016-08-19 13:50:21 -05:00
Alan Jeffrey
900518ae55 Filter reports for ones with title Error report. 2016-08-19 13:49:05 -05:00
Alan Jeffrey
8e7837159e Only include [PERF] lines in report. 2016-08-19 13:47:00 -05:00
Alan Jeffrey
a2a45a77fd Removed extra space from --userscripts option. 2016-08-19 13:46:28 -05:00
Alan Jeffrey
eb18665f5b Removed shell option from subprocess. 2016-08-19 13:45:55 -05:00
Alan Jeffrey
1096351543 Diabled never-terminating photobucket test from manifest. 2016-08-19 13:43:49 -05:00
Alan Jeffrey
53c0765bfb Diabled missing tp6 test from manifest. 2016-08-19 13:43:16 -05:00
Alan Jeffrey
4a77ef03c8 Added tp5n prefix to URLs in manifest. 2016-08-19 13:42:37 -05:00
Shing Lyu
13b8f2531c Bug 10452 - Page Load Time Test Runner 2016-08-08 16:08:13 +08:00