This makes it easier to run `update-wpt` based on results from the bots.
A future version of this could aggregate all unexpected results that
were not filtered as intermittents.
Bump euclid to 0.22
Major changes:
- All matrices are now stored in row major order. This means that parameters to rotation functions should no longer be negated.
- `post_...()` functions are now named `then()`. `pre_transform()` is removed, so `then()` is used and the order of operations changed.
In addition, this PR updates lyon_geom to the latest version.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#27424
- [x] There are tests for these changes
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Output test results as GitHub comments
After filtering intermittents, output the results as JSON. Update the
GitHub workflow to aggregate this JSON data into an artifact and use the
aggregated data to generate a GitHub comment with details about the try
run. The idea here is that this comment will make it easier to track
intermittent tests and notice when a change affects a test marked as
intermittent -- either causing it to permanently fail or fixing it.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they modify the CI infrastructure.
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
After filtering intermittents, output the results as JSON. Update the
GitHub workflow to aggregate this JSON data into an artifact and use the
aggregated data to generate a GitHub comment with details about the try
run. The idea here is that this comment will make it easier to track
intermittent tests and notice when a change affects a test marked as
intermittent -- either causing it to permanently fail or fixing it.
wpt: Make WPT result formatting logic independent of ServoHandler
This will allow results to be formatted by other parts of the code (such as the intermittent filtering code). Previously formatting was handled in ServoHandler, which was a bit strange as it's really only necessary for GroupingFormatter and the intermittent filtering code. This also allows the results to be properly typed by the Python typing system.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they are part of the testing infrastructure that is generally untested.
This will allow results to be formatted by other parts of the code (such
as the intermittent filtering) code. Previously, formatting was handled
in ServoHandler, which was a bit strange as it's really only necessary
for GroupingFormatter and the intermittent filtering code. This also
allows the results to be properly typed by the Python typing system.
ci: Produce a single WPT log artifact
GitHub supports adding files to an artifact in parallel, as long as the filenames are unique. This makes it easier to download build results when more than a single builder fails.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because this is a small change to the CI.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Bump unicode-segmentation from 1.9.0 to 1.10.1
Bumps [unicode-segmentation](https://github.com/unicode-rs/unicode-segmentation) from 1.9.0 to 1.10.1.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a href="https://github.com/unicode-rs/unicode-segmentation/commits/v1.10.1">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Run `test-tidy` sooner for pull requests
This lets the builder fail sooner when there is an issue with the style.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they are a small change to the CI.
GitHub supports adding files to an artifact in parallel, as long as the
filenames are unique. This makes it easier to download build results
when more than a single builder fails.
Upgrade the Rust toolchain to 'nightly-2023-02-01'
<!-- Please describe your changes on the following line: -->
This change should address the failing nightly [rustc test jobs](https://github.com/servo/servo/actions/workflows/nightly-rust.yml)
For reference, these are the [relevant](https://github.com/rust-lang/rust/pull/107206) [PRs](https://github.com/rust-lang/rust/pull/104170) in rustc that I could find.
Signed-off-by: Mukilan Thiyagarajan <mukilanthiagarajan@gmail.com>
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [X] These changes do not require tests because there are existing unit tests for script_plugins that do pass.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
wpt: retain raw results for unexpected tests and subtests
ServoHandler currently processes the raw results for unexpected tests and subtests on the fly, discarding the test_end (test) or test_status (subtest) event data once it has generated output for the events.
This patch retains that event data in ServoHandler.test_failures and ServoHandler.subtest_failures, allowing servowpt.py to send structured data about unexpected tests and subtests to the dashboard (servo/intermittent-tracker#5), e.g.
```diff
diff --git a/tests/wpt/servowpt.py b/tests/wpt/servowpt.py
index fee6dcf2df..ac3e46e36d 100644
--- a/tests/wpt/servowpt.py
+++ b/tests/wpt/servowpt.py
@@ -27,6 +27,7 @@ import update # noqa: F401,E402
TRACKER_API = "https://build.servo.org/intermittent-tracker"
TRACKER_API_ENV_VAR = "INTERMITTENT_TRACKER_API"
+TRACKER_DASHBOARD_SECRET_ENV_VAR = "INTERMITTENT_TRACKER_DASHBOARD_SECRET"
GITHUB_API_TOKEN_ENV_VAR = "INTERMITTENT_TRACKER_GITHUB_API_TOKEN"
@@ -145,6 +146,48 @@ def run_tests(**kwargs):
logger.add_handler(handler)
wptrunner.run_tests(**kwargs)
+
+ if TRACKER_DASHBOARD_SECRET_ENV_VAR in os.environ:
+ body = []
+ for failure in handler.test_failures:
+ # print(f'>>> {repr(failure)}')
+ body.append({
+ 'path': failure['test'],
+ 'subtest': None,
+ 'expected': failure['expected'],
+ 'actual': failure['status'],
+ 'time': failure['time'] // 1000,
+ 'message': failure.get('message'),
+ 'stack': failure.get('stack'),
+ 'branch': os.environ.get('SERVO_BRANCH'),
+ 'build_url': os.environ.get('SERVO_BUILD_URL'),
+ 'pull_url': os.environ.get('SERVO_PULL_URL'),
+ })
+ for (path, failures) in handler.subtest_failures.items():
+ for failure in failures:
+ # print(f'>>> {repr(failure)}')
+ body.append({
+ 'path': path,
+ 'subtest': failure['subtest'],
+ 'expected': failure['expected'],
+ 'actual': failure['status'],
+ 'time': failure['time'] // 1000,
+ 'message': failure.get('message'),
+ 'stack': failure.get('stack'),
+ 'branch': os.environ.get('SERVO_BRANCH'),
+ 'build_url': os.environ.get('SERVO_BUILD_URL'),
+ 'pull_url': os.environ.get('SERVO_PULL_URL'),
+ })
+ request = urllib.request.Request(
+ f'{os.environ.get(TRACKER_API_ENV_VAR, TRACKER_API)}/dashboard/attempts',
+ method='POST',
+ data=json.dumps(body).encode('utf-8'),
+ headers={
+ 'Authorization': f'Bearer {os.environ[TRACKER_DASHBOARD_SECRET_ENV_VAR]}',
+ 'Content-Type': 'application/json',
+ })
+ urllib.request.urlopen(request)
+
if handler.unexpected_results and filter_intermittents_output:
all_filtered = filter_intermittents(
handler.unexpected_results,
```
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [ ] ~~`./mach build -d` does not report any errors~~
- [ ] ~~`./mach test-tidy` does not report any errors~~
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___
Integrate filter-intermittents into test-wpt
This change integrates the `filter-intermittents` command into `test-wpt`. This is in preparation for future work on tracking intermittent failures. This change also:
- Removes `ServoJsonFormatter` and replaces it with a generic WPT log handler which tracks unexpected results.
- The intermittent filter is now controlled via environment variables and the GitHub version requires a token instead of credentials.
- Output is saved to a single file and is always text.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they are changes to build infrastructure.
Invalidate latest nightly files in CloudFront.
After a new nightly build is uploaded to S3, bust the cache in CloudFront Edge servers with the CreateInvalidation AWS API.
For each platform we use the `/nightly/<platform>/servo-latest.<ext>*` pattern to invalidate both package and sha256 files. As part of this change, a new policy has been attached to the "download.servo.org-uploads-from-travis" IAM user
to allow the "cloudfront:CreateInvalidation" action.
Since CloudFront [invalidates every version][1] of the cached file, regardless of the headers used for that version, this change should invalidate the different caches for 'Accept-Encoding' header.
[1]: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html#invalidation-specifying-objects
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#29034
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because they update nightly upload process. Changes have been *partially* validated using [manual run ](https://github.com/servo/servo/actions/runs/4012975660/jobs/6892315365)
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
After a new nightly build is uploaded to S3,
bust the cache in CloudFront Edge servers with
the CreateInvalidation AWS API.
For each platform we use the
`/nightly/<platform>/servo-latest.<ext>*`
pattern to invalidate both package and sha256
files. As part of this change, a new policy
has been attached to the
"download.servo.org-uploads-from-travis" IAM user
to allow the "cloudfront:CreateInvalidation" action.
Since CloudFront [invalidates every version][1]
of the cached file, regardless of the headers used
for that version, this change should invalidate the
different caches for 'Accept-Encoding' header.
[1]: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html#invalidation-specifying-objectsCloses#29034
Signed-off-by: Mukilan Thiyagarajan <mukilanthiagarajan@gmail.com>
- Also updates raqote to latest with an upgrade of font-kit to 0.11
applied on as a patch
- Update lyon_geom to the latest version
Major change:
- All matrices are now stored in row major order. This means that
parameters to rotation functions no longer should be negated.
- `post_...()` functions are now named `then()`. `pre_transform()` is removed,
so `then()` is used and the order of operations changed.
Fix upstreaming of WPT changes
The GitHub Action needs access to Servo repository secrets, so switch to using the 'pull_request_target' event. Since these PRs have more complete access to the Servo repository, do not execute the version of the upstream script that comes with the PR. Instead, simply fetch the changes. To make this work, the script no longer expects the PR commit to be checked out, merely that they exist in the repository somewhere.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] There are tests for these changes
Remove UWP/HoloLens CI & Nightly jobs.
Servo [TSC has decided][1] to halt support for UWP platform. This PR only removes the CI & Nightly jobs and doesn't modify any code related to UWP support.
[1]: https://github.com/servo/project/blob/master/governance/tsc/tsc-2023-01-23.md#supported-platforms
Signed-off-by: Mukilan Thiyagarajan <mukilanthiagarajan@gmail.com>
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [X] These changes fix#28721
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because they are removing CI jobs and not modifying code.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
The GitHub Action needs access to Servo repository secrets, so switch to
using the 'pull_request_target' event. Since these PRs have more
complete access to the Servo repository, do not execute the version of
the upstream script that comes with the PR. Instead, simply fetch the
changes. To make this work, the script no longer expects the PR commit
to be checked out, merely that they exist in the repository somewhere.
Revert LLVM downgrade & fix nighly build
<!-- Please describe your changes on the following line: -->
Windows MSCV nightly builds have been failing
since Jan 18. The failure is similar to the
one @delan identified & fixed, where CI build started
to fail after the version of LLVM included in
Github Actions runner images was bumped from
14 to 15.
As discussed on Zulip, we have existing support
in ./mach bootstrap to download the LLVM binaries
from our s3 bucket. Following the documented
[process][1], new LLVM 15.0.5 binaries have now been
uploaded to S3.
I've validated that the windows msvc build no longer fails:
https://github.com/mukilan/servo/actions/runs/3996453221/jobs/6856528077
[1]: https://github.com/servo/servo/wiki/Upgrading-the-windows-LLVM-binaries
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix#29269 (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [X] These changes do not require tests because they modify only CI steps that have been validated by manual run.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Approve PR automatically after WPT sync successfully
This PR will utilize the `gh pr comment` command to comment `r+` as @servo-wpt-sync so that we can merge WPT sync PRs more easily.
---
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because it's related to the WPT sync CI job
