Fixed bug where ./mach test-wpt was unable to run on windows due to servowpt…
….py not correctly setting default binary path
<!-- Please describe your changes on the following line: -->
Changed the default_binary_path variable to account for the binary_name change to "servo.exe" on win32 properly. Before, the target_dir was being changed after the default path variable was set. This would result in the following error when ./mach test-wpt was run:
"Binary path C:\Code\servo\target\release\servo does not exist"
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
<!-- Either: -->
- [x] These changes do not require tests because the validity of the change can be tested by running ./mach test-wpt.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Catch more exceptions when delivering build notifications
It seems that catching ImportError isn't enough, so we must catch any type of exception when trying to deliver a build notification.
Fixes#29645.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they fix a small build infrastructure issue.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Download prebuilt dependencies from Github instead of S3
After moving nightly builds to GitHub releases, the major contributor to AWS cost is now the S3 data transfer charges.
This PR only addresses the data transfer costs incurred due to the download of gstreamer dependency in the linux and the Windows dependencies (llvm, cmake, moztools, openssl etc) for MSVC builds. Android dependencies and UWP/Hololens dependencies will continue to be served from S3. These can be moved to GH as well in a future PR.
The new [servo-build-deps](https://github.com/servo/servo-build-deps/releases) repo now has two tagged releases - linux and msvc. For now, I've manually uploaded the latest versions of the dependencies referred in Servo code to the corresponding releases. We can add the older versions of the dependencies later if needed, potentially using a script to copy from s3.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because it is a small change in the CI/bootstrap code.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Reorganize Servo's WPT Python scripts
This change moves all of Servo's WPT Python support scripts into one directory as they were previously scattered throughout the directory structure. This should allow more code reuse and make it easier to understand how everything fits together.
The changes:
- `tests/wpt/update` → `python/wpt/importer`
- `etc/ci/upstream-wpt-changes/wptupstreamer` → `python/wpt/exporter`
- `etc/ci/upstream-wpt-changes/test.py` → `python/wpt/test.py`
- `etc/ci/upstream-wpt-changes/tests` → `python/wpt/tests`
- `tests/wpt/servowpt.py` → - `python/wpt/update.py` - `python/wpt/run.py`
- `tests/wpt/manifestupdate.py` → `python/wpt/manifestupdate.py`
This change also removes
- The ability to run the `update-wpt` and `test-wpt` commands without using `mach`. These didn't work very well, because it was difficult to get all of the wptrunner and mach dependencies installed outside of the Python virtualenv. It's simpler if they are always run through `mach`.
- The old WPT change sync / upstreaming script that was no longer used. This allows this code to pass `test-tidy` now.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] There are tests for these changes
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
This change moves all of Servo's WPT Python support scripts into one
directory as they were previously scattered throughout the directory
structure. This should allow more code reuse and make it easier to
understand how everything fits together.
The changes:
- `tests/wpt/update` → `python/wpt/importer`
- `etc/ci/upstream-wpt-changes/wptupstreamer` → `python/wpt/exporter`
- `etc/ci/upstream-wpt-changes/test.py` → `python/wpt/test.py`
- `etc/ci/upstream-wpt-changes/tests` → `python/wpt/tests`
- `tests/wpt/servowpt.py` →
- `python/wpt/update.py`
- `python/wpt/run.py`
- `tests/wpt/manifestupdate.py` → `python/wpt/manifestupdate.py`
This change also removes
- The ability to run the `update-wpt` and `test-wpt` commands without
using `mach`. These didn't work very well, because it was difficult
to get all of the wptrunner and mach dependencies installed outside
of the Python virtualenv. It's simpler if they are always run through
`mach`.
- The old WPT change upstreaming script that was no longer used.
After moving nightly builds to GitHub releases,
the major contributor to AWS cost is now the S3 data
transfer charges.
This PR only addresses the data transfer costs incurred
due to the download of gstreamer dependency in the
linux and the Windows dependencies (llvm, cmake, moztools,
openssl etc) for MSVC builds. Android dependencies and
UWP/Hololens dependencies will continue to be served
from S3. These can be moved to GH as well in a future PR.
Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
Fix nightly upload to GH release logic.
The boto3 S3 client automatically [closes the given fileobj](https://github.com/boto/s3transfer/blob/develop/s3transfer/upload.py#L106) after the transfer is complete. This prevents us from reusing the package_hash_fileobj between s3 and github upload methods. This is causing the [upload to github to fail](8303739124) with:
```
ValueError: I/O operation on closed file.
File "/home/runner/work/servo/servo/python/servo/package_commands.py", line 792, in upload_nightly
upload_to_github_release(platform, package, package_hash_fileobj)
File "/home/runner/work/servo/servo/python/servo/package_commands.py", line 635, in upload_to_github_release
package_hash_fileobj.getbuffer().nbytes,
```
This PR changes fixes the issue by creating fresh instances of io.BytesIO within the two upload_to_* methods.
I've triggered a [manual nightly job](4686470246) based on this branch. This PR can be kept open until the build completes, in case other issues are surfaced.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because they fix an error in nightly CI Job
Use notify-py to send notifications
- Use notify-py to send notifications, but use a custom notifier on
Linux since transient (ie non-sticky) notifications are not supported.
- Add an icon to the notification.
- Don't send a notification after doing `./mach check` because that
can trigger notifications when `rust-analyzer` is working.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they just change build notifications.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
- Use notify-py to send notifications, but use a custom notifier on
Linux since transient (ie non-sticky) notifications are not supported.
- Add an icon to the notification.
- Don't send a notification after doing `./mach check` because that
can trigger notifications when `rust-analyzer` is working.
The boto3 S3 client automatically closes the given
fileobj after the transfer is complete. This prevents
us from reusing the package_hash_fileobj between
s3 and github upload methods.
This PR changes fixes the issue by creating fresh
instances of io.BytesIO within the upload_to_*
methods.
Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
This change extends the `mach upload-nightly` command to
publish the nightly builds for all platforms as GH Release
assets.
The GH releases are made on a separate repository so
that we can persist older nightly builds without having
to accumulate git tags for them.
Some design tradeoffs in this approach are:
1. To allow the 'latest' link from servo.org to remain stable,
the release assets are named 'servo-latest.{ext}' instead of
containing the release tag/date.
2. The release is created as draft and published atomically
when all platforms have been built successfully. This allows
us to link to the 'latest' alias from servo.org while
gauranteeing that it contains builds for all platforms.
The other option here would be to have code in servo.org UI
that uses GH API to find the most recent release with a
successful build for a given platform.
3. The tags in the nightly repo are all based on the same
commit that has no relation to servo code base.
Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
Also organize some of the imports. Now that Servo only uses Python 3,
this module is unnecessary. This is part of the gradual migration to
using only Python 3.
This cleans up the GitHub actions yaml a bit and ensures that developers
are running this check locally before submitting changes. In addition,
it allows adding tests for this check. Finally, this change fixes the
tidy tests by upgrading voluptuous for Python 3.10 as well as by
reverting an inadvertent change for NixOS compatibility on one of the
dummy testing files.
Add try command to mach & try build partitioned
Adds `./mach try` command that enables anybody to easily test their changes without opening PR and requesting try from bors-servo, by force pushing HEAD to appropriate branch. Command accepts branches names to select only partial runs of CI (same like bors try command). So if you only want to test mac build (that would be `@bors-servo try=mac`) you run `./mach try mac`. If no job is specified, try branch is used.
As partitioned CI jobs were not working after migration to GitHub Actions I remade them by using if guards.
Also WPT jobs were failing due to empty `INTERMITTENT_TRACKER_DASHBOARD_SECRET` on my fork, so I added additional check to prevent failed run.
And that concludes my work on #29379🎉
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#29379
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because it's CI
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
The layout-2020 variant will be consumed by the
internal WPT trend [dashboard][1] only and won't
be made available on servo.org.
This change will be removed once the layout
engine we will use going forward is finalized.
[1]: https://servo.github.io/internal-wpt-dashboard/
Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
In old versions of GStreamer gstvideoconvertscale was split into two
plugins, gstvideoscale and gstvideoconvert. This change makes the use of
these three plugins conditional on their existence and has Windows use
the two plugins from the older version of GStreamer. In addition, it
starts to clean up the gstreamer.py file which is responsible for
managing GStreamer libraries and plugins. This is the first change in a
series of changes that seek to clean up the way these libraries are
managed during the build.
Aggregate unexpected results into logs
This makes it easier to run `update-wpt` based on results from the bots by writing a new version of the raw log that filters out all tests with expected results. Then aggregate this output into the filtered results archive. A future version of this could aggregate all unexpected results that were not filtered as intermittents.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they are infrastructure changes.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
This makes it easier to run `update-wpt` based on results from the bots.
A future version of this could aggregate all unexpected results that
were not filtered as intermittents.
This will allow results to be formatted by other parts of the code (such
as the intermittent filtering) code. Previously, formatting was handled
in ServoHandler, which was a bit strange as it's really only necessary
for GroupingFormatter and the intermittent filtering code. This also
allows the results to be properly typed by the Python typing system.
Integrate filter-intermittents into test-wpt
This change integrates the `filter-intermittents` command into `test-wpt`. This is in preparation for future work on tracking intermittent failures. This change also:
- Removes `ServoJsonFormatter` and replaces it with a generic WPT log handler which tracks unexpected results.
- The intermittent filter is now controlled via environment variables and the GitHub version requires a token instead of credentials.
- Output is saved to a single file and is always text.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they are changes to build infrastructure.
After a new nightly build is uploaded to S3,
bust the cache in CloudFront Edge servers with
the CreateInvalidation AWS API.
For each platform we use the
`/nightly/<platform>/servo-latest.<ext>*`
pattern to invalidate both package and sha256
files. As part of this change, a new policy
has been attached to the
"download.servo.org-uploads-from-travis" IAM user
to allow the "cloudfront:CreateInvalidation" action.
Since CloudFront [invalidates every version][1]
of the cached file, regardless of the headers used
for that version, this change should invalidate the
different caches for 'Accept-Encoding' header.
[1]: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html#invalidation-specifying-objectsCloses#29034
Signed-off-by: Mukilan Thiyagarajan <mukilanthiagarajan@gmail.com>
Windows MSCV nightly builds have been failing
since Jan 18. The failure is similar to the
one @delan identified where the CI build started
to fail after the version of LLVM included in
Github Actions runner images was bumped from
14 to 15.
As discussed on Zulip, we already have support
in ./mach bootstrap to download the LLVM binaries
from our s3 bucket. Following the documented
[process], new LLVM 15.0.5 binaries have been
uploaded to S3.
This PR updates the bootstrap script to use this
new LLVM version.
[process]: https://github.com/servo/servo/wiki/Upgrading-the-windows-LLVM-binaries
Signed-off-by: Mukilan Thiyagarajan <mukilanthiagarajan@gmail.com>
This change integrates the filter-intermittents command into test-wpt.
This is in preparation for future work on tracking intermittent
failures. This change also:
- Removes the SrvoJson logger and replaces it with a generic WPT log
handler which tracks unexpected results.
- The intermittent filter is now controlled via environment variables
and the GitHub version requires a token instead of credentials.
- Output is saved to a single file and is always text.
Fail faster on CI when Linux bootstrap fails
Raise an exception when dependencies fail to install. Also split the run phase of the Linux bootstrap so that either of these failing commands will cause the job to fail.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they just change minor script issues.
Combine `run.py` and `update.py` into `servowpt.py` in order to allow
them to share code. Import them directly into the mach script to avoid
having to call `compile` and `exec` on the code. This makes it clearer
how they are executed. In addition, move all of the setup into
`setupwpt.py` to avoid differences between tests executed via mach and
not. Finally, be more ambitious when detecting the build to use. If none
was specified, try to use the one that exists between "release" and
"debug."
Raise an exception when dependencies fail to install. Also split the run
phase of the Linux bootstrap so that either of these failing commands
will cause the job to fail.
