Catch more exceptions when delivering build notifications
It seems that catching ImportError isn't enough, so we must catch any type of exception when trying to deliver a build notification.
Fixes#29645.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they fix a small build infrastructure issue.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Rename `ComputedValuesExt::establishes_containing_block`
This renames the helper method to be a bit more accurate. For elements with static, relative, and sticky positioning, their containing block is always formed by their nearest block container ancestor. This method is really dealing with style that means an element will establish a containing block for absolutely positioned descendants.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they do not change behavior.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Download prebuilt dependencies from Github instead of S3
After moving nightly builds to GitHub releases, the major contributor to AWS cost is now the S3 data transfer charges.
This PR only addresses the data transfer costs incurred due to the download of gstreamer dependency in the linux and the Windows dependencies (llvm, cmake, moztools, openssl etc) for MSVC builds. Android dependencies and UWP/Hololens dependencies will continue to be served from S3. These can be moved to GH as well in a future PR.
The new [servo-build-deps](https://github.com/servo/servo-build-deps/releases) repo now has two tagged releases - linux and msvc. For now, I've manually uploaded the latest versions of the dependencies referred in Servo code to the corresponding releases. We can add the older versions of the dependencies later if needed, potentially using a script to copy from s3.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because it is a small change in the CI/bootstrap code.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Reorganize Servo's WPT Python scripts
This change moves all of Servo's WPT Python support scripts into one directory as they were previously scattered throughout the directory structure. This should allow more code reuse and make it easier to understand how everything fits together.
The changes:
- `tests/wpt/update` → `python/wpt/importer`
- `etc/ci/upstream-wpt-changes/wptupstreamer` → `python/wpt/exporter`
- `etc/ci/upstream-wpt-changes/test.py` → `python/wpt/test.py`
- `etc/ci/upstream-wpt-changes/tests` → `python/wpt/tests`
- `tests/wpt/servowpt.py` → - `python/wpt/update.py` - `python/wpt/run.py`
- `tests/wpt/manifestupdate.py` → `python/wpt/manifestupdate.py`
This change also removes
- The ability to run the `update-wpt` and `test-wpt` commands without using `mach`. These didn't work very well, because it was difficult to get all of the wptrunner and mach dependencies installed outside of the Python virtualenv. It's simpler if they are always run through `mach`.
- The old WPT change sync / upstreaming script that was no longer used. This allows this code to pass `test-tidy` now.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] There are tests for these changes
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Build on Mac with the dummy media stack
This is a temporary fix for broken GStreamer homebrew packages.
Fixes#29653.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix#29653.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Because empty set with `.all(predicate)` will always return `true`, it
will result in always filtering wrong elements when the classes is empty
set.
Thus, we return earlier with `always_empty` HTMLCollection in the mpety
classes case so that we can avoid filtering on empty sets.
This change moves all of Servo's WPT Python support scripts into one
directory as they were previously scattered throughout the directory
structure. This should allow more code reuse and make it easier to
understand how everything fits together.
The changes:
- `tests/wpt/update` → `python/wpt/importer`
- `etc/ci/upstream-wpt-changes/wptupstreamer` → `python/wpt/exporter`
- `etc/ci/upstream-wpt-changes/test.py` → `python/wpt/test.py`
- `etc/ci/upstream-wpt-changes/tests` → `python/wpt/tests`
- `tests/wpt/servowpt.py` →
- `python/wpt/update.py`
- `python/wpt/run.py`
- `tests/wpt/manifestupdate.py` → `python/wpt/manifestupdate.py`
This change also removes
- The ability to run the `update-wpt` and `test-wpt` commands without
using `mach`. These didn't work very well, because it was difficult
to get all of the wptrunner and mach dependencies installed outside
of the Python virtualenv. It's simpler if they are always run through
`mach`.
- The old WPT change upstreaming script that was no longer used.
After moving nightly builds to GitHub releases,
the major contributor to AWS cost is now the S3 data
transfer charges.
This PR only addresses the data transfer costs incurred
due to the download of gstreamer dependency in the
linux and the Windows dependencies (llvm, cmake, moztools,
openssl etc) for MSVC builds. Android dependencies and
UWP/Hololens dependencies will continue to be served
from S3. These can be moved to GH as well in a future PR.
Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
This renames the helper method to be a bit more accurate. For elements
with static, relative, and sticky positioning, their containing block is
always formed by their nearest block container ancestor. This method is
really dealing with style that means an element will establish a
containing block for absolutely positioned descendants.
Use ccache on windows workflow
Because `sccache` is not working when setting `CCACHE=sccache` I used ccache which is taking cache hits.
I would certainly love to share some timings, but I found out that windows builds have very inconsistent timings.
Try to fix the CI failures
It seems that the WPT update process added an unintentional flake to the results.
Fixes#29642.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#29642
- [x] There are tests for these changes
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Make errors during build status notification warnings
Instead of raising an exception, simply print a warning on stderr when notification fails.
Fixes#29635.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#29635
- [x] These changes do not require tests because they fix a minor tools issue.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Fix calculation of block axis for abspos elements in Layout 2020
The calculation of the block axis positioning of absolutely positioned elements was using the inline size of the containing block instead of the block size.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] There are tests for these changes
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
The calculation of the block axis positioning of absolutely positioned
elements was using the inline size of the containing block instead of
the block size.
Use GStreamer in Windows workflow
I accidental came across https://github.com/servo/servo/projects/26 and can't helped but to fix one. (Few were already fixed but not closed).
Test run: https://github.com/sagudev/servo/actions/runs/4687138840/jobs/8306052042
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#28277
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because it is only CI
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Fix nightly upload to GH release logic.
The boto3 S3 client automatically [closes the given fileobj](https://github.com/boto/s3transfer/blob/develop/s3transfer/upload.py#L106) after the transfer is complete. This prevents us from reusing the package_hash_fileobj between s3 and github upload methods. This is causing the [upload to github to fail](https://github.com/servo/servo/actions/runs/4685791796/jobs/8303739124) with:
```
ValueError: I/O operation on closed file.
File "/home/runner/work/servo/servo/python/servo/package_commands.py", line 792, in upload_nightly
upload_to_github_release(platform, package, package_hash_fileobj)
File "/home/runner/work/servo/servo/python/servo/package_commands.py", line 635, in upload_to_github_release
package_hash_fileobj.getbuffer().nbytes,
```
This PR changes fixes the issue by creating fresh instances of io.BytesIO within the two upload_to_* methods.
I've triggered a [manual nightly job](https://github.com/servo/servo/actions/runs/4686470246) based on this branch. This PR can be kept open until the build completes, in case other issues are surfaced.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because they fix an error in nightly CI Job
Use notify-py to send notifications
- Use notify-py to send notifications, but use a custom notifier on
Linux since transient (ie non-sticky) notifications are not supported.
- Add an icon to the notification.
- Don't send a notification after doing `./mach check` because that
can trigger notifications when `rust-analyzer` is working.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they just change build notifications.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
- Use notify-py to send notifications, but use a custom notifier on
Linux since transient (ie non-sticky) notifications are not supported.
- Add an icon to the notification.
- Don't send a notification after doing `./mach check` because that
can trigger notifications when `rust-analyzer` is working.
The boto3 S3 client automatically closes the given
fileobj after the transfer is complete. This prevents
us from reusing the package_hash_fileobj between
s3 and github upload methods.
This PR changes fixes the issue by creating fresh
instances of io.BytesIO within the upload_to_*
methods.
Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
Import WPT tests every week instead of every day
We often have multiple imports stacked up which makes it more confusing to land each one. In addition, giving a week to do the import will give us more margin to triage failing tests and hopefully to be able to use intermittent expectations for flaky tests.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because this just change the frequency of a scheduled action.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Upload nightly builds to Github Releases
This change extends the `mach upload-nightly` command to publish the nightly builds for all platforms as GH Release assets.
The GH releases are made on a separate repository so that we can persist older nightly builds without having to accumulate git tags for them.
Example releases on a fork can be seen here: https://github.com/mukilan/servo-nightly-builds/releases/
Assets of the *latest* nightly releases will available at a stable url e.g for linux
`https://github.com/mukilan/servo-nightly-builds/releases/latest/download/servo-latest.tar.gz`
Some design tradeoffs in this approach are:
1. To allow the 'latest' link from servo.org to remain stable, the release assets are named 'servo-latest.{ext}' instead of containing the release tag/date.
2. The release is created as draft and published atomically when all platforms have been built successfully. This allows us to link to the 'latest' alias from servo.org while gauranteeing that it contains builds for all platforms. The other option here would be to have code in servo.org UI that uses GH API to find the most recent release with a successful build for a given platform.
3. The tags in the nightly repo are all based on the same commit that has no relation to servo code base.
---
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because it extends the nightly CI job
This change extends the `mach upload-nightly` command to
publish the nightly builds for all platforms as GH Release
assets.
The GH releases are made on a separate repository so
that we can persist older nightly builds without having
to accumulate git tags for them.
Some design tradeoffs in this approach are:
1. To allow the 'latest' link from servo.org to remain stable,
the release assets are named 'servo-latest.{ext}' instead of
containing the release tag/date.
2. The release is created as draft and published atomically
when all platforms have been built successfully. This allows
us to link to the 'latest' alias from servo.org while
gauranteeing that it contains builds for all platforms.
The other option here would be to have code in servo.org UI
that uses GH API to find the most recent release with a
successful build for a given platform.
3. The tags in the nightly repo are all based on the same
commit that has no relation to servo code base.
Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
We often have multiple imports stacked up which makes it more confusing
to land each one. In addition, giving a week to do the import will give
us more margin to triage failing tests and hopefully to be able to use
intermittent expectations for flaky tests.
Handle non-UTF8 diffs in the WPT upstream script
The output of `git diff` and `git show` can include non-UTF8 text or binary data, so the WPT upstream script should handle this properly.
Fixes#29620.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#29620
- [x] There are tests for these changes
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Dedupe redox_syscall 0.1
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [ ] `./mach build -d` does not report any errors
- [ ] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Split quick check from pull request and rename WPT export actions
Split out the quick check GitHub action (which is really only meant for forks). This prevents the quick check from running twice for PRs from branches on the upstream repository. Also rename the WPT export action file to make it clearer that it runs for pull requests. This change also means we no longer trigger builds when simply editing pull request body or title.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#29603
- [x] These changes do not require tests because this is just a reorganization of the workflow files.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Split out the quick check GitHub action (which is really only meant for
forks). This prevents the quick check from running twice for PRs from
branches on the upstream repository. Also rename the WPT export action
file to make it clearer that it runs for pull requests.
