Use GStreamer in Windows workflow
I accidental came across https://github.com/servo/servo/projects/26 and can't helped but to fix one. (Few were already fixed but not closed).
Test run: https://github.com/sagudev/servo/actions/runs/4687138840/jobs/8306052042
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#28277
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because it is only CI
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Fix nightly upload to GH release logic.
The boto3 S3 client automatically [closes the given fileobj](https://github.com/boto/s3transfer/blob/develop/s3transfer/upload.py#L106) after the transfer is complete. This prevents us from reusing the package_hash_fileobj between s3 and github upload methods. This is causing the [upload to github to fail](https://github.com/servo/servo/actions/runs/4685791796/jobs/8303739124) with:
```
ValueError: I/O operation on closed file.
File "/home/runner/work/servo/servo/python/servo/package_commands.py", line 792, in upload_nightly
upload_to_github_release(platform, package, package_hash_fileobj)
File "/home/runner/work/servo/servo/python/servo/package_commands.py", line 635, in upload_to_github_release
package_hash_fileobj.getbuffer().nbytes,
```
This PR changes fixes the issue by creating fresh instances of io.BytesIO within the two upload_to_* methods.
I've triggered a [manual nightly job](https://github.com/servo/servo/actions/runs/4686470246) based on this branch. This PR can be kept open until the build completes, in case other issues are surfaced.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because they fix an error in nightly CI Job
Use notify-py to send notifications
- Use notify-py to send notifications, but use a custom notifier on
Linux since transient (ie non-sticky) notifications are not supported.
- Add an icon to the notification.
- Don't send a notification after doing `./mach check` because that
can trigger notifications when `rust-analyzer` is working.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they just change build notifications.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
- Use notify-py to send notifications, but use a custom notifier on
Linux since transient (ie non-sticky) notifications are not supported.
- Add an icon to the notification.
- Don't send a notification after doing `./mach check` because that
can trigger notifications when `rust-analyzer` is working.
The boto3 S3 client automatically closes the given
fileobj after the transfer is complete. This prevents
us from reusing the package_hash_fileobj between
s3 and github upload methods.
This PR changes fixes the issue by creating fresh
instances of io.BytesIO within the upload_to_*
methods.
Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
Import WPT tests every week instead of every day
We often have multiple imports stacked up which makes it more confusing to land each one. In addition, giving a week to do the import will give us more margin to triage failing tests and hopefully to be able to use intermittent expectations for flaky tests.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because this just change the frequency of a scheduled action.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Upload nightly builds to Github Releases
This change extends the `mach upload-nightly` command to publish the nightly builds for all platforms as GH Release assets.
The GH releases are made on a separate repository so that we can persist older nightly builds without having to accumulate git tags for them.
Example releases on a fork can be seen here: https://github.com/mukilan/servo-nightly-builds/releases/
Assets of the *latest* nightly releases will available at a stable url e.g for linux
`https://github.com/mukilan/servo-nightly-builds/releases/latest/download/servo-latest.tar.gz`
Some design tradeoffs in this approach are:
1. To allow the 'latest' link from servo.org to remain stable, the release assets are named 'servo-latest.{ext}' instead of containing the release tag/date.
2. The release is created as draft and published atomically when all platforms have been built successfully. This allows us to link to the 'latest' alias from servo.org while gauranteeing that it contains builds for all platforms. The other option here would be to have code in servo.org UI that uses GH API to find the most recent release with a successful build for a given platform.
3. The tags in the nightly repo are all based on the same commit that has no relation to servo code base.
---
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because it extends the nightly CI job
This change extends the `mach upload-nightly` command to
publish the nightly builds for all platforms as GH Release
assets.
The GH releases are made on a separate repository so
that we can persist older nightly builds without having
to accumulate git tags for them.
Some design tradeoffs in this approach are:
1. To allow the 'latest' link from servo.org to remain stable,
the release assets are named 'servo-latest.{ext}' instead of
containing the release tag/date.
2. The release is created as draft and published atomically
when all platforms have been built successfully. This allows
us to link to the 'latest' alias from servo.org while
gauranteeing that it contains builds for all platforms.
The other option here would be to have code in servo.org UI
that uses GH API to find the most recent release with a
successful build for a given platform.
3. The tags in the nightly repo are all based on the same
commit that has no relation to servo code base.
Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
We often have multiple imports stacked up which makes it more confusing
to land each one. In addition, giving a week to do the import will give
us more margin to triage failing tests and hopefully to be able to use
intermittent expectations for flaky tests.
Handle non-UTF8 diffs in the WPT upstream script
The output of `git diff` and `git show` can include non-UTF8 text or binary data, so the WPT upstream script should handle this properly.
Fixes#29620.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#29620
- [x] There are tests for these changes
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Dedupe redox_syscall 0.1
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [ ] `./mach build -d` does not report any errors
- [ ] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Split quick check from pull request and rename WPT export actions
Split out the quick check GitHub action (which is really only meant for forks). This prevents the quick check from running twice for PRs from branches on the upstream repository. Also rename the WPT export action file to make it clearer that it runs for pull requests. This change also means we no longer trigger builds when simply editing pull request body or title.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#29603
- [x] These changes do not require tests because this is just a reorganization of the workflow files.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Split out the quick check GitHub action (which is really only meant for
forks). This prevents the quick check from running twice for PRs from
branches on the upstream repository. Also rename the WPT export action
file to make it clearer that it runs for pull requests.
Fix the WPT export script
- Have the WPT exporter script use the WPT_SYNC_TOKEN for checking out
wpt.
- Make sure the local WPT repository is unshallow when pushing.
- When searching for existing PRs use the main GitHub search API,
as the pull request search does not seem to properly process
the "head" parameter.
- When deleting branches in the downstream WPT repository, use
the full URL to avoid trying to modify the upstream repository.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] There are tests for these changes OR
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
- Have the WPT exporter script use the WPT_SYNC_TOKEN for checking out
wpt.
- Make sure the local WPT repository is unshallow when pushing.
- When searching for existing PRs use the main GitHub search API,
as the pull request search does not seem to properly process
the "head" parameter.
- When deleting branches in the downstream WPT repository, use
the full URL to avoid trying to modify the upstream repository.
Pass S3 secrets for nightly builds and fix windows build
<!-- Please describe your changes on the following line: -->
Secrets need to be passed to reusable workflows either by name or by `secrets: inherit` from the caller workflow. The latter allows accessing all secrets in scope.
Windows build has a bug where the upload-nightly command is not run from the correct working directory.
[This manual build](https://github.com/servo/servo/actions/runs/4665552398) based on this branch has passed and the binaries have also been updated in servo.org for today.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#29614
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because it fixes bug in CI configuration.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Secrets need to be passed to reusable workflows either
by name or by `secrets: inherit` from the caller workflow.
The latter allows accessing all secrets in scope.
Windows build has a bug where the upload-nightly command is
not run from the correct working directory.
Closes#29614
Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
Bump openssl from 0.10.49 to 0.10.50
Bumps [openssl](https://github.com/sfackler/rust-openssl) from 0.10.49 to 0.10.50.
<details>
<summary>Commits</summary>
<ul>
<li><a href="8395a89532"><code>8395a89</code></a> Release openssl v0.10.50</li>
<li><a href="b03b9ea09b"><code>b03b9ea</code></a> Release openssl-sys v0.9.85</li>
<li><a href="a03a346d91"><code>a03a346</code></a> Merge pull request <a href="https://redirect.github.com/sfackler/rust-openssl/issues/1874">#1874</a> from vishwin/master</li>
<li><a href="1a52fa61a4"><code>1a52fa6</code></a> Bump LibreSSL to 3.7.2</li>
<li><a href="f74b8e6f4e"><code>f74b8e6</code></a> Merge pull request <a href="https://redirect.github.com/sfackler/rust-openssl/issues/1871">#1871</a> from haroldbruintjes/feature/cipher-inplace</li>
<li><a href="edf3a165c7"><code>edf3a16</code></a> Add in-place cipher update method</li>
<li><a href="05ce5bc241"><code>05ce5bc</code></a> Merge pull request <a href="https://redirect.github.com/sfackler/rust-openssl/issues/1870">#1870</a> from sfackler/alex-patch-1</li>
<li><a href="b8559cbaf8"><code>b8559cb</code></a> whoops</li>
<li><a href="12a0de5839"><code>12a0de5</code></a> Raise the minimum CC version</li>
<li>See full diff in <a href="https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.49...openssl-v0.10.50">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Remove more Taskcluster and Treeherder integration
Servo no longer uses Taskcluster and Treeherder, so this change removes script references to those services and support files.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because this just removes unused code and files.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Replace usage of six.moves.urllib with urllib
Also organize some of the imports. Now that Servo only uses Python 3, this module is unnecessary. This is part of the gradual migration to using only Python 3.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they do not change behavior.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Package in platform workflows & nightly switch to them
Packaging takes only 1min per OS which I think it is reasonable tradeoff.
Nightly builds are also getting speed up due to sccache.
Also organize some of the imports. Now that Servo only uses Python 3,
this module is unnecessary. This is part of the gradual migration to
using only Python 3.
Update shell.nix
<!-- Please describe your changes on the following line: -->
After https://github.com/NixOS/nixpkgs/pull/207938 was merged the option `pkgs.xlibsWrapper` was removed. This PR replacing it with `xorg.libX11` as this is enouth to build and test in `nix-shell` environment.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
<!-- Either: -->
- [X] These changes do not require tests because PR updates nix build environment.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Have all mach command accept "--with-layout-2020" and "--layout-2020"
Some command accepted one and some the other. This changes makes them all accept both so it's no longer necessary to remember which uses which.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they change the arguments accepted by mach.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Remove unnecessary steps from "multipart/form-data encoding algorithm"
The specifications for the "multipart/form-data encoding algorithm" has changed [1], and the "_charset_" is not now handled here.
Remove no longer necessary steps from the "multipart/form-data encoding algorithm".
Similar to the "text/plain encoding algorithm" case fixed by f0818aa3 (Remove unnecessary steps from "text/plain encoding algorithm").
[1]: https://github.com/whatwg/html/pull/3645
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#29593 (GitHub issue number if applicable)
<!-- Either: -->
- [x] These changes do not require tests because this patch doesn't expect actual behavior, just removing extra steps.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Enable CSS position tests and add results
This also removes imported position: sticky tests from the Mozilla directory. These were only supposed to be temporary until the upstream version from WPT were imported.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they update tests and results.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
This also removes imported position: sticky tests from the Mozilla
directory. These were only supposed to be temporary until the upstream
version from WPT were imported.
The specifications for the "multipart/form-data encoding algorithm" has
changed [1], and the "_charset_" is not now handled here.
Remove no longer necessary steps from the "multipart/form-data encoding
algorithm".
Similar to the "text/plain encoding algorithm" case fixed by f0818aa3
(Remove unnecessary steps from "text/plain encoding algorithm").
[1]: https://github.com/whatwg/html/pull/3645
Signed-off-by: Yutaro Ohno <yutaro.ono.418@gmail.com>
Remove unnecessary steps from "text/plain encoding algorithm"
For "text/plain encoding algorithm", the specification [1] has changed [2] and "_charset_" is not now handled here. It's supposed to be handled in "construct the form data set" algorithm, and we've already implemented that [3]. So we now have extra steps for "text/plain encoding" algorithm.
Remove no longer necessary steps from text/plain encoding algorithm so that it meets the specification.
[1]: https://html.spec.whatwg.org/multipage/form-control-infrastructure.html#text/plain-encoding-algorithm
[2]: https://github.com/whatwg/html/pull/3645
[3]: https://github.com/servo/servo/pull/25217
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#25221 (GitHub issue number if applicable)
<!-- Either: -->
- [x] These changes do not require tests because this patch doesn't expect actual behavior, just removing extra steps.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
