There were some issues with the way that the `--release` and `--dev`
arguments were handled in mach commands.
- Not all commands accepted them in the same way. For instance `./mach
test-wpt` didn't really accept them at all.
- If you did not pass either of them, mach would try to guess which
build you meant. This guess was often quite surprising as it wasn't
printed and it depended on the state of the your target directory,
which is difficult to remember.
- The `dev` profile is colloquially called a "debug" profile and some
commands accepted `-d` or `--debug...` like arguments, but `--debug`
with `./mach run` meant run in a debugger. It was easy to mix this
up.
This change:
- Centralizes where build type argument processing happens. Now it the
same shared decorator in CommandBase.
- Uses a `BuildType` enum instead of passing around two different
booleans. This reduces the error checking for situations where both
are true.
- Be much less clever about guessing what build to use. Now if you
don't specify a build type, `--dev` is chosen. I think this behavior
matches cargo.
- Makes it so that `./mach test-wpt` accepts the exact same arguments
and has the same behavior as other commands. In addition, the suite
correct for `test-wpt` is removed. There are only two suites now and
it's quite unlikely that people will confuse WPT tests for rust unit
tests.
PR #30048 switched `./mach update-wpt` to use 2020 layout engine by default. Since the WPT import job was not passing any flags when updating 2013 expectations, it was not using the correct metadata files, leading to failures when landing the recent [wpt sync PR](https://github.com/servo/servo/pull/30075)
Now that the new version of GStreamer fixes this issue, we can remove
the workarounds for this problem as well as all of the homebrew
bootstrapping logic.
Do not do the duplicate workflow checks for main workflow jobs triggered
by 'issue_comment' GitHub Actions. This prevents try jobs from being
skipped. This issue was not detected in my testing, because the main
workflow does not run in personal repositories and 'issue_comment'
actions cannot be tested from PRs.
This is the last piece of the puzzle to turning off bors. This makes
functionality provided by bors to understand "@bors-servo try" a GitHub
Action. For now the syntax is more or less the same, but we can modify
it in the future and even add support for custom configuration options
(more specific build combinations or even passing compiler flags).
The big difference between this and what bors does is that there is no
merge commit. GitHub simply runs tests on the version of the branch that
is on a pull request. There is always the risk that tests might start
failing when a branch is rebased, but this offers a bit more control
because you can easily rebase from the PR and the merge queue will check
this as well.
Much of the code used to import WPT tests from upstream has been moved
to the WPT repository itself, so this can be reused. In addition,
simplify the workflows by merging the entire process into mach and also
directly into the GitHub workflow. This should fix WPT imports after
combining compilation of layout and legacy layout.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
When the decision job decides that a workflow should be skipped because
an identical one has already run, that workflow should be marked as
successful and not unsuccessful.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Split the WPT workflow from the Linux one
This allows running tests for only one layout system at a time. This was
originally part of #29950, but was lost in the shuffle.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they improve the CI.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
When using "try-<platform>" trigger unit tests
There is currently no way to trigger unit tests using the bors-servo try
infrastructure. This change makes it so that using try-<platform>
triggers unit tests for that platform. This will be especially useful
when testing changes for Windows before trying to land them.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they are just GitHub Actions changes.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
This PR configures the main workflow (currently
used by bors via auto & try branches) to run
on 'merge_group' events emitted when pull requests
are added to GitHub merge queue.
Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
There is currently no way to trigger unit tests using the bors-servo try
infrastructure. This change makes it so that using try-<platform>
triggers unit tests for that platform. This will be especially useful
when testing changes for Windows before trying to land them.
Remove:
- tidy self test support from `./mach test`
- `./mach test-idl`
Adds a `./mach test-scripts` command that is responsible for running all
Python script tests. Run this during the CI to catch regressions in
changes to scripts. The WebIDL tests are still *very* slow and there are
from Gecko, so only run them when "-a" is passed meaning all tests.
Default nightly builds to layout 2020
This PR switches the filenames used by nightly builds so that the default `servo-latest.{ext}` packages use the 2020 engine and the 'servo-latest-legacy-layout.{ext}` packages use the 2013 engine.
Since the platform workflows are reused by the main, quick-check, PR and WPT import flows and since the 2013/2020 flags are also controlled by try branches, I've kept the changes isolated to `nightly.yml` and `package_commands.py` and have avoided changing the input parameters. If we want to deault all the workflows to 2020, I can decline this PR and raise a new one.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they modify packaging logic.
Run unit tests on the main Linux workflow
Not running unit tests on the main Linux workflow makes it faster, but also allows merging changes that break the unit tests. This change fixes that.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they are CI changes.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Implement `bootstrap-gstreamer` for all platforms
This change makes it so that the Platform classes can now handle installing GStreamer dependencies and properly setting up the environment including when cross-compiling. For Windows and Linux, prepackaged GStreamer is now installed into `target/dependencies/gstreamer` when not installed system-wide. In addition this change:
1. The Environment path append helper is moved to `util.py` and a new `prepend` version is added.
2. `set_run_env` and `build_dev` functions are combined and include more code from callers so environment setup is more often in the same place. Now code that used to call `set_run_env` calls `build_dev` and then `os.environ.update(...)`. We can further refine this with the `is_build` argument later.
4. Python typing information is added in many places.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#25335
- [x] These changes do not require tests because they are to the build scripts.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
This change makes it so that the Platform classes can now handle
installing GStreamer dependencies and properly setting up the
environment including when cross-compiling. For Windows and Linux
is now installed into `target/dependencies/gstreamer` when not installed
system-wide. In addition:
1. Creating and moving existing environment path append helpers to
`util.py`.
2. Combining the `set_run_env` and `build_dev` functions and moving
some outside code into them so that it can be shared. Now code that
used to call `set_run_env` calls `build_dev` and then
`os.environ.update(...)`.
3. Adding Python typing information in many places.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
We'd like to start offering servo nightly builds
with layout 2020 engine so that users can test
the new layout without building servo. These new
builds will be offered in addition to the current
2013 builds until we switch over to 2020 as the default.
Also fix report_aggregated_expected_results.py which was reporting an
error when there were no failing tests. This is more commonly an issue
with Layout 2020 because if runs fewer tests and was causing builds to
show up as failing even when they were not.
This PR re-enables support for the gstreamer mediastack
in macOS by consuming the official binary '.pkg' files
from gstreamer.freedesktop.org
To maintain symmetry with other platforms, the '.pkg'
files are uploaded to servo-build-deps and fetched from
there using the new script 'etc/install_macos_gstreamer.sh'.
Unlike the Homebrew version, the official GStreamer is
distributed as a 'relocatable' framework i.e the dylibs all
have @rpath-relative install names and also link to other
dylibs using @rpath relative path. To address this difference
the 'servo' binary needs to be patched with 'install_name_tool'
to add an LC_RPATH command that sets the relative paths
that the dynamic linker should search when trying to satify
dependencies. In Servo's case, this will be a path relative to
the 'servo' binary itself i.e '@executable_path/lib/'
The additional 'lib' is due to a flaw in the gstreamer
packaging where the install names of some of the dylibs
have the prefix '@rpath/lib' and some of them just have '@rpath'.
This PR also fixes a couple of issues present in the
`mach build` process on MacOS:
1. `mach build` process was not copying transitive dependencies
of servo binary but only the first level dylibs
2. `mach build` process didn't patch the links to dylibs
in servo binary (and dependencies). This meant though
(some) dylibs were copied to local path, the binary
still loaded the dylibs from system GStreamer installation
i.e homebrew instead of the copieds dylibs
The build and runtime dependencies in etc/homebrew/Brewfile
and etc/homebrew/Brewfile-build have also been removed in This
PR.
Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
Reorganize Servo's WPT Python scripts
This change moves all of Servo's WPT Python support scripts into one directory as they were previously scattered throughout the directory structure. This should allow more code reuse and make it easier to understand how everything fits together.
The changes:
- `tests/wpt/update` → `python/wpt/importer`
- `etc/ci/upstream-wpt-changes/wptupstreamer` → `python/wpt/exporter`
- `etc/ci/upstream-wpt-changes/test.py` → `python/wpt/test.py`
- `etc/ci/upstream-wpt-changes/tests` → `python/wpt/tests`
- `tests/wpt/servowpt.py` → - `python/wpt/update.py` - `python/wpt/run.py`
- `tests/wpt/manifestupdate.py` → `python/wpt/manifestupdate.py`
This change also removes
- The ability to run the `update-wpt` and `test-wpt` commands without using `mach`. These didn't work very well, because it was difficult to get all of the wptrunner and mach dependencies installed outside of the Python virtualenv. It's simpler if they are always run through `mach`.
- The old WPT change sync / upstreaming script that was no longer used. This allows this code to pass `test-tidy` now.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] There are tests for these changes
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
This change moves all of Servo's WPT Python support scripts into one
directory as they were previously scattered throughout the directory
structure. This should allow more code reuse and make it easier to
understand how everything fits together.
The changes:
- `tests/wpt/update` → `python/wpt/importer`
- `etc/ci/upstream-wpt-changes/wptupstreamer` → `python/wpt/exporter`
- `etc/ci/upstream-wpt-changes/test.py` → `python/wpt/test.py`
- `etc/ci/upstream-wpt-changes/tests` → `python/wpt/tests`
- `tests/wpt/servowpt.py` →
- `python/wpt/update.py`
- `python/wpt/run.py`
- `tests/wpt/manifestupdate.py` → `python/wpt/manifestupdate.py`
This change also removes
- The ability to run the `update-wpt` and `test-wpt` commands without
using `mach`. These didn't work very well, because it was difficult
to get all of the wptrunner and mach dependencies installed outside
of the Python virtualenv. It's simpler if they are always run through
`mach`.
- The old WPT change upstreaming script that was no longer used.
Use ccache on windows workflow
Because `sccache` is not working when setting `CCACHE=sccache` I used ccache which is taking cache hits.
I would certainly love to share some timings, but I found out that windows builds have very inconsistent timings.
Use GStreamer in Windows workflow
I accidental came across https://github.com/servo/servo/projects/26 and can't helped but to fix one. (Few were already fixed but not closed).
Test run: 8306052042
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#28277
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because it is only CI
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
