The Salt bootstrapper invokes Salt during `./mach bootstrap`
to install Servo's build dependencies.
It uses salt-call pinned to the same version of Salt as used in saltfs.
Currently, the implementation uses gitfs and reads directly from
the master branch of the saltfs repo;
in the future this should be changed when the relevant Salt states
are moved in-tree as part of Dockerization for TaskCluster.
We have not Salted our Windows machines, so the existing Windows
bootstrappers are retained. Currently this is only tested on
Ubuntu Trusty.
Salt uses various system python libraries,
including `python-apt` on Debian-based OSes to interact with apt.
`python-apt` does not seem to be installable via a requirements.txt
file, and the versions available on PyPI are far behind the versions
installed on actual Ubuntu machines.
Additionally, adding `python-apt` as an unconditional python dependency
would add bloat for users of other OSes, and lead to more churn
as additional OSes are supported.
However, as `python-apt` is already installed via apt on these machines,
we can allow Salt to instead use the module by using
`--system-site-packages` for the python virtualenv.
We also add the `-I` flag to `pip install` to ensure we have a local,
untouched copy of any other python packages used.
However, because this prints system-level Python packages in scope,
it slightly breaks isolation, so it is important to always pin
all dependencies in the requirements files.
- Default to interactive mode and remove the `--interactive` flag
- Use `--force` to skip interactivity
- Change MSVC dependency storage organization on disk: put each version
into its own folder and directly refer to the versioned folders,
providing immutability and making the installation list redundant
- Reuse `host_triple()` function to fix broken bootstrapper dispatching
- Simplify code:
- Remove or inline many unused and redudant functions and variables
- Prefer plain functions to classes
- Consolidate into fewer files, remove unnecessary bootstrapper/ dir
- Improve Python style
- Sort dependency list
Allow passing --nocapture argument to test-unit mach command
This adds the ` --nocapture` argument to the test-unit mach command.
Defaulting to false, when given it passes `-- --nocapture` to cargo test to show stdout during test-unit runs.
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#14595
<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because _____
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/14706)
<!-- Reviewable:end -->
Use MSVC toolchain if PLATFORM is defined
<!-- Please describe your changes on the following line: -->
`VSInstallDir` exists only when Visual Studio is installed.
`VS140COMNTOOLS` is defined by installing either Visual Studio or [Visual C++ Build Tools](http://landinghub.visualstudio.com/visual-cpp-build-tools)
It will allow to build MSVC based Servo without having to install whole Visual Studio (few GiB).
EDIT: Another (maybe cleaner) solution would be reverting 63ec96a57d
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [ ] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #__ (github issue number if applicable).
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because `VS140COMNTOOLS` variable is already used in [mach.bat](https://github.com/servo/servo/blob/master/mach.bat)
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/14346)
<!-- Reviewable:end -->
Tidy: Check Cargo.lock for packages with same version and different sources
<!-- Please describe your changes on the following line: -->
r? @Wafflespeanut
cc @SimonSapin
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix#14695
<!-- Either: -->
- [X] There are tests for these changes
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/14715)
<!-- Reviewable:end -->
Raise warning when OSMesa path is not set
In Ubuntu 14.04, I still get crashes due to non-existence of OsMesa library (#13515). It turned out that sometimes I have paths like `osmesa-src-<hash>` without `out/lib/gallium`. Now, we raise a warning whenever we don't find a proper path.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
<!-- Either: -->
- [x] These changes do not require tests because it's related to mach.
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/14601)
<!-- Reviewable:end -->
Define NATIVE_WIN32_PYTHON environment variable
r? @Wafflespeanut or @larsbergstrom
---
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/13650)
<!-- Reviewable:end -->
Various Windows installer fixes and improvements
r? @metajack or @larsbergstrom
---
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/14531)
<!-- Reviewable:end -->
Majority of build_gecko.rs is just the straightforward conversion from
regen.py. There are two differences that:
1. Side in whitelist is changed to mozilla::Side
2. std::atomic__My_base is added to opaque types for Windows
Part of #13551 - Create packages under release/debug directories as appropriate
r? @aneeshusa
This change implements fixes for #13551 for Linux and MacOS targets.
`/python/servo/package_commands.py` was modified so that:
- On MacOS, it creates all intermediate packaging directories like `dmg`, `brew`, and `brew-tmp` under `target/(release|debug)`, rather than in `target` directly.
- On MacOS, all packaging artifacts (`.dmg`, brew `.tar.gz`) are packaged under `target/(release|debug)`, rather than in `target` directly.
- On Linux, the resulting `tar.gz` Servo package is placed under `target/(release|debug)`, rather than in `target`.
- Also did some extra cleanup around path parsing in the MacOS packaging code, to use `os.path` methods rather than straight `'/'` parsing with `split` and `join` where it was applicable.
`/etc/ci/upload_nightly.sh` was modified to:
- Look for artifacts in `target/release` for `mac`, `macbrew`, and `linux` platforms, rather than just `target/`.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#13551
- [x] These changes do not require tests because it is based on CI and packaging tools. They were manually tested for correctness.
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
…lease/debug as specified or detected. Modify macos packaging to create all packages under release/debug directory. Updated etc/ci/upload_nightly.sh to support uploading from either release/debug directory, depending on what was built
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/13845)
<!-- Reviewable:end -->
This commit updates the 'mach package' command to generate output
under the target/release and target/debug directories when run on
macOS and Linux.
etc/ci/upload_nightly.sh has also been updated to upload packages from
the release/debug directories.
Adding linting checks for buildbot_steps.yml
This pull request adds some tidy checks around YAML files, and specifically `buildbot_steps.yml`.
Tidy checks added:
* YAML files are checked for well-formedness/parse-ability
* Whether a YAML file has duplicate keys
* Whether a `buildbot_steps.yml` file contains only mappings to list-of-strings.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [x] These changes fix#13838 (github issue number if applicable).
<!-- Either: -->
- [X] There are tests for these changes OR
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
…ing checking for correct mappings and duplicate YAML keys. Added unit tests to test_tidy.py.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/14051)
<!-- Reviewable:end -->
This commit adds tidy checks for buildbot_steps.yml, as well as unit
tests. These checks include:
* Checking buildbot_steps.yml can be parsed by a YAML loader
* buildbot_steps.yml does not contain duplicate keys
* buildbot_steps.yml keys map to a list of strings
Instead of once per day (or night) with a date in the URL,
Cargo binaries are now built and published for every merge to master
(with the merge commit hash in the URL).
Commit that fixes the issue #11074 by upgrading pip whenever virtuale…
<!-- Please describe your changes on the following line: -->
I have kind of resolved the issue #11074 by adding bool variable which is set to `True` if we had created the virtualenv and `False` otherwise.
Then it updates pip by executing `pip install --upgrade pip` in the same way as packages are updated. I am a little bit worried that I have almost duplicated the installation routine from the `for` loop but I am not sure whether I should add a function or not.
I think it is the best way of doing this because it does not need any Internet access for regular work (only for the first time you execute mach) as @larsbergstrom worried [here](https://github.com/servo/servo/pull/11149). It also doesn't add any extra latency on a no-op build.
I have checked the solution inside a docker container based on debian wheezy. Before the patch `./mach` failed to run because it wasn't able to install some packages. Now it runs successfully.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix#11074 (github issue number if applicable).
<!-- Either: -->
- [ ] There are tests for these changes OR
- [X] These changes do not require tests because it changes only mach_bootstrap.py
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/14452)
<!-- Reviewable:end -->
Implement tidy commit message test
<!-- Please describe your changes on the following line: -->
Makes tidy check commit messages since the latest merge, failing if one of them contains the string "WIP".
I have written a test for my changes, although it is a bit hacky.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#14388.
<!-- Either: -->
- [x] These changes do not require tests because this would require playing with a new/existing git repo
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/14450)
<!-- Reviewable:end -->
