`mach bootstrap` will finish succesfully,
but then segfault inside glibc while exiting.
It seems to have to do with the MarkupSafe library used by Mako;
delaying the Mako import from the top level to inside functions where it
is used avoids the problem.
(It seems to interact with the call to pip to install the Salt
requirements in the virtualenv, as commenting that pip call out
separately will also fix the segfault.)
Also, use the Mako installed in the Python virtualenv while running
packaging commands (instead of the one from the bundled zip file),
and cleanup imports in the package_commands.py file.
Add env var for local saltfs root during bootstrap
<!-- Please describe your changes on the following line: -->
This enables using the environment variable `SERVO_SALTFS_ROOT` to use a local copy of the saltfs tree for `./mach bootstrap`. The immediate use case is to add a test for `./mach bootstrap` to saltfs itself, to ensure Salt changes or refactors don't break `./mach bootstrap`.
I tested this locally in an Ubuntu Trusty VM.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [ ] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #__ (github issue number if applicable).
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because the test will be added to saltfs
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/16370)
<!-- Reviewable:end -->
Add `--dry-run` argument to `mach cargo-update`
This add ability to check which packages are outdated, also detect which updates are minor and which may contain breaking changes.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/15857)
<!-- Reviewable:end -->
`incremental = true` can be set in the `[build]` section of `.servobuild`
to enable incremental compilation.
In `./mach build-geckolib`, which uses a stable version of the compiler,
this would cause a warning repeated for each invocation of rustc:
```
warning: the option `Z` is unstable and should only be used on the nightly
compiler, but it is currently accepted for backwards compatibility;
this will soon change, see issue #31847 for more details
```
This PR disables incremental compilation for `build-geckolib`,
regardless of `.servobuild` settings.
Mach bootstrap: Improve and support more platforms
<!-- Please describe your changes on the following line: -->
r? @aneeshusa or @Wafflespeanut
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #__ (github issue number if applicable).
<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because _____
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/15230)
<!-- Reviewable:end -->
Rewrite the ban-type lint in Python
<!-- Please describe your changes on the following line: -->
Rewrite the ban-type lint in Python.
Question: Should the old ban-type lint written in rust be deleted?
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [x ] `./mach build -d` does not report any errors
- [x ] `./mach test-tidy` does not report any errors
- [x ] These changes fix#15591
<!-- Either: -->
- [ x] There are tests for these changes
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/15649)
<!-- Reviewable:end -->
Add 'use statements with extraneous spaces' tidy check
I added simple check routine for 'use statements with extraneous
spaces' and codes that breaks the check routine in rust_tidy.rs.
* Added a code that using 'use statements with extraneous spaces' code
in rust_tidy.rs
* Added assertion code in test_tidy.py.
* check_rust function in tidy.py now recognizes the simple case in
the 'use statements with extraneous spaces'.
* Ran tidy check on rust code and modified a
code(tests/unit/style/parsing/inherited_text.rs) that is not passing
on this new tidy check.
TODO: this code has to be refactored to support more general cases.
- [X] ./mach test-tidy does not report any errors
- [X] These changes fix#14898 (github issue number if applicable).
fixed an issue related with .DS_Store
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix#15581.
<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because _____
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/15665)
<!-- Reviewable:end -->
Sometimes clippy gets outdated by months, and its current support setup
means that each Servo component need to opt into it by depending on
the plugins crate manually, and not all components do that.
The Servo repository is now (mostly) vendored in the Firefox Mercurial
repository. For size and duplication reasons, the tests/wpt directory is
not included in the vendored copy.
This causes problems when running `mach` from the Firefox repository
because `mach` references pip requirements files and module search
paths from WPT.
This commit adds code to detect when Servo's mach is running from a
Firefox source tree and to resolve WPT paths to the Firefox location
if appropriate. This enables `mach` to "just work" when running
from the servo/ directory in the Firefox repository.
The file looked for to identify the Firefox repository is identical
to what Firefox's `mach` script uses.
A potential issue with using Firefox's WPT files is that they may be
different from those in the Servo repository and this could lead to
differences in behavior - possibly even an error when loading/running
`mach`. However, the behavior before this commit was that Servo's
`mach` never worked in the Firefox repository (due to missing WPT
files). And post-commit it does. So this seems like a "perfect is the
enemy of good" scenario.
Allow running servo in software rendering mode from ./mach run
<!-- Please describe your changes on the following line: -->
This adds a flag to ./mach run to use software rendering on Linux (through the LIBGL_ALWAYS_SOFTWARE env var).
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix#14982.
<!-- Either: -->
- [X] These changes do not require tests because like said in the issue "There's no way to add automated tests for this".
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/15138)
<!-- Reviewable:end -->
Use Salt for mach bootstrap
<!-- Please describe your changes on the following line: -->
This is currently WIP, but allows Salt for `mach bootstrap`. Not looking for review yet, just posting for visibility. You can run `./mach bootstrap` and Salt will run, letting you know what changes it would make for bootstrapping (doesn't actually run yet though).
Currently, this reads from saltfs using gitfs, meaning it always tracks the master branch. (Note that this is blocked on https://github.com/servo/saltfs/pull/577 landing; in the meantime, I've included a small workaround in this PR to pull from my updated saltfs branch, which will need to be removed.) In the future, the relevant Salt code may be merged in tree as part of Docker support for TC, and the bootstrapper should be updated to read from in tree.
Also, our Windows machines have not been Salted, so the existing Windows bootstrappers are retained.
TODO:
- [x] Hook into existing bootstrapping code less hackily
- [x] Actually bootstrap instead of always using `test=True` mode (includes sudo)
- [x] Default to interactive mode (test first, then ask), with a force flag
- [x] Don't require running from the repository root directory
- [x] Make it easy to add support for other distros
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [ ] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #__ (github issue number if applicable).
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because they should be verified manually
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/14974)
<!-- Reviewable:end -->
The Salt bootstrapper invokes Salt during `./mach bootstrap`
to install Servo's build dependencies.
It uses salt-call pinned to the same version of Salt as used in saltfs.
Currently, the implementation uses gitfs and reads directly from
the master branch of the saltfs repo;
in the future this should be changed when the relevant Salt states
are moved in-tree as part of Dockerization for TaskCluster.
We have not Salted our Windows machines, so the existing Windows
bootstrappers are retained. Currently this is only tested on
Ubuntu Trusty.
Salt uses various system python libraries,
including `python-apt` on Debian-based OSes to interact with apt.
`python-apt` does not seem to be installable via a requirements.txt
file, and the versions available on PyPI are far behind the versions
installed on actual Ubuntu machines.
Additionally, adding `python-apt` as an unconditional python dependency
would add bloat for users of other OSes, and lead to more churn
as additional OSes are supported.
However, as `python-apt` is already installed via apt on these machines,
we can allow Salt to instead use the module by using
`--system-site-packages` for the python virtualenv.
We also add the `-I` flag to `pip install` to ensure we have a local,
untouched copy of any other python packages used.
However, because this prints system-level Python packages in scope,
it slightly breaks isolation, so it is important to always pin
all dependencies in the requirements files.
- Default to interactive mode and remove the `--interactive` flag
- Use `--force` to skip interactivity
- Change MSVC dependency storage organization on disk: put each version
into its own folder and directly refer to the versioned folders,
providing immutability and making the installation list redundant
- Reuse `host_triple()` function to fix broken bootstrapper dispatching
- Simplify code:
- Remove or inline many unused and redudant functions and variables
- Prefer plain functions to classes
- Consolidate into fewer files, remove unnecessary bootstrapper/ dir
- Improve Python style
- Sort dependency list
