The Salt bootstrapper invokes Salt during `./mach bootstrap`
to install Servo's build dependencies.
It uses salt-call pinned to the same version of Salt as used in saltfs.
Currently, the implementation uses gitfs and reads directly from
the master branch of the saltfs repo;
in the future this should be changed when the relevant Salt states
are moved in-tree as part of Dockerization for TaskCluster.
We have not Salted our Windows machines, so the existing Windows
bootstrappers are retained. Currently this is only tested on
Ubuntu Trusty.
Salt uses various system python libraries,
including `python-apt` on Debian-based OSes to interact with apt.
`python-apt` does not seem to be installable via a requirements.txt
file, and the versions available on PyPI are far behind the versions
installed on actual Ubuntu machines.
Additionally, adding `python-apt` as an unconditional python dependency
would add bloat for users of other OSes, and lead to more churn
as additional OSes are supported.
However, as `python-apt` is already installed via apt on these machines,
we can allow Salt to instead use the module by using
`--system-site-packages` for the python virtualenv.
We also add the `-I` flag to `pip install` to ensure we have a local,
untouched copy of any other python packages used.
However, because this prints system-level Python packages in scope,
it slightly breaks isolation, so it is important to always pin
all dependencies in the requirements files.
In addition to minor changes for Windows, this forces Windows Python to
be used for all Windows builds (instead of using Windows Python only for
pc-windows-msvc builds).
* Split package commands into their own file
* Delete spurious files from build dir
* Create runservo.sh to invoke servo with the right browserhtml incantation
* Tar it all up with the date and time in the filename
When I originally rewrote Servo's mach bootstrapping (using virtualenv
w/ requirements.txt in #7103), I didn't specify mach as a requirement
because a new version hadn't been published in a while. Now that 0.6
is out, I asked the mach maintainers to publish a new version on PyPI,
so now we can fetch it like the other Python dependencies.
Fixes https://github.com/servo/servo/issues/10728.
There are two changes:
* remove quoting which causes virtuaenv not activate
* check virtualenv actually activated
If the quoting added in the fix in #8394 (4ff8d3ad9c) kicks in, it
causes virtualenv to fail to activate.
For the common case it is no op:
```python
>>> from pipes import quote
>>> print quote('common/case')
common/case
```
When the path actually needs quoting, this is what happens:
```python
>>> print quote('test spaces')
'test spaces'
>>> print quote('windows\\path')
'windows\\path'
```
Note the embedded quotes.
Virtualenv in activate_this.py uses __file__ to build the path that
should be added to PATH:
```python
>>> print os.getcwd()
C:\software\git
>>> print os.path.abspath(quote('windows\\path'))
C:\software\git\'windows\path'
>>>
```
The constructed path is not valid. Adding it at the beginning of PATH
has no effect. This issue affects any case when the call to `quote`
kicks in.
When mach_bootstrap got interrupted while it's setting up virtualenv or
calling out to pip, it wouldn't repeat that step on subsequent runs, and
mach fails because its environment isn't set up properly or dependencies
are missing.
So now we re-run virtualenv if activate_this.py doesn't exist, and only
create the marker file for required packages after pip has returned
successfully.
Prior to this commit:
* Our Python dependency story was a bit of a mess. We had complete
Python packages (wheels and directories) living in-tree, despite
not having any changes from upstream. This is particularly bad because
`setup.py` never gets run on these packages which could (sometimes
silently) unintended breakage.
* Python virtual environments (virtualenv) were only utilized for
testing web-platform tests
After this commit:
* A single virtualenv (`python/_virtualenv`) is activated upon *every*
call to mach
* A requirements file (`python/requirements.txt`) is added to describe
the dependencies needed by Python modules in `python/`. The child
commit immediately following this will remove all the dependencies
no longer needed in-tree (for the sake of keeping this commit
readable).
Relevant to https://github.com/servo/servo/issues/861
Fixes https://github.com/servo/servo/issues/6999
