The sys.platform check in need_pip_upgrade was doing effectively the same pip invocation regardless of what
platform we were evaluating to.
Additionally, removed some duplicate definitions of the python variable which was already in scope
higher up in the function.
Gecko is now using the wpt harness under the wpt tools directory
rather than a copy in an adjacent directory. Therefore the path to the
requirements files, and the required model paths, have changed.
The Servo repository is now (mostly) vendored in the Firefox Mercurial
repository. For size and duplication reasons, the tests/wpt directory is
not included in the vendored copy.
This causes problems when running `mach` from the Firefox repository
because `mach` references pip requirements files and module search
paths from WPT.
This commit adds code to detect when Servo's mach is running from a
Firefox source tree and to resolve WPT paths to the Firefox location
if appropriate. This enables `mach` to "just work" when running
from the servo/ directory in the Firefox repository.
The file looked for to identify the Firefox repository is identical
to what Firefox's `mach` script uses.
A potential issue with using Firefox's WPT files is that they may be
different from those in the Servo repository and this could lead to
differences in behavior - possibly even an error when loading/running
`mach`. However, the behavior before this commit was that Servo's
`mach` never worked in the Firefox repository (due to missing WPT
files). And post-commit it does. So this seems like a "perfect is the
enemy of good" scenario.
The Salt bootstrapper invokes Salt during `./mach bootstrap`
to install Servo's build dependencies.
It uses salt-call pinned to the same version of Salt as used in saltfs.
Currently, the implementation uses gitfs and reads directly from
the master branch of the saltfs repo;
in the future this should be changed when the relevant Salt states
are moved in-tree as part of Dockerization for TaskCluster.
We have not Salted our Windows machines, so the existing Windows
bootstrappers are retained. Currently this is only tested on
Ubuntu Trusty.
Salt uses various system python libraries,
including `python-apt` on Debian-based OSes to interact with apt.
`python-apt` does not seem to be installable via a requirements.txt
file, and the versions available on PyPI are far behind the versions
installed on actual Ubuntu machines.
Additionally, adding `python-apt` as an unconditional python dependency
would add bloat for users of other OSes, and lead to more churn
as additional OSes are supported.
However, as `python-apt` is already installed via apt on these machines,
we can allow Salt to instead use the module by using
`--system-site-packages` for the python virtualenv.
We also add the `-I` flag to `pip install` to ensure we have a local,
untouched copy of any other python packages used.
However, because this prints system-level Python packages in scope,
it slightly breaks isolation, so it is important to always pin
all dependencies in the requirements files.
In addition to minor changes for Windows, this forces Windows Python to
be used for all Windows builds (instead of using Windows Python only for
pc-windows-msvc builds).
* Split package commands into their own file
* Delete spurious files from build dir
* Create runservo.sh to invoke servo with the right browserhtml incantation
* Tar it all up with the date and time in the filename
When I originally rewrote Servo's mach bootstrapping (using virtualenv
w/ requirements.txt in #7103), I didn't specify mach as a requirement
because a new version hadn't been published in a while. Now that 0.6
is out, I asked the mach maintainers to publish a new version on PyPI,
so now we can fetch it like the other Python dependencies.
Fixes https://github.com/servo/servo/issues/10728.
There are two changes:
* remove quoting which causes virtuaenv not activate
* check virtualenv actually activated
If the quoting added in the fix in #8394 (4ff8d3ad9c) kicks in, it
causes virtualenv to fail to activate.
For the common case it is no op:
```python
>>> from pipes import quote
>>> print quote('common/case')
common/case
```
When the path actually needs quoting, this is what happens:
```python
>>> print quote('test spaces')
'test spaces'
>>> print quote('windows\\path')
'windows\\path'
```
Note the embedded quotes.
Virtualenv in activate_this.py uses __file__ to build the path that
should be added to PATH:
```python
>>> print os.getcwd()
C:\software\git
>>> print os.path.abspath(quote('windows\\path'))
C:\software\git\'windows\path'
>>>
```
The constructed path is not valid. Adding it at the beginning of PATH
has no effect. This issue affects any case when the call to `quote`
kicks in.
When mach_bootstrap got interrupted while it's setting up virtualenv or
calling out to pip, it wouldn't repeat that step on subsequent runs, and
mach fails because its environment isn't set up properly or dependencies
are missing.
So now we re-run virtualenv if activate_this.py doesn't exist, and only
create the marker file for required packages after pip has returned
successfully.
Prior to this commit:
* Our Python dependency story was a bit of a mess. We had complete
Python packages (wheels and directories) living in-tree, despite
not having any changes from upstream. This is particularly bad because
`setup.py` never gets run on these packages which could (sometimes
silently) unintended breakage.
* Python virtual environments (virtualenv) were only utilized for
testing web-platform tests
After this commit:
* A single virtualenv (`python/_virtualenv`) is activated upon *every*
call to mach
* A requirements file (`python/requirements.txt`) is added to describe
the dependencies needed by Python modules in `python/`. The child
commit immediately following this will remove all the dependencies
no longer needed in-tree (for the sake of keeping this commit
readable).
Relevant to https://github.com/servo/servo/issues/861
Fixes https://github.com/servo/servo/issues/6999
