mirror of
https://github.com/servo/servo.git
synced 2025-06-06 16:45:39 +00:00
Implement bootstrap-gstreamer for all platforms
This change makes it so that the Platform classes can now handle installing GStreamer dependencies and properly setting up the environment including when cross-compiling. For Windows and Linux is now installed into `target/dependencies/gstreamer` when not installed system-wide. In addition: 1. Creating and moving existing environment path append helpers to `util.py`. 2. Combining the `set_run_env` and `build_dev` functions and moving some outside code into them so that it can be shared. Now code that used to call `set_run_env` calls `build_dev` and then `os.environ.update(...)`. 3. Adding Python typing information in many places. Signed-off-by: Martin Robinson <mrobinson@igalia.com>
This commit is contained in:
Martin Robinson
parent
a56abe44e0
commit
7d20f16d9f
17 changed files with 437 additions and 291 deletions
2
.github/workflows/mac.yml
vendored
2
.github/workflows/mac.yml
vendored
|
|
@ -66,7 +66,7 @@ jobs:
|
|||
- name: Bootstrap
|
||||
run: |
|
||||
python3 -m pip install --upgrade pip virtualenv
|
||||
bash etc/install_macos_gstreamer.sh
|
||||
python3 ./mach bootstrap-gstreamer
|
||||
brew install gnu-tar
|
||||
- name: Release build
|
||||
run: |
|
||||
|
|
|
|||
8
.github/workflows/windows.yml
vendored
8
.github/workflows/windows.yml
vendored
|
|
@ -52,13 +52,6 @@ jobs:
|
|||
fetch-depth: 2
|
||||
- name: Copy to C drive
|
||||
run: cp D:\a C:\ -Recurse
|
||||
- name: Install GStreamer
|
||||
shell: powershell
|
||||
run: |
|
||||
Start-BitsTransfer -Source https://gstreamer.freedesktop.org/data/pkg/windows/1.16.0/gstreamer-1.0-msvc-x86_64-1.16.0.msi
|
||||
Start-Process msiexec.exe -Wait -ArgumentList "/i gstreamer-1.0-msvc-x86_64-1.16.0.msi /quiet /qn /norestart ADDLOCAL=ALL"
|
||||
Start-BitsTransfer -Source https://gstreamer.freedesktop.org/data/pkg/windows/1.16.0/gstreamer-1.0-devel-msvc-x86_64-1.16.0.msi
|
||||
Start-Process msiexec.exe -Wait -ArgumentList "/i gstreamer-1.0-devel-msvc-x86_64-1.16.0.msi /quiet /qn /norestart ADDLOCAL=ALL"
|
||||
- name: wix311-binaries
|
||||
shell: powershell
|
||||
run: |
|
||||
|
|
@ -72,6 +65,7 @@ jobs:
|
|||
run: |
|
||||
python -m pip install --upgrade pip virtualenv
|
||||
python mach fetch
|
||||
python mach bootstrap-gstreamer
|
||||
- name: Release build
|
||||
working-directory: "C:\\a\\${{ github.event.repository.name }}\\${{ github.event.repository.name }}"
|
||||
run: python mach build --release --with-${{ env.LAYOUT }}
|
||||
|
|
|
|||
74
README.md
74
README.md
|
|
@ -50,18 +50,10 @@ Please select your operating system:
|
|||
|
||||
#### macOS
|
||||
|
||||
Xcode version 10.2 or above is recommended.
|
||||
|
||||
##### On macOS(Intel based or ARM based) (Homebrew)
|
||||
|
||||
NOTE: run these steps after you've cloned the project locally.
|
||||
|
||||
``` sh
|
||||
cd servo
|
||||
bash etc/install_macos_gstreamer.sh
|
||||
brew bundle install --file=etc/homebrew/Brewfile
|
||||
pip install virtualenv
|
||||
```
|
||||
- Install Xcode (version 10.2 or above is recommended)
|
||||
- Run `./mach bootstrap-gstreamer`. This will install the recommended version of GStreamer globally on your system.
|
||||
- Run `brew bundle install --file=etc/homebrew/Brewfile`
|
||||
- Run `pip install virtualenv`
|
||||
|
||||
#### On Debian-based distros
|
||||
|
||||
|
|
@ -82,10 +74,15 @@ sudo apt install git curl autoconf libx11-dev libfreetype6-dev libgl1-mesa-dri \
|
|||
libgstreamer-plugins-bad1.0-dev autoconf2.13 llvm-dev
|
||||
```
|
||||
|
||||
Additionally, you'll need a local copy of GStreamer with a version later than 16.2. You can place it in `support/linux/gstreamer/gst`, or run `./mach bootstrap-gstreamer` to set it up. On **Ubuntu 20.04LTS**, you can use the system GStreamer if you install the necessary packages:
|
||||
``` sh
|
||||
sudo apt install gstreamer1.0-nice gstreamer1.0-plugins-bad
|
||||
```
|
||||
You will also need GStreamer which you can install in two ways:
|
||||
|
||||
1. On **Ubuntu 20.04 LTS or greater**: The system version of GStreamer is sufficient if you install the necessary plugins:
|
||||
|
||||
```
|
||||
sudo apt install gstreamer1.0-nice gstreamer1.0-plugins-bad
|
||||
```
|
||||
|
||||
2. Run `./mach bootstrap-gstreamer` to install the prepackaged binaries into the `target` directory.
|
||||
|
||||
If you are using **Ubuntu 16.04** or **Linux Mint 18**, run `export HARFBUZZ_SYS_NO_PKG_CONFIG=1` before building to avoid an error with harfbuzz.
|
||||
|
||||
|
|
@ -188,39 +185,40 @@ You will need to run this in every shell before running mach.
|
|||
|
||||
1. Install Python 3.9 for Windows (https://www.python.org/downloads/release/python-392/). The Windows x86-64 MSI installer is fine. This is required in order to build the JavaScript engine, SpiderMonkey.
|
||||
|
||||
You will also need to set the `PYTHON3` environment variable, e.g., to 'C:\Python39\python.exe' by doing:
|
||||
```
|
||||
setx PYTHON3 "C:\Python39\python.exe" /m
|
||||
```
|
||||
The `/m` will set it system-wide for all future command windows.
|
||||
You will also need to set the `PYTHON3` environment variable, e.g., to 'C:\Python39\python.exe' by doing:
|
||||
```
|
||||
setx PYTHON3 "C:\Python39\python.exe" /m
|
||||
```
|
||||
The `/m` will set it system-wide for all future command windows.
|
||||
|
||||
2. Install virtualenv.
|
||||
2. Install virtualenv. In a normal Windows Shell (cmd), do:
|
||||
```
|
||||
pip install virtualenv
|
||||
```
|
||||
If this does not work, you may need to reboot for the changed PATH settings (by the python installer) to take effect.
|
||||
|
||||
In a normal Windows Shell (cmd), do:
|
||||
```
|
||||
pip install virtualenv
|
||||
```
|
||||
If this does not work, you may need to reboot for the changed PATH settings (by the python installer) to take effect.
|
||||
3. Install GStreamer:
|
||||
```
|
||||
mach bootstrap-gstreamer
|
||||
```
|
||||
You can also install GStreamer manually using the MSVC (**not MingGW**) binaries from the [GStreamer](https://gstreamer.freedesktop.org/data/pkg/windows/) site. The currently recommended version is 1.16.0. i.e:
|
||||
|
||||
3. Install the most recent [GStreamer](https://gstreamer.freedesktop.org/data/pkg/windows/) MSVC packages. You need to download the two `.msi` files for your platform from the [GStreamer](https://gstreamer.freedesktop.org/data/pkg/windows/) website and install them. The currently recommended version is 1.16.0. i.e.:
|
||||
- [gstreamer-1.0-msvc-x86_64-1.16.0.msi](https://gstreamer.freedesktop.org/data/pkg/windows/1.16.0/gstreamer-1.0-msvc-x86_64-1.16.0.msi)
|
||||
- [gstreamer-1.0-devel-msvc-x86_64-1.16.0.msi](https://gstreamer.freedesktop.org/data/pkg/windows/1.16.0/gstreamer-1.0-devel-msvc-x86_64-1.16.0.msi)
|
||||
|
||||
- [gstreamer-1.0-msvc-x86_64-1.16.0.msi](https://gstreamer.freedesktop.org/data/pkg/windows/1.16.0/gstreamer-1.0-msvc-x86_64-1.16.0.msi)
|
||||
- [gstreamer-1.0-devel-msvc-x86_64-1.16.0.msi](https://gstreamer.freedesktop.org/data/pkg/windows/1.16.0/gstreamer-1.0-devel-msvc-x86_64-1.16.0.msi)
|
||||
|
||||
Note that the MinGW binaries will not work, so make sure that you install the MSVC ones.
|
||||
|
||||
Note that you should ensure that _all_ components are installed from gstreamer, as we require many of the optional libraries that are not installed by default.
|
||||
Note that you should ensure that _all_ components are installed from gstreamer, as we require many of the optional libraries that are not installed by default.
|
||||
|
||||
4. Install Git for Windows (https://git-scm.com/download/win). DO allow it to add git.exe to the PATH (default
|
||||
settings for the installer are fine).
|
||||
|
||||
5. Install Visual Studio Build Tools 2019 (https://visualstudio.microsoft.com/de/downloads/#build-tools-for-visual-studio-2019). It is easiest to install via [Chocolatey](https://chocolatey.org/install#installing-chocolatey) with:
|
||||
```
|
||||
choco install -y visualstudio2019buildtools --package-parameters="--add Microsoft.VisualStudio.Component.Roslyn.Compiler --add Microsoft.Component.MSBuild --add Microsoft.VisualStudio.Component.CoreBuildTools --add Microsoft.VisualStudio.Workload.MSBuildTools --add Microsoft.VisualStudio.Component.Windows10SDK --add Microsoft.VisualStudio.Component.VC.CoreBuildTools --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 --add Microsoft.VisualStudio.Component.VC.Redist.14.Latest --add Microsoft.VisualStudio.Component.VC.ATL --add Microsoft.VisualStudio.Component.VC.ATLMFC --add Microsoft.VisualStudio.Component.TextTemplating --add Microsoft.VisualStudio.Component.VC.CoreIde --add Microsoft.VisualStudio.ComponentGroup.NativeDesktop.Core --add Microsoft.VisualStudio.Workload.VCTools"
|
||||
```
|
||||
If you really need to use the Visual Studio Installer (UI), choose "Desktop development with C++" and add the optional "MSVC", "C++-ATL" and "C++-MFC" (latest).
|
||||
```
|
||||
choco install -y visualstudio2019buildtools --package-parameters="--add Microsoft.VisualStudio.Component.Roslyn.Compiler --add Microsoft.Component.MSBuild --add Microsoft.VisualStudio.Component.CoreBuildTools --add Microsoft.VisualStudio.Workload.MSBuildTools --add Microsoft.VisualStudio.Component.Windows10SDK --add Microsoft.VisualStudio.Component.VC.CoreBuildTools --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 --add Microsoft.VisualStudio.Component.VC.Redist.14.Latest --add Microsoft.VisualStudio.Component.VC.ATL --add Microsoft.VisualStudio.Component.VC.ATLMFC --add Microsoft.VisualStudio.Component.TextTemplating --add Microsoft.VisualStudio.Component.VC.CoreIde --add Microsoft.VisualStudio.ComponentGroup.NativeDesktop.Core --add Microsoft.VisualStudio.Workload.VCTools"
|
||||
```
|
||||
If you really need to use the Visual Studio Installer (UI), choose "Desktop development with C++" and add the optional "MSVC", "C++-ATL" and "C++-MFC" (latest).
|
||||
|
||||
The Visual Studio 2019 Build Tools MUST be installed to the default location or mach.bat will not find them.
|
||||
The Visual Studio 2019 Build Tools MUST be installed to the default location or mach.bat will not find them.
|
||||
|
||||
##### [Optional] Install LLVM for faster link times
|
||||
|
||||
|
|
|
|||
|
|
@ -24,6 +24,8 @@ import zipfile
|
|||
from time import time
|
||||
|
||||
import notifypy
|
||||
import servo.platform
|
||||
import servo.util
|
||||
|
||||
from mach.decorators import (
|
||||
CommandArgument,
|
||||
|
|
@ -33,9 +35,8 @@ from mach.decorators import (
|
|||
from mach.registrar import Registrar
|
||||
|
||||
from mach_bootstrap import _get_exec_path
|
||||
from servo.command_base import CommandBase, cd, call, check_call, append_to_path_env, gstreamer_root
|
||||
from servo.command_base import CommandBase, cd, call, check_call
|
||||
from servo.gstreamer import windows_dlls, windows_plugins, macos_plugins
|
||||
from servo.platform import host_triple
|
||||
|
||||
|
||||
@CommandProvider
|
||||
|
|
@ -92,7 +93,7 @@ class MachCommands(CommandBase):
|
|||
features += media_stack
|
||||
has_media_stack = media_stack[0] == "media-gstreamer"
|
||||
|
||||
target_path = base_path = self.get_target_dir()
|
||||
target_path = base_path = servo.util.get_target_dir()
|
||||
if android:
|
||||
target_path = path.join(target_path, "android")
|
||||
base_path = path.join(target_path, target)
|
||||
|
|
@ -141,8 +142,8 @@ class MachCommands(CommandBase):
|
|||
build_start = time()
|
||||
env["CARGO_TARGET_DIR"] = target_path
|
||||
|
||||
host = host_triple()
|
||||
target_triple = target or host_triple()
|
||||
host = servo.platform.host_triple()
|
||||
target_triple = target or servo.platform.host_triple()
|
||||
if 'apple-darwin' in host and target_triple == host:
|
||||
if 'CXXFLAGS' not in env:
|
||||
env['CXXFLAGS'] = ''
|
||||
|
|
@ -191,7 +192,7 @@ class MachCommands(CommandBase):
|
|||
|
||||
# Ensure that the NuGet ANGLE package containing libEGL is accessible
|
||||
# to the Rust linker.
|
||||
append_to_path_env(angle_root(target_triple, env), env, "LIB")
|
||||
servo.util.append_paths_to_env(env, "LIB", angle_root(target_triple, env))
|
||||
|
||||
# Don't want to mix non-UWP libraries with vendored UWP libraries.
|
||||
if "gstreamer" in env['LIB']:
|
||||
|
|
@ -224,12 +225,6 @@ class MachCommands(CommandBase):
|
|||
print(stderr.decode(encoding))
|
||||
exit(1)
|
||||
|
||||
# Ensure that GStreamer libraries are accessible when linking.
|
||||
if 'windows' in target_triple:
|
||||
gst_root = gstreamer_root(target_triple, env)
|
||||
if gst_root:
|
||||
append_to_path_env(os.path.join(gst_root, "lib"), env, "LIB")
|
||||
|
||||
if android:
|
||||
if "ANDROID_NDK" not in env:
|
||||
print("Please set the ANDROID_NDK environment variable.")
|
||||
|
|
@ -511,9 +506,8 @@ class MachCommands(CommandBase):
|
|||
assert os.path.exists(servo_bin_dir)
|
||||
|
||||
if has_media_stack:
|
||||
gst_root = gstreamer_root(target, env)
|
||||
print("Packaging gstreamer dylibs")
|
||||
if not package_gstreamer_dylibs(gst_root, servo_path):
|
||||
if not package_gstreamer_dylibs(target, servo_path):
|
||||
return 1
|
||||
|
||||
# On Mac we use the relocatable dylibs from offical gstreamer
|
||||
|
|
@ -776,7 +770,13 @@ def copy_dependencies(binary_path, lib_path, gst_root):
|
|||
need_checked.difference_update(checked)
|
||||
|
||||
|
||||
def package_gstreamer_dylibs(gst_root, servo_bin):
|
||||
def package_gstreamer_dylibs(cross_compilation_target, servo_bin):
|
||||
gst_root = servo.platform.get().gstreamer_root(cross_compilation_target)
|
||||
|
||||
# This might be None if we are cross-compiling.
|
||||
if not gst_root:
|
||||
return True
|
||||
|
||||
lib_dir = path.join(path.dirname(servo_bin), "lib")
|
||||
if os.path.exists(lib_dir):
|
||||
shutil.rmtree(lib_dir)
|
||||
|
|
@ -791,7 +791,7 @@ def package_gstreamer_dylibs(gst_root, servo_bin):
|
|||
|
||||
|
||||
def package_gstreamer_dlls(env, servo_exe_dir, target, uwp):
|
||||
gst_root = gstreamer_root(target, env)
|
||||
gst_root = servo.platform.get().gstreamer_root(cross_compilation_target=target)
|
||||
if not gst_root:
|
||||
print("Could not find GStreamer installation directory.")
|
||||
return False
|
||||
|
|
|
|||
|
|
@ -34,12 +34,12 @@ from subprocess import PIPE
|
|||
|
||||
import toml
|
||||
import servo.platform
|
||||
import servo.util as util
|
||||
|
||||
from xml.etree.ElementTree import XML
|
||||
from servo.util import download_file, get_default_cache_dir
|
||||
from mach.decorators import CommandArgument
|
||||
from mach.registrar import Registrar
|
||||
from servo.gstreamer import macos_gst_root
|
||||
|
||||
BIN_SUFFIX = ".exe" if sys.platform == "win32" else ""
|
||||
NIGHTLY_REPOSITORY_URL = "https://servo-builds2.s3.amazonaws.com/"
|
||||
|
|
@ -213,37 +213,6 @@ def is_linux():
|
|||
return sys.platform.startswith('linux')
|
||||
|
||||
|
||||
def append_to_path_env(string, env, name):
|
||||
variable = ""
|
||||
if name in env:
|
||||
variable = six.ensure_str(env[name])
|
||||
if len(variable) > 0:
|
||||
variable += os.pathsep
|
||||
variable += string
|
||||
env[name] = variable
|
||||
|
||||
|
||||
def gstreamer_root(target, env, topdir=None):
|
||||
if is_windows():
|
||||
arch = {
|
||||
"x86_64": "X86_64",
|
||||
"x86": "X86",
|
||||
"aarch64": "ARM64",
|
||||
}
|
||||
gst_x64 = arch[target.split('-')[0]]
|
||||
gst_default_path = path.join("C:\\gstreamer\\1.0", gst_x64)
|
||||
gst_env = "GSTREAMER_1_0_ROOT_" + gst_x64
|
||||
if env.get(gst_env) is not None:
|
||||
return env.get(gst_env)
|
||||
elif os.path.exists(path.join(gst_default_path, "bin", "ffi-7.dll")):
|
||||
return gst_default_path
|
||||
elif is_linux():
|
||||
return path.join(topdir, "support", "linux", "gstreamer", "gst")
|
||||
elif is_macosx():
|
||||
return macos_gst_root()
|
||||
return None
|
||||
|
||||
|
||||
class BuildNotFound(Exception):
|
||||
def __init__(self, message):
|
||||
self.message = message
|
||||
|
|
@ -343,14 +312,8 @@ class CommandBase(object):
|
|||
def get_top_dir(self):
|
||||
return self.context.topdir
|
||||
|
||||
def get_target_dir(self):
|
||||
if "CARGO_TARGET_DIR" in os.environ:
|
||||
return os.environ["CARGO_TARGET_DIR"]
|
||||
else:
|
||||
return path.join(self.context.topdir, "target")
|
||||
|
||||
def get_apk_path(self, release):
|
||||
base_path = self.get_target_dir()
|
||||
base_path = util.get_target_dir()
|
||||
base_path = path.join(base_path, "android", self.config["android"]["target"])
|
||||
apk_name = "servoapp.apk"
|
||||
build_type = "release" if release else "debug"
|
||||
|
|
@ -359,7 +322,7 @@ class CommandBase(object):
|
|||
def get_binary_path(self, release, dev, target=None, android=False, simpleservo=False):
|
||||
# TODO(autrilla): this function could still use work - it shouldn't
|
||||
# handle quitting, or printing. It should return the path, or an error.
|
||||
base_path = self.get_target_dir()
|
||||
base_path = util.get_target_dir()
|
||||
|
||||
binary_name = "servo" + BIN_SUFFIX
|
||||
|
||||
|
|
@ -532,56 +495,6 @@ class CommandBase(object):
|
|||
|
||||
return self.get_executable(destination_folder)
|
||||
|
||||
def needs_gstreamer_env(self, target, env, uwp=False, features=[]):
|
||||
if uwp:
|
||||
return False
|
||||
if "media-dummy" in features:
|
||||
return False
|
||||
|
||||
# MacOS always needs the GStreamer environment variable, but should
|
||||
# also check that the Servo-specific version is downloaded and available.
|
||||
if is_macosx():
|
||||
if servo.platform.get().is_gstreamer_installed():
|
||||
return True
|
||||
else:
|
||||
raise Exception("Official GStreamer framework not found (we need at least 1.21)."
|
||||
"Please see installation instructions in README.md")
|
||||
|
||||
try:
|
||||
if servo.platform.get().is_gstreamer_installed():
|
||||
return False
|
||||
except Exception:
|
||||
# Some systems don't have pkg-config; we can't probe in this case
|
||||
# and must hope for the best
|
||||
return False
|
||||
effective_target = target or servo.platform.host_triple()
|
||||
if "x86_64" not in effective_target or "android" in effective_target:
|
||||
# We don't build gstreamer for non-x86_64 / android yet
|
||||
return False
|
||||
if is_linux() or is_windows():
|
||||
if path.isdir(gstreamer_root(effective_target, env, self.get_top_dir())):
|
||||
return True
|
||||
else:
|
||||
raise Exception("Your system's gstreamer libraries are out of date \
|
||||
(we need at least 1.16). Please run ./mach bootstrap-gstreamer")
|
||||
else:
|
||||
raise Exception("Your system's gstreamer libraries are out of date \
|
||||
(we need at least 1.16). If you're unable to \
|
||||
install them, let us know by filing a bug!")
|
||||
return False
|
||||
|
||||
def set_run_env(self, android=False):
|
||||
"""Some commands, like test-wpt, don't use a full build env,
|
||||
but may still need dynamic search paths. This command sets that up"""
|
||||
if not android and self.needs_gstreamer_env(None, os.environ):
|
||||
gstpath = gstreamer_root(servo.platform.host_triple(), os.environ, self.get_top_dir())
|
||||
if gstpath is None:
|
||||
return
|
||||
os.environ["LD_LIBRARY_PATH"] = path.join(gstpath, "lib")
|
||||
os.environ["GST_PLUGIN_SYSTEM_PATH"] = path.join(gstpath, "lib", "gstreamer-1.0")
|
||||
os.environ["PKG_CONFIG_PATH"] = path.join(gstpath, "lib", "pkgconfig")
|
||||
os.environ["GST_PLUGIN_SCANNER"] = path.join(gstpath, "libexec", "gstreamer-1.0", "gst-plugin-scanner")
|
||||
|
||||
def msvc_package_dir(self, package):
|
||||
return path.join(self.context.sharedir, "msvc-dependencies", package,
|
||||
servo.platform.windows.DEPENDENCIES[package])
|
||||
|
|
@ -606,6 +519,10 @@ install them, let us know by filing a bug!")
|
|||
def build_env(self, hosts_file_path=None, target=None, is_build=False, test_unit=False, uwp=False, features=None):
|
||||
"""Return an extended environment dictionary."""
|
||||
env = os.environ.copy()
|
||||
|
||||
if not features or "media-dummy" not in features:
|
||||
servo.platform.get().set_gstreamer_environment_variables_if_necessary(env, cross_compilation_target=target)
|
||||
|
||||
if sys.platform == "win32" and type(env['PATH']) == six.text_type:
|
||||
# On win32, the virtualenv's activate_this.py script sometimes ends up
|
||||
# turning os.environ['PATH'] into a unicode string. This doesn't work
|
||||
|
|
@ -615,7 +532,6 @@ install them, let us know by filing a bug!")
|
|||
# it in any case.
|
||||
env['PATH'] = env['PATH'].encode('ascii', 'ignore')
|
||||
extra_path = []
|
||||
extra_lib = []
|
||||
if "msvc" in (target or servo.platform.host_triple()):
|
||||
extra_path += [path.join(self.msvc_package_dir("cmake"), "bin")]
|
||||
extra_path += [path.join(self.msvc_package_dir("llvm"), "bin")]
|
||||
|
|
@ -666,19 +582,6 @@ install them, let us know by filing a bug!")
|
|||
# Always build harfbuzz from source
|
||||
env["HARFBUZZ_SYS_NO_PKG_CONFIG"] = "true"
|
||||
|
||||
if is_build and self.needs_gstreamer_env(target or servo.platform.host_triple(), env, uwp, features):
|
||||
gst_root = gstreamer_root(target or servo.platform.host_triple(), env, self.get_top_dir())
|
||||
bin_path = path.join(gst_root, "bin")
|
||||
lib_path = path.join(gst_root, "lib")
|
||||
pkg_config_path = path.join(lib_path, "pkgconfig")
|
||||
# we append in the reverse order so that system gstreamer libraries
|
||||
# do not get precedence
|
||||
extra_path = [bin_path] + extra_path
|
||||
extra_lib = [lib_path] + extra_lib
|
||||
append_to_path_env(pkg_config_path, env, "PKG_CONFIG_PATH")
|
||||
if is_macosx():
|
||||
env["OPENSSL_INCLUDE_DIR"] = path.join(gst_root, "Headers")
|
||||
|
||||
if is_linux():
|
||||
distrib, version, _ = distro.linux_distribution()
|
||||
distrib = six.ensure_str(distrib)
|
||||
|
|
@ -687,17 +590,13 @@ install them, let us know by filing a bug!")
|
|||
env["HARFBUZZ_SYS_NO_PKG_CONFIG"] = "true"
|
||||
|
||||
if extra_path:
|
||||
append_to_path_env(os.pathsep.join(extra_path), env, "PATH")
|
||||
util.append_paths_to_env(env, "PATH", extra_path)
|
||||
|
||||
if self.config["build"]["incremental"]:
|
||||
env["CARGO_INCREMENTAL"] = "1"
|
||||
elif self.config["build"]["incremental"] is not None:
|
||||
env["CARGO_INCREMENTAL"] = "0"
|
||||
|
||||
if extra_lib:
|
||||
path_var = "DYLD_LIBRARY_PATH" if sys.platform == "darwin" else "LD_LIBRARY_PATH"
|
||||
append_to_path_env(os.pathsep.join(extra_lib), env, path_var)
|
||||
|
||||
# Paths to Android build tools:
|
||||
if self.config["android"]["sdk"]:
|
||||
env["ANDROID_SDK"] = self.config["android"]["sdk"]
|
||||
|
|
@ -737,6 +636,14 @@ install them, let us know by filing a bug!")
|
|||
# This wrapper script is in bash and doesn't work on Windows
|
||||
# where we want to run doctests as part of `./mach test-unit`
|
||||
env['RUSTDOC'] = path.join(self.context.topdir, 'etc', 'rustdoc-with-private')
|
||||
elif "msvc" in servo.platform.host_triple():
|
||||
# on MSVC, we need some DLLs in the path. They were copied
|
||||
# in to the servo.exe build dir, so just point PATH to that.
|
||||
util.prepend_paths_to_env(env, "PATH", path.dirname(self.get_binary_path(False, False)))
|
||||
|
||||
# FIXME: https://github.com/servo/servo/issues/26192
|
||||
if test_unit and "apple-darwin" not in servo.platform.host_triple():
|
||||
env["RUST_BACKTRACE"] = "1"
|
||||
|
||||
if self.config["build"]["rustflags"]:
|
||||
env['RUSTFLAGS'] = env.get('RUSTFLAGS', "") + " " + self.config["build"]["rustflags"]
|
||||
|
|
@ -1067,7 +974,7 @@ install them, let us know by filing a bug!")
|
|||
|
||||
def ensure_clobbered(self, target_dir=None):
|
||||
if target_dir is None:
|
||||
target_dir = self.get_target_dir()
|
||||
target_dir = util.get_target_dir()
|
||||
auto = True if os.environ.get('AUTOCLOBBER', False) else False
|
||||
src_clobber = os.path.join(self.context.topdir, 'CLOBBER')
|
||||
target_clobber = os.path.join(target_dir, 'CLOBBER')
|
||||
|
|
|
|||
|
|
@ -48,9 +48,8 @@ class MachCommands(CommandBase):
|
|||
|
||||
self.ensure_bootstrapped(target=target)
|
||||
self.ensure_clobbered()
|
||||
env = self.build_env()
|
||||
|
||||
status = self.run_cargo_build_like_command("check", params, env=env, features=features, **kwargs)
|
||||
status = self.run_cargo_build_like_command("check", params, features=features, **kwargs)
|
||||
if status == 0:
|
||||
print('Finished checking, binary NOT updated. Consider ./mach build before ./mach run')
|
||||
|
||||
|
|
@ -142,9 +141,8 @@ class MachCommands(CommandBase):
|
|||
|
||||
self.ensure_bootstrapped(target=target)
|
||||
self.ensure_clobbered()
|
||||
env = self.build_env()
|
||||
|
||||
return self.run_cargo_build_like_command("fix", params, env=env, features=features, **kwargs)
|
||||
return self.run_cargo_build_like_command("fix", params, features=features, **kwargs)
|
||||
|
||||
@Command('cargo-clippy',
|
||||
description='Run "cargo clippy"',
|
||||
|
|
@ -166,9 +164,8 @@ class MachCommands(CommandBase):
|
|||
|
||||
self.ensure_bootstrapped(target=target)
|
||||
self.ensure_clobbered()
|
||||
env = self.build_env()
|
||||
|
||||
return self.run_cargo_build_like_command("clippy", params, env=env, features=features, **kwargs)
|
||||
return self.run_cargo_build_like_command("clippy", params, features=features, **kwargs)
|
||||
|
||||
@Command('grep',
|
||||
description='`git grep` for selected directories.',
|
||||
|
|
|
|||
|
|
@ -9,9 +9,10 @@
|
|||
|
||||
import platform
|
||||
|
||||
from .base import Base
|
||||
from .windows import Windows
|
||||
|
||||
__platform__ = None
|
||||
|
||||
|
||||
def host_platform():
|
||||
os_type = platform.system().lower()
|
||||
|
|
@ -47,18 +48,27 @@ def host_triple():
|
|||
|
||||
|
||||
def get():
|
||||
# pylint: disable=global-statement
|
||||
global __platform__
|
||||
if __platform__:
|
||||
return __platform__
|
||||
|
||||
# We import the concrete platforms in if-statements here, because
|
||||
# each one might have platform-specific imports which might not
|
||||
# resolve on all platforms.
|
||||
# TODO(mrobinson): We should do this for Windows too, once we
|
||||
# stop relying on platform-specific code outside of this module.
|
||||
# pylint: disable=import-outside-toplevel
|
||||
if "windows-msvc" in host_triple():
|
||||
return Windows()
|
||||
if "linux-gnu" in host_triple():
|
||||
triple = host_triple()
|
||||
if "windows-msvc" in triple:
|
||||
__platform__ = Windows(triple)
|
||||
elif "linux-gnu" in triple:
|
||||
from .linux import Linux
|
||||
return Linux()
|
||||
if "apple-darwin" in host_triple():
|
||||
__platform__ = Linux(triple)
|
||||
elif "apple-darwin" in triple:
|
||||
from .macos import MacOS
|
||||
return MacOS()
|
||||
return Base()
|
||||
__platform__ = MacOS(triple)
|
||||
else:
|
||||
from .base import Base
|
||||
__platform__ = Base(triple)
|
||||
return __platform__
|
||||
|
|
|
|||
|
|
@ -7,28 +7,104 @@
|
|||
# option. This file may not be copied, modified, or distributed
|
||||
# except according to those terms.
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from typing import Dict, Optional
|
||||
|
||||
from .. import util
|
||||
|
||||
|
||||
class Base:
|
||||
def __init__(self, triple: str):
|
||||
self.environ = os.environ.copy()
|
||||
self.triple = triple
|
||||
self.is_windows = False
|
||||
self.is_linux = False
|
||||
self.is_macos = False
|
||||
|
||||
def set_gstreamer_environment_variables_if_necessary(
|
||||
self, env: Dict[str, str], cross_compilation_target: Optional[str], check_installation=True
|
||||
):
|
||||
# Environment variables are not needed when cross-compiling on any
|
||||
# platform other than Windows. UWP doesn't support GStreamer. GStreamer
|
||||
# for Android is handled elsewhere.
|
||||
if cross_compilation_target and (
|
||||
not self.is_windows
|
||||
or "uwp" in cross_compilation_target
|
||||
or "android" in cross_compilation_target
|
||||
):
|
||||
return
|
||||
|
||||
# We may not need to update environment variables if GStreamer is installed
|
||||
# for the system on Linux.
|
||||
gstreamer_root = self.gstreamer_root(cross_compilation_target)
|
||||
if gstreamer_root:
|
||||
util.prepend_paths_to_env(env, "PATH", os.path.join(gstreamer_root, "bin"))
|
||||
util.prepend_paths_to_env(
|
||||
env, "PKG_CONFIG_PATH", os.path.join(gstreamer_root, "lib", "pkgconfig")
|
||||
)
|
||||
util.prepend_paths_to_env(
|
||||
env,
|
||||
self.library_path_variable_name(),
|
||||
os.path.join(gstreamer_root, "lib"),
|
||||
)
|
||||
env["GST_PLUGIN_SCANNER"] = os.path.join(
|
||||
gstreamer_root,
|
||||
"libexec",
|
||||
"gstreamer-1.0",
|
||||
f"gst-plugin-scanner{self.executable_suffix()}",
|
||||
)
|
||||
env["GST_PLUGIN_SYSTEM_PATH"] = os.path.join(gstreamer_root, "lib", "gstreamer-1.0")
|
||||
if self.is_macos:
|
||||
env["OPENSSL_INCLUDE_DIR"] = os.path.join(gstreamer_root, "Headers")
|
||||
|
||||
# If we are not cross-compiling GStreamer must be installed for the system. In
|
||||
# the cross-compilation case, we might be picking it up from another directory.
|
||||
if check_installation and not self.is_gstreamer_installed(cross_compilation_target):
|
||||
raise FileNotFoundError(
|
||||
"GStreamer libraries not found (>= version 1.16)."
|
||||
"Please see installation instructions in README.md"
|
||||
)
|
||||
|
||||
def gstreamer_root(self, _cross_compilation_target: Optional[str]) -> Optional[str]:
|
||||
raise NotImplementedError("Do not know how to get GStreamer path for platform.")
|
||||
|
||||
def library_path_variable_name(self):
|
||||
raise NotImplementedError("Do not know how to set library path for platform.")
|
||||
|
||||
def executable_suffix(self):
|
||||
return ""
|
||||
|
||||
def _platform_bootstrap(self, _cache_dir: str, _force: bool) -> bool:
|
||||
raise NotImplementedError("Bootstrap installation detection not yet available.")
|
||||
|
||||
def _platform_bootstrap_gstreamer(self, _cache_dir: str, _force: bool) -> bool:
|
||||
raise NotImplementedError("GStreamer bootstrap support is not yet available for your OS.")
|
||||
def _platform_bootstrap_gstreamer(self, _force: bool) -> bool:
|
||||
raise NotImplementedError(
|
||||
"GStreamer bootstrap support is not yet available for your OS."
|
||||
)
|
||||
|
||||
def _platform_is_gstreamer_installed(self) -> bool:
|
||||
return subprocess.call(
|
||||
["pkg-config", "--atleast-version=1.16", "gstreamer-1.0"],
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE) == 0
|
||||
def is_gstreamer_installed(self, cross_compilation_target: Optional[str]) -> bool:
|
||||
env = os.environ.copy()
|
||||
self.set_gstreamer_environment_variables_if_necessary(
|
||||
env, cross_compilation_target, check_installation=False)
|
||||
return (
|
||||
subprocess.call(
|
||||
["pkg-config", "--atleast-version=1.16", "gstreamer-1.0"],
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
env=env,
|
||||
)
|
||||
== 0
|
||||
)
|
||||
|
||||
def bootstrap(self, cache_dir: str, force: bool):
|
||||
if not self._platform_bootstrap(cache_dir, force):
|
||||
print("Dependencies were already installed!")
|
||||
|
||||
def bootstrap_gstreamer(self, cache_dir: str, force: bool):
|
||||
if not self._platform_bootstrap_gstreamer(cache_dir, force):
|
||||
print("Dependencies were already installed!")
|
||||
|
||||
def is_gstreamer_installed(self) -> bool:
|
||||
return self._platform_is_gstreamer_installed()
|
||||
def bootstrap_gstreamer(self, _cache_dir: str, force: bool):
|
||||
if not self._platform_bootstrap_gstreamer(force):
|
||||
root = self.gstreamer_root(None)
|
||||
if root:
|
||||
print(f"GStreamer found at: {root}")
|
||||
else:
|
||||
print("GStreamer already installed system-wide.")
|
||||
|
|
|
|||
|
|
@ -9,12 +9,12 @@
|
|||
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
from typing import Tuple
|
||||
import tempfile
|
||||
from typing import Optional, Tuple
|
||||
|
||||
import distro
|
||||
import six
|
||||
|
||||
from .. import util
|
||||
from .base import Base
|
||||
|
||||
# Please keep these in sync with the packages in README.md
|
||||
|
|
@ -48,11 +48,21 @@ XBPS_PKGS = ['libtool', 'gcc', 'libXi-devel', 'freetype-devel',
|
|||
'clang', 'gstreamer1-devel', 'autoconf213',
|
||||
'gst-plugins-base1-devel', 'gst-plugins-bad1-devel']
|
||||
|
||||
GSTREAMER_URL = \
|
||||
"https://github.com/servo/servo-build-deps/releases/download/linux/gstreamer-1.16-x86_64-linux-gnu.20190515.tar.gz"
|
||||
PREPACKAGED_GSTREAMER_ROOT = \
|
||||
os.path.join(util.get_target_dir(), "dependencies", "gstreamer")
|
||||
|
||||
|
||||
class Linux(Base):
|
||||
def __init__(self):
|
||||
def __init__(self, *args, **kwargs):
|
||||
super().__init__(*args, **kwargs)
|
||||
self.is_linux = True
|
||||
(self.distro, self.version) = Linux.get_distro_and_version()
|
||||
|
||||
def library_path_variable_name(self):
|
||||
return "LD_LIBRARY_PATH"
|
||||
|
||||
@staticmethod
|
||||
def get_distro_and_version() -> Tuple[str, str]:
|
||||
distrib = six.ensure_str(distro.name())
|
||||
|
|
@ -116,7 +126,7 @@ class Linux(Base):
|
|||
f"{self.distro}, please file a bug")
|
||||
|
||||
installed_something = self.install_non_gstreamer_dependencies(force)
|
||||
installed_something |= self._platform_bootstrap_gstreamer(_cache_dir, force)
|
||||
installed_something |= self._platform_bootstrap_gstreamer(force)
|
||||
return installed_something
|
||||
|
||||
def install_non_gstreamer_dependencies(self, force: bool) -> bool:
|
||||
|
|
@ -157,15 +167,34 @@ class Linux(Base):
|
|||
|
||||
print("Installing missing dependencies...")
|
||||
if run_as_root(command + pkgs, force) != 0:
|
||||
raise Exception("Installation of dependencies failed.")
|
||||
raise EnvironmentError("Installation of dependencies failed.")
|
||||
return True
|
||||
|
||||
def _platform_bootstrap_gstreamer(self, _cache_dir: str, _force: bool) -> bool:
|
||||
if self.is_gstreamer_installed():
|
||||
def gstreamer_root(self, cross_compilation_target: Optional[str]) -> Optional[str]:
|
||||
if cross_compilation_target:
|
||||
return None
|
||||
if os.path.exists(PREPACKAGED_GSTREAMER_ROOT):
|
||||
return PREPACKAGED_GSTREAMER_ROOT
|
||||
# GStreamer might be installed system-wide, but we do not return a root in this
|
||||
# case because we don't have to update environment variables.
|
||||
return None
|
||||
|
||||
def _platform_bootstrap_gstreamer(self, force: bool) -> bool:
|
||||
if not force and self.is_gstreamer_installed(cross_compilation_target=None):
|
||||
return False
|
||||
|
||||
gstdir = os.path.join(os.curdir, "support", "linux", "gstreamer")
|
||||
if not os.path.isdir(os.path.join(gstdir, "gst", "lib")):
|
||||
subprocess.check_call(["bash", "gstreamer.sh"], cwd=gstdir)
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_name = os.path.join(temp_dir, GSTREAMER_URL.rsplit('/', maxsplit=1)[-1])
|
||||
util.download_file("Pre-packaged GStreamer binaries", GSTREAMER_URL, file_name)
|
||||
|
||||
print(f"Installing GStreamer packages to {PREPACKAGED_GSTREAMER_ROOT}...")
|
||||
os.makedirs(PREPACKAGED_GSTREAMER_ROOT, exist_ok=True)
|
||||
|
||||
# Extract, but strip one component from the output, because the package includes
|
||||
# a toplevel directory called "./gst/" and we'd like to have the same directory
|
||||
# structure on all platforms.
|
||||
subprocess.check_call(["tar", "xf", file_name, "-C", PREPACKAGED_GSTREAMER_ROOT,
|
||||
"--strip-components=2"])
|
||||
|
||||
assert self.is_gstreamer_installed(cross_compilation_target=None)
|
||||
return True
|
||||
return False
|
||||
|
|
|
|||
|
|
@ -9,28 +9,76 @@
|
|||
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
from typing import Optional
|
||||
|
||||
from .. import util
|
||||
from .base import Base
|
||||
from ..gstreamer import macos_gst_root
|
||||
|
||||
URL_BASE = "https://github.com/servo/servo-build-deps/releases/download/macOS"
|
||||
GSTREAMER_URL = f"{URL_BASE}/gstreamer-1.0-1.22.2-universal.pkg"
|
||||
GSTREAMER_DEVEL_URL = f"{URL_BASE}/gstreamer-1.0-devel-1.22.2-universal.pkg"
|
||||
GSTREAMER_ROOT = "/Library/Frameworks/GStreamer.framework/Versions/1.0"
|
||||
|
||||
|
||||
class MacOS(Base):
|
||||
def __init__(self):
|
||||
pass
|
||||
def __init__(self, *args, **kwargs):
|
||||
super().__init__(*args, **kwargs)
|
||||
self.is_macos = True
|
||||
|
||||
def _platform_is_gstreamer_installed(self) -> bool:
|
||||
# We override homebrew gstreamer if installed and always use pkgconfig
|
||||
# from official gstreamer framework.
|
||||
try:
|
||||
gst_root = macos_gst_root()
|
||||
env = os.environ.copy()
|
||||
env["PATH"] = os.path.join(gst_root, "bin")
|
||||
env["PKG_CONFIG_PATH"] = os.path.join(gst_root, "lib", "pkgconfig")
|
||||
has_gst = subprocess.call(
|
||||
["pkg-config", "--atleast-version=1.21", "gstreamer-1.0"],
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env) == 0
|
||||
gst_lib_dir = subprocess.check_output(
|
||||
["pkg-config", "--variable=libdir", "gstreamer-1.0"], env=env)
|
||||
return has_gst and gst_lib_dir.startswith(bytes(gst_root, 'utf-8'))
|
||||
except FileNotFoundError:
|
||||
def library_path_variable_name(self):
|
||||
return "DYLD_LIBRARY_PATH"
|
||||
|
||||
def gstreamer_root(self, cross_compilation_target: Optional[str]) -> Optional[str]:
|
||||
# We do not support building with gstreamer while cross-compiling on MacOS.
|
||||
if cross_compilation_target or not os.path.exists(GSTREAMER_ROOT):
|
||||
return None
|
||||
return GSTREAMER_ROOT
|
||||
|
||||
def is_gstreamer_installed(self, cross_compilation_target: Optional[str]) -> bool:
|
||||
if not super().is_gstreamer_installed(cross_compilation_target):
|
||||
return False
|
||||
|
||||
# Servo only supports the official GStreamer distribution on MacOS.
|
||||
env = os.environ.copy()
|
||||
self.set_gstreamer_environment_variables_if_necessary(
|
||||
env, cross_compilation_target, check_installation=False
|
||||
)
|
||||
gst_lib_dir = subprocess.check_output(
|
||||
["pkg-config", "--variable=libdir", "gstreamer-1.0"], env=env
|
||||
)
|
||||
if not gst_lib_dir.startswith(bytes(GSTREAMER_ROOT, "utf-8")):
|
||||
print("GStreamer is installed, but not the official packages.\n"
|
||||
"Run `./mach bootstrap-gtstreamer` or install packages from "
|
||||
"https://gstreamer.freedesktop.org/")
|
||||
return False
|
||||
return True
|
||||
|
||||
def _platform_bootstrap_gstreamer(self, force: bool) -> bool:
|
||||
if not force and self.is_gstreamer_installed(cross_compilation_target=None):
|
||||
return False
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
libs_pkg = os.path.join(temp_dir, GSTREAMER_URL.rsplit("/", maxsplit=1)[-1])
|
||||
devel_pkg = os.path.join(
|
||||
temp_dir, GSTREAMER_DEVEL_URL.rsplit("/", maxsplit=1)[-1]
|
||||
)
|
||||
|
||||
util.download_file("GStreamer libraries", GSTREAMER_URL, libs_pkg)
|
||||
util.download_file(
|
||||
"GStreamer development support", GSTREAMER_DEVEL_URL, devel_pkg
|
||||
)
|
||||
|
||||
print("Installing GStreamer packages...")
|
||||
subprocess.check_call(
|
||||
[
|
||||
"sudo",
|
||||
"sh",
|
||||
"-c",
|
||||
f"installer -pkg '{libs_pkg}' -target / &&"
|
||||
f"installer -pkg '{devel_pkg}' -target /",
|
||||
]
|
||||
)
|
||||
|
||||
assert self.is_gstreamer_installed(cross_compilation_target=None)
|
||||
return True
|
||||
|
|
|
|||
|
|
@ -10,14 +10,15 @@
|
|||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
from typing import Optional
|
||||
import urllib
|
||||
import zipfile
|
||||
|
||||
from distutils.version import LooseVersion
|
||||
|
||||
import six
|
||||
from .. import util
|
||||
from .base import Base
|
||||
from ..util import extract, download_file
|
||||
|
||||
DEPS_URL = "https://github.com/servo/servo-build-deps/releases/download/msvc-deps/"
|
||||
DEPENDENCIES = {
|
||||
|
|
@ -31,10 +32,22 @@ DEPENDENCIES = {
|
|||
"openxr-loader-uwp": "1.0",
|
||||
}
|
||||
|
||||
URL_BASE = "https://gstreamer.freedesktop.org/data/pkg/windows/1.16.0/"
|
||||
GSTREAMER_URL = f"{URL_BASE}/gstreamer-1.0-msvc-x86_64-1.16.0.msi"
|
||||
GSTREAMER_DEVEL_URL = f"{URL_BASE}/gstreamer-1.0-devel-msvc-x86_64-1.16.0.msi"
|
||||
DEPENDENCIES_DIR = os.path.join(util.get_target_dir(), "dependencies")
|
||||
|
||||
|
||||
class Windows(Base):
|
||||
def __init__(self):
|
||||
pass
|
||||
def __init__(self, triple: str):
|
||||
super().__init__(triple)
|
||||
self.is_windows = True
|
||||
|
||||
def executable_suffix(self):
|
||||
return ".exe"
|
||||
|
||||
def library_path_variable_name(self):
|
||||
return "LIB"
|
||||
|
||||
@staticmethod
|
||||
def cmake_already_installed(required_version: str) -> bool:
|
||||
|
|
@ -51,11 +64,11 @@ class Windows(Base):
|
|||
def prepare_file(cls, deps_dir: str, zip_path: str, full_spec: str):
|
||||
if not os.path.isfile(zip_path):
|
||||
zip_url = "{}{}.zip".format(DEPS_URL, urllib.parse.quote(full_spec))
|
||||
download_file(full_spec, zip_url, zip_path)
|
||||
util.download_file(full_spec, zip_url, zip_path)
|
||||
|
||||
print("Extracting {}...".format(full_spec), end='')
|
||||
print("Extracting {}...".format(full_spec), end="")
|
||||
try:
|
||||
extract(zip_path, deps_dir)
|
||||
util.extract(zip_path, deps_dir)
|
||||
except zipfile.BadZipfile:
|
||||
print("\nError: %s.zip is not a valid zip file, redownload..." % full_spec)
|
||||
os.remove(zip_path)
|
||||
|
|
@ -70,7 +83,7 @@ class Windows(Base):
|
|||
return os.path.join(deps_dir, package, version)
|
||||
|
||||
to_install = {}
|
||||
for (package, version) in DEPENDENCIES.items():
|
||||
for package, version in DEPENDENCIES.items():
|
||||
# Don't install CMake if it already exists in PATH
|
||||
if package == "cmake" and self.cmake_already_installed(version):
|
||||
continue
|
||||
|
|
@ -82,8 +95,8 @@ class Windows(Base):
|
|||
return False
|
||||
|
||||
print("Installing missing MSVC dependencies...")
|
||||
for (package, version) in to_install.items():
|
||||
full_spec = '{}-{}'.format(package, version)
|
||||
for package, version in to_install.items():
|
||||
full_spec = "{}-{}".format(package, version)
|
||||
|
||||
package_dir = get_package_dir(package, version)
|
||||
parent_dir = os.path.dirname(package_dir)
|
||||
|
|
@ -96,3 +109,67 @@ class Windows(Base):
|
|||
os.rename(extracted_path, package_dir)
|
||||
|
||||
return True
|
||||
|
||||
def gstreamer_root(self, cross_compilation_target: Optional[str]) -> Optional[str]:
|
||||
build_target_triple = cross_compilation_target or self.triple
|
||||
gst_arch_names = {
|
||||
"x86_64": "X86_64",
|
||||
"x86": "X86",
|
||||
"aarch64": "ARM64",
|
||||
}
|
||||
gst_arch_name = gst_arch_names[build_target_triple.split("-")[0]]
|
||||
|
||||
# The bootstraped version of GStreamer always takes precedance of the installed vesion.
|
||||
prepackaged_root = os.path.join(
|
||||
DEPENDENCIES_DIR, "gstreamer", "1.0", gst_arch_name
|
||||
)
|
||||
if os.path.exists(os.path.join(prepackaged_root, "bin", "ffi-7.dll")):
|
||||
return prepackaged_root
|
||||
|
||||
# The installed version of GStreamer often sets an environment variable pointing to
|
||||
# the install location.
|
||||
root_from_env = os.environ.get(f"GSTREAMER_1_0_ROOT_{gst_arch_name}")
|
||||
if root_from_env:
|
||||
return root_from_env
|
||||
|
||||
# If all else fails, look for an installation in the default install directory.
|
||||
default_root = os.path.join("C:\\gstreamer\\1.0", gst_arch_name)
|
||||
if os.path.exists(os.path.join(default_root, "bin", "ffi-7.dll")):
|
||||
return default_root
|
||||
|
||||
return None
|
||||
|
||||
def is_gstreamer_installed(self, cross_compilation_target: Optional[str]) -> bool:
|
||||
return self.gstreamer_root(cross_compilation_target) is not None
|
||||
|
||||
def _platform_bootstrap_gstreamer(self, force: bool) -> bool:
|
||||
if not force and self.is_gstreamer_installed(cross_compilation_target=None):
|
||||
return False
|
||||
|
||||
if "x86_64" not in self.triple:
|
||||
print("Bootstrapping gstreamer not supported on "
|
||||
"non-x86-64 Windows. Please install manually")
|
||||
return False
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
libs_msi = os.path.join(temp_dir, GSTREAMER_URL.rsplit("/", maxsplit=1)[-1])
|
||||
devel_msi = os.path.join(
|
||||
temp_dir, GSTREAMER_DEVEL_URL.rsplit("/", maxsplit=1)[-1]
|
||||
)
|
||||
|
||||
util.download_file("GStreamer libraries", GSTREAMER_URL, libs_msi)
|
||||
util.download_file(
|
||||
"GStreamer development support", GSTREAMER_DEVEL_URL, devel_msi
|
||||
)
|
||||
|
||||
print(f"Installing GStreamer packages to {DEPENDENCIES_DIR}...")
|
||||
os.makedirs(DEPENDENCIES_DIR, exist_ok=True)
|
||||
common_args = [
|
||||
f"TARGETDIR={DEPENDENCIES_DIR}", # Install destination
|
||||
"/qn", # Quiet mode
|
||||
]
|
||||
subprocess.check_call(["msiexec", "/a", libs_msi] + common_args)
|
||||
subprocess.check_call(["msiexec", "/a", devel_msi] + common_args)
|
||||
|
||||
assert self.is_gstreamer_installed(cross_compilation_target=None)
|
||||
return True
|
||||
|
|
|
|||
|
|
@ -15,6 +15,8 @@ import os.path as path
|
|||
import subprocess
|
||||
from shutil import copytree, rmtree, copy2
|
||||
|
||||
import servo.util
|
||||
|
||||
from mach.decorators import (
|
||||
CommandArgument,
|
||||
CommandProvider,
|
||||
|
|
@ -79,9 +81,15 @@ class PostBuildCommands(CommandBase):
|
|||
help="Command-line arguments to be passed through to Servo")
|
||||
def run(self, params, release=False, dev=False, android=None, debug=False, debugger=None,
|
||||
headless=False, software=False, bin=None, emulator=False, usb=False, nightly=None):
|
||||
self.set_run_env(android is not None)
|
||||
env = self.build_env()
|
||||
env["RUST_BACKTRACE"] = "1"
|
||||
if software:
|
||||
if not is_linux():
|
||||
print("Software rendering is only supported on Linux at the moment.")
|
||||
return
|
||||
|
||||
env['LIBGL_ALWAYS_SOFTWARE'] = "1"
|
||||
os.environ.update(env)
|
||||
|
||||
# Make --debugger imply --debug
|
||||
if debugger:
|
||||
|
|
@ -129,13 +137,6 @@ class PostBuildCommands(CommandBase):
|
|||
if headless:
|
||||
args.append('-z')
|
||||
|
||||
if software:
|
||||
if not is_linux():
|
||||
print("Software rendering is only supported on Linux at the moment.")
|
||||
return
|
||||
|
||||
env['LIBGL_ALWAYS_SOFTWARE'] = "1"
|
||||
|
||||
# Borrowed and modified from:
|
||||
# http://hg.mozilla.org/mozilla-central/file/c9cfa9b91dea/python/mozbuild/mozbuild/mach_commands.py#l883
|
||||
if debug:
|
||||
|
|
@ -251,7 +252,7 @@ class PostBuildCommands(CommandBase):
|
|||
toolchain_path = path.dirname(path.dirname(rustc_path))
|
||||
rust_docs = path.join(toolchain_path, "share", "doc", "rust", "html")
|
||||
|
||||
docs = path.join(self.get_target_dir(), "doc")
|
||||
docs = path.join(servo.util.get_target_dir(), "doc")
|
||||
if not path.exists(docs):
|
||||
os.makedirs(docs)
|
||||
|
||||
|
|
@ -293,4 +294,4 @@ class PostBuildCommands(CommandBase):
|
|||
self.doc([])
|
||||
import webbrowser
|
||||
webbrowser.open("file://" + path.abspath(path.join(
|
||||
self.get_target_dir(), "doc", "servo", "index.html")))
|
||||
servo.util.get_target_dir(), "doc", "servo", "index.html")))
|
||||
|
|
|
|||
|
|
@ -40,7 +40,6 @@ from servo.command_base import (
|
|||
call, check_call, check_output,
|
||||
)
|
||||
from servo_tidy_tests import test_tidy
|
||||
from servo.platform import host_triple
|
||||
|
||||
SCRIPT_PATH = os.path.split(__file__)[0]
|
||||
PROJECT_TOPLEVEL_PATH = os.path.abspath(os.path.join(SCRIPT_PATH, "..", ".."))
|
||||
|
|
@ -247,16 +246,6 @@ class MachCommands(CommandBase):
|
|||
|
||||
packages.discard('stylo')
|
||||
|
||||
env = self.build_env(test_unit=True)
|
||||
# FIXME: https://github.com/servo/servo/issues/26192
|
||||
if "apple-darwin" not in host_triple():
|
||||
env["RUST_BACKTRACE"] = "1"
|
||||
|
||||
if "msvc" in host_triple():
|
||||
# on MSVC, we need some DLLs in the path. They were copied
|
||||
# in to the servo.exe build dir, so just point PATH to that.
|
||||
env["PATH"] = "%s%s%s" % (path.dirname(self.get_binary_path(False, False)), os.pathsep, env["PATH"])
|
||||
|
||||
if len(packages) > 0 or len(in_crate_packages) > 0:
|
||||
args = []
|
||||
for crate in packages:
|
||||
|
|
@ -270,7 +259,7 @@ class MachCommands(CommandBase):
|
|||
|
||||
err = self.run_cargo_build_like_command("bench" if bench else "test",
|
||||
args,
|
||||
env=env,
|
||||
env=self.build_env(test_unit=True),
|
||||
with_layout_2020=with_layout_2020,
|
||||
**kwargs)
|
||||
if err:
|
||||
|
|
@ -393,7 +382,8 @@ class MachCommands(CommandBase):
|
|||
return self._test_wpt(android=True, **kwargs)
|
||||
|
||||
def _test_wpt(self, android=False, **kwargs):
|
||||
self.set_run_env(android)
|
||||
if not android:
|
||||
os.environ.update(self.build_env())
|
||||
return wpt.run.run_tests(**kwargs)
|
||||
|
||||
# Helper to ensure all specified paths are handled, otherwise dispatch to appropriate test suite.
|
||||
|
|
|
|||
|
|
@ -17,9 +17,12 @@ import stat
|
|||
import sys
|
||||
import time
|
||||
import urllib
|
||||
import urllib.request
|
||||
import zipfile
|
||||
from typing import Dict, List, Union
|
||||
|
||||
from io import BytesIO
|
||||
import six
|
||||
from io import BufferedIOBase, BytesIO
|
||||
from socket import error as socket_error
|
||||
|
||||
try:
|
||||
|
|
@ -27,6 +30,8 @@ try:
|
|||
except ImportError:
|
||||
HAS_SNI = False
|
||||
|
||||
SCRIPT_PATH = os.path.abspath(os.path.dirname(__file__))
|
||||
SERVO_ROOT = os.path.abspath(os.path.join(SCRIPT_PATH, "..", ".."))
|
||||
HAS_SNI_AND_RECENT_PYTHON = HAS_SNI and sys.version_info >= (2, 7, 9)
|
||||
|
||||
|
||||
|
|
@ -58,17 +63,17 @@ def delete(path):
|
|||
os.remove(path)
|
||||
|
||||
|
||||
def download(desc, src, writer, start_byte=0):
|
||||
def download(description: str, url: str, writer: BufferedIOBase, start_byte: int = 0):
|
||||
if start_byte:
|
||||
print("Resuming download of {} ...".format(src))
|
||||
print("Resuming download of {} ...".format(url))
|
||||
else:
|
||||
print("Downloading {} ...".format(src))
|
||||
print("Downloading {} ...".format(url))
|
||||
dumb = (os.environ.get("TERM") == "dumb") or (not sys.stdout.isatty())
|
||||
|
||||
try:
|
||||
req = urllib.request.Request(src)
|
||||
req = urllib.request.Request(url)
|
||||
if start_byte:
|
||||
req = urllib.request.Request(src, headers={'Range': 'bytes={}-'.format(start_byte)})
|
||||
req = urllib.request.Request(url, headers={'Range': 'bytes={}-'.format(start_byte)})
|
||||
resp = urllib.request.urlopen(req, **get_urlopen_kwargs())
|
||||
|
||||
fsize = None
|
||||
|
|
@ -79,7 +84,7 @@ def download(desc, src, writer, start_byte=0):
|
|||
chunk_size = 64 * 1024
|
||||
|
||||
previous_progress_line = None
|
||||
previous_progress_line_time = 0
|
||||
previous_progress_line_time = 0.0
|
||||
while True:
|
||||
chunk = resp.read(chunk_size)
|
||||
if not chunk:
|
||||
|
|
@ -88,7 +93,7 @@ def download(desc, src, writer, start_byte=0):
|
|||
if not dumb:
|
||||
if fsize is not None:
|
||||
pct = recved * 100.0 / fsize
|
||||
progress_line = "\rDownloading %s: %5.1f%%" % (desc, pct)
|
||||
progress_line = "\rDownloading %s: %5.1f%%" % (description, pct)
|
||||
now = time.time()
|
||||
duration = now - previous_progress_line_time
|
||||
if progress_line != previous_progress_line and duration > .1:
|
||||
|
|
@ -102,13 +107,13 @@ def download(desc, src, writer, start_byte=0):
|
|||
if not dumb:
|
||||
print()
|
||||
except urllib.error.HTTPError as e:
|
||||
print("Download failed ({}): {} - {}".format(e.code, e.reason, src))
|
||||
print("Download failed ({}): {} - {}".format(e.code, e.reason, url))
|
||||
if e.code == 403:
|
||||
print("No Rust compiler binary available for this platform. "
|
||||
"Please see https://github.com/servo/servo/#prerequisites")
|
||||
sys.exit(1)
|
||||
except urllib.error.URLError as e:
|
||||
print("Error downloading {}: {}. The failing URL was: {}".format(desc, e.reason, src))
|
||||
print("Error downloading {}: {}. The failing URL was: {}".format(description, e.reason, url))
|
||||
sys.exit(1)
|
||||
except socket_error as e:
|
||||
print("Looks like there's a connectivity issue, check your Internet connection. {}".format(e))
|
||||
|
|
@ -118,22 +123,22 @@ def download(desc, src, writer, start_byte=0):
|
|||
raise
|
||||
|
||||
|
||||
def download_bytes(desc, src):
|
||||
def download_bytes(description: str, url: str):
|
||||
content_writer = BytesIO()
|
||||
download(desc, src, content_writer)
|
||||
download(description, url, content_writer)
|
||||
return content_writer.getvalue()
|
||||
|
||||
|
||||
def download_file(desc, src, dst):
|
||||
tmp_path = dst + ".part"
|
||||
def download_file(description: str, url: str, destination_path: str):
|
||||
tmp_path = destination_path + ".part"
|
||||
try:
|
||||
start_byte = os.path.getsize(tmp_path)
|
||||
with open(tmp_path, 'ab') as fd:
|
||||
download(desc, src, fd, start_byte=start_byte)
|
||||
download(description, url, fd, start_byte=start_byte)
|
||||
except os.error:
|
||||
with open(tmp_path, 'wb') as fd:
|
||||
download(desc, src, fd)
|
||||
os.rename(tmp_path, dst)
|
||||
download(description, url, fd)
|
||||
os.rename(tmp_path, destination_path)
|
||||
|
||||
|
||||
# https://stackoverflow.com/questions/39296101/python-zipfile-removes-execute-permissions-from-binaries
|
||||
|
|
@ -198,3 +203,30 @@ def check_hash(filename, expected, algorithm):
|
|||
|
||||
def get_default_cache_dir(topdir):
|
||||
return os.environ.get("SERVO_CACHE_DIR", os.path.join(topdir, ".servo"))
|
||||
|
||||
|
||||
def append_paths_to_env(env: Dict[str, str], key: str, paths: Union[str, List[str]]):
|
||||
if isinstance(paths, list):
|
||||
paths = os.pathsep.join(paths)
|
||||
|
||||
existing_value = env.get(key, None)
|
||||
if existing_value:
|
||||
new_value = six.ensure_str(existing_value) + os.pathsep + paths
|
||||
else:
|
||||
new_value = paths
|
||||
env[key] = new_value
|
||||
|
||||
|
||||
def prepend_paths_to_env(env: Dict[str, str], key: str, paths: Union[str, List[str]]):
|
||||
if isinstance(paths, list):
|
||||
paths = os.pathsep.join(paths)
|
||||
|
||||
existing_value = env.get(key, None)
|
||||
new_value = paths
|
||||
if existing_value:
|
||||
new_value += os.pathsep + six.ensure_str(existing_value)
|
||||
env[key] = new_value
|
||||
|
||||
|
||||
def get_target_dir():
|
||||
return os.environ.get("CARGO_TARGET_DIR", os.path.join(SERVO_ROOT, "target"))
|
||||
|
|
|
|||
|
|
@ -116,7 +116,6 @@ directories = [
|
|||
# Upstream
|
||||
"./support/android/apk",
|
||||
"./support/hololens",
|
||||
"./support/linux/gstreamer",
|
||||
"./tests/wpt/harness",
|
||||
"./tests/wpt/web-platform-tests",
|
||||
"./tests/wpt/mozilla/tests/mozilla/referrer-policy",
|
||||
|
|
|
|||
2
support/linux/gstreamer/.gitignore
vendored
2
support/linux/gstreamer/.gitignore
vendored
|
|
@ -1,2 +0,0 @@
|
|||
gst/
|
||||
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
#!/usr/bin/env bash
|
||||
|
||||
# This Source Code Form is subject to the terms of the Mozilla Public
|
||||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at https://mozilla.org/MPL/2.0/.
|
||||
|
||||
set -o errexit
|
||||
|
||||
curl -L https://github.com/servo/servo-build-deps/releases/download/linux/gstreamer-1.16-x86_64-linux-gnu.20190515.tar.gz | tar xz
|
||||
sed -i "s;prefix=/opt/gst;prefix=$PWD/gst;g" $PWD/gst/lib/pkgconfig/*.pc
|
||||
Loading…
Add table
Add a link
Reference in a new issue