Commit graph

1972 commits

Author SHA1 Message Date
Martin Robinson
0c8e74dcec
mach: Don't print anything when deps are up-to-date (#31128)
I added this print statement, but after using `mach` for a while with
it, I think it is far too chatty. This change just removes the line.
Information is only printed when dependencies are installed.
2024-01-18 19:05:41 +00:00
Martin Robinson
c641c589e5
build: Use --keep-going in ./mach doc (#31119)
The documentation build for `gstreamer-gl-x11` depends on nightly Rust
for some reason, which breaks our doc build. Use `--keep-going` to
prevent breakage in dependencies from breaking our doc build. This
unfortunately hides issues in our own documentation build.

NB: It isn't possible to exclude dependencies with `--exclude` unless
you are using `--workspace`. We would probably like to do that, but we
have crates with the same name, preventing using `--workspace`.
2024-01-18 16:12:04 +00:00
Martin Robinson
d86e713a9c
build: Clean up post-build copy of Windows DLLs (#31092)
* build: Clean up post-build copy of Windows DLLs

- No longer use vcvarsall.bat at all. Instead find the Windows SDK
  directory by looking in the registry.
- Split logic for copying Windows dependencies into its own function and
  do some minor clean up, such as collecting all MSVC functionality into
  visual_studio.py.
- Remove support for Visual Studio 2015 and Visual Studio 2017.

This is a preparatory change in order to support Visual Studio 2022.

* More cleanup of the code
2024-01-17 10:53:34 +00:00
Martin Robinson
d43adb1a92
build: Only set PATH for GStreamer (#31078)
The documentation claims that this is the only environment variable that
needs to be set before build.
2024-01-17 09:11:04 +00:00
Martin Robinson
25a9f4560e
bootstrap: Improve pip dependency resolution (#31026)
We can now use the "new" pip resolver which should prevent the
installation of conflicting packages. Also, take this opportunity to
make bootstrap more resilient. Hash all dependencies to detect
situations where a newer marker file has been installed, but for an
older branch. This should ensure that dependencies are up to date even
when switching back and forth between older and new branches.

This also updates some dependencies to be the same as the ones used for
WPT tests, which is an issue caught be the resolver.

Fixes #10611.
2024-01-15 13:16:42 +00:00
Martin Robinson
0d240b8713
deps: Raise the Python requirement to 3.10 (#31088)
This will allow us to use the pip's new dependency resolver.
2024-01-15 11:56:15 +00:00
Martin Robinson
3c1ab65458
crown: Add a cargo config.toml file (#31090)
This makes it more foolproof to install crown from inside the Servo
directory, because the root Servo config.toml overrides the rustc to use
crown (an obvious circular dependency).
2024-01-15 11:18:48 +00:00
Martin Robinson
abb54f7832
build: Don't add the neon feature for arm and aarch64 (#31075)
This was enabled to allow using the simd / std::simd / packed-simd crate
in the glyph cache [1][2]. Support for simd in the gfx crate was removed
though [3], so this flag is not really doing anything -- and the Android
build is currently broken. Plus, it's unclear what target features we
can enable using stable Rust. We can explore adding neon support when
Android is working again.

This is part of a long-term effort to remove build complication and make
it so that `cargo build` is equivalent to `./mach build`.

1. https://github.com/servo/servo/pull/10916
2. https://github.com/servo/servo/pull/10900
3. https://github.com/servo/servo/pull/24304
2024-01-12 14:41:14 +00:00
Martin Robinson
54dd8becc1
build: Combine and simplify GStreamer shared object lists (#31038)
Add documentation and simplify the way that GStreamer shared objects
lists are stored. In addition, move the list of extra GStreamer DLL
dependencies to to the `gstreamer.py` file.

The conditional plugin logic is no longer required as we are having
already increased our GStreamer version requirements.
2024-01-09 14:27:14 +00:00
Martin Robinson
79a0f76d26
Fix the build on Ubuntu 20.04 (#31019)
Ubuntu 20.04 doesn't have a new enough version of GStreamer, so
automatically disable media when running on that platform.

This also cleans up the media detection a bit, putting the result in a
`enable-media` variable and moving some of the logic into the build
scripts themselves rather than the platform module.
2024-01-07 21:25:22 +00:00
Martin Robinson
9a1d7aabd7
bootstrap: Adding more output when installing dependencies (#31003)
It's often the case (especially with the taplo installation and on
Windows) that bootstrap is doing lots of stuff in the background for a
long amount of time. Without output it's hard to tell what exactly is
going on. This change adds more output to this process as well as
removing some Pythong 2.x era code.
2024-01-05 11:14:17 +00:00
Martin Robinson
7fa4ea9740
Upgrade media / GStreamer / GLib (#30750)
- Upgrade the version of GStreamer for Windows

This upgrades the Windows build to use the most recent version of
GStreamer. This is necessary to upgrade our GStreamer dependency.

- Stop shipping GStreamer binaries on Linux

The binary bundle of GStreamer that we package is not used to compile --
only to run layout tests. It's too old for the APIs that we are using
(as evidenced by needed 1.18 for WebRTC) and nowadays Linux
distributions carry a new version so it's unecessary for our build
machines. No longer using this binary bundle will allow us to upgrade
our GStreamer dependency -- which now has stricter checks that we
are using at least version 1.18.

- Upgrade media to use newer versions of GStreamer / GLib dependencies
2024-01-05 08:01:58 +00:00
Martin Robinson
0dbba9b6ff
Download Windows GStreamer dependencies from GitHub (#30973)
This works around issues where the server is rejecting our requests and
should be more reliable.
2024-01-03 13:06:08 +00:00
Martin Robinson
516cc2cbca
tidy: A few small improvements and fixes (#30941)
1. Make the tidy output easier to follow
2. Integrate the WPT manifest cleanliness step into tidy
   itself and don't run it if nothing has changed in the WPT
   directory.
3. Fix an issue where Python test requirements were not installed,
   which could cause issues with some modules not being found.

Fixes #30002.
2024-01-02 06:14:51 +00:00
Samson
8d121a0a1d
Update requirements.txt (#30940) 2023-12-26 09:52:14 +00:00
Taym Haddadi
554b35b705
Improve formatting of Python files (#30919)
Signed-off-by: Bentaimia Haddadi <haddadi.taym@gmail.com>
2023-12-24 23:58:37 +00:00
Delan Azabani
0be30b30ce
Fix the WPT exporter (#30870)
* Fix the WPT exporter

* apply fixes by @mrobinson

* fix mach test-scripts on NixOS

* rename main_branch_name to default_branch
2023-12-15 11:05:00 +00:00
Martin Robinson
569c4a8823
Integrate hyper_serde into Servo source (#30803)
hyper_serde changes very infrequently and typically only in order to
upgrade Servo dependencies. crates.io lists hyper_serde as having no
dependents, and its integration will not prevent releasing new versions.
Integrating it will both reduce the number of repositories we have to
maintain as well as making it easier to upgrade Servo dependencies.
2023-12-14 11:43:10 +00:00
Samson
1105eb66e9
Use os version in taplo cache-key & setuptools 65 in py3.8 (#30848) 2023-12-11 15:06:43 +00:00
Corey Farwell
117d59d393
Replace virtualenv with Python's built-in venv (#30377)
* Replace virtualenv with Python's built-in venv.

* Apply Delan's suggestions and make a couple small fixes

- Fix a tidy warning about directories that don't exist
- Use shutil instead of the redundant get_exec_path
- Miscellaneous cleanups

* Fix typo in environment variable

* fix bug where pip still tries to the wrong site-packages

---------

Co-authored-by: Martin Robinson <mrobinson@igalia.com>
Co-authored-by: Delan Azabani <dazabani@igalia.com>
2023-12-07 08:18:30 +00:00
Martin Robinson
7e82c5c957
Compile Servo with the latest version of rust stable (#30831)
This completes the transition to compiling Servo with rust stable. Some
nightly-only features are still used when compiling the `script` and
`crown` crates, as well as for some style unit tests. These will likely
break with newer compiler versions, but `crown` can be disabled for them
conditionally. This is just the first step.

This has some caveats:

1. We need to disable setting up the special linker on Linux. The -Z
   option isn't supported with stable rust so using this is out --
   meanwhile we can't be sure that lld is installed on most systems.
2. `cargo fmt` still uses some unstable options, so we need to rely on
   the unstable toolchain just for running `fmt`. The idea is to fix this
   gradually.
2023-12-06 17:36:07 +00:00
Martin Robinson
bbc35b682f
Remove thinlto servobuild.config option (#30830)
Rust uses a version of ThinLTO by default now [1]. This can be tweaked
by adjusting rust compiler flags, which is probably a better way of
controlling this than a custom servo configuration considering:

1. We want to remove the custom servo configuration eventually.
2. The -Z option that this configuration currently uses is unsupported
   by stable rust.

1. https://blog.rust-lang.org/inside-rust/2020/06/29/lto-improvements.html
2023-12-06 11:06:36 +00:00
Martin Robinson
23add0c1e5 Rename the master branch to main 2023-12-04 10:11:56 +01:00
Samson
604979e367
Replace script_plugins with a clippy like rustc driver (named crown) (#30508)
* Remove script_plugins

* Use crown instead of script_plugins

* crown_is_not_used

* Use crown in command base

* bootstrap crown

* tidy happy

* disable sccache

* Bring crown in tree

* Install crown from tree

* fix windows ci

* fix warning

* fix mac

libscript_plugins.dylib is not available anymore

* Update components/script/lib.rs

Co-authored-by: Martin Robinson <mrobinson@igalia.com>

* Update for nightly-2023-03-18

Mostly just based off https://github.com/servo/servo/pull/30630

* Always install crown

it's slow only when there is new version

* Run crown test with `mach test-unit`

* Small fixups; better trace_in_no_trace tests

* Better doc

* crown in config.toml

* Fix tidy for real

* no sccache on rustc_wrapper

* document rustc overrides

* fixup of compiletest

* Make a few minor comment adjustments

* Fix a typo in python/servo/platform/base.py

Co-authored-by: Samson <16504129+sagudev@users.noreply.github.com>

* Proper test types

* Ignore tidy on crown/tests

---------

Co-authored-by: Martin Robinson <mrobinson@igalia.com>
2023-12-01 15:50:52 +00:00
season
53b0fa827d
Fix build script error.because starting with Python 3.9, the util module was moved from importlib to a separate top-level module called importlib.util. Therefore, if you are using Python 3.9 or later and you receive an AttributeError stating that module 'importlib' has no attribute 'util', it is likely due to a compatibility issue with your code. You can fix this issue by updating your code to import importlib.util directly (#30796) 2023-11-29 16:45:51 +00:00
Samson
e620381736
Pin pyopenssl (#30728)
* Update requirements.txt

* Update requirements.txt

* Update python/requirements.txt

Co-authored-by: Martin Robinson <mrobinson@igalia.com>

---------

Co-authored-by: Martin Robinson <mrobinson@igalia.com>
2023-11-13 13:17:08 +00:00
Martin Robinson
e44ea6290a
bootstrap: Alphabetize and add 'm4' Ubuntu dependency (#30727)
It looks like `m4` is now required for builds on Debian-like systems.
2023-11-13 12:06:26 +00:00
Samson
96d37d3785
Add --production option to mach (#30707)
* --prod(uction) mach argument

* Use profile in workflows instead of production

* Use profiles in unit tests

* ups

* Build (${{ inputs.profile }})
2023-11-10 10:38:33 +00:00
Martin Robinson
c529bfa49e
Remove some stray references to OpenSSL (#30710)
These were left over from the change to use rustls.
2023-11-09 09:17:19 +00:00
Martin Robinson
8fd370784e
Update mozjs and setuptools (#30685)
This should fix issues with Python 3.12 on Mac builders.

Co-authored-by: Samson <16504129+sagudev@users.noreply.github.com>
2023-11-02 16:55:12 +00:00
Martin Robinson
f5627ce4cc
Remove __future__ imports that are no longer necessary (#30661)
These are no longer necessary as we always use Python 3.
2023-11-02 12:01:10 +00:00
Martin Robinson
af68769007
Remove an unused file (#30662)
This was probably used for the now removed homebrew package.
2023-11-01 20:01:01 +00:00
Martin Robinson
9d9c5d3ca9
Vendor mozdebug and stop using distutils (#30659)
The version of `mozdebug` installable via pip is two years old and
Mozilla is slow to release new versions. It also uses `distutils` which
doesn't work on newer Python versions. Vendor it and stop using
`distutils` there.
2023-10-31 09:42:14 +00:00
Mukilan Thiyagarajan
117cc1da6c
mach: use importlib module instead of imp (#30645)
* mach: use `importlib` module instead of `imp`

`imp` module has been deprecated since python 3.4
and has been removed in 3.12. The recommended alternative
is to use the `importlib` module that was introduced in
python 3.1

This is required to fix the CI failures in macos builds
since GitHub runner images for macos-13 now use python 3.12

Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>

* mach: use `importlib` module instead of `imp`

`imp` module has been deprecated since python 3.4
and has been removed in 3.12. The recommended alternative
is to use the `importlib` module that was introduced in
python 3.1

This is required to fix the CI failures in macos builds
since GitHub runner images for macos-13 now use python 3.12

Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>

---------

Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>
2023-10-30 07:53:20 +00:00
Martin Robinson
463c61f34d
Stop using distutils (#30638)
This is removed from Python 3.12 so we can no longer rely on it. All of
this functionality is either in `setuptools` or `shutil`.
2023-10-28 17:17:57 +00:00
Mukilan Thiyagarajan
6dd359193f
Fix upload-nightly to use correct path (#30632)
Nightly builds now use production profile and thus will be
available under target/production
2023-10-28 07:10:28 +00:00
Delan Azabani
a3d2f0c586
Enable debug assertions for all builds other than official releases (#30509)
* Run main and try jobs with debug assertions

* use single quotes in workflow expressions

* set force-debug-assertions in main.yml

* set force-debug-assertions as part of decision job

* fix typo in MachCommands.build

* fix more hardcoded profile names

* fix tidy

* split cargo_profile_option on windows

* Fix running servoshell and unit tests through a symlink

* rename steps to make them less confusing

* fix more hardcoded cargo profile options

* fix missing inputs in linux-wpt and mac-wpt

* make filename an inherent method of Resource

* rework release-with-debug-assertions profile to production profile

* rework resource logic to eliminate std_test_override

* set production flag in nightly release builds

* clean up servobuild.example and windows.yml

* oops forgot to check in embedder_traits/build.rs

* fix mach test-unit behaviour through symlink

* unit tests only need current_dir and ancestors

* fix macOS package smoketest breakage

* expect css/css-color/currentcolor-003 to crash under layout 2013

* fix more references to {force,release-with}-debug-assertions

* fix local build failures under --profile production
2023-10-26 08:22:14 +00:00
Servo WPT Sync
65beca94cd
Sync WPT with upstream (24-10-2023) (#30607)
* wpt sync: adapt code for new TestRoot class

Fixes #30558

Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>

* temp change to allow wpt sync build to pass

Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>

* Update web-platform-tests to revision b'0d5028f200c8651b17bb224657d6e1065adcff37'

* Revert "temp change to allow wpt sync build to pass"

This reverts commit bad72c7f87.

* revert metadata update for intermittent timeouts

Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>

---------

Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>
Co-authored-by: Mukilan Thiyagarajan <mukilan@igalia.com>
2023-10-25 07:26:53 +00:00
Delan Azabani
aabae55407
Fix filtered test counts in WPT summaries (#30591) 2023-10-20 07:20:19 +00:00
bc-universe
9d10d230ac
Add support for EndeavourOS (#30513) 2023-10-06 20:58:06 +00:00
Martin Robinson
aadf48bd4d
Bootstrap pkg-config and cmake on MacOS (#30497)
These need to be installed in order to build so we can install them via
Homebrew. Do this by simply restoring the Homebrew bootstrapping logic
we had in place previously.

Fixes #27171.
2023-10-05 01:12:29 +00:00
Mukilan Thiyagarajan
330f59a711
Fix WPT import by overriding product name to servo (#30457)
Upstream wpt script has bug where 'product' defaults to
'firefox' and this causes import of uninstalled python
modules specific to firefox runners.

Fixes #30452

Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>
2023-10-04 17:50:47 +00:00
Martin Robinson
f78d53daaa
Adjust the missing taplo message (#30473)
The current message doesn't have a line break where you would expect it
to and instead has a gap due to the indentation of the code in Python.
2023-10-03 10:23:02 +00:00
Samson
d74d51190a
Update WebGPU CTS (#30454)
* Fix webgpu-cts import script

* Update webgpu-cts to f2b59e0362

* Updated expectations
2023-09-30 16:28:46 +00:00
Delan Azabani
5d8d7c79c2
fix WebGPU runtime dependencies on linux (#30407) 2023-09-22 11:05:38 +00:00
Samson
7caac9790d
Enforce formatting of TOML files (#30128)
* Fmt all toml files

* bootstrap taplo

* enforce toml formatting with taplo

* Install taplo in CI using cargo-install action
2023-09-19 08:57:27 +00:00
Carlos Bentzen
0b86d65798
Add TUXEDO OS to mach bootstrap (#30363)
Signed-off-by: Carlos Bentzen <cadubentzen@gmail.com>
2023-09-15 04:20:32 +00:00
Samson
59d134133f
Use MozTools 4 and update mozjs (#30326)
* Update mozjs

* moztools4 in bootstrap

* no autoconf

* tidy

* switch to servo-build-deps

* update mozjs for real

* glue mozjs

* fmt

* move to servo/mozjs
2023-09-11 08:23:05 +00:00
Samson
9b1247b20f
No extern crate test/tidy & better RUSTFLAGS handling in mach (#30328)
* Remove unused extern crate rustflag

* Remove tidy check alphabetical order of extern crates

rustfmt's reorder_imports already does that

* fix

* better RUSTFLAGS handling
2023-09-10 18:32:10 +00:00
Atbrakhi
3df284cf54
Move user input logic into servoshell (#30238)
* cleanup and move user input logix into servoshell

* fix fmt

* moves test from servoshell file

* move command-line args into servoshell

* remove feature media-gstreamer

* fix fmt

* move user input logic code into lib to make it more testable

* remove opts_matches in fn instead get it from main2

* remove pub and fix import

* add licence in new file

* revert passing Matches, instead pass Option String

* review update, also move sanitize fn to parser file

* fmt fix

* review fix: remove extra line
2023-09-06 11:45:56 +00:00