Tungestenite upgrade
This updates our async-tungstenite/tungstenite dependencies to slightly newer versions that still rely on tokio 0.2 and should make #28661 easier.
docs(linux): added runtime dependencies
Closes#28657
<!-- Please describe your changes on the following line: -->
added info about runtime dependencies needed for linux
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix#28657 (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [X] These changes do not require tests because it's a README change
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Non-blocking network IO
The current networking strategy uses a fixed sized thread pool to perform blocking requests on a per call basis. This is inefficient, but can be ~easily~ improved by utilizing the existing tokio runtime instead to submit async networking tasks to its executor. However, since servo is currently using an outdated version of the `hyper` http library (`0.12`) we must use the [`tokio_compat`](https://github.com/tokio-rs/tokio-compat) and [`futures_compat`](https://docs.rs/futures/0.3.1/futures/compat/index.html) libraries to integrate with the older version of [`Future` used in `hyper`](https://docs.rs/hyper/0.12.1/hyper/rt/trait.Future.html).
~**NOTE**: This PR is just proof of concept at the moment. In addition to test failures, it appears that large javascript downloads are silently failing to stream entire payloads, and occasionally getting cutoff.~
Tests are passing.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix [#22813](https://github.com/servo/servo/issues/22813) (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Allow bootstrap on Pop!_OS 21.10
<!-- Please describe your changes on the following line: -->
bootstrap.py doesn't support Pop!_OS 21.x yet. Adding these lines should make the script works again.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
<!-- Either: -->
- [x] These changes do not require tests because it seems there no test requires certain version check and test-tidy has passed.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Update more deprecated python imports.
`mach` still uses some deprecated imports that were removed in Python 3.10, and currently fails when using Python 3.10.
This is in the same line as #28649.
concept-response-clone: Ensure header guard is cloned after headers
https://fetch.spec.whatwg.org/#concept-response-clone
If the header guard of the response to clone is `immutable`, then copying the headers to the new response will fail with `Guard is immutable` unless we ensure the guard is copied _after_ the headers.
8650794391/components/script/dom/response.rs (L331-L334)8650794391/components/script/dom/headers.rs (L202-L210)8650794391/components/script/dom/headers.rs (L73)
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
https://fetch.spec.whatwg.org/#concept-response-clone
If the header guard of the response to clone is `immutable`, then copying the headers to the new response will fail with `Guard is immutable` unless we ensure the guard is copied _after_ the headers.
8650794391/components/script/dom/response.rs (L331-L334)
fn Append(&self, name: ByteString, value: ByteString) -> ErrorResult {
// Step 1
let value = normalize_value(value);
// Step 2
let (mut valid_name, valid_value) = validate_name_and_value(name, value)?;
valid_name = valid_name.to_lowercase();
// Step 3
if self.guard.get() == Guard::Immutable {
return Err(Error::Type("Guard is immutable".to_string()));
}
Allow builds / tests on Ubuntu 21.10
<!-- Please describe your changes on the following line: -->
This PR updates the version check in `python/servo/bootstrap.py` for Ubuntu to `> 21.10` to accommodate the latest version. It also updates the `clang-format` check in `test_commands.py` to use `find()` instead of `startswith()` to account for possible distro-specific insertions into the `clang-format --version` command.
Ubuntu, for example, outputs `Ubuntu clang-format version 13.0.0-2`, which caused `version.startswith()` to throw an error as it looked for `clang-format` at the beginning of the string. This would cause `./mach test-tidy` to skip the CPP formatting portion.
These two changes should help run bootstrap and test commands on the latest Ubuntu version.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
<!-- Either: -->
- [X] These changes do not require tests because: I do not believe there are any current tests for the version checks (as the version checks are in-and-of themselves a test before running the bootstrap/test commands)
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Perform non-null checks on pointers used in js stacktrace captures
Need to ensure that external passed in pointers are non-null prior to using them unchecked. Seeing sporadic crashes when visiting `https:://cnn.com` and `https://www.bankofamerica.com`:
```
...
6: __sigtramp
7: core::ptr::non_null::NonNull<T>::as_ref
at /rustc/0fa3190394475a84360b34e074e719d519bc40f1/library/core/src/ptr/non_null.rs:317:20
8: alloc::rc::Rc<T>::inner
at /rustc/0fa3190394475a84360b34e074e719d519bc40f1/library/alloc/src/rc.rs:332:18
<alloc::rc::Rc<T> as core::clone::Clone>::clone
at /rustc/0fa3190394475a84360b34e074e719d519bc40f1/library/alloc/src/rc.rs:1479:9
9: <servo_url::origin::MutableOrigin as core::clone::Clone>::clone
at /Users/navgattu/Documents/dev-git/servo/components/url/origin.rs:92:26
10: script::dom::bindings::principals::ServoJSPrincipals::origin
at /Users/navgattu/Documents/dev-git/servo/components/script/dom/bindings/principals.rs:42:9
11: script::dom::bindings::principals::subsumes
at /Users/navgattu/Documents/dev-git/servo/components/script/dom/bindings/principals.rs:136:22
12: _ZN2jsL30SavedFrameSubsumedByPrincipalsEP9JSContextP12JSPrincipalsN2JS6HandleIPNS_10SavedFrameEEE
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/SavedStacks.cpp:617:10
...
```
d384ff7930/library/core/src/ptr/non_null.rs (L181-L188)
Full stacktrace:
[trace.txt](https://github.com/servo/servo/files/7631210/trace.txt)
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Do not use ParallelIterator if not using rayon
The rayon [`ParalllelIIterator`](https://github.com/rayon-rs/rayon/blob/master/src/iter/plumbing/README.md) is being used for inline content size calculations irrespective of the `use_rayon` flag, which can lead to an unbounded (dynamically determined) number of threads being spawned during iteration.
The check is performed for static position calculations, however:
6fced22e47/components/layout_2020/flow/mod.rs (L235-L242)
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Need to ensure the passed in pointers are non-null prior to using them unchecked. Seeing sporadic crashes:
```
Stack trace for thread "Script(2,36)"
0: backtrace::backtrace::libunwind::trace
at /Users/navgattu/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.63/src/backtrace/libunwind.rs:93:5
backtrace::backtrace::trace_unsynchronized
at /Users/navgattu/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.63/src/backtrace/mod.rs:66:5
1: <servo::backtrace::Print as core::fmt::Debug>::fmt
at /Users/navgattu/Documents/dev-git/servo/ports/winit/backtrace.rs:53:13
2: core::fmt::write
at /rustc/0fa3190394475a84360b34e074e719d519bc40f1/library/core/src/fmt/mod.rs:1117:17
3: std::io::Write::write_fmt
at /rustc/0fa3190394475a84360b34e074e719d519bc40f1/library/std/src/io/mod.rs:1667:15
4: servo::backtrace::print
at /Users/navgattu/Documents/dev-git/servo/ports/winit/backtrace.rs:17:5
5: servo::crash_handler::install::handler
at /Users/navgattu/Documents/dev-git/servo/ports/winit/crash_handler.rs:25:21
6: __sigtramp
7: core::ptr::non_null::NonNull<T>::as_ref
at /rustc/0fa3190394475a84360b34e074e719d519bc40f1/library/core/src/ptr/non_null.rs:317:20
8: alloc::rc::Rc<T>::inner
at /rustc/0fa3190394475a84360b34e074e719d519bc40f1/library/alloc/src/rc.rs:332:18
<alloc::rc::Rc<T> as core::clone::Clone>::clone
at /rustc/0fa3190394475a84360b34e074e719d519bc40f1/library/alloc/src/rc.rs:1479:9
9: <servo_url::origin::MutableOrigin as core::clone::Clone>::clone
at /Users/navgattu/Documents/dev-git/servo/components/url/origin.rs:92:26
10: script::dom::bindings::principals::ServoJSPrincipals::origin
at /Users/navgattu/Documents/dev-git/servo/components/script/dom/bindings/principals.rs:42:9
11: script::dom::bindings::principals::subsumes
at /Users/navgattu/Documents/dev-git/servo/components/script/dom/bindings/principals.rs:136:22
12: _ZN2jsL30SavedFrameSubsumedByPrincipalsEP9JSContextP12JSPrincipalsN2JS6HandleIPNS_10SavedFrameEEE
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/SavedStacks.cpp:617:10
_ZN2jsL20GetFirstMatchedFrameIFbP9JSContextP12JSPrincipalsN2JS6HandleIPNS_10SavedFrameEEEEEES8_S2_S4_RT_S9_NS5_20SavedFrameSelfHostedERb
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/SavedStacks.cpp:636:9
_ZN2jsL21GetFirstSubsumedFrameEP9JSContextP12JSPrincipalsN2JS6HandleIPNS_10SavedFrameEEENS4_20SavedFrameSelfHostedERb
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/SavedStacks.cpp:660:10
13: _ZN2js16UnwrapSavedFrameEP9JSContextP12JSPrincipalsN2JS6HandleIP8JSObjectEENS4_20SavedFrameSelfHostedERb
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/SavedStacks.cpp:744:10
14: _ZN2JS16BuildStackStringEP9JSContextP12JSPrincipalsNS_6HandleIP8JSObjectEENS_13MutableHandleIP8JSStringEEmN2js11StackFormatE
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/SavedStacks.cpp:1041:13
15: mozjs::rust::CapturedJSStack::as_string
at /Users/navgattu/.cargo/git/checkouts/rust-mozjs-8611526964119dd6/09edacd/src/rust.rs:1377:17
16: script::dom::webglrenderingcontext::capture_webgl_backtrace::{{closure}}
at /Users/navgattu/Documents/dev-git/servo/components/script/dom/webglrenderingcontext.rs:1939:46
17: core::option::Option<T>::and_then
at /rustc/0fa3190394475a84360b34e074e719d519bc40f1/library/core/src/option.rs:1043:24
18: script::dom::webglrenderingcontext::capture_webgl_backtrace
at /Users/navgattu/Documents/dev-git/servo/components/script/dom/webglrenderingcontext.rs:1939:27
19: script::dom::webglrenderingcontext::WebGLRenderingContext::send_command
at /Users/navgattu/Documents/dev-git/servo/components/script/dom/webglrenderingcontext.rs:397:28
20: script::dom::webglrenderingcontext::WebGLRenderingContext::get_gl_extensions
at /Users/navgattu/Documents/dev-git/servo/components/script/dom/webglrenderingcontext.rs:913:9
21: <script::dom::webglrenderingcontext::WebGLRenderingContext as script::dom::bindings::codegen::Bindings::WebGLRenderingContextBinding::WebGLRenderingContextBinding::WebGLRenderingContextMethods>::GetSupportedExtensions::{{closure}}
at /Users/navgattu/Documents/dev-git/servo/components/script/dom/webglrenderingcontext.rs:2330:27
22: script::dom::webgl_extensions::extensions::WebGLExtensions::init_once
23: <script::dom::webglrenderingcontext::WebGLRenderingContext as script::dom::bindings::codegen::Bindings::WebGLRenderingContextBinding::WebGLRenderingContextBinding::WebGLRenderingContextMethods>::GetSupportedExtensions
at /Users/navgattu/Documents/dev-git/servo/components/script/dom/webglrenderingcontext.rs:2329:9
24: script::dom::bindings::codegen::Bindings::WebGLRenderingContextBinding::WebGLRenderingContextBinding::getSupportedExtensions::{{closure}}::{{closure}}
at /Users/navgattu/Documents/dev-git/servo/target/debug/build/script-80d43fa6e481c605/out/Bindings/WebGLRenderingContextBinding.rs:1095:46
25: script::dom::bindings::codegen::Bindings::WebGLRenderingContextBinding::WebGLRenderingContextBinding::getSupportedExtensions::{{closure}}
at /Users/navgattu/Documents/dev-git/servo/target/debug/build/script-80d43fa6e481c605/out/Bindings/WebGLRenderingContextBinding.rs:1090:33
26: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &mut F>::call_once
at /rustc/0fa3190394475a84360b34e074e719d519bc40f1/library/core/src/ops/function.rs:280:13
27: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
at /rustc/0fa3190394475a84360b34e074e719d519bc40f1/library/core/src/panic/unwind_safe.rs:271:9
28: std::panicking::try::do_call
at /rustc/0fa3190394475a84360b34e074e719d519bc40f1/library/std/src/panicking.rs:403:40
29: <unknown>
at /Users/navgattu/.cargo/git/checkouts/rust-mozjs-8611526964119dd6/09edacd/src/glue.rs:299:6
30: std::panicking::try
at /rustc/0fa3190394475a84360b34e074e719d519bc40f1/library/std/src/panicking.rs:367:19
31: std::panic::catch_unwind
at /rustc/0fa3190394475a84360b34e074e719d519bc40f1/library/std/src/panic.rs:129:14
32: mozjs::panic::wrap_panic
at /Users/navgattu/.cargo/git/checkouts/rust-mozjs-8611526964119dd6/09edacd/src/panic.rs:22:11
33: script::dom::bindings::codegen::Bindings::WebGLRenderingContextBinding::WebGLRenderingContextBinding::getSupportedExtensions
at /Users/navgattu/Documents/dev-git/servo/target/debug/build/script-80d43fa6e481c605/out/Bindings/WebGLRenderingContextBinding.rs:1090:5
34: CallJitMethodOp
at /Users/navgattu/.cargo/git/checkouts/rust-mozjs-8611526964119dd6/09edacd/src/jsglue.cpp:663:12
35: script::dom::bindings::utils::generic_call
36: script::dom::bindings::utils::generic_method
37: _Z12CallJSNativeP9JSContextPFbS0_jPN2JS5ValueEEN2js10CallReasonERKNS1_8CallArgsE
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/Interpreter.cpp:435:13
_ZN2js23InternalCallOrConstructEP9JSContextRKN2JS8CallArgsENS_14MaybeConstructENS_10CallReasonE
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/Interpreter.cpp:520:12
38: _ZN2js13CallFromStackEP9JSContextRKN2JS8CallArgsE
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/Interpreter.cpp:584:10
_ZL9InterpretP9JSContextRN2js8RunStateE
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/Interpreter.cpp:3244:16
39: _ZN2js9RunScriptEP9JSContextRNS_8RunStateE
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/Interpreter.cpp:405:13
40: _ZN2js23InternalCallOrConstructEP9JSContextRKN2JS8CallArgsENS_14MaybeConstructENS_10CallReasonE
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/Interpreter.cpp:552:13
41: _ZN2js4CallEP9JSContextN2JS6HandleINS2_5ValueEEES5_RKNS_13AnyInvokeArgsENS2_13MutableHandleIS4_EENS_10CallReasonE
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/Interpreter.cpp:597:8
42: _ZN2js9fun_applyEP9JSContextjPN2JS5ValueE
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/JSFunction.cpp:1166:10
43: _Z12CallJSNativeP9JSContextPFbS0_jPN2JS5ValueEEN2js10CallReasonERKNS1_8CallArgsE
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/Interpreter.cpp:435:13
_ZN2js23InternalCallOrConstructEP9JSContextRKN2JS8CallArgsENS_14MaybeConstructENS_10CallReasonE
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/vm/Interpreter.cpp:520:12
44: _ZN2js3jit14DoCallFallbackEP9JSContextPNS0_13BaselineFrameEPNS0_15ICCall_FallbackEjPN2JS5ValueENS7_13MutableHandleIS8_EE
at /Users/navgattu/Documents/dev-git/mozjs/mozjs/js/src/jit/BaselineIC.cpp:1841:10
```
Preserve URI fragment across redirects
https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2
```
If the Location value provided in a 3xx (Redirection) response does
not have a fragment component, a user agent MUST process the
redirection as if the value inherits the fragment component of the
URI reference used to generate the request target (i.e., the
redirection inherits the original reference's fragment, if any).
```
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Updated cssparser dependency to 0.29.
Signed-off-by: Lewin Probst <info@emirror.de>
This commit updates the cssparser dependency to 0.29. The intention is to get the selectors crate to cssparser 0.29. This will reduce problems in other crates that belong on both, selectors and cssparser.
Would love to get a review and changes required to be able to merge this.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
