Commit graph

224 commits

Author SHA1 Message Date
Simon Wülker
0ebdf146fc
Cleanup blocking fetch operations with bad ports (#35324)
Blocking a fetch due to a bad port should be grouped together
with CSP blocks as per the spec, but these steps were previously
seperated.

Additionally, remove handling of ftp in
should_request_be_blocked_due_to_a_bad_port, since it did nothing
anyways.

Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
2025-02-06 16:47:29 +00:00
Shubham Gupta
1e164738d8
Add support for Upgrade request to a potentially trustworthy URL. (#34986)
* Add support for Upgrade request to a potentially trustworthy URL.

Signed-off-by: Shubham Gupta <shubham13297@gmail.com>

* script: Support inheritable insecure request policy in documents and workers.

Signed-off-by: Josh Matthews <josh@joshmatthews.net>

---------

Signed-off-by: Shubham Gupta <shubham13297@gmail.com>
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
Co-authored-by: Shubham Gupta <shubham.gupta@chromium.org>
Co-authored-by: Josh Matthews <josh@joshmatthews.net>
2025-02-05 12:49:56 +00:00
Delan Azabani
5e9de2cb61
Include WebViewId into EmbedderMsg variants where possible (#35211)
`EmbedderMsg` was previously paired with an implicit
`Option<WebViewId>`, even though almost all variants were either always
`Some` or always `None`, depending on whether there was a `WebView
involved.

This patch adds the `WebViewId` to as many `EmbedderMsg` variants as
possible, so we can call their associated `WebView` delegate methods
without needing to check and unwrap the `Option`. In many cases, this
required more changes to plumb through the `WebViewId`.

Notably, all `Request`s now explicitly need a `WebView` or not, in order
to ensure that it is passed when appropriate.

Signed-off-by: Delan Azabani <dazabani@igalia.com>
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
2025-01-30 11:15:35 +00:00
zhuhaichao518
a1326a7cf6
Implement WebResourceRequested Event. (#34961)
* Implement WebResourceRequested Event on the Embedder Layer

Signed-off-by: zhuhaichao518 <zhuhaichao518@gmail.com>

* fix and add test

Signed-off-by: zhuhaichao518 <zhuhaichao518@gmail.com>

* resolve comments

Signed-off-by: zhuhaichao518 <zhuhaichao518@gmail.com>

* remove sample code in webview

Signed-off-by: zhuhaichao518 <zhuhaichao518@gmail.com>

* remove typo

Signed-off-by: zhuhaichao518 <zhuhaichao518@gmail.com>

* ./mach format

Signed-off-by: zhuhaichao518 <zhuhaichao518@gmail.com>

* fix test fail caused by interception message

Signed-off-by: zhuhaichao518 <zhuhaichao518@gmail.com>

* update impl for is_for_main_frame

Signed-off-by: zhuhaichao518 <zhuhaichao518@gmail.com>

---------

Signed-off-by: zhuhaichao518 <zhuhaichao518@gmail.com>
2025-01-16 07:44:16 +00:00
arthmis
90c5685d61
add minimal implementation of FetchParams (#34833)
* add minimal implementation of fetchParams and fetch controller for cancellation support

fix something

removing fetch params from http network or cache fetch due to implementation difficult

Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>

* run formatter

Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>

* fix incorrect spec implementation and add comments with related step number and description

Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>

* fix double borrow issue

Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>

* remove unused code from FetchParams

Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>

* add workaround for double mutable borrow error

Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>

* remove unnecessary comments, move import and format

Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>

* fix comments that state spec instructions

Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>

* update comment

Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>

* refactor tests

Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>

* refactor tests

Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>

---------

Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
Co-authored-by: lazypassion <25536767+lazypassion@users.noreply.github.com>
2025-01-12 14:37:25 +00:00
Martin Robinson
748954d610
net: Use RequestId to cancel fetches instead of creating an IPC channel (#34883)
Instead of creating an IPC channel for every fetch, allow cancelling
fetches based on the `RequestId` of the original request. This requires
that `RequestId`s be UUIDs so that they are unique between processes
that might communicating with the resource process.

In addition, the resource process loop now keeps a `HashMap` or `Weak`
handles to cancellers and cleans them up.

This allows for creating mutiple `FetchCanceller`s in `script` for a
single fetch request, allowing integration of the media and video
elements to integrate with the `Document` canceller list -- meaning
these fetches also get cancelled when the `Document` unloads.

Signed-off-by: Martin Robinson <mrobinson@igalia.com>
2025-01-11 11:49:22 +00:00
Martin Robinson
fbd77b4524
script: Move navigation fetching to the ScriptThread (#34919)
This allows reusing the asynchrnous fetch mechanism that we use for page
resources and is likely a step toward removing the `FetchThread`.

Benefits:
 - Reduces IPC traffic during navigation. Now instead of bouncing
   between the constellation and the `ScriptThread` responses are sent
   directly to the `ScriptThread`.
 - Allows cancelling loads after redirects, which was not possible
   before.

There is the question of what to do when a redirect is cross-origin
(#23037). This currently isn't handled properly as the `Constellation`
sends data to the same `Pipeline` that initiated the load. This change
doesn't fix this issue, but does make it more possible for the
`ScriptThread` to shut down the pipeline and ask the `Constellation` to
replace it with a new one.

Signed-off-by: Martin Robinson <mrobinson@igalia.com>
2025-01-10 12:19:40 +00:00
Josh Matthews
76e0a1872b
Update all network-related dependencies to the latest versions (#34630)
* Update all network-related dependencies to the latest versions:
* rustls
* hyper
* http
* headers
* tungstenite
* async-tungstenite

Signed-off-by: Josh Matthews <josh@joshmatthews.net>

* net: Fix panics with 1xx responses in WPT tests.

Signed-off-by: Josh Matthews <josh@joshmatthews.net>

* net: Use reported response length when calculating available ranges.

Signed-off-by: Josh Matthews <josh@joshmatthews.net>

* net: Remove unreachable match arm.

Signed-off-by: Josh Matthews <josh@joshmatthews.net>

* net: Clean up commented fragments in blob and file handlers.

Signed-off-by: Josh Matthews <josh@joshmatthews.net>

* net: Remove unreachable match arm.

Signed-off-by: Josh Matthews <josh@joshmatthews.net>

* net: Fix clippy warning.

Signed-off-by: Josh Matthews <josh@joshmatthews.net>

* net: Cleanup.

Signed-off-by: Josh Matthews <josh@joshmatthews.net>

* net: Fix up unit tests for dependency upgrades.

Signed-off-by: Josh Matthews <josh@joshmatthews.net>

* Update aws-lc-sys to fix Windows builds.

Signed-off-by: Josh Matthews <josh@joshmatthews.net>

* net: Use ring instead of aws-lc-sys.

Signed-off-by: Josh Matthews <josh@joshmatthews.net>

* embedding: Require embedder to initialize a rustls CryptoProvider.

Signed-off-by: Josh Matthews <josh@joshmatthews.net>

* Disable aws-lc-rs pending OhOS build fixes.

Signed-off-by: Josh Matthews <josh@joshmatthews.net>

---------

Signed-off-by: Josh Matthews <josh@joshmatthews.net>
2025-01-08 05:47:58 +00:00
Wulan Seruniati Salim
2ab66ce678
Optimize mutex usage in fetch by locking once and using scoped MutexGuard (#34737)
Signed-off-by: Wulan Seruniati Salim <wulanseruniati@gmail.com>
2024-12-22 11:14:57 +00:00
shanehandley
bd37d5496d
Update response handling in main_fetch (#34570)
Signed-off-by: Shane Handley <shanehandley@fastmail.com>
2024-12-11 12:59:29 +00:00
shanehandley
975e2ae859
Remove referrer policy from document (#34263)
* Remove the referrer policy from document and rely on its policy container

Signed-off-by: Shane Handley <shanehandley@fastmail.com>

* Make ReferrerPolicy non-optional, instead using a new enum value to represent the empty string case

Signed-off-by: Shane Handley <shanehandley@fastmail.com>

* Fix clippy issue

Signed-off-by: Shane Handley <shanehandley@fastmail.com>

* Fix usage of Option<ReferrerPolicy> in unit test

Signed-off-by: Shane Handley <shanehandley@fastmail.com>

---------

Signed-off-by: Shane Handley <shanehandley@fastmail.com>
2024-11-19 12:45:10 +00:00
shanehandley
6451767428
Implement PolicyContainer and update the default ReferrerPolicy (#33977)
* Implement PolicyContainer

Signed-off-by: Shane Handley <shanehandley@fastmail.com>

* implement small parts of fetch that interact with policy container

Signed-off-by: Shane Handley <shanehandley@fastmail.com>

* fix: allow policy container's csp list to be unset

Signed-off-by: Shane Handley <shanehandley@fastmail.com>

* fix: use the correct default policy when parsing from a token

Signed-off-by: Shane Handley <shanehandley@fastmail.com>

---------

Signed-off-by: Shane Handley <shanehandley@fastmail.com>
2024-11-08 07:19:23 +00:00
Martin Robinson
036e74524a
net: Start reducing number of IPCs channels used for fetch with a FetchThread (#33863)
Instead of creating a `ROUTER` for each fetch, create a fetch thread
which handles all incoming and outcoming fetch requests. Now messages
involving fetches carry a "request id" which indicates which fetch is
being addressed by the message. This greatly reduces the number of file
descriptors used by fetch.

In addition, the interface for kicking off fetches is simplified when
using the `Listener` with `Document`s and the `GlobalScope`.

This does not fix all leaked file descriptors / mach ports, but greatly
eliminates the number used. Now tests can be run without limiting
procesess on modern macOS systems.

Followup work:

1. There are more instances where fetch is done using the old method.
   Some of these require more changes in order to be converted to the
   `FetchThread` approach.
2. Eliminate usage of IPC channels when doing redirects.
3. Also eliminate the IPC channel used for cancel handling.
4. This change opens up the possiblity of controlling the priority of
   fetch requests.

Fixes #29834.

Signed-off-by: Martin Robinson <mrobinson@igalia.com>
2024-10-16 16:53:24 +00:00
webbeef
58f34ad7a3
Create HttpStatus to safely deal with HTTP responses status. (#33581)
Signed-off-by: webbeef <me@webbeef.org>
2024-09-29 18:23:48 +00:00
webbeef
f57ae60056
Let protocol handlers decide if they are fetchable (#33573)
This adds a 'is_fetchable()' method on the ProtocolHandler trait that is then used in the fetch code.
The 'data:' protocol handler is updated to return true instead of hardcoding the scheme comparison, as well
as the 'urlinfo:' handler since it's just a testing one.

Signed-off-by: webbeef <me@webbeef.org>
2024-09-28 19:38:49 +00:00
Simon Wülker
10e5bb72d9
Initial support for <link rel="prefetch"> (#33345)
* Properly store link relations

Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>

* Send fetch request for prefetch links

We don't actually *do* anything with the response yet
(handle errors etc) but its a first step.

Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>

* Fire load/error events for prefetch loads

Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>

* Set prefetch destination/cors setting correctly

Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>

* Update WPT expectations

Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>

* Fix ./mach test-tidy errors

Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>

* Set correct "Accept" value for prefetch requests

Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>

* Add spec text to individual steps

Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>

---------

Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
2024-09-09 18:52:56 +00:00
webbeef
663a92a5df
make protocol handlers registrable (#33104)
Signed-off-by: webbeef <me@webbeef.org>
2024-08-22 04:11:16 +00:00
Martin Robinson
057873c94a
Remove unused constant from components/net/fetch/methods.rs (#33054)
This was revealed by the recent switch to `LazyLock`.

Signed-off-by: Martin Robinson <mrobinson@igalia.com>
2024-08-14 18:35:13 +00:00
Hayashi Mikihiro
65f90ff1fd
Replace the lazy_static crate with std::sync::LazyLock in components/net (#33046)
* replace in net/fetch/methods.rs

Signed-off-by: Hayashi Mikihiro <34ttrweoewiwe28@gmail.com>

* replace in net/hosts.rs

Signed-off-by: Hayashi Mikihiro <34ttrweoewiwe28@gmail.com>

* replace in net/async_runtime.rs

Signed-off-by: Hayashi Mikihiro <34ttrweoewiwe28@gmail.com>

* replace in net/tests/main.rs

Signed-off-by: Hayashi Mikihiro <34ttrweoewiwe28@gmail.com>

* remove lazy_static crate from components/net

Signed-off-by: Hayashi Mikihiro <34ttrweoewiwe28@gmail.com>

---------

Signed-off-by: Hayashi Mikihiro <34ttrweoewiwe28@gmail.com>
2024-08-14 12:15:55 +00:00
Josh Matthews
f38d1574bc
Allow navigations that include cross-origin redirects to succeed. (#32996)
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
2024-08-12 06:09:45 +00:00
Martin Robinson
7ea894774f
Add a directory listing feature for file URLs (#32580)
Signed-off-by: Bobulous <Bobulous@users.noreply.github.com>
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Co-authored-by: Bobulous <Bobulous@users.noreply.github.com>
2024-06-26 08:02:50 +00:00
Peter Mikola
3c06536cb6
clippy: fix some leftover warnings in components/net (#32476) 2024-06-11 17:50:21 +00:00
eri
da696b7e57
clippy: fix result_unit_err warnings (#31791)
* clippy: fix `result_unit_err` warnings

* feat: fix result warnings in script

* doc: document `generate_key` return type

Co-authored-by: Martin Robinson <mrobinson@igalia.com>

* feat: add back result to RangeRequestBounds::get_final

Co-authored-by: Martin Robinson <mrobinson@igalia.com>

---------

Co-authored-by: Martin Robinson <mrobinson@igalia.com>
2024-03-21 12:51:45 +00:00
eri
63527f56ca
clippy: Fix warnings in components/net (#31626)
* clippy: fix warnings in `components/net`

* fix: review comments
2024-03-13 09:40:04 +00:00
Mucha Naibei
0bc685ed97
Fix more clippy warnings in components/shared/net (#31548)
* Fix clippy warnings in components/shared

* Fix build error

* Fixes in order to solve some merge issues

---------

Co-authored-by: Martin Robinson <mrobinson@igalia.com>
2024-03-10 17:47:38 +00:00
eri
67b277c992
clippy: fix warnings in components/net (#31564)
* clippy: fix some warnings in components/net

* fix: review comments

* fix: tidy
2024-03-10 15:34:16 +00:00
Martin Robinson
5c1723c983
rustdoc: Fix many rustdoc errors (#31147)
This fixes many rustdoc errors that occur due to raw URLs in rustdoc
comments as well as unescaped Rust code that should be in backticks.
2024-01-22 13:13:48 +00:00
Samson
aad2dccc9c
Strict import formatting (grouping and granularity) (#30325)
* strict imports formatting

* Reformat all imports
2023-09-11 19:16:54 +00:00
Samson
711dbbd4af
remove extern crate (#30311)
* remove extern crate

* Update components/script_plugins/lib.rs

Co-authored-by: Martin Robinson <mrobinson@igalia.com>

---------

Co-authored-by: Martin Robinson <mrobinson@igalia.com>
2023-09-08 12:11:31 +00:00
Delan Azabani
c3c6c95a9b
constellation: crash to a new “sad tab” error page (#30290)
* constellation: crash to a new “sad tab” page

* check in resources/crash.html

* use a separate enum variant instead of keying on reason

* fmt + tidy

* rename Resource::Crash to Resource::CrashHTML

* clean up crash page and add details (reason + backtrace)

* avoid repeating crash errors in script::script_thread warn log

* make new LoadData init more idiomatic

* clarify comments and new fields

* fix doc comment style
2023-09-06 09:52:37 +00:00
Martin Robinson
bce7622cde
Switch to rustls and webpki-roots (#30025)
This change replaces OpenSSL with rustls and also the manually curated
CA certs file with webpki-roots (effectively the same thing, but as a
crate).

Generally speaking the design of the network stack is the same. Changes:

- Code around certificate overrides needed to be refactored to work with
  rustls so the various thread-safe list of certificates is refactored
  into `CertificateErrorOverrideManager`
- hyper-rustls takes care of setting ALPN protocols for HTTP requests,
  so for WebSockets this is moved to the WebSocket code.
- The safe set of cypher suites is chosen, which seem to correspond to
  the "Modern" configuration from [1]. This can be adjusted later.
- Instead of passing a string of PEM CA certificates around, an enum is
  used that includes parsed Certificates (or the default which reads
  them from webpki-roots).
- Code for starting up an SSL server for testing is cleaned up a little,
  due to the fact that the certificates need to be overriden explicitly
  now. This is due to the fact that the `webpki` crate is more stringent
  with self-signed certificates than SSL (CA certificates cannot used as
  end-entity certificates). [2]

1. https://wiki.mozilla.org/Security/Server_Side_TLS
2. https://github.com/briansmith/webpki/issues/114

Fixes #7888.
Fixes #13749.
Fixes #26835.
Fixes #29291.
2023-08-08 14:00:10 +00:00
Tuna
4c8db6af87
bump base64 from 0.10 to 0.21 (#29804)
* bump base64 from 0.10 to 0.21

* Fix configuration of bitflags

---------

Co-authored-by: Martin Robinson <mrobinson@igalia.com>
2023-08-02 21:25:37 +00:00
Naveen Gattu
a48a111cee Upgrade Hyper 2022-01-16 09:34:17 -08:00
Josh Matthews
08ff81b09a Fix warnings. 2022-01-03 13:05:19 -05:00
Naveen Gattu
903e0cd857 Non-blocking network IO 2021-12-23 11:16:24 -08:00
Josh Matthews
01681e79c4 Update nightly rustc. 2021-11-01 08:46:18 -04:00
Vincent Ricard
3148b8553e Block port 10080
See https://github.com/whatwg/fetch/issues/1191
2021-04-16 18:05:46 +02:00
Vincent Ricard
2e7be21ee1 Add port blocking for tftp, netbios-ns, snmp, rtsp, h323gatestat, h323hostcall, pptp, sane-port
Spec update: https://github.com/whatwg/fetch/pull/1148
2021-03-29 20:30:31 +02:00
Vincent Ricard
3b3824078d Fix some 'nosniff' tests 2021-03-08 21:24:08 +01:00
Vincent Ricard
9b31533728 Update the "bad ports" list
https://github.com/whatwg/fetch/issues/1108
2020-11-04 21:31:02 +01:00
Utsav Oza
310821d3b0 Update referrer computation
Update unit tests for determine_requests_referrer

Update wpt metadata

Add missing spec links
2020-07-02 14:29:26 +05:30
Matthias Deiml
fa18cf620f Make url for "client" referrer mandatory 2020-06-17 19:07:14 +02:00
Gregory Terzian
719b395c40 fix streaming request bodies, terminate fetch if the body stream errors 2020-06-16 13:14:38 +08:00
Josh Matthews
2550600131 net: Use a POST request for allowing certs temporarily. 2020-06-09 16:51:21 -04:00
Josh Matthews
6a6662195e net: Add option to temporarily accept certs that failed the handshake. 2020-06-09 15:03:18 -04:00
Gregory Terzian
7d51997214 net: update list of bad ports 2020-06-04 11:38:38 +08:00
Alexandrov Sergey
a7c5c97616 check http_state in determine_request_referrer 2020-05-19 20:06:59 +03:00
Josh Matthews
19216627d3 Update content-security-policy. 2020-03-25 09:11:09 -04:00
bors-servo
cd00e57c97
Auto merge of #25740 - gterzian:per_fetch_filemanager_handle, r=Manishearth
Per fetch file token for blob url

<!-- Please describe your changes on the following line: -->

Depends on https://github.com/servo/servo/pull/25724

First step of https://github.com/servo/servo/issues/25226

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [ ] `./mach build -d` does not report any errors
- [ ] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)

<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
2020-02-25 14:49:26 -05:00
Gregory Terzian
895500e68c add a fetch token for blob urls 2020-02-21 14:26:45 +08:00