Our persistent localstorage data can be meaningfully large after testing
real world sites. This change ensures it shows up in about:memory.
Testing: Opened about:memory after launching the browser with a
persistent config
Fixes: Part of #11559
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
This reduces the memory used by the preload list to just 1.9MB. The
total memory savings in HSTS from
pre-103cbed928
is now 62MB, or 96%. And in terms of total resident memory is a 7.5%
reduction. The DAFSA/DAWG used by Firefox is 1.1MB so there could be
additional gains available but this seems like the best option based on
maintained libraries available (I could not find a good maintained
library for DAFSAs in Rust).
The main trick is this: the FST map API is currently designed to map
byte sequences to u64 values. Because we only need to determine if a
preloaded domain has the `includeSubdomains` flag set, we encode that
into the lowest bit of the ids in the map. This way finding an entry in
the map directly provides us with the `includeSubdomains` flag and we
don't need to keep another mapping in memory or on disk.
Updated the `./mach update-hsts-preload` command to generate the new FST
map file. (Not sure if I need to update any dev-dependencies anywhere
for this change)
This change also replaces the use of "mozilla.org" with "example.com" in
the HSTS unit tests to make sure that entries in the preload list do not
influence the tests (since example.com should not ever end up on the
preload list)
Testing: Updated unit tests
Fixes: #25929
---------
Signed-off-by: Sebastian C <sebsebmc@gmail.com>
Plumbs in the memory reporting into resource_thread since that's where
the other user of the public suffix list (HSTS) reports.
Testing: Checked about:memory on servo.org
Signed-off-by: Sebastian C <sebsebmc@gmail.com>
Combines the 2 time values in the HSTS entry with a single timestamp for
expiration. (9MB savings per list)
The previous time representations were based on system boot time which
meant that the `hsts_list.json` round trip across boots resulted in
completely erroneous expiration times.
The preload list is now initialized separately from the public and
private lists and shared by both, cutting memory use in half.
Overall takes memory use from 64MB for HSTS to 24MB.
Expired HSTS entries are now removed from the list when updating an
entry and subdomains can be added to a list if the superdomain does not
already include them.
Testing: New unit tests added
Related to #25929 but the next step would be to attempt to use
https://github.com/BurntSushi/fst Which will be explored in a follow-up.
---------
Signed-off-by: Sebastian C <sebsebmc@gmail.com>
It seems sub-optimal to to sequentialise execution by grabbing a lock,
each time we want to spawn a task onto the tokio runtime. We don't need
the lock either, so it makes sense to just remove it, which also
simplifies a bunch of the using code.
Testing: Covered by existing tests
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
This removes a bunch of duplicated code needed to support
ConditionalMallocSizeOf correctly, and fixes multiple places where that
code was subtly wrong (the seen pointers hashset was never cleared).
Testing: Measuring https://www.nist.gov/image-gallery lots of times.
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
Records the memory usage of the HSTS lists in the network thread.
Testing: Verified the presence of the new reports for servo.org.
Fixes: #35059
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
So far the memory reporter aggregates reports from all processes, and
runs the system reporter only in the main process. Instead it is
desirable to have per-process reports. We do so by:
- creating a ProcessReports struct that holds includes the pid in
addition to the reports themselves.
- running the system memory reporter also in content processes.
- updating the about:memory page to create one report per process, and
add useful information like the pid and the urls loaded in a given
process.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by
`[X]` when the step is complete, and replace `___` with appropriate
data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
Signed-off-by: webbeef <me@webbeef.org>
- Remove `EmbedderMethods::get_user_agent_string`. This is now part of
the `Preferences` data structure, which should allow it to be
per-`WebView` in the future.
- Remove `EmbedderMethods::get_version_string`. This was used to include
some data along with WebRender captures about the Servo version. This
isn't really necessary and it was done to replace code in the past
that output the WebRender version, so also isn't what the original
code did. I think we can just remove this entirely.
The idea with these changes is that `EmbedderMethods` can be removed
in a followup and the rest of the methods can be added to
`ServoDelegate`. These two methods are ones that cannot be added to a
delegate as they are used during `Servo` initialization.
Testing: There is currently no testing for libservo. These changes are
meant
as preparation for adding a suite of `WebView` unit tests.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
* Create config_dir if none exist for caching
Signed-off-by: Euclid Ye <yezhizhenjiakang@gmail.com>
* remove specialized behaviour for ohos; copy prefs.json if necessary
Signed-off-by: Euclid Ye <yezhizhenjiakang@gmail.com>
* downgrade the log to trace verbosity
Signed-off-by: Euclid Ye <yezhizhenjiakang@gmail.com>
* update wpt-test
Signed-off-by: Euclid Ye <yezhizhenjiakang@gmail.com>
---------
Signed-off-by: Euclid Ye <yezhizhenjiakang@gmail.com>
* Use 2024 style edition
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
* Reformat all code
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
---------
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
Rework the `WebViewDelegate::intercept_web_resource_load` into
`WebViewDelegate::load_web_resource` and clean up internal messaging.
The main thing here is adding objects which manage the response to these
delegate methods. Now we have `WebResourceLoad` and
`InterceptedWebResourceLoad` which make it much harder to misuse the
API.
In addition, the internal messaging for this is cleaned up. Canceling
and finishing the load are unrelated to the HTTP body so they are no
longer subtypes of an HttpBodyData message. Processing of messages is
made a bit more efficient by collecting all body chunks in a vector and
only flattening the chunks at the end.
Finally, "interceptor" is a much more common spelling than "intercepter"
so I've gone ahead and made this change everywhere.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Flatten and simplify Servo's preferences code. In addition, have both
preferences and options passed in as arguments to `Servo::new()` and
make sure not to use the globally set preferences in `servoshell` (as
much as possible now).
Instead of a complex procedural macro to generate preferences, just
expose a very simple derive macro that adds string based getters and
setters.
- All command-line parsing is moved to servoshell.
- There is no longer the concept of a missing preference.
- Preferences no longer have to be part of the resources bundle because
they now have reasonable default values.
- servoshell specific preferences are no longer part of the preferences
exposed by the Servo API.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Instead of creating an IPC channel for every fetch, allow cancelling
fetches based on the `RequestId` of the original request. This requires
that `RequestId`s be UUIDs so that they are unique between processes
that might communicating with the resource process.
In addition, the resource process loop now keeps a `HashMap` or `Weak`
handles to cancellers and cleans them up.
This allows for creating mutiple `FetchCanceller`s in `script` for a
single fetch request, allowing integration of the media and video
elements to integrate with the `Document` canceller list -- meaning
these fetches also get cancelled when the `Document` unloads.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
* prompt user to get their credentials
Signed-off-by: Lloyd Massiah artmis9@protonmail.com
move credential prompt to a function
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
* add prompt for step 15.4
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
* add new prompt definition for user credentials
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
* remove default implementation for HttpState which allowed making the embedder_proxy non-optional
- default implementation was only used in tests so created an alternative create_http_state function
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
add credentials to authentication cache
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
* add tests that are successful for the happy path
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
* add test for user cancels prompt and user inputs incorrect credentials, and refactor shared code between tests
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
* handle error when setting username and password in Url and ran formatting
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
renaming test functions
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
* change authentication flag to false for proxy authentication. The spec doesn't specify that the flag should be true, and the flag is by default false
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
* clean up test code a bit
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
* add skeleton implementation to support open harmony and android
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
* update warning message to include Android
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
* fix build error for OH os and Android
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
* remove unused import to fix warning
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
---------
Signed-off-by: Lloyd Massiah <artmis9@protonmail.com>
Co-authored-by: lazypassion <25536767+lazypassion@users.noreply.github.com>
* Add prefs to limit threadpool sizes
Add preferences to control the size of threadpools,
so that we can easily reduce the amount of runtime
threads and test which pools benefit from more
threads.
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
* Add pref for Webrender threadpool
Add a preference to limit the size of the webrender threadpool.
Note: WebRender by default calls hooks which register the threads with
a profiler instance that the embedder can register with webrender.
Servo currently doesn't register such a profiler with webrender,
but in the future we might also want to profile the
webrender threadpool.
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
---------
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
This changes updates to the new version of the `cookie` crate in Servo
which no longer uses the old `time@0.1` data types. This requires using
a new version of `time` while we transition off of the old one. This is
the first step in that process.
In addition, the overloading of the `cookie::Cookie` name was causing a
great deal of confusion, so I've renamed the Servo wrapper to
`ServoCookie` like we do with `ServoUrl`.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
This change replaces OpenSSL with rustls and also the manually curated
CA certs file with webpki-roots (effectively the same thing, but as a
crate).
Generally speaking the design of the network stack is the same. Changes:
- Code around certificate overrides needed to be refactored to work with
rustls so the various thread-safe list of certificates is refactored
into `CertificateErrorOverrideManager`
- hyper-rustls takes care of setting ALPN protocols for HTTP requests,
so for WebSockets this is moved to the WebSocket code.
- The safe set of cypher suites is chosen, which seem to correspond to
the "Modern" configuration from [1]. This can be adjusted later.
- Instead of passing a string of PEM CA certificates around, an enum is
used that includes parsed Certificates (or the default which reads
them from webpki-roots).
- Code for starting up an SSL server for testing is cleaned up a little,
due to the fact that the certificates need to be overriden explicitly
now. This is due to the fact that the `webpki` crate is more stringent
with self-signed certificates than SSL (CA certificates cannot used as
end-entity certificates). [2]
1. https://wiki.mozilla.org/Security/Server_Side_TLS
2. https://github.com/briansmith/webpki/issues/114Fixes#7888.
Fixes#13749.
Fixes#26835.
Fixes#29291.
