Many object store related operations require the transaction to be
checked: to ensure it is still active, and, if the operation is a write,
that the transaction is not read-only. I've added the
`check_transaction` method to perform these checks.
Additionally `Clear` was still half-implemented, so I went ahead and
implemented that.
---------
Signed-off-by: Ashwin Naren <arihant2math@gmail.com>
The current behaviour is too strict according to the spec and is the
cause of many intermittent test errors.
Testing: Existing coverage is sufficient.
Fixes: #37706
---------
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
This change refactors how we notify DevTools about network activity so
that all fetches (even those served from cache) appear correctly in the
Network panel, and so that DevTools sees request metadata as soon as
possible rather than waiting until the end of a full HTTP cycle.
- Before, we only send DevTools events inside http_network_fetch, so
cached responses (which skip that path) never show up. By emitting a
minimal HttpRequest event at the very start of main_fetch (with URL,
method, pipeline and browsing IDs), we guarantee every fetch shows up
immediately.
- Then, by moving HttpResponse notifications out of http_network_fetch
into main_fetch (right after process_response and process_response_eof),
we ensure DevTools gets status, header, and completion events for both
network and cache hits. Leveraging nullable fields in NetworkEventActor
lets us incrementally fill in timing, header, and body data later,
improving DevTools’ visibility.
Testing: Ran servo with `--devtools=6080` flag, cached responses now
appear in the network panel
Fixes: https://github.com/servo/servo/issues/37869
---------
Signed-off-by: Uthman Yahaya Baba <uthmanyahayababa@gmail.com>
Digging into several crashing tests revealed that committing
transactions is a fallible operation. Propagating those errors led to
exposing many new errors caused by the IDBRequest implementation
assuming that all successful responses contained a structured clone. The
end result is a bunch of new test failures that were previously hidden.
Testing: Existing test coverage is sufficient.
---------
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
This refactors some of the cookie retrieval mechanism to be less
repetitive and separates the cookie-list from the cookie-string which
will also be needed for Cookie Store.
Testing: No new behavior, should be covered by existing WPT tests.
Signed-off-by: Sebastian C <sebsebmc@gmail.com>
This change allows the compiler to recognize if any read-only operations
are missing an implementation when processing a readonly transaction.
Testing: The existing behaviour is unchanged, so current tests suffice.
The new code is unused and cannot be tested.
Fixes: part of #6963
---------
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
The cookie-rs library parses the cookie expiry date based on the format
from RFC 2616 (for HTTP/1.1), which is stricter than the format from RFC
6265 (for HTTP cookie).
This patch implements the cookie expiry date algorithm from RFC 6265.
When Cookie::parse fails in parsing the expiry date, we try to parse the
expiry again with this algorithm, to provide extra compatibility with
legacy systems.
Testing: Pass a WPT test that was expected to fail before, and add a
unit test.
Fixes: #36452
---------
Signed-off-by: Kingsley Yung <kingsley@kkoyung.dev>
The will-navigate message tells the devtools client to expect a
navigation for a browsing context. This makes the network monitor clear
any previous entries and show the requests for the new page that is
loaded. In order to support this correctly, we need to send the
navigation notification from the constellation instead of the script
thread, otherwise we silently ignore navigations triggered by the
browser URL bar.
Testing: Ran servo in devtools mode , now the requests appear for new
loaded page
Fixes: https://github.com/servo/servo/issues/37334
---------
Signed-off-by: Uthman Yahaya Baba <uthmanyahayababa@gmail.com>
This pull request introduces changes to the storage subsystem to:
- Isolate sessionStorage per top-level browsing context (WebViewId), in
addition to origin.
- Copy sessionStorage when creating a new auxiliary browsing context
without
noopener, as required by the corresponding spec
These changes bring Servo closer to spec compliance, matching expected
browser
behavior.
Testing: This work affects observable behavior. As a result, some
previously
failing WPT tests now pass. No new tests are added, since the behavior
is
already covered by existing web-platform-tests.
Fixes: #21291
---------
Signed-off-by: Jan Varga <jan.varga@gmail.com>
This creates a new method in shared/compositing/lib to generate image
keys that are send over the webview. This does not immediately return
the keys but goes over the constellation to receive the keys from the
IOCompositor. To make this more efficient, we now cache the keys in
image_cache in a simple FIFO order. The old blocking method stays intact
for now but got renamed to make the blocking clear.
The blocking calls that are left are in:
- `components/canvas/canvas_data.rs`
- `components/script/dom/htmlmediaelement.rs`
Testing: WPT tests should cover this as this doesn't change any
functionality.
Fixes: Was mentioned in
https://github.com/servo/servo/issues/37161#issuecomment-2915750051 and
part of https://github.com/servo/servo/issues/37086
---------
Signed-off-by: Narfinger <Narfinger@users.noreply.github.com>
Signed-off-by: gterzian <2792687+gterzian@users.noreply.github.com>
Co-authored-by: gterzian <2792687+gterzian@users.noreply.github.com>
cd30b78 improved the memory efficiency of write_json_to_file, but this
causes visible (when profiling) perf regression on my mac, which can
easily be fixed by wrapping the file in a `BufWriter`. This still bounds
peak memory usage, but keeps writing efficient.
Testing: Manual profiling with samply.
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
Allows indexeddb backends to return errors on certain operations.
Currently the errors are not demarcated, as the result type is
`Result<(), ()>`. If this is not appropriate then perhaps having a
string error might be better.
Testing: Some tests might perhaps move from PANIC to FAIL
Fixes: Partially fixes a bit of #37647, more work needs to be done
however
---------
Signed-off-by: Ashwin Naren <arihant2math@gmail.com>
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
Co-authored-by: Josh Matthews <josh@joshmatthews.net>
Rejects version upgrade if the upgraded version is lower than the
current database version. Also returns the actual version instead of the
requested upgrade version.
Testing: WPT tests might cover this
Fixes: #25322
Signed-off-by: Ashwin Naren <arihant2math@gmail.com>
The settings for WPT IndexedDB coverage is incorrect. We never ran the
IndexedDB tests actually.
Testing: `tests/wpt/tests/IndexedDB`
---------
Signed-off-by: Wu Yu Wei <yuweiwu@pm.me>
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
Co-authored-by: Josh Matthews <josh@joshmatthews.net>
Resolves#37638. Fixes threadpool name and adds a config preference
(`threadpools_indexeddb_workers_max`).
Signed-off-by: Ashwin Naren <arihant2math@gmail.com>
Similar to #37638. It was using `threadpools_async_runtime_workers_max`,
and `threadpools_image_cache_workers_max` was unused, this PR fixes
that.
Signed-off-by: Ashwin Naren <arihant2math@gmail.com>
Instead of serializing everything at once in memory it is more efficient
to use `to_writer_pretty`.
The memory allocations from writing the HSTS file to disk showed up very
visibly in DHAT and
should be fixed by streaming the writing. This change should reduce
allocations.
Testing: This change should not modify behavior, and thus is covered by
existing tests.
---------
Signed-off-by: Jonathan Schwender <55576758+jschwe@users.noreply.github.com>
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
This is finishing some incomplete cleanup from #33044. Kitchen sink enum
types like IndexedDBThreadReturnType make code harder to read and
require ignoring variants that will never be sent in many cases.
Testing: No behaviour change; existing WPT tests suffice.
Fixes: part of #6963
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
Adds indexeddb support to servo. At the moment heed is being used as the
backend, although this can be swapped out by implementing `KvsEngine`.
This PR adds a thread + a thread pool for Indexeddb related operations.
Also `database_access_task_source` is added for Indexeddb related
operations.
This is a partial rewrite of #25214. (Reopened due to branching issue)
Fixes#6963
---------
Signed-off-by: Ashwin Naren <arihant2math@gmail.com>
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
Co-authored-by: Rasmus Viitanen <rasviitanen@gmail.com>
Co-authored-by: Josh Matthews <josh@joshmatthews.net>
#37021 exposed that our tests are trying to load a local file path for
Ahem.ttf, but this stops working when we implement the specified CORS
protections for our font requests. Luckily, the WPT suite already
provides a same-origin version of the same font, so we can just update
our CSS to use that font face instead.
Testing: Existing WPT coverage.
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
This change upgrades some dependencies that required some manual
intervention due to duplicate packages and breaking API changes. These
changes also allow us to upgrade to ipc-channel 0.20
(https://github.com/servo/ipc-channel/pull/390#discussion_r2070677101),
and allow us to upgrade other dependencies that have migrated to rand
0.9 while the ecosystem remains split.
Testing: Existing WPT tests.
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
While working on #37209 I discovered that the header was computed
incorrectly. After carefully reading the specification, I realized that
the link in the spec was wrong and we were missing the fact that for
host-domains, we should operate on the registrable domain.
Additionally, the same-site call was missing the negation.
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
Also add clarifying comments to the SRI WPT tests with
regards to the `www.` domain and how that interacts with
the integrity checks.
Lastly, adjust the casing for `Strict-Dynamic`, as in
the post-request check that should also be case-insensitive.
Closesservo/servo#37200Closesservo/servo#36760Fixesservo/servo#36499
Part of w3c/webappsec-csp#727Fixesw3c/webappsec-csp#728
Part of servo/servo#4577
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
Co-authored-by: Josh Matthews <josh@joshmatthews.net>
These changes allow a minimal set of checks for font-src
CSP checks to pass.
Part of #4577
Part of #35035
---------
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
This change adds support for rendering static SVG images using the
`resvg` crate, allowing svg sources in the `img` tag and in CSS
`background` and `content` properties. There are some limitations in
using resvg:
1. There is no support for animations or interactivity as these would
require implementing the full DOM layer of SVG specification.
2. Only system fonts can be used for text rendering. There is some
mechanism to provide a custom font resolver to usvg, but that is not
explored in this change.
3. resvg's handling of certain edge cases involving lack of explicit
`width` and `height` on the root svg element deviates from what the
specification expects from browsers. For example, resvg uses the values
in `viewBox` to derive the missing width or height dimension, but
without scaling that dimension to preserve the aspect ratio. It also
doesn't allow overriding this behavior.
Demo screenshot:
<details>
<summary>Source</summary>
```
<style>
#svg1 {
border: 1px solid red;
}
#svg2 {
border: 1px solid red;
width: 300px;
}
#svg3 {
border: 1px solid red;
width: 300px;
height: 200px;
object-fit: contain;
}
#svg4 {
border: 1px solid red;
width: 300px;
height: 200px;
object-fit: cover;
}
#svg5 {
border: 1px solid red;
width: 300px;
height: 200px;
object-fit: fill;
}
#svg6 {
border: 1px solid red;
width: 300px;
height: 200px;
object-fit: none;
}
</style>
</head>
<body>
<div>
<img id="svg1" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
</div>
<div>
<img id="svg2" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
<img id="svg3" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
<img id="svg4" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
</div>
<div>
<img id="svg5" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
<img id="svg6" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
</div>
</body>
```
</details>
---------
Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
Removed unused dependencies in various crates using cargo-machete, grep
and cargo-udeps.
Testing: The CI tests if any dependencies were used that are removed.
Signed-off-by: Narfinger <Narfinger@users.noreply.github.com>
This makes servo use less file descriptors for animated images and
avoids the crash described in
https://github.com/servo/servo/issues/36792.
Doing this also forces the end users to be more explicit about whether
they want to deal with all image frames or just the first one.
Previously, `Image::bytes` silently returned only the data for the first
frame. With this change there's now a `frames` method which returns an
iterator over all frames in the image.
Testing: No tests - this simply reduces the number of fds used. Servo
doesn't currently display animated gifs anyways.
Fixes: https://github.com/servo/servo/issues/36792
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
Our persistent localstorage data can be meaningfully large after testing
real world sites. This change ensures it shows up in about:memory.
Testing: Opened about:memory after launching the browser with a
persistent config
Fixes: Part of #11559
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
This reduces the memory used by the preload list to just 1.9MB. The
total memory savings in HSTS from
pre-103cbed928
is now 62MB, or 96%. And in terms of total resident memory is a 7.5%
reduction. The DAFSA/DAWG used by Firefox is 1.1MB so there could be
additional gains available but this seems like the best option based on
maintained libraries available (I could not find a good maintained
library for DAFSAs in Rust).
The main trick is this: the FST map API is currently designed to map
byte sequences to u64 values. Because we only need to determine if a
preloaded domain has the `includeSubdomains` flag set, we encode that
into the lowest bit of the ids in the map. This way finding an entry in
the map directly provides us with the `includeSubdomains` flag and we
don't need to keep another mapping in memory or on disk.
Updated the `./mach update-hsts-preload` command to generate the new FST
map file. (Not sure if I need to update any dev-dependencies anywhere
for this change)
This change also replaces the use of "mozilla.org" with "example.com" in
the HSTS unit tests to make sure that entries in the preload list do not
influence the tests (since example.com should not ever end up on the
preload list)
Testing: Updated unit tests
Fixes: #25929
---------
Signed-off-by: Sebastian C <sebsebmc@gmail.com>
Plumbs in the memory reporting into resource_thread since that's where
the other user of the public suffix list (HSTS) reports.
Testing: Checked about:memory on servo.org
Signed-off-by: Sebastian C <sebsebmc@gmail.com>
This changes includes two semi-related things:
1. Fixes some specification compliance issues when parsing mime
types and charsets for `XMLHttpRequest`.
2. Implements a `<stylesheet>` parsing quirk involving mime types.
Testing: There are tests for these changes.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
Combines the 2 time values in the HSTS entry with a single timestamp for
expiration. (9MB savings per list)
The previous time representations were based on system boot time which
meant that the `hsts_list.json` round trip across boots resulted in
completely erroneous expiration times.
The preload list is now initialized separately from the public and
private lists and shared by both, cutting memory use in half.
Overall takes memory use from 64MB for HSTS to 24MB.
Expired HSTS entries are now removed from the list when updating an
entry and subdomains can be added to a list if the superdomain does not
already include them.
Testing: New unit tests added
Related to #25929 but the next step would be to attempt to use
https://github.com/BurntSushi/fst Which will be explored in a follow-up.
---------
Signed-off-by: Sebastian C <sebsebmc@gmail.com>
Fix a IPC hang due to `ReadableStream::get_in_memory_bytes` could return
really huge chunk.
Testing: WPT on ReadableStream should pass
Fixes: IPC hang when transferring huge chunk bytes from `ReadableStream`
cc @gterzian @Taym95 since this is also related to ReadableStream.
---------
Signed-off-by: Yu Wei Wu <yuweiwu@YunoMacBook-Air.local>
Co-authored-by: Yu Wei Wu <yuweiwu@YunoMacBook-Air.local>
This deletes images from WR when dropping the ImageCacheStore for a
WebView.
Testing: Run `./mach run --enable-experimental-web-platform-features
unsplash.com` and then open `about:memory` in a new tab. On Linux we end
up with ~30MB of WR images. Then close the unsplash.com tab and measure
memory again, it will down to ~1.25MB
Fixes: https://github.com/servo/servo/issues/25927
Signed-off-by: webbeef <me@webbeef.org>
When slicing a blob that is already sliced we should reference it's
parent's data instead of creating a subview into the sliced blob. This
keeps the blob ancestry chain small and reduces the number of blobs that
we have to resolve.
Testing: Includes a new crashtest
Fixes: https://github.com/servo/servo/issues/36843
[try
run](https://github.com/simonwuelker/servo/actions/runs/14844873660)
---------
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
When all unit tests are run, the existing feature flag use from
net_traits was enough to ensure the default resources are baked into the
test binary. When only the unit tests for the `net` crate are run, that
feature flag was not enabled in the build so the tests that require the
resources fail.
Testing: Ran `./mach test-unit -p net`
Fixes: #36837
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
Rather than sharing the full image cache in a script_thread, the image
cache is now unique per document. This ensures that CSP factors no
longer affect whether the image is retrieved from the cache incorrectly.
To do so, the thread_pool is shared across all caches, but the store is
fresh. Except for the place_holder{image,url}, which are cloned. That's
because the `rippy_data` is only available in the constellation and no
longer accessible at the point that we need to create the document in
the script_thread.
Contrary to the description in #36505, the script_thread still has an
image_cache for this reason: so it has access to the store and
thread_pool to clone it.
With these changes, the two CSP tests no longer flake. Confirmed with
running the following commmand:
```
./mach test-wpt tests/wpt/tests/content-security-policy/generic/ --rerun=10
```
Fixes#36505
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
It seems sub-optimal to to sequentialise execution by grabbing a lock,
each time we want to spawn a task onto the tokio runtime. We don't need
the lock either, so it makes sense to just remove it, which also
simplifies a bunch of the using code.
Testing: Covered by existing tests
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
This turned out to be a full rabbit hole. The new header
is parsed in the new `parse_csp_list_from_metadata` which
sets `disposition` to `report.
I was testing this with
`script-src-report-only-policy-works-with-external-hash-policy.html`
which was blocking the script incorrectly. Turns out that there
were multiple bugs in the CSP library, as well as a missing
check in `fetch` to report violations.
Additionally, in several locations we were manually reporting csp
violations, instead of the new `global.report_csp_violations`. As
a result of that, they would double report, since the report-only
header would be appended as a policy and now would report twice.
Now, all callsides use `global.report_csp_violations`. As a nice
side-effect, I added the code to set source file information,
since that was already present for the `eval` check, but nowhere
else.
Part of #36437
Requires servo/rust-content-security-policy#5
---------
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
Signed-off-by: Tim van der Lippe <TimvdLippe@users.noreply.github.com>
Instead of returning true / false it's better to return a Result (even
if we continue ignoring possible error).
Testing: Doesn't change any behavior
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
Added a simple check to only perform metadata extraction and listener
notification when we haven't already processed the metadata for an image
Testing: Existing tests should cover if we break decoding image metadata
complete.
Fixes: #36502
---------
Signed-off-by: Barigbue <barigbuenbira@gmail.com>
Co-authored-by: Josh Matthews <josh@joshmatthews.net>
This removes a bunch of duplicated code needed to support
ConditionalMallocSizeOf correctly, and fixes multiple places where that
code was subtly wrong (the seen pointers hashset was never cleared).
Testing: Measuring https://www.nist.gov/image-gallery lots of times.
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
This allows removing a LazyLock around the resources.
We override the baked in resources unconditionally in servoshell
upon initialization anyway
([desktop](9f93ccd942/ports/servoshell/desktop/cli.rs (L15)),
[android](9f93ccd942/ports/servoshell/egl/android/simpleservo.rs (L49)),
[ohos](9f93ccd942/ports/servoshell/egl/ohos/simpleservo.rs (L43))
), meaning that the baked in resources
are unused in servoshell.
For 3rd-party embedders, we probably also want to the let them know
early that they should initialize the resources, instead of
restricting the panics to production mode.
Rippy is the only resource which was required. Since it is only
253 bytes large, we just bake that resource in as a fallback.
We do want to make using the resources easy from tests, so we add
some logic to keep the baked in resources for tests only and initialize
the resource reader on first access.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by
`[X]` when the step is complete, and replace `___` with appropriate
data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
These changes add a new report for image cache memory usage for each
script thread.
Testing: Looked at the numbers after browsing various stock photo sites
that show galleries of images.
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
Records the memory usage of the HSTS lists in the network thread.
Testing: Verified the presence of the new reports for servo.org.
Fixes: #35059
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
