Add vello backend by implementing Backend traits in canvas crate (so
this lives in canvas_paint_thread - embedded process). Current
implementation uses normal wgpu, so we block on GPU work. Vello backend
is gated behind `vello` feature and `dom_canvas_vello_enabled` pref.
Feature-wise this backend is on on par with raqote (sometimes better
sometimes worse), but performance wise it's worse.
## Known vello problems:
- image roundtrip does not work (fixed in
https://github.com/linebender/vello/pull/974)
- https://github.com/linebender/vello/issues/1066 (fixed)
- clip layers are not working properly:
https://github.com/linebender/vello/issues/1061
- `/html/canvas/element/pixel-manipulation/2d.imageData.put.*`
- `/html/canvas/element/path-objects/2d.path.clip.intersect.html`
- https://github.com/linebender/vello/issues/1056
-
`/html/canvas/element/fill-and-stroke-styles/2d.gradient.interpolate.coloralpha.html`
- `kurbo::Cap::Butt` is defect (only visible with big lineWidth)
https://github.com/linebender/vello/issues/1063
- `/html/canvas/element/line-styles/2d.line.cross.html`
- `/html/canvas/element/line-styles/2d.line.miter.acute.html`
- other lack of strong correct problems
(https://github.com/linebender/vello/issues/1063#issuecomment-2998084736):
- `/html/canvas/element/path-objects/2d.path.rect.selfintersect.html`
- There is currently no way to do put image properly in vello as we
would need to ignore all clips and other stuff (we try to work around
this on best effort basis)
https://github.com/linebender/vello/issues/1088
- `/html/canvas/element/pixel-manipulation/2d.imageData.put.*`
- precision problems
- `/html/canvas/element/path-objects/2d.path.stroke.scale2.html`
- `/html/canvas/element/path-objects/2d.path.arc.scale.1.html`
## Known servo problems
- bad performance due to blocking on GPU work
- some get/put intensive tests `TIMEOUT`
- proper shadow support (non-blocker as we already are living without it
now)
- support for rect shadow is there but unimplemented currently as that's
the state in raqote
Testing: `mach try vello` will run normal WPT (with raqote) +
vello_canvas subsuite that runs only on `/html/canvas/element`. All
subsuite expectations are stored separately.
Fixes: #36823Fixes: #35230
---------
Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>
Add support of the ImageBitmapRenderingContext as "bitmaprenderer"
canvas context mode to RenderingContext/OffscreenRenderingContext
https://html.spec.whatwg.org/multipage/#imagebitmaprenderingcontext
It is initial implementation with public interface API but without
any display presentation support for HTMLCanvasElement.
Testing: Improvements in the following tests:
-
html/canvas/element/manual/imagebitmap/createImageBitmap-origin.sub.html
- html/canvas/offscreen/manual/text/canvas.2d.offscreen*
-
html/canvas/offscreen/manual/the-offscreen-canvas/offscreencanvas.transfer.to.imagebitmap.nocrash.html
- imagebitmap-renderingcontext/*
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
The 'none' image orientation option was temporarily deprecated (in 2022)
according to HTML specification with the remark about going to be reused
with
a different meaning in future as the same semantics as CSS 'none'
image-orientation.
https://html.spec.whatwg.org/multipage/#dom-imagebitmapoptions-imageorientation-nonehttps://www.w3.org/TR/css-images-3/#valdef-image-orientation-none
Official MDN documentation added it back with new meaning (in 2024),
but it wasn't added back to HTML specification (still pending).
https://developer.mozilla.org/en-US/docs/Web/API/Window/createImageBitmap#none
At current moment this option is poor supported on all major browsers,
but there are some
existed WPT tests (ImageBitmap/WebGL) which are actively use it in
createImageBitmap() options.
Chromium (supported):
- stable: same as 'from-image' (but with deprecation warning)
- experimental: ignoring any orientation metadata
Firefox (supported as default option):
- stable: ignoring any orientation metadata ('from-image' do the same)
Testing: Improvements and fails (expects 'from-image' behaviour from
'none' option) in the following tests
-
html/canvas/element/manual/imagebitmap/createImageBitmap-exif-orientation_none.html
- webgl/tests/conformance/textures/misc/exif-orientation.html
Fixes (partialy): #34112
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Follow to the HTML specification and support of Blob as
ImageBitmapSource
to able use it as intermediate instance in "fetch() -> Blob ->
ImageBitmap" execution sequence.
https://html.spec.whatwg.org/multipage/#imagebitmapsource
The specification says what these steps must run in parallel
(outside the createImageBitmap task), but currently loading bytes from
Blob
and later image decoding happen in synchronous order while
promise is fullfilled or rejected on bitmap task source.
https://html.spec.whatwg.org/multipage/#the-imagebitmap-interface:blob-4
Testing: Improvements in the following WPT tests
- html/canvas/element/compositing/2d.composite*
- html/canvas/element/drawing-images-to-the-canvas/2d.drawImage*
- html/canvas/element/manual/imagebitmap/createImageBitmap*
- html/canvas/offscreen/compositing/2d.composite
- html/canvas/offscreen/drawing-images-to-the-canvas/2d.drawImage
- webgl/tests/conformance/textures/image_bitmap_from_blob/*
Fixes (partially): #34112
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
According to HTML specification the poster attribute determines the
element's poster frame
(regardless of the value of the element's show poster flag).
https://html.spec.whatwg.org/multipage/#poster-frame
So the poster frame and the show poster flag is orthogonal to each
other, the latest one only controls
when browser should display the poster frame and should do not block
accepting video frames
from media pipeline (e.g. on new_preroll callback from video sink).
During layout the video element should select the current frame which
will be presented
based on first matching condition from the list:
https://html.spec.whatwg.org/multipage/#the-video-element:the-video-element-7
Testing: Improvements in the following WPT tests
- html/canvas/element/manual/imagebitmap/createImageBitmap*
- html/semantics/embedded-content/the-canvas-element/*
Fixes: #37165
---------
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
Follow the ImageBitmap specification and use the global scope bitmap
task source
to fulfill resolved promise (asynchronously).
https://html.spec.whatwg.org/multipage/#bitmap-task-source
Any promise rejection must be done synchronously.
Testing: Improvements in the following WPT test
-
html/canvas/element/manual/imagebitmap/createImageBitmap-resolves-in-task.any.js
Fixes (partially): #34112
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Follow the ImageBitmap specification and make cropping of the bitmap
data to the source rectangle with formatting:
https://html.spec.whatwg.org/multipage/#cropped-to-the-source-rectangle-with-formatting
For now the next functionality not implemented:
- image orientation support (such as EXIF metadata)
- color space conversion (image, blob)
The convertion from ResizeQuality to "image" FilterType:
- pixelated/low/medium/high -> Nearest/Triangle/CatmullRom/Lanczos3
Other browsers use the following sample filtering:
- chromium (skia): Nearest/Linear/Linear/CatmullRom
- firefox (skia): Lanczos3
Testing: Improvements in the following WPT tests
- html/canvas/element/manual/imagebitmap/*
Fixes (partially): #34112
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
On execution media element load algorithm
https://html.spec.whatwg.org/multipage/media.html#media-element-load-algorithm
we can observe race condition between parallel running fetch requests:
R1: (media element) initial request (see "resource_fetch_algorithm"
function)
R2: (gstreamer) duration "seek-data" request (see PlayerEvent::SeekData)
R3: (gstreamer) continue on last interrupted time "seek-data" request
At time there are only one current fetch context for media element but
because resource fetch and cancellation are async operations need
to identify (by request id) and skip processing outdated fetch request
(fetch listener callbacks).
Async load in background sequence (stream content length = 2757913):
```
R1 (seek-offset=0): [------------------------------X]
R2 (seek-offset=2757866): [---------Y]
AS-****-+++-AE
R3 (seek-offset=98304): [----------------Z]
BS-****-+++BE
R1 (seek-offset=0): [------X]
R2 (seek-offset=2757866): [--------Y] (discarded data)
AS-****---------+++-AE
- A*/B* performed seeks (*** seek lock, +++ flush buffer queue + reset EOS)
(on Gstreamer streaming thread)
- X/Y/Z "end-of-stream" events from fetch requests
(on Servo script thread)
```
All incoming input data from different requests are mixed and
pushed to active media player, plus abnormal behaviour due to
intermixed X/Y/Z "end-of-stream" events.
To handle it properly we will unblock seek lock on "seek-data" event
(on script thread) as soon as possible (GStreamer will be able
to flush buffer queue and reset EOS state) and introduce buffered data
source
to delay pushing input data/EOS event until media player will be
actually ready.
Testing: Improvements to the following tests:
-
/html/canvas/element/manual/imagebitmap/createImageBitmap-origin.sub.html
-
/html/semantics/embedded-content/the-canvas-element/security.pattern.fillStyle.sub.html
Fixes: https://github.com/servo/servo/issues/31931
Fixes: https://github.com/servo/servo/issues/36989
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Limit the maximum image allocation size to 2GB to minimize the
possibility of out of memory errors on some `ImageBitmap`, `ImageData`,
`Canvas`, and `OffscreenCanvas` operations such as construction,
`toBlob`, and `toDataURL`. Other browsers have similar limits:
- Chromium: 2^32-1 (~4GB)
- Firefox: 2^31-1 (~2GB)
Testing: Improvements to the following tests:
-
`html/canvas/element/pixel-manipulation/2d.imageData.object.ctor.basics.html`
assert_throws_dom("INDEX_SIZE_ERR", function() { new ImageData(1 << 31,
1 << 31); });
-
`html/canvas/element/manual/imagebitmap/createImageBitmap-invalid-args.html`
makeOversizedCanvas + makeOversizedOffscreenCanvas
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
This change adds support for rendering static SVG images using the
`resvg` crate, allowing svg sources in the `img` tag and in CSS
`background` and `content` properties. There are some limitations in
using resvg:
1. There is no support for animations or interactivity as these would
require implementing the full DOM layer of SVG specification.
2. Only system fonts can be used for text rendering. There is some
mechanism to provide a custom font resolver to usvg, but that is not
explored in this change.
3. resvg's handling of certain edge cases involving lack of explicit
`width` and `height` on the root svg element deviates from what the
specification expects from browsers. For example, resvg uses the values
in `viewBox` to derive the missing width or height dimension, but
without scaling that dimension to preserve the aspect ratio. It also
doesn't allow overriding this behavior.
Demo screenshot:
<details>
<summary>Source</summary>
```
<style>
#svg1 {
border: 1px solid red;
}
#svg2 {
border: 1px solid red;
width: 300px;
}
#svg3 {
border: 1px solid red;
width: 300px;
height: 200px;
object-fit: contain;
}
#svg4 {
border: 1px solid red;
width: 300px;
height: 200px;
object-fit: cover;
}
#svg5 {
border: 1px solid red;
width: 300px;
height: 200px;
object-fit: fill;
}
#svg6 {
border: 1px solid red;
width: 300px;
height: 200px;
object-fit: none;
}
</style>
</head>
<body>
<div>
<img id="svg1" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
</div>
<div>
<img id="svg2" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
<img id="svg3" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
<img id="svg4" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
</div>
<div>
<img id="svg5" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
<img id="svg6" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
</div>
</body>
```
</details>
---------
Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
According to specification ImageBitmap objects are serializable objects
and transferable objects.
https://html.spec.whatwg.org/multipage/#the-imagebitmap-interface:imagebitmap-11
Testing:
- html/canvas/element/manual/imagebitmap/*
- html/infrastructure/safe-passing-of-structured-data/*
- html/webappapis/structured-clone/*
- workers/semantics/structured-clone/*
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Add mock SVGImageElement interface to fix TIMEOUT WPT tests
which are related to ImageBitmap (html/canvas/*).
https://svgwg.org/svg2-draft/embedded.html#InterfaceSVGImageElement
Rationality of this change to fire event "error" on any attempt to fetch
image resource on href attribute change to not block WPT tests
execution.
Some WPT tests use the legacy namespace attribute "xlink:href", so
support for it was added to source code.
https://svgwg.org/svg2-draft/linking.html#XLinkHrefAttribute
- setAttributeNS("http://www.w3.org/1999/xlink", 'xlink:href', src);
Testing: Covered by existed WPT tests
- fetch/metadata/generated/svg-image*
- html/canvas/element/manual/*
- html/dom/idlharness.https.html
- html/semantics/embedded-content/the-canvas-element/*
- html/webappapis/scripting/events/event-handler-all-global-events.html
- mozilla/interfaces.https.html
Fixes: https://github.com/servo/servo/issues/35881
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
https://github.com/web-platform-tests/wpt/pull/50041 allows us to start
running the webdriver conformance tests in Servo, which will make it
easier for us to track regressions/improvements in our webdriver server
implementation.
---
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes are part of #15274
- [x] There are tests for these changes
---------
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
* Do not change media element ready state when network response is complete.
* Do not fire multiple error events for the same media content.
* Inform media backend when media response is complete.
* Continue delaying the load event when a complete media response is received.
* Only mark a media response as complete when the response is the active one.
* Update expectations for imagebitmap tests using video element.
* Update fetch ORB video test expectations.
* Update media CSS selector test expectation for non-implemented feature.
* Update expectations for media element tests that now work.
* Updat expected result for failing reftest.
* Update expected failure for test that loads an audio file in a video element.
* Update media test expectation for unimplemented track feature.
* Do not process media element ready state changes that are unchanged.
* Reset media element ready state to Current when playback finishes.
* Set media element ready state to Enough when appropriate player event is received.
* Update test expectations.
Most tests were only being run for layout-2013, not for layout-2020.
This wasn't great since layout-2020 is now the default.
So this patch unifies the lists of included tests for both layouts.
For layout-2013 this implies adding css/css-content/, css/css-logical/
and css/css-masking/clip/.
For layout-2020 this implies adding several additional css tests, and
also tests like dom/, js/, html/, etc.
