Instead of doing font selection and text shaping in `canvas`, move this
to `script`. This allows canvas to use the shared `Document`
`FontContext`, which has access to web fonts. In addition, ensure that
there is a font style accessible for `OffscreenCanvas` in workers.
Testing: This causes a number of WPT tests to start to pass as web fonts
are
supported on canvas again. In addition, some start to fail as they
expose other
issues:
- The lack of support for the `Context2D.fontStretch` property
- Issues with zerosize gradient interpolation.
- Differences between quoted and unquoted font family names. This seems
like
a timing issue with the way we are handling web fonts. The test seems to
be
expecting Local fonts to be available immediately (without waiting for
them
to load). This isn't how Servo works ATM. Seems like an issue with the
test.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
We really want to remove font-kit from dep tree, so this is the first
step into removing raqote from servo. While vello_cpu is not perfect
replacement, I am confident that we will resolve all issues eventually:
https://github.com/servo/servo/issues/38345 (most important ones already
have PRs).
Reviewable per commit.
Testing: Existing WPT tests.
Try run: https://github.com/sagudev/servo/actions/runs/17138369290
---------
Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>
According to my tests `OptimizeSpeed` slight improves performance and it
does NOT affect WPT results in negative way.
Testing: Tested by existing WPT tests.
---------
Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>
As noted in #38345, vello scenes only grow. While we can reset them when
clearing viewport (#38356) that is not enough. We need to reset scene on
each render (~each frame) and providing old frame as backdrop to new
scene. Be do this lazily so multiple rendering without any changes
should be cheaper, we still do GPUBuffer mapping, because that would
require more complex work.
Testing: Code functionality is covered by existing WPT tests, but we do
not have any performance test.
---------
Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>
Add vello backend by implementing Backend traits in canvas crate (so
this lives in canvas_paint_thread - embedded process). Current
implementation uses normal wgpu, so we block on GPU work. Vello backend
is gated behind `vello` feature and `dom_canvas_vello_enabled` pref.
Feature-wise this backend is on on par with raqote (sometimes better
sometimes worse), but performance wise it's worse.
## Known vello problems:
- image roundtrip does not work (fixed in
https://github.com/linebender/vello/pull/974)
- https://github.com/linebender/vello/issues/1066 (fixed)
- clip layers are not working properly:
https://github.com/linebender/vello/issues/1061
- `/html/canvas/element/pixel-manipulation/2d.imageData.put.*`
- `/html/canvas/element/path-objects/2d.path.clip.intersect.html`
- https://github.com/linebender/vello/issues/1056
-
`/html/canvas/element/fill-and-stroke-styles/2d.gradient.interpolate.coloralpha.html`
- `kurbo::Cap::Butt` is defect (only visible with big lineWidth)
https://github.com/linebender/vello/issues/1063
- `/html/canvas/element/line-styles/2d.line.cross.html`
- `/html/canvas/element/line-styles/2d.line.miter.acute.html`
- other lack of strong correct problems
(https://github.com/linebender/vello/issues/1063#issuecomment-2998084736):
- `/html/canvas/element/path-objects/2d.path.rect.selfintersect.html`
- There is currently no way to do put image properly in vello as we
would need to ignore all clips and other stuff (we try to work around
this on best effort basis)
https://github.com/linebender/vello/issues/1088
- `/html/canvas/element/pixel-manipulation/2d.imageData.put.*`
- precision problems
- `/html/canvas/element/path-objects/2d.path.stroke.scale2.html`
- `/html/canvas/element/path-objects/2d.path.arc.scale.1.html`
## Known servo problems
- bad performance due to blocking on GPU work
- some get/put intensive tests `TIMEOUT`
- proper shadow support (non-blocker as we already are living without it
now)
- support for rect shadow is there but unimplemented currently as that's
the state in raqote
Testing: `mach try vello` will run normal WPT (with raqote) +
vello_canvas subsuite that runs only on `/html/canvas/element`. All
subsuite expectations are stored separately.
Fixes: #36823Fixes: #35230
---------
Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>
Add support of the ImageBitmapRenderingContext as "bitmaprenderer"
canvas context mode to RenderingContext/OffscreenRenderingContext
https://html.spec.whatwg.org/multipage/#imagebitmaprenderingcontext
It is initial implementation with public interface API but without
any display presentation support for HTMLCanvasElement.
Testing: Improvements in the following tests:
-
html/canvas/element/manual/imagebitmap/createImageBitmap-origin.sub.html
- html/canvas/offscreen/manual/text/canvas.2d.offscreen*
-
html/canvas/offscreen/manual/the-offscreen-canvas/offscreencanvas.transfer.to.imagebitmap.nocrash.html
- imagebitmap-renderingcontext/*
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
This removes assumption about pixel format from backend abstraction to
actual backend implementation. This is important for vello.
Testing: WPT tests
---------
Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>
Currently, HTMLImageElement uses as an image source
(ImageBitmapSource, CanvasImageSource, TexImageSource)
in various canvas2D/WebGL operations, and there is a small
inconsistency in how we get the image data of the 'img' element:
usability checking and retrieving the image data from the image cache.
To simplify and avoid state inconsistency between the window's image
cache and the 'img' element, let's retrieve the image data (as a raster)
from the HTMLImageElement itself.
Testing: No expected changes in testing results, except the
'drawimage_svg_image_with_foreign_object_does_not_taint.html'
which is 'false' passed because drawing of the non supported vector
image
is silently skip instead of throwing the 'InvalidState' exception
anymore.
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
The 'none' image orientation option was temporarily deprecated (in 2022)
according to HTML specification with the remark about going to be reused
with
a different meaning in future as the same semantics as CSS 'none'
image-orientation.
https://html.spec.whatwg.org/multipage/#dom-imagebitmapoptions-imageorientation-nonehttps://www.w3.org/TR/css-images-3/#valdef-image-orientation-none
Official MDN documentation added it back with new meaning (in 2024),
but it wasn't added back to HTML specification (still pending).
https://developer.mozilla.org/en-US/docs/Web/API/Window/createImageBitmap#none
At current moment this option is poor supported on all major browsers,
but there are some
existed WPT tests (ImageBitmap/WebGL) which are actively use it in
createImageBitmap() options.
Chromium (supported):
- stable: same as 'from-image' (but with deprecation warning)
- experimental: ignoring any orientation metadata
Firefox (supported as default option):
- stable: ignoring any orientation metadata ('from-image' do the same)
Testing: Improvements and fails (expects 'from-image' behaviour from
'none' option) in the following tests
-
html/canvas/element/manual/imagebitmap/createImageBitmap-exif-orientation_none.html
- webgl/tests/conformance/textures/misc/exif-orientation.html
Fixes (partialy): #34112
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Follow to the HTML specification and support of Blob as
ImageBitmapSource
to able use it as intermediate instance in "fetch() -> Blob ->
ImageBitmap" execution sequence.
https://html.spec.whatwg.org/multipage/#imagebitmapsource
The specification says what these steps must run in parallel
(outside the createImageBitmap task), but currently loading bytes from
Blob
and later image decoding happen in synchronous order while
promise is fullfilled or rejected on bitmap task source.
https://html.spec.whatwg.org/multipage/#the-imagebitmap-interface:blob-4
Testing: Improvements in the following WPT tests
- html/canvas/element/compositing/2d.composite*
- html/canvas/element/drawing-images-to-the-canvas/2d.drawImage*
- html/canvas/element/manual/imagebitmap/createImageBitmap*
- html/canvas/offscreen/compositing/2d.composite
- html/canvas/offscreen/drawing-images-to-the-canvas/2d.drawImage
- webgl/tests/conformance/textures/image_bitmap_from_blob/*
Fixes (partially): #34112
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
According to HTML specification the poster attribute determines the
element's poster frame
(regardless of the value of the element's show poster flag).
https://html.spec.whatwg.org/multipage/#poster-frame
So the poster frame and the show poster flag is orthogonal to each
other, the latest one only controls
when browser should display the poster frame and should do not block
accepting video frames
from media pipeline (e.g. on new_preroll callback from video sink).
During layout the video element should select the current frame which
will be presented
based on first matching condition from the list:
https://html.spec.whatwg.org/multipage/#the-video-element:the-video-element-7
Testing: Improvements in the following WPT tests
- html/canvas/element/manual/imagebitmap/createImageBitmap*
- html/semantics/embedded-content/the-canvas-element/*
Fixes: #37165
---------
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
Follow the ImageBitmap specification and use the global scope bitmap
task source
to fulfill resolved promise (asynchronously).
https://html.spec.whatwg.org/multipage/#bitmap-task-source
Any promise rejection must be done synchronously.
Testing: Improvements in the following WPT test
-
html/canvas/element/manual/imagebitmap/createImageBitmap-resolves-in-task.any.js
Fixes (partially): #34112
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Follow the ImageBitmap specification and make cropping of the bitmap
data to the source rectangle with formatting:
https://html.spec.whatwg.org/multipage/#cropped-to-the-source-rectangle-with-formatting
For now the next functionality not implemented:
- image orientation support (such as EXIF metadata)
- color space conversion (image, blob)
The convertion from ResizeQuality to "image" FilterType:
- pixelated/low/medium/high -> Nearest/Triangle/CatmullRom/Lanczos3
Other browsers use the following sample filtering:
- chromium (skia): Nearest/Linear/Linear/CatmullRom
- firefox (skia): Lanczos3
Testing: Improvements in the following WPT tests
- html/canvas/element/manual/imagebitmap/*
Fixes (partially): #34112
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
On execution media element load algorithm
https://html.spec.whatwg.org/multipage/media.html#media-element-load-algorithm
we can observe race condition between parallel running fetch requests:
R1: (media element) initial request (see "resource_fetch_algorithm"
function)
R2: (gstreamer) duration "seek-data" request (see PlayerEvent::SeekData)
R3: (gstreamer) continue on last interrupted time "seek-data" request
At time there are only one current fetch context for media element but
because resource fetch and cancellation are async operations need
to identify (by request id) and skip processing outdated fetch request
(fetch listener callbacks).
Async load in background sequence (stream content length = 2757913):
```
R1 (seek-offset=0): [------------------------------X]
R2 (seek-offset=2757866): [---------Y]
AS-****-+++-AE
R3 (seek-offset=98304): [----------------Z]
BS-****-+++BE
R1 (seek-offset=0): [------X]
R2 (seek-offset=2757866): [--------Y] (discarded data)
AS-****---------+++-AE
- A*/B* performed seeks (*** seek lock, +++ flush buffer queue + reset EOS)
(on Gstreamer streaming thread)
- X/Y/Z "end-of-stream" events from fetch requests
(on Servo script thread)
```
All incoming input data from different requests are mixed and
pushed to active media player, plus abnormal behaviour due to
intermixed X/Y/Z "end-of-stream" events.
To handle it properly we will unblock seek lock on "seek-data" event
(on script thread) as soon as possible (GStreamer will be able
to flush buffer queue and reset EOS state) and introduce buffered data
source
to delay pushing input data/EOS event until media player will be
actually ready.
Testing: Improvements to the following tests:
-
/html/canvas/element/manual/imagebitmap/createImageBitmap-origin.sub.html
-
/html/semantics/embedded-content/the-canvas-element/security.pattern.fillStyle.sub.html
Fixes: https://github.com/servo/servo/issues/31931
Fixes: https://github.com/servo/servo/issues/36989
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Limit the maximum image allocation size to 2GB to minimize the
possibility of out of memory errors on some `ImageBitmap`, `ImageData`,
`Canvas`, and `OffscreenCanvas` operations such as construction,
`toBlob`, and `toDataURL`. Other browsers have similar limits:
- Chromium: 2^32-1 (~4GB)
- Firefox: 2^31-1 (~2GB)
Testing: Improvements to the following tests:
-
`html/canvas/element/pixel-manipulation/2d.imageData.object.ctor.basics.html`
assert_throws_dom("INDEX_SIZE_ERR", function() { new ImageData(1 << 31,
1 << 31); });
-
`html/canvas/element/manual/imagebitmap/createImageBitmap-invalid-args.html`
makeOversizedCanvas + makeOversizedOffscreenCanvas
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
This change adds support for rendering static SVG images using the
`resvg` crate, allowing svg sources in the `img` tag and in CSS
`background` and `content` properties. There are some limitations in
using resvg:
1. There is no support for animations or interactivity as these would
require implementing the full DOM layer of SVG specification.
2. Only system fonts can be used for text rendering. There is some
mechanism to provide a custom font resolver to usvg, but that is not
explored in this change.
3. resvg's handling of certain edge cases involving lack of explicit
`width` and `height` on the root svg element deviates from what the
specification expects from browsers. For example, resvg uses the values
in `viewBox` to derive the missing width or height dimension, but
without scaling that dimension to preserve the aspect ratio. It also
doesn't allow overriding this behavior.
Demo screenshot:
<details>
<summary>Source</summary>
```
<style>
#svg1 {
border: 1px solid red;
}
#svg2 {
border: 1px solid red;
width: 300px;
}
#svg3 {
border: 1px solid red;
width: 300px;
height: 200px;
object-fit: contain;
}
#svg4 {
border: 1px solid red;
width: 300px;
height: 200px;
object-fit: cover;
}
#svg5 {
border: 1px solid red;
width: 300px;
height: 200px;
object-fit: fill;
}
#svg6 {
border: 1px solid red;
width: 300px;
height: 200px;
object-fit: none;
}
</style>
</head>
<body>
<div>
<img id="svg1" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
</div>
<div>
<img id="svg2" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
<img id="svg3" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
<img id="svg4" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
</div>
<div>
<img id="svg5" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
<img id="svg6" src="https://raw.githubusercontent.com/servo/servo/refs/heads/main/resources/servo.svg" alt="Servo logo">
</div>
</body>
```
</details>
---------
Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
According to specification ImageBitmap objects are serializable objects
and transferable objects.
https://html.spec.whatwg.org/multipage/#the-imagebitmap-interface:imagebitmap-11
Testing:
- html/canvas/element/manual/imagebitmap/*
- html/infrastructure/safe-passing-of-structured-data/*
- html/webappapis/structured-clone/*
- workers/semantics/structured-clone/*
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Add mock SVGImageElement interface to fix TIMEOUT WPT tests
which are related to ImageBitmap (html/canvas/*).
https://svgwg.org/svg2-draft/embedded.html#InterfaceSVGImageElement
Rationality of this change to fire event "error" on any attempt to fetch
image resource on href attribute change to not block WPT tests
execution.
Some WPT tests use the legacy namespace attribute "xlink:href", so
support for it was added to source code.
https://svgwg.org/svg2-draft/linking.html#XLinkHrefAttribute
- setAttributeNS("http://www.w3.org/1999/xlink", 'xlink:href', src);
Testing: Covered by existed WPT tests
- fetch/metadata/generated/svg-image*
- html/canvas/element/manual/*
- html/dom/idlharness.https.html
- html/semantics/embedded-content/the-canvas-element/*
- html/webappapis/scripting/events/event-handler-all-global-events.html
- mozilla/interfaces.https.html
Fixes: https://github.com/servo/servo/issues/35881
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
https://github.com/web-platform-tests/wpt/pull/50041 allows us to start
running the webdriver conformance tests in Servo, which will make it
easier for us to track regressions/improvements in our webdriver server
implementation.
---
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes are part of #15274
- [x] There are tests for these changes
---------
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
