Remove the `ucd` dependency which has not been updated in 8 years. In
addition, replace it with a generated UnicodeBlock enum which reflects
the modern Unicode standard. This is generated via a Python script which
is included in the repository. The generation is not part of the build
process, because the Unicode database is hosted on the web and it does
not change the frequently.
This is done instead of bringing in the more up-to-date `unicode_blocks`
dependency. `unicode_blocks` defines each block as constant, which means
that they cannot be used in match statements -- which we do in Servo.
Co-authored-by: Lauryn Menard <lauryn.menard@gmail.com>
The main change here is that collapsed and `text-transform`'d text is
computed as it's processed by DOM traversal. This single transformed
text is stored in the root of the `InlineFormattingContext`.
This will eventually allow performing linebreaking and shaping of the
entire inline formatting context at once. Allowing for intelligent
processing of linebreaking and also shaping across elements. This
matches more closely what LayoutNG does.
This shouldn't have any (or negligable) behavioral changes, but will
allow us to prevent linebreaking inside of clusters in a followup
change.
Co-authored-by: Rakhi Sharma <atbrakhi@igalia.com>
This change adds support for `white-space-collapse: break-spaces` and
adds initial parsing support for `overflow-wrap` and `word-break`. The
later two properties are not fully supported, only in their interaction
with `break-spaces`. This is a preliminary change preparing to implement
them.
In addition, `break_and_shape` is now forked and added to Layout 2020.
This function is going to change a lot soon and forking is preparation
for this. More code that is only used by Layout 2013 is moved from `gfx`
to that crate.
Co-authored-by: Rakhi Sharma <atbrakhi@igalia.com>
The `gfx_traits` crate is gradually become just about text and fonts and
this is one of the few things there that is standing in the way of this.
Eventually `webrender_traits` will be the general cross-process
compositor API, so this sort of makes sense as well.
- Better detect situations where emoji is necessary by looking ahead one
character while laying out. This allow processing Unicode presentation
selectors. When detecting emoji, put emoji fonts at the front of
fallback lists for all platforms.
This enables monochrome emoji on Windows. Full-color emoji on Windows
probably needs full support for processing the COLR table and drawing
separate glyph color layers.
- Improve the font fallback list on FreeType platforms. Ideally, Servo
would be able to look through the entire font list to find the best
font for a certain character, but until that time we can make sure the
font list contains the "Noto Sans" fonts which cover most situations.
Fixes#31664.
Fixes#12944.
* fonts: Add support for the CSS font matching algorithm
This is a port from Gecko of the CSS font matching algorithm distance
functions as well as the "simple family" concept for optimizing
matching when dealing with simple fonts.
Fixes#189.
Fixes#190.
Fixes#20686.
Fixes#20684.
Co-authored-by: Mukilan Thiyagarajan <mukilan@igalia.com>
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
* Ensure that simple faces are removed for removed stylesheets
---------
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Co-authored-by: Mukilan Thiyagarajan <mukilan@igalia.com>
This is the first part of ensuring that unused fonts do not leak. This
change makes it so that when a stylesheet is removed, the corresponding
web fonts are removed from the `FontContext`.
Note: WebRender assets are still leaked, which was the situation before
for all fonts. A followup change will fix this issue.
Fixes#15139.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Co-authored-by: Mukilan Thiyagarajan <mukilan@igalia.com>
Instead of sending a message to the script thread via IPC when a web
font loads and then sending another, just give the `FontContext` a
callback that send a single message to the script thread. This moves all
the cache invalidation internally into `FontContext` as well.
Additionally, the unused LayoutControlMessage::ExitNow enum variant is
removed.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Co-authored-by: Mukilan Thiyagarajan <mukilan@igalia.com>
Instead of depending on `rust-freetype`, depend directly on
`freetype-sys` which is a transitive dependency. This provides almost
everything we need (apart from one function call). This will help us
eliminate one crate in the dependency chain.
This moves mangement of web fonts to the per-Layout `FontContext`,
preventing web fonts from being available in different Documents.
Fixes#12920.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Co-authored-by: Mukilan Thiyagarajan <mukilan@igalia.com>
For a long time, `gfx_traits` has held a lot of things unrelated to graphics
and also unrelated to the `gfx` crate (which is mostly about fonts).
This is a cleanup which does a few things:
1. Move non `gfx` crate things out of `gfx_traits`. This is important in
order to prevent dependency cycles with a different integration between
layout, script, and fonts.
2. Rename the `msg` crate to `base`. It didn't really contain anything
to do with messages and instead mostly holds ids, which are used
across many different crates in Servo. This new crate will hold the
*rare* data types that are widely used.
Details:
- All BackgroundHangMonitor-related things from base to a new
`background_hang_monitor_api` crate.
- Moved `TraversalDirection` to `script_traits`
- Moved `Epoch`-related things from `gfx_traits` to `base`.
- Moved `PrintTree` to base. This should be widely useful in Servo.
- Moved `WebrenderApi` from `base` to `webrender_traits` and renamed it
to `WebRenderFontApi`.
Color emoji support with "Noto Color Emoji" requires two things:
1. Support for bitmap fonts in the FreeType backend. This requires
specially handling bitmap fonts which have different characteristics
in the FreeType API (such as requiring metrics scaling). This support
is generally ported from Gecko's implementation.
2. When a character is an emoji it "Noto Color Emoji" needs to be in the
fallback list. Ensure that this is high on the list -- this will be
improved in a later PR.
This is a speculative fix for #32161. A similar failure is
reproducible on the Android x86_64 emulator with API 35
system image. The fix has not been validated on the actual
device so potentially there might be other issues that need
to be fixed to complete #32161.
Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>
* clippy: Squish warnings and errors in gfx
warning: redundant closure (gfx/font.rs:415:18)
warning: useless conversion to the same type (gfx/font.rs:534:9)
warning: the following explicit lifetimes could be elided: 'a (gfx/font.rs:619:16)
error: this loop never actually loops (gfx/font_cache_thread.rs:112:9)
warning: this expression creates a reference which is immediately dereferenced by the compiler (gfx/font_cache_thread.rs:229:51)
warning: redundant closure (gfx/font_cache_thread.rs:551:18)
3 instances of:
warning: casting integer literal to `f64` is unnecessary (gfx/platform/freetype/font_list.rs:271-273)
* clippy: methods called `from_*` usually take no `self`
It reports that by standard convention, from_* methods should not take any `&self` parameter
* clippy: you should consider adding a `Default` implementation
It reports that public types with a pub fn new() -> Self should have a Default implementation since they can be constructed without arguments
* clippy: casting to the same type is unnecessary (`f32` -> `f32`)
* clippy: use of `unwrap_or_else` to construct default value
* clippy: methods called `is_*` usually take `self` by mutable reference or `self` by reference or no `self`
* clippy: manual `!RangeInclusive::contains` implementation
contains expresses the intent better and has less failure modes (such as fencepost errors or using || instead of &&)
* clippy: this function has an empty `#[must_use]` attribute, but returns a type already marked as `#[must_use]`
* clippy: Fix some new warnings
warning: this `if` statement can be collapsed (gfx/font.rs:468:130)
warning: this lifetime isn't used in the impl (gfx/platform/freetype/font.rs:341:6)
warning: field assignment outside of initializer for an instance created with Default::default() (compositor.rs:881:17)
* Add OpenHarmony support for allocator / profile
Signed-off-by: Jonathan Schwender <jonathan.schwender@huawei.com>
* gfx: Build harfbuzz from source on OHOS
Updates `freetype-sys` to v0.20.1, which includes a build
fix for OpenHarmony.
Signed-off-by: Jonathan Schwender <jonathan.schwender@huawei.com>
* gfx: Don't depend on fontconfig on OpenHarmony
Signed-off-by: Jonathan Schwender <jonathan.schwender@huawei.com>
* gfx: Add ohos font fallback
Hardcode HarmonyOS_Sans_SC_Regular for Chinese
Signed-off-by: Jonathan Schwender <jonathan.schwender@huawei.com>
* libservo: OHOS useragent, and explicitly opt out of sandboxing
Signed-off-by: Jonathan Schwender <jonathan.schwender@huawei.com>
* libservo: Disable get_native_media_display_and_gl_context on ohos
Signed-off-by: Jonathan Schwender <jonathan.schwender@huawei.com>
---------
Signed-off-by: Jonathan Schwender <jonathan.schwender@huawei.com>
* servoshell: Upgrade `egui` and many other dependencies
This upgrades:
- `core-graphics`
- `core-text`
- `egui` and friends
- `font-kit`
- `glow` and friends
- `harfbuzz-sys`
- `jni`
- `nix`
- `raqote`
- `raw-window-handle`
- `winit`
* Downgrade jni until we can properly upgrade
* Update some test results
It's unclear why these are now passing, but they are.
---------
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
This allows sharing font templates, fonts, and platform fonts across
layout threads. It's the first step toward storing web fonts in the
layout versus the shared `FontCacheThread`. Now fonts and font groups
have some locking (especially on FreeType), which will probably affect
performance. On the other hand, we measured memory usage and this saves
roughly 40 megabytes of memory when loading servo.org based on data from
the memory profiler.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Co-authored-by: Mukilan Thiyagarajan <mukilan@igalia.com>
* feat: Support font-relative `ch` and `ic` units
After #31966, which made it possible for the first time to resolve
font-relative CSS units, this change adds support for the `ch` and
`ic` units.
One difference with the `ex` unit that was added in that PR is that
these units must reflect the advance width of a character (the zero
digit in the case of `ch`, and the CJK water radical for `ic`) as it
would be rendered by the current font group. This means that the size
of these units don't only depend on the first available font, in the
case where that font does not contain a glyph for that character.
This is implemented by adding the advance width for these two
characters as optional fields of `FontMetrics`, so the advance width
computation happens in advance. Then, when the font metrics are
queried as part of unit resolution, the font group is searched for the
first font containing that character.
This change only implements support for these units in upright
typesetting modes, since Servo does not yet have support for vertical
writing modes. This means that many of the WPT tests that test for the
behavior of these units with vertical writing modes do not pass.
This change also makes a number of WPT tests pass, which relied on the
`ch` and `ic` units. It, however, also makes the test
`/css/css-text/white-space/text-wrap-balance-overflow-002.html` fail,
since it tests `text-wrap: balance`, which Servo does not yet
implement, and it was only previously passing by chance due to the
previous behavior of these units.
* Revert Python 3.10-related changes to wss
* Fix formatting
* Remove test expectation
There are a couple major changes here:
1. Support is added for the `weight`, `style`, `stretch` and
`unicode-range` declarations in `@font-face`.
2. Font matching in the font cache can return templates and
`FontGroupFamily` can own mulitple templates. This is due to needing
support for "composite fonts". These are `@font-face` declarations
that only differ in their `unicode-range` definition.
This fixes a lot of non-determinism in font selection especially when
dealing with pages that define "composite faces." A notable example of
such a page is servo.org, which now consistently displays the correct
web font.
One test starts to fail due to an uncovered bug, but this will be fixed
in a followup change.
Fixes#20686.
Fixes#20684.
Co-authored-by: Mukilan Thiyagarajan <mukilan@igalia.com>
This combines `style()`, `boldness()`, `stretchiness()` into a
`descriptor()` method which is used when creating `FontTemplate`s for
web fonts. Eventually this method will simply read font tables using
skrifa. This is the first step.
In addition, `family_name()` and `face_name()` are removed. They were
only used for debugging and the `FontIdentifier` serves for that. On
Windows, this was adding another way in which font loading could fail,
without buying us very much. The path or URL to the font is more
important when debugging than the names in the font tables.
Closes#15103.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by
`[X]` when the step is complete, and replace `___` with appropriate
data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they should not change
observable behavior.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox
is checked, so that we can help you if you get stuck somewhere along the
way.-->
<!-- Pull requests that do not address these steps are welcome, but they
will require additional verification as part of the review process. -->
This fixes two issues that were preventing emojis from being properly
selected from fonts on macOS.
1. `CTFont::get_glyphs_for_characters` takes the input characters as
UniChar which are UTF-16 encoded characters. We need to encode the
input `char` as UTF-16 before passing it to CoreText.
2. The font fallback list is updated with the latest logic from Gecko,
which importantly adds "Apple Color Emoji" to the list of fallback
fonts. Sorry for the big change, but this is just a direct port of
the code from Gecko.
With these two changes, emojis display but in grayscale. 😅 To fix this,
another part of the font stack will need to detect when the font
supports color and pass that information to WebRender when creating the
font instance. We will likely do this in platform independent way later
that will depend on some more preliminary changes.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by
`[X]` when the step is complete, and replace `___` with appropriate
data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes are part of #17267.
- [x] There are tests for these changes, but the macOS CI does not
currently run WPT so we cannot observe the updated results.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox
is checked, so that we can help you if you get stuck somewhere along the
way.-->
<!-- Pull requests that do not address these steps are welcome, but they
will require additional verification as part of the review process. -->
This change reworks the way that platform fonts are created and
descriptor data is on `FontTemplate` is initialized.
The main change here is that platform fonts for local font faces are
always initialized using the font data loaded into memory from disk.
This means that there is now only a single path for creating platform
fonts.
In addition, the font list is now responsible for getting the
`FontTemplateDescriptor` for local `FontTemplate`s. Before the font had
to be loaded into memory to get the weight, style, and width used for
the descriptor. This is what fonts lists are for though, so for every
platform we have that information before needing to load the font. In
the future, hopefully this will allow discarding fonts before needing to
load them into memory. Web fonts still get the descriptor from the
platform handle, but hopefully that can be done with skrifa in the
future.
Thsese two fixes together allow properly loading indexed font variations
on Linux machines. Before only the first variation could be
instantiated.
Fixes https://github.com/servo/servo/issues/13317.
Fixes https://github.com/servo/servo/issues/24554.
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
----
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#13317 and #24554
- [x] There are tests for these changes
---------
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
* Simplify `FontHandle` and rename it to `PlatformFont`
Rename it to `PlatformFont` and move the `FontTemplate` member to
`Font`, because it's shared by all platforms.
* Update components/gfx/platform/freetype/font.rs
Co-authored-by: Mukilan Thiyagarajan <mukilanthiagarajan@gmail.com>
* Fix build for MacOS and Windows
---------
Co-authored-by: Mukilan Thiyagarajan <mukilanthiagarajan@gmail.com>
Now that `FontTemplateData` is more or less the same on all platforms,
it can be removed. This is a preparatory change for a full refactor of
the font system on Servo. The major changes here are:
- Remove `FontTemplateData` and move its members into `FontTemplate`
- Make `FontTemplate` have full interior mutability instead of only
the `FontTemplateData` member. This is preparation for having these
data types `Send` and `Sync` with locking.
- Remove the strong/weak reference concept for font data. In practice,
all font data references were strong, so this was never fully
complete. Instead of using this approach, the new font system will
use a central font data cache with references associated to layouts.
- The `CTFont` cache is now a global cache, so `CTFont`s can be shared
between threads. The cache is cleared when clearing font caches.
A benefit of this change (apart from `CTFont` sharing) is that font data
loading is platform-independent now.
The `FontContextHandle` was really only used on FreeType platforms to
store the `FT_Library` handle to use for creating faces. Each
`FontContext` and `FontCacheThread` would create its own
`FontContextHandle`. This change removes this data structure in favor of
a mutex-protected shared `FontContextHandle` for an entire Servo
process. The handle is initialized using a `OnceLock` to ensure that it
only happens once and also that it stays alive for the entire process
lifetime.
In addition to greatly simplifying the code, this will make it possible
for different threads to share platform-specific `FontHandle`s, avoiding
multiple allocations for a single font.
The only downside to all of this is that memory usage of FreeType fonts
isn't measured (though the mechanism is still there). This is because
the `FontCacheThread` currently doesn't do any memory measurement.
Eventually this *will* happen though, during the font system redesign.
In exchange, this should reduce the memory usage since there is only a
single FreeType library loaded into memory now.
This is part of #32033.
Since the original version of the CoreText font code, it has scaled the
metrics from CoreText by an unusual scale:
```
let scale = px_to_pt(self.ctfont.pt_size()) / (ascent + descent);
```
It's unclear what this scale was trying to accomplish. Note that it's
passing the return value of `pt_size()` to `px_to_pt` which seems
backward. This scale seems bogus, but perhaps it's based on a
misconception about what its returned from CoreText. Unlike the return
values of `CGFont` methods, which are returned in font units, the ones
from `CTFont` are "scaled according to the point size and matrix of the
font reference."
Indeed, when just interpreting these values as pixel values, the results
more or less match Firefox and Chrome. This becomes much more obvious
now that we have support for `ex` units. Even when not using `ex`, you
can sometimes see the top parts of glyphs cut off due to this scaling.
This change removes the scaling and simply interpets the return values
of `CTFont` methods as pixels. It addresses all of the issues mentioned
above. Note that this path will eventually just be a fallback path and
metrics will come from sfnt tables in the future.
The only font relative unit that Servo knows how to resolve currently is
`rem` (relative to the root font size). This is because Stylo cannot do
any font queries. This adds a mechanism to allow this, exposing the
ability to properly render `ex` units in Servo.
This change only allows resolving some font size relative units thoug,
as Servo doesn't collect all the FontMetrics it needs to resolve them
all. This capability will be added in followup changes.
Some new tests fail:
- ex-unit-001.html: This test fails because Servo does not yet have
support for setting the weight using @font-face rules on web fonts.
- ex-unit-004.html: This test fails because Servo does not yet have
support for setting the Unicode range of a web font using @font-face
rules.
- first-available-font-001.html: This test fails because the above
two feature are missing.
This change also makes two fixes that are necessary to get WOFF2 fonts
working:
1. It adds support for loading web fonts from stylesheets included via
@import rules.
2. It ensure that when web fonts are loaded synchronusly they invalidate
the font cache. This led to incorrect font rendering when running
tests before.
Fixes#31598.
There is now platform-specific way to get metrics for `line-through` on
MacOS and currently striking through simply does not work. The correct
approach here is likely to first search for these metrics in font tables
and then falling back to deriving them. Searching the font tables is a
larger change, so this change adds the fallback mechanism first. This at
least makes sure that strike through renders at all on Mac.
In a followup change we can add support for getting metrics via HarfBuzz
in a platform-independent way, which is what Gecko does.
Fixes#942.
* rustdoc: Fix fontface broken link error
* Correct link to `Font`
* Reduce the diff a bit
---------
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
The old logic was assuming that all whitespace was a break opportunity,
and that no newlines would be preserved.
Note that text shaping considers the advance of a newline to be the same
as a space. This was problematic because if we have a segment with a
preserved space and newline, only the advance of the space should
contrinute to the size of the block container. Therefore, I'm changing
the breaker logic in other to have newline characters in their own
segment.
Then glyph_run_is_whitespace_ending_with_preserved_newline can just be
renamed to glyph_run_is_preserved_newline.
This patch is still not perfect because it doesn't check allow_wrap(),
so `nowrap` is treated like `normal`, and `pre-wrap` like `pre`.
This brings the version of WebRender used in Servo up-to-date with Gecko
upstream. The big change here is that HiDPI is no longer handled via
WebRender. Instead this happens via a scale applied to the root layer in
the compositor. In addition to this change, various changes are made to
Servo to adapt to the new WebRender API.
Co-authored-by: Mukilan Thiyagarajan <mukilan@igalia.com>
