Implement font fallback

Prior to this change, if none of the fonts specified in CSS contained a
glyph for a codepoint, we tried only one fallback font. If that font
didn't contain the glyph, we'd give up.

With this change, we try multiple fonts in turn. The font names we try
differ across each platform, and based on the codepoint we're trying to
match. The current implementation is heavily inspired by the analogous
code in Gecko, but I've used to ucd lib to make it more readable,
whereas Gecko matches raw unicode ranges.

This fixes some of the issues reported in #17267, although colour emoji
support is not implemented.

== Notes on changes to WPT metadata ==

=== css/css-text/i18n/css3-text-line-break-opclns-* ===

A bunch of these have started failing on macos when they previously
passed.

These tests check that the browser automatically inserts line breaks
near certain characters that are classified as "opening and closing
punctuation". The idea is that if we have e.g. an opening parenthesis,
it does not make sense for it to appear at the end of a line box; it
should "stick" to the next character and go into the next line box.

Before this change, a lot of these codepoints rendered as a missing
glyph on Mac and Linux. In some cases, that meant that the test was
passing.

After this change, a bunch of these codepoints are now rendering glyphs
on Mac (but not Linux). In some cases, the test should continue to pass
where it previously did when rendering with the missing glyph.

However, it seems this has also exposed a layout bug. The "ref" div in
these tests contains a <br> element, and it seems that this, combined
with these punctuation characters, makes the spacing between glyphs ever
so slightly different to the "test" div. (Speculation: might be
something to do with shaping?)

Therefore I've had to mark a bunch of these tests failing on mac.

=== css/css-text/i18n/css3-text-line-break-baspglwj-* ===

Some of these previously passed on Mac due to a missing glyph. Now that
we're rendering the correct glyph, they are failing.

=== css/css-text/word-break/word-break-normal-bo-000.html ===

The characters now render correctly on Mac, and the test is passing. But
we do not find a suitable fallback font on Linux, so it is still failing
on that platform.

=== css/css-text/word-break/word-break-break-all-007.html ===

This was previously passing on Mac, but only because missing character
glyphs were rendered. Now that a fallback font is able to be found, it
(correctly) fails.

=== mozilla/tests/css/font_fallback_* ===

These are new tests added in this commit. 01 and 02 are marked failing
on Linux because the builders don't have the appropriate fonts installed
(that will be a follow-up).

Fix build errors from rebase

FontTemplateDescriptor can no longer just derive(Hash). We need to
implement it on each component part, because the components now
generally wrap floats, which do not impl Hash because of NaN. However in
this case we know that we won't have a NaN, so it is safe to manually
impl Hash.
This commit is contained in:
Jon Leighton 2018-02-26 13:41:20 +00:00
parent 15a677c639
commit 691c6c6f1a
81 changed files with 1192 additions and 360 deletions

View file

@ -6,6 +6,8 @@ use std::cell::RefCell;
use std::fs::File;
use std::io::{self, Read};
use std::path::Path;
use text::util::is_cjk;
use ucd::{Codepoint, UnicodeBlock};
use xml5ever::Attribute;
use xml5ever::driver::parse_document;
use xml5ever::rcdom::*;
@ -470,12 +472,61 @@ pub fn system_default_family(generic_name: &str) -> Option<String> {
}
}
pub fn last_resort_font_families() -> Vec<String> {
vec!(
"sans-serif".to_owned(),
"Droid Sans".to_owned(),
"serif".to_owned(),
)
// Based on gfxAndroidPlatform::GetCommonFallbackFonts() in Gecko
pub fn fallback_font_families(codepoint: Option<char>) -> Vec<&'static str> {
let mut families = vec!();
if let Some(block) = codepoint.and_then(|c| c.block()) {
match block {
UnicodeBlock::Armenian => {
families.push("Droid Sans Armenian");
}
UnicodeBlock::Hebrew => {
families.push("Droid Sans Hebrew");
}
UnicodeBlock::Arabic => {
families.push("Droid Sans Arabic");
}
UnicodeBlock::Devanagari => {
families.push("Noto Sans Devanagari");
families.push("Droid Sans Devanagari");
}
UnicodeBlock::Tamil => {
families.push("Noto Sans Tamil");
families.push("Droid Sans Tamil");
}
UnicodeBlock::Thai => {
families.push("Noto Sans Thai");
families.push("Droid Sans Thai");
}
UnicodeBlock::Georgian |
UnicodeBlock::GeorgianSupplement => {
families.push("Droid Sans Georgian");
}
UnicodeBlock::Ethiopic |
UnicodeBlock::EthiopicSupplement => {
families.push("Droid Sans Ethiopic");
}
_ => {
if is_cjk(codepoint.unwrap()) {
families.push("MotoyaLMaru");
families.push("Noto Sans CJK JP");
families.push("Droid Sans Japanese");
}
}
}
}
families.push("Droid Sans Fallback");
families
}
pub static SANS_SERIF_FONT_FAMILY: &'static str = "sans-serif";

View file

@ -187,7 +187,7 @@ impl FontHandleMethods for FontHandle {
} else {
FontStretchKeyword::Normal
}.compute();
NonNegative(percentage)
FontStretch(NonNegative(percentage))
}
fn glyph_index(&self, codepoint: char) -> Option<GlyphId> {

View file

@ -10,10 +10,10 @@ use fontconfig::fontconfig::{FcFontSetList, FcObjectSetCreate, FcObjectSetDestro
use fontconfig::fontconfig::{FcObjectSetAdd, FcPatternGetInteger};
use libc;
use libc::{c_char, c_int};
use std::borrow::ToOwned;
use std::ffi::CString;
use std::ptr;
use super::c_str_to_string;
use text::util::is_cjk;
static FC_FAMILY: &'static [u8] = b"family\0";
static FC_FILE: &'static [u8] = b"file\0";
@ -132,12 +132,25 @@ pub fn system_default_family(generic_name: &str) -> Option<String> {
}
}
pub fn last_resort_font_families() -> Vec<String> {
vec!(
"Fira Sans".to_owned(),
"DejaVu Sans".to_owned(),
"Arial".to_owned()
)
}
pub static SANS_SERIF_FONT_FAMILY: &'static str = "DejaVu Sans";
// Based on gfxPlatformGtk::GetCommonFallbackFonts() in Gecko
pub fn fallback_font_families(codepoint: Option<char>) -> Vec<&'static str> {
let mut families = vec!(
"DejaVu Serif",
"FreeSerif",
"DejaVu Sans",
"FreeSans",
);
if let Some(codepoint) = codepoint {
if is_cjk(codepoint) {
families.push("TakaoPGothic");
families.push("Droid Sans Fallback");
families.push("WenQuanYi Micro Hei");
families.push("NanumGothic");
}
}
families
}

View file

@ -229,7 +229,7 @@ impl FontHandleMethods for FontHandle {
use style::values::generics::NonNegative;
let normalized = self.ctfont.all_traits().normalized_width(); // [-1.0, 1.0]
NonNegative(Percentage(normalized as f32 + 1.0))
FontStretch(NonNegative(Percentage(normalized as f32 + 1.0)))
}
fn glyph_index(&self, codepoint: char) -> Option<GlyphId> {

View file

@ -3,7 +3,8 @@
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
use core_text;
use std::borrow::ToOwned;
use text::util::unicode_plane;
use ucd::{Codepoint, UnicodeBlock};
pub fn for_each_available_family<F>(mut callback: F) where F: FnMut(String) {
let family_names = core_text::font_collection::get_family_names();
@ -28,8 +29,162 @@ pub fn system_default_family(_generic_name: &str) -> Option<String> {
None
}
pub fn last_resort_font_families() -> Vec<String> {
vec!("Arial Unicode MS".to_owned(), "Arial".to_owned())
// Based on gfxPlatformMac::GetCommonFallbackFonts() in Gecko
pub fn fallback_font_families(codepoint: Option<char>) -> Vec<&'static str> {
let mut families = vec!("Lucida Grande");
if let Some(codepoint) = codepoint {
match unicode_plane(codepoint) {
// https://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane
0 => {
if let Some(block) = codepoint.block() {
match block {
UnicodeBlock::Arabic |
UnicodeBlock::Syriac |
UnicodeBlock::ArabicSupplement |
UnicodeBlock::Thaana |
UnicodeBlock::NKo => {
families.push("Geeza Pro");
}
UnicodeBlock::Devanagari => {
families.push("Devanagari Sangam MN");
}
UnicodeBlock::Gurmukhi => {
families.push("Gurmukhi MN");
}
UnicodeBlock::Gujarati => {
families.push("Gujarati Sangam MN");
}
UnicodeBlock::Tamil => {
families.push("Tamil MN");
}
UnicodeBlock::Lao => {
families.push("Lao MN");
}
UnicodeBlock::Tibetan => {
families.push("Songti SC");
}
UnicodeBlock::Myanmar => {
families.push("Myanmar MN");
}
UnicodeBlock::Ethiopic |
UnicodeBlock::EthiopicSupplement |
UnicodeBlock::EthiopicExtended |
UnicodeBlock::EthiopicExtendedA => {
families.push("Kefa");
}
UnicodeBlock::Cherokee => {
families.push("Plantagenet Cherokee");
}
UnicodeBlock::UnifiedCanadianAboriginalSyllabics |
UnicodeBlock::UnifiedCanadianAboriginalSyllabicsExtended => {
families.push("Euphemia UCAS");
}
UnicodeBlock::Mongolian |
UnicodeBlock::YiSyllables |
UnicodeBlock::YiRadicals => {
families.push("STHeiti");
}
UnicodeBlock::Khmer |
UnicodeBlock::KhmerSymbols => {
families.push("Khmer MN");
}
UnicodeBlock::TaiLe => {
families.push("Microsoft Tai Le");
}
UnicodeBlock::GeneralPunctuation |
UnicodeBlock::SuperscriptsandSubscripts |
UnicodeBlock::CurrencySymbols |
UnicodeBlock::CombiningDiacriticalMarksforSymbols |
UnicodeBlock::LetterlikeSymbols |
UnicodeBlock::NumberForms |
UnicodeBlock::Arrows |
UnicodeBlock::MathematicalOperators |
UnicodeBlock::MiscellaneousTechnical |
UnicodeBlock::ControlPictures |
UnicodeBlock::OpticalCharacterRecognition |
UnicodeBlock::EnclosedAlphanumerics |
UnicodeBlock::BoxDrawing |
UnicodeBlock::BlockElements |
UnicodeBlock::GeometricShapes |
UnicodeBlock::MiscellaneousSymbols |
UnicodeBlock::Dingbats |
UnicodeBlock::MiscellaneousMathematicalSymbolsA |
UnicodeBlock::SupplementalArrowsA |
UnicodeBlock::SupplementalArrowsB |
UnicodeBlock::MiscellaneousMathematicalSymbolsB |
UnicodeBlock::SupplementalMathematicalOperators |
UnicodeBlock::MiscellaneousSymbolsandArrows |
UnicodeBlock::SupplementalPunctuation => {
families.push("Hiragino Kaku Gothic ProN");
families.push("Apple Symbols");
families.push("Menlo");
families.push("STIXGeneral");
}
UnicodeBlock::BraillePatterns => {
families.push("Apple Braille");
}
UnicodeBlock::Bopomofo |
UnicodeBlock::HangulCompatibilityJamo |
UnicodeBlock::Kanbun |
UnicodeBlock::BopomofoExtended |
UnicodeBlock::CJKStrokes |
UnicodeBlock::KatakanaPhoneticExtensions => {
families.push("Hiragino Sans GB");
}
UnicodeBlock::YijingHexagramSymbols |
UnicodeBlock::CyrillicExtendedB |
UnicodeBlock::Bamum |
UnicodeBlock::ModifierToneLetters |
UnicodeBlock::LatinExtendedD |
UnicodeBlock::ArabicPresentationFormsA |
UnicodeBlock::HalfwidthandFullwidthForms |
UnicodeBlock::Specials => {
families.push("Apple Symbols");
}
_ => {}
}
}
}
// https://en.wikipedia.org/wiki/Plane_(Unicode)#Supplementary_Multilingual_Plane
1 => {
families.push("Apple Symbols");
families.push("STIXGeneral");
}
// https://en.wikipedia.org/wiki/Plane_(Unicode)#Supplementary_Ideographic_Plane
2 => {
// Systems with MS Office may have these fonts
families.push("MingLiU-ExtB");
families.push("SimSun-ExtB");
}
_ => {}
}
}
families.push("Geneva");
families.push("Arial Unicode MS");
families
}
pub static SANS_SERIF_FONT_FAMILY: &'static str = "Helvetica";

View file

@ -163,7 +163,7 @@ impl FontInfo {
let weight = StyleFontWeight(weight_val as f32);
let stretch = NonNegative(match min(9, max(1, width_val)) {
let stretch = StyleFontStretch(NonNegative(match min(9, max(1, width_val)) {
1 => FontStretchKeyword::UltraCondensed,
2 => FontStretchKeyword::ExtraCondensed,
3 => FontStretchKeyword::Condensed,
@ -174,7 +174,7 @@ impl FontInfo {
8 => FontStretchKeyword::ExtraExpanded,
9 => FontStretchKeyword::UltraExpanded,
_ => return Err(()),
}.compute());
}.compute()));
let style = if italic_bool {
GenericFontStyle::Italic
@ -212,7 +212,7 @@ impl FontInfo {
// slightly blacker black
FontWeight::ExtraBlack => 1000.,
});
let stretch = NonNegative(match font.stretch() {
let stretch = StyleFontStretch(NonNegative(match font.stretch() {
FontStretch::Undefined => FontStretchKeyword::Normal,
FontStretch::UltraCondensed => FontStretchKeyword::UltraCondensed,
FontStretch::ExtraCondensed => FontStretchKeyword::ExtraCondensed,
@ -223,7 +223,7 @@ impl FontInfo {
FontStretch::Expanded => FontStretchKeyword::Expanded,
FontStretch::ExtraExpanded => FontStretchKeyword::ExtraExpanded,
FontStretch::UltraExpanded => FontStretchKeyword::UltraExpanded,
}.compute());
}.compute()));
Ok(FontInfo {
family_name: font.family_name(),

View file

@ -7,6 +7,8 @@ use servo_atoms::Atom;
use std::collections::HashMap;
use std::sync::Mutex;
use std::sync::atomic::{Ordering, AtomicUsize};
use text::util::unicode_plane;
use ucd::{Codepoint, UnicodeBlock};
lazy_static! {
static ref FONT_ATOM_COUNTER: AtomicUsize = AtomicUsize::new(1);
@ -19,10 +21,6 @@ pub fn system_default_family(_: &str) -> Option<String> {
Some("Verdana".to_owned())
}
pub fn last_resort_font_families() -> Vec<String> {
vec!("Arial".to_owned())
}
pub fn for_each_available_family<F>(mut callback: F) where F: FnMut(String) {
let system_fc = FontCollection::system();
for family in system_fc.families_iter() {
@ -69,3 +67,270 @@ pub fn font_from_atom(ident: &Atom) -> Font {
let fonts = FONT_ATOM_MAP.lock().unwrap();
FontCollection::system().get_font_from_descriptor(fonts.get(ident).unwrap()).unwrap()
}
// Based on gfxWindowsPlatform::GetCommonFallbackFonts() in Gecko
pub fn fallback_font_families(codepoint: Option<char>) -> Vec<&'static str> {
let mut families = vec!("Arial");
if let Some(codepoint) = codepoint {
match unicode_plane(codepoint) {
// https://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane
0 => {
if let Some(block) = codepoint.block() {
match block {
UnicodeBlock::CyrillicSupplement |
UnicodeBlock::Armenian |
UnicodeBlock::Hebrew => {
families.push("Estrangelo Edessa");
families.push("Cambria");
}
UnicodeBlock::Arabic |
UnicodeBlock::ArabicSupplement => {
families.push("Microsoft Uighur");
}
UnicodeBlock::Syriac => {
families.push("Estrangelo Edessa");
}
UnicodeBlock::Thaana => {
families.push("MV Boli");
}
UnicodeBlock::NKo => {
families.push("Ebrima");
}
UnicodeBlock::Devanagari |
UnicodeBlock::Bengali => {
families.push("Nirmala UI");
families.push("Utsaah");
families.push("Aparajita");
}
UnicodeBlock::Gurmukhi |
UnicodeBlock::Gujarati |
UnicodeBlock::Oriya |
UnicodeBlock::Tamil |
UnicodeBlock::Telugu |
UnicodeBlock::Kannada |
UnicodeBlock::Malayalam |
UnicodeBlock::Sinhala |
UnicodeBlock::Lepcha |
UnicodeBlock::OlChiki |
UnicodeBlock::CyrillicExtendedC |
UnicodeBlock::SundaneseSupplement |
UnicodeBlock::VedicExtensions => {
families.push("Nirmala UI");
}
UnicodeBlock::Thai => {
families.push("Leelawadee UI");
}
UnicodeBlock::Lao => {
families.push("Lao UI");
}
UnicodeBlock::Myanmar |
UnicodeBlock::MyanmarExtendedA |
UnicodeBlock::MyanmarExtendedB => {
families.push("Myanmar Text");
}
UnicodeBlock::HangulJamo |
UnicodeBlock::HangulJamoExtendedA |
UnicodeBlock::HangulSyllables |
UnicodeBlock::HangulJamoExtendedB |
UnicodeBlock::HangulCompatibilityJamo => {
families.push("Malgun Gothic");
}
UnicodeBlock::Ethiopic |
UnicodeBlock::EthiopicSupplement |
UnicodeBlock::EthiopicExtended |
UnicodeBlock::EthiopicExtendedA => {
families.push("Nyala");
}
UnicodeBlock::Cherokee => {
families.push("Plantagenet Cherokee");
}
UnicodeBlock::UnifiedCanadianAboriginalSyllabics |
UnicodeBlock::UnifiedCanadianAboriginalSyllabicsExtended => {
families.push("Euphemia");
families.push("Segoe UI");
}
UnicodeBlock::Khmer |
UnicodeBlock::KhmerSymbols => {
families.push("Khmer UI");
families.push("Leelawadee UI");
}
UnicodeBlock::Mongolian => {
families.push("Mongolian Baiti");
}
UnicodeBlock::TaiLe => {
families.push("Microsoft Tai Le");
}
UnicodeBlock::NewTaiLue => {
families.push("Microsoft New Tai Lue");
}
UnicodeBlock::Buginese |
UnicodeBlock::TaiTham |
UnicodeBlock::CombiningDiacriticalMarksExtended => {
families.push("Leelawadee UI");
}
UnicodeBlock::GeneralPunctuation |
UnicodeBlock::SuperscriptsandSubscripts |
UnicodeBlock::CurrencySymbols |
UnicodeBlock::CombiningDiacriticalMarksforSymbols |
UnicodeBlock::LetterlikeSymbols |
UnicodeBlock::NumberForms |
UnicodeBlock::Arrows |
UnicodeBlock::MathematicalOperators |
UnicodeBlock::MiscellaneousTechnical |
UnicodeBlock::ControlPictures |
UnicodeBlock::OpticalCharacterRecognition |
UnicodeBlock::EnclosedAlphanumerics |
UnicodeBlock::BoxDrawing |
UnicodeBlock::BlockElements |
UnicodeBlock::GeometricShapes |
UnicodeBlock::MiscellaneousSymbols |
UnicodeBlock::Dingbats |
UnicodeBlock::MiscellaneousMathematicalSymbolsA |
UnicodeBlock::SupplementalArrowsA |
UnicodeBlock::SupplementalArrowsB |
UnicodeBlock::MiscellaneousMathematicalSymbolsB |
UnicodeBlock::SupplementalMathematicalOperators |
UnicodeBlock::MiscellaneousSymbolsandArrows |
UnicodeBlock::Glagolitic |
UnicodeBlock::LatinExtendedC |
UnicodeBlock::Coptic => {
families.push("Segoe UI");
families.push("Segoe UI Symbol");
families.push("Cambria");
families.push("Meiryo");
families.push("Lucida Sans Unicode");
families.push("Ebrima");
}
UnicodeBlock::GeorgianSupplement |
UnicodeBlock::Tifinagh |
UnicodeBlock::CyrillicExtendedA |
UnicodeBlock::SupplementalPunctuation |
UnicodeBlock::CJKRadicalsSupplement |
UnicodeBlock::KangxiRadicals |
UnicodeBlock::IdeographicDescriptionCharacters => {
families.push("Segoe UI");
families.push("Segoe UI Symbol");
families.push("Meiryo");
}
UnicodeBlock::BraillePatterns => {
families.push("Segoe UI Symbol");
}
UnicodeBlock::CJKSymbolsandPunctuation |
UnicodeBlock::Hiragana |
UnicodeBlock::Katakana |
UnicodeBlock::Bopomofo |
UnicodeBlock::Kanbun |
UnicodeBlock::BopomofoExtended |
UnicodeBlock::CJKStrokes |
UnicodeBlock::KatakanaPhoneticExtensions |
UnicodeBlock::CJKUnifiedIdeographs => {
families.push("Microsoft YaHei");
families.push("Yu Gothic");
}
UnicodeBlock::EnclosedCJKLettersandMonths => {
families.push("Malgun Gothic");
}
UnicodeBlock::YijingHexagramSymbols => {
families.push("Segoe UI Symbol");
}
UnicodeBlock::YiSyllables |
UnicodeBlock::YiRadicals => {
families.push("Microsoft Yi Baiti");
families.push("Segoe UI");
}
UnicodeBlock::Vai |
UnicodeBlock::CyrillicExtendedB |
UnicodeBlock::Bamum |
UnicodeBlock::ModifierToneLetters |
UnicodeBlock::LatinExtendedD => {
families.push("Ebrima");
families.push("Segoe UI");
families.push("Cambria Math");
}
UnicodeBlock::SylotiNagri |
UnicodeBlock::CommonIndicNumberForms |
UnicodeBlock::Phagspa |
UnicodeBlock::Saurashtra |
UnicodeBlock::DevanagariExtended => {
families.push("Microsoft PhagsPa");
families.push("Nirmala UI");
}
UnicodeBlock::KayahLi |
UnicodeBlock::Rejang |
UnicodeBlock::Javanese => {
families.push("Malgun Gothic");
families.push("Javanese Text");
families.push("Leelawadee UI");
}
UnicodeBlock::AlphabeticPresentationForms => {
families.push("Microsoft Uighur");
families.push("Gabriola");
families.push("Sylfaen");
}
UnicodeBlock::ArabicPresentationFormsA |
UnicodeBlock::ArabicPresentationFormsB => {
families.push("Traditional Arabic");
families.push("Arabic Typesetting");
}
UnicodeBlock::VariationSelectors |
UnicodeBlock::VerticalForms |
UnicodeBlock::CombiningHalfMarks |
UnicodeBlock::CJKCompatibilityForms |
UnicodeBlock::SmallFormVariants |
UnicodeBlock::HalfwidthandFullwidthForms |
UnicodeBlock::Specials => {
families.push("Microsoft JhengHei");
}
_ => {}
}
}
}
// https://en.wikipedia.org/wiki/Plane_(Unicode)#Supplementary_Multilingual_Plane
1 => {
families.push("Segoe UI Symbol");
families.push("Ebrima");
families.push("Nirmala UI");
families.push("Cambria Math");
}
_ => {}
}
}
families.push("Arial Unicode MS");
families
}