For those who haven't been following Moonlight development, I'm the guy that has been implementing all the text layout and rendering.
Let me start off by saying that text is hard. Really hard. Especially layout.
Font Hinting
The aspect of text rendering that I've been thinking a lot about lately is hinting. For an example of the difference that hinting makes with text rendering, see the following screenshot:
In the screenshot above, I've posted side-by-side text renderings produced by both Moonlight and Pango (courtesy of Michael Dominic K.) for Arial, Verdana and Times New Roman fonts. You'll notice that especially at the smaller font sizes, Moonlight's text rendering gets a bit fuzzy and hard to read. On the contrary, Pango's rendering, using hinting, has improved readability over Moonlight.
Why don't I just use hinting, you ask?
The problem is that hinting changes the font metrics in such a way that as you scale to larger sizes, the metrics do not scale uniformly (you'll notice that as the font sizes increase, the width of the line gets uniformly wider using Moonlight, but not so with Pango). This causes a bit of a problem for Moonlight (and Silverlight) because:
- with hinting enabled, scaling the font size in an animation would look jumpy
- this is made even worse when you consider line wrapping and how changing font metrics would mean that line wrapping would also change with the changing font size
In most applications, the text in each UI element is unlikely to change size (unless the user manually changes the font settings), and so enabling the use of hinting has no adverse side effects. Silverlight (and thus Moonlight), however, are meant to do all sorts of animations which may apply matrix transforms (such as scaling) to any item in the canvas (even text!).
Since scaling has to look smooth, it's hard to use hinting because of the way it changes the metrics.
Hopefully I can find a way to improve rendering quality at the smaller sizes without getting the side effects I mentioned above, or at least have them be far less noticeable.
Line Breaking
While not thinking about the render quality of the text, the bane of my existence has been implementing layout algorithms for the 3 different TextWrapping modes used in Silverlight (Wrap, NoWrap, and WrapWithOverflow).
I've mostly been focusing on TextWrapping=Wrap lately as it's probably the hardest one to implement, especially if I want to emulate the Silverlight wrapping behavior (from what I can gather by reading the Pango source code, there are ambiguities in the Unicode specification for line-breaking, so I've been having to familiarize myself with the rules for line-breaking lately).
Going Forward
In the future, I hope to replace my text layout/rendering code with the use of Pango but in order for that to happen, Pango will need a few new features:
- The ability to specify fonts outside of the configured font directories (e.g. a way to bypass FontConfig). From what I understand, this is already happening so I expect to be able to cross this off the list soon ;-)
- The ability to mark the beginning/end of a sub-run of text. Silverlight's <TextBlock> element supports child <Run> elements. This normally wouldn't be a big deal, but Silverlight uses the joint between two runs as a possible break opportunity. For example, <Run Text="abcdefg"/><Run Text="hijklmn"/> might line-wrap differently than <Run Text="abcdefghijklmn"/> even though there are no spaces between the two runs. Currently, when rendering a string of text using Pango, you combine all runs into a single string to pass to a PangoLayout, so the layout engine doesn't have a way of knowing that a position mid-word should be treated as a break opportunity.
- Pango will need a way to specify alternative text wrapping modes (so that it would be possible to emulate Silverlight's TextWrapping behavior). Apparently Owen and Behdad have already discussed the desire to have this sort of feature, so I'm looking forward to working with them on designing a suitable API.
- A way to specify that font metrics should scale uniformly as the font size changes. Update: Behdad has notified me that this feature is actively being worked on and that I can enable it via CAIRO_HINT_METRICS_OFF.
- The last thing that I can think of at the moment that Pango will need is a way to reset the Foreground brush for each run and each time each of those runs line-breaks. For example, the following screenshot consists of a single TextBlock with two Run elements which share their parent's font and foreground brush properties. This means that the prepare_run() virtual method for the subclassed PangoRenderer will need to know the extents of the next sub-run of text to be rendered so that it can properly setup the custom foreground brush.
In the meantime, having to write my own layout/rendering engine for text has taught me a lot more about text and fonts than I ever wanted to know and I'm continuing to learn more every day.
Update: With all of these custom changes that I'll need in order to move to Pango, I'm beginning to wonder if it's really worth it? The only thing that it could get me is the Unicode line-breaking logic, but the thing is... if I have to implement my own pluggable TextWrapping modes, then wouldn't I still essentially be implementing my own line-breaking logic? This makes me very sad :(
15 comments:
The reason text jumps is not so much because of hinting as because of how freetype does hinting. It's doing it wrong - or more accurately, it's optimized for a use-case other than "correct layout."
This is a huge article covering this - I'd assume you've read it, but maybe you've missed it.
http://antigrain.com/research/font_rasterization/index.html
Hi,
I've faced the scaling of hinted font issue when working on the gnumeric graph engine, and solved it by not changing the font size, but just use a different cairo scale.
It's not a perfect solution, since the character are not nicely spaced, but at least the graph layout doesn't change when changing the zoom factor.
If you want to see how it behaves, open a recent gnumeric (1.8.x), create a graph with some text, then resize the guru window.
But what we really want is something that's described in Owen Taylor's Grid Fitting document:
http://people.redhat.com/otaylor/grid-fitting/
Freetype only does what you tell it to do. You can just as well disable hinting and then try to compare the results.
I think it would actually be in the interest of the general public to improve Pango to be more versatile, even if this is not trivial. Best to keep efforts concentrated.
Is line-breaking logic really the only thing you'd gain from using Pango? What about supporting right-to-left languages or complex scripts like Arabic or Devangari?
On the matter of additional break points between runs, wouldn't inserting U+200B ZERO WIDTH SPACE do what you want?
YES IT IS WORTH, if you consider being part of our community: Implementing that features in Moonlight benefits only Moonlight. Pushing that features to Pango benefits the entire community.
Ciao,
Mathias
Hmm, But hinting disabled (as Moonlight on the screenshot), some letters look _really_ clumsy.
That's not good....
Isn't there a midway?
Regarding font readability during animations..
I don't know if it's just on my hardware, but I think microsoft is actually "cheating" during animations in WPF.
If you assign e.g. a DoubleAnimation affecting the scale (in a layout or render transform) for a textblock, the text is blurry during the entire animation, and then at the end (like a few milliseconds after it), it becomes clear again.
marius: Silverlight does not support RTL, so line-breaking is pretty much the only thing that Moonlight would really gain by using Pango. But as I mentioned, I'd still have to override Pango's wrapping modes to do what I needed which would pretty much defeat the purpose :(
(unless we could design some API that allowed all that logic to stay within Pango but give me some means of overriding the behavior in a way that didn't require me to implement my own Unicode line-breaking logic... but not sure how that could be done)
Inserting a zero-width space would not get me the desired effect afaict, I've already been dealing with those in a test suite I've been running my text layout code against.
taschenorakel: whether I end up using Pango or not, some of the features I'd need in Pango would still be useful for other applications, sure. And these features should definitely be added to Pango regardless :)
isak: yea, someone else mentioned that to me on IRC. I wonder what they are doing...
Isak/Jeff: MS intentionally does low quality text rendering when the text is animated (there's an IsAnimated property someplace in PresentationCore.. ValueSource maybe). There's a timer that makes a pass once it's stopped which switches to a higher quality layout/rendering.
Well, deciding how (or if) to hint text in potentially dynamic contexts is problematic indeed.
Although freetype with David Turner's patches is incredibly helpful (in your screenshot, Pango+ft's render sucks), there's no denying that a compromise is needed. Many people are afraid because font hinting is a damn patent minefield. Ubuntu chose to assume the risks, at least since Gutsy, btw.
Something I think that would help is the ability to use different hinting modes for horizontal and vertical directions (I think agg's article explores that possibility).
agg's article mentioned by Sean is also helpful, but I think that there is always going to be a compromise between animation-enabled rendering and quality rendering... Flash has designer-specified rendering modes for this (so if you won't transform or animate, you can have sharper fonts). What Chris says MS does is not so nice, I think (too instrusive for the user).
I would try Light hinting with Turner's excellent patches, and if it's not enough, then switch to non-hinting mode. Of course that depends on the runtime environment.
Keep the good work,
nachokb
Sean: was just looking for the antigrain article.. this has to be the way forward, surely ?
For what its worth, I think the hinting makes the type look pretty bad. It makes the spacing between letters all uneven, and causes the shapes to get sort of lumpy and awkward.
And I don't think hinting really helps make the text more readable. Research has shown that we read more based on the overall shape of words than identifying individual features of letters. If anything, not-hinting preserves better the shape of the letters and words making reading easier.
Nice crisp verticals don't add much except a belief amongst developers that the rendering is somehow better because it has nice crisp verticals.
Aha, thanks Sean! I've read that article when it was published a while back, but that was before I had started working on text rendering for Moonlight so I hadn't paid it much attention.
BTW, the Agg article and the Turner patches amount to more or less the same: No horizontal stem snapping, etc.
Post a Comment