Skia – Design Documents

Docs: Analytic Anti-Alias

Mon, 01 Jan 0001 00:00:00 +0000

Docs: Canvas2D Shaped Text Extensions

Mon, 01 Jan 0001 00:00:00 +0000

Shaped Text is a proposal for exposing the Browser’s text shaping engine. It takes in a block of (annotated) text, and returns the low-level information needed to correctly measure, hit-test, and draw the text as positioned glyphs. This processing step is needed whenever text is measured or drawn in the browser, and this processing can be complex and time consuming. The output of this processing is, however, simple and can be rendered quite efficiently. It is runs (tied to a specific Typeface and size) of glyph IDs and x,y positions.

This proposal extends Canvas2D to allow it to draw those glyphs directly, and also includes utilities for querying attributes of the glyphs (not needed for drawing, but useful for other operations).

Principles

Drawing positioned glyphs should be at least as flexible as the existing fillText() method.
It is expected that drawing glyphs can be faster than fillText() – no shaping/processing is needed.
With the additional utilities, new effects should be easy and efficient.

Drawing glyphs

At the heart of the proposal is a parallel to fillText/strokeText…

context.fillGlyphs(glyphs, positions, Font);

context.strokeGlyphs(glyphs, positions, Font);

These respect all of the same settings as their ‘Text’ equivalents (e.g. current transform, clip, style) with the exception of the actual text attributes:

font
textAlign
textBaseline
direction

These are ignored, because they have already been ‘computed’ by the Shape Text processing, and their results are represented in the glyphs, positions, and Font parameters.

Font is far more specific in this extension that the existing context.font attribute. In today’s canvas2d, “font” holds a high-level description of the typeface(s): It is a string with the font’s name, which has to be resolved to find the actual (set of) resources. For Shaped Text, this resolution has already occurred. The glyph IDs are specific to exactly 1 resource (i.e. file/blob) and so the Font interface contains not the name, but a handle to the actual resource.

The upside to this specificity is performance: with all “fallback” and shaping having already occurred, the draw calls can execute faster.

Font utilities

Shaped Text introduced the Font interface, but for shaping, it only needed to specify the resource (Typeface object) sizing information, and (on input) optional font-features. After shaping, clients may want to query information about specific glyphs within that Font. Those extended methods are presented here.

interface Font {
    // return array of advance widths for the specified glyphs.
    //
    sequence<float> getGlyphAdvances(sequene<unsigned short> glyphs);

    // return array of [left, top, right, bottom] coordinates for the specified glyphs,
    //
    // If positions are provided, then the rectangles are relative to each glyph's postion.
    // If no positions are provided, then the rectangles are all relateive to (0,0).
    // Note: positions are stored as (x,y) pairs
    //
    sequence<float> getGlyphBounds(sequene<unsigned short> glyphs, sequence<float> positions?);

    // return array of Path2D objects for the specified glyphs,
    //
    // If positions are provided, then the paths are relative to each glyph's postion.
    // If no positions are provided, then the paths are all relateive to (0,0).
    // Note: positions are stored as (x,y) pairs
    //
    // If a glyph has no visual representation (e.g. a SPACE) then its path will be null.
    // If a glyph has an image for its representation, then its path will be undefined.
    //
    sequence<Path2D> getGlyphPaths(sequene<unsigned short> glyphs, sequence<float> positions?);

    // A glyph may be represented with an image (e.g. emoji). getGlyphImage() for these glyphs
    // will return a GlyphImage object. If the glyph does not have an Image, null is returned.
    //
    GlyphImage getGlyphImage(unsigned short glyphID);
};

interface GlyphImage {
    readonly attribute ImageBitmap image;
    readonly attribute DOMMatrix transform;
};

Overview document

Contributors:

mikerreed,

Docs: Dangling Pointer Detector

Mon, 01 Jan 0001 00:00:00 +0000

A pointer is dangling when it references freed memory. Dangling pointers are a source of Use-After-Free (UAF) bugs and are highly discouraged unless you can definitively ensure that they will never be dereferenced or used after the pointed-to object is freed.

Skia tests (run via dm) are configured to detect dangling raw_ptr<T> instances when built with PartitionAlloc enabled (skia_use_partition_alloc = true).

Motivation

Tracking the lifetime of interacting objects across a complex C++ codebase is difficult. Often, lifetime issues are discovered late when they cause hard-to-reproduce user crashes.

Enforcing the Dangling Pointer Detector on the Commit Queue (CQ) helps us:

Prevent Regressions: Catch accidental lifetime changes that invalidate prior ownership assumptions before they ship.
Promote Better Architecture: Flag ambiguous object lifetimes during code review.
Verify Cleanups: Give developers immediate confirmation when cleaning up dangling pointers.

`raw_ptr<T>`

A raw_ptr<T> is a non-owning smart pointer. When using raw_ptr<T>, the severity of UAFs is significantly mitigated because the underlying allocation is protected by MiraclePtr / BackupRefPtr.

A raw_ptr<T> works transparently like a raw T*. It should primarily be used for class and struct member variables.

Flavors & Annotations

When a pointer must temporarily dangle safely, or represents an untriaged legacy instance, you can use these annotations:

raw_ptr<T> ptr_never_dangling;
raw_ptr<T, DisableDanglingPtrDetection> ptr_allowed_to_dangle;
raw_ptr<T, DanglingUntriaged> ptr_dangling_to_investigate;

DisableDanglingPtrDetection: Used to annotate intentional and safe dangling pointers as a last resort if re-architecting ownership is impractical.
DanglingUntriaged: Indicates a pre-existing dangling pointer marked for future cleanup.

Docs: PDF Theory of Operation

Mon, 01 Jan 0001 00:00:00 +0000

Internally, SkPDFDocument and SkPDFDevice represents PDF documents and pages. This document describes how the backend operates, but these interfaces are not part of the public API and are subject to perpetual change.

See Using Skia’s PDF Backend to find out how to use SkPDF as a client calling Skia’s public API.

Typical usage of the PDF backend

SkPDFDevice is the main interface to the PDF backend. This child of SkDevice can be set on an SkCanvas and drawn to. Once drawing to the canvas is complete (SkDocument::onEndPage() is called), the device’s content and resources are added to the SkPDFDocument that owns the device. A new SkPDFDevice should be created for each page or layer desired in the document. After all the pages have been added to the document, SkPDFDocument::onClose() is called to finish serializing the PDF file.

PDF Objects and Document Structure

Background: The PDF file format has a header, a set of objects and then a footer that contains a table of contents for all of the objects in the document (the cross-reference table). The table of contents lists the specific byte position for each object. The objects may have references to other objects and the ASCII size of those references is dependent on the object number assigned to the referenced object; therefore we can’t calculate the table of contents until the size of objects is known, which requires assignment of object numbers. The document uses SkWStream::bytesWritten() to query the offsets of each object and build the cross-reference table.

Furthermore, PDF files can support a linearized mode, where objects are in a specific order so that pdf-viewers can more easily retrieve just the objects they need to display a specific page, i.e. by byte-range requests over the web. Linearization also requires that all objects used or referenced on the first page of the PDF have object numbers before the rest of the objects. Consequently, before generating a linearized PDF, all objects, their sizes, and object references must be known. Skia has no plans to implement linearized PDFs.

%PDF-1.4
…objects...
xref
0 31  % Total number of entries in the table of contents.
0000000000 65535 f
0000210343 00000 n
…
0000117055 00000 n
trailer
<</Size 31 /Root 1 0 R>>
startxref
210399  % Byte offset to the start of the table of contents.
%%EOF

The the virtual class SkPDFObject are used to manage the needs of the file format. Any object that will represent a PDF object must inherit from SkPDFObject and implement the methods to generate the binary representation and report any other SkPDFObjects used as resources. SkPDFTypes.h defines most of the basic PDF object types: bool, int, scalar, string, name, array, dictionary, and stream. (A stream is a dictionary containing at least a Length entry followed by the data of the stream.)

Streams are now handled in a slightly different way. The SkPDFStreamOut() function compresses and serializes the binary data immediately instead of creating a new object.

All of these PDF object types except the stream type can be used in both a direct and an indirect fashion, i.e. an array can have an int or a dictionary as an inline entry, which does not require an object number. The stream type, cannot be inlined and must be referred to with an object reference. Most of the time, other objects types can be referred to with an object reference, but there are specific rules in the PDF specification that requires an inline reference in some place or an indirect reference in other places. All indirect objects must have an object number assigned.

bools: true false
ints: 42 0 -1
scalars: 0.001
strings: (strings are in parentheses or byte encoded) <74657374>
name: /Name /Name#20with#20spaces
array: [/Foo 42 (arrays can contain multiple types)]
dictionary: <</Key1 (value1) /key2 42>>
indirect object:
5 0 obj (An indirect string. Indirect objects have an object number and a generation number, Skia always uses generation 0 objects) endobj
object reference: 5 0 R
stream: <</Length 56>> stream ...stream contents can be arbitrary, including binary... endstream

Indirect objects are either:

Serialized as soon as they are needed, and a new SkPDFIndirectReference is returned, or
Serialized later, but reserve a document-unique SkPDFIndirectReference to allow other objects to refer to it.

Example document:

%PDF-1.4
2 0 obj <<
  /Type /Catalog
  /Pages 1 0 R
>>
endobj
3 0 obj <<
  /Type /Page
  /Parent 1 0 R
  /Resources <>
  /MediaBox [0 0 612 792]
  /Contents 4 0 R
>>
endobj
4 0 obj <> stream
endstream
endobj
1 0 obj <<
  /Type /Pages
  /Kids [3 0 R]
  /Count 1
>>
endobj
xref
0 5
0000000000 65535 f
0000000236 00000 n
0000000009 00000 n
0000000062 00000 n
0000000190 00000 n
trailer
<</Size 5 /Root 2 0 R>>
startxref
299
%%EOF

PDF drawing

Most drawing in PDF is specified by the text of a stream, referred to as a content stream. The syntax of the content stream is different than the syntax of the file format described above and is much closer to PostScript in nature. The commands in the content stream tell the PDF interpreter to draw things, like a rectangle (x y w h re), an image, or text, or to do meta operations like set the drawing color, apply a transform to the drawing coordinates, or clip future drawing operations. The page object that references a content stream has a list of resources that can be used in the content stream using the dictionary name to reference the resources. Resources are things like font objects, images objects, graphic state objects (a set of meta operations like miter limit, line width, etc). Because of a mismatch between Skia and PDF’s support for transparency (which will be explained later), SkPDFDevice records each drawing operation into an internal structure (ContentEntry) and only when the content stream is needed does it flatten that list of structures into the final content stream.

4 0 obj <<
  /Type /Page
  /Resources <<
    /Font <</F1 9 0 R>>
    /XObject <</Image1 22 0 R /Image2 73 0 R>>
  >>
  /Content 5 0 R
>> endobj

5 0 obj <</Length 227>> stream
% In the font specified in object 9 and a height
% of 12 points, at (72, 96) draw ‘Hello World.’
BT
  /F1 12 Tf
  72 96 Td
  (Hello World) Tj
ET
% Draw a filled rectange.
200 96 72 72 re B
...
endstream
endobj

Interned objects

There are a number of high level PDF objects (like fonts, graphic states, etc) that are likely to be referenced multiple times in a single PDF. To ensure that there is only one copy of each object, the SkPDFDocument holds on to a mapping from type-specific keys onto the SkPDFIndirectReference for these objects.

Graphic States

PDF has a number of parameters that affect how things are drawn. The ones that correspond to drawing options in Skia are: color, alpha, line cap, line join type, line width, miter limit, and xfer/blend mode (see later section for xfer modes). With the exception of color, these can all be specified in a single pdf object, represented by the SkPDFGraphicState class. A simple command in the content stream can then set the drawing parameters to the values specified in that graphic state object. PDF does not allow specifying color in the graphic state object, instead it must be specified directly in the content stream. Similarly the current font and font size are set directly in the content stream.

6 0 obj <<
  /Type /ExtGState
  /CA 1  % Opaque - alpha = 1
  /LC 0  % Butt linecap
  /LJ 0  % Miter line-join
  /LW 2  % Line width of 2
  /ML 6  % Miter limit of 6
  /BM /Normal  % Blend mode is normal i.e. source over
>>
endobj

Clip and Transform

Similar to Skia, PDF allows drawing to be clipped or transformed. However, there are a few caveats that affect the design of the PDF backend. PDF does not support perspective transforms (perspective transform are treated as identity transforms). Clips, however, have more issues to contend with. PDF clips cannot be directly unapplied or expanded. i.e. once an area has been clipped off, there is no way to draw to it. However, PDF provides a limited depth stack for the PDF graphic state (which includes the drawing parameters mentioned above in the Graphic States section as well as the clip and transform). Therefore to undo a clip, the PDF graphic state must be pushed before the clip is applied, then popped to revert to the state of the graphic state before the clip was applied.

As the canvas makes drawing calls into SkPDFDevice, the active transform, clip region, and clip stack are stored in a ContentEntry structure. Later, when the ContentEntry structures are flattened into a valid PDF content stream, the transforms and clips are compared to decide on an efficient set of operations to transition between the states needed. Currently, a local optimization is used, to figure out the best transition from one state to the next. A global optimization could improve things by more effectively using the graphics state stack provided in the PDF format.

Generating a content stream

For each draw call on an SkPDFDevice, a new ContentEntry is created, which stores the matrix, clip region, and clip stack as well as the paint parameters. Most of the paint parameters are bundled into an SkPDFGraphicState (interned) with the rest (color, font size, etc) explicitly stored in the ContentEntry. After populating the ContentEntry with all the relevant context, it is compared to the the most recently used ContentEntry. If the context matches, then the previous one is appended to instead of using the new one. In either case, with the context populated into the ContentEntry, the appropriate draw call is allowed to append to the content stream snippet in the ContentEntry to affect the core of the drawing call, i.e. drawing a shape, an image, text, etc.

When all drawing is complete, SkPDFDocument::onEndPage() will call SkPDFDevice::content() to request the complete content stream for the page. The first thing done is to apply the initial transform specified in part in the constructor, this transform takes care of changing the coordinate space from an origin in the lower left (PDF default) to the upper left (Skia default) as well as any translation or scaling requested by the user (i.e. to achieve a margin or scale the canvas). Next (well almost next, see the next section), a clip is applied to restrict drawing to the content area (the part of the page inside the margins) of the page. Then, each ContentEntry is applied to the content stream with the help of a helper class, GraphicStackState, which tracks the state of the PDF graphics stack and optimizes the output. For each ContentEntry, commands are emitted to the final content entry to update the clip from its current state to the state specified in the ContentEntry, similarly the Matrix and drawing state (color, line joins, etc) are updated, then the content entry fragment (the actual drawing operation) is appended.

Drawing details

Certain objects have specific properties that need to be dealt with. Images, layers (see below), and fonts assume the standard PDF coordinate system, so we have to undo any flip to the Skia coordinate system before drawing these entities. We don’t currently support inverted paths, so filling an inverted path will give the wrong result (issue 40031223). PDF doesn’t draw zero length lines that have butt of square caps, so that is emulated.

Layers

PDF has a higher level object called a form x-object (form external object) that is basically a PDF page, with resources and a content stream, but can be transformed and drawn on an existing page. This is used to implement layers. SkPDFDevice has a method, makeFormXObjectFromDevice(), which uses the SkPDFDevice::content() method to construct a form x-object from the the device. SkPDFDevice::drawDevice() works by creating a form x-object of the passed device and then drawing that form x-object in the root device. There are a couple things to be aware of in this process. As noted previously, we have to be aware of any flip to the coordinate system - flipping it an even number of times will lead to the wrong result unless it is corrected for. The SkClipStack passed to drawing commands includes the entire clip stack, including the clipping operations done on the base layer. Since the form x-object will be drawn as a single operation onto the base layer, we can assume that all of those clips are in effect and need not apply them within the layer.

Fonts

There are many details for dealing with fonts, so this document will only talk about some of the more important ones. A couple short details:

We can’t assume that an arbitrary font will be available at PDF view time, so we embed all fonts in accordance with modern PDF guidelines.
Most fonts these days are TrueType fonts, so this is where most of the effort has been concentrated.
Because Skia may only be given a glyph-id encoding of the text to render and there is no perfect way to reverse the encoding, the PDF backend always uses the glyph-id encoding of the text.

Type1/Type3 fonts

Linux supports Type1 fonts, but Windows and Mac seem to lack the functionality required to extract the required information from the font without parsing the font file. When a non TrueType font is used any any platform (except for Type1 on Linux), it is encoded as a Type3 font. In this context, a Type3 font is an array of form x-objects (content streams) that draw each glyph of the font. No hinting or kerning information is included in a Type3 font, just the shape of each glyph. Any font that has the do-not embed copy protection bit set will also get embedded as a Type3 font. From what I understand, shapes are not copyrightable, but programs are, so by stripping all the programmatic information and only embedding the shape of the glyphs we are honoring the do-not embed bit as much as required by law.

PDF only supports an 8-bit encoding for Type1 or Type3 fonts. However, they can contain more than 256 glyphs. The PDF backend handles this by segmenting the glyphs into groups of 255 (glyph id 0 is always the unknown glyph) and presenting the font as multiple fonts, each with up to 255 glyphs.

Font subsetting

Many fonts, especially fonts with CJK support are fairly large, so it is desirable to subset them. Chrome uses the HarfBuzz subsetter to provide subsetting support to Skia for TrueType fonts.

Shaders

Skia has two types of predefined shaders, image shaders and gradient shaders. In both cases, shaders are effectively positioned absolutely, so the initial position and bounds of where they are visible is part of the immutable state of the shader object. Each of the Skia’s tile modes needs to be considered and handled explicitly. The image shader we generate will be tiled, so tiling is handled by default. To support mirroring, we draw the image, reversed, on the appropriate axis, or on both axes plus a fourth in the vacant quadrant. For clamp mode, we extract the pixels along the appropriate edge and stretch the single pixel wide/long image to fill the bounds. For both x and y in clamp mode, we fill the corners with a rectangle of the appropriate color. The composed shader is then rotated or scaled as appropriate for the request.

Gradient shaders are handled purely mathematically. First, the matrix is transformed so that specific points in the requested gradient are at pre-defined locations, for example, the linear distance of the gradient is always normalized to one. Then, a type 4 PDF function is created that achieves the desired gradient. A type 4 function is a function defined by a restricted postscript language. The generated functions clamp at the edges so if the desired tiling mode is tile or mirror, we have to add a bit more postscript code to map any input parameter into the 0-1 range appropriately. The code to generate the postscript code is somewhat obtuse, since it is trying to generate optimized (for space) postscript code, but there is a significant number of comments to explain the intent.

Xfer modes

PDF supports some of the xfer modes used in Skia directly. For those, it is simply a matter of setting the blend mode in the graphic state to the appropriate value (Normal/SrcOver, Multiply, Screen, Overlay, Darken, Lighten, !ColorDodge, ColorBurn, HardLight, SoftLight, Difference, Exclusion). Aside from the standard SrcOver mode, PDF does not directly support the porter-duff xfer modes though. Most of them (Clear, SrcMode, DstMode, DstOver, SrcIn, DstIn, SrcOut, DstOut) can be emulated by various means, mostly by creating form x-objects out of part of the content and drawing it with a another form x-object as a mask. I have not figured out how to emulate the following modes: SrcATop, DstATop, Xor, Plus.

At the time of writing [2012-06-25], I have a CL outstanding to fix a misunderstanding I had about the meaning of some of the emulated modes. I will describe the system with this change applied.

First, a bit of terminology and definition. When drawing something with an emulated xfer mode, what’s already drawn to the device is called the destination or Dst, and what’s about to be drawn is the source or Src. Src (and Dst) can have regions where it is transparent (alpha equals zero), but it also has an inherent shape. For most kinds of drawn objects, the shape is the same as where alpha is not zero. However, for things like images and layers, the shape is the bounds of the item, not where the alpha is non-zero. For example, a 10x10 image, that is transparent except for a 1x1 dot in the center has a shape that is 10x10. The xfermodes gm test demonstrates the interaction between shape and alpha in combination with the port-duff xfer modes.

The clear xfer mode removes any part of Dst that is within Src’s shape. This is accomplished by bundling the current content of the device (Dst) into a single entity and then drawing that with the inverse of Src’s shape used as a mask (we want Dst where Src isn’t). The implementation of that takes a couple more steps. You may have to refer back to the content stream section. For any draw call, a ContentEntry is created through a method called SkPDFDevice::setUpContentEntry(). This method examines the xfer modes in effect for that drawing operation and if it is an xfer mode that needs emulation, it creates a form x-object from the device, i.e. creates Dst, and stores it away for later use. This also clears all of that existing ContentEntry’s on that device. The drawing operation is then allowed to proceed as normal (in most cases, see note about shape below), but into the now empty device. Then, when the drawing operation in done, a complementary method is called,SkPDFDevice::finishContentEntry(), which takes action if the current xfer mode is emulated. In the case of Clear, it packages what was just drawn into another form x-object, and then uses the Src form x-object, an invert function, and the Dst form x-object to draw Dst with the inverse shape of Src as a mask. This works well when the shape of Src is the same as the opaque part of the drawing, since PDF uses the alpha channel of the mask form x-object to do masking. When shape doesn’t match the alpha channel, additional action is required. The drawing routines where shape and alpha don’t match, set state to indicate the shape (always rectangular), which finishContentEntry uses. The clear xfer mode is a special case; if shape is needed, then Src isn’t used, so there is code to not bother drawing Src if shape is required and the xfer mode is clear.

SrcMode is clear plus Src being drawn afterward. DstMode simply omits drawing Src. DstOver is the same as SrcOver with Src and Dst swapped - this is accomplished by inserting the new ContentEntry at the beginning of the list of ContentEntry’s in setUpContentEntry instead of at the end. SrcIn, SrcOut, DstIn, DstOut are similar to each, the difference being an inverted or non-inverted mask and swapping Src and Dst (or not). SrcIn is SrcMode with Src drawn with Dst as a mask. SrcOut is like SrcMode, but with Src drawn with an inverted Dst as a mask. DstIn is SrcMode with Dst drawn with Src as a mask. Finally, DstOut is SrcMode with Dst draw with an inverted Src as a mask.

Known issues

issue 40031257 SrcAtop Xor, and Plus xfer modes are not supported.
issue 40031248 drawVerticies is not implemented.
issue 40031251 Mostly, only TTF fonts are directly supported. (User metrics show that almost all fonts are truetype.)
issue 40031270 Page rotation is accomplished by specifying a different size page instead of including the appropriate rotation annotation.

Docs: Shaped Text

Mon, 01 Jan 0001 00:00:00 +0000

A series of object models for describing a low-level builder for multi-line formatted text, and the resulting objects that expose the results of shaping that text. These are done outside of DOM Text nodes, and outside of any particular rendering model (e.g. canvas2d or webgl).

A related explainer focuses on suggested extensions to canvas2d to allow it to efficiently render the shaped results, and to offer helper objects for inspecting useful properties from a typeface.

Overview document

Target audience

We want to target web apps that have already chosen to render their content either in canvas2d, or webgl, or in some other fashion, but still want access to the powerful international text shaping and layout services inherent in the browser. In the case of canvas2d, which already has some facilities for text, we want to address the missing services and low-level results needed for creating interactive text editing or high-performance rendering and animations.

Rather than ‘extend’ the existing canvas2d fillText() method, we propose an explicit 2-step model: process the ‘rich’ text input into shaped results, and then expose those results to the client, allowing them to draw or edit or consume the results as they choose.

JavaScript frameworks are another target audience. This proposal is heavily influenced by successful APIs on native platforms (desktop and mobile) and seeks to deliver similar control and performance. Thus it may be quite natural that sophisticated frameworks build upon these interfaces, providing more ‘friendly’, constrained versions of the features. This is expected, since multiple ‘high level’ models for text are valid, each with its own opinions and tradeoffs. The goal of this API is to expose the core services and results, and leave the opinionated layers to the JavaScript community.

Principles

An imperative JavaScript-friendly text representation.
Restrict input to only what is needed for shaping and metrics is provided.
Decorations (i.e. colors, underlines, shadows, effects) are explicitly not specified, as those can vary widely with rendering technologies (and client imagination).

Sequence of calls

For maximum re-use and efficiency, the process of going from rich-text description to final shaped and formatted results is broken into stages. Each ‘stage’ carries out specific processing, and in-turn becomes a factory to return an instances of the next stage.

TextBuilder, ShapedText and FormattedText objects are used in sequence:

const builder = new ParagraphBuilder(font-fallback-chain);
const shaped = builder.shape(DOMString text, sequence<TextBlock> blocks);
const formatted = shaped.format(double width, double height, alignment);

A Block is a descriptor for a run of text. Currently there are two specializations, but others may be added without breaking the design.

interface Typeface {
    // Number or opaque object: Whatever is needed for the client to know exactly
    // what font-resource (e.g. file, byte-array, etc.) is being used.
    // Without this, the glyph IDs would be meaningless.
    //
    // This interface is really an “instance” of the font-resource. It includes
    // any font-wide modifies that the client (or the shaper) may have requested:
    //    e.g. variations, synthetic-bold, …
    //
    // Factories to create Typeface can be described elsewhere. The point here
    // is that such a unique identifier exists for each font-asset-instance,
    // and that they can be passed around (in/out of the browser), and compared
    // to each other.
};

interface TextBlock {
    unsigned long length;  // number of codepoints in this block
};

interface InFont {
    attribute sequence<Typeface> typefaces; // for preferred fallback faces
    attribute double size;
    attribute double scaleX?;   // 1.0 if not specified
    attribute double skewX?:    // 0.0 if not specified (for oblique)

    attribute sequence<FontFeature> features?;
    // additional attributes for letter spacing, etc.
};

interface FontBlock : TextBlock {
    attribute InFont font;
};

interface PlaceholderBlock : TextBlock {
    attribute double width;
    attribute double height;
    attribute double offsetFromBaseline;
};

interface ShapedTextBuilder {
    constructor(TextDirection,          // default direction (e.g. R2L, L2R)
                sequence<Typeface>?,    // optional shared fallback sequence (after TextBlock's)
                ...);

    ShapedText shape(DOMString text, sequence<TextBlock>);
};

Here is a simple example, specifying 3 blocks for the text.

const fontA = new Font({family-name: "Helvetica", size: 14});
const fontB = new Font({family-name: "Times", size: 18});
const blocks = [
  { length: 6, font: fontA },
  { length: 5, font: fontB },
  { length: 6, font: fontA },
];

const shaped = builder.shape("Hello text world.", blocks);

// now we can format the shaped text to get access to glyphs and positions.

const formatted = shaped.format({width: 50, alignment: CENTER});

This is explicitly intended to be efficient, both for the browser to digest, and for the client to be able to reuse compound objects as they choose (i.e. reusing fontA in this example).

If there is a mismatch between the length of the text string, and the sum of the blocks' lengths, then an exception is raised.

Access the results of shaping and formatting

FormattedText has methods and the raw data results:

typedef unsigned long TextIndex;

interface TextPosition {
    readonly attribute TextIndex textIndex;
    readonly attribute unsigned long lineIndex;
    readonly attribute unsigned long runIndex;
    readonly attribute unsigned long glyphIndex;
};

interface FormattedText {
    // Interaction methods

    // Given a valid index into the text, adjust it for proper grapheme
    // boundaries, and return the TextPosition.
    TextPosition indexToPosition(TextIndex index);

    // Given an x,y position, return the TextPosition
    // (adjusted for proper grapheme boundaries).
    TextPosition hitTextToPosition(double x, double y);

    // Given two logical text indices (e.g. the start and end of a selection range),
    // return the corresponding visual set of ranges (e.g. for highlighting).
    sequence<TextPosition> indicesToVisualSelection(TextIndex t0, TextIndex t1);

    // Raw data

    readonly attribute Rect bounds;

    readonly attribute sequence<TextLine> lines;
};

The sequence of TextLines is really and array of arrays: each line containing an array of Runs (either Glyphs or Placeholders for now).

// Shared by all output runs, specifying the range of code-points that produced
// this run. Known subclasses: TextRun, PlaceholderRun.
interface TextRun {
    readonly attribute TextIndex startIndex;
    readonly attribute TextIndex endIndex;
};

interface GlyphRunFont {
    // Information to know which font-resource (typeface) to use,
    // and at what transformation (size, etc.) to use it.
    //
    readonly attribute Typeface typeface;
    readonly attribute double size;
    readonly attribute double scaleX?;   // 1.0 if not specified
    readonly attribute double skewX?:    // 0.0 if not specified (could be a bool)
};

interface GlyphRun : TextRun {
    readonly attribute GlyphRunFont font;

    // Information to know what positioned glyphs are in the run,
    // and what the corresponding text offsets are for those glyphs.
    // These “offsets” are not needed to correctly draw the glyphs, but are needed
    // during selections and editing, to know the mapping back to the original text.
    //
    readonly attribute sequence<unsigned short> glyphs;     // N glyphs
    readonly attribute sequence<float> positions;           // N+1 x,y pairs
    readonly attribute sequence<TextIndex> indices;         // N+1 indices
};

interface PlaceholderRun : TextRun {
    readonly attribute Rect bounds;
};

interface TextLine {
    readonly attribute TextIndex startIndex;
    readonly attribute TextIndex endIndex;

    readonly attribute double top;
    readonly attribute double bottom;
    readonly attribute double baselineY;

    readonly attribute sequence<TextRun> runs;
};

With these data results (specifically glyphs and positions for specific Typeface objects) callers will have all they need to draw the results in any fashion they wish. The corresponding start/end text indices allows them to map each run back to the original text.

This last point is fundamental to the design. It is recognized that a client creating richly annotated text will associate both shaping (e.g. Font) information, as well as arbitrary decoration and other annotations with each block of text. Returning in each Run the corresponding text range allows the client to “look-up” all of their bespoke additional information for that run (e.g. colors, shadows, underlines, placeholders, etc.). This frees the Browser from having to support or even understand the union of all possible decorations (obviously impossible).

Alternatives and Prior Art

This model is designed to be low-level, to appeal to performance sensitive applications (e.g. animations) and sophisticated text (editors). It is also intended to feel ‘natural’ to a developer coming from a native app environment (desktop or mobile).

We recognized that many (more casual) users may also want access to some of these services. That is appropriate, but we posit that with the right primitives and data exposed, such higher-level models can be constructed by the JavaScript community itself, either as formal Frameworks or as refined sample / example code.

One excellent example of a higher-level data model is Formatted Text and we hope to explore ways to layer these two proposals, allowing high-level clients to utilize their data model, but still have the option to access our lower level accessors (as they wish).

Rendering in Canvas2D

The next explainer describes how to take these results and render them into an (extended) Canvas context.

Contributors:

mikerreed,

Docs: Text API Overview

Mon, 01 Jan 0001 00:00:00 +0000

2D Graphics is a broad area. It consists of a host of related, but different ideas and primitives

rectangles, lines, paths
images (often scaled, transformed, filtered)
text (also scalable and transformable)
…

Native (desktop, mobile) frameworks combine these primitives with common rendering constructs

transforms and clipping
colors, gradients, patterns
blending and colorspaces

One primitive in particular stands out: Text. It can be drawn with the same common constructs as rects and paths, but it has another component unrelated to actual drawing, a ‘processing’ step that is surprisingly complex and expensive: Shaping. The native frameworks recognized this, and offer a separate set of APIs just around this:

CoreGraphics –> CoreText
Direct2D/X –> DirectWrite
Skia –> SkShaper

This proposal embraces this pattern, and aims to expose this functionality to the Web.

Background

Part of this complexity lies in language and alphabets themselves, and part is due to how our system of fonts and font formats have evolved. This document does not claim to detail either of these, but will briefly touch on each to motivate the proposed API suite.

Language and Alphabets

“The office 大楼 is in München.”

This simple sentence is illustrative of some of the richness (i.e. complexity) of displaying text. Some examples

The ‘ffi" in office (or possibly just the “fi”) could be drawn with a specialized form called a ligature.
The “ü” could have been drawn as a single shape, or it could have been drawn by separately drawing “u” followed by drawing the umlaut (¨) on top.
The third word may not be representable in the same font as the other words, so a different font may have been needed.

This example in no way attempts to be complete or exhaustive, but it suggests the processing needed to go from just the ‘letters’ in a sentence, to the actual ‘shapes’ that must be found or composed from various fonts to finally be able to draw it.

Fonts and Styles

This site has an excellent overview of some of the richess of specifying a “font” in CSS. While different native platforms have some variance, most support the following mapping from a description of a font, to the actual resource / file / blob containing the drawn shapes.

family-name : “Helvetica”, “Times New Roman”, “Lucida Typewriter Sans”, …
font-style : bold, italic
font-weight : 100 .. 900
font-stretch : ultra-condensed .. ultra-expanded
font-variation-setting : ‘wght’ 850 or ‘wdth’ 25 or …

All of these attributes (and more) go into the browser’s search for the best matching font resource/file/blob. Let’s call the resulting ‘resource’ a Typeface. Note: If the font supports variation-settings, our definition will also include the specific settings (e.g. a font resource + variation settings).

Typefaces and Glyphs

We define a Typeface to be a coherent collection of shapes in a given typograph style (including any variation settings). Typically a typeface is stored in a single file or blob (e.g. an OpenType file).

Along with a single Typeface we define the individual shapes or images in the typeface as Glyphs. Glyphs represent the smallest independent drawing element within a typeface. By convention, glyphs are identified by an index, ranging from 0 to however many are in a particular typeface.

Determining what glyphs, in what order and positions, are needed to represent a set of letters, is the heart of Shaping, and it is this process, and resulting typefaces + positioned glyphs, that we propose exposing to Web Apps.

Summary

We posit that drawing internationally correct Text is critical to most Web Apps, and that it is both complex to get correct, and can be computationally expensive. We propose exposing this processing to apps, providing them with results that can be efficiently drawn / animated.

The core Shaping APIs are detailed here.

Associated Canvas2D extensions are detailed here.

Note: it is an explicit goal to not tie Shaping or its results to Canvas2D. We envision multiple scenarios where a framework or app will want to shape text, but not need a Canvas2D context .

drawn using WebGL or WebGPU
drawn using DOM (but utilizing line-breaking and metrics results)
drawn using a bespoke (i.e. wasm) renderer

We are also proposing a lower level interface, one that just addresses exposing unicode properties.

Contributors:

mikerreed,

Docs: Text Properties API

Mon, 01 Jan 0001 00:00:00 +0000

A variety of (internationally correct) text processing requires know the properties of unicode characters. For example, where in a string are the word boundaries (needed for line-breaking), which need to be ordered right-to-left or left-to-right?

We propose a batch call that characterizes the code-points in a string. The method will return an array of bitfields packed in a 32bit unsigned long, containing the results of all of the Options.

Functional Requirement

One measure of the value/completeness of the API is the following:

*For sophisticated apps or frameworks (e.g. Flutter or Lottie) that need…

text shaping
line breaking
word and grapheme boundaries

Certainly this API could include more than is strictly required for those use cases, but it is important that it include at least enough to allow them to function without increasing their (WASM) download size by having to include an copy of ICU (or its equivalent).

Ergonomics

Associated with the above Function Requirements, another driver for the shape of the API is efficiency, esp. when called by WASM clients. There is a real cost for each JS <–> WASM call, more than the equivalent sequence between JS and the Browser.

Minimize # calls needed for a block of text
Homogenous arrays rather than sequence of objects

Given this, implementations are encouraged to use Uint32Array typed array buffer for the result.

//  Bulk call to characterize the code-points in a string.
//  This can return a number of different properties per code-point, so to maximize performance,
//  it will only compute the requested properties requested (see optional boolean request fields).
//
interface TextProperties {
    const unsigned long BidiLevelMask   = 31,       // 0..31 bidi level

    const unsigned long GraphemeBreak   = 1 << 5,
    const unsigned long IntraWordBreak  = 1 << 6,
    const unsigned long WordBreak       = 1 << 7,
    const unsigned long SoftLineBreak   = 1 << 8,
    const unsigned long HardLineBreak   = 1 << 9,

    const unsigned long IsControl       = 1 << 10,
    const unsigned long IsSpace         = 1 << 11,
    const unsigned long IsWhiteSpace    = 1 << 12,

    attribute boolean bidiLevel?;
    attribute boolean graphemeBreak?;
    attribute boolean wordBreak?;       // returns Word and IntraWord break properties
    attribute boolean lineBreak?;       // returns Soft and Hard linebreak properties

    attribute boolean isControl?;
    attribute boolean isSpace?;
    attribute boolean isWhiteSpace?;

    // Returns an array the same length as the input string. Each returned value contains the
    // bitfield results for the corresponding code-point in the string. For surrogate pairs
    // in the input, the results will be in the first output value, and the 2nd output value
    // will be zero.
    //
    // Bitfields that are currently unused, or which correspond to an Option attribute that
    // was not requested, will be set to zero.
    //
    sequence<unsigned long> characterize(DOMString inputString,
                                         DOMString bcp47?);
}

Example

const properties = {
    isWhiteSpace: true,
    lineBreak: true,
};

const text = "Because I could not stop for Death\nHe kindly stopped for me";

const results = properties.characterize(text);

// expected results

results[7,9,15,19,24,28,37,44,52,65] --> IsWhiteSpace | SoftLineBreak
results[34] --> HardLineBreak

Some facilities for characterizing Unicode already exist, either as part of EcmaScript or the Web api. See intl segmenter. This proposal acknowledges these, but suggests that any potential overlap in functionality is OK, given the design constraint spelled out in the Ergonomics section.

Similar to the contrast between canvas2d and webgl, this proposal seeks to provide very efficient, lower level access to unicode properties, specifically for sophisticated (possibly native ported to wasm) frameworks and apps. It is not intended to replace existing facilities (i.e. Segmenter), but rather to offer an alternative interface more suited to high-performance clients.

We also propose a higher level interface specfically aimed at Text Shaping.

Contributors:

mikerreed,

Docs: The Raster Tragedy in Skia

Mon, 01 Jan 0001 00:00:00 +0000

This is an extension of The Raster Tragedy at Low-Resolution Revisited as it applies to Skia. The Raster Tragedy describes a number of issues with typeface rasterization with a particular emphasis on proper hinting to overcome these issues. Since not all fonts are nicely hinted and sometimes hinting is not desired, there are additional hacks which may be applied. Generally, one wants to hint purely informational text laid out for a particular device, but not hint text which is art. Unless, of course, the hinting is part of the art like Shift_JIS art.

The Gamma Hack

First, one should be aware of transfer functions (of which ‘gamma’ is an example). A good introduction can be had at What Every Coder Should Know About Gamma.

In Skia, all color sources are converted into the destination color space and the blending is done in the destination color space by applying the linear blend function. Skia does not convert into a linear space, apply the linear blend, and convert back to the encoded space. If the destination color space does not have a linear encoding this will lead to ‘incorrect’ blending. The idea is that there are essentially two kinds of users of Skia. First there are existing systems which are already using a non-linear encoding with a linear blend function. While the blend isn’t correct, these users generally don’t want anything to change due to expectations. Second there are those who want everything done correctly and they are willing to pay for a linearly encoded destination in which the linear blend function is correct.

For bi-level glyph rendering a pixel is either covered or not, so there are no coverage blending issues.

For regular full pixel partial coverage (anti-aliased) glyph rendering the user may or may not want correct linear blending. In most non-linear encodings, using the linear blend function tends to make black on white look slightly heavier, using the pixel grid as a kind of contrast and optical sizing enhancement. It does the opposite for white on black, often making such glyphs a bit under-covered. However, this fights the common issue of blooming where light on dark on many displays tends to appear thicker than dark on light. (The black not being fully black also contributes.) If the pixels are small enough and there is proper optical sizing and perhaps anti-aliased drop out control (these latter two achieved either manually with proper font selection or ‘opsz’, automatically, or through hinting) then correct linear blending tends to look great. Otherwise black on white text tends to (correctly) get really anemic looking at small sizes. So correct blending of glyph masks here should be left up to the user of Skia. If they’re really sophisticated and already tackled these issues then they may want linear blending of the glyphs for best effect. Otherwise the glyphs should just keep looking like they used to look due to expectations.

For subpixel partial coverage (subpixel anti-aliased) glyph masks linear blending in a linear encoding is more or less required to avoid color fringing effects. The intensity of the subpixels is being directly exploited so needs to be carefully controlled. The subpixels tend to alleviate the issues with no full coverage (though still problematic if blitting text in one of the display’s primaries). One will still want optical sizing since the glyphs will still look somewhat too light when scaled down linearly.

So, if subpixel anti-aliased glyph masks (and sometimes full pixel anti-aliased glyph masks) need a correct blit how are they to be used with non-linearly encoded destinations?

One possible solution is to special case these blits. If blitting on the CPU it’s often fast and close enough to take the square root of the source and destination values, do the linear blend function, then square the result (approximating the destination encoding as if its transfer function is square). Many GPUs have a mode where they can blend in sRGB, though unfortunately this generally applies to the whole framebuffer, not just individual draws. For various reasons, Skia has avoided special casing these blends.

What Skia currently does is the gamma hack. When creating the glyph mask one usually knows the approximate color which is going to be drawn, the transfer function, and that a linear source-over blend is going to be used. The destination color is then guessed to be a contrasting color (if there isn’t any contrast the drawing won’t be able to be seen anyway) so assume that the destination color will be the perceptually opposite color in the destination color space. One can now determine the desired value by converting the perceptual source and guessed destination into a linear encoding, do the linear source-over blend, and convert to the destination encoding. The coverage is then adjusted so that the result of a linear source-over blend on the non-linear encoded values will be as close as possible to this desired value.

This works, but makes many assumptions. The first is the guess at the destination. A perceptual middle gray could equally well (or poorly) contrast with black or white so the best guess is drawing it on top of itself. Subpixel anti-aliased glyph masks drawn with this guess will be drawn without any adjustment at all, leaving them color fringy. On macOS Skia tweaks the destination guess to reduce the correction when using bright sources. This helps reducing ‘blooming’ issues with light on dark (aka dark mode) and better matches how CoreText seems to do blending. The second is that a src-over blend is assumed, but users generally aren’t as discriminating of the results of other blends.

The gamma hack works best with subpixel anti-aliasing since the adjustment can be made per-channel instead of full pixel. If this hack is applied to full pixel anti-aliased masks everything is essentially being done in the nearest gray instead of the nearest color (since there is only one channel through which to communicate), leading to poor results. Most users will not want the gamma hack applied to full pixel anti-aliased glyphs.

Since the gamma hack is logically part of the blend, it must always be the very last adjustment made to the coverage mask before being used by the blitter. The inputs are the destination transfer function and the current color (in the destination color space). In Skia these come from the color space on the SkSurface and the color on the SkPaint.

Optical Sizing Hack

In metal type a type designer will draw out on paper the design for the faces of the final sorts. Often there would be different drawings for different target sizes. Sometimes these different sizes look quite different (like Display and Text faces of the same family). Then a punch cutter takes these drawings and creates a piece of metal called a punch which can stamp out these faces. The punch is used to create a negative in a softer piece of metal forming a strike. This strike is made the correct width and called a matrix. This matrix is used as a mold to cast individual sorts. The sorts are collected into a box called a type case. A typesetter would take sorts from a type case and set them into a form. The printer would then ink the sorts in the form and press the paper. (Note that the terms ‘typeface’ and ‘font’ aren’t used in this description, there is a lot of disagreement on what they apply to.)

Every step of this process is now automated in some way. Unfortunately, knowledge embedded in the manual process has not always been replicated in the automation. This can be for a wide variety of reasons, such as being overlooked or being difficult to emulate. One of these areas is the art of optical sizing and managing thin features.

In general smaller type should be relatively heavier in weight than would be expected if taking a larger size and linearly scaling it down. The type designer will draw out the faces with this in mind, potentially with an eye toward how it would vary with size and with potential need for ink traps. The punch cutter would then cut the punches at a given size with this in mind, adjusting until the soot proofs looked good. There may even be slight adjustments to the matrix itself if something seemed off. The typesetter would often know which specific cases contained slightly heavier or lighter sorts. The printer would then adjust the ink and pressure to get a good looking print.

Popular digital font formats didn’t really support this until recently with the variable font ‘opsz’ axis. The presence of an ‘opsz’ axis is like the type designer giving really good instructions about how the faces should look at various sizes. However, not all fonts have or will have a ‘opsz’ axis. Since we don’t always have an ‘opsz’ what can be done to prevent small glyphs from looking all washed out?

One way the type designer could influence the optical size is through hinting. Any manual hinting is done by someone looking at the result at small sizes. Tweaks are made until it ‘looks good’. This often unconsciously bakes in optical sizing. Even autohinters eventually end up emulating this, generally making the text more contrasty and somewhat heavier at small pixel sizes (this depends on the target pixel size not the nominal requested size).

An alternate way the designer could provide optical sized fonts is using multiple font files for different optical sizes and expect the user to select the right one (like Display and Text variants). In theory the right one might also be selected automatically based on some criteria. Switching font files because of nominal size changes may seem drastic, but this is how the system font on macOS worked for a while.

One automatic way to do optical sizing is to artificially embolden the outline itself at small nominal sizes. This is the approach taken by CoreText. This is a lot like the dreaded fake-bold, but doesn’t have the issue of the automatic fake bolding being heavier than ultra bold, since this emboldening is applied uniformally based on requested nominal size.

Another automatic way to do optical sizing is to over-cover all the edges when doing anti-aliasing. The pixels are a fixed physical size so affect small features more than large features. Skia currently has rudimentary support for this in its ‘contrast hack’ which is described more later.

As a note on optical sizing, this is one place where the nominal or optical size of the font is treated differently from the final pixel size of the font. A SkCanvas may have a scale transform active. Any hinting will be done at the final pixel size (the font size mapped through the current transformation matrix), but the optical size is not affected by the transform since it’s actually a parameter for the SkTypeface (and maybe the SkFont).

The Contrast Hack

Consider the example of a pixel wide stroke at half-pixel boundaries. As stated in section 3.1.3 “if we were to choose the stroke positions to render the most faithful proportions and advance width, we may have to compromise the rendering contrast.” If the stroke lands on a pixel boundary all is well. If the stoke lands at a half pixel boundary and correct linear blending is used then the same number of photons are reaching the eye, but the reduction in photons is spread out over twice the area. If the pixels were small enough then this wouldn’t be an issue. Back of the envelope suggests an 8K 20inch desktop monitor or ~400ppi being the minimum. Note that RGB subpixel anti-aliasing brings a 100dpi display up to 300dpi, which is close, but still a bit short. Exploiting the RGB subpixels also doesn’t really increase the resolution as much when getting close to the RGB primaries for the fill color.

One way to try to compensate for this is to cover some of these partially covered pixels more until it visually looks better. Note that this depends on a lot of factors like the pixel density of the display, the visual acuity of the user, the distance of the user from the display, and the user’s sensitivity to the potential variations in stem darkness which may result. Automatically taking all these factors into account would be quite difficult. The correct function is generally determined by having the user look at the result of applying various amounts of extra coverage and having them to pick a setting that looks the least bad to them.

This specific form of over-covering is a form of drop out control for anti-aliasing and could be implemented in a similar way, detecting when a stem comes on in one pixel and goes out in the next and mark that for additional coverage. At raster time a cruder approximation could be made by doing a pass in each of the horizontal and vertical directions and finding runs of more than one non-fully-covered pixel and increasing their coverage.

If instead of doing these computationally expensive passes all coverage is boosted then in addition to the smeared stems the entire outside edge of the glyph will also be bolded. Making these outside edges heavier is a crude approximation of outsetting the initial path in a rather complicated way and amounts to an optical sizing tweak. Just as hinting can be used to approximate optical sizing if the user’s perception of the pixel sizes is known in advance, this is a pixel level tweak tied to a specific user and display combination.

Much like the gamma hack can be modified to reduce the correction for light on dark to fight blooming, the contrast hack can be reduced when the color being drawn is known to be light. Generally the contrast correction goes to zero as one approaches white.

Docs: Two-point Conical Gradient

Mon, 01 Jan 0001 00:00:00 +0000

(Please refresh the page if you see a lot of dollars instead of math symbols.)

We present a fast shading algorithm (compared to bruteforcely solving the quadratic equation of gradient $t$) for computing the two-point conical gradient (i.e., createRadialGradient in spec). It reduced the number of multiplications per pixel from ~10 down to 3, and brought a speedup of up to 26% in our nanobenches.

This document has 3 parts:

Problem Statement and Setup
Algorithm
Appendix

Part 1 and 2 are self-explanatory. Part 3 shows how to geometrically proves our Theorem 1 in part 2; it’s more complicated but it gives us a nice picture about what’s going on.

Problem Statement and Setup

Let two circles be $C_0, r_0$ and $C_1, r_1$ where $C$ is the center and $r$ is the radius. For any point $P = (x, y)$ we want the shader to quickly compute a gradient $t \in \mathbb R$ such that $p$ is on the linearly interpolated circle with center $C_t = (1-t) \cdot C_0 + t \cdot C_1$ and radius $r_t = (1-t) \cdot r_0 + t \cdot r_1 > 0$ (note that radius $r_t$ has to be positive). If there are multiple (at most 2) solutions of $t$, choose the bigger one.

There are two degenerate cases:

$C_0 = C_1$ so the gradient is essentially a simple radial gradient.
$r_0 = r_1$ so the gradient is a single strip with bandwidth $2 r_0 = 2 r_1$.

They are easy to handle so we won’t cover them here. From now on, we assume $C_0 \neq C_1$ and $r_0 \neq r_1$.

As $r_0 \neq r_1$, we can find a focal point $C_f = (1-f) \cdot C_0 + f \cdot C_1$ where its corresponding linearly interpolated radius $r_f = (1-f) \cdot r_0 + f \cdot r_1 = 0$. Solving the latter equation gets us $f = r_0 / (r_0 - r_1)$.

As $C_0 \neq C_1$, focal point $C_f$ is different from $C_1$ unless $r_1 = 0$. If $r_1 = 0$, we can swap $C_0, r_0$ with $C_1, r_1$, compute swapped gradient $t_s$ as if $r_1 \neq 0$, and finally set $t = 1 - t_s$. The only catch here is that with multiple solutions of $t_s$, we shall choose the smaller one (so $t$ could be the bigger one).

Assuming that we’ve done swapping if necessary so $C_1 \neq C_f$, we can then do a linear transformation to map $C_f, C_1$ to $(0, 0), (1, 0)$. After the transformation:

All centers $C_t = (x_t, 0)$ must be on the $x$-axis
The radius $r_t$ is $x_t r_1$.
Given $x_t$ , we can derive $t = f + (1 - f) x_t$

From now on, we’ll focus on how to quickly compute $x_t$. Note that $r_t > 0$ so we’re only interested in positive solutions for $x_t$. Again, if there are multiple $x_t$ solutions, we may want to find the bigger one if $1 - f > 0$, and smaller one if $1 - f < 0$, so the corresponding $t$ is always the bigger one (note that $f \neq 1$, otherwise we’ll swap $C_0, r_0$ with $C_1, r_1$).

Algorithm

Theorem 1. The solution to $x_t$ is

$\frac{x^2 + y^2}{(1 + r_1) x} = \frac{x^2 + y^2}{2 x}$ if $r_1 = 1$
$\left(\sqrt{(r_1^2 - 1) y ^2 + r_1^2 x^2} - x\right) / (r_1^2 - 1)$ if $r_1 > 1$
$\left(\pm \sqrt{(r_1^2 - 1) y ^2 + r_1^2 x^2} - x\right) / (r_1^2 - 1)$ if $r_1 < 1$.

Case 2 always produces a valid $x_t$. Case 1 and 3 require $x > 0$ to produce valid $x_t > 0$. Case 3 may have no solution at all if $(r_1^2 - 1) y^2 + r_1^2 x^2 < 0$.

Proof. Algebraically, solving the quadratic equation $(x_t - x)^2 + y^2 = (x_t r_1)^2$ and eliminating negative $x_t$ solutions gets us the theorem.

Alternatively, we can also combine Corollary 2., 3., and Lemma 4. in the Appendix to geometrically prove the theorem. $\square$

Theorem 1 by itself is not sufficient for our shader algorithm because:

we still need to compute $t$ from $x_t$ (remember that $t = f + (1-f) x_t$);
we still need to handle cases of choosing the bigger/smaller $x_t$;
we still need to handle the swapped case (we swap $C_0, r_0$ with $C_1, r_1$ if $r_1 = 0$);
there are way too many multiplications and divisions in Theorem 1 that would slow our shader.

Issue 2 and 3 are solved by generating different shader code based on different situations. So they are mainly correctness issues rather than performance issues. Issue 1 and 4 are performance critical, and they will affect how we handle issue 2 and 3.

The key to handle 1 and 4 efficiently is to fold as many multiplications and divisions into the linear transformation matrix, which the shader has to do anyway (remember our linear transformation to map $C_f, C_1$ to $(0, 0), (1, 0)$).

For example, let $\hat x, \hat y = |1-f|x, |1-f|y$. Computing $\hat x_t$ with respect to $\hat x, \hat y$ allows us to have $t = f + (1 - f)x_t = f + \text{sign}(1-f) \cdot \hat x_t$. That saves us one multiplication. Applying similar techniques to Theorem 1 gets us:

If $r_1 = 1$, let $x' = x/2,~ y' = y/2$, then $x_t = (x'^2 + y'^2) / x'$.
If $r_1 > 1$, let $x' = r_1 / (r_1^2 - 1) x,~ y' = \frac{\sqrt{r_1^2 - 1}}{r_1^2 - 1} y$, then $x_t = \sqrt{x'^2 + y'^2} - x' / r_1$
If $r_1 < 1$, let $x' = r_1 / (r_1^2 - 1) x,~ y' = \frac{\sqrt{1 - r_1^2}}{r_1^2 - 1} y$, then $x_t = \pm\sqrt{x'^2 - y'^2} - x' / r_1$

Combining it with the swapping, the equation $t = f + (1-f) x_t$, and the fact that we only want positive $x_t > 0$ and bigger $t$, we have our final algorithm:

Algorithm 1.

Let $C'_0, r'_0, C'_1, r'_1 = C_0, r_0, C_1, r_1$ if there is no swapping and $C'_0, r'_0, C'_1, r'_1 = C_1, r_1, C_0, r_0$ if there is swapping.
Let $f = r'_0 / (r'_0 - r'_1)$ and $1 - f = r'_1 / (r'_1 - r'_0)$
Let $x' = x/2,~ y' = y/2$ if $r_1 = 1$, and $x' = r_1 / (r_1^2 - 1) x,~ y' = \sqrt{|r_1^2 - 1|} / (r_1^2 - 1) y$ if $r_1 \neq 1$
Let $\hat x = |1 - f|x', \hat y = |1 - f|y'$
If $r_1 = 1$, let $\hat x_t = (\hat x^2 + \hat y^2) / \hat x$
If $r_1 > 1$, let $\hat x_t = \sqrt{\hat x^2 + \hat y^2} - \hat x / r_1$
If $r_1 < 1$:
1. return invalid if $\hat x^2 - \hat y^2 < 0$
2. let $\hat x_t = -\sqrt{\hat x^2 - \hat y^2} - \hat x / r_1$ if we’ve swapped $r_0, r_1$, or if $1 - f < 0$
3. let $\hat x_t = \sqrt{\hat x^2 - \hat y^2} - \hat x / r_1$ otherwise
$t$ is invalid if $\hat x_t < 0$ (this check is unnecessary if $r_1 > 1$)
Let $t = f + \text{sign}(1 - f) \hat x_t$
If swapped, let $t = 1 - t$

In step 7, we try to select either the smaller or bigger $\hat x_t$ based on whether the final $t$ has a negative or positive relationship with $\hat x_t$. It’s negative if we’ve swapped, or if $\text{sign}(1 - f)$ is negative (these two cannot both happen).

Note that all the computations and if decisions not involving $\hat x, \hat y$ can be precomputed before the shading stage. The two if decisions $\hat x^2 - \hat y^2 < 0$ and $\hat x^t < 0$ can also be omitted by precomputing the shading area that never violates those conditions.

The number of operations per shading is thus:

1 addition, 2 multiplications, and 1 division if $r_1 = 1$
2 additions, 3 multiplications, and 1 sqrt for $r_1 \neq 1$ (count subtraction as addition; dividing $r_1$ is multiplying $1/r_1$)
1 more addition operation if $f \neq 0$
1 more addition operation if swapped.

In comparison, for $r_1 \neq 1$ case, our current raster pipeline shading algorithm (which shall hopefully soon be upgraded to the algorithm described here) mainly uses formula

$$ t = 0.5 \cdot (1/a) \cdot \left(-b \pm \sqrt{b^2 - 4ac}\right)$$ It precomputes $a = 1 - (r_1 - r_0)^2, 1/a, r1 - r0$. Number $b = -2 \cdot (x + (r1 - r0) \cdot r0)$ costs 2 multiplications and 1 addition. Number $c = x^2 + y^2 - r_0^2$ costs 3 multiplications and 2 additions. And the final $t$ costs 5 more multiplications, 1 more sqrt, and 2 more additions. That’s a total of 5 additions, 10 multiplications, and 1 sqrt. (Our algorithm has 2-4 additions, 3 multiplications, and 1 sqrt.) Even if it saves the $0.5 \cdot (1/a), 4a, r_0^2$ and $(r_1 - r_0) r_0$ multiplications, there are still 6 multiplications. Moreover, it sends in 4 uniforms to the shader while our algorithm only needs 2 uniforms ($1/r_1$ and $f$).

Appendix

Lemma 1. Draw a ray from $C_f = (0, 0)$ to $P = (x, y)$. For every intersection point $P_1$ between that ray and circle $C_1 = (1, 0), r_1$, there exists an $x_t$ that equals the length of segment $C_f P$ over the length of segment $C_f P_1$. That is, $x_t = || C_f P || / ||C_f P_1||$.

Proof. Draw a line from $P$ that’s parallel to $C_1 P_1$. Let it intersect with the $x$-axis on point $C = (x', y')$.

Triangle $\triangle C_f C P$ is similar to triangle $\triangle C_f C_1 P_1$. Therefore $||P C|| = ||P_1 C_1|| \cdot (||C_f C|| / ||C_f C_1||) = r_1 x'$. Thus $x'$ is a solution to $x_t$. Because triangle $\triangle C_f C P$ and triangle $\triangle C_f C_1 P_1$ are similar, $x' = ||C_f C_1|| \cdot (||C_f P|| / ||C_f P_1||) = ||C_f P|| / ||C_f P_1||$. $\square$

Lemma 2. For every solution $x_t$, if we extend/shrink segment $C_f P$ to $C_f P_1$ with ratio $1 / x_t$ (i.e., find $P_1$ on ray $C_f P$ such that $||C_f P_1|| / ||C_f P|| = 1 / x_t$), then $P_1$ must be on circle $C_1, r_1$.

Proof. Let $C_t = (x_t, 0)$. Triangle $\triangle C_f C_t P$ is similar to $C_f C_1 P_1$. Therefore $||C_1 P_1|| = r_1$ and $P_1$ is on circle $C_1, r_1$. $\square$

Corollary 1. By lemma 1. and 2., we conclude that the number of solutions $x_t$ is equal to the number of intersections between ray $C_f P$ and circle $C_1, r_1$. Therefore

when $r_1 > 1$, there’s always one unique intersection/solution; we call this “well-behaved”; this was previously known as the “inside” case;
when $r_1 = 1$, there’s either one or zero intersection/solution (excluding $C_f$ which is always on the circle); we call this “focal-on-circle”; this was previously known as the “edge” case;

when $r_1 < 1$, there may be $0, 1$, or $2$ solutions; this was also previously as the “outside” case.

Lemma 3. When solutions exists, one such solution is

$$ x_t = {|| C_f P || \over ||C_f P_1||} = \frac{x^2 + y^2}{x + \sqrt{(r_1^2 - 1) y^2 + r_1^2 x^2}} $$

Proof. As $C_f = (0, 0), P = (x, y)$, we have $||C_f P|| = \sqrt{x^2 + y^2}$. So we’ll mainly focus on how to compute $||C_f P_1||$.

When $x \geq 0$:

Let $X_P = (x, 0)$ and $H$ be a point on $C_f P_1$ such that $C_1 H$ is perpendicular to $C_f P_1$. Triangle $\triangle C_1 H C_f$ is similar to triangle $\triangle P X_P C_f$. Thus $$||C_f H|| = ||C_f C_1|| \cdot (||C_f X_P|| / ||C_f P||) = x / \sqrt{x^2 + y^2}$$ $$||C_1 H|| = ||C_f C_1|| \cdot (||P X_P|| / ||C_f P||) = y / \sqrt{x^2 + y^2}$$

Triangle $\triangle C_1 H P_1$ is a right triangle with hypotenuse $r_1$. Hence $$ ||H P_1|| = \sqrt{r_1^2 - ||C_1 H||^2} = \sqrt{r_1^2 - y^2 / (x^2 + y^2)} $$

We have \begin{align} ||C_f P_1|| &= ||C_f H|| + ||H P_1|| \\\ &= x / \sqrt{x^2 + y^2} + \sqrt{r_1^2 - y^2 / (x^2 + y^2)} \\\ &= \frac{x + \sqrt{r_1^2 (x^2 + y^2) - y^2}}{\sqrt{x^2 + y^2}} \\\ &= \frac{x + \sqrt{(r_1^2 - 1) y^2 + r_1^2 x^2}}{\sqrt{x^2 + y^2}} \end{align}

When $x < 0$:

Define $X_P$ and $H$ similarly as before except that now $H$ is on ray $P_1 C_f$ instead of $C_f P_1$.

As before, triangle $\triangle C_1 H C_f$ is similar to triangle $\triangle P X_P C_f$, and triangle $\triangle C_1 H P_1$ is a right triangle, so we have $$||C_f H|| = ||C_f C_1|| \cdot (||C_f X_P|| / ||C_f P||) = -x / \sqrt{x^2 + y^2}$$ $$||C_1 H|| = ||C_f C_1|| \cdot (||P X_P|| / ||C_f P||) = y / \sqrt{x^2 + y^2}$$ $$ ||H P_1|| = \sqrt{r_1^2 - ||C_1 H||^2} = \sqrt{r_1^2 - y^2 / (x^2 + y^2)} $$

Note that the only difference is changing $x$ to $-x$ because $x$ is negative.

Also note that now $||C_f P_1|| = -||C_f H|| + ||H P_1||$ and we have $-||C_f H||$ instead of $||C_f H||$. That negation cancels out the negation of $-x$ so we get the same equation of $||C_f P_1||$ for both $x \geq 0$ and $x < 0$ cases:

$$ ||C_f P_1|| = \frac{x + \sqrt{(r_1^2 - 1) y^2 + r_1^2 x^2}}{\sqrt{x^2 + y^2}} $$

Finally

$$ x_t = \frac{||C_f P||}{||C_f P_1||} = \frac{\sqrt{x^2 + y^2}}{||C_f P_1||} = \frac{x^2 + y^2}{x + \sqrt{(r_1^2 - 1) y^2 + r_1^2 x^2}} $$ $\square$

Corollary 2. If $r_1 = 1$, then the solution $x_t = \frac{x^2 + y^2}{(1 + r_1) x}$, and it’s valid (i.e., $x_t > 0$) iff $x > 0$.

Proof. Simply plug $r_1 = 1$ into the formula of Lemma 3. $\square$

Corollary 3. If $r_1 > 1$, then the unique solution is $x_t = \left(\sqrt{(r_1^2 - 1) y ^2 + r_1^2 x^2} - x\right) / (r_1^2 - 1)$.

Proof. From Lemma 3., we have

\begin{align} x_t &= \frac{x^2 + y^2}{x + \sqrt{(r_1^2 - 1) y^2 + r_1^2 x^2}} \\\ &= { (x^2 + y^2) \left ( -x + \sqrt{(r_1^2 - 1) y^2 + r_1^2 x^2} \right ) \over \left (x + \sqrt{(r_1^2 - 1) y^2 + r_1^2 x^2} \right ) \left (-x + \sqrt{(r_1^2 - 1) y^2 + r_1^2 x^2} \right ) } \\\ &= { (x^2 + y^2) \left ( -x + \sqrt{(r_1^2 - 1) y^2 + r_1^2 x^2} \right ) \over -x^2 + (r_1^2 - 1) y^2 + r_1^2 x^2 } \\\ &= { (x^2 + y^2) \left ( -x + \sqrt{(r_1^2 - 1) y^2 + r_1^2 x^2} \right ) \over (r_1^2 - 1) (x^2 + y^2) } \\\ &= \left(\sqrt{(r_1^2 - 1) y ^2 + r_1^2 x^2} - x\right) / (r_1^2 - 1) \end{align}

The transformation above (multiplying $-x + \sqrt{(r_1^2 - 1) y^2 + r_1^2 x^2}$ to enumerator and denominator) is always valid because $r_1 > 1$ and it’s the unique solution due to Corollary 1. $\square$

Lemma 4. If $r_1 < 1$, then

there’s no solution to $x_t$ if $(r_1^2 - 1) y^2 + r_1^2 x^2 < 0$
otherwise, the solutions are $x_t = \left(\sqrt{(r_1^2 - 1) y ^2 + r_1^2 x^2} - x\right) / (r_1^2 - 1)$, or $x_t = \left(-\sqrt{(r_1^2 - 1) y ^2 + r_1^2 x^2} - x\right) / (r_1^2 - 1)$.

(Note that solution $x_t$ still has to be nonnegative to be valid; also note that $x_t > 0 \Leftrightarrow x > 0$ if the solution exists.)

Proof. Case 1 follows naturally from Lemma 3. and Corollary 1.

For case 2, we notice that $||C_f P_1||$ could be

either $||C_f H|| + ||H P_1||$ or $||C_f H|| - ||H P_1||$ if $x \geq 0$,
either $-||C_f H|| + ||H P_1||$ or $-||C_f H|| - ||H P_1||$ if $x < 0$.

By analysis similar to Lemma 3., the solution to $x_t$ does not depend on the sign of $x$ and they are either $\frac{x^2 + y^2}{x + \sqrt{(r_1^2 - 1) y^2 + r_1^2 x^2}}$ or $\frac{x^2 + y^2}{x - \sqrt{(r_1^2 - 1) y^2 + r_1^2 x^2}}$.

As $r_1 \neq 1$, we can apply the similar transformation in Corollary 3. to get the two formula in the lemma. $\square$

Skia – Design Documents

Docs: Analytic Anti-Alias

Docs: Canvas2D Shaped Text Extensions

Principles

Drawing glyphs

Font utilities

Contributors:

Docs: Dangling Pointer Detector

Motivation

raw_ptr<T>

Flavors & Annotations

Docs: PDF Theory of Operation

Typical usage of the PDF backend

PDF Objects and Document Structure

PDF drawing

Interned objects

Graphic States

Clip and Transform

Generating a content stream

Drawing details

Layers

Fonts

Type1/Type3 fonts

Font subsetting

Shaders

Xfer modes

Known issues

Docs: Shaped Text

Target audience

Principles

Sequence of calls

Access the results of shaping and formatting

Alternatives and Prior Art

Rendering in Canvas2D

Contributors:

Docs: Text API Overview

Background

Language and Alphabets

“The office 大楼 is in München.”

Fonts and Styles

Typefaces and Glyphs

Summary

Contributors:

Docs: Text Properties API

Functional Requirement

Ergonomics

Example

Related

Contributors:

Docs: The Raster Tragedy in Skia

The Gamma Hack

Optical Sizing Hack

The Contrast Hack

Docs: Two-point Conical Gradient

Problem Statement and Setup

Algorithm

Appendix

`raw_ptr<T>`