module Tput::Unicode

Overview

Unicode display-width support for terminal cells.

Terminal layout is measured in columns, not codepoints: a combining mark occupies 0 columns, an East-Asian-Wide / emoji glyph occupies 2, everything else 1. This module answers "how many columns does this text occupy", operating on grapheme clusters (via Crystal's UAX-#29 String#each_grapheme) so a user-perceived character is measured as a single unit.

Width rules follow the common wcwidth + East-Asian-Width convention plus emoji. Terminals disagree on a few cases (ambiguous-width chars, ZWJ emoji, flags); treat the values here as authoritative.

NOTE (known approximation): zero-width detection uses Char#mark?, which also matches spacing combining marks (general category Mc). True wcwidth only zero-widths Mn/Me. This is rare in TUI content and can be refined later by excluding Mc ranges.

Extended Modules

Defined in:

tput/unicode.cr

Constant Summary

WIDE = [{4352, 4447}, {8986, 8987}, {9001, 9002}, {9193, 9196}, {9200, 9200}, {9203, 9203}, {9725, 9726}, {9748, 9749}, {9800, 9811}, {9855, 9855}, {9875, 9875}, {9889, 9889}, {9898, 9899}, {9917, 9918}, {9924, 9925}, {9934, 9934}, {9940, 9940}, {9962, 9962}, {9970, 9971}, {9973, 9973}, {9978, 9978}, {9981, 9981}, {9989, 9989}, {9994, 9995}, {10024, 10024}, {10060, 10060}, {10062, 10062}, {10067, 10069}, {10071, 10071}, {10133, 10135}, {10160, 10160}, {10175, 10175}, {11035, 11036}, {11088, 11088}, {11093, 11093}, {11904, 12350}, {12353, 13311}, {13312, 19903}, {19968, 40959}, {40960, 42191}, {43360, 43391}, {44032, 55203}, {63744, 64255}, {65040, 65049}, {65072, 65135}, {65280, 65376}, {65504, 65510}, {110592, 110959}, {126980, 126980}, {127183, 127183}, {127374, 127374}, {127377, 127386}, {127488, 127743}, {127744, 128591}, {128640, 128767}, {128992, 129003}, {129008, 129008}, {129280, 129535}, {129648, 129791}, {131072, 262141}]

Sorted, non-overlapping ranges of East-Asian-Wide / Fullwidth codepoints plus emoji that render in 2 columns. Derived from the standard wcwidth/East-Asian-Width data.

Instance Method Summary

Instance Method Detail

def chop_grapheme(text : String) : String #

Returns text with its last grapheme cluster removed (e.g. a base + combining mark, or a wide emoji, comes off as one unit) — grapheme-aware backspace. Empty in, empty out.


[View source]
def cluster_size(g : String::Grapheme) : Int32 #

Codepoint count of a grapheme cluster, read from the stdlib-internal @cluster ivar (Char | String) so the common single-Char cluster costs no to_s allocation. Identical to g.to_s.size (a String-backed cluster is always multi-codepoint; a Char one is exactly 1). The same pinning spec that covers #width(String::Grapheme) covers this.


[View source]
def codepoint_width(char : Char) : Int32 #

Columns occupied by a single codepoint: 0 (control / combining / zero-width), 2 (East-Asian-Wide / emoji), or 1 (everything else).


[View source]
def display_width(string : String) : Int32 #

Display width, in terminal columns, of a whole string: the sum of the widths of its grapheme clusters.


[View source]
def extend_grapheme(content, ci : Int32, base : Char) : Tuple(String, Int32) #

Assembles the grapheme cluster beginning with base (codepoint at content[ci - 1]) by consuming following extending codepoints from content starting at ci: combining marks, ZWJ (and the codepoint it joins), variation selectors, emoji skin-tone modifiers, and a second regional indicator for flags. Returns {cluster, new_ci}.

A pragmatic subset of UAX-#29 covering cases that occur in terminal text; content is anything indexable by codepoint (#[]? returning Char?).


[View source]
def grapheme_extender?(c : Char) : Bool #

Whether c extends the grapheme cluster it follows: a combining mark, a zero-width joiner (U+200D), a variation selector (U+FE00..U+FE0F), or an emoji skin-tone modifier (U+1F3FB..U+1F3FF).


[View source]
def leading_byte_len(text : String, cols : Int32, full_unicode : Bool) : Int32 #

Byte length of the leading run of whole grapheme clusters of text that fits in width columns (never splitting a grapheme) — i.e. the end argument for text.byte_slice(0, …) when keeping a text's leading width columns. full_unicode selects the width metric per grapheme: true measures display columns (wide CJK/emoji count as 2), false counts codepoints (grapheme.size), preserving each caller's sizing.


[View source]
def needs_cluster?(base : Char, nxt : Char | Nil) : Bool #

Whether base begins a multi-codepoint grapheme cluster, given successor nxt — i.e. whether #extend_grapheme would assemble anything beyond base alone. Cheap pre-check letting a renderer skip cluster assembly for the common lone-codepoint cell. Mirrors #extend_grapheme's start conditions exactly.


[View source]
def pad(cell : String, width : Int32, align : AlignFlag | Nil) : String #

Pads cell with spaces to width display columns under horizontal alignment align (HCenter/Right, else left), returning a new String. A cell already at or over width is returned unchanged — this never clips.


[View source]
def regional_indicator?(c : Char) : Bool #

Whether c is a Unicode regional-indicator symbol (U+1F1E6..U+1F1FF); a pair of them forms a flag emoji.


[View source]
def trailing_byte_len(text : String, cols : Int32, full_unicode : Bool = true) : Int32 #

Byte offset at which the trailing run of whole grapheme clusters of text that fits in cols columns begins (never splitting a grapheme) — i.e. the start argument for text.byte_slice(…) when keeping a text's trailing cols columns. The suffix mirror of #leading_byte_len. full_unicode selects the width metric per grapheme: true measures display columns (wide CJK/emoji count as 2), false counts codepoints (grapheme.size).

Computed as "drop as few leading graphemes as needed so the remainder fits": one pass sums the total width, a second drops leading clusters until the remainder is within cols. Yields the same longest-fitting suffix a greedy scan-from-the-end would, with no per-grapheme allocation.


[View source]
def width(grapheme : String::Grapheme) : Int32 #

Display width of a single grapheme cluster (yielded by each_grapheme).

Reads the stdlib-internal @cluster ivar (Char | String) directly to avoid the fresh String that grapheme.to_s allocates for the common Char-backed cluster. Behavior-identical to #width(grapheme.to_s): a single codepoint admits no VS16 promotion (it cannot carry a following U+FE0F), so the String overload's VS16 scan would be a no-op; a lone regional indicator still renders wide. Multi-codepoint clusters take the String branch, preserving VS16/flag handling exactly. A spec pins the @cluster layout this depends on.


[View source]
def width(grapheme : String) : Int32 #

Display width of a single grapheme cluster given as a String.

The width is driven by the cluster's base codepoint; trailing combining marks / joiners / variation selectors add nothing. Two cluster shapes are special-cased to 2 columns: regional-indicator flags, and emoji whose presentation is forced wide by a VS16 (U+FE0F) selector.


[View source]
def width(char : Char) : Int32 #

Display width of a single codepoint, ignoring clustering. Prefer the grapheme-aware #width for user text; this is the low-level building block.


[View source]