module Noir::MaskedLexer

Overview

Structural helpers shared by the hand-rolled masking lexers (Noir::PhpLexer, Noir::CSharpLexer, Noir::ScalaLexer).

Each of those lexers exists for the same reason: the analyzers they feed used to count {/}/(/)/; per line with no string awareness, so a single delimiter inside a string literal, a comment or a heredoc body truncated a method block (dropping callees), made a signature run away (dropping parameters), or leaked route-shaped text as a phantom endpoint. Each lexer therefore makes ONE linear pass that blanks every non-code region to spaces — newlines preserved, character length unchanged — into @masked, and records each blanked region in @spans.

The language-specific part is that masking pass (PHP heredocs and #[…] attributes, the C# string zoo, Scala's nested block comments and triple-quoted strings). Everything below is what is left once masking is done: plain depth counters and span lookups over @masked/@spans with no string-state bookkeeping of their own, and therefore language-agnostic.

An includer must declare @masked, @size, @spans and @skip_ranges. They are declared here so the contract is enforced at compile time rather than restated (and potentially drifting) in each lexer.

Direct including types

Defined in:

minilexers/masked_lexer.cr

Instance Method Summary

Instance Method Detail

def in_code?(pos : Int32) : Bool #

[View source]
def masked : Array(Char) #

Code with strings, comments and any other non-code region (PHP heredoc bodies, Scala triple-quoted strings, …) blanked to spaces. Same character length as the source and newlines preserved, so line/offset math against the original content stays valid.


[View source]
def matching_delimiter(open_pos : Int32) : Int32 | Nil #

Index of the delimiter that closes the (/[/{ at open_pos, or nil. Counts only the matching pair type, which is correct for balanced code and mirrors the per-analyzer scanners this replaces (e.g. PHP's find_matching_php_close_brace).


[View source]
def skip_ranges : Array(Range(Int32, Int32)) #

Character ranges occupied by the non-code regions: strings, comments and whatever else the includer masks (PHP heredoc/nowdoc bodies, …).


[View source]
def statement_end(start_pos : Int32) : Int32 #

Index just after the top-level ; at or after start_pos, or the source size when none is found. Mirrors PHP's find_php_statement_end.


[View source]