class
Noir::PhpLexer
- Noir::PhpLexer
- Reference
- Object
Overview
PhpLexer is a hand-rolled structural lexer for PHP source. It exists to replace the per-analyzer character state machines that every PHP analyzer re-implements (balanced-brace matching, statement-end scanning, string/ comment skip ranges) with a single shared pass that is:
- open/close-tag aware — a
.phpfile is HTML with islands of code in it, so everything outside<?php … ?>is inert output text. Lexing it as code let a lone apostrophe in<h1>Today's report</h1>open a string that masked the rest of the file, silently dropping every route declared below it. - heredoc/nowdoc aware —
<<<EOT … EOT/<<<'EOT' … EOTbodies are masked, so route-shaped text or stray{};inside a heredoc can no longer leak as a false endpoint or corrupt a brace/statement bound. None of the pre-existing scanners handled<<<at all. - PHP-8 attribute aware —
#[Route(...)]is code, not a#comment. - linear on multi-byte input — the source is materialised once into an
Array(Char)with O(1) indexing, so CJK-commented controllers stay O(n) instead of the O(n^2) thatString#[](Int)caused.
The lexer masks every non-code region (inline HTML, strings, comments,
heredoc/nowdoc bodies) into spaces in @masked while preserving newlines and overall
length, so the structural helpers in Noir::MaskedLexer are plain depth
counters over @masked with no string-state bookkeeping of their own.
Included Modules
Defined in:
minilexers/php_lexer.crConstructors
-
.new(source : String, *, php_mode : Bool = false)
sourceis a whole.phpfile by default, so lexing starts in HTML mode: nothing is code until the first<?php/<?=.
Instance Method Summary
-
#expression_end(start_pos : Int32) : Int32
Index of the first top-level expression terminator (
,;or a closing) ] }that would pop above the starting level) at or afterstart_pos. -
#tokens : Array(PhpToken)
Lazily produce a flat token stream over the source: structural delimiters,
->/::/=>operators, identifiers,$variables, and one token per string/comment/heredoc span.
Instance methods inherited from module Noir::MaskedLexer
in_code?(pos : Int32) : Bool
in_code?,
masked : Array(Char)
masked,
matching_delimiter(open_pos : Int32) : Int32 | Nil
matching_delimiter,
skip_ranges : Array(Range(Int32, Int32))
skip_ranges,
statement_end(start_pos : Int32) : Int32
statement_end
Constructor Detail
source is a whole .php file by default, so lexing starts in HTML
mode: nothing is code until the first <?php / <?=. Pass
php_mode: true when source is a fragment already carved out of a PHP
region (a closure body handed back for a recursive pass, say), which by
construction has no opening tag of its own.
Instance Method Detail
Index of the first top-level expression terminator (, ; or a closing
) ] } that would pop above the starting level) at or after start_pos.
Mirrors find_arrow_expression_end.
Lazily produce a flat token stream over the source: structural
delimiters, ->/::/=> operators, identifiers, $variables, and one
token per string/comment/heredoc span. This is the reusable miniparser
surface for consumers that want to walk PHP structurally (e.g. following
a Route::a(...)->b(...)->group(...) method chain).