module Noir::JSLiteralScanner

Overview

JSLiteralScanner provides utilities for scanning JavaScript source code while properly skipping string literals, comments, template literals, and regex. This ensures parenthesis/brace matching doesn't get confused by literals.

Implementation note: every public entry point takes CHAR indices and returns CHAR indices (callers slice with char-based String#[]). The scanners themselves run over an indexed char source instead of probing the string with String#[](Int) — which is O(index) once a string contains any multi-byte UTF-8 — and accumulate through String::Builder instead of per-char String#+, which reallocated the whole prefix on every append. ASCII content (the overwhelmingly common case) scans its byte slice with zero allocation; non-ASCII content pays one up-front chars materialization and stays linear.

Defined in:

utils/js_literal_scanner.cr

Constant Summary

KEYWORD_WINDOW = 12

The regex-context checks only ever look at the tail of the scanned output: the last non-whitespace char, and ends_with? against the keywords above (longest: "instanceof", 10 chars). A rolling window of this size replaces re-materializing the whole accumulated prefix on every '/' encountered.

REGEX_PRECEDING_CHARS = Set {'(', '[', '{', ',', ':', ';', '=', '!', '&', '|', '?', '+', '-', '*', '%', '<', '>', '~', '^'}

Characters that can precede a regex literal (operators/punctuation expecting expression)

REGEX_PRECEDING_KEYWORDS = ["return", "case", "throw", "in", "of", "typeof", "instanceof", "void", "delete", "new"]

Keywords that can precede a regex literal in JavaScript

Class Method Summary

• .extract_paren_content(content : String, start_pos : Int32) : ScanResult | Nil

  Extract content between parentheses while skipping literals Returns the content inside the parentheses (not including the parens)

• .find_matching_brace(content : String, open_brace_idx : Int32) : Int32 | Nil

  Find matching closing brace, skipping literals

• .find_matching_paren(content : String, open_paren_idx : Int32) : Int32 | Nil

  Find matching closing paren, skipping literals

• .regex_context?(last_char : Char | Nil, last_word : String) : Bool

  The single rule for "can a '/' here start a regex literal?".

• .try_skip_literal(content : String, pos : Int32, accumulated : String) : NamedTuple(content: String, pos: Int32) | Nil

  Try to skip a literal at the current position Returns updated content string and position if a literal was skipped, nil otherwise

Class Method Detail