struct Noir::TopLevelSplit::Rules

Overview

The seven axes along which the hand-rolled copies differed.

Defined in:

utils/top_level_split.cr

Constant Summary

CPP = new(nest: (Nest::Paren | Nest::Bracket) | Nest::Brace, quotes: "\"", escape: Escape::InQuotes, strip: true, empties: Empties::DropTrailing, per_kind: true, clamp: true)

C++ call-argument lists. Serves: analyzer/analyzers/cpp/{drogon,httplib,oatpp}.cr split_top_level_args. Double quotes only (C++ single quotes are char literals and never wrap a route), per-kind counters and clamping both taken verbatim from those three bodies, <> deliberately NOT counted.

GENERICS_ONLY = new(nest: Nest::Angle, quotes: "", escape: Escape::None, strip: false, empties: Empties::Keep, per_kind: false, clamp: true)

Type lists: angle brackets only, no quote handling. Serves the three byte-identical split_top_level_commas clones in miniparsers/{java_route,jaxrs,micronaut}_extractor_ts.cr. All three run on the tail of an implements clause with the class body already truncated at {, so only generic arguments can nest and a stray " would swallow the rest of the type list. They keep empties and do not strip; every caller strips each part itself and skips the empties.

JAVA = new(nest: (Nest::Paren | Nest::Bracket) | Nest::Brace, quotes: "\"'", escape: Escape::InQuotes, strip: true, empties: Empties::DropTrailing, per_kind: false, clamp: true)

Java annotation and call-argument lists. Serves four sites in analyzer/analyzers/java/: armeria.cr split_top_level_args AND split_top_level_concat (same body, + instead of ,), dropwizard.cr split_top_level_args, and vertx.cr split_top_level, which takes the separator as a parameter and is called with both , and +. Note dropwizard's copy was written against a String::Builder and the other three against index slices; they are nevertheless observably identical, unbalanced input included. One shared depth is unanimous across the Java splitters, unlike Python and JS.

Three Java sites each disagree with this preset on one or two axes and so carry a file-local Rules constant rather than a preset here — each is used by exactly one splitter, so naming them centrally would put three single-use constants in this file: quarkus.cr split_top_level_args -> nest also includes Angle wicket.cr split_arguments -> nest also includes Angle, quotes """ only spring.cr split_top_level_concat -> quotes """, Empties::DropAll

JS = new(nest: (Nest::Paren | Nest::Bracket) | Nest::Brace, quotes: "\"'`", escape: Escape::InQuotes, strip: true, empties: Empties::DropAll, per_kind: true, clamp: true)

JavaScript/TypeScript object-literal and argument lists. Serves analyzer/analyzers/javascript/nestjs.cr and analyzer/analyzers/typescript/loopback.cr split_top_level, which are byte-identical and are called with both , and +. Backticks are quote characters because template literals wrap most route strings; missing them merges a whole ${...} route into one part.

Three more JS/TS sites each disagree on one to four axes and carry a file-local Rules constant rather than a preset here, each being the only user of its variant: typescript/trpc.cr split_top_level -> per_kind: false javascript/nextjs.cr split_top_level_commas -> no quotes, no escape, nest adds Angle, strip: false, Empties::Keep javascript/remix.cr split_flat_segments -> Nest::Bracket only, no quotes, strip: false, Empties::Keep, delimiter . javascript/express/router_mount_scanner.cr split_at_top_level_commas -> strip: false, Empties::DropTrailing

That last one was the only site converted with a DELIBERATE behavior change: its hand-rolled body closed a quoted run with a prev_char != '\\' lookback, which reads "a\\" as unterminated because the escaped backslash is taken as escaping the closing quote. No Escape value reproduces that, and reproducing it was not worth doing — it is a bug, and Escape::InQuotes is what the lookback was reaching for.

JS_POSITIONAL_ARGS = new(nest: (Nest::Paren | Nest::Bracket) | Nest::Brace, quotes: "\"'`", escape: Escape::InQuotes, strip: true, empties: Empties::Keep, per_kind: false, clamp: true)

JavaScript/TypeScript call-argument lists split with split_spans, where the caller needs each argument's absolute position as well as its text. Serves the three byte-identical split_top_level_args copies in analyzer/analyzers/javascript/{express,feathers,hono}.cr and miniparsers/js_http_route_extractor.cr split_top_level.

Differs from JS on two axes, both load-bearing:

  • one SHARED depth counter, not per-kind
  • Empties::Keep, because every caller indexes the result positionally (args[0] is the route, args[2] the handler), so an empty argument must hold its slot or the handler shifts left.
PYTHON = new(nest: (Nest::Paren | Nest::Bracket) | Nest::Brace, quotes: "\"'", escape: Escape::InQuotes, strip: false, empties: Empties::Keep, per_kind: true, clamp: true)

Python call-argument lists and expression terms. Serves the six byte-identical split_python_arguments clones in analyzer/analyzers/python/{bottle,cherrypy,django,pyramid,sanic, starlette}.cr plus django.cr split_python_expression_terms (same body, + instead of ,). Note these deliberately do NOT strip and keep every empty part — callers strip themselves and some index positionally, so an interior empty must hold its slot.

Near misses that need their own Rules when converted: flask.cr split_python_call_args -> Escape::Always, strip: true, Empties::DropAll, and one counter shared by [/{ only — not expressible here.

SHARED_DEPTH_RAW = new(nest: (Nest::Paren | Nest::Bracket) | Nest::Brace, quotes: "\"'", escape: Escape::InQuotes, strip: false, empties: Empties::Keep, per_kind: false, clamp: true)

PYTHON with one shared depth counter instead of per-kind counters. Serves analyzer/analyzers/python/{django_ninja,falcon}.cr split_python_arguments, fastapi.cr split_python_top_level and elixir/elixir_phoenix.cr split_top_level_commas.

A named preset rather than four inline Rules.new(...) literals because the split is not a per-file accident: the Python analyzers genuinely disagree on per_kind, and keeping the two variants adjacent is what makes that disagreement — and the fact that it is only observable on unbalanced input — legible. Four copies of the same seven-argument literal in four files is exactly the drift this module exists to end.

Named for the shape and not for Python because it turned out not to be a Python idiom at all: Phoenix's splitter, written independently in another language, agrees on all seven axes. It was PYTHON_SHARED_DEPTH while Python was its only user.

Constructors

Instance Method Summary

Constructor Detail

def self.new(nest : Nest = (Nest::Paren | Nest::Bracket) | Nest::Brace, quotes : String = "\"'", escape : Escape = Escape::InQuotes, strip : Bool = true, empties : Empties = Empties::DropTrailing, per_kind : Bool = false, clamp : Bool = true) #

[View source]

Instance Method Detail

def clamp? : Bool #

true => a closer at depth 0 is ignored (d -= 1 if d > 0) false => depth is allowed to go negative

Same reasoning as per_kind. Most replaced sites clamp, but three do a bare depth -= 1 — erlang/cowboy.cr split_top_level, php/wordpress.cr split_top_level_args, and engines/cfml_engine.cr split_arguments (the last one is easy to miss: it is on the shared engine, not an analyzer, so a survey scoped to analyzers/ reports only two). On a fragment that closes a bracket opened in a prefix the regex already discarded (e.g. ")x, y") that is the difference between one part and two. Splitting requires depth EXACTLY 0, so a negative depth suppresses every remaining split rather than re-enabling them.


[View source]
def empties : Empties #

Empty-part policy; see Empties.


[View source]
def escape : Escape #

Backslash policy; see Escape.


[View source]
def nest : Nest #

Bracket kinds that contribute depth.


[View source]
def per_kind? : Bool #

true => an independent depth counter per bracket kind false => one shared depth across all enabled kinds

Configurable rather than fixed because it is observable on UNBALANCED input, which is what these splitters actually receive: every caller hands them a regex-sliced fragment, not a parsed expression. On "[a)b, c" a shared counter reads [ as +1 and ) as -1 and splits at the comma; per-kind counters leave the bracket depth at 1 and emit one part. Normalizing to either value would have moved endpoints in whichever group lost.


[View source]
def quotes : String #

Characters that open and close a quoted run. A run is closed by the same character that opened it, so "'" + "\"" in one string handles both quote styles without letting "it's" open an apostrophe run. "" disables quote handling entirely.


[View source]
def strip? : Bool #

Whether each part is stripped. Applied BEFORE the Empties policy, so a whitespace-only part counts as empty.


[View source]