fix(awk): honor POSIX statement continuation across newlines (#206)
The awk lexer emitted a NEWLINE token unconditionally when it saw `\n`,
even when the previous token was one of the continuation-allowing tokens
POSIX awk specifies (`,`, `{`, `&&`, `||`, `?`, `:`, `do`, `else`, `if`,
`while`). Common multi-line idioms like
printf "%s=%d\n",
$1, $2
(comma at end-of-line followed by indented args on the next line) parsed
as two separate statements with a stray NEWLINE in the middle, surfacing
as "Unexpected token: NEWLINE" — even though gawk, mawk, and the BSD
one-true-awk all accept this form.
The lexer already tracks `lastTokenType` as an instance property, so the
fix is a small inject in `nextToken`: when the next character is `\n`
and `lastTokenType` is in `CONTINUES_ACROSS_NEWLINE`, swallow the
newline and recurse to the next real token instead of emitting one.
Adds eight regression tests covering each continuation-allowing token
plus the TSV → SQL INSERT printf idiom that motivated the fix. Z
Zach Smith committed
6ccc35f5a9b5c6f395b145ed2ec7ee71c4862057
Parent: 97d0c34
Committed by GitHub <noreply@github.com>
on 4/29/2026, 12:11:09 AM