diff options
| author | Cliff Click <cliffc@acm.org> | 2026-11-01 10:33:55 -0800 |
|---|---|---|
| committer | Cliff Click <cliffc@acm.org> | 2026-11-01 10:33:55 -0800 |
| commit | f4bd723d929e53475f7990374dca7b7dfcbbdc42 (patch) | |
| tree | 7bb080119e7b88ad4ac9b30af019fe6a5e4ba9dd /chapter29/docs/29-grammar.md | |
| parent | 0ac0c9e10287972ac3af5a4ee43d26f0d5b318fe (diff) | |
| download | Simple-main.tar.gz Simple-main.zip | |
Add Chapter 29 for tech debt cleanup with the parser splitmain
Keep Chapter 28b intact and carry the grammar-oriented parser package, Var move, IterPeeps move, and parser regression tests into Chapter 29. Defer privacy and exclude its experimental tests and fixture changes.
Register the chapter in the indexes, IntelliJ modules and run configurations, Maven reactor, roadmap, and linear-history publisher. Windows and Linux each pass 573 tests, release builds, 199 allocation replays, and 29 native checks.
Diffstat (limited to 'chapter29/docs/29-grammar.md')
| -rw-r--r-- | chapter29/docs/29-grammar.md | 233 |
1 files changed, 233 insertions, 0 deletions
diff --git a/chapter29/docs/29-grammar.md b/chapter29/docs/29-grammar.md new file mode 100644 index 00000000..58c9b30c --- /dev/null +++ b/chapter29/docs/29-grammar.md @@ -0,0 +1,233 @@ +# Grammar for Chapter 29 + +English | [日本語](29-grammar.ja.md) + +The handwritten recursive-descent implementation lives in +[`com.seaofnodes.simple.parser`](../src/main/java/com/seaofnodes/simple/parser/). +The production-to-code map below the grammar describes where syntax ends and +name resolution or graph construction begins. This grammar documents the +language; it is not input to a parser generator. + +```antlrv4 +grammar SimpleLanguage; + +// Name lookup, assignable operands, type compatibility, and access permissions +// are checked by the compiler in addition to these syntax rules. +// A file is a compilation unit; its path supplies its namespace. +// Qualified names resolve nested types and other units without an import keyword. +program : statement* EOF ; + +block : '{' statement* '}' ; + +statement + : block + | returnStatement + | ifStatement + | whileStatement + | forStatement + | breakStatement + | continueStatement + | structDeclaration + | declaration ';' + | assignment ';' + | ';' + ; + +returnStatement : 'return' assignment ';' ; +ifStatement : 'if' '(' assignment ')' statement ('else' statement)? ; +whileStatement : 'while' '(' assignment ')' statement ; +breakStatement : 'break' ';' ; +continueStatement : 'continue' ';' ; + +forStatement + : 'for' '(' (declaration | assignment)? ';' assignment? ';' assignment? ')' statement + ; + +structDeclaration : 'struct' IDENTIFIER '{' structMember* '}' ';' ; +structMember : constructorDeclaration | methodDeclaration | statement ; + +// The constructor name must match the enclosing struct's name. +constructorDeclaration : 'new' IDENTIFIER '=' functionExpression ';' ; + +// A method is an ordinary fixed function binding in a struct definition. +// The compiler supplies self; it is not written in parameterList. +// Function types and expressions both receive their implicit receiver here. +methodDeclaration : 'val' bindingModifier? IDENTIFIER '=' functionExpression ';' ; + +declaration : declarationType binding (',' binding)* ; +declarationType : type | 'var' | 'val' ; +binding : bindingModifier? IDENTIFIER ('=' initializer)? ; +bindingModifier : '!' | '~' ; +// A declaration initializer consisting of the string "C" declares an extern +// symbol: a function, or explicit numeric native storage. Ordinary strings +// elsewhere are array-valued expressions. +initializer : assignment ; + +// The prefix qualifies the named struct reference, not an enclosing array. +// Each [] / [~] suffix qualifies its own array layer. ? requires a reference. +type + : functionType + | primitiveType typeSuffix* + | accessModifier? typeName typeSuffix* + ; +accessModifier : '!' | '~' ; +typeSuffix : '?' | '[]' | '[~]' ; +typeName : IDENTIFIER ('.' IDENTIFIER)* ; + +primitiveType + : 'int' | 'i8' | 'i16' | 'i32' | 'i64' + | 'u1' | 'u8' | 'u16' | 'u32' | 'byte' | 'bool' + | 'flt' | 'f32' | 'f64' + ; + +assignment : expression (assignmentOperator assignment)? ; +assignmentOperator + : '=' | '+=' | '-=' | '*=' | '/=' | '&=' | '|=' | '^=' + | '<<=' | '>>=' | '>>>=' + ; + +expression : logicalExpression ('?' assignment (':' assignment)?)? ; + +// Both logical operators short-circuit and associate to the right here. +logicalExpression : bitwiseExpression (('&&' | '||') logicalExpression)? ; + +bitwiseExpression + : equalityExpression (('&' | '|' | '^') equalityExpression)* + ; + +equalityExpression : comparisonExpression (('==' | '!=') comparisonExpression)* ; + +// Chained comparisons test adjacent operands and short-circuit. A chain must +// keep its direction; equality has lower precedence and does not join a chain. +comparisonExpression + : shiftExpression + ( (('<' | '<=') shiftExpression)+ | (('>' | '>=') shiftExpression)+ )? + ; + +shiftExpression + : additiveExpression (('<<' | '>>' | '>>>') additiveExpression)* + ; + +additiveExpression + : multiplicativeExpression (('+' | '-') multiplicativeExpression)* + ; + +multiplicativeExpression + : unaryExpression (('*' | '/') unaryExpression)* + ; + +unaryExpression + : ('-' | '!') unaryExpression + | updateOperator IDENTIFIER + | postfixExpression + ; + +// Prefix updates require a local name; postfix updates also allow fields +// and array elements. All updates require an assignable operand. +updateOperator : '++' | '--' ; +postfixExpression : primaryExpression postfixSuffix* updateOperator? ; +postfixSuffix : '.' IDENTIFIER | '[' assignment ']' | '#' | callSuffix | methodCallSuffix ; + +// For a fixed function-valued member, object.name(args) passes object as self. +// This overlaps ordinary field selection followed by a call syntactically; +// name/type resolution determines which call form applies. +methodCallSuffix : '.' IDENTIFIER callSuffix ; + +primaryExpression + : INTEGER_LITERAL + | FLOAT_LITERAL + | STRING_LITERAL + | CHARACTER_LITERAL + | 'true' + | 'false' + | 'null' + | IDENTIFIER + | '(' assignment ')' + | newExpression + | functionExpression + ; + +// A function is an expression bound with an ordinary declaration. +// Its last statement supplies the implicit result; return can exit earlier. +functionExpression : '{' parameterList? '->' statement* '}' ; +parameterList : (parameter ','?)+ ; +parameter : type bindingModifier? IDENTIFIER ; + +// Function types have unnamed, space-separated argument types. +// {int} is the zero-argument type; its function expression still uses ->. +functionType + : '{' type+ '->' type '}' '?'? + | '{' type '}' '?'? + ; + +callSuffix : '(' argumentList? ')' ; +argumentList : assignment (',' assignment)* ','? ; + +// Allocation invokes the declared constructor; inline initializer blocks from +// earlier chapters are replaced by constructor arguments. +newExpression : 'new' type ('[' assignment ']' | callSuffix)? ; + +INTEGER_LITERAL : '0' | [1-9] DIGIT* ; +// The f suffix also accepts an integer-shaped token, e.g. 12f. +FLOAT_LITERAL : DIGIT+ ('.' DIGIT* ('e' DIGIT+)? | 'e' '-'? DIGIT+) 'f'? + | DIGIT+ 'f' ; +STRING_LITERAL : '"' (ESCAPE | ~["\\])* '"' ; +CHARACTER_LITERAL : '\'' (ESCAPE | ~['\\]) '\'' ; +fragment ESCAPE : '\\' [0ntr\\'"] ; +IDENTIFIER : NON_DIGIT (NON_DIGIT | DIGIT)* ; +fragment NON_DIGIT : [a-zA-Z_] ; +fragment DIGIT : [0-9] ; +WHITESPACE : [ \t\r\n]+ -> skip ; +LINE_COMMENT : '//' ~[\r\n]* -> skip ; +BLOCK_COMMENT : '/*' .*? '*/' -> skip ; +``` + +## Reading the parser alongside the grammar + +All paths below are within `src/main/java/com/seaofnodes/simple/parser/`. +The components share one `Parser` instance containing the cursor, active scope, +compilation unit, and graph utilities. They are package-private collaborators; +there is no inheritance chain, AST, or additional compiler pass. + +| Productions | Implementation | +| --- | --- | +| `program` | [`Parser.parse`](../src/main/java/com/seaofnodes/simple/parser/Parser.java): establish the source-unit scope, parse its initializer, resolve remaining file names, publish its class | +| `statement`, `block`, loops, jumps, `returnStatement` | [`Statements`](../src/main/java/com/seaofnodes/simple/parser/Statements.java), starting at `parseStatement` | +| `declaration`, `binding`, `bindingModifier` | [`Declarations`](../src/main/java/com/seaofnodes/simple/parser/Declarations.java): declaration/expression lookahead, then each binding and initializer | +| `structDeclaration`, `constructorDeclaration` | [`Structs`](../src/main/java/com/seaofnodes/simple/parser/Structs.java), starting at `parseStruct` and `parseConstructorDeclaration`; field discovery and initialization checks follow these entry points | +| `type`, `typeName`, `functionType` | [`Types`](../src/main/java/com/seaofnodes/simple/parser/Types.java): `parseType` and `parseFunctionType`; qualified-name lookup lives in `typeName`, `dottedName`, and `rootedName` | +| `assignment` through `primaryExpression` | [`Expressions`](../src/main/java/com/seaofnodes/simple/parser/Expressions.java): `parseAssignment`, `parseExpression`, `parseLogical`, `parseBitwise`, `parseEquality`, `parseComparison`, `parseShift`, `parseAddition`, `parseMultiplication`, `parseUnary`, `parsePrimary` | +| `postfixExpression`, member selection and updates | `Expressions.parsePostfix` / `parsePostfixMethod` dispatch to [`References.parseField`](../src/main/java/com/seaofnodes/simple/parser/References.java); `References.parseName` handles bare names, implicit receivers, and forward references | +| `functionExpression`, `parameterList`, calls, `argumentList` | [`Functions`](../src/main/java/com/seaofnodes/simple/parser/Functions.java): `parseFunction`, `parseConstructor`, shared `parameters` and `arguments`, then `parseBody` | +| `newExpression` | [`Allocations.parseNew`](../src/main/java/com/seaofnodes/simple/parser/Allocations.java); the allocation-helper graph builder follows the expression routines | +| Literals, identifiers, comments, token matching | [`Lexer`](../src/main/java/com/seaofnodes/simple/parser/Lexer.java); it also supplies source-location snapshots used by diagnostics and serialized IR | + +`ControlFlow` shares guarded scope merging between `if`, `?:`, `&&`, and `||`. +`Conversions` shares binding, field, and result conversions. These +operations are compiler semantics rather than additional grammar productions. +`Parser` retains the type registry and the small control/memory/peephole helpers +used by several productions and by the rest of the compiler. + +There are a few deliberate differences between a syntax-only parser and this +one-pass IR builder: + +- A statement beginning with `{` might be a block or a function-type binding. + `Declarations` and `Types` use speculative parsing and cursor restore; + qualified type lookup also stops before a suffix that belongs to an expression. +- Bare assignment, field assignment, and compound updates share the grammar's + precedence but use different IR paths. `References` decides between an SSA + local, a constructor field's local value, and an actual memory access. A + function-valued field may also supply an implicit receiver to a call. +- `for` updates are scanned first and parsed after the body, where their effects + belong. Explicit constructor bodies are scanned first and parsed after field + discovery, so later fields have local slots. Both use `Lexer.skipTo`, which + ignores delimiters inside comments and quoted text. +- `Structs.parseInitializer` parses a file or struct body as a function while + discovering fields. `Functions.parseBody` still owns function scope/return + construction; moving it does not change constructor or memory semantics. + +The syntax chart does not describe all name/type restrictions. In particular, +prefix updates require an already-known non-pointer binding, and a private-class +privacy policy is separate from recognizing a qualified type name. The existing +same-named function binding inside a struct is also recognized as a constructor; +`new Name = { ... };` is the explicit constructor spelling above. |
