diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/chapter28b/28b-grammar.ja.md | 175 | ||||
| -rw-r--r-- | docs/chapter28b/28b-grammar.md | 175 | ||||
| -rw-r--r-- | docs/chapter28b/README.ja.md | 146 | ||||
| -rw-r--r-- | docs/chapter28b/README.md | 233 |
4 files changed, 729 insertions, 0 deletions
diff --git a/docs/chapter28b/28b-grammar.ja.md b/docs/chapter28b/28b-grammar.ja.md new file mode 100644 index 00000000..65944fe2 --- /dev/null +++ b/docs/chapter28b/28b-grammar.ja.md @@ -0,0 +1,175 @@ +# 第28b章の文法 + +[English](28b-grammar.md) | 日本語 + +```antlrv4 +grammar SimpleLanguage; + +// Name lookup, assignable operands, type compatibility, and access permissions +// are checked by the compiler in addition to these syntax rules. +// A file is a compilation unit; its path supplies its namespace. +// Qualified names resolve nested types and other units without an import keyword. +program : statement* EOF ; + +block : '{' statement* '}' ; + +statement + : block + | returnStatement + | ifStatement + | whileStatement + | forStatement + | breakStatement + | continueStatement + | structDeclaration + | declaration ';' + | assignment ';' + | ';' + ; + +returnStatement : 'return' assignment ';' ; +ifStatement : 'if' '(' assignment ')' statement ('else' statement)? ; +whileStatement : 'while' '(' assignment ')' statement ; +breakStatement : 'break' ';' ; +continueStatement : 'continue' ';' ; + +forStatement + : 'for' '(' (declaration | assignment)? ';' assignment? ';' assignment? ')' statement + ; + +structDeclaration : 'struct' IDENTIFIER '{' structMember* '}' ';' ; +structMember : constructorDeclaration | methodDeclaration | statement ; + +// The constructor name must match the enclosing struct's name. +constructorDeclaration : 'new' IDENTIFIER '=' functionExpression ';' ; + +// A method is an ordinary fixed function binding in a struct definition. +// The compiler supplies self; it is not written in parameterList. +// Function types and expressions both receive their implicit receiver here. +methodDeclaration : 'val' bindingModifier? IDENTIFIER '=' functionExpression ';' ; + +declaration : declarationType binding (',' binding)* ; +declarationType : type | 'var' | 'val' ; +binding : bindingModifier? IDENTIFIER ('=' initializer)? ; +bindingModifier : '!' | '~' ; +// A declaration initializer consisting of the string "C" declares an extern +// symbol: a function, or explicit numeric native storage. Ordinary strings +// elsewhere are array-valued expressions. +initializer : assignment ; + +// The prefix qualifies the named struct reference, not an enclosing array. +// Each [] / [~] suffix qualifies its own array layer. ? requires a reference. +type + : functionType + | primitiveType typeSuffix* + | accessModifier? typeName typeSuffix* + ; +accessModifier : '!' | '~' ; +typeSuffix : '?' | '[]' | '[~]' ; +typeName : IDENTIFIER ('.' IDENTIFIER)* ; + +primitiveType + : 'int' | 'i8' | 'i16' | 'i32' | 'i64' + | 'u1' | 'u8' | 'u16' | 'u32' | 'byte' | 'bool' + | 'flt' | 'f32' | 'f64' + ; + +assignment : expression (assignmentOperator assignment)? ; +assignmentOperator + : '=' | '+=' | '-=' | '*=' | '/=' | '&=' | '|=' | '^=' + | '<<=' | '>>=' | '>>>=' + ; + +expression : logicalExpression ('?' assignment (':' assignment)?)? ; + +// Both logical operators short-circuit and associate to the right here. +logicalExpression : bitwiseExpression (('&&' | '||') logicalExpression)? ; + +bitwiseExpression + : equalityExpression (('&' | '|' | '^') equalityExpression)* + ; + +equalityExpression : comparisonExpression (('==' | '!=') comparisonExpression)* ; + +// Chained comparisons test adjacent operands and short-circuit. A chain must +// keep its direction; equality has lower precedence and does not join a chain. +comparisonExpression + : shiftExpression + ( (('<' | '<=') shiftExpression)+ | (('>' | '>=') shiftExpression)+ )? + ; + +shiftExpression + : additiveExpression (('<<' | '>>' | '>>>') additiveExpression)* + ; + +additiveExpression + : multiplicativeExpression (('+' | '-') multiplicativeExpression)* + ; + +multiplicativeExpression + : unaryExpression (('*' | '/') unaryExpression)* + ; + +unaryExpression + : ('-' | '!') unaryExpression + | updateOperator IDENTIFIER + | postfixExpression + ; + +// Prefix updates require a local name; postfix updates also allow fields +// and array elements. All updates require an assignable operand. +updateOperator : '++' | '--' ; +postfixExpression : primaryExpression postfixSuffix* updateOperator? ; +postfixSuffix : '.' IDENTIFIER | '[' assignment ']' | '#' | callSuffix | methodCallSuffix ; + +// For a fixed function-valued member, object.name(args) passes object as self. +// This overlaps ordinary field selection followed by a call syntactically; +// name/type resolution determines which call form applies. +methodCallSuffix : '.' IDENTIFIER callSuffix ; + +primaryExpression + : INTEGER_LITERAL + | FLOAT_LITERAL + | STRING_LITERAL + | CHARACTER_LITERAL + | 'true' + | 'false' + | 'null' + | IDENTIFIER + | '(' assignment ')' + | newExpression + | functionExpression + ; + +// A function is an expression bound with an ordinary declaration. +// Its last statement supplies the implicit result; return can exit earlier. +functionExpression : '{' parameterList? '->' statement* '}' ; +parameterList : (parameter ','?)+ ; +parameter : type bindingModifier? IDENTIFIER ; + +// Function types have unnamed, space-separated argument types. +// {int} is the zero-argument type; its function expression still uses ->. +functionType + : '{' type+ '->' type '}' '?'? + | '{' type '}' '?'? + ; + +callSuffix : '(' argumentList? ')' ; +argumentList : assignment (',' assignment)* ','? ; + +// Allocation invokes the declared constructor; inline initializer blocks from +// earlier chapters are replaced by constructor arguments. +newExpression : 'new' type ('[' assignment ']' | callSuffix)? ; + +INTEGER_LITERAL : '0' | [1-9] DIGIT* ; +FLOAT_LITERAL : DIGIT+ ('.' DIGIT* ('e' DIGIT+)? | 'e' '-'? DIGIT+) ; +STRING_LITERAL : '"' (ESCAPE | ~["\\])* '"' ; +CHARACTER_LITERAL : '\'' (ESCAPE | ~['\\]) '\'' ; +fragment ESCAPE : '\\' [0ntr\\'"] ; +IDENTIFIER : NON_DIGIT (NON_DIGIT | DIGIT)* ; +fragment NON_DIGIT : [a-zA-Z_] ; +fragment DIGIT : [0-9] ; +WHITESPACE : [ \t\r\n]+ -> skip ; +LINE_COMMENT : '//' ~[\r\n]* -> skip ; +BLOCK_COMMENT : '/*' .*? '*/' -> skip ; +``` diff --git a/docs/chapter28b/28b-grammar.md b/docs/chapter28b/28b-grammar.md new file mode 100644 index 00000000..c7a279fa --- /dev/null +++ b/docs/chapter28b/28b-grammar.md @@ -0,0 +1,175 @@ +# Grammar for Chapter 28b + +English | [日本語](28b-grammar.ja.md) + +```antlrv4 +grammar SimpleLanguage; + +// Name lookup, assignable operands, type compatibility, and access permissions +// are checked by the compiler in addition to these syntax rules. +// A file is a compilation unit; its path supplies its namespace. +// Qualified names resolve nested types and other units without an import keyword. +program : statement* EOF ; + +block : '{' statement* '}' ; + +statement + : block + | returnStatement + | ifStatement + | whileStatement + | forStatement + | breakStatement + | continueStatement + | structDeclaration + | declaration ';' + | assignment ';' + | ';' + ; + +returnStatement : 'return' assignment ';' ; +ifStatement : 'if' '(' assignment ')' statement ('else' statement)? ; +whileStatement : 'while' '(' assignment ')' statement ; +breakStatement : 'break' ';' ; +continueStatement : 'continue' ';' ; + +forStatement + : 'for' '(' (declaration | assignment)? ';' assignment? ';' assignment? ')' statement + ; + +structDeclaration : 'struct' IDENTIFIER '{' structMember* '}' ';' ; +structMember : constructorDeclaration | methodDeclaration | statement ; + +// The constructor name must match the enclosing struct's name. +constructorDeclaration : 'new' IDENTIFIER '=' functionExpression ';' ; + +// A method is an ordinary fixed function binding in a struct definition. +// The compiler supplies self; it is not written in parameterList. +// Function types and expressions both receive their implicit receiver here. +methodDeclaration : 'val' bindingModifier? IDENTIFIER '=' functionExpression ';' ; + +declaration : declarationType binding (',' binding)* ; +declarationType : type | 'var' | 'val' ; +binding : bindingModifier? IDENTIFIER ('=' initializer)? ; +bindingModifier : '!' | '~' ; +// A declaration initializer consisting of the string "C" declares an extern +// symbol: a function, or explicit numeric native storage. Ordinary strings +// elsewhere are array-valued expressions. +initializer : assignment ; + +// The prefix qualifies the named struct reference, not an enclosing array. +// Each [] / [~] suffix qualifies its own array layer. ? requires a reference. +type + : functionType + | primitiveType typeSuffix* + | accessModifier? typeName typeSuffix* + ; +accessModifier : '!' | '~' ; +typeSuffix : '?' | '[]' | '[~]' ; +typeName : IDENTIFIER ('.' IDENTIFIER)* ; + +primitiveType + : 'int' | 'i8' | 'i16' | 'i32' | 'i64' + | 'u1' | 'u8' | 'u16' | 'u32' | 'byte' | 'bool' + | 'flt' | 'f32' | 'f64' + ; + +assignment : expression (assignmentOperator assignment)? ; +assignmentOperator + : '=' | '+=' | '-=' | '*=' | '/=' | '&=' | '|=' | '^=' + | '<<=' | '>>=' | '>>>=' + ; + +expression : logicalExpression ('?' assignment (':' assignment)?)? ; + +// Both logical operators short-circuit and associate to the right here. +logicalExpression : bitwiseExpression (('&&' | '||') logicalExpression)? ; + +bitwiseExpression + : equalityExpression (('&' | '|' | '^') equalityExpression)* + ; + +equalityExpression : comparisonExpression (('==' | '!=') comparisonExpression)* ; + +// Chained comparisons test adjacent operands and short-circuit. A chain must +// keep its direction; equality has lower precedence and does not join a chain. +comparisonExpression + : shiftExpression + ( (('<' | '<=') shiftExpression)+ | (('>' | '>=') shiftExpression)+ )? + ; + +shiftExpression + : additiveExpression (('<<' | '>>' | '>>>') additiveExpression)* + ; + +additiveExpression + : multiplicativeExpression (('+' | '-') multiplicativeExpression)* + ; + +multiplicativeExpression + : unaryExpression (('*' | '/') unaryExpression)* + ; + +unaryExpression + : ('-' | '!') unaryExpression + | updateOperator IDENTIFIER + | postfixExpression + ; + +// Prefix updates require a local name; postfix updates also allow fields +// and array elements. All updates require an assignable operand. +updateOperator : '++' | '--' ; +postfixExpression : primaryExpression postfixSuffix* updateOperator? ; +postfixSuffix : '.' IDENTIFIER | '[' assignment ']' | '#' | callSuffix | methodCallSuffix ; + +// For a fixed function-valued member, object.name(args) passes object as self. +// This overlaps ordinary field selection followed by a call syntactically; +// name/type resolution determines which call form applies. +methodCallSuffix : '.' IDENTIFIER callSuffix ; + +primaryExpression + : INTEGER_LITERAL + | FLOAT_LITERAL + | STRING_LITERAL + | CHARACTER_LITERAL + | 'true' + | 'false' + | 'null' + | IDENTIFIER + | '(' assignment ')' + | newExpression + | functionExpression + ; + +// A function is an expression bound with an ordinary declaration. +// Its last statement supplies the implicit result; return can exit earlier. +functionExpression : '{' parameterList? '->' statement* '}' ; +parameterList : (parameter ','?)+ ; +parameter : type bindingModifier? IDENTIFIER ; + +// Function types have unnamed, space-separated argument types. +// {int} is the zero-argument type; its function expression still uses ->. +functionType + : '{' type+ '->' type '}' '?'? + | '{' type '}' '?'? + ; + +callSuffix : '(' argumentList? ')' ; +argumentList : assignment (',' assignment)* ','? ; + +// Allocation invokes the declared constructor; inline initializer blocks from +// earlier chapters are replaced by constructor arguments. +newExpression : 'new' type ('[' assignment ']' | callSuffix)? ; + +INTEGER_LITERAL : '0' | [1-9] DIGIT* ; +FLOAT_LITERAL : DIGIT+ ('.' DIGIT* ('e' DIGIT+)? | 'e' '-'? DIGIT+) ; +STRING_LITERAL : '"' (ESCAPE | ~["\\])* '"' ; +CHARACTER_LITERAL : '\'' (ESCAPE | ~['\\]) '\'' ; +fragment ESCAPE : '\\' [0ntr\\'"] ; +IDENTIFIER : NON_DIGIT (NON_DIGIT | DIGIT)* ; +fragment NON_DIGIT : [a-zA-Z_] ; +fragment DIGIT : [0-9] ; +WHITESPACE : [ \t\r\n]+ -> skip ; +LINE_COMMENT : '//' ~[\r\n]* -> skip ; +BLOCK_COMMENT : '/*' .*? '*/' -> skip ; +``` diff --git a/docs/chapter28b/README.ja.md b/docs/chapter28b/README.ja.md new file mode 100644 index 00000000..8f9c5f5d --- /dev/null +++ b/docs/chapter28b/README.ja.md @@ -0,0 +1,146 @@ +# 第28b章: 分離コンパイル + +[English](README.md) | 日本語 + +[Previous: Chapter 28a](../chapter28a/README.ja.md) + +前の章では、見つかったソースを一度にコンパイルしました。この章ではライブラリと +クライアントを別々にコンパイルします。クライアントにはライブラリのオブジェクト +ファイルが必要ですが、そのソースは不要です。 + +## オブジェクトに理想 IR を保存する + +通常の ELF オブジェクトには機械語、データ、シンボル、再配置情報があります。 +Simple はさらに `.simple` セクションへ型と理想 IR を保存します。クライアントは +宣言を読み込んで型を検査し、関数本体も読み込んで SCCP やインライン化に使います。 +残った呼び出しは、ネイティブリンクでライブラリの機械語につなぎます。 + +宣言型と解析中の型は別々に保存します。ある時点でフィールドの値が 0 でも、 +その宣言を定数 0 に変えてはいけません。引数の型の上限、数値演算のモード、 +Store の幅、元のソース位置も保存します。入れ子の構造体には、関数本体だけでなく +割り当て関数とコンストラクタの宣言も必要です。 + +別々のコンパイルで同じ番号になったエイリアスや関数は、同じものとは限りません。 +`GlobalBits` は定義元のファイルとファイル内の番号を保存し、インポート時にその +組み合わせを今回の番号へ写します。型とノードに同じ写像を使い、複数ファイルを +またぐ再帰型も復元します。 + +コンパイル単位ごとに `sys.o`、`sys/io.o`、`sys/libc.o` などを出力します。宣言と関数の +要約は本体より先に保存し、IR を復元せずに読めます。未ロードの関数は `FunNode`、 +`SummaryNode`、`ReturnNode` で表し、戻り値・メモリ・エスケープ情報を解析に使います。 +要約そのものはインライン化しません。ワークリストが空になった時点で、保存したサイズを +参考に本体を要求し、読み込んで解析してから実際のインライン化を判断します。 + +`sys.io.p` を通常の呼び出しとして残す場合、`io` の IR は不要です。`io.o` の本体を +読み込んでも、その中の外部参照には型があり、別の本体を自動的に要求しません。 +メタデータの探索は保守的で、要求したオブジェクト内の関数本体はまとめて読み込みます。 +さらに細かい関数単位の読み込みは今後の目標です。 + +要約には直接呼び出しとクラス初期化の依存辺も保存します。循環の可能性が見つかったら、 +残りの本体をすべて読み込み、最適化後に再検査してからエラーを報告します。循環の外の +ヘルパーが返す定数によって、循環内の辺が消える場合もあるためです。 + +コンパイル直後のネイティブリンクは CompUnit に保存されたオブジェクトのパスを使います。 +コンパイラの状態を持たず保存済みオブジェクトだけをリンクするテストでは、必要な +オブジェクトをすべて明示的に列挙します。必要なファイルが欠けていればリンクに失敗します。 +ネイティブ実行では実行時の動的ロードは行いません。 +Eval2 も実行前の解析時に依存オブジェクトを解決します。型検査済みプログラムの +オブジェクトは、互いに整合するビルドから提供されることを前提とします。 +実行しない分岐からの依存でも、必要なオブジェクトがなければ失敗します。 +Eval2 は型検査済みのプログラムを実行し、呼び出しで必要になったオブジェクトの IR だけを +復元して個別にスケジュールします。これはリンク済みプログラムの準備を遅らせる仕組みで、 +実行中に新しいライブラリを探索するものではありません。実行中のグラフの再最適化は行いません。 +理想ノードの CLInit を直接実行します。子の `<clinit>` は親の初期化を確認してから、 +自分の `$init` フラグを設定し、本体を実行します。Eval2 と Lower 後のコードでこの順序を +共有し、呼び出し側は対象クラスのフラグだけを検査します。 +IR の読み込みだけではクラスの初期化や本体の出力は起こりません。 + +## 実行できる例 + +英語版の例は次の二つのソースルートにあります。 + +```text +src/test/smp/separate/library/Counters.smp +src/test/smp/separate/client/Main.smp +``` + +ライブラリは、ループで final フィールドを計算する `Counter` と共有カウンタを +公開します。クライアントの `run(3)` は最初に 107 を返します。同じプロセスで +続けて `run(5)` を呼ぶと 231 になります。インスタンスは別々ですが、クラスの +カウンタは共有されます。`run(0)` はライブラリを使わずに戻ります。 + +このディレクトリでライブラリをコンパイルします。 + +```sh +make lib +make build/classes/test/.ttag +java -ea -cp "build/classes/main:build/classes/test:lib/*" \ + com.seaofnodes.simple.Simple --norun -o build/separate-example/lib/ \ + src/test/smp/separate/library/Counters.smp +``` + +次にオブジェクトを使ってクライアントを評価します。 + +```sh +java -ea -cp "build/classes/main:build/classes/test:lib/*" \ + com.seaofnodes.simple.Simple --eval \ + -L build/separate-example/lib/Counters.o \ + src/test/smp/separate/client/Main.smp 3 +``` + +結果は `107` です。Windows のクラスパス区切りには `;` を使います。クライアントの +ソースルートには `Counters.smp` がありません。`SeparateCompilationTest` はさらに +二つのオブジェクトを C ドライバとリンクし、繰り返し呼び出す動作を検査します。 + +## 解析とコードの所有者 + +インポートした本体は解析に参加しても、クライアントの出力には属しません。 +`Serialize` は `Lower` がクラス初期化を展開する前に理想 IR を保存します。 +`Unlink` は機械語生成に向けてインポートした本体と解析用の呼び出し辺を切り離します。 +インライン化でクライアントへコピーされたコードは、クライアント自身の本体として +出力されます。 + +初期化規則は前の章と同じです。型のインポートだけでは初期化しません。能動的な使用は +`CLInit` を通して初期化を要求し、ライブラリが持つクラスオブジェクトとフラグを使います。 +依存関係の検査にはインポートした本体も含めます。ネイティブコードへ渡した関数の +未知の呼び出し元も保持します。 + +コンストラクタのフィールド更新はローカル値と Phi のままで、ヘルパーの出口に Store を +置きます。未完成のオブジェクトでも初期化済みフィールドは読めます。 +アンダースコアは公開インターフェースの選択に使いますが、ソースのアクセス制限は +まだ実装されていません。 + +## 検証と今後 + +明示的に指定したソースは常にコンパイルします。他のローカルオブジェクトを再利用する前に、 +記録された依存関係をたどり、各オブジェクトのソースを検査します。 +入力の欠落や、より新しいソース・依存オブジェクトがあれば、保存した型や IR を使わず +再解析します。A が B をインライン化していれば、B のオブジェクトが未更新でも、 +B のソース変更で A は無効になります。循環は訪問済み集合で処理します。 + +ライブラリは検査と読み込みで同じ検索パスを使います。ソースのないライブラリは整合した +組み合わせで提供する必要があります。時刻だけでは互換性は判断できません。入力ハッシュ、 +コンパイラ・ABI のバージョン、自動再リンクを含むビルドドライバは今後の課題です。 + +リポジトリのルートから次を実行します。 + +```sh +make tests CHAPTERS=chapter28b +``` + +保存したコンストラクタ、インポートをまたぐ初期化循環、共有データ、ネイティブ関数の +サンク、ライブラリとクライアントの実行を検査します。Make はテスト用の `sys` も +オブジェクトにします。`make release` は jar と標準ライブラリを作り、`make view` は +共通のグラフビューアーを起動します。 + +`IncrementalBuildTest` は時刻を明示的に設定し、再利用、間接的な変更、オブジェクト間の +循環、入力の欠落、ライブラリ更新を検査します。変更後のネイティブ実行結果も +クリーンビルドと比較します。 +`SelectiveLoadingTest` は要約による呼び出し、インライン化のためのロード、静的リンク、 +初期化循環の再検査を確認します。 + +保存形式は `C0DE` 以外のバージョン番号をまだ持ちません。コンパイラや形式を変えたら +オブジェクトを再構築してください。高コストの往復一致検査は無効のままで、古い増分 +ビルドの実験 `testModule0` も実行対象外です。 + +ネイティブライブラリの構築と最適化の調整は[付録](../appendices/README.md)にまとめています。 diff --git a/docs/chapter28b/README.md b/docs/chapter28b/README.md new file mode 100644 index 00000000..921d99ee --- /dev/null +++ b/docs/chapter28b/README.md @@ -0,0 +1,233 @@ +# Chapter 28b: Separate Compilation + +English | [日本語](README.ja.md) + +[Previous: Chapter 28a](../chapter28a/README.md) + +The previous chapter compiled every discovered source file together. This +chapter lets a program and the libraries it uses compile separately, with +full type information. To compile a program that uses a library, we need the +library's *object* file, but not its source. + +In spirit this is similar to Java's *class* file, with its full execution and +type semantics stored in the file; we happen to be using an ELF file for the +same reason. The ahead-of-time compiled code is *also* available for the +linking step. + +An ordinary native object contains machine code, data, symbols, and +relocations. Simple adds a `.simple` section with types and the ideal IR. +Importing `.simple` lets the compiler check types and optimize calls into the +library. Calls that remain out of line use the library's machine code at +native link time. + +## A program that uses a library + +The runnable example has two separate source roots: + +```text +src/test/smp/separate/library/Counters.smp +src/test/smp/separate/client/Main.smp +``` + +`Counters.smp` declares shared state and an instance constructor: + +```simple +int calls=0; +val next_noInline = { -> calls++; return calls; }; + +struct Counter { + val value=1; + new Counter = { int n -> + value=0; + for( int i=0; i<n; i++ ) value=value*2+1; + }; + val get = { -> value; }; +}; +``` + +`Main.smp` uses both parts of the library: + +```simple +val run = { int n -> + if( !n ) return 0; + val c = new Counters.Counter(n); + return c.get()+100*Counters.next_noInline(); +}; +return run(arg); +``` + +At runtime, `run(0)` returns without reaching or initializing `Counters`. + +For `run(3)`, `Counters` initializes, the constructor computes 7 and +`calls`==1, so we get `(7+100*1)` or 107. + +Calling `run(3)`, then `run(5)`, in one process increments the shared `calls` +twice. The first call gives `(7+100*1)`, or 107; the second gives +`(31+100*2)`, or 231. + +(The `_noInline` suffix prevents inlining, so the example exercises native +linking as well as imported IR.) + +From this chapter directory, build the compiler and compile the library: + +```sh +make lib +make build/classes/test/.ttag +java -ea -cp "build/classes/main:build/classes/test:lib/*" \ + com.seaofnodes.simple.Simple --norun -o build/separate-example/lib/ \ + src/test/smp/separate/library/Counters.smp +``` + +Then compile and evaluate the program using that object: + +```sh +java -ea -cp "build/classes/main:build/classes/test:lib/*" \ + com.seaofnodes.simple.Simple --eval \ + -L build/separate-example/lib/Counters.o \ + src/test/smp/separate/client/Main.smp 3 +``` + + +## What crosses the separate compilation boundary + +Each source compilation unit gets its own object file: compiling `sys.smp` +produces `sys.o`, `sys/io.o`, `sys/libc.o`, and any other discovered classes. +Mutually dependent sources are compiled together and they output to separate +files with a common build timestamp. + +The object file has a compact encoding of the optimized IR in the `.simple` +section. Declarations and function summaries come first so they can be read +without reading all of the IR. The saved state includes canonical declarations +as well as optimized flow types. E.g. `int x=7` has declared type `int` and +initial value `7`. Struct field types (both declared and optimized), function +argument types (both declared and optimized), numeric modes, store widths, and +source locations also survive serialization. + +Each loaded `.simple` section restores `CompUnit`s into the shared tree, +mirroring the original code layout. A dependency shared by several objects +uses the same `CompUnit`. Loaded aliases and function indices are renumbered +to avoid collisions; all the dense indices end up in the same namespace. +`GlobalBits` has the mapping from the imported file indices to the local +compilation indices; this remapping extends everywhere through the loaded code, +including recursive types that span several files. Importing skips parsing the +original source, but loaded IR is still analyzed within its new context. + +`ParseAll` is the workhorse that discovers source and object dependencies. It +tracks missing symbol references, looking them up in the search path one by one, +and loading the matching files. + +The end result combines locally parsed and imported IR in one graph, while +keeping their separate `CompUnit`s. This combined compilation can take +advantage of e.g. constants in the library, or inline library code using the +normal inlining heuristics. The combined graph goes through the normal +compilation phases, including peepholes, Opto and TypeCheck. + +Dependencies between `CompUnit`s are recorded. An explicitly requested source +file is always compiled. Before reusing another local object, `CompUnit` walks +its recorded dependencies and checks each local object's source. +A missing input or a newer source or dependency object forces parsing +again, before saved types or IR are imported. This includes transitive +dependencies: if A has inlined B, editing B's source invalidates A even while +B's object is unchanged. A visited set lets this check follow cycles. + +Library dependencies use the same search paths for checking and loading. +Object-only libraries must be supplied as a coherent build; timestamps alone +cannot establish their compatibility. These checks are not a complete build +driver: explicit requests still compile, and input hashes, compiler/ABI +versions, and automatic relinking remain future work. + +An unloaded function has a `FunNode`, `SummaryNode`, and `ReturnNode`. Its +summary supplies result, memory, and escape information; calls keep the usual +graph structure without requiring the body. Summaries cannot be inlined. +At an empty optimization worklist, an interesting local call can request its +callee's object using saved size information. The compiler loads the bodies, +reconciles identities, and analyzes them before deciding whether to inline. + +For `sys.io.p`, the compiler can leave `p` out of line without loading `io` IR. +Loading `io.o` makes its functions available, but references inside them carry +types without automatically loading other bodies. An unused imported function +does not cause speculative loading of its callees. Metadata discovery remains +conservative, and a requested object still loads all of its function bodies. + + + +## Using library code and shared state + +The compiler can use library IR to optimize a program while leaving the +library's functions and shared data in their own object files. Separate +compilation must also preserve when each class is initialized, even when +several files use it. + +For example, compiling `helloWorld` may load the body of `sys.io.p` to decide +whether to inline it or not. Inlined code becomes part of `helloWorld.o`. +Ifif it was not inlined that call remains using the function in `sys/io.o`. +Loading did not add another stand-alone copy of `p` to `helloWorld.o`. + +In any case, loaded or not, inlined or not, the initialization semantics of +`sys.io` is unchanged; this means we might need CLInit checks in the inlined +code, or where we load `p` from `sys.io`. + +If we did load a bunch of IR, but then chose not to inline, that imported code +is still in our compiled graph. We use `Unlink` before code-gen to remove the +imported bodies. The not-inlined call sites will keep the target names around +for the system linker to eventually connect all the code. + +Shared state follows the same idea as function calls. In the earlier example, +all calls within the program use the same `Counters.calls` field. The library +also supplies a shared `$init` flag so its class initializer runs once. +Importing types or loading IR does not run that initializer: execution must +reach a use that needs the class. + +A nested class's initializer first ensures that its enclosing class is +initialized, then sets its own flag and runs its body. + +The IR represents an initialization check with a `CLInit` node. Simple saves +this form in the object file so the compiler can analyze it when compiling +a program that uses the library. For machine code, `Lower` turns the check +into a flag test and a conditional call; Eval2 performs the same steps +directly. Optimization can remove a check when the class is +already known to be initialized. + +The compiler must also reject cycles in which class initializers depend on one +another. Simple keeps *function summaries* that list calls and classes that +may need initialization, so the compiler can check for cycles before loading +bodies. If a possible cycle appears, it loads all remaining bodies, optimizes, +and checks again. A constant returned by a some other code outside the cycle +can make a branch disappear and remove the cycle. + +All required object files must be found before execution. Native code is +linked before the program starts. Eval2 can wait until a call needs a body +to decode its IR, but the object's types and dependencies have already been +read. These choices change when code is prepared; both execution paths use +the same shared state and class-initialization rules. + + +## Building and checking + +From the repository root: + +```sh +make tests CHAPTERS=chapter28b +``` + +The suite includes the preceding constructor and escape regressions, imported +constructors, initialization cycles across objects, shared static data, native +function thunks, and native programs linked with libraries. Make also builds +the modular `sys` objects used by the library tests. Programs can import +`sys.o` with `-L`, without parsing the standard library's source again. +`IncrementalBuildTest` checks reuse, transitive edits, cycles across objects, missing +inputs, and changed library objects using controlled timestamps. It also +compares native execution after an edit with a clean build. +`SelectiveLoadingTest` checks summarized calls, loading for inlining, native +link dependencies, and refinement of a potential initialization cycle. +`make release` builds the launcher jar and standard library; `make view` +starts the shared graph viewer. + +The serialized format is internal and currently unversioned beyond its `C0DE` +magic. Rebuild Simple objects when the compiler or layout changes. The expensive +serialization bijection check remains disabled, and `testModule0`, the older +incremental-rebuild experiment, remains ignored. They are not validation +guarantees for this snapshot. + +See the appendices on [native libraries](../appendices/native-libraries.md) +and [optimizer/backend tuning](../appendices/optimizer-tuning.md). |