From 51f1f8f4b871e3b52eb0c5d18b03f362a2b37791 Mon Sep 17 00:00:00 2001 From: Cliff Click Date: Mon, 10 Aug 2026 11:11:18 -0700 Subject: Chapter 25: add separate compilation and simplify SSA construction Introduce Chapter 25 with separately compiled Simple modules, serialized IR, cross-compilation-unit linking, and native system-library integration. This also completes a broad parser and type-system cleanup needed to make graph construction independent of incomplete parse-time type information. Separate compilation and linking - Add compilation-unit Start/Stop nodes and track functions, classes, aliases, and global indices across compilation-unit boundaries. - Serialize canonical types and ideal graph nodes into ELF objects, including dependency metadata, relocations, global/local bitset remapping, and dense memory-alias information. - Load dependent objects on demand, recursively resolve source and object paths, relink calls and function pointers, and preserve external C symbols. - Distinguish the post-optimization ownership hook from unknown callers so private call graphs can remain precise while public functions remain externally callable. - Add command-line and build support for compiling modules and the reusable `sys` object library. Parser, constructors, and SSA - Replace inline constructor expansion with a hidden default `` followed by an explicit user constructor declared as `new Type = { ... }`. - Keep the default constructor private, pass private object memory through constructor calls, and require user constructors to initialize non-default fields before the object escapes. - Track possibly-uninitialized fields in parser `Var` state and diagnose early reads without encoding parser-only constraints in Load and Store nodes. - Make ordinary parser memory bulk and recover precise alias slices in the optimizer using `MemMerge`, `MemPhi`, and `BulkMemPhi` nodes. - Remove parser-supplied type constraints from Loads and ordinary Phis, defer integer/float arithmetic selection, and keep call shape driven by syntax instead of provisional inferred types. - Preserve chained field declarations as static parser information when flow types are not yet available, including postfix field updates through calls. Types, functions, and escape analysis - Add `TypeScalar` as the common lattice family for integer, float, memory pointer, and function-pointer values, with scalar TOP/BOT values for generic Loads, Stores, parameters, and invalid intermediate expressions. - Represent unresolved struct definitions directly with the `TypeStruct` forward-reference flag while retaining the separate open-structure concept; close and canonicalize recursive structures without side maps. - Add explicit `FunPtrNode` references so constant function pointers retain a graph edge to their Return, and keep referenced functions alive until Opto proves the pointer dead. - Refine public-function and class-initializer escape handling, including public-only fallback escape sets and monotone compilation-unit summaries. - Add guards, proof casts, read-only propagation, pointer-to-integer conversion for non-null FFI arguments, and consistent deep mutability checks at calls. Optimization and code generation - Make weak and unresolved graph inputs compute conservatively and monotonically throughout iterative peepholes and SCCP. - Preserve independent memory slices when sharpening Stores, retain control on conditional Stores, and freeze Store widths from their target field types. - Extend serialization for new node kinds, scalar types, Store modes, function pointers, and memory partitions, preserving serialize/deserialize bijection. - Complete native selection and encoding support across x86-64, RISC-V, and ARM, including FFI pointer conversion and missing integer operations. - Fix x86 immediate multiply REX encoding and assorted scheduling, register allocation, spilling, relocation, and constant-pool issues exposed by the larger linked graphs. System library, tests, and documentation - Compile and link the Simple `sys` library, including libc bindings, I/O, scanners, strings, vectors, array helpers, and executable entry points. - Add Chapter 25 module, serialization, constructor, forward-reference, postfix-update, system, Hello World, and Bubble Sort coverage; update golden graphs and spill expectations for the new IR shape. - Add module and type-system documentation, parser-simplification design notes, grammar updates, examples, diagrams, and graph visualization assets. The resulting compiler passes the raw Chapter 1-24 suite and the Chapter 25 system and separate-compilation tests, including a freshly rebuilt `sys.o` and the native Bubble Sort example. --- chapter25/module.md | 277 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 277 insertions(+) create mode 100644 chapter25/module.md (limited to 'chapter25/module.md') diff --git a/chapter25/module.md b/chapter25/module.md new file mode 100644 index 00000000..d7bed18c --- /dev/null +++ b/chapter25/module.md @@ -0,0 +1,277 @@ + +# Modules for Simple + +Goals: +- 1-liner valid program (for easy tutorial/newbie starts) +- - Compile and run with minimal effort +- Separate compilation and namespaces; external libs +- - Compile to ELF or other linkable/executable format +- Easy private fields & classes +- Output files are compatible with existing .o/.exe/lib/dll infrastructure +- No `#include` or `import` files +- Structure / file system heirarchy per module +- - Perhaps many files make a module +- - easy namespaces within a module +- - Nested names in parent file or nested file +- Class initializers guarenteed: +- - execute exactly once +- - lazily on first touch +- - complete on or before first instances are made +- - - (an exception is allowed for a golden instance in ``) + + +--- +### Goal: Valid 1-liner, minimal effort + +This makes a module and a class `hello` out of the current working directory +name. Compiling without more arguments also links and executes. + +File: `hello.smp` +``` +sys.io.p("Hello World!"); +``` + +Compile and run: +``` +> smp hello.smp +Hello World! +``` + +Module and class name is `hello` extracted from file name, compiled in memory +and `exec'd`. No files made, this is strictly a run and gun mode. + + +--- +### Goal: separate compilation + +Name a build directory target during compilation to get `.o` files. +The build target directory structure will match the source, and +can be placed anywhere. + +As before, since no module name is mentioned on the compile line, `Pet` becomes +both the module and a class. + +File: Pet.smp +``` +// Class initializer is any code outside a struct declaration +// and is guaranteed to run-once on first touch. + +// static fields listed here: +int default_leggies = 4; + +// A default (no argument) is code inside struct with same name as the +// file; this is the same behavior as field initializers for Java, although +// any code is allowed, not just expressions. + +struct Pet { + str name; + int leggies = default_leggies; +} + +float yet_another_static_field; +``` + +Compile to `Pet.o` in the `build/` directory: +``` +> smp Pet.smp -o build/ +> ls build +Pet.o +``` + + +--- +### Goal: nested namespaces + +This makes a module and class `Pet` based on the file name in `myRepo`. It +also makes a nested class `Pet.Vet`. These are used from a different module +`Owner`. + + +Directory: `~/myRepo/` +``` +~/myRepo/ + Pet.smp + build/ + +~/otherRepo/ + Owner.smp +``` + +File: `myRepo/Pet.smp` +``` +int default_leggies = 4; + +struct Pet { + str name; + int leggies = default_leggies; +} + +// Nested class Vet inside of file Pet +struct Vet { + str name; + str address; + long credit_card_num; +} +``` + +File: `otherRepo/Owner.smp` +``` +Pet dog = Pet{name="Fido"}; +Pet snake = Pet{name="slither", leggies = 0; }; +Pet.Vet vet = Pet.Vet{name="Linda"; address="101 Main St"; credit_card_num=1234}; + +sys.io.p("I took "+dog.name+" to "+vet.name+"\n"); +``` + +Compile to `Pet.o` in the `build/` directory; since the compile line does not +mention some other module root, `Pet` must be the module (and is also a class). +Since `Pet` is the module, `Vet` is a class within the module `Pet`. + +``` +> smp Pet.smp -o build/ +> ls build +Pet.o +``` + +The code for both `Pet` and `Pet.Vet` are in the same `Pet.o` file. + +Compile and run `Owner.smp`, specifying the `Pet` repo to satisfy the free +variables `Pet` and `Pet.Vet`. + +``` +> smp Owner.smp -lib ~/myRepo/build +I took Fido to Linda +``` + +--- +### Goal: nested namespace in a nested file + +Same as the last example but with nested files: + +``` +~/myRepo + Pet.smp + Pet/ + Vet.smp + build/ + Pet.o + Pet/ + Vet.o + +~/otherRepo + Owner.smp +``` + + +File: Pet.smp +``` +int default_leggies = 4; + +struct Pet { + str name; + int leggies = default_leggies; +} +``` + +File: Pet/Vet.smp +``` +// Nested class Vet inside of directory Pet +struct Vet { + str name; + str address; + long credit_card_num; +} +``` + +Compile and run `Owner.smp`: +``` +> smp Owner.smp -lib ~/myRepo/build +I took Fido to Linda +``` + + +--- +### Goal: Class Initializers are Lazy + +``` +Parent.smp +Parent/ + ChildA.smp + ChildB.smp +``` + +File: `Parent.smp` +``` +sys.io.p("In Parent "); +sys.io.p(ChildB.str); +``` + +File: `ChildA.smp` +``` +// for ChildA +sys.io.p("In ChildA "); +str str = "A"; +``` + +File: `ChildB.smp` +``` +// for ChildB +sys.io.p("In ChildB "); +str str = "B"; +``` + +Compile and run: +``` +> smp Parent.smp +In Parent +In ChildB +B + +``` + +### Goal: Easy private fields + +Leading underscore makes field or class private. + +### Goal: similar to C compiler driver syntax + +``` +Simple --cpu x86_64_v2 --abi Win64 --norun -o lib/lib/sys_x86_64_v2_win64.o src/main/smp/sys.smp +``` + +Module not mentioned, so top-level file is module: `sys` +Module directory not mentioned, so is prefix of file name `src/main/smp`. +Build dir not mentioned, so is prefix of output file: `lib/` +Output file is default `smp.o`, here renamed. + + + +One can refer to another, and will be executed as first touched. +Parents must be touched before children by design. +However a child A can refer to a child B. +Touching Parent.B means +- Parent is touched first, and begins +- Anything it touches, gets recursively including e.g. A or B +- Parent completes. +- Nothing in a can refer to another *module*. +- Static functions *can* refer to another module, but then can be executed... + +Goal: deterministic within a module. + History: plenty of times I've had complex orders, and then discovered + some referred static-final-fields are not yet init'd when child uses. + SubGoal: deterministic *order*, Parent before Child, etc. + Issue: mid-Parent , refers to Child, so Child before Parent completes. + SubAnswer: Since parent first, and "parent knows child", this is OK. + SubIssue: this does not work across modules; one module is not a parent of another + Issue: Parent fcns called during can non-determistic init child. + "fcn = { -> rand ? A.fld : B.fld }" + Answer?: Parent *may not* direct or indirect refer to a child. + Example: `B b = new B; // Illegal ref to child in Parent.` + Example: `B? b = fcn(); fcn = { -> rand ? null : new B; } // Illegal call to fcn referring to B inside Parent ` + Example: `B? b; fcn = { -> new B; } // OK, no B. during Parent.` + +Goal: external linkage also enjoys the 1-shot init feature. + Answer?: extern call to 'Foo.foo()' first checks Foo. + has run, before calling `Foo.foo()`. Optional: Caller is + modified to bypass check after success. + \ No newline at end of file -- cgit v1.2.3