Chapter 25: add separate compilation and simplify SSA construction

Introduce Chapter 25 with separately compiled Simple modules, serialized IR,
cross-compilation-unit linking, and native system-library integration.  This
also completes a broad parser and type-system cleanup needed to make graph
construction independent of incomplete parse-time type information.

Separate compilation and linking

- Add compilation-unit Start/Stop nodes and track functions, classes, aliases,
  and global indices across compilation-unit boundaries.
- Serialize canonical types and ideal graph nodes into ELF objects, including
  dependency metadata, relocations, global/local bitset remapping, and dense
  memory-alias information.
- Load dependent objects on demand, recursively resolve source and object
  paths, relink calls and function pointers, and preserve external C symbols.
- Distinguish the post-optimization ownership hook from unknown callers so
  private call graphs can remain precise while public functions remain
  externally callable.
- Add command-line and build support for compiling modules and the reusable
  `sys` object library.

Parser, constructors, and SSA

- Replace inline constructor expansion with a hidden default `<init>` followed
  by an explicit user constructor declared as `new Type = { ... }`.
- Keep the default constructor private, pass private object memory through
  constructor calls, and require user constructors to initialize non-default
  fields before the object escapes.
- Track possibly-uninitialized fields in parser `Var` state and diagnose early
  reads without encoding parser-only constraints in Load and Store nodes.
- Make ordinary parser memory bulk and recover precise alias slices in the
  optimizer using `MemMerge`, `MemPhi`, and `BulkMemPhi` nodes.
- Remove parser-supplied type constraints from Loads and ordinary Phis, defer
  integer/float arithmetic selection, and keep call shape driven by syntax
  instead of provisional inferred types.
- Preserve chained field declarations as static parser information when flow
  types are not yet available, including postfix field updates through calls.

Types, functions, and escape analysis

- Add `TypeScalar` as the common lattice family for integer, float, memory
  pointer, and function-pointer values, with scalar TOP/BOT values for generic
  Loads, Stores, parameters, and invalid intermediate expressions.
- Represent unresolved struct definitions directly with the `TypeStruct`
  forward-reference flag while retaining the separate open-structure concept;
  close and canonicalize recursive structures without side maps.
- Add explicit `FunPtrNode` references so constant function pointers retain a
  graph edge to their Return, and keep referenced functions alive until Opto
  proves the pointer dead.
- Refine public-function and class-initializer escape handling, including
  public-only fallback escape sets and monotone compilation-unit summaries.
- Add guards, proof casts, read-only propagation, pointer-to-integer conversion
  for non-null FFI arguments, and consistent deep mutability checks at calls.

Optimization and code generation

- Make weak and unresolved graph inputs compute conservatively and monotonically
  throughout iterative peepholes and SCCP.
- Preserve independent memory slices when sharpening Stores, retain control on
  conditional Stores, and freeze Store widths from their target field types.
- Extend serialization for new node kinds, scalar types, Store modes, function
  pointers, and memory partitions, preserving serialize/deserialize bijection.
- Complete native selection and encoding support across x86-64, RISC-V, and
  ARM, including FFI pointer conversion and missing integer operations.
- Fix x86 immediate multiply REX encoding and assorted scheduling, register
  allocation, spilling, relocation, and constant-pool issues exposed by the
  larger linked graphs.

System library, tests, and documentation

- Compile and link the Simple `sys` library, including libc bindings, I/O,
  scanners, strings, vectors, array helpers, and executable entry points.
- Add Chapter 25 module, serialization, constructor, forward-reference,
  postfix-update, system, Hello World, and Bubble Sort coverage; update golden
  graphs and spill expectations for the new IR shape.
- Add module and type-system documentation, parser-simplification design notes,
  grammar updates, examples, diagrams, and graph visualization assets.

The resulting compiler passes the raw Chapter 1-24 suite and the Chapter 25
system and separate-compilation tests, including a freshly rebuilt `sys.o` and
the native Bubble Sort example.

1 files changed, 232 insertions, 0 deletions

diff --git a/chapter25/Makefile b/chapter25/Makefile
new file mode 100644
index 00000000..7497f98f
--- /dev/null
+++ b/chapter25/Makefile
@@ -0,0 +1,232 @@
+#
+# Makefile expects to run at the project root, and not nested inside anywhere
+#
+#  cd $DESK/Simple/chapterXX; make
+#
+#
+# first time only - this is the only thing that hits the Interwebs
+#
+#   make lib
+#
+# any time you want to nuke everything from space:
+#
+#   make clean tags
+#
+# run the tests
+#
+#   make tests
+
+#######################################################
+#
+# Boilerplate because make syntax isn't the best
+#
+
+# Use bash for recipes
+SHELL := /bin/bash
+
+# Remove target on error (no broken leftover files)
+.DELETE_ON_ERROR:
+
+# Keep partial builds but not partial recipes; keeping them speeds up the next `make`
+.NOTINTERMEDIATE:
+
+# for printing variable values
+# usage: make print-VARIABLE
+#        > VARIABLE = value_of_variable
+print-%  : ; @echo $* = $($*)
+
+# literal space
+space := $() $()
+
+# Decide OS-specific questions
+# jar-file seperator
+ifeq ($(OS),Windows_NT)
+	SEP = ;
+else
+	SEP = :
+endif
+# Find a reasonable ctags.
+CTAGS = $(shell which ctags)
+# Hack for MacOS: /usr/bin/ctags is unfriendly, so look for ctags from brew
+ifeq ($(UNAME),Darwin)
+	CTAGS = $(shell brew list ctags 2> /dev/null | grep bin/ctags)
+endif
+
+# Fun Args to javac.  Mostly limit to java8 source definitions, and fairly
+# aggressive lint warnings.
+JAVAC_ARGS = -g
+
+# Source code
+SIMPLE := com/seaofnodes/simple
+SRC := src/main/java
+TST := src/test/java
+CLZDIR:= build/classes
+main_javas   := $(wildcard $(SRC)/$(SIMPLE)/*java $(SRC)/$(SIMPLE)/*/*java $(SRC)/$(SIMPLE)/node/cpus/*/*java)
+test_javas   := $(wildcard $(TST)/$(SIMPLE)/*java $(TST)/$(SIMPLE)/*/*java)
+main_classes := $(patsubst $(SRC)/%java,$(CLZDIR)/main/%class,$(main_javas))
+test_classes := $(patsubst $(TST)/%java,$(CLZDIR)/test/%class,$(test_javas))
+# These tests do not rely on the stdlib
+test_alone := com.seaofnodes.simple.BrainFuckTest com.seaofnodes.simple.MergeSortTest com.seaofnodes.simple.TypeTest
+test_raw_nums0 := 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21
+test_raw_nums1 := 22 23 24
+# These tests DO rely on the stdlib
+test_raw_sys   := 25
+
+# Convert chapter numbers to test files
+test_raw0    := $(patsubst %,com.seaofnodes.simple.Chapter%Test,$(test_raw_nums0))
+test_raw1    := $(patsubst %,com.seaofnodes.simple.Chapter%Test,$(test_raw_nums1))
+test_sys     := $(patsubst %,com.seaofnodes.simple.Chapter%Test,$(test_raw_sys)  )
+# All the 'src/test/java' files ending in 'Test'
+test_cp0     := $(patsubst $(TST)/$(SIMPLE)/%.java,com.seaofnodes.simple.%,$(wildcard $(TST)/$(SIMPLE)/*Test.java))
+# Minus raw0, raw1, alone tests
+test_cp1     := $(filter-out $(test_raw0) ,$(test_cp0))
+test_cp2     := $(filter-out $(test_raw1) ,$(test_cp1))
+test_cp3     := $(filter-out $(test_alone),$(test_cp2))
+test_cp4     := $(filter-out com.seaofnodes.simple.Chapter25Test,$(test_cp3))
+test_cp      := $(filter-out $(test_sys)  ,$(test_cp4))
+# All the classes to compile
+classes = $(main_classes) $(test_classes)
+# All the libraries
+libs = $(wildcard lib/*jar)
+jars = $(subst $(space),$(SEP),$(libs))
+
+
+default_targets := TAGS release tests
+# Optionally add ctags to the default target if a reasonable one was found.
+ifneq ($(CTAGS),)
+default_targets += tags
+endif
+
+default: $(default_targets)
+
+#######################################################
+# Compile Java as a whole source set.  javac tracks Java dependencies
+# internally, so compiling only the directly out-of-date .java files can leave
+# stale dependent .class files behind.
+$(CLZDIR)/main/.mtag: $(main_javas)
+	@[ -d $(CLZDIR)/main ] || mkdir -p $(CLZDIR)/main
+	@# This crap is really just "javac", but with a beautiful error message.
+	@# The very very long list of java files is suppressed and counted.
+	@# The required output "Note: blah blah deprecated blah" is suppressed.
+	$(file > .argsM.txt, $(main_javas))
+	@echo "compiling main " `wc -w < .argsM.txt` " files"
+	@if ! javac $(JAVAC_ARGS) -cp "$(CLZDIR)/main$(SEP)$(jars)" -sourcepath $(SRC) -d $(CLZDIR)/main @.argsM.txt >& .out.txt ; then \
+            cat .out.txt ; \
+            exit 1; \
+	fi
+	@rm -rf .out.txt
+	@touch $(CLZDIR)/main/.mtag
+	@rm -f .argsM.txt
+
+
+#######################################################
+
+$(CLZDIR)/test/.ttag: $(test_javas) $(CLZDIR)/main/.mtag
+	@[ -d $(CLZDIR)/test ] || mkdir -p $(CLZDIR)/test
+	$(file > .argsT.txt, $(test_javas))
+	@echo "compiling test " `wc -w < .argsT.txt` " files"
+	@if ! javac $(JAVAC_ARGS) -cp "$(CLZDIR)/test$(SEP)$(CLZDIR)/main$(SEP)$(jars)" -sourcepath $(TST) -d $(CLZDIR)/test @.argsT.txt >& .out.txt ; then \
+            cat .out.txt ; \
+            exit 1; \
+	fi
+	@rm -rf .out.txt
+	@touch $(CLZDIR)/test/.ttag
+	@rm -f .argsT.txt
+
+#######################################################
+
+# Default Simple .o target is build/objs which is where all the test .o's get
+# dumped.  The default sys search needs to NOT include all the dumped test
+# files; move it down into lib/
+OBJS:= build/objs
+SYS_SMPS:= $(wildcard src/main/smp/*.smp src/main/smp/*/*.smp)
+CPU_PORT:= x86_64_v2
+CALL_CONV:= win64
+CPU_ABI:= $(CPU_PORT)_$(CALL_CONV)
+RELEASE:= build/release
+RELEASE_SYS:= $(RELEASE)/$(CPU_ABI)/sys.o
+
+$(OBJS)/lib_$(CPU_ABI)/sys.o:	$(CLZDIR)/main/.mtag $(SYS_SMPS)
+	@echo "simping " $@
+	@$(JVM) com.seaofnodes.simple.Simple --cpu $(CPU_PORT) --abi $(CALL_CONV) --norun -o $@ src/main/smp/sys.smp
+
+$(RELEASE_SYS):	$(OBJS)/lib_$(CPU_ABI)/sys.o
+	@echo "copying " $@
+	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
+	@cp $< $@
+
+#######################################################
+# Base launch line for JVM tests
+JVM=nice java -ea -cp "$(CLZDIR)/main${SEP}${jars}${SEP}$(CLZDIR)/test"
+
+# Tests that do NOT need sys.smp
+tests_raw0:	$(CLZDIR)/main/.mtag $(CLZDIR)/test/.ttag
+	@echo "testing " $(test_raw0)
+	@[ -d $(OBJS) ] || mkdir -p $(OBJS)
+	@$(JVM) org.junit.runner.JUnitCore $(test_raw0)
+
+tests_raw1:	$(CLZDIR)/main/.mtag $(CLZDIR)/test/.ttag
+	@echo "testing " $(test_raw1)
+	@[ -d $(OBJS) ] || mkdir -p $(OBJS)
+	@$(JVM) org.junit.runner.JUnitCore $(test_raw1)
+
+# Larger standalone tests, still JUnits, still no sys.smp
+tests_alone:	$(CLZDIR)/main/.mtag $(CLZDIR)/test/.ttag
+	@echo "testing " $(test_alone)
+	@[ -d $(OBJS) ] || mkdir -p $(OBJS)
+	@$(JVM) org.junit.runner.JUnitCore $(test_alone)
+
+# Tests that DO need sys.smp
+tests_sys:	$(CLZDIR)/main/.mtag $(CLZDIR)/test/.ttag $(RELEASE_SYS)
+	@echo "testing " $(test_sys)
+	@[ -d $(OBJS) ] || mkdir -p $(OBJS)
+	@$(JVM) org.junit.runner.JUnitCore $(test_sys)
+
+fuzzer: $(CLZDIR)/main/.mtag $(CLZDIR)/test/.ttag
+	@echo "fuzzing "
+	@$(JVM) org.junit.runner.JUnitCore com.seaofnodes.simple.FuzzerWrap
+
+tests:	$(CLZDIR)/main/.mtag $(CLZDIR)/test/.ttag $(RELEASE_SYS) tests_raw0 tests_raw1 tests_alone tests_sys
+	@echo "All tests"
+	@[ -d $(OBJS) ] || mkdir -p $(OBJS)
+	@$(JVM) org.junit.runner.JUnitCore $(test_cp)
+
+
+# Build a Simple jar and release stdlib
+release:	build/release/Simple.jar $(RELEASE_SYS)
+
+# Build a Simple jar
+build/release/Simple.jar:	$(CLZDIR)/main/.mtag $(CLZDIR)/test/.ttag
+	@echo "jarring " $@ " because " $?
+	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
+	@printf "Main-Class: com.seaofnodes.simple.Simple\nClass-Path: ../../lib/junit-4.12.jar ../../lib/hamcrest-core-1.3.jar\n\n" > build/release/MANIFEST.MF
+	@jar cfm $@ build/release/MANIFEST.MF -C $(CLZDIR)/main . -C $(CLZDIR)/test . -C $(SRC)/$(SIMPLE) . -C $(TST)/$(SIMPLE) .
+
+# Launch viewer
+view:	$(main_classes)
+	@echo "viewing "
+	@$(JVM) com.seaofnodes.simple.JSViewer
+
+#######################################################
+.PHONY: clean
+clean:
+	rm -rf build
+	rm -f TAGS
+	(find . -name "*~" -exec rm {} \; 2>/dev/null; exit 0)
+
+# Download libs from maven
+lib:	lib/junit-4.12.jar  lib/hamcrest-core-1.3.jar
+
+# Jars
+lib/junit-4.12.jar:
+	@[ -d lib ] || mkdir -p lib
+	@(cd lib; wget https://repo1.maven.org/maven2/junit/junit/4.12/junit-4.12.jar)
+
+lib/hamcrest-core-1.3.jar:
+	@[ -d lib ] || mkdir -p lib
+	@(cd lib; wget https://repo1.maven.org/maven2/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar)
+
+# Build emacs tags (part of a tasty emacs ide experience)
+TAGS:	$(main_javas) $(test_javas)
+	@rm -f TAGS
+	@$(CTAGS) -e --recurse=yes --fields=+fksaiS $(SRC) $(TST)