diff options
Diffstat (limited to 'skills')
| -rw-r--r-- | skills/chapter-split.md | 220 | ||||
| -rw-r--r-- | skills/chapter25-codex-notes.md | 336 | ||||
| -rw-r--r-- | skills/current-spill-matrix.md | 14 | ||||
| -rw-r--r-- | skills/graph-viewer-codex-notes.md | 11 | ||||
| -rw-r--r-- | skills/incremental-builds.md | 76 | ||||
| -rw-r--r-- | skills/japanese-translation.md | 22 | ||||
| -rw-r--r-- | skills/stage-audit.md | 113 |
7 files changed, 741 insertions, 51 deletions
diff --git a/skills/chapter-split.md b/skills/chapter-split.md new file mode 100644 index 00000000..67a974e4 --- /dev/null +++ b/skills/chapter-split.md @@ -0,0 +1,220 @@ +# Approved chapter split + +Cliff approved the seven-part split on 2026-10-19 and the final numbering on +2026-10-24. The old combined Chapters 24 and 25 are retired; their history is +available in Git. Every new directory is a complete compiler snapshot. + +| Previous directory | Final chapter | +|---|---| +| chapter-sccp | 24a: SCCP within a function | +| chapter-interprocedural-sccp | 24b: SCCP across functions | +| chapter-incomplete-types | 25: SSA before types are known | +| chapter-construction | 26: Construction and private memory | +| chapter-escape | 27: Escape analysis and unknown callers | +| chapter-modules | 28a: Modules and lazy class initialization | +| chapter-separate-compilation | 28b: Separate compilation | + +Keep 17a/17b. The other paired chapters extend one idea: 24a/24b goes from +local to interprocedural SCCP; 28a/28b goes from compiling modules together to +compiling them independently. Construction and escape remain separate topics; +neither is presented as requiring the other. + +Inherited Java test names such as `Chapter24Test` and `Chapter25Test` retain +their historical identities across snapshots. They are regression cohorts, +not an index of the current teaching sequence. Historical allocator tables +also retain their original labels and dates. + +Native library construction and optimizer tuning are now [appendices](../docs/appendices/README.md). +The [roadmap](../docs/roadmap.md) owns future work. + +The teaching pattern is to introduce a difficult idea, then show how to make +it useful in a larger setting. Each chapter must remain a complete, runnable +compiler snapshot that preserves the language supported by its predecessor. + +## Teaching sequence + +| Topic | Main idea | Endpoint | +|---|---|---| +| SCCP within a function | Optimistic types, reachable control, Phi inputs, worklists, fixed points, and loop widening. Keep calls conservative at their interfaces. | Fold loop-carried values and eliminate branches that pessimistic iteration cannot resolve. | +| SCCP across functions | Discover call edges during analysis; merge arguments and returns; account for unknown callers and function pointers. | Specialize parameters, resolve call targets, handle recursion, and remove unreachable functions. | +| SSA before types are known | Syntax and lexical binding choose graph shape. Keep scalar types, deferred numeric modes, conversions, guards, incomplete objects, symbolic fields, and removal of parser-supplied Load/Phi bounds together. | Compile forward definitions without provisional types determining graph topology; settle types and layouts afterward. | +| Construction and private memory | Allocation helpers, hidden initializers, user constructors, private self-memory, field-completion checks, and explicit publication. | Preserve initialization and memory facts through constructor calls and publish a completed object. | +| Escape analysis and unknown callers | Discover which aliases and functions can reach code outside the known graph. | Preserve exposed memory and callable functions while specializing private code. | +| Modules and lazy class initialization | Multiple source files, namespaces, privacy, ownership, singleton storage, first-use initialization, parent order, cycle diagnostics, and lowering. | Compile sources together with defined class-initialization behavior. | +| Separate compilation | Preserve types, ideal IR, and identities across independent compilations. | Build a library and client separately, optimize imported bodies, link remaining calls, and check initialization dependencies across imports. | + +The first incomplete-type chapter stays together. Its unresolved memory accesses +must work before construction introduces private memory; defer the private +construction protocol, not the machinery required to represent an unknown access. + +## SCCP boundary + +The first SCCP snapshot keeps unknown-caller Start inputs and disconnects call +edges during the optimistic solve. Parameters retain their declared types; +call results retain facts from the preceding pessimistic pass, which can be +sharper than the written signature. Calls still have whole-memory effects. +After SCCP, peepholes relink calls and FinalizeIR removes uncalled bodies while +preserving named exports. This is an intermediate compiler design, not a +textual division of Opto.java. + +Ordinary loop widening belongs in the first chapter. The second adds argument +and result widening for recursion. Use `f(x-1)+1` to explain why these are +separate data cycles: the recursive return value never crosses Parm(x). + +Chained comparisons do not need their own chapter. Their home is +beside Chapter 23 short-circuit expressions: evaluate each operand +once and evaluate later operands only when earlier comparisons succeed. +The current Chapter 24 cold-loop-split allocator improvement belongs with the +tuning appendix. + +## Construction and escape boundary to establish + +Separate an explicit memory-publication operation from the analysis that +discovers exposure to unknown code. + +| Construction chapter | Escape-analysis chapter | +|---|---| +| Tracks private memory while an object is built. | Tracks aliases and functions exposed through memory, calls, and returns. | +| Publishes initialized memory at an explicit boundary. | Propagates exposure through the reachable graph. | +| Conservatively preserves unknown callers for surviving function addresses, following the current Chapter 24 policy. | Uses escape facts to determine which functions need unknown callers. | +| Explains EscapeNode as a publication operation. | Explains the Start/Stop feedback cycle between known code and the external world. | + +Verify this split against constructor calls and memory summaries. The earlier +snapshot can publish conservatively; the later one refines that behavior. +The name EscapeNode does not require teaching the full escape analysis at +the point where publication is introduced. + +`chapter26` now implements that explicit boundary. Hidden initializer +and allocator functions carry whole private object memory; completion publishes +fields into public aliases. The new constructor functions reject early passing +or storing of `self`, including self-links, and access through a merged alias. +These are deliberate limits for review with the next escape snapshot. Normal +function addresses retain conservative unknown callers. Initializers capture +enclosing values as hidden arguments; writing back to captures is rejected. +Arrays follow the same construction/publication protocol. There are no modules +or class singletons in this snapshot. + +Review on 2026-10-22 removed an unnecessary representation detour. Constructor +fields now stay in local SSA values and ordinary Phis; helpers emit ordinary +Stores at exit, with precise aliases gathered in MemMerge. Private memory is +only the handoff between helpers and publication. No global escape discovery +is needed to build that representation. + +Construction and escape discovery need not be taught in this order. Keep the +current order provisionally, without inventing a constructor example to claim +that dependency. The callback example now lives in the EA chapter. + +## Modules before separate compilation + +Teach module semantics with all source files compiled in one invocation. +Include lazy initialization, parent requirements, and cycle checking there. +Separate compilation then adds serialized ideal IR and types, GlobalBits +identity remapping, imported type reconstruction and analysis, and Unlink +before machine compilation. Extend initialization checks across imported +bodies, preserving opaque CLInit nodes until lowering. + +Keep code ownership distinct from unknown-caller edges in both chapters. +Keeping a function for emission must not imply arbitrary callers can reach it. + +## Appendices and future work + +- Native library construction: numeric C data symbols, address handling, + modular sys, runtime and driver support, and larger examples. Preserve the + native functionality already taught in earlier chapters. +- Optimizer and backend tuning: existing cloning/inlining choices, spill + ranking by live-range area and copy cost, rematerialization, and splitting + policies. Compare equivalent programs when explaining costs. + +This split organizes implemented behavior. Deterministic inlining, argument +arrays, static pre-initialization of classes, CFG frequency estimates, and +constant instance-field storage remain future work. See the +[roadmap](../docs/roadmap.md). + +## Renumbering and validation + +The first runnable extractions are +[SCCP Within a Function](../chapter24a/README.md) and +[SCCP Across Functions](../chapter24b/README.md), followed by +[SSA Before Types Are Known](../chapter25/README.md), +[Construction and Private Memory](../chapter26/README.md) and +[Escape Analysis and Unknown Callers](../chapter27/README.md), and +[Modules and Lazy Class Initialization](../chapter28a/README.md), and +[Separate Compilation](../chapter28b/README.md). +All seven main snapshots are extracted and numbered. Chained comparisons now +start in Chapter 23 beside short-circuit expressions. The linear-history +publisher follows the numbered directories, including lettered pairs. + +Validate each intermediate compiler with its own Make suite, including execution +checks. Keep declarations, conversions, diagnostics, and type bounds consistent; +do not weaken checks to make an intermediate snapshot pass. Rebuild serialized +libraries when their compiler or object format changes. + +## Escape snapshot ready for boundary review + +`chapter27` adds function/alias exposure sets to TypeMem and closes the +Start/Stop feedback cycle during SCCP. Known calls propagate memory summaries; +exposed returns, fields, external arguments and native calls discover further +addresses. Native calls contribute even on paths with no live Return. Function +addresses kept entirely in known code no longer imply unknown callers. + +The [recursive callback example](../chapter27/README.md) now belongs in +that chapter. Its base parameter becomes 2 while recursive calls remain. +Returning the callback or a containing object, or handing it to native code, +restores conservative arguments. FinalizeIR separates those analysis facts +from the roots needed to emit code. + +Construction still keeps local field values and rejects early exposure of self. +Both runnable snapshots are available for Cliff's promised review/play session. +This extraction does not choose broader rules for partially initialized objects. +No modules, class singletons, serialized IR or identity remapping are included. + +## Modules snapshot ready for review + +`chapter28a` compiles all discovered source files together. It adds class +singletons, namespaces and export filtering, parent initialization, CLInit memory effects, +cycle diagnostics, runtime lowering, and shared native class storage. Source +discovery reads parents as well as referenced children. The bundled modular sys +is supplied as source; building reusable libraries remains appendix material. + +There is no Serialize phase, object reader, GlobalBits remapping, or imported +IR. LocalIds assigns dense IDs within the invocation. FinishCalls detaches +analysis call edges after loop analysis, before scheduling or instruction +selection. ELF emits all discovered live bodies in the selected entry object. + +The preceding construction representation remains: field values and loop Phis +stay local, with Stores at helper exits. Constructor reads may see an unfinished +object, but may never read an uninitialized field. The module snapshot checks +this separately from completion and publication. Escape analysis still restores +unknown callers when a private callback reaches native code, including on paths +with no Return. + +The README's lazy lookup-table example is executable. Existing tests also cover +source/type cycles, rejected initialization cycles, parent order, native cold/hot +calls, shared storage, and ARM/RISC-V encoding. The construction/escape border +remains open to Cliff's review; this extraction does not settle broader rules for +publishing unfinished objects. + +## Separate compilation snapshot ready for review + +`chapter28b` adds saved types and ideal IR, `GlobalBits` +identity remapping, object discovery, imported bodies, and `Unlink`. It carries +Modules' deferred constructor parsing, local final-field values and loop Phis, +converted private memory summaries, native callback exposure, and source-parent +initialization order. Source privacy enforcement remains a separate review. + +The runnable example compiles a library and a client from separate source roots. +The client allocates an imported struct whose constructor updates a final field +in a loop, then calls a retained library function that updates shared state. +The regression checks evaluation under three seeds and native cold/repeated +calls. A second import test preserves unknown callers for a callback passed to +native code. Existing import tests retain initialization-cycle diagnostics, +shared class data, public type reconstruction, and native function thunks. + +Serialization now saves the class declarations of owned nested structs even +when they are absent from the optimized graph's type closure. Otherwise a +client can load the instance type but cannot find its allocation signature. +An explicit source input is always recompiled; dependency objects may still +be reused. The old incremental-rebuild experiment remains ignored. + +All seven main teaching boundaries now have numbered snapshots. Broader +language choices and compiler improvements remain in the review queue. diff --git a/skills/chapter25-codex-notes.md b/skills/chapter25-codex-notes.md index 36c5ee56..f585f654 100644 --- a/skills/chapter25-codex-notes.md +++ b/skills/chapter25-codex-notes.md @@ -1,34 +1,27 @@ -# Codex notes for Simple Chapter 25 +# Codex notes for Simple This file is intentionally AI-facing. It records durable invariants, debugging habits, and project-owner preferences that are easy to miss when reading only -the implementation. Use the current checkpoint below rather than old session -handoffs or historical branch/status reports. -For cross-chapter work, read [the backport queue and validation record](../docs/chapter-backports.md). -It owns pending reproductions and a condensed completion record; these notes -capture reusable lessons. Detailed superseded validation history remains in Git. +the implementation. Check current code and Git state rather than treating old +session handoffs as live status. +The [roadmap](../docs/roadmap.md) owns unfinished compiler work and language +proposals. These notes capture reusable lessons. Completed work and superseded +validation history remain in Git. +The [completed split](chapter-split.md) maps the former combined 24/25 +to 24a/24b (SCCP), 25 (incomplete types), 26 (construction), 27 (escape), and +28a/28b (modules/separate compilation). The old combined snapshots are retired. +Earlier entries below retain historical chapter numbers and test names; old +Chapter 25 implementation references describe the lineage now in 25–28b, with +the complete implementation in 28b. Use current paths and teaching boundaries. +The construction/escape border remains open to semantic review. +The [incremental build design](incremental-builds.md) expands the roadmap's +dependency and output-ownership requirements. For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.md). -## Reset checkpoint (2026-10-03) - -- Cliff committed the chapter reorganization and mutability split as `22df8918` - (`Chapter renumber, and split ch 17`). Implementation and validation are done; - there is no unfinished compiler port or hung validation process to resume. - Check current Git state before treating this checkpoint as live status. -- Cliff subsequently committed the Chapter 18 README review and backup-file - deletion as `3c13b271` (`Update README.md`). The tree was clean before this - memory refresh; only these two notes files are left modified by the refresh. - Do not commit or push without a new instruction. -- Read the mutability section and validation record in - [chapter-backports.md](../docs/chapter-backports.md) for scope and results. - Full affected chapter suites, Chapter 25 native tests, standalone 17a/17b - builds, and exporter checks passed. Logs are disposable under - `build/mutability/`; no need to repeat suites without a new change or concern. -- Next work is Cliff's review, not another inferred cleanup task. The separate - pending queue still includes float-to-int syntax and deferred BOTTOM contracts. - ## Collaboration preferences +- For densely keyed integer maps, use indexed `Ary<T>` storage rather than + `HashMap<Integer,T>`. - Build with Make alone; Maven must never be required. Cliff prefers few tests: avoid tests for every small viewer edit or rapidly changing display experiment. Use focused existing tests and disposable probes when they answer a real risk. @@ -43,10 +36,18 @@ For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.m Static `_spillScaled` also counts each inlined loop copy separately, without credit for eliminated calls; inspect the optimized call graph before calling a higher cross-chapter score an allocator regression. -- Prefer concrete ArrayList to List, common base classes to unnecessary - interfaces, and short names (`pred`, `proj`, `def`, `use`). Avoid redundant +- Use concrete JDK collection types in fields, parameters, and return types: + ArrayList, HashMap, HashSet, etc., rather than List, Map, or Set interfaces. + Prefer common base classes to unnecessary interfaces, and short names + (`pred`, `proj`, `def`, `use`). Avoid redundant Objects.requireNonNull. Prefer int[]/AryInt to List<Integer>; avoid large boxed ID collections (use arrays/BitSet or NBHML as appropriate). Node ID 0 means null. +- When both branches assign the same field or variable, prefer one assignment + with a ternary expression over an if/else. +- Space declaration initializers as `Type name = expression;`. Preserve + deliberate column alignment when it makes related declarations easier to read. +- Do not introduce Java streams. Use loops and explicit sorts; ask Cliff before + making any exception. - Cliff often debugs the same failure in parallel. If asked to work in a side repository, update `tmp/` and do not edit the live files he is debugging. - Stop and discuss before changing a central lattice, constructor, memory, or @@ -70,6 +71,188 @@ For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.m ## Mutability and chapter boundaries +- Chapter 28b emits one object per source CompUnit. Imported functions have + Fun/Summary/Return graphs until an object is materialized. SummaryNodes are + never inlineable bodies. Only local calls request speculative IR; an unused + imported function must not recursively fetch its callees. Function identity + survives materialization through GlobalBits and rewired call/return edges. +- Initialization summaries retain direct call/touch edges and parent edges. + A potential cycle requires full IR refinement before diagnosis: a helper + outside the SCC can provide the constant that removes an internal edge. + Native linking uses the compiler's resolved CompUnit object paths when + available. Object-only tests without a CodeGen list all link inputs explicitly; + TestC does no dependency search. Eval2 trusts TypeCheck and a coherent set of + library objects; it does not revalidate cross-object compatibility. + SummaryNode.load uses CodeGen.getElf directly; + Eval2 needs no extra byte snapshot or protection from rewritten object files. + Tests must isolate output paths to avoid writer/reader races. + Eval2 materializes one object's checked IR on demand and schedules it + independently; never reoptimize graphs with active evaluation frames. Eval2 + executes CLInit directly using shared run-once flags. The child's clinit + body checks its parent before setting its own flag, so Eval2 and Lower need + only the requested class's flag-test/call at each active use. +- Summary escape information must include retention on nonreturning paths and + references passed by new callers. Result types alone do not capture this. + Import reanalysis releases prior SCCP memory keeps before starting another + solve; freshly deserialized nodes need parser-like construction semantics. + +- Prefix updates must share ordinary name resolution: a method's bare field + name denotes memory, while constructor fields may still be local SSA values. + Do not update the declaration's scope slot before resolving that distinction. + Keep the returned value alive across Store folding; a narrow Store can remove + a conversion it does not need, while the prefix expression still needs it. + +- Separate compilation retains the Modules parser's constructor local values + and escape fixes while adding saved IR and import ownership. Save owned nested + struct class declarations even when graph types omit them: callers need the + allocation and constructor signatures before bodies load. Preserve the nearest + file owner so imported children are not claimed by a compiled parent. The + library/client example checks three seeds and repeated native calls; native + callback exposure is also checked after saving and loading the library. + +- Final data fields are writable constructor locals, including in loops; final + local bindings and shared function declarations stay fixed. Do not classify + a constant instance default as a shared field or skip its loop Phi merely + because the published field is final. Class constants keep their static image. + +- The modules snapshot is source-only: LocalIds reserves class slots within one + invocation; ParseAll reads discovered parents and children; FinishCalls removes + analysis call edges before scheduling/selection. No Serialize, ElfReader, + GlobalBits or imported type reconstruction. Keep the constructor and native + exposure cases in BoundaryTest when extracting separate compilation. +- A symbolic Store with alias 1 is a barrier to Load forwarding: until its + field resolves it may write the queried alias. Skipping it can turn a byte + compound assignment at 255 into a stale read of 255 instead of 2. Both + incomplete-types and construction keep this barrier; exercise worklist seeds. +- Constructor fields stay in local SSA values and ordinary Phis. Emit ordinary + initializing Stores at helper exits and package precise aliases in MemMerge. + Private memory carries fields between helpers and into publication; it does + not carry each loop update. Keep converted field values in private summaries + and keep unwritten fields visible to completion checks. Constructor bodies + are parsed after the field list closes, so later fields also have local slots. + The discarded InitStore/InitPhi design must not be brought into the EA split. + Constructor reads may see an unfinished object, but never an uninitialized + field. Exact self aliases must read the current local value, including inside + loops, rather than loading the allocation before exit Stores have run. + Construction privacy follows language rules; global escape discovery is a + separate topic, with no required teaching order between them. + +- The escape snapshot carries function/alias exposure sets in TypeMem. Stop + feeds externally visible results/memory back to Start; SCCP restores unknown + callers only for exposed function identities. Their parameters use declared + bounds, never the narrower values inferred from known calls. External callers + already hold their object arguments, so writes through them expose addresses. + Native calls are exposure roots even when their path has no live Return. + FinalizeIR freezes this feedback before replacing call edges with emission + roots; the analysis cycle is not runtime CFG. Keep construction's local SSA + representation and early-self restrictions for the planned boundary review. + +- Native function addresses in AMD64 ELF use low-address tail-jump thunks. + Static fields and computed addresses must name the same weak entry across + objects; direct calls can still use the native PLT. A native FunPtr has no + local FunNode, so Encoding must check externFunc before the dead-function + identity fallback. Do not suppress relocation overflow checks or make class + storage read-only to accept a narrow dynamic relocation. Preserve constant + initialization Store folding. See StaticDataTest.testNativeFunctionIdentity. +- Long-term pointer direction: compact code and data arenas, which may be + separate; full-width native pointers have separate types and lifetime rules. + Keep static class images viable for ROM and eventual removal of initialization + checks. These are future changes, recorded in the Chapter 25 roadmap; the + native thunk fix keeps current four-byte function/eight-byte data pointers. +- The class-init-cycle project is closed. Cycle checking, diagnostics, + serialization/import handling, and runtime lowering are complete for the + agreed scope. Argument arrays and initialization optimizations remain + separate ROADMAP work; they do not keep this project open. +- Native boundary decision (2026-10-18): no classes for native code; do not + design a native-startup policy. Ordinary supported FFI calls are separate. + Chapter 25's README owns the tutorial explanation of class initialization; + ROADMAP owns future work. Do not recreate an initialization progress document. +- GCM memory-Phi anti-dependencies constrain paths whose input differs from + the load's memory, not paths carrying that memory unchanged. Bound the idom + walk by the memory definition's depth: an alternate Phi input need not be + dominated by that definition. Also stop at `defblk.idom()`: when the memory + definition is the function entry, its null idom is the valid sentinel. + Include the boundary depth; a strict `>` skips required same-block edges + and can make a read observe a later store. Keep `testCapitalize`, + `testConditionalClassInitX86`, and + `testReadBeforeConditionalStore` passing; dropping Phi handling entirely + miscompiles the latter (140 instead of 120). + This rule applies from Chapter 13 onward. The nested-branch regression + `SchedulerTest.testMemoryPhiAcrossSiblingBranch` checks the disjoint walk; + removing the depth bound reproduces a null-stblk crash. Retain each early + chapter's evaluator scheduling rules when porting the boundary. +- CLInit lowering (2026-10-17): TypeCheck queues surviving markers; optional + Serialize saves opaque ideal IR, then Lower runs Iter before LoopTree. + Serialization includes disconnected Returns and filters cross-object links + without mutating the live graph. In Chapter 28b the child's initializer owns + both its parent-init check and its subsequent true flag store. Runtime + flag reads follow ordinary Load idealization, including removing control. + Handle x86 condition-flag lifetime constraints in the backend, not by pinning + ideal Loads. Generated calls remain unlinked and + conservative; do not reopen parameter flow into SCCP-optimized bodies. + Private/imported initializer bodies retain Start because opaque CLInit has no + ordinary Call edge yet; this does not change public reachability. + The selected entry gets the external arg; other clinits get zero for now. + The planned array ABI uses non-null entry args and null elsewhere; not yet + implemented. Eval2 shares singleton class objects and retains imported bodies. + Execution coverage includes native cold/hot calls, parent order, + ARM/RISC-V encoding, and nonreturning object reload. +- Chapter 25 phase structure: Opto only runs SCCP, numeric-mode + resolution and worklist seeding. IterPostOpto does peepholes/inlining; + FinalizeIR does field layout/offset cleanup and function/CU finalization. + opto() requires an explicit prior iter(). Direct calls needing the old complete + optimization sequence also name iterPostOpto().finalizeIR(); driver() handles + the sequence for later targets. Iter setup, FinalizeIR, TypeCheck, Unlink, + and InstSelect own their algorithms in separate files. CodeGen's phase wrapper + handles assertions, timing (including failures), and successful viewer phase + notifications. Its driver lists the order and skips completed steps. + Serialize decides whether ideal IR is needed: explicit Serialize or file + Export, not Encoding or in-memory linking. A staged file export must request + Serialize before Lower; ideal IR cannot be recovered from machine nodes. + Unlink detaches imports for machine selection, including requests stopping at + Select, while ideal evaluation retains them. Schedule requires Unlink or Select; + use the driver instead of skipping Lower/LoopTree in hand-written test chains. + Earlier chapters expose only phases with real work. CodeGen and its resumable + driver start in Chapter 18, with explicit LoopTree and LocalSched boundaries. + Instruction selection arrives in 19, RegAlloc in 20, Encoding/Export in 21. + FinalizeIR starts in 22 with unused-function cleanup and gains layout in 23. + Opto (SCCP) and IterPostOpto start in 24; earlier peepholes are named Iter. + Serialize, Lower, and Unlink stay in 25. Use short explicit chains such as + parse().iter(), and driver(Phase.target, ...) for longer compilation paths. +- Class-init memory checkpoint (2026-10-17) is implemented: keep CLInit + opaque through Iter/Opto, carry whole-memory effects plus the known-true + `$init` alias result, and use normal parser memory for redundant checks. + The flag store belongs inside the initializer. Parser creates the ready load + up front; ordinary Load/FldOff nodes resolve forward receivers. The fourth + CLInit input is always present in a constructed graph. If the receiver resolves + to an instance, CLInit and its speculative flag load disappear together. + Precise alias recovery can still be deferred. See `chapter28a/README.md`. + Runtime lowering is implemented above; argument arrays remain deferred. Public function fields + on instances of public classes must retain Start, just like class-object + fields. publicFIDX now checks both; this fixes sys.scan.match losing Start + while building sys.o and subsequently widening after import. + Do not weaken assertions or reset imported types + to conceal incorrect public classification. + Future optimization: move final, always-constant instance fields to the class + or fold them to constants and omit their instance storage. Access semantics + and public reachability remain unchanged; do not implement layout changes yet. + Frequency estimates belong + after Opto and lowering: function entry 1, unknown branches 50/50, estimated + loop trips 10, rare init/failure edges, non-null/null 90/10; bounded, + flow-consistent estimates shared by GCM and Encoding. +- Class-init cycle validation (2026-10-14): reject an SCC only when it contains + more than one `<clinit>`. A self-edge or helper recursion with one initializer + is legal, regardless of inlining. Error reports list every internal edge of + the first rejected SCC, including helper calls; stack order is not a path. +- Class-cycle diagnostics use `SourceError.format(CompUnit, byteOffset, message)`. + Start `file:line:column: error: message` in column one for Emacs compilation + buffers; put the source and caret on separate following lines. With no source, + report the path and saved byte offset, without inventing line/column numbers. + Lexer snapshots keep their original CompUnit through inlining and Call/CLInit + serialization; recover representative sites by rescanning only on rejection. + Source text is assumed available. Imported source paths are diagnostic data, + not `_smp` rebuild/discovery state. General error-message migration is deferred. + - Constructors start in 16, independent binding/access permissions in 17a, syntax sugar and inference in 17b. Historical Chapter 17 references mean 17b. - `!Point` is writable access, `~Point` deep read-only access; `!p` allows @@ -87,6 +270,11 @@ For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.m - Resolving a forward struct reference must preserve pointer access/nullability. Phi idealization should report progress only when the referenced struct changes, not merely because the canonical declaration has different pointer permissions. +- In chapter25, complete named declarations as a group before + upgrading old graph types. Update stored declarations in New, Parm, Convert, + and similar nodes as well as their cached result types. A nested undeclared + struct must remain open, not become an empty finished definition. Unknown + expression types are distinct from explicit missing-initialization sentinels. - Construct cyclic pointers with their access bit already set before interning; creating a provisional pointer and then changing access can orphan temporary cyclic types. Serialized objects use the unversioned `C0DE` header; rebuild @@ -94,6 +282,10 @@ For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.m ## Tutorial backports +- Chapter 18 is the evaluator cutover: use `Eval2` and the compiler's schedule + from that chapter onward. The old `evaluator/Evaluator` and its private + `Scheduler` remain only in Chapters 10-17b; do not forward-port or revive them. + - Shared ISA encoding lives in `isa/` for Chapters 21-25: x86 bytes behind `CodeSink`, ARM/RISC-V word packing, and test-only evaluators. Chapter code still owns machine nodes, register masks, operand/type interpretation, layouts and @@ -111,11 +303,22 @@ For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.m deliberately retains immutable shallow struct references resolved by name; see its README's L0/L1 notes before trying to expand recursive types. Keep Chapter 25's separate constructor-memory and incomplete-type design. - See the concrete boundary notes - in `docs/chapter-backports.md`. GCM readiness and anti-dependency checks must both + GCM readiness and anti-dependency checks must both filter by alias and ignore MemMerge as a clobber; keep the evaluator's same alias filtering too. MemMerge still needs ordinary data-dependency placement. +- From Chapter 15, an array element Load's control stands for its range check. + Keep that edge during parsing and rewriting. A Load can push through a + memory Phi when its control is null or dominates the Region's idom; retain + that control on both copies. The Region itself or a later check cannot + supply control to predecessor loads. Dropping it permits an out-of-bounds + speculative read. The parser currently attaches the current control as a + placeholder, which can be the Region itself even in a valid graph. + `Chapter15Test.testArrayLoadControl` checks a guarded read after a memory + merge; `testArrayLoadDominatingControl` checks successful splitting under + an earlier common check. Length loads have no index; + Chapters 19+ already leave them unpinned, independently of element control. + - Chapter 15 New consumes `{ctrl, $mem, size}` and produces `{ptr, $mem}`: no control result, pointer slot 0, memory slot 1. Its TypeStruct/TypeMemPtr defines the covered field aliases. Use a partial input MemMerge (null default), @@ -230,25 +433,35 @@ For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.m User-count dependencies must wake from Node.delUse, including unkeep. - Generalized Phi factoring now starts in 10 and is forwarded through 25. - Memory-specific eligibility lives in `MemOpNode.canDrop`, with virtual Load - and Store checks. Load owns `clobbered`: all these chapters check only - immediate memory users, without recursion or a visited set. Stop at Stores, - Phis, MemMerges (11+), New (15+), and Calls (18+); do not chase aliases or aggregates just - to save one Load. Keep this teaching optimization simple and conservative. + Memory-specific eligibility lives in `MemOpNode.canDrop`. Store identity + equality now excludes Stores from this rewrite. Load owns `clobbered`: all + these chapters check only immediate memory users, without recursion or a + visited set. Stop at Stores, Phis, MemMerges (11+), New (15+), and Calls (18+); + do not chase aliases or aggregates just to save one Load. Keep this teaching + optimization simple and conservative. Preserve exact operation attributes with clone-based `copyEmpty`, type each operand Phi separately, reuse identical inputs, and reject type widening. - Stores require sole use by the Phi; Loads inspect memory users for clobbers - before GCM has made anti-dependence edges. From 11, create precise memory + Loads inspect memory users for clobbers before GCM has made anti-dependence + edges. From 11, create precise memory Phis for the factored operands. Do not factor aggregates or bulk Phis; wait for other BulkMemPhis at the Region to disappear, since their slice - lookup assumes Region/alias identifies the completed memory point. Existing - control-bound Stores in 13-14 remain ineligible. `delUse` wakes recorded - dependents for user-count queries; Region's empty-diamond fold depends on + lookup assumes Region/alias identifies the completed memory point. `delUse` + wakes recorded dependents for user-count queries; Region's empty-diamond fold depends on projection rewiring as well as projection types. These guards are already forwarded through 25; preserve the later chapters' existing `copyEmpty`. `subsume` must wake each rewired user's recorded dependents, just as `setDef` does: operand identity changes can enable a Phi fold without changing the intervening operation's type (`Chapter14Test.testCloneAnd` in 16). +- Stores use identity equality from Chapter 10 onward. Equal Stores on two of + three paths must not merge and run on the third path. Sharing requires the + original Stores to cover every exit path, or GCM to split the shared Store + back onto those paths. GVN checks neither, and GCM does not split Stores. + This also disables generic Phi factoring of Stores, but keeps explicit + store-after-store elimination and the closed-loop sink introduced in 17b. + `Chapter10Test.testConditionalStoreSharing` returns the object to expose the + actual write; returning a Load could hide it through forwarding. + `Chapter17bTest.testSinkLoopStores` checks exit placement and zero-trip values + in 17b and every later snapshot. Do not restore Store GVN to recover a fold. - Find the chapter where the relevant feature first appears, not just the chapter that can parse the original reproducer. Reduce away later syntax/features when possible. Put the regression in that earliest `ChapterNTest.java`, and forward @@ -258,13 +471,13 @@ For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.m top-level runner, e.g. `make -k tests CHAPTERS="chapter10 chapter11 chapter12"`, with all affected directories explicitly listed. Establish the destination baseline before changing it; older Makefiles may need a forced rebuild after API changes. -- Keep unrelated discoveries separate in `docs/chapter-backports.md`. Completed - fixes leave the pending queue; retain a concise validation summary and move - unresolved discoveries out of the history into the actual queue. A green suite +- Record confirmed unresolved bugs and future work in `docs/roadmap.md`. + Remove completed items and record validation in the change description. + Do not turn an unconfirmed textual difference into a backport task. A green suite after changing graph/test order does not prove an intermittent failure fixed. - Store disposable probes/logs under an appropriate ignored build directory. Preserve durable reproducers in tests or the review record, not only scratch - files. The B13 `chapter25/tmp` probes were removed at Cliff's request after he + files. The B13 `chapter28b/tmp` probes were removed at Cliff's request after he reviewed and pushed the fix; future sessions must not rely on those files. ## Dominator caches and searches @@ -466,6 +679,18 @@ For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.m before coloring, a fixed-register definition can clobber despite `_reg==-1`. Cloning is valid only if its output mask can satisfy the use. A fixed-register clone with flexible uses needs use-side splits, not a no-op def-side path. +- Chapters 20-25 flag spills use specialized splits that exclude FP registers. + x86 can transfer FLAGS directly to a GPR or stack slot. ARM's + `SplitFlagsARM` transfers NZCV through a GPR and overrides + `isNonStackSplit()`; RegAlloc includes these transfers in loop-depth + collection and subsequent splitting. `SplitGPRARM` then admits GPRs and + stack slots, preserving the FP exclusion through cascading spills. + Keep ordinary splits excluded from those allocator paths. Test both flags + clobbers and simultaneous flags/GPR clobbers: `testFlagSpills` executes the + allocated moves for all 64 x86 status-bit and 16 ARM NZCV combinations from + Chapter 21 onward. Chapter 20 checks allocation and masks only; encoding + begins in 21. Preserve each allocator's existing ordinary-split policy: + 20-22 always skip ordinary splits here, while 23-25 skip them when min!=max. - Check function ownership and register masks immediately after allocation, before encoding rewrites tail calls or adds untyped branches. Call.regmap can itself depend on CFG.fun/idom. The test-only CheckedCodeGen override keeps @@ -479,7 +704,7 @@ For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.m with documented constructor, receiver, library and mutability adaptations. These are allocation/legality replays, not historical runtime harnesses. Its own cohort includes current Chapter25Test allocations and a fresh library - encoding. See `chapter25/src/test/java/com/seaofnodes/simple/spill/README.md`. + encoding. See `chapter28b/src/test/java/com/seaofnodes/simple/spill/README.md`. Do not silently use current inherited tests as though their sources were frozen. - Rebuild and measure the system library even if sys.o is up to date. In the final 25 review, the older ranking and a multi-use-clone preference looked @@ -637,6 +862,13 @@ Consequences: ## Types and monotonicity +- Chapters 24a/24b retain typed fallback results for erroneous or unresolved + Loads and arithmetic. Chapter 25 introduces the surrounding Convert nodes + and removes parser-supplied bounds. Do not backport BOTTOM-on-error in + isolation: parser, conversions, loop bounds, and diagnostics must agree. + Preserve `testBrainFuck`, `testWhileWithNullInside`, `testNullGuardErrors`, + `testLinkedList0`, and `testCoRecur`, including their expected errors. + - Integer widening starts in Chapter 24. `nonZero()` preserves `_widen` when narrowing a range. B10's correction is in 24, with `TypeTest.testNonZeroWidening` in 24-25. Both now use widening 0-3 and dual widening `3-widen`, preserve @@ -644,6 +876,13 @@ Consequences: widen only below-center nonconstant ranges: above-center widening can fall to an unwidened constant, violating SCCP monotonicity. Existing Chapter14/21 tests exposed this during the backport. +- Recursion needs two widening points in Chapters 24-25: ParmNode for arguments, + and CallEndNode for results. The return cycle in `f(x-1)+1` never crosses the + Parm. Keep the comments at both sites and PhiNode cross-linked. Chapter 25's + FunPtrNode follows the live Return range; join that range before widening, + then bound widening by the callee signatures, not that live range. Apply the + same rule before and after linking, and retain narrow results such as i32. + `Chapter24Test.testRecursiveWidening` checks both cycles in both chapters. - TypeConAry's `_make(any,widen)` shares the backing array and returns an uninterned type. `xdual()` must leave interning to Type.intern's dual-pair setup; ordinary factories and ymeet's widening replacement intern their results. @@ -730,6 +969,11 @@ Consequences: checking rather than a pre-Opto deletion. - Extern C functions have no Simple FunNode/Return and are handled only while linking. +- From Chapter 22, retain native symbol names by function identity in CodeGen. + Selection and spill copies can turn ExternNode into a plain ConstantNode; + address relocation must consult that identity before looking for a local + function. ARM ADRP splits a signed page delta into unsigned instruction + fields: mask both pieces, including the high piece for backward addresses. - Unknown-caller Start edges and post-Opto compilation-unit ownership are separate concepts. Do not overload Start edges merely to keep surviving functions attached to a compilation unit. @@ -867,8 +1111,8 @@ serialize/deserialization bijection after persisted graph changes Default fuzzer wrappers use explicit regression seed lists; exploratory fuzzing is opt-in. A passing wrapper with an empty list is not exploratory coverage, and `OPEN_FAILING_SEEDS` remain unresolved until explicitly verified and promoted. -For dated suite counts and known failures, consult the backport validation record -rather than treating an old count as the current baseline. +For dated suite counts, consult the validation record in Git history; current +known work is in the roadmap. Do not treat an old count as the current baseline. Peepholes are always enabled in every chapter, by Cliff's request on 2026-09-26. Do not reintroduce an unoptimized compiler mode. Incomplete Loop/Phi nodes still @@ -889,7 +1133,7 @@ comparisons as equal runtime results. the SUB, not copy its inputs. ARM register 31 is XZR for load destinations, so narrow loads use WMASK, never RMASK. Logical-not uses CMP to XZR followed by CSET to its result register. Shared emulator fixes and Chapter 21-25 - backports are recorded in `docs/chapter-backports.md`. + backports are recorded in Git history. - Cohort 23's dynamic increase from compiler 24 to 25 is primarily class-field initialization: Ops 630->1106. RA 160->442 includes 224 extra pointer-address and 56 integer rematerialization instructions, but only two extra splits. diff --git a/skills/current-spill-matrix.md b/skills/current-spill-matrix.md new file mode 100644 index 00000000..ee77a77a --- /dev/null +++ b/skills/current-spill-matrix.md @@ -0,0 +1,14 @@ +# Current allocator comparison + +`make spill-stats` measures Chapters 20, 21, 22, 23, 24b, and 28b against the +historical test cohorts. Cohort IDs and test names retain their old numbers; +column headings identify the current compiler directories. + +These are separate from the dated [pre-split measurements](../docs/regalloc-spills.md). +After a successful run, `python build-support/spill-matrix.py --reuse --update-docs` +updates only this table. Fresh logs use 24b/28b names so old 24/25 results cannot +silently become measurements of new compiler snapshots. + +<!-- spill-matrix:start --> +No post-split measurement has been published yet. +<!-- spill-matrix:end --> diff --git a/skills/graph-viewer-codex-notes.md b/skills/graph-viewer-codex-notes.md index e4775e91..645efdd8 100644 --- a/skills/graph-viewer-codex-notes.md +++ b/skills/graph-viewer-codex-notes.md @@ -4,8 +4,8 @@ Updated 2026-09-26. Read graph/README.md for the current design and protocol. These are AI-facing continuity notes, not a new task list or authority to change compiler semantics. Cliff declared the viewer task complete on 2026-09-26. Peephole animations, inlining display, TypeCheck diagnostics, and arrow-key -playback are committed. Resume the independent bug-fix backport review from -docs/chapter-backports.md when requested. +playback are committed. Unfinished compiler work is tracked in +docs/roadmap.md. ## Working tree and latest change @@ -26,7 +26,8 @@ diagnostic. Implemented in chapters 10-25 (the chapters with a TypeCheck pass): - Chapters 10-17: SimpleGraphObserver calls STOP.typeCheck() after Iter; the previously package-private method is now public. Chapters 18-25 call - CodeGen.parse().opto().typeCheck(). Earlier chapters remain unchanged. + CodeGen.driver(CodeGen.Phase.TypeCheck), running the real ideal phases in + that chapter. Earlier chapters remain unchanged. - GraphObserver.error(node,msg) is an optional callback. Existing compiler type checking reports the node it diagnoses, rather than the viewer repeating the check or trying to infer the node from text. Chapter 25 retains its originating @@ -47,7 +48,7 @@ passed for real TypeCheck errors in chapters 10, 17, 18, 25, including final Nex folded error nodes, backward playback, direct jump, and a subsequent valid compile. Dead erroneous code was removed before TypeCheck. Existing JUnit checks passed: chapter10 Chapter10Test (27), chapter18 Chapter13Test + Chapter18Test (39), -chapter25 Chapter13Test (16). This was NOT an all-chapter full test-suite run. +chapter28b Chapter13Test (16). This was NOT an all-chapter full test-suite run. ## Durable design and owner preferences @@ -145,7 +146,7 @@ capture server on an ephemeral port. Scratch helpers may disappear after cleanup sibling-Phi exclusion, nested replacement ordering. Some older scratch scripts assumed adjacent capture frames and are stale after atomic peep playback. - build/check-folding-state.py: inlining FOLDING labels; folding-frames.json is a - 349-frame chapter25 capture predating the added final TypeCheck frame. + 349-frame chapter28b capture predating the added final TypeCheck frame. - build/check-graph-finish.py defines capture(ch) before its chapter05 invocation; helpers reuse that prefix. build/FinishFrames.java (build/graph-finish classes) calls GraphCapture.run on an ephemeral socket without launching a browser. diff --git a/skills/incremental-builds.md b/skills/incremental-builds.md new file mode 100644 index 00000000..47c06fd9 --- /dev/null +++ b/skills/incremental-builds.md @@ -0,0 +1,76 @@ +# Incremental builds + +Implementation notes and remaining build-driver work, reviewed 2026-10-25. + +## Current reuse check + +Chapter 28b's `CompUnit` decides whether to reuse a local object by walking its +recorded dependencies before loading saved types or IR. It checks all sources +owned by each local object, missing inputs, and dependency object timestamps. +Dependencies use the same local/library resolution for checking and loading. + +For example: + +1. A depends on B and has inlined some of B. +2. B's source changes, leaving B's object unchanged. +3. A client discovers A. The check follows A's recorded dependency to B and + finds B's newer source, even though B's object has not changed. +4. A is reparsed, so its saved inlined copy of B is never accepted. + +Use a fresh visited set for each candidate object. Cycles terminate safely, +but a provisional success inside a cycle must not be cached for another root. +Each source compilation unit now owns its own object; units compiled together +share a build timestamp so cycles do not invalidate themselves due to write order. +An unchanged public signature is insufficient: Simple imports implementation +details for optimization as well as declarations for type checking. + +Object-only libraries are still supplied as a coherent build. Do not reject +them merely because one dependency has a newer installation timestamp: existing +tests compile cyclic libraries against stubs and then combine their real +objects. Validating those combinations requires saved input versions, not a +guess from file times. A newer library object does invalidate a source-backed +local dependent that can be rebuilt. + +`IncrementalBuildTest` covers unchanged reuse, transitive source edits, +cycles, missing inputs, library dependencies, and clean/incremental native +execution parity. It sets file times directly, without sleeps. + +## Required pieces + +- Plan object targets and rebuilds across invocations. Handle cycles as groups + and newly discovered dependencies. The compiler's per-object reuse check is + not a complete scheduling or linking policy. +- Record actual input identities: source units and resolved library paths and + versions. A search + path change can select a different library even with an older timestamp. +- Record compiler and serialization versions, target, ABI, and relevant + options. Content hashes would also detect changes with preserved timestamps. + Specify which changes require rebuilding source and which reject an + incompatible source-free library. +- Make build targets account for every output of a compilation request, which + can discover several source units and produce a separate object for each. +- Provide an "ensure current" build operation. Explicit source compilation + currently always recompiles. An unchanged build should do no compilation or + linking; a changed object or link input should trigger relinking. +- Publish successful outputs without leaving a partial object accepted after + failure, and keep the recorded inputs consistent with those outputs. + +Initially, rebuilding every dependent of a changed object is a reasonable +conservative rule. Fine-grained interface/body fingerprints can come later. + +## Suggested first implementation + +Put a small build driver around the compiler with explicit object targets, +recorded inputs, and dependency propagation. Keep individual compilation +requests straightforward. A clean build and an incremental build must produce +the same observable program behavior. + +Cover unchanged builds, direct and transitive source edits, inlined constants +and bodies, cyclic dependencies, multiple outputs, removed inputs, library-path +changes, compiler/target changes, and failed builds. Use controlled timestamps +or hashes instead of sleeps. Replace the ignored historical experiment with +tests matching the chosen object ownership model. + +Library IR loading is independent of build and native link dependencies. +Summary-only imports still require coherent objects at link time, and inlining +does not erase the build dependency on the object supplying the copied body. diff --git a/skills/japanese-translation.md b/skills/japanese-translation.md new file mode 100644 index 00000000..39383345 --- /dev/null +++ b/skills/japanese-translation.md @@ -0,0 +1,22 @@ +# Japanese translation / 翻訳の保守方針 + +English is authoritative. The contributed Japanese translation is maintained on +a best-effort basis; English edits and chapter reorganizations never wait for +translation updates. Move Japanese files and update navigation when renumbering +chapters; prose updates can follow separately. Contributions are welcome, without +an ongoing maintenance obligation for the original translator. + +[英語版](../README.md)が正式な文書です。この日本語訳は寄贈されたもので、可能な範囲で保守されますが、 +英語版より更新が遅れることがあります。英語版の編集や章の再構成を、翻訳の更新待ちにすることはありません。 +章を移動・改番する際は、日本語ファイルも一緒に移動して章の見出しやリンクを更新し、本文の翻訳は後から更新できます。 +翻訳の改善は歓迎しますが、元の寄贈者に継続的な保守を義務付けるものではありません。 + +初回の翻訳は英語版のコミット +[`81b654d7`](https://github.com/SeaOfNodes/Simple/tree/81b654d76444d747fdf1d053760e507b50f26e18) +を基にしています。2026年10月の目次・文法見出し・コード例・命令エンコーディング・ +`C0DE` ヘッダに関する文書修正も反映しています。 +各文書の English リンクは現在の英語版を指します。更新されていない箇所の原文を確認するには、 +上記コミットの対応するファイルを参照してください。新しい章が未翻訳の場合は、その旨を示して英語版にリンクできます。 + + +[日本語の紹介に戻る](../README.ja.md) diff --git a/skills/stage-audit.md b/skills/stage-audit.md new file mode 100644 index 00000000..194a5178 --- /dev/null +++ b/skills/stage-audit.md @@ -0,0 +1,113 @@ +# Chapter split cleanup audit + +This record distinguishes checked behavior, existing gaps, and future designs +while retiring the old combined Chapters 24/25. The final mapping is in the +[split record](chapter-split.md). Historical Java test names are retained as +regression identities; they do not identify current chapter boundaries. + +## Corrections in this cleanup + +- Chained comparisons now begin in Chapter 23. The early SCCP implementations + incorrectly compared a boolean intermediate with the third operand: + `10 < 30 < 20` returned true. Later implementations preserved the middle + operand but reused the first operator, making `0 < 2 <= 2` false. All current + snapshots from 23 onward now compare adjacent values using each written + operator. Regression cases cover both directions, strict/non-strict mixtures, + single evaluation, side effects, short-circuiting, and mixed-direction errors. +- Chapter 23's inherited, explicitly described fuzzer scratch contains many + comparisons using the former left-associative grammar. It is retained as an + ignored `testJigFuzzer`, consistent with the scratch-test policy; its mixed + directions are no longer valid source. Later snapshots already omitted it. +- Native-library and tuning material is preserved in [appendices](../docs/appendices/README.md), + and unfinished work is collected in the [roadmap](../docs/roadmap.md). +- The recursive-return diagnostic already passed. It remains enabled in the + earlier snapshots and is enabled again in 28a/28b. The mutual-recursion test had an obsolete graph-string + expectation; it now checks its existing four execution results in 24b–28b. + In 24a, local SCCP cannot infer the recursive return type; the same example + checks that conservative diagnostic instead. This makes the boundary explicit. +- Allocation/runtime measurement runs passed after the comparison correction; + the changed spill goldens record the resulting smaller graphs. Historical + published tables retain their original compiler labels and dates. + +## Prefix-update follow-up + +Prefix `++` and `--` now use the same name lookup as ordinary reads and postfix +updates in Chapters 23 through 28b. A method's field goes through memory; a +constructor-local field stays in SSA. Previously, the prefix path updated the +field declaration's parser slot and silently lost the write. The two-call +reproducer now returns 21 instead of 11. Narrow update results stay alive while +Store folding drops a redundant conversion from its own input. + +`PrefixUpdateTest` covers increment/decrement, integer/float/byte fields, +multiple worklist seeds, local shadowing, final-field errors, native byte +wraparound, and module fields where supported. Existing final-constructor +coverage includes prefix updates in loops. + +The focused Linux checks pass in all eight affected chapters, including native +SystemV byte-wraparound execution and the existing final-constructor tests. +The early final-field rejection is retained so an illegal prefix update never +enters the optimizer as a Store; native data bindings still update their writable +contents despite having a fixed address. + +The Windows full suites cover 4,169 tests plus eight fuzzer regression runs; +shared printer and ISA checks pass. Focused reruns cover the final diagnostic +and prefix-result lifetime adjustments. The full Chapter 27 suite was rerun +after restricting the temporary keep to prefix results; its existing spill +expectations remain unchanged. No allocator goldens were updated. + +## Implemented, with narrower limits than earlier prose suggested + +The constructor-loop assertion is fixed in Chapters 25–27. Load forwarding now +waits for a stored loop Phi to acquire a type at least as precise as the Load, +and records a dependency to retry. The reduced case needs only a conditional +assignment in a loop; break/continue and final fields are not required. +Regressions cover the reduced case and the original constructor loop across +worklist seeds and execution inputs. Chapters 28a/28b already had the guard; +the corresponding earlier-chapter probes pass in 23/24a/24b. + +- Separate compilation imports types and optimized IR without source parsing; + it reruns analysis and type checking with the new callers. +- Loading is by object and recorded dependencies, not individual used members. + A call to `sys.io.p` loads all ten bundled sys units. Independent, unrelated + objects can remain unloaded. Loading does not imply class initialization. +- Timestamp checks do not yet provide a reliable incremental build system. + The [design note](incremental-builds.md) records the missing pieces. +- Underscores select exports in modules; source privacy checks are still a + proposal awaiting review. Do not treat hidden exports as inaccessible source. +- Constructor-local final fields use SSA values and exit Stores. Reads of an + unfinished object are allowed only for initialized fields. Passing unfinished + self to arbitrary helpers and accepting completion only after SCCP are not + promised by the current chapter boundary. + +## Existing failures and deferred validation + +- The ideal evaluator does not truncate narrow stores in Chapter 23. A byte + update can therefore disagree with generated code; the postfix-only + reproducer is retained in the [roadmap](../docs/roadmap.md#narrow-stores-in-the-ideal-evaluator). +- `testModule0` is an ignored historical incremental-build experiment. It + assumes per-source output objects and unchanged explicit-input builds, neither + of which matches the present compiler's contract. +- Serialization's expensive byte-for-byte round-trip assertion remains disabled. + Ordinary import tests do not establish that stronger property. +- Direct probes of the remaining ignored tests found a stale forward-method + graph expectation and a missing `progs/sieve.smp` fixture in 28b. These and + the interactive stdin test remain ignored. Ignored Jig and + random-fuzzer entry points are intentional scratch/long-running tools. + Audit these separately before advertising their behavior as tested. + +Selective loading, full incremental builds, source privacy, compact pointers, +argument arrays, deterministic inlining, and broader constructor rules remain +future work. They are not prerequisites for completing the chapter numbering. + +## Validation + +The affected snapshots (23, 24a, 24b, 25, 26, 27, 28a, 28b) were checked with +their full Make suites on Windows and Linux, plus focused reruns after updating +test expectations. The final Windows coverage is 4,143 tests plus eight fuzzer +regression runs, with the shared printer and ISA tests also passing. The +native library/client cases and final Chapter 28b release build pass on both OSes. + +A local linear-history dry run produced all 31 tags in teaching order. The +final source snapshot, chapter navigation, Maven module IDs, and reviewed local +documentation links were checked. Measurement capture in 24b and dynamic +measurements in 24b/28b pass; the historical published tables were not rewritten. |