diff options
Diffstat (limited to 'skills')
| -rw-r--r-- | skills/chapter-split.md | 220 | ||||
| -rw-r--r-- | skills/chapter25-codex-notes.md | 119 | ||||
| -rw-r--r-- | skills/current-spill-matrix.md | 14 | ||||
| -rw-r--r-- | skills/graph-viewer-codex-notes.md | 8 | ||||
| -rw-r--r-- | skills/incremental-builds.md | 75 | ||||
| -rw-r--r-- | skills/japanese-translation.md | 22 | ||||
| -rw-r--r-- | skills/stage-audit.md | 113 |
7 files changed, 554 insertions, 17 deletions
diff --git a/skills/chapter-split.md b/skills/chapter-split.md new file mode 100644 index 00000000..67a974e4 --- /dev/null +++ b/skills/chapter-split.md @@ -0,0 +1,220 @@ +# Approved chapter split + +Cliff approved the seven-part split on 2026-10-19 and the final numbering on +2026-10-24. The old combined Chapters 24 and 25 are retired; their history is +available in Git. Every new directory is a complete compiler snapshot. + +| Previous directory | Final chapter | +|---|---| +| chapter-sccp | 24a: SCCP within a function | +| chapter-interprocedural-sccp | 24b: SCCP across functions | +| chapter-incomplete-types | 25: SSA before types are known | +| chapter-construction | 26: Construction and private memory | +| chapter-escape | 27: Escape analysis and unknown callers | +| chapter-modules | 28a: Modules and lazy class initialization | +| chapter-separate-compilation | 28b: Separate compilation | + +Keep 17a/17b. The other paired chapters extend one idea: 24a/24b goes from +local to interprocedural SCCP; 28a/28b goes from compiling modules together to +compiling them independently. Construction and escape remain separate topics; +neither is presented as requiring the other. + +Inherited Java test names such as `Chapter24Test` and `Chapter25Test` retain +their historical identities across snapshots. They are regression cohorts, +not an index of the current teaching sequence. Historical allocator tables +also retain their original labels and dates. + +Native library construction and optimizer tuning are now [appendices](../docs/appendices/README.md). +The [roadmap](../docs/roadmap.md) owns future work. + +The teaching pattern is to introduce a difficult idea, then show how to make +it useful in a larger setting. Each chapter must remain a complete, runnable +compiler snapshot that preserves the language supported by its predecessor. + +## Teaching sequence + +| Topic | Main idea | Endpoint | +|---|---|---| +| SCCP within a function | Optimistic types, reachable control, Phi inputs, worklists, fixed points, and loop widening. Keep calls conservative at their interfaces. | Fold loop-carried values and eliminate branches that pessimistic iteration cannot resolve. | +| SCCP across functions | Discover call edges during analysis; merge arguments and returns; account for unknown callers and function pointers. | Specialize parameters, resolve call targets, handle recursion, and remove unreachable functions. | +| SSA before types are known | Syntax and lexical binding choose graph shape. Keep scalar types, deferred numeric modes, conversions, guards, incomplete objects, symbolic fields, and removal of parser-supplied Load/Phi bounds together. | Compile forward definitions without provisional types determining graph topology; settle types and layouts afterward. | +| Construction and private memory | Allocation helpers, hidden initializers, user constructors, private self-memory, field-completion checks, and explicit publication. | Preserve initialization and memory facts through constructor calls and publish a completed object. | +| Escape analysis and unknown callers | Discover which aliases and functions can reach code outside the known graph. | Preserve exposed memory and callable functions while specializing private code. | +| Modules and lazy class initialization | Multiple source files, namespaces, privacy, ownership, singleton storage, first-use initialization, parent order, cycle diagnostics, and lowering. | Compile sources together with defined class-initialization behavior. | +| Separate compilation | Preserve types, ideal IR, and identities across independent compilations. | Build a library and client separately, optimize imported bodies, link remaining calls, and check initialization dependencies across imports. | + +The first incomplete-type chapter stays together. Its unresolved memory accesses +must work before construction introduces private memory; defer the private +construction protocol, not the machinery required to represent an unknown access. + +## SCCP boundary + +The first SCCP snapshot keeps unknown-caller Start inputs and disconnects call +edges during the optimistic solve. Parameters retain their declared types; +call results retain facts from the preceding pessimistic pass, which can be +sharper than the written signature. Calls still have whole-memory effects. +After SCCP, peepholes relink calls and FinalizeIR removes uncalled bodies while +preserving named exports. This is an intermediate compiler design, not a +textual division of Opto.java. + +Ordinary loop widening belongs in the first chapter. The second adds argument +and result widening for recursion. Use `f(x-1)+1` to explain why these are +separate data cycles: the recursive return value never crosses Parm(x). + +Chained comparisons do not need their own chapter. Their home is +beside Chapter 23 short-circuit expressions: evaluate each operand +once and evaluate later operands only when earlier comparisons succeed. +The current Chapter 24 cold-loop-split allocator improvement belongs with the +tuning appendix. + +## Construction and escape boundary to establish + +Separate an explicit memory-publication operation from the analysis that +discovers exposure to unknown code. + +| Construction chapter | Escape-analysis chapter | +|---|---| +| Tracks private memory while an object is built. | Tracks aliases and functions exposed through memory, calls, and returns. | +| Publishes initialized memory at an explicit boundary. | Propagates exposure through the reachable graph. | +| Conservatively preserves unknown callers for surviving function addresses, following the current Chapter 24 policy. | Uses escape facts to determine which functions need unknown callers. | +| Explains EscapeNode as a publication operation. | Explains the Start/Stop feedback cycle between known code and the external world. | + +Verify this split against constructor calls and memory summaries. The earlier +snapshot can publish conservatively; the later one refines that behavior. +The name EscapeNode does not require teaching the full escape analysis at +the point where publication is introduced. + +`chapter26` now implements that explicit boundary. Hidden initializer +and allocator functions carry whole private object memory; completion publishes +fields into public aliases. The new constructor functions reject early passing +or storing of `self`, including self-links, and access through a merged alias. +These are deliberate limits for review with the next escape snapshot. Normal +function addresses retain conservative unknown callers. Initializers capture +enclosing values as hidden arguments; writing back to captures is rejected. +Arrays follow the same construction/publication protocol. There are no modules +or class singletons in this snapshot. + +Review on 2026-10-22 removed an unnecessary representation detour. Constructor +fields now stay in local SSA values and ordinary Phis; helpers emit ordinary +Stores at exit, with precise aliases gathered in MemMerge. Private memory is +only the handoff between helpers and publication. No global escape discovery +is needed to build that representation. + +Construction and escape discovery need not be taught in this order. Keep the +current order provisionally, without inventing a constructor example to claim +that dependency. The callback example now lives in the EA chapter. + +## Modules before separate compilation + +Teach module semantics with all source files compiled in one invocation. +Include lazy initialization, parent requirements, and cycle checking there. +Separate compilation then adds serialized ideal IR and types, GlobalBits +identity remapping, imported type reconstruction and analysis, and Unlink +before machine compilation. Extend initialization checks across imported +bodies, preserving opaque CLInit nodes until lowering. + +Keep code ownership distinct from unknown-caller edges in both chapters. +Keeping a function for emission must not imply arbitrary callers can reach it. + +## Appendices and future work + +- Native library construction: numeric C data symbols, address handling, + modular sys, runtime and driver support, and larger examples. Preserve the + native functionality already taught in earlier chapters. +- Optimizer and backend tuning: existing cloning/inlining choices, spill + ranking by live-range area and copy cost, rematerialization, and splitting + policies. Compare equivalent programs when explaining costs. + +This split organizes implemented behavior. Deterministic inlining, argument +arrays, static pre-initialization of classes, CFG frequency estimates, and +constant instance-field storage remain future work. See the +[roadmap](../docs/roadmap.md). + +## Renumbering and validation + +The first runnable extractions are +[SCCP Within a Function](../chapter24a/README.md) and +[SCCP Across Functions](../chapter24b/README.md), followed by +[SSA Before Types Are Known](../chapter25/README.md), +[Construction and Private Memory](../chapter26/README.md) and +[Escape Analysis and Unknown Callers](../chapter27/README.md), and +[Modules and Lazy Class Initialization](../chapter28a/README.md), and +[Separate Compilation](../chapter28b/README.md). +All seven main snapshots are extracted and numbered. Chained comparisons now +start in Chapter 23 beside short-circuit expressions. The linear-history +publisher follows the numbered directories, including lettered pairs. + +Validate each intermediate compiler with its own Make suite, including execution +checks. Keep declarations, conversions, diagnostics, and type bounds consistent; +do not weaken checks to make an intermediate snapshot pass. Rebuild serialized +libraries when their compiler or object format changes. + +## Escape snapshot ready for boundary review + +`chapter27` adds function/alias exposure sets to TypeMem and closes the +Start/Stop feedback cycle during SCCP. Known calls propagate memory summaries; +exposed returns, fields, external arguments and native calls discover further +addresses. Native calls contribute even on paths with no live Return. Function +addresses kept entirely in known code no longer imply unknown callers. + +The [recursive callback example](../chapter27/README.md) now belongs in +that chapter. Its base parameter becomes 2 while recursive calls remain. +Returning the callback or a containing object, or handing it to native code, +restores conservative arguments. FinalizeIR separates those analysis facts +from the roots needed to emit code. + +Construction still keeps local field values and rejects early exposure of self. +Both runnable snapshots are available for Cliff's promised review/play session. +This extraction does not choose broader rules for partially initialized objects. +No modules, class singletons, serialized IR or identity remapping are included. + +## Modules snapshot ready for review + +`chapter28a` compiles all discovered source files together. It adds class +singletons, namespaces and export filtering, parent initialization, CLInit memory effects, +cycle diagnostics, runtime lowering, and shared native class storage. Source +discovery reads parents as well as referenced children. The bundled modular sys +is supplied as source; building reusable libraries remains appendix material. + +There is no Serialize phase, object reader, GlobalBits remapping, or imported +IR. LocalIds assigns dense IDs within the invocation. FinishCalls detaches +analysis call edges after loop analysis, before scheduling or instruction +selection. ELF emits all discovered live bodies in the selected entry object. + +The preceding construction representation remains: field values and loop Phis +stay local, with Stores at helper exits. Constructor reads may see an unfinished +object, but may never read an uninitialized field. The module snapshot checks +this separately from completion and publication. Escape analysis still restores +unknown callers when a private callback reaches native code, including on paths +with no Return. + +The README's lazy lookup-table example is executable. Existing tests also cover +source/type cycles, rejected initialization cycles, parent order, native cold/hot +calls, shared storage, and ARM/RISC-V encoding. The construction/escape border +remains open to Cliff's review; this extraction does not settle broader rules for +publishing unfinished objects. + +## Separate compilation snapshot ready for review + +`chapter28b` adds saved types and ideal IR, `GlobalBits` +identity remapping, object discovery, imported bodies, and `Unlink`. It carries +Modules' deferred constructor parsing, local final-field values and loop Phis, +converted private memory summaries, native callback exposure, and source-parent +initialization order. Source privacy enforcement remains a separate review. + +The runnable example compiles a library and a client from separate source roots. +The client allocates an imported struct whose constructor updates a final field +in a loop, then calls a retained library function that updates shared state. +The regression checks evaluation under three seeds and native cold/repeated +calls. A second import test preserves unknown callers for a callback passed to +native code. Existing import tests retain initialization-cycle diagnostics, +shared class data, public type reconstruction, and native function thunks. + +Serialization now saves the class declarations of owned nested structs even +when they are absent from the optimized graph's type closure. Otherwise a +client can load the instance type but cannot find its allocation signature. +An explicit source input is always recompiled; dependency objects may still +be reused. The old incremental-rebuild experiment remains ignored. + +All seven main teaching boundaries now have numbered snapshots. Broader +language choices and compiler improvements remain in the review queue. diff --git a/skills/chapter25-codex-notes.md b/skills/chapter25-codex-notes.md index 9b861a18..be9768f8 100644 --- a/skills/chapter25-codex-notes.md +++ b/skills/chapter25-codex-notes.md @@ -1,12 +1,21 @@ -# Codex notes for Simple Chapter 25 +# Codex notes for Simple This file is intentionally AI-facing. It records durable invariants, debugging habits, and project-owner preferences that are easy to miss when reading only the implementation. Check current code and Git state rather than treating old session handoffs as live status. -For cross-chapter work, read [the backport queue](../docs/chapter-backports.md). -It owns pending work; these notes capture reusable lessons. Completed work and -superseded validation history remain in Git. +The [roadmap](../docs/roadmap.md) owns unfinished compiler work and language +proposals. These notes capture reusable lessons. Completed work and superseded +validation history remain in Git. +The [completed split](chapter-split.md) maps the former combined 24/25 +to 24a/24b (SCCP), 25 (incomplete types), 26 (construction), 27 (escape), and +28a/28b (modules/separate compilation). The old combined snapshots are retired. +Earlier entries below retain historical chapter numbers and test names; old +Chapter 25 implementation references describe the lineage now in 25–28b, with +the complete implementation in 28b. Use current paths and teaching boundaries. +The construction/escape border remains open to semantic review. +The [incremental build design](incremental-builds.md) expands the roadmap's +dependency and output-ownership requirements. For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.md). ## Collaboration preferences @@ -56,6 +65,69 @@ For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.m ## Mutability and chapter boundaries +- Prefix updates must share ordinary name resolution: a method's bare field + name denotes memory, while constructor fields may still be local SSA values. + Do not update the declaration's scope slot before resolving that distinction. + Keep the returned value alive across Store folding; a narrow Store can remove + a conversion it does not need, while the prefix expression still needs it. + +- Separate compilation retains the Modules parser's constructor local values + and escape fixes while adding saved IR and import ownership. Save owned nested + struct class declarations even when graph types omit them: callers need the + allocation and constructor signatures before bodies load. Preserve the nearest + file owner so imported children are not claimed by a compiled parent. The + library/client example checks three seeds and repeated native calls; native + callback exposure is also checked after saving and loading the library. + +- Final data fields are writable constructor locals, including in loops; final + local bindings and shared function declarations stay fixed. Do not classify + a constant instance default as a shared field or skip its loop Phi merely + because the published field is final. Class constants keep their static image. + +- The modules snapshot is source-only: LocalIds reserves class slots within one + invocation; ParseAll reads discovered parents and children; FinishCalls removes + analysis call edges before scheduling/selection. No Serialize, ElfReader, + GlobalBits or imported type reconstruction. Keep the constructor and native + exposure cases in BoundaryTest when extracting separate compilation. +- A symbolic Store with alias 1 is a barrier to Load forwarding: until its + field resolves it may write the queried alias. Skipping it can turn a byte + compound assignment at 255 into a stale read of 255 instead of 2. Both + incomplete-types and construction keep this barrier; exercise worklist seeds. +- Constructor fields stay in local SSA values and ordinary Phis. Emit ordinary + initializing Stores at helper exits and package precise aliases in MemMerge. + Private memory carries fields between helpers and into publication; it does + not carry each loop update. Keep converted field values in private summaries + and keep unwritten fields visible to completion checks. Constructor bodies + are parsed after the field list closes, so later fields also have local slots. + The discarded InitStore/InitPhi design must not be brought into the EA split. + Constructor reads may see an unfinished object, but never an uninitialized + field. Exact self aliases must read the current local value, including inside + loops, rather than loading the allocation before exit Stores have run. + Construction privacy follows language rules; global escape discovery is a + separate topic, with no required teaching order between them. + +- The escape snapshot carries function/alias exposure sets in TypeMem. Stop + feeds externally visible results/memory back to Start; SCCP restores unknown + callers only for exposed function identities. Their parameters use declared + bounds, never the narrower values inferred from known calls. External callers + already hold their object arguments, so writes through them expose addresses. + Native calls are exposure roots even when their path has no live Return. + FinalizeIR freezes this feedback before replacing call edges with emission + roots; the analysis cycle is not runtime CFG. Keep construction's local SSA + representation and early-self restrictions for the planned boundary review. + +- Native function addresses in AMD64 ELF use low-address tail-jump thunks. + Static fields and computed addresses must name the same weak entry across + objects; direct calls can still use the native PLT. A native FunPtr has no + local FunNode, so Encoding must check externFunc before the dead-function + identity fallback. Do not suppress relocation overflow checks or make class + storage read-only to accept a narrow dynamic relocation. Preserve constant + initialization Store folding. See StaticDataTest.testNativeFunctionIdentity. +- Long-term pointer direction: compact code and data arenas, which may be + separate; full-width native pointers have separate types and lifetime rules. + Keep static class images viable for ROM and eventual removal of initialization + checks. These are future changes, recorded in the Chapter 25 roadmap; the + native thunk fix keeps current four-byte function/eight-byte data pointers. - The class-init-cycle project is closed. Cycle checking, diagnostics, serialization/import handling, and runtime lowering are complete for the agreed scope. Argument arrays and initialization optimizations remain @@ -123,7 +195,7 @@ For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.m up front; ordinary Load/FldOff nodes resolve forward receivers. The fourth CLInit input is always present in a constructed graph. If the receiver resolves to an instance, CLInit and its speculative flag load disappear together. - Precise alias recovery can still be deferred. See `chapter25/README.md`. + Precise alias recovery can still be deferred. See `chapter28a/README.md`. Runtime lowering is implemented above; argument arrays remain deferred. Public function fields on instances of public classes must retain Start, just like class-object fields. publicFIDX now checks both; this fixes sys.scan.match losing Start @@ -167,6 +239,11 @@ For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.m - Resolving a forward struct reference must preserve pointer access/nullability. Phi idealization should report progress only when the referenced struct changes, not merely because the canonical declaration has different pointer permissions. +- In chapter25, complete named declarations as a group before + upgrading old graph types. Update stored declarations in New, Parm, Convert, + and similar nodes as well as their cached result types. A nested undeclared + struct must remain open, not become an empty finished definition. Unknown + expression types are distinct from explicit missing-initialization sentinels. - Construct cyclic pointers with their access bit already set before interning; creating a provisional pointer and then changing access can orphan temporary cyclic types. Serialized objects use the unversioned `C0DE` header; rebuild @@ -174,6 +251,10 @@ For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.m ## Tutorial backports +- Chapter 18 is the evaluator cutover: use `Eval2` and the compiler's schedule + from that chapter onward. The old `evaluator/Evaluator` and its private + `Scheduler` remain only in Chapters 10-17b; do not forward-port or revive them. + - Shared ISA encoding lives in `isa/` for Chapters 21-25: x86 bytes behind `CodeSink`, ARM/RISC-V word packing, and test-only evaluators. Chapter code still owns machine nodes, register masks, operand/type interpretation, layouts and @@ -359,13 +440,13 @@ For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.m top-level runner, e.g. `make -k tests CHAPTERS="chapter10 chapter11 chapter12"`, with all affected directories explicitly listed. Establish the destination baseline before changing it; older Makefiles may need a forced rebuild after API changes. -- Keep unrelated discoveries separate in `docs/chapter-backports.md`. Completed - fixes leave the pending queue; record validation in the change description. - Keep unresolved discoveries in the actual queue. A green suite +- Record confirmed unresolved bugs and future work in `docs/roadmap.md`. + Remove completed items and record validation in the change description. + Do not turn an unconfirmed textual difference into a backport task. A green suite after changing graph/test order does not prove an intermittent failure fixed. - Store disposable probes/logs under an appropriate ignored build directory. Preserve durable reproducers in tests or the review record, not only scratch - files. The B13 `chapter25/tmp` probes were removed at Cliff's request after he + files. The B13 `chapter28b/tmp` probes were removed at Cliff's request after he reviewed and pushed the fix; future sessions must not rely on those files. ## Dominator caches and searches @@ -592,7 +673,7 @@ For shared viewer work, read [the graph viewer notes](graph-viewer-codex-notes.m with documented constructor, receiver, library and mutability adaptations. These are allocation/legality replays, not historical runtime harnesses. Its own cohort includes current Chapter25Test allocations and a fresh library - encoding. See `chapter25/src/test/java/com/seaofnodes/simple/spill/README.md`. + encoding. See `chapter28b/src/test/java/com/seaofnodes/simple/spill/README.md`. Do not silently use current inherited tests as though their sources were frozen. - Rebuild and measure the system library even if sys.o is up to date. In the final 25 review, the older ranking and a multi-use-clone preference looked @@ -750,6 +831,13 @@ Consequences: ## Types and monotonicity +- Chapters 24a/24b retain typed fallback results for erroneous or unresolved + Loads and arithmetic. Chapter 25 introduces the surrounding Convert nodes + and removes parser-supplied bounds. Do not backport BOTTOM-on-error in + isolation: parser, conversions, loop bounds, and diagnostics must agree. + Preserve `testBrainFuck`, `testWhileWithNullInside`, `testNullGuardErrors`, + `testLinkedList0`, and `testCoRecur`, including their expected errors. + - Integer widening starts in Chapter 24. `nonZero()` preserves `_widen` when narrowing a range. B10's correction is in 24, with `TypeTest.testNonZeroWidening` in 24-25. Both now use widening 0-3 and dual widening `3-widen`, preserve @@ -850,6 +938,11 @@ Consequences: checking rather than a pre-Opto deletion. - Extern C functions have no Simple FunNode/Return and are handled only while linking. +- From Chapter 22, retain native symbol names by function identity in CodeGen. + Selection and spill copies can turn ExternNode into a plain ConstantNode; + address relocation must consult that identity before looking for a local + function. ARM ADRP splits a signed page delta into unsigned instruction + fields: mask both pieces, including the high piece for backward addresses. - Unknown-caller Start edges and post-Opto compilation-unit ownership are separate concepts. Do not overload Start edges merely to keep surviving functions attached to a compilation unit. @@ -987,8 +1080,8 @@ serialize/deserialization bijection after persisted graph changes Default fuzzer wrappers use explicit regression seed lists; exploratory fuzzing is opt-in. A passing wrapper with an empty list is not exploratory coverage, and `OPEN_FAILING_SEEDS` remain unresolved until explicitly verified and promoted. -For dated suite counts and known failures, consult the backport validation record -rather than treating an old count as the current baseline. +For dated suite counts, consult the validation record in Git history; current +known work is in the roadmap. Do not treat an old count as the current baseline. Peepholes are always enabled in every chapter, by Cliff's request on 2026-09-26. Do not reintroduce an unoptimized compiler mode. Incomplete Loop/Phi nodes still @@ -1009,7 +1102,7 @@ comparisons as equal runtime results. the SUB, not copy its inputs. ARM register 31 is XZR for load destinations, so narrow loads use WMASK, never RMASK. Logical-not uses CMP to XZR followed by CSET to its result register. Shared emulator fixes and Chapter 21-25 - backports are recorded in `docs/chapter-backports.md`. + backports are recorded in Git history. - Cohort 23's dynamic increase from compiler 24 to 25 is primarily class-field initialization: Ops 630->1106. RA 160->442 includes 224 extra pointer-address and 56 integer rematerialization instructions, but only two extra splits. diff --git a/skills/current-spill-matrix.md b/skills/current-spill-matrix.md new file mode 100644 index 00000000..ee77a77a --- /dev/null +++ b/skills/current-spill-matrix.md @@ -0,0 +1,14 @@ +# Current allocator comparison + +`make spill-stats` measures Chapters 20, 21, 22, 23, 24b, and 28b against the +historical test cohorts. Cohort IDs and test names retain their old numbers; +column headings identify the current compiler directories. + +These are separate from the dated [pre-split measurements](../docs/regalloc-spills.md). +After a successful run, `python build-support/spill-matrix.py --reuse --update-docs` +updates only this table. Fresh logs use 24b/28b names so old 24/25 results cannot +silently become measurements of new compiler snapshots. + +<!-- spill-matrix:start --> +No post-split measurement has been published yet. +<!-- spill-matrix:end --> diff --git a/skills/graph-viewer-codex-notes.md b/skills/graph-viewer-codex-notes.md index 0393cca9..645efdd8 100644 --- a/skills/graph-viewer-codex-notes.md +++ b/skills/graph-viewer-codex-notes.md @@ -4,8 +4,8 @@ Updated 2026-09-26. Read graph/README.md for the current design and protocol. These are AI-facing continuity notes, not a new task list or authority to change compiler semantics. Cliff declared the viewer task complete on 2026-09-26. Peephole animations, inlining display, TypeCheck diagnostics, and arrow-key -playback are committed. Resume the independent bug-fix backport review from -docs/chapter-backports.md when requested. +playback are committed. Unfinished compiler work is tracked in +docs/roadmap.md. ## Working tree and latest change @@ -48,7 +48,7 @@ passed for real TypeCheck errors in chapters 10, 17, 18, 25, including final Nex folded error nodes, backward playback, direct jump, and a subsequent valid compile. Dead erroneous code was removed before TypeCheck. Existing JUnit checks passed: chapter10 Chapter10Test (27), chapter18 Chapter13Test + Chapter18Test (39), -chapter25 Chapter13Test (16). This was NOT an all-chapter full test-suite run. +chapter28b Chapter13Test (16). This was NOT an all-chapter full test-suite run. ## Durable design and owner preferences @@ -146,7 +146,7 @@ capture server on an ephemeral port. Scratch helpers may disappear after cleanup sibling-Phi exclusion, nested replacement ordering. Some older scratch scripts assumed adjacent capture frames and are stale after atomic peep playback. - build/check-folding-state.py: inlining FOLDING labels; folding-frames.json is a - 349-frame chapter25 capture predating the added final TypeCheck frame. + 349-frame chapter28b capture predating the added final TypeCheck frame. - build/check-graph-finish.py defines capture(ch) before its chapter05 invocation; helpers reuse that prefix. build/FinishFrames.java (build/graph-finish classes) calls GraphCapture.run on an ephemeral socket without launching a browser. diff --git a/skills/incremental-builds.md b/skills/incremental-builds.md new file mode 100644 index 00000000..050bd1ba --- /dev/null +++ b/skills/incremental-builds.md @@ -0,0 +1,75 @@ +# Incremental builds + +Implementation notes and remaining build-driver work, reviewed 2026-10-25. + +## Current reuse check + +Chapter 28b's `CompUnit` decides whether to reuse a local object by walking its +recorded dependencies before loading saved types or IR. It checks all sources +owned by each local object, missing inputs, and dependency object timestamps. +Dependencies use the same local/library resolution for checking and loading. + +For example: + +1. A depends on B and has inlined some of B. +2. B's source changes, leaving B's object unchanged. +3. A client discovers A. The check follows A's recorded dependency to B and + finds B's newer source, even though B's object has not changed. +4. A is reparsed, so its saved inlined copy of B is never accepted. + +Use a fresh visited set for each candidate object. Cycles terminate safely, +but a provisional success inside a cycle must not be cached for another root. +Dependencies owned by the same object are checked through its bundled sources. +An unchanged public signature is insufficient: Simple imports implementation +details for optimization as well as declarations for type checking. + +Object-only libraries are still supplied as a coherent build. Do not reject +them merely because one dependency has a newer installation timestamp: existing +tests compile cyclic libraries against stubs and then combine their real +objects. Validating those combinations requires saved input versions, not a +guess from file times. A newer library object does invalidate a source-backed +local dependent that can be rebuilt. + +`IncrementalBuildTest` covers unchanged reuse, transitive source edits, bundled +cycles, missing inputs, library dependencies, and clean/incremental native +execution parity. It sets file times directly, without sleeps. + +## Required pieces + +- Plan object targets and rebuilds across invocations. Handle cycles as groups + and newly discovered dependencies. The compiler's per-object reuse check is + not a complete scheduling or linking policy. +- Record actual input identities. An object can own several source units, so + track all of their sources and resolved library paths and versions. A search + path change can select a different library even with an older timestamp. +- Record compiler and serialization versions, target, ABI, and relevant + options. Content hashes would also detect changes with preserved timestamps. + Specify which changes require rebuilding source and which reject an + incompatible source-free library. +- Make build targets match output ownership. The old ignored `testModule0` + expects separate A.o and A/B.o, while the current compiler can bundle several + units into one object. The target must identify the complete set it owns. +- Provide an "ensure current" build operation. Explicit source compilation + currently always recompiles. An unchanged build should do no compilation or + linking; a changed object or link input should trigger relinking. +- Publish successful outputs without leaving a partial object accepted after + failure, and keep the recorded inputs consistent with those outputs. + +Initially, rebuilding every dependent of a changed object is a reasonable +conservative rule. Fine-grained interface/body fingerprints can come later. + +## Suggested first implementation + +Put a small build driver around the compiler with explicit object targets, +recorded inputs, and dependency propagation. Keep individual compilation +requests straightforward. A clean build and an incremental build must produce +the same observable program behavior. + +Cover unchanged builds, direct and transitive source edits, inlined constants +and bodies, cyclic dependencies, bundled sources, removed inputs, library-path +changes, compiler/target changes, and failed builds. Use controlled timestamps +or hashes instead of sleeps. Replace the ignored historical experiment with +tests matching the chosen object ownership model. + +Selective library IR loading is separate future work. It can reduce compile +cost, but is not required to make conservative incremental rebuilding correct. diff --git a/skills/japanese-translation.md b/skills/japanese-translation.md new file mode 100644 index 00000000..39383345 --- /dev/null +++ b/skills/japanese-translation.md @@ -0,0 +1,22 @@ +# Japanese translation / 翻訳の保守方針 + +English is authoritative. The contributed Japanese translation is maintained on +a best-effort basis; English edits and chapter reorganizations never wait for +translation updates. Move Japanese files and update navigation when renumbering +chapters; prose updates can follow separately. Contributions are welcome, without +an ongoing maintenance obligation for the original translator. + +[英語版](../README.md)が正式な文書です。この日本語訳は寄贈されたもので、可能な範囲で保守されますが、 +英語版より更新が遅れることがあります。英語版の編集や章の再構成を、翻訳の更新待ちにすることはありません。 +章を移動・改番する際は、日本語ファイルも一緒に移動して章の見出しやリンクを更新し、本文の翻訳は後から更新できます。 +翻訳の改善は歓迎しますが、元の寄贈者に継続的な保守を義務付けるものではありません。 + +初回の翻訳は英語版のコミット +[`81b654d7`](https://github.com/SeaOfNodes/Simple/tree/81b654d76444d747fdf1d053760e507b50f26e18) +を基にしています。2026年10月の目次・文法見出し・コード例・命令エンコーディング・ +`C0DE` ヘッダに関する文書修正も反映しています。 +各文書の English リンクは現在の英語版を指します。更新されていない箇所の原文を確認するには、 +上記コミットの対応するファイルを参照してください。新しい章が未翻訳の場合は、その旨を示して英語版にリンクできます。 + + +[日本語の紹介に戻る](../README.ja.md) diff --git a/skills/stage-audit.md b/skills/stage-audit.md new file mode 100644 index 00000000..194a5178 --- /dev/null +++ b/skills/stage-audit.md @@ -0,0 +1,113 @@ +# Chapter split cleanup audit + +This record distinguishes checked behavior, existing gaps, and future designs +while retiring the old combined Chapters 24/25. The final mapping is in the +[split record](chapter-split.md). Historical Java test names are retained as +regression identities; they do not identify current chapter boundaries. + +## Corrections in this cleanup + +- Chained comparisons now begin in Chapter 23. The early SCCP implementations + incorrectly compared a boolean intermediate with the third operand: + `10 < 30 < 20` returned true. Later implementations preserved the middle + operand but reused the first operator, making `0 < 2 <= 2` false. All current + snapshots from 23 onward now compare adjacent values using each written + operator. Regression cases cover both directions, strict/non-strict mixtures, + single evaluation, side effects, short-circuiting, and mixed-direction errors. +- Chapter 23's inherited, explicitly described fuzzer scratch contains many + comparisons using the former left-associative grammar. It is retained as an + ignored `testJigFuzzer`, consistent with the scratch-test policy; its mixed + directions are no longer valid source. Later snapshots already omitted it. +- Native-library and tuning material is preserved in [appendices](../docs/appendices/README.md), + and unfinished work is collected in the [roadmap](../docs/roadmap.md). +- The recursive-return diagnostic already passed. It remains enabled in the + earlier snapshots and is enabled again in 28a/28b. The mutual-recursion test had an obsolete graph-string + expectation; it now checks its existing four execution results in 24b–28b. + In 24a, local SCCP cannot infer the recursive return type; the same example + checks that conservative diagnostic instead. This makes the boundary explicit. +- Allocation/runtime measurement runs passed after the comparison correction; + the changed spill goldens record the resulting smaller graphs. Historical + published tables retain their original compiler labels and dates. + +## Prefix-update follow-up + +Prefix `++` and `--` now use the same name lookup as ordinary reads and postfix +updates in Chapters 23 through 28b. A method's field goes through memory; a +constructor-local field stays in SSA. Previously, the prefix path updated the +field declaration's parser slot and silently lost the write. The two-call +reproducer now returns 21 instead of 11. Narrow update results stay alive while +Store folding drops a redundant conversion from its own input. + +`PrefixUpdateTest` covers increment/decrement, integer/float/byte fields, +multiple worklist seeds, local shadowing, final-field errors, native byte +wraparound, and module fields where supported. Existing final-constructor +coverage includes prefix updates in loops. + +The focused Linux checks pass in all eight affected chapters, including native +SystemV byte-wraparound execution and the existing final-constructor tests. +The early final-field rejection is retained so an illegal prefix update never +enters the optimizer as a Store; native data bindings still update their writable +contents despite having a fixed address. + +The Windows full suites cover 4,169 tests plus eight fuzzer regression runs; +shared printer and ISA checks pass. Focused reruns cover the final diagnostic +and prefix-result lifetime adjustments. The full Chapter 27 suite was rerun +after restricting the temporary keep to prefix results; its existing spill +expectations remain unchanged. No allocator goldens were updated. + +## Implemented, with narrower limits than earlier prose suggested + +The constructor-loop assertion is fixed in Chapters 25–27. Load forwarding now +waits for a stored loop Phi to acquire a type at least as precise as the Load, +and records a dependency to retry. The reduced case needs only a conditional +assignment in a loop; break/continue and final fields are not required. +Regressions cover the reduced case and the original constructor loop across +worklist seeds and execution inputs. Chapters 28a/28b already had the guard; +the corresponding earlier-chapter probes pass in 23/24a/24b. + +- Separate compilation imports types and optimized IR without source parsing; + it reruns analysis and type checking with the new callers. +- Loading is by object and recorded dependencies, not individual used members. + A call to `sys.io.p` loads all ten bundled sys units. Independent, unrelated + objects can remain unloaded. Loading does not imply class initialization. +- Timestamp checks do not yet provide a reliable incremental build system. + The [design note](incremental-builds.md) records the missing pieces. +- Underscores select exports in modules; source privacy checks are still a + proposal awaiting review. Do not treat hidden exports as inaccessible source. +- Constructor-local final fields use SSA values and exit Stores. Reads of an + unfinished object are allowed only for initialized fields. Passing unfinished + self to arbitrary helpers and accepting completion only after SCCP are not + promised by the current chapter boundary. + +## Existing failures and deferred validation + +- The ideal evaluator does not truncate narrow stores in Chapter 23. A byte + update can therefore disagree with generated code; the postfix-only + reproducer is retained in the [roadmap](../docs/roadmap.md#narrow-stores-in-the-ideal-evaluator). +- `testModule0` is an ignored historical incremental-build experiment. It + assumes per-source output objects and unchanged explicit-input builds, neither + of which matches the present compiler's contract. +- Serialization's expensive byte-for-byte round-trip assertion remains disabled. + Ordinary import tests do not establish that stronger property. +- Direct probes of the remaining ignored tests found a stale forward-method + graph expectation and a missing `progs/sieve.smp` fixture in 28b. These and + the interactive stdin test remain ignored. Ignored Jig and + random-fuzzer entry points are intentional scratch/long-running tools. + Audit these separately before advertising their behavior as tested. + +Selective loading, full incremental builds, source privacy, compact pointers, +argument arrays, deterministic inlining, and broader constructor rules remain +future work. They are not prerequisites for completing the chapter numbering. + +## Validation + +The affected snapshots (23, 24a, 24b, 25, 26, 27, 28a, 28b) were checked with +their full Make suites on Windows and Linux, plus focused reruns after updating +test expectations. The final Windows coverage is 4,143 tests plus eight fuzzer +regression runs, with the shared printer and ISA tests also passing. The +native library/client cases and final Chapter 28b release build pass on both OSes. + +A local linear-history dry run produced all 31 tags in teaching order. The +final source snapshot, chapter navigation, Maven module IDs, and reviewed local +documentation links were checked. Measurement capture in 24b and dynamic +measurements in 24b/28b pass; the historical published tables were not rewritten. |