summaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorCliff Click <cliffc@acm.org>2026-09-19 10:58:34 -0700
committerCliff Click <cliffc@acm.org>2026-09-19 10:58:34 -0700
commita78b1bd06f79652e268b1a530d579b0faf665031 (patch)
tree8d6f8a72d6c150e96165ee706b2d80f66f16d70a /docs
parent0c62c5a642132fc8b2de4905bb4adce31069ece1 (diff)
downloadSimple-a78b1bd06f79652e268b1a530d579b0faf665031.tar.gz
Simple-a78b1bd06f79652e268b1a530d579b0faf665031.zip
AI skills update
Diffstat (limited to 'docs')
-rw-r--r--docs/chapter-backports.md352
1 files changed, 88 insertions, 264 deletions
diff --git a/docs/chapter-backports.md b/docs/chapter-backports.md
index 1828b16b..09f1a14e 100644
--- a/docs/chapter-backports.md
+++ b/docs/chapter-backports.md
@@ -23,7 +23,6 @@ the correction into the chapter's representation, not the entire modern file.
| B10 | Preserve widening state in integer nonzero refinement | **Audit** first chapter with both widening and `nonZero`; no later than 24 | Chapter 25 `TypeInteger.nonZero` preserves `_widen`. Check lattice laws and loop refinement. Exclude TypeScalar, storage-type, and serialization changes. |
| B11 | Diagnose return types using the optimized return expression | 18; 19 already has the correction | `ReturnNode.err()` uses `expr()._type` instead of the parse-time `mt` aggregate. Chapter 18 rejects `struct S { u8 x; }; return new S; return 0;` with a mixed integer/reference error; Chapter 19 accepts it. Also test genuinely reachable incompatible returns. This is a candidate, not yet an applied or isolated-patch-verified fix. |
| B12 | Encode the actual destination register for two-address immediate multiply | 21 | `MulIX86` inherits `ImmX86.encoding`, whose ModRM.reg is fixed to zero and whose REX.R is clear. With source/destination both `rcx`, multiplying by 11 emits `48 6b c1 0b` (destination `rax`) instead of `48 6b c9 0b`. With both `r9`, it emits `49 6b c1 0b` instead of `4d 6b c9 0b`. Give multiply its proper register fields while preserving Chapter 21's two-address allocation contract; do not change the opcode-extension fields used by other `ImmX86` subclasses. Later chapters use a separate multiply encoder. |
-| B14 | Check leaf type kinds before cyclic equality | 23-24; 25 already checks | `Type.cycle_eq` calls leaf `eq` without checking `_type`. Intern a struct with a `Type.BOTTOM` field, then same-named structs with float constants 1.0 and 2.0; meeting the latter throws ClassCastException in `TypeFloat.eq`. Reproduced independently in 23 and 24; the adapted probe passes in 25. Add a deterministic TypeTest before backporting the type-kind check. This is separate from B13; no type implementation changed. |
After the first two reviews, batch only corrections with established independence
and regressions. Keep one logical correction per commit across affected chapters.
@@ -77,274 +76,99 @@ starts, so run `make lib` separately before `make tests`.
## Validation record
-### Issue #246: Chapter 25 reproduction and local correction
-
-On 2026-09-19, rebuilt Chapter 25 with
-`make build/classes/main/.mtag build/classes/test/.ttag` and ran the following
-through `new CodeGen(src).driver(CodeGen.Phase.TypeCheck)` with Java assertions
-enabled. Before the correction it failed in `MemOpNode.err()` with
-`Might be null accessing 'x'`.
-The original issue's `new Point { x = 42; }` syntax is no longer accepted;
-only object construction was adapted to Chapter 25's constructor syntax.
-
-```text
-struct Point { int x; new Point = { int v -> x = v; }; };
-Point?[] !points = new Point?[2];
-points[arg] = new Point(42);
-Point? p = points[1];
-if (p != null)
- return p.x;
-return -1;
-```
-
-That failure prevented evaluation of either expected runtime result.
-
-Debugging isolated the first bad phase to parsing. `Parser.parseEquality`
-builds `!(p == null)`, and `BoolNode.EQ.idealize` rewrites the inner comparison
-to `!p`. `ScopeNode._addGuards` records nonzero `!!p` and zero `!p` on the
-true arm, but does not descend again to record non-null `p`. Meanwhile,
-`CProjNode.idealize` strips the negations from the control test. The resulting
-graph therefore tests `p` directly without refining the pointer used by `p.x`.
-
-With default seed 126, the original's `LoadNode#150` uses nullable
-`ReadOnlyNode#126` directly after Parse, Iter, and Opto. Changing only the
-condition to `p` produces `GuardNode#132` on that same pointer, consumed by
-`LoadNode#135`, and returns `-1` and `42` as expected. Explicit `!!p` fails;
-`if (p == null) return -1; return p.x;` passes with both expected results.
-The original failure and successful `if (p)` control were also checked with
-seeds 1, 42, and 123, with assertions enabled.
-
-The local correction recurses into a Not operand when it is another Not or a
-short-circuit Phi, flipping the proven truth. Existing single-negation handling
-remains in place; its guards need not be duplicated. Each recursive call still
-checks dominance/schedulability, and the predicate is kept alive while recursive
-peepholes run. Boolean expression values retain their original meaning.
-
-`Chapter10Test.testNullGuards` covers both comparison orders,
-nested negations, early returns, a reused Boolean value, and negated
-short-circuit logic. Each case checks both runtime outcomes and scheduling with
-seeds 1, 42, and 126. `testNullGuardErrors` verifies rejection of
-unguarded access, a check on a different pointer, and a guard used beyond its
-branch. The positive regression failed before the compiler change.
-
-Final `make tests` passed all 427 tests with assertions enabled: 362 raw0,
-32 raw1, 7 standalone, 18 system, 1 fixed-seed fuzzer wrapper, and 7 remaining
-tests. The build regenerated `sys.o`. Log: `chapter25/build/issue246-tests.log`.
-A broader intermediate version that duplicated single-negation handling exposed
-a Dijkstra SCCP assertion; it was not retained. The original compiler passed
-Dijkstra in a comparison run with freshly rebuilt `sys.o`, and the final
-correction passes it too. No assertions or existing test expectations changed.
-
-Separate pre-existing diagnostic concern: using the same array setup with
-`if (p == null) return p.x; return -1;` was accepted before the correction,
-despite dereferencing null on the taken arm. This remains outside B13's fix.
-
-### B13: earliest chapter and regression placement
-
-Null-check refinement begins in Chapter 10, alongside nullable struct pointers;
-it is documented in `chapter10/README.md` and implemented by
-`ScopeNode.upcast`. Chapter 9 has no corresponding pointer guards. The reduced
-failure already reproduces in Chapter 10:
-
-```text
-struct Point { int x; };
-Point point = new Point;
-point.x = 42;
-Point? p = null;
-if (arg) p = point;
-if (p != null) return p.x;
-return -1;
-```
-
-The same program with `if (p)` passes. The array version reproduces in Chapter
-15, where arrays first appear; its `if (p)` control also passes.
-
-Substantial guard implementation differences before the backport:
-
-| Chapters | Representation and behavior |
-|---|---|
-| 10-16 | `ScopeNode.upcast` refines only values present directly in local scope inputs. A non-null pointer uses `CastNode(TypeMemPtr.VOIDPTR, ...)`; a zero/null fact replaces a local with a constant. There is no expression-guard table and no general nonzero integer refinement. |
-| 13 onward | The parser also refines the false arm without an explicit `else`, allowing a fact to survive an early return. Chapters 10-12 require an explicit `else` for that test. |
-| 17 | `addGuards`/`removeGuards` introduce a scoped predicate/cast table and `upcastGuard` applies facts to later matching expressions, including loads. Refinements use `nonZero`/`makeZero` and type-bearing CastNodes. |
-| 18-24 | The same scoped Cast architecture; `_addGuard` factors the two cases and rejects high joins. Chapter 23 adds short-circuit syntax, but these scopes do not recursively decompose its Phis as Chapter 25 does. |
-| 25 | GuardNode stores zero/nonzero intent and derives the value family from its input, supporting incomplete types. Scope guard discovery checks dominance, decomposes short-circuit Phis, and now follows nested negations. |
-
-The tests have been moved from Chapter 25's `Chapter25Test.java` to
-`Chapter10Test.java` and forward-ported into that same test class in every
-snapshot from 10 through 25. Chapters 10-14 use the reduced pointer setup;
-15-24 use arrays with the allocation syntax available there; 25 retains the
-constructor-based array setup, seed rotation, and scheduling checks. Boolean
-temporaries use `int` in earlier snapshots, early-return coverage uses an
-explicit `else` before Chapter 13, and short-circuit coverage starts at 23.
-The tests are active and assert successful guarded access, not the old error.
-
-The compiler backport now retains each chapter's existing guard representation
-and adds negation traversal there, as detailed below. Importing Chapter 25's
-GuardNode/incomplete-type architecture remains a separate design change.
-
-Before adding the tests, `make -k tests` passed in all snapshots 10-24.
-Log: `chapter25/build/issue246-review/baseline.log`.
-
-After moving/porting the tests, ran `make -k tests` across 10-25:
-
-- Chapters 10-23 each fail only `Chapter10Test.testNullGuards`, with
- `Might be null accessing 'x'`. The new rejection test and all existing tests
- in those suites pass. These were the exposed regressions before the compiler
- backport recorded below.
-- Chapter 24's isolated `Chapter10Test` runs 26 tests with only the same B13
- failure. Its full suite also exposes a type-equality failure, starting at
- `Chapter16Test.testSquare`: `TypeFloat.eq` receives a plain `Type` through
- `Type.cycle_eq`, causing a ClassCastException and later cascading failures
- (197 failures total, including B13). A forced `make -B tests CHAPTERS=chapter24`
- reproduces this, so a rebuild does not resolve it. No type implementation
- was changed. This additional failure must be reviewed separately.
-- Chapter 25 passes all 427 tests after relocation: 364 raw0, 32 raw1,
- 7 standalone, 16 system, 1 fuzzer wrapper, and 7 remaining tests.
-
-Logs: `chapter25/build/issue246-review/ported-tests.log`,
-`chapter25/build/issue246-review/chapter24-focused.log`, and
-`chapter25/build/issue246-review/chapter24-rebuilt.log`.
-
-### B13: completed compiler backport
-
-Applied locally on 2026-09-19 to every snapshot from Chapter 10 through 25;
-nothing pushed. The same paired-negation traversal starts in Chapter 10 and
-continues forward, with representation-specific adaptations:
-
-- 10-16 recurse within `upcast`, retaining local-binding casts and constants.
- No dominance machinery, expression-guard table, or modern type architecture
- is introduced.
-- 17-22 separate the public guard-set marker from recursive `_addGuards`.
- Nested negations add facts within the existing scoped CastNode table;
- recursive calls do not add extra scope markers.
-- 23-24 additionally use Chapter 25's short-circuit Phi decomposition, since
- short-circuit syntax starts in 23. A small `availableAt` helper walks data
- inputs and the existing CFG immediate-dominator chain to reject RHS-only
- values unavailable at the proposed guard. Phi availability is checked at
- its Region. This avoids importing 25's folding-aware `earlyCFG` machinery.
-- 25 retains its existing GuardNode, dominance, and short-circuit mechanisms;
- only the nested-negation discovery is added.
-
-All versions retain the original single-negation handling and keep the
-predicate alive during recursive peepholes. The added guards do not change
-the Boolean expression's value.
-
-The reused-Boolean regression also exposed `Eval2` casting a pointer to Long
-when evaluating Not in 18-24. Those test evaluators now share Chapter 25's
-null/integer/float-aware Not helper. Compiler type implementations are unchanged.
-
-`Chapter10Test` remains the home of the forward-ported regression in every
-snapshot 10-25. In 23-25, `testShortCircuitGuardScheduling` additionally checks
-an RHS-only call result through local scheduling and evaluates both branch
-outcomes and a zero result. Chapter 25's pointer cases retain seed rotation.
-
-Each full `make tests` target passes after the correction:
-
-| Chapters | Ordinary tests per chapter | Additional suites |
-|---|---|---|
-| 10-17 | 149, 164, 166, 181, 201, 212, 230, 282 | Each complete chapter target passed. |
-| 18-22 | 309, 350, 359, 374, 386 | Each also passed its fixed-seed fuzzer wrapper. |
-| 23-24 | 403, 425 | Each also passed its fixed-seed fuzzer wrapper. |
-| 25 | 428 total | 365 raw0, 32 raw1, 7 standalone, 16 system, 1 fuzzer wrapper, 7 remaining. |
-
-Commands were `make tests CHAPTERS=chapter10`, then `make -k tests` with
-`CHAPTERS='chapter11 ... chapter22'`, `CHAPTERS='chapter18 ... chapter24'`,
-and finally `CHAPTERS='chapter23 chapter24 chapter25'` (the ellipses here
-abbreviate explicitly enumerated chapter names). Logs under
-`chapter25/build/issue246-review/`: `chapter10-fixed.log`,
-`chapters11-22-fixed.log` (11-17 passed; 18-22 exposed the evaluator issue),
-`chapters18-24-fixed.log`, and `chapters23-25-final.log`.
-
-The separate cyclic-equality failure is not fixed by B13's green suites.
-It has a deterministic standalone reproducer in 23 and 24 and is queued as
-B14. The probe interns `Probe { x: BOTTOM }`, `Probe { x: 1.0 }`, and
-`Probe { x: 2.0 }`, then meets the latter two. It throws in `TypeFloat.eq`
-through `Type.cycle_eq`; Chapter 25's equivalent probe succeeds. This records
-the previously order-sensitive full-suite failure without expanding the guard
-fix into a lattice change.
-
-### Initial setup baseline
-
-On 2026-09-17, using Windows/Cygwin and OpenJDK 22.0.2:
-
-| Check | Result |
-|---|---|
-| All 25 chapter `tags` targets | Passed via `make -k tags`. |
-| All 25 chapter `release` targets | Passed, in two batches (1-22 and 23-25). |
-| `make -k tests`: Chapters 1-17, 19-21, 23-25 | Passed each complete chapter target. |
-| Chapter 18 | 300 ordinary tests passed; `fuzzPeepsSmall` failed on seed `973358943756616234`. This test chooses seeds from wall-clock time. Preserve the failure rather than rerunning until green. |
-| Chapter 22, initial run | `testLatticeTheory` failed with a TypeConAry/TypeRPC cast error. |
-| `make -B tests CHAPTERS=chapter22` | Passed after full rebuild: 369 ordinary tests and the fuzzer test. This is consistent with stale classes; the original failure remains recorded. |
-
-That initial test baseline was **not green** because of Chapter 18. The follow-up
-below records the explicitly requested change to default fuzzing coverage; the
-underlying compiler failure remains open.
-
-Chapter 25 initially could not run its system suite because the Hello World
-example was deleted in the working tree. Cliff restored it; the final full run
-passed (362 raw0, 32 raw1, 7 standalone, 16 system, and 1 fuzzer test).
-
-Build-only fixes made during setup: supply missing early Makefiles and release
-targets, alias Chapter 25's `TAGS` as `tags`, use the existing older-chapter ctags
-option spelling in 21-24, and correct malformed `$(file ...)` argument-list
-writes in 22-24. These changes do not move compiler behavior between chapters.
-
-Local logs (ignored build artifacts): `build/review/tests-verified.log`,
-`build/review/chapter22-rebuilt-tests.log`, `build/review/tags-verified.log`,
-`build/review/release-01-22.log`, and `build/review/release-23-25.log`.
-
-The Chapter 18 fuzzer printed this reduced input:
+Historical results below are dated evidence, not a substitute for a fresh
+baseline. Logs live in ignored build directories and may no longer exist.
+Reusable implementation lessons are in `skills/chapter25-codex-notes.md`.
+
+### B13 / issue #246: complete, 2026-09-19
+
+Cliff reports the correction pushed. Nested negation in `p != null` / `!!p`
+now discovers the underlying pointer guard in every snapshot 10-25, retaining
+each chapter's guard representation. Chapters 23-24 also decompose short-circuit
+Phis with availability checks; Eval2 in 18-24 now handles pointer/null Not.
+
+`Chapter10Test.testNullGuards` failed before the backport in 10-24 and passes
+afterward; `testNullGuardErrors` checks rejected uses. Both tests live in every
+snapshot 10-25. `testShortCircuitGuardScheduling` covers RHS-only call results
+in 23-25; Chapter 25 also rotates optimizer seeds. Full `make tests` passed in
+each directory 10-25. Ordinary test counts were 149, 164, 166, 181, 201, 212,
+230, 282, 309, 350, 359, 374, 386, 403, 425 in 10-24; Chapter 25 passed 428
+total. No assertions or existing expectations were weakened.
+Logs: `chapter25/build/issue246-review/`, particularly
+`chapters18-24-fixed.log` and `chapters23-25-final.log`.
+
+Independent findings retained for follow-up:
+
+- B14's cyclic-equality failure remained reproducible despite green B13
+ suites; it is corrected separately below.
+- A pre-existing Chapter 25 diagnostic concern remains outside B13: with the
+ issue's nullable array setup, `if (p == null) return p.x; return -1;` was
+ accepted despite dereferencing null on the taken arm. Original setup:
+ `struct Point { int x; new Point = { int v -> x = v; }; };`
+ `Point?[] !points = new Point?[2]; points[arg] = new Point(42);`
+ `Point? p = points[1];`. This needs separate investigation.
+
+### B14: complete locally, 2026-09-19
+
+Added the existing Chapter 25 leaf type-kind check to `Type.cycle_eq` in 23-24.
+Cyclic structural equality first appears in 23. Inspection of 9-22 found the
+older interned-child identity representation with kind checks in `Type.equals`;
+the equivalent BOTTOM/float-struct meet probe passes in 22 without a correction.
+No earlier compiler change is needed for B14.
+
+`TypeTest.testCyclicLeafKinds` in 23-25 interns same-named structs with `x`
+typed BOTTOM, 1.0, and 2.0, then meets the latter two. Before the correction it
+throws ClassCastException through `TypeFloat.eq` in 23 and 24; 25 passes.
+Afterward all three pass, checking F32 field type, canonical interning,
+commutativity, absorption by the BOTTOM-field struct, and the dual/join result.
+
+Validation with assertions enabled (Windows/Cygwin, JDK 21.0.4):
+
+- Unmodified baseline: `make -k tests CHAPTERS="chapter22 chapter23 chapter24 chapter25"`
+ passed each full target.
+- After a full Java rebuild of 23-24, the focused regression and full TypeTest
+ passed in 23-25 (6 TypeTest cases in each).
+- `make -k tests CHAPTERS="chapter23 chapter24 chapter25"` passed: 404 and 426
+ ordinary tests plus their fuzzer wrappers in 23 and 24; 429 total in 25
+ (365 raw0, 32 raw1, 8 standalone, 16 system, 1 fuzzer, 7 remaining).
+
+Logs: `chapter25/build/b14-review/{baseline,before-fix,fixed}.log`.
+
+### Build and fuzzer baseline, 2026-09-17
+
+Windows/Cygwin, OpenJDK 22.0.2: all 25 chapters passed `tags` and `release`.
+Build setup supplied missing early targets and corrected argument-list writes
+in 22-24; no compiler behavior was backported. Chapter 22 initially failed
+TypeTest with a TypeConAry/TypeRPC cast error, then passed a forced rebuild
+(369 ordinary tests plus fuzzer), consistent with stale classes.
+
+The initial all-chapter test baseline failed Chapter 18's random fuzzer on
+seed `973358943756616234`. At Cliff's request, wrappers in 8-24 now follow
+25's explicit seed-list structure, and 8-15 invoke their wrappers from
+`make tests`. Exploratory fuzzing is opt-in. All 25 full chapter targets
+passed after these harness changes; 8-15 passed again after wrapper invocation
+was added. Logs: `build/review/seed-lists-all-tests.log` and
+`seed-lists-08-15-tests.log`.
+
+**The Chapter 18 compiler failure remains open.** Its seed is in
+`OPEN_FAILING_SEEDS`, and explicitly running it still failed. Regression lists
+started empty, so green default wrappers added no fuzz regression coverage.
+Reduced input:
```java
-struct s0 {
- u8 v1;
-};
+struct s0 { u8 v1; };
s0? !UmPOLQK=null;
-if(0)
- while(0&UmPOLQK.v1) {}
+if(0) while(0&UmPOLQK.v1) {}
if(UmPOLQK.v1) {}
return new s0;
while(0) {}
```
-### Deterministic fuzzer follow-up
-
-Chapters 8-24 now follow Chapter 25's seed-list structure. Each wrapper has its
-own `REGRESSION_SEEDS` and ignored `OPEN_FAILING_SEEDS`. Exploratory fuzzing is
-opt-in, including the previously automatic random runs. Chapters 8-15 now invoke
-their wrappers from `make tests` as well. Fixed-seed failures throw immediately
-with the seed and original cause, without entering the exploratory reducer.
-Each chapter retains its generator and checking strategy; seeds are not copied
-between chapters.
-
-The regression lists start empty. Chapter 18's open list contains only
-`973358943756616234`; explicitly invoking that list still reproduces the failure.
-Passing default tests therefore does not mean that this compiler bug is fixed
-or that new fuzz regression coverage has been established. After a correction,
-move its seed to that chapter's regression list once the full seed passes.
-
-Validation: `make -k tests` passed across all 25 chapters after the harness
-changes. After adding wrapper invocation to Chapters 8-15, their complete
-`make tests` targets passed again. Logs: `build/review/seed-lists-all-tests.log`,
-`build/review/seed-lists-08-15-tests.log`, and
-`build/review/chapter18-open-seed.log` (intentional failure).
-
-The reduced Chapter 18 source exposes two separate concerns:
-
-- With peepholes disabled, nullable field lookup can fail during parsing;
- with peepholes enabled, the reduced original reaches a mixed-return error.
- Chapter 19 changes the fuzzer comparison to two optimizer worklist seeds,
- rather than peepholes disabled/enabled. That harness change deserves separate
- review; it is not proof that the compiler discrepancy was corrected.
-- The mixed-return diagnostic reduces further to
- `struct S { u8 x; }; return new S; return 0;`. Chapter 18 rejects it with
- peepholes both disabled and enabled. Chapter 19 accepts it under both fuzzer
- worklist orders. A plain `return new S;` is accepted in both chapters. The
- corresponding Chapter 19 `ReturnNode.err()` change is B11 above. Dead
- `if(0) return 0;` and a trailing `while(0) {}` also trigger the Chapter 18
- error. No compiler code has been changed yet.
-
-Additional source probes across Chapters 19-25 show differing treatment of
-unused nullable loads. A live `return p.x;` with null `p` was rejected in every
-tested snapshot. Keep the original seed open: B11 alone has not been shown to
-resolve its separate parser/checking discrepancy.
+This exposes two distinct concerns: nullable field lookup can fail during
+parsing with peepholes disabled; with peepholes enabled it reaches a
+mixed-return error. B11 isolates the latter further to
+`struct S { u8 x; }; return new S; return 0;`, rejected in 18 and accepted in
+19. Chapter 19 also changes the fuzzer comparison from peepholes off/on to
+two optimizer worklist seeds. Neither that harness change nor B11 alone proves
+the original seed fixed. Keep it open until the full seed passes.
+Log: `build/review/chapter18-open-seed.log` (intentional failure).