summaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorCliff Click <cliffc@acm.org>2026-10-07 19:10:51 -0700
committerCliff Click <cliffc@acm.org>2026-10-07 19:10:51 -0700
commit2eba63966bdc3d9cfdf41ee085f35cbb6d529bbc (patch)
treec297b19a448bc3e0780437122646183240437a30 /docs
parentaaec6f8136df6f7020bfc05f79ff5273adf73de1 (diff)
downloadSimple-2eba63966bdc3d9cfdf41ee085f35cbb6d529bbc.tar.gz
Simple-2eba63966bdc3d9cfdf41ee085f35cbb6d529bbc.zip
Towrds getting RA spill counts comparable
Ignore JIGs. Get test suites more conical, so reg alloc spill counts are comparable
Diffstat (limited to 'docs')
-rw-r--r--docs/chapter-backports.md129
-rw-r--r--docs/regalloc-spills.md108
2 files changed, 232 insertions, 5 deletions
diff --git a/docs/chapter-backports.md b/docs/chapter-backports.md
index 4b516036..fdc4eff4 100644
--- a/docs/chapter-backports.md
+++ b/docs/chapter-backports.md
@@ -465,6 +465,25 @@ lines beyond the original extraction.
## Pending corrections
+- **Chapter 25 deterministic inlining (deferred until after the Chapter 25 split).**
+ As of 2026-10-07, ordinary functions reject `_approxUIDs >= 100` before checking
+ the optimized body. `_approxUIDs` counts node IDs consumed during parsing and
+ is not refreshed after optimization. In the now-private Chapter 20 String
+ replay, `_hashCodeString` is rejected at 100 although its live body has 25
+ nodes. MergeSort's `_merge` is rejected at 182 although its body has 54;
+ both are below the ordinary live-body threshold of 100. A shadow build removing
+ only that early rejection inlines both helpers and drops their out-of-line
+ bodies; MergeSort retains its recursive `_split_merge` as expected. The
+ ordinary-function policy is unchanged. Cliff deferred the larger repair until
+ after splitting Chapter 25: converge shrinking rewrites, then choose one growing
+ inline deterministically using current body sizes cached per convergence round.
+ Stable call-site provenance must break ties rather than randomized worklist
+ order or newly assigned node IDs. Avoid dependencies on every body node and
+ repeated scans during each peephole. The bounded boilerplate check below is
+ the small interim heuristic; the larger constructor parse allowance was replaced.
+ Evidence:
+ `build/ra-repair/inlining{,-ablation}.log`.
+
- **Chapter 24 Load BOTTOM-on-error backport, unwound/deferred (2026-09-28).**
Cliff requested starting with 24 and analyzing failures before proceeding.
Literal global BOTTOM memory/pointer inputs already propagate BOTTOM. The
@@ -579,9 +598,11 @@ Chapter 25 suite, including `make -j 4 tests`.
## Register allocation: completed chapter progression
-The allocator review is complete through Chapter 25. Each README ends with the
-introduced technique, measured program cohorts, and commentary. `make spill-stats`
-reports actual retained moves, per target and cohort, with assertions enabled.
+The allocator review is complete through Chapter 25; Cliff reviewed Chapter 23
+on 2026-10-07. Each README ends with the introduced technique, a triangular
+matrix of fixed cohorts across compiler chapters, and commentary. The current
+[spill-cost matrix](regalloc-spills.md) is regenerated by root `make spill-stats`,
+which checks matching test/CPU/ABI membership with assertions enabled.
| Chapter | Additional technique | Final cohort sizes |
|---|---|---|
@@ -590,7 +611,7 @@ reports actual retained moves, per target and cohort, with assertions enabled.
| 22 | Stronger copy-chain/backedge bias and cheap-spill ordering | 39, 52, 26 |
| 23 | Group popular single-def uses by compatible register classes | 39, 52, 26, 30 |
| 24 | One cold-only attempt for loop-Phi self-conflicts, then mandatory fallback | 39, 52, 26, 30, 67 |
-| 25 | Existing area/cost ranking and later conflict strategies | 39, 52, 24, 30, 67, 14 |
+| 25 | Existing area/cost ranking and later conflict strategies | 39, 52, 26, 30, 67, 14 |
Shared corrections include RegMask/LRG bookkeeping, null use masks, kills without
an output LRG, self-conflict splitting, compatible rematerialization, clobber-aware
@@ -614,7 +635,7 @@ registers removes that conflict. Encoding must then use the allocated register
bank, with five-bit register numbers, for both immediate and indexed loads/stores.
The emulator must preserve floating bits and scale double offsets by eight.
-Chapter 25 freezes the 212 earlier compilation entries in
+Chapter 25 freezes the 214 earlier compilation entries in
[documented source fixtures](../chapter25/src/test/java/com/seaofnodes/simple/spill/README.md).
Constructor/library adaptations change IR, so this is a program-cohort comparison,
not identical machine graphs. Those rows replay allocation and legality checks;
@@ -656,6 +677,104 @@ The top-level runner accepts explicit chapter lists, e.g.
## Validation record
+- **Ignore debugging Jigs (2026-10-07).** All `testJig*` methods are scratch
+ placeholders, not regression tests. Added missing `@Ignore` annotations in
+ Chapters 14-25, preserving their bodies, and removed all 15 Jig entries and
+ four unused frozen sources from Chapter 25's replay corpus. Cohorts 23/24 now
+ have 24/58 allocations; the corpus has 199 entries. The debug Fibonacci in
+ Chapters 22-24's `progs/jig.smp` uses private `_fib`; `fib.smp` and the genuine
+ `testFibExport` fixtures retain their exports, as confirmed by Cliff.
+ Full Make suites pass in all 13 affected snapshots, including 17a/17b and
+ Chapter 25's native groups, plus shared printer/ISA checks. Fresh spill reports
+ pass in 20-25, and every retained raw/scaled allocation record matches its
+ prior measurement. Cohort 23 is now 103/103/152 across compilers 23/24/25;
+ cohort 24 is 1,387/1,503 across 24/25. The historical heuristic comparison was
+ recomputed from saved records with the same exclusions. All tables refreshed.
+ A Chapter 25 allocation probe confirms the private debug Fibonacci leaves
+ only `<clinit>`, with no separate Fibonacci body. Logs: `build/jig-cleanup/`.
+
+- **Tiny-body inlining backport, Chapters 20-24 (2026-10-07).** Preserved Cliff's
+ reusable static 15-node worklist and Return-input/prefix dependencies. These
+ chapters previously supported only single-caller inlining, so the port adds
+ restricted copying of straight-line Store/MemMerge bodies, substituting call
+ arguments and control directly. Growing inlines run from a separate worklist
+ after ordinary peepholes settle; existing single-caller inlining is unchanged.
+ Constructor defaults already expand directly in these chapters; they have no
+ Chapter 25-style constructor calls or parse-size gate to relax. All full Make
+ suites pass: 406/445/466/488/518 tests respectively, plus each fuzzer wrapper.
+ The existing `Chapter18Test.testInline` graph now contains literal `1` on one
+ branch; no new permanent tests were added. Disposable two-call store probes
+ pass with seeds 123/456/789 in all five chapters, including an intervening store
+ to another alias. All 39/91/117/147/214 allocation records, including raw and
+ scaled costs, match the prior reports exactly. Logs: `build/tiny-inline/final*.log`.
+ Chapter 25 code and the deferred general inliner redesign are unchanged.
+
+- **Bounded tiny-body inlining (2026-10-07).** Replaced the constructor-specific
+ 1,000-node parse allowance with a conservative at-most-15-node check when the
+ parse estimate exceeds the normal limit. It recognizes entry/parameter/store/
+ memory-merge/Return boilerplate, follows uses to avoid sparse alias arrays,
+ bounds fanout, and stops at Return. Dependencies watch Return inputs and only
+ the inspected prefix/frontier; failed checks can retry after folding. Actual
+ body limits remain 100, or 200 for `<new>` and instance `<init>`. All six replay
+ constructor calls still fold, and all 228 raw/scaled allocation records match
+ the preceding allowance trial. Full Chapter 25 Make tests pass, as do fresh
+ Win64/SystemV library compilations. Disposable probes force parse estimates
+ to 10,000 on Bubble Sort initializers and verify successful inlining/allocation
+ on all three targets. A dependency probe verifies requeueing without a Return
+ type change, rejection above the small-body bound, and acceptance after a
+ memory-chain fold. Logs: `build/tiny-inline/`. The general inliner redesign
+ remains deferred until after the chapter split.
+
+- **Chapter 25 constructor inlining allowance (2026-10-07).** `<new>` and
+ instance `<init>` share a 200-node optimized-body limit, including the cloning
+ assertion. A 1,000-node parse-time cutoff accommodates nested-method and
+ memory construction overhead; ordinary functions stay at 100 for both gates.
+ The initial 200/200 trial changed none of 228 allocation records. The larger
+ parse allowance removes all six remaining constructor calls in the frozen
+ replay sources (Bubble Sort), while public constructor definitions remain.
+ Cohorts 20-23 are unchanged; cohort 24 improves 1,637 to 1,634, while cohort 25
+ rises 1,829 to 1,862. Win64 sys improves 842 to 809, Dijkstra 584 to 582;
+ FileIO rises 293 to 361. Fresh SystemV sys improves 803 to 777. These static
+ move costs do not establish a runtime improvement. Full Chapter 25 Make tests
+ pass (405 + 39 + 9 + 23 + 1 + 19), and the complete spill report passes its
+ 214 replay allocations and native/system tests. No goldens, replay sources,
+ seeds or memberships changed during this experiment. Matrices are refreshed.
+ Earlier compiler snapshots are unchanged: 20-24 have single-use non-cloning
+ inlining rather than this size-based cloning gate. Logs: `build/ctor-inline/`.
+
+- **Chapter 25 function constants and private helpers (2026-10-07).**
+ Single-use, single-definition function addresses separated from their use in
+ the same block get priority for rematerialization over spilling callee-save
+ registers. Adjacent, multi-use and cross-block cases keep the existing ranking.
+ On unchanged inputs this removes all 28 C-return initializer moves, plus 13
+ weighted moves from cohort 24 and 22 from cohort 25, with no other cohort
+ changes. Eleven replay fixtures separately mark local helpers/classes private;
+ exported interfaces stay public. Final Ch25 weighted costs by cohort are
+ 491 / 1,077 / 71 / 380 / 1,637 / 1,829, with unchanged membership and counts.
+ The fixture README and matrix document the changed visibility baseline.
+ Full `make -C chapter25 tests` passes (405 + 39 + 9 + 23 + 1 + 19 tests),
+ as do both complete spill reports and fresh SystemV sys encoding on Windows
+ (327 raw / 803 weighted moves). The reports each include 214 replay allocations,
+ fresh Win64 library encoding and 23 native/system tests. No inlining-policy
+ change or earlier-chapter allocator change was made. Logs: `build/ra-repair/`.
+
+- **Spill matrices and bounded CI processes (2026-10-07).** Reran Chapter 20-25
+ statistics with fixed cohort sizes 39, 52, 26, 30, 67, and 14. Chapter 25 now
+ includes the two Chapter 22 C-return ABI cases missing from its old manifest.
+ The matrix generator checks the multiset of test/CPU/ABI identities and emits
+ no totals row; each README shows columns through its chapter. Chapter 23's
+ copied Java implementation snippets are replaced with direct source links.
+ CI run 37675851863 timed out in `nm` after 10 seconds; run 37678408500 passed.
+ The bounded GCC/nm/test-executable calls now allow 60 seconds and retain
+ forced termination and captured stdout/stderr, with PID/CPU-time diagnostics.
+ CLI subprocesses retain their existing unbounded behavior. Root `make
+ spill-stats` passes and reproduces the matrix; the reporter rejects changed
+ membership even when counts match, and rejects failed runs. All 23 Chapter 25
+ system tests pass. An 11-second subprocess passes the new bound; an intentional
+ hang is killed at a short test deadline with diagnostics and no live child.
+ Logs: `build/spill-stats/`, `build/ci-timeout-60-tests.log`, and
+ `build/ci-timeout-60-probe.log`.
+
- **Numeric assignment conversions and evaluators (2026-10-07).** Chapters
14-25 accept float-to-integer assignment using ToIntegerNode: truncate toward
zero, saturate to i64 on overflow, map NaN to zero, then apply the declared
diff --git a/docs/regalloc-spills.md b/docs/regalloc-spills.md
new file mode 100644
index 00000000..9b96a30b
--- /dev/null
+++ b/docs/regalloc-spills.md
@@ -0,0 +1,108 @@
+# Register allocation across chapters
+
+Read a row from left to right to follow a fixed test cohort through successive
+compilers. Columns identify the compiler/allocator chapter; each cell is its
+sum of `_spillScaled`. Blank cells precede the cohort's introduction.
+There is no total across different cohorts.
+
+<!-- spill-matrix:start -->
+| Test cohort | # tests | Ch 20 | Ch 21 | Ch 22 | Ch 23 | Ch 24 | Ch 25 |
+|---|---:|---:|---:|---:|---:|---:|---:|
+| Ch 20 | 39 | 360 | 576 | 430 | 442 | 441 | 491 |
+| Ch 21 | 52 | | 1,072 | 960 | 961 | 948 | 1,077 |
+| Ch 22 | 26 | | | 63 | 63 | 63 | 71 |
+| Ch 23 | 24 | | | | 103 | 103 | 152 |
+| Ch 24 | 58 | | | | | 1,387 | 1,503 |
+| Ch 25 | 14 | | | | | | 1,862 |
+<!-- spill-matrix:end -->
+
+Here **# tests counts compilations**, including the same program on different
+CPU/ABI combinations and separately recorded compilation phases. It is not the
+number of JUnit methods. Every filled cell in a row uses the same number of
+compilations and the same test/CPU/ABI membership, including zero-spill cases.
+The reporter checks the multiset of those identities before emitting the table.
+Jig methods are temporary debugging placeholders and are always ignored. Their
+15 replay entries are excluded consistently: six from cohort 23 and nine from
+cohort 24. The two cohort-23 Jigs duplicated one Fibonacci scratch program;
+Chapters 23-24 discarded its default main, while 25 compiled it and an exported
+function. Its removal eliminates 228 weighted moves from the Chapter 25 column
+without changing the compiler.
+Chapter 20's BrainFuck and MergeSort tests were moved into Chapter20Test in 21;
+the reporter recognizes that rename.
+
+`_spillScaled` counts retained SplitNodes, including register-to-register moves,
+weighted by `8^loopDepth`. Lower is better; these are estimated move costs,
+not measured memory traffic or execution time.
+
+Measured on Windows x86-64, 2026-10-07. The earlier cohorts combine x86-64
+SystemV/Win64 with ARM and RISC-V SystemV, at optimizer seed 123. Cohort 25
+contains 13 recorded native-test allocations at their existing seeds and a
+fresh system-library compilation at seed 456.
+
+The rows compare whole chapter compilers. Optimizations, scheduling, ABI
+lowering, and library representation also change, so a row need not decrease
+at every step. In particular, Chapter 25 replays earlier programs with the
+[documented syntax, constructor and visibility adaptations](../chapter25/src/test/java/com/seaofnodes/simple/spill/README.md).
+The two Chapter 22 C-return ABI cases are included there as well, keeping that
+row at 26 compilations. They cost no moves in 22-25 after Chapter 25's targeted
+function-address rematerialization preference.
+Use a controlled comparison within one compiler to isolate an allocator heuristic.
+
+The Chapter 25 comparison below separates that allocator change from making
+local helpers/classes private in eleven replay fixtures. The ranking trial uses
+the same sources, seeds, targets and ABIs as its baseline. Private names remove
+unneeded exported bodies and allocation factories; their effect is a change to
+the lowered program, not an allocator gain. Export/API fixtures stay public.
+
+| Test cohort | Before | Ranking only | + Private helpers | + Tiny-body check |
+|---|---:|---:|---:|---:|
+| Ch 20 | 608 | 608 | 491 | 491 |
+| Ch 21 | 1,170 | 1,170 | 1,077 | 1,077 |
+| Ch 22 | 127 | 99 | 71 | 71 |
+| Ch 23 | 152 | 152 | 152 | 152 |
+| Ch 24 | 1,558 | 1,545 | 1,506 | 1,503 |
+| Ch 25 | 1,851 | 1,829 | 1,829 | 1,862 |
+
+These four successful Chapter 25 reports have been recomputed from their saved
+per-allocation records with the Jig entries excluded, leaving 199 replay
+allocations. They also passed fresh library encoding and 23 native/system tests.
+The final compiler also passes its complete
+Make suite and a fresh SystemV library encoding on Windows. The earlier compiler
+columns are unchanged.
+
+The tiny-body check bypasses an oversized parse-time estimate only for recognized
+straight-line boilerplate containing at most 15 nodes. Parsing an initializer
+also builds its nested methods, so that estimate can greatly exceed the surviving
+body. Dependencies on Return inputs and the bounded inspected prefix allow retries
+after folding, without walking a large body on every change. The walk follows uses
+from entry to Return, with bounded fanout, avoiding sparse memory input arrays.
+This removes six constructor calls in the replay corpus, all in Bubble Sort:
+estimates 276, 352 and 479 represented bodies of only 11, 11 and 14 nodes. Public
+constructor bodies remain exported. The optimized-body limits remain 100 for
+ordinary functions and 200 for `<new>`/instance `<init>`; selection order is unchanged.
+It reproduces all 213 retained allocation records from the preceding 1,000-node constructor
+parse-allowance trial, replacing that allowance with a bounded structural check.
+
+This is a boilerplate reduction, not a uniform spill improvement. Bubble Sort's
+weighted moves change by -10 on x86, +3 on ARM and +4 on RISC-V. The fresh Win64
+library improves 842 to 809 and Dijkstra 584 to 582, while FileIO rises 293 to 361.
+The separately checked SystemV library improves 803 to 777. The broader inliner
+redesign is deferred until after the Chapter 25 split.
+
+Chapters 20-24 also recognize these tiny bodies, copying only straight-line
+stores and memory merges after ordinary peepholes settle. They already expand
+constructor defaults directly, so there are no analogous constructor calls to
+remove. Rerunning their full suites and allocation reports after this backport
+changed no raw or scaled allocation record; the matrix above remains unchanged.
+
+Run `make spill-stats` from the repository root to rerun all six chapters and
+write logs plus `matrix.md` under `build/spill-stats/`. A failed test or changed
+cohort membership makes the command fail; partial results are not published.
+To refresh these tables after a successful run:
+
+```sh
+python build-support/spill-matrix.py --reuse --update-docs
+```
+
+Each Chapter 20-25 README contains the matrix up to that chapter. The per-chapter
+`make spill-stats` commands still provide individual allocations and CPU/ABI sums.