diff options
Diffstat (limited to 'graph')
| -rw-r--r-- | graph/README.md | 65 | ||||
| -rw-r--r-- | graph/src/main/java/com/seaofnodes/graph/GraphAdapter.java | 11 | ||||
| -rw-r--r-- | graph/src/main/java/com/seaofnodes/graph/GraphCapture.java | 125 | ||||
| -rw-r--r-- | graph/src/main/java/com/seaofnodes/graph/GraphJson.java | 6 | ||||
| -rw-r--r-- | graph/src/main/java/com/seaofnodes/graph/GraphSocket.java | 148 | ||||
| -rw-r--r-- | graph/src/main/java/com/seaofnodes/graph/GraphViewer.java | 49 | ||||
| -rw-r--r-- | graph/test_browser.py | 9 | ||||
| -rw-r--r-- | graph/test_transport.py | 61 | ||||
| -rw-r--r-- | graph/web/viewer.js | 5 |
9 files changed, 448 insertions, 31 deletions
diff --git a/graph/README.md b/graph/README.md index 27a96c01..e9cef3aa 100644 --- a/graph/README.md +++ b/graph/README.md @@ -1,21 +1,22 @@ # Shared graph viewer -The browser application lives in `web/` and is shared by chapters 18–25. +The browser application lives in `web/` and is shared by chapters 4 and 18–25. `web/index.html` contains the page and `web/viewer.js` manages playback. `web/layout.js` runs ELK layout in a browser worker; `web/render.js` draws SVG and handles selection, pan and zoom. `web/legacy.js` loads Graphviz on demand for earlier chapters. Libraries live in `web/vendor/`, with one checked-in copy of each; chapters do not copy them during builds. -From a chapter directory, run `make view`. The chapter's `JSViewer` finds +From a chapter directory, run `make view`. For chapters 4 and 25, the shared +`GraphViewer` finds `graph/web/index.html` by walking up from the working directory, serves its directory through the JDK's HTTP server on a local ephemeral port, and opens that web address in the browser. This works from a chapter directory, the repository root, or the root of a linearized checkout. Java launches from elsewhere can specify `-Dsimple.graph.url=<viewer URL>` before the main class. -The viewer uses the existing WebSocket connection. Chapter 25 sends structured -JSON snapshots in each frame; earlier chapters send bare DOT. +The viewer uses a WebSocket connection. Chapters 4 and 25 send structured +JSON snapshots in each frame; chapters 18–24 still send bare DOT. No JavaScript package installation, frontend build, or separate server process is required. For development over HTTP, run this from the repository root: @@ -31,7 +32,7 @@ The initial program compiles on connection. For another program, click **Compile or press **Ctrl+Enter** (**Cmd+Enter** on macOS), then use the arrows to step through the captured frames. Keep `make view` running while using the tab. -In chapter 25, scroll to zoom, drag to pan, and use **Fit** to show the whole +In chapters 4 and 25, scroll to zoom, drag to pan, and use **Fit** to show the whole graph again. Clicking a node shows its full label, type and input slots, and highlights its edges. Selection follows that node's ID across frames; Escape clears it. **Associations** shows scope bindings and lifetime edges, which are @@ -69,10 +70,24 @@ for Firefox testing. Test logs/screenshots are written under `build/graph-browse ## Boundary between chapters and the viewer -Each chapter currently retains its Java graph exporter, compiler hooks, and -WebSocket transport. The browser code has no dependency on a chapter's Java -classes. The adapters below share snapshot capture; moving the transport into -the shared Java code is a separate step. +Chapters 4 and 25 share the Java viewer plumbing as well as the browser: + +- `GraphViewer` owns asset discovery, the source/compile loop, frame serialization, + and session shutdown. A chapter's `JSViewer` subclass only invokes its compiler + and attaches/detaches its observer. +- `GraphSocket` serves local assets, opens the browser, and handles WebSocket + messages. One buffered stream handles both the upgrade and frames. Payloads + use exact reads and their advertised length, including 64-bit lengths; + continuation frames assemble UTF-8 before decoding, and PING data is echoed. +- `GraphCapture<N>` owns nested attempts, buffered snapshots, event numbering, + neighborhood accumulation and disconnect handling. It extends `GraphObserver<N>`. + Chapter subclasses supply roots, phase, parser position and observer detachment. +- `GraphAdapter<N>` supplies node facts and indexed graph/dependency access. + Dependency access defaults to an empty set in chapters without those lists. + +Chapter 25's private WebSocket implementation is removed. Chapters 18–24 retain +their old transport until migrated. Shared Java has no dependency on chapter classes. +The WebSocket listener and built-in asset server bind to 127.0.0.1. The connection messages are: @@ -81,12 +96,13 @@ The connection messages are: | Compiler to browser | `!` | Request the source program | | Browser to compiler | Source text | Compile and capture a new sequence | | Compiler to browser | `digraph ...` | One complete DOT snapshot | -| Compiler to browser | JSON `{snap, pos, evt}` | Chapter 25's complete graph, parser position and event | +| Compiler to browser | JSON `{snap, pos, evt}` | Chapters 4/25's complete graph, parser position and event | +| Compiler to browser | JSON `{error}` | Compilation failed; the session accepts another program | | Browser to compiler | `+` | Acknowledge/request another frame | | Compiler to browser | `#` | End of sequence | | Browser to compiler | `null` | End the session | -Chapter 25's JSON frame has this shape: +The JSON frame has this shape: ```json { @@ -115,10 +131,10 @@ The browser caches the parsed objects in `frames`. It computes geometry on the first visit to a frame and keeps it in `frames[i].layout`; revisiting a frame uses exactly that geometry. Viewport changes do not rerun layout. Nodes and edges have stable SVG IDs within a compilation. There is no animation yet. -One `+` acknowledges the whole frame. The chapter 25 writer sends UTF-8 bytes +One `+` acknowledges the whole frame. The shared writer sends UTF-8 bytes and supports WebSocket's 64-bit length header for frames larger than 65535 bytes. -Earlier chapters' DOT frames may contain a `// POS:` comment identifying the +Legacy chapters' DOT frames may contain a `// POS:` comment identifying the parser position. The browser owns frame history and backward/forward playback. Chapter 25 no longer generates DOT for the interactive viewer. The browser @@ -207,8 +223,8 @@ Functions and compilation-unit boundaries. These are existing IR facts, not computed region membership or a SESE hierarchy. Scope bindings and constant lifetime edges are associations, so they need not constrain CFG layout. Types may be absent on newly constructed nodes. Source locations, grouping, -and delta encoding remain follow-up work. Chapter 25 adds observer events as -described below; the chapter 4 adapter is still independent of an observer. +and delta encoding remain follow-up work. Both chapters use the observer events +described below. Make compiles these shared sources directly into each participating chapter's classes using `javac`. From the repository root: @@ -223,22 +239,26 @@ browser. Shared source changes trigger recompilation. No Maven executable, Maven-built artifacts, generated source copies or separately installed graph JARs are needed. Existing `make tests`, `make release` and chapter 25's `make view` also include the shared sources through their normal prerequisites. +Chapter 4's `make view` depends on `build` and runs the same shared viewer. For the existing Maven/CI build path, `build-helper:add-source` describes the same source directory. The two chapter POMs select `../graph`, while the root POM defaults to `graph` for linearized checkouts. These entries are independent of the Make build. -## Peephole observer (chapter 25) +## Peephole observers `GraphObserver<N>` is a shared abstract base for `before`, `after`, `phase`, and -`dep` callbacks. `CodeGen._obs` owns the optional observer. Normal compilation +`dep` callbacks. Chapter 25's `CodeGen._obs` and chapter 4's `Parser._obs` own the +optional observer. Chapter 4 uses `Parser.PARSER` as its current compilation +context, alongside its existing global `START`. Normal compilation leaves it null, so it does no snapshot capture or serialization. Assertions that probe peepholes with `_midAssert` set do not generate observer calls. Neither `Node` nor `IterPeeps` refers to `JSViewer`. -`SimpleGraphObserver` supplies chapter 25's capture and neighborhood logic. -`JSViewer` installs one for each compilation and supplies only the frame sink. +`GraphCapture` supplies capture and neighborhood logic. Each chapter's small +`SimpleGraphObserver` supplies its compilation context. `JSViewer` installs one +for each compilation and forwards frames to `GraphViewer`. An I/O failure disables the observer and allows optimization to continue. The observer is detached in a `finally` block when compilation finishes or fails. @@ -249,7 +269,7 @@ The event kinds distinguish the actual capture boundaries: | `BEFORE` | Graph before an attempt that made progress, or contained a nested rewrite | | `RETURN` | Recursive `Node.peephole()` finished, including its local DCE; the caller has not yet attached the returned node | | `APPLY` | A worklist attempt finished after substitution, dependency draining, and unused-node cleanup | -| `PHASE` | Parse, Iter, or Opto completed, including work outside individual peepholes | +| `PHASE` | Parse completed in chapter 4; Parse, Iter, or Opto completed in chapter 25, including work outside individual peepholes | `peep` identifies an attempt; `up` identifies its enclosing attempt, or zero. `node` and `repl` identify the original and result nodes. A zero `repl` on @@ -288,8 +308,9 @@ Actual graph diffs and explicit rule annotations can refine this later. The browser keeps the event alongside its snapshot and layout. It labels and highlights the current step, but does not yet collapse nested rewrites or move -the camera to their neighborhoods. The chapter 4 observer hookup is follow-up -work; its adapter already targets the same graph model. +the camera to their neighborhoods. Chapter 4 uses the same event model, with +defs/uses for its neighborhoods and no worklist or dependency lists. Its parser +retains the final Return as a root after scope cleanup. Chapter 25's parser no longer accepts `#showGraph;`; use the browser to inspect the graph and **Save SVG** for a static figure. Earlier chapters retain their diff --git a/graph/src/main/java/com/seaofnodes/graph/GraphAdapter.java b/graph/src/main/java/com/seaofnodes/graph/GraphAdapter.java index 77672c45..3e840b54 100644 --- a/graph/src/main/java/com/seaofnodes/graph/GraphAdapter.java +++ b/graph/src/main/java/com/seaofnodes/graph/GraphAdapter.java @@ -17,6 +17,9 @@ public abstract class GraphAdapter<N> { protected abstract int nOuts(N node); protected abstract N out(N node, int idx); protected abstract GraphSnapshot.Node desc(N node); + protected abstract boolean dead(N node); + protected int nDeps(N node) { return 0; } + protected N dep(N node, int idx) { throw new IndexOutOfBoundsException(idx); } protected final int ref(N node) { return node == null ? 0 : id(node); } @@ -28,10 +31,16 @@ public abstract class GraphAdapter<N> { */ @SafeVarargs public final GraphSnapshot snap(String comp, long step, N... roots) { + var list = new ArrayList<N>(); + java.util.Collections.addAll(list, roots); + return snap(comp, step, list); + } + + public final GraphSnapshot snap(String comp, long step, ArrayList<N> roots) { // Node IDs are dense; index directly without boxed keys or map entries. var seen = new ArrayList<N>(); var todo = new ArrayDeque<N>(); - var rids = new int[roots.length]; + var rids = new int[roots.size()]; int len = 0; for( N root : roots ) if( enq(root, seen, todo) ) rids[len++] = id(root); diff --git a/graph/src/main/java/com/seaofnodes/graph/GraphCapture.java b/graph/src/main/java/com/seaofnodes/graph/GraphCapture.java new file mode 100644 index 00000000..0e2f538d --- /dev/null +++ b/graph/src/main/java/com/seaofnodes/graph/GraphCapture.java @@ -0,0 +1,125 @@ +package com.seaofnodes.graph; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.BitSet; +import java.util.UUID; + +/** Shared capture, event nesting and neighborhoods. Chapter hooks only read the IR. */ +public abstract class GraphCapture<N> extends GraphObserver<N> { + + private final GraphAdapter<N> _graph; + private final String _comp = UUID.randomUUID().toString(); + private final ArrayList<Peep> _peeps = new ArrayList<>(); + private long _next, _step; + private boolean _off; + + protected GraphCapture(GraphAdapter<N> graph) { _graph = graph; } + + protected abstract String phase(); + protected abstract int pos(); + protected abstract void roots(ArrayList<N> roots); + protected abstract void detach(); + + protected abstract void frame(GraphSnapshot snap, int pos, GraphEvent evt) throws IOException; + + private class Peep { + final N node; + final long id, up; + final String phase; + final int pos; + final BitSet near = new BitSet(); + GraphSnapshot pre; + boolean sent; + + Peep(N n, long id, long up, String phase, int pos) { + node = n; this.id = id; this.up = up; this.phase = phase; this.pos = pos; + } + } + + @Override public void before(N n) { + if( _off ) return; + var p = new Peep(n, ++_next, _peeps.isEmpty() ? 0 : _peeps.getLast().id, + phase(), pos()); + _peeps.add(p); + near(p, n); + // Buffer until we know that this attempt (or a nested one) made progress. + // Include n explicitly: parse-time nodes need not have any uses yet. + p.pre = snap(n); + } + + @Override public void after(N n, N repl, boolean applied) { + if( _off ) return; + Peep p = _peeps.getLast(); + assert p.node == n; + near(p, n); + near(p, repl); + for( Peep up : _peeps ) up.near.or(p.near); + if( repl != null || p.sent ) { + // Emit ancestors first so a nested rewrite always has its context. + for( Peep up : _peeps ) { + if( up.sent ) continue; + emit(up.pre, up.pos, evt(up, GraphEvent.Kind.BEFORE, 0)); + if( _off ) return; + up.pre = null; + up.sent = true; + } + N rez = repl == null ? n : repl; + emit(snap(_graph.dead(rez) ? null : rez), pos(), + evt(p, applied ? GraphEvent.Kind.APPLY : GraphEvent.Kind.RETURN, + _graph.dead(rez) ? 0 : _graph.id(rez))); + } + _peeps.removeLast(); + } + + @Override public void dep(N n, N def) { + if( _off || _peeps.isEmpty() ) return; + Peep p = _peeps.getLast(); + add(p, n); + add(p, def); + } + + @Override public void phase(String phase) { + if( _off ) return; + emit(snap(null), pos(), new GraphEvent(GraphEvent.Kind.PHASE, 0, 0, phase, 0, 0, new int[0])); + } + + private GraphEvent evt(Peep p, GraphEvent.Kind kind, int repl) { + return new GraphEvent(kind, p.id, p.up, p.phase, _graph.id(p.node), repl, p.near.stream().toArray()); + } + + private void add(Peep p, N n) { if( n != null ) p.near.set(_graph.id(n)); } + + private void near(Peep p, N n) { + if( n == null ) return; + add(p, n); + for( int i = 0; i < _graph.nIns(n); i++ ) add(p, _graph.in(n, i)); + for( int i = 0; i < _graph.nOuts(n); i++ ) add(p, _graph.out(n, i)); + // These are wake-up dependents, not all nodes inspected by a rule. + // Read before the compiler drains the list; never move/clear it here. + for( int i = 0; i < _graph.nDeps(n); i++ ) add(p, _graph.dep(n, i)); + } + + + private GraphSnapshot snap(N extra) { + var roots = new ArrayList<N>(); + roots(roots); + for( Peep p : _peeps ) if( !_graph.dead(p.node) ) roots.add(p.node); + roots.add(extra); + return _graph.snap(_comp, 0, roots); + } + + private void emit(GraphSnapshot snap, int pos, GraphEvent evt) { + if( _off ) return; + // Assign steps on emission; attempts without progress do not consume frames. + snap = new GraphSnapshot(snap.ver(), _comp, _step++, snap.roots(), snap.nodes()); + try { + frame(snap, pos, evt); + } catch( IOException e ) { + // A disconnected display must not interrupt optimization. + _off = true; + detach(); + System.err.println("Graph viewer disconnected: " + e.getMessage()); + } + } +} diff --git a/graph/src/main/java/com/seaofnodes/graph/GraphJson.java b/graph/src/main/java/com/seaofnodes/graph/GraphJson.java index c27076d3..8726d81b 100644 --- a/graph/src/main/java/com/seaofnodes/graph/GraphJson.java +++ b/graph/src/main/java/com/seaofnodes/graph/GraphJson.java @@ -2,6 +2,12 @@ package com.seaofnodes.graph; /** JSON for the shared graph model; no chapter or layout dependencies. */ public class GraphJson { + public static String error(String msg) { + var sb = new StringBuilder("{\"error\":"); + str(sb, msg); + return sb.append('}').toString(); + } + public static String write(GraphSnapshot snap) { var sb = new StringBuilder(); snap(sb, snap); diff --git a/graph/src/main/java/com/seaofnodes/graph/GraphSocket.java b/graph/src/main/java/com/seaofnodes/graph/GraphSocket.java new file mode 100644 index 00000000..b9859b28 --- /dev/null +++ b/graph/src/main/java/com/seaofnodes/graph/GraphSocket.java @@ -0,0 +1,148 @@ +package com.seaofnodes.graph; + +import java.io.*; +import java.net.*; +import java.nio.charset.StandardCharsets; +import java.nio.file.Path; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.util.Base64; +import com.sun.net.httpserver.HttpServer; +import com.sun.net.httpserver.SimpleFileServer; + +/** One local viewer session. HTTP assets and compiler messages share its lifetime. */ +class GraphSocket extends ServerSocket { + private DataInputStream _in; + private OutputStream _out; + private HttpServer _http; + private Socket _client; + + GraphSocket(URI uri, int port) throws IOException { + super(port, 1, InetAddress.getByName("127.0.0.1")); + try { + if( "file".equals(uri.getScheme()) ) { + Path page = Path.of(uri); + _http = SimpleFileServer.createFileServer( + new InetSocketAddress("127.0.0.1", 0), page.getParent(), + SimpleFileServer.OutputLevel.NONE); + _http.start(); + uri = URI.create("http://127.0.0.1:" + _http.getAddress().getPort() + "/" + page.getFileName()); + } + System.out.println("Graph viewer: " + uri); + if( Boolean.parseBoolean(System.getProperty("simple.graph.open", "true")) ) + java.awt.Desktop.getDesktop().browse(uri); + _client = accept(); + // Keep one buffer across the upgrade and all subsequent frames. + _in = new DataInputStream(new BufferedInputStream(_client.getInputStream())); + _out = new BufferedOutputStream(_client.getOutputStream()); + upgrade(); + } catch( IOException | RuntimeException | Error e ) { + try { close(); } catch( IOException ignored ) {} + throw e; + } + } + + private String line() throws IOException { + var sb = new StringBuilder(); + for( int b; (b = _in.read()) != '\n'; ) { + if( b == -1 ) throw new EOFException("Incomplete WebSocket upgrade"); + if( b != '\r' ) sb.append((char)b); + } + return sb.toString(); + } + + private void upgrade() throws IOException { + String key = null; + for( String line; !(line = line()).isEmpty(); ) { + int colon = line.indexOf(':'); + if( colon > 0 && line.substring(0, colon).equalsIgnoreCase("Sec-WebSocket-Key") ) + key = line.substring(colon + 1).trim(); + } + if( key == null ) throw new IOException("Missing WebSocket key"); + String accept; + try { + byte[] hash = MessageDigest.getInstance("SHA-1").digest( + (key + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11").getBytes(StandardCharsets.US_ASCII)); + accept = Base64.getEncoder().encodeToString(hash); + } catch( NoSuchAlgorithmException e ) { + throw new IllegalStateException(e); // SHA-1 is required by the JDK. + } + _out.write(("HTTP/1.1 101 Switching Protocols\r\nConnection: Upgrade\r\nUpgrade: websocket\r\n" + + "Sec-WebSocket-Accept: " + accept + "\r\n\r\n").getBytes(StandardCharsets.US_ASCII)); + _out.flush(); + } + + public String get() throws IOException { + ByteArrayOutputStream msg = null; + while( true ) { + int head = _in.read(); + if( head == -1 ) { + if( msg != null ) throw new EOFException("Incomplete WebSocket message"); + return null; + } + boolean fin = (head & 128) != 0; + int op = head & 15; + int size = _in.readUnsignedByte(); + if( (head & 112) != 0 || (size & 128) == 0 ) + throw new IOException("Expected a masked WebSocket frame without extensions"); + long len = size & 127; + if( len == 126 ) len = _in.readUnsignedShort(); + else if( len == 127 ) len = _in.readLong(); + if( len < 0 || len > Integer.MAX_VALUE ) throw new IOException("WebSocket frame too large"); + if( op >= 8 && (!fin || len > 125) ) throw new IOException("Invalid WebSocket control frame"); + byte[] mask = new byte[4]; + _in.readFully(mask); + byte[] data = new byte[(int)len]; + _in.readFully(data); // TCP may deliver any header or payload in pieces. + for( int i = 0; i < data.length; i++ ) data[i] ^= mask[i & 3]; + switch( op ) { + case 8: return null; + case 9: put(10, data); continue; // PONG echoes PING, even between fragments. + case 10: continue; + case 1: + if( msg != null ) throw new IOException("Text frame inside an unfinished message"); + if( fin ) return new String(data, StandardCharsets.UTF_8); + msg = new ByteArrayOutputStream(); + break; + case 0: + if( msg == null ) throw new IOException("Continuation without a text frame"); + break; + default: throw new IOException("Unsupported WebSocket opcode: " + op); + } + msg.write(data); + if( fin ) return msg.toString(StandardCharsets.UTF_8); + } + } + + public void put(String msg) throws IOException { put(1, msg.getBytes(StandardCharsets.UTF_8)); } + + private void put(int op, byte[] data) throws IOException { + int len = data.length; + _out.write(128 | op); + if( len <= 125 ) _out.write(len); + else if( len <= 65535 ) { + _out.write(126); + _out.write(len >>> 8); + _out.write(len & 255); + } else { + _out.write(127); + for( int shift = 56; shift >= 0; shift -= 8 ) + _out.write((int)((long)len >>> shift) & 255); + } + _out.write(data); + _out.flush(); + } + + @Override public void close() throws IOException { + try { + if( _out != null && !isClosed() ) put(8, new byte[0]); + } finally { + if( _http != null ) _http.stop(0); + try { + if( _client != null ) _client.close(); + } finally { + super.close(); + } + } + } +} diff --git a/graph/src/main/java/com/seaofnodes/graph/GraphViewer.java b/graph/src/main/java/com/seaofnodes/graph/GraphViewer.java new file mode 100644 index 00000000..dcb2e337 --- /dev/null +++ b/graph/src/main/java/com/seaofnodes/graph/GraphViewer.java @@ -0,0 +1,49 @@ +package com.seaofnodes.graph; + +import java.io.IOException; +import java.net.URI; +import java.nio.file.Files; +import java.nio.file.Path; + +/** Shared launcher and source/compile loop. Subclasses supply their compiler. */ +public abstract class GraphViewer implements AutoCloseable { + private final GraphSocket _server; + + protected GraphViewer() throws IOException { _server = new GraphSocket(viewerURI(), 12345); } + protected abstract void compile(String src); + + protected final void frame(GraphSnapshot snap, int pos, GraphEvent evt) throws IOException { + _server.put(GraphJson.frame(snap, pos, evt)); + } + + private static URI viewerURI() throws IOException { + String url = System.getProperty("simple.graph.url"); + if( url != null ) return URI.create(url); + for( Path dir = Path.of("").toAbsolutePath(); dir != null; dir = dir.getParent() ) { + Path page = dir.resolve("graph/web/index.html"); + if( Files.isRegularFile(page) ) return page.toUri(); + } + throw new IOException("Cannot find graph/web/index.html; run inside the Simple checkout " + + "or set -Dsimple.graph.url=<viewer URL>"); + } + + public final void run() throws IOException { + _server.put("!"); + while( true ) { + String src = _server.get(); + if( src == null || src.equals("null") ) return; + if( src.equals("+") ) continue; // Frames are currently sent eagerly. + System.out.println(src); + try { + compile(src); + } catch( RuntimeException e ) { + System.err.println(e); + _server.put(GraphJson.error(e.toString())); + } finally { + _server.put("#"); + } + } + } + + @Override public final void close() throws IOException { _server.close(); } +} diff --git a/graph/test_browser.py b/graph/test_browser.py index dbbfdd59..90223119 100644 --- a/graph/test_browser.py +++ b/graph/test_browser.py @@ -23,7 +23,7 @@ from playwright.sync_api import sync_playwright def check(chapter, browser_name): number = int(chapter.removeprefix("chapter")) - package = "com.seaofnodes.simple." + ("print." if number >= 20 else "") + package = "com.seaofnodes.simple." + ("print." if number == 4 or number >= 20 else "") artifacts = ROOT / "build/graph-browser" / (chapter + "-" + browser_name) artifacts.mkdir(parents=True, exist_ok=True) log = artifacts / "compiler.log" @@ -53,8 +53,9 @@ def check(chapter, browser_name): page.wait_for_function("current === 1 && !rendering") page.locator("#doPrev").click() page.wait_for_function("current === 0 && !rendering") - for source in ("return 1 + 2;", - "int x=0; while(x<arg) { x=x+1; } return x;"): + sources = ("return 1 + 2;", "int x=arg+1; return x+x+2;" if number == 4 else + "int x=0; while(x<arg) { x=x+1; } return x;") + for source in sources: previous = page.evaluate("generation") page.locator("#program").fill(source) page.locator("#compile").click() @@ -68,7 +69,7 @@ def check(chapter, browser_name): page.wait_for_function("index => current === index && !rendering", arg=index) assert page.locator("#doNext").is_disabled() assert "error" not in page.locator("#status").inner_text().lower() - if number == 25: + if number in (4, 25): assert page.locator("#elk g.node").count() == page.evaluate("frames[current].snap.nodes.length") # Revisit cached geometry and keep the selected node across steps. page.evaluate("""() => { diff --git a/graph/test_transport.py b/graph/test_transport.py index b33b4657..d0026b97 100644 --- a/graph/test_transport.py +++ b/graph/test_transport.py @@ -10,14 +10,45 @@ import subprocess import sys import time import urllib.request +import json +import struct + + +def packet(data, op=1, fin=True): + n = len(data) + head = bytes([op | (128 if fin else 0)]) + head += bytes([128 | n]) if n < 126 else (b'\xfe' + struct.pack('>H', n) if n <= 65535 + else b'\xff' + struct.pack('>Q', n)) + mask = b'abcd' + return head + mask + bytes(b ^ mask[i % 4] for i, b in enumerate(data)) + + +def frame(stream): + head = stream.read(2) + assert len(head) == 2, 'Incomplete server frame' + n = head[1] + if n == 126: n = struct.unpack('>H', stream.read(2))[0] + elif n == 127: n = struct.unpack('>Q', stream.read(8))[0] + data = stream.read(n) + assert len(data) == n + return head[0] & 15, data + + +def compile_frames(stream): + frames = [] + while True: + op, data = frame(stream) + assert op == 1 + if data == b'#': return frames + frames.append(json.loads(data)) def check(chapter): root = Path(__file__).resolve().parent.parent number = int(chapter.removeprefix("chapter")) - assert 18 <= number <= 25 + assert number == 4 or 18 <= number <= 25 chapter_dir = root / chapter - package = "com.seaofnodes.simple." + ("print." if number >= 20 else "") + package = "com.seaofnodes.simple." + ("print." if number == 4 or number >= 20 else "") log = root / "build" / "graph-transport" / (chapter + ".log") log.parent.mkdir(parents=True, exist_ok=True) with log.open("w") as output: @@ -62,7 +93,9 @@ def check(chapter): else: raise AssertionError("Responded before headers ended: " + repr(early)) client.settimeout(3) - client.sendall(b"\r\n") + shared = number in (4, 25) + # A frame in the same write as the upgrade must not get lost in a reader buffer. + client.sendall(b"\r\n" + (packet(b'hello', op=9) if shared else b'')) with client.makefile("rb") as stream: assert stream.readline() == b"HTTP/1.1 101 Switching Protocols\r\n" headers = [] @@ -74,6 +107,25 @@ def check(chapter): headers.append(line) assert b"Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=\r\n" in headers assert stream.read(3) == b"\x81\x01!", "Missing source request" + if shared: + assert frame(stream) == (10, b'hello') + # Larger than both the old 1 KB buffer and the 16-bit length field. + data = packet(b' ' * 70000 + b'return 3;') + client.sendall(data[:3]) + time.sleep(.05) + for start in range(3, len(data), 509): + client.sendall(data[start:start + 509]) + frames = compile_frames(stream) + assert frames and all('snap' in f for f in frames) + # Split a UTF-8 code point across fragments, with an interleaved PING. + client.sendall(packet(b'\xc3', fin=False) + packet(b'ping', op=9) + packet(b'\xa9', op=0)) + assert frame(stream) == (10, b'ping') + errors = compile_frames(stream) + assert any('error' in f for f in errors) + assert '\u00e9' in log.read_text() + client.sendall(packet(b'return 4;')) + frames = compile_frames(stream) + assert frames and all('snap' in f for f in frames) # Masked 'null' asks the compiler to exit normally. mask = b"abcd" payload = bytes(byte ^ mask[i % 4] for i, byte in enumerate(b"null")) @@ -91,7 +143,8 @@ def check(chapter): if process.poll() is None: process.terminate() process.wait(timeout=5) - print(chapter + ": HTTP assets, fragmented upgrade, greeting, and shutdown passed") + print(chapter + ": HTTP assets, fragmented upgrade, greeting, and shutdown passed" + + ("; large frames, UTF-8 fragments, ping, and compile-error recovery passed" if number in (4, 25) else "")) if __name__ == "__main__": diff --git a/graph/web/viewer.js b/graph/web/viewer.js index 9aaaa154..ed14651f 100644 --- a/graph/web/viewer.js +++ b/graph/web/viewer.js @@ -148,6 +148,11 @@ try { try { if (message.startsWith("{")) { frame = JSON.parse(message); + if (frame.error) { + failure = "Compile error: " + frame.error; + updateUI(); + return; + } if (frame.snap.ver !== 1) throw new Error("Unsupported graph version: " + frame.snap.ver); } else { // Earlier chapters still send bare DOT. |