Raku++ Internals
How a hand-written C++ implementation of Raku works — from source text to native code.
Download the PDF 320 pages · 1.1 MB
- Prefacewho this is for, and the conventions it uses
Ground
- Chapter 1What Raku++ Isthe goals, the four run modes, the shape of the system
- Chapter 2The Shape of the Sourcethe file map, and how to navigate it
- Chapter 3Where It Sits in the Taxonomywhy neither LL nor LR describes this compiler
The Front End
- Chapter 4The Lexertokens, significant whitespace, quotes, heredocs
- Chapter 5The Parserrecursive descent, a Pratt core, and no compile-time execution
- Chapter 6User-Defined Operators in a Single Passlive tables, precedence traits, and lexical rollback
- Chapter 7The ASTthe node hierarchy, and fields that answer questions about syntax
The Value Model
- Chapter 8
Value: the Fat Tagged Structone fat tagged struct for every runtime value - Chapter 9Strings:
CowStrthe copy-on-write string, and the quadratic it removed - Chapter 10Interning and Comparison:
IStrandMNameinterned tags and packed-prefix name comparison - Chapter 11Numbersnative ints, bignums, rationals, and the spill between them
- Chapter 12Containers, Binding, and Copy Semanticsscopes, binding, and where sharing is broken on purpose
The Interpreter
- Chapter 13The Tree Walkeval, exec, and the per-thread execution registers
- Chapter 14Calls and Parameter Bindingbuilding arguments, activating a routine, binding parameters
- Chapter 15Cooperative Control Flowreturn and friends without a C++ throw
- Chapter 16Dispatchsubs, multis, the method ladder, and re-dispatch
- Chapter 17The Object Systemclasses, roles, mixins, and the meta-object surface
- Chapter 18Laziness, Junctions, and the Wilder Valuesinfinite sequences, gather without coroutines, junctions
- Chapter 19Node Specializationone optimisation, measured end to end
The Regex and Grammar Engine
- Chapter 20Compiling a Regexa second recursive-descent compiler inside the runtime
- Chapter 21The Backtracking Matchercontinuation passing, backtracking, and a step budget
- Chapter 22Grammarsrules, packrat memoisation, actions, protos
- Chapter 23Longest-Token Matchingranking alternations by declarative prefix
Unicode
- Chapter 24Graphemes, Normalization, Collationgraphemes, normalization, collation — from pinned tables
The Back Ends
- Chapter 25Four Ways to Run a Programone program through interpret, bundle, AOT and native
- Chapter 26The Code GeneratorAST to C++, calling the same runtime the interpreter calls
- Chapter 27The
-OOptimizerthree passes, and three defaults that are not passes - Chapter 28Dispatch Cost in Compiled Codewhat each call shape costs, and an analysis that was wrong
- Chapter 29What a Compiled Binary Keepshow a binary drops what the program cannot reach
- Chapter 30AST Serialization and the Precompiled Parsethe binary AST format, and what invalidates a cache entry
- Chapter 31Raku in the Browserthe same runtime in a browser tab, Live mode, and embedded editors
Boundaries
- Chapter 32Moduleswhat use Foo actually does, and where it diverges
- Chapter 33The Installer and the Store
- Chapter 34
use nqp: a Compatibility Subseta compatibility subset that costs nothing when unused - Chapter 35NativeCall and the libffi Backendlibffi loaded at run time, and its ABI probed not tabulated
- Chapter 36The Extension ABIthe C ABI an extension compiles against, and why it never sees Value
- Chapter 37Concurrencythe GIL, per-thread registers, and switching the lock off
Around the Compiler
- Chapter 38Tooling Built on the ASTlint, highlight, profile, the REPL, and tools written in Raku
- Chapter 39Measuring, and Provingthe benchmark policy, the gates, and what did not pay
The Speed Campaign
Appendices
- Appendix AThe Tag Setsthe VT, NK and regex-node tag sets
- Appendix BFlags and Environment Variablesevery flag and environment variable
- Appendix CSource Map and Glossarya source map, the rules that are easy to break, a glossary