Raku++ Internals

How a hand-written C++ implementation of Raku works — from source text to native code.

Download the PDF 200 pages · 0.8 MB

  1. Prefacewho this is for, and the conventions it uses

Ground

  1. Chapter 1What Raku++ Isthe goals, the four run modes, the shape of the system
  2. Chapter 2The Shape of the Sourcethe file map, and how to navigate it

The Front End

  1. Chapter 3The Lexertokens, significant whitespace, quotes, heredocs
  2. Chapter 4The Parserrecursive descent, a Pratt core, and no compile-time execution
  3. Chapter 5User-Defined Operators in a Single Passlive tables, precedence traits, and lexical rollback
  4. Chapter 6The ASTthe node hierarchy, and fields that answer questions about syntax

The Value Model

  1. Chapter 7Value: the Fat Tagged Structone fat tagged struct for every runtime value
  2. Chapter 8Strings: CowStrthe copy-on-write string, and the quadratic it removed
  3. Chapter 9Interning and Comparison: IStr and MNameinterned tags and packed-prefix name comparison
  4. Chapter 10Numbersnative ints, bignums, rationals, and the spill between them
  5. Chapter 11Containers, Binding, and Copy Semanticsscopes, binding, and where sharing is broken on purpose

The Interpreter

  1. Chapter 12The Tree Walkeval, exec, and the per-thread execution registers
  2. Chapter 13Calls and Parameter Bindingbuilding arguments, activating a routine, binding parameters
  3. Chapter 14Cooperative Control Flowreturn and friends without a C++ throw
  4. Chapter 15Dispatchsubs, multis, the method ladder, and re-dispatch
  5. Chapter 16The Object Systemclasses, roles, mixins, and the meta-object surface
  6. Chapter 17Laziness, Junctions, and the Wilder Valuesinfinite sequences, gather without coroutines, junctions
  7. Chapter 18Node Specializationone optimisation, measured end to end

The Regex and Grammar Engine

  1. Chapter 19Compiling a Regexa second recursive-descent compiler inside the runtime
  2. Chapter 20The Backtracking Matchercontinuation passing, backtracking, and a step budget
  3. Chapter 21Grammarsrules, packrat memoisation, actions, protos
  4. Chapter 22Longest-Token Matchingranking alternations by declarative prefix

Unicode

  1. Chapter 23Graphemes, Normalization, Collationgraphemes, normalization, collation — from pinned tables

The Back Ends

  1. Chapter 24Four Ways to Run a Programone program through interpret, bundle, AOT and native
  2. Chapter 25The Code GeneratorAST to C++, calling the same runtime the interpreter calls
  3. Chapter 26The -O Optimizerthree passes, and three defaults that are not passes
  4. Chapter 27Dispatch Cost in Compiled Codewhat each call shape costs, and an analysis that was wrong
  5. Chapter 28AST Serialization and the Precompiled Parsethe binary AST format, and what invalidates a cache entry

Boundaries

  1. Chapter 29Moduleswhat use Foo actually does, and where it diverges
  2. Chapter 30use nqp: a Compatibility Subseta compatibility subset that costs nothing when unused
  3. Chapter 31NativeCall and the libffi Backendlibffi loaded at run time, and its ABI probed not tabulated
  4. Chapter 32The Extension ABIthe C ABI an extension compiles against, and why it never sees Value
  5. Chapter 33Concurrencythe GIL, per-thread registers, and switching the lock off

Around the Compiler

  1. Chapter 34Tooling Built on the ASTlint, highlight, profile, the REPL, and tools written in Raku
  2. Chapter 35Measuring, and Provingthe benchmark policy, the gates, and what did not pay

Appendices

  1. Appendix AThe Tag Setsthe VT, NK and regex-node tag sets
  2. Appendix BFlags and Environment Variablesevery flag and environment variable
  3. Appendix CSource Map and Glossarya source map, the rules that are easy to break, a glossary