← Contents

Chapter 2

The Shape of the Source

Before the mechanisms, the map. This chapter is the one to come back to when a later chapter names a file and you want to know what else lives near it.

The numbers

src/ holds about 147,000 lines of C++. That figure is misleading on its own, because 79,400 of them are generated Unicode tables — character names, properties, collation weights, normalization data — emitted from the pinned UCD and UCA 17.0 files in tools/ucd/. Nobody reads those, and nobody edits them.

The hand-written implementation is about 67,700 lines, and it is very unevenly distributed:

Two shapes stand out and both are deliberate.

Interpreter.cpp is enormous. It is the tree walk, and the tree walk touches everything: scopes, calls, operators, assignment, control flow, module loading, the FFI marshaller, the concurrency runtime. Splitting it by topic would mostly move #includes around, because the pieces share the interpreter's private state rather than a clean interface.

The method dispatcher is split across four files for one reason. It used to be a single 9,138-line function, methodCallInner, and that stopped being compilable in a reasonable time. It is now four ordered segments — Builtins.cpp holds the head, then MethodCallPart2, MethodCallPart3, MethodCallTail — each returning std::optional<Value>, where nullopt means "not handled here, try the next segment".

The critical property, stated in the source and worth repeating: these are segments, not categories. The chain is order-sensitive. Later arms deliberately catch what earlier ones decline. An arm belongs where its priority is, not where it reads nicely. MethodCallSegment.h exists so that the four files share one include prologue and cannot drift apart.

The library boundary

  src/main.cpp        the rakupp CLI          ─┐
                                               ├─► both link librakupp_rt.a
  generated stub.cpp  a --exe/--aot binary    ─┘

Everything except main.cpp and Repl.cpp compiles into librakupp_rt.a. The REPL lives in the executable rather than the library on purpose: nothing about an interactive session should be linked into the standalone binaries the compiling modes produce.

What each file is for

Front end

Values

Execution

Engines

Back ends and tooling

Boundaries

Reading conventions in the source

Three habits recur, and knowing them saves a lot of confusion.

*Comments explain why, and often carry the measurement.* A comment in this tree is rarely a restatement of the code. It is much more often the reason the obvious version was rejected, sometimes with a number attached:

// src/Value.h — the CowStr rationale, trimmed
// Value is copied by value everywhere ... so holding a bare std::string
// meant a long string was memcpy'd on each of those. The cost is
// O(length) per OPERATION, which makes any pure-Raku tokenizer O(n^2):
// JSON::Fast spent 13.9 s on a 421 KB document that Rakudo parses in
// 50 ms, and the profile was all copying, not parsing.

When this book explains a decision, it is usually expanding a comment like that one.

A "decided once" field is a fact about the syntax, not a cached result. Several node types carry a small mutable field that starts at a sentinel and is written on first evaluation:

// src/Ast.h
template <typename T> struct DecidedOnce {
    std::atomic<T> v;
    operator T() const { return v.load(std::memory_order_relaxed); }
    DecidedOnce& operator=(T x) {
        v.store(x, std::memory_order_relaxed); return *this;
    }
};

The atomic is not for synchronisation. It is there because a node is shared between threads, every writer computes the same idempotent answer, and a plain field would make that a data race that ThreadSanitizer correctly reports. Relaxed atomics make it defined at plain-load cost on the architectures that matter. Chapter 18 is entirely about what these fields hold and, more importantly, what they must never hold.

rt* functions are the compiled backend's vocabulary. Anything named rtAdd, rtIndexRef, rtAttrGet, rtCallB is a runtime entry point that Codegen emits calls to. They are declared in Interpreter.h and are the contract between the transpiler and the runtime — which is why they are inline where the fast path matters. Chapters 25 to 27 are about them.

Building it

cmake -S . -B build
cmake --build build -j

CMakeLists.txt builds librakupp_rt from all of src/ except main.cpp, then the rakupp executable. The glob is CONFIGURE_DEPENDS, but CMake caches it, so re-run the configure step after adding a source file.

The compiler matters more than usual here. Clang produces a binary between 1.2 and 2 times faster than GCC's on this codebase — the method dispatch chain and the tree walk are both inlining-sensitive in ways GCC handles less well — so Clang is what ships and GCC is kept as a portability gate. Link-time optimisation and -mcpu=native were both measured and both did nothing.

The test surface

The release checklist in docs/dev/RELEASING.md gates on all of them. Chapter 35 is about why the performance gate is there and what happens when it is skipped.