← Contents

Chapter 38

Concurrency

Raku has a full concurrency surface: start, await, Promise, Channel, Supply, react/whenever, Lock, atomics, and a scheduler. Raku++ implements it on real OS threads, running interpreter compute on all cores by default, with a global interpreter lock still there as an opt-in mode.

This chapter is the story of getting there in stages, because the stages are still visible in the code and each solved a specific class of bug.

Stage 1: per-thread execution registers

The state that belongs to one thread of Raku execution — its current lexical scope, the dynamic-variable chain, the recursion depth, the gather and supply collectors — was originally interpreter members. It is now a struct:

// src/Interpreter.h
struct ExecContext { /* Chapter 13 */ };
static thread_local ExecContext tctx_;
void saveCtx(ExecContext& c);
void loadCtx(ExecContext& c);

Being static thread_local, each real worker thread owns its own set, so interpreter execution needs no per-handover register swap. The save and load functions remain because the design allows a parked thread's registers to be stashed and restored; today they merely shuffle within a thread's own copy.

The same treatment was applied, thread by thread, to everything ThreadSanitizer reported:

// src/Interpreter.h
static thread_local Value* topicWriteback_;
static thread_local bool noAutothread_;
static thread_local int loopPhaserCtl_;
static thread_local std::vector<RedispatchCtx> redispatchStack_;
static thread_local std::vector<std::shared_ptr<ReactCtx>> reactStack_;
static thread_local bool t_isWorker;
static thread_local Value t_threadSelf;

(Two more live outside this header and are worth knowing about: the call-frame pool, and RVec's free list of small ValueList blocks — both per-thread caches of released memory rather than per-thread state, so neither needs a lock and neither is visible to another thread.)

The comment on the call registers records the scale: as plain members they were written by every call on every thread, and were TSan's top report — 2,761 lines on a program with no sharing in it at all.

One member deliberately stayed shared, and the reasoning is a good example of measuring rather than assuming:

// src/Interpreter.h — the current source line, for test diagnostics
// Written on EVERY statement, so a thread_local costs a TLV lookup per
// statement on macOS — measured +6% on loopsum. A relaxed atomic member is
// a plain store, defined under concurrency, and TSan-clean; the value being
// process-wide is the same arbitrariness diagnostics always had.
struct RelaxedLine {
    std::atomic<int> v{0};
    void operator=(int x) { v.store(x, std::memory_order_relaxed); }
    operator int() const { return v.load(std::memory_order_relaxed); }
} curLine_;

Thread-local storage is not free on every platform, and a diagnostic field does not need to be exactly right.

That lesson was learned again, expensively, on a field that does need to be right. Comparing this tree against the shipped v3.24.0 release found the plain assignment loop and the range-sum loop running 1.4 to 2 per cent slower than the release, and a bisect narrowed it to one line added to the ExprStmt arm of exec:

tctx_.curStmtExpr = e;   // which expression this statement is evaluating

tctx_ is thread-local, so on macOS that store is a real call to _tlv_get_addr — the second-heaviest symbol in an interpreter profile — and placing it at the top of the arm forces a resolution the rest of the arm would otherwise have folded into one. It costs about 34 instructions per statement executed: an empty loop body is unchanged, and every non-empty body moves by the same amount, which is how the cost was pinned to the statement rather than to the operation.

The difference from curLine_ is that this one cannot simply be made approximate. Cooperative next/last/redo read curStmtExpr to tell a bare loop control from one nested inside an expression, so the field has to be exact. The fix is to pay for it less often — set it only for statements whose expression can contain a bare loop control, which is a parse-time property of the node — or to hoist the thread-local resolution to the top of exec, where the function needs it anyway. Neither is done at the time of writing.

The counterweight is worth stating in the same breath, because it stops this reading as "thread-locals are bad". The ValueList free list of Chapter 12 is a new thread_local on the hottest path in the tree — every interpreted call touches it — and it is one of the largest wins in this book. A thread-local access costs what it costs; what matters is how much work rides on the access. One pointer store per statement is a bad trade. Removing a malloc per call is a good one.

Stage 2: the symbol-table freeze

The shared symbol tables — the class registry, the global environment, the named regex table, the loaded-module set, each class's method map — are mutated freely while the program is single-threaded. Once concurrency engages they must be treated as immutable, so worker threads can read them without a lock.

// src/Interpreter.h
std::atomic<bool> symbolsFrozen_{false};
void noteSymbolMutation(const char* what);

The flag flips when concurrency engages, and noteSymbolMutation is a tripwire wired into every structural writer. Under RAKUPP_FREEZE_TRACE it reports any post-freeze mutation and which thread did it.

That is the interesting part: the tripwire is empirical evidence for whether lock-free reads are safe, rather than an argument that they are. Behaviour is otherwise unchanged, so the instrumentation costs nothing in a normal build.

Stage 3: the GIL

// src/Interpreter.h
std::mutex gil_;          // held while running Raku
bool gilHeld_ = false;    // engaged once any thread is spawned
void engageGil();         // lazily lock on first async use

Only the holder may touch interpreter state. A thread drops it while blocked — in await, in a sleep, around a blocking syscall — so another can run.

The lock is lazy: a program that never spawns a thread never engages it and pays nothing.

Three operations manage the handover:

// src/Interpreter.h
void gilYieldNotify();              // unlock + bump a counter + notify
void yieldToWorker();               // drop the GIL until a worker progresses
bool yieldToWorkerFor(double secs); // …bounded
void sleepYield(double secs);       // sleep with the GIL released
bool gilPark();                     // release around a blocking syscall
void gilUnpark(bool wasParked);

gilPark has a strict contract, stated in the header: the parked window must touch no interpreter state — only thread-local buffers and syscalls. That is what lets a child-process wait release the lock so sibling workers can spawn their own children concurrently.

Safe points

A background worker doing pure compute has no I/O to yield at, so the interpreter weaves a cheap check into its hot loop:

// src/Interpreter.h
inline void safePoint() {
    if (!t_isWorker) return;                 // main-thread loops never park
    if (workerAbort_.load(std::memory_order_relaxed)) throw WorkerAbortEx{};
    if (++t_safePtCtr >= 4096) { t_safePtCtr = 0; workerYield(); }
}

Two jobs in four lines. It periodically hands the GIL back, so a compute-bound worker cannot starve the main thread. And it unwinds a worker whose result is no longer wanted, at shutdown, by throwing an exception that is deliberately not a RakuError — so a user's CATCH cannot swallow a shutdown.

The main-thread early return means the check is one predicted branch on a thread-local bool for every loop that is not in a worker.

Stage 3a: true parallelism, and then the flip

Worker threads run interpreter compute concurrently instead of serialising on the GIL — safe once the registers are thread-local and the symbol tables freeze. That arrived as an opt-in, and v3.0.0 made it the default:

// src/Interpreter.cpp — the constructor decides the mode
const char* g = std::getenv("RAKUPP_GIL");
const char* p = std::getenv("RAKUPP_PARALLEL");
bool gilWanted = (g && *g && std::string(g) != "0") ||
                 (p && std::string(p) == "0");
parallelMode_ = !gilWanted;

RAKUPP_GIL=1 selects the cooperative GIL — the escape hatch, the bisection tool and a CI leg — and RAKUPP_PARALLEL=0 is honoured as a synonym for symmetry with the old opt-in spelling. RAKUPP_PARALLEL=1 does nothing: it asks for what is already true.

The member's own initialiser still reads bool parallelMode_ = false;, which is a default the constructor overwrites on the next line. It is worth knowing because it is exactly the sort of line a reader — or a chapter — quotes as if it settled the question.

The few genuinely shared internals a parallel worker can still touch — the test counters and TAP output, the worker vectors — are guarded by one mutex. User data mutated without a Lock is the user's race, as it is in Rakudo.

For user containers there is a striped lock, and its design is the interesting part:

// src/Interpreter.h
struct ParStripe {
    std::unique_lock<std::recursive_mutex> l;
    ParStripe(const Interpreter& I, const void* p) {
        if (I.parallelMode_ &&
            I.liveWorkers_.load(std::memory_order_relaxed) > 0)
            l = std::unique_lock<std::recursive_mutex>(atomicStripe(p));
    }
};

Two conditions, not one. Parallel mode and live workers — because before the first spawn and after the last join, a single thread cannot race itself. A single-threaded program therefore pays two predicted branches and nothing else, default mode or not.

That second condition was not an optimisation for its own sake: the unconditional stripe tax was pushing compute-heavy Roast files past the timeout, in files with zero threads in them. The rule it encodes — the machinery must be free when one thread runs — is the gate every stage of this work had to pass.

Threads need big stacks

The tree walker recurses deeply, and the default stack for a non-main thread on macOS is 512 KB — a recursive Raku sub inside start {…} overflows it within about a hundred frames, producing a bus error and then a process that will not die at exit.

// src/Interpreter.h — BigStackThread
const size_t kStack = (size_t)256 << 20;   // 256 MiB, virtual
pthread_attr_setstacksize(&attr, kStack);
if (pthread_create(&h_, &attr, entry, fn) == 0) joinable_ = true;
else { std::unique_ptr<Fn> g(fn); g->f(); }   // creation failed: run inline

The reservation is virtual and committed only as used. Windows gets the same through a small shim kept in Runtime.cpp so <windows.h> stays out of the widely-included header. If thread creation fails, the work runs inline rather than being lost.

Worker bookkeeping, and two real crashes

// src/Interpreter.h
struct WorkerSlot {
    BigStackThread th; std::shared_ptr<std::atomic<bool>> done;
};
std::vector<WorkerSlot> workers_;
void addWorker(BigStackThread&& th, std::shared_ptr<std::atomic<bool>> fin);

addWorker is the only safe way to register a worker, and the header explains why in terms of a crash report. In parallel mode spawns happen from worker threads too — a start block tapping a supply spawns the interval ticker — and the old per-site "reap, then push" pattern mutated the vector from several threads at once. vector::erase corrupted it and the process segfaulted, with the report naming erase inside a supply-interval spawn on thread 5 of 292.

The second crash is inside the fix:

// Collect finished slots under the lock, JOIN THEM OUTSIDE IT. Joining
// under sharedMut_ deadlocked: the joinee had set its done flag but was
// still unwinding through code that itself wants sharedMut_ (a worker
// spawning a nested worker) — the reaper held the lock waiting for the
// joinee, the joinee waited for the lock.

That is the classic shape — hold a lock across a join — and it is worth remembering that the first fix for a concurrency bug introduced it.

There is also backpressure, parallel mode only:

void throttleSpawn() {
    if (!parallelMode_) return;
    while (liveWorkers_.load(std::memory_order_relaxed) >= 384)
        { /* reap finished workers, wait for the herd to thin */ }
}

A tap-and-close loop over interval supplies — four spawners times a thousand activations, each a real thread with a 256 MiB virtual stack — outran teardown and exhausted the address space. Above the cap a spawner reaps and waits.

Under the GIL this must stay off, and the reason is a nice illustration of how the two modes differ: a spawner spinning while holding the lock would keep the very workers it waits for from ever finishing.

The user-visible layer

TypeBacking
PromisePromiseState in ext: mutex, condition variable, result or cause, and a list of .then continuations fired once on settle
Channelshared state plus a stripe from the atomic pool
Supplyon-demand blocks run through a SupplyTapCtx; live suppliers register a tap record
react/whenevera ReactCtx with an event queue, a live-source count, and its own mutex and condition variable
Locka std::recursive_mutex, recursive because protect re-enters
Threada BigStackThread plus a per-thread identity value
$*SCHEDULER.cuea worker with a CueState cancellation flag

Two details in there earn a mention.

A tap's teardown lives on the context that owns it. An earlier design kept a close-callback stack as an interpreter member, which was shared across worker threads — two concurrent react blocks corrupted it. Making it thread-local fixed the crash and cost 6% on an unrelated benchmark by reshuffling thread-local storage layout on macOS. The right answer was neither: the context object already travels with the block, is already thread-correct, and is free.

whenever activations are deferred. Rakudo runs the react body first and only then activates subscriptions, so a say after a whenever prints before the first emitted value. The synchronous drains queue into ReactCtx::deferred, which the react implementation runs after the body.

Signals, and a thing that must be turned off

Signal supplies use the self-pipe trick: a handler writes a byte, a lazily spawned dispatcher thread reads it and runs the whenever block under the GIL. When the react block ends, externally wired taps must be closed explicitly, or the dispatcher keeps firing the handler after the block is gone — the symptom was a second Ctrl-C re-invoking a stop method on an already-stopped service.

Separately, and simply: SIGPIPE is ignored process-wide. Without that, a TCP server dies when a client disconnects mid-write. It is set once on the big-stack entry thread.

Testing it

Concurrency bugs do not appear in unit tests. What found the ones in this chapter:

Honest limitations