🦋 110. is rw vs is raw in Raku

The cryptic title should not stop you from seeing bits of the regular Raku code. Namely, the two traits that you can add to function arguments: is rw and is raw.

These two traits may look confusing because both allow changing the passed variable:

sub f1($x is rw) {
    say $x;
}

sub f2($x is raw) {
    say $x;
}

my $a = 42;
f1($a); # 42
f2($a); # 42

Neither of the functions changes the value of $a, but that’s not that important now. They both can, would you add, for example, $x++ in each function:

sub f1($x is rw) { 
    $x++;
    say $x;
}

sub f2($x is raw) {
    $x++;
    say $x;
}

my $a = 42;
f1($a); # 43
f2($a); # 44

But what if you pass a constant integer value instead of a variable container?

f1(42);
f2(42);

It is quite obvious that it is not possible to change a bare integer, which was not boxed in a variable. The call of the first function (having is rw) fails:

Parameter '$x' expected a writable container, but got Int value
   in sub f1 at rw.rk line 1
   in block <unit> at rw.rk line 15

Let’s comment it out and see what happens to the second call (with is raw):

Cannot resolve caller postfix:<++>(Int); the following candidates
 match the type but require mutable arguments:
     (Mu:D $a is rw)
     (Int:D $a is rw)
 
 The following do not match for other reasons:
     (Bool:D $a is rw)
     (Bool:U $a is rw --> Bool::False)
     (Mu:U $a is rw)
     (Num:D $a is rw)
     (Num:U $a is rw)
     (int $a is rw)
     (num $a is rw --> num)
   in sub f2 at rw.rk line 7
   in block <unit> at rw.rk line 16

Oh, my. The error message is big, but the main idea is clear. You cannot increment a constant integer.

Now step back to the original program. Remove increments and pass constant values:

sub f1($x is rw) { 
    say $x;
}

sub f2($x is raw) {
    say $x;
}

f1(42);
f2(42);

The first function still refuses to work:

Parameter '$x' expected a writable container, but got Int value
   in sub f1 at rw.rk line 1
   in block <unit> at rw.rk line 9

Even despite the fact that $x is not modified in the function, its signature tells the compiler that the argument must be mutable.

Comment that line and call f2(42) only.

$ raku rw.rk
42   

It works! The is raw trait did not affect the expectation: if you do not modify the variable ‘passed by reference’, just accept it.

Using Raku — a free book

Let me announce the second edition of my Using Perl 6 book. This time, it is published under the new name, Using Raku.

The book is available in the PDF format for free. Later, paper copies will also be available.

Download the Using Raku book now

This is not a bare text replacement s/perl6/raku/. This edition contains a number of changes in the text and program examples that the readers sent after reading the first edition.

Published by DeepText
Amsterdam, 13 October 2019
ISBN 978-90-821568-8-1

The book is free, but donations are much appreciated!

I hope you will enjoy both the book and the newly-named language!

OK, Raku

So, Larry Wall agreed to rename Perl 6 to Raku. Here’s the quote with a nested quote in it:

I am in favor of this change, because it reflects an ancient wisdom:

“No one sews a patch of unshrunk cloth on an old garment, for the patch will pull away from the garment, making the tear worse. Neither do people pour new wine into old wineskins. If they do, the skins will burst; the wine will run out and the wineskins will be ruined. No, they pour new wine into new wineskins, and both are preserved.”

This blog will soon be renamed from Perl 6 Inside Out to Raku Online, and most likely you will read it from raku.online instead of perl6.online (DNS changes are still not always that instantaneous as language renaming).

The major two ideas behind renaming is to:

  • Allow Perl 5 to continue its development, and
  • Avoid Perl in the name of Perl 6 to have new adopters.

There were a number of long and painful discussion rounds, and I would not say that they were human-oriented. There are still a few issues that you cannot cancel, but I am ready to reset it.

The Raku language is a great language and it is ways better than the Perl 6 as it was originally thought 19 years ago. It is not only about changing the syntax of the for loop; during the past two decades, the language design got some killing features, among which are, first of all:

  • Grammars — the next level of world-standard regular expressions, and
  • Features for parallel and concurrent computations (including a lot of syntactic pleasures such as reduction operator).

Even without the other features of the language, these two items can potentially attract new adopters of the Raku language.

But.

But what is missing from the rename proposal is the fact that the current Rakudo compiler is not actually production-ready. I’ll explain.

When I was promoting Perl 6, I was always mentioning that it has a compiler that can be used even in production. Indeed, it does not crash anymore (at all!), it is quite reliable, feature-complete and more or lest fast at the execution phase. But fast enough for test and joy. Not for production.

My plan for the next round of Perl 6/Raku life cycle is to:

  • Establish an annual international Raku conference (having the RakuCon name in mind), and
  • Initiate the compiler speed boost.

As for the conference, I propose we have the first RakuCon conference on Cyprus in May 2020, at the place we proposed for the PerlCon 2020 earlier this year in Riga. I am sure that in the long term, PerlCon and RakuCon will part, so there’s no need to wait before it happens naturally. I would prefer to force it. The presence of Raku at PerlCon is up to their organisers. From my side, RakuCon is open for Perl 5 talks (the most interesting topics are Perl 5 adopting the Perl 6/Raku features and changes in the Perl 5 compiler(s) to make it faster).

As for the compiler question, I am less confident in this area. I am confident in that if the compiler is faster, the language gets stronger perspectives.

Let me tell the truth, even if the Rakudo team will hate me once again. The Rakudo compiler is over-engineered. It has at least three layers that include NQP and MoarVM, which helped to build the compiler for the complex language, but slows up real-life applications.

I see two ways to achieve the goal (the goal is, if you missed it, to make Raku a popular programming language):

  • Create a new Raku compiler.
  • Learn Rakudo to produce binary or bytecode output that can be executed at machine speed. I think it is fine that only a subset of the language will be allowed here.

From the Perl 5 and the former Perl 6 community, I am seeking for help to implement the above items. In a few weeks, we will approach the companies to explain our view of Raku perspectives and to ask for supporting the conference and possibly the development. If you want to help my team, please indicate your intention.

I want to reset all previous grudges and start it over.

This is the only way Raku can prosper and thrive.

Ah, yeah, and tell me if it is Ráku or Rakú.

On renaming Perl 6

In short: I am against renaming.

The longer story

Once in a while, another round of the non-ending battle starts, and this time, unfortunately, I was the one who allowed to start it from a public stage. There were a lot of noise around the keynote speakers of PerlCon 2019 in Riga, and in the end, two speakers cancelled their participation, and we as the organisers replaced them with other prominent members. Actually, initially we were going to have three Perl 6 keynotes, and in the end it became more balanced: two Perl 6 and one Perl 5 talk.

I was very happy about the presence and the content of the Perl 5 talk, as well as about the Perl 6 Concurrency one. In the third keynote, actually, it was proposed to rename the language from Perl 6 to Camelia.

The same day, an issue in the problem-solving repository was open. Soon after that, a pull request was created to replace all mentions of Perl 6 (including the filename extensions and methods like .perl) to Raku.

Why Raku? Because the discussion led to the idea that Raku is better then Camelia. (Raku is the name that was proposed a year ago as an alias of Perl 6).

A bit of a history

Let me express what I feel about this all movement. I am following the Perl 6 development for many years, starting from its very beginning. Years ago, I was surprised how easily the development teams change their internal mechanisms. If you can recall that, there were IMC (intermediate code/compiler), PASM (Parrot assembler), PBC (Parrot bytecode) and a lot of other abbreviations which you can only understand if you follow the development of the language and is subscribed to different mailing lists of different development teams.

Originally, it was thought that Perl 6 compiler will use a virtual machine, and Parrot was the first attempt, which failed due to internal communication issues. Later, an attempt to switch to other virtual machines was made, including the JVM. As it was thought that Parrot virtual machine dit not serve the needs of Perl 6 and was instead aiming to supporting other languages, the MoarVM project appeared.

While the Perl 6 language is a beautiful language, the language design is quite complicated to be implemented, which resulted in the fact that we now have only one working compiler, Rakudo. It became very stable and reliable and can even work in production in a lot of applications. Even in those applications that require high performance. Rakudo is slow at compiling, but MoarVM is fast at executing.

Who is the owner of Perl 6

The Perl programming language is created by Larry Wall and it was released in 1987. Since then, it turned to its fifth version, which is still in active development (let me refer you to the Day 1 keynote of PerlCon 2019).

The Perl 6 programming language was originally thought as a continuation of the same Perl language, with a note that it should be a community rewrite. Does that mean that Larry is not the author of Perl 6 any more? No. He still is.

What is proposed in the renaming issue is to rename the language, and to vote on that within the current development team. The development team is working on both Rakudo compiler and the language specification, so it is a valid institute to raise discussion about the language. But. But there is one strong but: Larry is still the author of the language, and it’s his right to keep or break the language name.

In the renaming issue, it was only once mentioned that it is possible to rename the fork (read: implementation) and keep the language name unmodified. To support that, I am referring you to Larry’s words in his keynote given in 2017 at the Perl Conference in Amsterdam. He said there that using different names is fine when it is done for marketing purposes.

Up to date, we have no direct statement by the author of the language that he wants or agrees to rename it.

Do I need Perl 6?

I have been programming for the internet since 1999 and while my native language is C++, I am considering myself as a Perl programmer.

When Perl 6 was announced, I was among the first who wanted to start using that. I created the first web site in the world that was running on Perl 6 (give me a link if you disagree). The site was running on the very first implementation of Perl 6 written in C (even before Parrot) and was using pre-compiled byte code to serve pages.

Since that time, I changed a lot of places of work, being a developer, an CTO, a business owner and a bare life-lover. Today, I am much less connected with writing code (whatever code, not only Perl), and my activities shifted to other fields.

But still, Perl 6 for me is a beautiful language with a great potential. And it is very interesting and pleasant to blog about it, to help organising events, and to promote the language in different ways. So, despite I am not very much able to use it in my practice, I am willing the language to become widespread.

Back and forth

Earlier this year, Perl 6 entered the TIOBE index. Yes, somewhere at the end of the list and not very stable, but it was obvious that we can make its positions stronger if we make some simple SEO correction and include the Perl 6 programming phrase into our publications, as this is how TIOBE ranks the languages.

What is proposed now, effectively cancels all efforts of many people who was working during the recent years to make Perl 6 more popular and to demonstrate how great it is.

Once again, the Perl 6 path is making a sharp turn and loops itself. Intermediate language, Parrot assembler, Parrot bytecode, Java virtual machine, MoarVM, Perl 6, Raku, Camelia. Hey, what’s up? It is an endless game of naming and tool changes or a way to make the language popular? What currently happens, is another round of an idea that renaming can help. This is like inventing a name and buying a domain name for a start-up that has no chances to work as it mostly focuses on its name rather than being the best among the others.

What is behind the scene

Unfortunately, and what is mostly sad for me personally, I have seen a bit more than only public discussions that you can see too.

Conference attendance cancellations are not caused only by those reasons that were publicly announced. There are other things happened under the hood. People are being bullied, words are misinterpreted, and I have no wish to make that public. I can only hope that the current heroes will understand how badly they play.

Titus 3:10-11. 10 Warn a divisive person once, and then warn them a second time. After that, have nothing to do with them. 11 You may be sure that such people are warped and sinful; they are self-condemned.

It is not correct at this point to demand, to vote, and to approve to rename the language.

On a separate line, I want to thank all the developers that made Perl 6 possible. But I want to distinguish between the desire to constantly working on the process flow rather than making the product.

A simple solution

The best solution is to allow the Perl compiler (interpreter, translator, whatever) to accept both Perl 5 and Perl 6 code. This was, actually, the idea of the first Perl 6 compiler projects. It was supposed that the compiler will be able to determine the language even without a use vX instruction, but by just looking at the code. For example, does the file start with class? It is a Perl 6 program.

As the Perl 5 and Perl 6 languages became much more separated in their syntax, it is a very simple to determine the language by reading a few lines of code. This is not an impossible task for a computer program, isn’t it? Let the common compiler decide which backend language to use at this particular case. /usr/bin/perl stays in its place and does the job.

📘 The Brainfuck interpreter written in Perl 6

Create the interpreter for the Brainfuck language.

Brainfuck is an esoteric programming language that has a small set of instructions, each of them a single punctuation character.

It is assumed that the Brainfuck program has built-in data memory, which is an array of integers, and a pointer to the currently selected item. The two instructions, + and -, increment and decrement the current element. The < and > instructions move the data pointer one position to the left or to the right.

Another two instructions, . and ,, either print the current element using its values as the ASCII codepoint (in theory, it can be Unicode) or read a character from the standard input and put its numeric value to the current element of the data array.

Finally, [ and ] create loops. If when the program reads the closing bracket character, the current data element is not zero, the program returns to the corresponding opening bracket. In the case the program reads an opening bracket and the current data element is zero, the whole block between the two matching brackets is skipped. This option can also be used for embedding comments.

All other characters are ignored. This gives the ability to separate the program instructions with spaces or newlines, as well as to add comments just next to the main code. The comments should simply not include the main characters used as the code instructions.

Online, you can find many examples of the Brainfuck code. We’ll test our program on the following ‘Hello World!’ program:

++++++++[>++++[>++>+++>+++>+<<<<-]>+>+>->>+[<]<-]>>.>---.+++++++..+++.>>.<-.<.+++.------.--------.>>+.>++.

Now, we are ready to create the interpreter of Brainfuck in Perl 6.

First, read the source code to the $program variable, and pass it to the main interpreter subroutine:

my $program = $*IN.slurp;
brainfuck($program);

The parser first creates the containers it needs for the process: @program holds the program as an array of characters; the $program_pointer is set to the beginning of it; @data_memory keeps the data, and its current position is also set to 0 via $data_pointer.

sub brainfuck($program) {
    my @program = $program.comb('');
    my $program_pointer = 0;
    my @data_memory;
    my $data_pointer = 0;

Now, iterate over the program instructions.

    while $program_pointer < @program.elems {

At this point of the main loop, the @program[$program_pointer] element contains the current program instruction. We are using the givenwhen block to understand the meaning of it and make an action. The first four commands are straightforward:

        given @program[$program_pointer] {
            when '>' {$data_pointer++}
            when '<' {$data_pointer--}
            when '+' {@data_memory[$data_pointer]++}
            when '-' {@data_memory[$data_pointer]--}

Let’s skip the comma command for now and move on to the input dot. The input command is using the @data_memory array and the chr method to translate codepoints to characters.

            when '.' {
                print @data_memory[$data_pointer].chr
            }

Finally, the loop commands [ and ]. Their behaviour depends on the value of the current data element @data_memory[$data_pointer]. If the condition is met (i. e., if the current element is zero for [ and non-zero for ]), the $program_pointer must be moved to the position of the matching bracket.

To simplify the program, the code to find balancing brackets is placed to separate functions, _move_forward and _move_back. They modify the value of the program pointer, which is passed as an argument.

            when '[' {
                $program_pointer =
                    _move_forward(@program, $program_pointer)
                unless @data_memory[$data_pointer];
            }
            when ']' {
                $program_pointer =
                    _move_back(@program, $program_pointer)
                if @data_memory[$data_pointer];
            }
        }

All other instructions, which are not listed in the when clauses, are simply ignored. After the current instruction has been processed, the program pointer is moved to the next position:

        $program_pointer++;
    }
}

Finally, here is the code for the functions searching balancing brackets. They move either forward or backwards and count the opening and closing brackets. The $level variable is increased if the program finds the bracket, which is not the correct pair.

sub _move_back(@program, $program_pointer is copy) {
    my $level = 1;
    while $level && $program_pointer >= 0 {
        $program_pointer--;
        given @program[$program_pointer] {
            when '[' {$level--}
            when ']' {$level++}
        }
    }   
    return $program_pointer - 1;
}

sub _move_forward(@program, $program_pointer is copy) {
    my $level = 1;
    while $level && $program_pointer < @program.elems {
        $program_pointer++;
        given @program[$program_pointer] {
            when '[' {$level++}
            when ']' {$level--}
        }
    }   
    return $program_pointer - 1;
}

The subroutines use the same approach with the givenwhen keywords for dealing with command characters as in the main loop.

To prevent infinite loops in case of the incorrect program, both subs check if the $program_pointer reaches the beginning or end of the program. Notice that because the $program_pointer is modified inside the subs, it is declared as is copy in the signatures of the subs. The return value is intentionally decremented by one to compensate the subsequent increment of it in the main loop: $program_pointer++.

The interpreter is complete. Save the ‘Hello World!’ program in a file and pass it in the command line:

$ perl6 brainfuck.pl < helloworld.bf 
Hello World!

As an exercise, modify the interpreter so that it understands the , command. You need to update the givenwhen list in the main loop with the code that reads the character from the input:

when ',' {@data_memory[$data_pointer] = $*IN.getc.?ord}

The $*IN.getc returns Nil when there are no more characters in the input. Try to catch this situation to avoid filling the data memory with empty data. Here is a test program that copies the input to the output:

>+[[>],.-------------[+++++ +++++ +++[<]]>]<<[<]>>[.>]

Another useful modification would be error handling. There are a few places in the program where increments or decrements in one of the pointers may go out of the array ranges. Add the code that checks that to display an error message. To make theprocess easier, use some simple debugging code like the one below to visualise the position of the program pointer and data state at each iteration of the main loop:

say $program;
say ' ' x $program_pointer ~ '^';
say @data_memory[0..$data_pointer - 1] ~ ' [' ~
    @data_memory[$data_pointer] ~ '] ' ~
    @data_memory[$data_pointer + 1..*];

📘 Converting Morse to text using Perl 6

Convert the Morse sequence to plain text.

To save efforts in typing the decoding table, we can use the %code hash from Task 98, Text to Morse code, and create the ‘inversed’ hash, where the keys are the Morse sequences, and the values are letters or digits:

my %char = %code.kv.reverse;

Printing this variable shows its contents in the following way:

{- => t, -- => m, --- => o, ----- => 0, ----. => 9, ---.. => 8, 
--. => g, --.- => q, --.. => z, --... => 7, -. => n, -.- => k, 
-.-- => y, -.-. => c, -.. => d, -..- => x, -... => b, -.... => 6, 
. => e, .- => a, .-- => w, .--- => j, .---- => 1,.--. => p,
.-. => r, .-.. => l, .. => i, ..- => u, ..--- => 2, ..-. => f,
... => s, ...- => v, ...-- => 3, .... => h, ....- => 4, ..... => 5}

Despite the fact that Perl 6’s output does not print quotes, all the keys and values in %char are strings. The next step is to replace the sequences from the keys of the hash with its values. The small difficulty is that, unlike the text-to-Morse conversion, a regex has to search for the sequence of a few characters (dots and dashes), so it must anchor to the boundaries of the Morse characters.

The built-in << and >> regex anchors for word boundaries assume that the words are sequences of letters and digits, while Morse sequences are dots and dashes. Let’s use a space to serve as a separating character. To simplify the task, just add an additional space to the string before decoding it.

my $text = prompt('Morse phrase> ') ~ ' ';
$text ~~ s:g/(<[.-]>+) ' '/%char{$0}/;
$text ~~ s:g/\s+/ /;
say $text;

📘 Converting text to Morse code using Perl 6

Convert the given text to the Morse code.

Converting text to the Morse code is a relatively easy task. The solution is to replace all the alphanumeric characters with the corresponding representation in the Morse code.

In this solution, all the other characters are ignored and are removed from the source string. In the Morse code, letters are separated by the duration of one dash, and words are separated by the duration of approximately 2.5 dashes, so in the program, one space is used for separating characters, and three spaces separate the words.

The above logic is programmed in the series of replacements. First, lowercase the whole phrase (there is no distinction between lower- and upper-case letters) and then remove all the non-alphanumeric characters and increase the distance between the words. Finally, replace each remaining printable symbol with the corresponding Morse sequence.

my %code = (
    a => '.-',      b => '-...',    c => '-.-.',
    d => '-..',     e => '.',       f => '..-.',
    g => '--.',     h => '....',    i => '..',
    j => '.---',    k => '-.-',     l => '.-..',
    m => '--',      n => '-.',      o => '---',
    p => '.--.',    q => '--.-',    r => '.-.',
    s => '...',     t => '-',       u => '..-',
    v => '...-',    w => '.--',     x => '-..-',
    y => '-.--',    z => '--..',    0 => '-----', 
    1 => '.----',   2 => '..---',   3 => '...--',
    4 => '....-',   5 => '.....',   6 => '-....',
    7 => '--...',   8 => '---..',   9 => '----.'
);

my $phrase = prompt('Your phrase in plain text> ');

$phrase.=lc;
$phrase ~~ s:g/<-[a..z0..9]>/ /;
$phrase ~~ s:g/\s+/ /;
$phrase ~~ s:g/(<[a..z0..9]>)/%code{$0} /;

say $phrase; 

Let us test this on a random phrase:

$ perl6 morse.pl 
Your phrase in plain text> Hello, World!
.... . .-.. .-.. ---  .-- --- .-. .-.. -..  

The conversion table takes the biggest part of the program.

The regexes show how character classes are created in Perl 6.

A characters class with a range of symbols:

<[a..z0..9]>

A negative character class, which matches with any character other than the one from the range:

<-[a..z0..9]>

These character classes list all the allowed characters that can be encoded by the given %code hash. It is also possible to use \w and \W or <alnum> and <!alnum> instead of the above regexes if you are sure that the input string is pure ASCII. All the regexes in the program come with the :g adverb to make them global. Regex matching uses the double tilde ~~ operator for both matching and replacement.