May 15, 2008
Development of Rakudo Perl is continuing along quite nicely. Jonathan has been doing outstanding work with his one-day-per-week grant from Vienna.pm, primarily looking at classes, grammars, objects, etc., but also the occasional other features here and there.
We've also had some outstanding contributions from Stephen Weeks (Tene++ on IRC), who provided an implementation of placeholder parameters in Rakudo. In fact, Stephen's implementation works for both positional ($^a) and named ($:a) placeholder parameters, and even supports the @_ and %_ placeholders! There are still a few items to be tidied up -- especially when dealing with placeholder variables in the body of loops -- but I'm sure those fixes will be made soon also.
Chromatic has also continued to find
small fixes to Parrot that dramatically improve performance. For example, just yesterday he made a small tweak to Parrot's garbage collector that made a 15%-30% improvement in our "build rakudo" benchmark. We're continuing to find a lot of low-hanging fruit for improvement Parrot's overall speed.
Since Friday I've been able to do a fair amount of useful hacking on Rakudo and Parrot. I managed to close about fifteen or so tickets in the RT queue, which is nice.
In my last post I noted that the "fat arrow" syntax is now working, meaning that named arguments can be passed with:
foo(x => 1, y => 'hello');
The trick to getting this work was to get the operator precedence parser to again recognize "stop levels", such that it only parses expressions above (tighter) than a given precedence level. The code for this in the grammar looks something like
token fatarrow {
<key=ident> \h* '=>' <val=EXPR: 'i='>
{*}
}
The important part is the 'i=' parameter being passed to the <EXPR> subrule... it tells the operator precedence parser to only parse operators above "item assignment" precedence. (Look for %item_assignment in
STD.pm to see where the 'i=' comes from.) Yes, this approach doesn't exactly match the way STD.pm does it, but it's close enough until we get a more general expression parser into PGE's subrule calls.
Having the ability to control the range of expression parsing meant that I could also implement default values for parameters, as in:
sub foo($x = 3, $y = 'block') { ... }
Again, this involved getting the default_value rule to parse everything after an '=' up to a lower-precedence operator:
rule default_value {
'=' <EXPR: 'i='>
}
Since PCT already had support for default values on parameters, it was a fairly small change to the actions.pm module to get this to work.
Over the weekend I also looked a bit more at the official test suite, and cleaned up some tests in t/spec/S02-literals/ . In the process I found that Rakudo's Test.pm wasn't handling "todo" markers properly, and fixed that. The fudge-based preprocessor for the spec tests works extremely well -- it makes it very easy to mark sections of tests to be avoided when running the tests. A goal for May is to get the spectests and makefile targets updated enough such that "make spectest" in Rakudo tends to report relatively few errors.
As Stephen Weeks was performing his implementation of placeholder variables we also discovered a place where NQP (and PCT) run into problems trying to distinguish hash accesses from array accesses in various objects. At its core, Parrot uses integer versus non-integer indexes to distinguish between array and hash accesses, but NQP and PCT didn't provide a good way for high-level languages to make that distinction.
There was a lot of discussion about this on IRC, but after thinking about it a bit, I decided it was time for PCT to evolve a bit to do a better job of register handling. So, I spent the last 2-3 days doing just that, and we have some very good results -- about a 10% reduction in overall generated code size and substantially fewer gc-able objects created. There are a lot more details to share on this, but since this post is focused more about Rakudo and is getting a bit long, I'll write up the PCT details in a separate article.
Overall, there's a growing of things I'm eager to work on in Rakudo, and it's difficult to decide "what to do next", as all are useful or important. At this point I'm thinking that I may focus on updating protoobject and class handling next. Currently we have two separate protoobject systems in place: one for PGE and the compiler tools and another for Rakudo. On the surface they look fairly similar, but there are some minor differences and we really ought to be serving both needs from a single base. Plus getting the classes in Rakudo and PCT straightened out should make it easier for others to participate, so that seems like a Big Win for now.
Plus, I need to make sure all of my presentations are ready for this summer's conferences. Speaking of which, I have apparently neglected to provide links to materials for some of the talks I have already given this year, for those that are curious:
FOSDEM 2008 Presentation
"Perl 6"
http://video.fosdem.org/2008/maintracks/FOSDEM2008-perl6.ogg (video)
http://www.pmichaud.com/2008/pres/fosdem-perl6/ (slides)
Texas Open Source Symposium Talks
"An Introduction to Perl 6"
http://www.pmichaud.com/2008/pres/texasoss-perl6/
"Programming Parrot"
http://www.pmichaud.com/2008/pres/texasoss-parrot/
DFW.pm April 2008 meeting
"Rakudo Perl - Perl 6 on Parrot"
http://www.pmichaud.com/2008/pres/dfwpm-rakudo/Finally, on Saturday we're planning to have a "Parrot Bug Day" -- I hope to be around early in the morning and later in the evening to work with others to track down bugs and otherwise prepare for next Tuesday's monthly release.
Pm
May 15, 2008 01:46
May 10, 2008
For the past couple of weeks I've been mostly providing feedback and guidance to others on IRC, as well as trying to keep up with $otherjob. But this week I managed to get some coding in and continue progress.
Of course, it's great news that Jonathan++ has received a grant from Vienna.pm to continue his excellent work on Rakudo. He's making so much progress lately that I'm finding it difficult to keep up with him. Of course, that's a very good "problem" to have, so I'm not at all complaining.
Chromatic indicated that he had a
goal of reducing the number of open RT tickets for parrot to below 700, so I spent some time over the last couple of days closing out some of my older tickets. Earlier today were briefly down to 698, but then I found some additional Parrot bugs that brings us back to 700 as I'm writing this. But I'm sure we'll be able to close a few more quickly.
The Parrot developers could really use some help with reviewing the Parrot RT queue. Personally I think that a good number of those 700 open tickets must be irrelevant by now or otherwise already addressed. Coke wrote a nice set of guidelines for this at
http://www.parrotblog.org/2008/05/700-ticket-challenge.html, if you're interested in helping.
Several RT tickets had to do with updating PGE to include some syntax changes in Synopsis 5. Of course, changing PGE's syntax also means we have to update the various grammars that are using the older syntax, which takes a little while. I think I managed to convert all of the existing languages except for Plumhead, which looked a little more involved than most. So I'm hoping Bernhard will be able update that one soon.
I also cleared out a few RT tickets from the "perl6" queue, applying several useful patches and closing some out-of-date tickets. This leaves us with 22 new/open tickets for Rakudo, although I expect that number to grow as more people start playing with it and trying out various tests.
Tonight I updated Rakudo's interface into the operator precedence parser so that the 'fatarrow' syntax now works properly. So, we can use fat arrows for named parameters in subroutine and method calls. Soon I'll want to work on the hash and list composers, and then start to tackle list context and list assignment.
For the upcoming weekend I'm planning to update the spectests so that "make spectest" in Rakudo doesn't give back a ton of not-very-useful-error messages. As a result, we can't easily determine if a change to Rakudo causes something to break that was working previously. Fortunately, the 'fudge' script makes it much easier for us to maintain this. In an upcoming post I'll give an overview of spectests are being marked in Rakudo.
Spinclad on #parrot was remarking that was working on kjs'
Squaak tutorial, and wanted to initialize the @?BLOCK variable using pure NQP instead of the PIR List class that the tutorial currently requires. A couple of weeks ago I updated Parrot's ResizablePMCArray class to automatically provide the methods that the PIR List class was using, so I figured we'd be able to use those directly. Turns out it's not quite as simple as that, but NQP still lets us work around it a bit. What we ended up with was the following:
Protomaker.new_subclass('ResizablePMCArray', 'List');
our @?BLOCK := List.new();
The first line uses PCT's "Protomaker" object to create a new "List" class that is a subclass of Parrot's ResizablePMCArray. The second line then creates a new List object, and binds it to @?BLOCK. Since List inherits from ResizablePMCArray, we automatically get the shift/unshift methods that Squaak needs to do its work.
In fact, another good task will be to update the Squaak tutorial (or create another tutorial) and use some of the more recent PCT and NQP features that were added in April.
However, it's probably not a good idea to rely too strongly on Protomaker just yet. Currently we have two protoobject implementations -- one for PCT and another for Rakudo -- and after discussions with Jonathan and others I've decided that they really ought to be merged into a single implementation that is used by both. Protomaker may very likely disappear in the merged version (but we'll undoubtedly have something equivalent).
Anyway, lots to work on as usual, and I'm glad I have a few weeks of available coding time again.
Pm
May 10, 2008 04:13
May 06, 2008
JSON doesn't have native support for binary data? Really?
Someone, please tell me I'm wrong. (But only if you can back it up with documentation)
May 06, 2008 01:54
April 28, 2008
Last week I heard that someone was having trouble finding the Rakudo sources, and it was suggested that blogging about it here might provide an additional pointer. So, if you're looking for the sources to Rakudo Perl, it's currently held in the
languages/perl6/ directory of the Parrot repository.
Eventually Rakudo and other languages on Parrot will likely get separate repositories, but for now we all find it easier to keep everything in a single repository.
The architecture and layout of the Rakudo source code is described in
docs/compiler_overview.pod. This says what each source file does and how it all fits together.
Also, by popular demand, I've created a list of "milestones" for Rakudo Perl development and stuck them in the
ROADMAP. Reproducing the 2008-04-28 version here, we have:
* list context, list assignment
* return and control exceptions
* class, role, objects
* regex, token, rule, grammar
* selected libraries written in Perl 6
* modules
* junctions
* lazy lists
* slices
* multi sub & multi-method dispatch
* captures and signature handling
* operator overloading
* other S09 features (typed arrays, sized types)
* heredocs
* macros
* module versioning
While the milestones roughly in priority sequence, this list isn't meant to be rigid or strictly sequential in nature. If someone wants to work on a later milestone even though we haven't completed the earlier ones, that's certainly okay. This list just gives us a way to see where things are fitting in my head.
Suggestions and additions to the above list are welcome.
Pm
April 28, 2008 18:27
