Planet Perl Six

chromatic: Perl 6 Design Minutes for 19 November 2008

Mon, 01 Dec 2008 19:57:52 +0000

The Perl 6 design team met by phone on 19 November 2008. Larry, Allison, Patrick, Jerry, Jesse, Will, Nicholas, and chromatic attended.

will let someone else discuss the Parrot Developers Summit
released Parrot 0.8.1 yesterday
mostly smooth, but didn't get in all of the features we wanted
looking at the plan for the next few months to figure out my next features

Patrick:

even if Parrot 0.8.1 didn't give all of the features we wanted, it was a useful deprecation point
lots of things merging into trunk today
the Parrot Developers Summit went very well this weekend
concentrated on bugfixes and getting lexicals to work
we were 95% of the way on Saturday
then I added one last feature, and everything fell apart
reviewing how Parrot manages contexts to figure that out
something's wrong with the collection of contexts
haven't worked on it much since Sunday, but working on it again today
updated the Rakudo roadmap in the Parrot repository
that looked pretty good
otherwise, little changes here and there
closed 10 or 12 tickets yesterday
more of the same coming up for me

Jerry:

organized the PDS over this weekend
most important thing we did was create the roadmap for Parrot 1.0
release date will be March 2009
it'll be a stable platform for high-level language implementors
Allison, chromatic, and I posted more details on use.perl.org and parrot.org
the most visible work product is the Parrot roadmap, which lists important and critical tasks for Parrot 1.0 and beyond
haven't made much progress on my grant in the past week
turning back to that now that the conference is over
hope to do a whiteboarding session with a couple of local developers later this week
then I'll have a draft for review

Allison:

extremely pleased with the way the Summit went
makes a big difference to be able to say that we can deliver 1.0 in March
seems to be making a difference internally as well
working on the IO architecture
about to do the first merge back to trunk here shortly
have the copyright assignment from TPF to the Parrot Foundation as well

Jesse:

is Rakudo part of that?

Allison:

it's explicitly excluded
that copyright stays with TPF

Jesse:

but NQP stays with Parrot?

Patrick:

as does PGE and the Parrot Compiler Toolkit

Larry:

the tryfile program now reports time and memory usage of the compilation it does
spent a lot of time refactoring the LTM to get rid of string processing and storage
switched over to a trie implementation, which is closer to a DFA implementation
uses less memory and is faster
not a great deal of either, but it's progress
a bad bug in STD's operator precedence parser
built nested pre- and post-fix operators inside out
fixed that and hopefully improved the algorithm at the same time
building STD now builds a Storable grammar of all of the lexers
should start up faster now
lexers can now reuse the patterns of the lexers of the parent class if they haven't changed
cuts down memory usage
refined the LTM tiebreaking rules as part of the trie switch
if there's a tie, the first tiebreak is "which if the two rules had more literal characters matched at its front?"
if you match a specific keyword exactly, it takes precedence over a generic rule which matches \w*
seems to be DWIMmier
if that fails, it falls back on order tiebreaking
thinking a lot about iterator semantics and the API, mainly at Daniel Ruoso's request
several different ways to go
several different use cases
probably a matter of allowing any of those semantics underneath, but picking a good default
probably neither strictly lazy nor strictly eager
about four different gradations of laziness versus easiness

Jesse:

attempting to figure out where we are with Synopses and spec tests and implementations
when Parrot migrates to Trac, does Rakudo migrate to RT?

Allison:

it stays in RT on the perl.org infrastructure

Jesse:

is there a broad announcement of the plan?

Allison:

the roadmap is the main written information we produced
we should write out the general migration too
other languages will make their choices too as they move to public repositories

Patrick:

before the March release, Rakudo will move into its own repository on perl.org

Will:

in general, ripping stuff out of Parrot
trying to make ParTcl go faster
if anyone wants to help with that, let me know

Jesse:

Tim Bunce has been doing great in Perl 5
and he's positive about Parrot

Allison:

profiling tools aren't on the list for 1.0
but if they get done before then, great!

Nicholas:

did someone name S17 from "Threads" to "Concurrency"?

Jesse:

the canonical source for the Synopses is going to move to the Pugs repo

Patrick:

there will be announcements to the mailing lists

Jerry:

will there be any changes to version numbers and date updates?

Patrick:

they stay up to date pretty well
people can be mindful of that
we can put commit hooks on them there too

Jesse:

we won't meet next week
I'll send mail confirming that

Patrick Michaud: .subst now honors :g(lobal) flag

Mon, 01 Dec 2008 04:51:51 +0000

Today I added in support for the :global/:g flag in the .subst method (S05); now we can do things like:

> my $i = 0; say "There's more than one way".subst(/e/, { ++$i }, :global);
Th1r2's mor3 than on4 way

Previously the .subst method would only work on the first match in a string. Rakudo still doesn't recognize the case where the :global flag is part of the regex, but at least we now have at least one concise mechanism for performing global search/replace on a string.

Within the closure argument one can use $/, $0, $1, etc. to refer to the matched portions of the string in the replacement. Getting that to work meant that we needed to fix the binding of $/ within closures -- it turns out the mechanism we were trying to use previously (!OUTER) can't really work in Parrot. As a bonus, fixing this simplified actions.pm slightly, and removed several internal subroutine calls that were being performed at the start of each closure.

Of course, the pattern and replacement arguments to .subst can be simple strings, but any variable substitutions in the replacement string then refer to the outer environment and not the values from the matched string.

Now we just need to verify and add a few more tests for this. :-)

Patrick Michaud: infix: now works on Ranges in Rakudo

Fri, 28 Nov 2008 19:42:38 +0000

Moritz found a bug in the infix:<Z> operator -- it wouldn't properly zip over a range:

09:04 <moritz_> rakudo: say (<a b c> Z 1..10).perl
09:05 <p6eval> rakudo 33300: OUTPUT[elements() not implemented in class 'Range'␤current
instr.: 'infix:Z' pc 4039 (src/gen_builtins.pir:2555)␤]

Normally we would expect this to return a list of lists:

(("a", 1), ("b", 2), ("c", 3))

The problem here was that infix:<Z> was trying to determine the size of each of its elements by using the 'elements' opcode and then get the individual elements by using a subscript. This failed for two reasons: (1) Ranges don't support postcircumfix:<[ ]>, and (2) Rakudo's implementation of Range didn't work for the 'elements' opcode.

The real approach, as with most things dealing with lists, is to use iterators. I changed the algorithm that it creates an iterator for each argument to infix:<Z>, and then walks those iterators in parallel to build the elements of the final result list. When any of the argument iterators is empty, we're done building the list for infix:<Z>.

As a general rule, I suspect most list operators and builtin functions should not be counting the number of elements or using subscripts to access individual elements of a List. There are likely a few more functions like this in Rakudo that we should be replacing.

Pm

Patrick Michaud: Lexicals have arrived

Tue, 25 Nov 2008 20:10:33 +0000

Earlier today I finally got lexicals to work properly in Parrot (and Rakudo). This means that recursion now works properly, as do a whole host of other things. It resulted in closing at least eight tickets in the Rakudo and Parrot RT trackers, and we're still looking to see how many spectests it enables.

We're still missing list interpolation, list assignment, and a few other basic things, but lexicals support was a big blocker for many things we've wanted to do. I'm glad it's finally done.

Many thanks to (irc nicks) jonathan, infinoid, masak, moritz, particle, bacek, chromatic, rgrjr, Tene, tewk, allison, and many others who helped with debugging and testing.

chromatic: Automatic Variables in a GCd Language

Fri, 21 Nov 2008 08:22:36 +0000

One of the persistent bottlenecks in Parrot is that its naive, simple-to-describe stop-the-world mark-and-sweep garbage collector can be expensive. Even though it's mostly O(2n), there's one characteristic of almost every program which makes n far larger than you might want it to be: most GCable elements have short-to-very-short lifespans. That's even true in Perl.

Cleverer GCs use tricks such as memory partitioning or generations to handle this problem. When you need more memory, you first try to reclaim the youngest GCable elements, on the theory that most of them have very short lifespans. Everything that survives that collection gets promoted to an older generation or copied to an older partition which you sweep less frequently. Mark and sweep, at least as currently implemented in Parrot, always walks the entire live set of GCable elements every time the GC runs to completion. Even if a PMC anchored in the root set has survived the last thousand GC runs, it still gets marked during every run.

We haven't fixed that yet. (Replacing a garbage collector in a running system not exactly designed for pluggability of garbage collectors is not an easy task.) Andrew Whitworth's proof of concept for Google's Summer of Code 2008 gave us a lot of good information to improve our GC, but that task is still ahead of us.

If you've programmed in C or C++, you're familiar with automatic variables. These get allocated on the stack -- int x; char y;. If you've ever returned a pointer to a stack-allocated value, you've had fun debugging the results. (Parrot r32065 fixed an amusing bug related to this in Parrot.)

Heap variables you allocate yourself with malloc/calloc and must explicitly free yourself. The advantage here is that you can pass pointers to them with impunity, no matter where you are in the call graph with regard to the stack location where you allocated them.

The memory characteristics of individual programs vary, but it's likely that a program uses many more stack variables than it does heap variables. (I use the terms in a metaphorical sense. What I really mean is "Variables that don't escape the current compilation unit versus variables that do", but I don't want to talk about escape analysis.)

I don't know of many garbage collectors which can distinguish between stack-style GCables and heap-style GCables -- again, not without doing clever tricks with escape analysis. This is one place where I do wish sometimes we had a tiny kernel with the VM bootstrapped atop; the semantics of C just don't let you express or analyze the memory characteristics of the program in any way that lends itself to automatic lifetime tracking.

It's also difficult to reason from the code in general about the expected lifetime of a GCable element. Every PMC or STRING returned from a C function has effectively escaped, and we can't predict what any future consumer of that API will do with the results.

Like many problems which are unsolveable in general, there are specific exceptions. Sometimes you can read the code and verify that you have what would be in C a stack variable -- even if it's a GCable entity.

I added a pair of functions in Parrot r32313 called temporary_pmc_new and temporary_pmc_free. The documentation has quite a few caveats that I won't explain (figuring that they're scary enough that anyone who doesn't understand their implications should not use them). Basically, they're a way of creating a new PMC for the singular case where you know it's specific and entire lifespan.

I mention this to say that I teased a nice 5.14% speedup out of our multidispatch algorithm today by turning two PMCs into temporary PMCs. Parrot r32963 shows how easy the patch was to write, once I had identified the hotspot and verified that this was an appropriate optimization.

The ultimate solution is to improve our garbage collector so that it can recycle lots of young garbage very quickly, but I'll take a 5% optimization for now if I can spend two or three minutes coding and ten minutes reviewing profiling runs.

perl6.announce: Parrot 0.8.1 "Tio Richie" Released! by chromatic

Wed, 19 Nov 2008 06:39:30 +0000

On behalf of the Parrot team, I'm proud to announce Parrot 0.8.1 "Tio
Richie." [1] Parrot is a virtual machine aimed at running all dynamic
languages.

Rat Creature #1: Comrade! We are about to feast! Quick, get your fat
carcass behind this bush and get ready!

Rat Creature #1: Hello, small mammal.... Could you step in here for
a moment? I've got something to show you....

Fone Bone: Can't you show me out here, where I've got runnin' space?

Rat Creature #1: No! No! Please! Step in here -- your friend the
dragon isn't around, is he?

Fone Bone: Hey, Ted! Where you goin'?

Ted: You're on yer own, Bone! (exit stage left)

Rat Creature #1: Quick, comrade! Start the cooking fire!

Rat Creature #2: No. You called me fat.

Fone Bone: Ted! Wait for me!

Rat Creature #1: (pulls Bone into the bush) Well, well... Look who's
joined us for supper... Go start the cooking fire!!

Rat Creature #2: No. You called me fat.

Rat Creature #1: No?!!! What do you mean, no?!!!

Rat Creature #2: And it's not the first time you've done it
either....

Rat Creature #1: Comrade... be reasonable! I wasn't thinking -- I
was trying to catch our dinner -- this isn't the time -- I take it
back.... you're not fat.

Rat Creature #2: Too late!

Rat Creature #1: Please, comrade! I just want to chop him up for the
stew!

Rat Creature #2: And that's another thing. I'm tired of stew! I want
to put him in a crust and bake a light fluffy quiche!

Rat Creature #1: Quiche?! What kind of food is that for a monster to
eat?! .... Listen, do you think you could come back in half an hour?
We'll have this straightened out by then!

Bone: (runs away)

Rat Creature #1: (beat) Why didn't you stop me?

Rat Creature #2: Why should I? You're so smart!

time passes

Fone Bone: (hanging from a branch in a waterfall) Those rat
creatures would have to be pretty stupid to follow me on to this
frail, little branch!

Rat Creatures: (follow Bone on to the frail, little branch)

Fone Bone: Stupid, stupid rat creatures!! (branch breaks)

-- "Bone", by Jeff Smith

Parrot 0.8.1 is available via CPAN (soon [2]). You may also see the Parrot
download instructions[3]. For those who would like to develop on Parrot, or
help develop Parrot itself, we recommend using Subversion [4] on our source
code repository [5] to get the latest and best Parrot code.

Please note the updated Parrot Roadmap [6] and our goals leading to the 1.0
release.

Parrot 0.8.1 News:
- Implementation
+ added CPAN module Storable 2.12 as a configuration and build dependency
+ removed the pseudo PIR opcode 'addr'
+ added the 'box' opcode
+ fixed 'pop_eh' handling in PIR libraries and examples
+ removed usage of .return for tailcalls (use .tailcall instead)
+ removed 'get_hash' and 'get_array' from Capture PMC and Capture_PIR
+ improved debugger and HLL coordination
+ allowed MMD primitive autoboxing
+ fixed all known memory leaks in PIR "Hello, world!"
+ NCI signatures now JITted on x86-32 platforms (Windows and Linux)
+ made the .const directive take a quoted type name instead of a constant
+ made IMCC more re-entrant
- Languages
+ Rakudo
- refactored Junctions implementation
- added "fire and forget" tool to rebase/rebuild/test Parrot and Rakudo
- updated container/reference semantics
- added more builtin methods and functions
- improved support for multilevel namespaces
- added support for .Str, .succ, .pred in user-defined classes
- implemented pointy blocks on if/loops
- increased STD.pm convergence
- added %*VM hash
- improved MMD candidate sorting
- improved integration of Num and Int
- implemented increment on protoobjects
- added initial support for MAIN subs
- added .PARROT method
+ Pipp
- added some predefined constants.
- added implemention of the function basename().
+ Cardinal (Ruby)
- Added initial support for the classes Proc, Continuation, Queue, Dir,
File
, and FileStat
- fixed various minor bugs
- fixed broken Regexes
- Compilers
+ PCT
- added '.isa' method to PCT::Node
- cleaned up 'immediate block' handling
- allowed arguments to immediate blocks in loops/conditionals
- metaclass objects can now 'add_method'
+ PIRC
- integrated macro processing in PIRC's lexer
- integrated heredoc lexer in PIRC executable
- added preprocess and heredoc-preprocess commandline options
- integrated the PASM grammar into PIRC
- added a register allocator to optimize the built-in vanilla allocator
- code cleanups and documentation
- added 'make test' target
+ TGE
- updated to work with new namespace/classname syntax
- Deprecations
+ PARROT_API will become PARROT_EXPORT
+ :lexid will become :subid
- Miscellaneous
+ Documentation
- updates and additions to the Parrot book (see docs/book/)
+ OpenGL
- minor fixes to Parrot's base OpenGL bindings
- converted OpenGL PIR examples to Perl 6 (two complete, one WIP)
- added simplified OpenGL PIR example for NCI JIT testing
+ Tests
- further improvements to parallel tests
- tests no longer leave filesystem breadcrumbs

Thanks to all our contributors for making this possible, and our
sponsors for supporting this project. Our next release is 16 Dec 2008.

Enjoy!

References

1. http://parrotcode.org/
2. http://search.cpan.org/dist/parrot
3. http://parrotcode.org/source.html
4. http://subversion.tigris.org/
5. https://svn.perl.org/parrot/trunk/
6. https://trac.parrot.org/parrot/wiki/ParrotRoadmap

chromatic: Parrot 0.8.1 "Tio Richie" Released!

Wed, 19 Nov 2008 03:35:14 +0000

On behalf of the Parrot team, I'm proud to announce Parrot 0.8.1 "Tio Richie." Parrot is a virtual machine aimed at running all dynamic languages.

Rat Creature #1: Comrade! We are about to feast! Quick, get your fat carcass behind this bush and get ready!
Rat Creature #1: Hello, small mammal.... Could you step in here for a moment? I've got something to show you....
Fone Bone: Can't you show me out here, where I've got runnin' space?
Rat Creature #1: No! No! Please! Step in here -- your friend the dragon isn't around, is he?
Fone Bone: Hey, Ted! Where you goin'?
Ted: You're on yer own, Bone! (exit stage left)
Rat Creature #1: Quick, comrade! Start the cooking fire!
Rat Creature #2: No. You called me fat.
Fone Bone: Ted! Wait for me!
Rat Creature #1: (pulls Bone into the bush) Well, well... Look who's joined us for supper... Go start the cooking fire!!
Rat Creature #2: No. You called me fat.
Rat Creature #1: No?!!! What do you mean, no?!!!
Rat Creature #2: And it's not the first time you've done it either....
Rat Creature #1: Comrade... be reasonable! I wasn't thinking -- I was trying to catch our dinner -- this isn't the time -- I take it back.... you're not fat.
Rat Creature #2: Too late!
Rat Creature #1: Please, comrade! I just want to chop him up for the stew!
Rat Creature #2: And that's another thing. I'm tired of stew! I want to put him in a crust and bake a light fluffy quiche!
Rat Creature #1: Quiche?! What kind of food is that for a monster to eat?! .... Listen, do you think you could come back in half an hour? We'll have this straightened out by then!
Bone: (runs away)
Rat Creature #1: (beat) Why didn't you stop me?
Rat Creature #2: Why should I? You're so smart!
time passes
Fone Bone: (hanging from a branch in a waterfall) Those rat creatures would have to be pretty stupid to follow me on to this frail, little branch!
Rat Creatures: (follow Bone on to the frail, little branch)
Fone Bone: Stupid, stupid rat creatures!! (branch breaks)
"Bone", by Jeff Smith

Parrot 0.8.1 is available via CPAN (soon), or follow the Parrot download instructions. For those who would like to develop on Parrot, or help develop Parrot itself, we recommend using Subversion on our source code repository to get the latest and best Parrot code.

Please note the updated Parrot Roadmap and our goals leading to the 1.0 release.

Parrot 0.8.1 News:

- Implementation + added CPAN module Storable 2.12 as a configuration and build dependency + removed the pseudo PIR opcode 'addr' + added the 'box' opcode + fixed 'pop_eh' handling in PIR libraries and examples + removed usage of .return for tailcalls (use .tailcall instead) + removed 'get_hash' and 'get_array' from Capture PMC and Capture_PIR + improved debugger and HLL coordination + allowed MMD primitive autoboxing + fixed all known memory leaks in PIR "Hello, world!" + NCI signatures now JITted on x86-32 platforms (Windows and Linux) + made the .const directive take a quoted type name instead of a constant + made IMCC more re-entrant - Languages + Rakudo - refactored Junctions implementation - added "fire and forget" tool to rebase/rebuild/test Parrot and Rakudo - updated container/reference semantics - added more builtin methods and functions - improved support for multilevel namespaces - added support for .Str, .succ, .pred in user-defined classes - implemented pointy blocks on if/loops - increased STD.pm convergence - added %*VM hash - improved MMD candidate sorting - improved integration of Num and Int - implemented increment on protoobjects - added initial support for MAIN subs - added .PARROT method + Pipp - added some predefined constants. - added implemention of the function basename(). + Cardinal (Ruby) - Added initial support for the classes Proc, Continuation, Queue, Dir, File, and FileStat - fixed various minor bugs - fixed broken Regexes - Compilers + PCT - added '.isa' method to PCT::Node - cleaned up 'immediate block' handling - allowed arguments to immediate blocks in loops/conditionals - metaclass objects can now 'add_method' + PIRC - integrated macro processing in PIRC's lexer - integrated heredoc lexer in PIRC executable - added preprocess and heredoc-preprocess commandline options - integrated the PASM grammar into PIRC - added a register allocator to optimize the built-in vanilla allocator - code cleanups and documentation - added 'make test' target + TGE - updated to work with new namespace/classname syntax - Deprecations + PARROT_API will become PARROT_EXPORT + :lexid will become :subid - Miscellaneous + Documentation - updates and additions to the Parrot book (see docs/book/) + OpenGL - minor fixes to Parrot's base OpenGL bindings - converted OpenGL PIR examples to Perl 6 (two complete, one WIP) - added simplified OpenGL PIR example for NCI JIT testing + Tests - further improvements to parallel tests - tests no longer leave filesystem breadcrumbs

Thanks to all our contributors for making this possible, and our sponsors for supporting this project. Our next release is 16 Dec 2008.

Enjoy!

Patrick Michaud: Updated rakudo ROADMAP

Tue, 18 Nov 2008 06:03:52 +0000

Tonight I updated Rakudo's ROADMAP file to reflect the progress we've made since it was last done (August 2008, YAPC::EU). At first I thought there wouldn't be much to update, but it turns out we've accomplished a fair bit.

More importantly, more infrastructure pieces are falling into place so that the remaining items become even easier than they were before. I expect to see good progress during the next month or so.

This weekend many of us attended the Parrot Developer's Summit to plan the way forward to Parrot 1.0 (March 2009). In parallel several of us also made plans for Rakudo's development in that time -- our goal is to have Rakudo on its own release cycle by then. I very much like the way the time-based monthly release cycle has worked for Parrot development, so we will undoubtedly continue this in Rakudo. Development for Rakudo will continue to track the Parrot trunk for a while, and we'll time Rakudo releases to follow a few days after each Parrot release.

We'll put together more details as we get closer; until then we're continuing to work on feature development.

During my airplane travel and at the summit I was able to get a lot of work on fixing lexical variables in Parrot. Unfortunately I've also found several places where Parrot was not properly allocating and freeing contexts, so those need to be reviewed and fixed before we can merge the branch back into trunk. Initial tests show that the design itself appears to work as expected, so now it's just a matter of reviewing and repairing context handling in Parrot. I'm expecting (hoping) to merge the branch back into trunk sometime on Wednesday (after tomorrow's Parrot release), which will close out quite a few RT tickets.

chromatic: A Roadmap from the Parrot Developer Summit

Sun, 16 Nov 2008 06:40:03 +0000

I'm at the Parrot Developer Summit this weekend, with Allison, Patrick, Jerry Gay, Klass-Jan Stol, Stephen Weeks, Kevin Tew, Jonathan Worthington, Jeff Horwitz, and Abhijit Mahabal. Our main goal was to create a roadmap toward production-ready releases of Parrot.

One of my early suggestions was to define a single-sentence vision for each milestone release.

Our tentative plan (to which we all agreed) is to continue monthly releases indefinitely. Language implementors can (of course, as currently) track our trunk, or our monthly releases, but we've decided to produce two milestone releases each year which represent a good stable point on which to host a language.

Our plan is to produce the first of these milestone releases in March and the second in July. Subsequent milestone releases will occur ever six months: January and July. Here's our vision list for the next six milestone releases:

March 2009: a stable API for language implementors, and the point at which we believe Parrot is an attractive platform to begin implementing languages, at least for people who don't want to track our trunk.
July 2009: integration and interoperability, the point at which we provide the next generation of tools to make implementing languages even easier.
January 2010: production readiness, the point at which Parrot is ready for your business.
July 2010: portability, the point at which we branch out beyond our three core platforms to other devices and systems.
January 2011: independence, the point at which Parrot is independent of the last remnants of non-Parrot build systems.
July 2011: magical wishlist items, the point at which Parrot supports wonderful exotic features that many other virtual machines may never support.

We're concentrating our efforts on the March release. Our schedule is aggressive, but it's workable. (We invested two hours in two agile planning exercises to identify, prioritize, and cluster features into the six milestones we've identified.) There's plenty of work to do, especially for novices and interested neophytes. In particular, we could use people to help us identify missing documentation and to ensure that it's clear and appropriate -- not to mention people interested in running tests, triaging bugs, and learning how to use the system.

We'll have more to say in the coming weeks, but I think you'll be very impressed at our progress over the next couple of releases.

chromatic: Perl 6 Design Minutes for 12 November 2008

Fri, 14 Nov 2008 07:10:32 +0000

The Perl 6 design team met by phone on 12 November 2008. Larry, Allison, Patrick, Jerry, Will, Nicholas, Jesse, and chromatic attended.

Allison:

we have two sponsors now, BBC and ActiveState
starting conversations with a few others
it helps to have two already
mainly focused on the IO milestone milestone
will merge back in two stages
stage one is getting the new IO system working
you can use it as an object
adds just one test file
I can have all of the different architectures report any failures with that test file
will merge in the bigger change later
though it's a simpler change
replace calls to the existing system with calls to the new system
hope it'll be a smooth transition
fixed up some calling conventions stuff last night for Andrew Whitworth
some pieces weren't working for him
committed a small patch to fix them, and explained it

Jerry:

the first Parrot Developer Summit is this weekend
for people who don't know about it, it's too late to attend
there may be a dozen or so of us meeting
intent is to hammer out a roadmap for 1.0
lots of questions on the channel and mailing lists about when there will be Parrot 1.0
good, I suppose

Patrick:

Christmas Eve!

Jerry:

they've given up asking about Perl 6

Jesse:

good!

Jerry:

I don't have an answer yet
but our work this weekend will let us decide what we need to release 1.0
also had a grant approved
designing and implementing Perl 6 command line syntax
already started the work
researching other dynamic languages and compilers
most languages have just grown their syntax over time
it's kind of messy
have put together some ideas
discussed them with Larry
expect that to continue, until I have a design document that I can send for review
then I'll move on to implementation
all laid out in the grant proposal on TPF's site

Jesse:

expect me to harass you occasionally

Patrick:

things are going well
lots of additions to Rakudo this week
I fixed the string semantics
classes can override how objects report stringification
Jonathan, Jerry, and I fixed Rakudo's container semantics
it does references correctly
fixed a lot of bugs and closed a lot of tickets
Jonathan started and I finished getting pointy blocks to work on flow control statements other than for
waiting for answers on repeat while and repeat until

Larry:

me too

Patrick:

when one of us gets the answer, he should let the other one know
completely refactored Rakudo's junctions
worked on the assignment metaoperators, such as += and *=
work correctly on protoobjects
I'm refactoring those now
will cause regression in one test that doesn't work right anyway
Rakudo is now over the 4500 passing test mark
we should get quite a few more over the next week or so
Jonathan and I will continue closing things out
worked more on the design of lexicals
continuing to update the document on my website
need to figure out autoclose in my head
Jonathan and I will hack on that this weekend
I could use his expertise in Parrot internals
also had a grant approved

Larry:

went to the hackers conference this last weekend
high point was teaching Don Knuth how to decode his Korean email

Jesse:

I thought he had no email!

Larry:

refactored the compile-time variables
there are no version-specific ones remaining
now $?OS, $?DISTRO, and $?VM variable
generalized matching against pairs to call a method corresponding to the name on whatever the object is, matching against whatever the argument is
sort of inside out as to what named arguments in BUILD routines do
refactored the :pair-named methods
they were kind of grungy anyway
thinking about whether strings ought to have methods like .r and .w and .x
or whether you ought to coerce to some sort of File type

Patrick:

I lean toward coercion

Larry:

everyone does
I just want to think it through
decided that .perl isn't special to junctions
if you use it on one, it autothreads
if you stringify that junction, it produces a result as if you'd done .perl on the whole junction
stringification on a junction produces Perl-style syntax
if you smart match a hash against a pair pattern, it tests the key and the value for matching
factoring out various amounts of boilerplate from the Perl 5 emitted code
simplifying code
fixed bugs with node wrapping
confused people looking at ASTs
chopped out TRE
discovered that my stupid LTM emulation ran twice as fast as the TRE code
that was eye-opening
STD now recognizes ::T as defining a valid type for later in the same scope
did some work to set up the infrastructure for STD to handle lexical scopes
nested pads
should have declarations in the right scope
part of the motivation
besides better error handling
set up a mechanism to define a prelude
freeze the lexical environment for the prelude
then slurp that up into STD
mundane functions and operators will probably end up there
have most of an implementation of a trie engine
that'll replace my stupid LTM
should evolve eventually into a full DFA
almost works
should run faster
and cut down on the on-disk cache
for various definitions of "should"

mostly fixing bugs
trying to keep blockers from blocking people
adding a little polish here and there to make people work better
thinking about our release process post 1.0
will talk about that this weekend

Will:

spending most of my time deleting things from Parrot
ripping out deprecated features
everything that's in there, some goes back to 0.5
but everything in there now we can rip out before the next release
sad that I'll miss the summit this week

Nicholas:

I'm the new grant manager for Dave Mitchell doing 5.10.1
announced the end-of-life for 5.8.x
surprised that no one told me I couldn't say that
if the Parrot Foundation is capable of raising money from people, how come TPF isn't?

Allison:

I tried to do something similar for TPF four years ago
it was very uncomfortable for the Perl community
lots of pushback
they wanted guarantees that no company would have control over the development of Perl

no one has control over the development of Perl!

Allison:

also the amount of money required for Perl 5 is much larger
it's not about money for Parrot
four memberships for Parrot is enough to push toward 1.0
we're a small project

Jerry:

have Perl 5 releases been sponsored before?
seems like a good thing

Jesse:

last week in China for the Beijing Perl Workshop
it was a lovely workshop
chatted with a lot of people about the state of Perl 6 and how they could get involved
offered to make lots of introductions
haven't taken anyone up on that yet
the Perl problem in China is that they're having a hard time finding competent Perl hackers
people are afraid to learn Perl because that'll make them unpalatable to the workforce
universities only teach C
people study on their own to learn C# and Java
most of the hiring firms really want C# and Java from their developers

Patrick:

in Rakudo, over the past week, I'm seeing lots of segfaults in different places
suspect it might be something that Jonathan and I did
have no clue where to start looking
it shows up in the regression suites
can point you to specific instances of things that fail

Will:

I can help narrow it down to PIR if you have a specific test that fails

Patrick:

they really float around
showed up in the assignment tests this morning
a couple of revisions later, they don't show up later
only some of them does -G help
the weird one this morning
if I run it from Parrot with the command line, I get a segfault
from the test harness, I don't
that seems backwards

Allison:

the test harness has more strict flags turned on

Patrick:

at one time, it used a different runcore

if there's a short PIR test case, we may be able to do a binary search

Patrick:

it's hard to give a short PIR case
the segfault moves around so much

Patrick Michaud: The world according to .PARROT

Thu, 13 Nov 2008 22:45:36 +0000

In Rakudo we have a number of places where bits of Parrot poke out from underneath the covers at inconvenient times; masak, moritz, and chrisdolan have been particularly good at finding these, especially when dealing with strings in Rakudo.

When we get HLL mapping fully working in Rakudo then a lot of these will go away; in the meantime I've been trying to explain what's happening "behind the scenes" that cause the seemingly bizarre behaviors. But yesterday I decided it would be better to provide a way to remove the covers entirely...

So, Rakudo now has a .PARROT method that reports the underlying Parrot data type for an object. This is in contrast to .WHAT, which gives back the (sometimes imperfect) Perl 6 view of an object.

Here's an example involving %*ENV -- normally we expect values in %*ENV to be (Perl6) Str objects, but the current implementation of %*ENV tends to give us String PMCs. But rakudo considers Parrot String and Perl 6 Str to have the same .WHAT protoobject, so it looks like it's a 'Str'.

> say %*ENV; /home/pmichaud
> say %*ENV.WHAT;
Str

The new .PARROT method lets us view the true identity of an object:

> say %*ENV.PARROT;
String

This still doesn't fix the problem, but at least it gives some transparency into what's going on.

I also adjusted the value semantics for the String PMC so that it auto-promotes to a Perl 6 Str object whenever it's asked for its scalar value. That should make it easier to get things into the form needed to get work done, at least until we can complete our other transitions that make this more seamless.

Patrick Michaud: Rakudo any/all/none junctions now collapse

Thu, 13 Nov 2008 20:18:28 +0000

For those who are wondering what happened to my journal posts over the past couple of months -- I've been focusing on some underlying Rakudo/Parrot design issues and supporting others more than writing code or prose. Also, I think I needed a bit of a break.

I've decided that it'll be easier (and thus more likely) for me to increase visibility on Rakudo progress if I blog about new features and decisions as they happen rather than try to batch them up for a post every few days. I do still plan to come up with "weekly summaries" as well, but I figure the immediate posts will be more satisfying for me and more helpful for others.

So, here's the first such post -- earlier this week I refactored Rakudo's Junction implementation to be cleaner and more correct, and bacek++ provided a patch to collapse duplicate values of one, any, and none Junctions. We had to wait for confirmation from perl6-language for this, which we just got today. So, we now have:

> say any(1,2,2,1,3,2).perl;
any(1, 2, 3);
> say (any(1,2,3) % 2).perl;
any(1, 0);

Although any/all/none junctions collapse, one() junctions do not:

> say (one(1,2,2,1,3,2) % 2).perl
one(1, 0, 0, 1, 1, 0)

This is because duplicate values change the boolean meaning of a one() junction, so we can't simply collapse them. As Larry says, "... one() junctions produce bags rather than sets."

As a result we've been able to unfudge more spectest for rakudo -- we're currently (r32625) passing 4598 spectests.

Moritz Lenz (Perl 6): Perl 6 Tidings for November 2008

Sun, 09 Nov 2008 21:22:08 +0000

Specification

A couple of small improvements to the spec made me a bit happier this past week:

There's now a Nil type. A Nil represents the empty list, but unlike a normal List object it is undefined in scalar context. It is returned by a bare return statement.
A cleanup of the global variables added (among other things) a $?VM variable, which holds informations about the backend being used to execute some code. I like this very much because currently some tests use the unspecced $?PUGS_BACKEND variable, which of course breaks other implementions.
File test operators look like this in Perl 6 $filename ~~ :e. This used to boil down to calling the method :e in class Str, which was a bit messy, because colons are not part of an identifier, and thus :e wasn't really a valid method name. Now this is special-cased in the smart match operator instead, and the method is named without the colon.

The smart links in the synopsis documents at http://perlcabal.org/syn/ now contains syntax hilighted code.

Implementations

SMOP

In his report to the TPF grant comittee Daniel Ruoso informed us about his work to integrate SMOP into Perl 5 by means of writing an XS module.

I'm not sure how much you can actually do with it right now, but it seems like an important proof-of-concept.

Rakudo

Jonathan and Patrick refactored Rakudo's handling of containers and values, to great success. This resulted in quite a few tickets being closed, and some more passing spec tests. Great work!

It's now also possible to define custom stringification routines for your classes (what's called "overloading" in Perl 5).

Last but not least, both Patrick Michaud and Jerry Gay received grants from Ian Hague's fanastic donation. Patrick will work on protoregexes and longest token matching (LTM) as well as the prerequisites to use external libraries (which is needed if you want to write the prelude in Perl 6 - guess what the plan is...).

(I've tried to explain the importance of LTM in this article, but I'm not sure if I succeeded. Anyway, it's essential for parsing Perl 6 correctly.

chromatic: Marketing Position

Sat, 08 Nov 2008 11:29:48 +0000

Sometimes I wonder what would happen if we gave Perl 6 the slogan "Yes we can!" and promoted it as "Real, effective change!" Maybe people could stop feeling ashamed of programming again.