Parrot 2.4.0 "Sulfur Crest" Released | parrot.org | Log: irclog.perlgeek.de/parrot/today | Nopaste: nopaste.snit.ch:8001 | GSOC Students: trac.parrot.org/parrot/wiki/GSoCersStartHere
Set by moderator on 18 May 2010.
whiteknight purl msg allison Can you take a look at TT #1222 and TT #1219 for me? I'm cleaning up all this code in the ns_func_cleanup branch, but these comments are too cryptic and old for me to decipher. Looks like some design decisions might need to be made about the NameSpace VTABLE interface 00:02
purl Message for allison stored.
mikehh make corevm/make coretest FAILs - see: nopaste.snit.ch/20672 00:03
perlcritic fails on 18 files (out of 599) - all Flag comment TODO and/or XXX
all other tests PASS (pre/post-config, smoke (#34090), fulltest) at r47073 - Ubuntu 10.04 amd64 (gcc)
pmichaud whiteknight: (#1662) Perl 6 basically requires that namespaces be able to act like hashes, fwiw. 00:04
whiteknight pmichaud: They do act like hashes, mostly. But there are some very crypic comments throughout the codebase that suggest that comments either are not-hashlike enough, or weren't hashlike but now they are (in which case the comments can disappear) 00:05
I don't know which
pmichaud I suspect the latter
at one time I don't think that NameSpace extended Hash, but now it does. But I'm really speculating there. 00:06
whiteknight namespace uses a very confusing combination of set_pointer_keyed_str and set_pmc_keyed_str for various purposes 00:10
If I could just get final confirmation, I would delete a lot of these stupid TODO notes 00:12
nopaste "cotto" at 192.168.1.3 pasted "dependency-related parallel build failure" (34 lines) at nopaste.snit.ch/20675 00:13
cotto_work mispaste
pmichaud some of namespace's oddities with set_pmc_keyed_str and the like is because namespaces get used for attaching methods to classes (I think) 00:17
and in that respect, namespaces do more than act like simple hashes because they're tied to classes
anyway, afk -- dinner. back in 60. 00:18
dalek rrot: r47074 | whiteknight++ | branches/ns_func_cleanup/src/namespace.c:
cleanups. Move all the static functions togethers, fix some docs, general code cleanups, etc
00:20 tcurtis joined 00:23 cotto_work2 joined
darbelo whiteknight: That's mean, now I have to review your commit to find out what that 'etc is about. 00:25
cotto_work makes a note 00:27
jimk nods at #parrot and gives him a small kipper 00:28
darbelo decides to add an 'etc' at the end of all commit messages, to see if it works on other people.
00:30 kid51 joined
dalek rrot: r47075 | whiteknight++ | branches/ns_func_cleanup/src/namespace.c:
more fixes. Including fixing some stupid errors I introduced in a previous refactor.
 00:52
00:55 plobsing joined
tcurtis (whoever implemented load_bytecode of foo.pbc when foo.pbc doesn't exist but foo.pir does) += 50 00:59
whiteknight darbelo: I'm just scrolling up and down over that file, cleaning the living shit out of it 01:00
no rhyme nor reason
dalek rrot: r47076 | whiteknight++ | branches/ns_func_cleanup (5 files):
Parrot_find_name_op -> Parrot_ns_find_named_item
 01:10
rrot: r47077 | tcurtis++ | branches/gsoc_past_optimization (5 files):
Fix typos and codingstd, start adding tests for PAST::Transformer::Dynamic, regen manifest.
pmichaud tcurtis: I'm around now :) 01:15
tcurtis pmichaud: alright. 01:17
pmichaud: would you like to have our discussion here in #parrot or privately? 01:22
whiteknight in #parrot! very interested onlookers present :) 01:23
kid51 builds tcurtis' branch 01:24
kid51 builds whiteknight's branch too
whiteknight kid51++
my next step is to fix src/oo.c. That file is a goddamn cesspool 01:25
01:27 eternaleye joined
kid51 Are the t/compilers/opsc/*.t tests expected to be slow. 01:27
?
dalek kudo: da0437f | jonathan++ | src/core/operators.pm:
Remove cheating versions of whatever-ers that are no longer used.
 01:28
kudo: 0c65df6 | jonathan++ | build/PARROT_REVISION:
Bump to a PARROT_REVISION that's not busted on Win32.
kudo: e659dd5 | jonathan++ | src/Perl6/ (2 files):
Implement bracketed infixes.
kid51 whiteknight: PASS
pmichaud in #parrot is fine
whiteknight w00t
pmichaud (sorry, wife called me a way for a bit)
kid51 tcurtis: PASS
tcurtis kid51: what whiteknight said. 01:29
After you brain-dump your thoughts, I have a minor question I'd like to ask you. 01:31
kid51 whiteknight: smolder.plusthree.com/app/projects/...ails/34092 01:34
tcurtis: smolder.plusthree.com/app/projects/...ails/34093
whiteknight: You'll have to add documentation to src/namespace.c to get it to pass t/codingstd/c_function_docs.t 01:37
whiteknight kid51: yeah
dalek rrot: r47078 | whiteknight++ | trunk/src/oo.c:
[oo] no.
 01:43
rrot: r47079 | jkeenan++ | branches/ns_func_cleanup/src/namespace.c:
[codingstd] split function call to meet linelength standard.
kid51 tcurtis: make codetest PASS in your branch 01:45
01:49 jsut joined
pmichaud tirsprru. tjere 01:55
oops
tcurtis: sorry, there's a small storm brewing here at the house... have to go calm some tempers a bit
tcurtis pmichaud: alright.
cotto_work whiteknight++ #no 01:58
01:58 JimmyZ joined
pmichaud tcurtis: okay, have one short thing to finish here and then I'll be back in about fifteen for my brane dump 02:06
02:07 snarkyboojum joined 02:11 snarkyboojum_ joined 02:14 snarkyboojum joined 02:19 snarkyboojum_ joined 02:21 parthm joined 02:26 abqar joined
pmichaud okay, back 02:28
tcurtis: still around?
tcurtis pmichaud: I am.
pmichaud excellent
okay, I'll start writing. feel free to interrupt with questions or comments (this goes for all)
first I need to set a background, though, because having gone several times into the battles of writing dynamic language translators for parrot, I have a bit of a pessimistic bias regarding how much optimization we can get 02:30
that shouldn't be taken as a discouragement -- I'm hopeful that this is simply because I've been looking in the wrong places or at things the wrong way.
at the same time, I also know that many of the places that we typically think are good candidates for optimizing end up not working for dynamic languages, or at least not for Perl 6. 02:31
part of that is the nature of the language, part is some of the core design decisions Parrot has made when we collectively didn't know any better
so, as things progress I think I can be a voice of hard experience with Parrot, and that experience has taught me where a lot of the pitfalls lie for general-purpose optimization 02:33
but while from time to time I'll say "that can't possibly work", I hope you'll be able to ignore me long enough to prove that it can :)
(which is effectively how I managed to write the tools I've written -- by ignore the people who've said it couldn't be done :) 02:34
*ignoring
so, on to your email to parrot-dev
it sounds like the approach you're planning to use is a tree transformer 02:35
I think this is probably the correct approach
that is where we were originally headed with TGE (Tree Grammar Engine), but it never quite worked out
in your email you ask about syntax for matching -- to me, the lack of a good syntax for TGE is exactly what caused it to not work out 02:36
TGE provided some of the walking primitives, but underneath we still had to write the transformations in PIR. Worse, there weren't really any good standards for how to structure our transformations, so everyone's TGE grammar ended up looking radically different. 02:37
so, if there's one thing I think is vital, it's having a good syntax for specifying tree transformations 02:39
nopaste "kid51" at 192.168.1.3 pasted "config/gen/platforms/generic/dl.c: functional documentation: Is this reasonable?" (53 lines) at nopaste.snit.ch/20678
pmichaud at this point I don't have any idea what that should look like. many people have suggested using xpath as a model, but many people who use and study xpath often say "ugh" when they hear that. :)
because the xpath syntax can be kinda ugly.
but even an ugly syntax would probably be a better starting place than no syntax 02:40
kid51 is trying to reduce number of files being skipped by t/coding/c_function_docs.t. Can people take a look at what I wrote for a dl.c file (paste 20678)? Thanks. 02:41
PerlJam pmichaud: I think people suggest xpath simply because it's the first thing that occurs to them (at least, I know that's how it happened for me :)
pmichaud PerlJam: yes, I think that's so also.
anyway, if the gsoc project accomplished nothing else other than to explore some tree transformation syntaxes, I'd personally consider it a success. :)
(but of course we'd like much more than that :)
so, if we're going to have a good framework for building optimization stages, getting an expressive syntax or library to do it would seem to be important 02:44
at least for getting others to be able to understand it
as far as whether a transformer should build a new tree or transform it in place.... I can go either way on that.
tcurtis Regarding XPath: One reservation I have about it is that if I implement XPath-like syntax, people may expect all the semantics and functions of XPath, not all of which might map easily or usefully to Parrot. Not necessarily a big issue, but something that I'm thinking about. 02:45
pmichaud there's a lot of value in being able to treat data structures as being immutable, so that we use an existing structure to construct a new one rather than do in-place modifications.
at the same time, in-place modifications can be easier to deal with
my advice would be to start with whatever feels most natural to the problem at hand. 02:46
PerlJam Perl 6 grammars are meant for more than just text. It would be a big win if we can leverage that syntax ala NQP
pmichaud the problems of optimization are (to me) likely to be challenging enough without adhering too strongly to any given principal
*principle of program design
I, of course, would love to see NQP leveraged to write optimizers. It's one of my goals to eventually rewrite most of PCT in NQP. 02:47
02:48 janus joined
pmichaud in that sense, I think that writing optimization stages then becomes more a matter of coming up with the correct objects and API, rather than actual syntax 02:48
much in the same way that PAST is really more about datastructures and API than about any particular syntax used to create/transform the trees
PerlJam yep.
sounds just right to me.
pmichaud still, NQP is a very procedural view of transformation, whereas things like xpath and tree grammars might want to be more declarative 02:49
dalek kudo: 51874cc | jonathan++ | src/core/Any.pm:
Fix a spelink fel.
rrot: r47080 | jkeenan++ | trunk/t/codingstd/c_function_docs.t:
Delete non-existent file from list of TODO-ed files.
rrot: r47081 | jkeenan++ | trunk/config/gen/platform/win32/dl.c:
Correct spelling error in POD.
rrot: r47082 | jkeenan++ | trunk/config/gen/platform/ansi/time.c:
Add a little documentation.
pmichaud as far as using XPath syntax and having missing features.... you're absolutely right 02:50
some people will get frustrated that something they know should be available isn't present
PerlJam those people can contribute an implementation then :)
pmichaud on the flip side, though, NQP doesn't have nearly all of the features of Perl 6, but it's still a useful enough subset that it has great value for writing programs (IMO) 02:51
PerlJam (assuming it's something that could be implementable)
pmichaud I mean, someone sitting down to NQP would look at it and say "oh, this is Perl 6", but the fact that some key components of Perl 6 are missing doesn't seem to completely block people from being productive with it
so perhaps that's a counter-example that shows that limited subsets can still be useful
PerlJam As long as the limitations are clear. 02:52
tcurtis Good point. Note to self: consider "Not Quite XPath" as syntax for matching PASTs.
PerlJam (like "no assignment in NQP" for instance)
pmichaud anyway, what NQP tries to get is a syntax that is simple, expressive, and familiar. 02:53
so, we throw away listops (because those are complex)
and we require that every function call has parens, which is very un-Perlish but greatly simplifies the implementation 02:54
we eliminate a lot of the shortcuts that might otherwise be in Perl 6, because the goal is to remain simple but familiar 02:55
so, now to optimization itself
I know that in the past there are a lot of people that look at the PIR code that gets generated by PCT and think "oh, how inefficient". 02:56
"I could definitely do better."
what I've found is that parrot has some deep limitations that hugely constrain what PCT can do in the general case 02:57
perhaps those limitations are really limitations in my ability to see through the problems, though
but in general, the code that PCT generates is the only answer I could find that worked for the common cases in the HLLs I'm working with
a particularly illustrative example is the problem I've had in teasing out "assignment versus binding semantics" in Parrot 02:58
Parrot's vtable set, opcode set, and core PMCs don't have a well-thought out view of assignment and binding 02:59
in many places it tends to confuse the two concepts, and a code generator has to work around the confusion
I can come up with a number of examples where common Perl 5 or Perl 6 code really doesn't translate well to PIR. 03:00
03:00 snarkyboojum joined
pmichaud so, some of the ugliness and overhead of pct's generated output is dealing with that mismatch 03:01
03:01 eternaleye_ joined
pmichaud again, I don't want to be presenting a view that says "optimization is a lost cause", because I don't think that's true. 03:02
I just know that the general cases are far more complex than they seem at first glance. 03:03
if I were looking to optimize past/pct for hlls, I'd personally start by trying to optimize NQP. It's a relatively short and straightforward language 03:04
PerlJam or that some young turk hasn't come along to think about those general cases in just the right way that makes them actally simple rather than just apparently simple :)
pmichaud as you've already done, one can write a short bit of NQP code, look at the PIR output and see if there are ways that it could be immediately improved
PerlJam: +1 03:05
purl 1
pmichaud and, there are some optimizations that are still left to be done -- in many cases I haven't done them simply because (1) I didn't have a good framework for implementing the optimization, and/or (2) my primary goal has always been to get a working Rakudo, and to deal with optimizations a bit later. 03:07
(or, even better, let someone else deal with the optimizations :-)
dalek kudo: 63c5a5c | jonathan++ | src/core/operators.pm:
Make ne consistent with !eq and != consistent with !==.
 03:08
pmichaud so, one example of a potentially useful optimization would be to avoid extraneous find_lex statements in the Parrot sub where a given lexical has been declared
I don't think that'll be a huge optimization, but the generated PIR would certainly look nicer. :-)
(i.e., I don't think it'll be significant in terms of runtime performance)
tcurtis hopes to be able to be the "someone else" for at least some of these optimizations.
pmichaud one optimization that could be huge in terms of performance is to be able to optimize out return exceptions that are thrown from the same sub that would be catching them 03:09
for example, consider the fibonacci:
sub fib($n) { return .... } 03:10
and
sub fib($n) { ... }
the existence of that 'return' statement makes a *huge* difference in program execution
tcurtis My last unscientific benchmark indicated that "sub foo() { return 5; }" took about twice as long to run as "sub foo() { 5; }".
pmichaud that's correct
because 'return' requires building an exception, filling it with a payload, catching the exception, extracting the payload, and returning it 03:11
if we can find places where the exception is thrown in the same (parrot) sub that will be catching it, and convert that to a goto or some optimized form, that would be a huge win.
PerlJam sounds like a good "first optimization" :) 03:12
pmichaud (it won't immediately help rakudo, though, because 'return' in rakudo is really a subroutine call so we'd have to optimize only in the case where there's not a non-standard &return in scope)
and these only help in the case where the 'return' is in the outermost block 03:13
with something like:
sub foo() { my $b; if something() { my $a := 5; something_else($b); return xyz(); } } 03:14
the 'return' still has to generate an exception because it's in a lexically nested sub
(unless of course we find a way for called subs to quickly force returns from caller subs)
another potential optimization that could have huge benefits is if we could inline blocks where it's clear that there's not a lexical conflict with an outer block 03:17
for example, with something like: 03:18
sub xyz() { my @list := abc(); for @list { my $y := $_ * 3; dosomething($y); } } 03:19
currently PCT naively creates a separate parrot sub for the body of the for loop, because $y is lexically nested to that scope
if PCT can detect that there's no $y defined in some outer scope, it could optimize it to be 03:20
sub xyz() { my @list := abc(); my $y; for @list { $y := $_ * 3; dosomething($y); } }
and now the body of the for loop could be inlined into the body of xyz() rather than requiring a separate Parrot sub
so that there wouldn't be a separate sub invocation for each iteration of the for loop 03:21
but this would seem to be reliable only for immediate blocks -- declarations and closures would still need to become separate parrot subs
we also have to make sure that $y is properly initialized at the beginning of the inlined block -- we wouldn't want a previous version of $y to be "remembered" in a subsequent iteration of the loop 03:23
(unfortunately, PIR's .lex directive doesn't initialize a lexical, so we can't rely on it to do it for us.) 03:24
in the case of more dynamic languages, such inlining might be bad also, though, because if we have something like 03:25
tcurtis Inlining seems like it might be a more fruitful approach to optimizing returns in inner blocks, as well. (although perhaps one could do some continuation-stuff to allow called blocks, particularly for blocks like if blocks that aren't intended to be called outside of the containing function to return via the caller's continuation)
pmichaud sub xyz() { my @list := abc(); for @list { my $y := $_ * 3; dosomething($y); }; say(MY::{'$y'}); } 03:26
we don't want that MY::{'$y'} to be accidentally picking up the $y of the inner scope
(which would happen if we optimized the $y lexical into the outer block) 03:27
yes, inlining also takes a lot of pressure of the returns as well
sorear perhaps you should be heading over to #perl6 and convincing TimToady to kill MY ...
pmichaud I don't think there's any chance of killing MY, or OUTER, or CALLER :-)
but we might be able to say that the existence of MY/OUTER/CALLER causes the compiler to have to pessimize a few things in the code generation. 03:28
(which would mean things like flags on PAST::Block nodes that disable certain optimizations) 03:29
s/would/might/
03:30 LoganLK joined
pmichaud or, even worse: 03:30
PerlJam one pass parsing doesn't necessarily mean one-pass code generation ;)
pmichaud sub xyz() { my @list := abc(); for @list { my $y := $_ * 3; dosomething($y); }; eval('say($y)'); }
tcurtis It seems like situations like that might be able to be dealt with by, if there's an existing lexical named $y in scope in the outer block, putting its existing value somewhere for the duration of the inner block and replacing it when the inner block is finished.
pmichaud tcurtis: as long as you can make sure to put it back if an exception is thrown, sure :-) 03:31
sometimes the inner block finishes in ways other than normal finishing :-)
tcurtis pmichaud: that's not nearly the worst:
sub xyz() { my @list := abc(); for @list { my $y := $_ * 3; dosomething($y); }; eval($x); } 03:32
pmichaud right
in that case, we have to either not inline the block or we have to have some way of telling eval "this $y you see isn't really in scope" 03:33
anyway, at the moment NQP is smart enough to inline immediate block that don't declare any new lexicals. 03:34
but other forms of inlining seem a litle problematic for parrot 03:35
anyway, those are just a few thoughts/experiences I've had about optimizations with hlls and pct. There's plenty more 03:36
what I would say is that as you come up with ideas, feel free to ask me about them and I can point out pitfalls I've already discovered or reasons why I didn't already do them 03:37
sorear reading this, I think the solution is a shadow lexpad
pmichaud (and sometimes the reason is simply "didn't have time" or "didn't think of that" :-)
sorear we should have a special image of what the lexpad "should" be
a hash of aliases
this is used by eval, MY::, OUTER::, CALLER:: 03:38
ideally with some hacks to avoid generating it at all in most functions
functions which call unknown functions can generate it lazily
pmichaud sorear: at one time, lexpads weren't so intimately tied to subs -- i.e., one could get a new pad without having to invoke another sub to do it
that got torn out a few years ago, and several of us at the time wondered if that was perhaps a mistake
tcurtis Can lexicals be removed from a lexpad after they've been added? 03:39
sorear Lexicals can't be added to a lexpad
pmichaud lexpads in parrot are (currently) intimately tied to sub contexts
sorear .lex should be treated as an attribute of the sub
its location has no effect
pmichaud i.e., the keys of the lexpad are fixed at compile-time by the .lex declarations within the sub
tcurtis Ah. 03:40
pmichaud (there are special directives to change this, but that's Parrot's default)
not only that, but the .lex declarations tie the lexpad entry to a specific pmc register in the context
the "values" in a lexpad aren't the pmc registers, but rather the index of the pmc register in the context 03:41
(keys are lexical names, values are index into the pmc register frame)
so, even if you get hold of the current lexpad hash, you can't add new entries to the lexpad 03:42
I suspect you could delete entries from it, though.
(it wouldn't have any effect other than to make the lexical name disappear from view.) 03:43
actually, come to think of it, I doubt the delete_keyed vtable is even defined for Parrot LexPads 03:44
so no deletion :-)
tcurtis: anyway, that's about all I can dump for this evening. I'm hoping you can come up with some terrific stuff to help us all out here, and I'll be glad to help where I can. 03:47
over the next month I may be a bit preoccupied with rakudo-priority stuff, but I'll try to get responses to you as quickly as I can
tcurtis Thank you. It's been quite enlightening. Do you have time tonight for the minor question I mentioned earlier? 03:48
pmichaud sure, but I forgot the minor question :-)
tcurtis I didn't ask it yet. I didn't want to distract from your brain dump. :) Here it is:
Are there are any non-obvious attributes of any PAST::Node subclasses that can also contain other PAST::Nodes? 03:49
For example, I didn't expect that PAST::Val nodes might have other PAST's in their 'value' attribute. 03:50
03:50 parthm joined
pmichaud there's not a general rule that applies, no. (more) 03:50
in the case of PAST::Val, I originally wanted something that could get me a reference to a block 03:51
I went with ::Val because such a reference is effectively constant, and ::Val nodes are the things that represent constants
I agree it could be argued to go somewhere else
it could also be argued that :value really ought to be .[0] 03:52
i.e., that the actual value should be a child of the ::Val node instead of an attribute. :value is largely historical in that sense
as far as other attributes in other nodes go.... there are places where an attribute might contain a PAST tree, yes.
:viviself and :vivibase come to mind 03:53
also, I think it's possible for PAST::Op( :name(...) ) to contain a PAST tree that dynamically produces the name of a sub or method to be invoked
if pressed to come up with a general rule, at this point I'd err on the side of "any attribute can potentially contain a PAST tree" rather than prohibiting them altogether. 03:54
(it might not make sense for many attributes to contain a PAST tree, but there have been many times where an attribute that I originally envisioned as a constant string or int ultimately made perfect sense as a dynamic runtime-computed value from an ast) 03:55
s/from an ast/by an ast/ 03:56
a related aspect that occurred later was that originally children of a PAST node could only be other PAST nodes
but I soon discovered that constructing PAST::Val nodes for common string/int constants was a real pain, so I changed it such that string and integer children were treated as if they were PAST::Val nodes 03:57
(they aren't treated *exactly* like PAST::Val nodes, but conceptually that's the idea)
so, another interpretation might be "all children and attributes are PAST nodes, but PAST understands some string and integer constants as being allowed non-PAST-node shortcuts." 1/2 :-) 03:59
tcurtis Ah. That's very good to know.
pmichaud (clearly the current implementation doesn't completely adhere to this latter interpretation, but conceptually that might be a way we want to think of it) 04:00
ultimately PAST, PCT, and NQP were driven by practical considerations as opposed to any key design principles. I'm sure there are some key design principles involved, but mostly I was just trying to get to something simple enough to be achievable, flexible enough to handle things we'd come up with, and driven by immediate programming needs
perhaps I should say "as opposed to any explicitly articulated key design principles" 04:01
at the same time, I'm very happily surprised at how well PAST's original design has managed to hold up over time, even as we've evolved into increasingly complex components of Perl 6, NQP, NQP-rx, and Rakudo. 04:02
there just haven't been many significant refactors 04:03
anyway, hope that answers well enough for now :)
tcurtis There's one (hopefully last for now) question I have: it was in my message to parrot-dev but I don't think you mentioned it tonight. Do you think that PAST::Walker/PAST::Transformer should traverse attributes that contain other nodes just as they traverse children? 04:04
pmichaud it wouldn't take long to identify all of the attributes where PAST values are supported
I can argue that question both ways.
if we take the view that it's possible for any attribute to potentially accept a PAST tree as a value, then it might make sense for walker/transformer to automatically traverse attributes 04:05
on the other side, if there are actually very few attributes that reasonably take PAST tree values, then we can say that it doesn't happen by default but there are some transformations available in the grammar that cause a walker/transformer to operate on specifically identified attributes in the node types that support them 04:06
personally, I'd like to see the latter
i.e., that we have a transformer that says "if you get a node of PAST::XYZ and :pasttype<abc>, then transform its :def attribute." 04:07
s/then transform/start a past transformation of its :def attribute/
we might later decide that we really do want all attributes to be transformed, but knowing that there's a way to cause specific attribute transformations to be started would be a useful tool in the toolbox 04:08
tcurtis I like that latter idea as well. 04:09
pmichaud (and when I say "in the grammar" above, I mean in the sense of a TGE tree grammar -- i.e., whatever is specifying the available transformations to be performed)
we probably need a term for "collection of patterns and transformations" 04:10
"grammar" might indeed be the correct term to use -- in the TGE days we used to distinguish between "parse grammars" and "tree grammars"
a parse grammar was something that parsed source into a parse tree, a "tree grammar" was something that transformed one tree into another 04:11
it's very interesting to note the parallels between PAST::Walker/PAST::Transformer and the way that NQP-based compilers are built now (more)
for a compiler built using NQP, we have a grammar with token/rule/regexes and we have action methods 04:12
the grammar identifies the patterns we're looking for, and the action methods say what to do when a given pattern is found
PAST::Walker and PAST::Transformer sounds very much like an analogue to that 04:13
it might be really need if tree transformations could keep the same model
*really neat 04:14
PAST::Walker would want to have a domain-specific-syntax that makes it easy to identify patterns in the tree
(in the case of parsing, our domain-specific-syntax for recognizing patterns is the Perl 6 regular expression syntax)
a PAST::Transformer could just be an object with methods that correspond to the rule identifiers used in a PAST::Walker object 04:15
so when a given pattern is recognized, it automatically invokes the corresponding method of the PAST::Transformer object
just like tokens and rules in grammars automatically invoke a corresponding method of an action object
if something like that is done, then PAST::Transformer methods could just be written using NQP :-) 04:16
tcurtis One thing: PAST::Walker isn't really directly analogous to the tokens and rules in a grammar. Nor is PAST::Transformer exactly the same as the action methods. They're both on a lower level than those. 04:17
pmichaud and of equal value, by binding a PAST::Node to $/, its children would show up as $0, $1, $2, and its attributes would show up as $<name>, $<pasttype>, etc.
tcurtis While PAST::Walker can be used for matching subtrees, it's support for that purpose is primitive. It's intended more as the thing on which matching can be implemented. 04:18
pmichaud okay, it sounds more like PAST::Walker is really the engine that drives the rules 04:19
much like the regex engine in NQP is really the thing that drives the rules of the grammar
tcurtis PAST::Transformer is a lot more practical for the equivalent of action methods, but I'm not sure it would be as convenient as action methods are in PCT. It's essentially PAST::Walker with the addition of the ability to do more things to the tree(walking a node and deciding to remove it from the tree or replace it with a different type of node as opposed to having to decide that at the level of its parent). 04:22
nopaste "pmichaud" at 192.168.1.3 pasted "a potential transformer written in NQP (pseudocode)" (9 lines) at nopaste.snit.ch/20680 04:26
pmichaud note that nothing requires that $/ be bound only to Regex::Match objects -- it can be bound to anything that supports the array and hash accessor interfaces :-) 04:29
anyway, it's just an idea to play with
tcurtis I'm quite liking the idea of having a grammar-like thing that finds appropriate subtrees, and then calls the action method on them. That would be especially nice for more language-specific optimizations or more expansive analyses. And more independent optimizations could be provided as roles that a PAST::Grammar or whatever it would be called could add.
pmichaud it also goes back to what PerlJam++ mentioned earlier that Perl 6 grammars are meant for more than just text. 04:31
but by treating them as grammar/action like objects, we also get inheritance
I've found that I really like the method-based declarations; such that NQP::Grammar is a subclass of HLL::Grammar (that provides definitions common to many HLLs), and NQP::Actions is a subclass of HLL::Actions 04:32
so we could create optimizers that are just like this common set of optimizations, but also have these other specialized patterns and transformations as well
or some hll-specific variants of an already existing transformation 04:33
anyway, hope that observation sparks some useful ideas :)
it does mean that a transformation syntax is already available; and we'd just need to come up with a nice pattern-matching syntax 04:34
and writing parsers in NQP is really easy :-)
(and writing parsers that build object data structures is really easy :-) 04:35
anyway, I need to get some sleep now -- I didn't get much sleep last night
tcurtis Yeah. :) This discussion has also re-inspired me to look at more of S05 and related Perl 6 Regex/Grammar documentation.
Good night! Again, thank you. This has been very enlightening.
pmichaud it's also perhaps worthwhile to look at the NQP source itself. or ops2c or any of the other language implementations :-) 04:36
same here, it's been really enlightening for me also. I'm suddenly much less pessimistic about optimization :-)
or the possibilities for optimization
...and I'm very glad it's you working on it and not me :-) :-)
tcurtis Heh. I'm very glad to be working on something with such potential usefulness as what is essentially my first real contribution to any open source project other than random personal projects I put on github. :) 04:38
sorear tcurtis: As I understand it, that is the actual purpose of the GSoC. 04:41
Carrot-first OSS involvement.
pmichaud I think that whatever you discover in this project will be incredibly useful to all of us. 04:42
anyway, gotta run. catch you all online tomorrow
04:45 parthm left
tcurtis sorear: good point. Although that is the less obvious carrot compared to the money. I am finding myself much more excited about the project than about the money, however. 04:47
JimmyZ is excited when he sees any huge improvement here. 04:55
cotto ~~? 05:23
purl well, ~~ is the ambivalence operator or also called the mumble operator or the smart match operator, xrl.us/bhj89p
cotto time to catch up on pmichaud's epic brain dump
either that or hope for a summary from whiteknight
plobsing I'd wait for the summary. It was huge. 05:26
05:38 parthm joined
cotto It's a good read so far. 05:39
Wow. He spent 2 hours on that. 05:45
cotto continues
nopaste "plobsing" at 192.168.1.3 pasted "attempt at deep clone (untested)" (164 lines) at nopaste.snit.ch/20683 05:46
parthm does nqp-rx have a 'switch' like construct or should i just chain if/else? 05:55
plobsing parthm: if it did, it would likely be given/when (in the style of Perl 6) in stead of switch/case 05:56
parthm plobsing: ah. thanks. i will check that. 05:58
tcurtis I don't think given/when is in nqp(based on searching nqp-rx/src/NQP/Grammar.pm for "given" or "when").
cotto Given (no pun intended) that it can be reasonably emulated by if/else, it sounds like the kind of feature that would be omitted from nqp. 06:00
parthm cotto: makes sense.
plobsing When pmichaud gets a simple patch implementing the syntax, I'm sure he'd consider adding it. 06:01
(pun definitely intended)
parthm :)
Tene Puns? In *MY* Perl community?
(it's more likely than you think!)
cotto That's punpossilble! 06:02
plobsing Tene: bad scopping! it's *OUR* perl community!
06:14 aukjan joined
tcurtis plobsing, Tene: aren't you both missing a sigil there? Or is "perl" a kind of sub? 06:19
plobsing :D 06:20
JimmyZ and missing assignment operator ? or binding operator? 06:23
cotto NERDS 06:27
06:30 fperrad joined 06:35 fperrad_ joined 06:44 eternaleye joined 06:48 parthm left
cotto plobsing, ping 07:07
plobsing cotto: pong 07:08
cotto plobsing, do you have any suggestions for LoritoRoadmap, e.g. holes or unclear bits?
plobsing I'll check 07:09
cotto thanks
plobsing 1) Lorito Centric PBC: why does PBC have to change at all? Our ops should be implemented in terms of Lorito, but they should remain the same more or less (otherwise deprecation policy will bite you) 07:12
2) If we can get GC to play nice with IMCC, I am very interested in getting IMCC to start using more PMCs (especially PackFile and OpLib PMCs). After that PIRATE and IMCC should be on equal footing, paving the way for bacek awesomeness. 07:14
cotto 1 is one of the points I'm less certain about, but I'm not sure how dynops could be implemented if not in Lorito (or something that compiles to it) ops rather than pir ops.
I like 2 except that it involves finding someone to do said imcc hacking. Using more PMCs would definitely give us a smoother road to PIRATE. 07:15
plobsing 3) I think implementing core items should be moved up. Given a mechanism to have parrot ops always available (Parrot setting or whatever you want to call it), I'd gladly convert many core component I have written from C to <insert Parrot Language here>. 07:16
cotto: I am willing to work on #2, but I find myself torn in several directions at once. Perhaps I should start a tasklist and ask for advice on priorities.
cotto I'm working on a plan to make sure Lorito efforts are coordinated and well-planned. 07:17
plobsing As an example of #3, nopaste.snit.ch/20683 , shows how easily all of our freeze/thaw could be moved over to an HLL *right* *now*. 07:18
07:18 TiMBuS joined
cotto You make a good argument. 07:19
plobsing Make the "implement core functionality in a parrot language" functionality available early, but don't have a push to move over to it until later.
people will use it for new things. I know I will at least. 07:20
cotto I like that.
plobsing as an added bonus, people implementing core things in HLLs can probe for areas of functionality not currently available to them, and help guide what ops lorito gets 07:21
cotto It's a good idea to have some time when early adopters can play with moving core code to Lorito to find the sharp edges.
or what you said 07:22
plobsing cotto++ for taking the reins on Lorito 07:23
cotto I'd be a lot more intimidated if there weren't so many smart hackers around who are willing to help. 07:24
plobsing++ for some good advice
plobsing, what do you think about how dynops would work in PIR-based pbc? 07:28
plobsing PIR-based you say? I don't follow. 07:29
cotto PASM-based would be more accurate
sorry
plobsing I'm still confused. Are we talking text-based as opposed to binary format?
cotto no, I'm talking about the binary format. I'm just saying that it's fairly close to PIR as opposed to much lower-level like Lorito will be. 07:30
plobsing well, as I see it, dynamic things (ops and pmcs) will be referenced in a packfile segment. the client is expected to have them already. 07:31
plobsing considers the possibility of using negative numbers for core ops and positive for dynops so we don't get confusing errors like I get from time to time now 07:32
cotto So you're seeing Lorito dynops and dynpmcs as being separate from pbc, i.e. another binary format? 07:33
plobsing yes. 07:34
cotto ok
That's something to think about.
plobsing the responsibility for dynops and dynpmc management falls to the lorito backend
for example, the LLVM backend might want to store them in .bc files 07:35
cotto Unnecessary massive changes to PBC would be nice to avoid.
I've been thinking of Lorito as having its own binary format that'd be used to store dynops, dynpmcs and bytecode from compiled PIR/HLL code and that that would be the new pbc. 07:37
timtowtdi 07:38
purl There Is More Than One Way To Do It
07:38 plobsing joined
plobsing I think imposing a common low-level bytecode format on backends is a little restrictive. They all have to be able to speak in-memory format, but can do with that information whatever they please. 07:40
cotto I see the low-level bytecode format as sitting just above the backends and being the lowest-level thing that all backends know how work with. 07:43
s/how/how to/ 07:44
I'm realizing that this has been my unspoken assumption though. 07:45
plobsing so the communication to the backends is a bytecode stream?
cotto no; the bytecode stream is a serialization of the communication with the backend
s
plobsing I can see that working, but I wouldn't want any of it to leak out of the VM. 07:46
in other words, keep it out of bytecode 07:47
cotto I'd pictured that as being the way to distribute dynpmcs and dynops.
compile the Foo PMC to foo.lor and distribute that for use with whichever backend the user wants 07:48
plobsing I'm not sure that encouraging distributing something so low level is a good idea.
the images will be quite large
dalek rrot: r47083 | fperrad++ | trunk/t/library/osutils.t:
[osutils] test newer()
 07:49
rrot: r47084 | fperrad++ | trunk/runtime/parrot/library (2 files):
[distutils] cleanup
plobsing and security is hard to impose at that level
I always saw serialized lorito as a local-only caching optimization
would it be so bad to distribute source and have people compile locally? 07:50
cotto That's a valid question. 07:52
I'd be inclined against it but I have limited experience in areas where that issue comes up. 07:54
plobsing inclined against compile from source? 07:55
another reason to not make Lorito ops a supported code exchange medium is that bytecode compat will be at least 10 times harder at that lower level 07:57
and we fail at that hard enough already
cotto yes we do
I don't think I'd expect Lorito bytecode compatibility except at the basic level of "That's not my exact version. I refuse to run that." 07:59
Perhaps I'm overly optimistic about how quickly Lorito will stabilize, but I've been expecting that once we get a set of ops that work, we won't need to make any changes to it apart from bug fixes. 08:00
plobsing maybe not you. maybe not anyone in the Parrot Cabal, but some enterprising user will find the .lor files and try to distribute them, possibly with great success, until we break compat. Then its our fault for breaking things. 08:01
cotto users are always messing things up 08:02
plobsing needs to find a more obscure project on which noone will care if I break things 08:03
cotto You could hack on the kernel or perl 5 for a while. I'm sure that make our compatibility concerns seem pretty trivial. 08:04
plobsing you didn't specify which kernel. I think I'll try my hand at GNU Hurd 08:05
dalek rrot: r47085 | fperrad++ | trunk/runtime/parrot/library/osutils.pir:
partial revert r47084, (I don't want commit my debug stuff)
 08:06
cotto funny. I was just thinking about that.
or windows 08:07
;)
dalek tracwiki: v8 | cotto++ | LoritoRoadmap 08:13
tracwiki: incorporate suggestions from plobsing++
tracwiki: trac.parrot.org/parrot/wiki/LoritoR...ction=diff
tracwiki: v1 | plobsing++ | PlobsingTaskList
tracwiki: create an initial list of current tasks of the top of my head. this is most definitely incomplete.
tracwiki: trac.parrot.org/parrot/wiki/Plobsin...ction=diff
cotto plobsing, quick question before I become unconscious: Would the overhead of having two ops replace one op outweigh the advantage of having fewer ops to deal with const args? 08:16
plobsing cotto: for common operations, yes. for uncommon ones, it might be worth it.
cotto did you just say yes twice? 08:17
or was there a missing not in there?
plobsing no I said taht for common ops, the expansion is a bad idea. for uncommon ones, it is a good idea.
cotto got it 08:18
plobsing s/common/frequent/; s/uncommon/infrequent/
cotto I definitely need to sleep.
plobsing eg: how many times do you intend on calling 'subclass' or 'newclass'?
I bet it is a lot less than you intend on calling eg: 'branch'
cotto makes sense
good night
feel free to further edit LoritoRoadmap 08:19
nopaste "plobsing" at 192.168.1.3 pasted "[winxed] const expansion bug" (8 lines) at nopaste.snit.ch/20685 08:21
plobsing msg NotFound have a look at nopaste.snit.ch/20685 . when I uncomment either of the const specifiers, winxed chokes in emit. 08:25
purl Message for notfound stored.
08:26 clinton joined
dalek tracwiki: v9 | cotto++ | LoritoRoadmap 08:30
tracwiki: trac.parrot.org/parrot/wiki/LoritoR...ction=diff
TT #1663 created by fperrad++: Mixture of dyn opcodes 08:39
TT #1663: trac.parrot.org/parrot/ticket/1663
moritz from the LoritoRoadmap: 08:46
Modify nqp-based compilers to emit Lorito dynops. Once these are usable, start running them.
should that be PCT based compilers?
if not, in which way i nqp relevant here? 08:47
mikehh plobsing: ping 08:55
plobsing mikehh: pong # but will be sleeping soon 08:56
mikehh plobsing: the massa-cre kinda broke make corevm/make coretest - bunch of tests fail but pass make test - see nopaste.snit.ch/20672 08:58
plobsing: still failing at r47085 08:59
plobsing why is yaml_dumper part of corevm?
or Data/Dumper for that matter?
corevm has some feaping creatures methinks. 09:00
anyways, thanks for brining that to my attention. I will try to resolve it tomorrow.
mikehh no idea - me just runs the tests 09:01
plobsing frequently shoots the messenger. that's why I don't get mail delivered to my door anymore.
mikehh plobsing: some of the tests have had problems before - and some stuff got moved around a while back 09:02
for example t/pmc/nci.t needs to be skipped in make corevm/make coretest 09:04
mikehh good thing I am a long way away 09:05
plobsing wonders close the nearest parrot hacker is 09:11
dalek TT #1664 created by plobsing++: OS PMC should use stat.pasm equivalent indices for stat buffer 09:12
TT #1664: trac.parrot.org/parrot/ticket/1664
09:32 elmex joined
mikehh make corevm/make coretest FAILs - see: nopaste.snit.ch/20672 09:32
perlcritic fails on 18 files (out of 599) - all Flag comment TODO and/or XXX 09:33
all other tests PASS (pre/post-config, smoke (#34099), fulltest) at r47085 - Ubuntu 10.04 amd64 (g++)
09:58 snarkyboojum joined 10:13 parthm joined 10:51 masak joined 11:09 mikehh_ joined
bacek aloha 11:12
11:17 casan joined 11:21 AzureSto_ joined 11:34 khairul joined 11:39 whiteknight joined 11:41 parthm left
whiteknight good morning, #parrot 11:44
masak o/ 11:51
bacek aloha whiteknight 11:52
.oO( I do want to create bot "aloha" for auto-completion purposes )
 11:53
whiteknight good morning masak, bacek
darbelo bacek++ # auto-completion purposes 11:58
pmichaud good morning, #parrot 11:59
masak morning, pmichaud. 12:03
moritz \\o 12:04
masak in my GSoC work, I have run into a malloc error: gist.github.com/417084 12:05
seems to me that when I use a string of ASCII-only letters, it works.
when I introduce UTF-8 letters, it gives me the malloc error. 12:06
this is quite a blocker for me, so help is appreciated. 12:07
darbelo masak: What happens if you make the initial string non-empty?
masak tries
darbelo say, a space or somesuch.
darbelo guesses it still breaks. 12:08
masak darbelo: aye, still breaks.
darbelo Oh. Wait, that's perl6. Does it still happen with just parrot? 12:09
masak I'm not skilled enough to turn that into pure PIR.
but I do know that the malloc error occurs on the `sb .= $S1` line, so it would seem likely that pure PIR would produce the error too. 12:10
darbelo masak: That's very nearly pure PIR already :) 12:11
masak aye. just not sure what boilerplate to add to make it an actual PIR file. :)
12:11 aukjan joined
pmichaud tries 12:12
darbelo .sub 'blah' \\n ... \\n .end might just work.
It does :) 12:13
parrot in realloc(): error: bogus pointer
Abort trap (core dumped)
pmichaud ooooh, nice segfault!
yes, same here
masak so... do I open a TT ticket for this? 12:14
nopaste "pmichaud" at 192.168.1.3 pasted "stringbuilder alloc fail, in pure pir" (100 lines) at nopaste.snit.ch/20691
darbelo masak: Yes, please. Also include pmichaud's nopaste if you can.
masak *nod* 12:15
pmichaud marvels at how well masak++ wields his +2 Sledgehammer of Bug Detection
masak more 'attraction' than 'detection' :P
bugs just like me! 12:16
moritz in sovjet parrot bugs like *you*
erm 12:17
bacek erm... 12:19
Confirmed.
bacek happy to have some masak's attention on parrot :) 12:20
darbelo That bugs like masak?
masak darbelo: I accidentally 800 bugs for Rakudo... :)
darbelo buffer->_bufstart = buffer->strstart = mem_gc_realloc_n_typed(INTERP, 12:21
buffer->_bufstart, total_size, char);
masak trac.parrot.org/parrot/ticket/1665
moritz masak: how many of them are still open?
like, half of them?
darbelo That's not entirely right. _bufstart is not always the allocationstart. 12:22
masak moritz: no more than 639 :)
darbelo There's a macro in pobj.h for that IIRC.
I think it's Buffer_bufallocstart() 12:23
bacek darbelo, problem with unicode converting. I'm fixing it now 12:26
12:26 bluescreen joined
darbelo bacek++ 12:26
masak \\o/ 12:27
12:28 JimmyZ joined
dalek TT #1665 created by masak++: Pointer error when adding UTF-8 characters to StringBuilder byte-by-byte 12:31
TT #1665: trac.parrot.org/parrot/ticket/1665
bacek hoy
Good bug! 12:32
masak++
masak :)
Coke (xpath) may be ugly, but it's a known quantity. if it's good enough, let's use it. (I find it good enough when I need to use it.) 12:46
masak I like XPath, and I've used it (and XSLT) quite a bit. 12:48
just don't try to use '/' for division :P
I wouldn't say XPath is ugly. XSLT 1.0, however, is a sad combination of verbose and underpowered. 12:50
JimmyZ Don't use the full version :) 12:51
bacek masak, r47086 12:52
masak bacek: thanks. my branch is pre-massacre, so I'll have to merge now, I think.
bacek masak, yeah. Looks like it's inevitable :) 12:53
masak would have to do it sooner or later anyway :) 12:54
anyway, thanks. will try it out.
12:55 atrodo joined
bacek masak, thank you for using your magical bug-finding skills on parrot :) 12:56
darbelo . o O ( Magical bug-finding robot? )
masak GSoC++ # prime mover :)
dalek rrot: r47086 | bacek++ | trunk (2 files):
Fix memory allocation issues in StringBuilder. Closes TT#1665. masak++
 13:02
TT #1665 closed by bacek++: Pointer error when adding UTF-8 characters to StringBuilder byte-by-byte 13:04
TT #1665: trac.parrot.org/parrot/ticket/1665
13:16 plobsing joined, JimmyZ joined
Coke plobsing: ... I think you found your obscure project already. =-) 13:17
dalek rrot: r47087 | fperrad++ | trunk (3 files):
[languages] fix after r47051 (merge ops_massacre)
 13:18
13:20 gbacon joined
masak bacek++ # the patch did the trick! \\o/ 13:23
thanks for the quick turnaround: 40 minutes from ticket creation to it being closed.
darbelo plobsing: ping 13:24
plobsing darbelo: pong
dalek kudo: c5fdb17 | masak++ | (4 files):
[Buf] Str.encode and Buf.decode

A bunch of tests now pass in t/spec/S32-str/encode.rakudo.		 13:25
darbelo plobsing: What are your thoughts this days on freeze/thaw vs pbc format? I've had crazy thoughts on that are a few times, but they dissipate before I can get some tuits. 13:30
One of the things I've wanted to do for a while now is move away from stashing a packfile in the ATTRs and start using PackFile PMCs 13:31
13:31 mikehh joined
plobsing darbelo: unless we require PMCs to satisfy some other interface, PBC will contain PMC guts, which means PBC compat breaks every time a core PMC has its guts changed 13:32
I don't see any way around it
13:32 patspam joined
plobsing Using PackFile PMCs don't solve this problem. Although they are good for other reasons. 13:32
darbelo It's the other reasons that motivate me :) 13:33
plobsing Freeze/Thaw shares a PackFile with IMCC. I'd like to see that linkage expanded (eg: ImageIO able to directly write to PackFile in stead of using strings as wrappers) 13:34
When we're able to move IMCC over to PackFile PMCs, the ImageIO switchover will naturally happen as well. 13:35
darbelo We could ditch the Buffer and use a 'PackFileRawSegment' or somesuch to store our data.
plobsing You still need to use whatever IMCC uses to be able to run string rlookup 13:36
and I'd like ImageIO to be able to write to/read from IMCC's buffer directly at some point to avoid all the copying we have going on right now 13:37
darbelo So, we're stuck until IMCC becomes sane? 13:38
Ouch.
plobsing darbelo: many PBC issues hang on IMCC.
darbelo wants IMCC to die. 13:39
mikehh good idea +1
plobsing you can help make it that happen. Work on PIRATE. Or PIRC. Or get IMCC to work with GC so we can use core PMCs and share with the rest of the Parrot ecosystem enough to evacuate the functionality we need. 13:40
mikehh brought PIRC up a couple of times - told it is not a major priotity 13:41
plobsing Note that IMCC or the PackFile struct are not what is blocking PBC compat from magically getting better
mikehh priority 13:42
purl hmmm... priority is set to 0. "lowest/at-idle"
plobsing mikehh: volunteer project ... tuits ... grumble grumble
Coke PIRC was intended to replace IMCC - it had a lot of effort early on but has stagnated. I'm not sure it's going to help.
moritz if somebody revived it... why not? 13:43
Coke at this point, it's not even built by default, I think.
mikehh likes PIRATE
Coke moritz: it was basically a one man show.
much like IMCC was. =-)
plobsing Looking at what IMCC actually *does*, I'm convinced any software that does the same thing will be equally complex
Coke at least now we have 3 or 4 people who occasionally hack on IMCC>
plobsing I think maybe what we need to do is simplify our assembler. 13:44
mikehh that's what lorito is about
masak ok, so we have a problem. seems Parrot isn't capable (as in API) of decoding a list of bytes into a string. parts of my GSoC work depend on that. 13:45
mikehh think so anyway
dalek lscript: d15e945 | fperrad++ | dynext/ops/wmls.ops:
fix with new ops2c compiler
 13:46
lscript: da7b4a6 | fperrad++ | (7 files):
fix build after ops_massacre
plobsing masak: you could always code yourself up an encapsulation-breaking C extension that does what you want
NotFound plobsing: ping
masak plobsing: sounds scary but doable.
moritz ... and build technical debt
masak I'd probably need a bit of help or pointers or something.
mikehh masak: that's what we are here for 13:47
whiteknight plobsing: I'm strongly in favor of simplifying the assembler
NotFound plobsing: in Winxed new needs a class identifier or a string constant.
whiteknight I would get rid of macros tomorrow, and most directives if I had my way 13:48
and I would disallow pure-pasm inputs
plobsing those are not things that complicate IMCC terribly 13:49
whiteknight true, but they're lousy for parser performance
what things do you see as the major complicating factors?
plobsing 1.) make it work 2.) make it work right 3.) make it work fast
we're somewhere between 1 and 2. don't even talk about 3 yet 13:50
whiteknight: GC not working in IMCC means it has to re-invent a lot of what has now been built in to parrot core
whiteknight I didn't realize GC was inactive during IMCC 13:51
that's definitely not a good thing
plobsing it gets dissabled
mikehh it is
plobsing if you enable it... epic fail
whiteknight well that's a matter of fixing it to properly use GC
dalek rrot: r47088 | jimmy++ | trunk/src/pmc/stringbuilder.pmc:
[pmc/stringbuilder]remove unused codes
plobsing another thing: opcode selection. IMCC *needs* a constant folder to be able to select an opcode for "$I0 = 1 | 5" 13:52
because we don't have an op for that.
and users expect it to work (despite being an assembler)
moritz then we need to fix our users.
whiteknight it's user expectations that have gotten PIR into the mess it's currently
in
moritz and educate them to write more stuff in NQP
or winxed 13:53
or whatever
purl i heard whatever was easiest to type
whiteknight everybody expects it to be a friendly, usable programming language
NotFound moritz: thanks ;)
moritz purl: forget whatever
purl moritz: I forgot whatever
whiteknight but that goal is directly contrary to compiler performance
plobsing NotFound: I could try to make something to try and fix the constants.
NotFound plobsing: the need for const in the string is expected. In the int, I must take a look. 13:54
whiteknight actually, better idea would be to disallow PIR and only use PASM inputs to the compiler
plobsing whiteknight: that requires PASM to actually *work* 13:55
13:55 davidfetter joined
whiteknight though PASM has some "features" that are overly complex too 13:55
plobsing: true. We would need a working PASM compiler and a working PIR->PASM compiler
plobsing right now it lacks some functionality, most notably it does not have access to the constant table
whiteknight that would be easy enough to fix, I think, if it were desired 13:56
all of this is sort of moot if Lorito is really on the horizon
Coke we can't get rid of macros until we declare PIR isn't going to be written by humans.
NotFound Is a parrot intermediate language, so it must be written by parrots. 13:57
whiteknight Coke: it's a waste anyway. Long term goal in this strategy would be to drop PIR entirely and nobody uses it for anything ever
Coke I would argue that we have a fairly big code base that uses it, but that code base is currently borked, so my argument is invalid. 13:58
moritz Coke: are you talking about TCL?
Coke whiteknight: Still, can't drop it just because "someday". need the alternative first.
moritz: yes.
whiteknight Coke: right, I'm not talking about dropping it now
Coke whiteknight: excellent, then my non-functional code will continue to ... not ... function. 13:59
Coke hopes to have some time to dedicate to that this weekend.
whiteknight but I'm also not talking about making significant improvements to PIR. The language is shitty and the effort to improve it is wasted when we could be working on a better alternative
plobsing I'd prefer we not drop it, but relegate it to the level of mere mortal language, not blessed language of the gods
whiteknight plobsing: that would be fine too. In any case, PIR should not be the primary language of Parrot
Coke shitty is ... vague and harsh.
whiteknight Coke: harsh, but not inaccurate
Coke in an ideal world, it would have been scheme. ah well. 14:00
whiteknight: unhelpful?
Iunno. perhaps I'm just being overly sensitive.
whiteknight Coke: PIR is a language that was designed with several competing goals in mind. It does lots of things, but does none of them well
plobsing Coke: (re: particle) have you had a chance to look at that fix I sent you?
whiteknight A language cannot be simultaneously for humans to read and write, and for machines to read and write
For performance, we want the later. 14:01
Coke plobsing: yes. as you point out, it only addresses one small component, and partcl isn't unbroken after applying it.
even at the revision that it was broken at.
so it's not really a fix.
it's partcl, not particle.
just so he doesn't get confused. =-)
plobsing Coke: but it gets further. then you hit a problem with (I suspect) HLL mapped but not initialized namespaces
Coke plobsing: yes. which is broken /by the same revision of parrot that broke partcl in the first place./ 14:02
purl Hmm. No matches for that, Coke.
Coke so I suspect that something more fundamental is broken. 14:04
but whether I apply that patch or something like it and then more patches, or figure out a better fix, partcl is still borked, and I hope to spend tuits on it this weekend.
plobsing afk 14:05
Coke Thank you for supplying the patch. I appreciate the effort.
particle ?? 14:08
i'm not cunfosed
moritz that's good 14:09
mikehh particle: so you ain't broke 14:10
14:10 ruoso joined
particle i ain't broke, i's busted. 14:11
mikehh or should that be broken
14:13 bubaflub joined 14:15 GodFather joined
atrodo That was a fascinating conversation. 14:21
I never knew IMCC was that borked and PIR was so complicated
darbelo masak: ping 14:23
masak darbelo: pong
darbelo What is parrot missing in your bunch-of-ints -> string conversion. 14:24
masak darbelo: it doesn't add bytes to a StringBuilder, only chars. 14:25
darbelo I'm waist deep in parrot string internals now, maybe I can help out.
masak \\o/ 14:26
darbelo: my bunch of ints represent bytes, not chars.
moritz these ints represent a blob
and that's to be decoded as UTF-8
right?
masak the default is UTF-8, yes.
darbelo Is this the same as your previous nopaste? 14:29
masak aye.
NotFound Strings are made from chars, not bytes.
masak when it started running, the pertinent test showed the thinko.
dalek a: f97d78f | fperrad++ | (12 files):
fix after ops massacre
 14:30
masak NotFound: right. and Perl 6 has the Buf type for when you really do want to talk about bytes or equiv.
NotFound masak: if we want a byte buffer we must provide one, better than playing games with string, String or StringBuilder. 14:31
masak NotFound: agreed.
NotFound ResizableByteArray or something like that.
moritz aren't there "binary" strings in parrot already that could be used for that? 14:32
darbelo moritz: It's an encoding.
NotFound moritz: To add multibyte chars to it by undecoding, I think no, that's not his purpose.
They could be abused to that, but I don't think we must. In addition, strings are immutable, so as a buffer it will be sloooooooow. 14:34
darbelo I'm not sure it can be done at the PIR level.
NotFound darbelo: I hope not. 14:35
plobsing NotFound++ # compound statements work in winxed-embedded pir (was that intentional?) 14:36
masak in the long run, I need to decode from more than just bytes. but probably converting to bytes and then decoding could work.
NotFound plobsing: Compound? 14:38
purl Compound is, like, ok
moritz forget compound
purl moritz: I forgot compound
plobsing ${ say 2 + 3 }. ${ print array[x][y] }
unintentional i take it
NotFound plobsing: a, yes, the arguments are evaluated.
The only part taken literally is the opcode. 14:39
But maybe the generated pir is not exactly what you expect. 14:40
plobsing this makes winxed a candidate for being called "a better PIR than PIR"
NotFound plobsing: that is one of his goals from the beginning.
plobsing yes, I looked under the hood. not what I expected. but it it JFW, then WFM 14:41
NotFound At least for the "written by humans" aspect of pir.
plobsing: I looked at the new-const thing, is a bit complicated but maybe I can implement a good solutiion during thw weekend. 14:44
The current parsing of new was too simplistic, is time to fix it.
Coke darbelo: this string/bytes issue is needed for masak's GSOC project (but not for R*); is this something you can help unstick him on ?\\ 14:50
darbelo Coke: Not at the level he's working, sadly. That kind of trickery is better handled at the C level, than with PIR: 14:52
Coke darbelo: OK.
mikehh make corevm/make coretest FAILs - see: nopaste.snit.ch/20672 (same failures) 14:56
perlcritic fails on 18 files (out of 599) - all Flag comment TODO and/or XXX
all other tests PASS (pre/post-config, smoke (#34105), fulltest) at r47088 - Ubuntu 10.04 amd64 (gcc)
Coke I propose we disable the perlcritic XXX checker 14:57
too much pain, not enough gain. 14:58
or, "not everything is worth of a ticket."
whiteknight I can probably look into masak's issue 14:59
Coke does that now.
whiteknight: if you have tuits, that would be great.
mikehh Coke: it started when i got the latest Perl::Critic::Bangs from CPAN 15:00
darbelo masak, Coke: maybe we could add a push_integer vtable to the String PMC ?
masak I'm not savvy enough to grok what that means. :/
whiteknight masak: what is your issue, exactly? Is there a ticket I can view? 15:01
Coke mikehh: I disabled it.
masak whiteknight: I haven't made a ticket, but there's been a fair bit of discussion here and on #perl6 about my issue today. 15:02
whiteknight: I can summarize it for you, though.
Coke whiteknight: basically, needs to get at the bytes of a string and then convert the bytes back into a string.
masak aye.
and I'm aware of the thing called 'encodings'. :)
s/thing/things/ 15:03
NotFound Allowing to add raw bytes to a string or to a String is to reopen a can of worms we are just closing.
masak NotFound: right. the Perl 6 specification and my GSoC grant both contain this method, though. 15:04
NotFound masak: What method? 15:05
purl method is probably still not an action
whiteknight masak: would we need to modify the bytes inthe buffer when we attach an encoding/charset to it?
masak NotFound: Buf.decode. 15:06
whiteknight or is it a simple copy+paste of the buffer into a STRING and adding an encoding/charset?
I guess I'm not clear on what you mean by "Decode" and "encode" in this context
NotFound masak: We don't have Buf. If we want it, we must write it, not fubar string.
masak NotFound: it's in perlcabal.org/syn/S32/Containers.html
NotFound: I know Parrot doesn't have a Buf. it's a Perl 6 role.
Coke NotFound: so if we want to go from a string to bytes, we'd convert it to a Buf first? 15:07
NotFound Coke: we don'y want to grow string, they are immutable.
darbelo Coke: pretty much.
Coke NotFound: do you think that constructing a /new/ string out of bytes is a problem? Or just mucking with an existing one?
NotFound: I don't think anyone is suggesting growing a string.
masak whiteknight: I'm not sure how to clarify the 'decode' and 'encode' steps. to me, they imply converting between strings and byte sequences, using some encoding. 15:08
Coke I thought it was "here is an array of bytes. please convert to a string"
NotFound Coke: I'm fine with that.
whiteknight if it's a matter of taking the raw buffer and adding a STRING header, that's easy
if I have to modify the buffer, that's hard
Coke I wonder if the trans_encoding stuff we have covers this already.
darbelo Taking an arbitrary buffer and saying "Here be a $encoding string" is asking for trouble.
Coke (just make it a binary string and then force an encoding change.)
NotFound Let's write ResizableByteArray, then use any vtable or methods wanted to encode/decode from/to string. 15:09
whiteknight I would not agree to that
the long-term goal, as I understand it, is to unify all our *Array types into one
darbelo NotFound: How's that different from a ResizableIntegerArray ?
masak darbelo: I'm not suggesting we convert without acutally checking that the bytes follow the encoding.
whiteknight better would be to make something which explicitly is not an array, like ByteBuffer 15:10
NotFound darbelo: the same as char unsigned char is different from INTVAL
whiteknight masak: so we add a STRING header to it, and then iterate over the string to ensure compliance?
masak whiteknight: you're asking me about design details of which I have no knowledge. 15:11
darbelo NotFound: And where is PIR going to get a 'Char' or even store the output?
masak but if it works, I'm all for it.
darbelo masak: my comment was aimed at whiteknight's "taking the raw buffer and adding a STRING header" comment.
NotFound darbelo: from strings of from int, whatever the used wants and we provide means to. 15:12
masak darbelo: that sounded to me like it was done in-place, at which point I was sure I had misunderstood something :)
whiteknight masak: well, I need some kind of details. Either I need more knowledge about what "encode" and "decode" are, or I need information on the algorithm to implement
masak whiteknight: the specs contain information on what Str.encode and Buf.decode do. if I can help in any way... 15:13
whiteknight masak: okay. A link to the specs would be very handy. I'm not good at navigating them
masak I'll paste it again. 15:14
perlcabal.org/syn/S32/Containers.html
that's for Buf.
Str is in perlcabal.org/syn/S32/Str.html
dalek rrot: r47089 | coke++ | trunk/tools/util/perlcritic.conf:
Don't run flag comment check by default.
NotFound whiteknight: I don't see a reason for explicitly not be an array, the usages probably want the array interface.
whiteknight NotFound: but in the future, there will be only one "Array" PMC, and it might have incompatible internal storage 15:15
NotFound Good point. 15:16
whiteknight if we want a raw byte-addressable buffer, we need to be explicit
such a buffer could give us cool opportunities in the future too, such as mmap the whole thing
NotFound I'll better avoid the "raw" part. That must not be part of its spec, just an internal detail. 15:17
We can always provide pin/unpin methods if the need appears to grant that. 15:19
whiteknight Basically what we want is a pointer and a block of memory. From there we can write to it, read from it, mmap it, pin/unpin it, etc 15:25
would make for a nice implementation of memory-mapped files, for instance
NotFound whiteknight: and encode to/decode from string
whiteknight NotFound: yes, of course
NotFound +1 for ByteBuffer 15:28
dalek rrot: r47090 | darbelo++ | branches/gsoc_nfg/include/parrot/pobj.h:
Add a pointer to the string structure, so that we can hang arbitrary extra data off it.
 15:31
rrot: r47091 | darbelo++ | branches/gsoc_nfg/src/string/unicode.h:
Add new struct to represent graphemes.
rrot: r47092 | darbelo++ | branches/gsoc_nfg/src (4 files):
Make sure the new 'extra' member is NULL when unused.
rrot: r47093 | khairul++ | branches/gsoc_instrument (8 files):
Rewrote runtime library in NQP
Coke khairul: IWBNI if that build rule just used the standard PIR->PBC rule and a (NYI?) standard NQP->PIR rule. 15:34
particle can folks here live without parrot.org for an hour or so for some maintenance, or should i schedule it for some other time? 15:45
whiteknight I'm coolwith it
particle darbelo, coke, NotFound, anyone else planning imminent commits or trac work? 15:46
dalek rrot: r47094 | darbelo++ | branches/gsoc_nfg (1 files):
Add base files for the NFG encoding. It's UCS-4 with the serial numbers removed at the time.
 15:47
Coke particle: um, what maintenance? 15:50
purl well, maintenance is not covered by the warranty. But you need it for the warranty to be valid. "I wasn't at 30K miles yet!" Yet, last month when his other car was 2K over a warranty limit he whined to them that "it's only 2K over! You gotta fix it under warranty!" GRRRRR. or how it's spelled in french
Coke and now is most certainly not a good time, given how many committers and other folks we have who are not on #parrot. 15:51
cotto_work hola, meatbags
Coke perhaps if you schedule it a few days (at least HOURS) in advance?
whiteknight what maintenance? an hour of down-time for necessary work will be acceptable to almost everybody 15:52
Coke whiteknight: really? with no notice? 15:53
whiteknight really. with no notice.
Coke You must know different parrot folk than I. =-)
whiteknight The people who I suspect would have the biggest issue with it are either not here right now, or work primarily out of different repos 15:54
particle looks like a python 2.5->2.6 upgrade
whiteknight most of our powerusers are using git or git-svn, so they can continue working and dcommit later
Coke particle: where are you seeing this?
cotto_work +1 to get it over with. Now doesn't seem like that bad a time.
particle i don't think it's a great time for it, we should give users notice in advance
Coke particle: and why am I not seeing it? 15:55
particle coke: email from osuosl to parrot.org contacts, me allison jhorwitz
we should probably make directors@ the contact
darbelo For the record: I have a git-svn checkout here, I can live without parrot.org for a while.
Coke particle: yes.
(though then we'll need to whitelist the OSU folks) 15:56
particle we can manage to do that
whiteknight darbelo: I suspect the ->bufused field in STRING is unnecessary now with immutable strings 15:59
bufstart and buflen give us the raw buffer, for GC. strstart and strlen give us the length that's actually used by this instance. bufused is unnecessary
particle coke: can you whitelist @osuosl.org for parrot-directors@? 16:02
dalek rrot: r47095 | darbelo++ | branches/gsoc_nfg/src/string/encoding (2 files):
Hederize the nfg files.
 16:05
baest I get this in my Makefile: "LINKARGS = $(LDFLAGS) $(LD_LOAD_FLAGS) -lpthread -lm -L/opt/local/lib -licuuc -licudata -lpthread -lm \\c -lm -lutil -lgmp -lreadline -lintl" after Configure.pl on osx 10.6. Anybody have seen something simular?
Coke think it has to be individuals. checking.
baest: yes.
oh, you mean the \\c?
particle ah, that'd suck.
baest Coke: sorry, yes should have hightlighted that part somehow 16:06
Coke I know people have complained of that happening in the past. not sure if we figured it out.
baest it's the \\c that's the problem. The same is in the ICU_SHARED
Coke are you building from svn? a tarball?
baest svn
Coke ah, yah, the ICU thing sounds familiar. 16:07
darbelo whiteknight: bufused and strlen are two different things, and our internals are sometimes confused about which does what.
whiteknight darbelo: right, but without COW, I don't think we need bufused
Coke look at config/auto/icu.pm
whiteknight it has no purpose anymore. We can remove that field and simplify
Coke perhaps that regex needs tweaking?
darbelo bufused is the 'byte length' of the string. It's used for space calculations.
Coke whiteknight: StringBuilder is using it, neh? 16:08
whiteknight so strlen is the character lengh of the string?
darbelo Yep.
particle we'll schedule the parrot.org maintenance window for some time next week, with more and public advance notice.
baest Coke: will do thanks
whiteknight darbelo: so is bufused the bytelength of the string itself, or of the buffer?
Coke particle: we already whitelist hosting & parrot directors @ osuosl.org. 16:09
darbelo The string. It's the member you ask for 'Do I need to reallocate?'
whiteknight darbelo: immutable strings. Never need to reallocate
particle coke++ for checking 16:10
darbelo True, but all size calculations are still done with it.
khairul coke: i can't get it to generate the pir w/o the nqp->pir rule (or rather I'm not sure how), but i've removed the pir->pbc rule though.
Coke I'm assuming that their ZOMG emails are coming from hosting?
khairul: I'll see about adding an NQP -> PIR rule you can glom onto later.
khairul++
particle coke: you assume too much 16:11
this was sent from mythmon@ with a systems@ reply-to
dalek kudo: ac1571f | (Solomon Foster)++ | src/Perl6/Actions.pm:
Don't whatever-curry infix:<~~>.
 16:12
cotto_work khairul, what are you trying for nqp->pir?
Coke particle: my list was for recipients, not senders anyway - if they send to hosting, it's whitelisted. 16:13
I'll add this guy to the whitelist.
cotto_work it looks like what you have should work
khairul cotto_work: what do you mean?
cotto_work nm 16:14
Coke particle: wednesday is fine.
baest heh, it's icu-config that have: ECHO_C="\\c"
Coke ah. if you can suggest a better fix than what we have, awesome. 16:15
also, we seem to have spelled erroneous wrong. 16:16
baest well, I think it is a icu or mac ports problem... I'll bug them
Coke hokay. 16:19
dalek rrot: r47096 | khairul++ | branches/gsoc_instrument/config/gen/makefiles/root.in:
Removed unneeded PIR->PBC rule.
 16:21
cotto_work There are some nice Indian places around here. I wonder if they have whatever-curry. 16:23
16:26 aukjan joined
darbelo whiteknight: On further review of src/string/api.c bufused is mostly used in the various copying operations we do on string contents. But we might be able to kill strstart this time. 16:39
Of course, that basically means we won't have COW anymore, but I'm hopeful that immutable strings have changed our string usage enough that it won't be a net loss. 16:42
16:44 dmalcolm joined
dalek tracwiki: v21 | cotto++ | CottoTasklist 16:50
tracwiki: trac.parrot.org/parrot/wiki/CottoTa...ction=diff
rrot: r47097 | fperrad++ | trunk (2 files):
[distutils] add test
 16:54
particle heads westward & 17:13
17:14 lucian joined
darbelo realizes that normalization is going to cost *way* more than anticipated. 17:22
icu-- 17:29
17:29 ruoso joined
pmichaud good afternoon, #parrot 17:38
sjn o/ 17:54
17:55 whiteknight joined
Coke allo 17:59
darbelo \\o 18:02
18:04 GodFather joined
whiteknight does anybody here know what the status of Pipp is? 18:20
cotto_work inactive 18:22
purl inactive is probably allocated, but not used. free is probably unallocated.
whiteknight urg, I'm looking through the source code now 18:23
some very interesting things, but I would almost need to start over 18:24
the parser looks to be mostlycomplete 18:25
cotto_work PHP doesn't need a very sophisticated parser at all.
whiteknight no, I imagine not
I'm sort of a PHP newbie, but it's language I've been using recently for some MediaWiki-related work I'm doing 18:26
cotto_work There's not a lot there on a language level. I suspect that that's why it's so popular. 18:27
whiteknight right, but there's a huge library of built-ins. That's the hard thing to duplicate 18:28
moritz with all their wtfs :-)
cotto_work that's definitely the scary part
whiteknight What I would really love to see, is some of the big language communities starting to target Parrot, instead of us trying to come up with competing compilers
the PHP people for instance were talking about doing a big rewrite eventually to fix unicode support. If they targetted Parrot instead of writing a new interpreter from the ground-up they could save themselves a lot of work 18:29
moritz and make themselves a lot of new work at the same time 18:30
18:30 LoganLK joined
whiteknight moritz: maybe true. But you immediately get things like exceptions, GC, NCI, an object framework, data serialization, etc 18:31
cotto_work I'd love that but we have some major speed issues to overcome before such a suggestion could be taken seriously.
whiteknight you spend about 1/10th the effort on a parser, task your internals people with improving Parrot incrementally, and everybody wins
cotto_work The thing about the PHP community is that there's little or no central structure afaik. 18:32
whiteknight cotto_work: true, but that's part of the symbiosis. You start with a decent base and work to improve that base. Theoretically, it should be easier to speed Parrot up than to create a new interpreter and then speed that up
you help Parrot, Parrot helps you
cotto_work I agree.
whiteknight Rakudo has had some problems directly related to some of Parrot's recent growing pains. If Rakudo started today with Parrot in the condition it is in now, they would have a much much easier run of things 18:33
moritz and if rakudo started today, parrot wouldn't have a sane subsystem for lexicals 18:34
which basically proves your point, whiteknight
cotto_work Once we get Parrot to scream performance-wise, I'll be glad to throw myself at building a good base for PHP on Parrot development.
whiteknight moritz: arguably, Parrot still doesn't have a sane system for lexicals. 18:35
moritz whiteknight: well, enough to implement proper closure cloning on top of it
whiteknight but that's besides the point. Parrot has lexicals now, and it took the perl people much less effort to implement lexicals than it would have taken to write a whole damn VM 18:36
moritz whiteknight: not enough in the sense that it doesn't allow storage of non-PMCs
whiteknight moritz: true. Storage of non-PMCs would be nice, but that's a relatively small change at this point
moritz is it?
purl it's it!
whiteknight with a dedicated effort, one or two developers could prototype such a system in the space of a week, I estimate 18:37
cotto_work whiteknight, is that conclusion based on looking at the code? I got the impression from pmichaud that there'd be more effort than that. 18:39
whiteknight cotto_work: there are really two parts to it: how to store the data of multiple types without autoboxing (hash already does this), and then the opcodes to get/set them 18:41
probably have to add a bunch of vtables to LexPad and LexInfo too. There are some design decisions to be made, such as how strict we are about types 18:42
can "$foo" refer to any object, or must it be a PMC always?
The bigger lexicals issue that I am aware of is fixing reference semantics so we don't have to fetch/modify/store lexicals 18:43
18:43 jrtayloriv joined
whiteknight again, the hardest part there is the design for what the solution looks like 18:43
that could be as simple as straightening out set/assign semantics 18:44
18:44 Hunger joined
jrtayloriv I'm failing several of the spectests for the latest git revision -- pastebin.ca/1873454 -- don't know if this is useful ... just thought I'd let ya'll know. 18:45
whiteknight I say "simple" here because writing the code is easy. Again (and I'm kicking a dead horse here), the hardest part of all of it by far is the design 18:46
jrtayloriv: what OS are you on? Any idea how to trace that git hash back to an svn revision?
sorear jrtayloriv: that looks like a Rakudo failure, not a Parrot one 18:47
jrtayloriv oops! wrong room sorry
whiteknight oh, I didn't even notice the test names
jrtayloriv (It is a Rakudo bug)
moritz jrtayloriv: known, somebody/something broke Whatever stars as arguments to methods, or so
jrtayloriv: it's an obscure bug
jrtayloriv moritz, OK -- thanks. I'll be on my way now :) 18:48
dalek tracwiki: v10 | cotto++ | LoritoRoadmap 18:49
tracwiki: Don't conflate NQP and PCT. moritz++ for noticing.
tracwiki: trac.parrot.org/parrot/wiki/LoritoR...ction=diff
cotto_work reference semantics might be the bigger fix that I was thinking about 18:51
I know that that will be a big project if we decide it needs to be taken on. 18:52
sorear it needs to be taken on
cotto_work thanks for volunteering ;)
sorear I did, months ago 18:54
and failed horribly
it's just too hard
cotto_work Maybe that needs its own roadmap.
Thanks for trying though. I'm sure your experience will be valuable in the eventual cracking of that nut. 18:55
whiteknight, ping 18:58
whiteknight pong 19:01
cotto_work Have I bugged you for feedback on trac.parrot.org/parrot/wiki/LoritoRoadmap yet?
and if not, is now a good time?
whiteknight yeah, now is alright
cotto_work ok 19:02
whiteknight I'm reading now 19:03
this all looks very good, very comprehensive 19:05
cotto_work It's fun to see what kind of holes different people notice. 19:07
19:08 chromatic joined
cotto_work Actually, I've got a programmer friend who works at Zipper. I should see what he thinks. 19:08
whiteknight I wish more of the stage 0 design blanks were filled in
chromatic Parrot doesn't support non-PMC lexicals because adding them hasn't been a Rakudo Star priority. 19:09
If it were, Parrot would support them.
whiteknight chromatic: my point exactly. If people wanted them, they would be added
I could do it in a week, I think, if an API were provided
chromatic I've asked Patrick multiple times if he wanted them yet, and the answer has always been "After R*". Jonathan too.
whiteknight cotto_work: my idea is that we would have Lorito ops in bytecode only, but we could define repeated patterns of ops into "megaops". 19:10
cotto_work aka PIR?
whiteknight so PIR ops would be predefined megaops, but an optimization pass would identify repeated patterns and define new ones on the fly as well
cotto_work Oh. That's an interesting idea. 19:11
whiteknight so op numbers 0 to N would be built-in lorito ops. Everything else would be a lookup in a megaop table
it would be like lempel-ziv style compression of PBCs, but replacing sequences with tokens in a lookup table
cotto_work That'd be more of a bytecode compression strategy than anything else, right? 19:12
whiteknight not necessarily, we could JIT the megaops too, maybe even include machine code in the PBC file
if the PBC is read on the same machine that produced it, free speed bonus 19:13
otherwise, no harm no foul
19:13 joeri joined
atrodo Are there ideas floating around that would take JIT code and create a real binary out of the whole bytecode? 19:14
whiteknight atrodo: like an ahead-of-time compiler? I've been a big proponent of that
I think AOT is an area where Parrot could really become a standout among VMs and other interpreters 19:15
atrodo Exactly like that
cotto_work similar to what Facebook is doing with HipHop
whiteknight exactly
cotto_work except that HLLs get it for free just for being awesome enough to run on Parrot 19:16
Coke perl101?
purl perl101 is Andy's site perl101.org
whiteknight right. When you think that HLLs like Rakudo already are used through a compiled fakecutable, it makes a lot of sense to go the extra mile, compile it all down to proper machine code, and optimize the hell out of it
cotto_work Mmmmm. Real binary executables. 19:17
chromatic meh
whiteknight the problem with hiphop is that you lose eval(), which Parrot can't afford 19:18
chromatic Real binary executables... that link against libparrot. 19:19
whiteknight chromatic: true, but execute directly instead of indirectly through the runcore
there are real savings to be had, including cross-op optimizations 19:20
Coke can only tilt at so many windmills at at time.
chromatic There are advantages, yes, but real binary executables don't live on ou topos.
Coke going to yapc::na 2010?
purl going to yapc::na 2010 is qw(jhannah rbuels cfedde apeiron dha nacmac dhoss mst chargrill kyriel triddle DrForr coke packy dukeleto kolibrie colomon jasonmay arcanez dylan doy ology waltman autarch)
Coke no, going to yapc::na 2010 is qw(jhannah rbuels cfedde apeiron dha nacmac dhoss mst chargrill kyriel triddle DrForr coke packy dukeleto kolibrie colomon jasonmay arcanez dylan doy ology waltman autarch
purl okay, Coke.
Coke no, going to yapc::na 2010 is qw(jhannah rbuels cfedde apeiron dha nacmac dhoss mst chargrill kyriel triddle DrForr coke packy dukeleto kolibrie colomon jasonmay arcanez dylan doy ology waltman autarch pmichaud) 19:21
purl okay, Coke.
Coke chromatic: you in?
atrodo no, going to yapc::na 2010 is qw(jhannah rbuels cfedde apeiron dha nacmac dhoss mst chargrill kyriel triddle DrForr coke packy dukeleto kolibrie colomon jasonmay arcanez dylan doy ology waltman autarch pmichaud atrodo)
purl okay, atrodo.
Coke (you're presenting, but that doesn't mean anything this year.)
doy?
purl doy is getting warnings about non-inlined constructors elsewhere or Horselover Fat or a forbidden fruit monger
Coke doy is also like "duh"
purl okay, Coke.
whiteknight chromatic: of course it's not all unicorns and roses and skittles 19:22
chromatic: but it's an option Parrot will want to have eventually, and there are performance benefits to be had for people willing to trade the flexibility of full interpretation 19:23
chromatic Yeah, we're not writing Ruby here.
whiteknight A huge amount of runtime cost for Rakudo is in compilation. If we cut that cost out of subsequent runs we can improve significance significantly 19:24
er, improve performance significantly
(the significance will stay the same)
chromatic Does PGE do LTM?
Coke don't think so. 19:25
moritz chromatic: nope
chromatic Backtracking cuts?
19:26 hercynium joined
whiteknight I look at companies like Facebook with HipHop and I see huge potential. What if we had a run-of-the-mill PHP parser and runtime, but we throw in a huge library of PAST-level optimization passes, pass it to LLVM, optimize the hell out of it there, and spit out an extremely fast native executable? 19:27
server deployment load could become quite impressive indeed, for the same codebase
moritz chromatic: : and ::, not quite sure about :::
chromatic If cuts can throw away failed match trees, we can reclaim them rather than hanging on to them. That helps. 19:30
... and if we had a better register allocator, we wouldn't mark things we don't use anymore. 19:31
Coke chromatic: is anyone using PGE other than partcl at this point?
(probably not worth optimizing)
whiteknight a better register allocator would be a very nice thing to have
chromatic PGE is a synecdoche for "Whatever NQP-rx uses". 19:32
Coke (if it's just partcl). I was under the impression that PGE and TGE would get chopped around 3.0 or so.
chromatic: NQP-rx uses itself.
AIUI, it only used PGE the to generate stage0 once.
chromatic Sweet dreams are made of that.
whiteknight With tcurtis writing optimizations for PAST, maybe it sets the stage for a register allocator to be written in NQP/PIR
Coke s/the//
chromatic bacek, bacek_at_work, how does PIRATE allocate registers, if it does? 19:33
comma comma comma chameleon....
Coke chromatic: the only place PGE appears in the source for nqp-rx is in its Configure.pl; doesn't seem to be used.
so, it could be using something that depends on PGE tangentially, but I doubt it. 19:34
chromatic Does NQP-rx support LTM and cuts?
Coke pmichaud?
purl pmichaud is www.pmichaud.com/ or "Patrick R. Michaud" <mailto:pmichaud@pobox.com> or in charge of toaster experiments or a friendly guy :)
atrodo Are there many projects that are embedding parrot at this point? 19:35
cotto_work PL/Parrot is one
whiteknight atrodo: PL/PArrot
purl hmmm... PL/PArrot is a really cool project irc://irc.freenode.net/plparrot
chromatic mod_parrot (don't know its status)
Padre 19:36
atrodo Padre is? I'll have to check that out
or is it embedding from within perl5?
whiteknight chromatic: if we built a fast linear-scan allocator into IMCC, since it's fast, and build a graph-coloring allocator into PIR/NQP/PASM, I think we win
we get good average case allocation for interpreted and JIT systems, and can activate more efficient code through an optimization if we need it 19:37
cotto_work What problems does the current register allocator cause?
Coke biggest embedder was mod_parrot, but that's dead, SFAIK.
chromatic Mostly theoretical, cotto_work.
cotto_work That's what I thought.
chromatic Consider a PIR function that builds a big tree of PMCs.
That's in $P1.
It does its work and never refers to $P1 again. 19:38
It calls another function which triggers a GC run.
At that point, the tree of PMCs in $P1 might be dead. Let's assume it is.
Yet the CallContext containing that register set is still alive, and it needs to mark those registers.
Thus they stay alive. 19:39
If, however, we could reuse registers after we proved important characteristics about the lifespans of their contents, we wouldn't necessarily suffer from that.
(Though it would be interesting to reset $Px to PMCNULL and $Sx to STRINGNULL at the point of last use.) 19:40
pmichaud NQP does not use PGE for anything. 19:42
the Configure.pl part is likely a fossil.
sorear Coke: What about doy?
pmichaud nqp-rx does some ltm with protoregexes and a few other places. pge does non.
*none
(well, pge does some ltm-like stuff with the old optable mechanism, but it's obsolete) 19:43
both pge and nqp handle cuts
the language that would benefit from non-PMC lexicals is NQP. Rakudo won't benefit. 19:45
(which is why I've not been pushing for them)
moritz not even when we implement native typess?
pmichaud moritz: not even then 19:46
moritz why? need for binding?
pmichaud parrot registers don't give us a way to keep properties around
binding is another issue, yes.
but if I have:
sub xyz(int $x) { $x = 4; } 19:47
*something* has to know that $x is readonly
and the compiler can't be the final arbiter on that
chromatic Maybe lexicals could have properties.
pmichaud lexicals having properties could work, and would be incredibly useful from another set of perspectives
i.e., there would likely be other places that could take advantage of lexicals with properties 19:48
but the main point is that theoretically we'd get the speed improvement because we'd somehow optimize '&infix:<=>'($P0, $P1) into something like $I0 = $P1 19:49
but the set_i_p opcode doesn't have any idea how to check for readonlyness
or, perhaps it becomes store_lex '$i', $P1
that might have a bit more luck, since it would end up going through the lexpad's vtable
but we're still boxing and unboxing the value that was in $P1 19:50
so thus far I just don't see it resulting in a huge performance benefit until we can do a lot more code analysis at the HLL level 19:51
and I don't see that happening anytime before Q4 2010, tbh.
(Q4 2010 is the earliest possible date, not a likely start date)
moritz having properties like ro-ness in the lexpad would enable us to detect many cases of assignment to ro variables at compile time
pmichaud no, lexpads in parrot don't exist at compile time 19:52
in that case we have the PAST::Block symbol table. It could do some detection.
moritz ah
pmichaud but we still have to have a runtime check
moritz yes 19:53
pmichaud or, instead of giving the properties directly to the lexicals, if we could get the properties in the lexinfo and reference those from the lexpad, that might be useful too. 19:54
also, just to make an important observation 19:55
19:38 <chromatic> Yet the CallContext containing that register set is still alive, and it needs to mark those registers.
afaik, the reason that call contexts currently stay alive is because of lexicals -- i.e., that some inner closure exists that needs to reference the outer call contexts 19:56
chromatic I meant in the general case, but that also applies.
pmichaud if we could somehow clean up lexicals altogether, that problem might be able to go away
chromatic That requires us to separate static lexpads and dynamic lexpads.
pmichaud I think the lexicals issue is the general case, but could be mistaken. 19:57
chromatic You're talking of marking more CallContexts than we really *need* to mark.
Coke sorear: what about it?
chromatic I'm talking of marking more registers in CallContexts than we really need to mark.
pmichaud oh! 19:58
yes.
I see the difference now.
19:58 TimToady joined
chromatic It's two different optimizations, but they achieve similar goals. 19:58
pmichaud and your optimization definitely has the potential to be significant on its own 19:59
any temporary pmcs that were created during the execution of the sub still hang around
Coke (sorear: it's apparently someone's nick, but is also what I told purl.)
pmichaud and they definitely don't need to do that
here's a possibility: 20:00
based on a hack I did earlier this week
when a subroutine exits, it could replace its LexPad with a simple hash
key is lexical name, value is the pmc that was in the lexical register
then the register frame can go away
the hash that remains will keep the lexicals alive, while all of the other registers are now gone 20:01
chromatic It's worth profiling as a temporary workaround.
pmichaud it's what the REPL is doing to maintain lexicals
after eval() an input code, we then build a hash of the entire lexical environment from the call context and store that for later uses as an outer pad 20:02
it's then okay if the call context from the input code disappears, because the hash keeps the relevant values around for the next iteration of the REPL loop
handling continuations in such a case might also be a complicating factor.. I'm not sure about where things stand with those these days. 20:06
chromatic That's why I think we need to distinguish between static lex pads and dynamic lexpads. 20:07
The space between "lex" and "pad" is insignificant.
Coke I thought we already did that with dynlexpads.
... not that those necessarily work or anything. 20:08
darbelo stalexpads?
Coke DALEXPADS!
no one? *sigh* 20:10
chromatic Everyone's wary of what comes next.
moritz dyslexpads? 20:11
darbelo What a xyspledads? 20:12
pmichaud Coke: the replace-LexPad-with-Hash trick is what I did to get dynlexpads, since the others don't really work :) 20:13
in fact, I'd almost recommend that we eliminate the trick that maps LexPad entries to CallContext registers... afaik it's not really used much and I'm not sure it's going to be a significant optimization point 20:15
or, if we don't do that, I'd like to see that inner subs have a way of also mapping lexicals directly to callcontext registers, rather than having to do store_lex on every lexical fetch
sorry, find_lex on every lexical fetch
e.g., a directive like .outerlex '$a', $P1 somehow keeps $P1 tied to the contents of '$a' from the outer frame. (I admit this is not trivial.) 20:16
as a more general case, a facility that can map a register in a local CallContext to a register in another (caller) CallContext 20:18
chromatic I think we need some use cases for the two types of lexpads.
The only complicating factor I can see is that when we return a new instance of a closure from a function, we need to bind it to the proper lexpad. 20:20
I *think* we optimize for that case, which isn't the normal case.
20:34 TimToady joined
bubaflub i'm working on getting parrot to cross compile on RTEMS (for my GSoC) which means i have to generate a hints file. is there any configure options that *can't* be overridden in a hints file? 20:49
darbelo bubaflub: Not to my knowledge, but I've never experimented too hard with that. 20:59
Try it out and let us know ;) 21:00
bubaflub darbelo: yeah, i'm looking at the configure.pl docs, and it looks like i can provide it with a bunch of options via a file
my thought is that i'll use a script to generate that file, passing all the correct options
and that script will also generate the hints file so all the auto-probing stuff will work
darbelo There's an option to pass in the name of the hints file you want to use. 21:01
bubaflub darbelo: exactly. which is great so i won't have to patch Configure.pl but just provide it in the other file
darbelo I added that option in order to do exactly that :)
bubaflub bahaha. thank you.
darbelo: so for RTEMS, i have a Makefile right now that dumps a skeleton of a hints file 21:02
i'll have another Makefile that'll generate the configuration file for a specific RTEMS bsp 21:03
and it should compile
(for certain values of "should")
darbelo Nice.
bubaflub imma decommute here in a second, but i'll have some progress for the #ps on tuesday
darbelo Very cool.
dalek rrot: r47098 | darbelo++ | branches/gsoc_nfg (5 files):
Add nfg as an encoding on it's own, separate from ucs-4.
 21:06
21:08 whiteknight joined
chromatic LLVM dev summit videos, including those from Apple employees: llvm.org/devmtg/2009-10/ 21:26
GeJ Good morning everyone.
darbelo Good tomorow morning, GeJ. 21:32
sorear Help, I don't have WIKI_MODIFY 21:35
(yes, I'm logged in) 21:36
moritz Coke can probably fix that
21:48 tcurtis joined 21:50 bluescreen joined 22:02 jan joined
Coke I took WIKI MODIFY away from AUTH_USER because of spammers. 22:15
added. 22:17
sorear thanks.
bacek Good morning, humans 22:20
chromatic, PIRATE doesn't allocate registers (yet). 22:21
And will not.
Idea is PCT's POST::Compiler will allocate them in .to_pbc method.
dalek 9plus: 63757bc | fperrad++ | s (2 files):
old NQP is gone, use new one
 22:22
22:25 patspam joined
dalek lambda: 6ad353a | fperrad++ | unl.pir:
fix after ops massacre
 22:25
bacek seen pmichaud 22:35
purl pmichaud was last seen on #parrot 2 hours, 17 minutes and 19 seconds ago, saying: as a more general case, a facility that can map a register in a local CallContext to a register in another (caller) CallContext
bacek msg pmichaud I want a pon^W "augment" in nqp. It will be very helpful to mix PIR/NQP code for single class. 22:39
purl Message for pmichaud stored.
dalek zy-k: 6a4d660 | fperrad++ | lazy.pir:
fix after ops massacre
 22:40
zy-k: d6d1f26 | fperrad++ | setup.pir:
modernize setup.pir
l: d0904c9 | fperrad++ | gil/parser/actions.pm:
old NQP is gone, use the new one
 22:41
pmichaud bacek: "augment" in nqp is currently spelled "module". 22:56
bacek pmichaud, yay! :)
pmichaud so, module XYZ { ... } adds methods into the XYZ class 22:57
bacek pmichaud, excellent! One less problem :)
pmichaud bacek: that's the way we're able to add methods into Parrot's Hash, RPA, etc. classes
22:58 bubaflub joined
bacek pmichaud, yes, thanks. It was what I'm looking for. 22:58
How far "multis" in your todo list?
pmichaud near the top 22:59
very likely to be examined tomorrow
bacek good! :)
sorear I thought we fixed that [ namespaces and method tables being the same ] already? 23:00
23:00 Psyche^ joined
dalek rdinal: 5855443 | fperrad++ | src/classes/String.pir:
update some string opcodes
 23:04
rdinal: bac8e86 | fperrad++ | src/parser/actions.pm:
update NQP syntax for PIR inline
rdinal: 0222330 | fperrad++ | cardinal.pir:
fix after ops massacre
23:10 joeri joined
cotto_work bacek, does PIRATE have a register allocator? 23:12
dalek tracwiki: v22 | cotto++ | CottoTasklist 23:17
tracwiki: trac.parrot.org/parrot/wiki/CottoTa...ction=diff
sorear cotto_work: No 23:18
PIRATE doesn't use registers
it's PCT-based
PIR locals and "registers" on input become lexicals in the AST 23:19
cotto_work but if it's going to emit pbc or POST, it'll need to figure out which registers to use
sorear register allocation will be done by the hypothetical future POST->Packfile compiler
cotto_work I thought that'd be part of PIRATE
I don't know how you'd emit POST without knowing which registers the code would be using. 23:23
sorear POST doesn't work in terms of registers
23:24 bubaflub joined
sorear POST manipulats not-quite-registers, which are named or numbered lexicals that are not visible in inner subs 23:24
cotto_work now I know 23:25
bacek cotto_work, PIRATE will emit POST, POST::Compiler.to_pbc will implement register allocations. 23:27
"POST::Compiler.to_pbc" can be part of PIRATE though 23:28
cotto_work That makes sense.
whiteknight I really prefer that method, because it means we can do a register allocator in PIR instead of C 23:42
bacek whiteknight, there is no "C" in PIRATE at all. 23:50
dalek tracwiki: v11 | bacek++ | GCMassacre 23:51
tracwiki: Some notes about incremental tri-colour GC.
tracwiki: trac.parrot.org/parrot/wiki/GCMassa...ction=diff
chromatic bacek, I wonder if there's a cheap way to mark the colors of objects allocated after tri-color starts. 23:56
I don't understand yet how you want to track colors. 23:57
bacek chromatic, erm. I don't quite understand your question. 23:58
chromatic Where do you store the color for a given object? 23:59
bacek Is it about incremental tri-colour?
chromatic yes
bacek @live_objects, @grey_objects, @dead_objects
I just move PMCs between those lists

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!